For hiring managers
AI interview questions for engineering candidates
Most engineering candidates now use AI to write and read code. The interview has to find out how they use it. Some accept a plausible diff and trust the tests it came with. Others give the tool the right files, read every line it returns, and can explain each change in review.
The twelve questions below cover the six skills Emeri measures, two questions per skill. Each one comes with what a strong answer shows and what a weak answer shows. Two interviewers can then score the same answer the same way.
How to use these questions
- Ask for a real example
- Each question works best when the candidate describes work they have done. When an answer stays general, ask what they did the last time, and what happened next.
- Ask every candidate the same core questions
- Pick four to six questions before the interviews start. The same questions make answers comparable across candidates.
- Score the behavior, not the vocabulary
- A candidate who names many tools or uses the right terms has not shown the skill. Score against the strong and weak notes under each question.
- Pair the interview with a short work sample
- If the job uses AI, let the candidate use it in the sample. Ask for the requests they wrote and a note on what they changed. An interview shows how a candidate talks about the work. A sample shows the work.
Asking well
Asking for what you need clearly enough to get useful results. What this skill looks like for software engineers.
Question 1
A teammate asks an AI tool to “fix the flaky test in the checkout suite.” What would you add to that request before sending it?
A vague request invites a fix that hides the symptom. This question shows whether the candidate knows what a model needs to diagnose a failure.
A strong answer
Names the test and pastes the failure output. Says how often it fails and in which environment. Asks for likely causes and a way to confirm each one before any code change. Rules out added sleeps or automatic retries. Defines done as the test passing on repeated runs.
A weak answer
Adds words such as “carefully” or “properly” and nothing else. Or accepts the first change that makes the test pass once, even if that change is a longer timeout.
Question 2
Tell me about a time an AI tool changed more code than you asked it to. How did you word your next request?
Unrequested changes are hard to review and can break callers. This question tests whether the candidate controls scope through the request.
A strong answer
Describes the extra changes in specific terms, such as renamed variables or a rewritten error path. Explains the limits they added next: which files may change, which signatures stay fixed, and that formatting stays as it is. May ask for a plan or a list of files before any edit.
A weak answer
Reverted the change and wrote the code by hand, with no change to how they ask. Or kept the larger diff because the tests still passed.
Giving good context
Giving AI the background, data, and limits it needs. What this skill looks like for software engineers.
Question 3
You need AI help to add a feature to a service you have never worked in. What would you give the tool before you ask for code?
A model knows nothing about a codebase it has not been shown. This question tests whether the candidate selects the material a new teammate would need.
A strong answer
Supplies the files the change touches, the interfaces and callers they depend on, and the language and framework versions. Adds one existing function the team treats as a good example, plus the related tests. Asks for an explanation of the data flow before any code. Leaves out deprecated modules that could mislead the model.
A weak answer
Points the tool at the whole repository and asks it to work out the rest. Or pastes a single function with no versions, callers, or tests.
Question 4
An AI tool suggests a call to a library function that your installed version does not have. What does that tell you, and what would you change?
Version mismatch is a common source of broken AI-written code. This question tests whether the candidate traces the error to missing context.
A strong answer
Recognizes that the model drew on a different version of the library. States the installed version in the request and supplies the documentation or changelog for the functions in use. Keeps a short project note with versions and conventions to include in future requests.
A weak answer
Rewords the request until something compiles. Or upgrades the library to match the suggestion without checking what else the upgrade affects.
Checking the work
Spotting mistakes, checking sources, and knowing what to trust. What this skill looks like for software engineers.
Question 5
An AI tool writes a function and six unit tests, and every test passes. What do you check before you open a pull request?
Passing tests written by the same tool prove little on their own. This question tests whether the candidate reviews AI code as they would a new teammate’s code.
A strong answer
Reads every line of the function. Checks that each test asserts real behavior rather than restating the code. Breaks the function on purpose to confirm the tests fail. Looks at boundaries, empty input, and error paths. Confirms that every library call exists in the installed version. Runs the full suite and the linters.
A weak answer
Treats the passing tests as proof the code is correct. Or reviews the tool’s summary of the change instead of the change itself.
Question 6
An AI assistant explains what a legacy module does, and the explanation sounds right. How would you confirm it before you change that module?
A fluent explanation of code can still be wrong in the details that matter. This question tests whether the candidate verifies before acting.
A strong answer
Traces the main claims against the code. Runs the module with known inputs or writes characterization tests that record current behavior. Reads the commit history for the reasons behind odd sections. Asks the module’s owner if one exists. Notes any part of the explanation that covers code the model was not shown.
A weak answer
Accepts the explanation because it reads clearly. Or makes the change and relies on code review to catch any misunderstanding.
Putting it together
Turning prompts, tools, and steps into a repeatable workflow. What this skill looks like for software engineers.
Question 7
Your team needs the same dependency upgrade applied across many services. How would you use AI so the team can repeat the process and audit it?
Repeated engineering work is where an AI workflow saves the most time. This question tests whether the candidate can build a process with review points.
A strong answer
Does the first service by hand with AI help and writes down each step and the instructions used. Runs the same steps per service, each in its own pull request with tests and the changelog attached. Stops and investigates when a service fails rather than forcing the change. Keeps a list of which services are done, so a teammate can continue the work.
A weak answer
Sends one large request covering every service at once. Or lets an automated tool open and merge the pull requests with no human review.
Question 8
Which engineering tasks would you hand to an AI agent that edits files and runs commands, and which would you keep for yourself?
Knowing where automation fits is part of the skill. This question tests judgment about risk, review cost, and how well a task is specified.
A strong answer
Hands over tasks that are well specified, covered by tests, and easy to reverse, such as mechanical refactors or test scaffolding. Keeps design decisions, unclear requirements, and changes to production data. Runs the agent on a branch with limited permissions and reviews the full diff. Gives an example from their own work.
A weak answer
Says any task that compiles is safe to hand over. Or says they never use agents and cannot explain where one would help.
Getting real work done
Using AI to finish real work, not just make a first draft. What this skill looks like for software engineers.
Question 9
Walk us through a pull request you shipped with AI help. Which parts did the AI write, and what did you verify yourself?
The engineer owns the code that ships, whoever drafted it. This question tests whether the candidate can account for every part of the change.
A strong answer
Separates the parts clearly. AI may have drafted boilerplate, tests, or a first version of a function. The candidate chose the approach, checked the edge cases, and wrote the pull request description in their own words. Speaks honestly about any defect found later and what they changed after it.
A weak answer
Cannot say which lines came from the tool. Or describes a pull request description that restates the diff and gives no reasons for the change.
Question 10
You have a week to write a design doc for a new service. How would you use AI, and which decisions would you keep for yourself?
AI can produce a design doc that looks complete and decides nothing. This question tests whether the candidate keeps the engineering judgment.
A strong answer
Uses AI to outline the document, list alternatives, and question their assumptions. Writes the requirements, the chosen design, and the tradeoffs from their own analysis. Checks every estimate and external claim against a source. Sends the draft to reviewers who can challenge the decisions.
A weak answer
Asks the tool to choose the architecture and edits the wording. Or produces a long document that no one on the team can defend in review.
Keeping it safe
Protecting private information and using good judgment. What this skill looks like for software engineers.
Question 11
You are debugging a production issue and want AI help. What would you remove before you paste logs, configuration, or data into the tool?
Engineering material often holds secrets and customer data. This question tests whether the candidate knows what must stay out of an AI tool.
A strong answer
Names specific items: API keys, tokens, connection strings, environment files, and customer personal data in logs or database rows. Replaces real records with redacted or synthetic samples. Uses only the company’s approved tools. Says they would rotate a credential at once if one went into a tool by mistake.
A weak answer
Says they would “be careful” and gives no examples. Or pastes full logs because the tool is a company account.
Question 12
An AI tool suggests adding an open-source package you have not heard of. What do you check before you install it?
AI tools can name packages that do not exist, and attackers register such names. This question tests whether the candidate treats a new dependency as a security decision.
A strong answer
Confirms the package exists under that exact name and is the one intended. Checks the maintainers, recent activity, license, and known vulnerabilities. Asks whether an existing dependency already covers the need. Pins the version and follows the team’s dependency policy.
A weak answer
Runs the install command the tool suggested. Or checks only that the package has a high download count.
For what each skill looks like in this role at every band from E1 to E4, read AI skills for software engineers. These questions are written for interviews. They are not taken from the Emeri assessment.
Questions
What AI skills should I look for when hiring a software engineer?
Look for all six skills Emeri measures: asking well, giving good context, checking the work, putting it together into workflows, getting real work done, and keeping it safe. In engineering, give extra weight to checking the work and keeping it safe. AI-written code often looks correct and runs, so errors hide at boundaries, in invented calls, and in new dependencies.
Should engineering candidates be allowed to use AI in a coding exercise?
If the team uses AI in daily work, yes. Ask candidates to share the requests they wrote. Then ask them to walk through the submitted code and explain one change they made to the AI output. A candidate who cannot explain their own submission has not shown the skill.
Are these questions taken from the Emeri exam?
No. These questions are written for interviews and are never exam items. The Emeri assessment uses its own questions and scenarios, graded against the same six skills.
How do I confirm a candidate’s Emeri credential?
Every Emeri credential has a public verification page. It shows the band, the assessment date, and a profile across the six skills. The employers page explains how to read one. No account is needed.
Evidence beyond the interview
Emeri assesses the same six skills with graded work, and each credential has a public verification page. Read how to check a candidate’s credential, or see how teams use Emeri assessments.