Interview prep · 101 topics
AI engineer interview prep
The topics that come up in AI engineering interviews and are not in a trail yet: failure signs, refactorings, design patterns, and system design cases. Use it as a review before the technical round.
Spot the problem
The interviewer describes a system and asks what will go wrong.
- Routing System PromptOne prompt tries to handle sales, support, and billing with IF the user asks X blocks.
- Few-Shot HoardingThe prompt carries 30 fixed examples and nobody remembers why each one is there.
- Rule StackingThe prompt piles up ALWAYS, NEVER, and IMPORTANT in caps, and the rules fight each other.
- Prompt as State MachineThe prompt describes steps (first ask for the ID, then the address) and the model skips or repeats them.
- Subagent RecursionSubagents spawn subagents with no depth or cost limit.
- Model-Pinned PromptSwitching models or versions breaks everything because the prompt relies on the old model's quirks.
- Hardcoded Few-ShotsExamples live inside the code and changing one requires a deploy.
- Scattered Prompt StringsPrompt fragments assembled with string concatenation across many files.
- Unversioned PromptNobody knows which prompt version produced the answer the customer complained about.
- Redundant Self-CritiqueA review your answer step that doubles the cost and almost never changes anything.
- Unused ToolsTools in the schema that have not been called in months but take up context on every request.
- Shared ScratchpadSeveral agents write to the same history and one pollutes the other's reasoning.
- Raw HTML Tool OutputThe tool returns the whole page, menus and footer included, and the model gets lost.
- Provider Format LeakOne provider's message format has spread into the business code.
- Secret in PromptKeys, passwords, or internal data sit in the prompt and can be echoed back to the user.
- Unvalidated Executed OutputCode, SQL, or commands generated by the model run directly, with no check or sandbox.
- No Cost BudgetThere is no spending limit per user, per task, or per day.
- Vibe-Check EvalQuality is measured by trying five questions by hand before deploying.
Fix it without breaking it
How to change an LLM system in production, with an eval before and after.
- Extract SubpromptPull one responsibility out of the main prompt and give it its own call.
- Inline Trivial StepRemove a call that does not change the result and fold its work into the neighboring step.
- Replace Rules with ExamplesReplace a pile of rules with a few well chosen examples that show the behavior.
- Move Instruction to Tool DescriptionMove a tool's usage rule into its description, right where the model makes the choice.
- Extract Versioned TemplateGather scattered prompt fragments into one template with a name, version, and variables.
- Separate Instructions from DataClearly mark what is a system instruction and what is content to be read.
- Move Few-Shots to a DatasetTake examples out of the code into a versioned set, picked at request time.
- Replace Prompt Rule with Graph EdgeTurn an IF this, do that written in the prompt into an explicit transition in code.
- Replace Agent with ChainWhen the path is always the same, fix the order of steps in code.
- Introduce RouterClassify the input first and send it to a specialized prompt or agent.
- Move Retrieval Out of the PromptStop pasting documents into the system prompt and fetch only the relevant passage.
- Normalize Tool OutputMake every tool return only the useful fields in a stable format.
- Split God ToolBreak a generic tool (run_sql, call_api) into named, safe operations.
- Expose Tool via MCPPublish the integration as an MCP server so any agent uses the same contract.
- Validate Before ExecuteCheck generated code, SQL, or commands and run them in an isolated environment.
- Move Secret Out of the PromptKeep credentials inside the tool code, where the model never sees them.
- Introduce Model CascadeTry a cheap model first and escalate only when confidence is low.
- Move to Durable ExecutionPersist state at every step so the run can resume after a crash, a wait, or an approval.
Design patterns
The toolbox for designing LLM apps and agents.
Composition 5
- VotingRun the same task several times or with several models and decide by majority or threshold.
- Evaluator-OptimizerOne model generates, another grades against clear criteria, and the loop repeats until it passes.
- Few-Shot SelectorPick, at request time, the examples most similar to the current input.
- Prompt Template RegistryStore prompts as versioned artifacts with variables and history.
- Map-ReduceProcess chunks in parallel and then combine the results into one answer.
Context and knowledge 5
- GraphRAGWalk relationships between entities when the answer depends on connections, not one passage.
- Agentic RAGLet the agent decide when, where, and how many times to search.
- Context IsolationRun subtasks in separate contexts and bring back only the result.
- Progressive DisclosureShow the model only a summary of each skill and load the details when it asks.
- Citation GroundingRequire every claim to point at a source and check that the source says it.
Memory and state 8
- Short-Term MemoryKeep the current conversation's state explicit, outside the raw transcript.
- Long-Term MemoryStore facts and preferences across sessions and bring back only the relevant ones.
- Episodic MemoryRemember past runs, what worked and what failed, to act better next time.
- Memory ConsolidationReview memories in the background to merge, correct, and forget.
- ScratchpadGive the agent a place to note its plan and partial results outside the conversation.
- Checkpoint and ResumeSave state at safe points so the run can continue where it stopped.
- Durable WorkflowRun the agent on an engine that guarantees each step through crashes and long waits.
- Shared BlackboardA structured space where several agents post and read facts, each with an owner and a format.
Tools and action 8
- Code Execution SandboxLet the model write and run code in an isolated, disposable environment.
- Computer UseDrive graphical interfaces through screen, mouse, and keyboard when there is no API.
- Browser AgentNavigate, read, and fill web pages using the DOM structure.
- MCP AdapterExpose systems as MCP servers so any agent uses the same contract.
- Tool Result SummarizerShrink large tool responses to what matters before they reach the model.
- Dry RunProduce the action plan and show its effect before actually running it.
- Idempotent ActionMake sure repeating the same call never duplicates the effect.
- Compensating ActionFor every step with an effect, have a step that undoes it, as in a saga.
Agent behavior 9
- Plan-then-ExecuteWrite the full plan first and execute steps with a simpler model or with code.
- ReplanningRevise the plan when reality does not match expectations.
- ReflectionHave the model critique its own result against an external signal and fix it.
- Self-ConsistencyGenerate several reasoning paths and keep the most frequent answer.
- Tree of ThoughtsExplore alternatives as a tree, score each branch, and prune the bad ones.
- Multi-Agent HandoffTransfer the conversation to another specialist agent along with the needed state.
- SupervisorOne agent coordinates others, decides who works, and checks the delivery.
- DebateTwo or more models argue positions and a judge decides.
- Long-Running AgentKeep an agent working for hours or days with goals, checkpoints, and reports.
Safety and reliability 7
- Output GuardrailsCheck the response before it reaches the user or a tool.
- Least-Privilege ToolsEach tool with its own scope, credential, and limits.
- Constitutional CheckGrade the response against a short list of written principles and revise on violation.
- Fallback ModelKeep another model or provider ready to take over when the main one fails.
- Circuit BreakerStop calling a failing service and resume once it recovers.
- Output Validation and RepairValidate output with code and, on failure, send the error back to the model to fix.
- PII RedactionRemove or mask personal data before the model and the logs, in line with privacy law.
Operations and quality 5
- LLM-as-JudgeUse a model with a rubric to grade answers that have no exact expected output.
- Shadow DeployRun the new version alongside real traffic, hidden from users, and compare.
- Semantic CacheAnswer equivalent questions with an already generated response.
- Prompt CachingReuse the fixed prompt prefix to cut cost and latency.
- Cost BudgetMeasure and cap spending per task, user, and day, with alerts and a cut-off plan.
AI system design
Full cases, from the naive prototype to an architecture that holds up with real users.
- Automatic ticket triageClassify thousands of tickets a day by queue, priority, and product.
- Call center voice assistantHandle live calls with response time under one second.
- Automated PR reviewerComment on PRs with real findings and few false positives.
- Test generatorGenerate tests that compile, pass, and catch real bugs.
- Legacy code migrationMigrate hundreds of files off an old framework over several days.
- On-call incident agentInvestigate alerts, read logs, and suggest the likely cause before the human wakes up.
- Spreadsheet analysis with code executionAnswer spreadsheet questions by writing and running code, not by guessing numbers.
- Automated sales SDRResearch leads, write the first message, and book a meeting.
- Legal assistantReview contracts, flag risky clauses, and cite the legal basis.
- Resume screening with bias controlPre-screen candidates in an explainable, auditable way.
- Content moderationModerate text and images at scale with a clear policy and low cost.
- Conversational onboardingSign up customers over WhatsApp, collect documents, and resume days later.
- E-commerce semantic searchUnderstand running shoes for rainy days and find the right product in milliseconds.
- Educational tutorA tutor that remembers the student, adapts exercises, and never just hands over the answer.
- Browser agent for repetitive tasksFetch duplicate bills, download invoices, and fill portals that have no API.
- Content creation pipelineFrom brief to published post, with research, draft, review, and brand check.
- Health assistant with guardrailsAnswer patient questions without diagnosing and knowing when to refer to a doctor.
- Back-office automation with durable executionReconcile payments, issue invoices, and collect from customers in processes that last days.