AI and LLMs Middle interview questions and answers

75 questions for a AI and LLMs (Middle) interview - with correct answers, explanations and links to the docs.

Take this quiz
  1. 1.What is "context engineering" in LLM apps?
  2. 2.Why is chunk overlap often used when splitting documents for RAG?
  3. 3.What is hybrid search in RAG?
  4. 4.What does a reranker do in a retrieval pipeline?
  5. 5.What does recall@k measure for a retriever?
  6. 6.Which RAG metric checks whether the answer's claims are supported by the retrieved context?
  7. 7.Why must a RAG system filter documents by user permissions?
  8. 8.What happens to an existing vector index if you switch to a new embedding model?
  9. 9.Which similarity measure is most common for comparing text embeddings?
  10. 10.What makes a tool definition easy for a model to use correctly?
  11. 11.A tool call fails with an error. What should the app usually send back to the model?
  12. 12.Why set a maximum number of steps for an agent loop?
  13. 13.When is a fixed workflow better than an autonomous agent?
  14. 14.What is prompt chaining?
  15. 15.What is the routing pattern in LLM workflows?
  16. 16.What is the evaluator-optimizer pattern?
  17. 17.What three main things can an MCP server expose?
  18. 18.Why validate structured output with a schema library like Zod even when the model was asked for JSON?
  19. 19.What is insecure output handling in LLM apps?
  20. 20.What is "excessive agency" in the OWASP LLM Top 10?
  21. 21.Why shouldn't secrets be placed in a system prompt?
  22. 22.How can you reduce the impact of prompt injection in an agent that reads web pages?
  23. 23.Which judge bias makes it prefer the first of two answers shown?
  24. 24.How do you make an LLM judge trustworthy?
  25. 25.What is pairwise evaluation?
  26. 26.How should an LLM eval suite be used in CI?
  27. 27.Why run each eval case several times?
  28. 28.How do you unit-test code that calls an LLM without paying for each run?
  29. 29.What is a risk of using model-generated synthetic test data only?
  30. 30.How should multi-turn chatbot behaviour be tested?
  31. 31.Why is snapshot testing of exact LLM outputs usually a poor fit?
  32. 32.What is a semantic cache for LLM calls, and its main risk?
  33. 33.Which technique lowers cost for many requests with the same long instructions?
  34. 34.What is model routing for cost control?
  35. 35.Why add a fallback provider or model for an AI feature?
  36. 36.What does time to first token (TTFT) measure?
  37. 37.What is the "lost in the middle" effect?
  38. 38.Why prefer answers with citations in a RAG product?
  39. 39.What is sensitive information disclosure in LLM apps?
  40. 40.How can personal data be kept out of LLM requests?
  41. 41.What should production monitoring of an AI feature include?
  42. 42.How do you A/B test two prompts in production safely?
  43. 43.What is the trade-off of self-hosting open-weight models versus using an API?
  44. 44.What is quantization of a model?
  45. 45.What is model distillation?
  46. 46.Why count tokens before sending a request?
  47. 47.How can an agent keep useful memory across long tasks?
  48. 48.What should an AI feature's UI offer when answers can be wrong?
  49. 49.What is data poisoning?
  50. 50.What is a supply chain risk for LLM apps?
  51. 51.Why should an MCP server you install be treated like any dependency?
  52. 52.What is unbounded consumption in LLM apps?
  53. 53.How do you test a feature that summarises long documents?
  54. 54.What should evals for an extraction feature (e.g. invoice to JSON) measure?
  55. 55.What is the purpose of a refusal test set?
  56. 56.How should a team track prompt and model versions in production?
  57. 57.Why prefill or constrain the start of a model's answer in some cases?
  58. 58.What is the main advantage of putting examples in XML-like tags in a prompt?
  59. 59.What can parallel tool calls improve in an agent?
  60. 60.What is the orchestrator-workers pattern?
  61. 61.Why are vector or embedding weaknesses listed as an LLM risk?
  62. 62.How should an app handle a model answer that fails validation repeatedly?
  63. 63.What is misinformation in the OWASP LLM Top 10 sense?
  64. 64.What is a guardrail model?
  65. 65.Why test an AI feature with inputs in several languages?
  66. 66.What is the difference between code-graded and model-graded evals?
  67. 67.What does "pass@k" mean for code-generating models?
  68. 68.Why keep humans in evaluation even with automated judges?
  69. 69.What should happen to user feedback like a thumbs down on an AI answer?
  70. 70.What is a common mistake when building a first RAG prototype?
  71. 71.What does streaming change for error handling?
  72. 72.Why should an AI feature have a cost budget per user or tenant?
  73. 73.Why doesn't adding more retrieved chunks always improve RAG answers?
  74. 74.What should an agent do before an irreversible action like sending money?
  75. 75.Why run your own evals on a new model before switching to it?