AI and LLMs Interview Questions and Answers

Sample questions

A few questions from each level with the correct answer and an explanation.

AI and LLMs · Junior

1. What does a large language model (LLM) fundamentally do?
   ▸Show the answer

   Correct answer: Predicts the next token based on the text so far

   An LLM is trained to continue text one token at a time. Useful behaviour like answering questions comes from that prediction plus training on instructions, not from a fact database.

2. What is a token in the context of LLMs?
   ▸Show the answer

   Correct answer: A chunk of text, often part of a word, that the model processes

   Text is split into tokens by a tokenizer. In English one token is roughly three quarters of a word; other languages and code often need more tokens per word.

3. Why does the same text in Ukrainian often use more tokens than in English?
   ▸Show the answer

   Correct answer: Tokenizers favour English, so other languages split into more pieces

   More tokens mean higher cost, slower responses and less room in the context window for the same content.

4. What is the context window of a model?
   ▸Show the answer

   Correct answer: The most tokens it can see at once, input plus output

   Everything the model sees, such as the system prompt, history, documents and its answer, must fit in the window. Text beyond it is simply not visible to the model.

AI and LLMs · Middle

1. What is "context engineering" in LLM apps?
   ▸Show the answer

   Correct answer: Choosing what information goes into the context for each step

   Beyond the prompt wording, results depend on which documents, history, tool results and instructions the model sees, and in what order.

2. Why is chunk overlap often used when splitting documents for RAG?
   ▸Show the answer

   Correct answer: So facts on chunk borders aren't cut in half

   A sentence split between two chunks may never be retrieved whole. A small overlap keeps context together at the cost of some duplication.

3. What is hybrid search in RAG?
   ▸Show the answer

   Correct answer: Combining keyword search like BM25 with vector search

   Vectors catch meaning; keywords catch exact terms like product codes and names. Combining them often improves retrieval.

4. What does a reranker do in a retrieval pipeline?
   ▸Show the answer

   Correct answer: Reorders retrieved candidates by relevance with a stronger model

   A fast first stage fetches, say, 50 chunks; a cross-encoder reranker picks the best 5 for the prompt, improving precision.

AI and LLMs · Senior

1. When is an LLM the wrong tool for a feature?
   ▸Show the answer

   Correct answer: When rules or a simple model solve it reliably and cheaper

   Deterministic logic, regex or a small classifier are faster, cheaper and testable. LLMs pay off for fuzzy language tasks where rules break down.

2. What is eval-driven development for AI features?
   ▸Show the answer

   Correct answer: Write evals with success criteria first, then iterate prompts against them

   Like TDD: define what good looks like, measure the baseline and make each change prove it improves the score.

3. How should success criteria for an AI feature be defined?
   ▸Show the answer

   Correct answer: Specific, measurable and tied to the task, e.g. 95% accuracy

   Vague goals can't be tested. Concrete targets for accuracy, latency, cost and safety guide trade-offs.

4. What makes an eval dataset representative?
   ▸Show the answer

   Correct answer: It mirrors real inputs, including hard, rare and adversarial cases

   Sample from production (anonymized), add known failures and edge cases, and keep a held-out part you never tune against.