AI and LLMs · Senior · Question 4 of 75
What makes an eval dataset representative?
- 1It uses the same ten examples that were in the prompt
- 2It mirrors real inputs, including hard, rare and adversarial cases
- 3It contains only the easiest cases so scores look stable
Show the answer
Correct answer: It mirrors real inputs, including hard, rare and adversarial cases
Explanation
Sample from production (anonymized), add known failures and edge cases, and keep a held-out part you never tune against.