AI and LLMs · Junior · Question 70 of 75
Why are evals run on many examples instead of one?
- 1Outputs vary, so pass rates across cases show real quality
- 2Providers require at least 100 requests per test
- 3One example costs more than many examples
Show the answer
Correct answer: Outputs vary, so pass rates across cases show real quality
Explanation
A feature passing 92 of 100 cases tells you more than one good-looking answer. Track the rate over versions.