AI and LLMs · Middle · Question 27 of 75
Why run each eval case several times?
- 1The first run of any test is always ignored by the tools
- 2Outputs vary, so pass rates show reliability better than one sample
- 3Providers charge less for repeated identical requests
Show the answer
Correct answer: Outputs vary, so pass rates show reliability better than one sample
Explanation
A case passing 3 of 5 times is flaky in production too. Track variance, not just averages.