AI and LLMs · Senior · Question 6 of 75
Your new prompt scores 84% vs 82% on 50 cases. What's the right conclusion?
- 1The new prompt is clearly better and should ship now
- 2The gap may be noise; more cases or repeated runs are needed
- 3The old prompt is better, since smaller changes are safer
Show the answer
Correct answer: The gap may be noise; more cases or repeated runs are needed
Explanation
With small samples and non-deterministic outputs, a 2-point gap is often within variance. Confidence intervals help decide.