AI and LLMs · Senior · Question 5 of 75
Why keep a held-out eval set you don't look at while tuning prompts?
- 1To detect overfitting the prompt to the visible test cases
- 2Because providers require a hidden set for every app
- 3Because held-out sets are cheaper to run than others
Show the answer
Correct answer: To detect overfitting the prompt to the visible test cases
Explanation
Iterating on the same cases can make a prompt great there and worse elsewhere. A held-out set shows real generalization.