AI and LLMs · Middle · Question 67 of 75
What does "pass@k" mean for code-generating models?
- 1The chance that at least one of k samples passes the tests
- 2The number of tests that pass on the k-th attempt only
- 3The share of k users who accept the generated code
Show the answer
Correct answer: The chance that at least one of k samples passes the tests
Explanation
It measures capability when you can try several samples and check them, common in coding benchmarks.