AI and LLMs · Senior · Question 32 of 75
How do you test an AI feature for unfair treatment of groups?
- 1Remove all names from the inputs and stop there
- 2Ask the model whether it is biased and trust its answer
- 3Run matched inputs differing only in group attributes
Show the answer
Correct answer: Run matched inputs differing only in group attributes
Explanation
Counterfactual tests (same CV, different names) reveal disparities. Metrics need a definition of fairness agreed with stakeholders.