Fairness metrics
Measures for comparing a model’s outcomes across groups.
Examples: demographic parity (similar positive rates across groups), equal opportunity (similar true positive rates) and equalised odds (similar true and false positive rates). The metrics can conflict, so the choice depends on context and law.
On the exam
There is no single correct fairness metric. The audit question is whether the organisation chose and justified appropriate metrics and monitors them.
Related terms
- Bias and fairnessWhether an AI system treats groups of people unfairly.
- Proxy discrimination and disparate impactUnfair outcomes caused indirectly, through variables linked to protected characteristics.
AAIA Test yourself on this with timed AAIA practice questions in the AAIA Exam Simulation Hub, or read the free AAIA exam guide.
Test yourself with today’s question of the day.
