Model extraction and model inversion
Stealing a model, or recovering its training data, through its outputs.
In extraction, an attacker queries a model repeatedly to build a copy. In inversion, outputs are used to reconstruct sensitive training data. Both exploit unrestricted access to predictions.
On the exam
Controls include rate limiting, monitoring query patterns, limiting output detail (labels rather than full probability scores) and access controls on the model API.
Related terms
- Membership inferenceWorking out whether a specific person’s data was used to train a model.
- Security controls for AI systemsProtecting models, data, pipelines and AI interfaces.
AAIA Test yourself on this with timed AAIA practice questions in the AAIA Exam Simulation Hub, or read the free AAIA exam guide.
Test yourself with today’s question of the day.
