Adversarial examples and evasion attacks
Inputs crafted to fool a model.
Small, often invisible changes to an input can make a model misclassify it, for example altering an image or wording so malicious content passes a filter. Evasion attacks happen at inference time, after the model is deployed.
On the exam
Defences include robustness testing, adversarial training, input validation and monitoring for unusual inputs.
Related terms
- Robustness and stress testingChecking that a model holds up under unusual, noisy or extreme inputs.
- Data poisoningCorrupting training data to manipulate a model.
- AI red teamingDeliberately attacking an AI system to find weaknesses before others do.
AAIA Test yourself on this with timed AAIA practice questions in the AAIA Exam Simulation Hub, or read the free AAIA exam guide.
Test yourself with today’s question of the day.
