AI red teaming
Deliberately attacking an AI system to find weaknesses before others do.
Red teams try jailbreaks, prompt injection, data leakage, bias triggers and harmful outputs, then report findings for fixing. It is used before launch and periodically afterwards.
On the exam
Red teaming is adversarial testing by people trying to break the system, beyond normal functional testing.
Related terms
- JailbreakingTricking a generative AI system into ignoring its safety rules.
- Robustness and stress testingChecking that a model holds up under unusual, noisy or extreme inputs.
- Evaluating generative AI (evals)Testing generative AI systems for quality, accuracy and safety.
AAIA Test yourself on this with timed AAIA practice questions in the AAIA Exam Simulation Hub, or read the free AAIA exam guide.
Test yourself with today’s question of the day.
