red teaming

Deliberately attacking your own AI system, by hand or with automated attacks, to find failures before real attackers or users do.

In more detail

For LLM apps it covers prompt injection, jailbreaks, data leaks and harmful output. Turn every finding into a regression case in your eval set.

Related terms

Taught in

References

Certification courses