agent policy engine
Code outside the model that decides, for every proposed tool call, whether to allow it, deny it or ask a person, based on rules written as data.
In more detail
Deny by default, check arguments and not just tool names, rate limit costly calls, and log every decision. Because it never reads the model's reasoning, an injected instruction cannot argue its way past it.
Related terms
- excessive agency — Giving a model-driven system more tools, permissions or autonomy than its task needs, so a fooled or mistaken model can do real damage.
- deny by default — The rule that anything not explicitly allowed is refused, including when the facts needed to decide are missing.
- human approval — Pausing an agent before a risky action so a person can approve, edit or reject it, then resuming from saved state.
- tool dispatcher — The code that receives a model's tool call, checks the tool name and input, runs the matching function and builds the result.
Taught in
- Prompt Injection — AI Engineering: LLM Apps, RAG & Agents
- Excessive Agency — AI Engineering: LLM Apps, RAG & Agents
References
- OWASP GenAI Security Project: LLM06:2025 Excessive Agency
- Debenedetti et al. (2025): Defeating Prompt Injections by Design (CaMeL)