AI Engineer Roadmap All lessons Interactive Quiz GitHub

Agent Guardrails

Advanced · 9 min read · Updated · By Saurabh Pathak

In short: Guardrails are checks in your code that limit what an agent can see and do (allowed tools, valid arguments, step and cost limits, input and output filters), no matter what the model decides.

Play the interactive lesson: Agent guardrailsRead on GitHub
On this page
  1. Explain it like I'm new
  2. Watch it
  3. How it works
  4. Code example
  5. When to use it / when not to
  6. Common mistakes
  7. Recap
  8. Interview questions
  9. Go deeper

Level: Advanced · Reading time: 9 min · Phase: 4 · Agentic AI

Before you start: Human in the Loop and Function Calling. You should know how code validates a tool call and how risky actions wait for approval.

Explain it like I'm new

A new driver can steer, but the road has lane markings, speed limits, and barriers on dangerous bends. The driver mostly stays safe on their own; the barriers stop the worst outcomes when they don't.

An agent is a capable but unreliable driver. It can misunderstand, hallucinate tool arguments, or be tricked by prompt injection (instructions hidden in a web page, email, or file it reads). Guardrails are the barriers:

The golden rule: guardrails live in code, not in the prompt. "Never delete files" in a system prompt is a request. A delete tool that isn't on the allowlist is a fact.

Words to know:

Watch it

Agent guardrails: Code-level limits on what an agent can see and do.
Agent guardrails: Code-level limits on what an agent can see and do.

It plays on a loop. Read the caption in the box, one step at a time. Want to go at your own pace? Step through it with Previous and Next.

How it works

Layers, from outside in:

  1. Least privilege. Give the agent only the tools it needs, with the narrowest scope (read-only DB user, one folder, one API scope).
  2. Allowlist check on every tool call. Unknown or disallowed tool → refuse.
  3. Argument validation. Schema check, then business rules (amount ≤ limit, path inside the sandbox, recipient in the company domain).
  4. Budget limits. Max steps, tokens, cost, wall-clock time across the whole run.
  5. Approval gate for irreversible actions.
  6. Untrusted content handling. Mark tool outputs and documents as data. Don't let them change permissions. Consider stripping or flagging instruction-like text.
  7. Output checks. Validate format, scan for secrets or personal data, apply content policy.
  8. Audit log. Record every call, check, and decision.
flowchart LR
    M[model wants tool call] --> A{on allowlist?}
    A -- no --> X[refuse + log]
    A -- yes --> V{args valid + within rules?}
    V -- no --> X
    V -- yes --> B{within step/cost budget?}
    B -- no --> S[stop run]
    B -- yes --> R{risky?}
    R -- yes --> H{human approves?}
    H -- no --> X
    H -- yes --> RUN[run tool]
    R -- no --> RUN
    RUN --> O[output filter] --> LOG[(audit log)]

Worked example

Take Guardrails(["write_text"], max_steps=2) from the code below. Real output when we call check three times:

CallWhat happensResult
check("write_text")step 1, on the allowlistpasses silently
check("send_email")step 2, not on the allowlisttool 'send_email' is not on the allowlist
check("write_text")step 3, over the cap of 2max_steps=2 exceeded

Notice that every call counts as a step, even a refused one. That is on purpose: an agent that keeps asking for forbidden tools still hits the cap.

At pipeline level, run_pipeline("agents and email it", ..., allowed_tools=("write_text", "add_example")) returns status blocked and its log ends with guardrail: tool 'send_email' is not on the allowlist.

Code example

From agent_multi.py: a Guardrails object that enforces an allowlist and a step cap, and marks risky tools for approval.

class GuardrailError(Exception):
    pass

class Guardrails:
    def __init__(self, allowed_tools, max_steps=10, risky_tools=()):
        self.allowed, self.risky = set(allowed_tools), set(risky_tools)
        self.max_steps, self.steps = max_steps, 0

    def check(self, tool):
        self.steps += 1
        if self.steps > self.max_steps:
            raise GuardrailError(f"max_steps={self.max_steps} exceeded")
        if tool not in self.allowed:
            raise GuardrailError(f"tool '{tool}' is not on the allowlist")

    def needs_approval(self, tool):
        return tool in self.risky

g = Guardrails(["write_text"], max_steps=2)
g.check("write_text")                   # ok
try:
    g.check("send_email")
except GuardrailError as e:
    print(e)                            # tool 'send_email' is not on the allowlist

An argument-level business rule, added on top of schema validation from agent_react.py:

import os

SANDBOX = os.path.realpath("/tmp/agent-sandbox")

def safe_path(path):
    full = os.path.realpath(os.path.join(SANDBOX, path))
    if not full.startswith(SANDBOX + os.sep):
        raise GuardrailError("path escapes the sandbox")
    return full

In real projects, frameworks offer hooks for these checks (for example, tool-level callbacks or graph interrupts in LangGraph). Run code-executing tools in a real sandbox (container or VM with no secrets and restricted network).

When to use it / when not to

Always. The question is how strict.

Agent can...Minimum guardrails
Only read public dataallowlist, step cap, output format check
Read private data+ per-user access filters, output PII/secret scan
Write or act externally+ argument rules, approval for irreversible actions, audit log
Run code+ sandbox with no secrets, network limits, timeouts

Common mistakes

Recap

If you remember only three things:

Interview questions

1. Why can't a system prompt alone keep an agent safe? <details><summary>Answer</summary> The model may misunderstand, ignore, or be manipulated out of prompt instructions, for example by prompt injection in content it reads. Only checks in code around tool execution reliably block disallowed actions. </details>

2. List five guardrails you'd add to a production agent. <details><summary>Answer</summary> Tool allowlist with least privilege, argument schema plus business-rule validation, max steps / token / cost / time limits, human approval for irreversible actions, input and output filtering (injection, secrets, PII), sandboxing for code execution, and audit logging. </details>

3. How do you defend an agent that browses the web against prompt injection? <details><summary>Answer</summary> Treat page content as untrusted data, keep high-risk tools unavailable or approval-gated during browsing, validate every action in code, limit data the agent can send out (no secrets in context, restricted destinations), and monitor/log. No single filter fully solves it, so use layers. </details>

Go deeper

Prev: Agent Evaluation · Next: Phase 5 · Frameworks · Back to phase: Phase 4