Build an AI Agent Policy Engine: Guardrails That Survive Production

An agent policy that exists only in a system prompt is a suggestion. The model can misunderstand it, lose it in a long context, or be manipulated by hostile input.

Production guardrails belong in code that the model cannot rewrite. The model proposes an action. A policy engine decides whether that action is allowed, denied, transformed, rate-limited, or held for approval.

Start With Typed Actions

Do not hand an agent a generic execute_anything tool. Define narrow operations such as read_issue, create_draft_invoice, send_email, deploy_preview, or refund_order.

Each action request should include:

• authenticated actor and tenant
• agent and session identity
• operation and resource
• structured parameters
• data classification
• reason and evidence references
• requested deadline or budget

Validate the schema before policy evaluation. Reject unknown fields, oversized strings, malformed identifiers, and destinations outside the tenant’s scope.

Typed actions make intent inspectable. They also prevent a model from hiding a destructive command inside a harmless-looking free-text field.

Separate Authentication From Authorization

Authentication establishes who the user and agent are. Authorization decides what they may do to a specific resource right now.

An agent acting for a finance analyst should not inherit every permission of the analyst’s browser session. Give it a constrained delegated identity with short-lived credentials and explicit scopes.

Evaluate tenant, project, ownership, environment, and resource labels on every request. “Can deploy” is incomplete. The useful rule is “can deploy this repository to preview from an approved commit during this session.”

Keep secrets in a broker. The agent requests an operation; the broker uses the credential after policy approval. The model never receives a reusable token.

Design a Risk Ladder

Classify actions by consequence. Reading public documentation may be low risk. Modifying a draft is moderate. Sending messages, moving money, deleting data, or changing production is high risk.

Map each class to controls:

• low: allow with logging and rate limits
• moderate: allow only inside a reversible sandbox or draft state
• high: require fresh human approval and stronger identity checks
• prohibited: deny regardless of model confidence

Risk depends on parameters. Sending an email to the current user is different from sending 50,000 emails to imported contacts. A small refund within policy differs from a cross-border transfer.

Compute risk from action type, value, volume, destination, data sensitivity, novelty, and reversibility.

Make Approval Specific

“Approve this agent” is too broad. Approval should bind to an exact operation, resource, parameters, maximum amount, expiration, and version of the proposed artifact.

If the agent edits the email after approval, the approval should become invalid. Hash the approved content and compare it at execution time.

Show reviewers a human-readable summary and the machine-level payload. Highlight external recipients, irreversible effects, unusual permissions, and differences from the last approved version.

Avoid approval fatigue. If every read requires a click, users will rubber-stamp the dangerous actions too. Focus attention where consequence changes.

Evaluate Policy Outside the Model

Use deterministic policy code or a policy engine such as Open Policy Agent. Inputs can include identity claims, resource attributes, action parameters, environment, approval records, and organizational rules.

Return structured decisions:

{
  "decision": "require_approval",
  "reason": "external_recipient",
  "constraints": {"maxRecipients": 1},
  "policyVersion": "2026-08-03.4"
}

The model may explain the decision, but it cannot override it. Denials should not reveal sensitive rules that help an attacker probe boundaries.

Fail closed when the policy service is unavailable for consequential actions. A timeout is not permission.

Add Budgets and Rate Limits

Agents can cause damage through repetition even when each action is individually allowed. Enforce budgets for model tokens, tool calls, emails, API requests, compute time, and monetary value.

Use per-user, per-agent, per-tenant, and per-destination limits. Detect bursts, repeated denials, and loops that alternate among similar tools.

Reserve global emergency controls. Operators need to disable an action class, provider, tenant, or agent version without redeploying the entire product.

Protect Against Confused Deputies

A policy broker can become a confused deputy if it performs privileged work based on attacker-controlled references. Resolve resources server-side and verify that actor, tenant, and resource belong together.

Do not accept a raw URL and fetch it with internal credentials. Accept a resource ID, look it up inside the authorized tenant, and construct the request from trusted configuration.

Bind outputs to their origin. A document retrieved from one customer must never be accepted as authorization for an action in another customer.

Log Decisions, Not Secrets

Create immutable audit events for request, policy input summary, decision, approval, execution, and result. Include model, prompt, tool, and policy versions along with artifact hashes.

Redact credentials, personal data, and full sensitive prompts. Store enough evidence to reconstruct who requested what and why without turning the audit system into a secret warehouse.

Alert on unusual destinations, policy-denial spikes, approval bypass attempts, high-cost loops, and new action combinations. Policy telemetry should feed both security response and product evaluation.

Test the Boundary Adversarially

Build cases for prompt injection, forged tenant IDs, path traversal, stale approvals, parameter changes after approval, Unicode lookalikes, excessive batch sizes, broker timeouts, and replayed requests.

Test enforcement directly. Asking the model whether it would obey a rule is not a security test. Invoke the tool boundary with malicious payloads and verify the action never reaches the downstream service.

Version policies and run historical action logs through proposed changes. A small rule edit can unexpectedly block common workflows or open a path that was previously denied.

A Production Decision Flow

1. Authenticate the user and delegated agent.
2. Parse the request into a typed action.
3. Resolve resources inside the tenant boundary.
4. Validate parameters and classify risk.
5. Evaluate deterministic policy.
6. Obtain action-specific approval when required.
7. Revalidate payload hashes and budgets.
8. Execute through a credential broker.
9. Record the result and update limits.

The winning guardrail is boring. It does not depend on the model remembering a paragraph. It is a narrow, testable authorization boundary that remains intact when the prompt is hostile, the model is confused, and the agent is moving fast.