Policy as Code for AI Agents: Turn Safety Rules Into Testable Runtime Controls

A rule in a system prompt is advice; a rule in an authorization path is control. Production teams need policy as code for AI agent guardrails because a probabilistic model should never be the final authority on whether it may send money, expose a document, or delete a record.

This tutorial builds a small policy layer that sits between an agent and its tools. The model can propose an action. Deterministic code decides whether that action is allowed, denied, or requires approval.

Prerequisites

You need an agent runtime that represents tool calls as structured data, a stable user and tenant identity, and a server-side tool executor. The examples use TypeScript-style objects and Open Policy Agent concepts, but the architecture works with any policy engine.

Do not place the enforcement layer in the browser or rely on text the agent can edit. The policy decision and tool credentials belong in a trusted service.

Step 1: define the authorization envelope

Start with the smallest facts the policy needs. Never send the entire prompt transcript to the policy engine.

type ActionRequest = {
  principal: { userId: string; tenantId: string; roles: string[] };
  tool: string;
  operation: string;
  resource: { tenantId: string; id?: string; classification?: string };
  effect: "read" | "write" | "delete" | "external_send";
  amountUsd?: number;
  destinationDomain?: string;
  requestId: string;
};

Expected result: every proposed action can be evaluated without parsing free-form model prose.

Use server-derived values for identity, tenant, and current approvals. The model may suggest a resource ID, but it must not assert its own role or approval status.

Step 2: deny by default

A compact decision function is better than scattered if statements.

type Decision =
  | { outcome: "allow"; policyId: string }
  | { outcome: "deny"; policyId: string; reason: string }
  | { outcome: "approval"; policyId: string; reason: string };

function decide(a: ActionRequest): Decision {
  if (a.principal.tenantId !== a.resource.tenantId) {
    return { outcome: "deny", policyId: "tenant-boundary-v1", reason: "cross-tenant resource" };
  }
  if (a.effect === "delete") {
    return { outcome: "approval", policyId: "destructive-action-v2", reason: "human approval required" };
  }
  if (a.effect === "external_send" && a.destinationDomain !== "company.example") {
    return { outcome: "approval", policyId: "external-send-v3", reason: "untrusted destination" };
  }
  if (a.tool === "payments" && (a.amountUsd ?? 0) > 100) {
    return { outcome: "approval", policyId: "payment-limit-v1", reason: "amount above autonomous limit" };
  }
  return { outcome: "allow", policyId: "baseline-v1" };
}

Expected result: an unknown tool or missing field does not accidentally fall into an allowed branch. In a real system, create an explicit allowlist and treat incomplete data as a denial.

Step 3: separate capability from policy

The tool executor should hold narrowly scoped credentials. Policy answers whether an action may proceed; capabilities limit what can happen if policy code fails.

For example, an invoice-reading worker should receive a read-only token restricted to one tenant and a short expiration. A payment worker should receive a separate credential only after approval. Never give the language model a general API key and expect a prompt to contain it.

const decision = await policy.evaluate(action);
await audit.append({ action, decision, evaluatedAt: new Date().toISOString() });

if (decision.outcome === "deny") throw new Error("ACTION_DENIED");
if (decision.outcome === "approval") return approvals.create(action, decision);

const capability = await broker.issue({
  tenantId: action.principal.tenantId,
  tool: action.tool,
  operation: action.operation,
  ttlSeconds: 60,
});
return executor.run(action, capability);

Expected result: the model never receives broad credentials, and every executed call has a matching decision record.

Step 4: make approvals bind to exact actions

“Approved” is not a reusable boolean. Hash the normalized action and bind the approval to that hash, requester, approver, deadline, and policy version.

const approvalKey = sha256(canonicalJson({
  principalId: action.principal.userId,
  tool: action.tool,
  operation: action.operation,
  resource: action.resource,
  amountUsd: action.amountUsd,
  destinationDomain: action.destinationDomain,
}));

If the destination, amount, or resource changes, the old approval must not apply. Mark approvals consumed when the action is non-idempotent.

Step 5: test policy like product code

Create table-driven tests for allow, deny, and approval outcomes.

for (const test of cases) {
  it(test.name, () => expect(decide(test.input)).toMatchObject(test.expected));
}

Include adversarial cases: missing tenant IDs, Unicode-lookalike domains, negative amounts, scientific notation, unknown effects, duplicate requests, stale approvals, and resources moved between classifications.

Run mutation tests or deliberately invert conditions. If the suite still passes after > becomes <, it is not protecting the boundary you think it is.

Step 6: version and observe decisions

Log the policy bundle version, input hash, outcome, reason, request ID, and downstream execution ID. Avoid logging secrets or full sensitive payloads.

Before rollout, shadow the new policy against production-shaped traffic. Compare old and new outcomes without enforcing the new result. Review unexpected differences, then canary by tenant or tool.

Useful metrics include deny rate, approval rate, approval latency, attempted cross-tenant access, decisions missing required fields, and tool executions without a decision record. That last number should be zero.

Common pitfalls

Letting the model classify risk

The model can supply hints, but deterministic services should derive the real effect from the registered tool schema. An operation named archive may be destructive even if the agent labels it as a read.

Evaluating after execution

Post-hoc moderation is an audit, not prevention. Put the policy check immediately before capability issuance and tool execution.

Treating retrieval as harmless

Reads can expose confidential data or cross tenant boundaries. Apply authorization before retrieval, then filter returned fields again before adding them to model context.

Building an unreviewable mega-policy

Split rules by domain and keep decision reasons stable. Owners should be clear: security controls tenant and secret boundaries; finance controls spend; product controls user-facing sends.

The production checklist

Verify that every tool route passes through one enforcement point, unknown actions deny, approvals bind to exact normalized requests, capabilities are short-lived, policy bundles are signed and versioned, and decisions can be traced to executions.

Policy as code does not make an agent safe by itself. It makes critical boundaries deterministic, testable, reviewable, and independent of whatever persuasive text the model just read.