Structured Outputs for AI Apps: Turn Unreliable Text Into Production Data

“Return JSON” is a request. A schema is a contract.

Production AI applications need structured outputs because downstream code cannot safely depend on a model remembering commas, field names, enums, and business rules in free-form text. Schema-constrained generation removes a large class of parsing failures, but it does not make model judgment automatically correct.

Design the Schema Around the Action

Start from what the application will do with the result. A support classifier may need only a category, confidence, and escalation flag. Do not ask the model for a 20-field object because those fields might be useful someday.

Keep the first schema narrow:

import { z } from "zod";

export const TicketDecision = z.object({
  category: z.enum(["billing", "bug", "account", "other"]),
  urgency: z.enum(["normal", "high"]),
  requiresHuman: z.boolean(),
  summary: z.string().min(1).max(240),
});

Enums are better than unconstrained labels. Bounded strings are better than unlimited prose. Required fields are better than ambiguous combinations of optional values.

Avoid using a single nullable field to represent several states. If a value can be unavailable because it was not found, not requested, or forbidden, model those states explicitly.

Separate Syntax From Meaning

Schema enforcement can guarantee that urgency is either normal or high. It cannot guarantee that the model chose the right one.

Use three validation layers:

1. provider-level structured generation for valid shape
2. runtime schema validation in your application
3. business-rule validation for meaning and permissions

Business rules might require that refunds above a threshold always go to a human, or that a tool action cannot target an account outside the authenticated workspace.

const parsed = TicketDecision.parse(modelOutput);

if (parsed.category === "billing" && parsed.urgency === "high") {
  parsed.requiresHuman = true;
}

Treat model output as untrusted input even when it matches the schema. It can contain prompt injection fragments, unsafe URLs, or confident but incorrect identifiers.

Use Discriminated Unions for Workflows

When an agent may take different actions, use a discriminated union instead of one object with many optional fields.

const Decision = z.discriminatedUnion("action", [
  z.object({
    action: z.literal("answer"),
    message: z.string().max(2000),
  }),
  z.object({
    action: z.literal("search"),
    query: z.string().min(3).max(300),
  }),
  z.object({
    action: z.literal("escalate"),
    reason: z.enum(["policy", "uncertain", "user_request"]),
  }),
]);

This prevents impossible objects such as a response that simultaneously contains search arguments, a final answer, and an escalation reason.

Keep authorization outside the model. The model may propose search; deterministic code decides whether search is allowed and which data sources are in scope.

Handle Refusals and Incomplete Results

Do not force every response into a success object. Models may refuse, time out, or lack enough evidence. Your application needs explicit failure states.

const Result = z.discriminatedUnion("status", [
  z.object({
    status: z.literal("ok"),
    data: TicketDecision,
  }),
  z.object({
    status: z.literal("needs_input"),
    question: z.string(),
  }),
  z.object({
    status: z.literal("refused"),
    reason: z.string(),
  }),
]);

A well-designed needs_input branch is more reliable than encouraging the model to guess missing customer IDs or dates.

Cap repair attempts. If validation fails, one targeted retry with the validation error may be reasonable. Repeatedly feeding malformed output back into the same prompt creates latency and unpredictable cost. After the cap, fall back or escalate.

Version Schemas Like APIs

Prompt changes can alter behavior even when the JSON shape remains identical. Store the schema version, prompt version, and model with every result.

When changing a field:

• add a new schema version
• keep readers compatible during migration
• backfill only when necessary
• monitor validation and semantic error rates by version

Do not silently reinterpret an old enum. If high once meant “reply within four hours” and now means “page an on-call engineer,” that is a business migration, not a prompt tweak.

Generated TypeScript types help developers, but the runtime validator remains essential. Static types disappear at the network boundary.

Evaluate With Adversarial Inputs

Create a dataset that attacks both shape and meaning:

• missing context
• conflicting user instructions
• very long input
• multilingual text
• malicious text that asks the model to ignore the schema
• values near business thresholds
• unknown categories
• content containing JSON examples

Track:

• schema-valid rate
• correct-field rate
• abstention quality
• repair rate
• end-to-end task success
• unsafe-action rate

A 99.9% valid-JSON rate can coexist with poor decisions. Measure the product outcome.

Keep Humans at High-Risk Boundaries

Structured output makes automation easier, which increases the importance of restraint. A perfectly formatted object can still trigger a bad refund, delete the wrong file, or message the wrong person.

Require confirmation or deterministic policy checks for destructive, financial, security-sensitive, or externally visible actions. Preserve the proposed object in an audit log with sensitive values redacted.

The best production pattern is simple: constrain the shape, validate it again, enforce business rules in code, and treat uncertainty as a legitimate output. Structured generation is the start of reliability—not the finish line.