AI Agents Need an Evidence Plane, Not Just More Autonomy

Three releases this week look unrelated. Anthropic published internal metrics for supervising AI research agents. Cognition launched Code Scans, which fans out codebase investigations and turns findings into reviewable pull requests. Bend introduced a language design that asks AI-generated changes to carry machine-checkable proofs of declared laws.

Together they reveal the next bottleneck in agentic software: the industry has built action planes faster than evidence planes.

An action plane lets an agent search, edit, call tools, and deploy. An evidence plane records what it inspected, why a claim follows, which checks ran, where uncertainty remains, and who approved the consequential step. Without that second layer, more autonomy mostly produces more confident work to review.

The common pattern

Anthropic’s internal system tracks stable agent identities, pre-action monitoring, post-action review, and escalation. Its automation index distinguishes participation from leadership and full autonomy. The core idea is traceability across a fleet.

Code Scans applies a similar structure to repository analysis. A broad goal is converted into selection rules, the codebase is sharded, parallel agents inspect bounded areas, and a reducer deduplicates and prioritizes findings. The output is not supposed to be “trust us, there are 44 SEO problems.” It is a list tied to concrete files and proposed changes.

Bend pushes verification closer to the programming language. Developers declare laws and ask an implementation to include a proof that the program satisfies them. A type checker validates the proof. The promise is not that the model stops making mistakes; it is that a defined class of mistakes cannot cross the gate with a convincing paragraph alone.

These systems operate at different layers, but their architecture rhymes: decompose the work, preserve provenance, attach evidence, and enforce a gate.

Why agent confidence is the wrong interface

Language models are optimized to produce plausible continuations. A fluent explanation can be useful, but fluency is not a checksum. When an agent claims that a symbol is unused, a migration is complete, or a permission boundary is safe, a reviewer needs artifacts.

Those artifacts might include a reference graph, a reproducible query, a test result, a counterexample search, a proof object, or an immutable log of tool calls. The right artifact depends on the claim.

Confidence scores do not solve this. A model can be confidently wrong, and calibration changes across tasks and prompts. Evidence lets another process challenge the conclusion without reproducing the entire reasoning trace.

What the releases actually show

Cognition reports that a Philips group saw an estimated 96% merge rate across several repositories and more than 700 engineering hours saved during testing. In a public Dioxus example, proposed changes reduced a clean debug build from 58.6 seconds to 21.0 seconds. The company also reports shipping fixes from 44 SEO findings across its own sites.

Those are vendor-selected examples, not controlled trials. A high merge rate can reflect good findings, conservative scope, reviewer preference, or selection bias. The useful product detail is the plan–shard–map–reduce process and the ability to inspect evidence before requesting pull requests.

Bend drew stronger public debate: its September 17 Hacker News thread had 366 points and 186 comments at our check. Discussion focused on whether the proof burden is practical, what properties the language can express, and whether generated proofs become another maintenance surface. Those are the correct questions. Formal verification is powerful only for properties that are specified correctly and checked by a trusted kernel.

Anthropic’s disclosure adds the organizational layer. Monitoring every action is valuable, but the company acknowledges that coverage does not equal detection. A fleet can produce correlated errors that look benign one event at a time.

A practical evidence-plane architecture

Agent platforms should separate six objects that are often collapsed into one transcript:

1. Task contract

Define the outcome, permitted resources, excluded areas, risk class, and completion test. A vague goal creates evidence that cannot be judged.

2. Coverage ledger

Record which files, datasets, services, or cases were inspected and which were skipped. For a code scan, this means enumerating the selected surface rather than sampling silently.

3. Claim graph

Every material conclusion should point to supporting artifacts and dependencies. “Dead code” might link to the symbol definition, reference search, dynamic-loading check, and test result.

4. Counterexample phase

Use a separate pass—or ideally a separate model or deterministic tool—to try to falsify the finding. Discovery and verification have different incentives.

5. Policy gate

Deterministic checks should control irreversible or high-impact operations. A language proof, schema validator, permission check, test suite, or human approval can serve as the gate.

6. Outcome feedback

Track what merged, what was reverted, what caused incidents, and what reviewers rejected. Otherwise the platform optimizes task completion without learning whether its claims survived contact with production.

Where formal proofs fit

Proofs are strongest when requirements can be stated precisely: a balance never goes negative, a parser accepts only a grammar, or a function preserves an invariant. They are weaker when the goal is subjective, incomplete, or dependent on a changing environment.

Even a correct proof establishes only the declared property under the language and model assumptions. It does not show that the property captures user intent. A perfectly verified authorization rule can still encode the wrong roles.

That means formal methods should complement tests, reviews, and runtime controls. They are a high-quality evidence type, not a universal replacement for judgment.

Limitations

All three examples come from organizations presenting their own systems. EgoistAI reviewed the official material and linked discussions but did not run Code Scans, audit Anthropic’s internal platform, or verify Bend’s proof kernel.

Evidence systems also create cost. Logs can expose sensitive data. Coverage tracking can become bureaucratic. Proof obligations may slow change. Independent verification consumes compute and reviewer time.

The answer is risk-based depth. A disposable prototype does not need the same evidence as a payment migration or model-training pipeline. But the system should make that choice explicit.

Final verdict

The next generation of agents will not earn trust by sounding less uncertain. It will earn trust by making important claims cheap to inspect and hard to fake.

Anthropic, Cognition, and Bend each provide one piece: fleet observability, evidence-backed repository search, and machine-checked invariants. The winning agent platform will connect those pieces into a control plane where autonomy expands only as verification capacity expands with it.