1,068 Tokens Per Second: Can Mercury 3 Change How LLMs Generate Text?

Mercury 3’s headline number is difficult to ignore: 1,068 output tokens per second on NVIDIA GPUs. Inception says its new flagship diffusion language model also supports a one-million-token context window and costs $0.20 per million input tokens and $0.70 per million output tokens at list price. Launch pricing temporarily cuts both rates in half.

Those numbers make Mercury 3 look less like another benchmark update and more like a challenge to the way most language models generate text. But the careful answer to the headline question is not yet “yes.” The speed is a vendor-reported serving result, while independent quality, latency and reliability testing remains thin. Mercury 3 proves that text diffusion deserves serious production evaluation; it does not yet prove that autoregressive models are obsolete.

What changes when a model does not write one token at a time

Most commercial LLMs are autoregressive. They produce the next token from all previous tokens, append it, and repeat. That sequence is simple and effective, but generation has an inherent serial bottleneck: token 500 cannot be finalized before token 499 exists.

A diffusion language model starts from a noisier or incomplete representation and refines multiple token positions over a series of passes. The analogy is image diffusion, where a picture emerges through repeated denoising, although text has stricter ordering and discrete-token constraints. Parallel refinement creates an opportunity to use modern accelerators more efficiently and emit text far faster than strict next-token decoding.

The trade-off is that “tokens per second” does not describe the whole experience. A model can stream an enormous number of tokens after a long initial wait, revise internal candidates many times, or produce fast prose that fails the task. Developers need time to first useful token, end-to-end completion time, task success rate and cost per successful result—not only throughput.

What Inception actually announced

Inception describes Mercury 3 as more than 40 times larger than Mercury 1 while retaining very high generation speed. The company reports 1,068 tokens per second, a one-million-token context window and quality competitive with speed-optimized frontier models. It also says the model is available through its own API, OpenRouter, Baseten and Vercel using an OpenAI-compatible interface.

The production examples are aimed at latency-sensitive work: browser agents, interactive tutoring, real-time search, customer support and voice applications. Inception reports that Mercury 3 completed 50% of 50 live Mind2Web navigation tasks in its own web-agent harness, and cites early partners that observed sub-second or low-single-digit-second generation in particular setups.

These are useful signals, not neutral certification. The benchmark configuration, hardware allocation, batching, prompts, provider routing and competing-model settings can materially change both speed and quality. Early customer quotations also come from selected launch partners. Until independent evaluators reproduce the results, every superlative should carry “Inception says.”

Why 1,068 tokens per second could matter

For a normal chat answer, humans cannot read hundreds of tokens per second. The benefit appears elsewhere. An agent may generate code, inspect a result, revise a plan and call another tool dozens of times. Saving one or two seconds on every reasoning or action step can turn a minute-long workflow into something interactive.

High throughput can also change product design. Search systems can examine more documents before answering. Voice agents can plan without awkward silence. Coding tools can generate and test multiple candidates. A model can spend extra tokens exploring alternatives while still finishing before a slower model returns its first long draft.

The low token price strengthens that argument. At the temporary launch rate, ten million output tokens would cost $3.50 before platform fees or other infrastructure. The permanent list rate would make the same output $7. That is attractive for high-volume agents, but only if the model completes enough tasks correctly. A cheap failed run repeated three times is not cheaper than one successful expensive run.

The one-million-token context needs its own test

Context-window size is a capacity limit, not proof that a model uses every part reliably. Long-context evaluation should test retrieval at multiple positions, conflicting instructions, source attribution, codebase navigation and accuracy as the prompt grows. It should also measure time to first token and total cost at realistic context lengths.

A one-million-token request may be supported by the API yet still be impractical for an interactive product. Input processing, caching policy and provider limits matter. Inception says cached tokens receive a 90% input discount, which can help repeated work over the same repository or document set. Developers should verify how cache matching, retention and privacy work in their actual provider path.

The benchmark gap

At launch, the most detailed performance story comes from Inception itself. Independent model indexes and long-running agent evaluations have not yet accumulated enough Mercury 3 data to establish a stable ranking. That matters because diffusion systems may behave differently across short answers, exact-format output, tool calls, long code edits and high-temperature generation.

A useful evaluation should compare Mercury 3 with the exact model it would replace, through the same provider region and client. Measure median and tail latency, first-token delay, valid JSON rate, tool-call success, answer quality judged blind, and completed-work cost. Run long prompts separately from short ones, because throughput at a small context says little about a near-million-token request.

Verdict

Mercury 3 is important because it makes diffusion generation a credible product choice rather than a research curiosity. Its published speed and price could improve agents, search and voice systems where every model call adds delay.

But a fast architecture has not automatically won the market. The 1,068-token figure, quality comparisons and production examples are still primarily vendor evidence. Mercury 3 may change how LLMs generate text; the next step is independent proof that the speed survives real prompts, real providers and real failure costs.