A faster draft matters only if it becomes a correct result sooner. That is the useful way to read Mercury 3: its parallel text generation is interesting, but the person waiting for an order update cares about the whole task, not the model’s token counter.
Mercury 3 is Inception’s October 19 diffusion language model. This guide explains how that generation method works, why the company’s reported 1,068 tokens per second is not the same as application response time, and how to test speed, correctness and cost together.
First, understand what the model is generating
A token is a unit of text used by a model: it might be a whole word, part of a word or punctuation. A large language model (LLM) uses the supplied context to generate a response. A token rate describes text production; it does not tell us whether the response is useful.
In ordinary autoregressive generation, each next token is predicted using the text already available. Think of extending a sentence from its current end. This dependency is real, but “500 output tokens always require exactly 500 full-model passes” is too strong: speculative decoding research shows how a system can propose and verify several tokens together.
A diffusion LLM instead learns to refine a noisy text representation over repeated steps, updating several positions in parallel. “Denoising” here means improving those candidate text positions, not cleaning microphone noise. Inception’s 2025 Mercury report explains this family-level principle; it describes Mercury Coder, not a complete specification of Mercury 3’s private serving engine.
Read Figure 1 from one state to the next. On the right, several positions can change within a refinement step. The diagram uses words so the relationship is easy to see; actual token boundaries need not match them.
The opportunity is to do more useful generation work together on a graphics processing unit (GPU), rather than waiting on every next position separately. The trade-off still involves computation, refinement steps and output quality. Parallel does not mean the whole answer appears instantly, nor that every prompt receives the same speedup.
What changed with Mercury 3?
Inception’s launch announcement reports 1,068 output tokens per second on NVIDIA GPUs and stronger results than GPT-6 Luna and Claude Haiku 5.5 across its reported comparisons. These are vendor-reported results, not measurements made for this article. A benchmark result belongs to its task set, reasoning setting and serving conditions; it is not an application-level service guarantee.
The current model documentation lists a 1M-token context window, up to 131,072 output tokens, text input/output, tool calling and structured outputs. Mercury 2.5 is listed with 260K context and 65,536 maximum output tokens. Context is the model’s working text budget, not proof that it reliably finds every fact in a million-token prompt. Do not interpret these two ceilings as permission to supply 1M input tokens plus the full output maximum simultaneously; confirm how the endpoint enforces the request budget.
Access is through Inception’s API and the partners named in its announcement, including OpenRouter, Baseten and Vercel. API compatibility can reduce integration work, but it does not make route limits, defaults, supported parameters or data handling identical. This article evaluates that hosted workflow rather than assuming that local model weights or a particular enterprise deployment are available.
Separate token speed from the time a user waits
Throughput is an amount of output per unit of time. Latency is elapsed waiting time for a defined event. Time to first token measures the first visible output; time to a complete valid result includes enough output to use. Neither necessarily includes the database lookup, authorization check or retry that finishes the user’s task.
For a simple serial workflow, estimate elapsed time as other work plus output tokens divided by the assumed generation rate. In Figure 2, five model turns with 400 output tokens each total 2,000 tokens. Assume four seconds for all other serial work. At 80 tokens/s, generation takes 25 seconds; at 300 tokens/s, about 6.67 seconds. Total estimates are 29 and 10.67 seconds. These rates are chosen for the example, not assigned to Mercury 3 or a competitor.
Figure 2 also shows the opposite case: a short 25-token tool proposal waiting on the same four seconds of other work. Read the total, not just the shrinking teal segment.
For that short case, estimated total time only falls from 4.31 to 4.08 seconds. The practical question is therefore which part of our waiting time can this model change? A longer report or repeated generation-heavy loop is a different workload from a tiny lookup request. Real timing also includes queueing, prompt processing, reasoning and network variation; those must be measured rather than inferred from this simplified formula.
A gateway measurement and Inception’s headline may use different output lengths, concurrency and timing definitions. A lower observed rate alone does not establish that routing caused the gap. Record the route, model version, reasoning effort, input/output sizes, concurrency and measurement window before comparing results.
Streaming means either appending chunks or replacing a draft
Streaming sends pieces of a response while generation continues. Inception’s streaming guide documents two behaviors: with stream: true, the interface appends refined text blocks; adding diffusing: true exposes intermediate full-text snapshots. The latter should replace the displayed buffer, not be appended as though each event contained only new words.
Figure 3 follows the text shown on screen. It explains a frontend state update, not two different permission levels or guaranteed latency classes.
For a reader-facing demonstration, showing refinement may be useful. For a backend expecting a complete JSON object, intermediate text is not a ready-to-execute action. JSON is a text format for named values and lists; a schema specifies the expected fields and types. Finish receiving the proposal, parse it, check its structure and verify business rules before doing work.
A support example: fast text still needs a verified order state
Consider a fictional customer asking, “Why is my replacement order still pending?” The model can propose a check_order function call. The application—not the model’s wording—must verify that the order belongs to the authenticated customer before querying it. Inception’s tool-use documentation describes returning tool-call arguments, executing the function in the application and sending its result back.
Suppose the backend returns {"replacement_status":"manual_review_required"}. A helpful next answer explains that state and offers the allowed review path. It should not claim that the replacement shipped, or promise a 24-hour resolution, unless the service actually provides that fact. If the next action changes an order, apply the required approval separately.
In Figure 4, follow the message across each actor and then downward. The permission gate belongs to the application; it is not an extra reasoning prompt that the model can approve for itself.
Two kinds of parallelism should stay separate. Parallel token refinement happens inside generation. Parallel tool execution is an application choice when operations are independent. Fetching two unrelated read-only records can potentially overlap; a refund that depends on an order check must wait for that check and its authorization. Faster generation does not remove that dependency.
Structured outputs can constrain the response shape, but a well-formed object can still name the wrong order or request an impermissible action. Count both schema validity and correct task completion in the evaluation.
Keep launch prices separate from accepted-task cost
The October 20 Inception rate card shows launch prices of $0.10 per million input tokens, $0.01 for cached input and $0.35 for output. The displayed regular rates are $0.20, $0.02 and $0.70 respectively. Confirm the effective rate and any route-specific charges when budgeting; a promotion should not silently become a permanent assumption.
For a worked example, assume 100 tasks, one request each, with 10,000 uncached input tokens and 1,000 billed output tokens per request. There are no retries or cache hits. Token spend is $0.135 at launch rates or $0.27 at regular rates. Now assume only 80 tasks meet the acceptance criteria.
Figure 5 connects the billing ledger to the acceptance ledger. Failed tasks remain in the numerator because their requests still cost money.
The corresponding token cost per accepted task is about $0.00169 or $0.00338. These are estimates for the stated workload, not observed invoices. For full operating cost, add billed retries, external tools, infrastructure and human review before dividing by accepted tasks. Log the provider’s reported usage rather than treating visible answer length as the entire bill.
A practical evaluation before switching traffic
Begin with a small sanitized test set that covers the actual decisions: short lookups, long documents, malformed inputs and requests that must be refused or escalated. Define correct output and acceptable wait time first. A 30–50-case pilot can expose integration faults, but it is not enough by itself to establish rare-failure rates or production tail latency.
- Freeze the comparison. Record prompts, tool schemas, route, model identifier, token budgets and concurrency. Use equivalent tasks, not equal token counts as a substitute for equal correctness.
- Set reasoning effort explicitly. The reasoning guide lists
instant,low,mediumandhigh; Mercury 3 defaults tolow, unlike the documentedmediumdefault of Mercury 2.5. A silent default change can confound a migration comparison. - Measure useful completion. Capture first output, complete valid object, tool completion and final accepted task. Report median and a slow-tail percentile with sample counts, alongside errors and retries.
- Test difficult context and denied actions. Include conflicting documents, missing facts and unauthorized order identifiers. Large context and schema compliance do not replace factual or permission checks.
- Release narrowly. Compare total accepted-task cost and user wait, then start a limited rollout with logs and a rollback path. Expand only if the quality and timing targets hold.
Figure 6 makes the decision explicit: a speed win does not compensate for a failed quality gate. If results disagree, diagnose the stage that changed before moving more traffic.
Practical questions
Does a diffusion model generate the whole answer in one step?
No. The useful distinction is parallel refinement over steps, not zero-step generation. The conceptual sequence in Figure 1 is an explanation, not a disclosed Mercury 3 iteration schedule.
Is the 1,068-token rate the time to first response?
No. It is Inception’s reported generation rate. Measure first output and task completion separately under your own route and workload.
What is the strongest reason to try Mercury 3?
A workflow that spends a large share of its time generating text may benefit from a faster generation method. The reason to adopt it is stronger: the same workload must finish correctly sooner at an acceptable total cost. That is the distinction this evaluation is designed to test.
Official technical sources
- Mercury 3 launch announcement — release and attributed vendor claims.
- Model specifications and current pricing — limits, formats and rate card checked October 20, 2026.
- Streaming and diffusion modes, tool calling, structured outputs and reasoning controls — integration behavior.
- Mercury research report (2025) — family-level mechanism; speculative decoding research — why sequential prediction does not imply one target-model pass per output token.