Decisions API Scores 1,000 Questions—Can Its Confidence Travel?

A decision endpoint, with a boundary

Google introduced the Gemini Decisions API on November 11, making it a fresh implementation candidate for developers evaluating multimodal features this week. Powered by Gemini Decision 1, it accepts text, images, audio, video and PDFs, and evaluates up to 1,000 independent questions against shared input. The headline feature is bounded judgment: yes/no checks, choices from supplied options and scores against an ordered rubric. Google’s announcement describes an endpoint that scores candidates rather than composing an answer token by token.

The interesting builder question is whether those scores remain useful when the evidence gets messy. A clean product photograph and a straightforward caption make an agreeable demo. A customer claiming damage while uploading a photograph of the wrong parcel makes a better evaluation case. The API offers a way to express the decision; the application still needs a precise definition of the evidence that would justify it.

This article examines that implementation boundary, rather than repeating the API’s token-price pitch. The practical opportunity is a reusable decision layer for multimodal applications. The test is whether it can distinguish a supported conclusion from an attractive guess on the particular material your users submit.

Confidence needs a local meaning

Calibration describes the relationship between predicted probabilities and observed outcomes. A system can choose the correct category relatively often while overstating how certain it is. Guo and colleagues’ research, On Calibration of Modern Neural Networks, documents this distinction and investigates methods for improving probability estimates. That paper supplies background, not an independent evaluation of this new Google endpoint.

For a builder, the useful question is whether cases assigned similar probabilities resolve as expected on held-out traffic. That requires labels and enough examples to see where confidence becomes misleading. Separate clean photographs from blurred ones, supported languages from unfamiliar ones, and complete documents from partial uploads. A single aggregate result can conceal the slice that generates your support backlog.

The Decisions API documentation distinguishes answer values from confidence. For a yes/no check, the returned value represents the probability of yes; confidence corresponds to the more probable outcome. A low probability of yes can therefore accompany a confident no. Mixing up those fields would turn an otherwise sensible review rule inside out.

Keep that distinction visible in application code. A check asking whether damage is visible needs a rule about the probability of damage, while a rule deciding whether to defer needs a measure of uncertainty. Name variables for those meanings. A generic field called score is an invitation for a future maintainer to automate the opposite of what you intended.

Make conflicting evidence the first test

Consider a proposed returns-triage application. The shared input contains a customer message, a photograph and an order record. Treat this as a design example, not a reported Google deployment. Ask narrow questions about visible damage, whether the supplied material supports identifying the purchased item, and whether the submission contains enough evidence for the next step.

Now construct matched cases. Keep the photograph fixed while changing the message from a neutral description to an emphatic damage claim. Keep the message fixed while replacing the photograph with an unrelated item. Remove the order record. Include a screenshot containing instructions to approve the return. Have reviewers label these cases using the same written policy before comparing the endpoint’s answers.

These tests examine different failure modes. A change driven by the customer message may be appropriate for a question about the customer’s complaint, but inappropriate for a question specifically about visible cracks. An unrelated photograph may support a confident description of an object while providing no support for a claim about the purchased item. The question’s wording determines which behavior counts as success.

Write evidence requirements into the task definition. If the check concerns visible physical damage, distinguish that from a customer’s allegation. If identifying the item requires a readable identifier, specify that requirement. Otherwise, the evaluation cannot tell whether a disagreement came from visual interpretation, missing evidence or an ambiguous instruction. A probability attached to an unclear proposition is still unclear.

One request is not a chain of reasoning

The documentation says questions are evaluated independently against the shared input and cannot see one another’s answers. That matters when designing a workflow: a question about the next action does not automatically inherit the result of an earlier damage check. Google recommends either making a subsequent request for dependent decisions or discarding an answer that does not apply. The multiple-question guide makes this limitation explicit.

For the returns example, collect evidence judgments together, then combine them in ordinary application logic. If the photograph is unrelated to the order, a visible-damage result need not advance the claim. If required evidence is missing, route the submission to a request for clarification. The action rule becomes inspectable instead of being buried inside a broad instruction to decide what to do.

Also distinguish API independence from statistical independence. Separate questions can all rely on the same misleading photograph or ambiguous sentence. Agreement among their answers does not establish independent corroboration. Do not multiply their probabilities as though each answer came from a separate witness. A shared-input endpoint makes collecting judgments convenient; it does not manufacture additional evidence.

Keep a compact decision record: the relevant input reference, question definitions, returned values, application rule version and resulting action. This is a proposed implementation practice, not a feature claim. When a reviewer overturns a decision, that record helps locate the defect. You can change the evidence question, the action rule or the review threshold without treating the entire system as one mysterious prompt.

Compare it with the classifier you already have

Google’s structured-output documentation describes generating responses that conform to a JSON Schema, including predefined classification categories. It also directs developers toward Decisions API for closed-ended probabilities and rubric scores. The distinction is useful: obtaining a valid category-shaped response and obtaining a probability suitable for an action threshold are separate requirements.

Build the comparison around an existing product decision. Use identical held-out submissions for the current classifier and the proposed endpoint. Preserve the same evidence rules and action choices. Judge the resulting workflow by incorrect automatic actions, unnecessary escalations and unresolved cases. If one system can abstain while the other must always choose, account for that difference before declaring a winner.

Choose review thresholds using examples kept separate from the final evaluation set. For each proposed threshold, record how much work proceeds automatically and which mistakes survive. A setting that looks good on easy submissions may send every difficult case to staff. That can be a reasonable product choice, but it needs to be visible in the result rather than disappearing behind a headline success rate.

Measure the full path as well. Google reports end-to-end latency of 150–250 milliseconds under its described test conditions; that is a vendor measurement, not a delivery promise for your application. The launch post ties its performance discussion to specific context lengths and question counts. Your evaluation should include media preparation, network time, retries and any subsequent decision request.

Start where an incorrect choice is recoverable

A useful first trial is routing incoming submissions to review queues while leaving the existing workflow in charge. Run the proposed decision layer alongside that workflow and compare its recommendations with reviewer outcomes. Include the awkward submissions deliberately. Otherwise, a quiet trial may simply mean the difficult cases never reached the experiment.

Keep the candidate actions honest. A menu containing only approval and rejection forces missing evidence into a substantive verdict. An application can instead define a clarification path or defer when its evidence checks fail. Whether that path is triggered by an explicit option or application logic, evaluate it as a real outcome with a real handling cost.

The launch makes multimodal decision scoring worth testing now. Shipping it requires a narrower finding: on a named task, with specified evidence and a documented fallback, the resulting workflow meets your error and handling requirements. That is a result a builder can defend. A confidence field, however polished the demo, cannot defend itself.

Frequently asked questions

What does the Gemini Decisions API return?

It returns bounded decisions: yes/no probabilities, selections from defined choices, or scores against an ordered rubric, with confidence information.

Does a high confidence score guarantee a correct decision?

No. Validate confidence against labeled examples from your own traffic, including incomplete and conflicting multimodal inputs.

Can questions in one request use each other’s answers?

No. They are evaluated independently against shared input; combine results in code or make a subsequent request for dependent decisions.