Category comparison
Decision models vs LLMs
The difference is not typed against untyped, because a constrained LLM is typed too. It is what comes back alongside the answer, and whether you can trust it enough to automate on.
The short version
If you only need a typed answer, constrained decoding on a general model gets you most of the way. If you need to know which answers to trust without a human looking at them, you need a calibrated probability across the option set, and that is the thing decision models are built and measured to provide.
Almost every comparison published between these two is measuring the harness, not the models.
The usual setup runs a decision model through its native interface and a general model through a prompt the author wrote themselves, then reports the gap. That gap contains the model difference, the prompt quality, the parsing strategy, the retry policy and the choice of what counts as a failure. Change the prompt and the result moves. Nobody publishes the prompt.
There is a clean way to do it. TypeSafe open sourced an adapter that constrains an arbitrary LLM to the same typed interface, which removes the prompt from the comparison entirely and makes the two sides answer the same question in the same shape. That is how we intend to run our baseline. We have not run it yet, so every claim about general models on this page is qualitative, and the table below has no numbers in its right-hand column on purpose.
Where they differ, axis by axis
Figures in the left column are from our own run against a frozen 2,000-decision split. The right column is descriptive, because we have not measured it.
| Dimension | Decision model | General LLM, constrained |
|---|---|---|
| Where the option set comes from | The request body. It can differ on every call without retraining. | The prompt, or a schema passed to constrained decoding. Also per-call, so on this axis there is no real difference. |
| Can the output leave the space | No. There is nowhere outside the type to go. | Not under constrained decoding. Under plain prompting plus parsing, yes, and that is where most production incidents come from. |
| What comes back alongside the answer | Probability mass across every option, plus a confidence value. | A token probability for the sampled path. Related to confidence, but not an estimate of whether the decision is right. |
| Is the confidence calibrated | It is the central design claim, and it is measurable. We measured ECE 0.045 on Jev. | Rarely reported. Instruction tuning is known to push general models toward overconfidence, so this needs measuring per model rather than assuming. |
| Prompt sensitivity | No prompt to be sensitive to. The questions are passed through as fields. | Real, and awkward to control for. Two teams comparing the same model can get different numbers from wording alone. |
| Latency shape | One round trip, no generated tokens to wait on. We saw P50 687 ms and P95 777 ms end to end from a client, for five decisions per request. | Scales with output length. Short constrained outputs are fast, but reasoning traces are not, and the two are often conflated in benchmarks. |
| Cost shape | Input tokens only on the one we measured, at $0.042 per MTok with output free. 2,000 decisions cost us $0.0159. | Input plus output, and output is usually the expensive side. A reasoning model that thinks before answering pays for every thought. |
| What it is bad at | Anything open ended. It cannot explain itself, draft a reply, or handle a decision you could not enumerate in advance. | Nothing categorically, which is the appeal. The cost is that it is rarely the cheapest or most predictable way to do a decision you could enumerate. |
| What a wrong answer looks like | A valid option that is wrong, with a probability attached. Catchable by threshold. | The same, under constrained decoding. Without it, also malformed output, refusals and invented options, which fail differently and need different handling. |
Where the general model genuinely wins
A site that sells decision model benchmarks should be able to say this plainly, so here it is.
The output is genuinely open
Drafting the reply, summarising the thread, writing the incident note. If the answer cannot be enumerated, a closed type is the wrong tool and no amount of calibration helps.
You cannot specify the task yet
Early product work where the decision space is still moving. Enumerating options is itself a design commitment, and a general model lets you defer it while you find out what the categories actually are.
The reasoning is the deliverable
When a person has to read why, not just what. A decision model returns a number and a label; it has nothing to say about how it got there.
Where the decision model wins
All three have the same shape: the decision is enumerable, it happens constantly, and being wrong costs something specific.
High volume, fixed decision space
Ticket routing, tool selection, relevance filtering, approval gating. Millions of calls against a stable option set is where the cost and latency shape starts to dominate everything else.
You need to route on confidence
Any workflow where some cases go to automation and the rest go to a person. That split is only as good as the calibration behind it, and this is the class of model built to be measured on exactly that.
The decision sits inside a loop
An agent choosing its next tool makes that decision on every step. Latency and cost compound in a way they do not for a single user-facing call, and a generated reasoning trace per step is expensive.
The number that should decide it
Not accuracy. Accuracy is one number about a whole population, and the choice in front of you is per case: this ticket, automate or escalate. The number that answers that is accuracy at a given coverage, and it only exists if confidence is calibrated.
Concretely, from our Jev run: 0.740 accuracy across everything, 0.864 across the 57.6% of decisions above 0.7 confidence, 0.937 across the most confident 24.5%. If your error budget is 10%, the second row is a product and the first is not. A comparison that reports only the headline cannot tell you which model clears your budget, and that is the only question that matters when you are choosing one.
Full method, confidence intervals and the teacher-agreement caveat are on the Jev model page. Background on why the category works this way is in what is a decision model.
Common questions
Choosing between the two
What people ask when they are deciding whether to swap a general model call for a decision model, including the ones we cannot yet answer with our own numbers.
Can I just use constrained decoding on a general model instead?
Often yes, and anyone selling you a decision model should be able to say why not for your case. Constrained decoding closes the output space, which is most of the benefit. What it does not give you for free is a calibrated probability across the options, and that is the part worth testing before you decide. Run both through the same interface and compare accuracy at coverage, not accuracy.
Have you benchmarked an LLM baseline against a decision model?
Not yet, and we would rather say so than publish a comparison we do not trust. Doing it properly means putting a general model behind the same typed interface rather than a prompt of our own design, which TypeSafe’s own open-source adapter makes possible. Until that run exists, the LLM column on this page is deliberately qualitative.
What about the speed and cost multipliers in the vendor announcement?
Those figures (193.6 times faster, 444.6 times cheaper) come from TypeSafe and we have not reproduced them. We can confirm the absolute numbers we saw ourselves: 2,000 decisions in 34 seconds for $0.0159, at P50 687 ms per request from a client. What the comparable general model would have cost on the same split is exactly the measurement we have not made.
Does a decision model replace the LLM in my product?
Usually it replaces one call inside it. The common pattern is a general model doing the open-ended work and a decision model handling the routing, gating and filtering decisions around it, because those are the calls that run most often and matter most when they are wrong.
Settle it on your own data
We put the candidates behind one interface, run them against a frozen set of your historical decisions, and report accuracy at every coverage level so the comparison answers a question you actually have.