The category, explained
What is a decision model?
A model that answers with a value from a type you declared, plus a calibrated probability across that type. No prompt, no parsing, and a confidence number you can actually route on.
Definition
A decision model takes a document and a set of questions, and answers each one with a value drawn from a type supplied by the caller, returning a probability distribution across that type rather than a sample from it.
Three things have to be true at once before the label means anything, and plenty of things being marketed as decision models satisfy only the first.
The output space is closed and declared by the caller. The set of valid answers arrives in the request, not in the weights. The model cannot return anything outside it, not because it was asked nicely but because there is nowhere else to go.
The model returns a distribution, not a pick. Probability mass across every option, so you can see the difference between a model torn between two answers and a model with no idea. Those two failures want different fallbacks, and a single label plus a confidence score collapses them into one.
The probability is calibrated well enough to threshold on. This is the one that gets skipped. A confidence number that does not track the real probability of being right is decoration. If the model says 0.9 on a hundred cases, roughly ninety of them had better be right, or every downstream automation rule you build on it is quietly wrong.
The three primitives
Every decision in this shape reduces to one of three calls. The interface is deliberately small, and the smallness is the product rather than a limitation of it.
choiceOne option from a closed set the caller supplies.
Which queue does this ticket belong in. Which of six tools should the agent call next. The set of valid answers is an argument to the call, so it can change per request without retraining anything.
billing · technical · account · abuse · other
scoreA bounded number.
How severe is this security incident. How confident are we that this line item matches the purchase order. The bound is declared, so the answer cannot arrive as prose that has to be turned into a number.
0.0 to 1.0, or 1 to 5
noulA yes or no.
Does this invoice need a human to approve it. Did the agent actually complete the step it claimed. The degenerate case of choice, common enough in real workflows to be worth its own primitive.
true · false
What calibration actually buys you
This is the part that turns a benchmark score into an operational decision, and it is the reason the third condition above is not optional.
Take a real example. We ran Jev against a frozen 2,000-decision test split and it scored 0.740 overall. On its own that number cannot tell you whether to automate anything, because it averages the cases the model was sure about with the ones it was guessing at.
Its expected calibration error was 0.045, which is low enough that the confidence figure can be trusted. That lets you abstain below a threshold and look at what is left:
| Confidence threshold | Coverage | Accuracy within that coverage |
|---|---|---|
| None | 100% | 0.740 |
| 0.7 and above | 57.6% | 0.864 |
| 0.9 and above | 24.5% | 0.937 |
One model, three different products. Automate everything and accept a 26% error rate. Automate the confident quarter at 6% error and route the rest to a person. Which row you pick is a business question about your error budget, and the model cannot answer it for you. What it can do is make the rows trustworthy, and that is exactly what calibration is for. Full method and confidence intervals are on the Jev model page.
What a decision model is not
Not a small LLM
Size is not the distinction. An LLM with constrained decoding also produces a typed value. The distinction is that a decision model returns a distribution across the whole option set, and that the distribution is calibrated well enough to set a threshold on. Most constrained LLMs give you the option and a token probability, which is not the same thing.
Not a classifier
A classifier has a label set fixed at training time. A decision model takes the label set at call time. That is what lets one model serve forty different routing decisions across your product without forty training runs, and it is also why classifier benchmarks do not transfer.
Not a correctness guarantee
A typed interface removes the parsing failure and the invented option. It does not remove the wrong answer. What it adds is that wrong answers arrive with a number attached, so you can decide in advance which ones you are willing to act on unsupervised.
Where the category came from
The first commercial model of this shape shipped on 15 September 2026, exposing one endpoint and the three primitives above. Adoption was immediate enough to be worth noting: OpenRouter recorded 334 billion tokens through it inside four days.
What happened next is the more interesting part. Open-weight reproductions began appearing within days, on bases ranging from a 151M encoder to a 4B decoder, and a community benchmark set converged almost immediately so that everyone was at least reporting against the same 2,000 decisions. By the end of that week there were ten models worth tracking and, as far as we can find, not one independent measurement of any of them.
That gap is the whole problem. Every number in circulation right now was published by the people who trained the model. We keep the model directory specifically to say, per model, whether anyone has checked.
Four questions that separate a real one from a wrapper
Useful when a vendor tells you their product is a decision model, and equally useful on an open-weight repo with an impressive README.
Does the option set come from the caller?
If the labels are baked in at training time, you have a classifier with a new name. Ask whether you can change the option set in the request body without retraining.
Does it return the full distribution, or one label?
A single label plus a confidence number is much weaker than probability mass across every option. The full distribution is what tells you whether the model was torn between two plausible answers or had no idea at all, and those two cases want different fallbacks.
Is calibration published, with the binning scheme?
Expected calibration error is the number that decides whether the confidence is worth routing on. It is also trivially manipulated by the choice of bins, so a figure published without its method is not checkable. We ran into exactly this and could not reproduce a third-party ECE figure for Jev from its public description.
Is there a risk and coverage curve?
Headline accuracy answers a question nobody operationally has. What you need is accuracy at each coverage level, because the decision you are actually making is how much of the queue to automate and where to stop.
Common questions
Decision models, briefly answered
The questions we get asked most often about the category. Longer answers live in the pages linked from each one.
Is a decision model just an LLM with JSON mode turned on?
Not quite, though the gap is narrower than vendors imply. Constrained decoding does give you a typed value out of a general model. What it does not reliably give you is a calibrated probability across the option set, because a token probability is not an estimate of whether the decision is right. The honest comparison runs a general model through the same typed interface and measures both, which is what we intend to do rather than take either side’s word for it. More on this in decision models vs LLMs.
What does "typed" actually mean here?
That the shape of the answer is declared by the caller before the model runs, and the model cannot return anything outside it. Not a schema the model is asked to follow, which it may or may not; a space it is structurally unable to leave.
Why does calibration matter more than accuracy?
Because accuracy is a single number about a whole population and your automation decision is per case. If confidence is calibrated you can abstain below a threshold and hand the rest to a human, which converts one accuracy figure into a curve you can pick a point on. On our own Jev run, 0.740 overall became 0.937 on the quarter of decisions where the model was most confident.
Are there open-weight decision models?
Yes, and they appeared within days of the first commercial release. We track ten of them in the model directory, with base model, licence and an explicit note on each about whether anyone has actually measured it. At the time of writing, one has.
How new is all of this?
Very. The first commercial decision model shipped on 15 September 2026 and the open-weight wave followed inside a week. Treat every published number in this category, including vendor benchmarks, as unreplicated until someone outside the lab has run it.
The threshold is the deliverable
Knowing what a decision model is does not tell you where to set yours. That comes out of your own historical decisions, frozen into a test set and run against the candidates.