system-one-qwen3.5-4b-scorer
The largest open model in the directory and the only one aimed at a single primitive, score. Licensed cc-by-nc-4.0, which rules it out of commercial use regardless of how it benchmarks.
Community card
Read the license first
cc-by-nc-4.0 is non-commercial. If you are evaluating decision models to put one into a
product, this one is out, and no benchmark result changes that. We list it because a directory
that quietly drops models on licence grounds is a directory you cannot trust to be complete, and
because the licence is exactly the kind of detail that gets discovered three weeks into an
integration.
Everything else in this directory is apache-2.0 or MIT, with the exception of a few weights that inherit Gemma terms from their base.
What it is
A 4B Qwen3.5 fine-tune, the largest open model we track, and the only one that targets one
primitive rather than the full set. score returns a bounded number rather than a choice from a
closed set, and it is the primitive general models handle worst: they cluster on round values,
they drift with phrasing, and the error does not show up as a wrong label, only as a number
that is off.
Our own Jev run bears out that score is the hard one. It came in at 0.705 against 0.738 for choice and 0.787 for noul, the weakest of the three.
A public number is a shortlist, not a decision
The only benchmark that settles which model you ship is one built from your own historical decisions. We freeze that test set, run the candidates against it, and hand back the threshold your error budget supports.