18 projects · updated 2 November 2026
Evals and benchmarks for decision models
Head-to-head races, task suites and benchmark harnesses. Read the method before the headline: most of these compare on speed and cost, and very few report calibration.
16 of these 18 call the hosted API rather than running open weights.
Other things people build
Developer tooling 84
Code review gates, CI checks, agent harnesses and editor plugins. The densest cluster in the ecosystem, because a coding agent makes hundreds of small decisions per session and each one costs something.
Open reimplementations 80
People rebuilding the category from open weights: one-pass scorers, constrained decoding over local models, and small models trained to return a calibrated distribution. The fastest-moving cluster here.
Agents 30
Tool selection, routing and step gating. An agent choosing its next action is the purest version of the problem this class of model exists for.
Language integrations 30
Libraries that make a typed decision feel native to a language: pattern matching on an answer in Elixir, a probabilistic if in Ruby, a typed method via BAML.
Find out what your data actually supports
We turn your historical decisions into a frozen test set, then tell you which model, which threshold, and how much of it you can safely automate.