8 projects · checked 2026-09-20
Evals and benchmarks for decision models
Head-to-head races, task suites and benchmark harnesses. Read the method before the headline: most of these compare on speed and cost, and very few report calibration.
7 of these 8 call the hosted API rather than running open weights.
Other things people build
Developer tooling 35
Code review gates, CI checks, agent harnesses and editor plugins. The densest cluster in the ecosystem, because a coding agent makes hundreds of small decisions per session and each one costs something.
Browser 18
Extensions and automation that judge pages: what to hide, what to click, what is worth reading. Latency matters here in a way it does not on a server.
Open reimplementations 15
People rebuilding the category from open weights: one-pass scorers, constrained decoding over local models, and small models trained to return a calibrated distribution. The fastest-moving cluster here.
Content 10
Moderation, summarisation triage, voice and video. Judging content rather than producing it.
Find out what your data actually supports
We turn your historical decisions into a frozen test set, then tell you which model, which threshold, and how much of it you can safely automate.