Built with Evals and benchmarks
jev-benchmarks
Probability-aware evaluation for typed decision models: calibration, selective risk, latency, and reproducible benchmarks.
27 ★ on GitHub
What it is
Probability-aware evaluation for typed decision models.
Getting started
From the README:
git clone https://github.com/AbdelStark/jev-benchmarks.git
cd jev-benchmarks
uv sync --extra benchmark --devWhere the decision sits
jev-benchmarks calls the hosted decision API rather than running its own weights. That keeps the project small and means its behaviour moves whenever the vendor ships a new model version. It also means an API key and a network round trip sit in the path of every decision.
Models it references: Jev, jev.
How active it is
Created 2026-09-17, last pushed 2026-09-17, 27 stars and 5 forks when we read it on 2026-11-02. Everything in this category is weeks old at most, so a stale push date means something different here than it would on a mature project. Check the repository before you depend on it.
Find out what your data actually supports
We turn your historical decisions into a frozen test set, then tell you which model, which threshold, and how much of it you can safely automate.