Built with Evals and benchmarks
decision-index
Decision Index: reproduce the Jev decision-model benchmark suite locally or as one HF Job
83 ★ on GitHub
What it is
Reproduction kit for the Decision Index, a benchmark for typed decision engines: models that take a state and a set of typed questions (multiple-choice choice or yes/no noul primitives with explicit criteria) and return one answer per question with a probability for every supplied option. It lets anyone rebuild the…
Getting started
From the README:
pip install -e ".[transformers,rebuild]"
export HF_HUB_DISABLE_XET=1
python -m decision_index suite rebuild --work work
python -m decision_index suite import \Where the decision sits
decision-index calls the hosted decision API rather than running its own weights. That keeps the project small and means its behaviour moves whenever the vendor ships a new model version. It also means an API key and a network round trip sit in the path of every decision.
Models it references: Jev.
How active it is
Created 2026-09-22, last pushed 2026-10-20, 83 stars and 144 forks when we read it on 2026-11-02. Everything in this category is weeks old at most, so a stale push date means something different here than it would on a mature project. Check the repository before you depend on it.
Find out what your data actually supports
We turn your historical decisions into a frozen test set, then tell you which model, which threshold, and how much of it you can safely automate.