Built with Evals and benchmarks
jevals
Agent evals and guardrails as Jev decisions: one request per trace, a fraction of a cent, fast enough for the agent loop. Runs locally with Kev or Laya.
110 ★ on GitHub
What it is
Evals and guardrails for agents, using Jev-style decision models instead of an LLM judge. All the evals for a trace go out as one request that costs a few thousandths of a cent and comes back in a few hundred milliseconds, so you can run them on every trace and inside the agent loop.
Getting started
From the README:
pip install jevals
pip install "jevals[pii]" # Presidio, for PII / PHI entity detection
pip install "jevals[mcp]" # MCP server, so Cursor / Claude Code / Copilot can run and write evals
pip install "jevals[openai-agents]" "jevals[langgraph]" "jevals[claude]"Where the decision sits
jevals calls the hosted decision API rather than running its own weights. That keeps the project small and means its behaviour moves whenever the vendor ships a new model version. It also means an API key and a network round trip sit in the path of every decision.
Models it references: Jev, Alpha, laya-mlx, laya.
How active it is
Created 2026-09-20, last pushed 2026-10-15, 110 stars and 11 forks when we read it on 2026-11-02. Everything in this category is weeks old at most, so a stale push date means something different here than it would on a mature project. Check the repository before you depend on it.
Find out what your data actually supports
We turn your historical decisions into a frozen test set, then tell you which model, which threshold, and how much of it you can safely automate.