GGUF · 7 models available
Run a decision model with llama.cpp
GGUF is the format that made local inference ordinary. For decision models it matters more than usual, because the whole argument for the category is cost per call.
How to run it
Install a runtime
llama.cpp is the reference implementation. Ollama wraps it if you would rather not build anything.
brew install llama.cppPull a model straight from Hugging Face
The -hf flag downloads and caches the GGUF without a separate step.
llama-cli -hf <owner>/<repo>Watch memory, not disk
A quantised 400M encoder is a few hundred megabytes on disk and runs comfortably on a laptop. The 2B and larger decoders are where you start caring about RAM.
Models available in GGUF
Every entry in the directory that ships this format, read from its repository metadata on 20 September 2026. None has been benchmarked by us.
decider-2b-GGUF
A GGUF build of decider-2b, for running typed decisions on llama.cpp instead of a hosted API.
decider-2b-vision-GGUF
A GGUF build of decider-2b-vision, for running typed decisions on llama.cpp instead of a hosted API.
laya-GGUF
A GGUF build of laya, for running typed decisions on llama.cpp instead of a hosted API.
laya-multilingual-GGUF
A GGUF build of laya-multilingual, for running typed decisions on llama.cpp instead of a hosted API.
decider-0.8b-GGUF
A GGUF build of decider-0.8b, for running typed decisions on llama.cpp instead of a hosted API.
laya-typed-decisions-GGUF
A GGUF build of laya-typed-decisions, for running typed decisions on llama.cpp instead of a hosted API.
MobiMind-Decider-7B-GGUF
A GGUF build of MobiMind-Decider-7B, for running typed decisions on llama.cpp instead of a hosted API.
Also browsable as a facet of the directory: GGUF builds.
Other ways to run one
MLX
MLX uses unified memory on M-series chips. For a model in the few-hundred-megabyte range this is the fastest local option on a Mac, and it needs no GPU of its own.
ONNX
An ONNX export loads without PyTorch. That matters for slim containers, for edge devices, and for running a decision inside a browser.
Core ML
Core ML puts the model on the Neural Engine, so the decision happens on the phone or the laptop and nothing leaves it. The strongest privacy story in the category.
Compatible servers
16 open projects that reimplement the decision API, if you want the interface rather than just the weights.
Find out what your data actually supports
We turn your historical decisions into a frozen test set, then tell you which model, which threshold, and how much of it you can safely automate.