laya-multilingual
The only multilingual decision model we have found, a 322M encoder on mmBERT. Also the one our English-only test set cannot say anything useful about.
Community card
What it is
The Laya family’s multilingual branch, built on mmBERT at 322M parameters rather than the
ModernBERT-large base used by laya and
laya-typed-decisions. It is the only model in this directory
that targets non-English input at all.
Why we are not going to rank it
The benchmark everyone in this wave reports against, LocalLLaMA/typed-decisions, is English.
Running a multilingual model on an English split measures the English half of it and tells you
nothing about the capability it was built for. We could publish that number. It would be
accurate, comparable, and misleading, which is the worst combination available.
Evaluating this properly needs a decision set in the target languages with labels produced in those languages, not translated from English. Translated test sets leak the source language’s phrasing into the task and quietly make it easier.
A public number is a shortlist, not a decision
The only benchmark that settles which model you ship is one built from your own historical decisions. We freeze that test set, run the candidates against it, and hand back the threshold your error budget supports.