Jev alternatives
Laya
The 421M encoder everyone starred — and the one you fine-tune
Quick answer
Use Laya as a fast, CPU-friendly base you fine-tune on a few hundred of your own labelled examples — not as a zero-shot drop-in replacement for Jev. Its base checkpoints score near chance on typed decisions (0.362 against a 0.318 random baseline), and its ordinal `score` output has a measured option-position bias strong enough that it never once picked the first listed option in 300 Japanese test items.
Laya is the flagship open reproduction of Jev: a 421M ModernBERT-large encoder with a decision head, plus a 322M multilingual checkpoint that covers 100+ languages. It installs with a single pip command, answers choice / score / noul questions in one forward pass, and runs on a CPU. It is also the project whose own README is the most candid about what it cannot do.
Train a decision model
A small non-autoregressive encoder with decision heads. No text generation at all — the classic classifier shape, rebuilt for natural-language options.
Search aliases
Key specs
- Licence
- Apache-2.0
- Author
- Convai Innovations
- Backbone
- ModernBERT-large (421M) · mmBERT-base (322M) multilingual
- Latency
- 32.8–39.5 ms/question on a T4 · 7.2 ms for ten batched · 22 ms on an RTX 5090 · 66 ms for three questions on an M3 Pro CPU
- Languages
- 100+ with the multilingual checkpoint
- Wire format
- POST /v1/systemone via laya-serve
- Install
- pip install laya
Laya against Jev, on the numbers Laya itself publishes
Published figures, not our own run. TypeSafe’s Master Customer Agreement 2.3(b) forbids using Jev or its outputs to build similar products, so independent head-to-head runs are rare and every number here is a third-party claim.
| Laya against Jev, on the numbers Laya itself publishes | Laya | Jev |
|---|---|---|
| Base model, zero-shot | 0.362 / 0.342 (below the 0.461 majority baseline) | Not published for this split |
| Fine-tuned on the benchmark’s own train split | 0.766 | 0.727 (published) |
| 77 options in one question (Banking77) | 0.425 | 0.870 |
| Architecture | Encoder, 421M | Closed, hosted |
| Latency | Tens of milliseconds, on your own CPU or GPU | 70–500 ms over the network |
| Cost per call | Zero after hardware | $0.042 / MTok input |
When Laya is the right choice
Pick Laya when you have a few hundred labelled examples, a routing or moderation-shaped decision, and hardware you already own. It is tiny, it speaks 100+ languages, the multilingual router picks the right checkpoint in under a millisecond, and `laya-serve` exposes the same `/v1/systemone` contract, so the official TypeSafe SDK can be pointed at it. Fine-tuned on your data it is genuinely competitive — the honest caveat is that the headline number people quote is fine-tuned on the very benchmark it is measured on.
When to walk away
Do not ship Laya zero-shot, do not use it for ordinal `score` questions without measuring position bias on your own data, and do not use it for questions with dozens of options — the options share a fixed 192–256 token budget at three to four tokens per label. Recalibrate before trusting any probability it reports: on Japanese business email its `noul` answers scored below simply always answering "no", and its own model card records 51-language macro accuracy of 0.227, with Khmer reaching confidence 0.952 at accuracy 0.000.
Adopting Laya in three steps
import laya
agent = laya.load("convaiinnovations/laya")
state = {
"subject": "Duplicate charge on invoice #4411",
"body": "We were billed twice for March. Refund it today or we cancel our plan.",
}
questions = {
"department": {
"type": "choice",
"instructions": "Which team should handle this?",
"criteria": {
"billing": "invoices, payments, refunds",
"technical": "bugs, outages, system errors",
"sales": "pricing, new contracts",
},
},
"urgency": {
"type": "score",
"instructions": "How urgent is this?",
"criteria": ["not urgent", "soon", "blocking right now"],
},
"churn_risk": {
"type": "noul",
"instructions": "Does the user threaten to cancel?",
},
}
answers = agent.predict(state, questions)["answers"]
print(answers["department"]["choice"], answers["department"]["confidence"])
print(answers["urgency"]["score"], answers["churn_risk"]["noul"])Laya questions people actually ask
Is Laya a real drop-in replacement for Jev?
It is interface-compatible and architecturally the same idea, but it is not an accuracy drop-in. Its own README reports base checkpoints at 0.362 and 0.342 on typed decisions against a 0.318 random baseline and a 0.461 majority baseline. After fine-tuning on your labels it becomes competitive; out of the box it does not.
Why does Laya’s headline score beat Jev’s published number?
Because that 0.766 comes from `laya-typed-decisions`, a checkpoint fine-tuned on the training split of the very benchmark it is scored on, while Jev’s 0.727 is zero-shot. Read the two numbers as different experiments, not the same race. Independent 49-task testing puts open encoders far behind, with Laya at 0.583 against Jev’s 0.966.
Does Laya work in Japanese?
Partly. A 300-item Japanese business-email benchmark measured `laya-multilingual` at 0.747 on four-way routing, which is usable. The same run measured `score` at RPS 0.232 — worse than the majority baseline — and `noul` at 0.543 against 0.703 for always answering no. The cause was option-position bias: across five relabellings and reversals the first listed option was chosen in 0 or 1 of 300 items. Put the lowest ordinal label last, or do not use Laya for ordinal questions.
Does temperature calibration fix the biased probabilities?
It fixes the calibration error and nothing else. Fitting one temperature per (question type, option count) moved expected calibration error from 0.148 to 0.087, and for `noul` from 0.352 to 0.012 — while accuracy did not move by a single point. Temperature scaling changes how sharp a distribution is, not the ranking inside it, so it cannot repair a position bias.
What should I use instead of Laya?
If you want a faster, better-calibrated version of the same architecture, cbjev is fine-tuned from Laya and reports 0.2% answer flips on option reordering against Laya’s 7.8% — but note it is GPL-3.0. If you have no GPU, Von runs a 395M encoder on a CPU in under 15 ms. If you already host an open LLM and want no training at all, SemIf or AnyJev read the logits instead.
Sources
- Laya on GitHub — README, benchmarks, latency table
- convaiinnovations/laya model card (Apache-2.0)
- Japanese 300-item teardown: position bias in ordinal scales
- Independent 49-task classifier benchmark
- Best open source Jev alternatives — stars and hardware table
Every comparison number on this page is a third-party published figure or a read of a public repository — not a benchmark we ran. We have not tested TypeSafe Jev itself, and its customer agreement forbids using its outputs to build similar products.