Jev alternatives
AnyJev
Training-free debiasing — from Nokia Applied Research
Quick answer
AnyJev is what to add when you are already reading logits off an open LLM and your answers move when you reorder the options. Its L0 layer needs zero labels: cyclic-shift marginalization removes position bias and a label-free prior estimate removes the model’s label bias. On Qwen3-8B it cut the Banking77 order-flip rate from 0.227 to 0.077 and drove the injection set to 0.000.
AnyJev does not train a model and does not ship weights. It is a layer that sits on top of an LLM you already run and fixes the two things naive logit reading gets wrong: options that win because of their position, and answers that lean a certain way regardless of the input. Its headline demonstration is concrete — the same spam question flips from 0.62 to 0.41 once the model’s own label prior is divided out.
Read the logits
No new weights. Intercept the model right before it answers and read the probability of each option instead of letting it write.
Search aliases
Key specs
- Licence
- Apache-2.0
- Author
- Jiamu (Morris) Zhang, Tianze Yang, Liang Wu — Nokia; Yucheng Shi — Tencent
- Backbone
- Any open LLM — no weights of its own
- Size
- L2 heads are about 100 KB
- Latency
- One prefill per call; no generation
- Wire format
- Library + CLI + MCP; a Jev-compatible server is on the roadmap
- Install
- pip install "anyjev[hf]"
AnyJev against Jev and against naive logit reading
AnyJev re-measured Laya and several Qwen models on Laya’s own typed-decisions set, 2,000 decisions, rather than quoting other people’s numbers. Jev’s row is the published figure and was not re-run — the project says so explicitly.
| AnyJev against Jev and against naive logit reading | AnyJev | Jev |
|---|---|---|
| Order-flip rate, raw → L0 (Qwen3-8B, Banking77) | 0.227 → 0.077, zero labels used | Not published as a flip rate |
| typed-decisions, Qwen3-32B + L0 | 0.700 zero-shot | 0.727 (published) |
| Calibration error after L1 | ECE 0.034 with 200 labels | 0.144 (published) |
| Fine-tuned Laya on the same set | 0.768 but ECE 0.215 — accurate, poorly calibrated | 0.727 with ECE 0.144 |
| Training required | None for L0; 100–500 labels for L1; a closed-form head for L2 | Closed weights, RLCD-trained |
When AnyJev is the right choice
Add AnyJev the moment your decision pipeline already reads logits and you have noticed answers shifting when the option list is reordered. That instability is not a curiosity — it means a refactor can change production behaviour — and AnyJev removes it without a single label. Then, if you have a few hundred labels, L1 buys you the probability honesty that the whole System One idea is supposed to provide: on the same 2,000 decisions it reached ECE 0.034 against a fine-tuned Laya’s 0.215.
When to walk away
Do not expect it to raise raw accuracy much. L0 is a bias correction, not a better model: on typed-decisions it moved Qwen2.5-7B from 0.621 to 0.628 and Qwen3-8B from 0.626 to 0.640. If your base model cannot express the decision at all, debiasing will not save it. And do not deploy it against a consequential action without checking that a reported 0.9 really is right about 90% of the time on your own data.
Adopting AnyJev in three steps
from anyjev import Decider, Question
from anyjev.backends.hf import HFBackend
d = Decider(HFBackend("Qwen/Qwen3-8B"))
route = Question.choice(
"Which handler should process this request?",
["billing", "technical", "sales", "other"], name="route")
safe = Question.noul(
"Is the proposed tool call destructive or irreversible?", name="safe")
done = Question.score(
"How complete is the task on a 0 to 1 scale?", bins=5, name="done")
r = d.decide(state, [route, safe, done])
r["route"].argmax # "billing"
r["route"].distribution # {"billing": 0.81, ...}
r["safe"].p_true # 0.12
r.level # "L0" — debiased, not yet calibrated
# With 100-500 labels per question, get an L1 artifact:
art = d.calibrate(safe, calib_states, calib_labels)
r = d.decide(state, [safe], level="L1")AnyJev questions people actually ask
Is AnyJev affiliated with TypeSafe or Jev?
No. The package says so on its own first screen: not affiliated with, endorsed by, or derived from TypeSafe AI or Jev. All of its comparisons are measured and reproducible from the committed results rather than quoted from the vendor.
What is the difference between L0, L1 and L2?
L0 needs nothing: it removes option-position bias by cyclic-shift marginalization and label bias by estimating the model’s prior without labels. L1 adds temperature scaling per (model, question), fitted on 100–500 labels. L2 fits a closed-form head of about 100 KB. Every result is tagged with the level that produced it, so your code can refuse to act on a raw answer.
Does debiasing make the model more accurate?
Only slightly. On Laya’s typed-decisions set, L0 moved Qwen2.5-7B from 0.621 to 0.628 and Qwen3-8B from 0.626 to 0.640. The large gains are in stability and calibration, not in accuracy: order flips fall by roughly two thirds and ECE drops from as high as 0.331 to around 0.04–0.16 after L1.
Why is a stable answer more important than an accurate one?
Because unstable answers make refactoring dangerous. If reordering your option list changes the decision, then a routine code change silently alters production behaviour, and no amount of accuracy compensates for that. A model that reads position instead of meaning is broken in a way that is invisible in an aggregate accuracy score.
Sources
- nokia-applied-research/AnyJev — canonical repository
- anyjev on PyPI — full benchmark tables and level contract
- Laya — the typed-decisions benchmark AnyJev re-measures
Every comparison number on this page is a third-party published figure or a read of a public repository — not a benchmark we ran. We have not tested TypeSafe Jev itself, and its customer agreement forbids using its outputs to build similar products.