Jev 101 logo
Jev 101

Guides / illustrated walkthrough

Open Jev Models Are Here: Semif, Nimble, Decider, DiffusionGemma & Laya Hands-On

Sam Witteveen tests the open Jev ecosystem in 21 minutes: JevBench rankings, the Semif logit readout on a frozen Qwen3.5-4B, Nimble's contrastive LoRA recipe, Decider's ~33 ms typed answers, DiffusionGemma batch routing, and Laya fine-tuned on a free Colab T4.

Quick takeaway

Within ~24 hours of Jev taking off, 20-30 open implementations appeared. On JevBench (intelligence, calibration, speed, cost at 25% each), Jev 1.13.0 scores 75.3 with Semif (frozen Qwen3.5-4B) right behind at 74.6 — close on easy work, still behind on hard tiers. Zero-training option: read option-token logits like Semif. Narrow high-accuracy option: Nimble's contrastive LoRA on ~3,000 pairs. Fastest serving: Decider at ~33 ms per answer and DiffusionGemma at 27.4 decisions/s. Smallest local setup: Laya, a 421M-parameter ModernBERT, fine-tunable on a free T4. For hard multi-hop cases, cascade up to a reasoning model.

Video source

Sam Witteveen

21:3253wDOI_7x8I
Visit channelWatch original

Step-by-step walkthrough

  1. 1

    Start from the JevBench scoreboard to pick your open Jev

    JevBench v1.2.2 runs 534 decisions per system and scores intelligence, calibration, speed, and cost at 25% each with a geometric mean. Jev 1.13.0 tops the official board at 75.3; right behind sits Semif (Qwen3.5-4B) at 74.6, then djev (Maisa, diffusion-gemma) at 74.3 and Laya at 70.1. Frontier models flip the picture: GPT-5.6 Luna posts a 97 intelligence sub-score but only 66.0 overall while costing roughly six times as much as Jev and running far slower. Switch the weighting to accuracy-only and the gap to the smaller open models widens — and note Jev's own terms of service even forbid benchmarking it against other models.

    JevBench Score leaderboard ranking Jev 1.13.0 first among open models at 75.3, just above Semif at 74.6, djev at 74.3, and Laya at 70.1 on a 25 percent blend of intelligence, calibration, speed, and cost.
    The official JevBench board: Jev 75.3, Semif 74.6, djev 74.3 — the open pack is close on blended score.Watch at 4:02
  2. 2

    Run the zero-training Semif readout on a model you already host

    Semif (formerly OpenJev) is a frozen Qwen3.5-4B with no training at all. Put the question and options in the prompt, run the prefill once, then read the logits of the option tokens and softmax over them instead of letting the model generate text. In the demo, a double-charge support ticket scores billing 86.7%, account_access 9.1%, sales 1.6%, escalate 2.6% — a 31 ms forward pass, 35 ms total, zero output tokens, 209 input tokens. Sam points out this matches his guess that Jev itself predicts off the prefill stage, and changing the policy wording immediately moves the distribution.

    Semif demo reading billing routing probabilities straight from option-token logits in one forward pass, scoring a double-charge ticket billing at 86.7 percent in 31 ms with zero generated output tokens.
    One forward pass: billing 86.7% read off raw option-token logits — no generation.Watch at 6:16
  3. 3

    Fine-tune with Bespoke Nimble's contrastive pairs

    Bespoke Nimble puts a LoRA on Qwen3.5-9B and trains it with fewer than 3,000 examples, explicitly not distilled from Jev. The trick is contrastive data curation: write two nearly identical records and flip one fact so the correct answer flips — a refund signed by Mira is authorized, the same record signed by Noah is not. The playground returns refund_authorized true with customer_tone CALM in 141 ms (47 ms per field), and the Flip the fact button swaps Mira for Noah to demonstrate the flip. On their own holdout it hits ~90% where the base model gets ~66% and Jev ~93%, but only ~44 on JevBench's hard tier — and the Priya pre-approval example shows the adapter can miss until the wording matches, so expect strong-but-narrow behavior.

    Bespoke Nimble playground scoring a refund record signed by Mira as refund_authorized true in 141 ms with a CALM customer tone, next to a Flip the fact button that swaps in signed-by-Noah to verify the decision flips.
    Flip the fact: swap signed-by-Mira for signed-by-Noah and watch the verdict flip.Watch at 9:08
  4. 4

    Serve typed answers at ~33 ms with Decider

    Decider (from Mapika) runs on a much smaller Qwen3.5 base model and reads the hidden state at each answer slot, projecting it onto your options instead of using output logits. It speaks the same three question types as Jev (choice, score, null), takes up to 32K context, and serves the same wire format as the Type Safe API, so it can drop in behind existing clients. The ANSWERS dashboard routes a double-charge ticket to billing at 99.8% with urgency 1.78, escalate yes at 72.3%, and frustration 1.52 — averaging about 33 ms per answer across ten runs. Sam's caveat: the model hedges, sitting near the middle of its distributions where Jev commits with high confidence.

    Decider ANSWERS dashboard averaging 35 ms per decision while routing a double-charge support ticket to billing at 99.8 percent with an urgency score of 1.78 and a 72.3 percent probability to escalate.
    Decider's ANSWERS board: billing 99.8%, ~33 ms average, escalate 72.3%.Watch at 12:00
  5. 5

    Batch decisions at DiffusionGemma speed

    DiffusionGemma is a different class entirely: a 26B diffusion language model (diffusiongemma-26B-A4B, NVFP4 here on an RTX Pro 6000) that denoises a whole canvas per step and reads every answer slot's distribution at once, wired to Jev's schema format through a vLLM PR. In the demo, 16 support tickets fire concurrently at concurrency 16 and are simply done, with the MEASURED panel reporting about 27.4 decisions per second. Sam calls the throughput insane and notes it keeps quality on technical-vs-billing routing and annoyed-customer tone, though he found it under-trained next to Gemma or Qwen peers of similar size. If you need absurd question volume, this is the one to try.

    DiffusionGemma 26B structured-reads console queuing 16 support tickets at concurrency 16 for one-denoise-step classification in the Jev wire format on an RTX Pro 6000 GPU.
    16 tickets, one denoise pass each: DiffusionGemma measures 27.4 decisions/s.Watch at 14:48
  6. 6

    Run and fine-tune Laya on a free Colab T4

    Laya is the old-school path reborn: a 421M-parameter non-autoregressive decision model built on ModernBERT-large with a 2-layer decision head, answering choice, score, and null questions in one ~35 ms forward pass across roughly 100 languages. The team publishes a Google Colab that fine-tunes it with RLCD policy gradients (log-score + spherical score + ranked probability score) on a free T4, peaking at about 6.5 GB of VRAM with FP16. Sam's take: impressive for its size, but on accuracy-weighted JevBench it trails the bigger models, most likely because a single-author fine-tune of an older encoder generalizes less than a well-funded lab's model. Still, it is the cheapest way to touch a Jev-style model on hardware you already have.

    Google Colab notebook for fine-tuning Laya, a 421M-parameter ModernBERT-large decision model, on a free NVIDIA T4 GPU with a 6.5 GB VRAM footprint and roughly 35 ms single-forward-pass inference.
    Fine-tuning Laya (421M params) on a free T4: ~6.5 GB VRAM, ~35 ms inference.Watch at 18:22

Frequently asked questions

What are the best open source alternatives to Jev?

As of this video there are 20-30 open implementations. The strongest on JevBench are Semif (a frozen Qwen3.5-4B logit readout, 74.6 vs Jev's 75.3), djev (74.3), and Laya (70.1). For image inputs try Alex Wartega's OpenJev, which can play Doom and Minecraft from screen states; Nano Jev is a 600M-parameter model that plays a Pac-Man-style game from grid choices.

Can I run a Jev-style classifier locally for free?

Yes. Laya ships a Google Colab that fine-tunes and runs a 421M-parameter model on a free T4 GPU (~6.5 GB peak VRAM, ~35 ms forward pass). Semif's readout trick works on any model you already host, and the video runs six implementations on a single RTX Pro 6000 workstation.

How close do open Jev models get to the real Jev?

On blended JevBench scores, very close: Jev 75.3 vs Semif 74.6 and djev 74.3. The gap opens on hard tiers — Bespoke Nimble scores only ~44 there despite ~90% on its own holdout — and multi-hop reasoning plus date arithmetic still favor cascading to a reasoning model on low or medium effort.

How do I fine-tune an open Jev model on my own data?

Copy Bespoke Nimble's contrastive data curation: write pairs of nearly identical records where one flipped fact flips the correct answer, then train a LoRA — about 3,000 such examples took them to ~90% on their holdout vs ~66% for the base Qwen3.5-9B. Sam also suggests MiniCPM 5 2B as a promising base for this recipe.

Related guides

Jev Benchmarks

How intelligence, calibration, speed, and cost are scored — and where open models land.

Read

Awesome Jev Projects

Curated open Jev implementations, including Semif, Decider, DiffusionGemma, and Nano Jev.

Read

Jev Classification Quickstart

Try the hosted Jev API with Choice, Score, and Noul before you self-host an open model.

Read

More video walkthroughs