Jev 101 logo
Jev 101

Guides / illustrated walkthrough

Run Jev Locally: Kev, SemIf & Von on Your Own GPU (OpenJev Guide)

A teardown of running Jev-style decision models on hardware you own: the logit-reading trick behind System One models, Kev as a drop-in on Qwen 3.5, SemIf with no training, Von on a CPU, and exactly how much accuracy each option trades away.

Quick takeaway

Jev itself is hosted and closed — every decision ships your state to TypeSafe's servers, which is a non-starter for hospitals, banks, or anyone under a data agreement. The open ecosystem replicates the trick instead of the model: a System One decision is just reading the logits for your allowed answers off a model instead of letting it write, so one forward pass returns all answers and it cannot invent a fifth option. Kev (Jared Palmer, built on Alibaba's open Qwen 3.5) copies Jev's interface exactly — change base_url to localhost:8080 and the same SDK code works — scoring 0.852 (9B) and 0.837 (4B) on unseen tasks against Jev's 0.883, with 4B and 9B fitting on a 32GB Mac. SemIf (Theo Lee) needs no training at all: it scores logits on a Qwen 4B you already run, landing 81.3% vs Jev's 88.3% on 102 aligned rows while running 5.21× faster than writing JSON. Von drops the GPU entirely: a 395M-parameter ModernBERT encoder with three decision heads that answers in under 15ms on a CPU. The honest cost: local models land in the low-to-mid 80s versus Jev's high 80s — tiny for sorting a support queue, not tiny for approving a loan — so run your own 100 examples and check that "87% confident" really is right 87 times in 100.

Video source

Cloud Codes

13:444mCyUXqkTpI
Visit channelWatch original

Step-by-step walkthrough

  1. 1

    Why local at all: the state is the sensitive part

    The demo says it in one shot: "my card just got charged twice. What do I do?" lands in a local model, which picks escalate at 87% confidence in half a second — and nothing about that customer leaves the room. Hosted Jev is excellent but closed: asking it anything means sending your program state — your users' messages, internal documents, customer records — to TypeSafe's servers. For a hospital, a bank, or anyone under a data agreement, that is a hard no: if the decision cannot run in the building, the automation simply does not happen. Everything that follows is about getting Jev-style decisions onto the GPU or CPU you already own.

    Slide explaining that Jev is hosted, showing your program and its state being sent over the network to TypeSafe hosted servers — the reason teams run Jev-style decision models locally.
    Every hosted call ships your state — the users' messages themselves — off your machine.Watch at 2:20
  2. 2

    The trick: read the logits, throw away the sentence

    Before a language model writes anything, it scores every possible next word with a raw number called a logit. A typed decision skips the writing: you supply the allowed answers — refund, escalate, reply, ignore — and read each answer's logit straight off the model. Highest score wins, in one forward pass, and the model cannot invent a fifth option or wrap the answer in prose. Better yet, several questions can share one read: the model reads your document once, keeps its notes in the key-value cache, and every question reads from those same notes. On one open build, scoring decisions from scratch runs a couple per second; reusing the cache across questions climbs to about 20 per second on the same card.

    Diagram of asking several questions at once — refund, spam, and route questions reusing one document read through the key-value cache instead of re-reading for every question.
    One document read, many decisions — the key-value cache is why throughput jumps.Watch at 3:40
  3. 3

    Kev: the drop-in that speaks Jev's interface

    Kev comes from Jared Palmer — the developer behind Formik and Turborepo — and ships as a small family of decision models built on Alibaba's open Qwen 3.5 weights: 0.8B, 4B, and 9B parameters. You download the weights and run a little local server. The part that makes Kev the easy pick is interface fidelity: it copies Jev's API exactly, so you take TypeSafe's official SDK, change base_url from their cloud to http://localhost:8080, and the same application code just works — your app cannot tell the difference. Palmer built it from a public write-up of Jev's architecture with an AI coding agent in a couple of weekends, and published evals openly, which is rarer than it should be.

    Change one line slide showing decide.py with the TypeSafe client base_url switched to localhost:8080, and an arrow from your app to your own machine instead of the TypeSafe cloud endpoint.
    Same SDK, one changed line — the decision endpoint now lives on your machine.Watch at 5:56
  4. 4

    The accuracy bill, in numbers

    On tasks Kev has not seen, the 9B model scores 0.822 on dev and 0.852 on test, the 4B lands at 0.797 and 0.837, against Jev's 0.883 hosted reference — with a coin flip drawn at 0.50 for scale. Calibration improves with size too: on the Brier score, where lower is more honest confidence, the 0.8B model sits at 0.468-0.499, the 4B at 0.255-0.299, and the 9B at 0.237-0.286. On hardware, Palmer says both the 4B and 9B fit on a 32GB Mac; in practice the 4B on a mid-range card is the sweet spot for real work. The other open options fill the gaps: OpenJev (Alex Wartega) adds sight — its 4B reads screenshots and photos via entailment training — and Laya is tiny and speaks 100 languages but starts near random (36%) until you fine-tune it on your task, where it jumps to about 77%.

    Bar chart of Kev accuracy on tasks it has not seen: the 9B model at 0.822 dev and 0.852 test, the 4B at 0.797 and 0.837, versus hosted Jev at 0.883, with a coin flip line at 0.50.
    Unseen tasks: 9B Kev 0.852 and 4B 0.837 vs hosted Jev 0.883 — close, not equal.Watch at 6:10
  5. 5

    SemIf: no new model, just scoring one you already run

    SemIf, from developer Theo Lee, proves the trick needs no special model at all. Its tagline is the whole spirit: semantic ifs from open models on a 3090 at home. SemIf takes an open model you already have — a 4B Qwen — and runs the logit-scoring technique on it directly: no fine-tuning, no training run. On the shared benchmark it lands 81.3% against Jev's 88.3% over 102 aligned rows, and it runs 5.21× faster than making the same model write the answer as JSON (1.023s versus 5.332s) — because generating JSON is the slow part it deletes. Lee is careful in the right way: the comparison covers about 100 aligned questions rather than the full set, and he says outright that you must check calibration on your own workload.

    Bar chart comparing SemIf at 81.3 percent against hosted Jev at 88.3 percent over 102 aligned rows, with a badge reading 5.21x faster than JSON output, 1.023 seconds versus 5.332 seconds.
    SemIf 81.3% vs Jev 88.3% — and 5.21× faster than the same model writing JSON.Watch at 7:30
  6. 6

    Von: the CPU answer when there is no GPU

    Kev and SemIf want a real GPU. Von does not: it is built on ModernBERT, an encoder that reads text to judge it rather than to continue it — which is exactly what a decision is — pre-trained on 2 trillion tokens and topped with three decision heads (choose an option, yes or no, rating on a scale). The whole model is 395 million parameters, about 1.5GB on disk, small enough to run anywhere: a CPU, an old laptop, a phone-class chip, no graphics card required, with each decision coming back in under 15 milliseconds. The trade is sharpness: on a clean 20-bucket routing job Von scores about 83.4%, but spread across a messy 49-task suite it drops to 71.5% — excellent where it is pointed, shaky where it is not.

    Small enough to run anywhere slide showing Von decision model on a CPU, an old laptop, or a phone-class chip with no graphics card required and under 15ms per decision.
    395M parameters, ~1.5GB on disk, under 15ms per decision — no GPU required.Watch at 8:35

Frequently asked questions

Can you run Jev itself locally?

No — Jev is hosted and closed, and every call sends your program state to TypeSafe's servers. What you can run locally are open models that replicate the same System One technique of reading logits for allowed answers: Kev (a drop-in interface clone on Qwen 3.5), SemIf (logit scoring on any open model you already run), Von (a 395M encoder for CPUs), OpenJev (adds image inputs), and Laya (tiny, 100 languages, needs fine-tuning).

Which local Jev-style model should I pick?

The video's pick for most people is the 4B Kev — a true drop-in where you keep your code and swap the base URL to localhost, with public evals to check. Pick SemIf instead when you would rather score a model you already run, with no new weights to serve. No GPU at all? Von runs on a CPU in under 15ms per decision. The 9B Kev buys the last few points of accuracy if you have a workstation card or a 32GB Mac.

How much accuracy do you give up running locally?

On the measured tasks, local setups land in the low-to-mid 80s against Jev's high 80s — for example Kev-4B at 0.837 and SemIf at 81.3% versus 0.883 hosted. That gap is tiny for sorting a support queue and not tiny for approving a loan. Close choices with clear options and plenty of examples are the sweet spot; open-ended judgment, long messy context, or anything adversarial still deserves a person in the loop.

What does it cost to run a Jev-style model locally?

Close to nothing per decision: no per-token bill and no metered API — you pay for the GPU once (or borrow the one you have) and a million decisions cost electricity. Von runs on a CPU at under 15ms per decision; the 4B Kev or SemIf fit a gaming card with 8-12GB of VRAM; the 9B wants a workstation card or a 32GB Mac. Running one is mundane: download the weights, start a little local server, point your code at localhost.

Related guides

Open Jev Models Landscape

The wider open ecosystem: JevBench leaderboard, Laya fine-tuning, and more.

Read

Jev Model Router Guide

A real deployment that routes between local MiniCPM and cloud DeepSeek with a PII gate.

Read

Jev Pricing

What hosted Jev costs per decision — the baseline your local setup competes with.

Read

More video walkthroughs