Jev 101 logo
Jev 101

Guides / illustrated walkthrough

Decision Models on Jetson: Jev vs Strands Decider 2B vs Laya vs d1-3B — Skill Master Scores, Thor Latency and JevBench, Every Number Frame-Checked

JetsonHacks’ Skill Master test app turns a Jetson into a decision-model bench: hosted Jev beside Strands Decider 2B, Laya and d1-3B, the same fourteen Jetson skills fired at each. This walkthrough keeps every on-screen number — the 99% identity answer, the 45% maintenance trap, the 0–3 intervention scale, Orin Nano memory math, 30–50 ms direct-on-Thor latency, the FastAPI wrapper tax, and the JevBench 86.6/75.3/72.3 card.

Quick takeaway

The twelve-minute version, with every card number verified against the frames. The rig: Jim (jetsonhacks.com) built Skill Master, a local web app that sends one query to several models and renders each answer — probability bars, intervention scores, per-case grades — in the same UI. Tabs across the two recording sessions: Jev remote, CLM-8B local (first session only, never tested), Strands Decider 2B local, Laya local, d1-3B local. Test set: 14 examples — identity, processes, memory before/after, fit model, container, serve, LLM performance, speculate, codec support, video ready, transcode, video speed, vague, outside. The headline answer: “Which Jetson module and L4T release is this board running?” → Device info 99.0%, Ask for clarification 1.0%, all fourteen other options 0.0%, flagged MATCHES EXPECTED SKILL, 153 ms round trip, ≈ $0.000047 per call ($0.047 per 1,000) shown in the UI. The trust rule: on “Set up a vLLM server for this model on my Jetson Thor,” the winner LLM serving setup scores just 45.0% (clarify 27.0%, package compatibility 15.0%, device info 10.0%) — the narrator wants >60% before trusting a choice, more for important questions — while the intervention side answers 2.99 position on 0–3 at 99% confidence, most probable level 3, Change and verify; this is also the run the on-screen header marks JEV · 137 MS ROUND TRIP. The two outs: “Can you check it?” (no context) → Ask for clarification 100.0%, intervention 0.11 at 89% confidence, most probable level 0, Inspect — a rating he calls a bit off; “Write a poem about a robot dinosaur” → No matching skill 100.0% (“the request is clear, but this catalog has no suitable skill”), 134 ms. The request, on the wire: POST to https://api.typesafe.ai/v1/systemone with {"model": "jev-latest", "state": "<the user’s query>", "questions": {"skill": {"type": "choice", "instructions": "Which ONE available skill should handle the user’s request first? Select clarify when essential details are missing. Select no_matching_skill when the request is clear but outside this catalog. For a multi-step request, pick the first useful skill. Do not infer measurements or execute a skill.", "criteria": {"jetson-print-device-info": "Identify the Jetson module, L4T release, kernel, OS version, uptime, or current power mode.", "jetson-diagnostic": "Inspect current RAM, processes, GPU, thermal, power, storage, and service health.", "jetson-memory-audit": "Measure DRAM and NVMap use, and verify memory reclamation before and after a change.", "jetson-inference-mem-tune": "Choose an LLM or VLM serving stack and its memory settings for a Jetson workload.", "jetson-package": "Select Jetson compatible containers, vLLM images, and Python package indexes for a JetPack version.", "jetson-llm-serve": "Stand up a vLLM or SGLang server on a compatible Jetson.", …}}}} — sixteen options total: fourteen skills plus clarify and no_matching_skill. The response: {"model": "jev-1.13.0", "answers": {"intervention": {"type": "score", "score": 0, "confidence": 1, "legend": {"0": "Inspect: Identify or read existing device state.", "1": "Measure: Run a test, audit, or benchmark without requesting a configuration change.", "2": "Recommend: Propose settings, a recipe, or a plan without applying it.", "3": "Change and verify: Install, configure, start, stop, or modify something and check the result."}, "probabilities": {"0": 1, …}}}} — typed answers your code reads by key, no prose to parse. The concept card: classifier vs LLM vs decision model share one input phase (prompt → encoding/pre-fill → contextual representation); classifier answers one of its labels, fixed by training — new category means retraining; LLM answers in free text, one token per pass, and the framework must interpret the string; decision model asks any question at run time — noul, choice, score — and answers in one pass. “Flexible, typed, and fast.” The lineup: AWS Strands Decider 2B; Laya (smallest, weakest — the video does not dwell on it); Liquid AI d1-3B (lab with MIT CSAIL roots, built for fast single-pass decisions). Memory: on an Orin Nano all three run, but Strands and d1-3B eat most of it; larger Orin and AGX Thor are comfortable fits (schematic, not to scale). The scorecard: Jev 14/14 correct, average 150 ms per case, average 1,109 tokens in / 222 out, 0 not run — grades 12 A, 1 B (codec support), 1 C (serve); Strands 14/14, average 353 ms, 794 in / 354 out — 10 A, 4 B (identity, processes, serve, vague); Laya 8/14; d1-3B 14/14 — 9 A, 4 B, 1 C. Per question type: identity question Jev 99% vs Strands 61% (correct but unsure, low intervention rating) vs Laya wrong skill vs d1-3B correct, between Strands and Jev, intervention assessment right; maintenance question flips it — d1-3B highest of all four, Strands well ahead of Jev, Laya also above Jev. The LLM baseline: Claude averaged ~2.5 s per skill choice; Jev ~135 ms — an 18× gap, validating the “10–20× faster than a regular LLM” pitch. The deployment bill: direct on Thor, the small locals decide in 30–50 ms; wrapped in a FastAPI server emulating the TypeSafe Jev API, add 75–100 ms; hence “benchmark numbers actually underestimate local models”; running directly instead of through a server can be ~2× faster, and the narrator expects these locals to be at least as fast as hosted Jev in practice. JevBench: Jev 1.13.0 86.6% (200/231), d1-3B 75.3% (174/231), Strands V19 72.3% (167/231) — easy questions nearly tied, standard slightly Jev, hard decisively Jev; over 100 decision models now exist and benchmarks are multiplying, so do your own research; but constrain the comparison to routing, policy decisions, tool selection and complex routing and all three are essentially equivalent — and the locals can still be fine-tuned to your workload. The close: the application defines the decision table (noul yes/no, choice with clarify as a category, 0–3 intervention score), so new decisions need new context, not retraining — but the application writes the instructions, outlines the categories and tests the combinations, and over time that is more application code to maintain. The model gets simpler and much faster; the system doesn’t have to.

Video source

JetsonHacks

12:18UIdMIz0s9VI
Visit channelWatch original

Step-by-step walkthrough

  1. 1

    The test rig: one query, four models, fourteen examples

    The video opens in Skill Master, a local web app Jim wrote for exactly this comparison: one query box, a row of model tabs, and the same question fired at each. This session pits hosted Jev against Strands Decider 2B, Laya and d1-3B running locally (an early recording also shows a CLM-8B tab that never gets a test — the app grew between sessions). The sample set is fourteen examples covering the things a Jetson owner actually asks: identity, processes, memory before/after, fit model, container, serve, LLM performance, speculate, codec support, video ready, transcode, video speed, plus a deliberately vague case and one outside the catalog. The first shot answers “Which Jetson module and L4T release is this board running?” — Device info takes it at 99.0%, with Ask for clarification the only other option to register at 1.0%. The app flags MATCHES EXPECTED SKILL, shows a 153 ms round trip for this call, and even prices it: about $0.000047 per call, $0.047 per thousand.

    Skill Master’s decision panel returning Device info at 99.0 percent with Ask for clarification at 1.0 percent for the query Which Jetson module and L4T release is this board running, completed as a 153 ms Jev round trip.
    One query, one probability list: Device info 99.0%, clarify 1.0%, everything else zero — in 153 ms.Watch at 0:40
  2. 2

    When the top answer is 45%: Jev’s trust threshold

    Question two is meant to be hard: “Set up a vLLM server for this model on my Jetson Thor.” Jev still picks the right skill — LLM serving setup, “stand up a vLLM or SGLang server on a compatible Jetson” — but only at 45.0%, with Ask for clarification lurking at 27.0% and Package compatibility at 15.0%. Jim’s reading rule is the practical takeaway of the whole video: he usually wants a probability over 60% before trusting a choice, and higher than that for important questions. The second answer in the same response is the intervention score, and there Jev is anything but unsure: 2.99 on the 0–3 scale at 99% confidence, most probable level 3, “Change and verify” — install, configure, start, stop, or modify something and check the result. This is also the run the header clocks at JEV · 137 MS ROUND TRIP, the figure he quotes out loud; the round-trip number varies a few milliseconds call to call (130–178 ms across this video’s screens).

    Jev answering Set up a vLLM server for this model on my Jetson Thor with LLM serving setup at only 45.0 percent, ahead of Ask for clarification at 27 percent, under a Requested Intervention score of 2.99 in 137 ms.
    Right skill, low confidence: 45.0% is below the 60% trust line — but the 2.99 intervention score says real work is being asked for.Watch at 0:58
  3. 3

    Two built-in outs: ask for clarification, or no matching skill

    A skill selector that must always pick would be a liability, so the request format builds in two escapes, and the demo exercises both. The vague case — “Can you check it?”, no context — comes back as Ask for clarification at 100.0%: “the task needs more detail before a skill can be selected.” Its intervention rating is the one rough edge Jim calls out: 0.11 on the 0–3 scale, 89% confident, most probable level 0, Inspect — a read-only label he finds a bit off for a request this ambiguous. The out-of-catalog case is cleaner: “Write a poem about a robot dinosaur” returns No matching skill at 100.0% — “the request is clear, but this catalog has no suitable skill” — in 134 ms. Both land in the graded test set as the vague and outside cases, which is exactly why the summary tab can score all fourteen examples as correct-or-not: refusing correctly counts.

    No matching skill selected at 100.0 percent for the prompt Write a poem about a robot dinosaur, with the Skill Master examples row showing Can you check it? beside the robot-dinosaur request.
    The refusal is the product: clear-but-outside-catalog gets No matching skill at 100.0% instead of a confident wrong answer.Watch at 1:40
  4. 4

    Inside the request: state, questions, criteria

    The app’s details view shows the raw JSON sent to Jev, and it is the real TypeSafe SystemOne contract — the header even shows the endpoint, api.typesafe.ai/v1/systemone. Three top-level keys: model set to jev-latest; state carrying the user’s query verbatim (“Which Jetson module and L4T release is this board running?”); and questions, a named map whose skill entry is a choice-type question. The instructions field is worth reading in full because it encodes the refusals you just saw: pick the one available skill that should handle the request first, select clarify when essential details are missing, select no_matching_skill when the request is clear but outside this catalog, and — for multi-step requests — pick the first useful skill without inferring measurements or executing anything. Then criteria, an object mapping every option key to a one-line description: jetson-print-device-info identifies module, L4T release, kernel, OS version, uptime or power mode; jetson-diagnostic inspects RAM, processes, GPU, thermal, power, storage and service health; jetson-memory-audit measures DRAM and NVMap use; jetson-inference-mem-tune picks a serving stack and its memory settings; jetson-package selects containers, vLLM images and package indexes for a JetPack version; jetson-llm-serve stands up vLLM or SGLang. Sixteen options in all — fourteen skills plus clarify and no_matching_skill — defined entirely in the request, not baked into the model.

    Raw JSON request posted to api.typesafe.ai/v1/systemone with model jev-latest, the board-identity question as state, and a choice question named skill whose criteria describe sixteen Jetson skills.
    The whole contract in one screen: state in, a named choice question out — sixteen options described in plain English, refusals included.Watch at 2:22
  5. 5

    Inside the response: typed answers, confidence, a legend

    The response side is just as legible, and the frame catches a score-type answer end to end: model jev-1.13.0, answers.intervention with type score, score 0, confidence 1, a legend explaining the scale — 0 Inspect, identify or read existing device state; 1 Measure, run a test, audit or benchmark without requesting a configuration change; 2 Recommend, propose settings, a recipe or a plan without applying it; 3 Change and verify, install, configure, start, stop, or modify something and check the result — and probabilities per level. Choice answers mirror the shape: the selected option under the key you named, a confidence, and the full probability list across all options, which is what the decision panel renders as bars. Nothing arrives as prose; the query defines the structure and the response fills it, so the calling code reads keys and numbers and moves on. That is the entire reason a decision model slots into an agent loop without a parsing layer.

    Response from Jev showing model jev-1.13.0 with an intervention score answer of type score, score 0, confidence 1, a four-level legend from Inspect to Change and verify, and per-level probabilities.
    Typed in, typed out: a score answer with its legend and per-level probabilities — no generated sentence anywhere in the payload.Watch at 2:52
  6. 6

    Same representation, three ways to answer

    Jim’s diagram card is the clearest one-screen taxonomy of this site’s core category. Every text model starts identically: the prompt goes through an encoding phase (you will hear it called pre-fill) and becomes a contextual representation. The output phase is where the three species part ways. A classifier turns that representation into a label — but the label set is fixed by training, so adding a category means new labeled data and a retraining run. An LLM writes: free text, one token per pass, each token conditioned on the last, which is why generation is the slow path — and in an agent system the framework still has to interpret the string afterward. A decision model takes the third road: the questions are asked at run time, the answers come back typed — a noul yes/no, a choice from options you name, a score on a scale you define — in a single pass, with the input side heavily parallel. His caption is the thesis: flexible, typed, and fast — and the same representation is what makes one trained model useful across all three output styles.

    Comparison table titled Same representation, three ways to answer, contrasting a classifier fixed by training and one pass, an LLM answering in free text over many passes, and a decision model choosing noul, choice or score in one pass.
    One encoding, three outputs: labels fixed by training, prose one token at a time, or typed answers in a single pass.Watch at 5:50
  7. 7

    The local lineup: Strands 2B, Laya and d1-3B against Jetson memory

    For the local round, three open-weight challengers join hosted Jev: AWS Strands Decider 2B (“probably doesn’t need much introduction”), Laya, and Liquid AI’s d1-3B from a lab with MIT CSAIL roots, built for fast single-pass decisions. The selection criterion is blunt: small enough for Jetson-class hardware, where memory is the constraint. The schematic card makes it concrete — on an Orin Nano, Strands and d1-3B fill most of the memory; on a larger Orin and a Jetson AGX Thor they fit comfortably (drawn as boxes, explicitly not to scale). Laya squeezes in anywhere, being the smallest model in the group, but the video is upfront that it will not get much airtime: it underperforms in the tests. That honesty is worth keeping when you plan your own bench — model size, module tier and expected residency all have to agree before the first query fires.

    Memory schematic titled Strands and d1-3B, by memory, with Strands and d1-3B filling most of an Orin Nano while fitting comfortably on a larger Orin and a Jetson AGX Thor.
    The whole hardware question in one card: Orin Nano is tight, larger Orin and AGX Thor are comfortable — schematic, not to scale.Watch at 7:32
  8. 8

    The fourteen-case scorecard: Jev 14/14 at 150 ms average

    The summary tab grades every model on all fourteen cases with a letter per answer, based on the probability. Jev: 14/14 correct, average 150 ms per decision, averaging 1,109 tokens in and 222 out — twelve A grades, one B (codec support) and one C, and the C sits exactly where the low-confidence answer did, on the serve case. Strands Decider 2B also goes 14/14 but at an average 353 ms with 794 tokens in and 354 out — ten A, four B. Laya manages 8/14, the weakest score in the group. d1-3B finishes 14/14 with nine A, four B and one C. The per-question detail explains the letters: on the identity question Jev answered at 99%, Strands chose correctly but at an uneasy 61% with a low intervention rating, Laya picked the wrong skill outright, and d1-3B landed between Strands and Jev with its intervention assessment right. The maintenance question flipped the order — d1-3B posted the highest probability of all four, Strands ran well ahead of Jev, and even Laya beat Jev there. Same fourteen questions, four very different confidence profiles.

    Skill Master summary table for Jev reading 14 of 14 correct at an average 150 ms and 1109 input plus 222 output tokens, with case grades from identity at A to serve at C and codec support at B.
    Perfect run, honest grades: 14/14 at 150 ms average — with the one C sitting on the serve case that scored 45%.Watch at 8:38
  9. 9

    The latency bill: direct on Thor, behind FastAPI, next to Jev

    Speed is where the video earns its keep, and it comes in three numbers. First the baseline: the same skill-picking test run through Claude took about 2.5 seconds per choice; Jev does it in roughly 135 ms — an 18× gap that makes the vendor’s “10–20× faster than a regular LLM” claim fully hold in this test. Second, the locals: running directly on a Jetson AGX Thor, these small models typically decide in 30–50 ms. Third, the catch — for the comparison Jim wrapped them in a FastAPI server emulating the TypeSafe Jev API, and that wrapper added 75–100 ms of latency. His conclusion is the sentence to remember: the benchmark numbers actually underestimate local models, and if latency really matters, calling the model directly instead of through a server can make it twice as fast again. In practice, he expects these small local models to be at least as fast as hosted Jev — the architecture card draws exactly that: a blue direct-on-Thor bar, an orange extension for the FastAPI tax, and Jev’s 135 ms line to the right.

    Latency bar chart Speed depends on deployment showing 30 to 50 ms direct on Thor without a server, a longer bar behind FastAPI emulating Jev’s API, and a Jev 135 ms reference line.
    30–50 ms bare on Thor, +75–100 ms once FastAPI emulates the Jev API, against Jev’s 135 ms — deployment choice is the benchmark.Watch at 9:44
  10. 10

    JevBench: 86.6 vs 75.3 vs 72.3 — and where they separate

    The benchmarks card puts the three on the same public board: Jev 1.13.0 at 86.6% (200 of 231), d1-3B at 75.3% (174 of 231), Strands V19 at 72.3% (167 of 231). Jim’s reading is more useful than the ranking: simple questions are nearly a three-way tie; on standard questions Jev edges ahead; on hard questions it breaks away decisively. He also zooms out — over a hundred decision models now exist and benchmarks are multiplying, so it is worth researching whichever models match your criteria. Then the twist that matters for builders: if you focus the comparison on the task types these models are actually hired for — routing, policy decisions, tool selection, complex routing — all three are essentially equivalent, and the local ones have one lever the hosted model does not: you can fine-tune them on your own workload. Different weights, same job, different total cost — which is exactly why the latency bill from the previous step decides more than the leaderboard does.

    JevBench scoreboard card with Jev 1.13.0 at 86.6 percent on 200 of 231, d1-3B at 75.3 percent on 174 of 231, and Strands V19 at 72.3 percent on 167 of 231.
    One board, three rows: 86.6 / 75.3 / 72.3 — near-tied until the questions get hard.Watch at 10:22
  11. 11

    The decision context — flexible, but not free

    The closing cards are the video’s argument for the whole category. The decision context for “My board keeps running out of memory” is not one question but a table the application defines: a noul — does the request need the health snapshot skill, true or false; a choice — which one skill should handle this first, with jetson-diagnostic, jetson-memory-audit and clarify shown out of fourteen options; and a score — intervention on levels 0 to 3, how much change the request asks for. Clarify sitting inside the choice list is the quiet insight: “a way out is a category too.” Because the table lives in the application, a new decision needs a new context, not a retraining run — unlike a classifier, whose categories are frozen at training time — and open-weight locals can still be fine-tuned if a workload demands it. But the last card prices it: the application writes the instructions, outlines the categories, and tests the combinations — and over time that is more application code to maintain. Jim’s closing line is the honest summary of everything above: the model may be simpler and much faster, but the system doesn’t have to be.

    Decision context diagram for the prompt My board keeps running out of memory, laying out a noul question on jetson-diagnostic, a choice over the skill plus thirteen more options including clarify, and a 0-to-3 intervention score.
    The application writes the decision table — noul, choice with clarify as a category, a 0–3 score — and owns its upkeep.Watch at 11:30

Frequently asked questions

Can you run decision models on a Jetson?

Yes — that is the video’s whole point. Three small open-weight decision models (AWS Strands Decider 2B, Laya, Liquid AI d1-3B) all run on a Jetson Orin Nano, though Strands and d1-3B consume most of the available memory there; on a larger Orin or a Jetson AGX Thor they fit comfortably. Decisions take about 30–50 ms when the model is called directly on Thor. Hosted Jev answered the same skill-selection queries in roughly 135 ms round trip from the same desk (individual calls ranged 130–178 ms on screen), at a displayed cost of about $0.000047 per call. The only real gate is RAM: match the model to the module tier before you benchmark anything.

Which decision model won on Jetson — Jev, Strands, d1-3B or Laya?

It depends which question you ask, and the video shows all four outcomes. On the fourteen-case skills test: Jev 14/14 (12 A, 1 B, 1 C) at an average 150 ms; Strands 14/14 (10 A, 4 B) at 353 ms; d1-3B 14/14 (9 A, 4 B, 1 C); Laya 8/14. On the identity question Jev’s 99% crushed Strands’ 61% while Laya picked the wrong skill; on the maintenance question the order flipped — d1-3B posted the highest probability of all four, Strands beat Jev comfortably. On JevBench: Jev 86.6% (200/231), d1-3B 75.3% (174/231), Strands 72.3% (167/231). But constrained to the common workloads — routing, policy decisions, tool selection, complex routing — all three are essentially equivalent, so latency, memory and fine-tuning usually decide the pick.

How much faster is a decision model than an LLM for skill selection?

In this test, about 18×. Claude needed roughly 2.5 seconds on average to choose the right skill for the same queries; Jev decided in about 135 ms, which the narrator notes fully validates the “10–20× faster than a regular LLM” claim. The local models go further: 30–50 ms running directly on a Jetson AGX Thor. The subtlety worth remembering: for the comparison they were wrapped in a FastAPI server emulating the TypeSafe Jev API, which added 75–100 ms — so public benchmark numbers actually underestimate local models, and calling the model directly instead of through a server can roughly double the speed again.

What is Skill Master?

A test app Jim of jetsonhacks.com built for this comparison: a local web console that sends one query to several models and renders every answer in the same UI — probability bars for all options, a requested-intervention score on a 0–3 scale (Inspect / Measure / Recommend / Change and verify), per-case timing, and a summary tab that grades fourteen example tasks (identity, processes, memory before/after, fit model, container, serve, LLM performance, speculate, codec support, video ready, transcode, video speed, vague, outside) with letter grades. It also displays per-call cost for hosted Jev (≈ $0.000047). Model tabs included Jev remote, CLM-8B local, Strands Decider 2B local, Laya local and d1-3B local — though only Jev plus the three open models were actually tested.

What does a Jev request and response look like on the wire?

The app posts to api.typesafe.ai/v1/systemone with three keys: model (jev-latest), state (the user’s query, verbatim), and questions — a named map where each entry is a typed question. The skill question here is type choice, with instructions telling the model to pick one skill, choose clarify when essential details are missing, and choose no_matching_skill when the request is clear but outside the catalog, plus a criteria object describing all sixteen options in plain English. The response comes back as {"model": "jev-1.13.0", "answers": {...}}: choice answers give the selected option, a confidence and the full probability list; score answers give score, confidence, a legend explaining each level (0 Inspect, 1 Measure, 2 Recommend, 3 Change and verify) and per-level probabilities. Everything is keyed JSON — no prose to parse, which is exactly why it slots into an agent loop.

Should you wrap a local decision model in a server, or call it directly?

Both, for different goals. The video’s numbers: direct on Thor, 30–50 ms; behind a FastAPI server emulating the TypeSafe Jev API, add 75–100 ms — so direct is roughly twice as fast, and benchmarks that measure the wrapped path underestimate local models. But the server shape is what makes local and hosted interchangeable: your agent code speaks the same state-plus-questions contract either way, and you can switch models without touching the caller. The video’s own closing caution applies to the architecture too — flexible, but not free. Whoever defines the decision table (instructions, categories, the clarify and no_matching_skill escapes, combination tests) owns the maintenance, and that lives in your application either way. Start direct if latency is the product; wrap it when you want the contract.

Related guides

Liquid d1 Guide

The deep dive on d1-3B itself — architecture, the noul/choice/score contract and a MacBook Air M5 field test of the model that won the maintenance questions here.

Read

Strands Decider Guide

AWS Strands Decider on its own axis — install and first decisions for the 2B model that matched Jev’s 14/14 here, at a third of the speed.

Read

Run Jev Locally Guide

The self-hosted Jev route end to end — pair it with this page’s Jetson tier for the edge corner of the local-deployment map.

Read

Jev vs Ollama Guide

Whether to localize at all: hosted calibration against local weights — the axis this page’s latency bill speaks to.

Read

Ollaya Guide

Decision models through an Ollama-style runner — the desktop sibling of what this video wraps in FastAPI on Jetson.

Read

More video walkthroughs