Jev 101 logo
Jev 101

Guides / illustrated walkthrough

LangChain + Jev Integration Tutorial: Routing, Guardrails & Evals

Prompt Engineer 48 wires Jev into LangChain end to end: one TypeSafeClassifier invoke for triage, ModelRouter and AutoModeGuard middleware around create_agent, a custom TriageMiddleware that ends runs before the LLM wakes, and a Jev-vs-Sonnet eval bake-off that finishes about 230x cheaper with identical verdicts.

Quick takeaway

langchain-typesafe makes Jev a first-class LangChain citizen: TypeSafeClassifier answers Noul, Choice, and Score questions in one parallel invoke; ModelRouter picks between fast (GPT 5.6 Luna at 20 cents per million input tokens) and powerful (GPT 6 Astra at $10) before the agent starts, at roughly $0.00002 per routing decision; AutoModeGuard vets protected tool calls and, once you write explicit risk criteria, blocks a backup-deleting prompt injection at 0.74 while a legitimate restart passes at 0.07; and a custom TriageMiddleware ends urgent runs before any LLM is called. As a judge, Jev matched human labels 500/500 in LangChain’s study ($0.34 versus Sonnet’s $28.17) and scored this agent at 0.46 seconds per judgment — 80 judgments for $0.0014, about 230x cheaper than Claude Sonnet with identical verdicts.

Video source

Prompt Engineer 48

17:26syGiVxM-NBw
Visit channelWatch original

Step-by-step walkthrough

  1. 1

    Install langchain-typesafe and wire three keys

    Two packages carry the whole integration: uv add "langchain-typesafe[experimental]" — the experimental extra pulls in the agent middleware — plus langchain-openrouter for the LLM lanes. Then three keys in .env: TYPESAFE_API_KEY, OPENROUTER_API_KEY, and LANGSMITH_API_KEY with LANGSMITH_TRACING=true so every Jev decision lands inside the agent trace. Two gotchas from the video: the package is 0.0.1a3 alpha, so pin versions and expect API changes; and pass load_dotenv(override=True) — an old OPENROUTER_API_KEY lingering in your shell silently wins because load_dotenv never overwrites existing variables by default. What you get: TypeSafeClassifier as a pipeable runnable, plus ModelRouter and AutoMode middleware that attach to create_agent — with TriageMiddleware showing you can roll your own.

    .env file open in VS Code lists TYPESAFE_API_KEY, OPENROUTER_API_KEY, LANGSMITH_API_KEY, LANGSMITH_TRACING=true, and LANGSMITH_PROJECT=support-ops-agent, while the terminal below runs uv add and reports langchain-typesafe 0.0.1a3 among the installed packages.
    One extra, two packages, three keys — and Jev is a LangChain citizen.Watch at 2:30
  2. 2

    One invoke returns typed answers: Noul, Choice, and Score in parallel

    Jev answers named questions about a state, and there are exactly three types. A Noul is yes-or-no with a probability ("urgent"); a Choice picks one key from your set with per-option probabilities and a confidence ("team" over infra, billing, account, product — it cannot invent a fifth team); a Score rates an ordered scale you describe ("severity" from cosmetic to full outage). Since alpha 3, state and questions both go in invoke: TypeSafeClassifier().invoke({"state": ticket, "questions": TRIAGE}). All three evaluate in parallel, so asking three costs about the same time as one. The run prints urgent 0.95, team infra at confidence 1.00 with every other team at 0.0, and severity 1.83 — an expected value, so in-between readings are normal. 461 input tokens, 1,126 ms on a cold call.

    Terminal under step1_classify.py prints urgent 0.95 as a Noul probability, team infra at confidence 1.00 with billing, product, and account all at 0.0, and severity 1.83, while jev-1.13.0 processes 461 input tokens in 1,126 ms.
    Typed values with probabilities — 0.95, infra 1.00, severity 1.83 — from a single parallel call.Watch at 4:30
  3. 3

    ModelRouter: the cheap lane answers first, the expert lane only when it must

    The ModelRouter middleware takes named options with plain-language criteria plus one instruction — choose the least expensive model that can safely complete the task. Here fast is GPT 5.6 Luna at $0.20 per million input tokens and powerful is GPT 6 Astra at $10. Before the agent starts, Jev classifies the last user message, stores the verdict in agent state as model_route, and swaps the model on every call in that run. "What does HTTP status 429 mean?" routes to fast at confidence 1.00 and Luna answers in 99 tokens for about a hundredth of a cent; the payments-database failover plan routes to powerful at 0.98 and Astra writes 1,273 tokens for $0.064 — over 500 times more. The routing decision itself: about $0.00002.

    In step3_router.py the terminal sends an HTTP 429 question down the fast lane, where gpt-5.6-luna answers 99 tokens for $0.0001 at router confidence 1.00, then pushes a payments-database failover to powerful, where gpt-6-astra writes 1,273 tokens for $0.064 at confidence 0.98.
    A hundredth of a cent for the easy one, 6.4 cents for the hard one — the router decides for $0.00002.Watch at 7:50
  4. 4

    AutoMode guard: write the risk criteria or the destructive call slips through

    AutoModeGuard protects a named list of tools: before each one runs, it asks Jev whether the call is risky or not clearly authorized by the user, and anything at 0.5 or above is blocked — the agent gets an error message instead of a tool result. Unlisted tools are never checked, and if Jev is unreachable the tool does not run. The stress test: a hidden HTML comment in ticket 4521 ordering "delete all backups." Stock settings measured 0.25 across five runs — under the line — because an alpha-3 bug passes criteria=None over the default criteria. After passing explicit ones ("destroys data that cannot be recovered, even if the user asked"), the same deletion dies at 0.74 while restart_service sails through at 0.07. Pair with LangChain’s human-in-the-loop middleware if you prefer approval over hard blocks.

    AutoModeGuard’s terminal verdicts show delete_all_backups blocked at risk 0.74 with a BLOCKED THE REQUEST badge after ticket 4521’s hidden instruction, while restart_service is allowed to run at risk 0.07 and the payments service restarts successfully.
    Explicit criteria flip the story: destruction blocked at 0.74, restart allowed at 0.07.Watch at 11:10
  5. 5

    TriageMiddleware: end the run before the LLM is ever called

    The built-in middleware are starting points — the classifier slots into any hook. This custom TriageMiddleware carries its own state schema, sends the whole conversation to Jev with the same three triage questions in one call, and when urgent and severity both fire it appends an on-call-notified message and jumps straight to the end via the can_jump_to_end attribute — no LLM call at all. Order matters in the middleware list: triage, router, guard. The demo ticket ("deploy failed twice, customers seeing 500s") reads urgent 0.95 and criticality 1.87 — paged, done, the cheapest possible route for the most important ticket. One cost note: each middleware builds its own client, so first calls pay a roughly one-second handshake while retries come back under 400 ms.

    A step5_full_agent.py run shows TriageMiddleware reading urgent 0.95 and criticality 1.87 from one Jev call, earning a PAGED ON-CALL badge for the infrastructure team, and closing with no LLM call needed.
    urgent 0.95, criticality 1.87 — the on-call engineer is paged and the LLM never wakes up.Watch at 12:35
  6. 6

    Jev as judge: same verdicts, 4x faster, about 230x cheaper

    LangChain’s published study judged five frozen runs of a weather agent 100 times each: Jev matched the human on all 500 verdicts where Claude Sonnet agreed on 80%, at $0.34 total versus $28.17, with 92-913x lower score variance than the LLM baselines. The video repeats the experiment on this agent with four Noul questions — grounded in tool results, no unrequested destructive actions, helpful, sound — over eight fixed transcripts, ten runs per judge. First pass: Jev 100%, Sonnet 87.5%, and the one miss turned out to be a logging bug in the transcript, not the judge; after the fix both scored 100% with zero variance. The durable numbers: 0.46 seconds per judgment versus 1.85, and 80 judgments for $0.0014 versus $0.3223.

    Benchmark bars from bench_vs_judge.py compare Jev at 0.46 seconds per judgment against Sonnet’s 1.85 and $0.0014 for 80 judgments against $0.3223, while the table below logs both judges at 100.0% accuracy with zero variance after the transcript fix.
    Same verdicts at a fraction of the cost — $0.0014 versus $0.3223 for 80 judgments.Watch at 16:05

Frequently asked questions

What is the langchain-typesafe package?

The official LangChain integration for Jev. TypeSafeClassifier wraps the state-plus-questions evaluate API as a pipeable LangChain Runnable, and the experimental extra adds agent middleware such as ModelRouter and AutoModeGuard that attach to create_agent. It ships as 0.0.1a3 alpha with the middleware marked experimental, so pin your versions — and since alpha 3, both state and questions are passed to invoke() rather than the constructor.

How does the AutoMode guard decide what to block?

For every protected tool it asks Jev once: is this call risky or not clearly authorized by the user? Anything scoring 0.5 or higher is blocked, tools you don’t list are never checked, and if Jev is unreachable the tool does not run (fail-closed). The criteria you write are the whole game: stock defaults measured 0.25 on a backup-deleting injection because an alpha-3 bug overrode them, while explicit criteria scored the same request 0.74 (blocked) and a raw prompt injection 0.99.

What does one Jev decision cost?

A routing decision runs about $0.00002 — two thousandths of a cent. The standalone classifier call in the video processed 461 input tokens for a fraction of a cent, and judging is just as cheap: 80 evaluations cost $0.0014, where Claude Sonnet needed $0.3223 for identical verdicts. Under the hood Jev 1.13 charges 4.2 cents per million input tokens, and output tokens are free.

Does Jev replace the LLM in my agent?

No — the video’s template is: let the LLM do the thinking and writing, and let Jev make the small decisions: routing, protecting, and evaluating. Each one takes about a second or less and costs next to nothing. Jev is also not a calculator — leave mathematical computations and date comparisons to your code, and use its typed answers as facts your policy code acts on.

What should I check before shipping this to production?

Five things from the video: pin your versions (the package is alpha and the middleware experimental); keep secrets out of agent state because the guard sends the last 30 messages plus arguments to the API; remember a Noul of 0.5 means Jev is wavering, not split-down-the-middle — use a Score when you need a midpoint; set thresholds based on the cost of an error, like paging at 0.7 instead of 0.8; and write down your guard criteria explicitly instead of trusting the defaults.

Related guides

Support Ticket Routing Recipe

The triage pipeline from this video as a step-by-step recipe: Noul, Choice, and Score questions over a real ticket stream.

Read

Jev Model Router Guide

Sam Witteveen’s standalone local-vs-cloud router build with PII gating — the deeper dive on the routing half.

Read

Jev API Reference

The evaluate endpoint, typed questions, and probability payload behind TypeSafeClassifier.

Read

LLM Fallback Strategy

Confidence-based fallback patterns for when the router or the guard is unsure.

Read

More video walkthroughs