Jev 101 logo
Jev 101

Guides / illustrated walkthrough

Jev RAG Reranker: Policy-Steered Reranking for Retrieval-Augmented Generation

Prompt Engineering walks through Jev as a policy-steered reranker inside a RAG pipeline: criteria written as executable policy filter top-100 chunks down to a top-5, rankings flip when the written policy flips, and BM25’s 21% top-1 accuracy climbs to 54% at a cost that stays flat as chunks grow.

Quick takeaway

A RAG pipeline is full of decision points — rerank, filter, cite — and this 11-minute build puts Jev in charge of the rerank one. The setup: candidate chunks come back from vector or full-text search for a query like "how do I fix a 401 unauthorized error from the API", and a Jev reranker built on the binary Noul primitive scores them against criteria that act as an executable policy — true when a passage directly answers the question with the specific answer asked for, false when it only shares words or topics. Because the criteria are written down, the same corpus ranks differently under a strict authoritative-version-2 policy versus a looser fast-workaround policy: steerability a cross encoder cannot offer (it is frozen in time) at a cost an LLM reranker cannot match (Gemini Flash’s bill grows roughly linearly with chunk size while Jev stays flat; a batch of decisions that takes about 40 seconds sequentially finishes in 7.6 seconds with 64 concurrent pools). On the benchmark, BM25’s 21% top-1 accuracy rose to 54% with Jev reranking — and reranking a realistic top-K of 10–20 chunks would score higher still.

Video source

Prompt Engineering

11:27UhGH8cNG0qs
Visit channelWatch original

Step-by-step walkthrough

  1. 1

    Find the decision points where your RAG pipeline needs a policy

    RAG looks like plumbing, but the video opens by marking the three places where it is really decision-making: the rerank step that narrows retrieval from top-100 to top-5, a filter gate that passes or rejects chunks, and the citation check on what the LLM finally answers. Jev slots into each gate because every gate is a typed decision, not a generation task. The running example is a documentation system with several competing policy versions that must be resolved at query time. The failure mode is subtle: vector distance only measures semantic proximity, not correctness under your business logic, so answers silently drift when the official API docs change — or when the same query deserves an in-depth architectural explanation instead of a quick copy-paste snippet.

    Blackboard walkthrough of a RAG pipeline where a query flows through retrieve and top-100 into three numbered Jev decision points: a rerank Score, a pass-or-fail filter Gate, and a Cite check that validates generated answers.
    Three Jev gates in one pipeline: score the rerank, gate the filter, check the cite.Watch at 0:48
  2. 2

    Compare the three rerankers: frozen, expensive, or steerable

    Before reranking, the video frames Jev’s three primitives: a binary true/false classifier, a choice ranker driven by a criterion, and a score ladder that assigns different scores to options. Reranking exercises them, and the whiteboard compares the three tools you could reach for. A cross encoder scores question-and-chunk pairs and is accurate, but it cannot follow instructions — it is frozen in time, which is exactly why it fails when your organization’s policy changes while the underlying business logic stays the same. An LLM reranker does follow instructions, but it is prohibitively expensive and noticeably slower. Jev is pitched as the third option: steerable like the LLM, priced closer to the cross encoder, which turns a policy change into a configuration edit instead of a re-training job.

    Hand-drawn top-100 to top-5 funnel asking whose policy decides the cut, branching into a cross-encoder labeled cannot steer, an LLM reranker labeled $$$ cost, and Jev labeled steerable plus cheap.
    Cross-encoders cannot steer, LLM rerankers cost $$$ — Jev is steerable and cheap.Watch at 2:11
  3. 3

    Turn criteria into an executable policy, not a prompt suggestion

    The notebook (official Python SDK, or REST API if you prefer; signing up includes $5 of free credit) wires a reranker around Jev’s binary Noul primitive, which returns a calibrated probability instead of free text. The trick is treating criteria as an executable policy: rather than soft prompt instructions, the criteria draw strict decision boundaries — true when the passage directly answers the question with the specific answer it asks for, false when it merely shares words or topics without answering. A sanity check runs four passages — a direct refund answer, a topical overlap, a keyword match, and an unrelated chunk — and the direct answer takes the highest probability while the rest score low, giving you a threshold classifier: accept above the line, discard below. Throughput holds up too: the same batch of decisions drops from about 40 seconds sequentially to 7.6 seconds using the 64 concurrent pools.

    Google Colab cell titled Act 2 The Core Mechanism defining criteria as executable policy: a Noul that returns a calibrated 0-1 probability, true and false criteria asking whether a passage directly answers the question, four test passages including a refund answer, and a loop printing each score.
    Criteria as executable policy: strict true/false boundaries instead of prompt suggestions.Watch at 5:52
  4. 4

    Steer the same corpus into different rankings with two policies

    The benchmark query is deliberately mundane — how do I fix a 401 unauthorized error from the API — with candidate chunks returned by vector or full-text search carrying normal metadata. The reranker then runs under two written policies: an authoritative version two policy that is strict about sourcing, and a fast workaround policy that is deliberately less strict. The bar chart is the payoff: the same eight candidates, drawn from official docs, Stack Overflow, blog, and forum pages at v1 and v2, come back in a different order under each policy. That is the whole point of a steerable reranker — the ranking follows the criteria you write, so changing what your organization means by "best answer" is a one-line edit instead of a new model.

    Policy Steering in Jev Reranker bar chart ranking eight candidate chunks c1 through c8 from official docs, Stack Overflow, blog, and forum sources at v1 and v2, with Policy A Official and Policy B Fast bars flipping order across candidates.
    Same eight chunks, two policies, two different rankings — the steerability payoff.Watch at 8:20
  5. 5

    Price the reranker: flat against a linear LLM bill

    The cost benchmark pits Jev against Gemini Flash-Lite and Gemini Flash acting as LLM rerankers, running the same prompt and query through each. At small chunk sizes Jev’s cost and latency sit close to Flash-Lite, but the two charts diverge as chunks grow: the LLM cost curve climbs roughly linearly with chunk length while Jev stays nearly flat — and the video notes that larger chunks are exactly where this approach shines. Throughput matters as much as unit cost: the concurrency test measured about 17 decisions per second even running serially. The practical read is that you can afford to re-score the whole candidate list on every request, and the bill stops growing every time retrieval starts returning longer passages.

    Side-by-side benchmark charts pitting Jev against Gemini Flash-Lite and Gemini Flash: cost and latency bars for 200 chunks where Flash towers, and a cost-scaling curve where Gemini Flash climbs linearly with chunk length while Jev stays near zero.
    Jev stays flat as chunk size grows; the Gemini reranker bill climbs linearly.Watch at 9:20
  6. 6

    Check the ceiling: 21% to 54% top-1 accuracy

    The final chart is the reality check. A BM25 full-text-search baseline answers only 21% of test queries with its top-1 chunk; adding Jev as a reranker pushes top-1 accuracy to 54% — and the video is careful to note this is the hardest metric, since reranking over a normal top-K of 10 to 20 chunks scores considerably higher. The chart’s subtitle carries the second insight: the reranker achieves 100% of candidate recall, so the misses that remain are bounded by the initial retrieval, not the reranker. The closing argument generalizes beyond reranking: anywhere an existing workflow makes a decision — rerank, filter, cite, validate — Jev can be plugged in as a drop-in replacement, and the decisions do not have to be binary.

    Retrieval Recall Ceiling chart showing BM25 plus Jev Rerank reaching 54 percent top-1 accuracy over a 21 percent BM25 baseline, annotated with recall ceiling 54 percent and a note that the reranker achieves 100 percent of candidate recall.
    Top-1 accuracy: 21% on BM25 alone, 54% with Jev reranking.Watch at 10:17

Frequently asked questions

What is a Jev RAG reranker?

A rerank step in which Jev scores every retrieved chunk against written criteria using its binary Noul primitive, which returns a calibrated probability that the passage directly answers the query. Unlike a cross encoder it follows instructions, so the ranking enforces your business policy, and unlike an LLM reranker its cost stays flat as chunks grow.

Why not just use a cross encoder or an LLM as the reranker?

Cross encoders only measure semantic proximity between question and chunk — they cannot follow instructions, so they fail exactly when your policy changes while the business logic stays the same. LLM rerankers do follow instructions but are prohibitively expensive and slower, with cost climbing roughly linearly as chunk size grows. Jev combines instruction-following with a cost close to the cross encoder.

How does policy steering change the ranking?

You write the criteria as an executable policy — in the video, a strict authoritative version two policy and a deliberately looser fast workaround policy. The same candidate chunks then come back in a different order under each policy, so changing what your organization means by "best answer" is an edit to the criteria, not a model retrain.

How much accuracy does Jev reranking add?

In the video’s benchmark, a BM25 full-text-search baseline answered 21% of queries correctly at top-1; adding Jev as a reranker pushed that to 54%. The chart also notes the reranker achieves 100% of candidate recall, meaning remaining misses are bounded by initial retrieval — and over a realistic top-K of 10 to 20 chunks accuracy runs higher still.

Can Jev reranking keep up with production traffic?

Yes. The concurrency test measured about 17 decisions per second even running serially — roughly 40 seconds for the full batch — and the same batch finished in 7.6 seconds using the 64 concurrent pools, which is enough throughput to re-score a top-100 candidate list on every query.

Related guides

Jev Model Router Guide

The same Choice, Score, and Noul primitives driving a privacy-gated model router.

Read

LLM Fallback Strategy

Confidence-based degradation patterns for decisions that come back low-confidence.

Read

Jev API Reference

The evaluate endpoint and typed questions behind every Noul reranker call.

Read

What is Jev?

The binary, choice, and score primitives explained beyond the hype.

Read

Jev Playground

Write your own criteria and watch the ranking flip live.

Read

More video walkthroughs