Clef-Flash Tutorial: Install and Run Cloudflare's 9B Multimodal Decision Model Locally on Ubuntu

This is the from-zero install of Clef-Flash, Cloudflare's 9-billion-parameter multimodal decision model, following robotics professor Aleksandar Haber's video frame by frame — every number below was read off the screen of the recording. Positioning first: Clef is built for structured decision-making rather than free-form chatting. A test means defining a text state, optionally passing a raw image, asking multiple-choice questions, and reading calibrated JSON probabilities for every option. The full install path on Ubuntu: sudo apt update && sudo apt upgrade; mkdir testClef2 && cd testClef2; python3 -m venv env1 && source env1/bin/activate; pip install torch transformers accelerate pillow huggingface_hub torchvision — the minimum set, which the creator says took a while to figure out and totals roughly 2–3 GB. Optionally, pip install causal-conv1d flash-linear-attention speeds up inference; these two are not required, need the NVIDIA CUDA toolkit and GCC because they compile locally on first install, and the on-screen run finished with Successfully installed causal-conv1d-1.7.0 einops-0.8.2 fla-core-0.5.2 flash-linear-attention-0.5.2 ninja-1.13.2. The model itself arrives in Python: path = snapshot_download("Cloudflare/clef-flash"), sys.path.insert(0, path), then from joint_schema_model import load_release_model, systemone — the joint_schema_model module ships inside the repo, which is why the path goes on sys.path. The model card's Usage section visible in the video says the stack was tested with torch 2.11 and transformers 5.10.2 on a single H200, and that image and video inputs also need pillow. Demo one, text only: a record whose state is an invoice (vendor Acme, total 1250.0, currency USD, status overdue) with a choice question "What is the invoice status?" over paid/overdue/draft plus a noul (yes/no) question "Is the total above 1000 USD?" returns status {'draft': 0.00969, 'overdue': 0.97277, 'paid': 0.01754} and large {'true': 0.97344} — correct on both, and the verdict lands right after the weights load (760/760, about two seconds once cached). Demo two, multimodal: a photo of a smartphone with a cracked screen packed in a box with crumpled paper (the creator generated it with an AI image engine; any photo works) goes in as images: [Image.open("test_image.jpeg")] with the state "Analyze the condition of the smartphone and its packaging.", a device_damage choice (none / scratched / cracked), and a has_packing_materials noul. Output: {'choice': 'cracked', 'confidence': 0.9626, 'probabilities': {'none': 0.015, 'scratched': 0.0225, 'cracked': 0.9626}} and has_packing_materials noul 0.957 — the paper is visibly in the box, and the model saw it. Hardware: on the creator's RTX 3090, nvtop showed the python3 test2.py process at 18,666 MiB with total card usage at 20.2 of 24 GiB, 62°C; the code comment itself says the 9B model loads natively in 24 GB VRAM. One API gotcha shown live: the systemone record requires the "model" and "state" keys — leave "model" out and you get ValueError: model and state are required. Honesty boundaries: every probability here is the creator's demo run on his own demo data, recorded once; the like count (358) and library versions are as they were at recording; and the multimodal image input shown here is a Clef capability that this site's Jev coverage has no equivalent for — that difference is real, and so is the warning that a single demo photo proves the plumbing, not benchmark-grade accuracy.