How To Use Decision Models for Evals
Decision models have exploded onto the scene recently, starting with TypeSafe’s Jev, followed by Perplexity’s Decider, Liquid’s d1, Cloudflare’s Clef, and OpenAI’s Decisions API1.
Proof1 independent outlet
Key points
- Decision models are an appealing low-cost alternative to LLM judges2 for automated evals.
- In this post, we’ll start by showing how to tune thresholds to get the best results from decision models.
- The task is to identify traces containing an error, such as a failed booking or a missed handoff to a human.3 We used the same dataset and broad failure-finding task in our prior post on automated evals.
- We tested Jev 1.13, Perplexity Decider 1.1 27B, Liquid d1 and d1-ultra, Cloudflare Clef Flash, and OpenAI Decisions (GPT-6 Luna)1.
Sources (1)
- [1]How To Use Decision Models for EvalsHamel Husain · Oct 20, 07:00 AM
Decision models have exploded onto the scene recently, starting with TypeSafe’s Jev, followed by Perplexity’s Decider, Liquid’s d1, Cloudflare’s Clef, and OpenAI’s Decisions API1.
Decision models are an appealing low-cost alternative to LLM judges2 for automated evals.
Extractive summary: sentences quoted from the sources.
Before this
- Oct 20, 2026Do Diffusion LLMs Scale? Testing Mercury 3 + Decide
- Oct 19, 2026Introducing Clairic, a new System One class of extraction models
- Oct 19, 2026sgl-project/sglang v0.5.22
- Oct 19, 2026Clef is available on Ollama's cloud
- Oct 18, 2026Narrow Decision Models: A System-1 Architecture for Calibrated, Typed Decisions
- Oct 18, 2026P(Yes): 15$\times$ Improvements via Test-Time Scaling of Decision Models
