AI Decision Models: Latency, APIs, and Architecture
AI decision models replace autoregressive generation with single-pass scoring over fixed schemas, dropping median latency to 38.8 ms with zero output fees.
AI decision models evaluate structured choices in a single forward pass instead of generating autoregressive text, cutting median evaluation latencies to as low as 38.8 ms while charging zero output token fees. By outputting direct probabilities over discrete schemas rather than streaming conversational strings, these architectures eliminate decoding overhead for high-throughput routing.23
What is an AI decision model?
An AI decision model processes both a prompt and candidate options simultaneously in a single forward pass, calculating normalized probability distributions across discrete targets without an autoregressive decoding loop. Cloudflare, Amazon, and OpenAI have developed systems sharing this design, which avoids conversational JSON text generation and downstream parsing regexes.13

Hosted APIs: Jev versus OpenAI Decisions
TypeSafe AI released its Jev AI model on September 15, 2026, opening general signups on September 20, pausing them two days later, and restoring them on September 27. Within a week of release, over 100,000 users joined the Discord server, around 2,000 GitHub projects used the model, and approximately 13 percent of paid teams on Vercel's AI Gateway adopted it within 24 hours. On October 1, TypeSafe AI said it had onboarded a quarter of the Fortune 500.15
Both Jev and OpenAI Decisions support three question formats: yes/no, choices, or scores. Jev evaluates up to 255 candidate options in a single POST /v1/systemone request at 4.2 cents per million input tokens with zero output token fees. OpenAI's Decisions API charges 10 cents per million input tokens with zero output token fees, using the gpt-6-luna decision model to add multimodal image support over Jev's text-only 32k context interface.234
Benchmark accuracy and latency across Clef and Jev
Cloudflare open-sourced Clef and Clef-flash on Hugging Face under an Apache 2.0 license, running on Workers AI with a 64k context window and a vision encoder. Clef relies on a Qwen backbone with a prefill-only forward pass to score schema choices non-autoregressively. In functional evaluations, Clef handled website fetching, rendering, and classification in 2.2 seconds compared to 4.7 seconds for gpt-oss-120b.2

| Model | Median Latency | p95 Latency | BFCL Exact | API-Bank Acc |
|---|---|---|---|---|
| Clef-flash | 38.8 ms | 122.4 ms | 98.76 | 93.11 |
More on Clef-flash
| ||||
| Clef | 209.3 ms | 238.6 ms | 98.47 | 91.93 |
More on Clef
| ||||
| Jev | 524.1 ms | 536.0 ms | 95.75 | 88.19 |
More on Jev
| ||||
Clef-flash achieved 97.73 on Home appliances case exact evaluations and 90.93 macro-F1 on BANKING77, while Clef reached 97.43 macro-F1 on CLINC150+OOS and 69.19 on ToolRet nDCG@10. Across specialized enterprise tests, Clef scored 64.7 on invoice processing, 76.3 on customer service, and 62.9 on security incidents. Jev outperformed both Cloudflare models on the When2Call routing test at 80.97 accuracy.2
Local deployment with Strands, Laya, and Kev
Teams running self-hosted AI agents can deploy open-weight checkpoints to avoid external API calls. Strands Decider 2B replaces the LM head of Qwen3.5-2B with a pointer head containing just over 1 million parameters and fine-tunes the torso with a rank-16 LoRA adapter, recording a median latency of around 115 ms on an Nvidia RTX 3090 and around 153 ms on an M3 MacBook. It ranked 3rd of 33 in the 2B class on JevBench's public set, and 1st of 30 when excluding models just over 2B.6
Laya by Convai Innovations provides Apache 2.0 checkpoints between 322 million and 421 million parameters using ModernBERT-large and mmBERT-base. A single decision processes in 39.5 ms on its base English checkpoint and 32.8 ms on its multilingual checkpoint on an Nvidia Tesla T4 GPU, reaching throughputs of 103 to 332 decisions per second under batching. Jared Palmer's Kev project provides Qwen-based tiers at Kev-0.8B, Kev-4B, Kev-9B, and Kev-27B at 51 gigabytes.3
For front-door classification and agent tool calls, routing decisions should move from autoregressive LLMs to single-pass models whenever candidate sets are bounded. Edge setups with sub-50 millisecond budgets can start with Clef-flash or a local checkpoint such as Strands Decider 2B, the fastest options in the figures above.236