# AI Decision Models: Latency, APIs, and Architecture

URL: https://3ammarketer.com/news/ai-decision-models-benchmarks-pricing
Published: 2026-10-10

> AI decision models replace autoregressive generation with single-pass scoring over fixed schemas, dropping median latency to 38.8 ms with zero output fees.

AI decision models evaluate structured choices in a single forward pass instead of generating autoregressive text, cutting median evaluation latencies to as low as 38.8 ms while charging zero output token fees. By outputting direct probabilities over discrete schemas rather than streaming conversational strings, these architectures eliminate decoding overhead for high-throughput routing. [2][3]

## What is an AI decision model?

An AI decision model processes both a prompt and candidate options simultaneously in a single forward pass, calculating normalized probability distributions across discrete targets without an autoregressive decoding loop. Cloudflare, Amazon, and OpenAI have developed systems sharing this design, which avoids conversational JSON text generation and downstream parsing regexes. [1][3]

![Cloudflare company logo.](https://dygolixznokmekyvajbb.supabase.co/storage/v1/object/public/images/content/4b5a7a453a54840d707b0cdc.jpg)

## Hosted APIs: Jev versus OpenAI Decisions

TypeSafe AI released its Jev AI model on September 15, 2026, opening general signups on September 20, pausing them two days later, and restoring them on September 27. Within a week of release, over 100,000 users joined the Discord server, around 2,000 GitHub projects used the model, and approximately 13 percent of paid teams on Vercel's AI Gateway adopted it within 24 hours. On October 1, TypeSafe AI said it had onboarded a quarter of the Fortune 500. [1][5]

Both Jev and OpenAI Decisions support three question formats: yes/no, choices, or scores. Jev evaluates up to 255 candidate options in a single POST /v1/systemone request at 4.2 cents per million input tokens with zero output token fees. OpenAI's Decisions API charges 10 cents per million input tokens with zero output token fees, using the gpt-6-luna decision model to add multimodal image support over Jev's text-only 32k context interface. [2][3][4]

Input token pricing across commercial decision models, in cents per million tokens, October 2026. [4]

- Jev: 4.2 ¢ per 1M tokens
- OpenAI Decisions: 10 ¢ per 1M tokens

## Benchmark accuracy and latency across Clef and Jev

Cloudflare open-sourced Clef and Clef-flash on Hugging Face under an Apache 2.0 license, running on Workers AI with a 64k context window and a vision encoder. Clef relies on a Qwen backbone with a prefill-only forward pass to score schema choices non-autoregressively. In functional evaluations, Clef handled website fetching, rendering, and classification in 2.2 seconds compared to 4.7 seconds for gpt-oss-120b. [2]

![Cloudflare blog post announcing the open-source Clef and Clef-flash decision models.](https://dygolixznokmekyvajbb.supabase.co/storage/v1/object/public/images/content/58af34073431c81828c9c849.png)

Inference latency and accuracy figures across hosted decision models. [2]

| Model | Median Latency | p95 Latency | BFCL Exact | API-Bank Acc |
| --- | --- | --- | --- | --- |
| Clef-flash | 38.8 ms | 122.4 ms | 98.76 | 93.11 |
| Clef | 209.3 ms | 238.6 ms | 98.47 | 91.93 |
| Jev | 524.1 ms | 536.0 ms | 95.75 | 88.19 |

BFCL Exact benchmark scores across hosted decision models, October 2026. [2]

- Clef-flash: 98.76
- Clef: 98.47
- Jev: 95.75

Median inference latency across hosted decision models, in milliseconds, October 2026. [2]

- Clef-flash: 38.8 ms
- Clef: 209.3 ms
- Jev: 524.1 ms

Clef-flash achieved 97.73 on Home appliances case exact evaluations and 90.93 macro-F1 on BANKING77, while Clef reached 97.43 macro-F1 on CLINC150+OOS and 69.19 on ToolRet nDCG@10. Across specialized enterprise tests, Clef scored 64.7 on invoice processing, 76.3 on customer service, and 62.9 on security incidents. Jev outperformed both Cloudflare models on the When2Call routing test at 80.97 accuracy. [2]

## Local deployment with Strands, Laya, and Kev

Teams running [self-hosted AI agents](/blog/self-hosted-ai-agents) can deploy open-weight checkpoints to avoid external API calls. Strands Decider 2B replaces the LM head of Qwen3.5-2B with a pointer head containing just over 1 million parameters and fine-tunes the torso with a rank-16 LoRA adapter, recording a median latency of around 115 ms on an Nvidia RTX 3090 and around 153 ms on an M3 MacBook. It ranked 3rd of 33 in the 2B class on JevBench's public set, and 1st of 30 when excluding models just over 2B. [6]

Laya by Convai Innovations provides Apache 2.0 checkpoints between 322 million and 421 million parameters using ModernBERT-large and mmBERT-base. A single decision processes in 39.5 ms on its base English checkpoint and 32.8 ms on its multilingual checkpoint on an Nvidia Tesla T4 GPU, reaching throughputs of 103 to 332 decisions per second under batching. Jared Palmer's Kev project provides Qwen-based tiers at Kev-0.8B, Kev-4B, Kev-9B, and Kev-27B at 51 gigabytes. [3]

Limit: The cited sources do not report memory footprints or batch scaling metrics for Kev checkpoints, nor do they provide multi-node deployment benchmarks beyond single Nvidia RTX 3090 and Tesla T4 setups.

For front-door classification and agent tool calls, routing decisions should move from autoregressive LLMs to single-pass models whenever candidate sets are bounded. Edge setups with sub-50 millisecond budgets can start with Clef-flash or a local checkpoint such as Strands Decider 2B, the fastest options in the figures above. [2][3][6]

## Sources

1. [Understanding Jev, the new model everyone is talking about](https://www.understandingai.org/p/understanding-jev-the-new-model-everyone)
2. [Introducing Clef: our open-source decision models, and new RL fine-tuning platform | Cloudflare Blog](https://blog.cloudflare.com/clef-decision-models/)
3. [AI Decision Models: Jev, Laya, and Open Alternatives](https://www.layer3labs.io/guides/system-one-decision-models)
4. [Release: llm-openai-decisions 0.1a0](https://simonwillison.net/2026/Oct/6/llm-openai-decisions/)
5. [Decision Models: The Next Shoe to Drop | Jamin Ball (@jaminball) on X](https://x.com/jaminball/article/2106044133353853262)
6. [Introducing Strands Decider 2B: a small, open source, decision model | Strands Agents](https://strandsagents.com/blog/introducing-strands-decider/)
