FlashProve Delivers 100 Tokens/s Provable Inference

Anthropic has released FlashProve, an open-source prototype that achieves provable inference at over 100 tokens per second on a single NVIDIA H100 GPU for a 400B-parameter mixture-of-experts model. The system uses zero-knowledge succinct non-interactive arguments of knowledge (zkSNARKs) to let developers verify that an output originated from a specific model without exposing the underlying model weights.12

Cryptographic verification with hidden weights

FlashProve provides provable inference by combining cryptographic commitments with zkSNARKs, selected for their hiding properties and verification efficiency. The protocol enables developers to verify whether a given response came from a specific AI model. The commitment to the model and subsequent inference proofs hide the model weights entirely, though hiding the network architecture is explicitly not a goal of the architecture.12

Throughput, proof size, and the 70x latency trade-off

Generating cryptographic proofs introduces substantial computational overhead compared to standard serving engines. FlashProve's proof generation time takes roughly 70 to 95 times the GPU time required to run the model itself on vLLM.1

For a 400B-parameter mixture-of-experts model, specifically Llama 4 Maverick under 4-bit quantization, FlashProve maintains proof generation throughput exceeding 100 tokens per second on one H100 GPU. The resulting proof footprint is approximately 19 MB per 1,024-token request.1

Hardware limits, CPU verification, and Lean 4 soundness

Verifying outputs does not require high-end accelerators or proprietary access. Anyone can verify the proof on an ordinary CPU in seconds using the cp_verify tool without access to the model weights. A provider attempting to pass off output that is not the committed model's is caught, except with probability about q/2^124 + 2^-101 after q hash queries.3

Proving remains tightly constrained by hardware requirements. The FlashProve prover strictly requires an x86-64 Linux system (kernel 4.14 or later) on a CPU featuring SHA extensions alongside an NVIDIA H100 GPU. The system runs Llama 4 Maverick at full size with a 1,024-token context window on that single H100 setup.3

Research prototype status for developers

Developers evaluating the framework should note its operational status. FlashProve is released under the Apache 2.0 license, covering source code, documentation, and formal Lean proofs. However, version 0.16.0 is strictly a research artifact: the repository is unmaintained and is not accepting external contributions.13