Anthropic FlashProve: open-source proofs of which model answered, 110 tokens/s

Anthropic has released FlashProve, an open-source research prototype that lets a model provider prove, with a cryptographic proof, that a reply came from one specific committed set of model weights, without revealing those weights. In the benchmark Anthropic reports, a single Nvidia H100 produces proofs for Meta's Llama 4 Maverick at about 110 tokens per second, with a proof of roughly 19 MB per 1,024-token request. That is fast compared with earlier work, but it is still about 70 to 95 times the GPU time of simply running the model, and the release has real limits that Anthropic lists itself.

Key points

• Anthropic published FlashProve on 2 November 2026 as code, a technical report, a protocol specification and Lean proofs, under the Apache 2.0 licence.
• On one H100, it proves Llama 4 Maverick (400B parameters, 4-bit weights) at about 110 tokens per second, with proofs of about 19 MB for a 1,024-token request.
• Proving costs roughly 72 times the GPU time of running the model on vLLM (66 to 77 across runs), and about 94 times at the highest load Anthropic tried.
• A verifier needs no GPU: it downloads a commitment of about 6.5 MB and checks a proof in 17 seconds on 24 CPU threads.
• It is a research prototype: contexts up to 1,024 tokens, Mixture-of-Experts models, greedy decoding, and the repository says it is not maintained and not accepting contributions.

What happened

On 2 November 2026 Anthropic's Frontier Red Team published "Progress on provable inference for large language models". The authors are Roy Rinberg and Alex Allain of Anthropic, with collaborators at EPFL, MIT and fellowship programmes, and the post names FlashProve as the system. Anthropic says it is releasing five artifacts: the blog post, a technical report, a protocol specification that Claude wrote, executable code (a prover and two verifiers, one fast in C++ and one in Lean), and Lean proofs of the top-level security statements together with an interactive visualiser to help auditors. Code, Lean proofs and documentation are under Apache 2.0.

The problem it targets is simple to state. A user of an AI service cannot tell whether the reply came from the model the provider names or from a cheaper one swapped in by a routing service, or from a stream an attacker altered. Anthropic's post lists four settings where that matters: regulators who want to confirm hardware is serving approved models, companies wanting to detect a tampered inference server, developers wanting to show a claimed model really reviewed their code, and rules that require only approved models to be served.

A provider first publishes a commitment, a short fingerprint pinning exactly one set of weights. For each reply it can then attach a proof that the committed model produces these exact output tokens for this prompt. Anyone holding the commitment, the prompt and the reply can check the proof without the weights. Anthropic says the proof is zero-knowledge for the weights, the intermediate values and which experts ran at each layer, though the model's architecture is not hidden.

Background

The field is usually called zkML. Anthropic's post traces how slow it has been. Early systems proved small networks: zkCNN (2021) reached VGG-16 at 88.3 seconds per proof. Transformer proofs then arrived: a GPT-2 proof took about 159 seconds per token, and Modulus Labs' 1.5-billion-parameter GPT-2 XL needed over 90 hours. In 2024 zkLLM proved LLaMA-2-13B on a 2,048-token input in about 15 minutes on GPU. The only earlier proof of Llama 4 Maverick that Anthropic knows of, VerInf, reported 14.3 hours for 1,000 tokens on a DGX Spark, with a 93.6 GB proof that took almost a day to verify.

Anthropic also mentions a commercial effort, Attestable, which it says proved Gemma 4 31B at 53 to 77 tokens per second on a single H100, but has not released a matching open-source library. We could not check that figure independently; it is Anthropic's description of someone else's claim.

The post connects to an earlier commitment. Anthropic's Frontier Safety Roadmap set a goal of a provable-inference prototype by 30 September 2026. Anthropic says it met that goal through a separate project that signs its inference traffic and recomputes inference on a trusted server, and that FlashProve pursues the same aim by a different method. A footnote says an update on 19 October 2026 renamed the goal "inference verification", because "provable inference" suggests a narrower approach than Anthropic believes necessary.

How it works

Two phases. In the offline phase the prover commits to the model weights. Anthropic measured about 39 minutes for this, once per model on its test server (one H100 and 24 CPU cores), and says the resulting commitment is good for about 16 million requests before the provider must commit again. In the online phase a batch of 24 requests took the prover about 223 seconds, which is the 110 tokens per second figure, and each proof was 18.78 MB.

The underlying proof is a zkSNARK built from an interactive oracle proof compiled with hash functions, a recipe used by blockchain layer-2 networks such as Starknet and Mantle, and Anthropic says it offers post-quantum security. Three design choices make the speed possible, according to the post. The prover's work follows the experts that actually ran: a Maverick token uses about 17B of 400B parameters. Requests are proved in batches, after the fact, in one parallel pass similar to prefill. And weights are streamed from host memory a segment of layers at a time, so one 80 GB GPU with about 290 GB of host memory beside it suffices.

Proving after the fact has a cost: you get guarantees about your traffic with a delay, and recomputation adds complexity. Anthropic argues that in-line proving may sacrifice security for speed while post-hoc proving need not.

Because Claude helped write much of the system, Anthropic says it counters the risk of hard-to-evaluate decisions by supplying top-level statements in Lean for completeness, soundness and zero knowledge. Soundness, in the repository README, is stated as a forged proof passing with probability about q/2^124 + 2^-101 after q hash queries. Anthropic is explicit that the fast C++ verifier, the one to use day to day, carries no formal guarantee and merely agrees with the Lean verifier on every test proof.

The numbers

FlashProve (Llama 4 Maverick, 1 H100)110

DeepProve (Gemma 3, 1 machine)1.4

SASH (Llama 3.1 8B, 1 H100)1.25

VerInf (Llama 4 Maverick, DGX Spark)0.02

Reported proving speed, tokens per second (different models and hardware, not a like-for-like race) (tokens/s)

	tokens/s
FlashProve (Llama 4 Maverick, 1 H100)	110
DeepProve (Gemma 3, 1 machine)	1.4
SASH (Llama 3.1 8B, 1 H100)	1.25
VerInf (Llama 4 Maverick, DGX Spark)	0.02

Source: Anthropic, "Progress on provable inference for large language models" (2 November 2026)

The only like-for-like pair is FlashProve and VerInf, which both proved Llama 4 Maverick, on different hardware. The other rows are other models and settings, listed because Anthropic cites them. The 110 figure is prefill-style proving over finished text, not live generation speed.

What it means

For developers and researchers, the result changes what seems feasible. Anthropic's own framing is that overhead of roughly two orders of magnitude, about 70 to 95 times in its measurements, is far lower than previously expected. That is still expensive for routine traffic, so a realistic use is proving a sample of tokens, or high-stakes responses such as an audit or an evaluation run, rather than every chat message.

For evaluators and regulators, the appeal is auditing without access to weights. An independent evaluator could confirm that the model they tested is the one served. That is a claim Anthropic makes about potential uses; no regulator is reported to be using the system.

For everyone else, note what a proof does not say. It attests which model produced a reply, not that the reply is accurate or safe. The blockchain.news report on the release makes the same point and also notes Anthropic gave no independent benchmark and no date for a product.

What we don't know yet

• Whether others reproduce the numbers. Anthropic shares code for readers to verify, but no independent result has been published that we found.
• How well it carries to other models. Anthropic believes progress "should transfer", but the system relies on Mixture-of-Experts structure.
• Real-world accuracy. The proof covers an integer version of the weights under greedy decoding; a floating-point or sampled reply could depart from it, and Anthropic says it has not covered adversarially designed prompts.
• Whether the timing leaks information. Side-channel attacks such as measuring proof time are out of scope for this release.
• Whether a product will follow. The README states the project is not maintained.

What to watch next

Watch for independent reproductions and for contexts longer than 1,024 tokens, which Anthropic names as the main gap. Follow-up work on dense models and on hiding the architecture is flagged as future work. If you want to try it, the README offers a CPU-only route that verifies a stored proof in under two seconds on Linux with g++ 13, while the Llama 4 Maverick route needs an H100, about 330 GB of RAM and 245 GB of disk.

Sources

• Anthropic: Progress on provable inference for large language models (primary)
• FlashProve repository on GitHub (README, Apache 2.0)
• Anthropic: FlashProve technical report
• Blockchain.News: Anthropic reports progress on provable inference