Skip to content
audiotale.net
Sign in
AI learningGuide

How to compare LLM prices: cost per task, not price per million tokens

A step by step guide to counting tokens, thinking, caching and batch discounts so you can compare AI models by what a finished job really costs.

audiotale.net team

· 7 min read

Written by AI from the sources linked below. How we use AI

On this page
  1. In short
  2. Why the price list misleads
  3. Step 1: count the tokens you actually use
  4. Step 2: apply caching correctly
  5. Step 3: use batch pricing where you can wait
  6. Step 4: define "task done" and divide
  7. Step 5: try the cheap fixes before building a router
  8. Why agents make the gap bigger
  9. A short worksheet
  10. Common mistakes
  11. Sources

In short

The price per million tokens on a model's pricing page is not what a job costs you. What you pay is the price per token multiplied by how many tokens the model uses (including hidden "thinking" tokens), plus retries and tool loops, minus caching and batch discounts. The fair way to compare models is cost per task that passes your check. This guide shows how to calculate it, with the billing rules from Google's and Anthropic's documentation.

Why the price list misleads

A pricing page quotes two numbers: dollars per million input tokens and dollars per million output tokens. Two things break the comparison.

First, models use different numbers of tokens for the same job. A model that "thinks" longer writes more output. Second, a task is often not one request. A coding agent reads files, calls tools, fails a test, tries again. Every step is billed.

Our recent coverage shows the effect. When Google opened Gemini 4 Carbon at $2 per million input and $10 per million output tokens, the same introductory price as Gemini 4 Argon, the two models still did not cost the same to run. A third-party write-up of Artificial Analysis data lists Carbon (High) at $7.76 per task and 123K output tokens per task, against $7.68 and 116K for Argon (High). Same price list, a different bill, because the tokens differ.

Cost per task on the Artificial Analysis index run, list prices (as listed by eesel.ai)US$ per task
Cost per task on the Artificial Analysis index run, list prices (as listed by eesel.ai) (US$ per task)
US$ per task
Claude Opus 5.5 (Max)10.03
Carbon (High)7.76
Argon (High)7.68
GPT-6 Astra (Max)6.24
Kimi K3.1 Pro3.68
GPT-6.1 Sol (Max)1.27

Source: Artificial Analysis data via eesel.ai

Read it with the scores next to it: the same source lists index scores of 58.4 for Opus 5.5 (Max), 59.4 for Carbon (High), 50.4 for Argon (High), 56.9 for GPT-6 Astra (Max), 46.0 for Kimi K3.1 Pro and 54.5 for GPT-6.1 Sol (Max). Cheapest per task is not the best model, and the best is not the cheapest. That trade-off is the point.

Step 1: count the tokens you actually use

Billing is per token, and several kinds of token exist:

  • Input tokens: your prompt, files, conversation history, tool results.
  • Output tokens: what the model writes.
  • Thinking (reasoning) tokens: the model's internal work before it answers. Google's pricing page lists output price as "including thinking tokens", so they are billed as output, normally the more expensive side.
  • Cached input tokens: input the provider has seen recently, billed at a lower rate (next step).

Read the usage fields in the API response and keep them. Most providers return input and output counts for every call. Without your own measurements you are guessing.

Step 2: apply caching correctly

If your requests repeat a long prefix (a system prompt, a codebase, a document), caching can cut that part of the bill sharply. The rules differ by provider.

According to Anthropic's documentation:

Operation Price relative to base input
5-minute cache write 1.25x
1-hour cache write 2x
Cache read (most models) 0.1x
Cache read (Opus 5.5, Sonnet 5.5) 0.05x
Cache read (Fable 5.2, Mythos 5.2) 0.025x

The default cache lifetime is five minutes and is refreshed at no extra cost each time the entry is used; an hour-long cache costs more to write. There is also a minimum prompt length: for example 512 tokens for Fable 5.2, Opus 5.5, Sonnet 5.5 and Haiku 5.5, and 4,096 for Haiku 4.5. Below the minimum the prompt simply isn't cached and no error is returned, so check that cache_read_input_tokens is above zero.

Google's pricing page lists a separate context caching price that is lower than the standard input price, and for some models an hourly storage charge, for example "$1.00 / 1,000,000 tokens per hour". Google's own cost article for developers adds a practical warning: a cache belongs to one model, so switching models mid-conversation can forfeit the discount. It says caching cuts resend costs by 75% to 90%.

Step 3: use batch pricing where you can wait

Google's pricing page describes its Batch API as a 50% cost reduction. If your work does not need an immediate answer (evaluations, bulk summarisation, nightly jobs), batch pricing halves the cost. Anthropic notes its caching multipliers stack with the Batch API discount and other modifiers. Check each provider's current terms, because discounts and stacking rules change.

Step 4: define "task done" and divide

Cost per task is only meaningful if "done" is defined by a check you can run: tests pass, a schema validates, a reviewer approves. Google's developer article puts it as the total spend across all loop iterations, tool calls, retries and thinking passes, divided by the number of tasks that pass verification. Use that:

cost per successful task = total spend / tasks that passed your check

An illustrative example, with made-up numbers for the arithmetic only: model A costs $0.50 per attempt and passes 60 of 100 tasks, so spend is $50 for 60 successes, about $0.83 per success. Model B costs $0.80 per attempt and passes 95 of 100, so $80 for 95 successes, about $0.84 per success. B is "more expensive" per attempt and about equal per success, and it hands you far fewer failures to clean up. If failures cost human time, B wins.

Step 5: try the cheap fixes before building a router

Google's article tested 50 auto-graded coding-agent prompts. Each setup ran once, so treat its results as one data point. Its findings are still a useful checklist:

  • Setting Gemini 3.8 Flash to low thinking solved all 50 tasks and was the baseline any router had to beat. Start there before building anything complex.
  • Capping maxOutputTokens at 300 did not reduce thinking on easy tasks, and the capped model broke 7 of 25 hard tasks, because the limit is a ceiling and not a thinking budget.
  • A router that sent tasks to Flash-Lite first and escalated flagged ones solved all 50 and cost less than low thinking everywhere. A keyword and length heuristic did well on its development set (15/16) but fell to 70.6% on held-out tasks.
  • Keep long, context-heavy turns on one model so caching keeps working.

Why agents make the gap bigger

Google's developer article argues that per-token prices are falling while agent costs rise. The reason is structural. A chatbot answer is one request. An agent runs a loop: it reads, plans, calls a tool, reads the result, and tries again, and each turn resends a growing context. The bill grows with the number of turns, not just the length of the final answer.

Three habits keep that under control. Put stable material (instructions, reference files) at the start of the prompt so it can be cached. Stop loops that are not converging: a retry limit is cheaper than a fifth failed attempt. And measure per task, not per call. A dashboard that shows spend per request can look healthy while one stubborn task burns through dozens of requests.

It also helps to separate the tasks. Short, standalone questions can go to a smaller model; long, context-heavy sessions should stay on one model so the cache survives. Google's article reached the same split in its own test, and its authors caution that security-related prompts should be tested on whichever configuration you ship, because in their run one model declined two short code-security prompts that another answered.

A short worksheet

  1. Pick 30 to 100 real tasks with an automatic pass/fail check.
  2. Run each candidate model on all of them, with the same prompts and the settings you would use in production.
  3. Log input, output, thinking and cached tokens plus retries.
  4. Compute total spend (with caching and batch discounts you will really get) and divide by passes.
  5. Record latency too. A cheap but slow model can cost more in waiting.
  6. Re-run when providers change prices. Introductory prices expire: Google's Carbon rate doubles to $4 and $20 after 30 November 2026.

Common mistakes

  • Comparing only the price of output tokens. Reasoning models can write several times more of them.
  • Ignoring the model's published benchmark conditions. Artificial Analysis states that its index evaluates models zero-shot under identical conditions and counts tokens from each provider's API. Your prompts and settings differ.
  • Trusting a vendor's cost-per-task chart as proof. It is useful as a hint, and still the vendor's own measurement.
  • Forgetting that a cache from one model is not usable on another.
  • Treating one run as certain. Results vary run to run; repeat before committing.

Sources

SaveStart the discussion

Written by

audiotale.net team

The official account of the people who run audiotale.net: announcements, conversation starters and the newsroom's drafts. Posts written with AI say so. Every other account here is a real person.

Read next