Gemini 3.8 Flash cost per task: try thinking LOW first, then route (Google's 50-task test)

In short

If your Gemini agent feels expensive, measure cost per successful task (total spend including retries, tool calls and thinking, divided by tasks that pass a check) and start by setting Gemini 3.8 Flash to thinkingLevel: "LOW". A Google Cloud engineer's 3 November write-up found that a cheap classifier routing easy tasks to a lighter model matched that baseline on correctness in a small test and mainly helped latency. Do not cap thinking with maxOutputTokens: Google's docs say that limit includes thinking tokens and simply cuts the answer off. The test was 50 questions and one run per setup, so use it as a method to copy and re-measure, not as a result to quote.

Key points

• Google's metric is cost per successful task: all spend (loops, tool calls, retries, thinking) divided by the tasks that pass verification.
• In Google's test of 50 auto-graded coding-agent questions, Gemini 3.8 Flash at thinking level LOW answered all 50 correctly, which made it the baseline a router had to beat.
• A hand-written keyword rule that scored 15 of 16 on its development tasks fell to 70.6% on 34 unseen ones; the classifier it tested scored 34 of 34 on those.
• Setting maxOutputTokens low does not reduce thinking: the Gemini docs say it counts thought tokens and truncates the response when reached.
• Context caching can cut the cost of resent context by 75 to 90 percent, Google's post says, but a cache belongs to one model, so switching models mid-conversation can lose it.

What the post tested

The post, "Cost per task: Using cheap classifiers to route across Gemini models and thinking levels" by Sergio Villani of Google Cloud AI, was published on 3 November 2026. Its premise is that the price per token is the wrong number for an agent. An agent that loops, calls tools and retries can spend many times what one call costs, so the useful figure is what you pay for each task that actually succeeds.

The author built a benchmark of 50 auto-graded coding-agent questions and labelled each by hand as easy or hard. He ran them against the live Gemini API and a third-party classifier, TypeSafe's Jev. The post says Google has no partnership with TypeSafe and does not endorse it, and offers an open-weights alternative in a Google codelab (DiffusionGemma on a GPU VM or Cloud Run, billed per GPU-hour). The idea is general: a small, fast model reads the request and decides whether it needs the expensive configuration.

Step 1: set thinking before you build a router

Gemini 3 models think by default, and thinking tokens are billed. Google's thinking documentation says pricing "is the sum of output tokens and thinking tokens", and that you pay for the full thoughts even though the API returns only summaries. The default depth depends on the model: gemini-3.8-flash defaults to medium and supports low, medium and high; gemini-3.6-flash defaults to medium and also supports minimal; gemini-3.5-flash-lite defaults to minimal. The level is set with thinking_level in generation_config.

That is why the post's recommended first step is simple: try Flash at thinkingLevel: "LOW" for everything. In the test it answered all 50 questions correctly. Anything you build after that, including a router, has to beat it on cost or speed to be worth the extra moving part.

Step 2: do not use maxOutputTokens as a thinking budget

It is tempting to cut cost by capping output. The docs say max_output_tokens is the maximum number of tokens a response can generate, including thought tokens, and that it is a hard cutoff that does not change how the model allocates thinking. If the limit is hit during reasoning, the status is incomplete and the output may be truncated or empty. The docs' own advice is to lower thinking_level instead.

The post shows the effect. With gemini-3.8-flash, a cap of 300 tokens used almost the same number of thinking tokens as a cap of 4,096 on 27 easy tasks, so it saved nothing. On the hard tasks it broke 7 of 25: six ran out of tokens mid-reasoning and one answered wrongly.

Step 3: route with a narrow question, not a broad one

The router in the post asks a classifier about each request before choosing a model. A broad question ("is this simple or complex?") sent nearly everything to "simple". What worked was an atomic yes/no question scored as a probability: "multiple sequential calculation, simulation, or deduction steps". Requests flagged as multi-step go to Flash at LOW; the rest go to Flash-Lite first.

Classifier, 34 unseen tasks100

Keyword rule, 16 development tasks93.8

Keyword rule, 34 unseen tasks70.6

Share of tasks routed correctly (%)

	%
Classifier, 34 unseen tasks	100
Keyword rule, 16 development tasks	93.8
Keyword rule, 34 unseen tasks	70.6

Source: Google for Developers, Cost per task

The comparison with a plain rule is the most useful number. A regex and word-count heuristic scored 15 of 16 (93.8%) on the tasks it was tuned on, but 70.6% on 34 tasks it had not seen, a drop of 23.2 points. Jev scored 34 of 34 on those unseen tasks and 48 of 50 (96.0%) overall, matching the hand labels in 13 and 14 of the first 16 on two runs. Rules look good on the examples you wrote them from; that is the point to take away, whichever classifier you use.

Step 4: compare against the strongest no-router baseline

The post's end-to-end results, which give accuracy and timing but not dollar figures:

• Flash at LOW for everything: 50 of 50 correct.
• Jev router (Flash-Lite first, flagged tasks escalated to Flash at LOW): 50 of 50. The post says it can reduce both cost and response time against LOW everywhere, with the speed gain mattering more for interactive agents.
• Flash-Lite alone: simple tasks in about half a second, but it missed 6 of the 25 hard tasks, all of which the classifier had flagged as complex with high confidence.
• A keyword-heuristic router: 48 of 50, because it sent five hard tasks to Flash-Lite and two failed.
• Escalating to default-thinking Flash: cheaper than default thinking everywhere but costlier than LOW everywhere.

Step 5: protect your cache

Gemini applies implicit caching by default to 2.5 and newer models, with no configuration, and the caching documentation lists a minimum of 4,096 input tokens for gemini-3.8-flash. You can check hits in usage.total_cached_tokens. The post adds that context caching cuts the cost of resent context by 75 to 90 percent, and that a cache is tied to one model, so alternating between a light and a heavy model across turns can throw the discount away. Its advice is to route short, standalone tasks, and keep long, context-heavy turns on one model. The docs page we read does not state the discount or the per-model rule, so those two points rest on the post alone.

What it means

For teams running agents on Gemini, the transferable advice is a process, not a model name: define success automatically, measure cost per success, try the cheapest configuration that passes, and only then add routing. The post also warns that behaviour can vary by model: Gemini 3.1 Pro declined two short code-security review prompts that Flash answered, so security-related prompts should be tested on whichever configuration you pick.

What we don't know yet

• Dollar figures. The post reports accuracy, latency and relative cost but no prices, so we cannot say how much a router saves in money on your workload.
• Reproducibility. The benchmark code is promised after an open-source review and is not yet public.
• Scale. Fifty prompts, one run per setup, hand-assigned difficulty and simple automatic grading make the differences small and noisy. The author says latency, token counts and costs will vary with prompt length, caching and production load.
• Other classifiers. Only one third-party classifier was tested in the post.

What to watch next

Watch for the published benchmark code, replications on other task types than coding, and any change to the default thinking level of new Gemini models, since that single setting moves cost more than most routing schemes.

Sources

• Google for Developers: Cost per task, using cheap classifiers to route across Gemini models and thinking levels (primary)
• Gemini API thinking documentation
• Gemini API context caching documentation