Pay less for repeated context
Cache a stable prefix. Reuse can reduce input cost without shrinking the context window. A changing prefix or expired entry can erase those savings.
LECTURE 03 / LEARN BY EXPERIMENTING
See what an agent carries forward, what it pays to reuse, and what it risks forgetting. Change one thing. Watch the consequences.
Open the context engineering lab ↓01 / THE EXPERIMENT
Same model and workload, different memory policies.
If we double the number of calls, does the total cost always quadruple?
| Call | Input tk | Cached tk | Summary cost | Call cost | Total cost |
|---|
02 / THE MEMORY TEST
At the first call, the user adds a requirement: “Keep Python 3.10 compatibility.” Later, the agent still needs it. Select a strategy above and inspect what remains in the prompt.
Presence in the prompt does not guarantee a correct answer. A summary can lose or distort details; verify important constraints against the original source.
03 / TAKE IT BACK TO YOUR HARNESS
Cache a stable prefix. Reuse can reduce input cost without shrinking the context window. A changing prefix or expired entry can erase those savings.
Select useful tool results, retain recent exchanges, and compact older work. Include the summary call and the cost of reading that summary again.
Keep requirements, decisions, evidence, and open questions. Compare task success and rework alongside the token bill.
Take a real tool response. Keep the evidence needed for the next decision, remove repeated noise, and measure token use and task correctness before and after.
REFERENCE / PRICING SNAPSHOT
Prices retrieved
Open the provider’s pricing page to check today’s rates. Links open in a new tab.
Over 30 days old. Check the official rates before budgeting.
USD per million text tokens, standard API processing. This is a curated selection of general-purpose models from OpenAI, Anthropic, and Google. Preview and earlier-generation models are labeled. Restricted-access and specialized models are outside this lab.
| Model | Input | Cached read | Cache write¹ | Output² | Status |
|---|
¹ Anthropic uses the 5-minute write rate; its 1-hour rate differs. Google and older OpenAI rows show “—” where no separate premium is modeled. ² Includes the reasoning output you specify. Small nonzero rates may round to $0.0. Calculations and CSV retain full precision. Click a model name for its official pricing source.
Long context: OpenAI presets above 272,000 input tokens apply 2× input/cache and 1.5× output rates to the full request. Gemini 3.1 Pro Preview uses a 200,000-token threshold. Claude presets have no long-context premium.
Time-sensitive: GPT-5.6 Sol has promotional pricing at least through 21 Nov 2026. Gemini 3.8 Flash rates change on 1 Jan 2027. Prices are a dated snapshot; they do not refresh automatically.
Budget, not provider profit: API list prices estimate a hypothetical API bill. They do not reveal a lab’s infrastructure costs, subscription margins, or how many messages a subscription permits.
One turn means one model call. A human message can trigger multiple model calls through a tool loop. Token counts are fixed estimates for a synthetic text workload, not a tokenizer or a benchmark.
For call n with full history: I(n) = S + U + T + (n − 1)(U + T + A). S is the stable prompt, U user input, T tool results and A visible assistant output. Billed output is A + R, including the chosen reasoning amount R.
Without caching or tier changes, total cost for N calls is [pᵢ × (N(S + U + T) + (U + T + A)N(N − 1)/2) + pₒ × N(A + R)] / 1,000,000. The growing-history term is quadratic; fixed prompt and output terms are linear. Doubling N does not generally give exactly 4× cost.
Cached tokens are a user-selected fraction of the longest unchanged prefix shared with the previous call. An illustrative minimum of 1,024 tokens is used for OpenAI and a conservative 4,096 tokens for the other presets. Below that floor, no cache reads are credited. Fresh tokens in cache-enabled requests use the listed write rate when the request reaches that floor. The model assumes appropriate cache breakpoints, eligible content, and a reusable cache entry. It does not reproduce provider-specific boundary rounding, eviction, TTL, or partial-prefix lookup behavior.
Recent-only retains 3 previous exchanges. Summary mode replaces 6 older exchanges whenever 9 raw exchanges have accumulated, retaining 3 and then growing again. Each summary is a separate uncached call using the same model, the system prompt, previous summary, and those 6 exchanges. It outputs the configured summary size with zero extra reasoning. Its bill is included in that turn’s cost. This is an illustrative client-side policy, not an emulation of a vendor’s compaction API.
OpenAI and Claude capacity checks include the configured output reserve; Google checks its published input limit separately. A strategy stops at overflow; incomplete runs are labeled and do not receive a savings percentage. Curves are never extrapolated beyond a failed call.
Excluded: tool service fees, explicit Google cache storage, retrieval infrastructure, media, taxes, regional uplifts, batch/fast tiers, retries, and model-specific variation in tokenization or output length. Do not use this estimate as an invoice prediction.
Read more: OpenAI prompt caching · Claude compaction and custom instructions · Claude model context limits · Gemini model limits.