Understanding API Pricing (Token Math)

You now think in tokens. Time to turn tokens into dollars. API pricing is, at its core, simple arithmetic — but a few asymmetries decide whether a feature costs $50 a month or$5,000: output is far pricier than input, conversation history compounds every turn, and a couple of discounts can cut 50–90% off.

Every serious AI engineer can do this back-of-the-envelope math before writing code, because the answer often changes the design. Let's make it second nature.

You'll learn:

• The basic pricing formula (input vs output, per million tokens)
• Why output costs ~5× more than input
• How to estimate a feature's real cost (a worked example)
• The multi-turn trap that quietly inflates chat costs
• The big levers: prompt caching, the Batch API, and model choice

APIs charge per token, quoted per million tokens (MTok) — and there are two separate rates: one for input (everything you send) and one for output (everything the model generates).

cost  =  (input_tokens  ÷ 1,000,000 × input_rate)
       + (output_tokens ÷ 1,000,000 × output_rate)

That's the entire model. Two numbers, two rates, add them up. The subtlety is in the rates — and in how big those token counts get.

The first thing that surprises people: generating tokens costs much more than reading them — typically about 5×. Output must be produced one token at a time (sequentially), which is far more compute-intensive than ingesting a prompt in parallel.

Representative rates (per 1M tokens, mid-2026 — always check the live pricing page, these move):

Two patterns jump out: output is ~5× input within every model, and the cheapest model is ~25× cheaper than the priciest. Both facts are levers you'll pull constantly.

Say you're building a support bot. One answer involves:

• Input: system prompt (500) + retrieved context (2,000) + user question (100) = 2,600 input tokens
• Output: a 400-token answer

Plug into the formula on Claude Opus 4.8 ($5 /$25):

input :  2,600 / 1,000,000 × $5   = $0.0130
output:    400 / 1,000,000 × $25  = $0.0100
--------------------------------------------
per request                       ≈ $0.023

Notice the punchline: those 400 output tokens cost almost as much as 2,600 input tokens — because output is 5× the rate. Now scale it to 10,000 requests/day (~300k/month) and compare models:

Same feature, $6,900/month on Opus vs$1,380 on Haiku — a 5× swing from one decision. This is why estimating cost up front matters: it changes which model you pick and how you design the prompt.

Here's the cost gotcha that catches everyone, and it follows directly from last lesson: because models are stateless, your app re-sends the entire conversation history every turn. So the input tokens grow with each turn — and you pay for the whole transcript again, every time.

A 20-turn chat doesn't cost 20× a single turn — it costs much more, because turn 20 re-sends turns 1–19 as input. Conversation cost grows roughly quadratically with length. That's why long chat sessions get surprisingly expensive, and why real apps trim or summarize history (a key reason context engineering exists — covered later). Watch this whenever you build anything conversational.

Two built-in discounts can dramatically cut the bill — for free:

• Prompt caching. If part of your prompt is the same across many calls — a long system prompt, a big document, fixed few-shot examples — you can cache it. Cached input is read at roughly 10% of the normal input price (up to ~90% off). This is huge for RAG and chatbots, where the same system prompt and context get re-sent constantly. (There's a small one-time write premium, so it pays off after a couple of reuses.)
• Batch API. For work that doesn't need an instant answer (bulk classification, offline generation, evals), submit it asynchronously and get ~50% off both input and output, with results back within ~24 hours.

These stack with model choice. A cached, batched job on a small model can be 10–50× cheaper than the naive realtime call on a frontier model — for the same task.

Before optimizing prompts or caching, ask the cheapest question: does this task even need a frontier model? Recall the three flavors lesson — most tasks don't need your most powerful (and priciest) model.

• Classification, extraction, simple formatting, routing → a small/cheap model (Haiku, GPT-5.4-mini/nano) is often as good and 5–25× cheaper.
• Reserve Opus/GPT-5.5 for genuinely hard reasoning, nuanced writing, or complex agentic work.
• Reasoning models cost extra in a sneaky way: their hidden reasoning tokens are billed as output, so a single "think hard" answer can cost many times a normal one. Use them only when the task warrants it.

A common, effective pattern is routing: send easy requests to a cheap model and escalate only the hard ones — often cutting costs by an order of magnitude with no quality loss.

• Estimate before you build. per-request cost × expected volume is a 2-minute calculation that can save (or sink) a project. Do it before committing to a model.
• Output is where the money goes. Cap max_tokens, prompt for concise answers, and be deliberate with reasoning models — output is ~5× input.
• Cache your static prefix. Same system prompt/docs on every call? Caching is close to free money (up to 90% off those tokens).
• Mind multi-turn growth. Conversational features get pricier per turn; plan to trim/summarize history.
• Right-size the model. The single biggest cost lever is not using a frontier model where a cheap one suffices.

(The dedicated Cost Optimization container later goes deep on squeezing every dollar — this lesson gives you the fundamentals to reason about cost from day one.)

Token math is best felt. Drag the sliders below and watch the cost update live:

• Bump output tokens up — notice the bill climbs fast (output is ~5× the price of input).
• Switch from Opus to Haiku — watch the monthly cost drop ~5× for the same request.
• Crank requests/day to see how a tiny per-request cost becomes a real monthly bill.

The split bar shows how much of each request is output cost — usually the lion's share.

• "Input and output cost the same." No — output is ~5× input. A short prompt with a long answer can be dominated by output cost.
• "A 20-turn chat costs 20× one turn." No — much more; history is re-sent every turn, so cost grows ~quadratically.
• "The big model is always worth it." No — for simple tasks a model 5–25× cheaper is often just as good. Match model to task.
• "Caching is an advanced optimization." It's a basic, near-free win whenever a prompt prefix repeats (RAG, chat) — use it early.
• "Reasoning models cost the same per answer." No — their reasoning tokens bill as output, so answers can cost several times more.

• Pricing is per token (quoted per million), with separate input and output rates: cost = in/1M × in_rate + out/1M × out_rate.
• Output costs ~5× input, so answer length (and reasoning tokens) often dominate the bill.
• Estimate cost up front: per-request × volume. A simple model swap (e.g. Opus → Haiku) can be a ~5× difference.
• Multi-turn chats compound because history is re-sent every turn (cost grows ~quadratically) — trim/summarize.
• Free levers: prompt caching (up to ~90% off repeated prefixes), Batch API (~50% off async), and above all choosing the cheapest model that's good enough.

Next — the section finale: you'll put this whole stack together and build a working CLI chatbot — real API calls, multi-turn history, streaming, and the token/cost awareness you just gained.