Build vs Buy & the Model Router

Last lesson, you chose a model. But the best production systems make two more decisions that the leaderboard never mentions: do you rent the model (buy) or run it yourself (build)? — and why bet on a single model at all?

This finale covers both. First, the build-vs-buy call: managed API vs self-hosting an open model, and the economics that decide it (spoiler: it's not close for most teams). Then the technique that quietly powers cost-efficient AI products everywhere — the model router, which sends each request to the model that fits it best, often cutting costs by half or more with no drop in quality.

You'll learn:

• Buy vs build — managed API vs self-hosted, and the trade-offs
• The economics: when (and only when) self-hosting pays off
• The model router — and the routing patterns (tier, cascade, fallback)
• The savings routing unlocks, and the gateways that do it for you

There are two ways to get a model behind your product:

• Buy — a managed API (Claude, GPT, Gemini). You send requests; the provider runs everything. Fast to start, zero infrastructure, always the best models, instant scale — but you pay per token, your data leaves your machine, you depend on the vendor, and you live with rate limits.
• Build — self-host an open model (on your own/rented GPUs, served by something like vLLM for production throughput; Ollama is the dev-time version from earlier). You get full control, privacy, and a fixed cost that doesn't scale per token — but you take on hardware, ops, and engineering burden, and you're limited to open models (strong, but not the absolute frontier).

The real question isn't 'which is better' — it's 'which is right for your volume, privacy needs, and team.' And the economics are more lopsided than most people expect.

Self-hosting feels cheaper ('no per-token fees!'), but the full cost picture says otherwise for almost everyone. Beyond GPUs, self-hosting carries ops labor — roughly 10–20 engineer-hours/month for monitoring, updates, and firefighting, i.e. $750–$3,000/month before a single token is served.

The numbers (2026): API access is cheaper for the vast majority of teams. Self-hosting only flips at sustained, very high volume — roughly once predictable API spend runs into the tens of thousands of dollars a month (hundreds of millions to billions of tokens) — or when privacy/compliance forces on-prem deployment regardless of cost.

So the default is clear: Buy. Use a managed API until you have a specific reason not to — namely massive scale (where fixed GPU cost beats per-token) or a hard data-residency requirement. Don't take on a GPU cluster to save money you're not yet spending.

Run your situation through these:

Most startups and features should buy; build is for scale-stage, regulated, or privacy-critical workloads. A pragmatic middle path also exists: buy for the hard requests, self-host a small model for the cheap/high-volume ones — which leads directly to the router.

Here's the insight that ties this whole section together: you don't have to pick one model — you can pick per request. Most real workloads are a mix of easy and hard tasks, and using your most powerful (priciest, slowest) model for everything is wasteful.

A model router (or LLM gateway) sits in front of your models and sends each request to the best-fit one: a cheap, fast model handles the easy majority; a frontier or reasoning model handles the hard minority. You get most of the quality at a fraction of the cost and latency — the Iron Triangle, finessed by not committing to one corner for all traffic.

A few patterns cover most needs:

• Tier / classification routing. Classify each request's difficulty up front and send it to the right tier. The classifier can be a rule/keyword (<1ms), an embedding similarity (~5ms), or a small LLM classifier (~50–100ms).
• Cascade / escalation. Try the cheap model first; if its answer fails a quality check (low confidence, a validator, or a judge), escalate to the frontier model. Caveat: escalated requests pay the cheap call's latency plus the frontier call's — great for throughput/async work, riskier for latency-critical paths.
• Task / modality routing. Send code to a code model, images to a multimodal model, etc.
• Fallback / resilience. If a provider is down or rate-limiting, fail over to another — instant reliability win.

Match the pattern to your goal: cost (tier/cascade), capability (task routing), or uptime (fallback).

Routing isn't a micro-optimization — it's one of the largest cost levers in applied AI. Research (e.g. RouteLLM) shows you can keep ~95% of frontier-model quality while sending ~85% of queries to a cheaper model, for a 45–85% cost reduction depending on the workload.

In the wild: a customer-support platform cut its monthly LLM bill from $42,000 to$18,000 simply by routing simple queries to a small model and escalating only the hard ones — with no drop in customer-satisfaction scores.

That's the magic of routing: the easy 85% of your traffic never needed the expensive model, and most users can't tell the difference. You spend frontier money only where it actually buys you quality.

Routing, fallback, and a single unified interface are solved problems — use an LLM gateway:

• LiteLLM — a self-hosted proxy giving one OpenAI-style interface across all providers, with built-in caching, load-balancing, and fallback, and zero added per-token cost (it pays for itself at scale).
• OpenRouter — a hosted gateway: one API key, hundreds of models, easy fallback, and bulk-negotiated pricing often 15–25% cheaper than going direct. Less setup, a small markup.

Both deliver the provider-agnostic dream from the SDK lesson: change a model name (or routing rule), not your code. Start with a gateway early — even before you need fancy routing — so swapping models and adding fallback is trivial later.

Design a router. Your app's traffic is ~90% simple FAQ and ~10% complex multi-step questions. Sketch the routing rule (cheap model by default → escalate the hard 10% to a frontier model). Now estimate: if the cheap model is ~10× cheaper, roughly what fraction of the all-frontier bill do you pay? → Around ~20% — a huge saving, at (usually) no quality loss on the easy majority.

• "Self-hosting is cheaper." Rarely — after GPUs and ~$750–3,000/mo ops labor, APIs win for most teams until extreme scale or strict privacy.
• "Build gives me the best models." No — open models are strong but the frontier lives behind APIs; self-hosting trades quality for control/cost-at-scale.
• "I should pick one model for everything." No — route: cheap model for the easy majority, frontier for the hard minority.
• "Routing sacrifices quality to save money." Done well, you keep ~95% of quality while cutting cost 45–85% — the easy traffic never needed the big model.
• "I have to build routing myself." No — gateways (LiteLLM, OpenRouter) give routing, fallback, and one interface out of the box.

• Buy (managed API) by default — it's cheaper and faster for the vast majority. Build (self-host) only at sustained, very high volume (tens of thousands/month+ in API spend) or for strict privacy/compliance.
• A model router / gateway sends each request to the best-fit model — cheap & fast for the easy majority, frontier for the hard minority.
• Patterns: tier/classification, cascade (escalate on failed check — mind the latency), task/modality, and fallback for resilience.
• Payoff: ~95% of frontier quality at 45–85% lower cost (real case: $42k →$18k/mo, same CSAT).
• Use a gateway (LiteLLM self-hosted, OpenRouter hosted) — routing, fallback, and one provider-agnostic interface, no custom plumbing.

That completes 'Capabilities, Limits & Choosing a Model.' You can now judge what a model can do, read its limits honestly, and choose, deploy, and route it like a pro. Next section: with the right model in hand, we make it do what we want — Prompt Engineering Fundamentals.