Open vs Closed Models — The Tradeoffs

You've met the families; now the decision you'll actually make: do you build on a closed model via API, or an open-weight model you can run yourself? This is the model-level version of build-vs-buy (recall the mindset lesson) — and like all good engineering decisions, the answer is "it depends on your constraints," not hype.

The good news: there's a clear default and a short list of reasons to deviate from it. Get those straight and you'll make this call deliberately instead of by vibes.

You'll learn:

• Why it's really three options, not two
• The honest tradeoffs of closed (API) vs open-weight
• A simple decision framework (and the real cost numbers)
• Why the mature answer is often both (hybrid)

"Open vs closed" is a useful frame but slightly misleading. In practice you're choosing among three deployment options:

1. Closed API — call OpenAI / Anthropic / Google. The weights are secret; you just hit an endpoint.
2. Open model via a hosted API — let someone else run Llama / Qwen / DeepSeek for you (Bedrock, Together, Fireworks, Groq…). You get open models without owning any GPUs.
3. Open model self-hosted — you run the open weights on your own infrastructure.

So "using an open model" does not require self-hosting. Two more clarifications:

• "Open weights" ≠ "open source." You usually get the weights, but not the training data or full code — and licenses can restrict commercial use. Always check the license.
• Most real decisions are: closed API vs open-via-hosted-API vs self-hosted — and the tradeoffs differ for each.

The pros (why it's the default):

• Frontier capability with zero setup — the best models, one API call away.
• No infrastructure. No GPUs, no serving, no scaling, no ops. The provider handles uptime, updates, and safety.
• Instant + cheap to start, and it scales to large volume with a single line of code — you're buying scalability you didn't have to build.

The cons (why you might leave):

• Ongoing per-token cost that can dominate your bill at high volume.
• Your data leaves your infrastructure. Under GDPR the provider becomes a data processor (you need agreements, data-residency care). (Mitigated lately: OpenAI and Anthropic now offer zero-data-retention plans and SOC 2 — which resolves privacy concerns for many teams without self-hosting.)
• Vendor lock-in and no control — the model can change or be deprecated under you, and you can't deeply customize it.
• Rate limits you don't control.

The pros (why you'd choose them):

• Data stays on your infrastructure. Nothing leaves — the cleanest answer for strict privacy/compliance (HIPAA, data-residency, attorney-client privilege).
• Full control & deep customization. You can fine-tune on proprietary data — and a fine-tuned small model (7B–13B) often beats a general-purpose 70B on a narrow task.
• No per-token vendor markup (you pay infrastructure, not a margin) and no lock-in / reproducibility — the weights are yours; the model won't vanish.
• Can run offline / on the edge.

The cons (the catch):

• You own the ops. Serving, scaling, GPUs, uptime, monitoring — a real, ongoing burden (often 10–20 engineer-hours/month just to keep it healthy).
• The true cost is higher than it looks — GPU rental plus labor, idle-capacity waste, and depreciation typically runs 3–5× the napkin number.
• Needs expertise you may not have, and the very best open models still trail the absolute frontier (a gap that's narrowing fast).

Same rule as build-vs-buy: default to the closed (or hosted) API, and switch toward self-hosting open only when a concrete need forces it.

The numbers back the default: APIs are cheaper for the large majority of teams. Self-hosting only becomes economically cheaper at sustained, very high volume — roughly once predictable API spend runs into the tens of thousands of dollars a month (hundreds of millions to billions of tokens) — a threshold most production systems never hit. Below that, an API is both simpler and cheaper.

So reach for open / self-hosted when you have one of these real reasons:

• Privacy / compliance — data legally cannot leave your infrastructure (and zero-retention API plans aren't enough).
• Cost at scale — sustained, high, predictable volume past the break-even.
• Deep customization — you must fine-tune on proprietary data or control model behavior precisely.
• No lock-in / reproducibility / offline — you need the model to be permanent, auditable, or to run without internet.

If none of those apply, the API is almost always the right call. (Note: the EU AI Act reaches full enforcement in August 2026 — for high-risk systems, self-hosting can give you more control over the audit logs and oversight it requires. Both paths must comply.)

In reality, it's rarely either/or. The most cost-effective 2026 architecture for many teams is hybrid — buy-and-build applied to models:

• Route the hard, low-volume tasks to a frontier closed model (where capability matters most).
• Route the easy, high-volume tasks to a cheap open model (where cost matters most) — often self-hosted via vLLM/Ollama for predictable baseline load.
• Use a router to send each request to the right one.

This is the same heterogeneous, route-by-need pattern you'll see again in the cost-optimization lesson. The question isn't "open or closed?" so much as "which model for which job?"

Pick a side. For each scenario, open-weight or closed?

1. Hospital that must keep all patient data on-prem. → ?
2. Tiny team that needs the best possible quality, fast, no infra. → ?
3. High-volume internal tool where per-token API cost is killing you. → ?

→ 1: open (self-host for privacy). 2: closed (frontier quality, zero ops). 3: open (fixed cost at scale — if the volume justifies the infra). The trade is always control/cost vs quality/convenience.

• "Open models are free." No — you trade a per-token bill for infrastructure + engineering cost, which is usually higher until you hit serious scale.
• "Open weights = open source." No — you get weights, rarely the data/code, and licenses vary. Check before you build commercially.
• "Closed is always less private." Not necessarily — zero-retention API plans exist; but if data truly can't leave, self-hosting is the cleaner guarantee.
• "It's open vs closed." It's really closed API vs hosted-open vs self-hosted — and often hybrid is best.

• It's three options: closed API, open-via-hosted-API, and self-hosted open — and "open weights" ≠ "open source."
• Closed/API: frontier capability + zero ops, but per-token cost, data leaves your infra, and lock-in.
• Open self-hosted: privacy, control, fine-tuning, no lock-in — but you own the ops and the true cost is 3–5× the napkin number.
• Default to the API (it wins for the large majority); go open for privacy/compliance, sustained very-high-volume cost (tens of thousands/month+), deep customization, or no-lock-in/offline.
• The mature answer is usually hybrid — route each task to the right model.

Next: one more crucial distinction within every family — Base vs Instruct vs Reasoning models. We saw pretraining makes a base model; now we'll see exactly what the others are and when to use each.