The AI Stack: Application, Model & Infrastructure Layers

In the last lesson we said AI engineering is building applications on top of foundation models. But where, exactly, does that work sit? What do you build yourself, what do you rent, and where does each skill in this course live?

The answer is the AI stack — and like most of computing, it comes in three layers. Internalize this map now and the entire course will slot neatly into place: you'll always know which layer a topic belongs to and why it matters.

You'll learn:

• The three layers of every AI application — application, model, infrastructure
• What lives in each, in concrete terms
• Where AI engineering sits — and the big shift up the stack
• What you build vs. rent at each layer
• How the rest of this course maps onto the stack

Every AI application — from a weekend chatbot to an enterprise agent — is built on three stacked layers (this is the framing from Chip Huyen's AI Engineering):

A crucial practical note: you develop top-down. You start at the application layer (can a prompt solve it?) and only descend — into fine-tuning, into infrastructure — as the problem forces you to. Most AI products never need to touch the bottom of layer 2.

This is where most of the action is, and where AI engineers spend the bulk of their time. You take a capable, pre-trained model and turn it into a useful, reliable product. It has five pillars:

• Prompting & context engineering — what instructions and information the model sees on each call.
• Context & knowledge (RAG) — feeding the model the right external/private data at query time.
• Tools & agents — letting the model act: call functions, search, run code.
• Evaluation — rigorously measuring whether the output is actually good (the discipline most teams skip — we don't).
• The AI interface (UX) — streaming, citations, handling uncertainty; a great model behind a bad interface is a bad product.

Notice: none of this requires training a model. It's software engineering + a probabilistic layer — exactly the inversion from lesson 1.

This layer is about creating and adapting the models themselves:

• Pre-training — building a foundation model from scratch on massive data (done by a handful of labs; enormously expensive — you will almost never do this).
• Fine-tuning — adapting an existing model to your task or style (something AI engineers do reach for — a whole container later).
• Dataset engineering — curating, cleaning, and synthesizing the data that training/fine-tuning consumes. Data is so central it lives here.
• Inference optimization — making a model run faster and cheaper (quantization, KV cache, serving engines).

As an AI engineer you mostly rent this layer — you call a model a lab already trained. But you dip in deliberately: to fine-tune when prompting/RAG isn't enough, and to make inference choices that control latency and cost. You need to understand this layer well enough to make those calls — which is why this course teaches it — without having to live in it.

The foundation of the stack — the systems that keep everything running:

• Model serving — exposing models behind APIs that scale (managed endpoints, or self-hosted with vLLM/TGI).
• Compute & data management — GPUs/accelerators, data pipelines, storage.
• Supporting services — vector databases, model gateways/routers, caches, orchestration.
• Monitoring & observability — tracing calls, tracking cost/latency, catching drift and failures in production.

The good news for AI engineers: this layer is increasingly managed. Hosted model APIs, managed vector DBs, and observability SaaS mean you can lean on infrastructure rather than build it — freeing you to focus up the stack. You still must understand it enough to deploy, scale, secure, and observe your app (the LLMOps container).

Here's the punchline. AI engineering concentrates on Layer 1, reaches down into Layer 2 for fine-tuning and inference choices, and relies on Layer 3 to ship. Compared with classic machine learning, the center of gravity has moved up the stack:

• In traditional ML, most effort went into layers 2 and 3 — building and serving a bespoke model was the whole job.
• In AI engineering, the model (Layer 2) is largely a rented commodity, so the differentiation — and most of the work — moves to Layer 1: the prompts, the retrieved context, the tools, the evals, the experience.

Said simply: you rent the intelligence (Layer 2), build the product (Layer 1), on mostly-managed infrastructure (Layer 3). That's lesson 1's "great inversion," now drawn as a stack. It's also why the AI engineer role exists as something distinct from the ML engineer — the subject of the next lesson.

Every container you'll work through maps cleanly onto a layer — so you'll always know why you're learning something:

The weight sits in Layer 1 — exactly where AI engineering lives — while still teaching enough of Layers 2 and 3 to make you dangerous across the whole stack.

Place each piece in the stack. Sort the nine components into Application, Model, or Infrastructure — then watch the reveal. The pattern is the lesson: everything you build lands in Layer 1, the model is rented (Layer 2), and the boxes it runs on are managed (Layer 3).

That's why AI engineers rarely touch the bottom two layers — you rent the intelligence, build the product, on mostly-managed infrastructure, developing top-down.

Place it in the stack. Sort each into Application, Model, or Infrastructure:

your prompt · GPT-5.5 · a React chat UI · a vector database · a GPU/inference server · your RAG logic

Answers — Application: your prompt, React UI, RAG logic, vector DB usage. Model: GPT-5.5. Infrastructure: the GPU/inference server. Most of your work as an AI engineer lives in the application layer — which is exactly why you rarely touch the bottom two.

• "AI engineering is only the application layer." Layer 1 is your home, but you need enough of Layer 2 to choose and fine-tune models, and enough of Layer 3 to deploy, scale, and observe. The best AI engineers are T-shaped: deep in Layer 1, literate in 2 and 3.
• "I must master model training (Layer 2) first." No — you rent the model. Pre-training is a lab's job. Start building at Layer 1 today; learn fine-tuning when a real need appears.
• "Infrastructure is someone else's problem." Until your app is slow, expensive, or down at 2 a.m. Treat serving, cost, and observability as first-class from the start (we do).
• "Work bottom-up." Backwards. Start at the top with the simplest thing (a prompt) and descend only when the problem demands it.

• Every AI app is a three-layer stack: Application Development (Layer 1), Model Development (Layer 2), Infrastructure (Layer 3).
• AI engineers live in Layer 1, reach into Layer 2 (fine-tuning, inference choices), and rely on Layer 3 (serving, observability) to ship.
• The center of gravity has shifted up the stack: the model is a rented commodity, so the work — and the differentiation — moves to the application layer.
• Build top-down: start with a prompt; descend into fine-tuning or infrastructure only when the problem forces you.
• This course is weighted toward Layer 1 but teaches enough of all three to make you effective end-to-end.

Next: with the stack in hand, we can precisely contrast the roles — AI Engineer vs ML Engineer vs Full-Stack Engineer — and see who owns which layer.