System, User & Assistant Roles

You met the three message roles — system, user, assistant — back in the API-anatomy lesson, as mechanics. Now we use them as a prompt-engineering tool, because where you put an instruction changes how reliably the model follows it.

The single most useful idea: the system role is for what's always true; the user role is for what's true this time. Get that split right and you get a consistent, controllable assistant. Get it wrong — cramming everything into the user message, or putting per-request data in the system prompt — and you get a flaky one that forgets its rules and is impossible to maintain.

You'll learn:

• The constant-vs-variable split between system and user
• Exactly what belongs in the system prompt
• The instruction hierarchy — and why it differs by provider
• Power moves with the assistant role (history, examples, prefill)
• Why the system prompt should be treated like code

A quick refresher (mechanics from the API lesson; strategy is this lesson):

• system — standing instructions that govern the whole conversation: who the assistant is, how it behaves, its rules and format.
• user — what the human says on this turn: the question, the task, the input data.
• assistant — the model's replies. You include past assistant turns to continue a conversation — and, as we'll see, you can use this role deliberately for examples and output control.

(Recall the provider quirk: Anthropic's system is a top-level parameter; OpenAI's is the first message in the list. Same concept, different shape.)

Here's the principle that organizes everything: separate what's constant from what's variable.

• System = the constant. Everything that should hold true no matter what the user types: the persona, the rules, the output format, the scope. Write it once.
• User = the variable. The one thing that changes every call: this turn's request and its input data.

Think of it like a function: the system prompt is the function definition (fixed logic), and each user message is the argument you pass in. A customer-support bot's 'you are a polite support agent who only discusses our product and returns JSON' lives in system; the specific customer question lives in user.

This split has a bonus payoff from the pricing lesson: because the system prompt is identical across calls, it's exactly the static prefix you prompt-cache for up to 90% savings. Mixing variable data into it would break that — another reason to keep the split clean.

The system prompt is your assistant's constitution. A strong one usually defines:

• Role / persona — "You are a senior support agent for Acme."
• Tone & style — concise, friendly, formal, etc.
• Scope — what it should and must not do. (Scope is the most-skipped and most-valuable part for domain assistants — "only answer questions about Acme products.")
• Output format / contract — the shape of a good answer (prose, JSON schema, length).
• Safety & quality constraints — refusals, what to never reveal, accuracy rules.
• Fallback behavior — the explicit escape hatch: "If a request is out of scope, reply: 'I can only help with Acme products.'" "If unsure, say so rather than guessing" (your anti-hallucination guardrail).

That last one matters more than people expect: a model without a defined fallback will improvise on edge cases. Telling it exactly what to do when it can't comply is what makes behavior predictable.

Roles form a hierarchy: system instructions are more privileged than user instructions, so a user generally can't casually override your rules. But — a crucial 2026 reality — how strongly the system slot is privileged varies by provider:

• Claude: the system parameter is genuinely privileged and hard to override; Claude also follows XML-like tags very reliably.
• OpenAI (GPT): the system role is less privileged — a sufficiently determined user prompt can override it; GPT tends to prefer numbered lists and section headers over XML.
• Gemini: rewards layered prompts — put meta-instructions before task details.

Two consequences: (1) don't ship one identical system prompt to every provider and expect identical behavior — it's a common failure; tune per model. (2) Never rely on the system prompt alone for security. A determined user can often coax the model past it (jailbreaks) — real safety needs guardrails outside the prompt (a whole topic in the security/defensive-prompting section later).

The assistant role isn't only the model's past replies — you can write assistant messages yourself for three powerful effects:

1. Conversation history. Append past assistant turns so the model 'remembers' (the chatbot-loop pattern).
2. Few-shot examples. A user → assistant pair demonstrates the exact behavior you want — often more effective than describing it (next few-shot lesson).
3. Prefill (putting words in its mouth). Provide the start of the assistant's reply and let it continue. A classic trick: prefill with { to force JSON output, or with "Step 1:" to force step-by-step format:

messages=[
  {"role": "user", "content": "Extract the fields as JSON."},
  {"role": "assistant", "content": "{"},   # ← prefill: the reply must continue valid JSON
]

Prefill is especially powerful on Claude for locking output format and skipping preambles. Used well, the assistant role is a steering wheel, not just a transcript.

Your system prompt is often the single longest-lived piece of logic in your AI product — it runs on every request, for months. So treat it the way you treat code, not a sticky note:

• Version it (in your repo, not pasted into a console) and review changes.
• Test/evaluate it when you change it — a one-word tweak to a system prompt can shift behavior across all users (we'll do systematic iteration two lessons from now, and full evaluation in its own container).
• Keep it unambiguous and conflict-free — contradictory rules ('be brief' + 'be thorough') produce erratic output.

The mindset shift that defined shipped AI products in 2026: prompts aren't throwaway text, they're specifications — structured, reviewed, versioned, and evaluated.

Sort into roles. Put each into system or user:

the assistant's persona · today's specific question · the required JSON format · the document to analyze · the safety rules

→ system (the constant): persona, format, safety rules. user (the variable): the question, the document. Get this split right and you get a consistent assistant — and a cacheable system prompt. (Cram it all into user and behavior drifts.)

• "Put everything in the user message." No — constants (rules, persona, format) go in system; only the variable request goes in user.
• "System and user are equally weighted." No — system outranks user, but the strength varies by provider (Claude strong, GPT weaker).
• "One system prompt works everywhere." No — providers treat the system slot differently; tune per model.
• "The system prompt is solid security." No — determined users jailbreak it; real safety lives outside the prompt.
• "The assistant role is just the model's output." No — you can author it for examples and prefill to steer format.

• Split constant from variable: system holds what's always true (persona, scope, format, safety, fallback); user holds this turn's request + data. (System = the cacheable static prefix, too.)
• The system prompt is your assistant's constitution — define role, tone, scope, output contract, and explicit fallback behavior.
• System outranks user, but how much varies by provider (Claude strong + XML-friendly, GPT weaker + lists/headers) — don't ship one prompt to all, and don't trust it for security.
• The assistant role is a tool: conversation history, few-shot examples, and prefill to force output format.
• Treat the system prompt like code — version, review, and evaluate it; it runs on every request.

Next: whatever role it lives in, an instruction only works if it's clear — so we drill into writing clear, explicit instructions that leave the model nothing to guess.