Hands-On: Build a CLI Chatbot

Time to assemble the whole section into something real. In this lesson you'll build a working command-line chatbot — one you can actually talk to in your terminal — and you'll understand every line of it.

It's small (~40 lines), but it exercises everything you just learned: API keys & SDKs (setup lesson), messages & roles (anatomy lesson), the context window / statelessness (you'll implement the "memory" by hand), streaming (responsive output), and token/cost awareness (we'll print usage). This is the moment the theory becomes muscle memory.

You'll build it step by step:

• A skeleton: client, system prompt, and the message loop
• Multi-turn memory — maintaining the conversation yourself
• Streaming so replies appear token-by-token
• Polish: exit/reset commands, error handling, and token usage
• The complete program (plus the OpenAI variant)

A terminal chat loop: you type, the assistant streams a reply, it remembers the conversation, and you keep going until you quit. Like this:

Chat with Claude (type 'exit' to quit, '/reset' to clear)

You: My name is Aryan.
Claude: Nice to meet you, Aryan! How can I help?
  [in: 18 tok · out: 11 tok]

You: What's my name?
Claude: Your name is Aryan.
  [in: 41 tok · out: 6 tok]

Notice the second answer knows the name — proof our memory works. And notice the input tokens grew (18 → 41) on the second turn: that's the whole history being re-sent, exactly as the context-window and pricing lessons predicted.

Make sure you've done the setup lesson (pip install anthropic python-dotenv and a .env with ANTHROPIC_API_KEY). Then the bones:

import os
from dotenv import load_dotenv
from anthropic import Anthropic

load_dotenv()                # read .env into the environment
client = Anthropic()         # picks up ANTHROPIC_API_KEY automatically

SYSTEM = "You are a friendly, concise assistant."
messages = []                # 👈 THIS list is the conversation's memory

print("Chat with Claude (type 'exit' to quit, '/reset' to clear)\n")

The single most important line is messages = []. The model is stateless — it remembers nothing between calls — so we hold the conversation in this list and re-send it every turn. That list is the chatbot's memory.

Now the heart of it: a loop that reads your message, appends it to messages, sends the whole list, and appends the reply. Here's the non-streaming version first (simplest to read):

while True:
    user = input("You: ").strip()
    if user.lower() in {"exit", "quit"}:
        break

    messages.append({"role": "user", "content": user})   # remember user turn

    resp = client.messages.create(
        model="claude-opus-4-8",
        max_tokens=1024,
        system=SYSTEM,
        messages=messages,            # 👈 send the ENTIRE history
    )
    reply = resp.content[0].text
    print(f"Claude: {reply}\n")

    messages.append({"role": "assistant", "content": reply})  # remember reply

The rhythm to burn in: append user → send all of messages → append assistant. Forget to append the assistant's reply and the bot develops amnesia — it won't recall its own previous answers. (This append-user / append-assistant cycle is the visual below.)

Waiting for the whole reply feels sluggish. Streaming prints tokens as they arrive — the difference between a frozen cursor and a bot that 'types' at you. Swap the API call for the streaming form, and accumulate the text so we can still store it in history:

    print("Claude: ", end="", flush=True)
    reply = ""
    with client.messages.stream(
        model="claude-opus-4-8",
        max_tokens=1024,
        system=SYSTEM,
        messages=messages,
    ) as stream:
        for text in stream.text_stream:     # tokens arrive here, live
            print(text, end="", flush=True)
            reply += text                    # accumulate for history
    print()

    messages.append({"role": "assistant", "content": reply})

We print each chunk and concatenate it into reply, so the model still 'remembers' what it said. Streaming doesn't make generation faster — it just shows progress immediately, which feels dramatically better.

A few touches make it robust and educational:

• A /reset command to clear history (a fresh context — and cheaper, since history resets).
• Error handling so a network hiccup or rate-limit (429) doesn't crash the program — and we remove the failed user turn so history stays clean.
• Token usage printed each turn, so you see your cost growing (straight from the pricing lesson). With streaming, grab it from the final message:

        usage = stream.get_final_message().usage
    print(f"\n  [in: {usage.input_tokens} tok · out: {usage.output_tokens} tok]\n")

Watching input_tokens climb every turn is the multi-turn cost trap made visible — a great habit to build early.

Everything assembled — a real, streaming, multi-turn chatbot in ~40 lines:

import os
from dotenv import load_dotenv
from anthropic import Anthropic

load_dotenv()
client = Anthropic()

SYSTEM = "You are a friendly, concise assistant."
messages = []

print("Chat with Claude (type 'exit' to quit, '/reset' to clear)\n")

while True:
    user = input("You: ").strip()
    if user.lower() in {"exit", "quit"}:
        break
    if user == "/reset":
        messages.clear()
        print("(conversation cleared)\n")
        continue
    if not user:
        continue

    messages.append({"role": "user", "content": user})

    print("Claude: ", end="", flush=True)
    reply = ""
    try:
        with client.messages.stream(
            model="claude-opus-4-8",
            max_tokens=1024,
            system=SYSTEM,
            messages=messages,
        ) as stream:
            for text in stream.text_stream:
                print(text, end="", flush=True)
                reply += text
            usage = stream.get_final_message().usage
        print(f"\n  [in: {usage.input_tokens} tok · out: {usage.output_tokens} tok]\n")
    except Exception as e:
        print(f"\n[error: {e}]\n")
        messages.pop()        # drop the failed user turn
        continue

    messages.append({"role": "assistant", "content": reply})

Save it as chat.py, run python chat.py, and you have your own assistant. You built that — and you understand every line.

Same architecture, the dialect differences from the SDK lesson. The loop and messages memory are identical; only the call, the system placement, and the streaming shape change:

from openai import OpenAI
client = OpenAI()

# system is a MESSAGE at the front of the list (not a separate param):
messages = [{"role": "system", "content": "You are a friendly, concise assistant."}]

# ...inside the loop, after appending the user message:
print("GPT: ", end="", flush=True)
reply = ""
stream = client.chat.completions.create(
    model="gpt-5.5",
    messages=messages,
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content or ""
    print(delta, end="", flush=True)
    reply += delta
print()
messages.append({"role": "assistant", "content": reply})

The key takeaways from the SDK lesson show up live: system is a message (so it lives in messages), and streaming yields chunk.choices[0].delta.content instead of a clean text stream. Same program, two dialects.

Your chatbot is a perfect sandbox. Things to try — each previews a later topic:

• Run it on a local model: point the OpenAI client at Ollama (base_url="http://localhost:11434/v1", model="llama3.1:8b") — free and offline (Ollama lesson).
• Add a token budget: when messages gets too long, trim or summarize old turns to control cost and stay in the window (context engineering).
• Give it tools: let it call a calculator or search function (tool-use container).
• Save the transcript to a file, or add a richer TUI.
• Add a model router: cheap model by default, escalate hard questions (cost optimization).

Don't just read this lesson — run the code and tinker. Hands-on is where AI engineering actually sticks.

• This loop is the skeleton of nearly every chat app. Agents, RAG assistants, customer-support bots — they all start from this exact append-user / send-history / append-assistant cycle.
• You now own the 'memory' abstraction. Knowing that you maintain messages (not the model) is what lets you later trim it, summarize it, persist it, or inject retrieved context.
• You can feel cost and latency. Watching tokens grow and replies stream makes the previous two lessons concrete — and tunes your instincts for designing efficient apps.
• It's provider-portable. The same structure runs on Claude, GPT, or a local model — change a few lines, not your architecture.
• Confidence. You've gone from 'how do LLMs work?' to 'I built a working AI app.' Everything after this is extending this foundation.

Your turn — then break it. Run the chatbot and have a short conversation. Now delete the line messages.append({"role":"assistant", ...}) and re-run. Predict, then watch: → it forgets its own previous replies (it only remembers your turns), proving the model is stateless and messages is the memory. Put the line back, then add a /tokens command that prints usage each turn to watch cost climb.

• A chatbot is a loop: read input → append user → send the entire messages history → stream the reply → append assistant → repeat.
• Because the model is stateless, your messages list is the memory — and re-sending it each turn is why input tokens (and cost) grow.
• Streaming (messages.stream / stream=True) shows tokens live for a far better feel — accumulate the text to keep it in history.
• Polish matters: exit/reset commands, error handling (drop failed turns), and printing token usage to stay cost-aware.
• The OpenAI variant is the same loop with the dialect differences (system as a message, delta.content streaming).

That completes 'The AI Engineering Stack & Your Dev Environment' — you can now set up, call, run locally, budget, and build. Next, we turn to choosing which model and understanding its limits — starting with long-context models and where their context really ends.