Hands-On: A Memory-Aware Agent

Introduction

This is where Section 5 comes together. You've learned why agents need memory (L132), the two tiers (L133), the three kinds (L134), and how memory is stored and retrieved (L135). Now we build the thing: a working agent that remembers you across sessions — closes the tab, comes back a week later, and still knows your name, your preferences, and what you did last time.

A memory-aware agent is shockingly simple at its core: it's a normal LLM call wrapped in a four-step loop — retrieve → inject → respond → write — around a persistent store. Everything else is refinement.

We'll build it three ways, from the inside out: from scratch (so you understand every moving part), then with Claude's built-in memory tool, then with a framework (Mem0 / LangMem) for production. By the end you'll be able to add durable memory to any agent.

In this lesson:

The architecture — the memory loop, and the session boundary it crosses
Building it from scratch — the store, retrieve + inject, respond, write + consolidate
Watching it run across two sessions (hands-on), then the easier paths (memory tool, frameworks)
Production concerns, and the whole memory arc in one picture

This closes Section 5. Next, the course turns to the Model Context Protocol (MCP) — the open standard for connecting agents to tools and data (L137 — The Integration Problem MCP Solves).

An infographic titled 'Hands-On: A Memory-Aware Agent' showing the agent loop that remembers across sessions. Four numbered steps run left to right along the top: step 1 RETRIEVE, search the memory store for the few relevant memories scored by relevance, recency, and importance; step 2 INJECT, place the top-k memories into the context window; step 3 REASON and RESPOND, the LLM answers now grounded in what it remembers; step 4 WRITE and CONSOLIDATE, extract the durable facts and events from the turn and save them. Below sits a MEMORY STORE band that persists across sessions, the agent's disk. A read arrow runs up from the store into step 1 RETRIEVE, and a write arrow runs down from step 4 into the store, closing the loop: store, retrieve, inject, reason, respond, write, back to the store. The key idea the diagram conveys is the session boundary: the context window resets every session, blank again, but because the agent wrote to the store in one session and retrieves from it in the next, it remembers the user across time. The three cards show the three ways to build it: do it yourself with a vector store and the explicit retrieve-inject-respond-write loop; use Claude's built-in memory tool, where the model reads and writes a memories directory itself; or use a framework such as Mem0 or LangMem, two calls add and search that handle extraction, storage, and consolidation for you. The takeaway: a memory-aware agent is the simple loop retrieve, inject, respond, write, wrapped around a persistent store, and that store is what carries memory across the session boundary, turning a stateless model into an assistant that remembers, personalizes, and improves. A concrete trace makes it vivid: in session one a user named Aria says she is vegetarian and allergic to peanuts; the agent retrieves nothing because the store is empty, replies, and writes the dietary fact with high importance. A week later in session two the context window is blank, but the agent retrieves Aria's diet from the store and suggests a peanut-free vegetarian dish, then writes that event back. Each step maps to an earlier lesson: retrieve and inject load long-term memory into working memory, the tiers of lesson L133; the response is grounded by the kinds of memory from L134; the search uses the relevance, recency, and importance score from L135. Production refinements complete the picture: write asynchronously in the background so the user never waits on memory curation, scope every read and write by user id so one user's memories never leak into another's, consolidate repeated episodes into durable semantic facts, handle stale and contradictory memories with newest-wins versioning, and protect privacy with retention and deletion. The diagram's bottom band, the persistent store, is the single element that turns four ordinary steps into an agent that accumulates knowledge over time.

The Architecture — The Memory Loop

Strip a memory-aware agent down to its skeleton and you get one loop, four steps, wrapped around a persistent store:

Retrieve — given the user's message, search the store for the few relevant memories (scored by relevance + recency + importance, L135 — Memory Stores & Retrieval).
Inject — drop those top-k memories into the context window (usually the system prompt) — loading long-term memory into working memory (L133).
Reason & Respond — call the LLM. It now answers grounded in what it remembers, not as a stranger.
Write & Consolidate — extract the durable facts and events from this turn and save them to the store for next time (and periodically distill episodic → semantic, L134 — Episodic, Semantic & Procedural Memory).

The magic is in what persists and what resets. The context window resets every session — each new conversation starts blank (the model is stateless, L132 — Why Agents Need Memory). But the store persists. Because step 4 wrote to it last time and step 1 reads from it this time, memory survives the session boundary. That single fact — write then retrieve across sessions — is the whole difference between a chatbot and an agent that knows you.

Memory-aware vs memory-augmented: an augmented agent just retrieves + injects; an aware agent also actively manages its memory — writing, consolidating, and forgetting. We're building the aware kind.

Step 1–4 — Building It From Scratch

Here's the entire loop in ~12 lines. Nothing here is new — it's L133 (tiers), L134 (kinds), and L135 (retrieval) assembled into one function:

def memory_agent(user_msg, user_id, conversation):
    # 1) RETRIEVE — fetch the few relevant memories (relevance + recency + importance, L135)
    memories = store.search(query=user_msg, user_id=user_id, k=5)

    # 2) INJECT — put them into the working context (the system prompt)
    system = BASE_PROMPT + "\n\nWhat you remember about the user:\n" + format(memories)

    # 3) REASON & RESPOND — the model answers, now GROUNDED in memory
    reply = client.messages.create(
        model="claude-opus-4-8",
        system=system,
        messages=conversation + [{"role": "user", "content": user_msg}],
    )

    # 4) WRITE — extract durable facts/events from the turn; save for next time
    store.add(f"User: {user_msg}\nAssistant: {reply.text}", user_id=user_id)   # async in prod
    return reply
# Four steps, every turn. The STORE is what carries memory across sessions.

Read it against the four steps: store.search(...) is retrieve (the L135 — Memory Stores & Retrieval scored retrieval); building system is inject (long-term → working memory); client.messages.create(...) is reason & respond (the model, now grounded — note claude-opus-4-8); store.add(...) is write. That store can be any of the L135 (Memory Stores & Retrieval) backends — a vector DB, Mem0, a knowledge graph. The loop is identical; only the storage swaps.

Two production refinements you'd add: (1) make the write asynchronous — don't make the user wait while you extract memories (the hot-path-vs-background trade-off from L134 — Episodic, Semantic & Procedural Memory); (2) add consolidation — a periodic background job that distills repeated episodes into semantic facts and merges duplicates (L134/L135 — Memory Stores & Retrieval), so the store stays sharp instead of bloating.

See It Run — Across Two Sessions

Watch the loop carry memory across the session boundary. Step through two sessions a week apart — notice the context window is blank at the start of session 2, yet the agent still knows Aria:

That's the entire point in one trace. Session 1: empty store → respond → write the diet. Session 2: brand-new blank conversation → retrieve the diet → personalize. Nothing carried over in the context window — it carried over in the store. Four steps, two sessions, one agent that remembers.

The Easier Paths — Memory Tool & Frameworks

Building from scratch teaches you the mechanics — but in production you rarely write the loop yourself. Two shortcuts, in increasing order of "hand it off":

Claude's native memory tool — give the model a memory tool and it manages its own /memories directory: it decides what to write, reads it back before each task, and you just run the file operations (L132). No retrieval pipeline to build — the model is the memory manager.
A memory framework — Mem0, LangMem, Zep, Letta (L135) ship the whole curate→store→score→retrieve→consolidate pipeline behind two calls: add() and search(). LangMem even plugs memory tools straight into a LangGraph agent; its background "Memory Manager" merges similar memories, summarizes old ones, and deletes stale entries for you.

# PATH B — let CLAUDE manage its own memory (the native memory tool):
client.messages.create(
    model="claude-opus-4-8", max_tokens=2048, messages=conversation,
    tools=[{"type": "memory_20250818", "name": "memory"}],    # reads/writes a /memories dir
)

# PATH C — a memory FRAMEWORK (Mem0): two calls; extraction + storage handled for you:
from mem0 import Memory
m = Memory()
m.add(f"User: {user_msg}", user_id="aria")                     # write — auto-extracts facts
hits = m.search("dietary restrictions?", user_id="aria")       # read — relevance + recency

Which to pick? From scratch when you need full control over scoring and storage (or to understand it — which you now do). The memory tool for a Claude-native agent where you want the model to self-manage. A framework for production, when you want persistence, consolidation, and multi-tenant scoping handled — and to ship this week. All three implement the same loop you just built; they differ only in how much you hand off.

Production Concerns

A toy memory agent is a loop; a production one earns its trust on the details — most of which you met across the section:

When to write: cheap episodic appends inline (hot path); expensive consolidation in the background (L134) — never make the user wait on memory curation.
Scope every read & write by user_id (and session_id/agent_id). The fastest way to lose user trust is leaking one user's memories into another's context.
Curate the write path — filter low-signal turns, dedup, canonicalize, tag metadata, and handle staleness/contradictions (newest-wins, source attribution) (L135). Garbage in → garbage retrieved.
Privacy & security: a persistent store of user data is a liability — PII handling, retention/expiry, and the right to delete a user's memory. Treat the store as sensitive (L132).
Evaluate retrieval, not vibes: does the agent surface the right memory at the right time? Track recall on a memory benchmark (LoCoMo-style) and watch for memory poisoning (a wrong fact written once, recalled forever).

The loop is the easy 20%. The write-path curation, scoping, freshness, and privacy are the 80% that make memory reliable — exactly the failure modes L132–L135 (Why Agents Need Memory → Memory Stores & Retrieval) kept flagging.

Section 5, Synthesized

Step back and see the whole arc you just completed — the complete answer to "how does an agent remember?":

Why (L132): an LLM is stateless; the context window is volatile working memory, so an agent needs a persistent store. → Tiers (L133): short-term (the window, RAM) vs long-term (the store, disk), with retrieve/consolidate between them. → Kinds (L134): long-term splits into episodic (events), semantic (facts), procedural (skills), linked by consolidation. → Stores & retrieval (L135): store as vectors / summaries / graphs / skills; retrieve by relevance + recency + importance, reranked. → Build (L136): wrap it in the retrieve → inject → respond → write loop around a persistent store.

That progression is the mental model to keep. A memory system is just those five answers, assembled: a persistent store, organized by kind, searched by a memory-aware score, read into the window at the start of a turn and written back at the end. Master it and your agents stop being amnesiacs — they accumulate, the way anything intelligent does.

🧪 Try It Yourself

Reason through these — they wire the whole section together:

In the loop, which step loads long-term memory into working memory, and which does the reverse? Name the L133 (Short-Term vs Long-Term Memory) terms.
Your memory agent feels sluggish — every reply lags. The write step is synchronous. What's the fix, and what did L134 (Episodic, Semantic & Procedural Memory) call it?
A user says "actually I eat meat now" but the agent keeps suggesting vegetarian food. Which production concern failed, and name two fixes.
You want to ship a Claude agent with cross-session memory by Friday, minimal infra. Which of the three build paths, and why?
Across the two sessions in the lab, what physically carried Aria's diet from session 1 to session 2 — and what did not?

→ (1) Retrieve → inject loads long-term (the store) into short-term/working memory (the context window); write/consolidate does the reverse (working → long-term). (2) Make the write asynchronous / background (don't block the response on memory curation) — L134 (Episodic, Semantic & Procedural Memory)'s hot-path vs background trade-off. (3) Staleness/contradiction handling failed — fixes: temporal versioning (prefer the newest record) + contradiction detection / consolidation (flag the conflict, retire the stale "vegetarian" fact); source attribution helps. (4) Claude's memory tool (or a framework) — minimal infra, no retrieval pipeline to build; the model self-manages /memories. From-scratch would be slower to ship. (5) The persistent store carried her diet across; the context window did not — it reset to blank in session 2. Memory lived in the store, not the window.

Key Takeaways

A memory-aware agent is one loop, four steps, around a persistent store: retrieve (scored search, L135 — Memory Stores & Retrieval) → inject (long-term → working, L133 — Short-Term vs Long-Term Memory) → reason & respond (the grounded LLM) → write & consolidate (durable facts → store, L134 — Episodic, Semantic & Procedural Memory).
The store persists; the context window resets. Memory survives the session boundary because you write in one session and retrieve in the next — that's the whole chatbot-vs-agent difference.
Build it three ways: from scratch (full control / understanding) · Claude's memory tool (memory_20250818, the model self-manages /memories) · a framework (Mem0/LangMem add + search, production). Same loop, different amounts handed off.
Production is the write path, not the loop: async writes, scope by user_id, curate + dedup + handle staleness/contradictions, protect privacy (delete/expiry), and evaluate retrieval (+ guard against poisoning).
Section 5 in one line: a persistent store, organized by kind, searched by a memory-aware score, read into the window at the start of a turn and written back at the end.
Next — Section 6: Model Context Protocol (MCP) (L137) — the open standard for plugging agents into tools and data sources.