Episodic, Semantic & Procedural Memory

Introduction

In L133 (Short-Term vs Long-Term Memory) you split memory into two tiers — the short-term window and the long-term store. Now we open up that long-term store, because it isn't one thing. It comes in three kinds, each answering a different question:

Episodic — "what happened?" (events) · Semantic — "what is true?" (facts) · Procedural — "how do I do it?" (skills)

This isn't framework jargon — it's a 50-year-old map of human memory (Tulving, 1972), and it's the taxonomy the entire agent field converged on (CoALA, Mem0, Letta, LangMem all use it). Knowing which kind a piece of information is tells you how to store it, how to retrieve it, and how to use it. Confuse them and you build an agent that logs facts it should generalize, or hard-codes events it should forget.

In this lesson:

Each kind in depth — episodic, semantic, procedural — what it is, how it's used, and a real implementation
The cognitive roots (declarative vs non-declarative) and a hands-on sorter to train the distinction
How they connect — the crucial episodic → semantic consolidation loop — and hot-path vs background writing
Why you need all three, and the 2026 reality

Scope: this is the kinds of long-term memory and how to use them. How they're stored and searched under the hood — vector indexes, summaries, ranking — is the next lesson (L135 — Memory Stores & Retrieval), and the full build is L136 (Hands-On: A Memory-Aware Agent).

An infographic titled 'Episodic, Semantic & Procedural Memory' showing the three kinds of an agent's long-term memory as three columns. EPISODIC memory answers 'what happened': it stores specific past events and interactions, tagged with time and context, such as 'Tue 2:14pm the user requested a refund on order #4471'; it is used by replaying similar past episodes as dynamic few-shot examples, as in the memory stream of the Generative Agents system. SEMANTIC memory answers 'what is true': it holds durable, context-free facts and world knowledge that form the agent's model of the user and the world, such as 'the user is vegetarian and allergic to peanuts'; it is typically extracted from conversations by an LLM and injected into the system prompt to personalize replies. PROCEDURAL memory answers 'how to do it': it encodes learned skills, routines, and rules, such as a refund flow of verify order, check the policy window, issue credit, email confirmation; the Voyager agent implements this as a skill library where skills are stored as code, indexed by the embedding of their description, and retrieved and run when a similar task appears. The cognitive roots come from Endel Tulving: episodic and semantic memory are both DECLARATIVE, the explicit memory you can consciously recall, while procedural memory is NON-DECLARATIVE, the implicit know-how you simply run, like riding a bike. A curved arrow runs from episodic to semantic labeled CONSOLIDATE, because semantic memory is a derivative of episodic memory: repeated episodes, such as the user asking to cancel four times this month, distill into a general semantic fact, that the user is a churn risk, the way a person turns experiences into knowledge. The three infographic cards summarize each kind: episodic is events you can replay (learn from a specific past success), semantic is facts you know (personalize and ground), procedural is skills you run (improve how the agent operates). Memory can be written on the hot path, where the agent decides mid-conversation what to remember and pays latency, or in the background, where an async worker extracts and consolidates without slowing the response. The takeaway: episodic remembers events, semantic knows facts, procedural runs skills, and an agent needs all three, with episodic consolidating into semantic over time.

The Three Kinds at a Glance

Here's the whole map on one page. Keep this table in your head — the rest of the lesson is just filling it in:

	Episodic	Semantic	Procedural
Answers	"what happened?"	"what is true?"	"how do I do it?"
Holds	events, past interactions	facts, world/user knowledge	skills, routines, rules
Time	context-specific (timestamped)	context-free (timeless)	timeless procedure
Example	"refund issued for #4471 Tue"	"user is vegetarian"	"refund = verify→credit→email"
Used as	few-shot from the past	injected into the system prompt	retrieved + executed
Human analog	remembering your last birthday	knowing Paris is in France	knowing how to ride a bike

And in code, the three are written and read in completely different ways:

# EPISODIC — log an EVENT; recall similar past episodes as few-shot examples.
episodic.append({"t": now(), "event": "issued refund for #4471", "outcome": "ok"})
examples = episodic.search(similar_to=task, k=3)      # dynamic few-shot from the past

# SEMANTIC — extract a durable FACT; inject it into the prompt on future turns.
semantic.upsert(user_id, {"diet": "vegetarian", "allergy": "peanuts"})
profile = semantic.get(user_id)                        # -> personalize every reply

# PROCEDURAL — store a reusable SKILL (Voyager: code, indexed by its description).
skills.add(name="issue_refund", code=refund_fn, desc="verify -> policy -> credit -> email")
skill = skills.search(task)[0]                          # retrieve, then RUN deterministic code

The single sharpest test, item by item: is it an event (episodic), a fact (semantic), or a skill (procedural)? Replay it, know it, or run it. Hold that question — you'll use it in the sorter below.

The Cognitive Roots

The three kinds aren't arbitrary engineering buckets — they're lifted straight from cognitive science, and the structure tells you something useful. Psychologist Endel Tulving split long-term memory into two families:

Declarative memory — what you can consciously recall and state. It divides into episodic (autobiographical events — "my last birthday") and semantic (general facts — "water boils at 100°C"). Episodic is context-specific (tied to a time and place); semantic is context-free (the fact stands alone, stripped of when you learned it).
Non-declarative memory — what you just do without stating it. This is procedural memory: skills and habits — riding a bike. You can't write down how you balance; you simply run the skill.

That split maps cleanly onto agents and explains their mechanics: declarative memories (episodic + semantic) are data you read into the context window — logs and facts you pull up and state. Procedural memory is behavior you execute — code you run, or instructions baked into the system prompt that shape how the agent acts. "Data you recall" vs "behavior you run" is the deepest line through the three.

Episodic — "What Happened"

Episodic memory is the agent's autobiography: a log of specific past events — completed tasks, prior conversations, what was tried and how it turned out — each tagged with when it happened. It's the Generative Agents "memory stream" (Park et al. 2023): a time-stamped record of experience the agent retrieves from to act in character.

Its killer use in agents is learning from specific experience via dynamic few-shot prompting: when the agent faces a task, it retrieves the most similar past episodes and drops them into the prompt as worked examples — "last time this situation came up, here's what worked." As LangChain frames it, episodic memory is most valuable "if there is a correct way to perform specific actions that have been done before." Show the model its own past successes and it repeats them.

Episodic memory turns "I've never seen this" into "I've handled this before — here's how." It's the substrate for experiential learning (and it's exactly what Reflexion (L129) stored: episodic records of failures, recalled to avoid repeating them).

The catch: episodic memory is specific and accumulative — it grows without bound and contains raw, possibly sensitive interaction data. You rarely want all of it; you want the relevant episodes, retrieved on demand (L135) — and you eventually distill it (more on that below).

Semantic — "What Is True"

Semantic memory is the agent's knowledge base: durable, context-free facts about the user and the world — the user's name, preferences, role, constraints; domain facts; the agent's model of who it's talking to. Unlike an episode, a semantic fact has no timestamp that matters — "the user is vegetarian" is just true, whenever you ask.

The standard implementation is a two-step loop: (1) extract — an LLM reads the conversation and pulls out durable facts ("they mentioned they only use Python"); (2) inject — those facts are stored and later retrieved into the system prompt to shape every future reply. LangChain's example: Replit's coding agent remembering the "Python libraries that the user likes" to personalize its code. This is the engine of personalization — the difference between an assistant that greets a stranger every time and one that knows you.

Semantic memory shines "if there isn't necessarily a correct way to do things" — when the agent constantly faces novel tasks and what matters isn't a replayable procedure but stable knowledge to ground its reasoning. Where episodic asks "have I done this before?", semantic asks "what do I know that's relevant?"

Procedural — "How To Do It"

Procedural memory is the agent's skill set: the learned how-to — reusable routines, workflows, and rules for getting things done. In the strict CoALA sense it's "the combination of LLM weights and agent code" — the model's baked-in abilities plus the procedures you've written. But the practical, learnable procedural memory shows up in two places:

A skill library — the Voyager pattern (Wang et al. 2023). When the agent works out how to do something, it saves the solution as a reusable skill stored as code, indexed by an embedding of its description. Facing a new task, it retrieves the top-k relevant skills and executes them as deterministic code. Skills compound: simple ones combine into complex ones, and the agent gets monotonically more capable — lifelong learning.
The evolving system prompt — the most common form today: the agent (or a reflection step) rewrites its own instructions based on feedback — "always run the tests before finishing," "prefer the user's house style." Encoding a learned rule into the system prompt is procedural memory you can actually update at runtime (whereas updating weights or code is still rare).

Episodic and semantic are data you recall; procedural is behavior you run. It's the agent learning not just what, but how — and reusing it.

See It — Sort the Memories

Time to train the one skill this lesson is about: telling the three apart. For each item the agent observes, decide — episodic, semantic, or procedural? — then check the reasoning:

Three kinds of long-term memory, one skill: telling them apart. The agent observes items from its life — classify each as Episodic (what happened), Semantic (what’s true), or Procedural (how to do it), then check the reasoning. A timestamped refund request is an EVENT → episodic; “the user is vegetarian” is a context-free FACT → semantic; a “verify → policy → credit → email” refund flow is a reusable HOW-TO → procedural. The finale teaches the link: when the user asks to cancel 4 times, each ask is episodic — but repeated episodes CONSOLIDATE into a semantic fact (“churn risk”), exactly how a person turns experiences into general knowledge.

If the last item tripped you up, good — it's the important one. "The user asked to cancel 4 times" is a set of episodic events. But four of them in a month is a pattern, and patterns want to become facts. That's the bridge between the kinds — and the subject of the next section.

How They Connect — Consolidation

The three kinds aren't isolated silos — there's a flow between them, and it's the most important idea in this lesson. In Tulving's own framing, semantic memory is a derivative of episodic memory: repeated personal experiences (episodic) get distilled into general knowledge (semantic). Your agent should do the same:

Episodic → Semantic (consolidation): many specific episodes get summarized into one durable fact. "Asked to cancel 4×" (episodes) → "this user is a churn risk" (fact). "Chose the veggie option 3 times" → "prefers vegetarian."

This is exactly the reflection step from Generative Agents (synthesize low-level observations into higher-level insights) and the consolidation pattern every serious memory system implements: "identifying patterns across past interactions and distilling them into reusable knowledge." Why bother? Because it lets the agent generalize, reduce redundancy, and stay efficient — instead of re-reading ten episodes every time, it reads one consolidated fact. In code, it's a background job:

# CONSOLIDATION — distill repeated EPISODES into a SEMANTIC fact (run in the background).
episodes = episodic.search(user_id, about="cancellation")   # 4 cancel asks this month
if len(episodes) >= 3:
    fact = llm.summarize(episodes)        # "user is a churn risk; wants flexible billing"
    semantic.upsert(user_id, {"churn_risk": True, "note": fact})  # now KNOWN, not re-derived
# Repeated experience -> general knowledge. Episodic is the raw log; semantic is the lesson.

And the loop continues: a procedural skill often emerges from repeated successful episodes (you did X three times, so save X as a reusable skill). Episodic is the raw experience; semantic and procedural are the lessons you extract from it. Memory is not just storage — it's distillation.

Writing Memory — Hot Path vs Background

One practical decision cuts across all three kinds: when do you write a memory? There are two options, and the trade-off is latency vs freshness (LangChain's framing):

In the hot path — the agent decides what to remember mid-conversation, via a tool call, before it answers. Pro: the memory is available immediately, even later in the same session. Con: it adds latency to the response, and you're asking the model to multitask (answer + curate memory).
In the background — a separate process writes/consolidates memory after the interaction (or on a schedule). Pro: zero latency on the user's response; ideal for expensive work like episodic→semantic consolidation. Con: the memory isn't available until the job runs, and you need triggering logic (when to fire it).

The common production split: write raw episodic logs cheaply in the hot path (just append the event), and do the expensive consolidation (episodic → semantic, summarization, skill extraction) in the background. A user-feedback loop — thumbs up/down on a response — is also a powerful signal, especially for refining which episodic examples to surface next time.

Why You Need All Three

These aren't alternatives to pick between — a capable agent runs all three, and each covers a gap the others can't:

Without episodic → the agent can't learn from specific experience. It re-solves problems it already cracked and repeats mistakes it already made; no "I've handled this before."
Without semantic → no grounding, no personalization. It forgets who you are and what's true, treating every session as a first meeting.
Without procedural → it can't improve how it operates. It never accumulates skills or internalizes a rule; it's as clumsy on task #1000 as on task #1.

A quick routing heuristic for what to store as which: Is it a timestamped event? → episodic. A standalone fact that's just true? → semantic. A reusable way of doing something? → procedural. (And remember the flow: log it episodic, then let consolidation promote the patterns to semantic and procedural.)

🧪 Try It Yourself

Classify each, then justify it:

"The user's company uses PostgreSQL."
"At 4:02pm the deploy to staging failed with a timeout."
"To deploy: run tests → build image → push → smoke-test → promote."
The agent notices it has corrected the same lint error in 5 different sessions. What kind are the 5 corrections, and what should they consolidate into — of which kind?
You want an assistant to greet returning users by name and recall their preferences with zero added latency on the reply. Which memory kind, written when?

→ (1) Semantic — a durable, context-free fact about the user's stack, used to ground answers. (2) Episodic — a specific, timestamped event (it happened once); replay it as context if the deploy is retried. (3) Procedural — a reusable how-to / skill the agent runs (it could live as a skill or a system-prompt rule). (4) The 5 corrections are episodic events; repeated like that, they should consolidate into a procedural rule ("always apply this lint fix / run the linter first") — or a semantic fact about the codebase's style. Repeated episodes → a generalized lesson. (5) Semantic memory (name + preferences), and write/consolidate it in the background (post-session) so there's no hot-path latency; retrieve it into the system prompt on the next visit.

Mental-Model Corrections

"Long-term memory is one thing." It's three kinds — episodic (events), semantic (facts), procedural (skills) — each stored, retrieved, and used differently.
"Episodic and semantic are the same (both 'remembering')." Episodic is a context-specific event (timestamped, replayable); semantic is a context-free fact (timeless). One you replay, the other you know.
"Procedural memory = remembering steps." It's behavior you run (skills/code, or rules in the system prompt) — non-declarative. Episodic/semantic are data you recall; procedural is how you act.
"Just store everything as episodic logs." Raw logs don't generalize and they bloat. Consolidate the patterns into semantic facts and procedural skills — memory is distillation, not just storage.
"Semantic and episodic are independent." Semantic derives from episodic — repeated episodes consolidate into facts (the reflection loop). They're a pipeline, not silos.
"Always write memory during the turn." Hot-path writes add latency. Do cheap appends inline, but run expensive consolidation in the background.

Key Takeaways

Long-term memory has three kinds, each answering a different question: episodic — what happened (events); semantic — what is true (facts); procedural — how to do it (skills).
Cognitive roots (Tulving): episodic + semantic are declarative (data you recall — episodic is context-specific, semantic context-free); procedural is non-declarative (behavior you run).
Episodic = the agent's autobiography → dynamic few-shot from past successes (Generative Agents' memory stream; Reflexion's failure logs). Semantic = facts → extracted by an LLM and injected into the system prompt to personalize/ground. Procedural = skills → a Voyager-style skill library (code indexed by description) or an evolving system prompt.
They connect via consolidation: episodic → semantic — repeated episodes distill into durable facts (semantic is a derivative of episodic); skills emerge from repeated episodes too. Memory is distillation, not just storage.
Write timing: cheap episodic appends in the hot path; expensive consolidation in the background (latency vs freshness).
You need all three: no episodic → can't learn from experience; no semantic → no grounding/personalization; no procedural → can't improve how it operates. Route by: event → episodic, fact → semantic, skill → procedural.
Next — L135: Memory Stores & Retrieval — how these are actually stored and searched (vector indexes + summaries), so the right memory reaches the window at the right time.