---
title: Memory and State
url: https://doc.liz6.com/en/ai/02-Agent/03-memory-and-state
locale: en
area: ai
tags:
- ai
- Agent
date: 2026-06-30
modified: 2026-07-16
description: APIs are stateless; an agent’s "memory" is entirely engineered. Three layers of memory—re-sending history within a session, compaction/context editing to manage the window, and memory tools/stores for cross-session persistence—each use different mechanisms to solve different problems.
---
# Memory and State
> APIs are stateless; an agent’s "memory" is entirely engineered. Three layers of memory—re-sending history within a session, compaction/context editing to manage the window, and memory tools/stores for cross-session persistence—each use different mechanisms to solve different problems.
## Overview
APIs are **stateless** (see [Context Engineering](/ai/01-models-and-context/01-context-engineering.md)): what the model "knows" in a single turn is entirely contained in that request; it does not automatically remember previous interactions. Therefore, an agent’s "memory" is **entirely engineered**—a set of mechanisms maintained by the caller, layered according to their lifecycle. Categorizing them by "how long they last" provides the clearest framework for understanding this:
```mermaid
flowchart TD
A["① Within-session Re-sending full history (protocol-native)"] --> B["② Near window limit Compaction summary / context editing trimming"]
B --> C["③ Cross-session persistence Memory tool / memory stores (outside the session)"]
```
The first layer is "remembering this turn," the second is "preventing overflow when the conversation gets too long," and the third is "remembering after closing and reopening." These three layers use different mechanisms to solve different problems; long-running agents often employ all three. Understanding their boundaries prevents misusing tools (e.g., relying solely on re-sending history when trying to remember preferences across sessions).
## Layer 1: Within-Session — The Cost of Re-sending History
The model remembers the previous turn simply because **the entire history is sent again**. This layer is "free" (native to the protocol), but it has two costs:
- **Linear cost growth**: Re-sending all history each turn causes input tokens to accumulate with each round.
- **Inevitable overflow**: If history grows infinitely, it will eventually approach the window limit (`stop_reason: "model_context_window_exceeded"`).
`prompt caching` allows stable prefixes in re-sent history to be billed at ~0.1x (see [Context Engineering](/ai/01-models-and-context/01-context-engineering.md)), alleviating cost; but the overflow problem must be addressed by Layer 2.
## Layer 2: Managing the Approaching Window — Compaction vs. Editing
Two opposing API primitives:
| | Compaction | Context Editing |
|---|---|---|
| What it does | **Summarizes** early history into a compaction block | **Clears** old `tool_result` / thinking blocks |
| Information handling | Summarizes and retains | Deletes directly |
| Beta | `compact-2026-01-12` | `context-management-2025-06-27` |
| Key pitfall | You must return `response.content` (including the compaction block) **as a whole block**; returning only text will lose state | Only deletes blocks, does not alter conversation structure |
Mnemonic: compaction is "summarizing," context editing is "deleting." Both operate **within a single session** to free up window space and **do not solve cross-session persistence**—if you close the process, they are gone.
## Layer 3: Cross-Session Persistent Memory
To "remember after closing and reopening," key points must be **written outside the session**. There are two paths, chosen based on self-managed vs. managed approaches.
### Path A: Memory Tool (Backend Self-Managed)
Declare the tool `{"type": "memory_20250818", "name": "memory"}`, and the model gains read/write access to a `/memories` directory, with **six commands**:
| Command | Semantics |
|------|------|
| `view` | Read file / list directory |
| `create` | Create/overwrite file |
| `str_replace` | Replace a string in a file |
| `insert` | Insert after a specified line |
| `delete` | Delete file |
| `rename` | Rename/move |
**The storage backend is implemented by the user** (local FS, object storage, DB)—the model only sends commands, and the implementation executes them and returns `tool_result`. Python/TypeScript SDKs provide scaffolding (`BetaAbstractMemoryTool` / `betaMemoryTool` + handler); you only need to fill in the storage logic for `view/create/str_replace/...`. The model writes "user preferences," "project conventions," and "lessons learned" into files, and reads them back via `view` in the next session.
> This approach is currently being used in this very session: persistent memory resides under `~/.claude/projects/.../memory/`, with **one file per fact + a `MEMORY.md` index**—`MEMORY.md` remains persistent, and each fact is read on demand. This is the practical implementation of the memory tool pattern (this turn updated the entry regarding "this site's build method").
### Path B: Managed Agents Memory Stores (Managed)
A workspace-level persistent memory repository with a three-layer object model:
Key mechanisms:
- **Mounted as a filesystem**: The store is mounted via FUSE to the container at `/mnt/memory//`; the agent reads/writes using standard file tools; the system prompt automatically injects a description informing it of the mount.
- **Access control**: `access: "read_only" | "read_write"`, enforced at the filesystem layer.
- **Optimistic concurrency**: `memories.update` supports `precondition: {type:"content_sha256", ...}`; if it doesn't match, it returns 409—read-modify-write operations won't overwrite each other.
- **Audit and rollback**: Every change produces a `memory_version`, which can be listed, fetched, and **redacted** (stripping content but leaving actor + timestamp, useful for leaks/PII removal).
- Up to 8 stores can be mounted per session (can be layered by "shared read-only reference + per-user read/write").
## Memory vs. Context: Which to Use?
Beginners often mistake "long context" for "memory." Distinguish them:
- **Context**: What is stuffed into this single request, disappearing at the end of the turn (unless re-sent). The "window."
- **Memory**: What survives across requests/sessions, written outside the session. The "hard drive."
Use context for "seeing this turn" (RAG puts snippets into the window); use memory for "remembering later" (writing files). Compaction/context editing are management tools for the context layer, not memory.
## Design and Security
Good memory design (also how this session's memory is written):
- **One experience per file**, with a one-line summary at the top to facilitate relevance judgment during retrieval.
- **Store only non-obvious things**: Don't store what's already in the code/history (that's noise); store "why," "where the pitfalls are."
- **Update on write, delete on error**: Update existing entries rather than stacking duplicates; delete outdated ones.
Security red lines:
- **Path validation is mandatory**: The `path` provided by the model is untrusted output. Resolve to a canonical path (`realpath`/`Path.resolve()`), confirm it is still within the memory root directory, and **reject `..`, symbolic links, absolute out-of-bounds paths, and URL-encoded traversal (`%2e%2e%2f`)**. Do not directly `open()` the raw path.
- **Never store secrets**: API keys, passwords, and tokens must not enter memory. Be cautious with PII (GDPR/CCPA); `redact` in memory stores is a post-hoc remediation measure.
- **Multi-tenant isolation**: Independent memory directories + authentication per user; reference implementations lack built-in access control.
## Best Practices
- **Use the right tool for each layer based on "how long it lasts."** Re-send history within sessions, use compaction/context editing near the window limit, and use memory tools/stores for cross-session persistence—don't use re-sending history to "remember preferences."
- **One experience per file, with a one-line summary at the top.** Facilitates relevance judgment during retrieval; a single `MEMORY.md` index remains persistent, reading each entry on demand (this session does exactly this).
- **Store only non-obvious things.** Don't store what's already in the code/history (it's noise); store "why," "where the pitfalls are," and "established conventions."
- **Update on write, delete on error.** Update existing entries rather than stacking duplicates; delete outdated ones.
- **Normalize paths before validating boundaries.** The `path` provided by the model is untrusted; `resolve()` + confirm it is within the memory root; reject `..`/symlinks/URL-encoded traversal (see [Security and Protection](/ai/03-applications-and-production/04-security-and-protection.md)).
- **Secrets never enter memory; multi-tenants are isolated by user.** Reference implementations lack built-in access control; be cautious with PII, and remember `redact` is only a post-hoc remedy.
## Trade-offs and Failure Modes
- **Stuffing everything into memory**: Becomes noise, interferes with retrieval → store only high-value, non-obvious items.
- **Treating memory as a transactional database**: It is notes for the model → use `content_sha256` optimistic locking (memory stores) or self-managed versions for concurrent writes.
- **Cross-session tenant leakage**: Multi-tenant isolation not enforced → separate directories by user + authentication.
- **Unvalidated paths**: Directory traversal reading/writing arbitrary files → normalize + boundary checks.
## References
- **Anthropic Official Documentation**: Memory Tool, Managed Agents Memory Stores (platform.claude.com)
*Keywords: stateless, session history, prompt caching, compaction, context editing, model_context_window_exceeded, persistent memory, memory tool, memory_20250818, /memories, view/create/str_replace/insert/delete/rename, BetaAbstractMemoryTool, memory stores, memstore, FUSE mount, access, content_sha256, precondition, redact, memory version, path traversal, secret, PII, multi-tenant isolation*