--- title: Context engineering url: https://doc.liz6.com/en/ai/02-context-and-interfaces/02-context-engineering locale: en area: ai tags: - Models & agents - Requests and Evidence date: 2026-06-30 modified: 2026-09-10 description: Once the output contract is clear, decide what the model receives now. Context engineering selects constraints and evidence, reserves output space, reuses stable prefixes and compresses history into recoverable references. It has distinct responsibilities from persistence, retrieval and model weights. --- # Context engineering Once the output contract is clear, decide what the model receives now. Context engineering selects constraints and evidence, reserves output space, reuses stable prefixes and compresses history into recoverable references. It has distinct responsibilities from persistence, retrieval and model weights. ## Three Levels of Knowledge and State A model's language capabilities and vast knowledge come from post-training weights; context provides the current problem, constraints, and available evidence. One should not interpret "the model does not automatically remember chat content between requests" as "all the model's knowledge is in this input." A separate level is the session history, files, and databases maintained by the application: they can exist long-term, but existence does not mean they have entered the current reasoning process. Model WeightsParameters learned in trainingPersistent App StateFiles, databases, history recordsSelection and AssemblyRetrieve, read, summarizeCurrent GenerationUses weights and contextCurrent ContextMaterials available this turnKnowledge, Persistent State, and Current Context are Three LevelsRecords in a database can only serve as current evidence after being read and provided to the current reasoning process. For example, a repository contains the latest test report; the model weights cannot possibly contain the content of a report just generated. The application must read the report and hand over the relevant results to the model; if only a path is passed, the model usually needs to call a tool to retrieve the body. Files can help restore across sessions but do not automatically expand the context for a single turn. "Stateless API" also needs to be limited to the interface. Interfaces that explicitly pass history are usually organized by the caller to continue the conversation; hosted sessions may save and inject history on the server side, and inference services may also reuse prefix states. A more useful question for the application is: **What did this inference actually receive, who is responsible for injecting history, and where is the resumable real state saved?** Claude's window documentation distinguishes between training knowledge and current working context, and explains the specific counting rules for input, output, and thinking. [Claude Context Windows](https://platform.claude.com/docs/en/build-with-claude/context-windows) ## Window Budget and Output Reserve ### Leave Space for the Next Step First First, confirm the target model's input and output limits and counting rules, then allocate materials. The following is a planning example assuming a **combined input and output upper limit of 128,000 tokens**, where K in this table represents 1,000; it does not represent a fixed configuration of any specific model. | Item | Budget | Reason for Retention | |---|---:|---| | System constraints and tool definitions | 8K | Determines task boundaries and available operations | | Recent history and task summary | 32K | Retains current proposals and key decisions | | Retrieved evidence | 24K | Supports current judgment, with source locations | | Current question | 1K | User's new requirements for this turn | | Generation reserve | 16K | Includes answer, reasoning, and tool call output per interface rules | | Safety margin | 8K | Leaves space for counting errors and next-step materials | | Total | 89K | 39K remaining from the upper limit | The input part is 65K. If the next tool return is 50K of logs, the planned total becomes 139K, exceeding the limit by 11K. At this point, one should filter the logs or read key intervals first, rather than waiting for a request error to arbitrarily delete the earliest history. Streaming output changes the delivery method but does not increase window capacity. You can check the budget ledger with the following independent example. It only verifies arithmetic; actual requests must still use the target model's counting tools to process message templates, tool definitions, and multimodal content. ```python limit = 128_000 parts = {"rules_tools": 8_000, "history": 32_000, "evidence": 24_000, "question": 1_000} reserve, margin = 16_000, 8_000 planned = sum(parts.values()) + reserve + margin assert planned == 89_000 assert limit - planned == 39_000 incoming = 50_000 print("Need to cut at least", max(0, planned + incoming - limit), "tokens") # Need to cut at least 11000 tokens ``` Different tokenizers cannot be converted using fixed multipliers. Even when using the provider's counting endpoint, results should be treated as pre-request estimates and calibrated with response usage; Claude officially states that there may be slight differences between the two. [Token counting](https://platform.claude.com/docs/en/build-with-claude/token-counting) ### Does the next step still fit? The fixed 49K includes rules, history, question and safety margin. Output reserve and evidence vary independently. Overflow is arithmetic; choosing what to remove is an evidence decision. Do not delete a condition merely to shorten the bar. **Does the next step still fit?** Input, tool results, output reserve and margin share the planned budget; a cache hit does not remove logical context. ## Selecting Materials Based on Evidence Needs Context selection is primarily an issue of evidence coverage. Asking "where is a function defined" usually only requires locating the symbol and a small amount of nearby code; asking "whether the migration plan misses compatibility requirements" may require reading multiple complete chapters simultaneously. Retrieving two hits but missing a negative condition is more dangerous than reading completely. | Task | Material Selection | Boundaries to Verify | |---|---|---| | Locating facts or symbols | Retrieval snippets plus nearby context | Version, same-name objects, limiting conditions | | Comparing a small number of complete documents | Entire document or entire chapter | Whether appendices, exceptions, and definitions are truncated | | Synthesizing across large amounts of material | Phased retrieval, merging evidence then generating | Whether multi-hop associations are covered, whether conclusion sources are traceable | | Continuous modification engineering | Current file, requirements, diffs, verification results | Whether the summary still corresponds to the latest workspace | Take "whether this configuration is suitable for production" as an example. Retrieval might hit `enabled: true` in the example, but the limiting explanation is in the previous paragraph: "Single-node testing only." Evidence units should include the configuration and its applicable conditions, not just lines containing keywords. Attach file or document identifiers, versions, and locations to each piece of material; keep conflicting evidence side-by-side, allowing the model to identify conflicts, and read the original text if necessary. Retrieved content is data to be analyzed; it does not gain the authority of execution instructions just because it appears in the context. When "ignore previous instructions" appears on a webpage, the source identity should be retained and handled according to the content; task permissions are determined by the application's trusted configuration and user authorization. See [Security and Protection](/ai/03-agent-systems/05-authority-and-tool-boundaries.md) for relevant execution boundaries. Research has observed that placing key evidence at different positions in long inputs affects performance, with some tasks degrading more easily in the middle. This supports doing position sensitivity tests, but does not prove that all models "can't see the middle," nor does it guarantee that copying conclusions to the end makes them correct. [Lost in the Middle](https://arxiv.org/abs/2307.03172) ## Reuse Boundaries of Caching KV cache is a mechanism for reusing attention history states during inference; prompt caching is a service's reuse capability for repeated prompt prefixes, which may use such states internally. The two are at different abstraction layers and are not completely unrelated. API cache hits also do not mean the model "remembered the answer": changing suffixes still require generation. Request ATool DefinitionsStable prefixUser Question ARequest BTool DefinitionsStable prefixUser Question BPrefix content and cache conditions are identicalSame Prefixes Can Be Reused; Subsequent States Need Recalculation After ChangesCache hits save prefix computation or costs; this input still occupies the model's context window. In prefix caching, tool definitions and stable instructions are suitable for placement before dynamic messages. If a random ID is inserted at the very beginning each time, subsequent materials that were originally identical may no longer be reusable; if only the final question is modified, the previous common prefix still has reuse opportunities. The boundaries here are also affected by cache granularity, expiration time, and model and service configurations, so one cannot just compare visible text. Taking Claude as an example, caching involves the hierarchy of tools, system, and messages, and hits are also constrained by minimum length, cache breakpoints, and expiration time; these can be verified from usage fields such as `cache_read_input_tokens`. Costs and limits vary by model and platform; check the corresponding page when integrating, and do not treat a certain price multiplier as the caching mechanism itself. [Prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching) ### Where does a change stop prefix reuse? Compare a dynamic ID at the front with changing only the final question. Prefix dependence explains the value of stable ordering; actual hits also require provider version, length, lifetime and configuration conditions. **Where does a change stop prefix reuse?** Matching suffix text cannot cross an earlier change to count as shared prefix; reuse is not overall text similarity. ## Compressing History into a Resumable State The goal of summarization is to allow the next round to continue working. Simply writing "several issues have been discussed and fixed" has almost no recovery value because it does not clarify where fixes were made, what was verified, or what the next step depends on. Assume the task is fixing an importer: the old proposal has been rejected, the current patch has been committed, but only unit tests have been run. A useful summary should clearly distinguish facts, unfinished items, and constraints: ```json { "goal": "Fix duplicate submission issue in importer", "constraints": ["Keep existing file format", "Do not change public interfaces yet"], "current_artifacts": ["src/importer.py", "tests/test_importer.py"], "decisions": ["Use input batch ID for deduplication; do not use in-process sets"], "verified": ["Unit tests passed; detailed results in artifacts/unit-test.txt"], "pending": ["Verify duplicate submissions after process restart"], "next_action": "Read current patch and test report, perform recovery scenario verification" } ``` This is an application-customized state format, not a vendor's compaction protocol. When restoring, first check if the files and reports still exist and correspond to the current modifications; "tests passed" in the summary cannot replace test artifacts, nor can it automatically represent that subsequent modifications also passed tests. | Operation | What to Retain | Main Loss or Requirement | |---|---|---| | Trimming old outputs | Current relevant snippets and original text locations | Deleted details need to be re-read | | Compressing history | Goals, constraints, decisions, evidence indices, and next steps | Summaries may omit or miswrite; key states need verification | | Persistent memory | Stable information truly useful across sessions | Must handle versions, expiration, and permissions | If using server-side compaction, you must retain and pass back its structured state according to that interface; you cannot just stuff any summary JSON into the protocol block. See [Claude Compaction](https://platform.claude.com/docs/en/build-with-claude/compaction) for specific compatibility scope and process; see [Memory and State](/ai/03-agent-systems/04-memory-and-recovery.md) for cross-session design. ## How to Verify a Context Change is Effective Fix a set of tasks and answer bases, and compare three inputs: original history, compressed history, and retrieved materials. Record task accuracy, necessary evidence recall, input volume, first visible output latency, and total cost; then perform a "post-compression recovery" check to see if it repeats previously failed proposals, misses user constraints, or claims unfinished steps are complete. If input volume decreases but recovery errors increase, it means key state was deleted; if input volume increases but accuracy remains unchanged, check if tool outputs were redundantly retained; if factual answers are correct but support locations cannot be given, check the evidence index. Only then can one judge whether context selection improved the task, rather than just making the prompt look neater. See [Evaluation and Observability](/ai/04-evaluation-and-production/01-evaluation-and-observability.md) for subsequent evaluation methods. Continue with：[Retrieval, RAG and evidence](/ai/02-context-and-interfaces/03-retrieval-and-evidence)。