Context engineering

Once the output contract is clear, decide what the model receives now. Context engineering selects constraints and evidence, reserves output space, reuses stable prefixes and compresses history into recoverable references. It has distinct responsibilities from persistence, retrieval and model weights.

Three Levels of Knowledge and State

A model's language capabilities and vast knowledge come from post-training weights; context provides the current problem, constraints, and available evidence. One should not interpret "the model does not automatically remember chat content between requests" as "all the model's knowledge is in this input." A separate level is the session history, files, and databases maintained by the application: they can exist long-term, but existence does not mean they have entered the current reasoning process.

For example, a repository contains the latest test report; the model weights cannot possibly contain the content of a report just generated. The application must read the report and hand over the relevant results to the model; if only a path is passed, the model usually needs to call a tool to retrieve the body. Files can help restore across sessions but do not automatically expand the context for a single turn.

"Stateless API" also needs to be limited to the interface. Interfaces that explicitly pass history are usually organized by the caller to continue the conversation; hosted sessions may save and inject history on the server side, and inference services may also reuse prefix states. A more useful question for the application is: What did this inference actually receive, who is responsible for injecting history, and where is the resumable real state saved? Claude's window documentation distinguishes between training knowledge and current working context, and explains the specific counting rules for input, output, and thinking. Claude Context Windows

Window Budget and Output Reserve

Leave Space for the Next Step First

First, confirm the target model's input and output limits and counting rules, then allocate materials. The following is a planning example assuming a combined input and output upper limit of 128,000 tokens, where K in this table represents 1,000; it does not represent a fixed configuration of any specific model.

The input part is 65K. If the next tool return is 50K of logs, the planned total becomes 139K, exceeding the limit by 11K. At this point, one should filter the logs or read key intervals first, rather than waiting for a request error to arbitrarily delete the earliest history. Streaming output changes the delivery method but does not increase window capacity.

You can check the budget ledger with the following independent example. It only verifies arithmetic; actual requests must still use the target model's counting tools to process message templates, tool definitions, and multimodal content.

limit = 128_000
parts = {"rules_tools": 8_000, "history": 32_000,
         "evidence": 24_000, "question": 1_000}
reserve, margin = 16_000, 8_000
planned = sum(parts.values()) + reserve + margin
assert planned == 89_000
assert limit - planned == 39_000
incoming = 50_000
print("Need to cut at least", max(0, planned + incoming - limit), "tokens")
# Need to cut at least 11000 tokens

Different tokenizers cannot be converted using fixed multipliers. Even when using the provider's counting endpoint, results should be treated as pre-request estimates and calibrated with response usage; Claude officially states that there may be slight differences between the two. Token counting

Does the next step still fit?

The fixed 49K includes rules, history, question and safety margin. Output reserve and evidence vary independently. Overflow is arithmetic; choosing what to remove is an evidence decision. Do not delete a condition merely to shorten the bar.

Selecting Materials Based on Evidence Needs

Context selection is primarily an issue of evidence coverage. Asking "where is a function defined" usually only requires locating the symbol and a small amount of nearby code; asking "whether the migration plan misses compatibility requirements" may require reading multiple complete chapters simultaneously. Retrieving two hits but missing a negative condition is more dangerous than reading completely.

Take "whether this configuration is suitable for production" as an example. Retrieval might hit enabled: true in the example, but the limiting explanation is in the previous paragraph: "Single-node testing only." Evidence units should include the configuration and its applicable conditions, not just lines containing keywords. Attach file or document identifiers, versions, and locations to each piece of material; keep conflicting evidence side-by-side, allowing the model to identify conflicts, and read the original text if necessary.

Retrieved content is data to be analyzed; it does not gain the authority of execution instructions just because it appears in the context. When "ignore previous instructions" appears on a webpage, the source identity should be retained and handled according to the content; task permissions are determined by the application's trusted configuration and user authorization. See Security and Protection for relevant execution boundaries.

Research has observed that placing key evidence at different positions in long inputs affects performance, with some tasks degrading more easily in the middle. This supports doing position sensitivity tests, but does not prove that all models "can't see the middle," nor does it guarantee that copying conclusions to the end makes them correct. Lost in the Middle

Reuse Boundaries of Caching

KV cache is a mechanism for reusing attention history states during inference; prompt caching is a service's reuse capability for repeated prompt prefixes, which may use such states internally. The two are at different abstraction layers and are not completely unrelated. API cache hits also do not mean the model "remembered the answer": changing suffixes still require generation.

In prefix caching, tool definitions and stable instructions are suitable for placement before dynamic messages. If a random ID is inserted at the very beginning each time, subsequent materials that were originally identical may no longer be reusable; if only the final question is modified, the previous common prefix still has reuse opportunities. The boundaries here are also affected by cache granularity, expiration time, and model and service configurations, so one cannot just compare visible text.

Taking Claude as an example, caching involves the hierarchy of tools, system, and messages, and hits are also constrained by minimum length, cache breakpoints, and expiration time; these can be verified from usage fields such as cache_read_input_tokens. Costs and limits vary by model and platform; check the corresponding page when integrating, and do not treat a certain price multiplier as the caching mechanism itself. Prompt caching

Where does a change stop prefix reuse?

Compare a dynamic ID at the front with changing only the final question. Prefix dependence explains the value of stable ordering; actual hits also require provider version, length, lifetime and configuration conditions.

Compressing History into a Resumable State

The goal of summarization is to allow the next round to continue working. Simply writing "several issues have been discussed and fixed" has almost no recovery value because it does not clarify where fixes were made, what was verified, or what the next step depends on.

Assume the task is fixing an importer: the old proposal has been rejected, the current patch has been committed, but only unit tests have been run. A useful summary should clearly distinguish facts, unfinished items, and constraints:

{
  "goal": "Fix duplicate submission issue in importer",
  "constraints": ["Keep existing file format", "Do not change public interfaces yet"],
  "current_artifacts": ["src/importer.py", "tests/test_importer.py"],
  "decisions": ["Use input batch ID for deduplication; do not use in-process sets"],
  "verified": ["Unit tests passed; detailed results in artifacts/unit-test.txt"],
  "pending": ["Verify duplicate submissions after process restart"],
  "next_action": "Read current patch and test report, perform recovery scenario verification"
}

This is an application-customized state format, not a vendor's compaction protocol. When restoring, first check if the files and reports still exist and correspond to the current modifications; "tests passed" in the summary cannot replace test artifacts, nor can it automatically represent that subsequent modifications also passed tests.

If using server-side compaction, you must retain and pass back its structured state according to that interface; you cannot just stuff any summary JSON into the protocol block. See Claude Compaction for specific compatibility scope and process; see Memory and State for cross-session design.

How to Verify a Context Change is Effective

Fix a set of tasks and answer bases, and compare three inputs: original history, compressed history, and retrieved materials. Record task accuracy, necessary evidence recall, input volume, first visible output latency, and total cost; then perform a "post-compression recovery" check to see if it repeats previously failed proposals, misses user constraints, or claims unfinished steps are complete.

If input volume decreases but recovery errors increase, it means key state was deleted; if input volume increases but accuracy remains unchanged, check if tool outputs were redundantly retained; if factual answers are correct but support locations cannot be given, check the evidence index. Only then can one judge whether context selection improved the task, rather than just making the prompt look neater. See Evaluation and Observability for subsequent evaluation methods.

Continue with：Retrieval, RAG and evidence。