Reasoning and Thinking

Enable models to "think before answering"—spend more compute during inference on decomposition, trial-and-error, and self-checking, rather than relying solely on intuition compressed into weights during pre-training. Adaptive thinking and effort shift thinking depth from a manually filled token count to semantic tiers.

Overview

Enable models to "think before answering"—spend more compute during test-time on decomposition, trial-and-error, and self-checking, rather than relying solely on intuition compressed into weights during pre-training. This trajectory has evolved from Chain-of-Thought (2022, where asking models to write out reasoning steps significantly improved math/reasoning accuracy) to reasoning models, and now to Claude’s adaptive thinking: where the model decides for itself when to think and how deeply. Understanding this is key to tuning the trade-off between "fast and cheap" and "accurate and stable"—this is also the variable with the largest token budget fluctuations in Context Engineering.

Core intuition: Compute can be spent during training or during inference. Allowing the same model to "think" for a while before answering is equivalent to investing more computation during inference to search and verify—for multi-step reasoning and agentic long-horizon tasks, this yields tangible accuracy gains; for simple queries, it is pure waste. Therefore, the modern approach is neither "deep thinking throughout" nor "no thinking at all," but dynamically allocating thinking effort based on task difficulty, with a master switch to balance the trade-off.

Mechanism: The Thinking Block and Why It Works

When thinking is enabled, a response first presents a thinking block (the model's internal reasoning), followed by text (the answer for the user):

flowchart LR
    Q["Input"] --> TH["thinking block<br/>Decompose → Trial & Error → Self-Check"]
    TH --> A["text block<br/>Final Answer"]

Why "thinking" improves quality can be broken down into three actions:

• Decomposition: Breaking hard problems into smaller, manageable steps to reduce the error rate of individual steps.
• Search/Trial-and-Error: Exploring multiple paths and comparing them within the thought process, rather than committing to a single path immediately.
• Self-Check/Verification: Re-checking intermediate conclusions against constraints (especially for code and math).

For example, solving "27 × 453": answering directly often leads to alignment errors; whereas in the thinking block, it decomposes into 27×400 + 27×53, tries out breaking 27×53 further into 27×50+27×3, and finally self-checks by multiplying back 10800+1431=12231 to verify. The same three steps occur in coding (listing edge cases first, trying several implementations, then cross-referencing with requirements) and multi-step planning—this is precisely why Chain-of-Thought significantly boosts scores by making reasoning "explicit": it replaces a single high-stakes gamble with several smaller, higher-confidence bets.

The cost is also real: more tokens, more latency. Therefore, "how much to think" must be controllable, which is the reason for the existence of effort and thinking controls.

Claude's Thinking Control (Current API Behavior)

Modern Claude (Opus 4.6+) uses adaptive thinking, eliminating the need to manually fill in thinking budgets:

thinking = {"type": "adaptive", "display": "summarized"}   # Adaptive + Show Summary
output_config = {"effort": "high"}                         # Master Switch

Effort: The Master Switch for Depth

effort ∈ {low, medium, high, xhigh, max} (xhigh is new in Opus 4.7, sitting between high and max), default is high:

Higher effort makes the model more inclined to think more, explore more, and call more tools—this turns "thinking depth" from a token count into a semantic tier.

Thinking Visibility: Raw Text is Hidden by Default

• display defaults to omitted (Opus 4.7/4.8, Fable 5): The thinking block is still present in the response, but the thinking field is an empty string. To show reasoning to the user, set display: "summarized" to get a summary.
• Raw Chain-of-Thought (CoT) is never returned—the most you can get is a summary, not verbatim reasoning.
• Streaming UX Pitfall: With omitted, the stream looks like a "long pause followed by sudden text generation" (while it is actually thinking). To show progress, explicitly set summarized.

Multi-turn: Thinking Blocks Must Be Echoed Back

This is the most common engineering pitfall. In multi-turn conversations or agent loops, when feeding back the previous turn's thinking block:

• Continuing with the same model: You must echo it back exactly as is (including the signature, and even empty text blocks should be preserved). The API rejects modified blocks, but does not reject blocks that have already been read—showing a summary is fine, but editing/refactoring blocks will result in a 400 error.
• Switching models: When switching to another model, the API automatically discards the previous generation's (e.g., Fable 5) thinking block from the prompt (usually silently, and the discard happens before billing, so it is not charged); do not manually strip standard thinking blocks, as this may trigger order/signature 400 errors.

Interleaved Thinking and Task Budget

• Interleaved thinking: Under adaptive mode, the model thinks between tool calls (thinking while doing), without needing extra headers—this is particularly important for agent loops (thinking again after receiving each tool_result).
• Task budget (beta task-budgets-2026-03-13): Sets a token budget for the entire agent loop, allowing the model to see the countdown and self-terminate. It differs from max_tokens:

To Think or Not to Think: Treat Effort as a Dimension to Sweep

Do not reflexively max out xhigh/max. Stronger models have higher intelligence ceilings; start with high and sweep medium/high/xhigh on your own eval set to decide—the relationship is not monotonic: high effort often reduces the number of turns in agentic tasks, saving cost overall; while for some tasks, medium is equally good and faster (see Evaluation and Observability for evaluation).

Migration Tip: For older models, temperature was used to tune "creativity"; for new models, use effort to tune "thinking depth"—sampling parameters have been removed (see Tokens and Sampling).

Best Practices

• Use adaptive thinking, do not manually fill budgets. Modern Claude adapts thinking depth automatically; manually filling budget_tokens will result in a 400 error on Opus 4.7/4.8/Fable 5.
• Start effort at high, sweep tiers based on evals. The relationship is non-monotonic—high effort often reduces turns and saves cost in agentic tasks, while for some tasks medium is equally good and faster.
• Set effort starting points based on tasks. Classification/extraction → low; Math/planning → high; Long agentic/coding → high~xhigh with large max_tokens and streaming; Correctness over cost → max (beware of spinning/idling).
• Set display:"summarized" if you want to show reasoning to the user. With default omitted, the stream looks like a "long pause followed by sudden text generation."
• Echo thinking blocks exactly in multi-turns. For same-model continuations, include the signature exactly as is; for switching models, let the API discard them automatically, do not manually strip.
• If overthinking/over-exploring occurs, lower effort first, do not pile on prompt constraints. medium is often the sweet spot.

Trade-offs and Failure Modes (with Prompt Fixes)

• Overthinking: High effort on simple tasks leads to spinning and burning tokens → lower effort first (medium is often the sweet spot), do not rush to pile on prompt constraints. To suppress as needed, add: "Thinking increases latency; only think deeply when multi-step reasoning is truly needed; answer directly if you are confident."
• Over-exploration / Over-delegation (agent scenarios): Under high effort, the model is more prone to explore and spawn sub-agents → lower effort or provide clear boundaries.
• Thinking perceived as stuck: With omitted, the frontend appears unresponsive → use display: "summarized".
• Lost blocks in multi-turn: Manually modifying/deleting thinking blocks causes 400 errors → echo back exactly as is (including signature) for same-model, let API discard for switching models.
• Verbosity when thinking:{type:"disabled"}: When thinking is disabled, the model may write reasoning into the visible body → keep adaptive, or add "Only output the final answer, do not include intermediate reasoning."

References

• Anthropic Official Documentation: Extended Thinking, Adaptive Thinking, Effort, Task Budgets, Migration Guide (platform.claude.com)
• Paper: "Chain-of-Thought Prompting Elicits Reasoning in LLMs" (Wei 2022)

Keywords: chain-of-thought, CoT, reasoning model, test-time compute, extended thinking, adaptive thinking, effort, low, medium, high, xhigh, max, thinking block, signature, display, summarized, omitted, raw CoT, interleaved thinking, task budget, max_tokens, multi-turn echo-back, budget_tokens deprecated, overthinking