Prompt Engineering and Structured Output

After sampling parameters were removed, the "creativity/determinism" knob was replaced by two: prompt (clarify intent) and effort (provide sufficient compute). To make outputs consumable by programs, it relies not on "please return JSON", but on schema constraints.

Overview

Token and Sampling discussed a fork: Claude's managed API removed temperature/top_p/top_k starting from Opus 4.7; passing them results in a 400 error. The old-era feel of "tuning temperature to control creativity, using temperature 0 for determinism" is gone. Anthropic has moved control up to two higher-level, more controllable knobs:

• prompting——to be deterministic, hardcode the instructions; to be divergent, explicitly request diversity in the prompt;
• effort——controls how deep to go and how many tokens to spend (see Reasoning and thinking).

This article explains how to use these two knobs well (Part 1: Prompt Engineering), and when outputs need to be fed to downstream programs, how to use Structured Output to constrain "free text" into "guaranteed parseable JSON" (Part 2). These two often appear together: prompt defines behavior, schema defines shape.

Part 1: Prompt Engineering

What the model sees is a concatenated text

Context Engineering already covered the rendering order tools → system → messages. Prompt engineering is about deciding what to write in this text and how to arrange it:

• system: Stable role, rules, output conventions—freeze it, don't insert timestamps (preserve cache).
• user: Specific task for this turn + necessary dynamic context (retrieved snippets, state).
• few-shot examples: A few input→output examples teach format and style better than a long string of abstract rules. Place them between system and user.

Four principles that become apparent after a few passes

• Clarify "when/boundaries", don't pile up "musts". Recent Opus (4.7/4.8) has extremely strong instruction following; old-era CRITICAL: YOU MUST used to "subdue" the model will over-trigger. Soften "CRITICAL: Must use search tool" to "When the answer relies on information not in the conversation, search first then answer".
• Place instructions in high-attention positions. Put key constraints in the early system or late user sections, and wrap them with XML tags/delimiters (<rules>...</rules>), don't bury them in the middle (to counter lost in the middle).
• Positive examples are better than negative ones. "Write like this: ..." is more stable than "Don't do this". To be concise, give one concise example, which is more effective than listing ten "don't be verbose".
• Give intent, not just instructions. Explain "who this is for and what it's used for", and the model can fill in reasonable details itself—especially obvious for long tasks.

Migration Recipe: temperature → prompt + effort

Translate sampling parameters in old code according to intent:

A note on an old misconception: temperature=0 on old models also never guaranteed byte-for-byte reproduction (floating-point accumulation order, batching, MoE routing all fluctuate, see Token and Sampling). "Determinism" is an engineering goal (hardcoded prompt + low effort + accept minor drift), not a guarantee of a specific parameter.

Part 2: Structured Output

To feed model output to programs (store in database, call downstream API, make assertions), "please return JSON" is unreliable—the model might add preamble, miss fields, or drift in format. Structured Output uses schema at the API level to constrain the output into a guaranteed valid shape. Two levels:

1. JSON Output: Constrain the Entire Response

Pass output_config.format in messages.create (note: the old top-level output_format parameter is deprecated):

response = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=16000,
    messages=[{"role": "user", "content": "Extract: John (john@co.com), Enterprise plan"}],
    output_config={"format": {
        "type": "json_schema",
        "schema": {
            "type": "object",
            "properties": {
                "name":  {"type": "string"},
                "email": {"type": "string"},
                "plan":  {"type": "string"},
            },
            "required": ["name", "email", "plan"],
            "additionalProperties": False,   # Required
        },
    }},
)
# The first text block is valid JSON

The SDK also provides client.messages.parse() + Pydantic/Zod to directly get validated objects, avoiding manual parsing.

2. Strict Tool Calling: Constrain Tool Arguments

When the model calls tools, strictly validate arguments according to schema—add strict: true to the tool definition (on the tool, not tool_choice), and the schema must include additionalProperties: false and required:

tools=[{
    "name": "book_flight",
    "strict": True,
    "input_schema": {
        "type": "object",
        "properties": {
            "destination": {"type": "string"},
            "passengers":  {"type": "integer", "enum": [1,2,3,4]},
        },
        "required": ["destination", "passengers"],
        "additionalProperties": False,
    },
}]

This ensures tool_use.input strictly conforms to the schema—classification tasks using tools with enum/structured output are much more stable than letting the model "output a label word" and then parsing it.

Schema Capability Boundaries (Know this, or silently step into pitfalls)

Structured Output supports: basic types, enum/const/anyOf/$ref, string formats (date-time/email/uri/uuid...), additionalProperties: false (must be written for each object). Does not support: recursive schemas, numeric constraints (minimum/maximum), string length (minLength/maxLength), complex array constraints. Python/TS SDKs will automatically strip unsupported constraints and move validation to the client.

Two practical points: the first new schema has a one-time compilation cost, followed by 24-hour cache hits; structured output is mutually exclusive with citations (400 if both enabled, see RAG).

Structured Output × Evaluation: Making Validation Programmable

This is the most underestimated value of structured output. Evaluation and Observability mentioned scoring: "if it can be judged by code, don't leave it to the model". Structured output downgrades a large class of validations from "LLM-as-judge" to "code assertions": whether the return conforms to schema, whether enum values are valid, whether required fields are complete—all assert, deterministic, zero cost, no bias. First pin the output into an assertable shape with schema, then discuss whether to use an LLM judge.

Best Practices

• Delete all sampling parameters, translate intent into prompt + effort. temperature=0→effort:"low"+hardcode instructions; seek divergence→prompt explicitly "give N directions".
• Write instructions as "when/boundaries", don't pile up "musts". New Opus has strong instruction following, extreme wording will over-trigger; place key constraints in strong positions + wrap with XML.
• few-shot positive examples > a bunch of abstract rules. Give examples to teach format and style; if concise, give a concise example.
• If output feeds to programs, use schema. output_config.format constrains the whole segment, strict:true constrains tool arguments; don't rely on "please return JSON".
• Write additionalProperties:false + required for each object. Otherwise, strict mode doesn't take effect.
• Use schema to downgrade validation to code assertions. Don't ask for a judge if you can assert; structured output is the prerequisite for programmable eval.
• Remember schema boundaries and mutual exclusivity. Doesn't support recursive/numeric/length constraints; mutually exclusive with citations, choose one.

Trade-offs and Failure Modes

• Passing temperature/top_p/top_k on new models: 400 → delete, convert to prompt + effort.
• Old-style CRITICAL: YOU MUST bombardment: Tools/skills over-triggered → soften to conditional sentences "when to use".
• Relying on "please return JSON": Occasional preamble/missing fields/format drift → output_config.format strong constraint.
• Placing strict on tool_choice: Doesn't take effect → place on tool definition, configure additionalProperties:false.
• Schema writes minLength/minimum: Silently stripped, constraint didn't take effect → put this kind of range validation in client code.
• Enabling structured output and citations simultaneously: 400 → choose one; for traceability, go with plain text + citations; for parseability, go with schema.
• Verbose when thinking is off: Reasoning leaks into body under thinking:disabled → keep adaptive, or add "only output final answer" (see Reasoning and thinking).

References

• Anthropic Official Documentation: Structured Outputs (output_config.format / strict), Migration Guide (sampling parameter removal, prompt tuning), Tool Use (platform.claude.com, refer to official docs before implementation)
• Related: Token and Sampling, Reasoning and thinking, Context Engineering, Evaluation and Observability

Keywords: prompt engineering, system prompt, few-shot, XML tags, instruction placement, temperature migration, effort, structured output, structured outputs, output_config.format, json_schema, strict tool use, additionalProperties, required, messages.parse, Pydantic, Zod, enum, schema compilation cache, citations mutual exclusion, programmable validation, determinism