When an AI system behaves inconsistently, the instinct is often to edit the wording of the prompt. Sometimes that helps. More often, the real issue is that the system has not decided what information it should see, which source wins when facts conflict, how much evidence fits, or what output is valid.
Context engineering is the design of that information boundary. It includes system instructions, task context, examples, retrieved passages, tool results, constraints, user input, and the output schema. The model call is one component inside a pipeline that assembles and validates those pieces.
Learning objectives
You will design a context window as a budget, compare ambiguous and structured extraction contexts, separate evidence quality from answer quality, and version the template and selection policy that produced a request. The local example is in content/build-with-ai-agents/context-engineering.examples and does not call a provider.
The example uses deterministic rules rather than pretending a toy model can measure language-model quality. It demonstrates how to test the context assembly contract before attaching a model whose behavior may vary.
Prerequisites
You should understand token budgets, embeddings, retrieval, and structured outputs. No API key or GPU is required.
The context is a composition
A useful context usually contains several responsibilities:
- System constraints define non-negotiable behavior and boundaries.
- Task instructions describe the current operation and success condition.
- Examples and schemas show the shape of a valid result.
- Retrieved evidence supplies current, task-specific information.
- Tool results report observations made outside the model.
- The user request supplies the immediate goal, subject to authorization and validation.
These parts are not interchangeable. A retrieved document should not be allowed to override a system security rule. A tool result should carry provenance and timestamp rather than appearing as an unexplained assertion. A user request can ask for an action but cannot grant itself permission to perform it.
Context has a budget
Every token spent on context is a token unavailable for the response or later turns. More information can also make selection harder. Research on long contexts has found that relevant information placed in the middle can be used less effectively than information near the ends in some settings. The practical implication is not a universal “put everything at the top” rule; it is to measure placement and trim irrelevant material.
Rank candidates by task relevance, freshness, authority, access scope, and redundancy. Keep source identifiers beside passages. If the budget is exceeded, remove low-priority content according to an explicit policy. Do not silently cut the oldest message or the middle of a paragraph and call the result grounded.
Toy problem / implementation
The local extraction experiment scores whether a context contains required fields for a support-ticket record. Four cases make the mechanism visible: an ambiguous request, a structured instruction, a grounded request containing policy evidence, and an overlong context where the required information is buried in noise.
def score(text, required=("intent=", "amount=")):
return sum(marker in text for marker in required) / len(required)This is not a language-model evaluator. It is a contract test for context assembly. A real system would add schema validation, field-level correctness, evidence attribution, and a model-based or human evaluation set. The point is to catch regressions in the deterministic part before they are hidden by a model call.
Run it yourself
From the repository root:
cd content/build-with-ai-agents/context-engineering.examples
./verify.shThe verifier runs two tests and writes a transcript. The structured and grounded cases contain both required markers; the ambiguous and overlong cases do not. The result is intentionally simple enough to inspect line by line.
Use the same structure with a real model by storing the final assembled context as a redacted test artifact, then asserting that the output parses against the schema and cites the relevant evidence. Keep a separate test for context assembly. Otherwise a model regression and a retrieval regression can look identical.
Evaluate the context before the prose
A useful evaluation table separates at least four questions. Did the selector include the relevant evidence? Did it exclude unauthorized or stale evidence? Did the model extract the right fields? Did the final action obey the policy? A failure at the first stage should not be described as a model reasoning failure, and a correct extraction from an unauthorized document is not a success.
For each test case, record the expected evidence ids, allowed fields, required citations, and valid action. Run the context builder without a model and compare selected ids and token counts. Then run the model stage and validate the structured result. Finally, test the downstream action with a fake tool or dry-run mode. This decomposition makes regressions smaller and makes a review useful to someone who did not write the prompt.
When the context changes, compare versions side by side. Did a new example displace evidence? Did a ranking change move the decisive sentence into a less useful position? Did a longer system instruction leave too little room for the answer? A diff of the final redacted context is often more informative than a diff of the template source because retrieval and formatting decisions happen at runtime.
Examples, schemas, and constraints
Examples are useful when they demonstrate the exact boundary of acceptable outputs. They can also create accidental bias: an example’s formatting, names, or assumptions may be copied into unrelated cases. Keep examples short, representative, and versioned. Test behavior with and without each example so its effect is known.
A schema makes output expectations machine-checkable, but it does not make an invalid value true. JSON can be syntactically valid while the amount is negative, the currency is unsupported, or the cited policy is irrelevant. Validate types, ranges, enums, cross-field relationships, and evidence references after parsing.
Tool results should be treated as observations with metadata. Include the tool name, arguments, result timestamp, source identifiers, and any error state. Do not concatenate raw tool output into an instruction block where untrusted text can masquerade as a system rule. Keep trust boundaries visible in both code and traces.
Precedence is a policy, not a formatting trick
Putting text in a section called “system” does not make an arbitrary string trustworthy. The application must decide which component is allowed to supply which instruction. A useful precedence policy might allow application-owned system constraints to define safety and permissions, the task template to define the operation, retrieved documents to supply evidence, and the user to supply a request within those boundaries. Tool output reports observations; it does not rewrite the policy that authorized the tool.
Write that policy down and test it with harmless fixtures. Include a document that says it is the system, a tool result containing an instruction-like sentence, and a user request that asks for an unauthorized field. The expected result is not merely “the model ignored it.” The pipeline should preserve provenance, reject or quarantine conflicting content, and produce a trace that explains which boundary applied. This is a system property that can be tested around any model.
Context also needs lifecycle rules. Conversation history can contain stale assumptions, prior mistakes, or information the user no longer wants retained. Summarization may reduce length, but it can also erase a qualification that changes the meaning of a request. Store a compact state representation with timestamps and source references, and make deletion and correction explicit. Do not treat the entire transcript as an eternal memory store.
Real-world application: structured information extraction
Consider extracting a refund request from a ticket. The context should contain the task definition, allowed categories, a schema for intent, amount, currency, and evidence spans, the ticket text after redaction, and only the policy passages relevant to the request. The final result should be parsed and validated before a downstream action is considered.
If a field is missing, return an explicit missing value rather than guessing. If two retrieved policies conflict, surface the conflict and route to review. If the ticket asks for an action beyond the user’s authority, the system should refuse or create a review task; the context cannot turn an untrusted request into permission.
Version the context template, retrieval query, ranking policy, redaction rules, schema, and model settings together. A change to any one of them can change behavior. Store a trace with hashed or redacted inputs, selected evidence ids, token counts, validation results, and the final action.
This versioning is especially important when teams share a context component. A prompt fragment that is safe for a summarizer may be wrong for an action-taking assistant. Give each task a named contract, its own evaluation cases, and an owner. Reusing a context block is valuable only when its assumptions are explicit and still true in the new workflow.
Keep a clear boundary between data needed for the current decision and data retained for later analytics. Redact secrets before assembly, cap the size of user-controlled fields, and record truncation as an explicit event. A context budget is also a privacy budget: every copied field is another place where sensitive information can appear.
This makes context review concrete for security, operations, and editorial teams.
It also gives reviewers a stable artifact to compare across all future production releases safely.
Failure modes and debugging
- Instruction collision. Separate trusted instructions from untrusted documents and label provenance.
- Context bloat. Measure token counts and remove redundancy instead of appending more text.
- Buried evidence. Test placement, chunking, and ranking on a fixed set.
- Schema overconfidence. Validate semantic constraints after parsing.
- Example leakage. Ensure examples do not contain values that can be copied into the answer.
- Stale retrieval. Carry freshness and authority metadata into selection.
- Untraceable prompts. Version the assembled context and record a redacted trace.
Limitations / when not to use
Context engineering cannot repair a model that lacks the capability for the task, and it cannot make untrusted evidence authoritative. A longer prompt is not a substitute for a database query, a permission check, or a deterministic calculation. If the task has a small, stable input space, ordinary code may be clearer.
The local experiment measures markers, not language understanding. It does not prove that a provider will follow the same context hierarchy or that a schema guarantees correct values. Those claims require task evaluation with the actual model and data distribution.
Exercises
- Add a freshness score and trim stale evidence first.
- Add a JSON Schema validator and field-level error messages.
- Create an adversarial untrusted document and verify it cannot change the system policy in the local pipeline.
- Measure token counts for five context versions and report what was removed at each budget.
Next in this path
The next article is Tool-Using AI: From Free-Form Text to Reliable Actions. It takes the context contract one step further: selecting a tool, validating its arguments, executing it, and handling failure without treating generated text as permission.
Sources
The research dossier is context-engineering. Context and workflow design are informed by Anthropic’s Building effective agents and OpenAI’s prompt engineering guide. Long-context placement is discussed by Liu et al., Lost in the Middle. Structural constraints follow the JSON Schema specification, and programmatic generation patterns are cross-checked against Guidance. Retrieval and answer evaluation dimensions are documented by Ragas.
Research and code were last verified on 2026-08-05. The local scores test context assembly invariants, not general model quality.