Purpose
Design how context windows are filled—retrieval, history, tool results, and system state—so the model receives necessary, authorized, and bounded information.
Engineering philosophy
Context is a scarce, trusted resource. Overstuffing, under-filtering, and leaking privileged context are engineering defects, not model quirks. Structure beats volume.
Why it matters
Poor context design causes hallucinations, privacy leaks, prompt injection via retrieved content, and unbounded cost. Agents fail when context is noisy or when critical constraints are buried.
Common failures
- Dumping entire conversation history without summarization or TTL
- Retrieving documents without ACL enforcement at query time
- No provenance: model cannot distinguish user text from retrieved policy
- Tool results concatenated without size or sensitivity limits
Mandatory checks
Gate controls. Each check is pass/fail via artifact + pass condition. Expected from the annotated capability level when the system meets the minimum criticality tier.
Context assembly shall enforce maximum sizes and prioritization rules
- Artifact
- Context-budget config + unit/integration tests for overflow behavior
- Pass condition
- 100% of production context builders enforce a max token/byte budget; tests show oversized inputs are truncated/rejected per priority rules with 0 silent overflows
Retrieved and tool-sourced content shall be labeled and access-checked before inclusion
- Artifact
- Context assembly code/tests showing source labels + ACL checks on retrieval/tool results
- Pass condition
- Automated tests: unauthorized retrieval chunks are excluded at 100%; included chunks carry a source label/type field in 100% of sampled assembled contexts
Sensitive context classes (secrets, regulated data) shall have explicit inclusion policies
- Artifact
- Data-class inclusion policy + enforcement tests or DLP hooks on context assembly
- Pass condition
- Policy enumerates sensitive classes and allow/deny rules; tests show disallowed classes are stripped or blocked at ≥95% on the sensitive-class fixture suite
Recommended checks
Strengthen posture beyond the gate. Same measurable structure; non-blocking unless elevated by organizational policy.
Context budgets are monitored per request with alerts on saturation
- Artifact
- Per-request context token/budget metrics dashboard + alert rule export for saturation thresholds
- Pass condition
- ≥99% of production requests in a 24h sample emit context-budget usage; alert fires when usage exceeds the documented % of max context (or hard truncate rate exceeds threshold)
Summarization or compaction strategies are tested for information loss on critical facts
- Artifact
- Compaction/summarization eval set of critical facts + latest information-loss report
- Pass condition
- Critical-fact retention ≥ documented threshold after compaction on the eval set; last run ≤90 days; regressions block context-pipeline releases
Structured context blocks (JSON/XML sections) separate instructions from data
- Artifact
- Context assembly schema/spec separating instruction vs data blocks + sample rendered request
- Pass condition
- Production assembler emits labeled structured sections (e.g. JSON/XML); untrusted data cannot overwrite the instruction section in a red-team or unit test
Evidence required
- Context assembly design document
- ACL enforcement evidence on retrieval paths
- Samples of labeled context structures (redacted)
Severity & risk
- Severity
- high
- Impact if violated
- Risk level
- high
- Typical residual risk (impact × likelihood)
Engineering best practices
- Design explicit context slots: instructions, user, memory, retrieval, tools
- Prefer citations and IDs over pasting large documents when possible
- Apply the same authorization checks to context as to API responses
- Test adversarial documents in the corpus as first-class fixtures
Automatic validations
- Unit tests for context builders and ACL filters
- Metrics on context token composition by source
- Guards rejecting oversized tool payloads
Manual validations
- Review of retrieval corpora for injection and oversharing risk
- Spot checks that production traces show expected context structure
Examples
- A RAG system includes document IDs and ACL tags; the model only sees chunks the user may access
- Agent tool output is truncated and classified before re-entering the context window
References
Crosswalks
MAP Map
NIST AI Risk Management Framework · supports
Privacy-Enhanced Privacy-Enhanced
NIST AI Risk Management Framework · aligns-with
LLM02 Sensitive Information Disclosure
OWASP Top 10 for Large Language Model Applications · supports
LLM08 Vector and Embedding Weaknesses
OWASP Top 10 for Large Language Model Applications · supports
Performance Efficiency Performance Efficiency
AWS Well-Architected Framework · partial
Future evolution
Standard context schemas across agent frameworks and portable ACL-aware retrieval protocols.