Context Engineering

Every production context builder shall enforce a finite max token/byte budget and documented prioritization rules so oversized inputs are truncated or rejected—never silently overflowing the model window.

Artifact

Context-budget config (max tokens/bytes) for each production context builder + Unit/integration tests showing truncate/reject per priority rules with 0 silent overflows

Pass condition

100% of production context builders enforce a max token/byte budget; tests show oversized inputs are truncated/rejected per priority rules with 0 silent overflows (budget evidence measuredAt ≤90 days). If no production AI paths assemble model context (RAG, chat history, tool results, or prompt templates), score NOT_APPLICABLE.

Retrieved and tool-sourced content shall receive a source label/type and an access check before inclusion in assembled model context—unauthorized chunks must be excluded.

Artifact

Context assembly code enforcing source labels on retrieved and tool-sourced chunks + Automated tests showing ACL exclusion of unauthorized chunks at 100% and labels on included chunks

Pass condition

Automated tests: unauthorized retrieval/tool chunks are excluded at 100%; included chunks carry a source label/type field in 100% of sampled assembled contexts (label/ACL evidence measuredAt ≤90 days). If context is prompt-only with no retrieval/tools, score NOT_APPLICABLE.

Production context assembly shall apply a documented inclusion policy that enumerates sensitive classes—including secrets and regulated data—with allow/deny rules, and shall strip or block disallowed classes before they enter the model context.

Artifact

Data-class inclusion policy enumerating sensitive classes with allow/deny rules + Enforcement tests or DLP hooks showing ≥95% strip/block on the sensitive-class fixture suite

Pass condition

Policy enumerates sensitive classes and allow/deny rules; tests show disallowed classes are stripped or blocked at ≥95% on the sensitive-class fixture suite (policy/suite evidence measuredAt ≤90 days). If context is purely static non-sensitive templates with no such paths, score NOT_APPLICABLE.

Production AI requests should emit per-request context-budget usage metrics, and operators should be alerted when usage saturates a documented share of max context—or when hard-truncate rate exceeds a defined threshold.

Artifact

Per-request context token/budget usage metrics (dashboard or telemetry export) + Alert rule for context saturation and/or hard-truncate rate with notify proof

Pass condition

≥99% of production requests in a 24h sample emit context-budget usage; an alert fires when usage exceeds the documented % of max context (or hard truncate rate exceeds threshold) (metrics/alert evidence measuredAt ≤90 days). If no production AI requests assemble model context, score NOT_APPLICABLE.

When production context uses summarization or compaction, teams should run a critical-fact eval set and retain facts at or above a documented threshold; the last run should be ≤90 days old and regressions should block context-pipeline releases.

Artifact

Compaction/summarization eval set of critical facts with documented retention threshold + Latest information-loss / retention report ≤90 days with release-blocking gate evidence

Pass condition

Critical-fact retention ≥ documented threshold after compaction on the eval set; last run ≤90 days; regressions block context-pipeline releases (eval evidence measuredAt ≤90 days). If no compaction path exists, score NOT_APPLICABLE.

Production context assemblers should emit labeled structured sections (for example JSON or XML blocks) that keep instructions distinct from untrusted data, and untrusted data should not be able to overwrite the instruction section.

Artifact

Context assembly schema/spec separating instruction vs data blocks + sample rendered request + Red-team or unit test showing untrusted data cannot overwrite the instruction section

Pass condition

Production assembler emits labeled structured sections (e.g. JSON/XML); untrusted data cannot overwrite the instruction section in a red-team or unit test (structure/test evidence measuredAt ≤90 days). If context is a single trusted static template with no untrusted sections, score NOT_APPLICABLE.