Adversarial Security

Untrusted input (including model-generated text) shall never authorize privileged tool or side-effect actions without server-side policy—so prompt injection and privilege escalation cannot bypass the control plane.

Artifact

Server-side policy mediating privileged tool/actions (not prompt-only) + Versioned injection/privilege-escalation corpus + CI gate report + Policy-engine deny sample logs or harness results (denyRatePct + modelTextPrivilegeGrants)

Pass condition

≥95% of corpus cases that attempt privilege escalation via untrusted input are denied; 0 cases where model text alone granted a privileged tool call in the suite (measuredAt ≤90 days). If production AI cannot invoke tools or privileged side effects, score NOT_APPLICABLE.

High-risk model outputs shall be schema-validated or policy-filtered before any write, irreversible, or financial side effect—so non-conforming output cannot drive privileged downstream actions.

Artifact

Coverage inventory of high-risk side-effect paths (write/irreversible/financial) + Schema or policy filter definitions applied before those side effects + Contract test results showing non-conforming output rejected on 100% of inventoried paths

Pass condition

100% of high-risk side-effect paths (impact tier write/irreversible/financial) reject non-conforming model output in contract tests; coverage inventory lists every such path (measuredAt ≤90 days). If production AI cannot drive write, irreversible, or financial side effects, score NOT_APPLICABLE.

Customer-facing releases shall run an abuse/jailbreak/injection security suite—with those case classes present—as a blocking gate, or record a time-boxed waiver (owner + expiry ≤30 days) when the gate does not pass.

Artifact

Abuse/jailbreak/injection suite definition covering those case classes + CI/release gate configuration that blocks promote on suite fail + Last 30 days of release reports showing gate=pass, or owned waivers (expiry ≤30 days)

Pass condition

100% of production releases in the last 30 days show an abuse/jailbreak/injection suite gate = pass (suite must include those case classes), or a time-boxed waiver with owner and expiry ≤30 days (measuredAt ≤90 days). If no customer-facing / production AI releases exist, score NOT_APPLICABLE.

Network and identity boundaries shall prevent the model/tool runtime path from becoming a universal proxy to internal systems—so the model identity reaches only allowlisted destinations and has 0 unrestricted routes to internal admin APIs or data stores.

Artifact

Architecture diagram of model/tool trust boundaries + Network policy or egress allowlist export for the model/tool runtime identity + Automated or reviewed probe results (allowlisted-only reachability; unrestricted internal admin/data-store routes = 0)

Pass condition

Automated or reviewed probe shows the model/tool runtime can reach only allowlisted destinations; 0 unrestricted routes to internal admin APIs or data stores from the model identity (measuredAt ≤90 days). If no model or tool runtime can initiate network calls, score NOT_APPLICABLE.

Production systems that use multi-turn dialogue, RAG, or MCP should run a versioned red-team suite with ≥10 multi-turn and ≥10 indirect RAG/MCP injection cases—so single-turn jailbreak tests alone cannot claim coverage.

Artifact

Versioned red-team suite covering multi-turn and indirect RAG/MCP injection + Case counts (≥10 multi-turn, ≥10 indirect RAG/MCP) with documented pass thresholds + Latest scored run report ≤90 days with retention ≥90 days

Pass condition

Suite includes ≥10 multi-turn and ≥10 indirect RAG/MCP injection cases; latest run ≤90 days meets documented pass thresholds; report retained ≥90 days (measuredAt ≤90 days). If the system has no multi-turn, RAG, or MCP surfaces, score NOT_APPLICABLE.

Where production AI accepts multimodal inputs (images, files, or similar), content-safety and malware scanners should run before model ingest—so unscanned media cannot reach the model path.

Artifact

Multimodal input content-safety and/or malware scanner config (pre-model-ingest) + Inventory of image/file types accepted in production + Latest CI or batch scan report ≤90 days showing type coverage and 0 unscanned production paths

Pass condition

Where multimodal inputs are accepted, scanner runs before model ingest; last report ≤90 days shows coverage of image/file types in use and 0 unscanned production paths (measuredAt ≤90 days). If multimodal inputs are not accepted, score NOT_APPLICABLE.

Sensitive AI contexts should have mechanisms to detect data exfiltration attempts—canary/honeytokens are one strong implementation; equivalent DLP, SIEM/UEBA, egress monitoring, or similar controls also satisfy when validated.

Artifact

Documented exfiltration-detection mechanism for sensitive AI contexts (mechanism class named) + Coverage of production-sensitive AI paths (inventory or mapping) + Latest validation ≤90 days (detection test, alert-rule exercise, or equivalent) with expected alerts / 0 silent misses

Pass condition

Sensitive AI contexts have a documented exfiltration-detection mechanism (canary/honeytoken, DLP, SIEM/UEBA, egress monitoring, or equivalent); latest validation ≤90 days shows expected detection/alerts for covered paths with 0 silent misses in the validation suite or review (measuredAt ≤90 days). If no sensitive AI contexts exist, score NOT_APPLICABLE.