Model Governance

Critical production AI paths shall reference immutable, pinned model identifiers, and shall not use floating aliases such as “latest” (or equivalent provider defaults that can change without a deploy).

Artifact

Production model pin config (or registry) listing immutable model IDs on critical paths + Lint or CI rule that rejects “latest”/floating aliases on those paths

Pass condition

0 “latest”/floating aliases on critical production paths; 100% of those paths reference immutable model IDs; lint or CI rejects floating aliases (pin evidence measuredAt ≤90 days). If no production AI paths call hosted or self-hosted models, score NOT_APPLICABLE.

Every production model promotion (version bump, pin change, or registry cutover) shall link a passing eval artifact, and the promotion path shall block promote-without-eval—not rely on informal review alone.

Artifact

Promotion policy or CI/registry rule requiring an eval pass artifact on model version bumps + Promotion log (or CI matrix) for the last 30 days showing linked eval pass artifacts; 0 promote-without-eval

Pass condition

100% of production model promotions in the last 30 days have linked eval pass artifacts; promote-without-eval is blocked by gate (promotion evidence measuredAt ≤90 days). If no production model promotions occur (pin changes, registry cutovers, or version bumps), score NOT_APPLICABLE.

Production systems should publish a deprecation and sunset policy for models and embeddings that defines notice periods and forced-sunset rules, record sunset dates for superseded production pins in the registry, and keep undocumented pins past sunset at zero (exceptions time-boxed).

Artifact

Model/embedding deprecation and sunset policy (notice period + forced-sunset rules) + Registry export showing sunset dates for superseded pins; query of undocumented pins past sunset = 0

Pass condition

Policy defines notice period and forced-sunset rules; ≥1 superseded production model/embedding has a sunset date in the registry; undocumented pins past sunset = 0 without exception (deprecation evidence measuredAt ≤90 days). If production uses no hosted or self-hosted models/embeddings that can be superseded, score NOT_APPLICABLE.

Each production AI workload should declare an explicit allowlist of model capabilities it may use (for example code execution, vision, browsing, or tool-calling modes), and denied capability attempts should be recorded in test or production within 90 days.

Artifact

Per-workload capability allowlist config (code execution, vision, browsing, etc.) + Deny log or test fixture showing a denied capability attempt within 90 days

Pass condition

Each production workload has an explicit capability allowlist; ≥1 denied capability attempt is recorded in test or prod within 90 days (allowlist evidence measuredAt ≤90 days). If models are text-only with no optional capabilities, score NOT_APPLICABLE.

Every open-weight or fine-tuned model in production should have a completed license and provenance review ≤12 months old, and any blocked or restricted licenses should carry a documented exception with owner and expiry.

Artifact

License/provenance review checklist (or registry fields) for open-weight and fine-tuned models + Completed reviews ≤12 months old for each such production model; exceptions for blocked licenses with expiry

Pass condition

100% of open-weight/fine-tuned production models have a license+provenance review ≤12 months old; blocked licenses have documented exceptions with expiry (review evidence measuredAt ≤90 days). If all production models are proprietary hosted APIs with no local/fine-tuned weights, score NOT_APPLICABLE.

Production AI systems should keep a queryable model inventory (or registry) in which every production model records a named owner, data-residency constraint, and intended use—with zero incomplete rows on those fields.

Artifact

Model inventory / registry export listing production models + Query or attestation showing 0 incomplete owner/residency/intended-use rows

Pass condition

100% of production models have owner, residency, and intended-use fields; inventory query returns 0 incomplete rows (inventory evidence measuredAt ≤90 days). If no production AI paths call hosted or self-hosted models, score NOT_APPLICABLE.