Purpose
Control which models (hosted or self-hosted) may be used in which environments, how they are versioned, evaluated, deprecated, and constrained by capability.
Engineering philosophy
Models are dependencies with behavioral contracts. Pin versions, document capabilities and known failure modes, and never silently change the production model under a customer-facing workload.
Why it matters
Provider default model changes, shadow upgrades, and unvetted open weights introduce quality, cost, security, and compliance regressions that teams discover only after incidents.
Common failures
- Using 'latest' model aliases in production
- No inventory of models and where they run
- Self-hosted models without supply-chain or license review
- Routing to cheaper models without eval coverage for those routes
Mandatory checks
Gate controls. Each check is pass/fail via artifact + pass condition. Expected from the annotated capability level when the system meets the minimum criticality tier.
Production shall use pinned model identifiers; aliases like “latest” are forbidden on critical paths
- Artifact
- Production model pin config + lint/CI rule rejecting “latest” aliases
- Pass condition
- 0 “latest”/floating aliases on critical production paths in config lint; 100% of critical paths reference immutable model IDs
Model inventory shall exist with owners, data residency, and intended use
- Artifact
- Model inventory registry export
- Pass condition
- 100% of production models have owner, residency, and intended-use fields; query returns 0 incomplete rows
Model changes shall require evaluation evidence before promotion
- Artifact
- Promotion policy + CI/registry requiring eval artifact on model version bumps
- Pass condition
- 100% of production model promotions in last 30 days have linked eval pass artifacts; promote without eval is blocked by gate
Recommended checks
Strengthen posture beyond the gate. Same measurable structure; non-blocking unless elevated by organizational policy.
Deprecation and sunset policy for models and embeddings
- Artifact
- Model/embedding deprecation and sunset policy + registry entries showing sunset dates for superseded pins
- Pass condition
- Policy defines notice period and forced-sunset rules; ≥1 superseded production model/embedding has a sunset date in the registry; no undocumented pins past sunset without exception
Capability allowlists (e.g., code execution, vision) per workload
- Artifact
- Per-workload capability allowlist config (code execution, vision, browsing, etc.) + deny logs
- Pass condition
- Each production workload has an explicit capability allowlist; a denied capability attempt is recorded in test or prod within 90 days
License and provenance review for open-weight and fine-tuned models
- Artifact
- License/provenance review checklist + completed reviews for each open-weight or fine-tuned production model
- Pass condition
- 100% of open-weight/fine-tuned production models have a license+provenance review ≤12 months old; blocked licenses have documented exceptions with expiry
Evidence required
- Model inventory and version pins
- Change records for model promotions
- Eval comparisons for model swaps
Severity & risk
- Severity
- high
- Impact if violated
- Risk level
- high
- Typical residual risk (impact × likelihood)
Engineering best practices
- Abstract provider SDKs behind an internal model gateway with policy
- Track embedding model versions separately—index rebuilds are part of change
- Document fallback models and degraded modes
- Separate experimental models into non-production projects
Automatic validations
- Config lint rejecting unpinned model IDs in production configs
- Gateway deny rules for unapproved models
- CI requiring eval artifact for model version bumps
Manual validations
- Architecture review when introducing a new model family
- Legal/compliance review for data-processing terms of new providers
Examples
- Production chat uses provider-model-2025-03-01; canaries test the next pin before cutover
- A coding agent is restricted to models that meet internal code-exfiltration evals
References
Crosswalks
GOVERN Govern
NIST AI Risk Management Framework · supports
MAP Map
NIST AI Risk Management Framework · supports
§8 Operation
ISO/IEC 42001 · supports
Annex A AI system controls (selected themes)
ISO/IEC 42001 · partial
LLM03 Supply Chain
OWASP Top 10 for Large Language Model Applications · supports
LLM04 Data and Model Poisoning
OWASP Top 10 for Large Language Model Applications · supports
CC8 Change Management
SOC 2 Trust Services Criteria · evidence-for
Sustainability Sustainability
AWS Well-Architected Framework · partial
Generative AI Lens Generative AI Lens (themes)
AWS Well-Architected Framework · aligns-with
Level 1 Build process documented
SLSA (Supply-chain Levels for Software Artifacts) · partial
Provenance Artifact provenance
SLSA (Supply-chain Levels for Software Artifacts) · supports
Future evolution
Model bill of materials (MBOM) standards and portable evaluation attestations across vendors.