Purpose
Provide secure, reliable runtime and network infrastructure for model serving, gateways, vector stores, agents, and MCP—including supply-chain integrity.
Engineering philosophy
AI workloads inherit cloud security and SRE fundamentals, then add GPU/runtime and model-artifact supply-chain concerns. There is no AI exception to patching, isolation, or least privilege.
Why it matters
Compromised gateways, exposed vector DBs, or poisoned model artifacts undermine every application-layer control.
Common failures
- Vector databases publicly reachable
- Unpatched inference runtimes
- Unsigned model artifacts from unverified sources
- Overly broad network paths from agents to internal services
Mandatory checks
Gate controls. Each check is pass/fail via artifact + pass condition. Expected from the annotated capability level when the system meets the minimum criticality tier.
AI data stores and control planes shall not be publicly exposed without authenticated edge controls
- Artifact
- CSPM/network scan of AI data stores and control planes + edge auth config
- Pass condition
- 0 AI data stores or control-plane endpoints publicly reachable without authentication in the latest scan; findings severity ≥ high closed or waived with expiry
Base images and runtimes shall follow documented patching SLAs
- Artifact
- Patching SLA policy + image age/CVE backlog report for AI runtimes
- Pass condition
- 100% of production AI runtime images are within patching SLA (e.g. critical fixes ≤ 14 days); report shows 0 SLA breaches or open waivers with expiry
Network segmentation shall limit agent/tool reachability to required dependencies
- Artifact
- Network policy / security-group export for agent and tool runtimes + dependency allowlist
- Pass condition
- Egress/east-west allowlists for agent/tool identities match the documented dependency inventory; probe from agent identity to a non-allowlisted internal service fails at 100%
GPU/accelerator isolation and noisy-neighbor controls shall protect multi-tenant inference
- Artifact
- Isolation design + capacity/isolation test report for shared accelerators
- Pass condition
- PASS if isolation controls are documented and the latest isolation/capacity test (≤90 days) meets stated limits
Recommended checks
Strengthen posture beyond the gate. Same measurable structure; non-blocking unless elevated by organizational policy.
Signed model and container artifacts with verify-on-deploy
- Artifact
- Cluster/admission controller config requiring signed images + last failed-unsigned admission event or drill
- Pass condition
- Unsigned model/container images cannot schedule in production namespaces; policy verified by a failed admission test within 90 days
Infrastructure-as-code with policy checks (CIS-aligned)
- Artifact
- IaC modules for AI infra + CIS-aligned policy scan config + latest scan report for production stacks
- Pass condition
- Production AI infra is declared in IaC; CIS (or equivalent) policy checks run on every apply/PR; critical findings for production stacks are 0 or have dated exceptions ≤90 days
Evidence required
- Network diagrams for AI components
- Patch and image scan reports
- Artifact signing/verification configuration
Severity & risk
- Severity
- high
- Impact if violated
- Risk level
- high
- Typical residual risk (impact × likelihood)
Engineering best practices
- Prefer private networking to providers where available
- Harden MCP hosts as internet-facing services
- Separate training/fine-tune clusters from serving
- Apply CIS benchmarks to hosts and Kubernetes where used
Automatic validations
- CSPM/misconfiguration scanning
- Image CVE gates in CI
- Admission controllers verifying signatures
Manual validations
- Architecture review for new inference deployments
- Supply-chain review for third-party model downloads
Examples
- A self-hosted Llama deployment sits behind a private gateway with mTLS
- Vector DB accepts connections only from the RAG service identity
References
Crosswalks
Secure & Resilient Secure and Resilient
NIST AI Risk Management Framework · aligns-with
§7 Support
ISO/IEC 42001 · partial
LLM03 Supply Chain
OWASP Top 10 for Large Language Model Applications · supports
CC6 Logical and Physical Access
SOC 2 Trust Services Criteria · evidence-for
A1 Availability
SOC 2 Trust Services Criteria · evidence-for
Security Security
AWS Well-Architected Framework · aligns-with
Level 2 Hosted build + signed provenance
SLSA (Supply-chain Levels for Software Artifacts) · aligns-with
Level 3 Hardened builds
SLSA (Supply-chain Levels for Software Artifacts) · partial
Verify-on-deploy Verification before use
SLSA (Supply-chain Levels for Software Artifacts) · supports
Future evolution
Attested inference environments and standardized model artifact signing ecosystems.