Purpose
Prevent API keys, credentials, tokens, and other secrets from entering prompts, completions, logs, training/fine-tuning corpora, or client-visible surfaces.
Engineering philosophy
Secrets never belong in context windows as data. Treat any path that can echo or store model I/O as a potential secret exfiltration channel.
Why it matters
Models and logs are high-bandwidth leak channels. A single leaked provider key or cloud credential can escalate into full environment compromise and unbounded spend.
Common failures
- API keys in frontend code or mobile apps
- Secrets pasted into prompts for 'debugging'
- Tool results returning credentials into context
- Fine-tuning on datasets that include secrets
Mandatory checks
Gate controls. Each check is pass/fail via artifact + pass condition. Expected from the annotated capability level when the system meets the minimum criticality tier.
Production secrets shall live in a secrets manager and not in prompts, repos, or prod notebooks
- Artifact
- Secrets-manager config + CI/repo secret-scan report including prompt and fixture paths
- Pass condition
- 0 privileged production secrets found in repos, prompt registries, or client bundles in the latest scan; 100% of production runtime secrets resolve from the secrets manager
Logging and tracing pipelines shall redact secret-like patterns
- Artifact
- Redaction config + sample of redacted traces + synthetic secret-injection test results
- Pass condition
- Synthetic secrets (API key/bearer/AWS-key patterns) injected into a canary request are redacted in persisted logs/traces at 100% detection in the test harness
Provider and cloud keys shall be rotatable and scoped; client apps shall not hold privileged keys
- Artifact
- Key inventory with rotation dates + client bundle/scan report
- Pass condition
- 0 privileged provider/cloud keys embedded in client apps; every production key has a rotation date within policy (≤90 days or provider-managed short-lived credentials)
Recommended checks
Strengthen posture beyond the gate. Same measurable structure; non-blocking unless elevated by organizational policy.
Pre-commit and CI secret scanning including prompt/fixture files
- Artifact
- Pre-commit + CI secret-scan config including prompt/fixture globs + latest scan report with findings disposition
- Pass condition
- Secret scanning covers application code, prompts, and fixtures; blocking on high-confidence secrets; last green main-branch scan ≤7 days (or last PR merge evidence)
Egress controls limiting where runtime credentials can be used
- Artifact
- Egress allowlist/policy config for runtimes that hold credentials + sample deny/allow logs (24h)
- Pass condition
- Runtime credentials can only call documented destinations; ≥1 deny event is observed in test or production logs proving enforcement in the last 90 days
Dataset scanning before fine-tuning or eval corpus publish
- Artifact
- Dataset secret/PII scan gate config before fine-tune or eval corpus publish + last scan report for a published corpus
- Pass condition
- 100% of fine-tune/eval corpora published in the last 90 days have a linked scan report; publish is blocked when critical findings are open
Evidence required
- Secrets management design
- Redaction configuration evidence
- Key rotation records
Severity & risk
- Severity
- critical
- Impact if violated
- Risk level
- critical
- Typical residual risk (impact × likelihood)
Engineering best practices
- Use short-lived credentials for tools
- Strip Authorization headers and env dumps from error payloads to models
- Educate builders: never ask the model to 'remember' a key
- Separate build-time and runtime secrets
Automatic validations
- Secret scanners in CI and image builds
- Runtime detectors blocking known secret formats in prompts
- Alerts on anomalous provider API key usage
Manual validations
- Review of new logging fields for secret risk
- Incident drills for leaked model API keys
Examples
- A developer tool refuses to send .env contents into an LLM chat
- Tracing middleware redacts AWS keys and bearer tokens before persistence
References
Crosswalks
Secure & Resilient Secure and Resilient
NIST AI Risk Management Framework · aligns-with
§7 Support
ISO/IEC 42001 · partial
LLM02 Sensitive Information Disclosure
OWASP Top 10 for Large Language Model Applications · supports
LLM07 System Prompt Leakage
OWASP Top 10 for Large Language Model Applications · aligns-with
CC6 Logical and Physical Access
SOC 2 Trust Services Criteria · evidence-for
C1 Confidentiality
SOC 2 Trust Services Criteria · evidence-for
Security Security
AWS Well-Architected Framework · aligns-with
Future evolution
Native provider APIs that never accept raw secrets in prompt fields, with automatic scrubbing attestations.