APRF-27 · Data Governance & Quality

Purpose

Own lineage, quality, labeling, retrieval-index governance, and train/serve consistency for data that shapes AI behavior.

Mandatory checks

Gate controls. Each check is pass/fail via artifact + pass condition. Expected from the annotated capability level when the system meets the minimum criticality tier.

DG-M1L3 · DefinedTier 2 · ProductionhybridE3
Every production retrieval corpus and index shall have a named owner, a version ID, and a refresh cadence; indexes stale beyond cadence shall be flagged or rebuilt.
Artifact
Corpus/index inventory listing production indexes with owner, version ID, and refresh schedule + Evidence stale-beyond-cadence indexes are flagged or rebuilt
Pass condition
100% of production indexes have owner + version ID + refresh cadence; 0 indexes missing any field; stale beyond cadence are flagged or rebuilt (inventory measuredAt ≤90 days). If no production retrieval corpora or vector indexes exist, score NOT_APPLICABLE.
Why this control exists
Threat map
Establish accountable ownership and version control over the corpora that ground production answers.
Threats mitigated
Data and Model PoisoningMemory PoisoningMisinformation
Protects
DataContext
MITRE: ATLAS AML.T0070 · ATLAS AML.T0059 · ATLAS AML.T0064
An owned, versioned corpus makes unauthorized content changes attributable and reversible, which is what turns RAG poisoning from a persistent compromise into a recoverable one. Without versioning there is no baseline against which dataset integrity erosion can be detected.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.
DG-M2L3 · DefinedTier 2 · ProductionhybridE3
Every dataset used in production eval gates or fine-tunes shall have documented provenance and quality criteria, and promotion shall be blocked when those are missing.
Artifact
Dataset cards or registry entries for eval/fine-tune sets covering provenance and quality criteria + CI or review-checklist evidence that promotion is blocked when cards are missing
Pass condition
100% of datasets used in production gates or fine-tunes have provenance + quality criteria documented; promotion is blocked if missing (CI or review checklist evidence; inventory measuredAt ≤90 days). If no in-scope evaluation or fine-tuning datasets exist, score NOT_APPLICABLE.
Why this control exists
Threat map
Ensure data used for evaluation and fine-tuning comes from known, vetted sources.
Threats mitigated
Data and Model PoisoningSupply Chain CompromiseMisinformation
Protects
DataModels
MITRE: ATLAS AML.T0020 · ATLAS AML.T0019 · ATLAS AML.T0059 · ATLAS AML.T0010 · ATT&CK T1195
Poisoned training and evaluation data is introduced through unvetted third-party sources that look legitimate at ingest time. Documented provenance and acceptance criteria create the checkpoint at which poisoned or low-quality datasets can be rejected.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.
DG-M3L3 · DefinedTier 2 · ProductionhybridE3
Every path that promotes user feedback or working memory into durable memory or training data shall require a policy check or human approval, and ungated promotion shall be denied by test.
Artifact
Promotion policy listing feedback/memory→durable/training write paths + Write-path controls (policy engine or human approval) on those paths + Automated tests showing ungated promotion is denied
Pass condition
100% of feedback→durable-memory/training promotion paths require a policy check or human approval; automated tests show ungated promotion is denied (deny evidence measuredAt ≤90 days). If no such write path exists, score NOT_APPLICABLE.
Why this control exists
Threat map
Prevent unvetted feedback or memory from being promoted into content that shapes future answers.
Threats mitigated
Memory PoisoningData and Model PoisoningMisinformation
Protects
MemoryDataContext
MITRE: ATLAS AML.T0080 · ATLAS AML.T0080.000 · ATLAS AML.T0070 · ATLAS AML.T0031
Automatic promotion of user feedback or session memory hands an attacker a write path into the system's durable knowledge. Governing promotion forces review before content can influence other users' answers, which is the mechanism that makes context poisoning persistent.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.

Evidence required

Corpus/index inventory with owners
Data quality criteria and sample reviews
Lineage examples from answer to source version

Recommended checks

Strengthen posture beyond the gate. Same measurable structure; non-blocking unless elevated by organizational policy.

DG-R1L4 · Quantitatively ManagedTier 2 · ProductionhybridE3
Each critical corpus shall have a documented freshness SLO, ≥95% of sampled docs shall meet that SLO in the last 7 days, and an alert shall fire on freshness breach.
Artifact
Freshness SLO definitions for each critical corpus ID + Dashboard or job report showing ≥95% SLO compliance over the last 7 days + Alert config (or firing proof) covering freshness breaches
Pass condition
Each critical corpus has a documented freshness SLO (e.g. max age hours); ≥95% of sampled docs meet the SLO in the last 7 days; an alert fires on freshness breach (sample/alert evidence measuredAt ≤7 days). If no critical production corpora/indexes exist, score NOT_APPLICABLE.
Why this control exists
Threat map
Detect stale or incomplete corpora before they degrade grounded answers.
Threats mitigated
MisinformationData and Model Poisoning
Protects
DataContext
MITRE: ATLAS AML.T0059
Freshness and coverage telemetry surfaces silent index decay that no error rate reports. Sharp unexplained changes in coverage are also an early indicator of dataset integrity erosion.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.
DG-R2L4 · Quantitatively ManagedTier 2 · ProductionhybridE3
Every production embedding or feature pipeline shall run a train/serve skew job within the last 7 days, document at least one skew threshold, and open a tracked ticket or page on breach.
Artifact
Skew monitor config covering each production embedding/feature pipeline + Last weekly (or ≤7-day) skew report comparing train vs serve distributions + Threshold definition plus ticket/page routing on breach
Pass condition
A skew job ran within the last 7 days for every production embedding/feature pipeline; at least one documented threshold exists; breach creates a tracked ticket or page (job/alert evidence measuredAt ≤7 days). If no production embedding or feature pipelines exist, score NOT_APPLICABLE.
Why this control exists
Threat map
Detect divergence between offline and online pipelines that silently degrades results.
Threats mitigated
Vector and Embedding WeaknessesMisinformationConfiguration Drift
Protects
DataModelsContext
MITRE: no technique mapped — this control addresses governance or assurance rather than a specific adversary technique.
Skew between training and serving pipelines produces degradation that offline evaluation cannot reproduce, because the evaluated features are not the served ones. This is a correctness control with no adversary technique mapping.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.
DG-R3L4 · Quantitatively ManagedTier 2 · ProductionhybridE3
Every major eval or fine-tune set used in production promotion shall have a dataset card covering purpose, source, PII handling, and a last-updated date within 12 months.
Artifact
Dataset cards (or registry metadata) for each major eval and fine-tune corpus + Fields covering purpose, source, PII handling, and last-updated date
Pass condition
100% of major eval/fine-tune sets used in production promotion have a dataset card with purpose, source, PII handling, and last-updated date ≤12 months (inventory measuredAt ≤90 days; card last-updated ≤365 days). If no major eval or fine-tune sets are used in production promotion, score NOT_APPLICABLE.
Why this control exists
Threat map
Document dataset purpose, source, and PII handling so downstream reuse is an informed decision.
Threats mitigated
Data and Model PoisoningSensitive Information Disclosure
Protects
DataAudit Trail
MITRE: no technique mapped — this control addresses governance or assurance rather than a specific adversary technique.
Dataset cards make source and PII handling explicit at the point where a dataset is reused for a new purpose. This is documentation that supports provenance decisions rather than blocking a technique.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.

More detailPhilosophy, failures, practices, validations, examples, crosswalks, and evolution

Engineering philosophy

Privacy asks who may see data; governance asks whether the data is fit for purpose. Bad corpora and poisoned indexes create silent production failures.

Why it matters

Hallucinations, biased outcomes, and retrieval failures often originate in unmanaged data—not in the model alone.

Common failures

RAG indexes with no freshness or ownership
Eval and fine-tune sets without label quality review
Feedback loops that reinforce errors into memory or training
No lineage from production answer to source document version

Severity & risk

Severity: high; Impact if violated
Risk level: high; Typical residual risk (impact × likelihood)

Engineering best practices

Treat indexes as release artifacts with version IDs
Separate experimental corpora from production indexes
Review user-generated feedback before it becomes training signal

Automatic validations

Index build pipelines emit version and checksum
Alerts on stale critical corpora
CI checks that eval datasets meet schema/quality gates

Manual validations

Periodic corpus content audits
Label quality sampling

Examples

A policy RAG index is rebuilt nightly with a new version ID and canary before cutover
Fine-tune data requires dual review before entering the training set

References

Crosswalks

Show 17 peer mappingsexpand

MAP Map
NIST AI Risk Management Framework · supports
Fair Fair — Harmful Bias Managed
NIST AI Risk Management Framework · partial
§4 Context of the organization
ISO/IEC 42001 · aligns-with
Annex A AI system controls (selected themes)
ISO/IEC 42001 · partial
LLM04 Data and Model Poisoning
OWASP Top 10 for Large Language Model Applications · supports
LLM08 Vector and Embedding Weaknesses
OWASP Top 10 for Large Language Model Applications · supports
C1.1 Training Data Origin & Data Security
OWASP AI Application Security Verification Standard (AISVS) · supports
C1.2 Data Labeling and Annotation Security
OWASP AI Application Security Verification Standard (AISVS) · partial
C1.3 Training Data Quality and Security Assurance
OWASP AI Application Security Verification Standard (AISVS) · supports
C3.2 Model Validation & Testing
OWASP AI Application Security Verification Standard (AISVS) · supports
C8.1 Access Controls on Memory & RAG Indices
OWASP AI Application Security Verification Standard (AISVS) · supports
C8.2 Embedding Sanitization & Validation
OWASP AI Application Security Verification Standard (AISVS) · supports
C8.3 Memory Expiry & Revocation
OWASP AI Application Security Verification Standard (AISVS) · supports
C11.1 Model Alignment, Safety, and Robustness Testing and Training
OWASP AI Application Security Verification Standard (AISVS) · supports
CWE-89 SQL Injection
OpenCRE (Open Common Requirements Enumeration) · partial
L2 Data Operations
CSA MAESTRO (Multi-Agentic Threat Model) · supports
P-series Privacy (selected)
SOC 2 Trust Services Criteria · partial

Future evolution

Portable dataset and index manifests with integrity proofs across vendors.