APRF-26 · Explainability & Transparency

Purpose

Provide appropriate explanations, citations, and decision traces so users, operators, and auditors can understand why the system produced an outcome.

Mandatory checks

Gate controls. Each check is pass/fail via artifact + pass condition. Expected from the annotated capability level when the system meets the minimum criticality tier.

EXP-M1L3 · DefinedTier 2 · ProductionhybridE3
High-stakes or factual RAG answers shall include ≥1 valid source ID or citation that resolves to an authorized corpus document—so operators and users can verify where the claim came from.
Artifact
Factual/high-stakes RAG eval suite measuring citation presence + Eval report showing ≥90% answers with ≥1 resolvable citation/source ID + Sample production traces with citations (supporting evidence)
Pass condition
On the factual/high-stakes RAG eval set, ≥90% of answers include ≥1 valid source ID/citation that resolves to an authorized corpus document (measuredAt ≤90 days). If no factual or high-stakes RAG outputs exist, score NOT_APPLICABLE.
Why this control exists
Threat map
Ensure grounded answers carry verifiable source attribution so claims can be checked.
Threats mitigated
MisinformationHallucinated ActionsPrompt Injection
Protects
UsersContextDataSafety
MITRE: ATLAS AML.T0067 · ATLAS AML.T0067.000 · ATLAS AML.T0070 · ATLAS AML.T0071
Verifiable citations let users and reviewers check claims instead of trusting fluent prose. This directly counters manipulation of trusted output components such as fabricated citations, and makes poisoned or false RAG entries traceable back to their source.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.
EXP-M2L3 · DefinedTier 2 · ProductionhybridE3
Operators shall reconstruct the production decision path (model→retrieval/tools→outcome) for sampled traces within a documented time budget—so incidents and disputes can be explained from retained evidence, not guesswork.
Artifact
Operator reconstruction procedure with documented time budget + Timed drill record covering ≥3 sampled production traces (≤90 days) + Evidence each sample reconstructed model→retrieval/tools→outcome within budget
Pass condition
On-call or operator successfully reconstructs model→retrieval/tools→outcome for 3/3 sampled production traces within the documented time budget (e.g. ≤15 minutes each; measuredAt ≤90 days). If no production AI outcomes with reconstructable paths exist, score NOT_APPLICABLE.
Why this control exists
Threat map
Make it possible to reconstruct how a specific production AI outcome was reached.
Threats mitigated
RepudiationPrompt InjectionHallucinated Actions
Protects
Audit TrailLogsRuntimeUsers
MITRE: no technique mapped — this control addresses governance or assurance rather than a specific adversary technique.
A reconstructable decision path is the basis of incident investigation and of resolving disputes about an AI outcome. The capability is forensic: it supports after-the-fact detection of injection or tool misuse rather than preventing either.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.
EXP-M3L3 · DefinedTier 2 · ProductionhybridE3
Explanation payloads (user-facing rationales, operator reconstructions, citation snippets) shall redact or block secrets and unauthorized data—so explainability does not become a leak path.
Artifact
Explanation redaction/blocking policy covering explanation payloads + Automated test showing 100% redaction/block of synthetic secret/PII fixtures + Production explanation sample scan with 0 privileged secret pattern hits (≤90 days)
Pass condition
Synthetic secret/PII fixtures in explanation paths are redacted or blocked at 100% in tests; latest production explanation sample scan shows 0 privileged secret pattern hits (measuredAt ≤90 days). If no explanation payloads exist (user rationale, operator reconstruction, citation snippets, or equivalent), score NOT_APPLICABLE.
Why this control exists
Threat map
Prevent explanation and rationale surfaces from becoming a disclosure channel.
Threats mitigated
Sensitive Information DisclosureSystem Prompt LeakageSecret LeakageData Exfiltration
Protects
SecretsDataPromptsContext
MITRE: ATLAS AML.T0056 · ATLAS AML.T0057 · ATLAS AML.T0069 · ATLAS AML.T0069.002
Rationale and citation surfaces frequently echo system prompt text or quote documents the user is not authorized to read. Constraining explanation content closes a channel that is actively used to extract system prompts and leak unauthorized data.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.

Evidence required

Example cited answers (redacted)
Operator reconstruction procedure
Policy for explanation disclosure vs confidentiality

Recommended checks

Strengthen posture beyond the gate. Same measurable structure; non-blocking unless elevated by organizational policy.

EXP-R1L4 · Quantitatively ManagedTier 2 · ProductionhybridE3
Material automated decisions should return a user-facing rationale (API field or UI explanation) so affected people can understand why the system acted—not only operators reconstructing traces after the fact.
Artifact
Material automated-decision catalog (decision types marked material) + ≥20-case sample showing user-facing rationale for each material type (100% coverage) + Gap list with named owners for any missing rationale
Pass condition
100% of decision types marked material in the catalog return a rationale field or UI explanation in a ≥20-case sample; gaps are tracked with named owners (measuredAt ≤90 days). If no material automated decisions exist, score NOT_APPLICABLE.
Why this control exists
Threat map
Give users an intelligible reason for material automated decisions that affect them.
Threats mitigated
RepudiationBias and DiscriminationMisinformation
Protects
UsersAudit TrailSafety
MITRE: no technique mapped — this control addresses governance or assurance rather than a specific adversary technique.
Explaining material automated decisions is what lets an affected user contest one, and it is required outright in several regulated contexts. This is a transparency obligation with no adversary technique mapping.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.
EXP-R2L4 · Quantitatively ManagedTier 2 · ProductionhybridE3
Every regulated AI feature should list the required explanation type and evidence in a maintained matrix—so compliance obligations are explicit, owned, and reviewable rather than implied by adjacent controls.
Artifact
Explainability requirements matrix (feature × regulation/obligation) + Last compliance review record ≤12 months with named owner
Pass condition
Every regulated AI feature lists required explanation type and evidence; the matrix was reviewed ≤12 months ago with a named owner (attest measuredAt ≤90 days). If no regulated AI features exist, score NOT_APPLICABLE.
Why this control exists
Threat map
Map formal explainability obligations to the regulated features they apply to.
Threats mitigated
RepudiationBias and Discrimination
Protects
UsersAudit TrailSafety
MITRE: no technique mapped — this control addresses governance or assurance rather than a specific adversary technique.
Explainability requirements differ sharply by jurisdiction and feature, so a single generic approach under- or over-delivers. Mapping them per feature makes the obligation testable; no adversary technique maps.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.
EXP-R3L5 · OptimizingTier 3 · Mission CriticalhybridE3
Material model or prompt promotions should retain a change or counterfactual summary describing what differed and expected impact—so operators can explain version-to-version behavior without archaeology.
Artifact
Change/counterfactual summary tooling or template for model/prompt promotions + Retained summary for the last material model or prompt promotion (≤90 days)
Pass condition
The last material model or prompt promotion includes a retained change or counterfactual summary (measuredAt ≤90 days). If no material model or prompt promotions occur, score NOT_APPLICABLE.
Why this control exists
Threat map
Make the behavioural effect of a model or prompt change reviewable rather than inferred.
Threats mitigated
Configuration DriftMisinformation
Protects
ModelsPromptsAudit Trail
MITRE: no technique mapped — this control addresses governance or assurance rather than a specific adversary technique.
A textual diff of a prompt or a changed model pin says little about how behaviour will actually shift. Counterfactual and change summaries give reviewers the real delta; no adversary technique maps.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.

More detailPhilosophy, failures, practices, validations, examples, crosswalks, and evolution

Engineering philosophy

Observability serves operators; explainability serves people affected by decisions and those who govern them. Production AI must support both audiences.

Why it matters

Opaque decisions block dispute resolution, regulatory review, and user trust—especially in high-stakes or regulated contexts.

Common failures

No citations for RAG answers that claim factual authority
Tool-driven actions with no human-readable rationale
Explanations that leak sensitive internal data
No audit-facing decision summary for contested outcomes

Severity & risk

Severity: high; Impact if violated
Risk level: medium; Typical residual risk (impact × likelihood)

Engineering best practices

Prefer source-grounded answers over free-form claims when stakes are high
Separate debug traces (privileged) from user explanations (minimized)
Version explanation formats with the product

Automatic validations

Eval gates for citation presence/accuracy on RAG suites
Tests that explanation redaction strips secrets

Manual validations

Legal/compliance review of explanation content for regulated flows
UX review of explanation clarity

Examples

A support bot cites knowledge-base article IDs with every policy answer
An underwriting assistant produces an auditor-facing feature contribution summary

References

NIST AI RMF — Explainability and Interpretability

Crosswalks

MAP Map
NIST AI Risk Management Framework · supports
Explainable Explainable and Interpretable
NIST AI Risk Management Framework · aligns-with
Accountable Accountable and Transparent
NIST AI Risk Management Framework · supports
LLM09 Misinformation
OWASP Top 10 for Large Language Model Applications · aligns-with
C7.2 Hallucination Detection & Mitigation
OWASP AI Application Security Verification Standard (AISVS) · supports
C7.4 Source Attribution & Citation Integrity
OWASP AI Application Security Verification Standard (AISVS) · aligns-with
C11.1 Model Alignment, Safety, and Robustness Testing and Training
OWASP AI Application Security Verification Standard (AISVS) · supports
reasoning-collapse Reasoning Collapse
CSA MAESTRO (Multi-Agentic Threat Model) · aligns-with

Future evolution

Standard explanation schemas for agentic and tool-mediated decisions.