APRF-08 · Evaluation

Purpose

Institutionalize offline and online evaluation so releases are gated on measured quality, safety, and task success—not intuition or demo success.

Mandatory checks

Gate controls. Each check is pass/fail via artifact + pass condition. Expected from the annotated capability level when the system meets the minimum criticality tier.

EVL-M1L3 · DefinedTier 2 · ProductionhybridE3
Every journey marked critical shall have a versioned offline eval suite, and every relevant production change in the last 30 days shall have triggered that suite—or carry a documented waiver ≤14 days.
Artifact
Eval suite registry mapping critical journeys to versioned offline suites + CI (or equivalent) runs on prompt/model/tool changes for the last 30 days, or waivers ≤14 days
Pass condition
100% of journeys marked critical have a versioned offline suite; 100% of relevant production changes in the last 30 days triggered the suite (or documented waiver ≤14 days) (suite/CI evidence measuredAt ≤90 days). If no journeys are marked critical, score NOT_APPLICABLE.
Why this control exists
Threat map
Detect quality and safety regressions in critical journeys before they reach users.
Threats mitigated
MisinformationHallucinated ActionsHarmful Content Generation
Protects
UsersModelsPromptsSafety
MITRE: no technique mapped — this control addresses governance or assurance rather than a specific adversary technique.
Automated evaluation on change is what makes prompt and model updates reversible, measured decisions rather than blind ones. It targets correctness and safety regression; adversarial robustness is covered separately by SEC-M3 and EVL-R1.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.
EVL-M2L3 · DefinedTier 2 · ProductionhybridE3
Production release gates shall require numeric minimum thresholds for at least one quality metric and one safety metric on every critical journey, and shall block deploy in CI when those thresholds are not met.
Artifact
Gate config listing metric names and numeric thresholds per critical journey (quality + safety) + CI evidence that a failing gate blocks deploy (or equivalent required check)
Pass condition
Each critical journey has ≥1 quality and ≥1 safety metric with numeric threshold; failing gate blocks deploy in CI (gate evidence measuredAt ≤90 days). If no journeys are marked critical, score NOT_APPLICABLE.
Why this control exists
Threat map
Ensure releases cannot proceed when quality or safety metrics fall below agreed minimums.
Threats mitigated
MisinformationHarmful Content GenerationHallucinated Actions
Protects
UsersModelsSafety
MITRE: no technique mapped — this control addresses governance or assurance rather than a specific adversary technique.
Evaluation without an enforced numeric threshold is advisory and routinely overridden under delivery pressure. Hard gates make the release decision deterministic; no adversary technique maps directly.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.
EVL-M3L3 · DefinedTier 2 · ProductionhybridE3
Production AI workloads shall emit live metrics for task success and failure and for safety refusals, with a defined alert or review cadence and dashboard freshness no older than 24 hours.
Artifact
Dashboard or metric exports for task success/failure rates on production AI workloads + Dashboard or metric exports for safety refusal rates + Alert rule or documented review cadence; proof dashboard freshness ≤24 hours
Pass condition
Online metrics exist and are updating for task success/failure and safety refusals; alert or review cadence defined; freshness ≤24 hours on the dashboard (metrics evidence measuredAt ≤90 days). If no production AI workloads complete user-visible tasks or apply safety refusals, score NOT_APPLICABLE.
Why this control exists
Threat map
Observe real production outcomes, including safety refusals, rather than relying on pre-release testing alone.
Threats mitigated
Harmful Content GenerationJailbreakPrompt InjectionMisinformation
Protects
UsersModelsSafety
MITRE: ATLAS AML.T0054 · ATLAS AML.T0031
Refusal-rate and task-failure telemetry is one of the few live indicators that jailbreak techniques are succeeding at scale. It also surfaces gradual output-quality erosion that offline suites miss. The mapping is detective; these signals do not block the attempt.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.
EVL-M4L5 · OptimizingTier 3 · Mission CriticalhybridE3
High-risk AI production cutovers shall run a shadow or canary with a retained eval comparison against the incumbent, and shall promote to 100% traffic only after documented promotion criteria are met.
Artifact
Shadow/canary eval config for high-risk AI cutovers (promotion criteria documented) + Comparison report for the last high-risk cutover showing criteria met before full traffic
Pass condition
The last high-risk AI cutover retained a shadow/canary eval comparison that met promotion criteria before 100% traffic (cutover evidence measuredAt ≤90 days). If no high-risk AI cutover is in scope, score NOT_APPLICABLE.
Why this control exists
Threat map
Compare a candidate AI change against production behaviour before it serves users.
Threats mitigated
MisinformationHarmful Content GenerationDenial of Service
Protects
UsersAvailabilitySafety
MITRE: no technique mapped — this control addresses governance or assurance rather than a specific adversary technique.
Shadow comparison exposes regressions under the real traffic distribution, which curated suites do not reproduce. It reduces release risk; no adversary technique maps.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.

Evidence required

Eval suite definitions and ownership
Gate thresholds and recent pass/fail reports
Dashboards for online quality metrics

Recommended checks

Strengthen posture beyond the gate. Same measurable structure; non-blocking unless elevated by organizational policy.

EVL-R1L4 · Quantitatively ManagedTier 2 · ProductionhybridE3
Production AI systems should maintain three distinct offline eval tracks—regression, adversarial, and distribution-shift—each with its own corpus and named owner, and each should have run successfully on the last production model or prompt promotion.
Artifact
Eval catalog listing separate tracks for regression, adversarial, and distribution-shift with owners and corpora + Latest CI (or equivalent) matrix showing each track succeeded on the last production promotion
Pass condition
All three tracks (regression, adversarial, distribution-shift) exist with distinct corpora and named owners; each track ran successfully on the last production model/prompt promotion (catalog/CI evidence measuredAt ≤90 days). If no production AI model/prompt promotions exist, score NOT_APPLICABLE.
Why this control exists
Threat map
Ensure adversarial robustness is measured on its own and not averaged away by aggregate quality scores.
Threats mitigated
Prompt InjectionJailbreakMisinformation
Protects
ModelsPromptsUsersSafety
MITRE: ATLAS AML.T0043 · ATLAS AML.T0051 · ATLAS AML.T0054
Adversarial cases are a small fraction of any blended suite, so a single aggregate score can improve while robustness degrades. A dedicated adversarial track keeps injection and jailbreak resistance visible as its own release signal.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.
EVL-R2L4 · Quantitatively ManagedTier 2 · ProductionhybridE3
Production AI systems should run human preference or expert-review sampling on a documented cadence and sample size, covering production-like prompts, with disagreements adjudicated and recorded.
Artifact
Human preference / expert-review sampling protocol (cadence + sample size) + Last scored sample set with production-like prompts and inter-rater / adjudication notes
Pass condition
Cadence and sample size are defined; last sample ≤90 days covers production-like prompts; disagreements have adjudication recorded (sampling evidence measuredAt ≤90 days). If no production AI prompts/outputs exist for review, score NOT_APPLICABLE.
Why this control exists
Threat map
Apply human judgement to outputs on a cadence, where automated metrics cannot recognize the harm.
Threats mitigated
MisinformationHarmful Content GenerationBias and Discrimination
Protects
UsersSafety
MITRE: no technique mapped — this control addresses governance or assurance rather than a specific adversary technique.
Automated metrics miss harms that require domain expertise to even recognize, such as subtly wrong clinical or legal guidance. Sampled expert review supplies that judgement; no adversary technique maps.
Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.

More detailPhilosophy, failures, practices, validations, examples, crosswalks, and evolution

Engineering philosophy

If you cannot measure it, you cannot claim production readiness. Evaluation is a product dependency equal to tests in traditional software, extended for stochastic systems.

Why it matters

LLMs regress silently. Without evals, teams ship prompt or model changes that increase hallucination, toxicity, tool errors, or task failure rates under real traffic.

Common failures

Only manual spot checks before release
Eval sets that do not match production task distribution
No online monitoring of quality after deploy
Safety and quality evals owned by nobody

Severity & risk

Severity: critical; Impact if violated
Risk level: high; Typical residual risk (impact × likelihood)

Engineering best practices

Version eval datasets; treat label quality as seriously as code quality
Include tool-use and multi-turn scenarios, not only single-shot Q&A
Budget for eval compute as part of CI cost
Correlate eval failures to traces for fast debugging

Automatic validations

CI pipelines blocking on eval gate failures
Scheduled regression runs against production prompts/models
Drift detection when online metrics diverge from offline baselines

Manual validations

Periodic calibration of automated graders vs human labels
Product review of eval coverage gaps

Examples

A RAG support bot cannot ship a prompt change unless citation accuracy stays above threshold
An agent fails CI when tool-selection accuracy drops on the golden set

References

Crosswalks

Show 27 peer mappingsexpand

MEASURE Measure
NIST AI Risk Management Framework · supports
Safe Safe
NIST AI Risk Management Framework · aligns-with
Fair Fair — Harmful Bias Managed
NIST AI Risk Management Framework · partial
§6 Planning
ISO/IEC 42001 · partial
§9 Performance evaluation
ISO/IEC 42001 · supports
LLM04 Data and Model Poisoning
OWASP Top 10 for Large Language Model Applications · supports
LLM09 Misinformation
OWASP Top 10 for Large Language Model Applications · aligns-with
C1.1 Training Data Origin & Data Security
OWASP AI Application Security Verification Standard (AISVS) · supports
C1.2 Data Labeling and Annotation Security
OWASP AI Application Security Verification Standard (AISVS) · partial
C1.3 Training Data Quality and Security Assurance
OWASP AI Application Security Verification Standard (AISVS) · supports
C3.1 Model Authorization & Integrity
OWASP AI Application Security Verification Standard (AISVS) · supports
C3.2 Model Validation & Testing
OWASP AI Application Security Verification Standard (AISVS) · supports
C3.3 Controlled Deployment & Rollback
OWASP AI Application Security Verification Standard (AISVS) · partial
C3.4 Secure Development Practices
OWASP AI Application Security Verification Standard (AISVS) · partial
C3.5 Pipeline Fine-Tuning
OWASP AI Application Security Verification Standard (AISVS) · partial
C7.2 Hallucination Detection & Mitigation
OWASP AI Application Security Verification Standard (AISVS) · supports
C7.4 Source Attribution & Citation Integrity
OWASP AI Application Security Verification Standard (AISVS) · aligns-with
C11.1 Model Alignment, Safety, and Robustness Testing and Training
OWASP AI Application Security Verification Standard (AISVS) · supports
C11.2 Membership-Inference and Model-Inversion Mitigation
OWASP AI Application Security Verification Standard (AISVS) · partial
C11.3 Model-Extraction Defense
OWASP AI Application Security Verification Standard (AISVS) · partial
C11.4 Model Runtime Anomaly Detection
OWASP AI Application Security Verification Standard (AISVS) · supports
L5 Evaluation & Observability
CSA MAESTRO (Multi-Agentic Threat Model) · supports
reasoning-collapse Reasoning Collapse
CSA MAESTRO (Multi-Agentic Threat Model) · aligns-with
goal-drift Goal Drift in Delegated Chains
CSA MAESTRO (Multi-Agentic Threat Model) · aligns-with
CC3 Risk Assessment
SOC 2 Trust Services Criteria · evidence-for
PI1 Processing Integrity
SOC 2 Trust Services Criteria · evidence-for
Generative AI Lens Generative AI Lens (themes)
AWS Well-Architected Framework · aligns-with

Future evolution

Shared public eval protocols for agentic and MCP workloads, with portable scorecards.