← Domains & pillars

Reliability & OperationsView domain

APRF-28

Performance & SLO Engineering

Latency, throughput, and error budgets for AI features—SRE discipline applied to GenAI.

Purpose

Define SLIs/SLOs for AI latency, availability, and quality signals; manage error budgets so AI features meet user experience and capacity targets.

Mandatory checks

Gate controls. Each check is pass/fail via artifact + pass condition. Expected from the annotated capability level when the system meets the minimum criticality tier.

  • PERF-M1L3 · DefinedTier 2 · ProductionhybridE3

    Every journey marked critical must have documented numeric availability and latency-percentile SLO targets in a maintained catalog.

    Artifact
    SLO catalog listing critical AI journeys with numeric availability and latency targets + Coverage showing 100% of marked-critical journeys have both target types
    Pass condition
    100% of journeys marked critical have availability % and latency percentile targets recorded in an SLO catalog (catalog evidence measuredAt ≤90 days). If no AI journey is marked critical, score NOT_APPLICABLE.

    Why this control exists

    Threat map

    Define the availability and latency commitment for critical AI user journeys.

    Threats mitigated

    Denial of Service

    Protects

    AvailabilityUsers

    MITRE: no technique mapped — this control addresses governance or assurance rather than a specific adversary technique.

    SLOs define the baseline against which degradation, abuse impact, and incident severity are measured. This is a reliability control with no adversary technique mapping.

    Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.

  • PERF-M2L3 · DefinedTier 2 · ProductionhybridE3

    Production AI services must collect and make available operational metrics for latency, error rate, and at least one AI-specific quality or task-success indicator.

    Artifact
    Metric definitions or exporters covering latency, error rate, and ≥1 AI quality/task-success signal + Proof metrics are queryable/available for operational monitoring
    Pass condition
    Metrics for latency, error rate, and at least one AI-specific quality or task-success indicator are collected and available for operational monitoring (metrics evidence measuredAt ≤90 days). If no production AI services are in scope, score NOT_APPLICABLE.

    Why this control exists

    Threat map

    Expose latency, error, and AI quality metrics so degradation is measurable.

    Threats mitigated

    Denial of ServiceMisinformation

    Protects

    AvailabilityUsersLogs

    MITRE: no technique mapped — this control addresses governance or assurance rather than a specific adversary technique.

    Metrics that are not emitted cannot be alerted on or investigated later. Instrumentation is the prerequisite for PERF-M3; no adversary technique maps directly.

    Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.

  • PERF-M3L3 · DefinedTier 2 · ProductionhybridE3

    Every critical AI journey SLO must have burn-rate (or equivalent) alert policies, and a test or documented fire must prove the notification path works.

    Artifact
    Burn-rate or SLO alert policies covering each critical AI journey SLO + Alert test or documented fire proving notification path
    Pass condition
    Alert policies exist for each critical journey SLO; an alert test or documented fire demonstrates the notification path works (alert evidence measuredAt ≤90 days). If no critical AI journeys with SLOs are in scope, score NOT_APPLICABLE.

    Why this control exists

    Threat map

    Alert when SLO burn indicates the system is degrading beyond agreed limits.

    Threats mitigated

    Denial of ServiceDenial of Wallet

    Protects

    AvailabilityCost

    MITRE: ATLAS AML.T0029

    Sustained SLO burn is often the first visible symptom of resource-exhaustion abuse against an AI endpoint. Alerting turns that into an actionable signal; it is detective and does not itself restore capacity.

    Informative threat context — mappings reduce exposure and do not guarantee mitigation; not certification.

Evidence required

  • SLO definitions for AI journeys
  • Dashboard and alert configuration
  • Recent burn-rate or incident examples (redacted)
More detailPhilosophy, failures, practices, validations, examples, crosswalks, and evolution

Engineering philosophy

If it is not measured with an SLO, it is not operationally owned. Stochastic systems still need latency and quality budgets.

Why it matters

Unbounded TTFT, streaming stalls, and silent quality burn destroy UX and hide regressions that “availability” alone will not catch.

Common failures

  • No p95/p99 latency targets for model calls
  • Quality treated as a one-time eval, not an online SLO
  • No error budget for AI feature releases
  • Capacity planning ignores bursty agent tool loops

Severity & risk

Severity
high
Impact if violated
Risk level
medium
Typical residual risk (impact × likelihood)

Engineering best practices

  • Separate SLOs for AI features vs core non-AI paths
  • Budget tokens and wall-clock together
  • Correlate performance regressions with model/prompt version changes

Automatic validations

  • Synthetic latency probes
  • Burn-rate alerts
  • CI performance budgets for critical paths where feasible

Manual validations

  • Quarterly SLO review with product owners
  • Post-incident SLO recalibration

Examples

  • Chat p95 TTFT < 1.5s with error-budget policy that freezes prompt experiments when burned
  • Agent task-success rate tracked as a quality SLO alongside HTTP availability

References

Crosswalks

  • MEASURE Measure

    NIST AI Risk Management Framework · supports

  • §9 Performance evaluation

    ISO/IEC 42001 · supports

  • LLM10 Unbounded Consumption

    OWASP Top 10 for Large Language Model Applications · supports

  • C9.1 Execution Budgets, Loop Control, and Circuit Breakers

    OWASP AI Application Security Verification Standard (AISVS) · supports

  • L5 Evaluation & Observability

    CSA MAESTRO (Multi-Agentic Threat Model) · supports

  • A1 Availability

    SOC 2 Trust Services Criteria · evidence-for

  • Reliability Reliability

    AWS Well-Architected Framework · aligns-with

  • Performance Efficiency Performance Efficiency

    AWS Well-Architected Framework · partial

Future evolution

Standard GenAI SLI catalogs (TTFT, tool-loop duration, citation accuracy) across ecosystems.