What should a runbook include?

Symptoms, checks (what to look at), actions (what to try), escalation (when to call someone else), and links to dashboards and logs.

How do I create a runbook for production outages?

Identify common failures. For each, document: symptoms, checks, actions, escalation. Include links to dashboards. Keep it updated.

What is a runbook template?

A template structure: Title, Symptoms, Checks, Actions, Escalation, Links. Copy and fill in for each failure type. Customize for your system.

Runbook Template for Production Outages | StackRail

What this problem means

A runbook is a step-by-step guide for fixing common failures. When production breaks, you don't want to figure it out from scratch. A template helps you create runbooks for your system.

Why this matters

- Faster resolution: Runbooks cut time-to-fix from hours to minutes.

- Consistency: Anyone on the team can follow the same steps.

- Onboarding: New team members can respond to incidents with a runbook.

Real-world example

A startup had no runbooks. When the database went down, the on-call engineer spent 45 minutes figuring out what to check. A simple runbook—"1. Check RDS status, 2. Check connection pool, 3. Restart if needed"—would have cut that to 10 minutes.

How to fix it

1. Identify common failures: Database down, API down, high error rate, high latency.

2. Document steps: For each, write step-by-step: what to check, what to try, when to escalate.

3. Include links: Dashboards, logs, provider status pages.

4. Keep updated: When the system changes, update the runbook.

5. Test: Run through the runbook during a drill or low-severity incident.

Runbook template structure

Title: e.g., "Database Down" Symptoms: What users see, what alerts fire. Checks: 1. RDS status, 2. Connection pool, 3. Recent deploys. Actions: 1. Restart RDS if needed, 2. Scale connection pool, 3. Rollback deploy. Escalation: When to call someone else. Who to call. Links: CloudWatch, RDS console, status page.

Common mistakes

- Runbooks that are too vague ("check the database").

- Outdated runbooks (system changed, runbook didn't).

- No escalation path.

Quick checklist

- [ ] Create runbooks for: database down, API down, high error rate

- [ ] Include specific steps and links

- [ ] Define escalation path

- [ ] Review and update quarterly

- [ ] Test during a drill

Need help with production readiness? Get a free 30-minute audit.

Book Free 30-Min Production Audit

View our DevSecOps services

Check if your system has this risk

Take the 60-second production readiness assessment to identify gaps in your infrastructure.

Start Assessment

Frequently asked questions

What should a runbook include?: Symptoms, checks (what to look at), actions (what to try), escalation (when to call someone else), and links to dashboards and logs.
How do I create a runbook for production outages?: Identify common failures. For each, document: symptoms, checks, actions, escalation. Include links to dashboards. Keep it updated.
What is a runbook template?: A template structure: Title, Symptoms, Checks, Actions, Escalation, Links. Copy and fill in for each failure type. Customize for your system.