Runbook = technical steps for one service. Playbook = roles and process for any incident.

After any incident where steps were wrong or missing. Quarterly minimum for tier-1 services.

Include shell commands?

Yes for safe, repetitive ops. Mark destructive commands and note required approvals.

Preview or copy each section below. Paste into Notion, Confluence, or your wiki.

Service runbook

Runbook: [Service / alert]

Service card

Field	Value
Owner	[Team]
On-call schedule	[Schedule name]
Escalation policy	[Policy name]
Status page component	[Component name]
Monitors	[Links]
Last tested	YYYY-MM-DD

What it does

[1–2 sentences: why a page here matters]

If you see this

[Alert condition or symptom]
[User-visible behavior]

Severity hint

Condition	Level
[All users down]	Critical
[Elevated errors > X%]	High
Otherwise	Medium: see severity matrix

Diagnose (in order)

[Dashboard/monitor]: check [metric]
Recent deploys: [CI/release link]
Dependencies: [DB, queue, vendor status]
Logs: [query]

Mitigate

Step	Action	Safe?	Expected result
1	[Restart workers / scale]	Yes	Errors drop
2	[Rollback / feature flag]	Needs approval	Service recovers
3	Escalate after [X] min	—	Bring in [expert/secondary]

Rollback

[Link or exact steps · who approves]

If customer-visible

Publish to [component] on status page:

We are investigating [symptom] affecting [component].

After resolution

Timeline notes in incident
Follow-up for missing monitors/docs
Update this runbook

Known false positives

[Looks like outage but is not: how to tell]

# Runbook: [Service / alert]

## Service card
| Field | Value |
|-------|-------|
| Owner | [Team] |
| On-call schedule | [Schedule name] |
| Escalation policy | [Policy name] |
| Status page component | [Component name] |
| Monitors | [Links] |
| Last tested | YYYY-MM-DD |

## What it does
[1–2 sentences: why a page here matters]

## If you see this
- [Alert condition or symptom]
- [User-visible behavior]

## Severity hint
| Condition | Level |
|-----------|-------|
| [All users down] | Critical |
| [Elevated errors > X%] | High |
| Otherwise | Medium: see severity matrix |

## Diagnose (in order)
1. [Dashboard/monitor]: check [metric]
2. Recent deploys: [CI/release link]
3. Dependencies: [DB, queue, vendor status]
4. Logs: `[query]`

## Mitigate
| Step | Action | Safe? | Expected result |
|------|--------|-------|-----------------|
| 1 | [Restart workers / scale] | Yes | Errors drop |
| 2 | [Rollback / feature flag] | Needs approval | Service recovers |
| 3 | Escalate after [X] min | — | Bring in [expert/secondary] |

## Rollback
[Link or exact steps · who approves]

## If customer-visible
Publish to **[component]** on status page:
> We are investigating [symptom] affecting [component].

## After resolution
- [ ] Timeline notes in incident
- [ ] Follow-up for missing monitors/docs
- [ ] Update this runbook

## Known false positives
- [Looks like outage but is not: how to tell]

How to use this template

One runbook per noisy alert or tier-1 service, not one mega-doc.
Test during game days; stale commands waste minutes.
Link from monitor runbook field or team wiki.

Open an incident from the alert, follow this runbook, log on the timeline, and publish status updates from the same record.

Explore incident management

On-Call Runbook