Free DevOps template

Runbook

Symptoms, first checks with commands, fixes and when to escalate, for one alert.

Open in the DevOps & TechOps LabSave and export

It opens with this template loaded and runs in your browser. You don't need an account to use it; a free account saves your work and exports it.

The job it does

For each alert that pages someone at night: the steps a tired on-call engineer can follow.

What's inside

This is the template exactly as the DevOps & TechOps Lab opens it, read-only. Open it to run it, change it and export it.

  • 13 checklist items
  • Exports as Markdown
The brief: Runbook: checkout errors

A checklist you tick in the lab and export as Markdown for the wiki, the release ticket or the on-call handbook.

Runbook: checkout error rate above 2%

Alert: checkout_errors_rate > 2% for 5 min · Severity: 2 · Owner: Orders team on call

What it means

Customers cannot pay or place orders. Every minute costs about 40 orders at peak.

First 5 minutes

  • Acknowledge the page and post in the incident channel: "Looking at checkout errors"
  • Open the checkout dashboard: is it all stores or one region?
  • Check recent deploys: deploy history mangrove-web --last 3
  • If a deploy went out in the last hour, roll it back: deploy rollback mangrove-web --to previous

Find the cause

  • Payment gateway errors? grep "gateway timeout" /var/log/app.log | tail -20
  • Database slow? Check p95 query time and connections on the DB dashboard
  • Out of disk? df -h /var

Fix

  • Gateway down: switch to the backup gateway (feature flag payments.backup)
  • Disk full: systemctl restart log-rotate, then delete old logs in /var/log/archive
  • App servers stuck: systemctl restart checkout-api one server at a time

Escalate

  • Not better in 30 minutes: page the engineering manager and the payments team lead
  • Customers affected for more than 15 minutes: tell customer support

After

  • Write the timeline while it is fresh; schedule the post-mortem within 2 working days
Open in the DevOps & TechOps Lab

The lesson behind it

Learn why it works, with a podcast and a quiz for each lesson.

  • Pipeline template

    Build with a dependency cache, unit and end-to-end tests, then staging and production gated on the tests.

  • Release checklist

    Before, during and after a deploy: owners, go/no-go checks and the rollback trigger, as a checklist.

All DevOps templates · Every template