1. Cheat sheets
  2. Engineering Roles

10 minReference

DevOps and SRE cheat sheet

CI/CD and reliability on a few screens: a GitHub Actions workflow to copy, Docker, kubectl and Terraform commands, release strategies, SLO and burn-rate numbers, incident roles, a postmortem template, and the Git and Linux commands you reach for on release day and on call.

If you see X, reach for Y

Start with the first fix in each row; most teams jump to the expensive one too early.

You seeReach for firstThen, if neededLesson
CI takes 25 minutesCache dependencies; run lint, unit and build jobs in parallelSplit tests across runners; run only affected packagesTrusted CI, Actions CI
"Works on my machine"A Dockerfile and a lockfile; npm ci in CIDev containers for the whole teamContainers
Deploys are scary, so they are rareSmaller changes, deployed more often, with a one-command rollbackCanary releases and feature flagsSafe releases
A bad deploy reached all 320 storesRings: pilot stores first, then a share, then everyoneAutomatic rollback on error-rate or SLO burnSafe releases
Servers configured by hand drift apartInfrastructure as code, applied only from CIDrift detection with a scheduled planIaC
Long-lived cloud keys in CI secretsOIDC from GitHub Actions to the cloudPer-environment roles with least privilegeSupply chain, Deploy and secure
Pods restart again and againkubectl logs --previous and describeFix probes, memory limits or the crashKubernetes
Alerts fire all night, nobody actsAlert on SLO burn and user symptoms, not on every CPU spikeDelete or demote alerts with no runbook actionBurn-rate alerts
"Is it down?" takes 20 minutes to answerA dashboard of the four golden signals per serviceTraces with a request id end to endOpenTelemetry, Observability
Incidents are chaoticNamed roles, one channel, regular status updatesSeverity levels and paging rules written downIncident command
The same incident keeps coming backBlameless postmortem with owned actionsTrack action completion in reviewsPostmortems
The team spends days on manual choresMeasure toil, automate the biggest item firstCap toil at about half of each engineer's timeToil

A GitHub Actions workflow to copy

Test on every pull request, deploy from main through a protected environment. Version tags such as @v4 are shown for readability; in a hardened repo pin each action to a full commit SHA (Actions CI, Trusted CI, Supply chain).

YAML
name: ci
on:
  pull_request:
  push:
    branches: [main]

permissions:
  contents: read            # least privilege for every job by default

concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: true  # a new push cancels the old run

jobs:
  test:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: postgres
        ports: ['5432:5432']
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - run: npm run lint
      - run: npm test
        env:
          DATABASE_URL: postgres://postgres:postgres@localhost:5432/postgres

  deploy:
    needs: test
    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    environment: production   # required reviewers and secrets live here
    permissions:
      contents: read
      id-token: write         # lets the job sign in to the cloud with OIDC
    steps:
      - uses: actions/checkout@v4
      - run: ./scripts/deploy.sh
I want to...Workflow keyLesson
Run by hand from the Actions tabon: workflow_dispatchActions CI
Run every night (UTC)on: schedule: - cron: '0 2 * * *'Actions CI
Test on several Node versionsstrategy: matrix: node: [20, 22], then ${{ matrix.node }}Actions CI
Run a job after anotherneeds: testActions CI
Skip a step unless on mainif: github.ref == 'refs/heads/main'Actions CI
Use a secret${{ secrets.SENTRY_TOKEN }} passed through env:, never echoedDeploy and secure
Pass files between jobsactions/upload-artifact and actions/download-artifactTrusted CI
Require approval before productionenvironment: production with required reviewers in repo settingsSafe releases
Stop a hung jobtimeout-minutes: 15Trusted CI

Docker: "I want to..."

Compose v2 is a plugin, so the command is docker compose with a space; the old docker-compose binary is retired (Containers).

I want to...CommandLesson
Build and tag an imagedocker build -t mangrove/api:1.8.0 .Containers
Run it, mapping a port, removed on exitdocker run --rm -p 3000:3000 -e NODE_ENV=production mangrove/api:1.8.0Containers
See running containersdocker ps (all, including stopped: -a)Containers
Follow a container's logsdocker logs -f --tail 100 <name>Containers
Open a shell inside itdocker exec -it <name> shContainers
See what uses disk, then clean updocker system df, then docker system prune (asks first)Containers
See image layers and sizesdocker history mangrove/api:1.8.0Containers
Start the local stack in the backgrounddocker compose up -dContainers
Follow one service's logsdocker compose logs -f apiContainers
Stop the stack (and wipe its volumes)docker compose down (add -v to delete volumes)Containers

A Dockerfile that follows the rules

Multi-stage build, dependency files copied before source so the install layer is cached, a slim base, and a non-root user. Add a .dockerignore with node_modules, .git and .env (Containers).

Dockerfile
FROM node:20-slim AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build && npm prune --omit=dev

FROM node:20-slim
WORKDIR /app
ENV NODE_ENV=production
COPY --from=build /app/node_modules ./node_modules
COPY --from=build /app/dist ./dist
USER node
EXPOSE 3000
CMD ["node", "dist/main.js"]

kubectl: "I want to..."

Add -n <namespace> to any command, or set it once with kubectl config set-context --current --namespace=api. Check which cluster you are on before you change anything (Kubernetes).

I want to...CommandLesson
See which cluster I am talking tokubectl config current-context; switch with config use-contextKubernetes
List pods with their node and IPkubectl get pods -o wideKubernetes
Find out why a pod is not runningkubectl describe pod <pod> (read Events at the bottom)Kubernetes
Read logs, or the logs of the last crashkubectl logs -f deploy/api, kubectl logs <pod> --previousKubernetes
Recent events, newest lastkubectl get events --sort-by=.metadata.creationTimestampKubernetes
Open a shell in a podkubectl exec -it <pod> -- shKubernetes
Reach a service from my laptopkubectl port-forward svc/api 8080:80Kubernetes
Preview, then apply a changekubectl diff -f k8s/, then kubectl apply -f k8s/Kubernetes
Watch a rollout, or undo itkubectl rollout status deploy/api, kubectl rollout undo deploy/apiSafe releases
Scale by handkubectl scale deploy/api --replicas=6Kubernetes
See CPU and memory usekubectl top pods (needs metrics-server)Kubernetes
Check what I am allowed to dokubectl auth can-i delete podsKubernetes
Pod showsUsually meansCheck
PendingNo node has room, or a volume cannot be attacheddescribe pod events; resource requests
ImagePullBackOffWrong image name or tag, or no pull permissionThe image reference and the registry secret
CrashLoopBackOffThe app starts and exits again and againlogs --previous; missing env var or config
OOMKilled (exit code 137)The container used more memory than its limitMemory limit vs real use; a leak
Running but not ReadyThe readiness probe fails, so it gets no trafficThe probe path and timing; the app's dependencies

Terraform workflow

Plan in the pull request, apply only from CI, state stored remotely with locking. OpenTofu takes the same commands with tofu in place of terraform (IaC).

StepCommandWhy
Download providers, connect to stateterraform initOnce per checkout, and after adding a provider
Format and validateterraform fmt -check, terraform validateCheap checks for CI
Preview the change and save itterraform plan -out=tfplanReviewers read exactly what will happen
Apply the reviewed planterraform apply tfplanApplies only what was reviewed, nothing newer
See what state tracksterraform state list, terraform state show <address>Investigate without editing state by hand
Adopt something built by handAn import block (Terraform 1.5 and later) or terraform importBring drift under code
Read outputsterraform outputFeed values to deploy scripts

Release strategies

Separate deploying code from releasing a feature. For Mangrove's tills, which must keep selling offline, a new POS build goes to a few pilot stores first and waits a full trading day before the next ring (Safe releases, Shipping).

StrategyHowGood forRollback
RollingReplace instances a few at a timeStateless APIs, the default in KubernetesRoll back the same way; old and new run side by side for a while
Blue-greenBring up the full new copy, switch traffic at onceFast, clean switch; easy to test the new copy firstSwitch back; costs double capacity during the switch
CanarySend 1%, then 10%, then 50% of traffic to the new version while watching metricsHigh-traffic services where real users find what tests missSend the canary's traffic back; automate on error rate
RingsPilot group, then a wider group, then everyoneDevices and stores: 5 pilot stores, then 50, then all 320Stop the ring; roll back the pilot devices
Feature flagCode ships turned off; turn on by user, store or percentageBusiness-timed launches, gradual exposure, kill switchesTurn the flag off, no deploy needed; remove old flags
Expand and contractAdd new schema, move code, remove old schema laterDatabase changes that must not stop the appThe old path still works until the contract step

SLO and error-budget numbers

An error budget is 100% minus the SLO. Burn rate is how fast you spend it: a burn rate of 1 uses exactly the whole budget over the SLO window (SLOs and budgets, Burn-rate alerts).

SLOBudget per 30 daysPer yearLesson
99%7 h 12 minabout 3.65 daysSLOs and budgets
99.5%3 h 36 minabout 1.8 daysSLOs and budgets
99.9%43.2 minabout 8 h 46 minSLOs and budgets
99.95%21.6 minabout 4 h 23 minSLOs and budgets
99.99%4.3 minabout 53 minSLOs and budgets
Alert (30-day SLO)Burn rateLong and short windowBudget spent when it fires
Page14.41 hour, with a 5-minute check2%
Page66 hours, with a 30-minute check5%
Ticket13 days, with a 6-hour check10%

What to watch and measure

  • Four golden signals per service: latency (p50, p95, p99), traffic, errors, saturation (Observability).
  • RED for request-driven services (rate, errors, duration) and USE for resources (utilisation, saturation, errors) (OpenTelemetry).
  • DORA metrics for delivery: deployment frequency, lead time for changes, change failure rate, and time to restore after a failed change (Safe releases).
  • Capacity: requests in progress = rate × time per request; plan for Mangrove's festival peak at about 4 times a normal day (Capacity and game days, Estimation).

Incident roles and severities

One person commands; they do not also debug. Roles can be one person each in a small team, but name them out loud (Incident command).

RoleDoesDoes not
Incident commanderSets priorities, assigns work, decides on rollback, calls for helpType commands or chase one theory alone
Operations leadRuns the technical investigation and changes, with respondersMake changes without saying them in the channel
Communications leadPosts status to stakeholders and customers on a fixed schedulePromise a fix time nobody gave
ScribeKeeps the timeline: what was seen, decided and changed, with timesRely on memory after the fact
SeverityMangrove exampleResponse
SEV1App checkout down, or tills in many stores cannot take card paymentsPage now, all hands, status updates every 30 minutes
SEV2Delivery tracking broken; one city's stores sync sales hours latePage the owning team, updates every hour
SEV3Back-office report slow; a minor feature fails with a workaroundTicket, fix in working hours

Status update template

Output
[SEV1] App checkout failing for about 40% of orders. Update 3, 19:45 UTC.
Impact: customers in all cities see "payment failed"; stores are not affected.
What we know: started 19:05 after release 1.8.0 of the payments API.
What we are doing: rolling back to 1.7.4, about 10 minutes.
Next update: 20:15 UTC or sooner if this changes.
Incident lead: Nadia

Postmortem template

Blameless: ask how the system let a reasonable person make this mistake, not who made it. Publish within a week (Postmortems).

Markdown
# Postmortem: checkout failures after payments API 1.8.0

Date: 2026-10-14   Severity: SEV1   Duration: 19:05 to 19:58 UTC (53 min)
Authors: Nadia, Ken      Status: actions in progress

## Summary
Two or three sentences a manager can forward.

## Impact
Orders failed, customers affected, revenue at risk, error budget used.

## Timeline (UTC)
19:05 deploy finished   19:09 burn-rate page   19:21 rollback decided ...

## Root cause and contributing factors
What happened and why the safeguards did not stop it.

## What went well / what was hard

## Actions
| Action | Type (prevent / detect / mitigate) | Owner | Due |

Git on release day

The everyday commands are in the Developer cheat sheet. These are the ones for releases and incidents (Branches and releases, Code detective).

I want to...CommandLesson
Tag a release and push the taggit tag -a v1.8.0 -m "Release 1.8.0", git push origin v1.8.0Branches and releases
See what changed between two releasesgit log --oneline v1.7.4..v1.8.0, git diff --stat v1.7.4 v1.8.0Branches and releases
Find which release a build came fromgit describe --tagsBranches and releases
Undo a merged pull request safelygit revert -m 1 <merge-sha> (a new commit; history stays)Undo
Copy a hotfix onto a release branchgit cherry-pick -x <sha> (-x records where it came from)Branches and releases
Find the commit that broke itgit bisect start, git bisect bad, git bisect good v1.7.4, then git bisect run npm test; finish with git bisect resetCode detective
Clone fast in CIgit clone --depth 1 <url>Actions CI

Linux on call

The first commands to run on a sick host. journalctl and systemctl need a systemd distribution; the TechOps cheat sheet has more (Linux CLI).

I want to know...CommandLesson
How busy the box has beenuptime (load averages for 1, 5, 15 minutes)Linux CLI
Memory and diskfree -h, df -h, du -sh /var/log/* | sort -hLinux CLI
What uses the CPU or memorytop (or htop), ps aux --sort=-%mem | headLinux CLI
Whether a service is up, and its logssystemctl status api, journalctl -u api --since "15 min ago" -fLinux CLI
What is listening on which portsudo ss -tlnpLinux CLI
Whether the kernel killed something for memorysudo dmesg -T | grep -i oomLinux CLI
Status code and time of an endpointcurl -sS -o /dev/null -w '%{http_code} %{time_total}\n' https://api.example.test/healthLinux CLI

Glossary

Plain definitions of the DevOps and SRE terms used across the Engineering Roles line.

Artifact
The built output of CI, such as a container image, built once and promoted unchanged from staging to production. Trusted CI
Blue-green deployment
Running the new version beside the old one and switching all traffic at once. Safe releases
Burn rate
How fast the error budget is being spent compared with spending it evenly over the window. Burn-rate alerts
Canary
A new version given a small share of real traffic first, watched before it gets more. Safe releases
CI/CD
Continuous integration (merge small changes often, tested automatically) and continuous delivery or deployment (every good build can go, or goes, to production). Trusted CI
Container image
A packaged app with everything it needs to run, built from a Dockerfile in layers. Containers
Deployment (Kubernetes)
The object that keeps a set number of pod copies running and rolls out new versions. Kubernetes
Drift
Real infrastructure that no longer matches its code because someone changed it by hand. IaC
Error budget
The failure an SLO allows, such as 43 minutes a month at 99.9%, which the team can spend on change. SLOs and budgets
Feature flag
A switch that turns a feature on or off at run time without a deploy. Safe releases
Game day
A planned exercise that breaks something on purpose to test systems and people. Capacity and game days
Infrastructure as code (IaC)
Servers, networks and permissions described in files, reviewed and applied like code. IaC
Liveness and readiness probes
Kubernetes checks: liveness restarts a stuck container, readiness keeps traffic away until it is ready. Kubernetes
MTTR
Mean time to restore: the average time from an incident starting to service being back. Incident command
OIDC (in CI)
CI proves its identity to the cloud with a short-lived token, so no long-lived keys are stored. Supply chain
OpenTelemetry
The open standard and libraries for producing traces, metrics and logs. OpenTelemetry
Pod
The smallest thing Kubernetes runs: one or more containers sharing a network address. Kubernetes
Postmortem
A written, blameless review of an incident with actions that have owners and dates. Postmortems
Rollback
Returning to the last good version; the first move when a deploy causes an incident. Safe releases
Runbook
Step-by-step instructions for one alert or task, written so anyone on call can follow them. Incident command
SLI, SLO, SLA
What you measure, the target you set, and the promise with penalties you sign. SLOs and budgets
Terraform state
Terraform's record of which real resources match which code; kept remote, locked and never edited by hand. IaC
Toil
Manual, repetitive work that grows with the system and could be automated. Toil

Courses on Kolom

  • Site Reliability Engineering and The Site Reliability Workbook from Google, free at sre.google/books. The workbook's alerting chapter is the source of the burn-rate table.
  • Accelerate by Nicole Forsgren, Jez Humble and Gene Kim. The research behind the DORA metrics.
  • The DevOps Handbook by Gene Kim, Jez Humble, Patrick Debois and John Willis. How flow, feedback and learning fit together.
  • GitHub Actions documentation at docs.github.com/actions, including the security hardening guide.
  • kubectl reference at kubernetes.io, with the official quick reference of common commands.
  • Terraform documentation at developer.hashicorp.com.