Engineering Roles line, station 4 of 8
Site Reliability Engineering
Run reliable services with SLOs and error budgets, OpenTelemetry observability, burn-rate alerts, calm incident command, blameless postmortems, capacity planning and less toil.
- 7lessons
- 4.5 hto read and do
- Advancedlevel
- 7podcast episodes
- 7checked exercises
Tools you'll use in this course
The lessons open these with a template ready. They work for your own job too, no course needed.
About this course
Site reliability engineering treats operations as a software problem. You decide how reliable a service must be, measure it from the user's side, spend the remaining error budget on change, and use engineering to remove the repetitive work that grows with every new service. This course teaches the practices SRE teams use every day, at the depth you need to own a service on call.
You own reliability for the checkout of Mangrove Market's app, a fictional grocery chain whose app takes about 60,000 orders a day and four times that during the festival rush. Last festival, checkout failed for 40 minutes at the evening peak. This year you define SLIs and SLOs for placing and paying for an order, write the error budget policy, instrument the services with OpenTelemetry, replace noisy threshold alerts with multi-window burn-rate alerts, lead an incident as incident commander, run a blameless postmortem whose actions get done, plan capacity for 4× traffic with load tests and a game day, and measure and automate away the toil that eats the team's week.
Lessons have diagrams, quick checks, a Build it section with real configuration (OpenTelemetry, Prometheus-style rules, load tests, templates), auto-checked SQL and JavaScript exercises in your browser, flashcards and a podcast episode. The capstone gets checkout ready for the festival rush, with an SLO document, alert rules, a capacity plan, a game-day report and an incident playbook.
What you will be able to do
- Define SLIs, SLOs and error budgets that match what users feel, and write an error budget policy the business agrees to
- Instrument services with OpenTelemetry metrics, logs and traces that answer new questions during an outage
- Alert on error-budget burn with multi-window rules that page rarely and accurately, and keep on-call healthy
- Lead incidents calmly as incident commander and run blameless postmortems whose actions get done
- Plan capacity for peaks with load tests, test resilience with game days, and measure and remove toil with automation
Syllabus
7 stops, about 4.5 h in total. Each stop fills in when you complete the lesson.
How reliable is reliable enough?
Last festival, Mangrove's checkout failed for 40 minutes at the evening peak, and nobody could say how much failure was acceptable or what should change afterwards. This lesson turns "checkout must be reliable" into numbers you can measure, a budget you can spend, and a written policy that decides what happens when the budget runs out.
SLOsError budgetsObservability
Observability with OpenTelemetry
During last year's festival outage, Mangrove's dashboards said checkout was failing but could not say why, and it took far too long to find the payment retries behind it. This lesson builds the telemetry that answers new questions during an outage, with OpenTelemetry from the app to the Collector, at a cost you can defend.
ObservabilityOpenTelemetry
Alerting that does not burn people out
Last festival, Mangrove's on-call phone buzzed dozens of times a week for CPU and memory warnings, and the alert that mattered came late. This lesson replaces those alerts with a few that watch how fast checkout's error budget is burning, routes them to the right person, and keeps the people on call healthy enough to answer.
SLOsError budgetsObservabilityIncident management
Leading an incident
When checkout breaks at the festival peak, about 450 orders fail every minute, and the first ten minutes decide how long the outage lasts. This lesson teaches you to run that response as incident commander: declare early, give everyone one job, keep people informed on a fixed rhythm, and stop the damage before anyone looks for the root cause.
Incident managementSLOsPostmortems
Postmortems and learning
An incident ends when checkout works again. The learning starts after that. This lesson shows how to turn last festival's 40-minute checkout outage into a blameless postmortem: impact in numbers, a timeline built from evidence, several contributing factors instead of one culprit, and action items that are actually finished before the next festival.
PostmortemsIncident managementError budgets
Capacity, resilience and game days
The festival rush brings about four times Mangrove's normal orders, squeezed into three evening hours, and last year checkout gave way for 40 minutes. This lesson turns history into a peak forecast, finds the first thing that breaks, sizes every service to survive losing a zone, and proves the plan with load tests and a game day before the customers arrive.
Capacity planningChaos engineeringSLOs
Toil and automation
Every week, Mangrove's on-call engineers reset store price caches, push stuck payments through by hand and grow database disks, and every new store and every festival makes that pile bigger. This lesson shows you how to measure that work, decide which parts to automate first, and build automation and self-service that cannot make an outage worse.
Toil reductionAutomationIncident management
Capstone: Get checkout ready for the festival rush
A realistic project with a rubric and a model answer.
Final exam and certificate
10 questions drawn at random, 70% to pass.
Capstone project
Get checkout ready for the festival rush
Last festival, the Mangrove app's checkout failed for 40 minutes at the evening peak: immediate retries against a slow payment provider exhausted the order service's database connection pool, the first page came nine minutes after customers started failing, and twenty minutes went on theories before anyone looked at the retries. This year's festival is six weeks away and orders will peak at about four times normal. Prepare checkout for it. Use the system as described in the lessons (API gateway, order, payment and stock services, PostgreSQL, Redis, a job queue); you may build a small stand-in service to instrument and load test, or work from the numbers given in the lessons.
You will hand in
- An SLO document for placing and paying for an order: SLIs with exact good and valid events, targets, windows, the error budget and the error budget policy
- Dashboards and burn-rate alert rules (fast and slow burn, page and ticket) with a runbook link for each page
- A capacity plan for 4× peak from load-test results, with headroom, the first bottleneck and what will be pre-scaled
- A game-day plan and report: two failure scenarios, the steady-state hypothesis, abort conditions and what was found
- An incident playbook: severities, roles, the communication cadence and templates, and the postmortem template the team will use
Check what you know
Already know some of this? Answer 7 quick questions, one from each lesson, and we will mark the lessons you can skim. It takes about 4 minutes and nothing is graded.
Questions
What should I take first?
Running Software in Production and DevOps: CI/CD, Containers and Infrastructure as Code. Scaling Building Blocks helps for the capacity lesson, and the TechOps lesson on incidents, problems and changes is a gentle start for the incident lessons.
Is SRE different from DevOps?
They overlap. DevOps is a broad way of working that joins development and operations; SRE is a specific practice of it with SLOs, error budgets, toil limits and blameless postmortems. The first lesson explains how teams combine them.
Do I need a big system to practise on?
No. The exercises run in your browser, and the Build it sections work on a small service on your laptop. The capstone can use a stand-in service or the numbers given in the lessons.
How long does the course take?
About 5 to 6 hours for the seven lessons with their checkpoints and practice tasks, plus roughly 70 to 90 minutes of podcast episodes. Allow 8 to 12 hours for the capstone.
Is it really free?
Yes. Every lesson, podcast episode, exercise, quiz, final exam and certificate is free. The labs and their templates for real work are free too. Plus and Premium add every lab challenge, weekly extras and a mentor on top, and they never lock the tools or the courses.
How does the final exam work?
You get 15 questions drawn at random from a larger bank. You need 70% to pass. If you do not pass, you can try again after 24 hours. The exam is graded on the server and you never see the answer key.