AI Engineering line, station 3 of 6
AI Systems for Builders
Learn how AI features really work, how the biggest models are served, and how to build and code with AI safely.
- 4lessons
- 2.5 hto read and do
- Advancedlevel
- 4podcast episodes
- 5checked exercises
Tools you'll use in this course
The lessons open these with a template ready. They work for your own job too, no course needed.
About this course
A language model is a slow, costly and sometimes wrong dependency that happens to be very clever. This course treats it that way. You design AI features with the same care you give any other system: the right data in, permissions enforced in code, quality measured with evals, cost estimated before launch, and a fallback for the day the model API is down.
The four lessons go in a deliberate order. First, how an AI feature works inside your own product: tokens, retrieval-augmented generation (RAG), vector search in Postgres, tool calling and evals. Next, how the giants serve their models: what OpenAI has published about running ChatGPT for hundreds of millions of people, and what Anthropic's documentation and postmortems show about serving Claude, from GPU memory and batching to prompt caching, rate limits and errors. Last, working with AI coding assistants: briefing them well, reviewing what they write, and letting CI enforce the rules.
Every lesson is set at Mangrove Market, the fictional grocery chain used across Kolom, and every practice task runs in your browser. This is the first course of the AI & Agents track. When you finish, the coming Building AI Agents course goes deeper into models that use tools over many steps.
What you will be able to do
- Explain tokens, context windows, streaming and non-determinism, and design around each one
- Design a RAG feature with permissions in the retrieval query, a minimum similarity, citations and an eval set that runs in CI
- Explain how large models are served (prefill, decode, KV cache, batching) and why GPU capacity shapes rate limits and queues
- Build a client for an LLM API that streams, retries only the right errors, respects a time budget and keeps prompts cache-friendly
- Work with AI coding assistants using an instruction file, test-first prompts, a review checklist and CI rules
Syllabus
4 stops, about 2.5 h in total. Each stop fills in when you complete the lesson.
Designing AI features: LLMs, RAG and vector search
Welcome to AI Systems for Builders. You need no AI background to start here. A language model is one more slow, costly, sometimes wrong dependency, like a payment API that is very clever. Building a useful AI feature is mostly normal system design around it: finding the right data, checking what goes in and what comes out, measuring quality, and controlling cost. In this lesson you design an assistant that answers store staff's questions from the company's how-to guides, and you keep it free to switch AI providers later.
LLM fundamentalsRetrieval-augmented generationVector searchTool callingAI evals
ChatGPT: serving a large language model to hundreds of millions
In October 2025, OpenAI's CEO said more than 800 million people use ChatGPT every week, and its API processes over 6 billion tokens a minute. Most of ChatGPT's internal design is private, but OpenAI has published real pieces: its PostgreSQL setup, its Kubernetes clusters, its outages, and how its engineers think about GPUs. This lesson puts those pieces together with how LLM serving works in general, and then you build a streaming chat proxy you can put in front of any model.
LLM servingStreaming responsesRate limitingCapacity planning
Claude: how Anthropic serves Claude at scale
In 2025 Anthropic said it serves Claude to millions of users on three kinds of chips, from three cloud partners, and to more than 300,000 business customers. Anthropic publishes little about its internal systems, but its API documentation and one detailed postmortem show a lot about how a model service behaves from the outside, and what went wrong inside. This lesson reads those sources carefully, separates them from general knowledge, and ends with a resilient client you can use with any LLM API.
Prompt cachingResilient API clientsLLM servingAI evals
Working with AI coding assistants
AI assistants now write a real share of the code in many teams, from one-line completions to whole pull requests. They are fast and tireless, and they are confidently wrong often enough that the team's habits matter more than the tool. This lesson explains how they work in plain words and gives you the instruction file, prompt, review checklist and CI check that keep them useful and safe.
AI-assisted developmentAI code reviewCI guardrailsPrompt writing
Capstone: Design and prototype a store-staff assistant
A realistic project with a rubric and a model answer.
Final exam and certificate
10 questions drawn at random, 70% to pass.
Capstone project
Design and prototype a store-staff assistant
Mangrove Market wants an assistant in its Store app that answers staff questions from about 40 SOPs, the step-by-step guides for jobs like receiving deliveries, counting stock and handling customer returns. Store managers and cashiers may read different SOPs, and the assistant may look up the status of a goods received note (GRN) for the user's own store. About 320 stores ask roughly 2,000 questions a day, four times that during festival rushes, and many staff work on patchy store Wi-Fi. Write a short design, build a small working prototype of retrieval and the API client in JavaScript, and set up the repository rules so that AI-written code for this feature is reviewed and tested properly.
You will hand in
- A design document of 1 to 2 pages: the ingest and query paths, where permissions are enforced, the tools the model may call, the fallback when the model API is down, and a monthly cost estimate with your assumptions
- A retrieval prototype in JavaScript: 10 to 20 SOP chunks with small made-up vectors, cosine similarity, top-k with a role filter and a minimum similarity, and an eval set of at least 12 cases with its pass rate
- A client wrapper (it can call a fake API) that streams text, retries only retryable errors with backoff and jitter, honours retry-after, never restarts after text is shown, and logs request IDs and cache usage
- An AGENTS.md and a review checklist for the assistant's repository, plus a CI rule that blocks source changes without tests
Check what you know
Already know some of this? Answer 4 quick questions, one from each lesson, and we will mark the lessons you can skim. It takes about 2 minutes and nothing is graded.
Questions
How long does the course take?
About 2.5 hours for the four lessons, including the checkpoints and practice tasks. The podcast episodes add about 50 minutes if you listen to all of them. Allow 2 to 6 hours more for the capstone project.
What do I need to know before I start?
No AI background. You will follow the code best if you can read basic JavaScript or TypeScript and simple SQL. When a lesson uses an idea from another Kolom course, such as caching, rate limiting or retries, it links to that lesson so you can read it when you need it. How the Web Works and JavaScript Deep Dive are good companions if those ideas are new.
Do I need to install anything or pay for an AI API key?
No. Every practice task runs in your browser. The Build-it sections show real NestJS, Postgres and SDK code you can run later on your own machine, and they show how to test against a fake model server, so you can learn the plumbing without an API key.
Is the certificate free, and how does the final exam work?
Yes, the course, the exam and the certificate are free. Each attempt gives you 15 questions picked at random from the course's question bank, and you pass with 70% or more. If you do not pass, you can try again after 24 hours.
Will this still be accurate when the AI tools change?
The design ideas (permissions in retrieval, evals, admission control, safe retries, cache-friendly prompts, test-first briefs) change slowly. Product details such as limits, model names and file names change faster, so the lessons say when they were checked and link to the official documentation in their Sources sections.
Is it really free?
Yes. Every lesson, podcast episode, exercise, quiz, final exam and certificate is free. The labs and their templates for real work are free too. Plus and Premium add every lab challenge, weekly extras and a mentor on top, and they never lock the tools or the courses.