AI Engineering line, station 4 of 6
Building AI Agents
Design, build and evaluate AI agents that use tools and real data safely, from a first prototype to production.
- 6lessons
- 4 hto read and do
- Intermediatelevel
- 6podcast episodes
- 6checked exercises
Tools you'll use in this course
The lessons open these with a template ready. They work for your own job too, no course needed.
About this course
An agent is a model in a loop: it decides the next step, calls a tool, reads the result and decides again. That loop can do real work, and it can also loop forever, act on the wrong store or follow an instruction hidden in a document. This course teaches you to build agents that do useful work and stay inside the lines, and to prove it before anyone relies on them.
You build an operations assistant for the store managers of Mangrove Market, the fictional grocery chain used across Kolom. It answers questions from Mangrove's store procedures with the version and step it relied on, looks up stock, deliveries and goods received notes, creates follow-up tasks, and asks the manager for approval before anything risky such as a stock transfer. You design its tools and a policy layer that decides allow, approve or deny, keep its plan and state in the database so it survives a restart or a two-hour wait for approval, evaluate it on golden tasks in a sandbox store, and take it to a staged pilot in five stores with budgets, traces, an audit trail and a kill switch.
The course builds on LLM Apps: Prompts, RAG and Evals and links to it rather than repeating it. Every lesson has runnable code, an auto-checked JavaScript exercise in your browser and a podcast episode.
What you will be able to do
- Decide when a fixed workflow beats an agent, and build an agent loop with step, budget and stop guards
- Design job-shaped tools with scopes from the session, a policy layer and approval steps for risky actions
- Build retrieval over versioned procedures that answers with the version and step it used, or says it does not know
- Keep plans and state in the database so a task survives restarts and waits for approval without repeating actions
- Evaluate agents with golden tasks in a sandbox, outcome and trajectory checks, and reliability over repeated runs
- Ship an agent in a staged pilot with cost and latency budgets, traces, an audit trail, gates and a kill switch
Syllabus
6 stops, about 4 h in total. Each stop fills in when you complete the lesson.
What makes an agent
An agent is a model in a loop that picks its own next step. This lesson shows what that loop is in code, what it costs, how it fails, and when a fixed workflow does the same job cheaper and more safely. You will build a small agent loop with guards, and the same task as a workflow, for the operations assistant that Mangrove Market's store managers will use.
AI agentsTool callingLLM APIsGuardrails
Designing tools and permissions
An agent can only do what its tools let it do, so the tool set is where an agent becomes useful or dangerous. In this lesson you design the tools for Mangrove's operations assistant, make every call run as the signed-in person, put a policy layer between the model and the tools, and build an approval flow with idempotency keys and an audit log.
Tool callingGuardrailsAI agents
Retrieval over procedures
Ama is receiving a chilled delivery at store 112 and the probe reads 7 °C. She asks the operations assistant whether she can accept the yoghurt. The answer must come from the procedure in effect today for her store, quote the safety step word for word, and say exactly where it came from. This lesson shows what changes when an agent retrieves from procedures: versions with effective dates, numbered steps, memos that override, and documents only managers may read.
Retrieval (RAG)AI agentsTool callingGuardrails
Memory, state and multi-step plans
A model remembers nothing between calls, yet Ama wants an assistant that can work on one problem for a whole morning, wait two hours for her approval and survive a deploy in the middle. This lesson shows you where an agent's memory really lives, how to build each step's context so the model keeps the thread, and how to store plans in Postgres so a task can stop and resume safely.
AI agentsLLM APIsGuardrailsContext engineeringDurable agent state
Evaluating agents
An agent takes many steps, calls tools that change things, and can reach the same goal by different routes. A list of questions with expected answers can't tell you whether it is safe. This lesson shows how to test an agent with golden tasks in a sandbox store, check both where it ended up and how it got there, run every task several times, and turn the results into a gate for Ama's five-store pilot.
EvalsAI agentsGuardrailsTool calling
Shipping an agent safely
Your store-manager operations assistant passes its evals. This lesson takes it to a pilot in five Mangrove stores: what one task really costs and how long it takes, the budgets that stop it in time, the traces and audit trail that tell you what happened, the switches that turn it off in seconds, and the staged plan that earns each next step with numbers.
AI agentsGuardrailsLLM APIsEvals
Capstone: Ship an operations assistant
A realistic project with a rubric and a model answer.
Final exam and certificate
10 questions drawn at random, 70% to pass.
Capstone project
Ship an operations assistant
Mangrove's store managers spend part of every morning chasing short deliveries, searching procedures and raising follow-ups by phone. Build an assistant that answers their questions from a set of versioned store procedures with citations, looks up store data through at least two read tools, creates follow-up tasks, and raises at least one risky action (such as a stock transfer request) only with the manager's approval. Use a sandbox with sample stores, stock, deliveries and procedures that you create. Prove with an eval set of at least 30 golden tasks, run several times each, that it is safe to pilot in five stores, and write the pilot plan with budgets, gates and stop rules.
You will hand in
- A repository with the working agent: at least two read tools, one write tool behind approval, a policy layer, durable task state and a README
- An eval set of at least 30 golden tasks in a sandbox, with outcome and trajectory checks and results per category over repeated runs
- A pilot plan for five stores: stages, entry and exit gates, stop rules, budgets per task and per day, and who reviews what
- A cost and latency estimate per task from measured traces
Check what you know
Already know some of this? Answer 6 quick questions, one from each lesson, and we will mark the lessons you can skim. It takes about 3 minutes and nothing is graded.
Questions
What should I take first?
LLM Apps: Prompts, RAG and Evals. This course links to it for API basics, tool calling, retrieval and evals, and spends its time on what is new for agents. You also need comfortable JavaScript with async functions.
Which model or framework does it use?
It stays vendor-neutral and framework-free. You build the loop, the policy layer and the eval runner in plain TypeScript or JavaScript around a small model wrapper, so you understand what any agent framework does for you and can use one later with open eyes.
Do I need a paid API key?
Not for the lessons: the exercises run in your browser with a fake model that returns scripted steps. For the capstone you need a model to call; free or cheap tiers, or a small open model on your own computer, are enough for a sandbox.
How is this different from AI Agents in Operations?
That lesson, in AI Product Management, is the product view: which tasks to automate, autonomy levels, approvals people read and return on investment. This course is the builder's view: the code, the permissions, the state, the evals and the pilot.
Is there an exam and a certificate?
Yes, and both are free. Each attempt mixes scenario questions with hands-on tasks. You pass with 70% on the questions and an average of 70% on the tasks.
Is it really free?
Yes. Every lesson, podcast episode, exercise, quiz, final exam and certificate is free. The labs and their templates for real work are free too. Plus and Premium add every lab challenge, weekly extras and a mentor on top, and they never lock the tools or the courses.
How long does it take?
Each lesson shows an honest time for reading and doing. Most people finish a course in two to four weeks at three or four hours a week.