1. Courses
  2. LLM Apps: Prompts, RAG and Evals
  3. Capstone

Capstone project for LLM Apps: Prompts, RAG and Evals

Ship the product-question assistant

Customers ask the Mangrove app about ingredients, allergens, substitutes, delivery slots and returns, and today they wait for the call centre. Build an assistant that answers from Mangrove's catalogue and help pages (use the sample data you create, or your own small catalogue of at least 50 products and 20 help-page sections), returns a structured reply the app can render with cited sources, checks live stock or delivery slots with at least one read-only tool that takes the store and zone from the session, and hands over to a person when it cannot answer safely. Build a golden set of at least 40 questions across question types, including allergen questions with missing data and prompt injections hidden in product text, run it before launch, and estimate cost and latency per 1,000 questions from measured token counts.

What you hand in

  1. A repository with the working assistant: retrieval, the prompt and output schema, one or more tools, and a README that explains how to run it
  2. A golden set of at least 40 questions with results per question type, retrieval recall@k, and the release gates you chose
  3. A cost and latency estimate per 1,000 questions from measured tokens and timings, with one cost lever applied
  4. A one-page safety note: the injection and allergen cases you tested, what failed and what you changed

Check your work

Tick each item when your project meets it. Be strict: this is the rubric the reviewer uses.

  • Every answer cites the ids of sources that were actually retrieved, and the assistant says it does not know when nothing relevant was found
  • Allergen questions are answered only from structured catalogue data; missing data gives 'unknown' and a handover, never a guess
  • Tools are read-only, validate their arguments and take the store, zone and customer from the session, never from the model
  • The golden set has 40+ questions, results are reported per question type, and the release gates are stated with numbers
  • Retrieval is measured on its own (recall@k) so a failure can be traced to retrieval or to generation
  • Prompt injection in user input and in retrieved product text is tested, with the results and the defences added
  • Cost and latency per 1,000 questions come from measured token counts and timings, and at least one lever (prompt caching, routing, trimming) is applied and measured

0 of 7 checked

Model answer

The model answer opens once your capstone passes the review, so you can compare your work with an outline of a strong submission.

Hand in your capstone

Your links are checked automatically, then the work is scored against the rubric above, item by item, with comments. A pass counts toward the course certificate and shows on your profile.

Loading…