Free AI template
Eval set for a chatbot
Replies a bot really gave, labelled good or bad, and checks that agree with every label, exportable as CSV.
It opens with this template loaded and runs in your browser. You don't need an account to use it; a free account saves your work and exports it.
The job it does
Before you change a prompt or a model: a set of real replies and checks, so you can tell in seconds whether the change helped. It comes with its own checks, so after you change it you can see at a glance that it still does the job.
How to use it with your own data
Import your own cases as a CSV (name, input, output, expect, checks) or paste replies from the chatbot you use.
What's inside
This is the template exactly as the AI Lab opens it, read-only. Open it to run it, change it and export it.
- 6 labelled replies
- 3 checks on every reply
Replies a store-hours bot really gave, each labelled by a person. Good replies give an exact time and are short; bad ones guess or ramble. The checks agree with every label, so any new prompt can be scored in seconds: paste its replies and Run.
- My checks agree with every good / bad label
- Eval pass rate is at least 60% (6+ scored cases)
- Matches /\b\d{1,2}:\d{2}\b/
- At most 40 words
- Does not contain "always open"
| Case | Question | Reply the bot gave | Label |
|---|---|---|---|
| Opening time | When does the Riverside store open on Sunday? | The Riverside store opens at 8:00 on Sunday and closes at 21:00. | good |
| Holiday hours | Are you open on the festival day? | On the festival day the Riverside store is open from 10:00 to 16:00. | good |
| Late closing | How late is Harbour Road open on Friday? | Harbour Road stays open until 23:00 on Fridays. | good |
| Pharmacy counter | When is the pharmacy counter open? | The pharmacy counter is open from 9:00 to 18:00, Monday to Saturday. | good |
| Guess with no time | Is Lake View open now? | Yes, Lake View is always open, come any time! | bad |
| Rambling reply | When do you open tomorrow? | Thanks for asking! Our stores have different hours depending on the day of the week, the season, local rules and the festival calendar, and some counters insid… | bad |
The lesson behind it
Learn why it works, with a podcast and a quiz for each lesson.
More AI templates
Structured output with a schema
An extraction prompt, the JSON Schema its replies must follow, and sample replies that should and should not pass.
RAG chunking for a help page
A help page cut into chunks under a token budget, with questions that must find their answer in the top results.
Agent flow with guardrails
An order-status agent with a step limit, input and output guardrails and read-only tools, that passes the validator.