1. Courses
  2. AI Engineering
  3. Building AI Agents
  4. Stop 2

40 minIntermediate

Designing tools and permissions

An agent can only do what its tools let it do, so the tool set is where an agent becomes useful or dangerous. In this lesson you design the tools for Mangrove's operations assistant, make every call run as the signed-in person, put a policy layer between the model and the tools, and build an approval flow with idempotency keys and an audit log.

By the end you can

  • Design a small tool set around the user's jobs, with descriptions, parameters, responses and errors a model can use well
  • Sort tools into read, write and irreversible, and run each call with the signed-in person's own short-lived scopes
  • Write a policy check that returns allow, approve or deny from the role, tool, store and amount
  • Build an approval flow: a stored pending action, a card built from a dry run, expiry, and execution with an idempotency key
  • Record every call in an append-only audit log that answers "who did what, and why was it allowed?"

The everyday picture

Think of a new teller at a bank branch. She does not get the vault keys and a login that opens every account. She gets a few forms, one for each job: deposit, withdrawal, balance check. Each form has boxes for exactly what that job needs, and the bank's system stamps her branch and her name on it. She cannot type in someone else's.

She can pay out up to a set amount on her own. Above it, a supervisor walks over, reads the slip with the exact amount on it, and turns a key. If the customer wanders off for an hour, the slip is void and they start again. Every slip carries a number, so if the printer jams and she presses the button twice, the customer is paid once. At the end of the day, every slip is in the journal with her name on it.

An agent needs the same things: job-shaped tools, an identity it cannot change, limits that a supervisor's key unlocks, slips that expire, numbered requests that cannot run twice, and a journal.

Tools shaped around jobs, not endpoints

In What makes an agent you built the loop: the model asks for a tool, your code runs it, the result goes back. How tool calling and MCP work is covered in Tools and the Model Context Protocol. This lesson is about what goes inside that loop: which tools exist, what they accept and return, and who decides whether a call runs.

Start from the jobs, not from your API. Ama, the store manager who pilots the assistant, asks things like "where is my bread delivery?", "how many 1-litre UHT milk cases do I have?", "what does the procedure say about a short delivery?" and "make sure someone recounts the rice tonight". Mangrove's platform already has REST endpoints for purchase orders, advance shipping notices, DC dispatch, truck arrival times and goods received notes (GRNs). If you wrap each endpoint as a tool, the model needs five rounds to answer "where is my delivery?", has to join ids across five responses itself, and reads thousands of tokens of fields it does not need. One get_deliveries tool that does the joining in your code answers in one round.

Five endpoint tools One job-shaped tool Model get_po get_asn get_dispatch get_truck_eta get_grn 5 rounds ~15,000 tokens model joins the ids Model 1 call summary get_deliveries store 112, today fans out Your server orders, ASN, dispatch, ETA, GRN APIs joined in code 1 round ~400 tokens ids joined in code
Wrapping each endpoint as a tool makes the model do your integration work, one slow and costly round at a time. A job-shaped tool moves the fan-out and the joining into your server and returns only what answers the question. Token counts are rough, for illustration.

Cost grows faster than it looks. In most model APIs the whole conversation is sent again on every round of the loop, so a 3,000-token response is paid for again on each later round (prompt caching lowers this with some providers). At an illustrative $3 per million input tokens (check your provider's current price list), the difference per question is small, but across 320 stores asking dozens of questions a day it adds up, and long contexts also make the model slower and less accurate.

Then design each tool for a reader that cannot ask you questions:

  • Names and descriptions say when to use it and when not to. "Status of deliveries to one store. Use for 'where is my delivery'. Do not use for stock on the shelf (use get_stock)." The model chooses tools by reading these words, so write them like a note to a new colleague, and fix them when your evals show the wrong tool being picked.
  • Small parameter sets with enums and ids. day: "yesterday" | "today" | "tomorrow" beats a free-text date. articleId beats an article name, because "milk" matches 40 of Mangrove's 25,000 articles. Set additionalProperties: false so invented fields fail validation.
  • Short, relevant, paginated responses. Return the five fields that answer the job, in words the model can quote ("delayed, now expected Sunday 07:00"), with a default page of 10 rows and a nextCursor. Never return a 40-field record "in case".
  • Errors that tell the model how to fix the call. A bare 400 Bad Request makes the model guess or give up. Say what was wrong and what to do next:
JSON
{
  "error": "ambiguous_article",
  "message": "3 articles match 'milk 1L'. Call get_stock again with one articleId.",
  "candidates": [
    { "articleId": 40211, "name": "UHT milk 1 L, case of 12" },
    { "articleId": 40207, "name": "Fresh milk 1 L" },
    { "articleId": 40388, "name": "Lactose-free milk 1 L" }
  ],
  "retryable": true
}

Quick check

The agent keeps calling get_stock when Ama asks "when does the bread arrive?". What is the best first fix?

Read, write and irreversible tools, run as the user

Sort every tool by what it can change, because that decides how much checking it needs.

  • Read: search_procedures, get_stock, get_deliveries, get_grn. A wrong call wastes a round; nothing changes in the world.
  • Write, easy to undo: create_task. A wrong task is closed in one tap.
  • Write, hard to undo: request_stock_transfer. Once store 118 has packed 40 cases and a van has left, undoing it costs a second trip.
  • Irreversible or customer-facing: change_price. A wrong price hits every till in the store and every app order within minutes, and customers have already paid it.

Next, decide who each call runs as. The rule is simple: the agent acts as the signed-in person, never as itself. Ama can read and act for her own store, 112. The area manager can act for her 12 stores. The assistant gets exactly that, and no more.

In practice, when Ama opens the assistant, your backend issues a for her session: her user id, her store list, and only the the assistant needs, such as procedures:read stock:read deliveries:read tasks:write transfers:request. There is no prices:write scope in it at all. Every tool calls Mangrove's normal APIs with that token, so the stock API applies the same row-level rules it applies when Ama uses the Store app. A shared admin key would turn one confused or manipulated step into access to all 320 stores; the OWASP Top 10 for LLM applications calls this .

One more rule follows: identity comes from the session, not from the model. No tool schema has a userId or role field. Where a store must be named (the area manager picks one of her 12), the gateway fills in the session's default store and checks any store the model passes against the token's list. The model can be wrong, and text it reads (a supplier's note, a procedure someone edited) can try to steer it, as Security essentials and AI features people can trust explain. Nothing the model writes should be able to change who is acting.

Quick check

The area manager asks about store 131, which is not one of her 12. The model calls get_stock with storeId: 131. Where should this be stopped?

A policy layer between the model and the tools

Scopes say what Ama may do in general. A decides, for each call, whether it runs now, waits for a person, or is refused. It sits in your gateway, after schema validation and before any tool runs. Its inputs are facts the model cannot change: the user's role and stores from the session, the tool name, the validated arguments, and the amount computed by your own system.

Write the policy as data, one row per tool, so it can be reviewed like any other config:

ToolKindDecisionWho approvesLimits
search_proceduresReadAllowNobody60 calls per user per hour
get_stock, get_deliveries, get_grnReadAllow, own storesNobodyPages of 10 to 20 rows
create_taskWrite, easy to undoAllow, own storesNobody30 per store per day
request_stock_transferWrite, hard to undoApprove, up to $2,000; deny aboveStore manager up to $500; area manager above5 per store per day; $2,000 per store per day
change_priceCustomer-facingDenyNot available to the agentPrices go through the pricing team
Anything elseDeny

The check runs in a fixed order and stops at the first failure: an unknown tool is denied (deny by default); a tool whose row says deny is denied; a role the row does not list is denied; a store outside the user's list is denied; an amount over the row's hard limit is denied. Only then does the row's own decision apply: allow, or approve.

Two details make this hold up. First, the amount comes from your system, not the model. For a transfer, the gateway runs a that picks the source store and prices the quantity at cost; the model only says which article and how many. Second, change_price is never sent to the model in the tool list, and its row says deny. If a later config change or an injected instruction ever produces that call, the policy still refuses it. This is what OWASP calls complete mediation: the downstream system decides, every time.

Limits per tool sit next to the policy. They do not judge one call; they cap the damage of many. Count calls and money from the audit log: 30 tasks per store per day, 5 transfers and $2,000 of transfers per store per day. When a loop goes wrong at 2 am, these limits stop it at the fifth transfer instead of the five-hundredth. The step and token budget from What makes an agent caps what the model spends; these caps limit what the tools can do. Rate limiting shows how to count efficiently.

Model tool call Gateway schema, ids from session call + value Policy check role, tool, store, value allow runs now approve pending action deny error to model how to fix user's token card Tool API runs as Ama Approval same args, key Audit log every decision
The model only proposes. The gateway validates the call and fills ids from the session, the policy decides, allowed calls run with the user's own token, approvals wait as a pending action, and denials go back as an error that tells the model what to do instead. Every outcome is written to the audit log.

Quick check

Ama's agent calls request_stock_transfer for her own store, and the dry run values it at $2,400. The policy row says approve, with a hard limit of $2,000. What does the policy return?

Approvals that mean something

"Approve" is only a control if the person approving sees the real effect and the system runs exactly what was approved. Five mechanisms make that true.

  1. A stored pending action. When the policy says approve, the gateway writes a row with the exact arguments that will run, a hash of them, the approver role, a status of pending and an expiry time. The model gets back "waiting for approval", not a result.
  2. A card built from the dry run. The card text comes from the preview, never from the model's summary: "Move 40 cases of UHT milk 1 L ($384.00 at cost) from store 118 to store 112, arriving tomorrow by 10:00." Plain words, the real numbers, the source the system chose. If the model wrote "move some milk from nearby", that sentence goes nowhere near the card.
  3. Expiry. Stock, prices and the approver's attention all go stale. A card expires after 30 minutes; after that it cannot be approved, and the agent must propose again with a fresh preview. A card found the next morning is not an approval.
  4. Execute what was approved. On approve, your code loads the stored arguments, checks their hash, and runs those. It never asks the model again, because the model might now produce different arguments.
  5. An idempotency key. Each pending action gets a random key when it is created. The transfer service stores keys with a unique constraint and, for a key it has seen, returns the first result instead of creating a second transfer. This is the same pattern payment APIs use for safe retries.

Why the last one matters: agents retry. The network drops after the transfer service committed but before the reply arrived; the loop sees a timeout and calls again; Ama double-taps Approve on a slow connection. Without a key, each of these creates a second request, and store 118 packs 80 cases. With a key, the retry gets back the same transfer number. Use the same idea for create_task, keyed by the step that created it, so a retried step does not give the night shift two recount tasks.

pending card shown reject approve after 30 min rejected nothing runs approved args hash same expired nothing runs run with key executed one transfer retry with same key
A pending action has one way to run: approved by the right role before it expires, with its stored arguments unchanged. Rejected and expired actions never run. A retry or a double tap reuses the idempotency key, so the transfer service returns the first transfer instead of making a second.

Who approves follows the rules people already use. Transfers up to $500 go to the receiving store's manager, which for Ama's own request means Ama confirms the card. From $500 to $2,000 they go to the area manager, as they do today on paper. The approver's role and stores are checked when they tap Approve, against their own session, not the requester's. How to design cards people actually read, and how to spot rubber-stamping, is covered from the product side in AI agents in operations.

Quick check

Ama opens an approval card 45 minutes after it was created (expiry is 30 minutes) and taps Approve. What should happen?

The audit log

Every decision writes one row: allowed, sent for approval, denied, approved, rejected, expired and executed. Each row says who (user, role, session), which tool, the arguments after the gateway filled the session ids, the decision and its reason (store_scope, over_limit, ok), the pending action id and idempotency key when there is one, and the result. The table is append-only: the app's database role can insert and read, never update or delete.

The log has three jobs. During an incident it answers "why did store 112 get two milk transfers on Friday?" in one query, as Observability describes for any service. It feeds the rate and spend limits, which count today's rows per tool and store. And it is the raw material for Evaluating agents: denied calls show where the model misunderstands its tools, and rejected cards show where its proposals are wrong.

Build it

The tool definitions, the policy table, the gateway, the approval tables and flow, and a test run, in TypeScript and Postgres.

  1. Define the read tools

    Each tool is a name, a description that says when and when not to use it, and a JSON Schema for its arguments. Providers name the schema field differently (for example input_schema or parameters); the content is the same.

    TypeScript
    // tools-read.ts: what the model sees for reading
export const searchProcedures = {
  name: 'search_procedures',
  description: 'Search Mangrove store procedures (SOPs). Use for "how do I" and "what is the rule" ' +
    'questions. Returns up to 5 short passages with their SOP id; cite the id in your answer.',
  inputSchema: { type: 'object', additionalProperties: false, required: ['query'],
    properties: { query: { type: 'string', maxLength: 200 } } },
};

export const getDeliveries = {
  name: 'get_deliveries',
  description: 'Status of supplier and DC deliveries to one store: expected, on_the_way, arrived, ' +
    'received (with GRN id) or delayed, with ETA and cartons. Use for "where is my delivery" or ' +
    '"when does X arrive". Not for stock on the shelf (use get_stock) or for differences on a ' +
    'received delivery (use get_grn).',
  inputSchema: {
    type: 'object', additionalProperties: false,
    properties: {
      storeId: { type: 'integer', description: 'Leave out to use your own store.' },
      day: { type: 'string', enum: ['yesterday', 'today', 'tomorrow'], default: 'today' },
      status: { type: 'string', enum: ['expected', 'on_the_way', 'arrived', 'received', 'delayed'] },
      limit: { type: 'integer', minimum: 1, maximum: 20, default: 10 },
      cursor: { type: 'string', description: 'nextCursor from the previous page.' },
    },
  },
};
// get_stock and get_grn follow the same pattern: storeId optional, articleId or grnId required.

  2. Define the write tools

    Write tools say what they change, that the system chooses the details the model must not, and whether a person approves.

    TypeScript
    // tools-write.ts
export const createTask = {
  name: 'create_task',
  description: 'Create a follow-up task for staff at one store, such as a recount or a shelf check. ' +
    'Use when the user wants a person to do something later. It does not change stock or prices.',
  inputSchema: {
    type: 'object', additionalProperties: false, required: ['kind', 'dueBy'],
    properties: {
      storeId: { type: 'integer', description: 'Leave out to use your own store.' },
      kind: { type: 'string', enum: ['recount', 'shelf_check', 'expiry_check', 'call_supplier', 'pricing_desk'] },
      articleId: { type: 'integer', description: 'From get_stock results. Never guess an id.' },
      dueBy: { type: 'string', enum: ['today', 'tomorrow', 'this_week'] },
      note: { type: 'string', maxLength: 200 },
    },
  },
};

export const requestStockTransfer = {
  name: 'request_stock_transfer',
  description: 'Ask for stock to be moved to a store from a nearby Mangrove store. The system picks ' +
    'the source store and the value. A person always approves a card first. Use only when stock ' +
    'will run out before the next delivery; check get_stock and get_deliveries first.',
  inputSchema: {
    type: 'object', additionalProperties: false, required: ['articleId', 'quantity', 'reason'],
    properties: {
      storeId: { type: 'integer', description: 'Receiving store. Leave out for your own.' },
      articleId: { type: 'integer' },
      quantity: { type: 'integer', minimum: 1, maximum: 500 },
      reason: { type: 'string', enum: ['stockout_risk', 'festival_peak', 'damaged_delivery'] },
    },
  },
};

  3. Write the policy table and limits as data

    Anything not in the table is denied. Approver routing and limits live next to it so one review covers all of them.

    TypeScript
    // policy.ts
export type Decision = 'allow' | 'approve' | 'deny';
export type Role = 'store_manager' | 'area_manager';
type Row = { roles: Role[]; base: Decision; maxCents?: number };

const managers: Role[] = ['store_manager', 'area_manager'];
export const policy: Record<string, Row> = {
  search_procedures:      { roles: managers, base: 'allow' },
  get_stock:              { roles: managers, base: 'allow' },
  get_deliveries:         { roles: managers, base: 'allow' },
  get_grn:                { roles: managers, base: 'allow' },
  create_task:            { roles: managers, base: 'allow' },
  request_stock_transfer: { roles: managers, base: 'approve', maxCents: 200_000 },
  change_price:           { roles: [], base: 'deny' },   // never sent to the model either
};

// Who approves a transfer, from the value the dry run computed
export const approverFor = (valueCents: number): Role =>
  valueCents <= 50_000 ? 'store_manager' : 'area_manager';

// Caps on many calls, counted from today's audit rows
export const limits: Record<string, { perUserPerHour?: number; perStorePerDay?: number; centsPerStorePerDay?: number }> = {
  search_procedures:      { perUserPerHour: 60 },
  create_task:            { perStorePerDay: 30 },
  request_stock_transfer: { perStorePerDay: 5, centsPerStorePerDay: 200_000 },
};

  4. Route every call through one gateway

    It validates, fills ids from the session, prices writes with a dry run, asks the policy, applies limits, and records the outcome. You write authorize itself in Practice.

    TypeScript
    // gateway.ts: the only path from a model's tool call to a real action
export async function handleToolCall(session: Session, raw: { name: string; args: unknown }) {
  const tool = toolsByName[raw.name];                     // the allow-list of tools
  const parsed = tool?.schema.safeParse(raw.args);       // JSON Schema or zod validation
  if (!tool || !parsed?.success) {
    return audit(session, raw, 'denied', 'bad_call', explainBadCall(raw, parsed));
  }
  // Identity and default store come from the session; the schema has no userId field
  const args = { storeId: session.storeIds[0], ...parsed.data };
  // Dry run with the user's own token: picks the source and values it, writes nothing
  const preview = tool.preview ? await tool.preview(session.token, args) : null;
  const call = { tool: raw.name, args, valueCents: preview?.valueCents ?? 0 };

  const { decision, reason } = authorize(session.user, call, policy);
  if (decision === 'deny') return audit(session, call, 'denied', reason, denyMessage(call, reason));
  if (await overLimit(session, call)) {
    return audit(session, call, 'denied', 'rate_limit',
      { error: 'rate_limit', message: 'Daily limit reached for this tool. Tell the user; do not retry.' });
  }
  if (decision === 'approve') {
    const pending = await createPending(session, call, preview);   // step 6
    return audit(session, call, 'needs_approval', reason,
      { status: 'waiting_for_approval', card: pending.card, expiresAt: pending.expires_at });
  }
  const result = await tool.run(session.token, args);             // runs as the user
  return audit(session, call, 'allowed', reason, result);
}

  5. Create the pending action and audit tables

    The pending action holds exactly what will run. The audit log is append-only: grant the app role insert and select, nothing else.

    SQL
    CREATE TABLE pending_action (
  id              uuid PRIMARY KEY DEFAULT gen_random_uuid(),
  requested_by    uuid NOT NULL,              -- the signed-in user the agent acts for
  store_id        int  NOT NULL,
  tool            text NOT NULL,
  args            jsonb NOT NULL,             -- exactly what will run, incl. the chosen source
  args_hash       text NOT NULL,              -- sha256 of the canonical args
  card_text       text NOT NULL,              -- built from the dry run, not from the model
  value_cents     int  NOT NULL DEFAULT 0,
  approver_role   text NOT NULL CHECK (approver_role IN ('store_manager', 'area_manager')),
  status          text NOT NULL DEFAULT 'pending'
    CHECK (status IN ('pending', 'approved', 'rejected', 'expired', 'executed', 'failed')),
  idempotency_key uuid NOT NULL UNIQUE DEFAULT gen_random_uuid(),
  expires_at      timestamptz NOT NULL DEFAULT now() + interval '30 minutes',
  decided_by      uuid,
  decided_at      timestamptz,
  result          jsonb
);

CREATE TABLE audit_log (
  id                bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
  at                timestamptz NOT NULL DEFAULT now(),
  user_id           uuid NOT NULL,
  user_role         text NOT NULL,
  session_id        uuid NOT NULL,
  tool              text NOT NULL,
  args              jsonb NOT NULL,
  event             text NOT NULL CHECK (event IN ('allowed', 'needs_approval', 'denied',
                      'approved', 'rejected', 'expired', 'executed')),
  reason            text NOT NULL,
  pending_action_id uuid REFERENCES pending_action (id),
  result            jsonb
);
CREATE INDEX audit_log_store_day ON audit_log (tool, ((args ->> 'storeId')::int), at);
GRANT SELECT, INSERT ON audit_log TO agent_app;   -- no UPDATE or DELETE grant

  6. Store, approve and execute with an idempotency key

    The approve step is one guarded UPDATE, so a double tap, an expired card or the wrong approver all fail in the database. Execution sends the stored key.

    TypeScript
    // approvals.ts
const canonical = (a: Record<string, unknown>) =>
  JSON.stringify(Object.keys(a).sort().map((k) => [k, a[k]]));   // same args, any order
const hashArgs = (a: Record<string, unknown>) =>
  createHash('sha256').update(canonical(a)).digest('hex');           // from node:crypto

export async function createPending(session: Session, call: Call, p: Preview) {
  const args = { ...call.args, sourceStoreId: p.sourceStoreId };    // run what the card shows
  const card = `Move ${p.quantity} cases of ${p.articleName} (${dollars(p.valueCents)} at cost) ` +
    `from store ${p.sourceStoreId} to store ${call.args.storeId}, arriving ${p.arrives}.`;
  return db.one(`INSERT INTO pending_action (requested_by, store_id, tool, args, args_hash,
      card_text, value_cents, approver_role) VALUES ($1,$2,$3,$4,$5,$6,$7,$8)
      RETURNING id, card_text AS card, expires_at`,
    [session.user.id, call.args.storeId, call.tool, args, hashArgs(args), card,
     call.valueCents, approverFor(call.valueCents)]);
}

export async function approve(actionId: string, approver: User) {
  const action = await db.oneOrNone(`UPDATE pending_action
      SET status = 'approved', decided_by = $2, decided_at = now()
      WHERE id = $1 AND status = 'pending' AND expires_at > now()
        AND approver_role = $3 AND store_id = ANY($4)
      RETURNING *`, [actionId, approver.id, approver.role, approver.storeIds]);
  if (!action) return { ok: false, message: 'Expired, already handled, or not yours to approve.' };
  if (hashArgs(action.args) !== action.args_hash) throw new Error('args changed after the card');
  await auditEvent(action, approver, 'approved');
  const result = await transfers.create(action.args, { idempotencyKey: action.idempotency_key });
  await db.none(`UPDATE pending_action SET status = 'executed', result = $2 WHERE id = $1`,
    [action.id, result]);
  await auditEvent(action, approver, 'executed', result);
  return { ok: true, result };
}

    Inside the transfer service, the key is a unique column, so a retry inserts nothing and reads the first row:

    SQL
    INSERT INTO transfer_request (idempotency_key, from_store, to_store, article_id, quantity, value_cents)
VALUES ($1, $2, $3, $4, $5, $6)
ON CONFLICT (idempotency_key) DO NOTHING
RETURNING id;
-- no row returned: this key was seen before, so return the existing transfer
SELECT id FROM transfer_request WHERE idempotency_key = $1;

  7. Test the gateway and read the audit log

    Run these as Ama (store 112) in a test database, then query the log: a read at her store, a read at store 118, a transfer she approves, and a change_price call you send to the gateway by hand to prove the policy refuses it even though the model never sees that tool.

    SQL
    SELECT to_char(at, 'HH24:MI:SS') AS at, tool, event, reason, args ->> 'storeId' AS store
FROM audit_log WHERE user_id = $1 ORDER BY id;
    Output
       at    |          tool          |     event      |   reason    | store
---------+------------------------+----------------+-------------+-------
16:41:02 | get_stock              | allowed        | ok          | 112
16:41:05 | get_stock              | denied         | store_scope | 118
16:41:09 | request_stock_transfer | needs_approval | ok          | 112
16:41:30 | request_stock_transfer | approved       | ok          | 112
16:41:31 | request_stock_transfer | executed       | ok          | 112
16:42:10 | change_price           | denied         | tool_denied | 112

Bring it to your team

Ask these when someone proposes a new tool or a new agent action.

  • Which user job does this tool serve, and could an existing tool serve it with a better description?
  • Does the description say when not to use it, and what does the model see when a call is wrong?
  • Is this tool read, write or irreversible, and what is its row in the policy table?
  • Whose token does it run with, and which scope does it need that the session does not already have?
  • Does any argument let the model choose who is acting, or set an amount the policy relies on?
  • What happens if this call is retried three times, and which idempotency key stops a duplicate?

Red flags

  • "The agent uses the service account so it can see everything."
  • A tool per REST endpoint, or one tool that runs any SQL or any HTTP request.
  • Limits and store lists written only in the system prompt.
  • Approval cards that show the model's summary instead of the exact effect.
  • An approve button that re-asks the model for the arguments.
  • An audit table the app can update or delete.

Practice

Write the policy check

Write authorize(user, call, policy) for the gateway in step 4. Return { decision, reason }, checking in this order and stopping at the first failure: the tool is not an own key of policy ('deny', 'unknown_tool'); the row's base is 'deny' ('deny', 'tool_denied'); the user's role is not in row.roles ('deny', 'role'); call.args.storeId is not in user.storeIds ('deny', 'store_scope'); the row has maxCents and call.valueCents is missing or above it ('deny', 'over_limit'). Otherwise return the row's base with reason 'ok'.

JavaScript exerciseRuns in your browser with tests
// user:   { id, role: 'store_manager' | 'area_manager', storeIds: [112] }
// call:   { tool, args: { storeId, ... }, valueCents }   (valueCents comes from your dry run)
// policy: { toolName: { roles: [...], base: 'allow' | 'approve' | 'deny', maxCents? } }
// Return { decision, reason }. Reasons: 'unknown_tool', 'tool_denied', 'role',
// 'store_scope', 'over_limit', or 'ok' when the row's base decision applies.
function authorize(user, call, policy) {
  return { decision: policy[call.tool].base, reason: 'ok' };
}
Ctrl + Enter runs
Show a solution
JavaScript
function authorize(user, call, policy) {
  // Deny by default: only tools with their own row exist (not 'constructor' from the prototype)
  if (!Object.prototype.hasOwnProperty.call(policy, call.tool)) {
    return { decision: 'deny', reason: 'unknown_tool' };
  }
  const row = policy[call.tool];
  if (row.base === 'deny') return { decision: 'deny', reason: 'tool_denied' };
  if (!row.roles.includes(user.role)) return { decision: 'deny', reason: 'role' };
  if (!user.storeIds.includes(call.args.storeId)) {
    return { decision: 'deny', reason: 'store_scope' };
  }
  // !(x <= max) is also true when x is undefined or NaN, so a missing value fails closed
  if (row.maxCents !== undefined && !(call.valueCents <= row.maxCents)) {
    return { decision: 'deny', reason: 'over_limit' };
  }
  return { decision: row.base, reason: 'ok' };
}
Pass the exercise to finish this task.

Redesign a proposed tool set

A teammate proposes four tools for the assistant: call_api(method, path, body) "so it can reach anything"; get_store(storeId), which returns the full 40-field store record; update_stock(storeId, articleId, newQuantity) so the agent can "fix" counts it thinks are wrong; and notify(userId, text) to message any member of staff. For each one, say what is wrong and what you would build instead: the tool name, its kind (read, write, irreversible), its parameters, and its row in the policy table.

  • You replace call_api with job-shaped tools and explain that an open-ended tool gives the model every permission the key has
  • You cut get_store down to the fields a job needs, with a short, paginated response
  • You replace update_stock with a recount task (or an approved adjustment), because a stock change based on the model's belief is a hard-to-undo write
  • You limit messaging to people in the user's own stores, with the sender taken from the session and a daily limit
  • No parameter lets the model choose who is acting
Show a model answer
Output
call_api       Remove. Any method on any path = every permission the key has.
               Replace with get_deliveries, get_stock, get_grn (read, allow, own stores).
get_store      Replace with get_store_hours(storeId?) or the fields a real job needs.
               Read, allow, own stores; 5 fields, no staff phone numbers.
update_stock   Replace with create_task(kind: 'recount', articleId, dueBy).
               Write, easy to undo, allow, 30 per store per day. A count changes
               only when a person scans the shelf in the Store app.
notify         Replace with message_store_staff(storeId?, role: enum, text max 300).
               Write, allow, own stores only; sender = session user; 10 per day.
               No userId parameter: the model never picks who is acting.

The pattern behind every change: each tool does one job, takes ids and enums, returns little, and runs as Ama inside her store list. Writes that act on what the model believes about the world become tasks for a person, unless they go through an approval card built from a dry run.

Check yourself

Before a festival weekend, the agent proposes a rice transfer to Ama's store that the dry run values at $1,850. Transfers are approve up to $2,000, with the store manager approving up to $500 and the area manager above. What happens next?

Your policy has a limit on transfer value. Where should the value that the policy checks come from?

On Monday, the area manager asks why store 112 received two milk transfers on Friday evening. Which design answers that fastest and most reliably?

Flashcards

Open my review

Key takeaways

  • Build a few tools around the user's jobs, with descriptions that say when not to use them, enums and ids, short paginated responses and errors that say how to fix the call.
  • The agent acts as the signed-in person with a short-lived, narrow token; identity and default stores come from the session, never from the model.
  • A policy layer in code decides allow, approve or deny from role, tool, store and a system-computed amount, denying by default, with per-tool rate and spend limits.
  • Approvals mean something when the card comes from a dry run, it expires, and execution runs the stored arguments with an idempotency key.
  • Every decision goes into an append-only audit log, which also feeds your limits and your evals.

Sources

Episode transcriptForty cases of milk, approved once · about 12 minutes · tap any line to play from there
  1. NadiaImran, picture Ama's store on a rainy Friday afternoon. The milk will run out tomorrow morning, and the assistant offers to move forty cases from a store nearby. She taps approve, and the Wi-Fi drops.
  2. ImranAnd the assistant retries, because it never heard back. So now the other store packs eighty cases?
  3. NadiaOnly if we designed it badly. Today is about the tools an agent gets, who they run as, who decides whether a call runs, and how an approval stays honest even when the network doesn't.
  4. ImranLast time we built the loop itself. The model asks for a tool, our code runs it, the result goes back. So this is what goes inside that loop.
  5. NadiaRight. Start with a new teller at a bank. She doesn't get the vault keys or a login to every account. She gets a few forms, one per job: deposit, withdrawal, balance check. And the system stamps her branch and her name on each one. She can't type in someone else's.
  6. ImranAnd above a certain amount, a supervisor walks over and turns a key.
  7. NadiaAfter reading the slip with the exact amount. If the customer wanders off for an hour, the slip is void. Every slip has a number, so if the printer jams and she presses twice, the customer is paid once. And everything lands in the day's journal with her name on it.
  8. ImranJob forms, a fixed identity, limits, slips that expire, numbered requests, a journal. That's the whole lesson in one branch.
  9. NadiaIt is. Let's take the forms first. Your platform has REST endpoints for purchase orders, shipping notices, DC dispatch, truck arrival times and goods received notes. What's the tempting move?
  10. ImranWrap each endpoint as a tool. Five endpoints, five tools. It feels thorough.
  11. NadiaIt makes the model do your integration work. The first diagram has two columns. On the left, the model calls five tools, one per round, and joins the ids itself: roughly fifteen thousand tokens. On the right, one tool, get deliveries, and your server does the joining: roughly four hundred tokens.
  12. ImranWhy does the token count grow so fast on the left?
  13. NadiaBecause in most model APIs the whole conversation is sent again on every round. A big response early on is paid for again on each later round. Prompt caching helps with some providers, but long contexts also make the model slower and less accurate.
  14. ImranSo design around Ama's jobs. Where's my delivery, how much milk do I have, what does the procedure say, get someone to recount the rice.
  15. NadiaExactly. Then write each tool for a reader who can't ask you questions. The description says when to use it and when not to. Get stock says it's not for deliveries and points to get deliveries.
  16. ImranParameters?
  17. NadiaFew, with fixed choices and ids. A day that is yesterday, today or tomorrow beats a free-text date. An article id beats the word milk, because milk matches about forty of Mangrove's twenty-five thousand articles. Responses are short and come in pages of ten.
  18. ImranAnd when the model gets it wrong?
  19. NadiaThe error teaches it. Don't send back a bare four hundred, bad request. Say three articles match, here are their ids, call again with one of them. The model fixes the call on the next round instead of guessing or giving up.
  20. ImranOkay. Now who does the tool run as? My first guess would be a service account for the assistant, so it can see everything it might need.
  21. NadiaThat's the trap. The agent acts as the signed-in person, never as itself. Ama can act for store one twelve. The area manager can act for her twelve stores. When Ama opens the assistant, the backend issues a short-lived token with her stores and only the scopes the assistant needs.
  22. ImranSo read stock, read deliveries, write tasks, request transfers. And no scope for prices at all.
  23. NadiaRight. The tools call Mangrove's normal APIs with that token, so the same rules apply as in the Store app. A shared admin key turns one confused step into access to all three hundred and twenty stores. OWASP calls that excessive agency.
  24. ImranWhat if the model just passes a different store id? Or a different user?
  25. NadiaNo tool has a user id or role parameter. Identity comes from the session. Where a store can be named, the gateway fills in the default from the session and checks anything else against the token's list. A line in the system prompt saying only use store one twelve is a request, not a control.
  26. ImranSo who decides whether a call runs at all?
  27. NadiaA policy layer, in code, between the model and the tools. The second diagram shows it top to bottom. The model sends a tool call. The gateway validates it and fills ids from the session. The policy check returns one of three answers: allow, approve or deny.
  28. ImranAnd those three branches go to different places.
  29. NadiaAllow runs now, with Ama's token. Approve becomes a stored pending action and a card. Deny goes back to the model as an error that says how to fix things. And every one of them writes a row to the audit log at the bottom.
  30. ImranWhat does the policy actually look at?
  31. NadiaFacts the model can't change: role and stores, the tool, the validated arguments, and an amount your system computed. It checks in order and stops at the first failure. Unknown tool, denied tool, wrong role, wrong store, over the hard limit: all deny. Then the row's own decision.
  32. ImranFor Mangrove, reads are allowed at your own stores. Creating a task is allowed, up to thirty per store a day. A stock transfer always needs approval, up to two thousand dollars, and is denied above that.
  33. NadiaAnd change price is denied for the agent. It isn't even in the tool list the model sees, and the policy row still says deny. If a config mistake or an injected instruction ever produces that call, it's refused anyway.
  34. ImranYou said the amount comes from your system. Why not let the model say how much the transfer is worth?
  35. NadiaBecause a limit is only as good as the number it checks. The gateway runs a dry run first. It picks the source store and prices the quantity at cost, without changing anything. The model only says which article and how many.
  36. ImranNow approvals. What stops an approval from being a rubber stamp on something nobody understood?
  37. NadiaFive things. The pending action stores the exact arguments and a hash of them. The card is built from the dry run, never from the model's summary. It expires after thirty minutes. Approving runs the stored arguments, without asking the model again. And each action carries an idempotency key.
  38. ImranThe third diagram shows that lifecycle. Pending at the top, then three ways out: rejected, expired after thirty minutes, or approved. Only approved moves on to executed.
  39. NadiaAnd there's a small loop next to executed: retry with the same key. That's our rainy Friday. The transfer service stores keys with a unique constraint. A retry with a key it has seen returns the first transfer instead of creating a second one.
  40. ImranSame for a double tap on approve, I guess. The approve step only works while the status is still pending.
  41. NadiaYes. It's one guarded update in the database: still pending, not expired, the right approver role, and a store the approver covers. A second tap finds nothing to update.
  42. ImranSo let's replay Ama's Friday with all of that in place.
  43. NadiaShe asks if she'll run out of milk. The assistant reads her stock, six cases, about eleven sold a day. It reads her deliveries: the DC truck is delayed by flooding until Sunday morning. It offers a transfer.
  44. ImranShe says yes. The model asks for forty cases, no source store. The dry run picks store one eighteen, two point four kilometres away, and values it at three hundred eighty-four dollars.
  45. NadiaUnder five hundred, so the store manager approves, which here means Ama. The card says move forty cases from store one eighteen to store one twelve, arriving Saturday by ten. She approves after twenty seconds.
  46. ImranThe Wi-Fi drops, the gateway retries with the same key, and store one eighteen still sees one request.
  47. NadiaThen she asks the assistant to mark down tomorrow's bread by thirty percent. There's no price tool, so it says price changes go through the pricing team and creates a task for them. Six audit rows tell the whole story.
  48. ImranWhat do we actually build?
  49. NadiaFirst the tool definitions: a name, a description with when to use and when not to, and a JSON Schema for the arguments. Then the policy table as data, with approver routing and limits next to it. Then one gateway function that every call passes through.
  50. ImranAnd the database side?
  51. NadiaTwo Postgres tables. Pending action holds the exact arguments, their hash, the card text, the approver role, the status, the expiry and the idempotency key. The audit log is append-only: the app's role can insert and read, never update or delete. Then you test a few calls and read the log back.
  52. ImranAnd the practice exercise is the policy check itself, with tests for own-store scope, amount limits and unknown tools.
  53. NadiaIncluding one sneaky unknown tool called constructor. Check that the tool is really in your policy table, not something inherited.
  54. ImranFor my next design review, I'm asking three things. Which user job does this tool serve? Whose token does it run with? And what happens if this call is retried three times?
  55. NadiaAdd one more. Does any argument let the model choose who is acting, or set an amount the policy relies on? If yes, that's the first thing to fix.
  56. NadiaHere's one to think about. The area manager asks on Monday why store one twelve got two milk transfers on Friday. What would you look for first? Take a second.
  57. ImranThe audit rows for those transfers. If there are two different idempotency keys, there were two approved actions. If there's one key, something retried and the transfer service didn't respect the key.
  58. NadiaExactly. One query separates a design problem from a bug. That's why every decision gets a row.
  59. ImranLet me sum up. One, build a few tools around the user's jobs, with descriptions that say when not to use them, and errors that say how to fix the call. Two, the agent runs as the signed-in person with a short-lived narrow token, and identity never comes from the model.
  60. ImranThree, a policy layer in code decides allow, approve or deny, and approvals use a dry-run card, an expiry and an idempotency key.
  61. NadiaOne nuance on the third. The policy denies by default. Anything not in the table, including a tool that someone forgot to remove, is refused. And the audit log also feeds your daily limits and your evals.
  62. ImranNext time?
  63. NadiaRetrieval over procedures. How the assistant answers from Mangrove's own procedures, with the source it used, and says it doesn't know when the procedures don't cover the question.
Finished reading?Mark it complete to fill in its stop on your course line.

Use it at work

Threat model checklist

STRIDE questions for one feature as data, with tests that every threat has an owner and a mitigation.

Open in the Security Lab
Next stopRetrieval over procedures
Was this lesson helpful?
Questions and discussion

Loading the discussion…