Start from the decision

"Use AI for out-of-stocks" describes a worry. A model can't act on a worry. Your first job is to find the decision a prediction could improve, because a prediction is only ever an input to a decision. If no decision changes, the model's accuracy doesn't matter to anyone.

At Mangrove Market, out-of-stocks touch several decisions, each with a different owner and rhythm:

• The nightly replenishment order from each of about 320 stores to its distribution centre (DC). Supply chain owns it, a system computes it at 22:00, and it covers every product each store carries: millions of order lines a night.
• The DCs' weekly purchase orders to suppliers, owned by the buyers.
• Which substitute the app offers when an item is cut from an order, owned by the app team.
• How much shelf space each product gets, revised a few times a year.

Pick one. They need different predictions (daily demand per store, weekly demand per DC, an order of substitutes, long-run sales rates), and a project that tries to serve all four serves none. Tomás, the data analyst who is moving into data science, picks the nightly store order for three reasons. It runs every night, so gains add up fast. Its formula already exists, so a better forecast can drop in without changing anyone's job. And the stock-out it causes is the one customers see: about 9% of app orders have an item cut today.

Here is today's rule, written down with one store's numbers:

order tonight = 4 days x average daily sales of the last 28 days - stock on hand

Store 117, milk 1 litre
average daily sales, last 28 days   30 units
cover for 4 days                    4 x 30 = 120 units
stock on hand at close              45 units
order tonight                       120 - 45 = 75 units

Look closely and you can see a forecast hidden inside the rule. "Average daily sales of the last 28 days" is a prediction of each of the next four days' demand. That is the part a model can replace. The rest (four days of cover, minus stock on hand) is a policy that supply chain owns. Keep that split clear. If you change the forecast and the policy at once and stock-outs fall, nobody can tell which change worked.

Before going further, write down five answers. Who acts on the prediction? How often? What exactly can they change? What does a wrong prediction cost in each direction? How long is it between deciding and seeing the result? For the store order: the replenishment system, every night, the order quantity, waste and crowded back rooms when it is too high or empty shelves and cut app orders when it is too low, and about a day of lead time.

Target, unit and horizon

Once the decision is fixed, three choices define the prediction. The unit is one row: here one store, one product, one day, because that is the grain at which the order is placed. The target is the number the model outputs for that row: units customers wanted to buy. The horizon is how far ahead you predict. The order goes at 22:00, the truck arrives the next morning, and the order has to cover tomorrow to day 4, so the forecast must cover days 1 to 4 after the order night.

The horizon fixes what you may use. At 22:00 you know sales up to today's close, tonight's stock on hand, promotions planned for next week and the festival calendar. You don't know tomorrow's sales, tomorrow's weather or tomorrow's closing stock. A forecast for day 4 rests on older information than one for day 1, so it is usually worse; in Forecasting demand you will report error per horizon.

This is supervised learning: you collect past examples where you know both the inputs and the outcome, and the model learns the link between them. Each training row is one store-product-day from history. The features are what was known at 22:00 the night before: sales over the last 28 days, the same weekday last week, day of the week, whether a promotion was planned, days to the next festival. The label is what happened: units sold that day. In Mangrove's tables a store or DC is a site and a product is an article, so the history lives in daily_sales(store_id, article_id, sale_date, units_sold).

The shape of the target decides the kind of task. Most business machine learning falls into four kinds:

Forecasting is regression where the rows are ordered in time. The past predicts the future, and the future must never leak back into the features; Features and leakage is about exactly that.

The same worry can be framed several ways. "Will milk run out at store 117 tomorrow?" is classification. It sounds closer to the problem, but its answer depends on tonight's order, which is the thing you are trying to set, and a yes or no can't tell the formula how many units to order. Demand per day is regression, and it plugs straight into the order. Frame the prediction as the input the decision needs.

Labels that lie: censored demand

The label is supposed to be what customers wanted. What the data records is what they bought. On most days the two match. On a day the shelf goes empty at 3 pm, they don't: the till records 12 units sold, but customers who came after 3 pm wanted more and left without it. That is censored demand: the true number is at least what you saw, and you don't know by how much. For Mangrove's fast movers, about 1 in 9 store-product-days has some out-of-stock time (approximate).

Censored labels do quiet damage. A model trained on raw sales learns that demand is lower than it really is, most of all for the products that run out most often. A low forecast produces a small order, the small order empties the shelf again, and the empty shelf produces another low sale to learn from. That is the dashed arrow in the first diagram. Today's 28-day average has the same flaw, and it is part of why it under-forecasts by about 9%.

You have three options, from cheap to careful:

• Drop censored days from training and from scoring. It is easy and it removes the worst lies. The catch: stock-outs cluster on the busiest days (festival eves, promotions), so dropping them can remove exactly the peaks the model most needs to learn.
• Keep them with a lower weight in training, and leave them out when you score the model. The model still sees the busy days, but trusts them less. Don't add "was out of stock that day" as a feature for the day you predict: at 22:00 the night before, nobody knows it yet.
• Estimate the lost demand. The app gives a direct signal: when an item is cut from an app order, a customer asked for it and didn't get it, so cut app lines are a floor for lost demand. For store sales, scale up from the sales before the shelf emptied. If 12 units sold by 3 pm on a day that usually sells 60% of its units by 3 pm, the full-day demand was about 12 ÷ 0.6 = 20.

How do you know a day was censored? The cleanest signal is on_hand_close in stock_snapshot. If the store closed with stock left, the shelf probably never emptied. It is a proxy, and an imperfect one: with stock accuracy around 82%, some "in stock" days had stock the system believed in that wasn't on the shelf. Write down the rule you used and how many rows it removed, because every later number depends on it. You will build this flag in SQL in the practice task.

Two metrics: one for the model, one for the business

A model metric tells you how close the forecast is. A business metric tells you whether anything got better. You need both, and you need to agree them before you see any results, so nobody picks the metric that flatters the model afterwards.

For demand forecasts the main model metric in this course is WAPE. It adds up every miss, ignoring direction, and divides by the total actual units, so a product that sells 300 units a day counts more than one that sells 2. Pair it with bias, which keeps the sign and tells you whether you are too high or too low overall. A forecast can have a decent WAPE and still under-order every single day.

day    actual   forecast   |actual - forecast|
Mon      10        12               2
Tue      20        15               5
Wed      30        30               0
sum      60        57               7

WAPE = 7 / 60          = 11.7%
bias = (57 - 60) / 60  = -5.0%   (too low overall)

Why not the familiar MAPE, the average of each row's percentage error? It divides by each row's actual, so a slow mover with 0 units sold makes it divide by zero, and small sellers dominate it. Forecasting demand compares the error measures properly.

The business metric is what the decision owner cares about. For the store order there are three: the out-of-stock rate (the share of store-product-days with an empty shelf at some point), waste (perishables thrown away) and stock on hand (money tied up and back-room space). They pull against each other. You could drive stock-outs to zero by ordering three times as much, and the waste bill would show why that is a bad idea. So report them as a set and agree limits up front, for example "fewer out-of-stock days, with stock on hand up by no more than about 8% and no rise in milk and bread waste".

The two kinds of metric are measured in different places. Model metrics come from offline evaluation: a backtest on history, where you stand at 22:00 on each past night, forecast with only what was known then, and compare with what happened. Features and leakage shows how to split by time without leaking the future. Business metrics need online evaluation: real orders in real stores. At Mangrove that means a store pilot, where 20 stores order from the new forecast and 20 similar stores keep the old rule for several weeks (Experiments: A/B tests, holdouts and store pilots). Offline gains often shrink online. A better forecast doesn't help if the DC can't pick the extra units, the shelf has no room, or the stock waits in the back room.

The baseline, and whether it is worth building

A baseline is the simplest honest prediction, and every model must beat it on the same rows, the same period and the same metric. Use two:

• Today's rule as a forecast: average daily sales over the last 28 days. The business already runs on it, so it is the bar that matters.
• Seasonal naive: the same weekday last week. It knows nothing but the weekly pattern, which makes it a good test of whether a fancier model learned anything more.

On Tomás's backtest (50 fast-moving products in 20 stores, trained on data up to 30 June 2026 and validated on the eight weeks from 1 July to 25 August), the baselines score like this:

The table already shapes the project. The 28-day average beats the seasonal naive overall, because averaging four weeks smooths out noise. Its festival-week bias of −41% shows where the money is: it is slow to notice a jump in demand, so it under-orders exactly when demand peaks. A model has to win there. Later lessons build models to beat these rows, and the best reaches about 24% (Training and comparing models). The rule is plain: a model that doesn't beat the baseline doesn't ship, and a model that beats it by a hair isn't worth a new system to run and watch.

Google's Rules of Machine Learning starts with this rule: "Don't be afraid to launch a product without machine learning." A good rule often gets you most of the value, and it gives you the baseline and the data you will need later. So before building, size the value roughly:

App orders a day                          60,000
with an item cut (about 9%)               about 5,400
share caused by the top fast movers       about 50%   (assumption to check)
share a better forecast removes           about 1/3   (assumption to check)
orders saved from a cut item              about 900 a day

Cost: 2 people for about 6 months at about $5,000 a month each = about $60,000

Every number after the first two is an assumption, so label it and check it. Each saved cut keeps a sale and spares a customer the frustration that makes people stop ordering (ds-05). The store shelves add more value on top, since they serve about 400,000 checkout transactions a day. If the sizing comes out small, or the decision can't change (say the DC can't deliver any more often or any more units), stop here. Picking the right problems for AI covers that choice in general.

Last, write down what would make you stop: a backtest that doesn't beat 34% WAPE by at least a few points, or a pilot that cuts stock-outs only by piling up stock. A project without a stop rule keeps going on hope.