It is a rainy evening and you order groceries in the Mangrove app: rice, eggs and a bag of mangoes. You press Place order and a green tick appears at once: "Order placed". Four minutes later a notification arrives: "Sorry, mangoes are out of stock at your store. We removed them and you won't be charged $1.80."
Did the first screen lie? Not quite. It told you something true, just not what you assumed. It meant "we have your order and we will not lose it". You read it as "your mangoes are in a bag". The gap between those two meanings is one of the most common design choices in apps that sell physical things.
What the app could do instead
To show a screen that is true in your sense, the order API would have to do everything before it answers: check stock in the store's system, hold the items, take payment, maybe find a picker. That is four or five systems in one request, some slow, some far away. Mangrove gets about 60,000 app orders on a normal day and around four times that at a festival peak. One slow store system would make every checkout slow, and one that is down would make checkout fail.
So most order systems split the work in two:
- Accept the order, quickly and safely. This part must never lose anything.
- Confirm it in the background: stock, payment, picking. This part can take seconds or minutes, and it can retry.
The first screen reports step 1. Everything after it arrives as an update.
The path of one order
- The app sends the order with an idempotency key, so a retry on a bad connection does not place it twice. The tap Pay twice breakdown explains how that works.
- The API writes two rows in one transaction: the order, with status
placed, and an event "order 1042 placed" in anoutboxtable. Either both are saved or neither is. - The API answers "Order placed". That costs a couple of database writes, not a tour of every system.
- A relay reads the outbox and publishes the event to a queue. If the relay crashes, the event is still in the table, and it goes out when the relay comes back.
- Workers do the slow parts: hold the stock at the store, take payment, assign a picker.
- When something changes the order, a worker updates its status and the app gets a push: "confirmed", or "1 item removed".
Why the outbox, and not "save the order, then send the event"? Because the server can crash between the two. Then you have an order that nobody works on, the worst kind of bug because it is silent. Writing both in one transaction turns that crash into a delay that fixes itself. Events, the outbox pattern and sagas walks through it, and Queues, background jobs and pub/sub covers the workers.
Why items still go missing
Mangrove's stock numbers are right only about 82% of the time, and about 9% of app orders have an item cut for being out of stock. The database says there are 3 bags of mangoes. The shelf has none, because a shopper in the store took them an hour ago, or a bag was damaged and nobody scanned it out.
No backend design fixes a wrong count at the source. What the system can decide is when to find out, and how to tell you.
- Hold stock early. The stock worker reserves the items within seconds of the order, so two app orders can't both get the last bag. Case study: ordering without overselling stock shows the reservation table and its trade-offs.
- Charge for what is packed. With cards, the payment can be held for the full amount and only the packed amount taken, so a cut item never needs a refund. With cash on delivery, about 35% of Mangrove's app orders, the rider simply collects less.
- Say what happened, in plain words. "1 item removed: mangoes, $1.80 not charged" beats a total that quietly changes at the door.
The real fix is the words
The honest version of the first screen does not need new servers. It needs order states a customer can understand, and screen text that matches each one:
| Status | What is true | What the screen says |
|---|---|---|
| placed | We saved your order. Nothing is checked yet. | Order received |
| confirmed | Stock is held and payment is approved. | Order confirmed |
| picking | Someone is packing it. | Packing your order |
| out for delivery | A rider has it. | On the way |
"Order received" is a promise the system can keep in a fraction of a second. "Order confirmed" follows a few seconds later, and most customers barely notice the gap. When an item is cut, the news arrives as a new state, not as a broken promise.
This is eventual consistency seen from the customer's side: some parts of the system learn the truth a little later than others. The design question is never "how do we make the gap zero?" It is "what do we show while the gap is open?"
The trade-offs, side by side
- Answer only after every check. The screen is always true. Checkout is as slow as the slowest system behind it, and it fails when any of them is down.
- Accept, then confirm. Fast, and it keeps working when a store system is slow. The first screen is a promise, so you need clear states, push updates and a clean way to cut items.
- Accept without an outbox. Simpler code, but now and then an order is saved with no event and sits untouched until a customer complains. Never worth it for money or stock.
Mangrove uses the second, like most apps that deliver physical goods. The price is the work on honest states above, and it is far cheaper than a checkout that falls over on festival night.
Next time an app says "Done!", ask: done with what?