1. Courses
  2. Data
  3. Dashboards and Business Intelligence
  4. Stop 1

40 minIntermediate

From questions to a reporting model

Every Monday, Mangrove Market's leadership sees three different "sales" numbers for the same week. This first lesson of the course starts the rebuild of their weekly trading dashboard at the bottom: you turn the decisions people make into questions, and the questions into a reporting model of facts, dimensions and a declared grain that any later definition can be computed from.

By the end you can

  • Turn a decision into questions with a measure, "by" words, a period and a comparison
  • Declare the grain of a fact table and keep every column true at that grain
  • Choose between a transaction, a periodic snapshot and an accumulating snapshot fact table
  • Share conformed dimensions with surrogate keys, and keep history with slowly changing dimensions
  • Write SQL that joins a fact to its dimensions and sums it at the grain the question needs

The everyday picture

Picture a woman who runs a fruit and vegetable stall in a busy market. Every sale gets a small paper slip: the date, the time, the item code, the quantity and the price. The slips go into a box. On the wall hangs one card per item code, with the name and the category (fruit, vegetables, herbs), and a calendar where she marks holidays and rainy days.

Her nephew keeps the accounts. He writes one line per day in a notebook: the date and the day's total. Every total in it is correct.

One evening she asks: how much fruit did we sell on rainy Saturday mornings this year? The notebook cannot answer. It has no items, no hours and no weather. The box of slips can. She pulls the Saturday morning slips, keeps the fruit by checking the wall cards, checks the calendar for rain, and adds up.

That is a reporting model. The slips are a fact table: one row per thing that happened, at a level of detail she decided once. The wall cards and the calendar are dimensions: the descriptions she filters and groups by. The item code on each slip is the key that joins them. The nephew's notebook is a summary: any question it was not built for is lost for good.

Start from the decisions

Every Monday at 9 am, Mangrove Market's leadership team reviews last week's trading. Three reports are on the screen, and each shows a different "sales" number for the same week. Finance says about $35.2 million. Store operations says about $31.4 million. The app team says about $7.9 million. The first twenty minutes go on arguing about which is right. You and Tomás, Mangrove's data analyst, have been asked to rebuild the weekly trading dashboard: sales, out-of-stock losses and delivery times for 320 stores and the app.

The tempting first step is to open the warehouse and start drawing charts. Good BI teams start earlier, with the people in the meeting and the decisions they make each week. A dashboard earns its place only if it changes what someone does on Monday afternoon.

For each person, write down the decision, then the questions that feed it. A useful question has four parts: a measure (the number), the "by" words (how to slice it), a period, and a comparison.

  • Head of stores decides which stores get a visit or extra staff this week. Question: "Sales by store, last week, against the same week last year."
  • Head of supply decides which products and distribution centres (DCs) to fix first. Question: "Sales lost to empty shelves by category and DC, last week, against the 4-week average."
  • App operations lead decides where to add riders. Question: "Median minutes from order placed to delivered, by city and hour, last week, against the 30-minute promise."

Now read each question like a grammar exercise. The numbers you add up (sales, units, minutes) are , also called measures. The words after "by", "per" and "for each" (store, category, DC, city, hour, week) point to . And each measure comes from a : selling at a till, taking an app order, counting stock on a shelf. One business process usually becomes one fact table.

This is the core of , described by Ralph Kimball and Margy Ross in The Data Warehouse Toolkit. The Kimball Group's design process has four steps: select the business process, declare the grain, identify the dimensions, identify the facts.

You designed tables for an app in Modeling data in tables. A reporting model is built for a different job, so it makes different choices:

App tables (sd-07)Reporting model
Built forMany small writes, one receipt at a timeLarge reads that add up millions of rows
ShapeNormalised: each fact in one place, many joinsA star: one fact table and a few wide dimensions
HistoryThe current state, updated in placeKept: new rows are added, old ones stay
ReadersThe app's own codeAnalysts, dashboards and BI tools

Quick check

The head of stores asks, "Which stores sold less fresh produce before noon last week than in the same week last year?" Which split is right?

Declare the grain, then the facts

The is a sentence that says what one row of a fact table means. Write it before you choose a single column: "one row per sale line", meaning one product line on one till receipt or one app order. Everything else follows from it. A column belongs in the fact only if it has exactly one true value at that grain.

Choose the lowest grain the source system records, called the atomic grain. You can always add rows up; you can never split a row that was stored already added up. Finance's ledger keeps one row per store per day per account. It is correct, and it can never tell you how much fresh produce sold before noon, because the product and the hour were gone before the data arrived. Mangrove's stores ring up about 400,000 baskets a day with roughly 4 lines each, so a sale-line fact grows by about 1.6 million rows a day, close to 600 million a year. A modern analytical warehouse handles that size; no computing power can recover detail the grain threw away.

With the grain fixed, add one key column per dimension and the numbers as fact columns. The result is a .

Sales by city by week dim_date week_start dim_store city fact_sales_line grain: one row per sale line sum(gross_cents) dim_article category dim_channel store or app date_key store_key article_key channel_key
Each line is a key column on the fact pointing at one dimension row. To answer "gross sales by city by week", a query joins only the two highlighted dimensions, groups by city and week_start, and sums gross_cents. Article and channel stay unused until a question needs them. A customer dimension joins the same way.

Every query against a star has the same shape: join the fact to the dimensions you need, filter and group on dimension columns, and sum fact columns. "Units of bakery sold through the app in East city" uses three dimensions instead of two, with no new tables. Because the shape never changes, BI tools such as Power BI, Looker and Metabase can write the query for you when you drag a field onto a chart. Microsoft's Power BI guidance puts it this way: dimension tables are for filtering and grouping, fact tables for summarising.

Store facts that add up. qty, gross_cents, discount_cents and tax_cents are : sum them across stores, products and days and the total still means something. A unit price is not. So store the line's extended amount (quantity times price) and compute an average price later as total cents divided by total units. Ratios such as average basket work the same way: store the parts, divide at the end. Stock on hand is a third kind, which adds up across stores but not across days. Metrics everyone agrees on, later in this course, deals with those kinds in depth.

Two more habits keep the fact usable. Keep the receipt number and line number on the fact with no table of their own, so anyone can trace a row back to the till. Kimball calls such a column a . And never leave a key empty. The customer is known only when someone scans a loyalty card or orders in the app. An empty key silently drops rows from an inner join, so point those lines at a default dimension row, such as customer_key = -1, "Unknown customer".

Quick check

A line-grain fact has a column basket_total_cents copied from the receipt onto every line. A dashboard sums it to show "sales". What goes wrong?

Three kinds of fact table

The dashboard's three measures come from three business processes with different rhythms. Kimball names three basic kinds of fact table, and the dashboard needs one of each.

A has one row per event, written when the event happens and never changed. Sale lines are the classic case. Nothing is recorded when nothing happens: a product that sold no units on Tuesday has no rows for Tuesday.

That gap is why out-of-stock losses need a : one row per store per article per day, recording stock on hand at closing, written every day whether or not anything moved. An empty shelf is a fact you can only see in a table that records the shelf on the days nothing sold. If a store stocks about 8,000 of Mangrove's 25,000 articles, that is about 2.6 million rows a day across 320 stores. Snapshots are dense on purpose.

Delivery times need the third kind. An app order passes through steps: placed, picked, dispatched, delivered (or cancelled). An has one row per order with a column for each milestone. The row is inserted when the order is placed and updated each time the order reaches the next step. Durations such as "minutes from placed to delivered" become plain fact columns, so a median by city and hour is one query. About 60,000 app orders a day means 60,000 new rows a day, each updated a few times in its first hour.

Transaction a row per event one row per sale line time Periodic snapshot a row per day Mon Tue Wed Thu Accumulating snapshot one row, updated placed 18:02 picked 18:09 dispatched 18:14 delivered 18:31 filled in step by step
Top: sale lines arrive whenever a customer pays; a quiet hour has no rows. Middle: stock is recorded every day for every store and article, even with no sales, so an empty shelf becomes visible. Bottom: one app order keeps a single row whose milestones fill in as it moves; this one took 29 minutes.

When you are unsure which kind you need, ask what the question counts. Events that happen (sales, refunds, deliveries received) suggest a transaction fact. A level that you want to see at regular points (stock, account balances, open tickets) suggests a periodic snapshot. A process with a known start, known steps and an end (an order, a delivery to a store, a job application) suggests an accumulating snapshot.

Dimensions everyone shares, and how they keep history

The trading dashboard puts sales, out-of-stock losses and delivery times on one page and filters them all with one "City" selector. That works only if the three fact tables use the same store dimension and the same date dimension. A dimension shared like this, with the same keys, names and meanings for every fact, is called . If the app team keeps its own store list and a calendar whose week starts on Sunday, "East city, week 42" means two different things on one page.

Kimball's planning tool for this is the : one row per business process, one column per dimension, and a mark where they meet. Each column with more than one mark is a dimension that must be built once and shared. The Do it section below builds one for the trading dashboard.

The date dimension is the one every fact uses. It has one row per day, with columns for the week start, month, fiscal period and festival weeks, so "the same week last year" becomes a join and a filter. One date dimension can also play several roles. A sale line has a sale date and a posting date (the day finance books it), so the fact carries two date keys that point at the same dim_date. Kimball calls this a role-playing dimension.

Dimensions get their own , created by the warehouse. You met surrogate keys in sd-07 as protection against changing codes. The warehouse adds two reasons. Data arrives from several systems: the POS knows a store as S118, the app's order system has its own store ids, and both map to one store_key. And the warehouse keeps history, so one store may need several rows. Keep the source code as an ordinary column; it is the store's durable key, the same in every version.

History is the last design choice. Store attributes change: a manager leaves, or head office redraws its regions. A dimension whose attributes change now and then is a , and for each attribute you choose a rule. The Kimball Group lists types 0 to 7; two cover most needs.

  • Type 1: overwrite. Update the value in place. Every past fact now shows the new value. Right for corrections, such as a misspelled store name.
  • Type 2: add a row. Close the current row (set valid_to and is_current = false) and insert a new row with a new surrogate key. Facts loaded before the change keep pointing at the old row, and new facts get the new key. Right for changes that matter to analysis, such as the region a store reports to.
S118: North to Central Type 1: overwrite key 7 · S118 · Central past sales now Central Type 2: add a row key 7 · North until 30 Jun key 42 · Central from 1 Jul Jan–Jun sales store_key 7 Jul–Sep sales store_key 42 sales keep their region
Type 1 rewrites the one row, so a report on North for January to June gives a smaller number in August than it gave in June, with no sales data changed. Type 2 closes the old row and adds a new one with a new key; each sale points at the version that was true when it happened.

Type 2 gives you history "as it was" for free: join the fact to the dimension on the surrogate key. Sometimes a manager wants history "as it is", every past sale under today's region, to compare like with like. Then join through the durable key to the current version of the same store, as the second practice exercise does.

Quick check

A store's name was entered as "Harbour Raod" by mistake. Which rule should fix it?

Do it

Write the first version of the reporting model for the trading dashboard: questions, a bus matrix, grain statements, the sales star in SQL, and two checks that prove it.

  1. Write the decisions and questions

    One row per person in the Monday meeting. If a chart can't be traced to a decision here, it doesn't go on the dashboard.

    Markdown
    # Weekly trading dashboard: decisions and questions
    | Who            | Decision this week          | Measure            | By                 | Compare with         |
    |----------------|-----------------------------|--------------------|--------------------|----------------------|
    | Head of stores | Which stores to visit/staff | Sales              | store, week        | same week last year  |
    | Head of supply | Which products/DCs to fix   | Out-of-stock loss  | category, DC, week | 4-week average       |
    | App operations | Where to add riders         | Minutes to deliver | city, hour         | 30-minute promise    |
    Definitions of "sales" and "out-of-stock loss": open, agreed in bi-03.
  2. Draw the bus matrix

    Rows are business processes, columns are dimensions. Every column with two or more marks is a conformed dimension you build once.

    Markdown
    | Business process (fact)       | Date | Store | Article | Channel | Customer | DC | Rider |
    |-------------------------------|------|-------|---------|---------|----------|----|-------|
    | Sales: fact_sales_line        |  X   |   X   |    X    |    X    |    X     |    |       |
    | Stock: fact_stock_daily       |  X   |   X   |    X    |         |          | X  |       |
    | App orders: fact_app_order    |  X   |   X   |         |    X    |    X     |    |   X   |
  3. Write one grain sentence per fact

    If two people read the sentence and picture different rows, rewrite it.

    Markdown
    fact_sales_line:  one row per line on a till receipt or app order, at the moment of sale.
    fact_stock_daily: one row per store per article per day, as counted at store closing.
    fact_app_order:   one row per app order, updated at each milestone until delivered or cancelled.
  4. Create the sales star

    Surrogate keys on every dimension, the store code kept as the durable key, type 2 columns on the store, and the grain written as the fact's primary key. dim_article, dim_channel and dim_customer follow the same pattern as dim_store.

    SQL
    CREATE TABLE dim_date (
      date_key     integer PRIMARY KEY,          -- 20261012
      full_date    date    NOT NULL UNIQUE,
      week_start   date    NOT NULL,             -- the Monday of that week
      month_start  date    NOT NULL,
      is_festival  boolean NOT NULL DEFAULT false
    );
    
    CREATE TABLE dim_store (
      store_key   bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,  -- warehouse key
      store_code  text    NOT NULL,              -- durable key from the POS: 'S118'
      name        text    NOT NULL,
      city        text    NOT NULL,
      region      text    NOT NULL,              -- type 2: a change adds a row
      valid_from  date    NOT NULL,
      valid_to    date    NOT NULL DEFAULT '9999-12-31',
      is_current  boolean NOT NULL DEFAULT true
    );
    SQL
    CREATE TABLE fact_sales_line (
      channel_key       integer NOT NULL REFERENCES dim_channel (channel_key),
      sale_id           bigint  NOT NULL,        -- degenerate dimension: receipt or order
      line_no           integer NOT NULL,
      sale_date_key     integer NOT NULL REFERENCES dim_date (date_key),
      posting_date_key  integer NOT NULL REFERENCES dim_date (date_key),  -- same dimension, second role
      store_key         bigint  NOT NULL REFERENCES dim_store (store_key),
      article_key       bigint  NOT NULL REFERENCES dim_article (article_key),
      customer_key      bigint  NOT NULL REFERENCES dim_customer (customer_key),  -- -1 = unknown
      order_status      text    NOT NULL,        -- 'completed', 'cancelled', 'returned'
      qty               integer NOT NULL,
      gross_cents       bigint  NOT NULL,        -- qty x price, before discount
      discount_cents    bigint  NOT NULL DEFAULT 0,
      tax_cents         bigint  NOT NULL DEFAULT 0,
      PRIMARY KEY (channel_key, sale_id, line_no)   -- the grain, written down
    );
  5. Prove the grain and reconcile to the source

    Many cloud warehouses accept a primary key but do not enforce it, so test the grain with a query. Then check that the fact adds up to the source lines day by day (sale and sale_line from the POS). Both queries should return no rows; any row is a bug to fix before anyone sees a chart.

    SQL
    -- 1. The grain holds: no two rows for the same line
    SELECT channel_key, sale_id, line_no, count(*)
    FROM fact_sales_line
    GROUP BY channel_key, sale_id, line_no
    HAVING count(*) > 1;
    
    -- 2. Store-channel facts add up to the POS lines, day by day
    WITH src AS (
      SELECT (s.sold_at AT TIME ZONE 'Asia/Dhaka')::date AS day,   -- the stores' zone
             sum(l.qty * l.price_cents) AS source_cents
      FROM sale s JOIN sale_line l ON l.sale_id = s.id
      GROUP BY 1
    ), fact AS (
      SELECT d.full_date AS day, sum(f.gross_cents) AS fact_cents
      FROM fact_sales_line f
      JOIN dim_date d    ON d.date_key = f.sale_date_key
      JOIN dim_channel c ON c.channel_key = f.channel_key
      WHERE c.channel = 'store'
      GROUP BY 1
    )
    SELECT * FROM src FULL JOIN fact USING (day)
    WHERE source_cents IS DISTINCT FROM fact_cents;

Bring it to your team

Ask these whenever someone proposes a new report table or dashboard.

  • Which decision will this change, and who makes it each week?
  • What is the grain of this fact table, in one sentence?
  • Is every column true at that grain, or is something copied down from the order or the day?
  • Do all our fact tables use the same store and date dimensions?
  • When a store changes region, do its old sales stay in the old region or move? Which do we want, and where is that written down?
  • Does the fact reconcile to the source system every day, and who looks when it doesn't?

Red flags

  • Dashboards built straight on the app's tables, with a different hand-written query per chart.
  • Facts stored already summed by store and day "to save space".
  • Order-level amounts repeated on every line row.
  • Each team keeping its own store list or its own calendar.
  • A store's region or manager overwritten with no history.
  • Unit prices or averages stored in a fact and then summed.

Practice

Sum a star at the grain the question needs

A small sales star: fact_sales_line(sale_id, line_no, date_key, store_key, article_key, channel_key, qty, gross_cents), with dim_date(date_key, full_date, week_start), dim_store(store_key, store_code, name, city), dim_article(article_key, name, category) and dim_channel(channel_key, channel). Return gross sales and units by city and category for the week starting 12 October 2026, both channels. Columns: city, category, gross_cents, units, one row per city and category.

SQL exerciseRuns in a real PostgreSQL inside your browser
-- Gross sales and units by city and category, week starting 2026-10-12
SELECT s.city,
       sum(f.gross_cents) AS gross_cents
FROM fact_sales_line f
JOIN dim_store s ON s.store_key = f.store_key
-- 1. join dim_article (for category) and dim_date (for week_start)
-- 2. keep only the week starting 2026-10-12
-- 3. add category and units (the sum of qty), and group by city and category
GROUP BY s.city;
Ctrl + Enter runs
Show a solution
SQL
SELECT s.city,
       a.category,
       sum(f.gross_cents) AS gross_cents,
       sum(f.qty)         AS units
FROM fact_sales_line f
JOIN dim_store   s ON s.store_key   = f.store_key
JOIN dim_article a ON a.article_key = f.article_key
JOIN dim_date    d ON d.date_key    = f.date_key
WHERE d.week_start = '2026-10-12'      -- filter on the dimension, not on date_key ranges
GROUP BY s.city, a.category;

Every star query has this shape: join the dimensions the question names, filter and group on their columns, sum the fact. The two sales on 19 October belong to the next week, so the week filter drops them. Channel isn't in the question, so dim_channel isn't joined and both channels are counted.

Pass the exercise to finish this task.

History as it is today

On 1 July 2026 Mangrove moved store S118 from the North region to the new Central region, and dim_store is type 2, so S118 has two rows. fact_store_month(month_start, store_key, gross_cents) holds monthly sales from May to August. The Central regional manager wants every month restated to today's regions, so she can compare May and June with July and August like for like. The starter joins on store_key, which counts each month under the region the store had at the time. Change it so every month counts under the store's current region. Columns: region, gross_cents.

SQL exerciseRuns in a real PostgreSQL inside your browser
-- Gross sales by region, May to August 2026.
-- Today this joins each month to the store version current at the time ("as it was").
-- Change it so each month counts under the store's region today ("as it is"):
-- join a second copy of dim_store on store_code, keeping only the current row.
SELECT s.region,
       sum(f.gross_cents) AS gross_cents
FROM fact_store_month f
JOIN dim_store s ON s.store_key = f.store_key
GROUP BY s.region;
Ctrl + Enter runs
Show a solution
SQL
SELECT cur.region,
       sum(f.gross_cents) AS gross_cents
FROM fact_store_month f
JOIN dim_store s   ON s.store_key    = f.store_key    -- the version at the time of sale
JOIN dim_store cur ON cur.store_code = s.store_code   -- every version of the same store
                  AND cur.is_current                  -- keep only today's version
GROUP BY cur.region;

The starter gives North $2.32 million and Central $2.48 million, because S118's May and June sales stay with North. Restated to today's regions, all four of S118's months count for Central: $3.28 million against $1.52 million. Both answers are correct; they answer different questions, so a dashboard must say which one it shows. The durable key (the store code) is what lets one type 2 dimension answer both.

Pass the exercise to finish this task.

Model the other two measures

The trading dashboard also needs (a) which articles were on an empty shelf at closing, by store and DC, every day, and (b) minutes from placed to delivered for each app order, by city and hour, including how many orders were cancelled. For each, write the fact table type, the grain sentence, the dimensions and the fact columns. Don't define "out-of-stock loss" yet; just make sure the model can compute it later.

  • (a) is a periodic snapshot with the grain "one row per store per article per day"
  • (a) notes that stock on hand adds up across stores but not across days
  • (b) is an accumulating snapshot with one row per order and a time or date key for each milestone
  • (b) stores durations in minutes as fact columns and keeps cancelled orders with a status
  • Both reuse the same store and date dimensions as the sales fact
Show a model answer
Markdown
(a) fact_stock_daily: periodic snapshot.
Grain: one row per store per article per day, as counted at store closing.
Dimensions: date, store, article, supplying DC.
Facts: on_hand_qty (semi-additive: sum across stores, never across days),
       is_out_of_stock (1 or 0, so a sum counts empty shelves),
       expected_daily_units (for the loss estimate defined later).

(b) fact_app_order: accumulating snapshot.
Grain: one row per app order, updated at each milestone.
Dimensions: placed date and hour, store, city (through the store),
            customer, rider, order status.
Milestones: placed_at, picked_at, dispatched_at, delivered_at, cancelled_at.
Facts: minutes_to_pick, minutes_to_deliver (empty until delivered),
       order_value_cents, is_cancelled.

Both use the conformed dim_store and dim_date, so one city filter works across sales, stock and delivery times on the same page.

Check yourself

The app operations lead wants the median minutes from order placed to delivered, by city and hour, and a count of orders stuck at "picked" right now. Which fact table fits best?

In June, finance's board pack showed North region sales of $41 million for January to June. In August the same report, for the same months, shows $38 million. No sales were loaded or deleted. What is the most likely cause?

Finance offers its ledger table (one row per store per day per account, net amounts) as the base for the whole trading dashboard. The buying team will ask for fresh produce sales before noon. What do you say?

Flashcards

Open my review

Key takeaways

  • Start from the decisions people make; each question gives you a measure, "by" words that become dimensions, a period and a comparison.
  • Declare the grain in one sentence before choosing columns, choose the lowest grain the source records, and keep every column true at that grain.
  • Events go in transaction facts, levels in periodic snapshots, and processes with milestones in accumulating snapshots.
  • Share conformed date and store dimensions with warehouse surrogate keys; use type 2 for changes that matter to history and type 1 for corrections.
  • Mangrove's three Monday numbers come from three grains and three dates; one line-grain fact with every amount and both dates lets one definition be agreed later.

Sources and further reading

Episode transcriptThree sales numbers and one grain · about 12 minutes · tap any line to play from there
  1. NadiaImran, how many sales numbers did you see in Monday's leadership meeting?
  2. ImranThree. Finance said about thirty-five point two million dollars. Store operations said about thirty-one point four million. The app team said about seven point nine million.
  3. NadiaFor the same week.
  4. ImranFor the same week. And we spent the first twenty minutes arguing about which one was right.
  5. NadiaThat's the job for this whole course. Tomás and a new analyst are rebuilding the weekly trading dashboard. Sales, out-of-stock losses and delivery times, for three hundred and twenty stores and the app.
  6. ImranSo today we decide which sales number is the right one?
  7. NadiaThat comes in lesson three. Today we build the model underneath, so that whichever definition wins, it can be computed from one place.
  8. NadiaPicture a woman who runs a fruit and vegetable stall in a busy market. Every sale gets a small paper slip. Date, time, item code, quantity, price. The slips go into a box.
  9. ImranAnd I'm guessing there are cards on the wall. One per item.
  10. NadiaExactly. Name and category. And a calendar where she marks holidays and rainy days. Her nephew keeps the accounts in a notebook, one line per day, the date and the total.
  11. ImranTidy and correct. I'd keep it that way too.
  12. NadiaThen one evening she asks, how much fruit did we sell on rainy Saturday mornings this year?
  13. ImranAnd the notebook has no fruit, no mornings and no rain in it. The box of slips has all three.
  14. NadiaThat's the lesson in small. The slips are a fact table. The cards and the calendar are dimensions. The item code on each slip is the key that joins them. And the notebook is a summary. Any question it wasn't built for is gone for good.
  15. ImranSo where do you start in real life? I'd open the warehouse and look at what tables we have.
  16. NadiaMost people do. Good teams start one step earlier, with the people in the meeting and the decisions they make each week.
  17. NadiaThe head of stores decides which stores get a visit or extra staff. The head of supply decides which products and distribution centres to fix first. The app operations lead decides where to add riders.
  18. ImranAnd each decision gives you questions.
  19. NadiaRight. A useful question has four parts. A measure, the by words, a period and a comparison. Sales by store, last week, against the same week last year.
  20. ImranSo sales, the number, is a fact. And store and week, the by words, are dimensions.
  21. NadiaThat's the trick. Read the question like a grammar exercise. The numbers you add up are facts. The words after by, per, or for each point to dimensions. And the activity that produces the number, like selling at a till, usually becomes one fact table.
  22. ImranThis is Kimball, right? Dimensional modelling.
  23. NadiaYes. Ralph Kimball and Margy Ross. Their design process has four steps. Select the business process, declare the grain, identify the dimensions, identify the facts.
  24. ImranGrain. The data team says that word a lot, and I've never been quite sure what it means.
  25. NadiaThe grain is one sentence that says what a row means. For sales, one row per sale line. One product line on one till receipt or one app order. You write it before you choose a single column.
  26. ImranAnd the first diagram shows that table in the middle, with four boxes around it.
  27. NadiaIf you can see the screen, yes. The sales fact sits in the middle, with date, store, article and channel around it, each joined by a key. If you can't, picture a star with four points. For sales by city by week, you join only the date and store points, group by city and week, and add up the sales column.
  28. ImranWhy the line level, though? Finance's ledger is already summed by store and day. It's smaller, and it's the official number.
  29. NadiaBecause you can always add rows up, and you can never split them. The ledger can't tell you how much fresh produce sold before noon. The product and the hour were gone before the data arrived.
  30. ImranHow big does the line level get?
  31. NadiaRoughly four hundred thousand baskets a day across the stores, about four lines each. So about one point six million rows a day, close to six hundred million a year. A modern analytical warehouse handles that.
  32. ImranOkay, here's my instinct. An app order has a delivery fee. I'd copy the fee onto every line so it's handy.
  33. NadiaThat's the classic trap. Two grains in one table. Take an order with five lines and a two dollar fee. Copy the fee onto each line, and the sum says ten dollars of fees.
  34. ImranBecause the fee belongs to the order, and the row is a line.
  35. NadiaRight. Every column has to be true at the grain. Keep order amounts in an order-level table, or split the fee across the lines so they add back up to two dollars.
  36. ImranSame with unit price, I suppose. Adding up prices gives you nonsense.
  37. NadiaYes. Store the line amount and the units, and divide the totals when you want an average price. Lesson three goes deeper into what adds up and what doesn't.
  38. ImranSo, back to our three Monday numbers. What did Tomás find when he traced them?
  39. NadiaThree sources, three grains, three dates. Finance's number comes from the ledger, one row per store per day per account, by posting date. So Sunday's sales often post on Monday.
  40. NadiaStore operations' number comes from the tills, one row per sale line, by sale date, stores only. And the app number counts every order placed, including orders that were cancelled later.
  41. ImranSo no join can make them agree.
  42. NadiaNone. The ledger has no products, and the order system has no till returns. So nobody picks a winner yet. They design one sales fact at line grain for both channels.
  43. NadiaIt holds every amount a definition might need. Gross, discount, tax, a return flag, the order status. And two dates, the sale date and the posting date.
  44. ImranTwo date columns pointing at the same calendar table.
  45. NadiaYes. That's called a role-playing dimension. And the ledger becomes the weekly check the fact has to reconcile to.
  46. ImranThe second diagram has three lanes. What are they?
  47. NadiaThree kinds of fact table. The top lane is a transaction table. A dot appears whenever a sale happens, and nothing appears in a quiet hour.
  48. ImranWhich is a problem for out-of-stock, isn't it? An empty shelf sells nothing, so it leaves no rows.
  49. NadiaExactly. So stock goes in the middle lane, a periodic snapshot. One row per store per article per day, written every night whether anything moved or not. If a store carries about eight thousand articles, that's about two point six million rows a day.
  50. ImranAnd the bottom lane is for delivery times.
  51. NadiaThat's an accumulating snapshot. One row per app order, with a column for each step. Placed, picked, dispatched, delivered. The row is inserted when the order is placed and updated as it moves. The order in the picture took twenty-nine minutes.
  52. ImranSo median minutes by city and hour is one query. And I can also see orders stuck at picked.
  53. NadiaRight. A quick rule. Events go in transaction tables. Levels you check at regular points go in periodic snapshots. Processes with steps go in accumulating snapshots.
  54. ImranAll three tables have a store and a date. Does each one get its own store list?
  55. NadiaPlease don't. That's how one team ends up with a week that starts on Sunday. The store and date dimensions must be conformed. Same keys, same names, same values, shared by every fact. Then one city filter works across the whole page.
  56. ImranAnd the bus matrix is how you plan that. Business processes down the side, dimensions across the top.
  57. NadiaYes. Any dimension with two or more marks is one you build once and share.
  58. ImranLast diagram. Store S one eighteen moves from the North region to Central on the first of July.
  59. NadiaThis is the slowly changing dimension problem. Type one overwrites the region. It's easy, but now every past sale of that store shows up as Central.
  60. ImranSo the North report for January to June gives a smaller number in August than it gave in June.
  61. NadiaWith no sales changed at all. Type two closes the old row and adds a new row with a new key. Old sales point at the North version, new sales at the Central version. On screen, that's the two boxes side by side, with sales pointing up at each.
  62. ImranWhen would I ever use type one, then?
  63. NadiaFor corrections. A store name typed as Harbour Raod. Nobody wants history split by a typo.
  64. ImranAnd if the Central manager wants all of that store's months counted under her region, to compare like with like?
  65. NadiaThen you join through the store code, the durable key, to the store's current row. That's the second practice exercise.
  66. NadiaHere's one to think about. The POS already has a store code. Why does the warehouse give each store its own surrogate key? Take a second.
  67. ImranBecause the app's system has its own store ids?
  68. NadiaThat's one reason. Both map to one warehouse key. The other is history. With type two, one store can have several rows, so the code alone can't pick a row.
  69. ImranSo what do I actually do on Monday?
  70. NadiaFive steps. Write each decision and its questions in a table. Draw the bus matrix. Write one grain sentence per fact table. Create the sales star, keeping the store code as the durable key and type two columns on the store. Then run two checks.
  71. NadiaOne check proves no line appears twice. The other proves the fact adds up to the till data, day by day.
  72. ImranBecause many cloud warehouses accept a primary key but don't enforce it.
  73. NadiaRight. So you test the grain with a query, and any row it returns is a bug to fix before anyone sees a chart.
  74. ImranFor my next review, I'll ask three things. What's the grain, in one sentence? Is every column true at that grain? And when a store changes region, do its old sales move or stay?
  75. NadiaGood list. Add one more. Does the fact reconcile to the source every day, and who looks when it doesn't?
  76. ImranLet me sum up. One, start from decisions, and read each question for facts and dimensions. Two, declare the grain first, at the lowest level the source records. Three, pick the fact table type by what you count, events, levels or processes, and share the store and date dimensions.
  77. NadiaOne nuance on the third. Shared dimensions also need a history rule for each column. Type two for changes that matter, type one for corrections.
  78. ImranAnd next time?
  79. NadiaNext time we turn raw tables into this model, layer by layer, with tested SQL inside the warehouse.
Finished reading?Mark it complete to fill in its stop on your course line.

Use it at work

Monthly sales report

Revenue, orders and average order value per month, with the change on the month before.

Open in the SQL Lab
Next stopTransforming data in the warehouse
Was this lesson helpful?
Questions and discussion

Loading the discussion…