--- title: 'What Is Data Analytics' source: 'https://academia.sh/en/courses/introduction-to-data-analytics/what-is-data-analytics' course: 'Introduction to Data Analytics' language: en updated: '2026-10-07T04:31:12+00:00' license: 'CC BY-SA 4.0' objectives: - 'Produces three different numbers for the same question from the same raw dataset using three defensible definitions' - 'Measures separately the difference in totals between two definitions and the number of records they disagree on' - 'Derives the list of decisions that must be written next to a number when it is reported' --- # What Is Data Analytics From a single subscriber table, the 'active subscriber count' is computed with three defensible definitions and yields three different numbers: 3,849 where the contract record is active, 3,548 where a reading was sent in the last 30 days, 4,211 where the contract is open. The totals of the first two definitions differ by only 301, yet they disagree on 645 subscribers; when the window, the one free number, is widened from 14 days to 90 days, the count rises from 1,859 to 3,750. No calculation changed, only the definition. When someone asks where a number in a report came from, the first answer is usually a description of the calculation: which rows were summed, which division was performed. This curriculum looks somewhere else. A number is often at its value not because of the calculation but because of a **definition decision** made before the calculation began. The calculation may have been done correctly and the number could still have been a different number. Data analytics is the work of making a question answerable with data and tying the resulting number to a decision. What this lesson measures is the first link of that chain: how far the decisions made while a question is made measurable move the number. The unit of measure is **the difference**: how many different numbers come out of the same data, for the same question, under two defensible decisions. - **AC1.** The dataset is the subscriber records of a **regional measurement network**: a network that collects readings from water meters, turns the readings into bills, and opens work orders for the field crew. Both the network and the records are **fiction**; every number below is the output of a model, not of a measured installation. - **AC2.** The table is ready and is used in this course as the **dataset**. Schema design, normalization, and query writing were established in the Databases curriculum and are not repeated here. - **AC3.** The reference date is day **0**. Every field ending in `Day` is the number of days counted back from that date: if `lastReadingDay` is 4, the meter sent a reading four days ago. - **AC4.** The reading cycle of the meters is monthly in the model. It is not normal for a meter to send no reading for 30 days; it is normal for it to send none for 14 days. - **AC5.** The contract status takes three values: `active`, `suspended`, `closed`. This field is the state in the **record system**; it has no necessary link to the behavior of the meter. - **AC6.** The question is this: **how many active subscribers are there in the network.** As it stands, the question is not a calculation but a request for a definition. ## What Is in the Raw Record Before answering the question, look at what is at hand. The block below builds the dataset and prints a single cross table: contract status against the time of the meter's last reading. The mismatch between the two is the source of the rest of the lesson. ```js // dataset.mjs — subscriber records of the regional measurement network; a MODEL, the records are fiction. // The reference date is day 0; fields ending in "Day" are the number of days counted back from that date. export const SEED = 20260601, SUBSCRIBERS = 4800, REGION = ["north", "south", "east", "west", "central"]; const rng = (t) => { let s = t >>> 0; return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; }; const r = rng(SEED); const pick = (u, d) => { let t = 0; for (const [a, p] of d) { t += p; if (u < t) return a; } return d.at(-1)[0]; }; export const RECORDS = Array.from({ length: SUBSCRIBERS }, (_, i) => { const region = REGION[Math.floor(r() * 5)]; const tier = pick(r(), [[1, 0.46], [2, 0.34], [3, 0.20]]); const connectionDay = 30 + Math.floor(r() * 2900); // The closure probability is high in the lowest tariff tier and closures pile up in recent months. const closureP = [0.16, 0.09, 0.05][tier - 1]; const closureDay = r() < closureP ? Math.min(connectionDay - 1, Math.floor(r() * r() * 420)) : null; const suspended = closureDay === null && r() < 0.085; const contract = closureDay !== null ? "closed" : suspended ? "suspended" : "active"; // Days since the meter's last reading. Some closed and suspended subscribers still // send readings; some active subscribers (faulty meter, empty home) do not. const g = contract === "active" ? (r() < 0.13 ? 31 + Math.floor(r() * 300) : Math.floor(r() * 29)) : contract === "suspended" ? (r() < 0.34 ? Math.floor(r() * 29) : 31 + Math.floor(r() * 250)) : (r() < 0.11 ? Math.floor(r() * 29) : 31 + Math.floor(r() * 500)); const consumption = Math.round((2 + tier * 4 + r() * 9) * 10) / 10; const fee = Math.round((6 + tier * tier * 7) * (0.85 + r() * 0.3)); return { i, region, tier, connectionDay, closureDay, contract, lastReadingDay: g, consumption, fee, support: r() < 0.22 ? 1 + Math.floor(r() * 3) : 0 }; }); const gen = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); if (import.meta.url === `file://${process.argv[1]}`) { console.log(`seed ${SEED}: ${SUBSCRIBERS} rows, ${Object.keys(RECORDS[0]).length} fields`); console.log("fields: " + Object.keys(RECORDS[0]).join(", ") + "\n"); const y = gen([12, 13, 11, 12, 11]); console.log(y(["contract", "subscribers", "0-30 days", "31-90 days", "90+ days"])); for (const s of ["active", "suspended", "closed"]) { const l = RECORDS.filter((k) => k.contract === s); console.log(y([s, l.length, l.filter((k) => k.lastReadingDay <= 30).length, l.filter((k) => k.lastReadingDay > 30 && k.lastReadingDay <= 90).length, l.filter((k) => k.lastReadingDay > 90).length])); } const t = (f) => RECORDS.filter(f).length; console.log(y(["total", SUBSCRIBERS, t((k) => k.lastReadingDay <= 30), t((k) => k.lastReadingDay > 30 && k.lastReadingDay <= 90), t((k) => k.lastReadingDay > 90)])); } ``` ``` seed 20260601: 4800 rows, 10 fields fields: i, region, tier, connectionDay, closureDay, contract, lastReadingDay, consumption, fee, support contract subscribers 0-30 days 31-90 days 90+ days active 3849 3376 88 385 suspended 362 121 55 186 closed 589 51 59 479 total 4800 3548 202 1050 ``` The table is read by its four corners. The top left corner is the expected case: the meters of the 3,376 subscribers whose contract is active sent a reading within the last 30 days. The top right corner is the unexpected one: the meters of 385 subscribers whose contract is active have been silent for more than 90 days. The bottom left corner is unexpected too: the meters of 51 subscribers whose contract is closed sent a reading within the last 30 days. These two corners are not a data error; both have an ordinary explanation. A silent meter may be faulty, or the home may be empty. A reading coming from a closed contract is the time that passes between the closure record being processed and the meter being removed. **The record system and the physical world move at different speeds**, and the table keeps the two in separate columns. When the question is put to this table, the answer is not single, because the phrase "active subscriber" does not say which column to look at. ## Three Definitions, Three Numbers The block below applies three definitions to the same dataset separately and prints the results side by side. All three are defensible definitions: the first looks at the record system, the second at the behavior of the meter, the third at the legal status of the contract. ```js // active.mjs — "active subscriber count" from the same raw dataset: three defensible definitions side by side. import { RECORDS, SUBSCRIBERS, SEED } from "./dataset.mjs"; const DEFINITION = [ { name: "A: contract record active", f: (k) => k.contract === "active" }, { name: "B: reading in 30 days", f: (k) => k.lastReadingDay <= 30 }, { name: "C: contract open", f: (k) => k.contract !== "closed" }, ]; const gen = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const S = DEFINITION.map((t) => new Set(RECORDS.filter(t.f).map((k) => k.i))); const y = gen([26, 13, 11]); console.log(`seed ${SEED}: ${SUBSCRIBERS} subscriber records, one table, one reference date\n`); console.log(y(["definition", "subscribers", "% records"])); DEFINITION.forEach((t, a) => console.log(y([t.name, S[a].size, ((100 * S[a].size) / SUBSCRIBERS).toFixed(1)]))); const z = gen([10, 9, 11, 11, 13]); console.log("\n" + z(["pair", "diff", "only first", "only last", "disagreement"])); for (const [a, b] of [[0, 1], [2, 0], [2, 1]]) { const i1 = [...S[a]].filter((i) => !S[b].has(i)).length; const i2 = [...S[b]].filter((i) => !S[a].has(i)).length; console.log(z([`${"ABC"[a]} - ${"ABC"[b]}`, S[a].size - S[b].size, i1, i2, i1 + i2])); } // The only free number in definition B is the window length; since the reading cycle is monthly, // a window shorter than 30 days asks about a frequency the meter could never send at. const w = gen([9, 13, 12]); console.log("\n" + w(["window", "subscribers", "ratio to A"])); for (const p of [14, 21, 30, 45, 60, 90]) { const n = RECORDS.filter((k) => k.lastReadingDay <= p).length; console.log(w([`${p} days`, n, (n / S[0].size).toFixed(2)])); } ``` ``` seed 20260601: 4800 subscriber records, one table, one reference date definition subscribers % records A: contract record active 3849 80.2 B: reading in 30 days 3548 73.9 C: contract open 4211 87.7 pair diff only first only last disagreement A - B 301 473 172 645 C - A 362 362 0 362 C - B 663 714 51 765 window subscribers ratio to A 14 days 1859 0.48 21 days 2733 0.71 30 days 3548 0.92 45 days 3602 0.94 60 days 3654 0.95 90 days 3750 0.97 ``` Three definitions gave three numbers: 3,849, 3,548, and 4,211. The largest gap between them is 663 subscribers, 13.8% of the records. No calculation changed: all three definitions are a single filtering operation and none is more complicated than another. The only thing that changed is which column was looked at. The second table says something more important. The totals of A and B differ by only **301**, but the **number of subscribers the two definitions disagree on is 645**. 473 subscribers are only in A, 172 subscribers are only in B. The difference in totals looks small because most of the two-way disagreement cancels itself out. **Totals being close does not mean the two definitions count the same people.** If a report is going to carry the sentence "both methods give roughly the same result," what needs to be measured is not the difference in totals but the disagreement at the record level. ## The Window Is Also a Decision Definition B contains a number: 30 days. This number did not come from the data; it was chosen. The third table writes down the cost of the choice: when the window is cut to 14 days, the active subscriber count falls to 1,859, and when it is widened to 90 days, it rises to 3,750. Same table, same question, same calculation; the result moves by more than a factor of two. Not every value of the window is defensible. AC4 says the reading cycle is monthly; a 14-day window makes a frequency the meter is not expected to send at the criterion and counts half of the active subscribers as silent. A 90-day window falls the other way: it counts a meter that has sent no reading for three months as active. Values between 30 and 60 move the result by only 106 subscribers, so in this range the definition is sound. **The defensibility of a threshold is measured by how much the result moves around that threshold.** Moving the threshold from 14 to 21 moves the count by 874 subscribers; moving it from 45 to 60 moves it by 52. The first region is fragile, the second is stable. ## The Ranking Moves Too A single total rarely stands alone in a report. Next to the total there is a breakdown, and someone acts on the top row of that breakdown: which region to visit first, where the budget goes first. If a definition decision moves the total, it moves the ranking as well. ```js // region.mjs — the definition decision moves not only the total but the ranking: five regions, two definitions. import { RECORDS, REGION } from "./dataset.mjs"; const ratio = (l, f) => l.filter(f).length / l.length; const rows = REGION.map((b) => { const l = RECORDS.filter((k) => k.region === b); return { b, n: l.length, a: ratio(l, (k) => k.contract === "active"), c: ratio(l, (k) => k.lastReadingDay <= 30) }; }); // Rank 1 = lowest ratio, that is, the region "to look at first" in the report. const rank = (field) => new Map([...rows].sort((x, y) => x[field] - y[field]) .map((s, i) => [s.b, i + 1])); const rA = rank("a"), rB = rank("c"); const gen = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join(""); const y = gen([9, 13, 10, 8, 10, 8, 8]); console.log(y(["region", "subscribers", "A ratio", "A rank", "B ratio", "B rank", "shift"])); for (const s of rows) { console.log(y([s.b, s.n, s.a.toFixed(4), rA.get(s.b), s.c.toFixed(4), rB.get(s.b), rB.get(s.b) - rA.get(s.b)])); } const first = (field) => [...rows].sort((x, y2) => x[field] - y2[field]); const spread = (field) => Math.max(...rows.map((s) => s[field])) - Math.min(...rows.map((s) => s[field])); console.log(`\nThe first region to look at under definition A is ${first("a")[0].b}, under B ${first("c")[0].b}`); console.log(`Under B the gap separating the first two regions is ${(first("c")[1].c - first("c")[0].c).toFixed(4)}, ` + `the spread across all five regions is ${spread("c").toFixed(4)}`); ``` ``` region subscribers A ratio A rank B ratio B rank shift north 929 0.7976 3 0.7277 3 0 south 913 0.7985 4 0.7240 1 -3 east 999 0.8248 5 0.7648 5 0 west 969 0.7905 1 0.7245 2 1 central 990 0.7970 2 0.7525 4 2 The first region to look at under definition A is west, under B south Under B the gap separating the first two regions is 0.0005, the spread across all five regions is 0.0408 ``` The head of the list changed: the first region to look at is west under definition A and south under definition B. Central moved up two places, south down three. East stayed the best region under both definitions: the shift does not happen on every row, but it happens on the row where the decision is made. The reason is written on the last row. The ratios of the five regions spread over a range of only 0.0408 under definition B. The difference between the two definitions, on the other hand, is 6.3 points across the network: the difference between 80.2% and 73.9% in the second table. **When the difference between groups is smaller than the shift a definition change produces, the ranking is a property of the definition and not of the regions.** The gap separating the first two regions under B is 0.0005; a single meter in the south sending a reading (662 instead of 661) reverses the ranking. Directing the field crew by the head of this list is directing it by a rounding difference. ## The Number Without Its Decisions The number this lesson produced is 3,548: the count of subscribers who sent a reading within the last 30 days. Three decisions stand next to it, and the number cannot be reported without all three written down: - **Which column is the criterion.** The behavior of the meter was taken as the criterion, not the contract record. Reason: if the question is really "how many subscribers can be billed," there is no bill without a reading. - **The window is 30 days.** Reason: the reading cycle is monthly, and between 30 and 60 the result moves by 106 subscribers, so the choice does not determine the result in this range. - **Contract status was not filtered.** The 51 subscribers whose contract is closed stayed inside the count because they sent a reading. Reason: in the question of whether a bill can be issued these are counted too; that the closure was not processed is a separate problem. When these three lines are not written, what is left is the number 3,548, and that number is not a finding but a choice. **A number without its decisions counts as unmeasured.** This rule will continue throughout the curriculum and will be applied in the same form in every lesson: the reported number, the decisions taken while producing it, the difference that appears when a decision is changed. A second conclusion follows, and it is more uncomfortable. There is no such thing as "the data speaks." In this table there are three correct numbers and none of them refutes another; what determines which one will be reported is not the data but what the question is being asked for. If a meter replacement list is to be drawn from the same table, the 645 subscribers on which A and B disagree are the target; if a bill run is to be made, it is B; if a contract count is to be reported, it is C. The job of analytics is not to choose one of these three numbers but to write down which of them corresponds to which question. ## Summary - Data analytics is the work of making a question measurable; a number is often determined not during the calculation but during the definition. - From a single subscriber table, "active subscriber" gives three different numbers under three defensible definitions: 3,849 (contract record active), 3,548 (reading in the last 30 days), 4,211 (contract open). - Totals being close does not show agreement: the totals of A and B differ by 301, but the number of subscribers they disagree on at the record level is 645. - A threshold inside a definition is also a decision; when the window is widened from 14 days to 90 days, the count rises from 1,859 to 3,750. The defensibility of a threshold is measured by how much the result fluctuates around it. - A definition decision moves the ranking as well: the first region to look at is west under definition A and south under definition B, and under B a single meter's reading separates the first two regions. - If a number does not have next to it the decision that produced it, that number is a choice and not a finding; a number without its decisions counts as unmeasured. ## Next Step This lesson made a single question measurable: **how many active subscribers are there.** The question itself was not questioned. Yet the questions that can be put to the same table are not of a single kind. "How many subscribers are there" describes the past; "why did the subscriber count fall" asks for a cause; "how many subscribers will there be next quarter" asks for a forecast; "which subscriber should be visited first" recommends an action. All four can be put to the same table, but all four cannot be answered with the same data. The next lesson separates these four kinds of question and counts how many in a set of questions can be answered with the dataset at hand.