Data Types

The previous topic showed that making a question measurable is the real decision: when the definition of “active subscriber” changed, the number moved without a single calculation changing. But across those four lessons one thing was never asked — what kind of thing the data itself is, the data the definition works on. Can a field be averaged, and if it is, what does the result mean; is the difference between two of its values a magnitude or only a difference of labels.

This question looks like a technical detail and is not one. In a dataset every field can sit there as a number; a calculator will return the average of all of them and produce an error message on none. A field with the wrong type assigned shows itself not with an error but with a number that means nothing. This lesson measures that number in three different forms.

• SD1. The dataset is the subscriber records of the regional measurement network: 1,400 rows, eight fields, five regions. The set and the numbers are fictional; every value below is the output of a model. The same set is used in all the lessons of this topic.
• SD2. Quantitative data reports a magnitude: the difference between two values is an amount. Qualitative data reports a category; the values are distinct but the difference between them is not an amount.
• SD3. If a quantitative field is discrete, it takes only countable values and the in-between values have no counterpart; if it is continuous, there is always another value between any two, and measurement is cut off at some precision.
• SD4. A field’s storage type and its data type are not the same thing. A field stored as an integer can be qualitative; a field stored with decimals can in fact be discrete.
• SD5. The tariff tier thresholds are 10, 25, and 40 m³. These thresholds are data; on this page they are not measured but given to the model as input.

Storage Type and Data Type Are Not the Same Thing

// dataset.mjs — regional measurement network record: each row is a subscriber, its fields
// are a meter reading, a field visit, and a satisfaction score. A MODEL, the dataset is fictional.
export const SEED = 20260218, RECORD_COUNT = 1400;
const mix = (x) => { x = Math.imul(x ^ (x >>> 16), 2246822507);
  x = Math.imul(x ^ (x >>> 13), 3266489909); return (x ^ (x >>> 16)) >>> 0; };
const rng = (t) => { let s = mix(t);
  return () => (s = (s * 1664525 + 1013904223) >>> 0) / 4294967296; };
const pick = (u, w) => { let t = 0;
  for (let i = 0; i < w.length; i++) { t += w[i]; if (u < t) return i; } return w.length - 1; };

export const REGION = [
  { name: "north", code: 4, share: 0.28, base: 21, sat: [0.05, 0.10, 0.25, 0.40, 0.20] },
  { name: "south", code: 3, share: 0.22, base: 17, sat: [0.02, 0.08, 0.30, 0.45, 0.15] },
  { name: "east", code: 2, share: 0.18, base: 26, sat: [0.20, 0.08, 0.05, 0.12, 0.55] },
  { name: "west", code: 1, share: 0.14, base: 14, sat: [0.10, 0.15, 0.25, 0.30, 0.20] },
  { name: "central", code: 5, share: 0.18, base: 23, sat: [0.06, 0.12, 0.28, 0.34, 0.20] },
];

export const RECORDS = Array.from({ length: RECORD_COUNT }, (_, i) => {
  const r = rng(SEED + i);
  const b = REGION[pick(r(), REGION.map((x) => x.share))];
  const consumption = Math.max(1.2, b.base * Math.exp((r() + r() + r() - 1.5) * 0.62));
  return { subscriber_id: 10001 + i, region: b.name,
    consumption_m3: Math.round(consumption * 100) / 100,
    meter_count: pick(r(), [0.62, 0.24, 0.10, 0.04]) + 1,
    visit_count: pick(r(), [0.38, 0.32, 0.16, 0.08, 0.04, 0.02]),
    satisfaction: pick(r(), b.sat) + 1,
    temperature_c: Math.round((6 + r() * 22) * 10) / 10,
    days_elapsed: Math.floor(Math.pow(r(), 1.9) * 210) };
});

// Coding A: the region codes fixed in the subscriber register (each REGION row carries its code).
export const CODE_A = Object.fromEntries(REGION.map((b) => [b.name, b.code]));
export const mean = (a) => a.reduce((x, y) => x + y, 0) / a.length;
const distinct = (f) => new Set(RECORDS.map(f)).size;

export const FIELDS = [
  { name: "subscriber_id", stored: "integer", kind: "qualitative", measure: "-", f: (k) => k.subscriber_id },
  { name: "region_code", stored: "integer", kind: "qualitative", measure: "-", f: (k) => CODE_A[k.region] },
  { name: "satisfaction", stored: "integer", kind: "qualitative", measure: "ordinal", f: (k) => k.satisfaction },
  { name: "meter_count", stored: "integer", kind: "quantitative", measure: "discrete", f: (k) => k.meter_count },
  { name: "visit_count", stored: "integer", kind: "quantitative", measure: "discrete", f: (k) => k.visit_count },
  { name: "days_elapsed", stored: "integer", kind: "quantitative", measure: "discrete", f: (k) => k.days_elapsed },
  { name: "consumption_m3", stored: "decimal", kind: "quantitative", measure: "continuous", f: (k) => k.consumption_m3 },
  { name: "temperature_c", stored: "decimal", kind: "quantitative", measure: "continuous", f: (k) => k.temperature_c },
];

const gen = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j])
  : String(v).padEnd(B[0]))).join("");

if (import.meta.url === `file://${process.argv[1]}`) {
  console.log(`model (seed ${SEED}): ${RECORD_COUNT} subscriber records, ${REGION.length} regions, ` +
    `${FIELDS.length} fields; all eight fields are stored as numbers`);
  const y = gen([16, 11, 14, 12, 10, 12]);
  console.log("\n" + y(["field", "stored", "type", "measure", "distinct", "mean"]));
  for (const a of FIELDS) {
    console.log(y([a.name, a.stored, a.kind, a.measure, distinct(a.f),
      mean(RECORDS.map(a.f)).toFixed(2)]));
  }
  const meaningful = FIELDS.filter((a) => a.kind === "quantitative").length;
  console.log(`\nmeans were computed for all eight fields; ` +
    `${meaningful} are quantitative, ${FIELDS.length - meaningful} are not`);
}

model (seed 20260218): 1400 subscriber records, 5 regions, 8 fields; all eight fields are stored as numbers

field                stored          type     measure  distinct        mean
subscriber_id       integer   qualitative           -      1400    10700.50
region_code         integer   qualitative           -         5        3.19
satisfaction        integer   qualitative     ordinal         5        3.56
meter_count         integer  quantitative    discrete         4        1.55
visit_count         integer  quantitative    discrete         6        1.20
days_elapsed        integer  quantitative    discrete       209       69.39
consumption_m3      decimal  quantitative  continuous      1099       21.31
temperature_c       decimal  quantitative  continuous       221       16.98

means were computed for all eight fields; 5 are quantitative, 3 are not

The last column is the real finding of this lesson: an average came out for all eight fields. The average of the subscriber_id field is 10,700.50, and this number is correct — it really is the arithmetic mean of the 1,400 subscriber numbers. The same number has no counterpart at all: there is no such concept as an average subscriber number, because the difference between subscriber numbers is not an amount but a distinction. The number was computed, computed correctly, and there is nothing being measured.

The distinct column is a hint for guessing the type but does not decide it. The region_code field takes five distinct values, and from this one can sense that it is qualitative. meter_count also takes only four values, yet it is quantitative and discrete: a household with two meters really does have one meter more than a household with one. What makes the distinction is not the number of values but whether the difference between values reports an amount. The consumption_m3 field takes 1,099 distinct values; this shows not that the field is continuous but that the measurement was cut off at two decimal places. A continuous quantity always looks discrete the moment it is stored; continuity is a property not of the stored value but of the thing measured.

The satisfaction field is in a third situation. Its five values are ordered — a score of 4 is better than a score of 3 — but no measurement says that the distance between 4 and 3 equals the distance between 2 and 1. This lesson counts it as qualitative and notes that it is ordered; which summaries ordering permits is the subject of the next lesson.

The Number a Wrongly Typed Field Produces

Type assignment is not a labeling chore; it moves the number that goes into the report directly. The block below measures three wrong assignments in turn.

// type-error.mjs — the two numbers produced when the same field is read as two types, side by side.
import { RECORDS, REGION, CODE_A, mean } from "./dataset.mjs";

// Coding B: regions are numbered from largest to smallest by record count.
const counts = Object.fromEntries(REGION.map((b) => [b.name,
  RECORDS.filter((k) => k.region === b.name).length]));
const CODE_B = Object.fromEntries(Object.keys(counts)
  .sort((x, y) => counts[y] - counts[x]).map((name, j) => [name, j + 1]));
const gen = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j])
  : String(v).padEnd(B[0]))).join("");

console.log("region_code is a qualitative field; both codings are defensible");
const y1 = gen([10, 9, 9, 9]);
console.log(y1(["region", "records", "code A", "code B"]));
for (const name of Object.keys(counts)) console.log(y1([name, counts[name], CODE_A[name], CODE_B[name]]));
const mA = mean(RECORDS.map((k) => CODE_A[k.region]));
const mB = mean(RECORDS.map((k) => CODE_B[k.region]));
console.log(`code A mean ${mA.toFixed(2)}, code B mean ${mB.toFixed(2)}, ` +
  `difference ${(mB - mA).toFixed(2)} — not a single record changed in the data`);

// Discrete field: the mean and the distribution side by side.
const z = RECORDS.map((k) => k.visit_count), mz = mean(z);
const VALUES = [0, 1, 2, 3, 4, 5];
const count = (v) => z.filter((x) => x === v).length;
// Decimal equality is not tested directly; the 0.005 threshold is half a percent of one visit
// and in this set can never count two different integer values as the same.
const EPS = 0.005, equal = z.filter((v) => Math.abs(v - mz) < EPS).length;
const below = z.filter((v) => v < mz).length;
console.log(`\nvisit_count: mean ${mz.toFixed(2)}, median ` +
  `${[...z].sort((a, b) => a - b)[z.length / 2]}, mode ` +
  `${VALUES.reduce((a, v) => (count(v) > count(a) ? v : a), 0)}`);
const y2 = gen([10, 9, 10, 12]);
console.log(y2(["visits", "records", "percent", "cumulative"]));
let cum = 0;
for (const v of VALUES) { cum += count(v);
  console.log(y2([v, count(v), (100 * count(v) / z.length).toFixed(1), cum])); }
console.log(`records equal to the mean ${equal}; records below the mean ${below}, ` +
  `${(100 * below / z.length).toFixed(1)}%`);

// Continuous field read as discrete: rounding to 5 m3 steps.
const THRESHOLDS = [10, 25, 40];
const tier = (v) => (v < THRESHOLDS[0] ? 1 : v < THRESHOLDS[1] ? 2 : v < THRESHOLDS[2] ? 3 : 4);
const raw = RECORDS.map((k) => k.consumption_m3), rounded = raw.map((v) => Math.round(v / 5) * 5);
const shifted = raw.filter((v, i) => tier(v) !== tier(rounded[i])).length;
const sum = (a) => a.reduce((x, y) => x + y, 0);
console.log(`\nconsumption_m3 is continuous; rounded to 5 m3 steps the total becomes ` +
  `${sum(rounded).toFixed(1)} m3 instead of ${sum(raw).toFixed(1)} m3, a difference of ` +
  `${(sum(rounded) - sum(raw)).toFixed(1)} m3 (${(1000 * (sum(rounded) - sum(raw)) / sum(raw)).toFixed(1)} per thousand)`);
console.log(`mean ${mean(rounded).toFixed(2)} instead of ${mean(raw).toFixed(2)}; ` +
  `${shifted} records change tariff tier`);
const y3 = gen([12, 9, 14, 9]);
console.log(y3(["tier", "raw", "rounded", "diff"]));
for (const b of [1, 2, 3, 4]) {
  const h = raw.filter((v) => tier(v) === b).length;
  const u = rounded.filter((v) => tier(v) === b).length;
  console.log(y3([`${b} (${b === 4 ? "40+" : `<${THRESHOLDS[b - 1]}`} m3)`, h, u, u - h]));
}

region_code is a qualitative field; both codings are defensible
region      records   code A   code B
north           408        4        1
south           282        3        2
east            240        2        4
west            209        1        5
central         261        5        3
code A mean 3.19, code B mean 2.69, difference -0.51 — not a single record changed in the data

visit_count: mean 1.20, median 1, mode 0
visits      records   percent  cumulative
0               514      36.7         514
1               442      31.6         956
2               237      16.9        1193
3               106       7.6        1299
4                62       4.4        1361
5                39       2.8        1400
records equal to the mean 0; records below the mean 956, 68.3%

consumption_m3 is continuous; rounded to 5 m3 steps the total becomes 29870.0 m3 instead of 29828.2 m3, a difference of 41.8 m3 (1.4 per thousand)
mean 21.34 instead of 21.31; 229 records change tariff tier
tier              raw       rounded     diff
1 (<10 m3)         48             3      -45
2 (<25 m3)        970           853     -117
3 (<40 m3)        348           488      140
4 (40+ m3)         34            56       22

The first block measures the average applied to a qualitative field. The region code can be given in two forms and both are defensible: numbering the regions with the codes fixed in the subscriber register, or numbering them from largest to smallest by record count. Under the first coding the average region code comes out 3.19, under the second 2.69. No record in the data has changed; the only thing that changed is the numbers attached to the labels. This is the shortest proof that the average of a qualitative field belongs not to the data but to the coding. If a report has a line saying “average region code 3.19”, that line is not a finding but the shadow of an ordering decision.

The second block shows what happens when a discrete field is treated as continuous. The average number of visits is 1.20; the calculation is correct. But none of the 1,400 records received 1.20 visits, because there is no such thing as half a visit. The number of records equal to the average is zero. Moreover, 68.3% of the records fall below this average: the largest group, at 514 records, is the subscribers who received no visit at all. The sentence “1.20 field visits per subscriber” does not describe even a single subscriber; the sentence “36.7% of subscribers received no visit at all” comes out of the same data and says something entirely different. Both are correct and the two suggest opposite decisions — one shows that the visit load is balanced, the other that a segment is never seen. For a discrete field, what goes into the report is not the average but the distribution; the average is only a summary placed next to that distribution.

The third block measures a continuous field being read as discrete, and its surprising result is here. When consumption is rounded to 5 m³ steps, the total goes from 29,828.2 m³ to 29,870.0 m³: a difference of 1.4 per thousand. The total barely stirs, because the upward and downward roundings cancel each other out. The same operation changes the tariff tier of 229 records. The 48 records in the first tier drop to 3, and the third tier rises from 348 to 488. Rounding preserves the total and corrupts the classification. Someone reporting on total consumption can say that rounding is harmless, and is right; someone reporting on the tariff tier distribution says that the same operation moves one in six records to another tier, and is also right. Which decision is harmless depends on the question being asked.

All three measurements write the same rule: a number whose type assignment is not written next to it cannot be read. Under which coding the region code average was computed, next to which distribution the visit average sits, whether the consumption average was rounded — without these three pieces of information, none of the three numbers can be defended.

Summary

• Storage type does not determine data type: all eight fields sit there as numbers, three are qualitative, and even though an average can be computed for them it does not report a magnitude.
• What separates quantitative from qualitative is not the number of values but whether the difference between two values reports an amount; meter_count is quantitative with four values, region_code is qualitative with five.
• The average of a qualitative field belongs to the coding: the same 1,400 records give 3.19 under the register coding and 2.69 under the size coding, and nothing in the data has changed.
• When a discrete field is treated as continuous, the average falls to a value with no counterpart: no record equals the 1.20 visit average, and 68.3% of the records fall below that value.
• Rounding a continuous field moves the total by 1.4 per thousand but changes the tariff tier of 229 records; whether the same decision is harmless changes with the question being asked.

Next Step

This lesson asked whether the average of a field can be taken, and was left with a half answer on the satisfaction field: its values are ordered, but nobody has measured that the distance between 4 and 3 equals the distance between 2 and 1. The same uncertainty exists in the temperature_c field too — can two temperatures be added, can one be counted as twice the other. The quantitative and qualitative distinction is not enough to answer these questions, because the distinction is too coarse. The next lesson places the fields on four measurement scales and counts which summary is legitimate on each scale: two defensible numeric codings given to an ordinal field reverse the satisfaction ranking of the regions from start to finish.