Analysis Workflow

The previous two lessons may have left an impression: first a question is chosen, then its type is determined, then one checks whether the data is enough, then the calculation is done. It looks like a sequential list. In reality this list does not run once from start to finish. The 30-day window in the first lesson became defensible only after the table was looked at and the reading cycle was seen to be monthly; the depth requirement in the second lesson gained a number only after the question set was written.

This lesson builds the workflow in five steps and shows with a number that the workflow is not linear. The measure is the number of rework loops: how many times it was necessary to return to an earlier step because of a decision seen in a later step. A second number is placed next to it: how many times in total the five steps of the workflow were run.

• AC12. The workflow is five steps: question, data, analysis, finding, communication. Both the workflow and the decisions are fiction; the decisions are taken from the subscriber counts of the previous two lessons.
• AC13. A rework loop is returning to the step a decision belongs to, when that decision is seen in a later step, and rerunning the steps in between.
• AC14. A step visit is counted as follows: if the workflow runs once from start to finish it is 5 visits; a loop from step $f$ back to step $a$ adds $f - a + 1$ visits. This is an accounting decision and will be changed at the end of the lesson.
• AC15. A decision counts as writable in advance if it can be written in the step it belongs to, without looking at the output of a later step. If it cannot be written, the content of the decision is learned only in a later step.
• AC16. Time is not measured. What is counted is loops and visits.

Five Steps

The question step makes the thing to be asked measurable: which definition, which reference date, which decision the result will feed, who will read it. The whole subject of the first lesson was this step.

The data step finds the fields the question asks for and sees the real state of the dataset: which field is filled, which row is duplicated, which value is missing. The inventory of the second lesson is the output of this step.

The analysis step filters, groups, and summarizes; the number comes out here. Table design and query writing were established in the Databases curriculum and are not repeated here; what is measured in this step is not syntax but the filtering and grouping decisions.

The finding step turns the number into a sentence: breakdown, ranking, comparison. The communication step ties the finding to a decision and writes it for the party who will read it. The last two steps do not produce decisions, they consume them, and for exactly this reason a missing decision seen there sends the workflow back to the very beginning.

Where Decisions Are Born, Where They Are Seen

// workflow.mjs — a five-step analysis workflow and the decisions that send it back. A MODEL:
// the decisions are taken from the subscriber counts of the previous two lessons, the workflow is fiction.
export const STEPS = ["question", "data", "analysis", "finding", "communication"];

// home: the step in which the decision has to be written. seen: the step in which the decision is noticed.
// advance: whether the decision can be written in its own step without looking at the data.
export const DECISIONS = [
  { name: "active subscriber definition", home: 1, seen: 1, advance: true },
  { name: "reference date", home: 1, seen: 1, advance: true },
  { name: "reading window 30 days", home: 1, seen: 2, advance: false },
  { name: "readings from closed contracts", home: 1, seen: 2, advance: false },
  { name: "region breakdown requested", home: 1, seen: 4, advance: true },
  { name: "who will read the report", home: 1, seen: 5, advance: true },
  { name: "comparison period", home: 1, seen: 5, advance: true },
  { name: "rows with empty region", home: 2, seen: 3, advance: false },
  { name: "duplicate subscriber record", home: 2, seen: 3, advance: false },
  { name: "tier field: text or number", home: 2, seen: 3, advance: true },
  { name: "ranking criterion: ratio or count", home: 3, seen: 4, advance: true },
  { name: "ranking is volatile near threshold", home: 3, seen: 4, advance: false },
];

// Decisions whose content is learned by looking at the raw data: a preview pulls these forward.
export const FROM_DATA = new Set(["reading window 30 days", "readings from closed contracts",
  "rows with empty region", "duplicate subscriber record"]);

// A rework loop is going back from the seen step to the home step and rerunning what is in between.
export const distance = (k) => k.seen - k.home;
export const cost = (k) => k.seen - k.home + 1;
export const policy = (written, preview = false) => {
  const d = DECISIONS.filter((k) => distance(k) > 0 && !(written && k.advance)
    && !(preview && FROM_DATA.has(k.name)));
  return { loops: d.length, list: d,
    visits: STEPS.length + (preview ? 1 : 0) + d.reduce((s, k) => s + cost(k), 0) };
};

const gen = (B) => (h) => h.map((v, j) => (j ? String(v).padStart(B[j])
  : String(v).padEnd(B[0]))).join("");

if (import.meta.url === `file://${process.argv[1]}`) {
  console.log(`workflow of ${STEPS.length} steps: ${STEPS.join(" -> ")}`);
  console.log(`${DECISIONS.length} decisions need to be written\n`);
  const y = gen([40, 12, 17, 9, 12]);
  console.log(y(["decision", "belongs to", "seen at", "distance", "in advance"]));
  for (const k of DECISIONS) {
    console.log(y([k.name, `${k.home} ${STEPS[k.home - 1]}`, `${k.seen} ${STEPS[k.seen - 1]}`,
      distance(k), k.advance ? "yes" : "no"]));
  }
  const g = DECISIONS.filter((k) => distance(k) > 0);
  console.log(`\n${g.length} of ${DECISIONS.length} decisions are seen after the step they belong to`);
  console.log(`of these, ${g.filter((k) => FROM_DATA.has(k.name)).length} are learned only by ` +
    `looking at the raw data: ${g.filter((k) => FROM_DATA.has(k.name)).map((k) => k.name).join("; ")}`);
}

workflow of 5 steps: question -> data -> analysis -> finding -> communication
12 decisions need to be written

decision                                  belongs to          seen at distance  in advance
active subscriber definition              1 question       1 question        0         yes
reference date                            1 question       1 question        0         yes
reading window 30 days                    1 question           2 data        1          no
readings from closed contracts            1 question           2 data        1          no
region breakdown requested                1 question        4 finding        3         yes
who will read the report                  1 question  5 communication        4         yes
comparison period                         1 question  5 communication        4         yes
rows with empty region                        2 data       3 analysis        1          no
duplicate subscriber record                   2 data       3 analysis        1          no
tier field: text or number                    2 data       3 analysis        1         yes
ranking criterion: ratio or count         3 analysis        4 finding        1         yes
ranking is volatile near threshold        3 analysis        4 finding        1          no

10 of 12 decisions are seen after the step they belong to
of these, 4 are learned only by looking at the raw data: reading window 30 days; readings from closed contracts; rows with empty region; duplicate subscriber record

The table is read with two columns. The left column says where the decision belongs, the right column says where it is seen. Of the twelve decisions, only two are seen where they are born: the active subscriber definition and the reference date. The remaining ten emerge after the step they belong to.

The distance column says that the loops are not equal. Readings arriving from closed contracts send the workflow back one step; noticing in the communication step who will read the report sends it back four steps. Where the second one is seen has nothing to do with what the question is; it has to do with the step of the workflow at which someone first asks “who is this for.”

The five decisions seen in the last two steps are also worth noting. The finding and communication steps produce no decisions at all: 4 and 5 never appear in the belongs to column of the table. But five decisions are seen there. The steps that consume decisions are the steps that notice the absence of the decisions they did not produce.

Three Working Modes

The same workflow, the same 12 decisions, one difference: when the decisions are written. In the first mode every decision is written the moment it is seen. In the second mode, in each step, the decisions that belong to that step and can be written in advance are written before leaving the step. The third mode adds one thing to the second: before leaving the question step, the raw data is looked at once. This look costs one step visit and is not an analysis; it only shows whether the fields are filled and which values occur.

// loop.mjs — the same workflow, two working modes: rework loops and step visits side by side.
import { STEPS, DECISIONS, FROM_DATA, distance, cost, policy } from "./workflow.mjs";

const A = policy(false), B = policy(true), C = policy(true, true);
const gen = (C) => (h) => h.map((v, j) => (j ? String(v).padStart(C[j])
  : String(v).padEnd(C[0]))).join("");

const y = gen([34, 14, 11, 13, 11]);
console.log(y(["working mode", "rework loops", "visits", "per step", "farthest"]));
for (const [name, p] of [["decisions written when seen", A], ["decisions written in their step", B],
  ["written in step + preview", C]]) {
  console.log(y([name, p.loops, p.visits, (p.visits / STEPS.length).toFixed(1),
    Math.max(...p.list.map(distance))]));
}

const z = gen([17, 10, 10, 10]);
console.log("\n" + z(["returned to", "mode 1", "mode 2", "mode 3"]));
for (let a = 1; a <= STEPS.length; a++) {
  const s = (p) => p.list.filter((k) => k.home === a).length;
  console.log(z([`${a} ${STEPS[a - 1]}`, s(A), s(B), s(C)]));
}

console.log(`\ntotal visits ${A.visits} -> ${B.visits}, rework loops ${A.loops} -> ${B.loops}`);
const long = A.list.filter((k) => distance(k) >= 3);
console.log(`in mode 1, ${long.length} loops with a distance of 3 or more take up ` +
  `${long.reduce((s, k) => s + cost(k), 0)} of the ${A.visits - STEPS.length} extra visits`);
console.log(`the ${B.loops} loops left in mode 2: ${B.list.map((k) => k.name).join("; ")}`);
console.log(`these ${B.loops} decisions are all of the ${DECISIONS.filter((k) => !k.advance).length} ` +
  `decisions that cannot be written in advance; they are learned by looking, not by writing`);
console.log(`\nthe preview pulls ${FROM_DATA.size} decisions forward, at a cost of 1 visit: ` +
  `${B.loops} loops fall to ${C.loops}, ${B.visits} visits fall to ${C.visits}`);
console.log(`the only loop left in mode 3: ${C.list.map((k) => k.name).join("; ")} ` +
  `— its content is in the output of the analysis, not in the raw data`);

// A second accounting decision: a rework loop reruns only the step it belongs to, not the steps in between.
const cheap = (p) => STEPS.length + p.loops;
console.log(`\nif the accounting decision changes (only the home step reruns): ` +
  `mode 1 ${cheap(A)} visits, mode 2 ${cheap(B)} visits`);
console.log(`mode 1 with cheap accounting ${cheap(A)}, mode 2 with full accounting ${B.visits}: ` +
  `the same number, a different workflow`);

working mode                        rework loops     visits     per step   farthest
decisions written when seen                   10         33          6.6          4
decisions written in their step                5         15          3.0          1
written in step + preview                      1          8          1.6          1

returned to          mode 1    mode 2    mode 3
1 question                5         2         0
2 data                    3         2         0
3 analysis                2         1         1
4 finding                 0         0         0
5 communication           0         0         0

total visits 33 -> 15, rework loops 10 -> 5
in mode 1, 3 loops with a distance of 3 or more take up 14 of the 28 extra visits
the 5 loops left in mode 2: reading window 30 days; readings from closed contracts; rows with empty region; duplicate subscriber record; ranking is volatile near threshold
these 5 decisions are all of the 5 decisions that cannot be written in advance; they are learned by looking, not by writing

the preview pulls 4 decisions forward, at a cost of 1 visit: 5 loops fall to 1, 15 visits fall to 8
the only loop left in mode 3: ranking is volatile near threshold — its content is in the output of the analysis, not in the raw data

if the accounting decision changes (only the home step reruns): mode 1 15 visits, mode 2 10 visits
mode 1 with cheap accounting 15, mode 2 with full accounting 15: the same number, a different workflow

The five-step workflow runs 33 times in the first mode. 6.6 visits per step means that describing the workflow as “a five-step process” describes the diagram and not the work. In the second mode the number falls to 15, to 3.0 per step. The difference of 18 visits between them is gained without collecting any additional data, only by changing when the decisions are written.

Where the gain is collected is written in the third group of lines. The three loops with a distance of three or more take up 14 of the 28 extra visits in the first mode: a third of the loops carry half of the cost. All three are the same kind of decision: whether a region breakdown will be requested, who will read the report, which period it will be compared with. None of them asks anything about the data; all three ask whom and what the question is being asked for, and all three can be written in the first step without looking at any table.

The Loop That Does Not Go to Zero

In the second mode the rework loops stop at five, and all five are a property of the dataset: the reading window, readings from closed contracts, rows with an empty region, duplicate subscriber records, the ranking being volatile near the threshold. None of them can be written at a desk; that the reading cycle is monthly was seen when the cross table of the first lesson was printed, and that the ranking turns on a difference of 0.0005 was seen when the region table was printed.

The third mode pulls four of these five forward. The preview pays one visit and in return brings 15 visits down to 8 and five loops down to one. The first two steps are never returned to. The gain comes from all four decisions being visible in the raw data: an empty field, a duplicated identifier, or a date distribution is seen by looking, without any calculation.

The one remaining loop does not close, and cannot. The ranking being volatile near the threshold does not stand in the raw data; it appears after the analysis has run and the region ratios have been computed. At which step a decision can be learned depends on the content of that decision and cannot be changed by the working arrangement. The workflow is not linear; it is not linear not because of disorder but because of the order of information. This is why the advice “define the question correctly first” is only half a piece of advice: part of the definition can be written only after the data is seen, and part only after the calculation has run. What can be done is to pull each decision forward to the earliest step at which it can be learned, and to leave room, knowing that the rest will loop back.

The last two lines apply the same test to the lesson’s own number. AC14 was an accounting decision: a loop reruns all the steps in between. A second accounting decision is also defensible: only the step the decision belongs to reruns, and the steps in between are left alone if they have not been broken. Under this decision the first mode produces 15 visits and the second mode 10. That is, the first mode comes out at 15 with cheap accounting, and the second mode comes out at 15 with full accounting. Two different working modes, with two different accounting decisions, give the same number. Unless it is written which 15 it is, this number says nothing; the lesson’s own measure is also subject to the rule of the course.

Summary

• The analysis workflow is five steps: question, data, analysis, finding, communication. The finding and communication steps consume decisions rather than produce them; but they are also the steps that notice a missing decision.
• In this run, 10 of the 12 decisions are seen after the step they belong to; only two are seen where they are born.
• When decisions are written as they are seen, the workflow produces 10 rework loops and 33 step visits, 6.6 per step. When every decision is written in its own step it falls to 5 loops and 15 visits, and when a preview of the raw data is added, to 1 loop and 8 visits.
• The gain is collected in the long loops: the three loops with a distance of three or more take up 14 of the extra visits, and all three concern whom the question is asked to, not the data.
• The content of the one loop that does not close is in the output of the analysis, not in the raw data; at which step a decision can be learned cannot be changed by the working arrangement.
• The number of visits also depends on an accounting decision: the first mode gives 15 with cheap accounting and the second mode gives 15 with full accounting; the number cannot be read without writing which 15 it is.

Next Step

This lesson assumed that a single person ran the workflow. In reality the steps of the workflow are often divided among people who have different jobs, and where the division falls changes what the result will be. The same question, whether the active subscriber count is falling, turns into a number in the hands of someone charged with measuring, into a threshold in the hands of someone charged with tying the decision, and into a prediction interval in the hands of someone charged with generalizing to the future. All three read the same table, and all three deliver a different output. The next lesson separates these three job descriptions and prints side by side how the same question turns into three different numbers in three hands.