Personal Data Protection

One of the previous lesson’s four data types did not behave like the others: personal data appears in eleven systems, does not disappear through any narrowing, and even after the removal attempts, it alone carried nine of the remaining thirteen controls. The source of the difference is not prevalence. For the other types, the obligation is written to the party holding the data; for personal data, part of the obligation is given to the person the data is about, as a right. The right is exercised not on the organization’s schedule but the moment its holder wants it, and it demands an action from the organization.

This lesson counts rights and processing principles as items. No law, article, institution, or country name is written down; what is measured is not the name of the obligation but how many systems a request travels through in the organization and how many minutes it takes. The inventory from the previous lessons becomes decisive here: a request can only be searched for in the systems that are in the inventory.

• FW27. The fictional institution has 15 systems carrying personal data; three of them are not registered in the inventory. The systems, record counts, and durations are fictional.
• FW28. Every system holds one or more data classes: identity, contact, contract, transaction, location, complaint, profile.
• FW29. Seven right-request types are defined; each type searches specific data classes and touches only the systems holding those classes.
• FW30. A request’s duration is the sum of the search times of the systems it touches in the inventory, plus a fixed processing time. A system not in the inventory is not searched, and it does not enter the duration either.
• FW31. The probability that the data subject’s record exists in a system that is not in the inventory is the ratio of that system’s record count to the subscriber count. If a request has left a record in any of these systems, it has been answered incompletely.
• FW32. The request team’s capacity is 260 person-hours per month (fictional).

// personal/request.mjs — counting data subject rights as items.
// MODEL: regional measurement network (fictional); systems, record counts, and durations are fictional.
export const row = (s) => s.trim().split("\n").map((l) => l.trim().split("|"));
export const gen = (B) => (...h) => h.map((v, j) =>
  (j ? String(v).padStart(B[j]) : String(v).padEnd(B[0]))).join("");

export const SUBSCRIBERS = 420000;   // fictional subscriber count; the coverage ratio is divided by this
// system | in inventory? | records | search minutes | data classes it holds
export const SYSTEM = row(`
  subscriber-portal|yes|420000|4|identity contact contract
  billing-application|yes|380000|6|identity contract transaction
  billing-database|yes|380000|8|identity contract transaction
  call-center-recording|yes|95000|12|identity contact complaint
  field-crew-app|yes|22000|9|identity location
  work-order-service|yes|140000|7|location contract
  identity-provider|yes|31000|3|identity
  directory-service|yes|31000|3|identity contact
  reporting-warehouse|yes|1200000|25|transaction profile
  backup-store|yes|2400000|60|identity contact contract transaction
  archive-store|yes|900000|40|contract
  log-collection-pipeline|yes|5400000|18|transaction
  marketing-list|no|60000|10|contact profile
  field-spreadsheet-files|no|8000|35|location complaint
  old-call-backup|no|75000|30|contact complaint`)
  .map(([name, inv, records, minutes, cls]) => ({ name, inventoried: inv === "yes",
    records: +records, minutes: +minutes, class: cls.split(" "),
    coverage: Math.min(1, +records / SUBSCRIBERS) }));

// right | data classes searched | extra processing minutes per item | monthly requests (fictional)
export const RIGHT = row(`
  R1 copy of record|identity contact contract transaction location complaint profile|30|40
  R2 correction|identity contact|20|25
  R3 deletion|identity contact contract transaction location complaint profile|45|30
  R4 restriction of processing|transaction profile|25|6
  R5 objection to processing|contact profile|20|12
  R6 portability|contract transaction|40|5
  R7 objection to automated decision-making|profile|30|3`)
  .map(([name, s, extra, monthly]) =>
    ({ name, code: name.split(" ")[0], class: s.split(" "), extra: +extra, monthly: +monthly }));

export const touched = (h, set = SYSTEM) =>
  set.filter((s) => s.class.some((x) => h.class.includes(x)));
export const duration = (h, set = SYSTEM) =>
  h.extra + touched(h, set).filter((s) => s.inventoried).reduce((a, s) => a + s.minutes, 0);
// The probability that the data subject's record exists in a system outside the inventory is the coverage ratio.
export const missingRate = (h, set = SYSTEM) => 1 - touched(h, set)
  .filter((s) => !s.inventoried).reduce((a, s) => a * (1 - s.coverage), 1);

if (import.meta.url === `file://${process.argv[1]}`) {
  const outside = SYSTEM.filter((s) => !s.inventoried);
  console.log(`model: ${SYSTEM.length} systems (${outside.length} not in inventory), ` +
    `${RIGHT.length} right-request types, ${RIGHT.reduce((a, h) => a + h.monthly, 0)} ` +
    `requests/month (fictional)`);

  const y = gen([43, 10, 9, 14, 18, 13, 11]);
  console.log("\n" + y("right request type", "searched", "skipped", "duration min",
    "records searched", "requests/mo", "missing %"));
  for (const h of RIGHT) {
    const d = touched(h), searched = d.filter((s) => s.inventoried);
    console.log(y(h.name, searched.length, d.length - searched.length, duration(h),
      searched.reduce((a, s) => a + s.records, 0).toLocaleString("en-US"), h.monthly,
      (100 * missingRate(h)).toFixed(1)));
  }
  const missingRequests = RIGHT.reduce((a, h) => a + h.monthly * missingRate(h), 0);
  const total = RIGHT.reduce((a, h) => a + h.monthly, 0);
  console.log(`\nright types touching a system outside the inventory ` +
    `${RIGHT.filter((h) => missingRate(h) > 0).length}/${RIGHT.length}; per month ` +
    `${missingRequests.toFixed(1)}/${total} requests (${(100 * missingRequests / total).toFixed(1)}%) ` +
    `are answered incompletely`);
  console.log(`${outside.length} systems outside the inventory: ` +
    `${outside.map((s) => `${s.name} (${(100 * s.coverage).toFixed(1)}%)`).join(", ")}`);
  const costliest = [...SYSTEM].filter((s) => s.inventoried).sort((a, b) => b.minutes - a.minutes)
    .slice(0, 3);
  console.log(`three systems that set the search duration: ` +
    `${costliest.map((s) => `${s.name} ${s.minutes} min`).join(", ")}; total workload ` +
    `${(RIGHT.reduce((a, h) => a + h.monthly * duration(h), 0) / 60).toFixed(1)} person-hours/month`);
}

model: 15 systems (3 not in inventory), 7 right-request types, 121 requests/month (fictional)

right request type                           searched  skipped  duration min  records searched  requests/mo  missing %
R1 copy of record                                  12        3           225        11,399,000           40       30.9
R2 correction                                       8        2           125         3,759,000           25       29.6
R3 deletion                                        12        3           240        11,399,000           30       30.9
R4 restriction of processing                        5        1           142         9,760,000            6       14.3
R5 objection to processing                          5        2           124         4,146,000           12       29.6
R6 portability                                      8        0           208        11,220,000            5        0.0
R7 objection to automated decision-making           1        1            55         1,200,000            3       14.3

right types touching a system outside the inventory 6/7; per month 33.9/121 requests (28.0%) are answered incompletely
3 systems outside the inventory: marketing-list (14.3%), field-spreadsheet-files (1.9%), old-call-backup (17.9%)
three systems that set the search duration: backup-store 60 min, archive-store 40 min, reporting-warehouse 25 min; total workload 381.2 person-hours/month

The table turns rights from a single sentence into a set of items. The seven right types do not do the same work: copy of record and deletion search all seven data classes and travel through twelve systems, objection to automated decision-making only looks at the profile class and ends at a single system. The duration ranges from 55 minutes to 240 minutes — for the same organization, for the same person, doing work that varies more than fourfold depending on which right is exercised. What sets the duration is not the request itself but the shape of the inventory: the backup store alone takes 60 minutes and is searched by six right types at once.

The second column is the lesson’s main measure. Six of the seven right types touch a system that is not in the inventory, and that system is never searched. A system that is not searched does not enter the duration; that is why an incomplete answer gives no sign at all in the response time. The request closes on time, its holder is told “your record was found in these three systems,” and the record in the shadow system is never mentioned. 33.9 of the 121 monthly requests — 28 percent — are in this state.

This is a defect different from what the previous three lessons measured. An unevidenced requirement shows up in an audit, an out-of-scope item is written into an applicability statement — both are recorded somewhere. The gap the shadow system produces is recorded nowhere: it leaves no trace in the request record, in the duration measurement, or in the evidence file. The only way to measure it is to search separately for what is outside the inventory; it cannot be found by the inventory itself.

The rate varies by right type, and the reason is the coverage ratio of the shadow systems: the old call backup contains 17.9 percent of subscribers, the marketing list 14.3 percent, the field spreadsheet files 1.9 percent. Rights touching the contact class have two shadow systems active at once, and the incomplete-answer rate rises to 29.6 percent. Portability, on the other hand, is the one clean right: the contract and transaction classes are held in no shadow system at all.

Two Options, One Capacity

There are two ways to lower the incomplete-answer rate, and they work in opposite directions: completing the inventory (registering the shadow systems and including them in the search), or reducing the data (not holding a class at all, emptying a store whose retention period has expired).

// personal/scenario.mjs — the effect of completing the inventory and reducing data on the request load.
import { SYSTEM, RIGHT, row, gen, duration, missingRate } from "./request.mjs";

const CAPACITY = 260;   // person-hours/month (two people, fictional)
const workload = (k) => RIGHT.reduce((a, h) => a + h.monthly * duration(h, k), 0) / 60;
const missing = (k) => RIGHT.reduce((a, h) => a + h.monthly * missingRate(h, k), 0);

// The scenarios are transformations defined on the same system set.
const completeInventory = (k) => k.map((s) => ({ ...s, inventoried: true }));
const minimizeData = (k) => k.filter((s) => s.name !== "archive-store")
  .map((s) => s.name === "reporting-warehouse"
    ? { ...s, class: s.class.filter((x) => x !== "profile") } : s);

const SCENARIO = [["baseline", (k) => k], ["inventory completed", completeInventory],
  ["data minimization", minimizeData], ["both combined", (k) => minimizeData(completeInventory(k))]];

const y = gen([21, 9, 12, 11, 15, 12]);
console.log(`${RIGHT.reduce((a, h) => a + h.monthly, 0)} requests/month, capacity ` +
  `${CAPACITY} person-hours/month (fictional)`);
console.log("\n" + y("scenario", "systems", "missing/mo", "missing %", "person-hrs/mo",
  "capacity %"));
for (const [name, transform] of SCENARIO) {
  const k = transform(SYSTEM), w = workload(k), m = missing(k);
  const total = RIGHT.reduce((a, h) => a + h.monthly, 0);
  console.log(y(name, k.length, m.toFixed(1), (100 * m / total).toFixed(1), w.toFixed(1),
    (100 * w / CAPACITY).toFixed(0)));
}

// Processing principles: each principle is tied to an evidence item and a set of data classes.
// principle | data classes it covers | evidence items produced in the period
const ALL_CLASSES = "identity contact contract transaction location complaint profile";
const PRINCIPLE = row(`
  lawful basis|contact profile|120
  purpose limitation|transaction profile|4
  data minimization|${ALL_CLASSES}|4
  accuracy|identity contact|12
  retention limit|contract transaction complaint|365
  integrity and confidentiality|${ALL_CLASSES}|365
  accountability|${ALL_CLASSES}|4`)
  .map(([name, s, item]) => ({ name, class: s.split(" "), item: +item }));

const z = gen([31, 14, 16, 26, 12]);
console.log("\n" + z("processing principle", "data classes", "evidence items",
  "system outside inventory", "verifiable"));
for (const i of PRINCIPLE) {
  const outside = SYSTEM.filter((s) => !s.inventoried && s.class.some((x) => i.class.includes(x)));
  console.log(z(i.name, i.class.length, i.item, outside.length, outside.length ? "no" : "yes"));
}
const dependent = PRINCIPLE.filter((i) =>
  SYSTEM.some((s) => !s.inventoried && s.class.some((x) => i.class.includes(x))));
console.log(`\n${dependent.length} of ${PRINCIPLE.length} processing principles depend on the ` +
  `completeness of the inventory; once the inventory is completed, all ${PRINCIPLE.length} ` +
  `principles become verifiable, and the workload rises by ` +
  `${(workload(completeInventory(SYSTEM)) - workload(SYSTEM)).toFixed(1)} person-hours/month`);
console.log(`evidence items the principles produce in the period ` +
  `${PRINCIPLE.reduce((a, i) => a + i.item, 0)}; annual load of fulfilling requests ` +
  `${(workload(SYSTEM) * 12).toFixed(0)} person-hours`);

121 requests/month, capacity 260 person-hours/month (fictional)

scenario               systems  missing/mo  missing %  person-hrs/mo  capacity %
baseline                    15        33.9       28.0          381.2         147
inventory completed         15         0.0        0.0          494.8         190
data minimization           14        33.9       28.0          324.9         125
both combined               14         0.0        0.0          438.6         169

processing principle             data classes  evidence items  system outside inventory  verifiable
lawful basis                                2             120                         2          no
purpose limitation                          2               4                         1          no
data minimization                           7               4                         3          no
accuracy                                    2              12                         2          no
retention limit                             3             365                         2          no
integrity and confidentiality               7             365                         3          no
accountability                              7               4                         3          no

7 of 7 processing principles depend on the completeness of the inventory; once the inventory is completed, all 7 principles become verifiable, and the workload rises by 113.7 person-hours/month
evidence items the principles produce in the period 874; annual load of fulfilling requests 4574 person-hours

The first table shows the trade-off in its bare form. In the baseline, there is 147 percent of the team’s capacity worth of work, and requests are already falling behind — and this backlog is happening while answering 28 percent of requests incompletely. Once the inventory is completed, the incomplete-answer rate goes to zero, but the workload rises to 494.8 person-hours — 190 percent of capacity. Answering correctly is 30 percent more expensive than answering incorrectly, and this difference never shows up anywhere as a line-item cost.

Placing the second and third rows side by side is the lesson’s result. Data minimization lowers the workload to 324.9 and brings it closer to capacity, but it does not change the incomplete-answer rate at all: the items removed were already in the inventory, the shadow systems stay right where they are. The two options solve different problems. Choosing only one leaves either a correct but backlogged process, or a fast but incomplete one; applying both at once zeroes out the incomplete answers and brings the load down to 438.6 — still above capacity. There is no situation resolved without growing the capacity, and that too is a decision.

The second table shows the same gap on the evidence side. All seven of the seven processing principles touch at least one system that is not in the inventory, and none of them are in a verifiable state. Purpose limitation is caught on a single shadow system, data minimization and accountability on three at once. This does not mean the principles are not applied; it means that their being applied cannot be shown. The third lesson’s distinction applies here exactly the same way: a principle can have a control, it can even be producing an evidence item, but the scope of the evidence is bounded by the scope of the inventory. That is why, for personal data, the first item is not a control but the inventory itself; everything counted before it has been counted on top of an incomplete set.

Summary

• Data subject rights are seven separate request types and do not do the same work: copy of record and deletion travel through twelve systems and take 225–240 minutes, while objection to automated decision-making finishes at a single system in 55 minutes.
• A system that is not in the inventory is not searched and does not enter the duration; that is why an incomplete answer gives no sign at all in the response time. Six of the seven right types are in this state, and 28 percent of the 121 monthly requests are answered incompletely.
• The coverage ratio of the shadow systems determines the incomplete-answer rate: rights touching the contact class have two shadow systems active at once, and the rate rises to 29.6 percent; portability is the one clean right.
• Answering correctly is expensive: once the inventory is completed, the incomplete-answer rate goes to zero, but the monthly workload rises from 381.2 to 494.8 person-hours — 190 percent of the 260-person-hour capacity.
• Completing the inventory and reducing the data solve different problems: minimization lowers the load to 324.9 but does not change the incomplete-answer rate at all; even with both applied at once, the load stays above capacity.
• All seven of the seven processing principles are unverifiable because they touch a system that is not in the inventory: the issue is not that the principle is not applied, but that its being applied cannot be shown.

Next Step

What the last two tables have in common is that everything measured rests on a document: the inventory is a document, the retention period is written in a document, the steps of the request flow are defined in a document. Up to this lesson, documents were treated as a single mass — yet there is a sharp difference between them from an audit standpoint. One document says “we protect personal data” and proves nothing; another says “the retention period is 1,095 days” and gives a measurable threshold; a third defines a procedure step by step and produces a record every time it runs.

The next lesson separates these three levels — policy, standard, and procedure — along the axis of evidence production: which document level counts as evidence in an audit, which merely states intent, and how many links of the chain need to be complete to show that a requirement has been satisfied.