---
title: 'State Splitting'
source: 'https://academia.sh/en/courses/infrastructure-as-code/state-splitting'
course: 'Infrastructure as Code'
language: en
updated: '2026-09-13T05:08:25+00:00'
license: 'CC BY-SA 4.0'
objectives:
    - 'Measures plan duration and blast radius in infrastructure running on a single state'
    - 'Repeats the same measurements after state is split'
    - 'Counts cross-part dependencies and compares two ways of resolving them'
    - 'Computes how many parts reapply when one part changes'
---

# State Splitting

The plan duration and blast radius of infrastructure growing in a single state, the same measurements in split form, the count of cross-part dependencies that is the cost of splitting, the difference between reading an output and querying a data source, and how many parts reapply when one part changes.

The lock rescued correctness but also built a queue: everyone writing to the same state file
waits. The only way to shrink the area a lock covers is to split the state. This lesson puts the
numbers from before and after splitting side by side, then works out the bill for splitting.

## Infrastructure Growing in a Single State

The fictional measurement network grew over the course of a year. **SM13.** The model builds a
structure that can be split into thirteen parts: network, data, compute and observability layers
in each of the three environments, plus a regional backbone shared across all three.

```js
// network-gen.mjs — the grown regional measurement network: thirteen parts, items and dependency edges.
// Output is written in two forms: a single state file, and one state file per part.
import { writeFileSync, mkdirSync, rmSync } from "node:fs";

const environments = ["test", "staging", "production"];
const layers = ["network", "data", "compute", "observability"];
const parts = [{ name: "shared", layer: "shared", count: 20 }];
for (const e of environments)
  for (const l of layers) parts.push({ name: `${e}_${l}`, env: e, layer: l, count: 50 });

// How many items a layer pulls information from other layers through.
const crossDeps = {
  network: [["shared", 2]], data: [["network", 3]],
  compute: [["network", 6], ["data", 4]], observability: [["compute", 5], ["data", 2]],
};
const targetPart = (p, layer) => (layer === "shared" ? "shared" : `${p.env}_${layer}`);

const items = [], innerEdges = [], crossEdges = [];
for (const p of parts) {
  for (let i = 0; i < p.count; i++) {
    items.push({ name: `${p.name}/i${i}`, part: p.name });
    if (i > 0) innerEdges.push([`${p.name}/i${i}`, `${p.name}/i${i - 1}`]);
  }
  for (const [layer, count] of crossDeps[p.layer] ?? [])
    for (let i = 0; i < count; i++)
      crossEdges.push([`${p.name}/i${i}`, `${targetPart(p, layer)}/i${i}`]);
}

rmSync("split", { recursive: true, force: true });
mkdirSync("split");
const item = (k) => ({ ...k, deps: [...innerEdges, ...crossEdges].filter((e) => e[0] === k.name).map((e) => e[1]) });
writeFileSync("single-state.json", JSON.stringify({ serial: 1, items: items.map(item) }));
for (const p of parts)
  writeFileSync(`split/${p.name}.json`, JSON.stringify(
    { serial: 1, items: items.filter((k) => k.part === p.name).map(item) }));

// Reality on the provider: the items in the record, plus three out-of-record resources in the production compute layer.
const provider = [...items.map((k) => k.name),
  "production_compute/manual-1", "production_compute/manual-2", "production_compute/manual-3"];
writeFileSync("provider.json", JSON.stringify(provider));

console.log(`parts: ${parts.length}, items: ${items.length}, ` +
  `inner edges: ${innerEdges.length}, cross-part edges: ${crossEdges.length}`);
console.log(`single state file: 1 x ${items.length} items | split: ` +
  `${parts.length} files, largest ${Math.max(...parts.map((p) => p.count))} items`);
console.log(`${provider.length} resources on provider, ${items.length} in record ` +
  `-> out of record ${provider.length - items.length}`);
```

```
parts: 13, items: 620, inner edges: 607, cross-part edges: 66
single state file: 1 x 620 items | split: 13 files, largest 50 items
623 resources on provider, 620 in record -> out of record 3
```

Six hundred twenty items, six hundred seventy-three edges. Only sixty-six of the edges cross a
part boundary — the rest stay inside their parts. This ratio says why splitting is possible at
all: ninety percent of the links are already local.

The last line sets up a difference that stays constant through the rest of the lesson: there are
six hundred twenty-three resources on the provider, six hundred twenty in the record. Three
resources are in no part's state at all. Splitting does not change this number — infrastructure
the record does not cover stays out of scope no matter how many files the record is split into.
The only effect it will feel below is which connection style accidentally pulls those three items
back in.

## Plan Duration and Blast Radius

A plan refreshes every item in the record from the provider and compares it against the
definition. **SM14.** One refresh read per item takes one millisecond; the real duration depends
on the provider and network latency, but it stays linear in the number of items.

The measurement rule is the same as the previous lesson's: the measured duration is not printed,
only whether it falls between 0.7 and 2.0 times the expected value.

```js
// plan.mjs — reads the state of the given scope, refreshes each item, and measures plan duration.
// Everything printed is run-independent: the refresh count and the expected duration derived
// from it, plus a band check on the measured duration. Raw milliseconds are not printed.
// usage: node plan.mjs single | node plan.mjs part <name>
import { readFileSync } from "node:fs";
const REFRESH_MS = 1, LOW = 0.7, HIGH = 2.0;     // modeled cost of a refresh read, and the band
const wait = (ms) => { const b = Date.now() + ms; while (Date.now() < b); };
const [scope, name] = [process.argv[2] ?? "single", process.argv[3]];
const file = scope === "single" ? "single-state.json" : `split/${name}.json`;

const t0 = Date.now();
const d = JSON.parse(readFileSync(file, "utf8"));
for (const k of d.items) wait(REFRESH_MS);       // one refresh read per item
const measured = Date.now() - t0;
const expected = d.items.length * REFRESH_MS;

console.log(`${(scope === "single" ? "single state" : name).padEnd(24)} | ` +
  `${String(d.items.length).padStart(3)} refreshes, expected plan ${String(expected).padStart(3)} ms | ` +
  `measured in band: ${measured >= LOW * expected && measured <= HIGH * expected ? "yes" : "no"} | ` +
  `items locked ${String(d.items.length).padStart(3)}`);
```

The work done is the same in both scopes: changing a single item in the observability layer.

```sh
node plan.mjs single
node plan.mjs part production_observability
```

```
single state             | 620 refreshes, expected plan 620 ms | measured in band: yes | items locked 620
production_observability |  50 refreshes, expected plan  50 ms | measured in band: yes | items locked  50
```

The carrier of the measurement is the **refresh count**; that number depends not on the run but
on the scope's item count. Changing one item in the single state requires six hundred twenty
refreshes, fifty in the split form. Against one changed item, the wasted-read ratio is above
ninety-nine percent. Expected plan duration is derived from this number — six hundred twenty
milliseconds against fifty — and the measurement stayed inside the band in both scopes; duration
tracks the refresh count directly.

**Blast radius** is sharper still, and does not depend on time at all: the lock covers all six
hundred twenty items. Someone adding a threshold to the observability layer holds all three
environments' network, data store and archive locked for the duration of the plan.

In split form, the same work refreshes fifty items and locks fifty items. Plan duration drops to
one-twelfth, blast radius drops from one hundred percent to eight percent. Everyone touching the
other twelve parts can work at the same time; the concurrency ceiling that fell to one in the
previous lesson rises to thirteen.

The difference accumulates in daily work. If the team triggers an apply twenty times a day, total
plan duration in the single state climbs to twelve seconds, and all of that time passes under
lock; in split form, most of the same twenty applies land on different parts, so each one is both
shorter and does not wait on the others. The real meaning of the measurement is not duration, it
is **how many people are waiting**: in the single state, every apply keeps the whole team waiting;
in split form, only the people touching the same part.

Another effect is the readability of the plan output. Finding the one changed item in a plan that
refreshed six hundred twenty items is harder than finding it in a plan of fifty; drift gets lost
in the same pile. As scope shrinks, the odds that the plan actually gets read go up, and a read
plan is the one thing that can still be stopped before it is applied.

## The Cost of Splitting

Splitting does not eliminate dependencies, it makes them visible. Each of the sixty-six edges that
cross a part boundary now has to be built between two separate state files.

```js
// dependency.mjs — counts cross-part edges, how they are resolved, and reapply propagation.
import { readFileSync, readdirSync } from "node:fs";

const parts = readdirSync("split").map((f) => f.replace(".json", ""));
const state = {}, itemPart = new Map();
for (const p of parts) {
  state[p] = JSON.parse(readFileSync(`split/${p}.json`, "utf8")).items;
  for (const k of state[p]) itemPart.set(k.name, p);
}
// Parts that publish outputs: they broadcast their values by name. Others can only be queried.
const outputParts = new Set(parts.filter((p) => /(^shared$|_network$|_data$)/.test(p)));

const edges = [];
for (const p of parts)
  for (const k of state[p])
    for (const b of k.deps) { const h = itemPart.get(b); if (h !== p) edges.push([p, h]); }
const byOutput = edges.filter(([, h]) => outputParts.has(h)).length;

console.log(`cross-part dependencies: ${edges.length} edges, ` +
  `${new Set(edges.map((e) => e.join(">"))).size} distinct part pairs`);
console.log(`  resolved by reading an output: ${byOutput}, ` +
  `resolved by data source: ${edges.length - byOutput}`);
const incoming = {};
for (const [, h] of edges) incoming[h] = (incoming[h] ?? 0) + 1;
console.log(`  most connected: ` + Object.entries(incoming).sort((a, b) => b[1] - a[1])
  .slice(0, 3).map(([p, n]) => `${p} ${n} edges`).join(", "));

// A data source returns everything matching the query: the record's uncovered resource too.
const provider = JSON.parse(readFileSync("provider.json", "utf8"));
const returned = provider.filter((a) => a.startsWith("production_compute/")).length;
const recorded = state["production_compute"].length;
console.log(`  data source query (production_compute): ${returned} items returned, ` +
  `recorded ${recorded}, out of record ${returned - recorded}`);
console.log(`  same link built by reading an output: ${recorded} items, out of record 0`);

const dependents = {};                           // target part -> parts that depend on it
for (const [k, h] of edges) (dependents[h] ??= new Set()).add(k);
const propagate = (p) => {
  const seen = new Set([p]), stack = [p];
  while (stack.length) {
    const x = stack.pop();
    for (const y of dependents[x] ?? []) if (!seen.has(y)) { seen.add(y); stack.push(y); }
  }
  return seen;
};
for (const p of ["production_observability", "production_data", "production_network", "shared"]) {
  const g = propagate(p);
  const items = [...g].reduce((t, x) => t + state[x].length, 0);
  console.log(`  ${p.padEnd(24)} changes: ${String(g.size).padStart(2)} parts reapplied, ` +
    `${String(items).padStart(3)} items refreshed, ${String(g.size).padStart(2)} locks taken`);
}
```

**SM15.** The network and data layers, along with the regional backbone, publish their values by
name as **outputs**; the compute layer does not publish, and connecting to it requires a **data
source** query. **SM16.** Three resources the record does not cover sit in the production compute
layer.

```
cross-part dependencies: 66 edges, 18 distinct part pairs
  resolved by reading an output: 51, resolved by data source: 15
  most connected: production_network 9 edges, staging_network 9 edges, test_network 9 edges
  data source query (production_compute): 53 items returned, recorded 50, out of record 3
  same link built by reading an output: 50 items, out of record 0
  production_observability changes:  1 parts reapplied,  50 items refreshed,  1 locks taken
  production_data          changes:  3 parts reapplied, 150 items refreshed,  3 locks taken
  production_network       changes:  4 parts reapplied, 200 items refreshed,  4 locks taken
  shared                   changes: 13 parts reapplied, 620 items refreshed, 13 locks taken
```

Sixty-six edges spread across eighteen part pairs. Fifty-one are resolved by **reading an
output**: the target part publishes a value by name, the connecting part reads that name. The
link is recorded and both sides are known; its cost is a version bond — if the target part removes
an output, every edge that depends on it breaks at the same time, and the break is only learned at
apply time.

The distribution of edges says something too: the three most connected parts are the network
layers of the three environments, and each carries nine edges. These three parts are the split's
breaking point. A naming change in the network layer's output list is written in a single part,
felt across nine edges, and forces three parts to reapply. Output names are therefore the parts'
**interface**; resource names inside a part can change freely, output names cannot.

The remaining fifteen are resolved by **data source**: the target part has no idea it is being
connected to, the connecting side queries the provider instead. The measurement shows the cost of
this path: the query returned fifty-three items, and fifty are in the record. Three items are
resources the record does not cover, and the data source pulls them in too. If the same link had
been built by reading an output, the returned item count would be fifty, out of record zero. A
query reads **reality**, an output reads **the record**; the two diverge by exactly these three
items.

## Reapply Propagation

The last four lines are the real bill for splitting. **SM17.** When a part's output changes, the
parts that depend on it reapply, and this is transitive; whoever depends on the dependent gets in
line too. Changing the observability part forces one part to reapply; changing the network part
forces four; changing the regional backbone forces all thirteen.

The last line deserves a careful read: when the shared part changes, six hundred twenty items
refresh — the exact same number as the single state file — but this time thirteen separate locks
are taken and thirteen applies are put in a queue. Splitting does not cheapen a wide change; it
**makes it more expensive**. In the single state, one apply did everything in the right order; in
split form, the person doing the work is now the one who has to build that order.

From this comes a rule: where to split is chosen by **frequency of change**. A layer that changes
often should stay in its own part so blast radius stays small; a layer that changes rarely and
that everyone connects to should be collected into a single part so its propagation is not
triggered often. Split in the wrong place and two harms combine: the count of cross-part edges
grows, and every change propagates to many parts.

## The Multiplication of the Record

Splitting multiplies the record too, and that is the record's own risk. One file became thirteen:
thirteen locks, thirteen access points, thirteen separate loss scenarios. The previous lesson's
measurement applies to each of them separately. Losing one part's state only makes that part
unmanageable — which is a gain — but the loss becomes harder to notice, because the other twelve
parts keep applying without a problem.

The second risk is the boundary itself. The part boundary is written down in no state file; which
item belongs to which part is visible only in how the files are distributed. If an item ends up in
two parts' definitions by mistake, two records claim ownership of the same resource, and the two
applies take turns undoing each other's work. The lock does not prevent this: two different files
have two different locks.

The third risk is **changing** the boundary. Splitting is not a one-time act; over time an item
turns out to sit in the wrong part and has to be moved. Moving is not copying the definition from
one file to another. The moment the item is removed from the old part's definition, that part's
plan shows it as **to be deleted**; the new part's plan shows it as **to be created**. Both plans
are correct on their own, but if they are applied one after the other, the resource is really
deleted and recreated. The right way is to remove the item's record from the old state by id and
add it to the new state by id — the reverse and continuation of the import from the first lesson.
Because this touches two files at once, it needs both locks at the same time, and if it is caught
in between, the item shows up in no record at all: the count of resources the record does not
cover rises from three to four.

## Summary

- Of the six-hundred-twenty-item network's six hundred seventy-three edges, only sixty-six cross
  a part boundary; ninety percent of the links are already local.
- In the single state, a one-item change refreshed six hundred twenty items and locked six
  hundred twenty items; in split form, the same work refreshed fifty items and locked fifty
  items.
- Of the sixty-six cross-part edges, fifty-one are resolved by reading an output, fifteen by
  data source; the data source query returned fifty-three items, three of which were outside the
  record.
- The number of parts that reapply when one part changes ranges from one to thirteen; changing
  the regional backbone requires thirteen locks and six hundred twenty refreshes.
- Splitting shrinks the lock, grows coordination; where to split is chosen by frequency of
  change.
- Thirteen files mean thirteen loss scenarios, and the part boundary is written down in no state
  file.

## Next Step

This lesson's split was made by layer: network, data, compute, observability. Environment was
also in the part names, but how the environment distinction itself is built was never examined.
If the definition of the three environments is the same, does the same text have to be written
three times, or does a single definition keep three instances? Both setups have a cost, and it
collects in the same place: whether the person starting the apply knows which environment they
are in. The next lesson builds the three environments in two separate forms, measures the risk of
applying to the wrong environment by step count, counts environment-specific differences, and
compares how many environments are forced to change together when the definition changes.