--- title: 'Store Types' source: 'https://academia.sh/en/courses/scaling-the-data/store-types' course: 'Scaling the Data Layer' language: en updated: '2026-08-23T07:01:34+00:00' license: 'CC BY-SA 4.0' --- # Store Types Measuring the key-value, document, wide-column, and graph store against the same access patterns: the query count and record count each type requires for a pattern, which pattern the structure that cheapens one makes expensive, and the cost the choice charges to the introductory course's store request rate and write/read ratio. Every decision up to this point was made on a single store shape with rows and columns. Yet the measured patterns want very different things: the tracking query wants a single record by a single key, the state event does continuous appends, the period scan sums over a wide range. When a store type writes one job into its structure, it leaves the others to the application, and this lesson counts that trade-off. The families' definitions — what the key-value, document, wide-column, and graph store give up and what they give — were established in the Data Modeling and Relational Theory course; the relational-versus-non-relational choice and the rule that the system of record stays singular were measured in this topic's first lesson. Neither is retold. The question here is narrow: **how many queries each type needs per pattern, and how many records it touches.** ## What Separates the Types A store type is defined by the access shape it can answer natively. Every shape it cannot answer falls to the application as a separate store request. A **key-value store** knows only the primary key. The inside of the value is meaningless to it, so secondary access runs through a key list the application keeps, and each record on that list becomes a separate request. A **document store** sees the value's fields and can build a secondary index on one. A secondary filter finishes in a single request. A **wide-column store** groups rows by partition key and reads a sorted range within the group in one request. Filtering by a different key requires writing the same data again under a second partition key. A **graph store** makes nodes and edges first-class objects; a traversal runs inside the store and appears to the client as one request. For measurement, a fourth pattern joins the first three, since none of the first three lands where the graph store wins. **P4 — route traversal:** when a transfer hub goes down, finding the shipments passing through it, deriving their next stops, and advancing three steps to the shipments passing through those. It is a question the delivery operation asks, concerning not a record itself but the links between records. ```js // type/access.mjs — the queries and records touched that the same access patterns require // across four store types. MODEL: stores are objects, counters are explicit; not a product, // a model of access capabilities. Results are deterministic. const SHIPMENT = 60_000, SELLER = 20, HUB = 8; // lesson 01's scale: 3000 per seller const shipments = Array.from({ length: SHIPMENT }, (_, i) => ({ id: `TR-${i}`, seller: i % SELLER, route: [0, 1, 2].map((j) => (i * 3 + j) % HUB), })); const sellerIndex = new Map(), hubIndex = new Map(), routeNext = new Map(); for (const g of shipments) { if (!sellerIndex.has(g.seller)) sellerIndex.set(g.seller, []); sellerIndex.get(g.seller).push(g); for (let j = 0; j < g.route.length; j += 1) { const m = g.route[j]; if (!hubIndex.has(m)) hubIndex.set(m, []); hubIndex.get(m).push(g); if (j + 1 < g.route.length) { if (!routeNext.has(m)) routeNext.set(m, new Set()); routeNext.get(m).add(g.route[j + 1]); } } } class Store { // every call is one store request; touched records are counted constructor() { this.query = 0; this.record = 0; } request(n) { this.query += 1; this.record += n; return n; } } // P1: single-record read by tracking number (state + last three route steps) // P2: writing a state event from the carrier // P3: a seller's period scan // P4: route traversal starting from a hub and advancing H steps const H = 3; // traversal depth const PATTERN = { "key-value": { // only the primary key; secondary index in the application P1: (d) => d.request(1), // the value comes back whole P2: (d) => { d.request(1); d.request(1); }, // read, then write back the whole value P3: (d) => { const n = sellerIndex.get(3).length; d.request(1); // the index key first for (let i = 0; i < n; i += 1) d.request(1); }, // then one request per record P4: (d, depth) => { let m = new Set([0]); for (let h = 0; h < depth; h += 1) { const s = new Set(); for (const x of m) { for (const g of hubIndex.get(x)) d.request(1); for (const y of routeNext.get(x) ?? []) s.add(y); d.request(1); } m = s; } }, }, "document": { // secondary index on document fields P1: (d) => d.request(1), P2: (d) => d.request(1), // appending to the document's event array P3: (d) => d.request(sellerIndex.get(3).length), P4: (d, depth) => { let m = new Set([0]); for (let h = 0; h < depth; h += 1) { const s = new Set(); let k = 0; for (const x of m) { k += hubIndex.get(x).length; for (const y of routeNext.get(x) ?? []) s.add(y); } d.request(k); m = s; } }, }, "wide-column": { // partition key + sorted range within the partition P1: (d) => d.request(3), // last three event rows P2: (d) => { d.request(1); d.request(1); }, // event table + seller table P3: (d) => d.request(sellerIndex.get(3).length), P4: (d, depth) => { let m = new Set([0]); for (let h = 0; h < depth; h += 1) { const s = new Set(); let k = 0; for (const x of m) { k += hubIndex.get(x).length; for (const y of routeNext.get(x) ?? []) s.add(y); } d.request(k); m = s; } }, }, "graph": { // nodes and edges; the traversal runs inside the store P1: (d) => d.request(1 + 3), // shipment node + last three edges P2: (d) => d.request(2), // event node + edge P3: (d) => d.request(1 + 2 * sellerIndex.get(3).length), // seller node, edge, shipment P4: (d, depth) => { let m = new Set([0]), k = 0; for (let h = 0; h < depth; h += 1) { const s = new Set(); for (const x of m) { k += 1 + hubIndex.get(x).length; for (const y of routeNext.get(x) ?? []) s.add(y); } m = s; } d.request(k); }, }, }; console.log(`model: ${SHIPMENT} shipments, ${SELLER} sellers, ${HUB} hubs, ` + `route 3 steps, traversal ${H} steps; numbers are query/record`); const labels = ["P1 single read", "P2 event write", "P3 period scan", "P4 route traversal"]; console.log(`\n${"store type".padEnd(16)}${labels.map((a) => a.padStart(21)).join("")}`); for (const [type, o] of Object.entries(PATTERN)) { const cell = ["P1", "P2", "P3", "P4"].map((p) => { const d = new Store(); o[p](d, H); return `${d.query}/${d.record}`; }); console.log(type.padEnd(16) + cell.map((h) => h.padStart(21)).join("")); } console.log(`\nP4 traversal depth and query count`); console.log(`${"store type".padEnd(16)}${[1, 2, 3].map((h) => `${h} step(s)`.padStart(10)).join("")}`); for (const [type, o] of Object.entries(PATTERN)) console.log(type.padEnd(16) + [1, 2, 3].map((h) => { const d = new Store(); o.P4(d, h); return String(d.query).padStart(10); }).join("")); ``` ``` model: 60000 shipments, 20 sellers, 8 hubs, route 3 steps, traversal 3 steps; numbers are query/record store type P1 single read P2 event write P3 period scan P4 route traversal key-value 1/1 2/2 3001/3001 67503/67503 document 1/1 1/1 1/3000 3/67500 wide-column 1/3 2/2 1/3000 3/67500 graph 1/4 1/2 1/6001 1/67503 P4 traversal depth and query count store type 1 step(s) 2 step(s) 3 step(s) key-value 22501 45002 67503 document 1 2 3 wide-column 1 2 3 graph 1 1 1 ``` These numbers belong to the computed-value class: they come from the model's counters, are deterministic, and are independent of the machine. The model is not a product; it is an abstraction of access capabilities. ## No Column Wins Start to Finish **P1 does not separate any type.** All four types have a query count of 1. Records touched differ as 1, 1, 3, and 4, but fetching a single record by a single key is a job every type does. Choosing a design by the tracking query alone cannot tell the four options apart. **P2 separates two types.** The document and graph store append an event in one request. The key-value store needs two: it cannot reach into the value, so it reads the whole value and writes it whole. The wide-column store also needs two, but for a different reason — serving P3 means writing the same data a second time under a second partition key. **Two types produce the same number for two separate reasons, and the reason determines how it grows**: the key-value store's second request scales with record size, the wide-column store's with the number of supported queries. **P3 eliminates the key-value store.** The document and wide-column store read 3000 records in a single request; the graph store touches 6001, since from the seller node each shipment is reached through an edge and then a node. The key-value store needs 3001 requests: the secondary index lives in the application, so whatever is on the list is pulled one at a time. Records touched are the same order of magnitude, **request count differs by three thousand times.** **P4 flips the table.** Records touched are of the same order across all four types: between 67,500 and 67,503. The separation is in the query column: document and wide-column spend one request per step, the graph store finishes in a single request regardless of depth, the key-value store needs 67,503. The table below makes this clear: as depth goes from one to three, document and wide-column's request count advances 1, 2, 3, while the graph store's stays at 1. **The graph store's win is not records touched, but a request count independent of traversal depth.** A traversal of unknown depth becomes, in the other types, a job of unknown request count. ## Back to the Numbers Query counts convert directly into K01's rows: each pattern's request rate is multiplied by the query count that pattern requires. ```js // type/cost.mjs — the effect of the measured query and record counts on K01's rows. // All of it is arithmetic; the input is the output of the type/access.mjs run. const READ = 41.67, WRITE = 97.22, STORE_REQUEST = 138.89, RATIO = 2.33; // K01 computed values const SCAN = 833.33, SELLER = 4000, WINDOW = 4 * 3600; // K01 computed values and V10 const MEASURE = { // type/access.mjs: [query, record] "key-value": { P1: [1, 1], P2: [2, 2], P3: [3001, 3001] }, "document": { P1: [1, 1], P2: [1, 1], P3: [1, 3000] }, "wide-column": { P1: [1, 3], P2: [2, 2], P3: [1, 3000] }, "graph": { P1: [1, 4], P2: [1, 2], P3: [1, 6001] }, }; console.log(`${"store type".padEnd(16)}${"store req/s".padStart(16)}${"x".padStart(7)}` + `${"write/read".padStart(13)}${"batch req/s".padStart(15)}${"batch rec/s".padStart(15)}` + `${"x".padStart(7)}`); for (const [type, o] of Object.entries(MEASURE)) { const read = READ * o.P1[0], write = WRITE * o.P2[0]; const request = (o.P3[0] * SELLER) / WINDOW, record = (o.P3[1] * SELLER) / WINDOW; console.log(type.padEnd(16) + (read + write).toFixed(2).padStart(16) + ((read + write) / STORE_REQUEST).toFixed(2).padStart(7) + (write / read).toFixed(2).padStart(13) + request.toFixed(2).padStart(15) + record.toFixed(2).padStart(15) + (record / SCAN).toFixed(2).padStart(7)); } console.log(`K01 baseline: requests reaching the store ${STORE_REQUEST} req/s, write/read ${RATIO}, ` + `batch scan ${SCAN} records/s (batch req/s at a single store ${(SELLER / WINDOW).toFixed(2)})`); ``` ``` store type store req/s x write/read batch req/s batch rec/s x key-value 236.11 1.70 4.67 833.61 833.61 1.00 document 138.89 1.00 2.33 0.28 833.33 1.00 wide-column 236.11 1.70 4.67 0.28 833.33 1.00 graph 138.89 1.00 2.33 0.28 1666.94 2.00 K01 baseline: requests reaching the store 138.89 req/s, write/read 2.33, batch scan 833.33 records/s (batch req/s at a single store 0.28) ``` Two types grow `requests reaching the store` from 138.89 to 236.11, 1.70 times, and push `write/read ratio at store` from 2.33 to 4.67. The reason is the same in both: P2 needs two requests. This topic's first lesson produced the same arithmetic — stretching the write path to two targets gave the same 236.11. **The store type changes, but the number comes from the same place: how many targets the write path touches.** The batch column carries the real warning. In the document and wide-column store, the end-of-day job produces 0.28 requests/s in the four-hour window, exactly repeating K01's computed 833.33 records/s scan. In the key-value store, the same job produces **833.61 requests/s**: the store sees six times the 138.89 req/s built up over the whole day. A single architectural decision moves K01's largest item into a different column — record count did not change, request count did. The graph column is the mirror image: request count matches the best types, but its batch record rate climbs from 833.33 to 1666.94, doubling, since every shipment is reached through an edge. The structure that cheapens traversal makes aggregation more expensive. Read together, the four columns yield one sentence: **no type cheapens all four patterns at once.** The choice is made against the most expensive pattern, and the first lesson's rule holds — the system of record stays singular, and the other types are secondary stores derived from it. ## Summary - A store type is defined by the access shape it answers natively; every shape it cannot answer falls to the application as a separate store request. - The single-record read does not distinguish the four types: all take one request, between 1 and 4 records. - The key-value store needs 3001 requests for the period scan, the document and wide-column store need 1; records touched are the same order, request count differs by three thousand times. - The graph store's gain is in request count, not records touched: in a three-step traversal, document and wide-column spend 3 requests while the graph store stays at 1, unchanged as depth grows. - Back to K01: the key-value and wide-column store push `requests reaching the store` from 138.89 to 236.11 (1.70 times) and `write/read ratio at store` from 2.33 to 4.67. - The end-of-day job produces 833.61 requests/s in the key-value store and pushes the scan rate to 1666.94 records/s in the graph store, from 833.33; no type cheapens all four patterns at once. ## Next Step Eight lessons designed **where** and **in what shape** data stands: type was chosen against the access pattern, data was replicated, federated, and sharded; shards were placed under three separate rules; the read path was shortened by denormalization, and a repeated sum was precomputed. The gain is real: the period scan dropped from 833.33 records/s to 8.33, a read's secondary records from 9000 to 40. The one number that did not gain is the read side of `requests reaching the store`. Even in this lesson's best column, the tracking query still knocks on the store's door at 41.67 requests/s, because none of the eight lessons kept the read from **reaching the store** — each only arranged what it would find once there. The next topic takes on meeting the read **in front of** the store, and its first question is which layer the copy stands in.