--- title: 'Introduction to System Design' source: 'https://academia.sh/en/courses/introduction-to-system-design' language: en updated: '2026-08-23T07:01:25+00:00' license: 'CC BY-SA 4.0' lessons: 13 outcomes: - 'Breaks a design problem down into its requirements and constraints' - 'Estimates the resource need with back-of-the-envelope calculation' - 'Expresses the consistency and service availability trade-off in the right terms' --- # Introduction to System Design Breaking a design problem down into its requirements and constraints: treating functional and non-functional requirements as separate determinants, estimating the request, data, and bandwidth need with back-of-the-envelope calculation, communicating the design with diagrams and decision rationale, and expressing the tensions among latency, throughput, scalability, service availability, and consistency in the right terms. ## The Approach How a system design problem is approached: separating component, contract, and quality decisions, narrowing scope through requirement gathering, functional requirements being separate determinants from non-functional ones, estimating request, data, and bandwidth need with back-of-the-envelope estimation, and communicating a design through diagram, trade-off, and decision rationale. ### What Is System Design Source: https://academia.sh/en/courses/introduction-to-system-design/what-is-system-design Breaking a system design down into three kinds of decisions: component boundaries, the contracts that cross them, and the thresholds placed on qualities; separating the three number classes as the answer to where a threshold comes from, and showing that an unclassified number carries no decision. The Architectural Styles course compared arrangements of the same shipment library and reduced every comparison to three parts: quality scenario, threshold, and measured value. The thresholds were given in that course; a number such as "at most 100 bytes cross the boundary" was an input to the exercise. The course's closing sentence marked this: a style choice is still defended with the same three parts here, and what changes is only where the thresholds come from. This course opens with that question. In a system, a threshold is not arbitrary; it comes from somewhere, and where it comes from can be written down. This lesson separates the kinds of decisions a threshold depends on, and it sets the rule that holds through every lesson of the course: every number in a design text states its class. #### Three Kinds of Decisions **System design** is the work of determining how software split across multiple components is divided, and the constraints that division brings. A program's parts run together and stop together; a system's **components** can scale separately and fail separately. The difference between these two sentences produces every question system design asks. A design is made of three kinds of decisions, and each is justified separately. A **component decision** states which work sits in which unit. Its measure is simple: whether a component can scale on its own, and how much of the rest stays standing when it stops. A **contract decision** states what crosses a component boundary: which fields, in which direction, in what shape. Every field crossing the boundary is a bond between two components, and changing it later requires changing both together. A **quality decision** places a threshold on a quality attribute: the tracking query's response time, the accepted rate of event loss, the hour by which the end-of-day job must finish. Quality attribute and quality scenario were established in the Quality Attributes lesson of the Quality and Testing Fundamentals course; they are not redefined here, only used as the carrier of a threshold. The first two kinds of decisions are made even on a single machine, and the Architectural Styles course measured them there. The third is the decision that makes a system a system, because a threshold comes from outside the software, not from inside it: from load, from expectation, and from a fault-tolerance decision. #### Shipment Tracking as a System The course works on a single system throughout: the **shipment tracking and pricing service**. A user asks about a shipment's status with a tracking number, a carrier sends a state event as a shipment moves, and a seller requests pricing at the end of the day. The three flows are kept separate because each wants a separate threshold: the read flow is frequent and small, the write flow is continuous and small, and the batch flow is infrequent and large. The first version of the design can be written as a data structure. Once components, contracts, and flows are listed, countable facts about the design come into view. ```js // design/system.mjs — shipment tracking and pricing service: components, contracts, and three flows export const COMPONENT = { "tracking-endpoint": "answers the tracking query", "event-receiver": "accepts the state event coming from the carrier", "shipment-store": "holds the shipment and delivery state", "tariff-rule": "calculates a shipment's fee", "batch-worker": "produces the seller's end-of-day pricing", }; export const CONTRACT = { "tracking-request": { direction: "client -> tracking-endpoint", field: ["trackingNo"] }, "tracking-response": { direction: "tracking-endpoint -> client", field: ["trackingNo", "state", "zone", "updatedAt"] }, "state-event": { direction: "carrier -> event-receiver", field: ["trackingNo", "state", "route", "time"] }, "state-read": { direction: "tracking-endpoint -> shipment-store", field: ["trackingNo"] }, "state-write": { direction: "event-receiver -> shipment-store", field: ["trackingNo", "state", "route", "time"] }, "day-scan": { direction: "batch-worker -> shipment-store", field: ["contractNo", "day"] }, "fee-request": { direction: "batch-worker -> tariff-rule", field: ["weight", "zone", "rate"] }, }; export const FLOW = { read: { name: "tracking query", component: ["tracking-endpoint", "shipment-store"], contract: ["tracking-request", "state-read", "tracking-response"] }, write: { name: "state event", component: ["event-receiver", "shipment-store"], contract: ["state-event", "state-write"] }, batch: { name: "end-of-day pricing", component: ["batch-worker", "shipment-store", "tariff-rule"], contract: ["day-scan", "fee-request"] }, }; ``` The first measure that comes out of this list is the **design surface**: how many components, how many contracts, and how many fields the contracts carry. ```js // design/surface.mjs — the components each flow touches, the contracts it crosses, and the fields it carries import { COMPONENT, CONTRACT, FLOW } from "./system.mjs"; console.log(`component = ${Object.keys(COMPONENT).length}, contract = ${Object.keys(CONTRACT).length}`); console.log(`${"flow".padEnd(23)}${"component".padStart(10)}${"contract".padStart(9)}${"field".padStart(6)}`); for (const f of Object.values(FLOW)) { const field = f.contract.reduce((t, c) => t + CONTRACT[c].field.length, 0); console.log(`${f.name.padEnd(23)}${String(f.component.length).padStart(10)}${String(f.contract.length).padStart(9)}${String(field).padStart(6)}`); } for (const b of Object.keys(COMPONENT)) { const flows = Object.values(FLOW).filter((f) => f.component.includes(b)).map((f) => f.name); console.log(`${b.padEnd(17)} flows touching = ${flows.length}${flows.length > 1 ? ` (${flows.join(", ")})` : ""}`); } ``` ``` component = 5, contract = 7 flow component contract field tracking query 2 3 6 state event 2 2 8 end-of-day pricing 3 2 5 tracking-endpoint flows touching = 1 event-receiver flows touching = 1 shipment-store flows touching = 3 (tracking query, state event, end-of-day pricing) tariff-rule flows touching = 1 batch-worker flows touching = 1 ``` The last lines give the design's most important fact: four of the five components serve a single flow, while `shipment-store` serves all three. This component is where three separate thresholds meet, and in the course's later topics every new constraint shows up here first. A design's most fragile point is the component the most flows touch. #### Where the Threshold Comes From A quality decision carries a number, and that number has a source. The source falls into one of three classes; until the text states which one, the decision cannot be checked. An **assumption** is a number chosen as an input to the design: daily active users, tracking queries per user, record size. Its accuracy cannot be shown from inside the design, so its rationale and what happens if it doubles are written down. A **computed value** is a number that comes out of assumptions through arithmetic: requests per second, daily data growth, the bandwidth required. A computed value is only as valid as the assumptions it rests on. A **measurement** is a number that comes out of an apparatus: how long a function takes on this machine, the peak size of a buffer. A measurement is tied to its environment, and that dependence is marked in the text. The script below checks four decisions against these three classes. The fourth decision is left without a class on purpose. ```js // design/decision.mjs — the class of the number next to a decision: assumption, computed value, or measurement const CLASS = ["assumption", "computed value", "measurement"]; const FEE_BUDGET_US = 50; // budget chosen for the tariff call (microseconds) // Two assumptions; the full assumption table is built in the Back-of-the-Envelope Estimation lesson. const V = { dailyUsers: 2_000_000, queriesPerUser: 6, dailyShipments: 400_000, eventsPerShipment: 7 }; const query = V.dailyUsers * V.queriesPerUser; const event = V.dailyShipments * V.eventsPerShipment; const TIER = [[1, 3000], [5, 4800], [20, 9600]]; const ZONE = { "34": 100, "06": 115, "35": 125 }; function tariff(s) { const tier = TIER.find(([k]) => s.weight <= k) ?? [0, 9600]; const base = Math.max(2500, Math.round((tier[1] * ZONE[s.zone]) / 100)); return base - Math.round(base * Math.min(s.rate, 0.4)); } function medianMicroseconds(runs = 9) { const durations = []; for (let i = 0; i < runs; i += 1) { const start = process.hrtime.bigint(); tariff({ weight: 4, zone: "35", rate: 0.15 }); durations.push(Number(process.hrtime.bigint() - start) / 1000); } durations.sort((a, b) => a - b); return durations[Math.floor(runs / 2)]; } const DECISION = [ { name: "read scales independently of write", measure: "read/write request ratio", value: (query / event).toFixed(2), class: "computed value" }, { name: "tracking response served from cache", measure: "cache hit ratio", value: "0.90", class: "assumption" }, { name: "tariff rule called inside the process", measure: `${FEE_BUDGET_US} us budget`, value: medianMicroseconds() < FEE_BUDGET_US ? "passed" : "missed", class: "measurement" }, { name: "batch job runs in the night window", measure: "window hours", value: "4", class: null }, ]; console.log(`${"decision".padEnd(40)}${"measure".padEnd(27)}${"value".padStart(8)} class`); for (const k of DECISION) { console.log(`${k.name.padEnd(40)}${k.measure.padEnd(27)}${String(k.value).padStart(8)} ${k.class ?? "-"}`); } const unclassified = DECISION.filter((k) => CLASS.includes(k.class) === false); console.log(`\ndecisions carrying an unclassified number = ${unclassified.length}: ${unclassified.map((k) => k.name).join("; ")}`); console.log(`daily tracking queries = ${query}, daily state events = ${event}`); console.log(`if the assumption doubles: queries per user 12 -> ratio ${((query * 2) / event).toFixed(2)}`); ``` ``` decision measure value class read scales independently of write read/write request ratio 4.29 computed value tracking response served from cache cache hit ratio 0.90 assumption tariff rule called inside the process 50 us budget passed measurement batch job runs in the night window window hours 4 - decisions carrying an unclassified number = 1: batch job runs in the night window daily tracking queries = 12000000, daily state events = 2800000 if the assumption doubles: queries per user 12 -> ratio 8.57 ``` Four rows require four separate readings. The read/write request ratio of 4.29 is a **computed value**: it is the quotient of two assumptions, and the last line shows its sensitivity — if queries per user were 12 instead of 6, the ratio would be 8.57. The cache hit ratio of 0.90 is an **assumption**; its rationale is that the same tracking number is asked several times before delivery, and its accuracy can only be tested once the system is running. The tariff call staying under budget is a **measurement**, and the output records a verdict rather than a raw duration, because a raw duration is tied to this machine and this runtime version. The fourth row has no class. "The batch job runs in a four-hour night window" reads as reasonable in a design text, but it cannot be checked: it is unclear whether four hours is an assumption, a computed value that follows from the seller's expectation, or a duration measured in some run. An unclassified number carries no decision, in the same way an unmeasured axis carries no weight in an argument. #### Component Decisions Depend on Quality Decisions The three kinds of decisions are not made in sequence; they determine each other. The first version of the component list comes from functional expectation: an endpoint that answers the tracking query, a receiver that accepts the event, a store that holds the state. This list is not an arrangement, only how the work is divided. The component count grows past this list because of quality decisions. If the read flow's threshold is to stay independent of the write flow's threshold, the two flows go into separate, independently scalable components, and the component count rises. If the batch job's end-of-day window must not disturb the read flow's response time, the batch job moves into its own unit. Every such separation grows the contract count and the number of fields crossing a boundary; that is the cost the Architectural Styles course measured. The direction of the argument therefore reads backward: a component count cannot be defended before the thresholds are known. Describing a design as "five components" is not a decision unless it says which threshold forced which separation. The next two lessons open the source of a threshold in order: first how requirements are gathered, then how the functional is treated as a determinant separate from the non-functional. #### Summary - System design is made of three kinds of decisions: component boundaries, the contracts that cross them, and the thresholds placed on qualities; the first two are made even on a single machine, the third comes from load and expectation. - A design surface is countable: the example system has 5 components and 7 contracts; the tracking query carries 3 contracts and 6 fields, the state event 2 contracts and 8 fields, the end-of-day pricing 2 contracts and 5 fields. - The component the most flows touch is the most fragile point: `shipment-store` sits in all three flows, and the other four components sit in one flow each. - The number next to a decision falls into one of three classes: an assumption is chosen and its sensitivity is written down, a computed value comes out of assumptions, and a measurement comes out of an apparatus and marks its dependence on the environment. - An unclassified number carries no decision; in the example, one of four decisions stayed unclassified and was counted as unverifiable. - The component count grows from quality thresholds, not from the functional split; a component count cannot be defended before the thresholds are known. #### Next Step This lesson said a threshold has a source, but it did not go looking for that source. The five components and seven contracts in the example were given ready-made; a real design task instead starts with a single-line request: "a service where sellers can track their shipments and be billed at the end of the day." That line contains no component name, no number, no boundary. The next lesson takes up how a design is reached from that line: which questions get asked, why every question left unanswered turns into an assumption, and how much room scope narrowing opens on the design surface. ### Approach to the Design Problem Source: https://academia.sh/en/courses/introduction-to-system-design/approach-to-the-design-problem Turning a single-line request into a design: separating questions into four families, showing which design step each family blocks, turning an unanswered question into an assumption, and measuring the room scope narrowing opens on the design surface. The previous lesson handed over a five-component design ready-made and measured that design's surface. Real design work does not start this way. The starting point is most often a single sentence: "a service where sellers can track their shipments and be billed at the end of the day." That sentence has no component name, no number, no boundary. This lesson takes up the move from that sentence to a design. The move consists of two tasks: finding the gaps by asking questions, and shrinking the design by narrowing the scope. Both produce a measurable result — the first the number of assumptions that must be written, the second the shrinkage in the design surface. #### The Request Is Incomplete, and the Gap Is Countable The single-line request says three things: one actor (the seller), two capabilities (tracking, pricing), and one timing (end of day). What it does not say is larger, and it does not scatter randomly — it groups into four families. **Actor and flow** questions define the outside of the system: who sends a request, who sends data, who waits for the result. Their answers determine the component list. **Volume** questions define the load: how many requests a day, how many records, how many bytes per record. Without their answers, no calculation can be made. **Threshold** questions define the acceptance criterion: how long a response can take, how much loss is accepted, when the result must be ready. Their answers are the input to quality decisions. **Constraint** questions define the obligations coming from outside the design: how long data is retained, where a rule changes, which component cannot be touched. The reason this separation is useful is that each family blocks a specific design step. When the question list is written together with this link, which step can start becomes something that can be calculated. ```js // design/question.mjs — the questions the problem line leaves open, and the step each question blocks export const STEP = { "actor and flow": "the component list cannot be written", volume: "back-of-the-envelope estimation cannot start", threshold: "a quality decision cannot be made", constraint: "a data retention decision cannot be made", }; export const QUESTION = [ { family: "actor and flow", question: "who makes the tracking query", answer: "the recipient and the seller, with separate authority" }, { family: "actor and flow", question: "who sends the state event", answer: "the carrier" }, { family: "actor and flow", question: "who starts pricing", answer: "the seller, at day's end" }, { family: "volume", question: "how many tracking queries arrive per day", answer: null }, { family: "volume", question: "how many shipments are created per day", answer: null }, { family: "volume", question: "how many state events arrive per shipment", answer: null }, { family: "threshold", question: "how long can the tracking response take at most", answer: null }, { family: "threshold", question: "is state event loss accepted", answer: "no, the event is resent" }, { family: "threshold", question: "when must the fee report be ready", answer: null }, { family: "constraint", question: "how long is a state record retained", answer: null }, { family: "constraint", question: "in how many places does the tariff rule change", answer: "one place" }, ]; ``` ```js // design/requirement.mjs — answered questions, unanswered questions, and the number of assumptions that must be written import { STEP, QUESTION } from "./question.mjs"; console.log(`${"question family".padEnd(19)}${"answer".padStart(6)} blocked step`); for (const [family, step] of Object.entries(STEP)) { const group = QUESTION.filter((q) => q.family === family); const answered = group.filter((q) => q.answer !== null).length; const block = answered === group.length ? "-" : step; console.log(`${family.padEnd(19)}${`${answered}/${group.length}`.padStart(6)} ${block}`); } const unanswered = QUESTION.filter((q) => q.answer === null); console.log(`\ntotal questions = ${QUESTION.length}, answered = ${QUESTION.length - unanswered.length}, unanswered = ${unanswered.length}`); console.log(`assumptions that must be written = ${unanswered.length}`); for (const q of unanswered) console.log(` assumption needed: ${q.question}`); ``` ``` question family answer blocked step actor and flow 3/3 - volume 0/3 back-of-the-envelope estimation cannot start threshold 1/3 a quality decision cannot be made constraint 1/2 a data retention decision cannot be made total questions = 11, answered = 5, unanswered = 6 assumptions that must be written = 6 assumption needed: how many tracking queries arrive per day assumption needed: how many shipments are created per day assumption needed: how many state events arrive per shipment assumption needed: how long can the tracking response take at most assumption needed: when must the fee report be ready assumption needed: how long is a state record retained ``` The most useful line in the output is the first table. Because the actor and flow family is complete, the component list can be written; this is why the previous lesson's five components could be written at all. In the volume family, the answer ratio is 0/3, so back-of-the-envelope estimation cannot start. In the threshold family, 1/3 is answered, meaning two-thirds of the quality decisions are pending. #### An Unanswered Question Turns Into an Assumption The six unanswered questions correspond to six assumptions. This is the step required so that design work does not stop: if a question cannot be answered, a number is chosen in its place, that number is written as an **assumption**, and its rationale is given in one sentence. An assumption's cost is not zero. Every computed value built on a wrong assumption is wrong by the same factor, and the only way to make this visible is to write down the sensitivity: if the assumption doubles, which computed value multiplies by how much. The previous lesson gave one example of this — when queries per user were taken as 12 instead of 6, the read/write ratio rose from 4.29 to 8.57. An assumption must not be confused with a threshold. An assumption is a guess about the world outside the system, and it is corrected by measurement. A threshold, on the other hand, is a decision: it is the accepted limit, and it is not corrected by measurement — it changes through discussion. "Twelve million queries arrive a day" is an assumption; "the tracking response does not exceed 200 milliseconds" is a threshold. #### Narrowing the Scope The second task is to leave a portion of the capabilities that surface while answering questions outside the design. The rationale for narrowing is not time — it is measure: every capability adds components, contracts, and fields to the design surface. The list below carries seven scope items and each one's contribution to the surface. Three are in, four are out; every item left out must carry a reason and a re-entry condition. The last item is left unrecorded on purpose. ```js // design/scope.mjs — each scope item's contribution to the design surface, and the record of narrowing const ITEM = [ { name: "tracking query", scope: "in", component: ["tracking-endpoint", "shipment-store"], contract: 3, field: 6 }, { name: "state event acceptance", scope: "in", component: ["event-receiver", "shipment-store"], contract: 2, field: 8 }, { name: "end-of-day pricing", scope: "in", component: ["batch-worker", "shipment-store", "tariff-rule"], contract: 2, field: 5 }, { name: "recipient notification", scope: "out", component: ["notification-endpoint", "template-store"], contract: 3, field: 9, reason: "does not change the tracking query's threshold", reentry: "if every state event were to trigger a notification" }, { name: "seller interface", scope: "out", component: ["interface-server", "session-store"], contract: 4, field: 14, reason: "the fee report can also be delivered as a file", reentry: "if the report is requested during the day" }, { name: "carrier contract management", scope: "out", component: ["contract-endpoint"], contract: 2, field: 7, reason: "the tariff rule comes from the pricing library and does not change here", reentry: "if the tariff is versioned by zone" }, { name: "route planning", scope: "out", component: ["route-planner", "location-receiver"], contract: 3, field: 11, reason: null, reentry: null }, ]; function surface(items) { const component = new Set(items.flatMap((m) => m.component)); return { component: component.size, contract: items.reduce((t, m) => t + m.contract, 0), field: items.reduce((t, m) => t + m.field, 0), }; } const wide = surface(ITEM); const narrow = surface(ITEM.filter((m) => m.scope === "in")); console.log(`${"measure".padEnd(10)}${"wide".padStart(7)}${"narrow".padStart(8)}${"ratio".padStart(7)}`); for (const a of ["component", "contract", "field"]) { console.log(`${a.padEnd(10)}${String(wide[a]).padStart(7)}${String(narrow[a]).padStart(8)}${(narrow[a] / wide[a]).toFixed(2).padStart(7)}`); } const outOfScope = ITEM.filter((m) => m.scope === "out"); const recorded = outOfScope.filter((m) => m.reason !== null && m.reentry !== null); console.log(`\nout-of-scope items = ${outOfScope.length}, with reason and re-entry condition written = ${recorded.length}`); for (const m of outOfScope.filter((x) => recorded.includes(x) === false)) { console.log(` unrecorded narrowing: ${m.name} (${m.component.length} components, ${m.field} fields could come back)`); } for (const m of recorded) console.log(` ${m.name.padEnd(30)} re-entry condition: ${m.reentry}`); ``` ``` measure wide narrow ratio component 12 5 0.42 contract 19 7 0.37 field 60 19 0.32 out-of-scope items = 4, with reason and re-entry condition written = 3 unrecorded narrowing: route planning (2 components, 11 fields could come back) recipient notification re-entry condition: if every state event were to trigger a notification seller interface re-entry condition: if the report is requested during the day carrier contract management re-entry condition: if the tariff is versioned by zone ``` The gain from narrowing shows up in three numbers: components drop from 12 to 5, contracts from 19 to 7, fields crossing a boundary from 60 to 19. The three ratios differ from each other, and that difference carries information. The component ratio is 0.42, the field ratio 0.32; that is, the items left out were growing the contract surface more than the component count. Narrowing the scope's biggest gain is not reducing the component count — it is reducing the number of bonds between components. It is not a coincidence that the third row matches the previous lesson's numbers: the narrowed scope's surface is 5 components and 7 contracts, meaning the previous lesson's design is the result of this narrowing. #### The Record of Narrowing Three of the four out-of-scope items are recorded, one is not. The record consists of two fields, and both are mandatory. The **reason** states why the item was removed, and the correct reason always has the same shape: which threshold this item does not change. Recipient notification was removed because it does not change the tracking query's response time threshold; the seller interface was removed because the fee report can also be delivered as a file. The **re-entry condition** states under what circumstance the item re-enters scope. This field protects the design going forward: if the condition is written down, it is known when the design will reopen, and the decisions in between are made with that knowledge. Recipient notification's condition is that every state event would trigger a notification; if that condition occurs, the write flow's volume changes and the previous lesson's read/write ratio becomes invalid. The item left unrecorded is a risk for exactly this reason. Route planning is out of scope, but because why it was removed and when it will return are not written down, the design does not account for it: when it comes back, 2 components and 11 fields are added to the surface all at once. Leaving something out of scope is a decision, and like every decision it is written down with its reason; an unwritten narrowing is indistinguishable from a forgotten capability. #### Summary - A single-line request separates into four question families: actor and flow, volume, threshold, constraint; each family blocks a specific design step. - In the example, 5 of 11 questions are answered: because actor and flow is 3/3, the component list could be written; because volume is 0/3, back-of-the-envelope estimation does not start. - The 6 unanswered questions turn into 6 assumptions; until an assumption's rationale and sensitivity are written down, the computed value built on it cannot be checked. - An assumption is a guess about the world and is corrected by measurement; a threshold is a decision and changes through discussion. - Scope narrowing opened measurable room: components dropped from 12 to 5 (0.42), contracts from 19 to 7 (0.37), fields crossing a boundary from 60 to 19 (0.32). - An out-of-scope item must carry a reason and a re-entry condition; because one of the four items in the example stayed unrecorded, 2 components and 11 fields get added unaccounted-for when it comes back. #### Next Step Two of the four families in the question list seem to ask the same thing: actor and flow questions ask what the system will do, threshold questions ask how it will do it. Because these two can be answered separately, they are separate determinants and are written separately. The next lesson establishes this separation: functional requirements produce the component list, non-functional requirements produce how many parts that list splits into. The two can be changed independently, and the proof is measurable — the same functional list gives two different designs with two different threshold sets. ### Functional and Non-Functional Requirements Source: https://academia.sh/en/courses/introduction-to-system-design/functional-and-non-functional-requirements Showing that the two kinds of requirements are separate determinants: the functional list determining which kind of component is required, thresholds determining how many parts the same work splits into, the change a single threshold produces in the component count when played one at a time, and a threshold's ability to shrink a design. The previous lesson separated questions into four families and marked that two families ask different things: actor and flow questions ask what the system will do, threshold questions ask within what limit it will do it. If two sets of answers can change independently of each other, they are two separate determinants. This lesson shows that independence. The functional requirement list states which kind of component must exist; thresholds state how many parts the same work splits into. When the two are played separately, different things change in the design, and that difference is countable. #### Two Determinants A **functional requirement** states what the system does and is answered as true or false: the state is returned given a tracking number, the same event arriving a second time is swallowed. The Quality Attributes lesson of the Quality and Testing Fundamentals course had established the same concept under the name "functional expectation"; here the catalog term, functional requirement, is used, and the meaning intended is the same. A **non-functional requirement** states within what limit the system does this work, and it is answered by degree. The quality attribute and quality scenario concepts established in that same lesson are not redefined here; a non-functional requirement is the written form of a quality scenario that places a threshold on a quality attribute. The test of the distinction is this: if a requirement can be checked with a single input–output pair, it is functional. If checking it requires a load, an environment, and a threshold, it is non-functional. #### A Threshold Has No Class, but It Has a Source The first lesson separated numbers into three classes: assumption, computed value, measurement. A threshold is none of these three, and this is deliberate. An assumption is a guess about the world, a computed value comes out of assumptions, a measurement comes out of an apparatus; all three report a fact. A threshold, by contrast, reports not a fact but an acceptance: which value is counted as sufficient. This is why what is written next to a threshold is not a class but a **source**: which expectation the threshold comes from. The source of the threshold "the tracking response does not exceed 200 milliseconds" is the response the recipient expects from the tracking page; the source of "event loss is zero" is the delivery obligation in the carrier contract. A threshold with no source written down cannot be argued about, because it is not known on whose behalf it was set. A non-functional requirement is written in exactly this form — four parts from the quality scenario, and the fifth line is the source field this course adds: - **Stimulus:** the recipient asks about the shipment's status with a tracking number. - **Environment:** the system is operating at peak load. - **Response:** the shipment's final state, its zone, and its update time are returned. - **Response measure:** the median response time does not exceed 200 milliseconds. - **Threshold source:** the response the recipient expects from the tracking page. The second part is empty for now. "At peak load" is not a number, and this scenario is currently unmeasurable; turning the environment into a number is the next lesson's job. #### The Same Function, Two Threshold Sets The way to show independence is to hold one side fixed and play the other. Six functional requirements are held fixed, and two different threshold sets are tried. The move from threshold to component is made with a set of rules. These rules are a **model**, not a universal derivation: each rule ties a threshold condition to a component change, and its rationale stands beside it. The fourth of the rules works in reverse of the other three and removes a component. ```js // design/requirement.mjs — functional list, threshold sets, and the model producing a component list from both export const FUNCTIONAL = [ { no: "F1", flow: "read", action: "return the shipment's state given trackingNo" }, { no: "F2", flow: "read", action: "return the shipment's route history" }, { no: "F3", flow: "write", action: "record the state event coming from the carrier" }, { no: "F4", flow: "write", action: "swallow the same event arriving a second time" }, { no: "F5", flow: "batch", action: "scan the seller's day and produce a fee" }, { no: "F6", flow: "batch", action: "deliver the fee report as a file" }, ]; export const THRESHOLD = { loose: { readMs: 900, eventLoss: 0.01, batchHours: 12, feeMs: 40 }, tight: { readMs: 200, eventLoss: 0, batchHours: 4, feeMs: 1 }, }; export const SOURCE = { readMs: "the response the recipient expects from the tracking page", eventLoss: "the delivery obligation in the carrier contract", batchHours: "the seller wanting the report in the morning", feeMs: "the time the scan can spend per shipment", }; // The functional list determines which kind of component must exist. export function baseline(functional) { const b = new Set(["service", "shipment-store"]); if (functional.some((i) => i.action.includes("fee"))) b.add("tariff-service"); return b; } // Thresholds determine how many parts the same work splits into. export const RULE = [ { name: "read threshold under 300 ms", condition: (e) => e.readMs <= 300, add: ["tracking-endpoint", "event-receiver"], remove: ["service"] }, { name: "event loss is not tolerated", condition: (e) => e.eventLoss === 0, add: ["event-queue"], remove: [] }, { name: "batch window under 6 hours", condition: (e) => e.batchHours < 6, add: ["batch-worker"], remove: [] }, { name: "fee call under 1 ms", condition: (e) => e.feeMs <= 1, add: [], remove: ["tariff-service"] }, ]; export function design(functional, threshold) { const component = baseline(functional); const triggered = []; for (const r of RULE) { if (r.condition(threshold) === false) continue; for (const b of r.remove) component.delete(b); for (const b of r.add) component.add(b); triggered.push(r.name); } return { component: [...component].sort(), triggered }; } ``` ```js // design/derive.mjs — the same functional list gives two different designs with two threshold sets import { FUNCTIONAL, RULE, SOURCE, THRESHOLD, design } from "./requirement.mjs"; const count = (a) => FUNCTIONAL.filter((i) => i.flow === a).length; console.log(`functional requirement = ${FUNCTIONAL.length} (read ${count("read")}, write ${count("write")}, batch ${count("batch")})`); console.log(`${"threshold".padEnd(11)}${"loose".padStart(7)}${"tight".padStart(6)} threshold source`); for (const a of Object.keys(SOURCE)) { console.log(`${a.padEnd(11)}${String(THRESHOLD.loose[a]).padStart(7)}${String(THRESHOLD.tight[a]).padStart(6)} ${SOURCE[a]}`); } for (const [name, threshold] of Object.entries(THRESHOLD)) { const t = design(FUNCTIONAL, threshold); console.log(`\n${name}: rules triggered ${t.triggered.length}/${RULE.length}, components ${t.component.length}`); console.log(` ${t.component.join(", ")}`); } ``` ``` functional requirement = 6 (read 2, write 2, batch 2) threshold loose tight threshold source readMs 900 200 the response the recipient expects from the tracking page eventLoss 0.01 0 the delivery obligation in the carrier contract batchHours 12 4 the seller wanting the report in the morning feeMs 40 1 the time the scan can spend per shipment loose: rules triggered 0/4, components 3 service, shipment-store, tariff-service tight: rules triggered 4/4, components 5 batch-worker, event-queue, event-receiver, shipment-store, tracking-endpoint ``` The six functional requirements are the same in both runs. With loose thresholds, no rule triggers and the design stays at three components; with tight thresholds, four rules trigger and the design rises to five components. Whether a design is "three components" or "five components" therefore cannot be read from the functional list. The composition of the five components is also worth noting: `service` and `tariff-service` are not in the list, and `tracking-endpoint`, `event-receiver`, `event-queue`, and `batch-worker` stand in their place. Because the fourth rule removes a component, tightening does not raise the component count in a single direction. The event queue appearing as a component does not mean the asynchronous delivery mechanism is explained here; the message queue was established in the Caching, Queues and Asynchronous Processing course, and here it is only counted as a component. #### Two Directions of Independence Independence must run in both directions: when an addition is made to the functional list, the thresholds must stay in place; when a threshold is played, the functional list must stay in place. The second script tests both. ```js // design/independence.mjs — playing the functional list and each threshold one at a time and measuring the change in component count import { FUNCTIONAL, THRESHOLD, design } from "./requirement.mjs"; const base = design(FUNCTIONAL, THRESHOLD.loose); console.log(`start: functional ${FUNCTIONAL.length}, threshold loose, components ${base.component.length}`); const F7 = { no: "F7", flow: "read", action: "return the shipment's estimated delivery day" }; const wider = design([...FUNCTIONAL, F7], THRESHOLD.loose); console.log(`functional added (${F7.no}): functional ${FUNCTIONAL.length + 1}, components ${wider.component.length}, ` + `change ${wider.component.length - base.component.length}`); console.log(`\n${"single threshold change".padEnd(28)}${"components".padStart(11)}${"change".padStart(8)} removed/added`); for (const a of Object.keys(THRESHOLD.loose)) { const t = design(FUNCTIONAL, { ...THRESHOLD.loose, [a]: THRESHOLD.tight[a] }); const removed = base.component.filter((b) => t.component.includes(b) === false); const added = t.component.filter((b) => base.component.includes(b) === false); const note = [removed.length ? `-${removed.join(" -")}` : "", added.length ? `+${added.join(" +")}` : ""].filter(Boolean).join(" "); console.log(`${`${a} ${THRESHOLD.loose[a]} -> ${THRESHOLD.tight[a]}`.padEnd(28)}${String(t.component.length).padStart(11)}` + `${String(t.component.length - base.component.length).padStart(8)} ${note}`); } ``` ``` start: functional 6, threshold loose, components 3 functional added (F7): functional 7, components 3, change 0 single threshold change components change removed/added readMs 900 -> 200 4 1 -service +event-receiver +tracking-endpoint eventLoss 0.01 -> 0 4 1 +event-queue batchHours 12 -> 4 4 1 +batch-worker feeMs 40 -> 1 2 -1 -tariff-service ``` The first two lines give one direction: when the seventh functional requirement is added, the component count stays at 3, a change of 0. The new function falls inside the existing read flow and does not move the component list. The intuition that a design grows as the functional list grows is falsified by this line. The table gives the other direction. When each of the four thresholds is played on its own, the functional list stays at six items, yet the component count changes: three thresholds raise it from 3 to 4, one lowers it from 3 to 2. The last row is the most instructive. When the fee call's budget is pulled from 40 milliseconds to 1 millisecond, the tariff service cannot stay a separate component, because being a separate component costs a network hop, and that hop does not fit the budget. A tight threshold does not always ask for more parts; sometimes it asks for the opposite. The removed-versus-added distinction in the first row also carries information. When the read threshold tightens, `service` is removed and two components take its place, meaning this is not an addition but a **split**. The other two thresholds only add. Whether a design change is a split or an addition determines its effect on the contract surface: a split divides an existing boundary in two, an addition brings in a new boundary. #### Summary - A functional requirement states what the system does and is checked with a single input–output pair; a non-functional requirement states within what limit it does it and requires a load, an environment, and a threshold. - A threshold has no assumption, computed value, or measurement class; a threshold is an acceptance, and what is written next to it is not a class but a source — which expectation it comes from. - A non-functional requirement is written with the quality scenario's four parts plus the threshold's source; the example scenario's environment part is still numberless. - The same six functional requirements gave 3 components with loose thresholds and 5 with tight thresholds; the component count cannot be read from the functional list. - The seventh functional requirement did not change the component count (a change of 0); each of the four thresholds changed it on its own (three by +1, one by −1). - A tight threshold does not always ask for more components: when the fee call's budget was pulled to 1 millisecond, the tariff service stopped being a separate component. #### Next Step Three of the quality scenario's four parts were written, one stayed empty: the environment. The sentence "operating at peak load" leaves the threshold unmeasurable, because how many requests peak load is has not been written down. The same gap exists in the thresholds themselves — whether 200 milliseconds is achievable cannot be said without knowing how many requests arrive per second. The next lesson fills this gap: the six unanswered questions that turned into six assumptions in the previous lesson are gathered into an assumption table, and request rate, data growth, and bandwidth need are derived from the table by arithmetic. ### Back-of-the-Envelope Estimation Source: https://academia.sh/en/courses/introduction-to-system-design/back-of-the-envelope-estimation Gathering thirteen assumptions into a single table, and deriving request rate, data growth, and bandwidth need from the table by arithmetic: calculating peak load, finding the load balance behind the cache, a sparse batch job dominating bandwidth, and measuring which result each assumption multiplies by how much. The previous two lessons left two gaps with no number. The quality scenario's environment part read "at peak load," and how many requests peak load is stayed unclear; whether the thresholds were achievable depended on that same uncertainty. In the second lesson, six unanswered questions had turned into six assumptions, but the assumptions had not been written down anywhere. This lesson fills that gap. A **back-of-the-envelope estimate** is the work of deriving a design's resource need from a small number of assumptions by arithmetic. Its purpose is not to find the correct number but to find the correct **order of magnitude** and to make visible which assumption a design decision rests on. #### The Assumption Table Every calculation has a single source. The thirteen assumptions below are also used through the rest of the course; no lesson chooses a new volume number on its own. | Code | Assumption | Value | Rationale | |---|---|---|---| | V1 | daily active users | 2,000,000 | recipients and sellers who open the tracking page at least once a day | | V2 | tracking queries per user per day | 6 | a shipment is asked about several times before delivery | | V3 | new shipments created per day | 400,000 | the sum of daily shipment counts per seller | | V4 | state events per shipment | 7 | pickup, transfer, out for delivery, delivered, and intermediate states | | V5 | tracking response body | 480 bytes | state, zone, update time, and the last three route steps | | V6 | a state event's record | 220 bytes | tracking number, state, route step, timestamp | | V7 | a shipment's record | 900 bytes | weight, volume, zone, contract, and tariff fields | | V8 | peak factor | 3 | the ratio of the peak-hour rate to the daily average | | V9 | tracking query cache hit ratio | 0.90 | the same tracking number is asked again within a short interval | | V10 | end-of-day job's window | 4 hours | the seller wants the report in the morning | | V11 | days the pricing job scans | 30 | pricing is reconciled over a monthly period | | V12 | how long a record is retained | 730 days | contract disputes can reach back two years | | V13 | items collected in one invoice line | 100 | one seller's one day is a single invoice line; how period records are partitioned depends on this ratio | Every row in the table is an assumption: its accuracy cannot be shown from inside the design. The bandwidth concept was established in the Computer Networks curriculum and is not re-explained here; the calculation counts only body bytes, and header and protocol overhead would be a separate assumption. ```js // design/assumption.mjs — the course's single assumption table; every calculation comes out of this table export const V = { dailyUsers: 2_000_000, // V1 queriesPerUser: 6, // V2 dailyShipments: 400_000, // V3 eventsPerShipment: 7, // V4 trackingResponseBytes: 480, // V5 eventRecordBytes: 220, // V6 shipmentRecordBytes: 900, // V7 peakFactor: 3, // V8 cacheHitRate: 0.9, // V9 batchWindowHours: 4, // V10 batchSpanDays: 30, // V11 retentionDays: 730, // V12 itemsPerInvoiceLine: 100, // V13 }; export const DAY_SECONDS = 86_400; export function calculate(v) { const dailyQueries = v.dailyUsers * v.queriesPerUser; const dailyEvents = v.dailyShipments * v.eventsPerShipment; const readAvg = dailyQueries / DAY_SECONDS; const writeAvg = dailyEvents / DAY_SECONDS; const readPeak = readAvg * v.peakFactor; const writePeak = writeAvg * v.peakFactor; const storeReads = readPeak * (1 - v.cacheHitRate); const batchBytes = v.batchSpanDays * v.dailyShipments * v.shipmentRecordBytes; const dailyBytes = v.dailyShipments * v.shipmentRecordBytes + dailyEvents * v.eventRecordBytes; return { "daily tracking queries": dailyQueries, "daily state events": dailyEvents, "average read requests/s": readAvg, "peak read requests/s": readPeak, "peak write requests/s": writePeak, "peak edge requests/s": readPeak + writePeak, "reads behind cache/s": storeReads, "requests reaching store/s": storeReads + writePeak, "write/read ratio at store": writePeak / storeReads, "read egress Mbit/s": (readPeak * v.trackingResponseBytes * 8) / 1e6, "write ingress Mbit/s": (writePeak * v.eventRecordBytes * 8) / 1e6, "batch job read GB": batchBytes / 1e9, "batch job scan records/s": (v.batchSpanDays * v.dailyShipments) / (v.batchWindowHours * 3600), "batch job Mbit/s": (batchBytes * 8) / (v.batchWindowHours * 3600 * 1e6), "daily data growth MB": dailyBytes / 1e6, "stored data GB": (dailyBytes * v.retentionDays) / 1e9, "daily invoice lines": v.dailyShipments / v.itemsPerInvoiceLine, "period seller-days": (v.batchSpanDays * v.dailyShipments) / v.itemsPerInvoiceLine, }; } ``` Units are counted in decimal: MB is $10^6$ bytes, GB is $10^9$ bytes, Mbit/s is $10^6$ bit/s. To avoid confusion, only bytes and seconds are used inside the calculation, and the conversion is done in the last step. #### The Calculations ```js // design/scale.mjs — every calculation that comes out of the assumption table import { V, calculate } from "./assumption.mjs"; const r = calculate(V); const format = (x) => (Number.isInteger(x) ? String(x) : x.toFixed(2)); for (const [name, value] of Object.entries(r)) console.log(`${name.padEnd(27)}${format(value).padStart(11)}`); const msPerShipment = 1000 / r["batch job scan records/s"]; console.log(`\nbatch job time per shipment = ${msPerShipment.toFixed(2)} ms (assuming a single worker)`); console.log(`fee call threshold 1 ms -> remaining margin = ${(msPerShipment - 1).toFixed(2)} ms`); console.log(`batch job bandwidth / peak read egress = ${(r["batch job Mbit/s"] / r["read egress Mbit/s"]).toFixed(2)}`); ``` ``` daily tracking queries 12000000 daily state events 2800000 average read requests/s 138.89 peak read requests/s 416.67 peak write requests/s 97.22 peak edge requests/s 513.89 reads behind cache/s 41.67 requests reaching store/s 138.89 write/read ratio at store 2.33 read egress Mbit/s 1.60 write ingress Mbit/s 0.17 batch job read GB 10.80 batch job scan records/s 833.33 batch job Mbit/s 6 daily data growth MB 976 stored data GB 712.48 daily invoice lines 4000 period seller-days 120000 batch job time per shipment = 1.20 ms (assuming a single worker) fee call threshold 1 ms -> remaining margin = 0.20 ms batch job bandwidth / peak read egress = 3.75 ``` All twenty-one of these numbers belong to the **computed value** class: each comes out of the assumption table by arithmetic, and none is taken from outside. Five of them affect the design directly. **Peak load is not where it seems to be.** The peak request rate at the edge is 513.89 per second. Of this, 416.67 is read, 97.22 is write. The read flow looks dominant — but behind the cache the table flips: reads reaching the store are 41.67, writes are 97.22, and the write/read ratio at the store is 2.33. The system described as read-heavy is write-heavy where the store sees it. The cache appears here not as a strategy but only as a multiplier in the calculation; caching was established in the Caching, Queues and Asynchronous Processing course. **A sparse flow dominates bandwidth.** Peak read egress is 1.60 Mbit/s, the end-of-day job's read rate is 6 Mbit/s: a ratio of 3.75. A job that runs once a day demands three and a half times the bandwidth of the read flow that runs every second of the day, because it squeezes 10.80 GB into a four-hour window. A design's largest resource line item does not have to be its most frequently running flow. **A threshold can be verified from a calculation.** The previous lesson's fee call threshold was 1 millisecond and looked arbitrary. Given a scan rate of 833.33 records per second, there are 1.20 milliseconds per shipment; the 1-millisecond threshold leaves a margin of 0.20 milliseconds. The threshold's source thereby becomes a computed value rather than an expectation. The number holds under the single-worker assumption: if the work is split across parallel workers, the margin grows, but that is a design decision, not a result of the arithmetic. **A coincidence requires re-running the calculation.** The peak request rate reaching the store is 138.89, and the average read rate is also 138.89. The two numbers are equal because $12 \cdot 10^6 \cdot 0.3 + 2.8 \cdot 10^6 \cdot 3 = 12 \cdot 10^6$; this is not an identity, it is a coincidence produced by the chosen assumptions. When one of the assumptions changes, the equality breaks, which is why the calculation is not memorized — it is re-run. **A single division gives two different numbers.** At the V13 ratio, 400,000 shipments a day works out to 4,000 invoice lines a day; since each line corresponds to one seller's one day, this number is also the seller count. The pricing job, which scans thirty days, instead encounters 120,000 lines. The two numbers come out of the same division but do not count the same thing: 4,000 sellers, 120,000 seller-days. This course's later Fundamental Properties topic splits the end-of-day job into partitions using this second number, so the two must not be used in place of each other. #### Sensitivity An assumption table's most important output is not the numbers but the numbers' sensitivity to the assumptions. The script below plays the thirteen assumptions one at a time and writes how much each of seven results multiplies by. Because the cache hit rate cannot be doubled, its play is defined as doubling the miss rate instead; the batch window and the items collected per invoice line are halved instead, because tightening either of them means shrinking it. ```js // design/sensitivity.mjs — playing each assumption on its own and measuring which result multiplies by how much import { V, calculate } from "./assumption.mjs"; const PLAY = { dailyUsers: (v) => v * 2, queriesPerUser: (v) => v * 2, dailyShipments: (v) => v * 2, eventsPerShipment: (v) => v * 2, trackingResponseBytes: (v) => v * 2, eventRecordBytes: (v) => v * 2, shipmentRecordBytes: (v) => v * 2, peakFactor: (v) => v * 2, cacheHitRate: () => 0.8, // hit rate 0.90 -> 0.80: the miss rate doubles batchWindowHours: (v) => v / 2, // window is halved: the same work must finish in half the time batchSpanDays: (v) => v * 2, retentionDays: (v) => v * 2, itemsPerInvoiceLine: (v) => v / 2, // half as many items are collected per line }; const RESULT = [["peak edge requests/s", "edge/s"], ["requests reaching store/s", "store/s"], ["read egress Mbit/s", "egress"], ["batch job Mbit/s", "batch"], ["daily data growth MB", "growth"], ["stored data GB", "storage"], ["period seller-days", "sel-day"]]; const baseline = calculate(V); const touched = {}; console.log(`${"assumption played".padEnd(26)}${RESULT.map(([, k]) => k.padStart(8)).join("")}`); for (const [name, play] of Object.entries(PLAY)) { const changed = calculate({ ...V, [name]: play(V[name]) }); touched[name] = RESULT.filter(([s]) => Math.abs(changed[s] - baseline[s]) > 1e-9).map(([s]) => s); const row = RESULT.map(([s]) => (touched[name].includes(s) ? `x${(changed[s] / baseline[s]).toFixed(2)}` : "-").padStart(8)); console.log(`${name.padEnd(26)}${row.join("")}`); } const widest = Object.entries(touched).sort((a, b) => b[1].length - a[1].length)[0]; console.log(`\nassumption touching the most results = ${widest[0]} (${widest[1].length}/${RESULT.length})`); for (const [s] of RESULT) { const source = Object.keys(touched).filter((a) => touched[a].includes(s)); console.log(`${s.padEnd(27)} dependent assumptions = ${source.length}`); } ``` ``` assumption played edge/s store/s egress batch growth storage sel-day dailyUsers x1.81 x1.30 x2.00 - - - - queriesPerUser x1.81 x1.30 x2.00 - - - - dailyShipments x1.19 x1.70 - x2.00 x2.00 x2.00 x2.00 eventsPerShipment x1.19 x1.70 - - x1.63 x1.63 - trackingResponseBytes - - x2.00 - - - - eventRecordBytes - - - - x1.63 x1.63 - shipmentRecordBytes - - - x2.00 x1.37 x1.37 - peakFactor x2.00 x2.00 x2.00 - - - - cacheHitRate - x1.30 - - - - - batchWindowHours - - - x2.00 - - - batchSpanDays - - - x2.00 - - x2.00 retentionDays - - - - - x2.00 - itemsPerInvoiceLine - - - - - - x2.00 assumption touching the most results = dailyShipments (6/7) peak edge requests/s dependent assumptions = 5 requests reaching store/s dependent assumptions = 6 read egress Mbit/s dependent assumptions = 4 batch job Mbit/s dependent assumptions = 4 daily data growth MB dependent assumptions = 4 stored data GB dependent assumptions = 5 period seller-days dependent assumptions = 3 ``` The matrix says three things. First, no assumption plays every result: the assumption with the widest effect is the daily shipment count, and even that reaches only six of the seven results. Second, the expectation that "doubling the input doubles the result" is wrong in most cells. When daily active users double, the peak request rate at the edge rises by a factor of 1.81, because the write flow does not depend on this assumption; the load the store sees rises by only a factor of 1.30, because ten percent of reads reach the store. The same growth shows up at a different size at different layers. Third, how many assumptions a result depends on is the measure of its reliability. The request rate reaching the store depends on six assumptions at once; if one of them is wrong, the number is wrong. Stored data depends on five assumptions. This is why, in a design discussion, the sentence "one hundred thirty-nine requests a second arrive at the store" cannot be defended on its own; the defense points back to the assumption table. #### Summary - A back-of-the-envelope estimate looks for the correct order of magnitude, not the correct number; its input is an assumption table gathered in a single place. - The calculations coming out of the thirteen assumptions: 513.89 requests/s peak at the edge (read 416.67, write 97.22), daily data growth of 976 MB, 712.48 GB across 730 days, 4,000 invoice lines a day, and 120,000 seller-days over the period — the last two come out of the same division but do not count the same thing. - Behind the cache, the load balance flips: reads reaching the store 41.67, writes 97.22, and a write/read ratio of 2.33 — a read-heavy system is write-heavy at the store. - A sparse flow can dominate bandwidth: the end-of-day job demands 6 Mbit/s because it squeezes 10.80 GB into four hours, 3.75 times the peak read egress. - A threshold can be verified with a calculation: since the scan runs at 833.33 records a second, there are 1.20 ms per shipment, and the previous lesson's 1 ms threshold leaves a margin of 0.20 ms. - The sensitivity matrix falsifies the "double the input, double the result" expectation: when daily users double, the peak at the edge rises by 1.81, the load at the store by 1.30. #### Next Step What is in hand is a component list, a threshold set, and twenty-one computed numbers. These are not a design; they are a design's material. A design exists once it has been explained to someone else and defended. The next lesson takes up that explanation: how components and flows go into a diagram, which number stands on which edge, how a trade-off is written, and how writing each number's class next to it makes the explanation checkable. ### Communicating the Design Source: https://academia.sh/en/courses/introduction-to-system-design/communicating-the-design Presenting a design in a defensible form: a box-and-arrow diagram whose edge labels are generated from the calculation, a five-field decision record made of scenario, threshold, value, the value's class, and the cost paid, counting missing fields, and screening unmeasured sentences out of the narrative. The previous four lessons produced a design's material: five components and seven contracts, a narrowed scope, four thresholds, and twenty-one computed values coming out of thirteen assumptions. This material is not a design on its own. A design exists to the extent that it can be defended once it has been explained to someone else and that person has pushed back. This lesson takes up the form of that explanation. The form consists of three parts: a diagram showing the flows and boundaries, a decision record that logs each decision with five fields, and the class standing next to each number. The style selection and decision table established in the Architectural Styles course are not re-explained here; that course's three-part defense form is used together with this course's number classes. #### What a Diagram Must Show A design diagram is not decoration; it is a document that lets a reader agree with a decision or push back on it. To do this, it must carry four things: **components**, the flow's **direction**, which **flow** an edge belongs to, and the **number** on the edge. The fourth is usually missing, and when it is missing the diagram is just an arrangement of boxes. Writing edge numbers by hand carries two risks: the diagram is not updated when an assumption changes, and which class a number belongs to gets lost. The fix for both is to generate the diagram from the calculation. The first file below recalculates, from the previous lesson's assumption table, only the numbers that appear in this narrative. ```js // design/calculation.mjs — the numbers used in the narrative; inputs come from the Back-of-the-Envelope Estimation lesson's assumption table const DAY_SECONDS = 86_400; export const V = { dailyUsers: 2_000_000, queriesPerUser: 6, dailyShipments: 400_000, eventsPerShipment: 7, peakFactor: 3, cacheHitRate: 0.9, batchSpanDays: 30, batchWindowHours: 4, shipmentRecordBytes: 900, eventRecordBytes: 220, retentionDays: 730 }; const dailyBytes = V.dailyShipments * V.shipmentRecordBytes + V.dailyShipments * V.eventsPerShipment * V.eventRecordBytes; export const C = { readPeak: ((V.dailyUsers * V.queriesPerUser) / DAY_SECONDS) * V.peakFactor, writePeak: ((V.dailyShipments * V.eventsPerShipment) / DAY_SECONDS) * V.peakFactor, storeReads: ((V.dailyUsers * V.queriesPerUser) / DAY_SECONDS) * V.peakFactor * (1 - V.cacheHitRate), scan: (V.batchSpanDays * V.dailyShipments) / (V.batchWindowHours * 3600), msPerShipment: (V.batchWindowHours * 3600 * 1000) / (V.batchSpanDays * V.dailyShipments), storedGB: (dailyBytes * V.retentionDays) / 1e9, }; ``` The drawing consists of a box-and-arrow layout, and the edge labels come from the values above. The class letters stand at the end of the label: `A` assumption, `C` computed value, `M` measurement. ```js // design/diagram.mjs — box-and-arrow drawing whose edge labels come from the calculation (A assumption, C computed value, M measurement) import { C } from "./calculation.mjs"; const e = (x, unit, cls) => `${x.toFixed(2)} ${unit} ${cls}`; const box = (label, inner) => `|${(` ${label}`).padEnd(inner)}|`; const B = { EDGE: "+-------------------+", STORE: "+------------------+" }; function line(...part) { let s = ""; for (const [col, text] of part) s = s.padEnd(col) + text; return s; } const drawing = [ line([2, "outside actor"], [30, "edge component"], [59, "shared component"]), "", line([2, "recipient, seller"], [30, B.EDGE]), line([3, "tracking query ----> "], [30, box("tracking-endpoint", 19)], [51, " ---+"], [59, e(C.storeReads, "requests/s", "C")]), line([3, e(C.readPeak, "requests/s", "C")], [30, B.EDGE], [55, "|"]), line([55, "|"]), line([2, "carrier"], [30, B.EDGE], [55, "|"], [59, B.STORE]), line([3, "state event ----> "], [30, box("event-receiver", 19)], [51, " ---+-> "], [59, box("shipment-store", 18)]), line([3, e(C.writePeak, "requests/s", "C")], [30, B.EDGE], [55, "|"], [59, `| ${e(C.storedGB, "GB", "C").padEnd(17)}|`]), line([55, "|"], [59, B.STORE]), line([2, "seller"], [30, B.EDGE], [55, "|"]), line([3, "end-of-day request ----> "], [30, box("batch-worker", 19)], [51, " ---+"], [59, e(C.scan, "records/s", "C")]), line([3, "1 job/day A"], [30, B.EDGE]), ]; for (const s of drawing) console.log(s); console.log(`\nbatch job margin per shipment = ${C.msPerShipment.toFixed(2)} ms C`); ``` ``` outside actor edge component shared component recipient, seller +-------------------+ tracking query ----> | tracking-endpoint | ---+ 41.67 requests/s C 416.67 requests/s C +-------------------+ | | carrier +-------------------+ | +------------------+ state event ----> | event-receiver | ---+-> | shipment-store | 97.22 requests/s C +-------------------+ | | 712.48 GB C | | +------------------+ seller +-------------------+ | end-of-day request ----> | batch-worker | ---+ 833.33 records/s C 1 job/day A +-------------------+ batch job margin per shipment = 1.20 ms C ``` The first thing the diagram says is that three outside actors enter three separate edge components, and all three converge on the same component. This makes visible the fact the first lesson measured: `shipment-store` is the component all three flows touch, and every constraint accumulates there first. The second thing is the **asymmetry** of the edge numbers. The left edge carries 416.67 requests a second; the same flow's right edge carries 41.67. That difference is a component decision in its own right, and it shows up in the diagram as a single pair of numbers. A diagram with no number on its edges hides this difference; a reader assumes the three arrows carry an equal load. Third is the distribution of the class letters. Four of the five edges are `C`, one is `A`. The end-of-day request arriving once a day is not a computed result but a chosen assumption, and its letter says so. If an edge carried `M`, it would read that the number was measured on this machine and would change in another environment. #### The Decision Record The diagram shows the state, not the decision. The answer to "why three separate edges" or "why is the tariff rule not a separate component" stands in a separate record. The record consists of five fields, and three of them come from the defense form in the Architectural Styles course. - **Scenario:** which quality is being asked about, under which stimulus. - **Threshold:** the accepted value for that quality and its source. - **Value:** the number the decision achieves on that measure. - **Value's class:** assumption, computed value, or measurement. - **Cost:** the number that grows in exchange — the other half of the trade-off. Without the fifth field, the record is a declaration, not a defense. Four decisions are written below with these five fields; two fields in the fourth are left empty on purpose. ```js // design/narrative.mjs — the decision record's five fields and the measure check on narrative sentences import { C } from "./calculation.mjs"; const DECISION = [ { name: "read and write are placed in separate components", scenario: "the recipient makes a tracking query while a write wave is running", threshold: "tracking response median at most 200 ms", value: `${C.writePeak.toFixed(2)} requests/s of write does not fall on the same process`, class: "computed value", cost: "components 3 -> 4, 1 new boundary" }, { name: "tariff rule is called inside the process", scenario: "the end-of-day job scans thirty days of shipments", threshold: "at most 1 ms per shipment", value: `${C.msPerShipment.toFixed(2)} ms margin`, class: "computed value", cost: "the tariff cannot be published separately, 2 parts published together" }, { name: "tracking response is served from cache", scenario: "the same tracking number is asked again within a short interval", threshold: "reads reaching the store at most 50 requests/s", value: `${C.storeReads.toFixed(2)} requests/s`, class: "computed value (depends on the 0.90 hit rate assumption)", cost: "the response can be as stale as one cache age" }, { name: "end-of-day job runs in a four-hour window", scenario: "the seller wants the report in the morning", threshold: `scan at least ${C.scan.toFixed(2)} records/s`, value: null, class: null, cost: "the scan cannot run outside the window" }, ]; const FIELD = ["scenario", "threshold", "value", "class", "cost"]; for (const k of DECISION) { const missing = FIELD.filter((a) => k[a] === null); console.log(`${k.name}\n ${5 - missing.length}/5 fields${missing.length ? ` — missing: ${missing.join(", ")}` : ""}`); } const complete = DECISION.filter((k) => FIELD.every((a) => k[a] !== null)); console.log(`\ndefensible decisions = ${complete.length}/${DECISION.length}`); console.log(`decisions with cost written = ${DECISION.filter((k) => k.cost !== null).length}/${DECISION.length}`); const SENTENCE = [ "the read path is independent of the write wave", "peak request rate reaching the store is 138.89/s", "the design scales", "the end-of-day job scans 10.80 GB", ]; console.log(""); for (const s of SENTENCE) console.log(`${/\d/.test(s) ? "measured " : "unmeasured"}: ${s}`); console.log(`unmeasured sentences = ${SENTENCE.filter((s) => /\d/.test(s) === false).length}/${SENTENCE.length}`); ``` ``` read and write are placed in separate components 5/5 fields tariff rule is called inside the process 5/5 fields tracking response is served from cache 5/5 fields end-of-day job runs in a four-hour window 3/5 fields — missing: value, class defensible decisions = 3/4 decisions with cost written = 4/4 unmeasured: the read path is independent of the write wave measured : peak request rate reaching the store is 138.89/s unmeasured: the design scales measured : the end-of-day job scans 10.80 GB unmeasured sentences = 2/4 ``` Three of the four decisions carry all five fields, one carries three. What the missing one lacks also matters: its scenario, threshold, and cost paid are written, but the value achieved against the threshold and that value's class are empty. This is the most common gap in design texts: a decision is declared together with a threshold, but that the threshold is met is not shown. For the four-hour window decision to be defensible, it must be shown that the scan keeps up with 833.33 records a second, and that number's class must be stated — a computed value with a single worker, a measurement once the worker count is chosen. The cost lines are also worth reading. All four decisions write a cost, and none of the costs is an unmeasured word like "complexity": a rise in the component count, the number of parts published together, the response's chance of going stale, the scan's inability to run outside the window. A decision record that writes its cost in unmeasured words is counted as not having written a cost at all. #### The Unmeasured Sentence The last check looks at the narrative itself. Two of the four sentences carry no number: "the read path is independent of the write wave" and "the design scales." Both appear often in design texts, and neither says anything. The first can be fixed: independence's measure is how much of the write flow's growth passes through to the read response, and that can be written as a number. The second cannot be fixed, because it does not say which quality degrades by how much under which load; unless a quality, a load, and a threshold are written in its place, the sentence is discarded. The two sentences that do carry a number must also carry their class. "Peak request rate reaching the store is 138.89" is a computed value that depends on six assumptions, and it must be presented as one; "the end-of-day job scans 10.80 GB" is the same. For a design narrative to be checkable means that every number inside it can be traced back to its source. #### Summary - A design narrative consists of three parts: a diagram showing flow and boundary, a five-field decision record, and the class standing next to each number. - A diagram must carry component, direction, flow, and edge counts; when edge labels are generated from the calculation, they stay current as assumptions change and their class is never lost. - In the example diagram, edge asymmetry makes a component decision visible: the same flow carries 416.67 requests/s on its left edge and 41.67 requests/s on its right edge. - The decision record's five fields are scenario, threshold, value, the value's class, and the cost paid; in the example, three of four decisions carried all five fields, and one was counted undefensible because its value and class fields stayed empty. - A trade-off cannot be written in an unmeasured word; the four decisions' costs were written as the component count, parts published together, the chance of staleness, and the window constraint. - Two of the four sentences in the narrative carried no number; a sentence with no number is either rewritten with a measure or discarded. #### Next Step This topic showed how a design is built and how it is explained: separating components, the two halves of a requirement, the move from assumption to computed value, and tying every decision to a rationale. What the rationale rests on has still not been named. This lesson's thresholds used measures like "tracking response median," "requests reaching the store," and "finishing inside the window" as if they were already known concepts, but the qualities behind sentences like "fast," "scales," and "stays up" were not defined one by one. The next topic defines these measures in turn, and its first lesson opens with the pair most often confused with each other: latency and throughput are the answers to two separate questions about the same system, and one improving does not mean the other improves. ## Fundamental Properties The metrics that define a system's behavior under load and failure: the independence of latency and throughput, the difference between performance and scalability, the limits of horizontal and vertical scaling, expressing service availability numerically as an outage budget, reading the CAP theorem correctly, the latency trade-off outside a partition, the models between strong and eventual consistency, and active-active versus active-passive failover patterns. ### Latency and Throughput Source: https://academia.sh/en/courses/introduction-to-system-design/latency-and-throughput Measuring that the two metrics are two independent axes: throughput saturating and latency worsening as concurrency rises, verifying by measurement the relation among concurrency, throughput, and latency, showing the latency cost of approaching saturation with a queueing model, and classifying every number as an assumption, a computed value, or a measurement. The previous topic covered how a design is built and communicated: separating components, splitting the requirement into functional and non-functional halves, roughly computing the resource need, and presenting every decision with a rationale. What the rationale rests on, however, went unnamed. "This design is fast," "this design scales," and "this design stays up" contain no metric; all three start an argument rather than end one, because once the other side reverses the same sentence, nothing measurable is left on the table. This topic defines those metrics. The first lesson starts with the pair most often confused: "median tracking response" and "request rate reaching the store," both of which appeared among the previous topic's thresholds, name two separate metrics with no necessary link between them. The lesson measures this independence in a setup, then shows where it ends and why. #### Three Flows and the Class of a Number The same system is used throughout the course: a **shipment tracking and billing service**. It has three flows. The **read** flow is the tracking query: the user supplies a tracking number and receives the shipment's latest state. The **write** flow is the state event coming from the carrier: small records stream in continuously. The **batch** flow is the seller's end-of-day billing: once a day, a very large job. This topic does not select new assumptions. The twelve-row assumption table from the Back-of-the-Envelope Estimation lesson, and the computed values it produces, are used here as input. The numbers needed for the three flows are: peak request rate at the edge **513.89 req/s** — 416.67 of it reads, 97.22 writes; peak rate reaching the store behind the cache **138.89 req/s**; the end-of-day job scans **12 million records spanning thirty days in a four-hour window**, that is **833.33 records/s** and 1.20 ms per record. All of these belong to the computed-value class, and none of them was taken from outside. The classification from the What Is System Design lesson applies to every number in this topic: a number is either a chosen **assumption**, a **computed value** derived from assumptions, or a **measurement** taken from a setup. This lesson's contribution is the third class — the computed values above state the required capacity but not what a single process can carry. That number gets measured. #### Two Metrics Are Two Separate Axes **Latency** is the time elapsed from the start of a piece of work to its end; its unit is time. The Network Models and Protocols course split it into four components: transmission, propagation, queueing, and processing. **Throughput** is the number of items of work completed per unit time; its unit is work divided by time. The two metrics have different units, so one cannot stand in for the other. The three flows place these two axes in separate corners. The tracking query wants low latency; the user waits for the response, and a single response's duration is directly visible to them. End-of-day billing wants high throughput; 12 million records must be scanned in four hours, and when any single record finishes does not matter. In one flow of the same system the unit of scale is the millisecond, in the other it is the hour. The critical consequence: increasing a flow's throughput can worsen its latency. The end-of-day job can multiply its throughput by grouping records, but every record in a group then waits for the group to finish. The reverse also happens: shortening latency can lower throughput, because starting every job immediately means giving up the gain grouping provides. There is one more asymmetry between the two metrics. Throughput is a single number: items of work completed per unit time are counted, and that is the end of it. Latency is not a single number; it is a **distribution** — under the same load some requests are answered quickly, others slowly. This is why latency is reported in **percentiles**: the median is the duration below which half of requests fall, p99 is the duration the top one percent exceeds. Reporting an average latency carries the least information, because a single very slow request skews the average and hides the distribution's shape. #### What Happens as Concurrency Increases The setup below sets up the tracking endpoint as a local server. Every request spends two things: a wait standing in for a record lookup, and a CPU share. Both are parameters of the setup, not measurements. ```js // tracking/server.mjs — tracking query endpoint: each request spends a wait and a CPU share import { createServer } from "node:http"; const WAIT = 3; // wait standing in for a record lookup (ms), parameter of the setup const CPU = 0.20; // CPU share per request (ms), parameter of the setup function burnCpu(ms) { // keeps the CPU busy for ms const end = performance.now() + ms; let t = 0; while (performance.now() < end) t += 1; return t; } const port = Number(process.argv[2]); if (Number.isInteger(port) === false) console.log("usage: node tracking/server.mjs "); else createServer(async (req, res) => { const no = new URL(req.url, "http://local").searchParams.get("no") ?? "-"; await new Promise((c) => setTimeout(c, WAIT)); burnCpu(CPU); const body = JSON.stringify({ no, state: "at transit hub", zone: "35", step: 4 }); res.writeHead(200, { "content-type": "application/json" }); res.end(body); }).listen(port); ``` The client runs at seven **concurrency** levels. At each level, the stated number of requests is kept in flight at all times; the number of requests sent over two seconds gives the throughput, and the individually measured durations give the latency percentiles. ```js // tracking/load.mjs — sends requests at increasing concurrency; each level prints throughput and latency import { Agent, request } from "node:http"; const port = Number(process.argv[2]); const LEVEL = [1, 2, 4, 8, 16, 32, 64]; const DURATION = 2000; // measurement duration per level (ms) const agent = new Agent({ keepAlive: true, maxSockets: 256 }); function oneRequest(no) { return new Promise((resolve, reject) => { const start = performance.now(); const r = request({ port, path: `/tracking?no=${no}`, agent }, (res) => { res.resume(); res.on("end", () => resolve(performance.now() - start)); }); r.on("error", reject); r.end(); }); } function percentile(arr, p) { const s = [...arr].sort((a, b) => a - b); return s[Math.min(s.length - 1, Math.floor((p / 100) * s.length))]; } async function level(n) { const latency = []; const end = performance.now() + DURATION; const start = performance.now(); await Promise.all(Array.from({ length: n }, async (_, k) => { let i = 0; while (performance.now() < end) latency.push(await oneRequest(`G${k}-${i++}`)); })); const elapsed = (performance.now() - start) / 1000; return { n, requests: latency.length, throughput: latency.length / elapsed, median: percentile(latency, 50), p99: percentile(latency, 99) }; } if (Number.isInteger(port) === false) console.log("usage: node tracking/load.mjs "); else { await oneRequest("warmup"); console.log("concurr requests throughput(req/s) median(ms) p99(ms) throughput*median"); for (const n of LEVEL) { const s = await level(n); const little = (s.throughput * s.median) / 1000; console.log(`${String(s.n).padStart(6)} ${String(s.requests).padStart(6)} ` + `${s.throughput.toFixed(0).padStart(13)} ${s.median.toFixed(2).padStart(10)} ` + `${s.p99.toFixed(2).padStart(7)} ${little.toFixed(2).padStart(22)}`); } agent.destroy(); } ``` The driver script starts the server, runs the measurement, and stops the server. The port number depends on the environment; it should be changed if it is already in use on the machine. {/* run: skip */} ```bash # measure.sh — starts the server, measures seven concurrency levels, stops the server node tracking/server.mjs 8811 & SERVER=$! sleep 1 node tracking/load.mjs 8811 kill $SERVER ``` Output varies from machine to machine; a sample run: ``` concurr requests throughput(req/s) median(ms) p99(ms) throughput*median 1 463 231 4.33 4.78 1.00 2 936 468 4.46 5.05 2.09 4 1764 880 4.80 5.38 4.23 8 3038 1515 5.67 6.14 8.59 16 4849 2417 6.65 7.57 16.06 32 8089 4031 8.21 9.56 33.10 64 8593 4268 15.15 16.46 64.64 ``` These numbers are measurements, taken on this machine. Their absolute values will not repeat on another machine; what repeats is the table's **shape**. #### Two Regions The table splits into two distinct regions, and the lesson's claim rests on this split. In the first region, as concurrency rises from 1 to 4, throughput climbs from 231 to 880, that is 3.81x; median latency climbs from 4.33 ms to 4.80 ms, that is 1.11x. Throughput grew nearly in step with concurrency while latency stood still. **In this region the two metrics are independent:** throughput is rising and says nothing about latency. In the second region the table reverses. As concurrency rises from 32 to 64, throughput climbs from 4031 to 4268, only 1.06x; median latency climbs from 8.21 ms to 15.15 ms, 1.85x. Doubling concurrency buys 6% more throughput and double the latency. **In this region independence ends:** throughput has hit an upper bound, and every added request only adds to the wait. Where the upper bound comes from is written into the setup. Every request spends 0.20 ms of CPU share, and the server runs on a single thread; the CPU share cannot be divided. Waits can overlap; a CPU share cannot. This is the distinction from the Server-Side Fundamentals course: waiting I/O is cheap, computation that occupies the CPU locks the queue. #### The Relation The last column ties the two metrics together. In every row, the product of throughput and median latency, once the latency is converted to seconds, comes out equal to the concurrency level: 1.00, 2.09, 4.23, 8.59, 16.06, 33.10, 64.64. This is not a coincidence; it is queueing theory's fundamental relation. The Caching, Queues and Asynchronous Processing course set it up as $L = \lambda W$; its counterpart here is: $$ N = X \times R $$ $N$ is the number of items of work in flight, $X$ is throughput, $R$ is latency. The design consequence of the relation is direct: **if two of the three quantities are chosen, the third cannot be chosen freely.** If the number of items in flight is held fixed and throughput cannot rise, latency must grow — there is no other way out. The behavior in the second region is this necessity made visible. The same relation also gives the rule for reading a measurement. A report that states only throughput says nothing about latency unless it also states the concurrency at which it was taken; a report that states only latency cannot be read either unless it states the load. Reporting a single metric on its own is an incomplete report. #### The Cost of Approaching Saturation A model shows what happens at the entrance to the second region. In a single-server queue, with service rate $\mu$ and arrival rate $\lambda$, **utilization** is defined as $\rho = \lambda / \mu$, and the average time spent in the system is $1 / (\mu - \lambda)$. The computation below feeds this model with the measured upper bound. **This is a model, not a measurement**; it carries assumptions about the distribution of arrivals and service times and does not predict a real server's latency. What it shows is the direction of the ratios. ```js // calc/queue.mjs — single-server queue MODEL: the latency cost of approaching saturation const MU = 4268; // measured upper bound: this setup's saturation throughput for a process (req/s) console.log(`service rate mu = ${MU} req/s (measured upper bound)`); console.log("utilization arrival rate(req/s) modeled average time(ms) ratio to 30% case"); const base = 1000 / (MU - 0.30 * MU); for (const rho of [0.30, 0.50, 0.70, 0.80, 0.90, 0.95, 0.99]) { const T = 1000 / (MU - rho * MU); console.log(`${(rho * 100).toFixed(0).padStart(6)}% ${(rho * MU).toFixed(0).padStart(19)} ` + `${T.toFixed(2).padStart(27)} ${(T / base).toFixed(1).padStart(18)}`); } const peak = [416.67, 833.33]; // Back-of-the-Envelope Estimation: peak read rate, and if the query assumption doubles for (const t of peak) { console.log(`peak ${t} req/s -> utilization ${((t / MU) * 100).toFixed(1)}%, ` + `upper bound is ${(MU / t).toFixed(1)}x peak, ` + `modeled average time ${(1000 / (MU - t)).toFixed(2)} ms`); } ``` ``` service rate mu = 4268 req/s (measured upper bound) utilization arrival rate(req/s) modeled average time(ms) ratio to 30% case 30% 1280 0.33 1.0 50% 2134 0.47 1.4 70% 2988 0.78 2.3 80% 3414 1.17 3.5 90% 3841 2.34 7.0 95% 4055 4.69 14.0 99% 4225 23.43 70.0 peak 416.67 req/s -> utilization 9.8%, upper bound is 10.2x peak, modeled average time 0.26 ms peak 833.33 req/s -> utilization 19.5%, upper bound is 5.1x peak, modeled average time 0.29 ms ``` The model says one thing: as utilization rises, throughput grows linearly while time does not. Going from 30% to 99% multiplies the arrival rate by 3.3 and the time by 70. The last slice of throughput is bought with unbounded growth in latency. Keeping utilization high looks like an efficiency win, and it is a latency decision. The last two lines join the computation to the measurement. Peak read rate is 416.67 req/s, the upper bound measured in the setup is 4268 req/s; the ratio is 10.2 and utilization stays at 9.8%. When queries per user double, the ratio becomes 5.1. What matters here is not the number itself but the **method**: the peak rate is computed from assumptions, the upper bound is measured from the setup, and their ratio gives the margin. A real tracking endpoint does more than this setup, so its upper bound will be lower; the ratio is still read the same way. When it falls below 1, the design no longer works; between 2 and 3, a single failure is narrow enough to erase the entire margin. #### Summary - Latency is the duration of a piece of work, throughput is the number of items of work completed per unit time; their units differ and one cannot substitute for the other. - In the measurement, as concurrency rose from 1 to 4, throughput climbed to 3.81x while median latency stayed at 1.11x: in this region the two metrics are independent. - As concurrency rose from 32 to 64, throughput climbed to 1.06x, latency climbed to 1.85x: once throughput hits its upper bound, every added request only adds to the wait. - $N = X \times R$ held in every measured row; if two of the three quantities are chosen the third cannot be chosen freely, so throughput or latency reported alone cannot be read. - The queueing model showed that time does not grow linearly with utilization: from 30% to 99%, arrival rate rises 3.3x while time rises 70x; keeping utilization high is a latency decision. - The computed peak read rate is 416.67 req/s, the measured upper bound is 4268 req/s; the ratio is 10.2 and falls to 5.1 when queries per user double. #### Next Step This lesson held a single process fixed and varied the load. The two regions that emerged show why "this system is fast" is insufficient: a system's speed depends on the load at which it was measured, and a number taken at low load says nothing about behavior at high load. Two separate questions follow. First, how good is the system at a given load. Second, how long does that goodness hold up as load grows. The two questions are often answered with a single word, and should not be: one is named performance, the other scalability. The next lesson separates the two concepts: it measures two implementations of the end-of-day job at four separate loads, shows that both meet today's threshold, and reveals that the difference between them can only be counted once the load is scaled up. ### Performance and Scalability Source: https://academia.sh/en/courses/introduction-to-system-design/performance-and-scalability Separating two concepts under load: showing that performance is the metric's value at a given load and scalability is how the metric changes with load, measuring two implementations of the end-of-day job at four loads, computing the largest load meeting the threshold as capacity, and separating a constant-factor gain from an order-of-growth gain. The previous lesson held a single process fixed and varied the load, producing two regions: one where throughput grows with concurrency, and one where throughput hits an upper bound and latency grows instead. The same system showing two different behaviors at two different loads also showed how little a single measurement says about a system. Two questions follow from this. How good is the system at a given load, and how long does that goodness hold up as load grows. This lesson answers the two questions with separate names and counts the difference on the end-of-day billing job. The only way to make the distinction visible is **measuring at more than one load**; a measurement at a single load does not even say which question it answers. #### A Point Versus a Curve **Performance** is the metric value measured at a given load: "this job takes 460 milliseconds on this machine with this many records" is a performance statement. It is a single point and says nothing outside that point. **Scalability** is how the metric changes with load: how many times the duration grows when the load doubles. It is not a point but a curve. Nothing about a system's scalability can be inferred from a single measurement, because a curve cannot be drawn through one point. Confusing the two concepts leads to a practical mistake: a change that improves performance is treated as having improved scalability. The measurement below shows why that is wrong. In the Quality Attributes lesson's list of quality families, performance falls under the performance-efficiency family; scalability is not in the list, and since the list is not closed, it is added here as a separate family. The two also differ in the measurement part of their quality scenarios: one states a threshold at a fixed load, the other states behavior as the load grows. #### Two Implementations of the Same Job End-of-day billing sums items into invoice lines. An invoice line corresponds to one seller's one day; from here on this grouping unit is called a **partition**, and the seller number together with the day form the **partition key**. The two implementations produce the same result by different routes. **A** scans every item for each partition. **B** passes over the items once and uses the partition key as an index. The number of items and the number of partitions are not independent. The assumption that gives the ratio stands as V13 in the Back-of-the-Envelope Estimation lesson's table: **100 items are grouped into one invoice line**. This produces a chain that holds with V3: 400,000 shipments a day, at this ratio, means 4000 invoice lines a day, that is 4000 sellers. The billing job scanning thirty days therefore sees 120,000 partitions: 4000 sellers times 30 days. The assumption's sensitivity will be shown in the computation. ```js // batch/grouping.mjs — two implementations of end-of-day billing measured at the same four loads const TARIFF = { tier: [[1, 3000], [5, 4800], [20, 9600]], zone: { "34": 100, "06": 115, "35": 125 }, minimum: 2500 }; const ZONES = ["34", "06", "35"]; const PER_PARTITION = 100; // V13: items grouped into one invoice line function items(n, partitions) { const d = new Array(n); for (let i = 0; i < n; i += 1) { d[i] = { partition: i % partitions, weight: (i % 20) + 1, zone: ZONES[i % 3], rate: (i % 5) / 20 }; } return d; } function net(k) { const tier = TARIFF.tier.find(([s]) => k.weight <= s) ?? [0, 9600]; const base = Math.max(TARIFF.minimum, Math.round((tier[1] * TARIFF.zone[k.zone]) / 100)); return base - Math.round(base * Math.min(k.rate, 0.4)); } function billingA(data, partitions) { // every item is scanned for each partition const result = new Array(partitions).fill(0); for (let b = 0; b < partitions; b += 1) { for (const k of data) if (k.partition === b) result[b] += net(k); } return result; } function billingB(data, partitions) { // single pass, aggregated by the partition key const result = new Array(partitions).fill(0); for (const k of data) result[k.partition] += net(k); return result; } function measure(f, data, partitions) { const start = performance.now(); const r = f(data, partitions); return { ms: performance.now() - start, signature: r[0] + r[partitions - 1] }; } // A ratio: how many times the duration grows when the load doubles. A order: log2(ratio) — 1 // linear, 2 quadratic growth. Raw ms depends on the machine, order is a property of the // implementation. console.log("items part. A(ms) B(ms) A/B A us/item B us/item A ratio order same"); let previousA = 0; for (const n of [20_000, 40_000, 80_000, 160_000]) { const partitions = n / PER_PARTITION; const data = items(n, partitions); const a = measure(billingA, data, partitions); const b = measure(billingB, data, partitions); const ratio = previousA === 0 ? null : a.ms / previousA; previousA = a.ms; console.log(`${String(n).padStart(7)}${String(partitions).padStart(10)}` + `${a.ms.toFixed(1).padStart(11)}${b.ms.toFixed(1).padStart(10)}${(a.ms / b.ms).toFixed(0).padStart(8)}` + `${((a.ms * 1000) / n).toFixed(2).padStart(12)}${((b.ms * 1000) / n).toFixed(2).padStart(12)}` + `${(ratio === null ? "-" : ratio.toFixed(2)).padStart(8)}${(ratio === null ? "-" : Math.log2(ratio).toFixed(2)).padStart(9)}` + `${String(a.signature === b.signature).padStart(6)}`); } ``` These numbers are measurements, taken on this machine; another machine gives different values. The last two columns exist for this reason: `A ratio` and `order` are quantities **derived** from the raw durations, and they do not change from machine to machine. The `same` column confirms the two implementations produce the same totals; the durations of two implementations that give different results cannot be compared. ``` items part. A(ms) B(ms) A/B A us/item B us/item A ratio order same 20000 200 13.3 1.4 9 0.67 0.07 - - true 40000 400 32.2 0.7 45 0.81 0.02 2.42 1.28 true 80000 800 121.9 1.8 66 1.52 0.02 3.78 1.92 true 160000 1600 471.7 2.9 160 2.95 0.02 3.87 1.95 true ``` #### What the Table Says Absolute durations are not read in the fourth and fifth columns, but in the **per-item** columns. B's per-item duration is 0.07 at the first load and 0.02 microseconds three times after; the difference in the first row comes from warm-up, the rest is constant. When the load rises eightfold, per-item duration does not change: **the job's cost is directly proportional to the number of items.** A's per-item duration rises from 0.67 to 2.95 microseconds in this run. As the load rises eightfold, per-item cost also grows, because every item is scanned for more partitions. The real reading is in the `order` column. A value of 1 would mean duration grows in direct proportion to load; 2 means duration grows with the square of the load. In the measurement, the last two doublings give 1.92 and 1.95, that is, **when the load doubles, the duration quadruples.** The first doubling stays at 1.28; the reason is that the runtime has not warmed up yet at the smallest load, and the computation below therefore takes its coefficient from the largest measurement, not the smallest. Absolute durations vary by machine, order does not — that is why the column was added. Now the critical observation. In the first row, the ratio between A and B is around 10, and A's absolute duration does not exceed fifteen milliseconds — no problem at all for a batch job. A performance measurement taken at this load finds A acceptable, and is right to. In the fourth row the ratio exceeds a hundred (160 in this run). **Same system, same code, same machine; the only thing that changed is the load.** The ratio itself is a function of the load, so the sentence "A is this many times slower than B" is meaningless when stated without a load. #### Capacity What turns the curve into a decision is a threshold. The Back-of-the-Envelope Estimation lesson computed that the end-of-day job runs in a four-hour window and scans 12 million records spanning thirty days. **Capacity** is the largest load that meets the threshold. The computation below takes the measured coefficients, finds each implementation's duration at today's volume and at two and four times that volume, then derives each one's capacity. ```js // calc/capacity.mjs — capacity in a four-hour window from measured coefficients, and the effect of a constant-factor speedup const WINDOW = 4 * 3600 * 1000; // V10: batch job window (ms) const DAILY = 400_000; // V3: new shipments created per day const DAYS = 30; // V11: days billing scans const TODAY = DAILY * DAYS; // scanned records const PER_PARTITION = 100; // assumption: items grouped into one invoice line console.log(`scanned records = ${TODAY}, daily partitions = ${DAILY / PER_PARTITION} (= seller), ` + `partitions over ${DAYS} days = ${TODAY / PER_PARTITION}`); const MEASURED = { n: 160_000, partitions: 1_600, aMs: 471.7, bMs: 2.9 }; // batch/grouping.mjs, last row const aUnit = MEASURED.aMs / (MEASURED.n * MEASURED.partitions); // ms / check const bUnit = MEASURED.bMs / MEASURED.n; // ms / item console.log(`input: a single run's measurement on this machine (A ${MEASURED.aMs} ms, B ${MEASURED.bMs} ms, ` + `n = ${MEASURED.n}); every duration and capacity below depends on these two numbers`); const aTime = (n) => aUnit * n * (n / PER_PARTITION); const bTime = (n) => bUnit * n; const hours = (ms) => ms / 3_600_000; console.log(`A unit = ${aUnit.toExponential(2)} ms/check, B unit = ${bUnit.toExponential(2)} ms/item`); console.log("records A time(hr) B time(hr) A fits window"); for (const mult of [1, 2, 4]) { const n = TODAY * mult; console.log(`${String(n).padStart(9)} (x${mult}) ${hours(aTime(n)).toFixed(3).padStart(13)} ` + `${hours(bTime(n)).toFixed(5).padStart(15)} ${String(aTime(n) <= WINDOW).padStart(17)}`); } const aCapacity = (unit) => Math.sqrt((WINDOW * PER_PARTITION) / unit); const bCapacity = (unit) => WINDOW / unit; console.log(`\nA capacity = ${(aCapacity(aUnit) / 1e6).toFixed(1)} million records ` + `(${(aCapacity(aUnit) / TODAY).toFixed(2)}x today)`); console.log(`B capacity = ${(bCapacity(bUnit) / 1e9).toFixed(0)} billion records ` + `(${(bCapacity(bUnit) / TODAY).toFixed(0)}x today)`); console.log(`if the implementation were 100x faster: A capacity x${(aCapacity(aUnit / 100) / aCapacity(aUnit)).toFixed(0)}, ` + `B capacity x${(bCapacity(bUnit / 100) / bCapacity(bUnit)).toFixed(0)}`); const half = aUnit * TODAY * (TODAY / (PER_PARTITION / 2)); console.log(`if the assumption were 50 instead of 100: A time ${hours(half).toFixed(3)} hr ` + `(x${(half / aTime(TODAY)).toFixed(0)}), B time unchanged`); ``` ``` scanned records = 12000000, daily partitions = 4000 (= seller), partitions over 30 days = 120000 input: a single run's measurement on this machine (A 471.7 ms, B 2.9 ms, n = 160000); every duration and capacity below depends on these two numbers A unit = 1.84e-6 ms/check, B unit = 1.81e-5 ms/item records A time(hr) B time(hr) A fits window 12000000 (x1) 0.737 0.00006 true 24000000 (x2) 2.948 0.00012 true 48000000 (x4) 11.793 0.00024 false A capacity = 28.0 million records (2.33x today) B capacity = 794 billion records (66207x today) if the implementation were 100x faster: A capacity x10, B capacity x100 if the assumption were 50 instead of 100: A time 1.474 hr (x2), B time unchanged ``` The first line shows the partition count holds with V3: 4000 partitions a day, that is 4000 sellers, 120,000 partitions over thirty days. The partition count should not be confused with the seller count — one is a daily figure, the other a period figure, and A's cost depends on the period figure. The second line states what the entire computation rests on: the input is the measurement of a single run. Every hour and capacity value below changes together with those two numbers; what stays fixed is the ratios between them and the shape of the curves. The third line is the lesson's most important result: **A meets today's threshold.** It scans 12 million records in 0.737 hours, that is forty-four minutes; well within the four-hour window. An acceptance test passes this implementation, and a performance report shows it as flawless. When volume doubles, A rises to 2.948 hours — still within the window, but its margin has shrunk fourfold. At four times volume it is 11.793 hours, and the window breaks. Duration quadrupling while volume doubles is the definition of a scalability problem; if it were a performance problem, duration would have doubled too. The capacity lines reduce the same thing to a single number. By this measurement, A carries 28.0 million records, 2.33 times today; the number changes on another machine, but being a small multiple of today does not. B carries 794 billion — this number is an arithmetic extrapolation, and the memory limit arrives long before that volume; its meaning is "in practice this job no longer runs into a time limit." The decision between the two implementations rests on these margins, not on the measured durations. #### A Constant Factor Does Not Beat the Order The last two lines separate two distinct kinds of improvement. If the implementation is sped up 100x — a cheaper comparison, a tighter loop, a better data layout — A's capacity rises only 10x. Because duration grows with the square of the load, capacity grows with the square root of the speedup. The same 100x gain raises B's capacity 100x. **A constant-factor gain does not pay back in kind on a solution with a poor order of growth.** The last line gives the assumption's sensitivity. If 50 items were grouped into an invoice line instead of 100, the partition count would double; A's duration rises to 1.474 hours, that is doubles, B's duration does not change at all. The assumption concerns only one implementation. When an assumption being wrong breaks one design without affecting the other, that is the real difference between the two designs. #### Summary - Performance is a single value measured at a given load; scalability is how the metric changes with load and cannot be inferred from a single measurement. - In the measurement, B's per-item duration stayed constant across an eightfold load increase (0.02 microseconds), while A's duration grew; the derived `order` column gave 1.92 and 1.95 in the last two doublings, that is, duration grows with the square of the item count. Raw durations depend on the machine, order does not. - The ratio between A and B is a function of the load: around 10 at the smallest load, exceeding a hundred at the largest. A ratio reported without a load is meaningless. - A meets today's threshold: by this measurement it scans 12 million records in 0.737 hours. At four times volume it rises to 11.793 hours and breaks the four-hour window. - By this measurement, capacity for A is 28.0 million records, 2.33 times today; for B the time limit is not binding in practice and the memory limit takes its place. - If the implementation speeds up 100x, A's capacity rises 10x, B's capacity rises 100x; a constant-factor gain does not beat the order of growth. #### Next Step This lesson varied the load and held the resource fixed: every measurement was taken with a single process, a single worker. The capacity computation is valid under this same assumption — so was the Back-of-the-Envelope Estimation lesson's 1.20-millisecond-per-record share. A resource is a quantity that can be changed, and there are two ways to change it: grow the same unit, or increase the number of units. The two hit different limits, require different conditions, and their costs grow along different curves. The next lesson measures the end-of-day job's speedup as the worker count rises from 1 to 16, finds where the speedup stops, and compares the cost of the two paths under the same threshold. ### Horizontal and Vertical Scaling Source: https://academia.sh/en/courses/introduction-to-system-design/horizontal-and-vertical-scaling The two ways to change a resource and each one's hard limit: measuring the speedup and efficiency of splitting the same job across one to sixteen workers, counting how the partitioning decision changes the serial fraction and the upper bound, measuring the limit that core count places on vertical scaling, and comparing the two paths' cost curves under the same target. The previous lesson varied the load and held the resource fixed; every measurement was taken with a single worker. The capacity computation was valid under that same assumption. The resource, however, is a quantity that can be changed, and there are two ways to change it: grow the same unit, or increase the number of units. This lesson separates the two paths, measures where each one stops, and compares their costs under the same target. The names of the distinction are as they appear in the catalog. **Vertical scaling** is growing the scaling unit itself: more CPU, more memory to the same unit. **Horizontal scaling** is increasing the number of units. The Server-Side Fundamentals course established that the scaling unit is the process, and that processes do not share memory; this lesson turns that rule's consequence into a measurement. #### Same Job, Growing Worker Count The job measured is end-of-day billing: 12 million records spanning thirty days are priced, and an audit trail is produced for every invoice line. The grouping unit set up in the previous lesson still holds — a **partition** is one seller's one day, so the thirty-day scan has 120,000 partitions (4000 sellers times 30 days). The job is split across one to sixteen workers, and the total duration is measured at each level. Records are not read, they are generated deterministically; what is measured is the parallelization of the compute share, not I/O. ```js // scale/worker.mjs — a worker: prices the items in its share, signs them, and returns partition totals import { parentPort, workerData } from "node:worker_threads"; const TARIFF = { tier: [[1, 3000], [5, 4800], [20, 9600]], zone: { "34": 100, "06": 115, "35": 125 }, minimum: 2500 }; const ZONES = ["34", "06", "35"]; function net(weight, zone, rate) { const tier = TARIFF.tier.find(([s]) => weight <= s) ?? [0, 9600]; const base = Math.max(TARIFF.minimum, Math.round((tier[1] * TARIFF.zone[zone]) / 100)); return base - Math.round(base * Math.min(rate, 0.4)); } function signature(text) { // audit trail of an invoice line let h = 2166136261; for (let i = 0; i < text.length; i += 1) h = Math.imul(h ^ text.charCodeAt(i), 16777619) >>> 0; return h; } function share({ start, end, step, partitions }) { const totals = new Map(); for (let i = start; i < end; i += step) { const b = i % partitions; const value = net((i % 20) + 1, ZONES[i % 3], (i % 5) / 20); const sig = signature(`G${i};${ZONES[i % 3]};${value}`); const previous = totals.get(b) ?? { net: 0, sig: 0 }; totals.set(b, { net: previous.net + value, sig: (previous.sig ^ sig) >>> 0 }); } return [...totals]; } if (workerData === null) console.log("this file is run as a worker by scale/run.mjs"); else parentPort.postMessage(share(workerData)); ``` ``` this file is run as a worker by scale/run.mjs ``` The item count is the product of V3 and V11 and comes from the Back-of-the-Envelope Estimation lesson's table; this lesson does not choose its own volume figure. The partition count is derived from V13: since a hundred items are grouped into an invoice line, twelve million items come down to 120,000 partitions. The partition count is not the seller count — 4000 sellers' thirty days make 120,000 seller-days. The driver splits the job in two different ways. **Index** partitioning divides items into contiguous blocks; a single partition's items are spread across every worker. **Key** partitioning advances with a stride; every partition falls to a single worker. Both process the same number of items and must produce the same result. ```js // scale/run.mjs — the same job split across 1..16 workers; speedup, efficiency, and merge share are measured import { Worker } from "node:worker_threads"; import { availableParallelism } from "node:os"; const ITEMS = 12_000_000; // V3 x V11: record count spanning thirty days const PARTITIONS = ITEMS / 100; // assumption: 100 items per invoice line -> 4000 sellers x 30 days const WORKERS = [1, 2, 4, 8, 12, 16]; function worker(data) { return new Promise((resolve, reject) => { const w = new Worker(new URL("./worker.mjs", import.meta.url), { workerData: data }); w.on("message", resolve); w.on("error", reject); }); } function ranges(n, mode) { // "index": contiguous block, "key": strided return mode === "index" ? Array.from({ length: n }, (_, s) => ({ start: Math.floor((s * ITEMS) / n), end: Math.floor(((s + 1) * ITEMS) / n), step: 1, partitions: PARTITIONS })) : Array.from({ length: n }, (_, s) => ({ start: s, end: ITEMS, step: n, partitions: PARTITIONS })); } async function run(n, mode) { const start = performance.now(); const shares = await Promise.all(ranges(n, mode).map(worker)); const parallel = performance.now() - start; const mergeStart = performance.now(); const invoice = new Map(); let collision = 0; for (const share of shares) { for (const [b, v] of share) { const previous = invoice.get(b); if (previous === undefined) invoice.set(b, { ...v }); else { previous.net += v.net; previous.sig = (previous.sig ^ v.sig) >>> 0; collision += 1; } } } const merge = performance.now() - mergeStart; const signature = [...invoice].reduce((a, [, v]) => (a ^ v.sig) >>> 0, 0); return { parallel, merge, total: parallel + merge, partitions: invoice.size, collision, signature }; } console.log(`cores = ${availableParallelism()}, items = ${ITEMS}, partitions = ${PARTITIONS}`); for (const mode of ["index", "key"]) { console.log(`\npartitioning = ${mode}`); console.log("worker parallel(ms) merge(ms) total(ms) speedup efficiency colliding records signature"); let base = 0; for (const n of WORKERS) { const r = await run(n, mode); if (n === 1) base = r.total; console.log(`${String(n).padStart(4)} ${r.parallel.toFixed(0).padStart(11)} ${r.merge.toFixed(1).padStart(15)} ` + `${r.total.toFixed(0).padStart(10)} ${(base / r.total).toFixed(2).padStart(8)} ` + `${(base / r.total / n).toFixed(2).padStart(10)} ${String(r.collision).padStart(13)} ${r.signature}`); } } ``` {/* mask: [ ]*[0-9][0-9.]*(?=(?:[ ]+[^ ]+){2,6}$) */} {/* mask: (?<=cores[ ]=[ ])[0-9]+ */} ``` cores = 12, items = 12000000, partitions = 120000 partitioning = index worker parallel(ms) merge(ms) total(ms) speedup efficiency colliding records signature 1 1674 8.4 1683 1.00 1.00 0 799471432 2 941 13.3 954 1.76 0.88 120000 799471432 4 602 24.6 626 2.69 0.67 360000 799471432 8 580 42.9 622 2.70 0.34 840000 799471432 12 883 70.6 953 1.76 0.15 1320000 799471432 16 882 68.3 950 1.77 0.11 1800000 799471432 partitioning = key worker parallel(ms) merge(ms) total(ms) speedup efficiency colliding records signature 1 1662 9.9 1672 1.00 1.00 0 799471432 2 879 5.6 885 1.89 0.94 0 799471432 4 549 5.5 554 3.02 0.75 0 799471432 8 321 6.8 328 5.11 0.64 0 799471432 12 284 5.5 289 5.79 0.48 0 799471432 16 364 6.9 370 4.51 0.28 0 799471432 ``` These numbers are measurements, taken on this machine; the core count is 12, and on another machine the table's break point falls elsewhere. The signature column gives the same value in all twelve runs (799471432), meaning every partitioning produced the same invoice; the durations of runs that give different results cannot be compared. The two single-worker base durations are 1683 and 1672 milliseconds; the difference between them is measurement noise, which is why speedup is read against each mode's own base. #### A Hard Limit With key partitioning, speedup grows from 1 to 12 workers: 1.89, 3.02, 5.11, and 5.79. At sixteen workers it **drops**: 4.51. This drop is not a measurement error; it is the vertical-scaling limit itself. The machine has 12 cores; sixteen workers are forced to share twelve cores, and the sharing itself is a cost. That growing a unit stops at some point is not an intuitive claim but a number read off the table. The efficiency column says the same thing from another angle. **Parallel efficiency** is the speedup obtained per worker, that is, speedup divided by worker count; it should not be confused with the Quality Attributes lesson's performance-efficiency family, which is the name of a quality family. In the measurement it is 0.94 at two workers, 0.75 at four, 0.64 at eight, 0.48 at twelve. **Every added worker returns less than the one before it.** The value dropping below 1 is not a flaw but the nature of parallel work; what matters for the decision is how fast it drops. #### The Condition for Horizontal Scaling The difference between the two partitionings is the lesson's second result. Same job, same worker count, same machine — at eight workers, speedup is 5.11 with key partitioning, 2.70 with index partitioning. A difference of nearly two times comes solely from **how the job is split**. The source of the difference is in the colliding-records column. With index partitioning, sixteen workers produced 1,800,000 colliding records: because a partition's total sits in pieces across multiple workers, every piece has to be summed at the merge step. With key partitioning this number is zero. Merge duration splits accordingly: 68.3 milliseconds versus 6.9 milliseconds at sixteen workers. The Server-Side Fundamentals course laid down the stateless-process rule: state that must survive between requests is moved outside the process. The measurement here is that rule's counterpart on the scaling side. Horizontal scaling does not happen just by increasing the number of units; it happens because **the job can be split in a way that requires no state shared between units.** The choice of partition key is therefore not a detail but the scalability decision itself. The terms partition and partitioning were set up in the Advanced SQL course in the context of table partitioning; the context here is splitting work across workers, not data across disk. #### Serial Fraction, Upper Bound, and Cost One more number can be derived from the measured speedup. Part of the job cannot be split — starting the workers, summing the shares, writing the result. If this **serial fraction** is $s$, the best possible speedup with $n$ workers is: $$ S(n) = \frac{1}{s + \dfrac{1 - s}{n}} $$ Solving the relation backward turns a measured speedup into a serial fraction, and as $n$ goes to infinity, speedup is bounded by $1/s$. The computation below does this for both partitionings, then compares the cost of meeting a target under the two paths. The vertical cost needs an **assumption**: how many times the unit's cost multiplies when its power doubles. Three values are given together because the result is sensitive to this assumption. ```js // calc/cost.mjs — serial fraction, upper bound, and the cost of the two paths from measured speedup const MEASURED = { key: { workers: 8, speedup: 5.11 }, index: { workers: 8, speedup: 2.70 } }; const MACHINE_LIMIT = 5.79; // measured: this machine's highest speedup for a single unit (12 cores) const serialFraction = (n, S) => (1 / S - 1 / n) / (1 - 1 / n); const amdahl = (s, n) => 1 / (s + (1 - s) / n); const requiredUnits = (s, K) => (1 - s) / (1 / K - s); console.log("partitioning measured(workers/speedup) serial fraction upper bound(1/s)"); for (const [name, o] of Object.entries(MEASURED)) { const s = serialFraction(o.workers, o.speedup); console.log(`${name.padEnd(12)}${`${o.workers} / ${o.speedup.toFixed(2)}`.padStart(21)}${(s * 100).toFixed(1).padStart(11)}%` + `${(1 / s).toFixed(1).padStart(17)}`); } const s = serialFraction(MEASURED.key.workers, MEASURED.key.speedup); console.log("\ntarget horizontal units horizontal cost vertical x2.0 vertical x2.5 vertical x3.0 vertical reachable"); for (const K of [2, 4, 8, 16]) { const units = requiredUnits(s, K); const vertical = (c) => c ** Math.log2(K); console.log(`${`x${K}`.padStart(5)}${(units > 0 ? units.toFixed(1) : "unreachable").padStart(13)}` + `${(units > 0 ? units.toFixed(1) : "-").padStart(16)}${vertical(2.0).toFixed(1).padStart(13)}` + `${vertical(2.5).toFixed(1).padStart(13)}${vertical(3.0).toFixed(1).padStart(13)}` + `${String(K <= MACHINE_LIMIT).padStart(15)}`); } console.log(`\namdahl forecast up to 16 workers (key partitioning, s = ${(s * 100).toFixed(1)}%):`); console.log([1, 2, 4, 8, 12, 16].map((n) => `${n}:${amdahl(s, n).toFixed(2)}`).join(" ")); ``` ``` partitioning measured(workers/speedup) serial fraction upper bound(1/s) key 8 / 5.11 8.1% 12.4 index 8 / 2.70 28.0% 3.6 target horizontal units horizontal cost vertical x2.0 vertical x2.5 vertical x3.0 vertical reachable x2 2.2 2.2 2.0 2.5 3.0 true x4 5.4 5.4 4.0 6.3 9.0 true x8 20.8 20.8 8.0 15.6 27.0 false x16 unreachable - 16.0 39.1 81.0 false amdahl forecast up to 16 workers (key partitioning, s = 8.1%): 1:1.00 2:1.85 4:3.22 8:5.11 12:6.35 16:7.23 ``` The serial-fraction lines reduce the partitioning decision's cost to a single number: 8.1% and an upper bound of 12.4 with key partitioning; 28.0% and an upper bound of 3.6 with index partitioning. Same job, same hardware — the upper bound diverges by more than three times. In a system where index partitioning was chosen, adding a thousand units instead of sixteen still would not push speedup past 3.6. The cost table shows the shape of the two curves. Horizontal cost is paid linearly in unit count, but the required unit count is not linear: 2.2 units for double, 5.4 for four times, 20.8 for eight times. The sixteen-times row shows "unreachable" instead of a number, because the target sits above the 12.4 upper bound the serial fraction imposes — no number of added units reaches it. Vertical cost, in contrast, depends entirely on the assumption: at a multiplier of 2.0, eight times costs 8.0 units — cheaper than horizontal; at 3.0 it costs 27.0 units — more expensive than horizontal. The last column settles the argument. Reaching eight times through the vertical path is **not possible** on this machine, because the highest measured speedup is 5.79. Vertical scaling's real problem is not the cost curve but the curve ending somewhere. The horizontal path is expensive, but its limit comes from the serial fraction, not the hardware — and the serial fraction is a design decision, as the measurement table's first row shows. The last line shows the model's own limit. The relation forecasts a speedup of 6.35 for twelve workers, 7.23 for sixteen; the measured values are 5.79 and 4.51. The model does not know the core count and never drops. **When a model's forecast diverges from a measurement, what needs correcting is not the measurement but the model's assumption** — here the missing assumption is that there is one core per worker. #### Summary - Vertical scaling is growing the scaling unit, horizontal scaling is increasing the number of units; the two hit different limits. - With key partitioning, speedup rose to 5.79 at 12 workers and dropped to 4.51 at 16: core count is the measured hard limit of the vertical path. - Parallel efficiency drops with every added worker: 0.94 at 2 workers, 0.64 at 8, 0.48 at 12. - How the job is split changed speedup by nearly two times: 5.11 versus 2.70 at eight workers. Index partitioning produced 1,800,000 colliding records and a 68.3 ms merge at 16 workers, key partitioning produced 0 and 6.9 ms. - The serial fraction derived from the measured speedup is 8.1% with key partitioning (upper bound 12.4), 28.0% with index partitioning (upper bound 3.6); the partition key is a scalability decision. - Reaching an eightfold target requires 20.8 units on the horizontal path, and is unreachable on the vertical path on this machine; sixteenfold is unreachable on the horizontal path too, because the serial fraction holds the upper bound at 12.4. #### Next Step The three lessons so far measured under the assumption that the system **works**. Load, capacity, speedup, and cost — all of these are numbers for a system that is up. The rationale for scaling is not load alone either: when a setup that scans twelve million records in 289 milliseconds is already sufficient, there must be another reason to increase the unit count. That reason is units not staying up. When a unit goes down, how much of the system goes down, for how long, and is that duration acceptable? The third question is answered with a percentage, and a percentage alone says nothing: the difference between ninety-nine percent and ninety-nine point nine percent stays invisible until it is written in minutes. The next lesson turns service availability into an outage budget and computes what combining components does to that budget. ### Availability in Numbers Source: https://academia.sh/en/courses/introduction-to-system-design/availability-in-numbers Converting a service availability percentage into an outage budget: converting percentages into per-year, per-month, per-week, and per-day duration, splitting the budget between planned work and failure, raising the target making planned work impossible, and computing the effect of multiplication in a series chain and of the complement in replication on composite service availability. The previous three lessons measured under the assumption that the system works: load, capacity, speedup, and cost. When a unit goes down, different questions apply. How much of the system goes down, for how long, and is that duration acceptable? The third question is answered with a percentage, and a percentage alone says nothing. This lesson converts the percentage into a duration, treats that duration as a spendable budget, and computes what combining components does to the budget. Every number belongs to the computed-value class; its inputs are explicitly named assumptions. #### Two Terms and Two Definitions **Service availability** is the ratio of the time the system can respond to requests to the total time. The term should not be confused with interface **accessibility**, which is an interface's usability with assistive technologies and the subject of the Accessible Component Patterns course. In this lesson, the term always means service availability, never accessibility. The ratio has two bases, and the same outage looks different under each. The **time-based** measurement divides uptime by total time; outage minutes are counted. The **request-based** measurement divides successful requests by total requests; the number of affected users is counted. The two diverge because traffic is not evenly distributed: since the Back-of-the-Envelope Estimation lesson's peak factor is 3, five minutes of downtime falling in the peak hour affects three times as many requests as five minutes falling at average load. This lesson builds the budget on the time base, because a budget is a quantity set aside in advance, allocated without knowing the traffic. #### Converting a Percentage into a Duration The percentage format misleads the comparison: the 0.09-point gap between 99.9% and 99.99% looks small. The same gap stops being invisible once it is converted into minutes. ```js // service/budget.mjs — converting a service availability percentage into a per-period outage budget const PERIOD = [["year", 365 * 24 * 60], ["month", 30 * 24 * 60], ["week", 7 * 24 * 60], ["day", 24 * 60]]; const PERCENT = [99, 99.5, 99.9, 99.95, 99.99, 99.999]; const format = (min) => (min >= 60 ? `${(min / 60).toFixed(1)} hr` : min >= 1 ? `${min.toFixed(1)} min` : `${(min * 60).toFixed(1)} sec`); console.log(`${"percent".padEnd(9)}${PERIOD.map(([a]) => a.padStart(12)).join("")}`); for (const p of PERCENT) { const fraction = 1 - p / 100; console.log(`${`${p}%`.padEnd(9)}${PERIOD.map(([, min]) => format(fraction * min).padStart(12)).join("")}`); } const first = 1 - 99.9 / 100; const second = 1 - 99.99 / 100; console.log(`\n99.9% -> 99.99%: monthly budget ${format(first * 43200)} instead of ${format(second * 43200)}, ` + `${(first / second).toFixed(0)}x narrower`); console.log(`adding one nine divides the outage share by 10 each time; a 0.09-point gap ` + `means ${format((first - second) * 43200)} monthly`); ``` ``` percent year month week day 99% 87.6 hr 7.2 hr 1.7 hr 14.4 min 99.5% 43.8 hr 3.6 hr 50.4 min 7.2 min 99.9% 8.8 hr 43.2 min 10.1 min 1.4 min 99.95% 4.4 hr 21.6 min 5.0 min 43.2 sec 99.99% 52.6 min 4.3 min 1.0 min 8.6 sec 99.999% 5.3 min 25.9 sec 6.0 sec 0.9 sec 99.9% -> 99.99%: monthly budget 43.2 min instead of 4.3 min, 10x narrower adding one nine divides the outage share by 10 each time; a 0.09-point gap means 38.9 min monthly ``` A month is taken as thirty days; since months are not equal in length, the budget's period must be defined up front, or the same percentage gives a different minute count in February. The table's first result is a difference of scale. In spoken discussion, "ninety-nine percent available" sounds strong; its counterpart is 87.6 hours a year, an outage exceeding three and a half days. Ninety-nine point nine nine nine percent is 5.3 minutes in the same year — a single restart spends the entire budget. The second result is a measure of the period choice. 99.9% gives 43.2 minutes in a month, 1.4 minutes in a day. The two numbers describe the same target but permit different behavior: the monthly budget accepts a single 43-minute outage, the daily budget does not. **A target's period is as decisive as the target itself.** #### The Budget Gets Spent The real use of an outage budget is turning an outage from something that just happens into a resource that gets spent. Part of the budget is spent deliberately: processes shut down and restart at every release, the service stops during planned maintenance. What remains is the share set aside for failure. The computation below takes four assumptions. **Target 99.9%**; its rationale is that the tracking query carries a commercial commitment but is not a payment flow. **20 deployments a month** and **15 seconds of unresponsiveness per deployment**; the rationale is the old process shutting down and the new one starting at every release. **10 minutes of planned maintenance a month.** And a **recovery time of 10 minutes**; the same order of magnitude as the recovery time target in the Relational Database Administration course. ```js // service/spend.mjs — splitting the monthly outage budget into planned work and failure const MONTH = 30 * 24 * 60; // minutes const TARGET = 99.9; // assumption: service availability target const DEPLOYS = 20; // assumption: monthly deployment count const DEPLOY_SEC = 15; // assumption: unresponsiveness per deployment (seconds) const MAINT_MIN = 10; // assumption: monthly planned maintenance (minutes) const RECOVERY_MIN = 10; // assumption: recovery time of a failure (minutes) function budget(target, deploys) { const total = (1 - target / 100) * MONTH; const planned = (deploys * DEPLOY_SEC) / 60 + MAINT_MIN; return { total, planned, failure: total - planned, count: (total - planned) / RECOVERY_MIN }; } console.log("target deploys monthly budget planned left for failure failures absorbed"); for (const [target, deploys] of [[TARGET, DEPLOYS], [TARGET, DEPLOYS * 2], [99.99, DEPLOYS]]) { const b = budget(target, deploys); console.log(`${`${target}%`.padEnd(9)}${String(deploys).padStart(7)}` + `${b.total.toFixed(1).padStart(14)} min${b.planned.toFixed(1).padStart(9)} min` + `${b.failure.toFixed(1).padStart(16)} min${b.count.toFixed(2).padStart(19)}`); } const b = budget(TARGET, DEPLOYS); console.log(`\nplanned work's share of the budget = ${((b.planned / b.total) * 100).toFixed(1)}%`); console.log(`if the target were 99.99%, planned work would be ${((budget(99.99, DEPLOYS).planned / budget(99.99, DEPLOYS).total) * 100).toFixed(0)}% of the budget`); ``` ``` target deploys monthly budget planned left for failure failures absorbed 99.9% 20 43.2 min 15.0 min 28.2 min 2.82 99.9% 40 43.2 min 20.0 min 23.2 min 2.32 99.99% 20 4.3 min 15.0 min -10.7 min -1.07 planned work's share of the budget = 34.7% if the target were 99.99%, planned work would be 347% of the budget ``` The first line shows how the budget splits: 15.0 of the 43.2 minutes goes to planned work, that is, 34.7% of the budget is already spent before a single failure occurs. The 28.2 minutes left for failure, at a ten-minute recovery time, means 2.82 failures a month: two failures fit the budget, a third breaks the target. The second line is the assumption's sensitivity: when the deployment count doubles, the failure share drops from 28.2 to 23.2 minutes, the absorbable failures from 2.82 to 2.32. Releasing more often means less headroom for failure — both are paid from the same budget. The third line is the lesson's harshest result. When the target is raised to 99.99%, the monthly budget shrinks to 4.3 minutes, and planned work's 15.0 minutes takes up **347% of the budget**; the failure share is negative 10.7 minutes. The target cannot be met even with zero failures. **Raising a service availability target requires changing the deployment pattern as well**; otherwise the target remains a statement of intent. What makes an outage budget a metric is its ability to flag a target as impossible. #### Multiplication in a Composite System Answering a request requires multiple components to work together. The tracking query arrives at the edge process, checks the cache, goes to the shipment store if needed, and is fed by the state event stream. If the request cannot be answered when any one component goes down, the components are connected **in series**, and composite service availability is a product. ```js // service/composite.mjs — multiplication in a series chain, complement in replication: the tracking path's composite service availability const MONTH = 30 * 24 * 60; // minutes const CHAIN = ["edge process", "cache", "shipment store", "event stream"]; const COMPONENT = 0.999; // assumption: service availability per component const minutes = (a) => (1 - a) * MONTH; const series = (a, n) => a ** n; const replicated = (a, k) => 1 - (1 - a) ** k; console.log(`chain = ${CHAIN.length} components, ${(COMPONENT * 100).toFixed(1)}% per component`); console.log("components composite service availability monthly outage"); for (let n = 1; n <= CHAIN.length; n += 1) { const a = series(COMPONENT, n); console.log(`${String(n).padStart(14)} ${(a * 100).toFixed(4).padStart(21)}% ${minutes(a).toFixed(1).padStart(9)} min`); } console.log("\nreplicas component outage share chain outage share monthly outage"); for (const k of [1, 2, 3]) { const b = replicated(COMPONENT, k); const z = series(b, CHAIN.length); console.log(`${String(k).padStart(12)} ${(1 - b).toExponential(2).padStart(20)} ` + `${(1 - z).toExponential(2).padStart(19)} ${(minutes(z) * 60).toFixed(3).padStart(9)} sec`); } const better = 0.9999; console.log(`\nsensitivity: if the component were ${(better * 100).toFixed(2)}%, the unreplicated chain would be ` + `${(series(better, CHAIN.length) * 100).toFixed(4)}%, ${minutes(series(better, CHAIN.length)).toFixed(1)} min monthly`); console.log(`weakest link: the chain cannot exceed any single component (upper bound ${(COMPONENT * 100).toFixed(1)}%)`); ``` ``` chain = 4 components, 99.9% per component components composite service availability monthly outage 1 99.9000% 43.2 min 2 99.8001% 86.4 min 3 99.7003% 129.5 min 4 99.6006% 172.5 min replicas component outage share chain outage share monthly outage 1 1.00e-3 3.99e-3 10352.458 sec 2 1.00e-6 4.00e-6 10.368 sec 3 1.00e-9 4.00e-9 0.010 sec sensitivity: if the component were 99.99%, the unreplicated chain would be 99.9600%, 17.3 min monthly weakest link: the chain cannot exceed any single component (upper bound 99.9%) ``` The first table gives the direction of the multiplication: even though each component is 99.9%, the four-component chain is 99.6006%, and the monthly outage rises from 43.2 to 172.5 minutes, nearly fourfold. Outage shares add up approximately. The design consequence is direct: **adding a component lowers service availability**, because in a series chain every new component is a new single point. The Architectural Styles course counted a microservice layout's bill as a state count that grows with the unit count; its counterpart here is multiplication. The same table also gives an upper bound. A series chain cannot exceed its weakest component: without replication, a path containing a 99.9% component cannot be 99.95%. If a target is set without being broken down into component targets, whether it holds cannot be tested by looking at the components. The second table shows the one tool that works in the opposite direction: placing multiple copies of the same component. Copies are connected **in parallel** — the component stays up unless all of them go down together — and the outage share is raised to a power by the copy count. A component's outage share falls from one in a thousand to one in a million with a second copy, one in a billion with a third; the four-component chain's monthly outage drops from 10,352 seconds to 10.4 seconds, then to 0.010 seconds. This second table carries an assumption that does not fully hold in reality: **that copies fail independently.** Two copies running the same faulty release go down together on the same request; the same configuration error breaks both at once. To the extent independence does not hold, real service availability falls below the computed value, so these numbers are an **upper bound.** Adding a copy is also not as cheap as it looks in the arithmetic; keeping copies unaware of each other's failures incurs another cost, and that cost is the subject of the rest of this topic. The last line gives the sensitivity: if the component's service availability were 99.99%, the unreplicated chain would be 99.9600% and the monthly outage 17.3 minutes. The same outcome can be reached two ways — improving the component or replicating it — and the cost of each is compared using the previous lesson's cost curves. #### Summary - Service availability is the ratio of the time the system can respond and is a concept separate from interface accessibility; time-based and request-based measurement show the same outage differently. - 99.9% is 8.8 hours a year, 43.2 minutes a month, 1.4 minutes a day of outage; adding one nine divides the share by 10, and a 0.09-point gap means 38.9 minutes a month. - A target's period is as decisive as the target: a monthly 43.2 minutes permits a single long outage, a daily 1.4 minutes does not. - 34.7% of the budget goes to planned work (20 deployments and 10 minutes of maintenance); the 28.2 minutes left for failure, at a ten-minute recovery time, is 2.82 failures a month. - When the target is raised to 99.99%, planned work takes up 347% of the budget and the target cannot be met even with zero failures; raising the target requires changing the deployment pattern. - In a series chain, service availabilities multiply: four 99.9% components give 99.6006% and 172.5 minutes a month. In parallel copies, the outage share is raised to a power, but the computation assumes copies fail independently, so it is an upper bound. #### Next Step This lesson converted service availability into a budget and counted how the budget gets spent: deployment, maintenance, failure. The open question is **why** the budget gets spent. Part of the outage comes from failure — a component goes down, and minutes leave the budget until it recovers. Part comes from a deliberate decision, and that part is invisible in the arithmetic. The bill for adding a copy was left unfinished in the last table: when copies are unaware of each other, the system has to choose between two things. Does it choose to respond, or to respond correctly? Choosing to respond preserves the budget and returns a wrong value; waiting or refusing in order to respond correctly spends from the budget. The next lesson names this choice, measures two policies under the same failure scenario, and counts how many minutes of the outage budget go to which policy. ### Availability and Consistency Tension Source: https://academia.sh/en/courses/introduction-to-system-design/availability-and-consistency-tension Reading the CAP theorem correctly: showing that partition tolerance is not a choice, measuring the answered-request and stale-response counts of the respond and reject policies in the same network partition scenario, calculating the outage budget the reject policy spends and the stale-response count the respond policy produces, and distinguishing the theorem's common misreadings. The previous lesson turned service availability into a budget: percentages became annual outage duration, and how that budget gets spent was counted. The open question is why it gets spent. Part of an outage comes from failure; part comes from a deliberate decision. When one replica loses touch with the other, the system must choose between two things: responding, or responding correctly. This lesson names that choice. It is not an arbitrary preference but a decision every system standing on a network is forced to make, usually without realizing it. Its name is the **CAP theorem**, the field's most misread sentence. #### Three Letters and the Letter Not Chosen The theorem names three properties. **Consistency**: every read sees the most recently acknowledged write; the system behaves as though it had only one copy. **Service availability**: every request to a running replica gets a response — not an error, a response. This must not be confused with interface accessibility, covered in the Accessible Component Patterns course and absent from this lesson. **Partition tolerance**: the system keeps working even when messages between replicas are lost. The network partition in the third term is not the partitioning covered in the Relational Database Administration course: partitioning is deliberately splitting data into pieces; a network partition is replicas losing sight of each other. Same root, two separate phenomena. The misreading starts here, because the three letters get presented as three options. They are not. A network partition is not a design choice but a fact: a cable breaks, a switch restarts, a zone cannot hear another for a few seconds. There is no "not choosing" this fact — only what the system does once it occurs. The theorem's actual statement: **while a network partition lasts, consistency and service availability cannot both be provided.** With no partition, the theorem is silent. Replication and replication lag were established in the Relational Database Administration course and are not retold here. What gets measured is what a policy does once the replicas' link is broken. #### The Two-Zone Model Instead of standing up a real cluster, the behavior is shown with an in-process model: replicas are objects, and whether the link is down is an explicit parameter. Every number below comes from running this model, not a real network; the model reads no clock, so it comes out the same every run. The shipment tracking service has two zones: carrier status events arrive at zone `b34`, and tracking queries can arrive at either zone. ```js // cap/model.mjs — two-zone replica model: replicas are objects, link and messages are open parameters export function build() { const state = { link: true, messages: 0 }; const replica = { b34: new Map(), b35: new Map() }; const neighbor = { b34: "b35", b35: "b34" }; function write(zone, key, value, round) { replica[zone].set(key, { value, round }); if (state.link === false) return { propagated: 0 }; state.messages += 1; replica[neighbor[zone]].set(key, { value, round }); return { propagated: 1 }; } // "respond": returns the local record without verifying it. "reject": a response is given // only when it can be verified with the neighbor; if the link is down the request is rejected. function read(zone, key, policy) { const local = replica[zone].get(key) ?? null; if (policy === "respond") return { response: local, rejected: false }; if (state.link === false) return { response: null, rejected: true }; state.messages += 1; return { response: local, rejected: false }; } return { state, replica, write, read }; } ``` The two policies are separated in the code by a single line. The reject policy returns nothing while the link is down; even holding a value, it will not respond, because it cannot verify that value is the latest one. The entire trade-off lives inside that one line. #### Same Scenario, Two Policies The driver script runs a twelve-round scenario. In every round, a carrier status event is written to zone `b34` and a tracking query arrives; the query lands on the two zones in turn. The link is down from round 3 through round 8. ```js // cap/measure.mjs — same scenario with two read policies: answered, rejected, and stale counts import { build } from "./model.mjs"; const TRACKING = "TR-9042"; const EVENTS = ["accepted", "departed", "transfer-34", "transfer-41", "line-35", "out-for-delivery", "delivery-attempt", "address-verification", "redelivery", "delivered", "signed", "closed"]; const PARTITION = [3, 8]; // link is down between these rounds (both ends included) function run(policy) { const m = build(); const s = { requests: 0, responses: 0, rejected: 0, stale: 0, trace: [] }; for (let round = 1; round <= EVENTS.length; round++) { m.state.link = round < PARTITION[0] || round > PARTITION[1]; m.write("b34", TRACKING, EVENTS[round - 1], round); // the carrier event always arrives at b34 const actual = m.replica.b34.get(TRACKING); const zone = round % 2 === 1 ? "b35" : "b34"; // the tracking query alternates between the two zones const r = m.read(zone, TRACKING, policy); s.requests += 1; if (r.rejected) { s.rejected += 1; s.trace.push(`${zone}:rejected`); continue; } s.responses += 1; const stale = r.response.round < actual.round; if (stale) s.stale += 1; s.trace.push(`${zone}:${r.response.value}${stale ? "~" : ""}`); } return { ...s, messages: m.state.messages }; } const windowOf = (a) => a.slice(PARTITION[0] - 1, PARTITION[1]); console.log(`scenario: ${EVENTS.length} rounds, link down in rounds ${PARTITION[0]}-${PARTITION[1]}`); console.log("policy | resp. | rej | stale | msgs. | partition window"); console.log("------------|-------|-----|-------|-------|------------------"); const results = {}; for (const p of ["respond", "reject"]) { const s = run(p); results[p] = s; console.log(`${p.padEnd(11)} | ${String(`${s.responses}/${s.requests}`).padStart(5)} | ` + `${String(s.rejected).padStart(3)} | ${String(s.stale).padStart(5)} | ` + `${String(s.messages).padStart(5)} | ${windowOf(s.trace).join(" ")}`); } const pw = PARTITION[1] - PARTITION[0] + 1; const count = (p, k) => windowOf(results[p].trace).filter(k).length; console.log(`in the partition window (${pw} rounds): respond stale = ` + `${count("respond", (x) => x.endsWith("~"))}/${pw}, reject rejected = ` + `${count("reject", (x) => x.endsWith("rejected"))}/${pw}`); console.log(`outside the partition (${EVENTS.length - pw} rounds): both policies gave the same response`); ``` ```sh node cap/measure.mjs ``` ``` scenario: 12 rounds, link down in rounds 3-8 policy | resp. | rej | stale | msgs. | partition window ------------|-------|-----|-------|-------|------------------ respond | 12/12 | 0 | 3 | 6 | b35:departed~ b34:transfer-41 b35:departed~ b34:out-for-delivery b35:departed~ b34:address-verification reject | 6/12 | 6 | 0 | 12 | b35:rejected b34:rejected b35:rejected b34:rejected b35:rejected b34:rejected in the partition window (6 rounds): respond stale = 3/6, reject rejected = 6/6 outside the partition (6 rounds): both policies gave the same response ``` #### Reading the Numbers The last line shows where the theorem stays silent: in the six rounds outside the partition, the two policies cannot be told apart. The trade-off exists only inside the window, and there it is sharp. The respond policy answered 12/12 requests and rejected none; in exchange it produced 3 stale responses. The `~` mark in the trace flags a stale response: zone `b35` kept returning `departed` throughout the partition, while the shipment had already moved to `out-for-delivery` and `address-verification`. The query got a response, but the response was not telling the truth. The reject policy answered 6/12 requests, turning back all six inside the partition window and producing not one stale response. One detail is easy to miss: it also rejected requests arriving at zone `b34` — even though `b34` held the newest value every round. The replica cannot know its value is the newest; it cannot learn, without asking its neighbor, whether the neighbor holds a write it knows nothing about. The guarantee requires not returning the correct value but being able to verify it; cut the verification channel and the guarantee is cut too. The message column counts a third cost. Respond spent 6 messages: the six writes propagated only while the link was up. Reject spent 12 — the same six propagations plus one verification message per answered read. This has nothing to do with the partition; it would be there without one too. That difference is the next lesson's subject. #### Where the Budget Goes The rates the model produces convert into an annual budget. The inputs are **assumptions**: how many times a year a network partition occurs and how long it lasts, and how many tracking queries arrive per second during it. The second run shows the assumption's sensitivity: what happens if the partition duration doubles. ```js // cap/budget.mjs — converts the model's rates into an annual budget; the inputs are assumptions const YEAR = 365 * 24 * 3600; const ASSUMPTION = { partition: 12, duration: 40, query: 30 }; // times/year, seconds, tracking queries/second const MODEL = { rejectRate: 6 / 6, staleRate: 3 / 6 }; // rates from the measurement const availabilityPercent = (s) => (100 * (1 - s / YEAR)).toFixed(4); for (const factor of [1, 2]) { const secs = ASSUMPTION.partition * ASSUMPTION.duration * factor; const queries = secs * ASSUMPTION.query; console.log(`partition ${secs} sec/year (${ASSUMPTION.partition} times x ${ASSUMPTION.duration * factor} sec)` + ` -> ${queries} tracking queries in the window`); console.log(` reject : rejected = ${Math.round(queries * MODEL.rejectRate)} requests,` + ` service availability = ${availabilityPercent(secs * MODEL.rejectRate)}%, stale = 0`); console.log(` respond : rejected = 0 requests,` + ` service availability = ${availabilityPercent(0)}%, stale = ` + `${Math.round(queries * MODEL.staleRate)} responses`); } ``` ``` partition 480 sec/year (12 times x 40 sec) -> 14400 tracking queries in the window reject : rejected = 14400 requests, service availability = 99.9985%, stale = 0 respond : rejected = 0 requests, service availability = 100.0000%, stale = 7200 responses partition 960 sec/year (12 times x 80 sec) -> 28800 tracking queries in the window reject : rejected = 28800 requests, service availability = 99.9970%, stale = 0 respond : rejected = 0 requests, service availability = 100.0000%, stale = 14400 responses ``` The two policies pay for the same fact in two different currencies. Reject spent 480 seconds of budget and cut service availability to 99.9985%, producing no wrong information in exchange. Respond spent nothing from the budget but told 7,200 tracking queries a stale status. When the assumption doubled, both numbers grew linearly: 99.9970% and 14,400. The real conclusion: the two costs are not comparable. An outage ends; an incorrectly reported delivery status has already interfered with whoever looked at it and made a decision. So the decision rests on which error the flow can tolerate, not the size of the numbers. A tracking query showing a status a few seconds stale is tolerable; end-of-day billing invoicing against a stale status is not. Two flows in the same system, two separate policies. #### The Theorem's Misreadings Four misreadings are common; the measurement refutes all four. **"Pick two of three."** The theorem offers no menu. Partition tolerance is not a chosen property but the consequence of standing on a network. The choice is between two letters, and it applies only while a partition lasts. **"Our system is CA."** For a system distributed over a network this is not a class but a design that never considered a partition. When a partition arrives the system makes a choice regardless; if the code never wrote that choice down, the library's default behavior makes it instead. The measurement's one-line difference shows exactly where that default is hiding. **"The choice applies to the whole system."** Both runs in the measurement used the same model; only the read policy differed. A policy can be given per request — the tracking query can run under respond while the billing read runs under reject. **"Choosing consistency just makes the system slower — that is all."** The message column shows this is half true: verification does spend messages. But what a partition spends is not time, it is the response itself — 6 requests were rejected. Two separate costs, handled separately. #### Summary - The CAP theorem offers no three-way choice: a network partition is a fact, and the choice between consistency and service availability applies only while it lasts. - In the same twelve-round scenario, respond answered 12/12 requests and produced 3 stale responses; reject answered 6/12, rejected 6 requests, and produced 0 stale responses. - In the six rounds outside the partition, the two policies could not be told apart; the trade-off exists only inside the window. - Reject also rejected requests at the zone holding the newest value: the guarantee requires not the correct value but the ability to verify it. - Under the assumption of 12 partitions a year lasting 40 seconds each, reject rejected 14,400 requests and cut service availability to 99.9985%; respond spent nothing from the budget but produced 7,200 stale responses. Doubling the assumption doubled both numbers. - The two costs are not in the same unit; the decision is made per request, based on which error the flow can tolerate. #### Next Step One column in the measurement was left unexplained. Reject spent 12 messages, respond spent 6, and that difference had nothing to do with the partition: verification messages were sent while the link was up too. So the choice does not end even without a network partition. There remains a decision between having every write acknowledged by both replicas and writing to one while waiting for it to propagate to the other; the cost is not a rejected request but a number of rounds waited. The next lesson names this region where CAP stays silent, and measures three things there: the acknowledgment-wait step, the staleness window seen on reads, and the number of messages per write. ### PACELC Source: https://academia.sh/en/courses/introduction-to-system-design/pacelc The latency trade-off outside a partition: naming the region where the CAP theorem stays silent, measuring the synchronous-acknowledgment and asynchronous-propagation policies with no partition in terms of the acknowledgment-wait step, messages per write, and the staleness window seen on reads, showing the sensitivity of the arrival-interval and latency assumptions, and classifying a system separately on its two branches. One column in the previous lesson's measurement was left unexplained. Reject spent 12 messages, respond spent 6; the difference came from every answered read being verified with the neighbor, and those messages were sent while the link was up too. So the decision about consistency does not end even if the network never partitions. CAP is silent in this region: with no partition, the theorem has nothing to say. Yet the decision is still there, paid on every request. The **PACELC** theorem fills the place where CAP stays silent, and that is this lesson's subject. #### The Theorem's Second Branch PACELC is a two-branch sentence. The first branch is what CAP says: if there is a **partition** (P), a choice is made between **service availability** (A) and **consistency** (C). The second branch is new: **else** (E — with no partition), a choice is made between **latency** (L) and **consistency** (C). The second branch's rationale is a simple fact: for a replica to verify its held value is current, it must talk to its neighbor. That conversation crosses the network and has a cost, independent of any partition — even with a perfectly healthy link, every verification needs a round trip. So consistency is not free even with no outage; only the currency paid changes — not a rejected request, but time waited. #### Two Write Policies With No Partition The choice being measured lies on the write path. A carrier status event can be handled two ways: have both replicas acknowledge every write (`acknowledged`), or write to one replica, acknowledge the carrier right away, and propagate to the neighbor afterward (`propagated`). The model below runs both policies over the same event sequence, with the link never down. This is a model, not a measurement: a round is an abstract time step, latency `D` is a parameter in rounds, and no clock is read anywhere — so the output is the same on every run. ```js // pacelc/model.mjs — two WRITE policies while there is NO partition. A round is an abstract time // step; the latency D is a model parameter in rounds, not a measured duration. Events for the // same tracking number are ordered, so at most one write is ever in flight. export function run({ rounds: N, latency: D, interval, policy }) { const write = new Map(); // writeNo -> { arrival, accepted, visible } const queue = []; let flight = null, messages = 0, seq = 0; for (let t = 1; t <= N; t++) { if ((t - 1) % interval === 0) { seq += 1; write.set(seq, { arrival: t, accepted: null, visible: null }); queue.push(seq); } if (policy === "acknowledged") { if (flight !== null && flight.ends === t) { const w = write.get(flight.no); w.accepted = t; // acknowledgment arrived: the write becomes visible on both replicas in the same round w.visible = t; messages += 2; // outbound + acknowledgment flight = null; } if (flight === null && queue.length > 0) flight = { no: queue.shift(), ends: t + 2 * D }; } else { while (queue.length > 0) { const w = write.get(queue.shift()); w.accepted = t; // written to the local replica, the carrier is acknowledged right away w.visible = t + D; // reaches the neighbor D rounds later messages += 1; } } } const all = [...write.values()]; const accepted = all.filter((w) => w.accepted !== null); const visible = accepted.filter((w) => w.visible <= N); const max = (a, f) => Math.max(0, ...a.map(f)); const avg = (a, f) => (a.length === 0 ? 0 : a.reduce((t, w) => t + f(w), 0) / a.length); return { arrived: all.length, accepted: accepted.length, messages, avgWait: avg(accepted, (w) => w.accepted - w.arrival), messagesPerWrite: accepted.length === 0 ? 0 : messages / accepted.length, maxStaleness: max(visible, (w) => w.visible - w.accepted), maxEndToEnd: max(visible, (w) => w.visible - w.arrival), }; } ``` The model's four numbers each need their own definition. **Wait** is the difference between the round an event arrives and the round the carrier is acknowledged. **Staleness window** is the difference between the round a write is acknowledged and the round it becomes visible on the second replica. **End-to-end** is the difference between arrival and visibility on the second replica. **Messages** is the number spent per accepted write. ```js // pacelc/measure.mjs — the trade-off on the E branch: two policies, two arrival intervals, three latencies import { run } from "./model.mjs"; const ROUNDS = 24; const pad = (x, n) => String(x).padStart(n); console.log(`no partition. ${ROUNDS} rounds. arrival interval and latency D are assumptions; a round is an abstract step.`); console.log(); console.log("interval | D | policy | accepted | avg wait | pending | msgs/write | staleness | end-to-end"); console.log("---------|---|--------------|----------|-------------|----------|-------------|-----------|-----------"); for (const interval of [1, 4]) { for (const D of [1, 2, 4]) { for (const policy of ["acknowledged", "propagated"]) { const r = run({ rounds: ROUNDS, latency: D, interval, policy }); console.log(`${pad(interval, 8)} | ${pad(D, 1)} | ${policy.padEnd(12)} | ` + `${pad(`${r.accepted}/${r.arrived}`, 8)} | ${pad(r.avgWait.toFixed(1), 11)} | ` + `${pad(r.arrived - r.accepted, 8)} | ${pad(r.messagesPerWrite.toFixed(2), 11)} | ` + `${pad(`${r.maxStaleness} round`, 9)} | ${pad(`${r.maxEndToEnd} round`, 10)}`); } } } ``` ```sh node pacelc/measure.mjs ``` ``` no partition. 24 rounds. arrival interval and latency D are assumptions; a round is an abstract step. interval | D | policy | accepted | avg wait | pending | msgs/write | staleness | end-to-end ---------|---|--------------|----------|-------------|----------|-------------|-----------|----------- 1 | 1 | acknowledged | 11/24 | 7.0 | 13 | 2.00 | 0 round | 12 round 1 | 1 | propagated | 24/24 | 0.0 | 0 | 1.00 | 1 round | 1 round 1 | 2 | acknowledged | 5/24 | 10.0 | 19 | 2.00 | 0 round | 16 round 1 | 2 | propagated | 24/24 | 0.0 | 0 | 1.00 | 2 round | 2 round 1 | 4 | acknowledged | 2/24 | 11.5 | 22 | 2.00 | 0 round | 15 round 1 | 4 | propagated | 24/24 | 0.0 | 0 | 1.00 | 4 round | 4 round 4 | 1 | acknowledged | 6/6 | 2.0 | 0 | 2.00 | 0 round | 2 round 4 | 1 | propagated | 6/6 | 0.0 | 0 | 1.00 | 1 round | 1 round 4 | 2 | acknowledged | 5/6 | 4.0 | 1 | 2.00 | 0 round | 4 round 4 | 2 | propagated | 6/6 | 0.0 | 0 | 1.00 | 2 round | 2 round 4 | 4 | acknowledged | 2/6 | 10.0 | 4 | 2.00 | 0 round | 12 round 4 | 4 | propagated | 6/6 | 0.0 | 0 | 1.00 | 4 round | 4 round ``` #### Reading the Numbers The two cleanest columns are staleness and messages. The acknowledged policy's staleness window is 0 rounds in all six runs; the propagated policy's is exactly `D` rounds — 1, 2, and 4. This turns the second branch's definition into a number: asynchronous propagation trades consistency for a window as wide as the latency parameter. In exchange, messages per write drop from 2.00 to 1.00. The acknowledgment message is the fee consistency pays even with no outage. The wait column shows who that fee is billed to. In the propagated policy, wait is 0.0 rounds in every run: the carrier gets its acknowledgment right away. In the acknowledged policy, wait at arrival interval 4 is exactly $2D$ rounds — 2.0 and 4.0. At sparse arrivals, the cost of synchronous acknowledgment is predictable and constant. The runs at arrival interval 1 show something quite different: wait climbs to 7.0, 10.0, and 11.5, and pending reaches 13, 19, and 22. The reason: events under the same tracking number must stay ordered, so at most one write is ever in flight, and the policy can accept a write only once every $2D$ rounds. Once the arrival interval falls below $2D$, the queue grows. This is the same independence seen in the Latency and Throughput lesson: per-write latency stays fixed at $2D$ while throughput falls, and falling throughput inflates the wait. What to do with the queue — reject it, slow it down, apply back pressure — is the subject of the Caching, Queues and Asynchronous Processing course and is not revisited here. There is a reading trap in the end-to-end column. At arrival interval 1 and $D = 4$, this number is 15, while at $D = 2$ it is 16 — smaller, but not better: the column is computed only over accepted writes, and in that row only 2 of 24 events were accepted. The end-to-end latency of the 22 events still queued is not yet defined. Near saturation, an average latency is misleading; the latency column should not be read before the pending column is. Finally, the acknowledged policy's end-to-end number is not 0. Synchronous acknowledgment does not let the reader see the truth sooner; it keeps the system from promising something it cannot yet verify. Staleness comes out 0 because that measure sits between what the system promises and what gets read. The distance between the event and the value read does not close — it just becomes invisible. #### Classifying the Two Branches Separately A system is not named with a single letter but classified separately for its two branches. The read policies from the previous lesson and the write policies from this one produce four combinations. | Class | Under partition | With no partition | Policy pair | |---|---|---|---| | PC/EC | consistency | consistency | reject + acknowledged | | PA/EL | service availability | latency | respond + propagated | | PC/EL | consistency | latency | reject + propagated | | PA/EC | service availability | consistency over latency | respond + acknowledged | The third row looks strange at first but is a coherent design: writes propagate asynchronously while the link is up, and a read is rejected once it goes down. The system is fast, and instead of getting something wrong during a partition it stays silent; the price is a `D`-round staleness window even while the link is up. Classification, again, is per flow rather than for the whole system. In the tracking service, the tracking query can choose the PA/EL branch and end-of-day billing the PC/EC branch; both run over the same replicas. Billing a shipment against a stale status produces a wrong invoice even within a two-round window; a tracking query showing a status two rounds stale is a tolerable error. #### Summary - PACELC is a two-branch sentence: under a partition (P), the choice is between service availability (A) and consistency (C); with no partition (E), between latency (L) and consistency (C). - In the model, the acknowledged policy's staleness window was 0 rounds in every run; the propagated policy's was exactly as wide as the latency parameter — 1, 2, and 4 rounds. - Consistency's fee outside a partition showed up in the message column: 2.00 messages per write versus 1.00. - At arrival interval 4, the acknowledged policy's wait stayed fixed at $2D$ rounds (2.0 and 4.0); once the interval dropped to 1, the queue grew and wait climbed from 7.0 to 11.5, with pending events rising from 13 to 22. - Synchronous acknowledgment does not let the reader see the truth sooner; end-to-end latency remains — the system only stops promising a value it cannot verify. - Classification is done separately for the two branches and per flow: the tracking query can sit on the PA/EL branch, billing on the PC/EC branch. #### Next Step In this lesson's measurement, the staleness window collapsed to a single number: 0 or `D` rounds. That is only the coarse shape of the problem. What a reader sees inside a `D`-round window depends less on the window's width than on which guarantee is given. Two clients querying the same shipment at once can see different statuses; one can see a sequence that goes backward instead of forward; a third can fail to find the event it just wrote on its next query. All of this happens inside the same `D` window, and none of it substitutes for the rest. The next lesson names these guarantees — strong, weak, and eventual consistency — reading the same event sequence under three read policies and counting how often different values are seen, how many rounds convergence takes, and how many monotonic-read violations occur. ### Consistency Models Source: https://academia.sh/en/courses/introduction-to-system-design/consistency-models Separating strong, weak, and eventual consistency: stating that distributed consistency is a different question from transaction isolation, measuring the same event sequence read under three models for the number of times two clients see different values, the number of rounds needed to converge, and the number of monotonic-read violations, and showing that sticky routing fixes monotonic reads while widening the gap between clients. The previous lesson collapsed the staleness window to a single number: 0 or `D` rounds. That number is a window's width; it says nothing about what can happen inside it. In the same `D`-round window, two clients querying the same shipment can see different statuses; one can see a sequence that goes backward instead of forward; a third can fail to find the event it just wrote on its next query. These are separate phenomena, and none substitutes for another. What defines the inside of the window is called a **consistency model**: what the system promises about reads. This lesson names three models and reads the same event sequence with all three, counting the difference between their promises. #### A Model, Not an Isolation Level The word consistency has appeared in two separate places in this repository. In the Data Modeling and Relational Theory course it was one of the ACID properties; in the Advanced SQL course it was the measure of isolation levels. The question here is different, and neither substitutes for the other: transaction isolation governs whether concurrent transactions see each other's half-finished work; distributed consistency governs when an acknowledged write becomes visible across replicas. A system can run the strictest isolation level while being only eventually consistent; the two are chosen separately. Isolation levels are not retold here. How replication works was also established in the Relational Database Administration course. What gets measured here is the promise sitting on top of that same replication. #### Three Models **Strong consistency**: every read sees the most recently acknowledged write. The system looks single-copy from the outside. Its cost is that the local replica verifies its held value is the newest. **Eventual consistency**: a read may not see the latest write, but once writing stops, all replicas are guaranteed to converge on the same value. The term was introduced in the Data Modeling and Relational Theory course; what is added here is that convergence is not automatic — it requires a repair mechanism that resends a lost propagation. **Weak consistency**: no promise is made at all. A read may see a stale value, and replicas may never converge. This is not a looser form of eventual consistency; it is the absence of the convergence guarantee, and the difference is measurable. #### Three-Replica Setup The model sets up a three-zone replica set. Writes arrive at zone `b34` and propagate to `b35` and `b06`; the propagation latencies and a loss rule are model parameters. Two clients read the same tracking number every round and are routed to different zones in turn. ```js // consistency/model.mjs — three-replica in-process model. A round is an abstract step; latency, // the loss rule, and the repair interval are model parameters, not measured durations. const LATENCY = { b35: 1, b06: 3 }; // rounds for a propagation to reach its target const LOST = (write) => write % 5 === 0; // propagation to b06 is lost on these writes const REPAIR = 2; // resend interval in the repaired models const ROUTE = { A: ["b34", "b35", "b06"], B: ["b06", "b35", "b34"] }; const STICKY = { A: ["b35"], B: ["b06"] }; // sticky routing: the client stays on one replica export function run({ rounds: N, writeEnd: W, model, sticky = false }) { const version = { b34: 0, b35: 0, b06: 0 }; const seen = { A: 0, B: 0 }; const trace = { A: [], B: [] }; let flight = [], messages = 0, differing = 0, violation = 0, converged = null; for (let round = 1; round <= N; round++) { if (round <= W) { version.b34 = round; // the carrier event is written to b34, the carrier is acknowledged right away for (const h of ["b35", "b06"]) { messages += 1; flight.push({ target: h, write: round, arrives: round + LATENCY[h], lost: h === "b06" && LOST(round) }); } } const arriving = flight.filter((m) => m.arrives === round); flight = flight.filter((m) => m.arrives !== round); for (const m of arriving) { if (m.lost === false) { version[m.target] = Math.max(version[m.target], m.write); continue; } if (model === "weak") continue; // a loss is not repaired: there is no convergence guarantee messages += 1; flight.push({ ...m, arrives: round + REPAIR, lost: false }); } const read = (client) => { const route = sticky ? STICKY[client] : ROUTE[client]; const zone = route[(round - 1) % route.length]; if (model !== "strong") return version[zone]; if (zone !== "b34") messages += 2; // the local replica verifies the latest write with b34 return version.b34; }; const value = { A: read("A"), B: read("B") }; if (value.A !== value.B) differing += 1; for (const c of ["A", "B"]) { if (value[c] < seen[c]) violation += 1; seen[c] = Math.max(seen[c], value[c]); trace[c].push(value[c]); } if (round > W && converged === null && version.b34 === version.b35 && version.b35 === version.b06) { converged = round - W; } } return { differing, violation, messages, converged, version, trace }; } ``` The three models are separated in the code by two lines: whether a lost propagation gets resent, and whether a read is verified against `b34`. A **monotonic-read violation** is a client seeing a version older than one it has already seen. ```js // consistency/measure.mjs — the same event sequence read with three models: diff, violation, convergence, messages import { run } from "./model.mjs"; const ROUNDS = 20, WRITE_END = 10; const pad = (x, n) => String(x).padStart(n); console.log(`${ROUNDS} rounds, one write per round through round ${WRITE_END}; no writes after that.`); console.log("two clients read the same tracking number every round, routed to different replicas."); console.log(); console.log("model | differing val | monotonic viol. | converged | msgs | final version b34/b35/b06"); console.log("----------------|---------------|------------------|---------------|------|--------------------------"); const CONFIGS = [["strong", false], ["eventual", false], ["eventual", true], ["weak", false]]; const results = {}; for (const [model, sticky] of CONFIGS) { const r = run({ rounds: ROUNDS, writeEnd: WRITE_END, model, sticky }); const label = sticky ? `${model}+sticky` : model; results[label] = r; console.log(`${label.padEnd(15)} | ${pad(`${r.differing}/${ROUNDS}`, 13)} | ${pad(r.violation, 16)} | ` + `${pad(r.converged === null ? "not converged" : `${r.converged} round`, 13)} | ${pad(r.messages, 4)} | ` + `${r.version.b34}/${r.version.b35}/${r.version.b06}`); } console.log(); for (const label of Object.keys(results)) { console.log(`${label.padEnd(15)} A: ${results[label].trace.A.join(" ")}`); console.log(`${label.padEnd(15)} B: ${results[label].trace.B.join(" ")}`); } ``` ```sh node consistency/measure.mjs ``` ``` 20 rounds, one write per round through round 10; no writes after that. two clients read the same tracking number every round, routed to different replicas. model | differing val | monotonic viol. | converged | msgs | final version b34/b35/b06 ----------------|---------------|------------------|---------------|------|-------------------------- strong | 0/20 | 0 | 5 round | 76 | 10/10/10 eventual | 9/20 | 8 | 5 round | 22 | 10/10/10 eventual+sticky | 13/20 | 0 | 5 round | 22 | 10/10/10 weak | 13/20 | 12 | not converged | 20 | 10/10/9 strong A: 1 2 3 4 5 6 7 8 9 10 10 10 10 10 10 10 10 10 10 10 strong B: 1 2 3 4 5 6 7 8 9 10 10 10 10 10 10 10 10 10 10 10 eventual A: 1 1 0 4 4 3 7 7 6 10 10 9 10 10 10 10 10 10 10 10 eventual B: 0 1 3 1 4 6 4 7 9 7 10 10 9 10 10 10 10 10 10 10 eventual+sticky A: 0 1 2 3 4 5 6 7 8 9 10 10 10 10 10 10 10 10 10 10 eventual+sticky B: 0 0 0 1 2 3 4 4 6 7 8 9 9 9 10 10 10 10 10 10 weak A: 1 1 0 4 4 3 7 7 6 10 10 9 10 10 9 10 10 9 10 10 weak B: 0 1 3 1 4 6 4 7 9 7 10 10 9 10 10 9 10 10 9 10 ``` #### Reading the Numbers The strong row delivers exactly what it promises: the two clients never saw different values in any round (0/20), monotonic violations were 0, and both traces are exactly the write sequence itself. The cost shows up in the message column: 76 messages, roughly four times the cheapest row. That count comes from reads, not writes — every read landing on the local replica adds a round trip to `b34`. In a read-heavy flow, the cost grows with the number of reads. The eventual row shows the limit of the promise. Convergence takes 5 rounds: five rounds after writing stops, all three replicas settle on version 10. Convergence happened, but inside the window the clients saw different values in 9 of 20 rounds and produced 8 monotonic violations. The traces show this directly: client A starts `1 1 0` — in round three it gets a response that looks as though it had never seen the shipment's status at all, because that round routed to zone `b06`. For a tracking query, this means the status shown on screen goes backward. The weak row's only difference is the absence of a mechanism, and the result shows in the last column: the replicas stayed at `10/10/9`, and convergence never happened. The tenth write is lost on its way to `b06` and never resent, so that zone shows the shipment one step behind forever. The convergence guarantee costs two messages here (20 versus 22); what it buys is the absence of a permanent divergence. Treating eventual consistency as a model with no guarantee at all means missing those two messages. #### Session Guarantees A monotonic violation can be fixed on an axis separate from the consistency model. In the eventual+sticky row, the model stayed the same; the only change is that the client is routed to the same replica every round — the read-path application of the **sticky routing** pattern introduced in the Caching, Queues and Asynchronous Processing course. Violations dropped from 8 to 0, because a single replica's version number never goes backward. Promises of this kind are called **session guarantees**; the two best known are monotonic reads (a client never sees an older value than one it has already seen) and read-your-writes (a client sees its own write on its next read). The same row also says something second and more important: the differing-value count rose from 9 to 13. Once sticky routing pins a client to a single replica, A always reads `b35` and B always reads `b06`, and because the two zones have different latencies, the two nearly diverge every round. A session guarantee fixes what one client sees over time; it does not fix the gap between clients — it can widen it. This maps to two separate requirements in the tracking service. A user querying the same shipment repeatedly needs the status not to go backward; sticky routing provides that. End-of-day billing basing itself on the same instant for every shipment cannot be provided by a session guarantee — that needs strong consistency, at the cost of a round trip per read. The next lesson ties these two requirements to separate patterns. #### Summary - Distributed consistency is a question separate from transaction isolation: one governs whether concurrent transactions see each other, the other governs when an acknowledged write shows up across replicas. - In the same twenty-round sequence, strong produced 0/20 differing values and 0 violations at a cost of 76 messages; eventual produced 9/20 differing values and 8 violations at a cost of 22 messages. - Eventual consistency's guarantee is convergence, and it is not free: with repair removed (weak), the replicas stayed at 10/10/9 and never converged; repair cost two messages. - Convergence took 5 rounds; during that time the promise is only "eventually the same," saying nothing about what reads will see. - Sticky routing dropped monotonic violations from 8 to 0 but raised the differing-value count from 9 to 13; a session guarantee does not fix the gap between clients. #### Next Step All three lessons up to this point carried the same implicit assumption: every replica is up, and only the link or the latency between them causes trouble. When a replica goes down entirely, the question changes: now a replacement must be found, requests must be routed to it, and the requests dropped during that transition must be counted. How the transition is handled is a choice of pattern: one replica takes writes while the other waits, or all of them take writes at once. The next lesson measures these two patterns: the failover round and the requests dropped during it in an active–passive setup, and the number of conflicting writes and the lost-write count a resolution rule produces in an active–active setup. ### Availability Patterns Source: https://academia.sh/en/courses/introduction-to-system-design/availability-patterns Comparing active–active and active–passive failover as patterns: measuring the failover round and the number of requests dropped during it in the same outage scenario, counting the conflicting writes and the lost writes the last-writer-wins rule produces in the active–active setup, converting round counts into requests using the course's assumption table, and showing how key ownership removes the conflict. The three lessons so far carried the same implicit assumption: every replica is up, and the only trouble comes from the link or the latency between them. When a replica goes down entirely, the question changes: a replacement must be found, requests routed to it, and the requests dropped during that transition counted. How the transition is handled is a choice of pattern, and this lesson measures two in the same outage scenario. The mechanisms underneath were established elsewhere: majority rule, fencing, and the health-check-frequency-versus-false-failover trade-off belong to the Relational Database Administration course; whether the rest of a request survives when one component goes down — fault isolation — was measured in the Architectural Styles course. What gets measured here is how much the pattern costs in requests and writes. #### Two Patterns **Active–passive**: only one replica accepts writes, the other stands ready. Because the passive replica takes no writes, conflict is impossible. When the active replica goes down, no replica is left to accept writes; writes are dropped until failover completes. **Active–active**: every replica accepts writes. When one goes down, the others keep working, and there is no failover window. In exchange, two replicas can write to the same key without knowing about each other; this is a **conflicting write** and needs a resolution rule. This lesson's rule is **last writer wins**: of two values written in the same window, one is chosen and the other shows up nowhere. In the tracking service, conflict is not contrived: the same shipment can pass through both zones' scanners at a transfer point, and both zones can write in the same round. #### The Setup The model sets up a two-zone replica set. Scan events originate at a specific zone; zone `b34` is unreachable from round 9 through round 12. Latency, detection, and promotion are model parameters in rounds. ```js // pattern/model.mjs — two-zone failover model. A round is an abstract step; latency, detection, // and promotion are model parameters in rounds, not measured durations. export const EVENTS = [ // [round, scan zone, status] [1, "b34", "accepted"], [2, "b34", "departed"], [3, "b34", "transfer-34"], [3, "b35", "line-35"], [4, "b34", "transfer-41"], [5, "b34", "out-for-delivery"], [5, "b35", "out-for-delivery-35"], [6, "b35", "delivery-attempt"], [7, "b34", "address-verification"], [7, "b35", "redelivery"], [8, "b34", "line-06"], [9, "b34", "transfer-06"], [10, "b35", "delivered"], [11, "b34", "signed"], [12, "b35", "closed"], [13, "b34", "invoice-ready"], [14, "b34", "invoice-issued"], [14, "b35", "invoice-approved"], [15, "b35", "archived"], [16, "b34", "archive-approved"], ]; export const OUTAGE = { zone: "b34", start: 9, end: 12 }; const LATENCY = 1, DETECTION = 2, PROMOTION = 1; const other = (z) => (z === "b34" ? "b35" : "b34"); const alive = (zone, round) => zone !== OUTAGE.zone || round < OUTAGE.start || round > OUTAGE.end; export function run({ pattern, rounds: N }) { const s = { accepted: 0, dropped: 0, conflicting: 0, lostConflict: 0, lostUnpropagated: 0, messages: 0 }; let flight = [], active = "b34", failoverStart = null, failoverRounds = 0; for (let round = 1; round <= N; round++) { if (round === OUTAGE.start) { // propagations that left the downed zone but have not yet arrived are lost s.lostUnpropagated += flight.filter((m) => m.source === OUTAGE.zone).length; flight = flight.filter((m) => m.source !== OUTAGE.zone); } flight = flight.filter((m) => m.arrives !== round); // arriving propagations become visible at the target if (pattern === "active-passive" && alive(active, round) === false) { if (failoverStart === null) failoverStart = round; if (round === failoverStart + DETECTION + PROMOTION) { active = other(active); failoverRounds = DETECTION + PROMOTION; } } const written = new Set(); for (const [, scan] of EVENTS.filter(([t]) => t === round)) { const target = pattern === "active-passive" ? active : (alive(scan, round) ? scan : other(scan)); if (alive(target, round) === false) { s.dropped += 1; continue; } // no replica available to accept it if (target !== scan) s.messages += 1; // the scan was forwarded to the remote replica s.accepted += 1; s.messages += 1; // the accepting replica propagates to its neighbor written.add(target); flight.push({ source: target, target: other(target), arrives: round + LATENCY }); } if (written.size === 2) { // both zones wrote in the same round, each unaware of the other s.conflicting += 1; s.lostConflict += 1; // last writer wins: the losing write shows up nowhere } } return { ...s, failoverRounds, active }; } ``` ```js // pattern/measure.mjs — same outage scenario with two patterns: failover, dropped, conflict, lost import { run, EVENTS, OUTAGE } from "./model.mjs"; const ROUNDS = 16; const pad = (x, n) => String(x).padStart(n); console.log(`${ROUNDS} rounds, ${EVENTS.length} scan events. zone ${OUTAGE.zone} is ` + `unreachable in rounds ${OUTAGE.start}-${OUTAGE.end}.`); console.log(); console.log("pattern | accept | drop | failover | conflict | lost(conflict) | lost(unprop.) | msgs"); console.log("----------------|--------|------|----------|----------|----------------|----------------|-----"); const results = {}; for (const pattern of ["active-passive", "active-active"]) { const r = run({ pattern, rounds: ROUNDS }); results[pattern] = r; console.log(`${pattern.padEnd(15)} | ${pad(`${r.accepted}/${EVENTS.length}`, 6)} | ${pad(r.dropped, 4)} | ` + `${pad(`${r.failoverRounds} round`, 8)} | ${pad(r.conflicting, 8)} | ${pad(r.lostConflict, 15)} | ` + `${pad(r.lostUnpropagated, 14)} | ${pad(r.messages, 4)}`); } console.log(); console.log(`active-passive: active replica after failover = ${results["active-passive"].active}`); console.log(`active-active: dropped requests during the outage rounds = ${results["active-active"].dropped}`); ``` ```sh node pattern/measure.mjs ``` ``` 16 rounds, 20 scan events. zone b34 is unreachable in rounds 9-12. pattern | accept | drop | failover | conflict | lost(conflict) | lost(unprop.) | msgs ----------------|--------|------|----------|----------|----------------|----------------|----- active-passive | 17/20 | 3 | 3 round | 0 | 0 | 1 | 24 active-active | 20/20 | 0 | 0 round | 4 | 4 | 1 | 22 active-passive: active replica after failover = b35 active-active: dropped requests during the outage rounds = 0 ``` #### Reading the Numbers The accept column confirms the difference the patterns advertise. Active–passive accepted 17 of 20 scan events; failover took 3 rounds, and the 3 events arriving in those rounds were dropped. Active–active accepted 20/20 and dropped none during the outage, because `b34`'s scanners were rerouted to `b35`. Which three events dropped matters: `transfer-06` in round 9, `delivered` in round 10, `signed` in round 11. Two were scans at zone `b35`, which stayed up — dropped anyway. The reason is the definition of a passive replica: it accepts no writes. Being up does not mean a zone's write gets accepted. The conflict column shows where that cost moves instead. In active–active, zones wrote the same key without seeing each other in rounds 3, 5, 7, and 14: 4 conflicts. Last writer wins deleted one write per conflict, so 4 lost writes — 0 in active–passive, which instead carries 3 dropped requests. The two patterns pay for the same outage in different places: one as an outage, the other as lost data. The last two columns show the cost the patterns share. `lost(unprop.)` is 1 in both: when `b34` went down, one propagation had already left it but had not arrived, and it was lost. This comes from asynchronous propagation, not the pattern; with synchronous acknowledgment it would be 0, at the cost of the wait measured in the PACELC lesson. Message counts are close too (24 versus 22); the extra in active–active comes from forwarding a scan to the remote replica. Choosing a pattern is not a decision about saving messages. #### Converting Rounds to Requests A round is an abstract step. Converting the numbers into requests needs two assumptions: how many seconds a round is, and how many failovers happen per year. The volume figures come from the Back-of-the-Envelope Estimation lesson's assumption table and are recalculated here. ```js // pattern/budget.mjs — converts the model's round counts into requests using the course's assumption table const V = { dailyUsers: 2_000_000, queriesPerUser: 6, dailyShipments: 400_000, eventsPerShipment: 7, peakFactor: 3 }; // Back-of-the-Envelope Estimation lesson: V1-V4, V8 const YEAR = 365 * 24 * 3600, DAY = 86_400; const SEC_PER_ROUND = 1; // assumption: one round equals one second const FAILOVERS_PER_YEAR = 6; // assumption: how many failovers per year const DUAL_SCAN = 0.05; // assumption: rate of events produced at both zones const peakWrites = (V.dailyShipments * V.eventsPerShipment / DAY) * V.peakFactor; const peakTotal = peakWrites + (V.dailyUsers * V.queriesPerUser / DAY) * V.peakFactor; console.log(`peak writes = ${peakWrites.toFixed(2)} req/s, peak total = ${peakTotal.toFixed(2)} req/s`); console.log(); for (const failoverRounds of [3, 5]) { const secs = failoverRounds * SEC_PER_ROUND; const annual = secs * FAILOVERS_PER_YEAR; console.log(`active-passive, failover ${failoverRounds} rounds (${secs} sec):`); console.log(` dropped writes in window = ${(secs * peakWrites).toFixed(2)}, dropped total requests = ${(secs * peakTotal).toFixed(2)}`); console.log(` ${FAILOVERS_PER_YEAR} failovers/year -> ${annual} sec, service availability = ` + `${(100 * (1 - annual / YEAR)).toFixed(5)}%`); } console.log(); for (const rate of [DUAL_SCAN, DUAL_SCAN * 2]) { const conflicting = V.dailyShipments * V.eventsPerShipment * rate; console.log(`active-active, dual-scan rate ${rate.toFixed(2)}: conflicting writes = ${conflicting}/day, ` + `last writer wins -> lost writes = ${conflicting}/day`); } ``` ``` peak writes = 97.22 req/s, peak total = 513.89 req/s active-passive, failover 3 rounds (3 sec): dropped writes in window = 291.67, dropped total requests = 1541.67 6 failovers/year -> 18 sec, service availability = 99.99994% active-passive, failover 5 rounds (5 sec): dropped writes in window = 486.11, dropped total requests = 2569.44 6 failovers/year -> 30 sec, service availability = 99.99990% active-active, dual-scan rate 0.05: conflicting writes = 140000/day, last writer wins -> lost writes = 140000/day active-active, dual-scan rate 0.10: conflicting writes = 280000/day, last writer wins -> lost writes = 280000/day ``` The active–passive failover window takes up very little of the budget: 6 failovers a year come to 18 seconds and leave service availability at 99.99994%. At 5 rounds, that becomes 30 seconds and 99.99990%. By the budget, this pattern is nearly free. The cost is not spread across the annual percentage but concentrated in a single window: every failover drops 291.67 status events and 1,541.67 requests at the peak, all within three seconds. An outage budget measures an average; it does not measure a loss concentrated in one window. The active–active number is on an entirely different order. A 5 percent dual-scan rate produces 140,000 conflicting writes a day, and last writer wins turns every one into a lost write; doubling the rate gives 280,000. This loss is not an outage — it does not show up in the budget and does not announce itself as an error — it just leaves some shipments' status history one step short. #### Choosing the Pattern and Key Ownership The measured numbers decide the choice per flow. For carrier events, active–passive fits: the 291.67 events dropped in a three-second window can be resent by the carrier, while a lost status update cannot be recovered. The same holds for end-of-day billing, where a missing piece of status history produces a wrong invoice. The tracking query, though, is a read, and either pattern can serve it from the passive replica — a read has no failover window. Active–active's conflict cost is not unavoidable. Conflict arises from two zones writing the same key; if writes are routed to a single zone by key — adding **key ownership** to the pattern — conflicting writes drop to 0, and each zone becomes the active replica for its own key set. The system is then active–active as a whole and active–passive for any single key. The 4 conflicts measured are what appears without an ownership rule — not the pattern's unavoidable cost, but the result of a missing decision. #### Summary - In active–passive, only one replica accepts writes: conflict is 0, but once the active replica goes down, no writes are accepted until failover completes. - In the same outage scenario, active–passive accepted 17/20 events, failover took 3 rounds, and 3 events were dropped; two of those originated at the zone that stayed up, because a passive replica takes no writes. - Active–active accepted 20/20 and dropped no requests during the outage; in exchange it produced 4 conflicting writes, and last writer wins deleted 4 of them. - The loss from asynchronous propagation came out the same in both patterns (1 unpropagated write); this cost comes from the write policy, not the pattern. - Converted with the assumptions, active–passive failover costs 18 seconds a year (99.99994%) but drops 291.67 status events and 1,541.67 total requests in every window; an outage budget does not measure a loss concentrated in one window. - Conflict is not the pattern's unavoidable cost: once key ownership is added, every key has a single active replica and conflicting writes drop to 0. #### Course Wrap-Up The course took on a design problem in two steps. The Approach topic broke a one-line request into requirements: component boundaries, contracts, and quality thresholds were separated, five of eleven questions were counted answered, scope narrowing brought the components from 12 down to 5, and the assumption table gave a peak total of 513.89 req/s at the edge with 976 MB of daily data growth. Its last lesson gave design communication its form: a five-field decision record and a diagram whose edges are labeled from the calculation — the same flow carrying 416.67 req/s on the left edge and 41.67 req/s on the right. Fundamental Properties turned each property into a number on top of those estimates: latency and throughput were measured as independent, throughput growing close to linearly as concurrency rose while median latency stayed nearly fixed; performance was separated from scalability, and the limits of horizontal and vertical scaling were drawn; service availability became a budget; why it gets spent was named with the CAP theorem; the latency trade-off outside a partition came from PACELC; consistency models were separated by convergence and monotonic reads; failover was measured as two patterns. The course's real rule stood beside these numbers: every number carried its class. An **assumption** is a chosen input with its sensitivity shown — 2,000,000 daily active users is one such number, and doubling it raises the peak total by 1.81 times. A **computed value** comes from the assumptions by arithmetic, produced with `node` — peak reads at 416.67 req/s, 712.48 GB of stored data, this lesson's 1,541.67 dropped requests. A **measurement** comes from a setup and is flagged for its environment dependence — the throughput upper bound from the concurrency setup, and the last three lessons' model runs: 6 rejected requests in a twelve-round partition, 8 monotonic-read violations in the three-replica model, 4 conflicting writes here. A number with no class carries no decision; that is why one of four decisions in the first lesson was unauditable. The question left behind sits one layer down. Every property was defined, thresholds set, and patterns chosen — but the request has not yet entered the system. A tracking query starts with a domain name resolving, may pass through an edge cache, lands on a load balancer, and reaches the application through a gateway. None of that path was designed here — caching was only a multiplier in the calculation, routing a one-line choice in the model. The next course, **The Traffic Layer**, takes on that path: every stop a request passes through before reaching the application, the decisions made there, and how those decisions change the numbers calculated in this course.