--- title: 'Chat System' source: 'https://academia.sh/en/courses/case-studies/chat-system' course: 'Case Studies' language: en updated: '2026-08-23T07:01:24+00:00' license: 'CC BY-SA 4.0' --- # Chat System The first case where writing dominates: connection and delivery scale drawn from nine assumptions, the effect of the delivery guarantee choice on lost and duplicated message counts measured in a one-second peak window, ordering violations counted with inversions, the sequential convoy's cost in wait rounds, and the polling model eliminated by its breakeven interval. The Read-Heavy Systems topic produced four cases, and in all four reading outdid writing by a wide margin. That balance makes one thing cheap: a response being a few seconds stale was an affordable cost, because the same record was read thousands of times and its freshness depended on a single write. The staleness window was a design tool there. This topic takes the reverse balance. In chat, a record is written once and read a small number of times; the record itself is a message a user sent. Its being lost, appearing a second time, or appearing in the wrong order is not acceptable. The same patterns are still on hand, but their parameters sit elsewhere. ## Constraints and Scope The functional requirements are five items: one-to-one message sending, group messages with at most 50 members, presence (online or last seen), delivery and read receipts, and holding messages for an offline recipient. What is not designed is stated explicitly: voice and video calling, file attachments, in-message search, end-to-end encryption, and history export are outside this case. The non-functional requirements are written with a threshold and the threshold's source: lost messages are zero (source: the message being assumed to exist once "sent" is shown to the sender), the same message being shown a second time is zero (source: the recipient's seen list being equal to the sent list), an ordering violation within a chat is zero (source: messages being replies to one another), and the median delivery delay to a connected recipient does not exceed 200 milliseconds (source: the concurrency the sender expects from the peer's typing indicator). ## Assumptions and Scale | Code | Assumption | Value | Rationale | |---|---|---|---| | SO1 | daily active users | 5,000,000 | account that opens the app at least once a day | | SO2 | messages sent per user per day | 40 | short exchanges spread across a few chats | | SO3 | message body | 220 bytes | text, sender, chat ID, timestamp | | SO4 | peak factor | 4 | ratio of evening hours to the daily average | | SO5 | concurrently connected user share | 0.25 | connection held while the app is open | | SO6 | recipients per message | 2.4 | average of the one-to-one and group mix | | SO7 | offline recipient share | 0.35 | share of recipients not connected at delivery time | | SO8 | retention period | 365 days | history opens a year back | | SO9 | connections per node | 50,000 | open connections a single node carries | ```js // chat/scale.mjs — the back-of-envelope calculation drawn from the SO assumption table export const SO = { dailyUsers: 5_000_000, // SO1 messagesPerUser: 40, // SO2 messageBytes: 220, // SO3 peakFactor: 4, // SO4 connectedShare: 0.25, // SO5 avgRecipients: 2.4, // SO6 offlineShare: 0.35, // SO7 retentionDays: 365, // SO8 connectionsPerNode: 50_000, // SO9 }; const DAY = 86_400; export function calculate(v) { const dailyMessages = v.dailyUsers * v.messagesPerUser; const avgSend = dailyMessages / DAY; const peakSend = avgSend * v.peakFactor; const peakDelivery = peakSend * v.avgRecipients; const connected = v.dailyUsers * v.connectedShare; return { "daily messages": dailyMessages, "average sends/s": avgSend, "peak sends/s": peakSend, "peak deliveries/s": peakDelivery, "peak mailbox writes/s": peakDelivery * v.offlineShare, "concurrent connected users": connected, "connection nodes": connected / v.connectionsPerNode, "peak egress Mbit/s": (peakDelivery * v.messageBytes * 8) / 1e6, "daily data growth GB": (dailyMessages * v.messageBytes) / 1e9, "stored data TB": (dailyMessages * v.messageBytes * v.retentionDays) / 1e12, }; } const r = calculate(SO); const format = (x) => (Number.isInteger(x) ? String(x) : x.toFixed(2)); for (const [name, d] of Object.entries(r)) console.log(`${name.padEnd(28)}${format(d).padStart(13)}`); const RESULT = ["peak deliveries/s", "connection nodes", "daily data growth GB"]; console.log(`\n${"assumption doubled".padEnd(25)}${RESULT.map((s) => s.split(" ")[0].padStart(11)).join("")}`); for (const name of ["messagesPerUser", "connectedShare", "avgRecipients"]) { const y = calculate({ ...SO, [name]: SO[name] * 2 }); console.log(`${name.padEnd(25)}${RESULT.map((s) => `x${(y[s] / r[s]).toFixed(2)}`.padStart(11)).join("")}`); } ``` ``` daily messages 200000000 average sends/s 2314.81 peak sends/s 9259.26 peak deliveries/s 22222.22 peak mailbox writes/s 7777.78 concurrent connected users 1250000 connection nodes 25 peak egress Mbit/s 39.11 daily data growth GB 44 stored data TB 16.06 assumption doubled peak connection daily messagesPerUser x2.00 x1.00 x2.00 connectedShare x1.00 x2.00 x1.00 avgRecipients x2.00 x1.00 x1.00 ``` These numbers belong to the **calculation** class. Four of them shape the design. First, peak sends are 9259.26 while peak deliveries are 22,222.22: every write produces 2.4 deliveries on average, and the load the system sees is not the write rate but the **delivery** rate. Second, 1,250,000 concurrent connections mean 25 connection nodes; that is 25 units of state held somewhere. Third, peak mailbox writes are 7777.78 records per second: more than a third of deliveries go to a recipient who is not connected and get written to a persistent mailbox. Fourth, bandwidth is small: 39.11 Mbit/s of egress, 44 GB of daily growth. In this case the bottleneck is not bytes but **record count** and **connection count**. The sensitivity table separates two sources: when the connected share doubles, only the node count doubles; when the recipient count doubles, only the delivery rate doubles. ## Design **Connection node.** The long-lived connection turns the stickiness covered in the Traffic Layer course's Session Stickiness lesson from a choice into a requirement: a connection is bound to a node and stays on that node. The parameter is the stickiness key: the device ID, not the user ID, because a second device for the same user is a separate connection. The mapping from device to node is done with the consistent hashing from the same course's Balancing Algorithms lesson; on a 25-node ring, adding one node moves a connection share of 1/26. **Routing directory.** The sender's node must know which node the recipient is connected to. This is the state store from the Application Layer and Service Interaction course's Stateless Services lesson; the record it holds is user ID, node name, and last heartbeat timestamp — the same record as presence. **Delivery path.** Every delivery passes through a message broker. Order within a chat is kept with the lane from the same course's Sequential Convoy lesson, and **the lane key is the chat ID**, not the sender ID: the ordering constraint exists within the chat, not across all of a sender's chats. **Uniqueness.** The client generates the message ID, and a resend carries the same ID. The scope of the uniqueness key from the Resilience and Reliability course's Idempotent Operations lesson is the `(sender, client message ID)` pair here; the chat ID is not part of the scope, because there is no case of the same message going to two chats. **Offline mailbox.** Messages for a recipient who is not connected are written to a persistent mailbox; the partition key from the Scaling the Data Layer course's Partitioning Strategies lesson is the recipient ID. **Deliberately unused pattern.** Edge caching (Traffic Layer, Entry Points) has no place in this design: every message goes to a single set of recipients exactly once, and no shared hit ever forms. ## Delivery Guarantee and Order Are Measured The measurement is an in-process model: there is no real socket, network, or broker; the channel is a probability and a round is an abstract step. Three more assumptions are added. **SO10 — channel loss probability 0.02**, justified by short cellular-connection drops; **SO11 — acknowledgment loss probability 0.01**, justified by the acknowledgment's return path being shorter; **SO12 — resend backoff 3 rounds**. The model takes a one-second peak window: 22,222 deliveries, 40 chat lanes. ```js // chat/delivery.mjs — in-process model of the delivery guarantee and ordering breakdown. // There is no real network, socket, or queue: the channel is a probability, the round is an abstract step. export function generator(seed) { // 32-bit linear congruential generator let s = seed >>> 0; return () => { s = (Math.imul(s, 1103515245) + 12345) >>> 0; return s / 4294967296; }; } const DELIVERIES = 22222, CHATS = 40; // calculation: peak deliveries/s x 1-second window const LOSS = 0.02, ACK_LOSS = 0.01; // SO10, SO11 const BACKOFF = 3; // SO12: resend backoff rounds export function run({ attempts, keyed, convoy, seed }) { const rnd = generator(seed); const laneLastArrival = new Array(CHATS).fill(0); const laneStream = Array.from({ length: CHATS }, () => []); const s = { channelAttempts: 0, lost: 0, duplicated: 0, waitRounds: 0 }; for (let i = 0; i < DELIVERIES; i += 1) { const chat = i % CHATS; const seq = Math.floor(i / CHATS); // the message's position in the lane and its send round let round = convoy ? Math.max(seq, laneLastArrival[chat]) : seq; s.waitRounds += round - seq; // waiting on the earlier message in the lane let reached = 0, arrival = null; for (let d = 0; d < attempts; d += 1) { s.channelAttempts += 1; if (rnd() < LOSS) { round += BACKOFF; continue; } // delivery dropped on the channel reached += 1; if (arrival === null) arrival = round; if (rnd() < ACK_LOSS) { round += BACKOFF; continue; } // ack dropped, sender resends break; } if (reached === 0) s.lost += 1; else { if (keyed === false) s.duplicated += reached - 1; laneStream[chat].push({ seq, arrival }); laneLastArrival[chat] = arrival; } } s.inversions = laneStream.reduce((t, stream) => { const arrivals = stream.map((o, i) => ({ ...o, i })).sort((a, b) => a.arrival - b.arrival || a.i - b.i); let n = 0; for (let a = 0; a < arrivals.length; a += 1) for (let b = a + 1; b < arrivals.length; b += 1) if (arrivals[a].seq > arrivals[b].seq) n += 1; return t + n; }, 0); return s; } const SCHEMES = [ ["at-most-once / free", { attempts: 1, keyed: false, convoy: false }], ["at-least-once / free", { attempts: 5, keyed: false, convoy: false }], ["at-least-once + key", { attempts: 5, keyed: true, convoy: false }], ["same + sequential convoy", { attempts: 5, keyed: true, convoy: true }], ]; console.log(`model: ${DELIVERIES} deliveries, ${CHATS} chat lanes, loss ${LOSS}, ack loss ${ACK_LOSS}, seed 20260730`); console.log(`\n${"scheme".padEnd(26)}${"channel attempts".padStart(17)}${"lost".padStart(10)}` + `${"duplicated".padStart(11)}${"inversions".padStart(11)}${"wait rounds".padStart(14)}`); const R = {}; for (const [name, choice] of SCHEMES) { const r = (R[name] = run({ ...choice, seed: 20260730 })); console.log(`${name.padEnd(26)}${String(r.channelAttempts).padStart(17)}${String(r.lost).padStart(10)}` + `${String(r.duplicated).padStart(11)}${String(r.inversions).padStart(11)}${String(r.waitRounds).padStart(14)}`); } const mostOnce = R["at-most-once / free"], atLeastOnce = R["at-least-once / free"]; const convoy = R["same + sequential convoy"]; console.log(`\nexpected loss = ${DELIVERIES} x ${LOSS} = ${(DELIVERIES * LOSS).toFixed(2)}; measured ${mostOnce.lost}`); console.log(`cost of resending: channel attempts x${(atLeastOnce.channelAttempts / mostOnce.channelAttempts).toFixed(4)}, ` + `duplicate display rate ${((100 * atLeastOnce.duplicated) / DELIVERIES).toFixed(2)}%`); console.log(`cost of the convoy: ${(convoy.waitRounds / DELIVERIES).toFixed(3)} wait rounds per message; ` + `inversions ${atLeastOnce.inversions} -> ${convoy.inversions}`); ``` ``` model: 22222 deliveries, 40 chat lanes, loss 0.02, ack loss 0.01, seed 20260730 scheme channel attempts lost duplicated inversions wait rounds at-most-once / free 22222 408 0 0 0 at-least-once / free 22870 0 229 827 0 at-least-once + key 22870 0 0 827 0 same + sequential convoy 22870 0 0 0 1414 expected loss = 22222 x 0.02 = 444.44; measured 408 cost of resending: channel attempts x1.0292, duplicate display rate 1.03% cost of the convoy: 0.064 wait rounds per message; inversions 827 -> 0 ``` These numbers belong to the **measurement** class; they reproduce with seed 20260730. The first row shows what at-most-once delivery gets: in the one-second peak window, 408 messages are lost, and in exchange no duplication and no ordering violation arise. Expected loss was 444.44; the measured 408 is this run's count. Lost messages do not disrupt the order of the rest, because nothing is resent. At-most-once delivery being clean with respect to order is not a coincidence — it is the loss itself. The second row hands over the bill for at-least-once delivery. Loss drops to zero; the cost is channel attempts rising to 1.0292 times, 229 messages reaching the recipient a second time (1.03% of deliveries), and 827 inversions. The duplication comes from the acknowledgment being lost: the message has arrived, the sender does not know it. The third row isolates the uniqueness key's contribution: the 229 duplicate displays drop to zero, inversions stay at 827. The key ledger resolves duplication, **not order**; the two are separate problems and call for separate patterns. The fourth row is the sequential convoy's contribution: the 827 inversions drop to zero, in exchange for 0.064 wait rounds paid per message, 1414 rounds total in the window. The wait arises from loss — when one delivery drops, the messages behind it wait in the lane. In a round without loss, the convoy's cost is zero. ## Eliminated Alternative: Polling A design that holds no connection can be built: messages are written to the mailbox, and the client asks at a fixed interval. This design removes the 25 connection nodes, the routing directory, and session stickiness all at once. The reason for its elimination is a ratio. ```js // chat/alternative.mjs — numeric comparison of the push model against the polling model const CONNECTED = 1_250_000, PEAK_DELIVERY = 22222.22; // scale.mjs calculation: connected users, peak deliveries/s const THRESHOLD_MS = 200; // delivery delay threshold (source: the write-response expectation) console.log(`${"poll interval".padStart(16)}${"poll requests/s".padStart(17)}${"ratio to push".padStart(14)}` + `${"non-empty polls".padStart(16)}${"avg delay ms".padStart(17)}`); for (const interval of [(2 * THRESHOLD_MS) / 1000, 1, 3, 10, 30]) { const requests = CONNECTED / interval; const nonEmpty = Math.min(1, (PEAK_DELIVERY * interval) / CONNECTED); console.log(`${`${interval} s`.padStart(16)}${requests.toFixed(0).padStart(17)}` + `${`x${(requests / PEAK_DELIVERY).toFixed(2)}`.padStart(14)}${`${(100 * nonEmpty).toFixed(2)}%`.padStart(16)}` + `${((interval * 1000) / 2).toFixed(0).padStart(17)}`); } const breakeven = CONNECTED / PEAK_DELIVERY; console.log(`\nbreakeven interval = connected / peak delivery = ${breakeven.toFixed(2)} s`); console.log(`average delivery delay at this interval = ${((breakeven * 1000) / 2 / 1000).toFixed(2)} s`); console.log(`interval needed for a ${THRESHOLD_MS} ms threshold = ${((2 * THRESHOLD_MS) / 1000).toFixed(1)} s, ` + `that is, x${(CONNECTED / ((2 * THRESHOLD_MS) / 1000) / PEAK_DELIVERY).toFixed(2)} of push's requests`); ``` ``` poll interval poll requests/s ratio to push non-empty polls avg delay ms 0.4 s 3125000 x140.63 0.71% 200 1 s 1250000 x56.25 1.78% 500 3 s 416667 x18.75 5.33% 1500 10 s 125000 x5.63 17.78% 5000 30 s 41667 x1.88 53.33% 15000 breakeven interval = connected / peak delivery = 56.25 s average delivery delay at this interval = 28.13 s interval needed for a 200 ms threshold = 0.4 s, that is, x140.63 of push's requests ``` The elimination number is the last line: a 200-millisecond average delay threshold demands a 0.4-second interval, which is 3,125,000 requests per second — 140.63 times the delivery rate in the push model. What is more, 99.29% of these requests return empty. Polling is a rate paid for work that is not carried. What the alternative wins is written too. The breakeven interval is 56.25 seconds: if the delay threshold rises above this, polling produces fewer requests, and the 25 connection nodes and the routing directory drop out of the design. The elimination is a delivery-delay-threshold decision; if the threshold were one minute, the choice would reverse. ## Failure Behavior and What Is Sacrificed When a connection node goes down, 50,000 connections drop at once and all of them try to reconnect. This is the scenario from the Performance Anti-Patterns and Monitoring course's Retry Storm lesson; its counterpart is the jitter and retry budget from the Resilience and Reliability course's Retries and Storm Risk lesson. The parameter is a ceiling: the 50,000 connections spread across the remaining 24 nodes mean 2083 extra connections per node, that is, 4.17% utilization. When the broker goes down, the design follows the path from the Graceful Degradation lesson: the connection stays up, presence and read receipts stop, and message sending falls through to the offline mailbox. What is sacrificed, in one sentence: a single lane per chat accepts giving up chat-level parallelism for the sake of in-chat order, and a 1414-round wait during rounds with loss. ## Summary - When writing dominates, the measure is record count: peak sends are 9259.26 while peak deliveries are 22,222.22, and bandwidth stays small at 39.11 Mbit/s. - 1,250,000 concurrent connections mean 25 nodes; when the connected share doubles, the delivery rate does not change, only the node count doubles. - At-most-once delivery lost 408 messages in the one-second peak window and never disrupted order; at-least-once delivery zeroed out loss but produced 229 duplicate displays and 827 inversions. - The uniqueness key zeroed out duplication and left inversions at 827; the sequential convoy zeroed out inversions and took 0.064 wait rounds per message. - Polling demands 140.63 times push's requests at the 200-millisecond threshold, and 99.29% of them return empty; the breakeven interval is 56.25 seconds. - Edge caching was deliberately not used: every message goes to a single set of recipients and never produces a shared hit. ## Next Step In this case delivery moved along a single path: the recipient was either connected or the message was written to their mailbox, and failure was met with a resend. The next case asks the same question where more than one path exists. The same notification can go out over different channels, the channels' failure rates differ from one another, and one channel's failure is grounds for trying another. The number to ask then is this: how much does trying the channels in sequence versus at the same time move the total delivery rate and the resend load.