--- title: 'Session Stickiness' source: 'https://academia.sh/en/courses/traffic-layer/session-stickiness' course: 'The Traffic Layer' language: en updated: '2026-08-23T14:25:25+00:00' license: 'CC BY-SA 4.0' --- # Session Stickiness Choice becoming a constraint and the scaling bill that follows: measuring how much sticky binding breaks distribution, counting the sessions lost when a replica drops and the sessions whose binding breaks when a replica is added, the busiest replica barely dropping despite the added replica, and converting the unsplittable share into an upper bound with the introductory course's serial fraction relation. The five lessons so far never questioned one thing: the balancer's choice was free. A request could go to any replica, because the replicas were each other's equals. Even consistent hashing, while it wanted the choice to stay stable, did not make it mandatory — if a key landed on a different replica, the response would still be correct, only the cache would run cold. The picture changes if state left over from a client's previous request sits in a particular replica's memory. That client's next request can no longer go to just any replica; choice stops being a preference and becomes a **constraint**. This binding is called **session stickiness**. The **sticky routing** from the Caching, Queues and Asynchronous Processing course is the same thing: the client is always routed to the same instance. Session state itself and its management are the Server-Side Fundamentals course's subject and are not re-derived here; this lesson's job is to measure stickiness's **scaling bill**. ## Measuring the Binding The model takes a client mix and compares three rules under the same mix. Under the **non-sticky** rule, the decision is made per request. Under **sticky-modulo**, the client identity's hash is divided by the replica count. Under **sticky-consistent**, the binding is read from the consistent hashing ring from the previous lesson. The client mix carries the shape of a real parcel tracking service: one major seller integration sends many queries from a single session, nine medium sellers send fewer, ninety minor sellers very few. The numbers are model parameters. ```js // sticky/model.mjs — the effect of stickiness on distribution, session loss, and scale-up. // This is a MODEL: the client mix and request counts are model parameters. const CLIENTS = [ // [name prefix, client count, requests per client] ["major", 1, 2000], ["medium", 9, 400], ["minor", 90, 50], ]; const VIRTUAL = 200; // consistent hashing virtual node count function hash(text) { // FNV-1a plus a final mix step; avalanche effect defined in let h = 2166136261; // the Hash Tables lesson: similar keys must give distant values for (let i = 0; i < text.length; i += 1) h = Math.imul(h ^ text.charCodeAt(i), 16777619) >>> 0; h ^= h >>> 15; h = Math.imul(h, 2246822507) >>> 0; h ^= h >>> 13; return h >>> 0; } const clients = CLIENTS.flatMap(([name, count, requests]) => Array.from({ length: count }, (_, i) => ({ name: `${name}-${i}`, requests }))); const TOTAL = clients.reduce((a, c) => a + c.requests, 0); function ring(replicas) { const h = []; for (const k of replicas) for (let v = 0; v < VIRTUAL; v += 1) h.push([hash(`${k}#${v}`), k]); return h.sort((x, y) => x[0] - y[0]); } function onRing(a, h) { const c = hash(a); if (c > h[h.length - 1][0]) return h[0][1]; let lo = 0, hi = h.length - 1; while (lo < hi) { const m = (lo + hi) >> 1; if (h[m][0] < c) lo = m + 1; else hi = m; } return h[lo][1]; } function binding(rule, replicas) { // client -> replica mapping (no mapping for non-sticky) if (rule === "sticky-modulo") return new Map(clients.map((c) => [c.name, replicas[hash(c.name) % replicas.length]])); if (rule === "sticky-consistent") { const h = ring(replicas); return new Map(clients.map((c) => [c.name, onRing(c.name, h)])); } return null; } function load(rule, replicas) { // requests per replica const y = new Map(replicas.map((k) => [k, 0])); const b = binding(rule, replicas); if (b === null) { // per-request round robin: each request decided separately let s = 0; for (const c of clients) for (let i = 0; i < c.requests; i += 1) { const k = replicas[s++ % replicas.length]; y.set(k, y.get(k) + 1); } } else for (const c of clients) y.set(b.get(c.name), y.get(b.get(c.name)) + c.requests); return y; } const imbalance = (y, n) => (Math.max(...y.values()) / TOTAL) * n; const THREE = ["k1", "k2", "k3"], FOUR = [...THREE, "k4"]; console.log(`${clients.length} clients, ${TOTAL} requests; largest client's share = ` + `${(Math.max(...clients.map((c) => c.requests)) / TOTAL).toFixed(4)}\n`); console.log("rule 3 replicas load imbalance 4 replicas load imbalance busiest replica drop"); for (const rule of ["non-sticky", "sticky-modulo", "sticky-consistent"]) { const u = load(rule, THREE), d = load(rule, FOUR); const pu = Math.max(...u.values()) / TOTAL, pd = Math.max(...d.values()) / TOTAL; console.log(`${rule.padEnd(18)} ${[...u.values()].join("/").padEnd(20)} ${imbalance(u, 3).toFixed(3).padStart(11)} ` + `${[...d.values()].join("/").padEnd(25)} ${imbalance(d, 4).toFixed(3).padStart(11)} ` + `${(1 - pd / pu).toFixed(4).padStart(21)}`); } console.log("\nclients whose binding changes when a replica is added, and their request share:"); for (const rule of ["sticky-modulo", "sticky-consistent"]) { const bu = binding(rule, THREE), bd = binding(rule, FOUR); const moved = clients.filter((c) => bu.get(c.name) !== bd.get(c.name)); console.log(` ${rule.padEnd(18)} clients ${String(moved.length).padStart(3)}/${clients.length} ` + `request share ${(moved.reduce((a, c) => a + c.requests, 0) / TOTAL).toFixed(4)}`); } console.log("\nsessions lost when a replica drops, and that replica's request share (3 replicas):"); for (const rule of ["non-sticky", "sticky-modulo", "sticky-consistent"]) { const b = binding(rule, THREE); if (b === null) { console.log(` ${"non-sticky".padEnd(18)} sessions 0/${clients.length} request share 0.0000 (no state sits on the replica)`); continue; } const dropped = clients.filter((c) => b.get(c.name) === "k1"); console.log(` ${rule.padEnd(18)} sessions ${String(dropped.length).padStart(3)}/${clients.length} ` + `request share ${(dropped.reduce((a, c) => a + c.requests, 0) / TOTAL).toFixed(4)}`); } ``` ``` 100 clients, 10100 requests; largest client's share = 0.1980 rule 3 replicas load imbalance 4 replicas load imbalance busiest replica drop non-sticky 3367/3367/3366 1.000 2525/2525/2525/2525 1.000 0.2501 sticky-modulo 2700/2250/5150 1.530 4450/2400/1800/1450 1.762 0.1359 sticky-consistent 2200/3150/4750 1.411 1250/2850/4300/1700 1.703 0.0947 clients whose binding changes when a replica is added, and their request share: sticky-modulo clients 70/100 request share 0.7475 sticky-consistent clients 27/100 request share 0.1683 sessions lost when a replica drops, and that replica's request share (3 replicas): non-sticky sessions 0/100 request share 0.0000 (no state sits on the replica) sticky-modulo sessions 40/100 request share 0.2673 sticky-consistent sessions 30/100 request share 0.2178 ``` ## Three Separate Bills The tables show stickiness being paid for in three separate items. **Distribution.** Under the non-sticky rule, imbalance is 1.000 — requests split by count and client identity plays no part. Under sticky-modulo it is 1.530, under sticky-consistent 1.411. The deviation's source is that clients do not send equal request counts: the major seller's 2000 requests land on a single replica, and the balancer has no authority to correct that. This is the same connection skew as the second lesson, except there the binding was a connection's lifetime, here it is a session. **Replica drop.** Under the non-sticky rule, a dropped replica carries no session at all; requests route to the remaining replicas and nothing is lost. Under the sticky rules, sessions bound to the dropped replica drop too: under sticky-consistent, 30 of 100 sessions and 21.78 percent of requests. The health check from the first lesson does not reduce this loss — dropping a replica from the pool gets requests answered, but it does not bring back the sessions that lived in that replica's memory. The health check's 34.26-request window does not apply here; the loss is measured in sessions, not a window. **Replica addition.** The third table is stickiness's most insidious item. Under sticky-modulo, when a fourth replica is added, 70 of 100 clients bind to a different replica, and those clients carry 74.75 percent of the total requests — scaling up breaks nearly three-quarters of the load's session bindings. Under sticky-consistent the ratio drops to 27 clients and 16.83 percent — the previous lesson's movement result repeats here in terms of sessions. The column at the end of the same row gives the actual result. When the fourth replica is added, the busiest replica's load drops 25.01 percent under the non-sticky rule — the figure a perfect split gives. Under the sticky rules the drop is 13.59 percent and 9.47 percent. **Adding a replica does not lower the peak load**, because the large session that creates the busiest replica's load stays put. Sticky-consistent being better at preserving sessions works against it here: keeping the binding means not moving the load. ## Back to the Computation This behavior's name was already set in C01. The Horizontal and Vertical Scaling lesson named the unsplittable part of a job the **serial fraction** and wrote the best possible speedup with $n$ units as: $$ S(n) = \frac{1}{s + \dfrac{1 - s}{n}} $$ Stickiness's counterpart to the serial fraction is direct: **a single session cannot be split.** The largest session's ratio to the total load stays on the busiest replica no matter how many replicas are added. ```js // sticky/scale.mjs — converting stickiness to a serial fraction and its counterpart in C01's peak rate const CLIENTS = [["major", 1, 2000], ["medium", 9, 400], ["minor", 90, 50]]; // same as model.mjs const PEAK_EDGE = 513.89; // C01 Back-of-the-Envelope Estimation: peak edge requests/s const SATURATION = 400; // lesson 01 assumption Y1 const SAFE = 200; // lesson 01: Y1 x Y2 const MEASURED = { 3: 1.411, 4: 1.703 }; // model.mjs output: sticky-consistent imbalance const total = CLIENTS.reduce((a, [, count, req]) => a + count * req, 0); const s = Math.max(...CLIENTS.map(([, , req]) => req)) / total; // the part that cannot be split const amdahl = (n) => 1 / (s + (1 - s) / n); // C01 Horizontal and Vertical Scaling lesson's relation console.log(`total requests = ${total}, largest session's share (serial fraction s) = ${s.toFixed(4)}`); console.log(`upper bound 1/s = ${(1 / s).toFixed(2)}: no matter the replica count, peak load cannot be split beyond this factor\n`); console.log("replicas non-sticky speedup sticky speedup busiest replica lower bound req/s utilization now"); for (const n of [3, 4, 8, 16, 64]) { const lowerBound = PEAK_EDGE / amdahl(n); console.log(`${String(n).padStart(8)} ${String(n).padStart(18)} ${amdahl(n).toFixed(2).padStart(14)} ` + `${lowerBound.toFixed(2).padStart(34)} ${(lowerBound / SATURATION).toFixed(3).padStart(15)}`); } console.log(`${"infinite".padStart(8)} ${"infinite".padStart(18)} ${(1 / s).toFixed(2).padStart(14)} ` + `${(PEAK_EDGE * s).toFixed(2).padStart(34)} ${((PEAK_EDGE * s) / SATURATION).toFixed(3).padStart(15)}\n`); console.log("replicas measured imbalance busiest replica req/s utilization replicas needed"); for (const [n, d] of Object.entries(MEASURED)) { const rate = (d * PEAK_EDGE) / n; console.log(`${n.padStart(8)} ${d.toFixed(3).padStart(18)} ${rate.toFixed(2).padStart(22)} ` + `${(rate / SATURATION).toFixed(3).padStart(11)} ${String(Math.ceil((d * PEAK_EDGE) / SAFE)).padStart(16)}`); } console.log(`\nsensitivity: if the largest session were 10% of the total, the upper bound would be ${(1 / 0.10).toFixed(2)}; ` + `at 30% it would be ${(1 / 0.30).toFixed(2)}`); ``` ``` total requests = 10100, largest session's share (serial fraction s) = 0.1980 upper bound 1/s = 5.05: no matter the replica count, peak load cannot be split beyond this factor replicas non-sticky speedup sticky speedup busiest replica lower bound req/s utilization now 3 3 2.15 239.14 0.598 4 4 2.51 204.79 0.512 8 8 3.35 153.28 0.383 16 16 4.03 127.52 0.319 64 64 4.75 108.20 0.270 infinite infinite 5.05 101.76 0.254 replicas measured imbalance busiest replica req/s utilization replicas needed 3 1.411 241.70 0.604 4 4 1.703 218.79 0.547 5 sensitivity: if the largest session were 10% of the total, the upper bound would be 10.00; at 30% it would be 3.33 ``` These numbers are in the **computation** class. The serial fraction is 0.1980 and the upper bound is 5.05: even if the replica count goes to infinity, the peak load cannot be split more than fivefold. The gap widens next to the non-sticky column — at 16 replicas the non-sticky rule splits the load 16-fold, the sticky rule 4.03-fold. Most of the extra replicas added do nothing for the busiest replica. The utilization column ties this to a threshold. The first lesson chose a target utilization of 0.50 per replica (Y2). Under the sticky regime, three replicas do not hold that target: the busiest replica carries at least 239.14 req/s and 0.598 utilization. At four replicas, 204.79 and 0.512 — still above target. Reaching below target takes eight replicas, and at that point eight replicas do the work of three. **Stickiness more than doubles the replica count needed, and no capacity arrives in return for the increase.** The second table runs the same computation with the measured imbalance. At three replicas the busiest replica is 241.70 req/s and 0.604 utilization; at four, 218.79 and 0.547. Adding one replica lowers the busiest replica's load by only 9.4 percent. Holding the imbalance measured at three replicas against the safe per-replica rate implies 4 replicas are actually needed; the imbalance measured at four implies 5. The point of comparison is the first lesson's number: without stickiness 3 replicas were enough; the second lesson's connection skew raised that to 5; here it comes out to 4 or 5 — the session binding's own share. The result is this layer's counterpart to C01's horizontal-scaling rule. That lesson said horizontal scaling does not happen just by increasing the number of units, but because the job can be split in a way that requires no state shared between units, and it measured index partitioning's serial fraction at 28.0 percent. Session stickiness is the traffic layer's version of the same statement: as long as state stays in a replica's memory, the balancer cannot split the load no matter how smart its rule is. Removing stickiness is not a balancing decision — it is a decision to move the state out of the process. ## Summary - Session stickiness binds a client's requests to a particular replica and turns the distribution choice into a constraint; sticky routing from the Caching, Queues and Asynchronous Processing course is the same binding. - Under the same client mix, imbalance came out to 1.000 for non-sticky, 1.530 for sticky-modulo, and 1.411 for sticky-consistent; the deviation's source is that clients do not send equal request counts. - When a replica drops, sticky-consistent loses 30 of 100 sessions and 21.78 percent of requests; the health check saves the request, not the session. - When a fourth replica is added, the busiest replica's load drops 25.01 percent under non-sticky, 13.59 percent under sticky-modulo, and 9.47 percent under sticky-consistent; keeping the binding means not moving the load. - The largest session's share, 0.1980, is a serial fraction and holds the scaling upper bound at 5.05: at 16 replicas the non-sticky rule splits the load 16-fold, the sticky rule 4.03-fold. - The 0.50 utilization target set by Y2 does not hold under the sticky regime at three replicas (0.598) or at four (0.512); the measured imbalance implies 4 replicas are needed at the three-replica reading, versus 3 without stickiness. ## Next Step This topic settled which replica a request lands on. The balancer is a routing device: it makes its decision at the connection level or the content level, picks its rule by turn, queue length, or key, drops a failed replica from the pool through the health check, and loses its power to choose once state stays in a replica's memory. The work piling up at the edge does not end there. The same point authenticates identity, applies rate limits, translates into the shape a client expects, and gathers responses from several services to satisfy a single user request. Whether this work belongs to the balancer, to the application, or to a separate layer was left open — the third option has a name, and the next topic starts with it: what a gateway, where edge responsibilities gather, should take on, and what it should not.