Security

Every measurement in this course assumed that the store talks only to its own clients; every cost counted so far was paid without asking who sent the command. This lesson closes that gap and covers not the mechanics of identity but the store’s configuration: who can connect, which command classes a connection can run, and what these decisions cost in the memory budget.

Configuration Is a Surface

The default setup is kept simple: a process with authentication off, listening on every network interface, accepting every command class. This setup has a measurable surface, and its size is a key count and a byte count.

The three decisions that determine the surface do not substitute for one another. Authentication decides who can connect, command restriction decides what a connection can do, and network isolation decides whether a connection attempt reaches the store at all. The fourth decision, encryption, does not narrow the surface; it makes data in transit unreadable, and it is the only cost line item of the four.

Mechanism

The mechanism is a model: no real store, network, or connection is set up, access is a number. Command names are not counted, command classes are; a step is an abstract passage, not measured time.

CU17 — the key space consists of the catalog (3,000 × 240 bytes), session (8,000 × 120), loan (12,000 × 96), queue (800 × 128), and branch counters (6 × 64); session, loan, and queue are secret. Rationale: these three regions carry data tied to a reader’s identity, the catalog and counters do not. CU18 — the encryption buffer is 32,768 bytes per connection, the frame overhead is 29 bytes, the extra steps per connection are 3 (handshake 2 + auth 1). Rationale: the three line items depend on three separate scales — the buffer is allocated per connection, the extra bytes repeat on every frame, the handshake is paid only when a connection is established. CU19 — while server-side scripting is on, every step the caller can reach can also be run from inside a script. Rationale: a script is not a separate authority domain.

// security.mjs — open surface and configuration cost is an IN-PROCESS MODEL. No real store, network, or
// connection is set up; access is a number. Command NAMES are not counted, command CLASSES are. A step
// is an abstract passage, not measured time.
const SPACE = [                      // [name, entries, bytes per entry, secret?]
  ["catalog", 3000, 240, false], ["session", 8000, 120, true], ["loan", 12000, 96, true],
  ["queue", 800, 128, true], ["counter", 6, 64, false]];
const CLASS = [                      // [name, destructive?, script?]
  ["read", false, false], ["write", false, false], ["delete", true, false],
  ["flush", true, false], ["config", true, false], ["script", true, true]];
const BUFFER = 32768, FRAME = 29, HANDSHAKE = 2, AUTH_STEP = 1;
const ROLES = 4, ROLE_BYTES = 128, PERMISSION_BYTES = 24;

const keys = SPACE.reduce((a, [, n]) => a + n, 0);
const bytes = SPACE.reduce((a, [, n, b]) => a + n * b, 0);
const secret = SPACE.reduce((a, [, n, b, g]) => a + (g ? n * b : 0), 0);

// configuration: [auth, destructive classes open, script open, network isolation, encryption]
const CONFIG = [
  ["default", false, true, true, false, false], ["auth", true, true, true, false, false],
  ["restrict+script", true, false, true, false, false], ["restrict", true, false, false, false, false],
  ["isolation", true, false, false, true, false], ["isolation+encryption", true, false, false, true, true]];

function surface([, auth, destructive, script, isolation]) {
  const open = CLASS.filter(([, d, s]) => (d ? destructive : true) && (s ? script : true));
  // Stranger: an unauthenticated network call. If isolation is on, it never even arrives.
  const strangerOpen = isolation || auth ? 0 : 1;
  // Application: authenticated, a restricted role. Classes closed off get reopened while scripting is on.
  const appDestructive = destructive || script ? bytes : 0;
  return { open: open.length, strangerKeys: strangerOpen * keys, strangerBytes: strangerOpen * bytes,
           strangerSecret: strangerOpen * secret, appDestructive };
}

const s = (x, n) => String(x).padStart(n);
const e = (x, n) => String(x).padEnd(n);
const pct = (x, n) => s((x * 100).toFixed(1) + "%", n);
const row = (cells, widths) => cells.map((c, i) => s(c, widths[i])).join(" | ");

console.log(`Key space: ${keys} keys, ${bytes} bytes; ${secret} of those bytes are secret`);
console.log("(session, loan, queue). Six command classes: read, write, delete, flush,");
console.log("config, script. Stranger = an unauthenticated network call.\n");
const W1 = [21, 12, 15, 14, 12, 18];
console.log(e("configuration", W1[0]) + " | " +
  row(["open classes", "stranger keys", "stranger bytes", "secret bytes", "app can destroy"], W1.slice(1)));
console.log(W1.map((w) => "-".repeat(w)).join("-|-"));
for (const a of CONFIG) {
  const r = surface(a);
  console.log(e(a[0], W1[0]) + " | " +
    row([r.open + "/6", r.strangerKeys, r.strangerBytes, r.strangerSecret, r.appDestructive + " bytes"], W1.slice(1)));
}

const REQUESTS = 200000;
console.log(`\nStep cost (${REQUESTS} requests; handshake ${HANDSHAKE} + auth ${AUTH_STEP} = 3 steps/connection):`);
const W2 = [24, 11, 11, 24, 19];
console.log(row(["requests/connection", "connections", "extra steps", "extra steps/request", "frame overhead"], W2));
console.log(W2.map((w) => "-".repeat(w)).join("-|-"));
for (const k of [1, 10, 100, 1000]) {
  const c = Math.ceil(REQUESTS / k), extra = c * (HANDSHAKE + AUTH_STEP);
  console.log(row([k, c, extra, (extra / REQUESTS).toFixed(3), REQUESTS * 2 * FRAME + " B"], W2));
}

console.log("\nEncryption's MEMORY cost (16 KiB read + 16 KiB write buffer per connection):");
const W3 = [23, 12, 10, 13, 20];
console.log(row(["concurrent connections", "buffer bytes", "data bytes", "buffer/data", "role+permission table"], W3));
console.log(W3.map((w) => "-".repeat(w)).join("-|-"));
for (const n of [8, 32, 128, 512]) {
  console.log(row([n, n * BUFFER, bytes, ((n * BUFFER / bytes) * 100).toFixed(1) + "%",
    ROLES * ROLE_BYTES + ROLES * CLASS.length * PERMISSION_BYTES + " B"], W3));
}

const ledger = ROLES * ROLE_BYTES + ROLES * CLASS.length * PERMISSION_BYTES;
console.log("\nquantities independent of the run:");
console.log("  frame overhead    = requests x 2 x " + FRAME + " bytes; independent of connection count");
console.log("  steps per request = 3 / (requests per connection)");
console.log("  encryption buffer = concurrent connections x " + BUFFER + " bytes");
console.log("  role+permission ledger = " + ROLES + "x" + ROLE_BYTES + " + " + ROLES + "x" + CLASS.length + "x" +
  PERMISSION_BYTES + " = " + ledger + " bytes; independent of connections");
console.log("  buffer overtakes data: ceil(" + bytes + " / " + BUFFER + ") = " + Math.ceil(bytes / BUFFER) +
  " concurrent connections");

Key space: 23806 keys, 2934784 bytes; 2214400 of those bytes are secret
(session, loan, queue). Six command classes: read, write, delete, flush,
config, script. Stranger = an unauthenticated network call.

configuration         | open classes |   stranger keys | stranger bytes | secret bytes |    app can destroy
----------------------|--------------|-----------------|----------------|--------------|-------------------
default               |          6/6 |           23806 |        2934784 |      2214400 |      2934784 bytes
auth                  |          6/6 |               0 |              0 |            0 |      2934784 bytes
restrict+script       |          2/6 |               0 |              0 |            0 |      2934784 bytes
restrict              |          2/6 |               0 |              0 |            0 |            0 bytes
isolation             |          2/6 |               0 |              0 |            0 |            0 bytes
isolation+encryption  |          2/6 |               0 |              0 |            0 |            0 bytes

Step cost (200000 requests; handshake 2 + auth 1 = 3 steps/connection):
     requests/connection | connections | extra steps |      extra steps/request |      frame overhead
-------------------------|-------------|-------------|--------------------------|--------------------
                       1 |      200000 |      600000 |                    3.000 |          11600000 B
                      10 |       20000 |       60000 |                    0.300 |          11600000 B
                     100 |        2000 |        6000 |                    0.030 |          11600000 B
                    1000 |         200 |         600 |                    0.003 |          11600000 B

Encryption's MEMORY cost (16 KiB read + 16 KiB write buffer per connection):
 concurrent connections | buffer bytes | data bytes |   buffer/data | role+permission table
------------------------|--------------|------------|---------------|---------------------
                      8 |       262144 |    2934784 |          8.9% |               1088 B
                     32 |      1048576 |    2934784 |         35.7% |               1088 B
                    128 |      4194304 |    2934784 |        142.9% |               1088 B
                    512 |     16777216 |    2934784 |        571.7% |               1088 B

quantities independent of the run:
  frame overhead    = requests x 2 x 29 bytes; independent of connection count
  steps per request = 3 / (requests per connection)
  encryption buffer = concurrent connections x 32768 bytes
  role+permission ledger = 4x128 + 4x6x24 = 1088 bytes; independent of connections
  buffer overtakes data: ceil(2934784 / 32768) = 90 concurrent connections

Reading the Open Surface

In the default setup, the surface is the entire data set. The first row shows 23,806 keys and 2,934,784 bytes; 2,214,400 bytes of that is information about which reader borrowed what. All six of the six command classes are open: the same call does not just read these bytes, it can flush the key space.

Authentication removes the stranger, it does not restrict the application. In the second row, the stranger columns drop to zero, but the last column does not change: 2,934,784 bytes can still be destroyed. Authentication answers the question “who,” not the question “what can they do.” If the batch job that updates the catalog and the application that writes loan records connect with the same identity, a bug in one takes down the entire key space.

Command restriction closes nothing while scripting is on. The third row is this lesson’s central finding: open classes drop from 6 to 2, destructive classes are closed, but the bytes the application can destroy stay at 2,934,784. Server-side scripting runs the caller’s steps; the door is shut, the window is left open. In the fourth row, once scripting is restricted too, the destructive surface zeroes out.

Network isolation is the next layer, not a substitute for one. The fifth row does not produce a different number from the fourth; even so, isolation prevents a fallback to the first row if authentication gets disabled. The sixth row shows that encryption does not narrow the surface at all.

Identity Is Cheap, Encryption Is Expensive

The second and third tables write out the bill line by line.

The role and permission ledger is practically free: 1,088 bytes, and it is independent of connection count besides. The decision that takes the surface from 2,934,784 bytes to zero occupies a thousandth of the memory budget.

The handshake is per connection, so its cost erodes with reuse. When a single request lands per connection, 200,000 connections and 600,000 extra steps get paid — 3 steps per request, more expensive than the work itself. At a thousand requests per connection, the same line item drops to 600 steps, 0.003 per request. Encryption’s step cost is not an encryption problem, it is a connection-management problem.

Frame overhead is unaffected by reuse: 11,600,000 bytes for 200,000 requests, the same across all four rows.

The real memory line item is the connection buffer. The buffer is 8.9 percent of the data at 8 concurrent connections, 142.9 percent at 128, 571.7 percent at 512. The run-independent row gives the turning point — past 90 concurrent connections, encryption buffers take up more room than the data being stored. In an in-memory store, this is a number that changes when the eviction policy kicks in.

Summary

• A default setup’s surface is the entire data set: 23,806 keys, 2,934,784 bytes (2,214,400 of them secret), and all six of the six command classes are open.
• Authentication zeroes the bytes a stranger can reach but leaves the 2,934,784 bytes an authenticated call can destroy; the two decisions are separate.
• Command restriction did not shrink the destructive surface at all while scripting was on: even with open classes dropping from 6 to 2, destructible bytes stayed at 2,934,784, and only zeroed out once scripting was restricted too.
• The role and permission ledger is 1,088 bytes and independent of connection count: the decision that zeroes the surface takes up no memory to speak of.
• Encryption’s step cost erodes with reuse (3.000 → 0.003 per request), but frame overhead is fixed: 11,600,000 bytes for 200,000 requests.
• Encryption’s memory cost is per connection; past 90 connections, buffers exceed the data stored, and at 512 connections the ratio is 571.7 percent.

Course Wrap-Up

Lesson	Bytes Held	What It Bought	What It Lost
Key-Value Basics	20,000 entries, 2,397,230 bytes (120 bytes/entry)	expiration cut the peak to 262,936 bytes	16,951 extra re-entries
Strings and Counters	200,000 counters, 15,545,901 bytes (77.7 bytes/number)	query 119,993 → 1 step; zero loss on atomic increment	77.2 steps per byte (6,410 for a partial counter)
Lists	linked node 43.0 bytes/entry (array 19.0)	operation 1,673 → 1 step	a rank query is still a scan (14,500 steps); trimming narrows the window to two hours
Hash Structures	200,000 records 91.2 MB (monolithic 43.8 MB)	daily writes 39.2 → 5.4 MB, reads 105.8 → 9.3 MB	237 bytes of overhead per record
Sets and Sorted Sets	38,000 members 2,052,056 bytes (list 532,056)	membership 28,464.2 → 1.0 step	order information; 3,135,704 bytes more in the skip list
Bitmaps and Probabilistic Structures	bitmap 250,001, estimate 4,096 bytes	16 and 1,024 times less space than the full set	2.53% false positive rate; which reader got counted
Streams	5,078 KiB at the length limit, 9,031 KiB at the safe trim	read position 16 bytes; three groups’ ledger 48 bytes	38,920 and 46,220 entries missed under bounded trimming
Spatial Indexes	cell bucket 5.09, sorted set 25.48 bytes/member	candidates scanned 120,000 → 821.22	in a cell bucket, granularity is fixed at insertion time
Server-Side Scripting	script cache 642, lock table peaks at 88,000 bytes	exactly three loans at every concurrency level (31.93 on the client)	a long script left 8,000 commands waiting in the queue
Snapshot Persistence	file 717,260 bytes (1,593,100 in memory)	average loss 10,001 → 501 writes	disk writes 2 → 30 MB; peak of 59,565 bytes during the copy
Append Log Persistence	39.7 bytes per write, the file grows linearly with writes	zero loss on fsync at every write	60,000 fsyncs; a file 3.32 times the snapshot
Persistence Selection	combined use 4,229,117 bytes, 23 fsyncs	recovery 55,000 → 32,330 steps, 1,247,842 bytes read	9,816 entries missing or stale with the snapshot alone
Memory Limit and Eviction Policies	a 400,000-byte budget, overhead share 9.9%-24.7%	random 4,938, least-frequently-used 4,123 entries	4.4-point hit rate under a tight budget; 709,241 dead-entry bytes
Expiration Management	4,027,669 bytes under lazy, 1,464,802 under active cleanup	dead share 73.6% → 27.4%	0.774 entry touches per access
Replication	2,343.8 KiB at four replicas (single node 468.8) + buffer 7.5-60 KiB	reads per node 400 → 80	11,420 stale reads; a 47.96% rate at delay 8
Automatic Failover	hot backup 937.5 KiB (cold 468.8)	outage 17 → 5 rounds, loss 760 → 80 records	40/80/160/320 accepted writes lost per failover
Clustering	1,539.6 untagged, 2,899.0 KiB with the branch tag	multi-key transaction 1.15% → 100%	slots 8,437 → 6, imbalance 1.085 → 4.913
Transactions and Optimistic Locking	watch overhead 768 bytes (counters 384)	over-limit 86 → 0	480 → 4,314 ticks; 96,414 client-ticks waiting at block 16
Publish-Subscribe	52 bytes per channel (36,865-145,243 in a persistent structure)	600 publishes, 753 delivered copies, no ledger kept	74 publishes vanished unsubscribed; 720 publishes lost for a slow subscriber
Typical Use Cases	hash form 60,333 bytes (string 45,825)	writes over a thousand updates 90,728 → 8,000 bytes	14,508 bytes more; sorted set 2,894.4 bytes/user
Metrics and Slow Command Analysis	slow command log 108 records at threshold 20, 6,340 bytes	longest wait 3,999 → 38, total 58,681 units	scan finished at 8,001 instead of 6,001 units, 398 entries never seen
Security	role+permission ledger 1,088 bytes; buffer 4,194,304 at 128 connections	2,934,784 bytes open to a stranger, and the destructive surface zeroed out	3 steps per connection, 11,600,000 bytes of frame overhead

All twenty-two lessons are an application of the same rule: memory is a budget; every structure buys something and costs something. The table’s second column has no zeros at all — even a structure that estimates asks for 4,096 bytes. Nor does the third; even the cheapest gain gives back a certainty, an order, a window, or a guarantee. The store being fast was not this course’s subject, it was its assumption; the subject was what that speed costs.

Every access so far carried one shared assumption: the key was known. Even in the most complex measurement, the first step was a known key; the spatial index only appeared to stretch this, and there too, a coordinate got converted into a key. Yet this is the most common question asked of a library catalog: a reader does not know the book’s identity, only a word that appears in its title. Going from a word inside a text to a document was never covered in this course, and it cannot be covered with key-value access. M17/K07 Search Engines and Text Retrieval opens with exactly this question.