Object Storage

The previous course ended by renting the machine and the network: instances launched, settled into subnets, passed through a filter, and opened outward. One question stayed open — where the data of everything running on those machines lives. The local disk is not the answer: when the instance shuts down the disk goes with it, and within a scaling group there is no telling which instance holds which data. In the rented layer, data has a place of its own, and choosing that place hands a set of decisions to the provider.

This is the course’s measure: every lesson counts which decision is delegated, which one stays, which of the ones that stay cannot be undone, and what data costs on the way out. The first type is the most common default: object storage. It is true that it is cheap, and that carries no decision; the decision is carried by the fact that object storage is not a file system.

Bucket, Object, and Key

There are three concepts. A bucket is a namespace; an object sits inside it; every object is reached by a key. A key looks like a path but is not one: meter/2026/03/14/A118.json is a plain string, and the slashes inside it carry no meaning for the store. The store looks this string up in a dictionary. Slashes only matter in a prefix query: keys that start with a given string can be listed. There is no such object as a directory, so there is no empty directory either, no directory delete, and no field to read for a directory’s size.

This single difference has four consequences, and all four can be counted: prefix instead of directory, copy instead of rename, full rewrite instead of partial update, and listing — assumed free — turning out to be a billed operation. The model below counts these over the fictional measurement network’s object set.

• ST1 — The regional measurement network is fictional: 4,000 meters, 310 days of history, four object families. Counts and average sizes are written into the model.
• ST2 — Fees are denominated in a fictional unit, not a real provider’s price; storage, request, and egress rates are written out explicitly in the model.
• ST3 — In the model a list request returns at most 1,000 keys; the page size varies from provider to provider. A request’s round trip takes 45 ms and requests are sequential on a single stream.
• ST4 — Copying happens server-side: it incurs no egress fee, only a request fee.

// object/measurement-network.mjs — the regional measurement network is FICTIONAL: the object set and the fees are both a model.
// Every fee is denominated in a fictional "unit"; it is not a real provider's price.
export const METERS = 4000, DAYS = 310;             // fictional network: 4,000 meters, 310 days of history
export const FAMILIES = [
  { name: "meter-readings", count: METERS * DAYS, avg: 2400, cadence: "daily" },
  { name: "invoice-documents", count: 86_000, avg: 380 * 1024, cadence: "monthly" },
  { name: "field-photos", count: 42_000, avg: Math.round(3.6 * 1024 * 1024), cadence: "once" },
  { name: "work-order-records", count: 310_000, avg: 11 * 1024, cadence: "daily" },
];
export const RATE = {                               // fictional unit
  storage: 0.023,                                    // unit / GiB / month
  writeRequest: 0.0050 / 1000,                       // unit / request (put, copy, list)
  readRequest: 0.0004 / 1000,                        // unit / request (get, head)
  egressInternet: 0.085,                             // unit / GiB
  egressCrossRegion: 0.020,                          // unit / GiB
};
export const PAGE = 1000;                            // a list request returns at most 1,000 keys
export const ROUND_TRIP_MS = 45;                     // round-trip time of a request in the model

export const gib = (bytes) => bytes / 1024 ** 3;
export const totalBytes = FAMILIES.reduce((a, f) => a + f.count * f.avg, 0);
export const totalCount = FAMILIES.reduce((a, f) => a + f.count, 0);
export const listRequests = (keys) => Math.ceil(keys / PAGE);

Differences from a File System, Item by Item

• ST5 — Invoice documents receive 4,300 edits a month; each edit appends 2 KiB to the document.
• ST6 — Range requests are supported: only the requested byte range of an object can be read.

// object/model-difference.mjs — file system operations vs. object storage equivalents, item by item
import { FAMILIES, RATE, PAGE, ROUND_TRIP_MS, gib, totalBytes, totalCount, listRequests } from "./measurement-network.mjs";

const g = (x, n, d = 2) => x.toFixed(d).padStart(n);
const s = (x, n) => String(x).padStart(n);
console.log(`fictional bucket: ${totalCount.toLocaleString("en-US")} objects, ${g(gib(totalBytes), 0, 1)} GiB, ` +
  `${PAGE} keys per page, round-trip time in the model ${ROUND_TRIP_MS} ms`);
console.log("\nfamily             |    count |    GiB | list requests | list time (s)");
console.log("-------------------|----------|--------|---------------|--------------");
for (const f of FAMILIES) {
  const li = listRequests(f.count);
  console.log(`${f.name.padEnd(18)} | ${s(f.count, 8)} | ${g(gib(f.count * f.avg), 6, 1)} | ` +
    `${s(li, 13)} | ${g(li * ROUND_TRIP_MS / 1000, 13, 1)}`);
}

// Four operations: work that is a single call on a file system, translated into object storage.
const invoice = FAMILIES[1], photo = FAMILIES[2], meter = FAMILIES[0];
const EDITS = 4300, APPEND = 2 * 1024, RANGE = 64 * 1024;
const OP = [
  { name: "learn directory size", fsCalls: 1, fsBytes: 0,
    objRequests: listRequests(meter.count), objBytes: 0 },
  { name: "rename directory", fsCalls: 1, fsBytes: 0,
    objRequests: listRequests(meter.count) + 2 * meter.count, objBytes: meter.count * meter.avg },
  { name: "append 2 KiB to a file", fsCalls: 1, fsBytes: APPEND,
    objRequests: 1, objBytes: invoice.avg },
  { name: "monthly edits", fsCalls: EDITS, fsBytes: EDITS * APPEND,
    objRequests: EDITS, objBytes: EDITS * invoice.avg },
  { name: "read a 64 KiB range", fsCalls: 1, fsBytes: RANGE, objRequests: 1, objBytes: RANGE },
  { name: "read the whole photo", fsCalls: 1, fsBytes: photo.avg, objRequests: 1, objBytes: photo.avg },
];
console.log("\noperation                | on a file system    | on object storage        | byte amplification");
console.log("                         |  calls /      bytes |  requests /        bytes | ");
console.log("-------------------------|---------------------|--------------------------|-------------------");
for (const o of OP) {
  const factor = o.fsBytes === 0 ? (o.objBytes === 0 ? "1x" : "unmeasured") : (o.objBytes / o.fsBytes).toFixed(1) + "x";
  console.log(`${o.name.padEnd(24)} | ${s(o.fsCalls, 6)} / ${s(o.fsBytes, 10)} | ` +
    `${s(o.objRequests, 9)} / ${s(o.objBytes, 12)} | ${factor.padStart(18)}`);
}

const rn = OP[1];
const requestFee = rn.objRequests * RATE.writeRequest;
console.log(`\nrename: ${rn.objRequests.toLocaleString("en-US")} requests = ${g(requestFee, 0, 2)} units ` +
  `(fictional), moving ${g(gib(rn.objBytes), 0, 2)} GiB; on the file system 1 call, 0 bytes.`);
const ed = OP[3];
console.log(`monthly edits: ${g(ed.fsBytes / 1024 ** 2, 0, 1)} MiB changed, ` +
  `${g(gib(ed.objBytes), 0, 2)} GiB written; with no partial update the factor is ${(ed.objBytes / ed.fsBytes).toFixed(0)}x.`);
console.log(`range reads work: requesting ${RANGE / 1024} KiB returns ${RANGE / 1024} KiB — ` +
  `partial reads exist, partial writes do not.`);

fictional bucket: 1,678,000 objects, 184.8 GiB, 1000 keys per page, round-trip time in the model 45 ms

family             |    count |    GiB | list requests | list time (s)
-------------------|----------|--------|---------------|--------------
meter-readings     |  1240000 |    2.8 |          1240 |          55.8
invoice-documents  |    86000 |   31.2 |            86 |           3.9
field-photos       |    42000 |  147.7 |            42 |           1.9
work-order-records |   310000 |    3.3 |           310 |          13.9

operation                | on a file system    | on object storage        | byte amplification
                         |  calls /      bytes |  requests /        bytes | 
-------------------------|---------------------|--------------------------|-------------------
learn directory size     |      1 /          0 |      1240 /            0 |                 1x
rename directory         |      1 /          0 |   2481240 /   2976000000 |         unmeasured
append 2 KiB to a file   |      1 /       2048 |         1 /       389120 |             190.0x
monthly edits            |   4300 /    8806400 |      4300 /   1673216000 |             190.0x
read a 64 KiB range      |      1 /      65536 |         1 /        65536 |               1.0x
read the whole photo     |      1 /    3774874 |         1 /      3774874 |               1.0x

rename: 2,481,240 requests = 12.41 units (fictional), moving 2.77 GiB; on the file system 1 call, 0 bytes.
monthly edits: 8.4 MiB changed, 1.56 GiB written; with no partial update the factor is 190x.
range reads work: requesting 64 KiB returns 64 KiB — partial reads exist, partial writes do not.

The table separates the four differences from one another. Learning a directory’s size is a single call on a file system; on object storage it is 1,240 list requests and takes 55.8 seconds in the model — while moving zero bytes. The meter family holds 1.5% of the bucket’s bytes but generates most of the list requests: listing depends not on bytes but on key count. Renaming a directory is a single call and zero bytes on a file system; on object storage it is 2,481,240 requests and moves 2.77 GiB start to finish. There is no equivalent, no multiplier — the operation type has changed.

The partial-update difference is the sneakiest of the four because it throws no error, it only inflates the bill: appending 2 KiB to a 380 KiB document rewrites the whole document, a factor of 190x. At 4,300 edits a month, the changed 8.4 MiB becomes 1.56 GiB written. The last two rows balance the picture: partial reads exist. Requesting a 64 KiB range returns 64 KiB. Object storage allows reading in pieces but not writing in pieces; this asymmetry is the one-sentence reason small, frequently updated records do not belong in object storage.

Key Layout Is an Unrecoverable Item

Because there is no directory, the only thing that determines the access pattern is the key itself. The same data can be kept in two layouts and both are correct; the difference is which query can narrow by prefix.

// object/key-layout.mjs — same data, two key layouts: how many list requests do the queries need
import { METERS, DAYS, FAMILIES, RATE, ROUND_TRIP_MS, gib, listRequests } from "./measurement-network.mjs";

const READING = FAMILIES[0], TOTAL = READING.count;
// Prefix tree: a query narrows to few keys only if it can narrow by prefix; otherwise it scans all.
const LAYOUT = {
  "date-first": { pattern: "meter/<year>/<month>/<day>/<meter>.json", day: METERS, meter: TOTAL },
  "meter-first": { pattern: "meter/<meter>/<year>/<month>/<day>.json", day: TOTAL, meter: DAYS },
};
const QUERY = [
  { name: "all readings of one day", field: "day", monthly: 30 },
  { name: "all history of one meter", field: "meter", monthly: 900 },
];
const g = (x, n, d = 2) => x.toFixed(d).padStart(n);
const s = (x, n) => String(x).padStart(n);

console.log(`${TOTAL.toLocaleString("en-US")} reading objects = ${METERS} meters x ${DAYS} days. ` +
  `Data is the same, only the key's first field changes.`);
for (const [name, d] of Object.entries(LAYOUT)) console.log(`  ${name.padEnd(11)} ${d.pattern}`);

console.log("\nquery                     | layout      | keys read | list requests  | time (s)");
console.log("--------------------------|-------------|-----------|----------------|---------");
const cost = {};
for (const q of QUERY)
  for (const [name, d] of Object.entries(LAYOUT)) {
    const keys = d[q.field], li = listRequests(keys);
    cost[name] = (cost[name] ?? 0) + q.monthly * li;
    console.log(`${q.name.padEnd(25)} | ${name.padEnd(11)} | ${s(keys, 9)} | ` +
      `${s(li, 14)} | ${g(li * ROUND_TRIP_MS / 1000, 7, 1)}`);
  }

console.log("\nmonthly total (30 daily queries + 900 meter queries):");
for (const [name, requests] of Object.entries(cost))
  console.log(`  ${name.padEnd(11)} ${s(requests.toLocaleString("en-US"), 11)} list requests = ` +
    `${(requests * RATE.writeRequest).toFixed(2).padStart(7)} units (fictional)`);
const [a, b] = Object.values(cost);
console.log(`  difference ${(Math.max(a, b) / Math.min(a, b)).toFixed(0)}x; both are correct, both hold the same data.`);

// Changing the layout later: no directory rename — every object is copied, then the old one deleted.
const migration = listRequests(TOTAL) + 2 * TOTAL, bytes = TOTAL * READING.avg;
console.log(`\ncost of changing the layout: ${listRequests(TOTAL)} listings + ${TOTAL.toLocaleString("en-US")} copies + ` +
  `${TOTAL.toLocaleString("en-US")} deletes = ${migration.toLocaleString("en-US")} requests`);
console.log(`  request fee ${(migration * RATE.writeRequest).toFixed(2)} units, moving ${gib(bytes).toFixed(2)} GiB, ` +
  `${(migration * ROUND_TRIP_MS / 3600000).toFixed(1)} hours in a single stream in the model`);
console.log(`  the same bytes moved out of the region: cross-region ${(gib(bytes) * RATE.egressCrossRegion).toFixed(3)} units, ` +
  `to the internet ${(gib(bytes) * RATE.egressInternet).toFixed(3)} units`);
console.log(`  migration fee equals ${(migration * RATE.writeRequest / (gib(bytes) * RATE.storage)).toFixed(0)} months of storage fees.`);

1,240,000 reading objects = 4000 meters x 310 days. Data is the same, only the key's first field changes.
  date-first  meter/<year>/<month>/<day>/<meter>.json
  meter-first meter/<meter>/<year>/<month>/<day>.json

query                     | layout      | keys read | list requests  | time (s)
--------------------------|-------------|-----------|----------------|---------
all readings of one day   | date-first  |      4000 |              4 |     0.2
all readings of one day   | meter-first |   1240000 |           1240 |    55.8
all history of one meter  | date-first  |   1240000 |           1240 |    55.8
all history of one meter  | meter-first |       310 |              1 |     0.0

monthly total (30 daily queries + 900 meter queries):
  date-first    1,116,120 list requests =    5.58 units (fictional)
  meter-first      38,100 list requests =    0.19 units (fictional)
  difference 29x; both are correct, both hold the same data.

cost of changing the layout: 1240 listings + 1,240,000 copies + 1,240,000 deletes = 2,481,240 requests
  request fee 12.41 units, moving 2.77 GiB, 31.0 hours in a single stream in the model
  the same bytes moved out of the region: cross-region 0.055 units, to the internet 0.236 units
  migration fee equals 195 months of storage fees.

Both layouts hold the same bytes, house the same objects, and pay the same storage fee. The difference is in the query: with date first, one day’s readings arrive in 4 requests, and one meter’s whole history is scanned in 1,240; with meter first, it is exactly reversed. This workload has 900 meter queries and 30 daily queries a month, so the wrong layout multiplies the monthly list-request count by 29x.

The real item is in the last block. Changing the layout is not a configuration task: 2,481,240 requests, 2.77 GiB rewritten, and 31 hours on a single stream in the model. The migration’s request fee equals 195 months of the same data’s storage fee — sixteen years of storage money spent on a one-time rename. Key layout is this course’s first unrecoverable item: it is technically reversible, but in practice not without moving the data. The exit side stands apart too: if the migration stays inside the region it incurs no egress fee; moving the same bytes out of the region costs 0.055 units, downloading them to the internet costs 0.236 units. Data is cheap coming in, expensive going out.

The Delegated Decision and What Stays

When object storage is rented, disk placement, replica count, repair of a corrupted block, disk replacement, and the durability target all pass to the provider. These are not unrecoverable decisions, because they are not decisions at all: they have no visibility, no dial to turn.

What stays is four things, and all four are measured in this course’s later lessons: key layout (in this lesson), storage class, versioning, and access scope. What the four have in common is that once chosen, none can change without moving the data. The bucket name is a fifth, and the harshest one: the namespace is shared, the name cannot be changed later, and changing it means opening a new bucket and copying every object — the same 2,481,240 requests as above.

Summary

• Object storage is a dictionary: a bucket is a namespace, a key is a plain string, and a slash only means something in a prefix query. There is no such concept as a directory, an empty directory, or a directory’s size.
• The four differences separate out by number: directory size is 1,240 requests instead of 1 call, rename is 2,481,240 requests instead of 1 call, appending 2 KiB is a 190x rewrite, and range reads are one to one.
• Listing depends on key count, not bytes: the meter family, holding 1.5% of the bucket’s bytes, generates most of the list requests and takes 55.8 seconds for a single scan in the model.
• Key layout fixes the access pattern in place: keeping the same data in two layouts changes the monthly list-request count by 29x, and changing the layout later costs 195 months of storage fees.
• The delegated decisions are the invisible ones (placement, replication, repair); what stays is key layout, storage class, versioning, and access scope — none of the four can be undone without moving the data.

Next Step

Every number in this lesson came out under a single storage fee: every object was billed at the same rate. Yet the four families inside the fictional bucket resemble each other not at all — meter readings are written daily and read daily, invoice documents are written once a month and opened a few times a year, field photos are written once and almost never read. The next lesson carries this difference over to the price side: storage classes exist by access frequency, and as the storage fee drops the access fee rises. Three things will be measured — each class’s break-even access frequency, the cost of keeping rarely accessed data in the frequent class, and the cost of dropping frequently accessed data into the infrequent class.