---
title: 'Object Storage'
source: 'https://academia.sh/en/courses/cloud-storage/object-storage'
course: 'Storage and Data in the Cloud'
language: en
updated: '2026-09-13T05:07:37+00:00'
license: 'CC BY-SA 4.0'
objectives:
    - 'Compares the bucket, object, and key model against a file system operation by operation'
    - 'Translates the differences in directory, rename, partial update, and listing into requests and bytes'
    - 'Shows that the key layout is an unrecoverable item by counting the cost of changing it'
    - 'Separates the delegated decision from the decision that stays, and records the exit item separately'
---

# Object Storage

Separating the bucket, object, and key model from a file system item by item: prefix instead of directory, copy instead of rename, no partial update, and listing as a billed operation; translating every difference into request counts and bytes, and counting the key layout as an unrecoverable item.

The previous course ended by renting the machine and the network: instances launched, settled into
subnets, passed through a filter, and opened outward. One question stayed open — where the **data**
of everything running on those machines lives. The local disk is not the answer: when the instance
shuts down the disk goes with it, and within a scaling group there is no telling which instance holds
which data. In the rented layer, data has a place of its own, and choosing that place hands a set of
decisions to the provider.

This is the course's measure: every lesson counts **which decision is delegated, which one stays**,
which of the ones that stay **cannot be undone**, and what data costs **on the way out**. The first
type is the most common default: object storage. It is true that it is cheap, and that carries no
decision; the decision is carried by the fact that object storage **is not** a file system.

## Bucket, Object, and Key

There are three concepts. A **bucket** is a namespace; an **object** sits inside it; every object is
reached by a **key**. A key looks like a path but is not one: `meter/2026/03/14/A118.json` is a plain
string, and the slashes inside it carry no meaning for the store. The store looks this string up in a
dictionary. Slashes only matter in a **prefix** query: keys that start with a given string can be
listed. There is no such object as a directory, so there is no empty directory either, no directory
delete, and no field to read for a directory's size.

This single difference has four consequences, and all four can be counted: prefix instead of
directory, copy instead of rename, full rewrite instead of partial update, and listing — assumed
free — turning out to be a billed operation. The model below counts these over the fictional
measurement network's object set.

- **ST1** — The regional measurement network is fictional: 4,000 meters, 310 days of history, four
  object families. Counts and average sizes are written into the model.
- **ST2** — Fees are denominated in a fictional **unit**, not a real provider's price;
  storage, request, and egress rates are written out explicitly in the model.
- **ST3** — In the model a list request returns at most 1,000 keys; the page size varies from
  provider to provider. A request's round trip takes 45 ms and requests are sequential on a
  single stream.
- **ST4** — Copying happens server-side: it incurs no egress fee, only a request fee.

```js
// object/measurement-network.mjs — the regional measurement network is FICTIONAL: the object set and the fees are both a model.
// Every fee is denominated in a fictional "unit"; it is not a real provider's price.
export const METERS = 4000, DAYS = 310;             // fictional network: 4,000 meters, 310 days of history
export const FAMILIES = [
  { name: "meter-readings", count: METERS * DAYS, avg: 2400, cadence: "daily" },
  { name: "invoice-documents", count: 86_000, avg: 380 * 1024, cadence: "monthly" },
  { name: "field-photos", count: 42_000, avg: Math.round(3.6 * 1024 * 1024), cadence: "once" },
  { name: "work-order-records", count: 310_000, avg: 11 * 1024, cadence: "daily" },
];
export const RATE = {                               // fictional unit
  storage: 0.023,                                    // unit / GiB / month
  writeRequest: 0.0050 / 1000,                       // unit / request (put, copy, list)
  readRequest: 0.0004 / 1000,                        // unit / request (get, head)
  egressInternet: 0.085,                             // unit / GiB
  egressCrossRegion: 0.020,                          // unit / GiB
};
export const PAGE = 1000;                            // a list request returns at most 1,000 keys
export const ROUND_TRIP_MS = 45;                     // round-trip time of a request in the model

export const gib = (bytes) => bytes / 1024 ** 3;
export const totalBytes = FAMILIES.reduce((a, f) => a + f.count * f.avg, 0);
export const totalCount = FAMILIES.reduce((a, f) => a + f.count, 0);
export const listRequests = (keys) => Math.ceil(keys / PAGE);
```

## Differences from a File System, Item by Item

- **ST5** — Invoice documents receive 4,300 edits a month; each edit appends 2 KiB to the document.
- **ST6** — Range requests are supported: only the requested byte range of an object can be read.

```js
// object/model-difference.mjs — file system operations vs. object storage equivalents, item by item
import { FAMILIES, RATE, PAGE, ROUND_TRIP_MS, gib, totalBytes, totalCount, listRequests } from "./measurement-network.mjs";

const g = (x, n, d = 2) => x.toFixed(d).padStart(n);
const s = (x, n) => String(x).padStart(n);
console.log(`fictional bucket: ${totalCount.toLocaleString("en-US")} objects, ${g(gib(totalBytes), 0, 1)} GiB, ` +
  `${PAGE} keys per page, round-trip time in the model ${ROUND_TRIP_MS} ms`);
console.log("\nfamily             |    count |    GiB | list requests | list time (s)");
console.log("-------------------|----------|--------|---------------|--------------");
for (const f of FAMILIES) {
  const li = listRequests(f.count);
  console.log(`${f.name.padEnd(18)} | ${s(f.count, 8)} | ${g(gib(f.count * f.avg), 6, 1)} | ` +
    `${s(li, 13)} | ${g(li * ROUND_TRIP_MS / 1000, 13, 1)}`);
}

// Four operations: work that is a single call on a file system, translated into object storage.
const invoice = FAMILIES[1], photo = FAMILIES[2], meter = FAMILIES[0];
const EDITS = 4300, APPEND = 2 * 1024, RANGE = 64 * 1024;
const OP = [
  { name: "learn directory size", fsCalls: 1, fsBytes: 0,
    objRequests: listRequests(meter.count), objBytes: 0 },
  { name: "rename directory", fsCalls: 1, fsBytes: 0,
    objRequests: listRequests(meter.count) + 2 * meter.count, objBytes: meter.count * meter.avg },
  { name: "append 2 KiB to a file", fsCalls: 1, fsBytes: APPEND,
    objRequests: 1, objBytes: invoice.avg },
  { name: "monthly edits", fsCalls: EDITS, fsBytes: EDITS * APPEND,
    objRequests: EDITS, objBytes: EDITS * invoice.avg },
  { name: "read a 64 KiB range", fsCalls: 1, fsBytes: RANGE, objRequests: 1, objBytes: RANGE },
  { name: "read the whole photo", fsCalls: 1, fsBytes: photo.avg, objRequests: 1, objBytes: photo.avg },
];
console.log("\noperation                | on a file system    | on object storage        | byte amplification");
console.log("                         |  calls /      bytes |  requests /        bytes | ");
console.log("-------------------------|---------------------|--------------------------|-------------------");
for (const o of OP) {
  const factor = o.fsBytes === 0 ? (o.objBytes === 0 ? "1x" : "unmeasured") : (o.objBytes / o.fsBytes).toFixed(1) + "x";
  console.log(`${o.name.padEnd(24)} | ${s(o.fsCalls, 6)} / ${s(o.fsBytes, 10)} | ` +
    `${s(o.objRequests, 9)} / ${s(o.objBytes, 12)} | ${factor.padStart(18)}`);
}

const rn = OP[1];
const requestFee = rn.objRequests * RATE.writeRequest;
console.log(`\nrename: ${rn.objRequests.toLocaleString("en-US")} requests = ${g(requestFee, 0, 2)} units ` +
  `(fictional), moving ${g(gib(rn.objBytes), 0, 2)} GiB; on the file system 1 call, 0 bytes.`);
const ed = OP[3];
console.log(`monthly edits: ${g(ed.fsBytes / 1024 ** 2, 0, 1)} MiB changed, ` +
  `${g(gib(ed.objBytes), 0, 2)} GiB written; with no partial update the factor is ${(ed.objBytes / ed.fsBytes).toFixed(0)}x.`);
console.log(`range reads work: requesting ${RANGE / 1024} KiB returns ${RANGE / 1024} KiB — ` +
  `partial reads exist, partial writes do not.`);
```

```
fictional bucket: 1,678,000 objects, 184.8 GiB, 1000 keys per page, round-trip time in the model 45 ms

family             |    count |    GiB | list requests | list time (s)
-------------------|----------|--------|---------------|--------------
meter-readings     |  1240000 |    2.8 |          1240 |          55.8
invoice-documents  |    86000 |   31.2 |            86 |           3.9
field-photos       |    42000 |  147.7 |            42 |           1.9
work-order-records |   310000 |    3.3 |           310 |          13.9

operation                | on a file system    | on object storage        | byte amplification
                         |  calls /      bytes |  requests /        bytes | 
-------------------------|---------------------|--------------------------|-------------------
learn directory size     |      1 /          0 |      1240 /            0 |                 1x
rename directory         |      1 /          0 |   2481240 /   2976000000 |         unmeasured
append 2 KiB to a file   |      1 /       2048 |         1 /       389120 |             190.0x
monthly edits            |   4300 /    8806400 |      4300 /   1673216000 |             190.0x
read a 64 KiB range      |      1 /      65536 |         1 /        65536 |               1.0x
read the whole photo     |      1 /    3774874 |         1 /      3774874 |               1.0x

rename: 2,481,240 requests = 12.41 units (fictional), moving 2.77 GiB; on the file system 1 call, 0 bytes.
monthly edits: 8.4 MiB changed, 1.56 GiB written; with no partial update the factor is 190x.
range reads work: requesting 64 KiB returns 64 KiB — partial reads exist, partial writes do not.
```

The table separates the four differences from one another. Learning a directory's size is a single
call on a file system; on object storage it is 1,240 list requests and takes 55.8 seconds in the
model — while moving zero bytes. The meter family holds **1.5% of the bucket's bytes** but generates
most of the list requests: listing depends not on bytes but on **key count**. Renaming a directory is
a single call and zero bytes on a file system; on object storage it is 2,481,240 requests and moves
2.77 GiB start to finish. There is no equivalent, no multiplier — the operation type has changed.

The partial-update difference is the sneakiest of the four because it throws no error, it only
inflates the bill: appending 2 KiB to a 380 KiB document rewrites the whole document, a factor of
190x. At 4,300 edits a month, the changed 8.4 MiB becomes 1.56 GiB written. The last two rows balance
the picture: partial **reads** exist. Requesting a 64 KiB range returns 64 KiB. Object storage allows
reading in pieces but not writing in pieces; this asymmetry is the one-sentence reason small,
frequently updated records do not belong in object storage.

## Key Layout Is an Unrecoverable Item

Because there is no directory, the only thing that determines the access pattern is the key
**itself**. The same data can be kept in two layouts and both are correct; the difference is which
query can narrow by prefix.

```js
// object/key-layout.mjs — same data, two key layouts: how many list requests do the queries need
import { METERS, DAYS, FAMILIES, RATE, ROUND_TRIP_MS, gib, listRequests } from "./measurement-network.mjs";

const READING = FAMILIES[0], TOTAL = READING.count;
// Prefix tree: a query narrows to few keys only if it can narrow by prefix; otherwise it scans all.
const LAYOUT = {
  "date-first": { pattern: "meter/<year>/<month>/<day>/<meter>.json", day: METERS, meter: TOTAL },
  "meter-first": { pattern: "meter/<meter>/<year>/<month>/<day>.json", day: TOTAL, meter: DAYS },
};
const QUERY = [
  { name: "all readings of one day", field: "day", monthly: 30 },
  { name: "all history of one meter", field: "meter", monthly: 900 },
];
const g = (x, n, d = 2) => x.toFixed(d).padStart(n);
const s = (x, n) => String(x).padStart(n);

console.log(`${TOTAL.toLocaleString("en-US")} reading objects = ${METERS} meters x ${DAYS} days. ` +
  `Data is the same, only the key's first field changes.`);
for (const [name, d] of Object.entries(LAYOUT)) console.log(`  ${name.padEnd(11)} ${d.pattern}`);

console.log("\nquery                     | layout      | keys read | list requests  | time (s)");
console.log("--------------------------|-------------|-----------|----------------|---------");
const cost = {};
for (const q of QUERY)
  for (const [name, d] of Object.entries(LAYOUT)) {
    const keys = d[q.field], li = listRequests(keys);
    cost[name] = (cost[name] ?? 0) + q.monthly * li;
    console.log(`${q.name.padEnd(25)} | ${name.padEnd(11)} | ${s(keys, 9)} | ` +
      `${s(li, 14)} | ${g(li * ROUND_TRIP_MS / 1000, 7, 1)}`);
  }

console.log("\nmonthly total (30 daily queries + 900 meter queries):");
for (const [name, requests] of Object.entries(cost))
  console.log(`  ${name.padEnd(11)} ${s(requests.toLocaleString("en-US"), 11)} list requests = ` +
    `${(requests * RATE.writeRequest).toFixed(2).padStart(7)} units (fictional)`);
const [a, b] = Object.values(cost);
console.log(`  difference ${(Math.max(a, b) / Math.min(a, b)).toFixed(0)}x; both are correct, both hold the same data.`);

// Changing the layout later: no directory rename — every object is copied, then the old one deleted.
const migration = listRequests(TOTAL) + 2 * TOTAL, bytes = TOTAL * READING.avg;
console.log(`\ncost of changing the layout: ${listRequests(TOTAL)} listings + ${TOTAL.toLocaleString("en-US")} copies + ` +
  `${TOTAL.toLocaleString("en-US")} deletes = ${migration.toLocaleString("en-US")} requests`);
console.log(`  request fee ${(migration * RATE.writeRequest).toFixed(2)} units, moving ${gib(bytes).toFixed(2)} GiB, ` +
  `${(migration * ROUND_TRIP_MS / 3600000).toFixed(1)} hours in a single stream in the model`);
console.log(`  the same bytes moved out of the region: cross-region ${(gib(bytes) * RATE.egressCrossRegion).toFixed(3)} units, ` +
  `to the internet ${(gib(bytes) * RATE.egressInternet).toFixed(3)} units`);
console.log(`  migration fee equals ${(migration * RATE.writeRequest / (gib(bytes) * RATE.storage)).toFixed(0)} months of storage fees.`);
```

```
1,240,000 reading objects = 4000 meters x 310 days. Data is the same, only the key's first field changes.
  date-first  meter/<year>/<month>/<day>/<meter>.json
  meter-first meter/<meter>/<year>/<month>/<day>.json

query                     | layout      | keys read | list requests  | time (s)
--------------------------|-------------|-----------|----------------|---------
all readings of one day   | date-first  |      4000 |              4 |     0.2
all readings of one day   | meter-first |   1240000 |           1240 |    55.8
all history of one meter  | date-first  |   1240000 |           1240 |    55.8
all history of one meter  | meter-first |       310 |              1 |     0.0

monthly total (30 daily queries + 900 meter queries):
  date-first    1,116,120 list requests =    5.58 units (fictional)
  meter-first      38,100 list requests =    0.19 units (fictional)
  difference 29x; both are correct, both hold the same data.

cost of changing the layout: 1240 listings + 1,240,000 copies + 1,240,000 deletes = 2,481,240 requests
  request fee 12.41 units, moving 2.77 GiB, 31.0 hours in a single stream in the model
  the same bytes moved out of the region: cross-region 0.055 units, to the internet 0.236 units
  migration fee equals 195 months of storage fees.
```

Both layouts hold the same bytes, house the same objects, and pay the same storage fee. The
difference is in the query: with date first, one day's readings arrive in 4 requests, and one
meter's whole history is scanned in 1,240; with meter first, it is exactly reversed. This workload
has 900 meter queries and 30 daily queries a month, so the wrong layout multiplies the monthly
list-request count by 29x.

The real item is in the last block. Changing the layout is not a configuration task: 2,481,240
requests, 2.77 GiB rewritten, and 31 hours on a single stream in the model. The migration's request
fee equals **195 months of the same data's storage fee** — sixteen years of storage money spent on a
one-time rename. **Key layout is this course's first unrecoverable item:** it is technically
reversible, but in practice not without moving the data. The exit side stands apart too: if the
migration stays inside the region it incurs no egress fee; moving the same bytes out of the region
costs 0.055 units, downloading them to the internet costs 0.236 units. Data is cheap coming in, expensive
going out.

## The Delegated Decision and What Stays

When object storage is rented, disk placement, replica count, repair of a corrupted block, disk
replacement, and the durability target all pass to the provider. These are not unrecoverable
decisions, because they are not decisions at all: they have no visibility, no dial to turn.

What stays is four things, and all four are measured in this course's later lessons: **key layout**
(in this lesson), **storage class**, **versioning**, and **access scope**. What the four have in
common is that once chosen, none can change without moving the data. The bucket name is a fifth, and
the harshest one: the namespace is shared, the name cannot be changed later, and changing it means
opening a new bucket and copying every object — the same 2,481,240 requests as above.

## Summary

- Object storage is a dictionary: a bucket is a namespace, a key is a plain string, and a slash only
  means something in a prefix query. There is no such concept as a directory, an empty directory, or
  a directory's size.
- The four differences separate out by number: directory size is 1,240 requests instead of 1 call,
  rename is 2,481,240 requests instead of 1 call, appending 2 KiB is a 190x rewrite, and range reads
  are one to one.
- Listing depends on key count, not bytes: the meter family, holding 1.5% of the bucket's bytes,
  generates most of the list requests and takes 55.8 seconds for a single scan in the model.
- Key layout fixes the access pattern in place: keeping the same data in two layouts changes the
  monthly list-request count by 29x, and changing the layout later costs 195 months of storage fees.
- The delegated decisions are the invisible ones (placement, replication, repair); what stays is key
  layout, storage class, versioning, and access scope — none of the four can be undone without moving
  the data.

## Next Step

Every number in this lesson came out under a single storage fee: every object was billed at the same
rate. Yet the four families inside the fictional bucket resemble each other not at all — meter
readings are written daily and read daily, invoice documents are written once a month and opened a
few times a year, field photos are written once and almost never read. The next lesson carries this
difference over to the price side: storage classes exist by access frequency, and as the storage fee
drops the access fee rises. Three things will be measured — each class's break-even access frequency,
the cost of keeping rarely accessed data in the frequent class, and the cost of dropping frequently
accessed data into the infrequent class.