Archive Storage

When access history was scanned in the second lesson, one family stood apart: 97% of field photos were never opened all year. For such data the frequent class is an obvious waste, and the third lesson’s lifecycle rule can send it to archive automatically. What data sent to archive actually becomes has not yet been measured.

The archive tier is a storage class, but it departs from the others at one point: data cannot be read right away. Retrieving an object first requires a retrieval request, the request is processed, and only then does the object become readable. This wait ranges from minutes to hours, and it can be sped up — for a fee. The wait is not the rule for every archive class: some providers’ cheapest class is read in milliseconds and differs only in fee and minimum storage duration; this lesson measures the one with the wait. This is the lesson’s measure: where what was gained in storage gets paid back, and in which item.

• ST31 — The archive tier, retrieval speeds, and fees are fictional; tiers are called by access frequency.
• ST32 — There are three retrieval speeds; each one’s first-response latency, streaming rate, byte fee, and request fee are written into the model.
• ST33 — A retrieval request takes at most 10,000 objects, and four batches are processed at once.

// archive/model.mjs — the archive tier, retrieval speeds, and the audit request are a MODEL. Fees are
// a fictional unit; tiers are called by access frequency, no commercial name is used.
export const CLASS = {                  // storage unit/GiB/month, transition unit/request, minDays: minimum retention
  frequent: { storage: 0.0230, transition: 0.0050 / 1000, minDays: 0 },
  archive: { storage: 0.0010, transition: 0.0500 / 1000, minDays: 180 },
};
// Retrieval speeds: first-response latency, streaming rate, byte and request fee.
export const SPEED = {
  slow: { firstHour: 12, gibPerHour: 250, perGib: 0.0025, request: 0.030 / 1000 },
  medium: { firstHour: 4, gibPerHour: 600, perGib: 0.0100, request: 0.060 / 1000 },
  fast: { firstHour: 5 / 60, gibPerHour: 1500, perGib: 0.0300, request: 0.120 / 1000 },
};
export const BATCH = 10_000, CONCURRENT = 4;        // 10,000 objects a request, 4 batches at once
export const EGRESS = { "in-region": 0, "cross-region": 0.020, internet: 0.085 };
// Fictional audit request: one province's three years of invoice documents and field photos.
export const AUDIT = { invoice: { count: 46_000, kib: 380 }, photo: { count: 80_000, kib: 3686 } };
export const gib = (bytes) => bytes / 1024 ** 3;
export const auditBytes = Object.values(AUDIT).reduce((a, k) => a + k.count * k.kib * 1024, 0);
export const auditCount = Object.values(AUDIT).reduce((a, k) => a + k.count, 0);

Time and Fee, Two Ends of the Same Dial

• ST34 — The fictional audit request covers one province’s three years of invoice documents and field photos; counts and sizes are written into the model.

// archive/retrieval.mjs — retrieval speeds are scanned: time and fee are counted together
import { SPEED, BATCH, CONCURRENT, EGRESS, gib, auditBytes, auditCount, AUDIT } from "./model.mjs";

const g = (x, n, d = 2) => x.toFixed(d).padStart(n);
const s = (x, n) => String(x).padStart(n);
const auditGib = gib(auditBytes), batches = Math.ceil(auditCount / BATCH), rounds = Math.ceil(batches / CONCURRENT);
console.log(`fictional audit request: ${Object.entries(AUDIT).map(([a, k]) => `${k.count} ${a}`).join(" + ")} = ` +
  `${auditCount.toLocaleString("en-US")} objects, ${g(auditGib, 0, 1)} GiB.`);
console.log(`the request is split into ${BATCH}-object batches: ${batches} batches, ${CONCURRENT} at once, ${rounds} rounds.`);
console.log("\nspeed  | first resp | stream GiB/hr | one batch (hr) | total wait (hr)  | retrieval fee");
console.log("-------|------------|---------------|----------------|------------------|--------------");
const FEE = {}, WAIT = {};
for (const [name, sp] of Object.entries(SPEED)) {
  const batchGib = auditGib / batches, roundHours = sp.firstHour + (batchGib * CONCURRENT) / sp.gibPerHour;
  const fee = FEE[name] = auditGib * sp.perGib + auditCount * sp.request;
  WAIT[name] = rounds * roundHours;
  console.log(`${name.padEnd(6)} | ${g(sp.firstHour, 10, 2)} | ${s(sp.gibPerHour, 13)} | ${g(sp.firstHour + batchGib / sp.gibPerHour, 14, 2)} | ` +
    `${g(WAIT[name], 16, 1)} | ${g(fee, 13, 2)}`);
}
console.log(`same request: wait shrinks ${(WAIT.slow / WAIT.fast).toFixed(0)}x, fee ` +
  `grows ${(FEE.fast / FEE.slow).toFixed(1)}x. Time and fee are two ends of the same dial.`);
console.log(`the request fee dominates: ${auditCount.toLocaleString("en-US")} objects at slow speed cost ` +
  `${g(auditCount * SPEED.slow.request, 0, 2)} units, the byte fee for the same bytes is ${g(auditGib * SPEED.slow.perGib, 0, 2)} units.`);
console.log("\nonce retrieval finishes, egress is paid separately (same bytes):");
for (const [where, rate] of Object.entries(EGRESS))
  console.log(`  ${where.padEnd(15)} ${g(auditGib * rate, 7)} units`);
console.log(`downloading to the internet costs ${g(auditGib * EGRESS.internet / FEE.slow, 0, 1)}x the cheapest retrieval: ` +
  `archive's cheapness only lasts until the data leaves.`);

// Decision: the cheapest speed that meets a given response-time commitment. What narrowing the request changes.
const solve = (count, bytes, limitHours) => {
  const G = gib(bytes), p = Math.ceil(count / BATCH), t = Math.ceil(p / CONCURRENT);
  let best = null;
  for (const [name, sp] of Object.entries(SPEED)) {
    const hours = t * (sp.firstHour + (G / p * CONCURRENT) / sp.gibPerHour), fee = G * sp.perGib + count * sp.request;
    if (hours <= limitHours && (best === null || fee < best.fee)) best = { name, hours, fee };
  }
  return best;
};
console.log("\ncheapest speed that meets a response-time commitment:");
console.log("commitment | full request (126,000 objects)    | invoices only (46,000 objects)");
console.log("-----------|-----------------------------------|-------------------------------");
for (const limit of [1, 6, 24, 72]) {
  const a = solve(auditCount, auditBytes, limit);
  const f = solve(AUDIT.invoice.count, AUDIT.invoice.count * AUDIT.invoice.kib * 1024, limit);
  const format = (x) => x ? `${x.name.padEnd(6)} ${g(x.hours, 5, 1)} hr ${g(x.fee, 7)} units` : "infeasible";
  console.log(`${s(limit + " hr", 10)} | ${format(a).padEnd(33)} | ${format(f)}`);
}

fictional audit request: 46000 invoice + 80000 photo = 126,000 objects, 297.9 GiB.
the request is split into 10000-object batches: 13 batches, 4 at once, 4 rounds.

speed  | first resp | stream GiB/hr | one batch (hr) | total wait (hr)  | retrieval fee
-------|------------|---------------|----------------|------------------|--------------
slow   |      12.00 |           250 |          12.09 |             49.5 |          4.52
medium |       4.00 |           600 |           4.04 |             16.6 |         10.54
fast   |       0.08 |          1500 |           0.10 |              0.6 |         24.06
same request: wait shrinks 86x, fee grows 5.3x. Time and fee are two ends of the same dial.
the request fee dominates: 126,000 objects at slow speed cost 3.78 units, the byte fee for the same bytes is 0.74 units.

once retrieval finishes, egress is paid separately (same bytes):
  in-region          0.00 units
  cross-region       5.96 units
  internet          25.32 units
downloading to the internet costs 5.6x the cheapest retrieval: archive's cheapness only lasts until the data leaves.

cheapest speed that meets a response-time commitment:
commitment | full request (126,000 objects)    | invoices only (46,000 objects)
-----------|-----------------------------------|-------------------------------
      1 hr | fast     0.6 hr   24.06 units     | fast     0.2 hr    6.02 units
      6 hr | fast     0.6 hr   24.06 units     | fast     0.2 hr    6.02 units
     24 hr | medium  16.6 hr   10.54 units     | medium   8.0 hr    2.93 units
     72 hr | slow    49.5 hr    4.52 units     | slow    24.1 hr    1.42 units

The audit request is 126,000 objects and 297.9 GiB. None of the three speeds is wrong; their difference collects into two columns. At slow speed the total wait is 49.5 hours — more than two days — and the retrieval fee is 4.52 units. At fast speed the wait drops to 0.6 hours, the fee rises to 24.06 units. The wait shrinks 86 times, the fee grows 5.3 times. In an audit request the real decision is not which speed is “good,” it is how many hours of waiting is acceptable; without that answer, no speed can be chosen.

The reason the total wait exceeds the first-response latency is the batch calculation. The request is split into 10,000-object batches: 126,000 objects makes 13 batches, four batches are processed at once, so four rounds are needed. At slow speed a single batch takes 12.09 hours, but the total is 49.5 hours. First-response latency is paid again on every round, and as object count grows the wait grows not with bytes but with rounds.

The line below shows the price side of the same structure: 126,000 objects’ request fee at slow speed is 3.78 units, while the same bytes’ byte fee is 0.74 units. Four-fifths of the retrieval bill comes from object count. This is archive’s counterpart to the rule from the first and second lessons — small, numerous objects cost through the request fee at every tier.

Egress is a separate item. Retrieval makes data readable; it does not move it anywhere. The same 297.9 GiB incurs no egress fee if it stays in the region, costs 5.96 units across regions, and 25.32 units downloaded to the internet. Downloading to the internet is 5.6 times the cheapest retrieval: archive’s cheapness lasts only until the data leaves.

The last table ties the decision to a response-time commitment and reduces the choice to a single line. At a one-hour commitment only the fast speed suffices, costing 24.06 units; at a twenty-four-hour commitment the medium speed drops to 10.54 units; at a seventy-two-hour commitment the slow speed suffices at 4.52 units. The right column opens a second lever: when the audit request is narrowed to only invoice documents, both time and fee drop at every commitment — 2.93 units instead of 10.54 units at the twenty-four-hour commitment. Narrowing the request is cheaper than raising the speed; the first thing to measure in archive is what actually needs to be retrieved.

Minimum Retention Period

• ST35 — The archive tier’s minimum retention period is 180 days, and it starts the moment the object enters the class.
• ST36 — The lifecycle rule transitions the object to archive on day 30 and deletes it when its lifetime ends.

// archive/minimum.mjs — minimum retention: an object deleted early is still billed for the full period
import { CLASS, gib } from "./model.mjs";

const DAYS_PER_MONTH = 30, archive = CLASS.archive, frequent = CLASS.frequent;
const minMonths = archive.minDays / DAYS_PER_MONTH;
// Two families, two lifetimes: a small, short-lived document and a large, long-lived photo.
const FAMILIES = [
  { name: "invoice-documents", count: 86_000, kib: 380, lifeMonths: 3 },
  { name: "field-photos", count: 42_000, kib: 3686, lifeMonths: 60 },
];
const TRANSITION_MONTH = 1;                          // the rule moves to archive on day 30
const g = (x, n, d = 2) => x.toFixed(d).padStart(n);
const s = (x, n) => String(x).padStart(n);
console.log(`the archive tier's minimum retention period is ${archive.minDays} days = ${minMonths} months. ` +
  `The rule transitions to archive in month ${TRANSITION_MONTH}, deletes when the lifetime ends.`);
console.log("\nfamily            |  life | in archive | billed | transition fee  | no archive | archive | ratio");
console.log("------------------|-------|------------|--------|-----------------|------------|---------|------");
for (const f of FAMILIES) {
  const dataGib = gib(f.count * f.kib * 1024), inArchive = f.lifeMonths - TRANSITION_MONTH;
  const billed = Math.max(inArchive, minMonths);
  const transitionFee = f.count * archive.transition;
  const withoutArchive = dataGib * frequent.storage * f.lifeMonths;
  const withArchive = dataGib * frequent.storage * TRANSITION_MONTH + transitionFee + dataGib * archive.storage * billed;
  console.log(`${f.name.padEnd(17)} | ${s(f.lifeMonths + " mo", 5)} | ${s(inArchive + " mo", 10)} | ${s(billed + " mo", 6)} | ` +
    `${g(transitionFee, 15)} | ${g(withoutArchive, 10)} | ${g(withArchive, 7)} | ${g(withArchive / withoutArchive, 5, 2)}`);
}

// Early deletion: even if the object is deleted on day 45, the 180-day debt stands.
const photo = FAMILIES[1], dataGib = gib(photo.count * photo.kib * 1024), DELETE_DAY = 45;
const used = DELETE_DAY / DAYS_PER_MONTH, owed = minMonths - used;
console.log(`\nearly deletion: ${photo.count.toLocaleString("en-US")} photos moved to archive, deleted on day ${DELETE_DAY}.`);
console.log(`  used ${g(used, 0, 1)} months, billed ${minMonths} months, paid for ${g(owed, 0, 1)} idle months.`);
console.log(`  the bill for deleting on day ${DELETE_DAY} is ${g(dataGib * archive.storage * minMonths, 0, 3)} units; ` +
  `never deleting and holding it for ${minMonths} months bills the same ${g(dataGib * archive.storage * minMonths, 0, 3)} units.`);
console.log(`  the ${g(owed, 0, 1)} months in between, ${g(dataGib * archive.storage * owed, 0, 3)} units, are paid for objects ` +
  `that no longer exist: early deletion gains nothing.`);
console.log(`archive is not a storage class for data that will live under ${minMonths} months, it is a debt paid up front.`);

the archive tier's minimum retention period is 180 days = 6 months. The rule transitions to archive in month 1, deletes when the lifetime ends.

family            |  life | in archive | billed | transition fee  | no archive | archive | ratio
------------------|-------|------------|--------|-----------------|------------|---------|------
invoice-documents |  3 mo |       2 mo |   6 mo |            4.30 |       2.15 |    5.20 |  2.42
field-photos      | 60 mo |      59 mo |  59 mo |            2.10 |     203.74 |   14.21 |  0.07

early deletion: 42,000 photos moved to archive, deleted on day 45.
  used 1.5 months, billed 6 months, paid for 4.5 idle months.
  the bill for deleting on day 45 is 0.886 units; never deleting and holding it for 6 months bills the same 0.886 units.
  the 4.5 months in between, 0.664 units, are paid for objects that no longer exist: early deletion gains nothing.
archive is not a storage class for data that will live under 6 months, it is a debt paid up front.

The table applies a single rule to two families and gets opposite results. Field photos live sixty months; the fifty-nine months spent in archive are far above the minimum retention period, so the billed period is also fifty-nine months. The family that costs 203.74 units without archive drops to 14.21 units with it — a ratio of 0.07, a fourteenfold gain. Invoice documents live three months; they spend two months in archive but are billed for six months. The family that costs 2.15 units without archive rises to 5.20 units with it: a 2.42x loss. Same rule, same tier, opposite result. Two things drive the difference — the object’s lifetime and the object’s size. The transition fee for 86,000 small documents alone is 4.30 units, twice the archive-free total.

The early-deletion block names the item. Forty-two thousand photos are placed in archive and deleted on day forty-five: 1.5 months used, 6 months billed. The bill for deleting is 0.886 units, and the bill for never deleting and holding it six months is also 0.886 units — identical. The 0.664 units paid for the 4.5 months in between are for objects that no longer exist. The minimum retention period is this lesson’s unrecoverable item: the debt is born the moment the object enters the class, and neither deletion nor a class change closes it.

The delegated-decision side is at its widest here. In the archive tier, durability, placement, storage medium, repair, and keeping the data readable all belong entirely to the provider; none of it is visible, none of it has a dial. What stays comes down to three numbers: which object enters archive and when, how long it lives, and at what speed it gets retrieved when needed. All three are decided before the data is written, because all three are paid for afterward.

Summary

• Storage in the archive tier is cheap, but data cannot be read right away: the same audit request waits 49.5 hours and costs 4.52 units at slow speed, waits 0.6 hours and costs 24.06 units at fast speed.
• The wait grows with rounds, not bytes: 126,000 objects split into 10,000-object batches make four rounds, and first-response latency is paid again on every round.
• Four-fifths of the retrieval bill comes from object count: 126,000 objects’ request fee is 3.78 units, the same bytes’ byte fee is 0.74 units.
• Retrieval makes data readable, it does not move it: the same 297.9 GiB, downloaded to the internet, adds a 25.32-unit egress fee on top of retrieval, 5.6 times the cheapest retrieval.
• The minimum retention period is an unrecoverable item: archive gains fourteen times for photos that live sixty months, and costs 2.42 times more for documents that live three months; an object deleted on day forty-five still pays for six months.

Next Step

Six lessons measured where data goes: object, block, shared file, and archive. None of them asked a single question — who can read this data. In the fictional measurement network, field teams open photos from their phones, auditors request invoice documents from outside, and neither group has bucket access. Granting an identity opens a permanent door; what is usually wanted, though, is a time-limited and narrow door. The next lesson measures this: a presigned URL is generated, and its expiry and scope are actually verified. Three things will be measured — the regeneration load that grows as the link’s lifetime shortens, the leak window that grows as it lengthens, and the object count a single link opens as scope widens.