Mounting and Persistent Mounts

The previous lesson separated a file system’s two limits and built all of its measurements on a single assumption: that the directory being looked at really is that file system’s directory. This assumption is not free. A file system is mounted at a point in the tree, and that point is a writable directory both before and after mounting.

This lesson measures mounting itself and automatic mounting at boot. Two numbers will come out: in how many boots a persistent mount entry can mount the wrong file system, and how many files a single mount option can make a maintenance job wrongly delete. Neither produces an error message.

Mounting Is an Attach Operation

In this system, every file system sits not under a separate letter but on a branch of a single tree. Mounting is attaching the file system on a block device to a point in this tree; that point is called the mount point, and a mount point is always a directory that already exists.

Mounting adds an entry to a table in the kernel; nothing changes on the device. The same device can be mounted at multiple points, one point can be mounted twice in a row, and unmounting does not affect the data. What is mounted is read in tree form with findmnt, in flat-list form with mount. The transcript below is a sample transcript showing the shape of findmnt’s output; it was not run, and no numeric claim in this lesson comes from it.

TARGET   SOURCE    FSTYPE  OPTIONS
/        /dev/sdb1 ext4    rw,relatime
├─/data  /dev/sdb2 xfs     rw,noatime,nodev
└─/log
         /dev/sdb3 ext4    rw,relatime,noexec,nosuid

The mount point’s content before mounting is the lesson’s first trap. The /data directory does not have to be empty; if it has files in it, the moment the device is mounted there, those files become invisible. They are not deleted, not moved, and keep taking up space on the underlying file system; only their names become unreachable. The most common form of this is: a unit starts running before the device is mounted and writes its output to the mount point; the device is mounted later, and that data stays buried inside the root file system. du cannot see it when it walks the mount point, because it is no longer in the tree; df shows unaccounted-for usage on the root file system. This gap between the two outputs is the twin of the previous lesson’s file with a deleted name, and it is found the same way — temporarily unmounting and looking at the same directory again is enough.

Persistent Mounts and Device Names

A manual mount is lost on restart. Automatic mounting at boot is defined by a persistent mount entry that binds the device to a mount point, a file system type, and an option list. The entry’s first field names the device, and how this field is written is what this section measures.

Device names are assigned in discovery order. If one controller responds before another, the names swap; when a device is added or fails, the whole order shifts. A device’s identity, on the other hand, is written when the file system is built and does not change no matter where the device is attached.

• ST20 — The fictional server has five block devices; three carry the same type of data file system, one carries swap space, one carries a file system of a different type.
• ST21 — The discovery order can change on every boot; the order is drawn from the shared definition’s generator, and names are assigned according to this order. 60 boots are observed.
• ST22 — The /data entry is written to the name device-b, and its type field is the data file system. It is counted as correct when the entry mounts the right identity, silently wrong when it mounts a different device without incident because the type matches, noisily wrong when it fails because the type does not match.

"""At boot, device name and device identity are not the same thing."""
SEED = 20260218
BOOTS = 60           # number of boots observed
DEVICES = ({"id": "k1", "type": "ext4", "content": "root"},
           {"id": "k2", "type": "ext4", "content": "data"},
           {"id": "k3", "type": "ext4", "content": "log"},
           {"id": "k4", "type": "swap", "content": "swap"},
           {"id": "k5", "type": "xfs",  "content": "backup"})
NAMES = ("device-a", "device-b", "device-c", "device-d", "device-e")
RECORD = {"name": "device-b", "id": "k2", "type": "ext4"}   # the /data entry


def generator(seed):
    d = seed

    def next_value(n):
        nonlocal d
        d = (d * 1103515245 + 12345) % 2147483648
        return d % n
    return next_value


def discovery_order(r):
    """Devices are named in discovery order; the order can change."""
    remaining, order = list(DEVICES), []
    while remaining:
        order.append(remaining.pop(r(len(remaining))))
    return order


def boot(seed=SEED, count=BOOTS):
    r = generator(seed)
    by_name = {"correct": 0, "silent_wrong": 0, "noisy_error": 0}
    by_id = {"correct": 0, "silent_wrong": 0, "noisy_error": 0}
    for _ in range(count):
        selected = dict(zip(NAMES, discovery_order(r)))[RECORD["name"]]
        if selected["id"] == RECORD["id"]:
            by_name["correct"] += 1
        elif selected["type"] == RECORD["type"]:
            by_name["silent_wrong"] += 1
        else:
            by_name["noisy_error"] += 1
        by_id["correct"] += 1
    return by_name, by_id


A, K = boot()
print("entry form     correct  silent wrong  noisy error  wrong diagnosis")
for name, s in (("device name", A), ("device id", K)):
    print(f"  {name:12s} {s['correct']:6d}  {s['silent_wrong']:13d}"
          f"  {s['noisy_error']:14d}  {s['silent_wrong']:11d}")
print("boots observed:", BOOTS)
print()
for seed in (SEED, 20260219):
    a, _ = boot(seed)
    print(f"seed {seed}: correct {a['correct']:3d}  silent wrong"
          f" {a['silent_wrong']:3d}  noisy error {a['noisy_error']:3d}")

entry form     correct  silent wrong  noisy error  wrong diagnosis
  device name      13             27              20           27
  device id        60              0               0            0
boots observed: 60

seed 20260218: correct  13  silent wrong  27  noisy error  20
seed 20260219: correct  10  silent wrong  27  noisy error  23

Three numbers side by side. The oracle: the file system that should be mounted at /data is the same identity on every boot. The tool’s output: the mount table shows a source for /data on every boot, and the name it shows is device-b every time; anyone looking at the table sees no difference. Wrong diagnosis: in twenty-seven of sixty boots, the wrong file system mounts without error.

The twenty failing boots are actually good news: because the type does not match, the mount fails, the failure is seen, and it gets fixed. What is dangerous is the twenty-seven silent cases — the unit runs, data gets written, no log line is produced, and the written data does not sit where it is expected. In an entry written to the identity, this number is zero; in the second seed, the correct-boot count comes out to 10 instead of 13, silent wrong is again 27. What writing to the identity buys is not performance, but determinism.

Writing the identity has a cost too, and it is not hidden: the identity is unreadable, writing it by hand is error-prone, and it changes when the device is reformatted. This is why entries are mostly produced not by hand but by reading them off the device. A middle path is the label given when the file system is built; a label is a readable name and, unlike a device name, does not depend on discovery order, but if two devices are given the same label, determinism is lost again.

The entry’s last fields affect the diagnosis too. When an entry fails, boot can halt, and the machine comes up only into a rescue shell; the option that prevents this makes the failure invisible, and the system keeps running with a file system missing. The choice between the two is a preference: halting noisily, or running silently incomplete.

The Option That Silently Changes Meaning

The option list is the least-read field of a mount. Mounting a partition read-only or closing it to running programs produces noisy results: writes are rejected, execution is rejected, an error message appears. The option that does not update access time, on the other hand, rejects nothing; it only freezes a recorded number.

• ST23 — The /data partition has 2000 files; the creation, last-write, and last-read moments are drawn from the generator within a 600-second window.
• ST24 — The recorded access time depends on the option: with full updating, the real read moment is written; with relaxed updating, only the write moment; with the no-update option, the creation moment.
• ST25 — The cleanup job deletes files whose recorded access time is older than 300 seconds. The oracle is the file’s real last-read moment.

"""A mount option changes recorded access time; a cleanup job deletes wrongly."""
SEED = 20260218
PERIOD = 600
THRESHOLD = 300
FILES = 2000


def generator(seed):
    d = seed

    def next_value(n):
        nonlocal d
        d = (d * 1103515245 + 12345) % 2147483648
        return d % n
    return next_value


def files(seed=SEED, count=FILES):
    """Oracle: every file's REAL last-read moment."""
    r = generator(seed)
    records = []
    for _ in range(count):
        creation = r(400)
        last_write = creation + (r(PERIOD - creation) if r(10) < 3 else 0)
        base = max(creation, last_write)
        last_read = base + (r(PERIOD - base) if r(10) < 6 else 0)
        records.append({"creation": creation, "last_write": last_write,
                         "last_read": last_read})
    return records


def recorded(d, option):
    if option == "atime":
        return d["last_read"]
    if option == "relatime":
        return d["last_write"]
    return d["creation"]          # noatime: never updated


def cleanup(records, option, threshold=THRESHOLD):
    """The job deletes files whose recorded access time is older than the threshold."""
    wrong_delete = wrong_keep = deleted = 0
    for d in records:
        delete = recorded(d, option) < threshold
        stale = d["last_read"] < threshold           # oracle
        deleted += delete
        wrong_delete += delete and not stale
        wrong_keep += stale and not delete
    return {"deleted": deleted, "wrong_delete": wrong_delete,
            "wrong_keep": wrong_keep,
            "wrong": wrong_delete + wrong_keep}


L = files()
print("oracle: files really stale:", sum(d["last_read"] < THRESHOLD for d in L),
      "/", FILES)
print()
print("option     deleted  wrong delete  wrong keep  wrong diagnosis")
for o in ("atime", "relatime", "noatime"):
    t = cleanup(L, o)
    print(f"  {o:9s} {t['deleted']:7d}  {t['wrong_delete']:12d}"
          f"  {t['wrong_keep']:13d}  {t['wrong']:11d}")
print()
for seed in (SEED, 20260219):
    l2 = files(seed)
    print(f"seed {seed}: stale {sum(d['last_read'] < THRESHOLD for d in l2):4d}"
          f"  atime {cleanup(l2, 'atime')['wrong']:4d}"
          f"  relatime {cleanup(l2, 'relatime')['wrong']:4d}"
          f"  noatime {cleanup(l2, 'noatime')['wrong']:4d}")

oracle: files really stale: 717 / 2000

option     deleted  wrong delete  wrong keep  wrong diagnosis
  atime         717             0              0            0
  relatime     1204           487              0          487
  noatime      1504           787              0          787

seed 20260218: stale  717  atime    0  relatime  487  noatime  787
seed 20260219: stale  728  atime    0  relatime  502  noatime  821

Three numbers side by side. The oracle: 717 of the 2000 files really have not been read for three hundred seconds. The tool’s output: with the no-update option, when the files’ access time is read, 1504 of them look stale. Wrong diagnosis: the cleanup job wrongly deletes 787 files and produces not a single error message.

The difference between the three rows is a design trade-off. Updating access time on every read turns a read-only workload into a write workload; every file read produces a metadata write. Relaxed updating lowers this cost and brings 487 wrong deletions; the no-update option zeroes the cost and brings 787 wrong deletions. The wrong-keep column being zero on all three rows shows the error is one-directional: the recorded time lags behind reality, it never runs ahead of it. In the second seed, the numbers come out to 502 and 821; the ordering does not change.

The rule that follows is this: a mount option changes the assumption of the jobs running on top of it. The option list appears in findmnt’s output, and if whoever looks at that list does not know which job relies on which field, they cannot draw any diagnosis from the row they see.

Switching to Read-Only on Error

Some mount options set not operations but failure behavior. A file system learns at mount time what to do when it detects inconsistency in its own ledgers or an error from the device: it can ignore it and continue, switch itself to read-only, or halt the machine. The common default behavior is the middle one.

Switching to read-only has an odd side from the operator’s point of view. The device is still mounted, still visible in the tree, reads work, and df keeps showing usage; only writes are rejected. Units that write take this rejection as an error, and the restart policy covers this error over — the condition measured in the Services topic holds here too and is not repeated. The result is a system that looks like it is running but writes nothing.

The record of this transition sits in exactly one place: a single line dropped into the system log. The line is written at the moment of transition and never written again; it disappears once the log is rotated. This is the most expensive example of the third claim from the Logs topic — the only evidence of the failure is kept in an arrangement that erases evidence by itself. In the mount table’s option list, the ro marker appears, and that marker does not say the file system transitioned into this state afterward; it would look the same if the entry had been written read-only from the start.

A Wrong Entry and the Way Back

The file holding persistent mount entries is one of the least-written and most damaging files on the system. A wrongly written line can halt boot; if the system is remote and there is no console access, the machine becomes unreachable. This is why the process of writing an entry is not done alone, but through a three-step procedure.

The first is syntax and target verification: findmnt --verify reads the entries, checks whether the device identities have a match, and checks whether the mount points exist; it mounts nothing. The second is the trial: the entries are reread, whatever is not yet mounted gets mounted, and the result is seen in the same session. The third is the way back: a copy of the file is taken before the change is made, and the restart is attempted at a time when rescue-shell access is possible.

The order entries are read in is a trap too. Mount points can be nested, and an inner point cannot be mounted before the outer one is; when the order breaks, the inner file system mounts onto a directory on the root file system and does not sit where it is expected. What resolves the dependency is not the entries’ order in the file but the mount points’ path depth; tools do this ordering themselves, but a manually issued mount command does not.

Unmounting is not as harmless as it seems either. A file system cannot be detached while a process has an open handle on it, and forcing the detach with the forcing option leaves whatever those processes were writing half-done. This lesson does not give the forcing detach command in complete form; the correct order is stopping the units that use the file system first, then verifying no open handle remains, and detaching last.

Summary

• Mounting attaches the file system on a device to a point in the single tree; the mount point’s previous content is not deleted, it becomes invisible and keeps taking up space on the underlying file system.
• Device names are assigned in discovery order: a persistent mount entry written to the name device-b mounts the correct file system in only 13 of sixty boots, silently mounts the wrong one in 27, and produces a noisy error in 20. Written to the identity, the wrong diagnosis is zero.
• The mount option that does not update access time rejects no operation; it only freezes the recorded time. While 717 files are really stale, the job deletes 1504 files, and 787 of these are wrong deletions.
• The error is one-directional: the recorded access time lags behind reality, it never runs ahead of it; this is why the wrong-keep column is zero across all three options.
• When a persistent mount entry is written wrongly, boot can halt; the verification command, trying it in the same session, and a copy of the file measurably lower this risk.

Next Step

All of this lesson’s measurements assumed a partition’s boundaries are fixed: a range drawn on the device and a file system built inside it. When it fills, the only thing to be done was deleting. The next lesson adds a layer that removes this fixedness — a layer that turns the partition into a pool made of volumes, can grow a volume while it runs, and can hold a copy of a moment. That layer has its own limit too, and what will be measured is the moment a snapshot fills.