--- title: 'Adding and Expanding a Disk' source: 'https://academia.sh/en/courses/system-administration/adding-and-expanding-a-disk' course: 'System Administration' language: en updated: '2026-08-17T18:10:05+00:00' license: 'CC BY-SA 4.0' --- # Adding and Expanding a Disk The chain for joining a new device to the system is nine steps, and thirteen of forty-five readings say the work is done when it is not; a persistent entry written to a device name mounts the correct file system in only ten of forty boots. Throughout this topic, storage was always something that already existed: a device, partitions on it, file systems inside them, mount points above them. The course's last lesson returns to the start of the chain and follows a device joining the system from start to finish. What is followed is not the steps themselves but the **gap** between them. At every step a tool says "done," and a layer above is not yet aware of it. The number to be measured is how many readings of the chain give a wrong "the work is done" diagnosis. ## The Chain's Links When a new device is physically attached, the system does not use it by itself. There are nine steps in between, and every step depends on the one before it. The device must first be **recognized**: the kernel scans the bus and adds the device to its tree. Then it is **partitioned**; the partition table gets written to the device, but the kernel can keep holding the old table in memory, and a **reread** is needed. After that, a **file system is built** on the partition. The built file system is not visible in the tree; it needs to be **mounted**. Because a manual mount is lost on restart, a **persistent mount entry** is written. Finally, the unit that will produce the data must **start writing** to the new path; the path changing does not by itself change the unit's configuration. - **ST38** — The chain is nine steps, and the order of steps is fixed in this measurement; the persistent mount entry is written **before** the manual mount, because this is common practice. - **ST39** — The oracle is the state "the new space is persistently in use": the file system is mounted, the entry is persistent, and the unit is writing to the new path. Until all three hold at once, the work is not done. - **ST40** — Five tools are read, and each sees a single link in the chain: `lsblk` the device, `blkid` the file system, `findmnt` and `df` the mount, entry checking the persistence. - **ST41** — In the verification boot, the discovery order is drawn from the shared definition's generator; **40 boots** are observed, and the entry is written in two forms. ```python """Chain for joining a new device to the system: what each tool says at every step.""" SEED = 20260218 BOOTS = 40 FIELDS = ("visible", "partition", "kernel_read", "fs", "mounted", "volume_writes", "persistent") STEP = ( ("0 device attached", 0, 0, 0, 0, 0, 0, 0), ("1 rescanned", 1, 0, 0, 0, 0, 0, 0), ("2 partition created", 1, 1, 0, 0, 0, 0, 0), ("3 kernel read table", 1, 1, 1, 0, 0, 0, 0), ("4 file system built", 1, 1, 1, 1, 0, 0, 0), ("5 persistent entry written", 1, 1, 1, 1, 0, 0, 1), ("6 manually mounted", 1, 1, 1, 1, 1, 0, 1), ("7 unit writes to new path", 1, 1, 1, 1, 1, 1, 1), ("8 restarted", 1, 1, 1, 1, 1, 1, 1), ) TOOL = {"lsblk": "visible", "blkid": "fs", "findmnt": "mounted", "df": "mounted", "entry check": "persistent"} def state(a): d = dict(zip(FIELDS, a[1:])) d["oracle"] = d["mounted"] and d["persistent"] and d["volume_writes"] return d print("step ", " ".join(f"{k:>14s}" for k in TOOL), " oracle") for a in STEP: d = state(a) print(f"{a[0]:28s}", " ".join(f"{str(bool(d[v])):>14s}" for v in TOOL.values()), f" {str(bool(d['oracle'])):>6s}") print() print("tool correct reads wrong reads") total = 0 for tool, field in TOOL.items(): y = sum(bool(state(a)[field]) != bool(state(a)["oracle"]) for a in STEP) total += y print(f" {tool:14s} {len(STEP) - y:11d} {y:12d}") print("wrong diagnosis:", total, "/", len(STEP) * len(TOOL)) print() def generator(seed): d = seed def next_value(n): nonlocal d d = (d * 1103515245 + 12345) % 2147483648 return d % n return next_value def verify(seed, entry_format, device_count=5, count=BOOTS): """Persistent entry written to a DEVICE NAME shifts when discovery order changes.""" r = generator(seed) correct = 0 for _ in range(count): remaining, order = list(range(device_count)), [] while remaining: order.append(remaining.pop(r(len(remaining)))) correct += 1 if entry_format == "id" else order[1] == 1 return correct for seed in (SEED, 20260219): print(f"seed {seed}: {BOOTS} boots |" f" entry written to name correct {verify(seed, 'name'):3d}" f" | entry written to id correct {verify(seed, 'id'):3d}") ``` ``` step lsblk blkid findmnt df entry check oracle 0 device attached False False False False False False 1 rescanned True False False False False False 2 partition created True False False False False False 3 kernel read table True False False False False False 4 file system built True True False False False False 5 persistent entry written True True False False True False 6 manually mounted True True True True True False 7 unit writes to new path True True True True True True 8 restarted True True True True True True tool correct reads wrong reads lsblk 3 6 blkid 6 3 findmnt 8 1 df 8 1 entry check 7 2 wrong diagnosis: 13 / 45 seed 20260218: 40 boots | entry written to name correct 10 | entry written to id correct 40 seed 20260219: 40 boots | entry written to name correct 6 | entry written to id correct 40 ``` Three numbers side by side. **The oracle:** the work finishes at the **seventh step**; in none of the states before that is the new space persistently in use. **The tool's output:** `lsblk` shows the device from the first step onward, and what it shows is correct — just not the answer to the question being asked. **Wrong diagnosis:** **thirteen** of forty-five readings say the work is done. The distribution shows the chain's shape. The lowest-level tool is wrong the most: `lsblk` six times, `blkid` three times. The topmost tools are wrong the least, but not zero — `findmnt` and `df` once each, because at the sixth step the file system is mounted and the unit is still writing to the old path. Entry checking is wrong twice, because the persistent entry is written before mounting and the file is correct on paper. **No tool sees the whole chain**, and each tells the truth about its own link. This distribution's practical consequence is an ordering rule: verification starts not from the bottom of the chain but from the top. First it is asked whether the unit is really writing to the new path, then whether the mount is persistent, and last whether the device is visible; asked in reverse order, the first three answers come back positive and the questioning stops there. The last two lines measure the chain's eighth step. In the verification boot, if the entry is written to the device name, the correct file system mounts in only **ten** of forty boots; in the second seed, this number drops to **six**. With the entry written to the identity, forty of forty are correct. A chain not verified by a restart does not count as complete. ## Irreversible Steps Three of the chain's nine steps are irreversible, and all three gather in the same place: the steps that write to the device. Writing the partition table erases the old layout, building a file system erases everything on the partition, and formatting as swap does the same thing. The common risk of all three is choosing the wrong target, and none of them asks "are you sure." This lesson does not give these commands in a **runnable, complete form**. The safe path is four steps, and all four are measurable. The first is **verifying the target by its identity**: in `lsblk` and `blkid`'s output, it is read that the target device is empty, has no mounted file system on it, and is the expected size. The second is a **dry run**: the option that shows partitioning tools' result without applying it, and the table's text dump, print out beforehand what the change will be. The third is **testing on a separate device**: mounting a file as a loop device makes it possible to run the whole chain away from real data. The fourth is a **snapshot**: if the change is on a logical volume, a snapshot taken beforehand makes recovery possible. The expansion direction follows the same rule too, and it has an added ordering. When a device is grown, the kernel does not see the new size by itself; the logical volume cannot grow before the partition grows, the file system cannot grow before the logical volume grows. The order runs upward, and it reverses on shrinking. What comes out when it is not reversed is not an error message but a silent data loss. ## Summary - Joining a new device to the system is a nine-step chain: recognition, partitioning, rereading the table, building the file system, the persistent entry, mounting, and directing the unit to the new path. - Five tools see five separate links of the chain, and none sees the whole; thirteen of forty-five readings say the work is done when it is not. The lowest-level tool is wrong the most: `lsblk` six, `blkid` three times. - The chain does not count as complete until it is verified by a restart: an entry written to the device name mounts the correct file system in ten of forty boots, in six in the second seed; with the entry written to the identity, forty of forty are correct. - Three of the chain's steps are irreversible, and none of them asks for confirmation; the safe path is verifying the target by its identity, reading the dry-run output, testing on a separate device, and taking a snapshot. - Expansion is done bottom to top, shrinking top to bottom; reversing the order produces not an error message but silent data loss. ## Course Wrap-Up This course asked one question twenty-two times: **how big is the gap between what the output says and the system's reality.** In every lesson, three numbers stood side by side — the oracle, what the tool showed, and how many times the diagnosis drawn from that showing was wrong. The course's rule was this: a command's number is not the row it shows, but how many times the diagnosis drawn from those rows is wrong. The table below collects the measure of the twenty-two lessons in one place. **The rows left blank are not made up**: every row is written only from its own source lesson, and those lessons were not produced in this batch. | Lesson | Oracle | Tool's Output | Wrong Diagnosis | |---|---|---|---| | Listing and Searching Processes | **14** of 24 processes are heavy | interval 1/5/15/30/60 → 14, 14, 11, 9, 9 | **0, 0, 3, 5, 5**; lifetime-average column **7** | | Foreground and Background Jobs | of 13 jobs, `&` keeps **0** alive, detaching keeps **4** alive | `jobs` **13 lines** | `&` **13**, detaching **9**, the path that severs the hangup signal **0** | | Signals | **2** accept the request, **5** resist; with the forcing signal **7/7** | exit code **0** on every attempt | wait 0/10/30 seconds → **7, 6, 5** | | Process Priorities | total work **237,906 → 238,244**, load steady at 3.97 | `NI` column changed **6/6** | **6/6**; **94 percent** of the freed share goes to the other heavy processes | | Resource Limits | **8 failure causes** | **3 messages + 1 silence**, each message maps to two causes | message alone **21**, limit alone **29**, combined **13** | | The Service Manager Model | 6 units, **7** transitive dependencies | flat **0** / direct **4** / transitive **7** / ordered **9** lines | **3 / 2 / 0 / 2** — the reading that prints the most lines does not give the best diagnosis | | Service Status and Control | 6 of 6 units failed, **60** restarts | **4** units read "active," hidden time **600** | **4/6**; polling 1/5/15/30/60 → **480/99/33/18/10** | | Defining a New Service | **60** crashes, 9 of 9 combinations failed | 5 "active," 2 "exited," 2 "failed" | **7/9**; start limit none/20/10/5/3 → hidden time **600/600/106/62/41** | | The Boot Process | `ingest` → **4** affected, **5** total impacted | "**1 unit**"; **4** units never tried | **3/6** | | Bootloaders | **11** of 24 changes work | generator output **24**, command line reading **21**, behavior test **11** | **13 / 10 / 0**; remaining log lines are **0** in both | | Log Architecture | 86 errors in 4000 lines, real source **31** with `process` | six unit files: 28, 0, 31, 0, 27, 0 | **5/6**; three files **completely silent** | | Authentication Logs | **41** attack lines in 601 lines | four filters: 153, 67, 51, 43 lines | **114 / 28 / 12 / 2** | | Log Filtering and Rotation | **86 errors**, the first at the 6.1th second | kept after rotation: 86, 42, **19**, 12, **5** | of six questions **0/3/4/4/4**; ratio 0.2209 and **0.0581** | | System Health Metrics | instantaneous load crosses the threshold for **54 seconds** | the averaged band 6.18–6.25 **never** crosses the threshold | **54**; false alarm **0** | | Tracing Performance Problems | **18 events** (4/4/10) | five snapshots: 432, **1728**, 432, 54, 54 lines | **12 / 12 / 8 / 2 / 0** — the diagnosis **does not change** when lines quadruple | | Block Devices and Partitions | free space in four pieces, largest contiguous 2,881,536 sectors | partition list 3,702,784 free sectors | 6 / 12 decisions | | File Systems | 54 reads touching a corrupted block | 0 errors in a design with no checksum | 54 silent reads | | Inodes and Metadata | write fails at 32,768 files | block usage 0.0125 | 2 / 12 warnings | | Mounting and Persistent Mounts | 717 files really stale | 1504 files look stale | 787 wrong deletions | | Logical Volume Management | writable space 296 blocks | lvs 4392, vgs 8488 blocks | 7 / 21 readings | | Swap Space | memory pressure 84 seconds | used swap sits at 533 | 272 false alarms | | Adding and Expanding a Disk | work finishes at the seventh step | lsblk shows the device from the first step | 13 / 45 readings | The Storage topic's seven lessons showed one pattern over and over. Tools do not lie; each tells the truth about its own layer, and the question asked is not that layer's question. A partition list is not a gap map, a usage percentage is not writability, a logical volume's size is not a file system's size, used swap is not memory pressure. The wrong diagnosis is born in these gaps, and its count does not shrink by adding a column: in the inode measurement, all three separate warning rules gave two wrong readings each in twelve cases, only the type of the error changed. A second pattern concerned time. The snapshot filled at the four-hundred-thirteenth second, and monitoring that looked once every sixty seconds saw a number that had not budged until then; when the swap rate fell between two looks, the missed seconds rose from 2 to 49. Evidence disappears on its own, and in storage this happens through a filled area or a dropped snapshot leaving no trace behind. One who looks late sees little. Throughout the course, destructive commands were not written in complete form, and this was not an omission but part of the measurement. What the commands that write the partition table, build a file system, delete a logical volume, and format swap have in common is that they produce no warning at all when the target is chosen wrongly. The safe path was the same four steps in every lesson: verifying the target by its identity, reading the dry-run output, doing the trial on a separate device, and leaving a way back. The next course, **Linux Network Administration and Troubleshooting**, carries the same question outside a single machine. There too is an interface list, and an address appearing in the list does not mean a connection can be established; there too is a routing table, and a row in the table does not mean a packet reaches its destination. The layered-diagnosis method is this course's storage-chain layering carried over onto the network: a sequence of queries narrowing from the physical link toward the application, and at every layer a tool saying "correct here, not above." And there, evidence disappears even faster, because a packet passes through and leaves, sitting nowhere unless it is captured.