--- title: 'The Service Manager Model' source: 'https://academia.sh/en/courses/system-administration/service-manager-model' course: 'System Administration' language: en updated: '2026-08-17T18:10:05+00:00' license: 'CC BY-SA 4.0' --- # The Service Manager Model The unit, target, and dependency concepts are established; the diagnosis given by the unit list is compared against the diagnosis given by the dependency graph at four levels of detail, and wrong diagnoses are counted. The process management topic looked at individual processes: which one is really heavy, which one responds to a termination request, which one hits an imposed resource limit. Most of the processes on a server, though, are not started by an operator. The machine boots, dozens of programs come up with no one watching, one crashes and another takes its place, and by morning everything looks in order. What does this is a **service manager**, and what it manages is not the process itself but the process's **definition**. This lesson builds that definition and the graph between definitions. Its question is: how much of the system's real structure does the manager's printed list show? The list prints six lines, and all six lines are correct. What is measured is how many times the diagnosis drawn from those six correct lines is wrong. ## Unit, Target, and Two Kinds of Edge The object the service manager manages is called a **unit** (service unit). A unit is a text file: it says which program runs as which user, what happens if it fails, and which other units it needs. The manager reads these files, turns the relationship between them into a graph, and starts processes by following the graph. This distinction carries the rest of the course: **the unit file sits on disk, the process runs in memory.** Changing the file does not change the running process, killing the process does not delete the file, and a unit can be "loaded" without ever having run. There are unit types besides services, and all of them sit in the same graph: a unit representing a mount point, a unit that listens on a network socket and wakes a service on an incoming connection, a unit carrying a scheduled task, a unit that watches a file path. This topic measures only service units; the other types are named and passed over. When units need to be referred to as a group rather than one by one, a **target** is used. A target does no work itself and starts no process; it is a name representing the units grouped under it. Stages like "network ready," "multi-user system ready," and "shutdown" are named with targets. Moving to a target means enabling the units under it. That a target does no work raises the question of how much information an "arrived at target" notification carries; that question is measured in the boot process lesson. Edges in the graph are of two kinds, and confusing them is the error this lesson measures. A **requires edge** says one unit is useless without another: if the required unit stops, the requiring unit stops too. An **ordering edge** only says timing: this unit should start after that one. An ordering edge does not create a dependency. Two units can have ordering without requiring, or requiring without ordering. Command names and unit-file syntax vary by service manager and distribution; the model here describes common behavior. Requiring also has a weak form, called **wanting**: the wanted unit is attempted, and if it cannot be started, the wanting unit keeps running anyway. The weak edge is commonly preferred because it stops crash propagation; in exchange it silences the breakage of an edge in the graph. The mock server's graph carries no weak edge, because the confusion this lesson measures sits between ordering and requiring. Printing all three edge types in the same tree, in the same form, is the subject of the next section. Process isolation from one another, how the kernel side of resource accounting is kept, and control-group and namespace mechanisms are **not covered in this course**; they are left to the M03/K05 Kernel Interfaces and Isolation course. Here, a unit is a record in the manager's bookkeeping ledger. The manager's need for a graph is not only bookkeeping. Without a graph, units would have to be started one after another, each waiting on the last; with a graph, units with no edge between them can start at the same time. This distinction determines boot time and is counted layer by layer in lesson `04`. ## The Mock Server's Unit Graph The same mock server is used throughout the course. It has six units: `ingest` receives data from outside, `queue` buffers the data it receives, `process` reads from the queue, `report` produces output from the processed data, `metrics` collects the ingestion pipeline's counters, and `backup` runs without depending on anyone. **SV1 — a unit is a definition**, not the running process itself. **SV2 — the requires edge is transitive:** if `report` requires `process` and `process` requires `queue`, then `report` is also indirectly tied to `queue`. **SV3 — the ordering edge is not counted as transitive** and creates no dependency. **SV4 — `report` starts only after `backup`** but does not require it; **SV5 — `metrics` starts only after `queue`** but does not require it. **SV6 — the graph is acyclic.** **SV7 — stopping a unit stops every unit that requires it.** **SV8 — the graph carries no weak wanting edge;** only requiring and ordering exist. **SV9 — every unit belongs to the same manager** and sits on a single machine; no edge reaches out to a remote node. **SV10 — the graph does not change during the measurement;** no unit file is added, removed, or reread. The last of these assumptions does not always hold on a real system, and what happens when it does not is covered separately later. The rest of the measurement rests on a fixed graph: the oracle **is** this graph. The manager's unit list does not show the graph; it prints units in alphabetical order, one per line. The dump below has not been run on this machine; it is written as an example to show the format: ```text UNIT LOAD ACTIVE SUB DESCRIPTION backup.service loaded active running Backup job ingest.service loaded active running Data ingestion unit metrics.service loaded active running Metrics collector process.service loaded active running Processing unit queue.service loaded active running Queue unit report.service loaded active running Report generator ``` Six lines, six units, zero edges. The only information available to an operator looking at this list is which units exist. The answer to "what happens if I stop this unit" is not in the list and cannot be extracted from it. ## The Command That Prints the Graph, and Which Way It Prints Service managers offer a command that prints dependencies as a tree. This command has two directions, and they answer different questions. The default direction is **downward**: it shows what a unit needs in order to run. The reverse direction is **upward**: it shows who needs that unit. The answer to an operator's question "what breaks if I stop this" lies only in the reverse direction, while the parameterless call prints the downward direction. The two dumps below have not been run; they are written to show what each direction of the command prints: ```text $ systemctl list-dependencies report.service report.service * +-process.service * +-queue.service * +-ingest.service $ systemctl list-dependencies --reverse ingest.service ingest.service * +-queue.service * | +-process.service * | +-report.service * +-metrics.service ``` The second dump lists four units, and that is the correct count. But the tree prints the requires edge, the ordering edge, and the weak wanting edge **in the same form**; which line comes from which kind of edge cannot be read from the tree itself. Whether a unit shown in the tree will really stop when stopped, or only start later, requires looking at the unit files. What the next section measures is exactly the cost of this ambiguity. ## Four Readings, Four Diagnoses The same graph can be read at four levels of detail. The **flat list** gives only the units. The **direct** reading counts, for each unit, the units that directly require it. The **transitive graph** follows the chain to its end. The **with ordering** reading counts ordering edges as requiring too. The oracle is the transitive result computed over requires edges; the metric is how many units each reading diverges from that result on. ```python UNIT = { "ingest": {"requires": [], "restart": "always"}, "queue": {"requires": ["ingest"], "restart": "always"}, "process": {"requires": ["queue"], "restart": "on-failure"}, "report": {"requires": ["process"], "restart": "no"}, "metrics": {"requires": ["ingest"], "restart": "always"}, "backup": {"requires": [], "restart": "no"}, } AFTER = {"report": ["backup"], "metrics": ["queue"]} # ordering only def dependents_of(unit, units=None): units = UNIT if units is None else units direct = [a for a, v in units.items() if unit in v["requires"]] all_ = set(direct) for d in direct: all_ |= dependents_of(d, units) return all_ def with_ordering(unit): """Reading that counts the ordering edge as a dependency too.""" direct = [a for a, v in UNIT.items() if unit in v["requires"] or unit in AFTER.get(a, [])] all_ = set(direct) for d in direct: all_ |= with_ordering(d) return all_ oracle = {b: len(dependents_of(b)) for b in UNIT} diagnoses = { "flat list": {b: 0 for b in UNIT}, "direct": {b: sum(1 for v in UNIT.values() if b in v["requires"]) for b in UNIT}, "transitive graph": dict(oracle), "with ordering": {b: len(with_ordering(b)) for b in UNIT}, } print("unit oracle direct flat list") for b in UNIT: print(f" {b:9s} {oracle[b]:5d} {diagnoses['direct'][b]:6d}" f" {diagnoses['flat list'][b]:9d}") print(" oracle total:", sum(oracle.values())) print() print("reading output lines wrong diagnosis") for ad, t in diagnoses.items(): wrong = sum(1 for b in oracle if t[b] != oracle[b]) print(f" {ad:16s} {sum(t.values()):12d} {wrong:15d}") ``` ``` unit oracle direct flat list ingest 4 2 0 queue 2 1 0 process 1 1 0 report 0 0 0 metrics 0 0 0 backup 0 0 0 oracle total: 7 reading output lines wrong diagnosis flat list 0 3 direct 4 2 transitive graph 7 0 with ordering 9 2 ``` ## Three Numbers and the Second Claim Three numbers sit side by side. **Oracle:** the graph's real state is **7** dependency relationships across six units; if `ingest` stops, four units stop with it. **Tool output:** the flat list prints **0** edge lines, the direct reading **4**, the transitive graph **7**, the with-ordering reading **9**. **Wrong diagnosis:** **3** for the flat list, **2** for the direct reading, **0** for the transitive graph, and **2** again for the with-ordering reading. The with-ordering reading prints the most lines, and its diagnosis is worse than the transitive graph's. The two extra lines say that stopping `queue` also stops `metrics`, and that stopping `backup` also stops `report`; both are wrong. This is the first payment of the course's second claim: **more output does not mean a better diagnosis.** What leads to the right count is not how many lines are printed, but which kind of edge is counted. The flat list's **3** wrong diagnoses come from the same place. Because the list shows no edge at all, it produces the answer "stopping this stops only this" for every unit; that answer is wrong for `ingest`, `queue`, and `process`, and happens to be right for the remaining three. The word "happens" is not used loosely here: the reason the answer holds for `report`, `metrics`, and `backup` is not that the reading is good, it is that those units sit at the graph's edge. The same reading errs more often on a graph with fewer edges at the tip. **Output whose diagnosis is not tested counts as unmeasured**; the list is correct, the conclusion drawn from it is not. The direct reading's **2** wrong diagnoses come from a different flaw. This reading sees edges but cuts the chain after one step: **2** for `ingest`, real value **4**; **1** for `queue`, real value **2**. The direction of error here is always the same, always **undercounting**. The with-ordering reading's error direction is always **overcounting**. Because the two error types land in the same column, the counts can come out equal, even though their operational consequences are opposite: the operator who undercounts is caught unprepared, the operator who overcounts does unnecessary work. The working habit that follows can be written in one sentence: before stopping a unit, the **reverse dependency tree** is read, and every line in that tree is verified against the unit files as either requiring or ordering. When either step is skipped, the direction of the error is known. If the tree is not read, units are undercounted and unexpected units stop; if the tree is read but the edge type is not verified, units are overcounted and an unnecessary maintenance window is opened. Both land in the same "wrong diagnosis" column in the measurement. ## The Graph Is a Snapshot Unit files do not sit in a single directory on disk. The definition a package installs is in one directory, a locally written definition is in another, a definition generated at runtime is in a third, and a file with the same name **shadows** another across these directories. Alongside shadowing there are also **drop-in files**: these do not replace the whole file, they only add a few lines or change a single setting. As a result, a unit's effective definition sits not in one file but in the combination of several; only the command that prints the **merged definition** says which line came from where. Opening and reading a unit's file does not mean seeing which settings it actually runs with. There is a second fact that catches people out even more: the manager does not reread these files on every command. It reads them at startup, builds the graph in memory, and works with that copy. Changing the file on disk does not change the graph in memory; the change only enters the graph once the manager is told to reread. So the observation "I fixed the file but the behavior did not change" is not a malfunction, it is the expected result. The status output does not distinguish between the two states either: a unit shows as "active" even while the file on disk and the definition in memory differ. The gap between the oracle and the tool output opens here not in edge count but in **time**; the graph is a snapshot of the disk, and when the copy was taken does not appear in the output. ## Second Seed The numbers might be tied to a single graph. When the same measurement is repeated with two seeds over twenty random six-unit graphs built with a seeded generator, total wrong diagnoses come out to **60** and **69** for the flat list, **16** and **21** for the direct reading, **0** and **0** for the transitive graph, and **52** and **36** for the with-ordering reading. The order of magnitude holds: the transitive graph is error-free in both seeds, and the reading that counts ordering as dependency produces an error close to the flat list's in both seeds too. The individual counts depend on the mock data; **ordering is not requiring** is not among them. One point deserves underlining. The transitive-graph reading gives **zero** wrong diagnoses in both seeds, and this is not a property of the mock data, it is a consequence of the definition: the oracle was already defined as the transitive closure. The measurement's meaning is not in the correct reading coming out error-free, it lies in **the size of the wrong readings**. Across the forty graphs' total, the cost of only confusing edge types is eighty-eight wrong diagnoses; the cost of not reading edges at all, in the same forty graphs, is one hundred twenty-nine. That the two numbers sit this close says counting the ordering edge as a dependency is not much better than not reading the graph at all. ## Summary - The service manager's object is the **unit**; a unit is not the running process but the process's definition. A **target** is a name that groups units and does no work itself. - The graph has two kinds of edge: the **requires** edge creates a dependency and is transitive, the **ordering** edge only says timing and creates no dependency. - The mock server has six units and **7** dependency relationships; if `ingest` stops, four more units stop. - The flat unit list shows **0** edges and produces a wrong diagnosis on **3** of six units; the direct-dependency column brings this down to **2**, the transitive graph to **0**. - The reading that counts ordering edges as dependencies prints **9** lines and still gives **2** wrong diagnoses: the reading with the most output does not give the best diagnosis. - Namespaces and control groups are not covered in this course; they are left to the M03/K05 Kernel Interfaces and Isolation course. ## Next Step This lesson built the graph and counted how it does not show up in the list. The next question is even more unsettling: how reliable is the **status word**, the one thing that does show up in the list? A unit showing "active" against its name means the unit is running; it does not mean it is healthy. The next lesson measures the status word of a unit stuck in a failure loop on the same mock server and counts how many of sixty crashes are seen at which polling interval.