The Case for Network Automation

The previous lesson measured end-to-end health indicators, and every lever touched so far was the same kind: the lever of observation. Sampling rate, capture filter, polling interval — all three determined not what the network did, but how much of it we saw. A misconfigured sampling rate does not break a device, it only hides it; turn the lever back and the device is still there.

This topic changes the lever. The lever is now configuration itself. A misconfigured template does not hide a device, it changes it; and once applied, turning the lever back does not restore the old state, because the old state no longer exists anywhere. This lesson’s question is not whether automation is a good idea — it is what manual configuration’s error grows with, how automation cuts that growth, and what tail remains after it does.

The Touch: The Unit of Manual Configuration

The unit of manual configuration is not the device, it is the field. An operator configuring forty devices does not make forty decisions; on each device, they make and write as many decisions as that device requires. Because devices are not identical to one another, this count is not the same either.

# taught configuration transcript, not run

device 14                          device 33
  interface  : main                  interface   : main
  address    : /24                   address     : /24
  scheduler  : 400 ms                priority    : interactive
                                     local_rule  : present

Two devices, not four and four, but three and four fields. A high-capacity device wants an aggregation field, a high-latency device wants a scheduler, a device in the interactive class wants a priority field, and a special device that must fall outside the template wants a local rule no template carries. The total surface of configuration is not the device count, it is the sum of these fields.

The direct consequence: in manual configuration, error grows with touch count, not device count — and since touch count grows with device count, the result lands at the same door. A forty-device change does not produce four times the error of a ten-device change; it produces however many times the touch count grew.

Manual Configuration’s Error Modes

Four modes can be distinguished, and each can be counted separately.

A field written wrong. The value is written, the syntax is correct, the device accepts it, and it is wrong. The device reports no fault because there is no fault; the gap between intent and what was written is a gap the device cannot see.

A skipped device. Thirty-nine of forty devices change, one does not. It keeps running for a long time, and the difference only becomes visible when an event singles it out from the others.

Order-dependent partial application. If the change is interrupted halfway, some devices are left with the new configuration and some with the old. The network behaves inconsistently between these two sets.

Documentation and device diverging. The written configuration says one thing, the actual configuration on the device says another. This gap is called configuration drift, established in the Wireless Networks and Network Security course as the gap between a boundary’s intended state and what actually passes. Here, the same gap is measured in the configuration itself.

What the four have in common is that none of them is reported by the device. The device applies what is written to it; whether what was written is correct lies outside the device’s knowledge.

The Template Is a Single Touch

Automation makes a single structural claim: making touch count independent of device count. The configuration is written once, reviewed once, and applied to forty devices in a single operation. What is written is not a sequence of commands but the desired end state — this approach is called declarative; it defines not what the tool will do, but what the result will be.

The infrastructure-as-code procedure and tool selection are the subject of the Infrastructure as Code course and are not repeated here. Here the object is not a server or a cloud resource, it is a network device; and what is measured is not the procedure’s efficiency, but the minority that falls outside the template.

Bringing the touch count down to one has a cost: the spread of error. In manual configuration, a field written wrong breaks one device. In a template, a field written wrong breaks forty at once. The probability of error does not fall, the size of the error changes: manual configuration produces many small errors, a template produces few very large ones.

The assumptions the measurement rests on:

• AC1 — The forty-device subject set is the course’s constant and is not changed in this lesson. The oracle is the setup itself: we know which device is special because we wrote it that way.
• AC2 — The fields a device must have written by hand are derived from its attributes: interface and address on every device, priority in the interactive class, a scheduler if latency is above 30 units, aggregation if capacity is 80 units or above, a local rule on a special device.
• AC3 — Every hand-written field has an error probability of 80 per thousand, and fields are independent. This rate is not a measurement but an assumption of the setup; if it changes, every row scales in the same direction.
• AC4 — The template carries four fields and is written once. The template’s probability of coming out faulty is computed at the same per-field rate; if the template is faulty, all forty of forty devices are faulty at once.
• AC5 — The template is written for a device class. A special device whose class does not match the template’s deviates; the deviation is not the device’s fault, it is the template’s scope decision.
• AC6 — The report is built on the interactive class’s indicators; a deviation in another class never appears in the report. This is an assumption, not a measurement.

The Measurement

"""Manual configuration vs. the template's error model.

Part 1 - manual campaign: touch count grows with subject count, errors with it.
Part 2 - the template is a single touch; error spreads, the expected count does not fall.
Part 3 - the template's own tail: what falls outside it and what the report cannot see.
"""
SEED = 20260812
ERROR_PER_MILLE = 80


def make_rng(seed):
    d = seed % 2147483646 + 1

    def r(n):
        nonlocal d
        d = (d * 48271) % 2147483647
        return d % n
    return r


def subjects(count=40, seed=SEED):
    r, out = make_rng(seed), []
    for i in range(count):
        out.append({"no": i + 1, "capacity": 20 + r(81),
                    "latency": 5 + r(45),
                    "class": ("interactive", "batch", "standby")[r(3)],
                    "special": r(9) == 0})
    return out


def fields(o):
    """The configuration fields a device requires to be written by hand."""
    f = ["interface", "address"]
    if o["class"] == "interactive":
        f.append("priority")
    if o["latency"] > 30:
        f.append("scheduler")
    if o["capacity"] >= 80:
        f.append("aggregation")
    if o["special"]:
        f.append("local_rule")
    return f


def manual(subj, count, seed):
    """Count devices are touched one by one; each field has ERROR_PER_MILLE error."""
    r, faulty = make_rng(seed), set()
    touches = errors = 0
    for o in subj[:count]:
        for _ in fields(o):
            touches += 1
            if r(1000) < ERROR_PER_MILLE:
                errors += 1
                faulty.add(o["no"])
    return touches, errors, len(faulty)


def apply_template(subj, template_class):
    """A single template is applied to forty devices; special devices deviate."""
    matched = deviated = hidden = 0
    for o in subj:
        if o["class"] == template_class or not o["special"]:
            matched += 1
        else:
            deviated += 1
            if o["class"] != "interactive":
                hidden += 1
    return matched, deviated, hidden


subj = subjects()
rate = ERROR_PER_MILLE / 1000
print(f"subjects {len(subj)} | total fields to write by hand "
      f"{sum(len(fields(o)) for o in subj)} | error per field {rate:.3f}")
print()
print(f"{'campaign':>9s} {'touches':>8s} {'errors':>6s} {'faulty devices':>14s} "
      f"{'errors per touch':>17s}")
for count in (10, 20, 40):
    t, e, a = manual(subj, count, SEED + 7)
    print(f"{count:9d} {t:8d} {e:6d} {a:14d} {e / t:17.3f}")

print()
p = 1 - (1 - rate) ** 4
print(f"{'procedure':<12s} {'touches':>8s} {'faulty prob':>12s} "
      f"{'affected devices':>17s} {'expected faulty devices':>24s}")
t, e, a = manual(subj, 40, SEED + 7)
print(f"{'manual':<12s} {t:8d} {'-':>12s} {a:17d} {t * rate:24.2f}")
print(f"{'template':<12s} {1:8d} {p:12.3f} {len(subj):17d} {p * len(subj):24.2f}")

print()
print(f"{'template class':<15s} {'matched':>7s} {'deviated':>8s} "
      f"{'hidden from report':>19s}")
for c in ("interactive", "batch", "standby"):
    m, d, h = apply_template(subj, c)
    print(f"{c:<15s} {m:7d} {d:8d} {h:19d}")

subjects 40 | total fields to write by hand 122 | error per field 0.080

 campaign  touches errors faulty devices  errors per touch
       10       30      3              3             0.100
       20       60      5              5             0.083
       40      122     12             10             0.098

procedure     touches  faulty prob  affected devices  expected faulty devices
manual            122            -                10                     9.76
template            1        0.284                40                    11.34

template class  matched deviated  hidden from report
interactive          34        6                   6
batch                34        6                   3
standby              34        6                   3

What Manual Configuration Costs

The top table gives the claim directly. As the campaign grows from ten devices to forty, touches climb from 30 to 122, and errors from 3 to 12. The right-hand column, though, stays nearly constant: errors per touch of 0.100, 0.083, and 0.098. This constancy is the measurement’s real result — error is not a skill problem, it is a multiplication. When the operator is more careful, the rate falls; when device count rises, the multiplier grows, and no indicator says how much attention has dropped.

The count of faulty devices is smaller than the count of errors: 12 errors spread across 10 devices, so two devices carry more than one wrong field. This matters in operations, because the unit of repair is the device, not the field; you go back to ten devices, not twelve.

None of the three rows reads zero. Shrinking the campaign does not zero out error, only shrinks it — and the cost is splitting the same change into more pieces.

What Automation Costs

The middle table puts the two procedures side by side, and they meet at an unexpected place. Manual configuration leaves 10 faulty devices across 122 touches; the expected value is 9.76. The template brings touches down to 1, but the template’s probability of coming out faulty is 0.284, and if the template is faulty, the affected devices number 40. Expected faulty devices: 11.34.

So automation did not lower the expected error count — in this setup, it raised it slightly. What changed is the distribution. Manual configuration produces around ten faulty devices every time. The template produces zero most of the time and forty the rest. Same average, different operation: the first is a steady, bearable repair load; the second is rare but covers the entire network when it happens.

So the case for automation is not “fewer errors.” The case rests on three things: error sits in one place and can be reviewed there; the change can be tested before it is applied; and it can be rolled back the same way. Manual configuration has none of the three, because there is no written form of the intent.

The Tail Automation Does Not Remove

The bottom table carries this lesson’s real result, unchanged for the rest of the topic. Whichever class the template is written for, matched devices are 34 and deviated devices are 6. Nine of the forty devices are special, and the six that fall outside the three matching the template’s class are outside the lever every time. The deviated count does not change with which class the lever targets — the deviation does not come from the template being written wrong, it comes from a single template claiming to represent forty different devices.

The right-hand column, though, does change: deviations hidden from the report are 6, 3, and 3. In the template written for the interactive class, every deviation is in another class, and none appear in the report; in the other two templates, half of the six are visible. The tail’s size is constant; what is visible is not. This distinction is the topic’s axis, paid off in full in the fourth lesson.

In a set of forty subjects, the smallest measurable difference is 1/40 = 0.025. The gap between six and three is one hundred twenty times that, comfortably inside the measurement band.

The Limit of the Inventory

The three tables above share the same silent assumption: all forty of the forty devices are known. The measurement runs over forty subjects because the subject set is the setup; in operations, this set comes from an inventory, and the inventory itself is written either by hand or through a discovery procedure.

This concerns both procedures: a device absent from the inventory is touched by neither the operator nor the template. In manual configuration, this device at least has a chance of being found by accident one day. In a network managed by templates, the device never appears in any report, application summary, or deviation list, because all of these documents are produced by walking the inventory. The lever does not reach beyond the inventory, and because it falls outside the inventory, it does not tell anyone that it does.

This is the lever’s tail in its second form. The first was the six deviated devices measured above: a known minority that does not fit the template. The second is a minority not known at all; in this lesson’s setup its count is zero, because we wrote the inventory ourselves. In a real network that count is not zero, and measuring it means comparing the inventory against an independent discovery — asking what one lever does by using another lever.

Irreversibility and Three Protections

Every bulk change in this topic is an irreversible operation: a template applied to forty devices erases the previous state on all forty the moment it is applied. A change that empties a device’s configuration, shuts down its interface, or cuts off management access can split the network in two, and the other half may become unreachable for repair. This course does not write such a command in runnable form; what is written is what it does and that it is irreversible.

Three protections exist against this, used together.

Dry run computes the change’s outcome without applying it: it lists in advance which field will change on which device. It is the only procedure that can show the six deviated devices above even where the report cannot, because a dry run compares intent against the device’s actual state, not against a report.

Staged rollout applies the change in batches instead of all forty at once. The goal is not to prevent error, it is to limit how many devices an error affects.

Rollback keeps an applicable copy of the previous state. Without a copy there is no rollback; rollback is therefore not an operation, it is a decision made before the change.

Credentials, keys, and access tokens are not written in any example in this topic; where access to a device comes from is a separate operational decision, and it does not live inside the configuration text.

Summary

• The unit of manual configuration is not the device, it is the field; forty devices require 122 field touches, and error grows with touch count — 3/5/12 errors across 30/60/122 touches.
• Errors per touch stay nearly constant at 0.100 / 0.083 / 0.098; error is not a skill problem, it is a multiplication by touch count.
• The template brings touches down to 1 but spreads the error: expected faulty devices rises from 9.76 to 11.34. What changes is not the average, it is the distribution — 0 most of the time, 40 the rest.
• The case for automation is not fewer errors; it is that intent sits in one place, can be tested before it is applied, and can be rolled back the same way.
• Whichever class the template targets, matched devices are 34 and deviated devices are 6; the only thing that changes is deviations hidden from the report being 6, 3, and 3.

Next Step

This lesson’s template rested on an assumption: that forty devices’ configuration can be described with the same fields. The measurement showed this does not hold for six of them, and the reason was the last row in the field list — the local rule a special device wants, a field no template carries. The next lesson follows that field: when device configuration is defined by a data model, what can the model represent, which devices do not fit it, and where does the field outside the model go.