Network Simulation Environments

The previous lesson counted a path choice’s result: the choice was made, applied to forty subjects, the tail was measured. Every measurement in the course followed this order. In operations, this order is meant to run in reverse — what the tail will be should be known before touching the lever.

The lab environment is built for exactly this: a copy of the real network is produced, the change is applied there first, and the result is read there. This lesson’s question is not how the lab is set up. The question is: how much does the lab represent reality, and on how many subjects does a change validated there behave differently in the field.

What the Lab Promises

The lab delivers three things in full. Topology can be copied exactly: which device connects to which, which subnet sits where, which path carries which cost. Configuration is text, so it transfers identically. And the lab is disposable: a broken setup is deleted and recreated, which is why things that cannot be tried in the field are tried there.

What it does not deliver is also three things, and this is the part that concerns the measurement. Timing is not real: virtual links’ latency is fabricated, and queue behavior does not come out the way it does under real load. Load is not real: traffic generated in the lab imitates the field’s scale, not its shape. And hardware is not real: a device’s own resource limits, chip-level table capacity, and vendor-specific behavior are represented by a generic stand-in.

This third one gives rise to the lever. A lab does not model every device separately; a modeling scale is chosen, and that scale applies to all forty subjects.

# taught lab definition, not run

topology:
  node:
    - name: core-1
      model: generic router
      interface: [ge-0, ge-1]
    - name: edge-07
      model: generic switch
      interface: [ge-0, ge-1]
  link:
    - core-1:ge-0 <-> edge-07:ge-0
      latency: 5ms           # fabricated, not measured
      capacity: 1g            # fabricated

validation:
  - reachability: edge-07 -> core-1
  - configuration diff: field vs. lab
  - rollback rehearsal: change applied and rolled back

The model: generic lines in the transcript carry this lesson’s entire question. The object sitting in the lab is not the field device itself, it is its class. Class behavior is correct; device behavior is approximate.

The Lever: The Lab’s Scale

Scale is a cost decision. Modeling forty subjects one-to-one means forty nodes, forty configurations, and forty times the resources; modeling four subjects is a tenth of that. The operator chooses a single number: how many subjects will be represented in the lab.

The change measured is something concrete: a configuration that reserves a fixed amount of resource on every device. If a device’s capacity is below this amount, the change fails on it. Alongside this, a second source of failure exists, and it has nothing to do with scale: because special devices are represented by a generic stand-in in the lab, the lab’s verdict about them is not evidence.

The assumptions the measurement rests on:

• AC74 — The forty subjects are the course’s fixed set and are not changed. A subject’s capacity is the upper limit of the resource the change can consume.
• AC75 — The change reserves 45 units on every device; a device whose capacity is below this fails in the field.
• AC76 — A special device is represented by a generic stand-in in the lab. Even if its capacity is sufficient, it behaves differently in the field, and the lab cannot see this at any scale.
• AC77 — Scale is choosing modeled subjects at even intervals by subject number: at scale four, every tenth is modeled, at forty, all of them. The choice does not use the oracle.
• AC78 — The oracle is the setup itself: we know capacity and which device is special because we wrote them. The lab has no access to this information.
• AC79 — The measurement is not a lab setup; no node is started, no device command runs. What is counted is the coverage of the representation.

The Measurement

"""Lab environment: a single scale decision determines the representation of forty subjects.

Lever      - the number of subjects modeled one-to-one in the lab.
Tail       - a subject where the change fails in the field.
Invisible  - a subject that fails in the field and never shows up in the lab at all.
"""
SEED = 20260812
ALLOCATED = 45      # units the change allocates on every device


def make_rng(seed):
    d = seed % 2147483646 + 1

    def r(n):
        nonlocal d
        d = (d * 48271) % 2147483647
        return d % n
    return r


def subjects(count=40, seed=SEED):
    r, out = make_rng(seed), []
    for i in range(count):
        out.append({
            "no": i + 1,
            "capacity": 20 + r(81),
            "latency": 5 + r(45),
            "class": ("interactive", "batch", "standby")[r(3)],
            "special": r(9) == 0,
        })
    return out


def in_field(o):
    """Does the change fail on this subject?"""
    return o["capacity"] < ALLOCATED or o["special"]


def in_lab(o, modeled):
    """Does the lab foresee this failure in advance?

    A special device is represented by a generic stand-in; its behavior cannot be reproduced.
    """
    return o["no"] in modeled and o["capacity"] < ALLOCATED and not o["special"]


subj = subjects()
print(f"subjects {len(subj)} | units the change allocates {ALLOCATED} | "
      f"insufficient capacity {sum(o['capacity'] < ALLOCATED for o in subj)} | "
      f"special devices {sum(o['special'] for o in subj)} | "
      f"fails in the field {sum(in_field(o) for o in subj)}")
print()
print(f"{'scale':>6s} {'modeled':>11s} {'unmodeled':>14s} "
      f"{'fails in field':>17s} {'foreseen in lab':>22s} {'missed':>6s}")
for step in (10, 5, 2, 1):
    modeled = {o["no"] for o in subj if o["no"] % step == 0}
    foreseen = sum(in_lab(o, modeled) for o in subj)
    failed = sum(in_field(o) for o in subj)
    print(f"{len(modeled):6d} {len(modeled):11d} "
          f"{len(subj) - len(modeled):14d} {failed:17d} "
          f"{foreseen:22d} {failed - foreseen:6d}")

print()
full = {o["no"] for o in subj}
missed = [o for o in subj if in_field(o) and not in_lab(o, full)]
print(f"at full scale, {len(missed)} subjects missed:",
      [(o["no"], o["capacity"], "special" if o["special"] else "generic") for o in missed])

subjects 40 | units the change allocates 45 | insufficient capacity 12 | special devices 9 | fails in the field 17

 scale     modeled      unmodeled    fails in field        foreseen in lab missed
     4           4             36                17                      0     17
     8           8             32                17                      1     16
    20          20             20                17                      3     14
    40          40              0                17                      8      9

at full scale, 9 subjects missed: [(4, 30, 'special'), (7, 78, 'special'), (12, 55, 'special'), (20, 29, 'special'), (25, 31, 'special'), (26, 56, 'special'), (31, 57, 'special'), (33, 24, 'special'), (40, 70, 'special')]

What Scale Changed

The middle column does not move from scale to scale at all: subjects failing in the field are 17 at every scale. The lab does not change the field, because the lab does not touch the field. The only thing the lever changes is the two right-hand columns.

At scale 4, the lab foresees none of the failures. Validation completes, the report comes back clean, the change is approved — and it fails on seventeen devices in the field. At scale 8, foreseen is 1, at scale 20, 3. A lab modeling half of the forty subjects catches three of the seventeen failures.

At scale 40, no subject is left unmodeled, and foreseen climbs to 8. Missed is still 9. The last row states who these nine are: all nine of the nine are special devices. Five have capacity above 45 — their failure comes not from capacity but from behavior the generic stand-in cannot represent. The remaining four already have insufficient capacity, and the lab would see this if it were in a position to measure it; it cannot, because its verdict about those four devices is not evidence in either direction.

The result that follows is the course’s third claim in its final form. The lab’s scale did not reduce failure; it changed how many of them were foreseen. And growing scale all the way to the end still does not close the tail, because part of the tail comes not from scale but from representation. A forty-node lab wants ten times the nodes of a four-node one, and in return, missed drops from 17 to 9.

This does not make the lab unnecessary; it changes what it is used for. Where the lab is reliable is topology — reachability, configuration diff, rollback rehearsal. Where it is unreliable is capacity and device-specific behavior. This is why, for every bulk change applied in the field, the lab alone is not enough: dry run, staged rollout, and a written rollback step stand alongside it. If nine subjects do not show up in the lab, they need to be seen in the last wave, not the first.

Summary

• The lab reliably delivers topology, configuration, and rollback rehearsal; it does not deliver timing, real load, or device-specific behavior.
• The lab’s scale is a single lever and does not change the field’s failure: 17 of forty subjects fail at every scale.
• At scale 4, the lab foresees none of these seventeen, at scale 20, 3, at full scale, 8.
• Even at full scale, 9 subjects are missed, and all nine are special devices; this part of the tail comes not from scale but from being represented by a generic stand-in.
• The lab is therefore not approval itself, it is one part of it; dry run, staged rollout, and written rollback have to stand alongside it.

Course Wrap-Up

A single question was asked throughout the course: when the operator touches a lever, how many of the forty subjects does it misserve. Sixteen lessons asked this question on sixteen separate objects, and every time it produced two numbers — the tail, and the part of the tail invisible on the indicator.

Lesson	Lever	Tail	Invisible
Load Balancers	the balancer’s layer	layer 4: 17, layer 7: 16 subjects over	the connection-count indicator says 10 for everyone; the 17 over-subjects are not there
Load Balancing Algorithms	the distribution rule	at load 1200: 6 / 0 / 0 / 7, at load 2600: 23 / 40 / 40 / 22	1160 idle units at load 1200
Reverse Proxy and Forward Proxy	a single limit number	unprotected subjects on reverse proxy 6 to 34, throttled subjects on forward proxy 27 to 4	9 subjects and 430 units the forward proxy never sees
Content Delivery Networks	which content sits at the edge	misses on pull climb from 33 to 245, bounded push gives 379	objects the edge never sees, 7 to 25; 402 units idle under push
Quality of Service	the scheduling rule	unmet demand of 433 units is the same across all three rules; queued subjects 40 / 9 / 29	overall average wait falls from 1.19 to 0.40 while the longest wait climbs from 3 to 13
Network Telemetry	the sampling rate	missed heavy events 0 / 13 / 15 / 20 / 20	subjects never seen at all 0 / 1 / 8 / 17 / 26
Device Monitoring Protocols	polling timeout and notification threshold	timed-out devices 30 to 0; missed heavy events 18 to 4	never polled 11, never reported climbs as high as 35
Packet Analysis Tools	capture filter narrowness and buffer size	the narrowest filter leaves 91 flows outside and misses 21 heavy events; unfiltered, 32 flows do not fit the buffer	33 subjects never seen at all when the filter is narrow, 5 even unfiltered
Reachability and Latency Measurement	the probe list’s coverage	36 subjects never probed across four targets; the indicator oscillates between 37.5 and 62.5 while the real value holds at 57.5%	invisible unreachable subjects 15 / 12 / 11 / 3 / 0, missed heavy events as high as 20
The Case for Network Automation	a single template instead of manual touches	manual: 12 errors across 122 touches, 10 faulty devices	of the template’s 6 deviated devices, hidden from the report 6 / 3 / 3
Configuration Models	the configuration model’s coverage	fields outside the model 42 / 17 / 9	9 field instances no model can represent
Programmable Interfaces	the request form	the append form is not idempotent and changes state on 40 devices on a second application; on retry it breaks 5 devices	the out-of-model field full replace erases on 9 devices without reporting it
Managing Devices with Scripts	a single template applied to forty devices	matched 34, deviated 6 — whichever class the template targets	deviations hidden from the report 6 / 3 / 3, report coverage 0.000 / 0.500 / 0.500
Cloud Networking Concepts	a single subnet prefix length	17 subnets that do not fit their block at /26	idle addresses: 2584 at /25, 9 at /27
Hybrid Connectivity	the internal traffic’s path choice	both together: 9 subjects, all nine from the batch class	5 subjects with less than 15 ms of margin left
Network Simulation Environments	the lab’s scale	17 subjects fail in the field, at every scale	9 special devices invisible even at full scale

Every one of the sixteen rows was read from its own lesson’s measurement; no number was copied from a shared definition.

The table repeats a single pattern. The lever has no “good” setting; it has settings that move where the tail sits. And every row’s right-hand column says the same thing: a lever’s most expensive effect falls not on what it measures, but on what it cannot measure.

With this course, the M04 curriculum closes. Six courses passed through the same network with six separate measures.

Course	Measurement Axis
How the Internet Works	a single request’s end-to-end trace, and the single thing each stage hands off to the next
Network Models and Protocols	the promise a layer makes against the promise it explicitly does not make
Application Layer Protocols	the decision an intermediary can take from a message without a round trip
Switching and Routing	the packet’s fate for as long as the tables disagree
Wireless Networks and Network Security	the gap between the boundary’s intended state and what actually passes
Network Operations and Automation	the tail a single lever leaves across forty subjects

The order is not arbitrary. The first course traced a request from start to finish and showed that every stage forgets what came before it. The second split this forgetting into layers and separated what each layer promises from what it explicitly does not promise. The third counted how the promise is written into a message and what the reader can extract from it. The fourth showed that the path carrying the message has its own settling time. The fifth measured where the boundary placed on that path diverges between intent and outcome. The sixth counted the single hand applying all of this to forty subjects at once.

Something changes at the last step. Every measurement in this course had a single operator: they touched the lever, they read the tail, they wrote the rollback. In reality, it is not like this. One person writes the template, another reviews it, a third applies it, and only a fourth knows the ninth special device in the field. Running a network is a team’s job — and the right-hand column of these tables is most often something one person knows but the team does not, because it was never written down.

Team work has its own discipline, and that discipline is established not in this curriculum but in the Software Development Practice curriculum: keeping a change’s history, recording it in a revertible form, having it read by someone else before it is applied, and writing down the reasoning behind a decision. Each of the three steps written in this course as “dry run, staged rollout, rollback” corresponds to a practice there. The hand touching the lever is one person’s hand; whether the team that owns the forty subjects knows what that hand did is the subject of a separate discipline.