--- title: 'Network Simulation Environments' source: 'https://academia.sh/en/courses/network-operations/network-simulation-environments' course: 'Network Operations and Automation' language: en updated: '2026-08-17T18:07:15+00:00' license: 'CC BY-SA 4.0' --- # Network Simulation Environments The lab environment's scale is a lever: the seventeen subjects on which the change fails in the field stay constant regardless of scale; the only thing that changes is how many the lab foresees, and even at full scale, nine subjects cannot be represented. The previous lesson counted a path choice's result: the choice was made, applied to forty subjects, the tail was measured. Every measurement in the course followed this order. In operations, this order is meant to run in reverse — what the tail will be should be known before touching the lever. The **lab environment** is built for exactly this: a copy of the real network is produced, the change is applied there first, and the result is read there. This lesson's question is not how the lab is set up. The question is: how much does the lab represent reality, and on how many subjects does a change validated there behave differently in the field. ## What the Lab Promises The lab delivers three things in full. Topology can be copied exactly: which device connects to which, which subnet sits where, which path carries which cost. Configuration is text, so it transfers identically. And the lab is **disposable**: a broken setup is deleted and recreated, which is why things that cannot be tried in the field are tried there. What it does not deliver is also three things, and this is the part that concerns the measurement. **Timing is not real**: virtual links' latency is fabricated, and queue behavior does not come out the way it does under real load. **Load is not real**: traffic generated in the lab imitates the field's scale, not its shape. And **hardware is not real**: a device's own resource limits, chip-level table capacity, and vendor-specific behavior are represented by a **generic stand-in**. This third one gives rise to the lever. A lab does not model every device separately; a modeling **scale** is chosen, and that scale applies to all forty subjects. ```text # taught lab definition, not run topology: node: - name: core-1 model: generic router interface: [ge-0, ge-1] - name: edge-07 model: generic switch interface: [ge-0, ge-1] link: - core-1:ge-0 <-> edge-07:ge-0 latency: 5ms # fabricated, not measured capacity: 1g # fabricated validation: - reachability: edge-07 -> core-1 - configuration diff: field vs. lab - rollback rehearsal: change applied and rolled back ``` The `model: generic` lines in the transcript carry this lesson's entire question. The object sitting in the lab is not the field device itself, it is its **class**. Class behavior is correct; device behavior is approximate. ## The Lever: The Lab's Scale Scale is a cost decision. Modeling forty subjects one-to-one means forty nodes, forty configurations, and forty times the resources; modeling four subjects is a tenth of that. The operator chooses a single number: how many subjects will be represented in the lab. The change measured is something concrete: a configuration that reserves a fixed amount of resource on every device. If a device's capacity is below this amount, the change fails on it. Alongside this, a second source of failure exists, and it has nothing to do with scale: because special devices are represented by a generic stand-in in the lab, the lab's verdict about them is **not evidence**. The assumptions the measurement rests on: - **AC74** — The forty subjects are the course's fixed set and are not changed. A subject's capacity is the upper limit of the resource the change can consume. - **AC75** — The change reserves **45** units on every device; a device whose capacity is below this fails in the field. - **AC76** — A special device is represented by a generic stand-in in the lab. Even if its capacity is sufficient, it behaves differently in the field, and the lab cannot see this at **any** scale. - **AC77** — Scale is choosing modeled subjects at even intervals by subject number: at scale four, every tenth is modeled, at forty, all of them. The choice does not use the oracle. - **AC78** — The oracle is the setup itself: we know capacity and which device is special because we wrote them. The lab has no access to this information. - **AC79** — The measurement is not a lab setup; no node is started, no device command runs. What is counted is the coverage of the representation. ## The Measurement ```python """Lab environment: a single scale decision determines the representation of forty subjects. Lever - the number of subjects modeled one-to-one in the lab. Tail - a subject where the change fails in the field. Invisible - a subject that fails in the field and never shows up in the lab at all. """ SEED = 20260812 ALLOCATED = 45 # units the change allocates on every device def make_rng(seed): d = seed % 2147483646 + 1 def r(n): nonlocal d d = (d * 48271) % 2147483647 return d % n return r def subjects(count=40, seed=SEED): r, out = make_rng(seed), [] for i in range(count): out.append({ "no": i + 1, "capacity": 20 + r(81), "latency": 5 + r(45), "class": ("interactive", "batch", "standby")[r(3)], "special": r(9) == 0, }) return out def in_field(o): """Does the change fail on this subject?""" return o["capacity"] < ALLOCATED or o["special"] def in_lab(o, modeled): """Does the lab foresee this failure in advance? A special device is represented by a generic stand-in; its behavior cannot be reproduced. """ return o["no"] in modeled and o["capacity"] < ALLOCATED and not o["special"] subj = subjects() print(f"subjects {len(subj)} | units the change allocates {ALLOCATED} | " f"insufficient capacity {sum(o['capacity'] < ALLOCATED for o in subj)} | " f"special devices {sum(o['special'] for o in subj)} | " f"fails in the field {sum(in_field(o) for o in subj)}") print() print(f"{'scale':>6s} {'modeled':>11s} {'unmodeled':>14s} " f"{'fails in field':>17s} {'foreseen in lab':>22s} {'missed':>6s}") for step in (10, 5, 2, 1): modeled = {o["no"] for o in subj if o["no"] % step == 0} foreseen = sum(in_lab(o, modeled) for o in subj) failed = sum(in_field(o) for o in subj) print(f"{len(modeled):6d} {len(modeled):11d} " f"{len(subj) - len(modeled):14d} {failed:17d} " f"{foreseen:22d} {failed - foreseen:6d}") print() full = {o["no"] for o in subj} missed = [o for o in subj if in_field(o) and not in_lab(o, full)] print(f"at full scale, {len(missed)} subjects missed:", [(o["no"], o["capacity"], "special" if o["special"] else "generic") for o in missed]) ``` ``` subjects 40 | units the change allocates 45 | insufficient capacity 12 | special devices 9 | fails in the field 17 scale modeled unmodeled fails in field foreseen in lab missed 4 4 36 17 0 17 8 8 32 17 1 16 20 20 20 17 3 14 40 40 0 17 8 9 at full scale, 9 subjects missed: [(4, 30, 'special'), (7, 78, 'special'), (12, 55, 'special'), (20, 29, 'special'), (25, 31, 'special'), (26, 56, 'special'), (31, 57, 'special'), (33, 24, 'special'), (40, 70, 'special')] ``` ## What Scale Changed The middle column does not move from scale to scale at all: subjects failing in the field are **17** at every scale. The lab does not change the field, because the lab does not touch the field. The only thing the lever changes is the two right-hand columns. At scale **4**, the lab foresees **none** of the failures. Validation completes, the report comes back clean, the change is approved — and it fails on seventeen devices in the field. At scale **8**, foreseen is **1**, at scale **20**, **3**. A lab modeling half of the forty subjects catches three of the seventeen failures. At scale **40**, no subject is left unmodeled, and foreseen climbs to **8**. Missed is still **9**. The last row states who these nine are: all nine of the nine are **special devices**. Five have capacity above **45** — their failure comes not from capacity but from behavior the generic stand-in cannot represent. The remaining four already have insufficient capacity, and the lab would see this if it were in a position to measure it; it cannot, because its verdict about those four devices **is not evidence in either direction**. The result that follows is the course's third claim in its final form. **The lab's scale did not reduce failure; it changed how many of them were foreseen.** And growing scale all the way to the end still does not close the tail, because part of the tail comes not from scale but from **representation**. A forty-node lab wants ten times the nodes of a four-node one, and in return, missed drops from **17** to **9**. This does not make the lab unnecessary; it changes what it is used for. Where the lab is reliable is **topology** — reachability, configuration diff, rollback rehearsal. Where it is unreliable is **capacity and device-specific behavior**. This is why, for every bulk change applied in the field, the lab alone is not enough: dry run, staged rollout, and a written rollback step stand alongside it. If nine subjects do not show up in the lab, they need to be seen in the **last** wave, not the first. ## Summary - The lab reliably delivers topology, configuration, and rollback rehearsal; it does not deliver timing, real load, or device-specific behavior. - The lab's scale is a single lever and does not change the field's failure: **17** of forty subjects fail at every scale. - At scale 4, the lab foresees **none** of these seventeen, at scale 20, **3**, at full scale, **8**. - Even at full scale, **9** subjects are missed, and all nine are special devices; this part of the tail comes not from scale but from being represented by a generic stand-in. - The lab is therefore not approval itself, it is one part of it; dry run, staged rollout, and written rollback have to stand alongside it. ## Course Wrap-Up A single question was asked throughout the course: when the operator touches a lever, how many of the forty subjects does it misserve. Sixteen lessons asked this question on sixteen separate objects, and every time it produced two numbers — the tail, and the part of the tail invisible on the indicator. | Lesson | Lever | Tail | Invisible | |---|---|---|---| | Load Balancers | the balancer's layer | layer 4: **17**, layer 7: **16** subjects over | the connection-count indicator says **10** for everyone; the 17 over-subjects are not there | | Load Balancing Algorithms | the distribution rule | at load 1200: **6 / 0 / 0 / 7**, at load 2600: **23 / 40 / 40 / 22** | **1160** idle units at load 1200 | | Reverse Proxy and Forward Proxy | a single limit number | unprotected subjects on reverse proxy **6** to **34**, throttled subjects on forward proxy **27** to **4** | **9** subjects and **430** units the forward proxy never sees | | Content Delivery Networks | which content sits at the edge | misses on pull climb from **33** to **245**, bounded push gives **379** | objects the edge never sees, **7** to **25**; **402** units idle under push | | Quality of Service | the scheduling rule | unmet demand of **433** units is the same across all three rules; queued subjects **40 / 9 / 29** | overall average wait falls from **1.19** to **0.40** while the longest wait climbs from **3** to **13** | | Network Telemetry | the sampling rate | missed heavy events **0 / 13 / 15 / 20 / 20** | subjects never seen at all **0 / 1 / 8 / 17 / 26** | | Device Monitoring Protocols | polling timeout and notification threshold | timed-out devices **30** to **0**; missed heavy events **18** to **4** | never polled **11**, never reported climbs as high as **35** | | Packet Analysis Tools | capture filter narrowness and buffer size | the narrowest filter leaves **91** flows outside and misses **21** heavy events; unfiltered, **32** flows do not fit the buffer | **33** subjects never seen at all when the filter is narrow, **5** even unfiltered | | Reachability and Latency Measurement | the probe list's coverage | **36** subjects never probed across four targets; the indicator oscillates between **37.5** and **62.5** while the real value holds at **57.5%** | invisible unreachable subjects **15 / 12 / 11 / 3 / 0**, missed heavy events as high as **20** | | The Case for Network Automation | a single template instead of manual touches | manual: **12** errors across **122** touches, **10** faulty devices | of the template's **6** deviated devices, hidden from the report **6 / 3 / 3** | | Configuration Models | the configuration model's coverage | fields outside the model **42 / 17 / 9** | **9** field instances no model can represent | | Programmable Interfaces | the request form | the append form is not idempotent and changes state on **40** devices on a second application; on retry it breaks **5** devices | the out-of-model field full replace erases on **9** devices without reporting it | | Managing Devices with Scripts | a single template applied to forty devices | matched **34**, deviated **6** — whichever class the template targets | deviations hidden from the report **6 / 3 / 3**, report coverage **0.000 / 0.500 / 0.500** | | Cloud Networking Concepts | a single subnet prefix length | **17** subnets that do not fit their block at `/26` | idle addresses: **2584** at `/25`, **9** at `/27` | | Hybrid Connectivity | the internal traffic's path choice | both together: **9** subjects, all nine from the batch class | **5** subjects with less than 15 ms of margin left | | Network Simulation Environments | the lab's scale | **17** subjects fail in the field, at every scale | **9** special devices invisible even at full scale | Every one of the sixteen rows was read from its own lesson's measurement; no number was copied from a shared definition. The table repeats a single pattern. The lever has no "good" setting; it has settings that **move where the tail sits**. And every row's right-hand column says the same thing: a lever's most expensive effect falls not on what it measures, but on **what it cannot measure**. With this course, the M04 curriculum closes. Six courses passed through the same network with six separate measures. | Course | Measurement Axis | |---|---| | How the Internet Works | a single request's end-to-end trace, and the single thing each stage hands off to the next | | Network Models and Protocols | the promise a layer makes against the promise it explicitly does not make | | Application Layer Protocols | the decision an intermediary can take from a message without a round trip | | Switching and Routing | the packet's fate for as long as the tables disagree | | Wireless Networks and Network Security | the gap between the boundary's intended state and what actually passes | | Network Operations and Automation | the tail a single lever leaves across forty subjects | The order is not arbitrary. The first course traced a request from start to finish and showed that every stage forgets what came before it. The second split this forgetting into layers and separated what each layer promises from what it explicitly does not promise. The third counted how the promise is written into a message and what the reader can extract from it. The fourth showed that the path carrying the message has its own settling time. The fifth measured where the boundary placed on that path diverges between intent and outcome. The sixth counted the single hand applying all of this to forty subjects at once. Something changes at the last step. Every measurement in this course had a single operator: they touched the lever, they read the tail, they wrote the rollback. In reality, it is not like this. One person writes the template, another reviews it, a third applies it, and only a fourth knows the ninth special device in the field. **Running a network is a team's job** — and the right-hand column of these tables is most often something one person knows but the team does not, because it was never written down. Team work has its own discipline, and that discipline is established not in this curriculum but in the Software Development Practice curriculum: keeping a change's history, recording it in a revertible form, having it read by someone else before it is applied, and writing down the reasoning behind a decision. Each of the three steps written in this course as "dry run, staged rollout, rollback" corresponds to a practice there. The hand touching the lever is one person's hand; whether the team that owns the forty subjects knows what that hand did is the subject of a separate discipline.