Hybrid Connectivity

The previous lesson’s entire measurement stayed inside a single virtual network. Forty subnets were cut from the same block, shared the same gateways, and fit a single prefix. But the reason cloud objects exist is most often that they work together with a network we own — and that network is somewhere else, in a different address space, under a different team’s responsibility.

Hybrid connectivity is the connection layer that forces two networks to behave as a single addressing and access plane. Two basic options exist, and both do the same job: a site-to-site tunnel opens an encrypted channel over a shared path, and a dedicated link gives a separated path between two networks. This lesson’s question is not which one is better. The question is: when the choice is applied to forty subjects as a single lever, which subject gets what, and who is left behind.

Two Paths

What separates the two options is not technology, it is whose capacity is shared.

A site-to-site tunnel is built on top of a path that already exists. The path is shared: its capacity is unpredictable, its latency changes together with others’ load, and the tunnel itself adds a fixed extra latency because of encryption and encapsulation. In return, setting it up is a configuration matter; opening a new tunnel means creating a new record.

A dedicated link, on the other hand, is a separated path between two endpoints. Its capacity is known and not shared with anyone else, its latency is stable, and traffic over it never touches the shared path. In return, its capacity is fixed: the number chosen at setup can only change through a new installation, not a later configuration change.

# taught configuration transcript, not run

site-to-site tunnel:
  peer: company-backbone
  tunnel selector:
    - 10.40.0.0/20 -> 172.20.0.0/16
  key material: read from a secret store, not written here
  extra latency: depends on the shared path's condition

dedicated link:
  peer: company-backbone
  reserved capacity: chosen at setup, cannot be grown later
  path: never touches the shared path

routing priority:
  1. dedicated link
  2. site-to-site tunnel      # if dedicated is full or down

Key material, credentials, and pre-shared secrets are not written in any example in this lesson; where they are read from is stated in a single line in the transcript above and passed over.

The tunnel’s routing side — which destinations the tunnel selector admits, how unselected flows never touch the rule sequence at all, how the two directions of an assigned zone pull in opposite directions — was measured in the Wireless Networks and Network Security course’s VPN Technologies lesson and is not repeated here. That lesson counted which flow the tunnel admits. This lesson counts what the tunnel gives the flow that enters it.

The Lever: Path Choice

The operator is handed three options, and one is chosen. The choice applies to all forty of forty subjects at once: dedicated link alone, tunnel alone, or both together.

The third option needs a priority rule, because the dedicated link’s capacity is smaller than the forty subjects’ total demand. The rule is taken in its most common operational form: class priority. The class most sensitive to latency is placed on the dedicated link first, and the rest fall to the tunnel.

Every subject has a latency budget, and the budget comes from its class. If a subject exceeds its budget, it is in the tail. If it finds no path at all, it is in a heavier form of the tail: it gets no service.

The assumptions the measurement rests on:

• AC67 — The forty subjects are the course’s fixed set; the set is not changed. A subject’s capacity is the units it wants from the dedicated link, its latency is the path’s baseline latency without a tunnel.
• AC68 — The dedicated link’s capacity is 900 units, and total demand is 2336; the capacity was chosen at setup and cannot be grown throughout the measurement.
• AC69 — The tunnel adds a fixed 30 ms on top of baseline latency. The tunnel’s capacity is not limited; when the shared path becomes a bottleneck, it shows this through latency, not capacity.
• AC70 — Latency budgets are set by class: interactive 45 ms, batch 60 ms, standby 120 ms. The budget is a service decision, not a finding of the measurement.
• AC71 — In the both-together choice, the dedicated link is filled by class priority; within a class, order follows subject number, and a subject whose capacity does not fit the remaining room passes to the tunnel.
• AC72 — A subject with less than 15 ms left in its budget counts as thin margin. This subject does not appear in the tail on the indicator; only the path degrading is enough to put it in the tail.
• AC73 — The oracle is the setup itself: we know capacity, baseline latency, and class because we wrote them. The measurement is not a network measurement; no tunnel is set up, no request is sent to any peer.

The Measurement

"""Hybrid connectivity: a single path choice applied to forty subjects at once.

Lever      - which path the company's internal traffic is routed over.
Tail       - a subject exceeding its latency budget, or finding no path at all.
Invisible  - a subject with less than MARGIN_THRESHOLD ms left in its budget.
"""
SEED = 20260812
DEDICATED = 900          # the dedicated link's unit capacity
TUNNEL_EXTRA = 30        # ms the tunnel adds over the shared path
TUNNEL_DEGRADED = 80     # extra ms when the shared path degrades
BUDGET = {"interactive": 45, "batch": 60, "standby": 120}
PRIORITY = ("interactive", "batch", "standby")
MARGIN_THRESHOLD = 15    # less than this much budget left counts as invisible tail


def make_rng(seed):
    d = seed % 2147483646 + 1

    def r(n):
        nonlocal d
        d = (d * 48271) % 2147483647
        return d % n
    return r


def subjects(count=40, seed=SEED):
    r, out = make_rng(seed), []
    for i in range(count):
        out.append({
            "no": i + 1,
            "capacity": 20 + r(81),
            "latency": 5 + r(45),
            "class": ("interactive", "batch", "standby")[r(3)],
            "special": r(9) == 0,
        })
    return out


def place(subj, choice):
    """Assigns each subject a path: 'dedicated', 'tunnel', or 'none'."""
    if choice == "tunnel":
        return {o["no"]: "tunnel" for o in subj}
    path, remaining = {}, DEDICATED
    ordered = sorted(subj, key=lambda o: (PRIORITY.index(o["class"]), o["no"]))
    for o in ordered:
        if o["capacity"] <= remaining:
            path[o["no"]] = "dedicated"
            remaining -= o["capacity"]
        else:
            path[o["no"]] = "tunnel" if choice == "both" else "none"
    return path


def measure(subj, path, extra):
    """Subjects exceeding budget, unserved, and with a narrowing margin."""
    over = over_ms = unserved = thin = 0
    for o in subj:
        p, budget = path[o["no"]], BUDGET[o["class"]]
        if p == "none":
            unserved += 1
            continue
        latency = o["latency"] + (extra if p == "tunnel" else 0)
        if latency > budget:
            over += 1
            over_ms += latency - budget
        elif budget - latency < MARGIN_THRESHOLD:
            thin += 1
    return over, over_ms, unserved, thin


subj = subjects()
print(f"subjects {len(subj)} | total demand {sum(o['capacity'] for o in subj)} | "
      f"dedicated link {DEDICATED} units | tunnel extra latency {TUNNEL_EXTRA} ms")
print("latency budget: " + ", ".join(f"{s} {BUDGET[s]} ms" for s in PRIORITY))
print()
print(f"{'choice':<16s} {'on dedicated':>12s} {'on tunnel':>9s} {'unserved':>8s} "
      f"{'over budget':>12s} {'over ms':>8s} {'thin margin':>12s}")
for choice in ("dedicated", "tunnel", "both"):
    path = place(subj, choice)
    a, ms, h, d = measure(subj, path, TUNNEL_EXTRA)
    print(f"{choice:<16s} {sum(1 for v in path.values() if v == 'dedicated'):12d} "
          f"{sum(1 for v in path.values() if v == 'tunnel'):9d} "
          f"{sum(1 for v in path.values() if v == 'none'):8d} "
          f"{a:12d} {ms:8d} {d:12d}")

print()
print(f"when the shared path degrades (extra latency {TUNNEL_EXTRA} -> {TUNNEL_DEGRADED} ms)")
print(f"{'choice':<16s} {'over budget':>12s} {'over ms':>8s} {'thin margin':>12s}")
for choice in ("dedicated", "tunnel", "both"):
    path = place(subj, choice)
    a, ms, h, d = measure(subj, path, TUNNEL_DEGRADED)
    print(f"{choice:<16s} {a:12d} {ms:8d} {d:12d}")

print()
path = place(subj, "both")
for cls in PRIORITY:
    group = [o for o in subj if o["class"] == cls]
    ded = sum(1 for o in group if path[o["no"]] == "dedicated")
    over = sum(1 for o in group
               if path[o["no"]] == "tunnel"
               and o["latency"] + TUNNEL_EXTRA > BUDGET[cls])
    print(f"both together | {cls:<11s} subjects {len(group):2d} | "
          f"dedicated {ded:2d} | tunnel {len(group) - ded:2d} | "
          f"over budget {over:2d}")

subjects 40 | total demand 2336 | dedicated link 900 units | tunnel extra latency 30 ms
latency budget: interactive 45 ms, batch 60 ms, standby 120 ms

choice           on dedicated on tunnel unserved  over budget  over ms  thin margin
dedicated                  17         0       23            0        0            2
tunnel                      0        40        0           14      144           10
both                       17        23        0            9       76            5

when the shared path degrades (extra latency 30 -> 80 ms)
choice            over budget  over ms  thin margin
dedicated                   0        0            2
tunnel                     32     1508            1
both                       16      758            3

both together | interactive subjects 11 | dedicated 11 | tunnel  0 | over budget  0
both together | batch       subjects 20 | dedicated  5 | tunnel 15 | over budget  9
both together | standby     subjects  9 | dedicated  1 | tunnel  8 | over budget  0

The Tail of Three Choices

The first row is a trap, and it needs to be read carefully. In the dedicated-alone choice, subjects over budget are 0, over ms 0. An indicator that measures latency shows this deployment as flawless. The column next to it tells you what was paid: 23 of forty subjects find no path at all. The dedicated link has 900 units, total demand is 2336; once capacity runs out, the next subject in line is left out.

The tail did not disappear, it stepped outside the indicator. The latency indicator counts only served subjects, and an unserved subject has no latency. This is the sharpest form of what has been counted since the course’s first lesson: a good-looking number’s denominator can have been shrunk by the tail itself.

In the tunnel-alone choice, the denominator is fixed — all forty of forty subjects are served, unserved is 0. This time the tail is inside the indicator: 14 subjects exceed their budget, total overage 144 ms. There is no capacity problem, there is a latency problem. The 30 ms the tunnel adds is fixed and applied the same way to everyone; classes with a tight budget cannot absorb it.

The both-together choice is best on both indicators: unserved 0, over budget 9, over ms 76. The dedicated link takes the latency-sensitive subjects, the rest pass over the tunnel. The tail drops from 14 to 9.

Where the Tail Collected

The bottom transcript gives the both-together choice’s real result, and this result matters more than the number itself. All 11 of the interactive class’s 11 subjects are on the dedicated link; over budget 0. Because the standby class’s budget is 120 ms, its eight subjects on the tunnel are comfortable too; over 0. All nine of the nine in the tail come from the batch class.

The priority rule did this. The rule put the interactive class first, that class took two-thirds of the dedicated link with 580 units, and the remaining 320 units met the demand of only five of the batch class’s twenty subjects. Fifteen batch subjects fell to the tunnel, and nine of them exceeded their budget.

This is the course’s second claim in this lesson’s form. None of the three choices removed the tail. Dedicated alone placed the tail on the unserved, tunnel alone on everyone with a tight budget, both together on a single class. The choice giving the lowest number is the choice that concentrates the pain into the narrowest place. Which one is correct is not a measurement decision, it is a policy decision — and whether the batch class’s nine subjects accept this cannot be read from the table.

Failover and Two Side Effects of Two Paths

The real rationale for a both-together setup is most often not capacity, it is failover: when the dedicated link fails, traffic falls to the tunnel and service is not interrupted. This cost is already written into the measurement. The moment the dedicated link goes down, all forty of forty subjects move to the tunnel, meaning the deployment drops to the tunnel row: over budget 14, over ms 144. Failover keeps the service running, but it raises the tail from nine to fourteen, and it does this at the moment of a single failure. An operation planning failover has to write the row that comes after the failover too.

Two paths existing at the same time has two more side effects, and both fall outside this measurement.

The first is address overlap. Hybrid connectivity forces two separate address spaces to behave as a single plane; if the two spaces use the same block, no path can be established. This is why the previous lesson’s virtual network block is a decision chosen once and irreversibly: the way to resolve an overlap later is to renumber the subnets or add a translating layer. The second option only works by breaking the address the rule sequence reads.

The second is path asymmetry. The outbound leg can go over the dedicated link and the return over the tunnel; the two directions taking separate paths is not, by itself, a fault, but it breaks every object that keeps state. A security group that sees one direction does not see the other, and recognizes the flow only by half. Narrowing: pin the flow, not the direction, to a path — both directions of traffic between two endpoints are pinned to the same path, and the pinning is written the same way on both ends through routing priority.

The Invisible Tail: Thin Margin

The last column counts subjects that have not entered the tail but stand at its edge. A subject with less than 15 ms left in its budget is not in the tail; it produces no warning, appears in no report.

The second table shows what happens when the shared path degrades. When the tunnel’s extra latency climbs from 30 ms to 80 ms, the tail in the tunnel-alone choice jumps from 14 to 32, over ms from 144 to 1508. In the both-together choice, the tail climbs from 9 to 16, overage from 76 to 758. The dedicated row does not move at all: 0 and 0. This is exactly what a separated path is worth, and it shows precisely here.

The movement in the thin-margin column reads backwards at first glance. In the tunnel-alone choice, this number falls from 10 to 1, in both-together from 5 to 3. This is not an improvement. The thin-margin subjects crossed into the tail along with the degradation; the number falling is because the subjects standing at the edge ran out. An indicator improving can come from the set it counts emptying out.

The operational rule that follows is singular: thin margin has to be counted as a separate indicator. The indicator counting the tail speaks after degradation; the indicator counting thin margin speaks before degradation, and the two are not the same number.

Summary

• A site-to-site tunnel is built over a shared path and adds a fixed extra latency; a dedicated link gives a separated path, but its capacity is chosen at setup and cannot be grown later.
• In the dedicated-alone choice, subjects over budget are 0, because 23 of forty subjects find no path at all and never enter the indicator’s denominator.
• Tunnel alone gives a path to all forty of forty subjects but throws 14 outside their budget; both together bring the tail down to 9.
• All nine of the nine in the tail are from the batch class: once the priority rule puts the interactive class’s eleven on the dedicated link, the remaining 320 units meet the demand of only five batch subjects.
• Subjects with less than 15 ms of margin left do not appear in the tail; when the shared path degrades from 30 ms to 80 ms, they cross into the tail, and the thin-margin count falling is not an improvement, it is the edge set emptying out.

Next Step

Every number in this lesson gave a choice’s result: the path was chosen, applied to forty subjects, the tail was counted. In operations, this order is meant to run in reverse — what the tail will be should be known before touching the lever. The next lesson measures the place built for exactly this: the lab environment where a copy of the real network is produced. What is measured is not the lab itself, it is how much it represents the real thing: how many subjects can be modeled, how many cannot, and how many subjects a change validated in the lab behaves differently on in the field.