Integrating with Development Tools

The table of the previous lesson was for a single client, and for that client the protocol was pure cost: 10 requests, 10 responses, 289 tokens and 17 round trips were paid and the same result was received that a tool defined inside the lesson gives. What the definition standing on the other side returns was never counted. The return becomes visible only when a second client appears, because the definition is then not written a second time. This lesson connects the same fictional server to four separate kinds of client and counts two things separately: how many lines of definition are rewritten and how many messages are paid again. This is the measure of portability.

The own measurement of code assistants was made in the Building Applications with Model APIs course and is not repeated here. The clients are mentioned here by kind: a client running inside a code editor, a client running over the shell, a client running in a continuous integration job and a client behind a support interface. No product, plug-in or editor name appears; what is measured is the number of clients, not their identity.

• IM46. The environment, the three servers, the envelope count and capability negotiation are fictional; no real server, client or editor is called. Seed 20260218.
• IM47. A definition line is the sum of the implementation lines of a tool and one schema line. The number is read from the source, not written by hand.
• IM48. The subscriber table itself is not counted as a definition line. Had it been counted, the no-protocol column would have grown even larger; the measure is therefore set up against the protocol, not in its favor.
• IM49. Every client does its own setup: the identity step, the authorization step, the handshake and the chain. No item is shared between clients.
• IM50. The four clients work on four separate subscribers; their jobs are different, their contracts are the same.

One Server, Four Clients

The first block sets up the common definition: the fictional measurement network, three fictional servers, the envelope counter, capability negotiation and the tool call.

# FICTIONAL. The subscriber records of the measurement network, three servers, the envelope counter
# and capability negotiation are defined inside the lesson; no real server or client is called.
SEED, M32 = 20260218, 0xFFFFFFFF


def rng(t):
    x = ((t ^ (t >> 16)) * 2246822507) & M32
    x = ((x ^ (x >> 13)) * 3266489909) & M32
    s = [(x ^ (x >> 16)) & M32]

    def next_value():
        s[0] = (s[0] * 1664525 + 1013904223) & M32
        return s[0] / 4294967296
    return next_value


def tokenize(m):
    b = []
    for s in str(m).lower().replace(",", " , ").split():
        while len(s) > 5:
            b.append(s[:5])
            s = s[5:]
        b.append(s)
    return b


REGION = ["dogu", "bati", "kuzey", "guney", "merkez"]
TYPE = ["mesken", "isyeri"]
SUBSCRIBERS = {}
for i in range(60):
    r = rng(SEED + 31 * i)
    previous = 10000 + int(r() * 80000)
    SUBSCRIBERS[f"A{i + 1:03d}"] = {"bolge": REGION[int(r() * 5)], "tip": TYPE[int(r() * 2)],
                                    "onceki": previous, "simdiki": previous + int(r() * 70)}
TIERS, ABOVE, COMMERCIAL_FACTOR = ((10, 4), (25, 7), (40, 11)), 16, 2


def _charge(m3, type_):
    remaining, lower, total = m3, 0, 0
    for upper, unit in TIERS:
        band = min(remaining, upper - lower)
        if band <= 0:
            break
        total += band * unit
        remaining -= band
        lower = upper
    return (total + remaining * ABOVE) * (COMMERCIAL_FACTOR if type_ == "isyeri" else 1)


def t_meter(abone=None, **_):
    a = SUBSCRIBERS.get(abone)
    return {"hata": "abone yok"} if a is None else {
        "onceki": a["onceki"], "simdiki": a["simdiki"], "tip": a["tip"]}


def t_consumption(onceki=None, simdiki=None, **_):
    if onceki is None or simdiki is None:
        return {"hata": "eksik alan"}
    f = simdiki - onceki
    return {"m3": f + 100000 if f < 0 else f}


def t_tariff(m3=None, tip="mesken", **_):
    return {"hata": "eksik alan"} if m3 is None else {"tutar": _charge(m3, tip)}


def t_region(bolge=None, **_):
    if bolge not in REGION:
        return {"hata": "bolge yok"}
    n = sum(1 for a in SUBSCRIBERS.values() if a["bolge"] == bolge)
    return {"bolge": bolge, "abone": n}


def t_record(abone=None, sutun=None, deger=None, **_):
    if not (abone and sutun and deger is not None):
        return {"hata": "eksik alan"}
    return {"yazildi": True}


SERVERS = {
    "olcum": {"version": 2, "transport": "local",
              "tools": {"sayac_oku": {"function": t_meter, "fields": ("abone",), "side_effect": False},
                        "tuketim": {"function": t_consumption, "fields": ("onceki", "simdiki"),
                                    "side_effect": False}},
              "resources": ("abone_listesi",), "prompts": ()},
    "tarife": {"version": 1, "transport": "remote",
               "tools": {"tarife": {"function": t_tariff, "fields": ("m3", "tip"),
                                    "side_effect": False}},
               "resources": ("basamak_tablosu",), "prompts": ("bedel_aciklamasi",)},
    "kayit": {"version": 3, "transport": "remote",
              "tools": {"bolge_ozet": {"function": t_region, "fields": ("bolge",),
                                       "side_effect": False},
                        "kayit_yaz": {"function": t_record, "fields": ("abone", "sutun", "deger"),
                                      "side_effect": True}},
              "resources": (), "prompts": ()},
}
CLIENT_VERSION = {"olcum": 2, "tarife": 1, "kayit": 2}      # the kayit server is ahead
ALLOW = ("sayac_oku", "tuketim", "tarife", "bolge_ozet")     # kayit_yaz is not on the allowlist

COUNT = {"request": 0, "response": 0, "notification": 0, "tokens": 0, "round_trips": 0}


def message(kind, body, transport="local"):
    """FICTIONAL envelope. No real protocol is implemented; what is counted is the cost of the envelope."""
    COUNT[kind] += 1
    b = len(tokenize(body)) + 8                          # envelope constant
    COUNT["tokens"] += b
    if kind == "request":
        COUNT["round_trips"] += 2 if transport == "remote" else 1
    return b


def handshake(client_version=None, allow=None):
    """Capability negotiation: a version mismatch and the allowlist narrow the tool set."""
    client_version = CLIENT_VERSION if client_version is None else client_version
    allow = ALLOW if allow is None else allow
    agreed, dropped = {}, []
    for name, s in SERVERS.items():
        message("request", f"yetenek sor {name}", s["transport"])
        message("response", f"yetenek bildir {name} surum {s['version']} "
                            + " ".join(s["tools"]) + " " + " ".join(s["resources"]))
        if client_version.get(name, 0) < s["version"]:
            dropped.append((name, "version", s["version"], client_version.get(name, 0)))
            continue
        for t, spec in s["tools"].items():
            if t not in allow:
                dropped.append((name, "allowlist", t, None))
                continue
            agreed[t] = dict(spec, server=name, transport=s["transport"])
    return agreed, dropped


def call(agreed, name, state):
    if name not in agreed:
        return {"hata": "arac bildirilmedi"}
    t = agreed[name]
    kw = {a: state[a] for a in t["fields"] if a in state}
    message("request", f"{name} " + " ".join(f"{k} {v}" for k, v in kw.items()), t["transport"])
    s = t["function"](**kw)
    message("response", " ".join(f"{k} {v}" for k, v in s.items()))
    return s

Two Columns: Definition Lines and Messages

The definition line is not written by hand, it is read from the source: the implementation lines of a tool plus one schema line. The no-protocol column multiplies this number by the number of clients, the protocol column keeps it constant. The message columns behave the other way around.

import inspect

CHAIN = ("sayac_oku", "tuketim", "tarife")
IMPLEMENTATION = {"sayac_oku": (t_meter,), "tuketim": (t_consumption,), "tarife": (t_tariff, _charge)}
CLIENTS = (("code editor", "A017"), ("shell client", "A042"),
           ("continuous integration", "A003"), ("support interface", "A055"))


def definition_lines(tools):
    """The lines a client without the protocol has to carry: the implementation
    lines plus one schema line per tool."""
    n = 0
    for name in tools:
        for f in IMPLEMENTATION[name]:
            n += len(inspect.getsource(f).rstrip().splitlines())
        n += 1
    return n


def auth_steps(servers):
    for name, s in servers.items():
        if s["transport"] == "remote":
            message("request", f"kimlik sun {name}", "remote")
            message("response", f"kimlik kabul {name}")
            message("request", f"yetki iste {name} arac kumesi", "remote")
            message("response", f"yetki verildi {name} izin listesi")


def connect_client(subscriber):
    """A client's own setup: identity and authorization steps, handshake, chain."""
    auth_steps(SERVERS)
    agreed, _ = handshake()
    state = {"abone": subscriber}
    for name in CHAIN:
        state.update(call(agreed, name, state))
    return state.get("tutar")


L = definition_lines(CHAIN)
print("definition lines per tool:", {a: definition_lines([a]) for a in CHAIN}, "total:", L)
print("\nclients  added type               no protocol protocol request response tokens round trips")
for k in range(1, len(CLIENTS) + 1):
    for j in COUNT:
        COUNT[j] = 0
    results = [connect_client(a) for _, a in CLIENTS[:k]]
    print(f"{k:>7}  {CLIENTS[k - 1][0]:<22} {k * L:>11} {L:>8} {COUNT['request']:>7}"
          f" {COUNT['response']:>8} {COUNT['tokens']:>6} {COUNT['round_trips']:>11}")
print("results of the four clients:", results)

CHANGED = ("tarife",)
print("\nlines rewritten when the interface changes (changed tool: " + CHANGED[0] + ")")
for k in (1, 2, 4):
    print(f"  clients {k}: no protocol {k * definition_lines(CHANGED):>3} lines,"
          f" change points {k};  protocol {definition_lines(CHANGED):>3} lines, change points 1")

definition lines per tool: {'sayac_oku': 5, 'tuketim': 6, 'tarife': 13} total: 24

clients  added type               no protocol protocol request response tokens round trips
      1  code editor                     24       24      10       10    289          17
      2  shell client                    48       24      20       20    578          34
      3  continuous integration          72       24      30       30    867          51
      4  support interface               96       24      40       40   1156          68
results of the four clients: [844, 68, 211, 684]

lines rewritten when the interface changes (changed tool: tarife)
  clients 1: no protocol  13 lines, change points 1;  protocol  13 lines, change points 1
  clients 2: no protocol  26 lines, change points 2;  protocol  13 lines, change points 1
  clients 4: no protocol  52 lines, change points 4;  protocol  13 lines, change points 1

The Marginal Cost of the Second Client

The first row is the plainest form of the second claim of the course: standardizing for a single client is pure loss. The no-protocol column and the protocol column both show 24 lines, so the definition is written once in both cases; but the setup with the protocol pays 10 requests, 10 responses, 289 tokens and 17 round trips on top of it. The gain is zero, the cost is twenty messages. Moving a server to the protocol for a single client is not a decision that can be defended once it is measured.

The second row turns the table. The definition column goes up to 48 without the protocol while it stays at 24 with the protocol; the marginal cost of the second client is 0 definition lines and 20 messages (10 requests, 10 responses, 289 tokens, 17 round trips). At the fourth client the no-protocol column reaches 96 lines and the protocol column is still 24; on the message side, in return, the count climbs to 40 requests, 1156 tokens and 68 round trips. The two columns share no common unit: one is source lines, the other is messages. For this reason no break-even point is computed here; what is written down is which measure weighs more. The line count is a cost paid once and looked at forever, while a message is paid again in every run.

The measure of portability is exactly these two numbers: every new client brings in 24 definition lines and costs 20 messages. The gain grows linearly with the number of clients, and so does the cost; but the gain belongs to what is written, the cost to what runs. Of the 96 lines carried by the no-protocol side in a four-client setup, 72 are copies of the same three functions, and the drifting apart of these copies produces no error at all; it shows itself only when the four clients give four different results.

The Change Point

The last output gives the real number of the contract. If the calculation of the tarife tool changes, for instance the tier limit shifts or the coefficient for businesses becomes different, 13 lines are rewritten in every client in the setup without the protocol: 26 lines and two change points at two clients, 52 lines and four change points at four clients. In the setup with the protocol the same change is 13 lines and a single point for any number of clients.

The number of change points is the direct measure of the rule of the course: the number of tool connections is not the number of tools but the number of contracts, and an interface for which it is not written down who will break when it changes counts as unmeasured. In the setup without the protocol, the answer to the question “who will break” is written nowhere; only someone who finds and reads all four copies knows it. In the setup with the protocol the answer is single, and the handshake itself restates it in every session.

None of the numbers in the table depends on the seed. The four clients work on four separate subscribers and give four separate results (844, 68, 211, 684), but the message count per client is the same in all four: 10 requests, 10 responses, 289 tokens, 17 round trips. What is measured is not the size of the job but the shape of the contract.

The Cost of the Unused Tool

The table above assumes that all four clients use all three tools. In reality clients do different jobs: the shell client asks for consumption, the continuous integration job computes only the tariff. In the setup without the protocol this lowers the lines carried, because a tool that is not used is not written. In the protocol, however, the handshake is at the server level, not at the tool level.

USAGE = {"code editor": CHAIN, "shell client": ("sayac_oku", "tuketim"),
         "continuous integration": ("tarife",), "support interface": CHAIN}
START = {"abone": "A017", "m3": 30, "tip": "mesken"}


def partial(tools):
    """The cost of a client that calls only the tools it needs."""
    for j in COUNT:
        COUNT[j] = 0
    auth_steps(SERVERS)
    agreed, _ = handshake()
    state = dict(START)
    for name in tools:
        state.update(call(agreed, name, state))
    return dict(COUNT, unused=len(agreed) - len(tools))


print("client type            tools used  no-protocol lines  request tokens unused")
total_lines, total_requests, total_tokens = 0, 0, 0
for kind, tools in USAGE.items():
    s = partial(tools)
    total_lines += definition_lines(tools)
    total_requests += s["request"]
    total_tokens += s["tokens"]
    print(f"{kind:<22} {len(tools):>10}  {definition_lines(tools):>17}  {s['request']:>7}"
          f" {s['tokens']:>6} {s['unused']:>6}")
print(f"{'total':<22} {'':>10}  {total_lines:>17}  {total_requests:>7} {total_tokens:>6}")
print("definition lines on the server:", definition_lines(CHAIN))

client type            tools used  no-protocol lines  request tokens unused
code editor                     3                 24       10    289      0
shell client                    2                 11        9    264      1
continuous integration          1                 13        8    234      2
support interface               3                 24       10    289      0
total                                             72       37   1076
definition lines on the server: 24

When the needs part ways, the no-protocol column falls from 96 to 72 lines: because an unused tool is not written, it is not copied either. The protocol column is still 24 lines; the gain is 72 against 24, that is 48 lines. On the message side, however, parting ways gains little: a client using a single tool pays 8 requests and 234 tokens, one using three tools pays 10 requests and 289 tokens. The difference is two requests, because most of the cost is the handshake itself, and the handshake asks all three servers.

The last column counts this by name: two of the tools that the continuous integration client has negotiated are unused, one of the shell client’s. An unused tool is not free; it has been carried in the capability declaration, it has passed the allowlist and it sits in the client’s tool set. The second claim of the course shows up here for the second time: standardization makes a client with a narrow need pay for the whole of a broad contract.

Summary

• The definition of the three-tool chain is 24 lines; in the setup without the protocol every client carries these lines again, in the setup with the protocol the number is independent of the number of clients.
• The marginal cost of the second client is 0 definition lines and 20 messages; this is the measure of portability.
• With a single client the protocol is pure cost: the same 24 lines, plus 289 tokens and 17 round trips.
• When the interface changes, the setup without the protocol rewrites 52 lines at four points at four clients, the setup with the protocol rewrites 13 lines at a single point.
• The handshake is at the server level: a client using a single tool pays 8 requests and 234 tokens, and negotiates the two tools it does not use.
• Definition lines and messages share no common unit; the two columns cannot be compared without writing down by which measure they are read.

Next Step

In this lesson the server was always ours: we wrote the definition, we built the allowlist, and we never questioned whether what the other side declared was true. The number that portability gained gives the same convenience to a server that we did not write: a server beyond the network also declares its capabilities, chooses the tool names itself and describes what each tool does in its own description. The party reading that description is the client, and the client has nothing in hand to verify it. The next lesson counts the definitions declared by an untrusted server: how many are outside the allowlist, how many have side effects, how many carry a suspect pattern, how many the audit catches, and how many findings the same audit gives on the trusted servers.