Publishing

The previous lesson produced the distribution and named its content with a digest. That digest was a number computed on the production side, and it stayed there; the installing side never knew about it.

Once a package is sent to a package registry, this arrangement changes. The installing side no longer asks for the content, it asks for the name: “version 1.2 of package metrics.” The registry gives back the name’s counterpart. This lesson’s question grows from here: can the same name carry two distinct contents, and if it can, what does the version number the lock file writes actually guarantee?

What the Registry Stores

A package registry is a mapping from an (name, version) pair to a distribution. This course does not write a registry’s product name; the registry is modeled here with a single dictionary and a single operation is defined on it: publish. The model does not connect to a real registry, nothing is sent, and no package is downloaded.

Version marking itself — tagging and semantic versioning — was established in the Introduction to Version Control course and is not repeated here. This lesson does not ask how the version number is chosen; it asks how binding a chosen number is.

Two Policies

What the publish operation does when the (name, version) pair has already been used is a policy question, and it has two answers.

An overwritable registry puts the new body in the old one’s place. The submission always succeeds. An immutable registry rejects the submission when a distinct body arrives for the same pair. It does not reject the same body being submitted again — that submission does not change the registry anyway.

This is exactly where the distinction lies: immutability forbids not resubmission, it forbids resubmission with distinct content. Both can sit side by side in a publish pipeline, and the measurement includes both.

The measurement defines nine publish attempts. Three publish the shared setup’s lock-file versions for the first time, one publishes an older version, one resubmits the same body, two send distinct bodies for the same version, one sends the same body once more, one opens a new version. Two points in time are read: t1 after the fifth attempt, t2 after the ninth.

The measurement’s assumptions:

• QP22 — The lock file is the result of the shared setup’s “newest” resolution: metrics 1.2, common 3.2, report 1.1. The lock’s content does not change over the course of the measurement.
• QP23 — The distribution’s content is represented by a single body string; its digest is a truncated eight-hex-digit value, and two digests being equal is read as content being equal.
• QP24 — Install is reading the registry’s digests for the three versions in the lock file. No other side effect of install is modeled.
• QP25 — t1 and t2 are two separate installs done with the same lock file. The only difference between them is the publish attempts that reach the registry in that span.
• QP26 — A lock that also writes the digest compares the expected and the arriving digest at install time; if it finds a mismatch, install stops. It does not try to fix the mismatch.
• QP27 — Yanking closes a version to resolution but does not touch its content in the registry; because a lock file asks for the version directly, it never runs resolution at all.
• QP28 — No duration is measured, no real submission is sent to a registry. What is counted is accepted, rejected, mismatch, and distinct install count.

Measurement

"""Publishing: can the same name carry two distinct contents."""

import hashlib

LOCK = {"metrics": (1, 2), "common": (3, 2), "report": (1, 1)}
REGISTRY = {"report": [(0, 9), (1, 0), (1, 1)]}
DECLARATION = {"report": ((1, 0), (2, 0))}
YANKED = {("report", (1, 1))}

ATTEMPTS = [
    ("report", (1, 0), "report-body-0"),
    ("metrics", (1, 2), "metrics-body-1"),
    ("common", (3, 2), "common-body-1"),
    ("report", (1, 1), "report-body-1"),
    ("metrics", (1, 2), "metrics-body-1"),
    ("report", (1, 1), "report-body-2"),
    ("metrics", (1, 2), "metrics-body-2"),
    ("common", (3, 2), "common-body-1"),
    ("metrics", (2, 0), "metrics-body-3"),
]
T1 = 5
POLICIES = (("overwritable", False), ("immutable", True))


def digest(text):
    return hashlib.sha256(text.encode()).hexdigest()[:8]


def publish(attempts, immutable):
    """Package registry model: (name, version) -> distribution digest."""
    registry, accepted, rejected, conflicts = {}, 0, 0, []
    for package, version, body in attempts:
        key, d = (package, version), digest(body)
        existing = registry.get(key)
        if existing is not None and existing != d:
            conflicts.append(f"{package} {version[0]}.{version[1]}")
            if immutable:
                rejected += 1
                continue
        registry[key] = d
        accepted += 1
    return registry, accepted, rejected, conflicts


def install(registry):
    """Versions in the lock file are read from the registry."""
    return {p: registry[(p, v)] for p, v in LOCK.items()}


print(f"{'policy':<20s} {'accepted':>8s} {'rejected':>8s} {'t1 install':>28s}"
      f" {'t2 install':>28s} {'distinct':>8s}")
for name, immutable in POLICIES:
    b = install(publish(ATTEMPTS[:T1], immutable)[0])
    registry, accepted, rejected, conflicts = publish(ATTEMPTS, immutable)
    g = install(registry)
    d1 = " ".join(b[p] for p in sorted(LOCK))
    d2 = " ".join(g[p] for p in sorted(LOCK))
    print(f"{name:<20s} {accepted:8d} {rejected:8d} {d1:>28s} {d2:>28s}"
          f" {len({d1, d2}):8d}")
print(f"conflicting submission: {', '.join(publish(ATTEMPTS, True)[3])}")

print()
print(f"{'lock format':<16s} {'policy':<20s} {'mismatch':>9s} "
      f"{'caught':>7s} {'install':>9s} {'distinct content':>16s}")
for kb, with_digest in (("version only", False), ("version and digest", True)):
    for name, immutable in POLICIES:
        b = install(publish(ATTEMPTS[:T1], immutable)[0])
        g = install(publish(ATTEMPTS, immutable)[0])
        mismatched = sum(1 for p in LOCK if b[p] != g[p])
        caught = mismatched if with_digest else 0
        stopped = caught > 0
        installed = ({tuple(sorted(b.items()))} if stopped
                      else {tuple(sorted(b.items())), tuple(sorted(g.items()))})
        print(f"{kb:<16s} {name:<20s} {mismatched:9d} {caught:7d} "
              f"{'stops' if stopped else 'continues':>9s} {len(installed):16d}")

print()


def resolve(package, apply_yank):
    lo, hi = DECLARATION[package]
    return [s for s in REGISTRY[package] if lo <= s < hi
            and not (apply_yank and (package, s) in YANKED)][-1]


registry = publish(ATTEMPTS, True)[0]
print(f"{'install path':<18s} {'before yank':>20s} {'after':>20s} "
      f"{'distinct':>8s}")
for name, chooser in (("with resolution", lambda y: resolve("report", y)),
                       ("with lock file", lambda y: LOCK["report"])):
    o = [f"{'.'.join(map(str, chooser(y)))}·{registry[('report', chooser(y))]}"
         for y in (False, True)]
    print(f"{name:<18s} {o[0]:>20s} {o[1]:>20s} {len(set(o)):8d}")

policy               accepted rejected                   t1 install                   t2 install distinct
overwritable                9        0   2600b1c5 03525db7 30aad9cf   2600b1c5 7fdd5554 9388981e        2
immutable                   7        2   2600b1c5 03525db7 30aad9cf   2600b1c5 03525db7 30aad9cf        1
conflicting submission: report 1.1, metrics 1.2

lock format      policy                mismatch  caught   install distinct content
version only     overwritable                 2       0 continues                2
version only     immutable                    0       0 continues                1
version and digest overwritable                 2       2     stops                1
version and digest immutable                    0       0 continues                1

install path                before yank                after distinct
with resolution            1.1·30aad9cf         1.0·86e9c6b2        2
with lock file             1.1·30aad9cf         1.1·30aad9cf        1

Nine Attempts, Two Rejections

The top table compares the two policies on the same flow.

The overwritable registry accepts all nine of the nine attempts: 9 accepted, 0 rejected. No submission fails, no warning comes out. And yet the registry’s content changes, and that change shows up in the t1 and t2 columns: metrics 1.2’s digest goes from 2600b1c5 to 7fdd5554, report 1.1’s digest goes from 03525db7 to 9388981e. The same lock file gives 2 distinct installs.

The immutable registry produces 7 accepted, 2 rejected on the same flow. The two rejected are report 1.1 and metrics 1.2; both are pairs already published earlier with a distinct body. The two attempts that resubmit the same body, by contrast, are accepted — the fifth and eighth attempts do not change the registry, so there is nothing to reject. The t1 and t2 columns are exactly identical: 1 distinct install.

The number reads like this: 9 accepted, 0 rejected is not a health indicator. Both registries took the same nine attempts; one handled the two conflicts loudly, the other silently. The one that stayed quiet looks like it has the cleaner report, and precisely because of that it carries less information.

The guarantee immutability gives can be written in one sentence: a name-and-version pair names at most one content over its lifetime. Without this guarantee, a version number is not an identifier, it is only a label; the same label can be stuck onto two separate boxes.

The same body being accepted twice does not contradict this guarantee and is practically necessary. A submission can be interrupted, its result may never reach the sender; the sender then resubmits the same body. If the immutable registry rejected this, the sending side would first have to ask whether the submission had actually gone through. Because the rule is built by looking at content, the second submission does not change the registry and nothing arises to reject — the measurement’s 7 accepted count includes both of these submissions too. The forbidden operation is not “resubmitting,” it is changing the registry.

The Two Promises a Version Number Carries

A version number makes two separate promises, and they are often mistaken for one.

The first promise concerns compatibility: which part of the number increased tells whether the change is backward compatible. This promise was established in the Introduction to Version Control course’s tagging lesson and is not repeated here.

The second promise concerns identity: the number names a single content. This lesson measures only the second promise, and the measurement shows that this promise does not come from the number itself; it comes from the registry’s policy. In an overwritable registry, the number can give the first promise but not the second.

That the two are independent can be seen directly in the measurement’s top table. Both rejected submissions are trying to send a distinct body to an existing pair; neither has anything to do with the compatibility promise. Even if the body were a fully compatible fix, the rejection would be the same, because what is checked is not compatibility, it is identity.

The cost a rejection makes the publishing side pay is clear too: if a change is to be published, a new version number has to be opened. The measurement’s ninth attempt does exactly this and is accepted as metrics 2.0. Immutability is not a prohibition, it is a redirection: it does not block the fix, it makes the fix’s naming mandatory. In return, the installing side knows the meaning of the number it holds does not change over time.

What the Lock Writes

The middle table measures how much the installing side can protect itself against this situation.

A lock that writes only the version catches 0 out of 2 mismatches in the overwritable registry. Install continues and 2 distinct contents get installed. The lock file has, as far as it is concerned, done its job — the requested versions were installed — but what it installed was two separate things across two separate runs. The lock unified resolution, it did not unify content.

A lock that also writes the version and the digest catches both of the 2 mismatches in the same registry, and install stops. Installed distinct-content count drops to 1, because the second install never completes. What is gained here is not one install’s success, it is the visibility of failure: where the registry stayed silent, the lock speaks up.

The bottom two rows show the situation where the policy is correctly built: in the immutable registry, mismatch 0, caught 0, install continues, distinct content 1. Writing the digest has no cost here and shows no gain either — because there is nothing to catch.

From this, two separate guarantees turn out to be independent. Immutability is a guarantee the registry gives, and the installing side cannot verify it. The digest is the installing side’s own verification and works independently of the registry’s policy. When both are built together, the installing side does not have to trust the registry; if only the first is present, it has to trust it; if only the second is present, it does not trust it but often stops.

Yanking Is Not Deletion

The bottom table measures a third operation. After a version is published, it can turn out that it should not be used. Yanking closes a version to resolution but does not touch its content in the registry.

The measurement separates the two. The installing side with resolution gets report version 1.1 and digest 30aad9cf before the yank; afterward it gets version 1.0 and digest 86e9c6b2. 2 distinct versions. The installing side with a lock file, by contrast, gets 1.1·30aad9cf in both cases: 1.

The difference comes from the lock file never running resolution at all. Yanking changes resolution’s candidate list; a lock file does not use a candidate list, it asks for the version directly and the registry keeps giving it. This is both the lock’s strength and its limit: it continues using a yanked version, and it does so silently.

Deletion is a separate operation and gives a separate result: on a deleted version, install with a lock file fails too. Three operations — overwriting, yanking, deletion — produce three separate outcomes, and only two of them notify the installing side.

Placing the three side by side defines what the registry owes the installing side. Overwriting changes the content and says nothing; the installing side only notices through the digest it keeps itself. Yanking does not change the content and is visible only to new resolutions; a locked install is unaffected. Deletion removes the content and stops both paths. The ordering is not one of severity, it is one of visibility: the quietest operation is the most damaging one, because in the other two the installing side gets a response either way.

Where yanking stays silent has a name too. A project that does not refresh its lock file keeps using the yanked version and does so without an error message; the yank decision only reaches that project once the lock is refreshed. A lock is a decision that freezes resolution’s result — and it also leaves out any decisions made afterward about what it froze.

Summary

• A package registry is a mapping from an (name, version) pair to a distribution; it can be built with two policies, overwritable and immutable.
• The same nine attempts give 9 accepted, 0 rejected in the first registry, 7 accepted, 2 rejected in the second; immutability forbids not resubmission but resubmission with distinct content.
• The same lock file gives 2 distinct installs in the overwritable registry, 1 in the immutable one; the version number alone does not determine content.
• A lock that writes only the version catches 0 of 2 mismatches, one that also writes the digest catches 2 and stops install; the registry’s policy and the installing side’s verification are independent guarantees.
• Yanking affects resolution, it does not affect the lock: the installing side with resolution sees 2 distinct versions, the one with a lock sees 1.

Next Step

A published package carries code to the installing side. It does not carry how to use that code. That information comes from documentation, and documentation can be produced two ways: written by hand, or generated from the source. The difference between the two ways can be measured with the same question asked in every lesson of this course. The last lesson asks it: which outcome does generating documentation from source unify, which does it not unify, and where it does not unify, what cannot take its place?