--- title: 'Advanced Git' source: 'https://academia.sh/en/courses/advanced-git' language: en updated: '2026-08-17T04:42:42+00:00' license: 'CC BY-SA 4.0' lessons: 12 outcomes: - 'Uses history-rewriting tools with an understanding of their risks' - 'Finds systematically which commit introduced a defect' - 'Manages large and multi-component repositories' --- # Advanced Git The tools that rewrite history, interrogate history, and manage a growing repository; the course measures the cost of each operation not by the length of the history but by the number of objects it touches. ## Rewriting History Squashing, editing, and reordering commits; the cost of changing the past repository-wide; force push discipline on shared branches; and recovering lost commits with the reflog. ### Interactive Rebase Source: https://academia.sh/en/courses/advanced-git/interactive-rebase Changing a commit changes the identity of every commit after it; rewriting from the middle touches 252, 362, 3002, and 11322 objects in 50-, 200-, 1000-, and 4000-commit histories respectively, and the ratio stays at roughly half at every scale. The previous course read history with a single measure: an integration format's value was not the code it produced but the number of questions the history it left behind could answer. Four formats produced the same code, and the six questions were answered five, three, two, and three times respectively. In that course, history was a **given record**: questions were asked of it, answers were counted. Once the record was written, changing it was not on the table. This course treats the same record as an object. History can be rewritten: a commit message can be fixed, two commits can be reduced to one, the order can be changed, a commit can be dropped entirely. The measure changes along with it. There, what was measured was the **answered question**; here, what is measured is the **touched object**. This lesson's question is this: how much of history does going back and changing a commit touch, and does that ratio drop as history grows? #### Identity Propagates Through the Chain A commit's identity is derived from its content. The fields that go into the identity are fixed: the file tree at that point, the author, the committer, the timestamp, the message, and the **parent's identity**. This last field determines this entire course. When a parent's identity changes, the child's identity is forced to change too, because the child's content carries the parent's identity. This is unavoidable because identity is produced from content: there is no way to keep the same identity while changing the content, because if there were, an identity would not be an identity. The consequence can be written in one sentence: **changing a commit changes the identity of every commit after it.** In this respect there is no difference between fixing a single typo and removing an entire file from history; both regenerate everything from that point onward. The **size** of the change does not determine the cost — its **location** does. The old commits are not deleted at this point. They stay in the repository; because no branch or tag points to them anymore, they become **unreachable**. Being unreachable is not the same as ceasing to exist, and this distinction is the entire subject of this topic's fourth lesson. For now, this much is enough: rewriting is not a deletion but a **duplication** — the old chain stays in place, a new chain is written beside it, and the branch tip points to the new one. #### Interactive Mode Rebase was established in the previous course as an **integration format**: the branch's commits are turned into patches and applied in order onto a new base. That comparison is not repeated here. What is new is the same family's **interactive** mode, which never appeared there. In interactive mode, the tool does not apply the patches directly. It first puts the commits to be staged in front of you as a list; each line in the list carries a commit and the action to be applied to it. Once the list is saved and closed, the tool executes the lines **top to bottom**. ```text # taught command and example dump — not executed $ git rebase -i HEAD~3 pick 9c4d7e1 metrics: print threshold value pick 4a5b6c7 report: fix field width pick 5b6c7d8 report: add summary line # r, reword = keep the commit, edit its message # s, squash = merge the commit into its predecessor # d, drop = drop the commit ``` Four actions form this lesson's measurement axis. **Reword** keeps the commit and changes only its message. **Reorder** changes the lines' positions; the order in the list becomes the order in history. **Squash** merges a commit into its predecessor and reduces the resulting commit count by one. **Drop** removes the commit entirely, and that too reduces the count by one. The four do very different jobs: one only fixes a piece of text, another removes a change from history entirely. What the measurement asks is whether this difference is reflected in the **cost**. One of these actions was already established on its own in the Introduction to Version Control course: the ability to amend the last commit in place produces a new commit from the staging area's current state and leaves the old one unreachable. That, too, was a rewrite, only it was the **shallowest** kind — it sat at the tip of history and had no descendant. Interactive mode does the same job anywhere in history; when it is done on a commit that has descendants, the cost comes along with it. A patch may fail to apply while the plan is being executed. The mechanics of this case were already established with the rebase format in the previous course and are not repeated here. Two points are specific to interactive mode: on a squash line, the tool puts the merged commits' messages in front of you in a single text and waits for you to fix it; and if the operation is canceled midway, the branch tip returns to where it was before entering the plan. Canceling is the one intact exit available while the plan is running. #### Where the Cost Starts Interactive mode asks how many commits you will put on the list. If you plan the last three commits, the tool shows three lines and the rewrite starts from the third commit. If you want to fix a commit in the middle of history, you are forced to put **everything** written since that commit into the list — even if only one line needs fixing. The cost is therefore measured not by the number of actions in the plan but by **where the plan starts**. The measurement asks this in two separate ways: four different actions at the same position, and the same action at four different depths. Two limits of the plan should be stated here. First, commits below the listed lines cannot be touched; once a starting point is chosen, anything before it is outside the plan, and going further back means a new plan. Second, if there is a fork edge in history, a linear plan does not carry it as it is: merge commits are flattened unless requested otherwise, and the two-parent structure collapses to a single parent. The measurement's setup is linear, so this second limit does not enter the measurement; but it needs to be known before building a plan in a repository whose history has a fork. The measurement's assumptions: - **RH1** — The setup is a single history built at four scales: **50, 200, 1000, 4000** commits. Each commit touches one of five files; one in eleven also carries a binary asset, and every version of that asset stays in history as a separate object. - **RH2** — A commit's object cost is **2** for the tree and blob; if it carries a binary asset, **40** is added to that. The count is the repository's object count, not its file count. - **RH3** — Changing a commit changes the identity of every commit after it. The touched set is the closed interval from the changed position to the tip of history. - **RH4** — "Rewriting from the middle" means the position `n // 2`; the rule does not change even as the scale changes. - **RH5** — Interactive mode's four actions differ only in **the length of the result**: squash and drop remove one commit, reword and reorder do not. The touched set is the same in all four. - **RH6** — This lesson's unit of cost is the **touched object**. The number of commits whose identity changes is not a cost, it is the width of the touched set; the step unit is not used in this lesson. - **RH7** — If the number of commits staged is `k`, the rewrite starts at position `n - k + 1`. - **RH8** — The set's resolution at the four scales is **1/50, 1/200, 1/1000, 1/4000** respectively. #### Measurement ```python """Interactive rebase: the cost of rewriting from the middle. Part 1 - rewriting from the middle at four scales: commits, objects, ratio. Part 2 - interactive mode's four actions at the same position. Part 3 - the number of commits staged determines the cost. """ SEED = 20260814 FILES = ("metrics.py", "report.py", "identity.py", "config.py", "document.md") SCALES = (50, 200, 1000, 4000) ACTIONS = ("reword", "reorder", "squash", "drop") def rng(seed): state = seed % 2147483646 + 1 def draw(n): nonlocal state state = (state * 48271) % 2147483647 return state % n return draw def history(n, seed=SEED): """n-commit linear history; each commit touches one file.""" draw, commits = rng(seed), [] for i in range(n): file = FILES[draw(5)] binary = draw(11) == 0 commits.append({"no": i + 1, "file": file, "binary": binary, "objects": 2 + (40 if binary else 0)}) return commits def rewrite(t, position): """Changing a commit changes the identity of every commit after it.""" after = [x for x in t if x["no"] >= position] return len(after), sum(x["objects"] for x in after) def clone_cost(t): return sum(x["objects"] for x in t) def interactive(t, position, action): """Four actions: only the result's length changes, the touched set does not.""" changed, touched = rewrite(t, position) return len(t) - (1 if action in ("squash", "drop") else 0), changed, touched print(f"{'commits':>7s} {'rewritten':>16s} {'touched objects':>16s}" f" {'ratio':>6s} {'object ratio':>12s}") for n in SCALES: t = history(n) changed, touched = rewrite(t, n // 2) print(f"{n:7d} {changed:16d} {touched:16d} {changed / n:6.3f}" f" {touched / clone_cost(t):12.3f}") print() T = history(1000) print(f"{'action':<18s} {'result commits':>14s} {'identity changed':>17s}" f" {'touched objects':>16s}") for a in ACTIONS: remaining, changed, touched = interactive(T, 500, a) print(f"{a:<18s} {remaining:14d} {changed:17d} {touched:16d}") print() print(f"{'staged':>13s} {'starting position':>17s} {'identity changed':>17s}" f" {'touched objects':>16s} {'ratio':>6s}") for k in (3, 10, 50, 500): changed, touched = rewrite(T, len(T) - k + 1) print(f"{k:13d} {len(T) - k + 1:17d} {changed:17d} {touched:16d}" f" {changed / len(T):6.3f}") ``` ``` commits rewritten touched objects ratio object ratio 50 26 252 0.520 0.663 200 101 362 0.505 0.393 1000 501 3002 0.501 0.532 4000 2001 11322 0.500 0.501 action result commits identity changed touched objects reword 1000 501 3002 reorder 1000 501 3002 squash 999 501 3002 drop 999 501 3002 staged starting position identity changed touched objects ratio 3 998 3 6 0.003 10 991 10 20 0.010 50 951 50 260 0.050 500 501 500 3000 0.500 ``` #### Rewriting Does Not Get Proportionally Cheaper The top table does the same operation at four scales: changing the commit in the middle of history. Touched objects climb from **252** to **11322** — forty-five times. The ratio column on the right does not budge: **0.520 / 0.505 / 0.501 / 0.500**. As history grows eightyfold, rewriting from the middle still touches roughly **half** of the commits. This is a counterintuitive result. One might expect that fixing a single commit in a growing history would become **proportionally** smaller work; the opposite happens. Because the ratio stays constant, the absolute cost grows linearly with history. **Rewriting does not get proportionally cheaper;** the only thing that gets cheaper is staying close to the tip. The object-ratio column carries a separate warning. While the commit ratio approaches 0.500, the object ratio bounces around as **0.663 / 0.393 / 0.532 / 0.501**. At small scales, where the commits carrying a binary asset happen to fall decides the result: in the fifty-commit history most of those commits fell in the second half, in the two hundred-commit one most fell in the first half. Counting commits and counting objects are not the same thing, and this distinction is itself the measure of the `large-repositories` topic. Here only this is recorded: **the ratio depends on the unit it is counted in**, and this lesson's unit is the object. The middle table answers the second question. When the four actions are applied at the same position, **identity changed is 501**, **touched objects is 3002** — the same in all four. The one column that diverges is the resulting commit count: squash and drop give **999**, the other two give **1000**. So fixing a typo in a message and removing a commit from history entirely have **the same cost**. The tool does not care which action you chose; it cares where you started. The bottom table measures this from the other direction. When **3** commits are staged, touched objects are **6**; at **10**, **20**; at **50**, **260**; at **500**, **3000**. A fifty-commit plan's object cost is thirteen times a ten-commit plan's, while its commit count is only five times as large — the difference comes from the binary assets that fall within the staged range. Commits near the tip are therefore cheap, and this is exactly where interactive mode's everyday use lies: tidying up the last few commits that have not yet been shared. Read together, the three tables give a single rule: **keep the plan close to the tip.** The rule has both a cost side and a risk side. The cost side has been measured — a three-commit plan touches six objects, a five-hundred-commit plan touches three thousand. The risk side has not been measured yet: the further back the staged range reaches, the higher the chance that the commits in that range have already been picked up by others. Both sides point in the same direction and say the same thing; the second one is measured in this topic's third lesson. #### Before Rewriting Rewriting is not a reversible operation; the old chain staying unreachable does not mean there is no way to bring it back, but it does mean you will go **looking** for one. Three habits make that search unnecessary. **Backup branch.** Opening a branch that points at the branch's current tip before the operation is a single line, and it reduces recovery to rolling back one pointer. A backup branch is not a local record; it is an ordinary branch, and it stays until deleted. **Dry run.** There are comparison abilities that show how far two chains have diverged before the plan is applied. Placing the rewritten chain next to its backup side by side shows which commit actually changed and which one only changed identity. **Separate clone.** If a bulk transformation is going to be tried, it is tried on a separate copy of the repository. This is the next lesson's subject, and there it becomes a required step. ```text # taught command and example dump — not executed $ git branch backup/metrics metrics $ git rebase -i --onto main 9c4d7e1 metrics $ git range-diff main...backup/metrics main...metrics ``` The fourth safeguard is the **reflog**: the tool writes every move of a branch tip to a local log, and the old tip can be read from there. The log has two limits — it is local, and it is not kept indefinitely. This topic's fourth lesson measures exactly that limit: how many commits stay reachable after a rewrite, and when the recovery window closes. #### Summary - A commit's identity contains its parent's identity; this is why changing a commit changes the identity of **every commit after it**. What determines the cost is not the size of the change but its **location**. - Interactive mode puts commits into a plan list; the four actions are reword, reorder, squash, and drop. - All four actions at the same position change the identity of **501** commits and touch **3002** objects; the only thing that diverges is the resulting commit count (**999** or **1000**). - Rewriting from the middle touches **252 / 362 / 3002 / 11322** objects at four scales and the ratio stays at **0.520 / 0.505 / 0.501 / 0.500**: rewriting does not get proportionally cheaper. - The number of commits staged directly determines the cost: **3 / 10 / 50 / 500**-commit plans touch **6 / 20 / 260 / 3000** objects respectively. - Rewriting cannot be undone; a backup branch, a dry run, and a separate clone are the three safeguards. #### Next Step In this lesson rewriting stayed within a single branch's last few commits, and even its deepest attempt started from the middle of history. Sometimes what is needed is broader than that: removing every version of a file from history, fixing an author field across every commit, moving a directory to the root. What these jobs share is that they are forced to start from the **first** commit of history. The next lesson measures the cost of this expansion: in a thousand-commit history, how many identities does a transformation starting from the first commit change, how many objects does it touch, and what happens to the repository's copies once the operation completes. ### Bulk History Transformation Source: https://academia.sh/en/courses/advanced-git/bulk-history-transformation A repository-wide transformation starts from the first commit: in a thousand-commit history, 1000 identities change, 5640 objects are touched, and the new history does not share a single commit with the old one — so every copy has to be re-cloned and the operation cannot be undone. The previous lesson kept rewriting on top of a single branch. When the plan stayed close to the tip, the cost was small: a three-commit plan touched six objects. Even when started from the middle of history, only about half of the commits changed identity, and the rest stayed as they were. That half was the deepest point the measurement reached. Some jobs do not leave even that much untouched. An asset committed in a repository's early days that should never have been there, an author field misspelled across every commit, a directory that needs to move under the root — all of these are forced to start from the **first** commit of history. These are called **bulk history transformations**, and although they come from the same family as the previous lesson's tool, they operate at a different scale. This lesson's question is this: what does the cost become when the transformation starts from the first commit, and what is left in the repository's other copies once the operation is done? #### Transformation as a Rule In interactive mode, the plan was a list: you saw the lines and wrote an action for each one. In bulk transformation there is no plan; there is a **rule**. The rule is applied to every commit in order, the commit's tree is run through the rule, and a new commit is produced from the result. The difference between ten commits and a thousand is only duration; the rule written is the same rule. Four typical requirements fall into this class. **Removing an asset:** if a binary file or a leaked secret that should never have been committed sits in history, deleting it from the last commit is not enough; it stays readable from the old commits. **Metadata correction:** if the author or committer field is wrong throughout history. **Tree relocation:** turning a subdirectory into the repository's root, or the reverse. **Retroactive normalization:** applying a line-ending or encoding fix not only to new commits but to old ones as well. The tool has a subcommand that does this job, and there are also filtering tools that run outside the repository. The command is **not given in full form** in this lesson. The reason is not pedagogy but responsibility: a copy-and-run bulk-transformation line changes the repository's history irreversibly and invalidates every copy of it. The dump below carries a placeholder instead of a filter and cannot be run. ```text # example dump — not executed, the command is not given in full form $ git filter-branch -- --all Rewrite 9c4d7e1 (1/1000) (0 seconds passed, remaining 0 predicted) Rewrite 4a5b6c7 (2/1000) (0 seconds passed, remaining 0 predicted) ... Ref 'refs/heads/main' was rewritten ``` The dump's counter shows this lesson's measure directly: the counter runs from one to a thousand, because a thousand of the thousand commits are being regenerated. The line at the end of the dump says a second thing. Interactive mode operates from a single branch's tip; bulk transformation can be applied to **all references at once**, and usually is. The reason is directly tied to the measurement: a branch or a tag left out of the transformation keeps pointing at the old chain, that chain stays reachable, and the asset meant to be removed does not leave the repository. Forgetting a branch gives the same result as not doing the transformation at all. This is why the scope decision comes before the rule itself: which references will enter the transformation, which will be deleted before the transformation runs. #### Where the Transformation Starts The rule is applied to every commit, but **identity change** does not start at every commit. Commits produced up to the first one where the rule's content actually changes something come out identical to the old ones and get the same identity. The change starts at that first commit and, by the previous lesson's rule, runs all the way to the tip. This reduces the job of removing a file from history to a single question: when was that file **first** touched. The answer is usually unpleasant. If a repository has only a few files and every commit touches one of them, every file's first appearance sits near the very beginning of history. The measurement asks this separately for five files and one binary asset. The measurement's assumptions: - **RH9** — The setup is the previous lesson's setup and is unchanged: four scales (**50, 200, 1000, 4000**), each commit touches one of five files, one in eleven carries a binary asset. - **RH10** — Bulk transformation is a rule applied to every commit. The measurement does not count **what** the rule does, only how many commits it regenerates; this is why an author correction and a file removal sit in the same column. - **RH11** — Identity change starts at the first commit whose content the rule actually changes, and runs to the tip of history. - **RH12** — Removing a file starts at the first commit that touches that file; for a binary asset, the start is the first commit that carries the binary. - **RH13** — This lesson's unit of cost is the **touched object**. The number of commits whose identity changes is the width of the set; the step unit is not used in this lesson. - **RH14** — The shared prefix is the commits the new history keeps in common with the old one: the range up to one before the starting position. - **RH15** — Objects to fetch are the objects the other side does not already hold; every object outside the shared prefix is retransferred. This is not a network measurement, it is an object count. - **RH16** — The set's resolution at the four scales is **1/50, 1/200, 1/1000, 1/4000** respectively. #### Measurement ```python """Bulk history transformation: the cost of rewriting from the first commit. Part 1 - transformation from the first commit at four scales. Part 2 - which asset is removed determines where the transformation must start. Part 3 - the prefix the old and new history share. """ SEED = 20260814 FILES = ("metrics.py", "report.py", "identity.py", "config.py", "document.md") BINARY_ASSET = "presentation.bin" SCALES = (50, 200, 1000, 4000) def rng(seed): state = seed % 2147483646 + 1 def draw(n): nonlocal state state = (state * 48271) % 2147483647 return state % n return draw def history(n, seed=SEED): """n-commit linear history; each commit touches one file.""" draw, commits = rng(seed), [] for i in range(n): file = FILES[draw(5)] binary = draw(11) == 0 commits.append({"no": i + 1, "file": file, "binary": binary, "objects": 2 + (40 if binary else 0)}) return commits def rewrite(t, position): after = [x for x in t if x["no"] >= position] return len(after), sum(x["objects"] for x in after) def first_touch(t, name): """The first commit that touches the asset to be removed: the transformation starts there.""" for x in t: if (x["binary"] if name == BINARY_ASSET else x["file"] == name): return x["no"] return 0 print(f"{'commits':>7s} {'identity changed':>17s} {'touched objects':>16s}" f" {'ratio':>6s} {'shared prefix':>14s}") for n in SCALES: t = history(n) changed, touched = rewrite(t, 1) print(f"{n:7d} {changed:17d} {touched:16d} {changed / n:6.3f}" f" {len(t) - changed:14d}") print() T = history(1000) print(f"{'removed asset':<17s} {'first touch':>12s} {'identity changed':>17s}" f" {'touched objects':>16s} {'ratio':>6s}") for name in FILES + (BINARY_ASSET,): k = first_touch(T, name) changed, touched = rewrite(T, k) print(f"{name:<17s} {k:12d} {changed:17d} {touched:16d} {changed / len(T):6.3f}") print() print(f"{'starting position':>18s} {'shared prefix':>14s} {'shared objects':>15s}" f" {'objects to fetch':>17s}") for position in (1, 2, 9, 501): shared = sum(x["objects"] for x in T if x["no"] < position) print(f"{position:18d} {position - 1:14d} {shared:15d}" f" {sum(x['objects'] for x in T) - shared:17d}") ``` ``` commits identity changed touched objects ratio shared prefix 50 50 380 1.000 0 200 200 920 1.000 0 1000 1000 5640 1.000 0 4000 4000 22600 1.000 0 removed asset first touch identity changed touched objects ratio metrics.py 1 1000 5640 1.000 report.py 8 993 5586 0.993 identity.py 4 997 5634 0.997 config.py 3 998 5636 0.998 document.md 2 999 5638 0.999 presentation.bin 6 995 5630 0.995 starting position shared prefix shared objects objects to fetch 1 0 0 5640 2 1 2 5638 9 8 56 5584 501 500 2640 3000 ``` #### The Ratio Locks to One The top table does the same operation at four scales, and the ratio column gives a single value: **1.000**. In a thousand-commit history, **1000** identities change and **5640** objects are touched. The previous lesson's rewrite from the middle of the same history meant 501 identities and 3002 objects; starting from the first commit roughly doubles that number. But the real difference is in the column on the right: shared prefix **0**. In rewriting from the middle, the old and new history shared the first 500 commits; here, not a single commit stays in common. The ratio locking in place does not mean scale is unimportant — it says the opposite. Because the ratio is constant, the absolute cost grows linearly with history: touched objects are **22600** in the four-thousand-commit history, sixty times the fifty-commit figure. The measurement does not measure duration, but its practical implication is clear: the transformation is not a momentary task, it has to be planned. The middle table shows that the starting point is not something that can be chosen. The five files' and the binary asset's first touches sit at commits **1, 8, 4, 3, 2**, and **6**. The latest to appear is the eighth commit, and removing even that one still rewrites **993** commits — **0.993** of history. This is not a coincidence; it is a direct consequence of the count: in a repository shared among a handful of files, every file appears within the first few commits. **In bulk transformation there is no such thing as "part of history";** the choice is between all of history and nearly all of it. The bottom table shows where the cost falls. When the starting position is **1**, none of the objects on the other side is any use: shared objects are **0**, objects to fetch are **5640** — the entire repository. If the position were **501**, shared objects would be **2640** and the amount to transfer would drop to **3000**. The difference is not only quantitative but qualitative: with a common ancestor in place the other side can **fetch**; without one, the only thing it can do is **clone**. Fetching adds on top of what is already held; once there is no base left to add onto, fetching stops being a fetch in anything but name. #### What the Transformation Does Not Change The numbers give only the cost; they do not say what the transformation does to history. On that side, the situation is surprisingly calm. If the rule touches only metadata — such as the author field — every commit's actual change stays as it is, order is preserved, branch structure is preserved. The previous course's measured questions get the same answers; what changes is only the identities those answers are attached to. **In most cases, a bulk transformation changes history's addresses, not its answers.** In one case this is not true: if the rule removes an asset, the change carried by the commits that touched that asset is lost along with it. That file's line history can no longer be asked; the commits that touched it no longer appear to have touched it. This is the information that is lost, and it is usually exactly what the transformation is for. The distinction has to be made: a metadata correction does not delete information, removing an asset does. The two pay the same cost and are equally irreversible, but they are not the same operation as far as history's readability goes. #### Irreversibility and the Safe Path Bulk transformation is an irreversible operation, and for two separate reasons. **Locally**, the old chain stays unreachable. The reflog shows it for a while, then garbage collection cleans the objects up; the recovery window is measured in this topic's fourth lesson. This in-between period has a consequence that is easy to miss: **the repository that ran the transformation is not clean the moment it finishes.** The removed asset's objects are unreachable but stay in place; anyone who knows their identity can still read them. What is clean is a fresh clone taken from that repository, because cloning only transfers reachable objects. This is why "work on a separate clone" is the correct path for outcome, not just for safety. **In the copies**, there is no such thing as recovery, because there is no loss to recover from: every copy still holds the old chain intact. That is the problem. If a copy keeping the old chain is merged with the new history, the removed asset comes back and the transformation is wasted. This is why **every copy has to be re-cloned**, and this rule is not a suggestion, it is part of the operation's definition. Until your copies are replaced, the bulk transformation is not complete. The transformation's effect reaches beyond the repository too. If old identities are written into an issue tracker, a release note, or a build record, those identities no longer point at anything. Tags bound to old commits need to be handled separately. The safe path has four parts. **First is a separate clone:** the transformation is tried not on the actual repository but on a fresh copy of it, and the actual repository waits untouched until verification finishes. **Second is verification:** on the new history, it is measured whether the removed asset is truly gone and whether the commit count matches expectations. Verification looks for three answers — the commit count after the transformation, whether the removed asset's name appears in any commit's tree, and whether a reference was left out of the transformation. All three are read operations with no destructive side; the measurement already does this counting, and the same counting can be done in a real repository. **Third is coordination:** when the copies will be changed is announced in advance, because a copy that misses the change can bring the old chain back. **Fourth is the leak rule:** a secret removed from history does not become invalid by being removed. Once a key has been written, it has to be rotated; the transformation makes it harder to read, not never having happened. The coordination item turns into a sequence in practice. The transformation is bound to a moment: everyone writes their in-hand work to the other side up to that moment, from that moment on no one derives new work from the old chain, the transformation is applied, the copies are refreshed, and only after that does work continue. Whichever step in the sequence gets skipped, the result is the same: the moment a single commit derived from the old chain enters the new history, the transformation is broken and has to be redone. This turns bulk transformation from a technical operation into a **scheduling** task, and it is also the largest unmeasured line item in its cost. #### Summary - Bulk transformation is not a plan but a **rule**: it is applied to every commit and regenerates every commit. The tool's subcommand and outside filtering tools do this job; the command is not given in full form here. - Identity change starts at the first commit whose content the rule changes. Removing a file requires going back to that file's **first appearance**. - Transformation from the first commit touches **380 / 920 / 5640 / 22600** objects at four scales and the ratio is **1.000** at every scale; in a thousand-commit history, **1000** identities change. - The five files' and the binary asset's first touches lie between **1** and **8**; removing even the latest-appearing asset rewrites **0.993** of history. - Once the shared prefix drops to **0**, there is no common ancestor left: the other side cannot fetch and has to re-clone the entire **5640**-object set. - The operation cannot be undone. The safe path is a separate clone, verification, coordination, and rotating the leaked key. #### Next Step In this lesson the cost was counted once and stayed within a single repository: a thousand identities, five thousand six hundred forty objects, zero common ancestor. What the number leaves out is this — writing the rewritten history to the other side means **overwriting** the chain on the other side, and the person who does this does not pay the share of the cost that falls to the other side. The next lesson measures that distinction: in a thousand-commit history, while fixing the last commit and fixing the first commit differ by a thousandfold in identity terms, whether there is any difference at all in terms of the repository's copies. ### Force Push Discipline Source: https://academia.sh/en/courses/advanced-git/force-push-discipline In a thousand-commit history, changing the last commit changes 1 identity, changing the first changes 1000; but in both cases all seven of seven copies have to resync, and the stranded local work stays the same — the cost scales with the number of copies, not the size of the change. The previous lesson counted the cost once and it stayed within a single repository: a thousand identities, five thousand six hundred forty objects, zero common ancestor. The number described the local half of the operation. For a rewritten history to be of any use, it has to be written to the other side too — and the other side does not accept it the ordinary way. This lesson covers why that rejection is placed, how it is overridden, and who loses what once it is overridden. It is not a procedure lesson: below, no force push line is written in runnable form. What is measured is not how the tool is invoked but the cost the invocation leaves **on whom**. #### The Reason for the Rejection The distinction between fetch, pull, and push was established in the previous course and is not repeated here. One point from it is needed here: a push asks to change where a reference on the other side points. The other side accepts this request under only one condition — the requested new tip **contains** the current tip. If it does, nothing is lost; every commit on the other side can still be found by walking backward from the new tip. Rewriting breaks exactly this condition. The new chain does not contain the old chain's commits; it puts commits with different identities in their place. The other side rejects the request and states its reason. ```text # example dump — not executed $ git push source metrics ! [rejected] metrics -> metrics (non-fast-forward) error: failed to push some refs to 'source' hint: Updates were rejected because the tip of your current branch is behind hint: its remote counterpart. ``` The rejection message reports not a defect but a rule doing its job. This rejection is the repository's one built-in protection. It allows the chain a reference on the other side points to to be changed **only by growing**; it does not allow shrinking it or replacing it with a different chain. #### Overriding the Rejection The tool has an option that disables this rejection, and its name is **force push**. What it does can be said in one sentence: it moves the reference on the other side to the tip of the new chain without asking its old value at all. The old chain is not deleted; because no reference points to it anymore it stays unreachable, and the other side's garbage collection routine cleans it up after a while. This line is not written in full form here. The reason is that the line itself is copyable: run on a shared branch, it deletes others' work from that branch's record and **the operation cannot be undone** — undoability is provided only by the copies in others' hands, and what that means is written separately at the end of the lesson. A second option in the same family adds the other side's reference's **expected value** to the request as well: it writes if the tip is where expected, rejects if not. This option does not make force push safe; it only prevents you from overwriting work you **did not see**. There is no protection for work you saw and overwrote anyway. #### Who Pays the Cost The cost of rewriting sits in two separate columns. The first is the writer's column: the rewritten chain's objects are produced once. The second is the copies' column: every copy has to re-obtain the same chain. The measurement places the two side by side and changes the position of the changed commit four times. The measurement's assumptions: - **RH17** — The setup is the previous lessons' setup and is unchanged; the measurement is done on a **1000**-commit history. - **RH18** — The repository lives in **seven** copies. The copy count is an input to the measurement, not a choice made by the person doing the rewrite. - **RH19** — The moment the branch tip's identity changes, every copy has to resync. The measurement counts the copy that **has to** resync; whether they actually do is outside the measurement. - **RH20** — The writer's cost is the object count of the chain they rewrote; the copies' cost is that same count multiplied by the number of copies. The unit is the **touched object**. - **RH21** — Each copy has some amount of local work written on top of the old tip (drawn from the setup, zero to three commits) and that work has not yet been written to the other side. - **RH22** — After force push, this local work's foundation disappears. The work left stranded is **independent of the position** of the changed commit: the moment the tip changes, every local chain loses its foundation. - **RH23** — Identity and copy counts are not a cost, they are the width of the set; the cost is read only in the object column. - **RH24** — The resolution is **1/1000** in commits, **1/7** in copies. #### Measurement ```python """Force push: the person who rewrites does not pay the cost. Part 1 - four positions: identity changed, objects touched by the writer and the copies. Part 2 - the local work copies wrote on top of the old tip. Part 3 - sharing status: which condition is the one safe one. """ SEED = 20260814 FILES = ("metrics.py", "report.py", "identity.py", "config.py", "document.md") COPIES = 7 def rng(seed): state = seed % 2147483646 + 1 def draw(n): nonlocal state state = (state * 48271) % 2147483647 return state % n return draw def history(n, seed=SEED): """n-commit linear history; each commit touches one file.""" draw, commits = rng(seed), [] for i in range(n): file = FILES[draw(5)] binary = draw(11) == 0 commits.append({"no": i + 1, "file": file, "binary": binary, "objects": 2 + (40 if binary else 0)}) return commits def rewrite(t, position): after = [x for x in t if x["no"] >= position] return len(after), sum(x["objects"] for x in after) def resyncing(copy_count, position, t): """Who has to resync the rewritten history.""" changed, _ = rewrite(t, position) return copy_count if changed else 0 def copies(count=COPIES, seed=SEED): """Each copy's local commit count written on top of the old tip.""" draw = rng(seed) return [{"no": i + 1, "local": draw(4)} for i in range(count)] T = history(1000) K = copies() print(f"{'changed position':>17s} {'identity changed':>17s}" f" {'writer objects':>15s} {'resyncing copies':>17s}" f" {'copies objects':>15s}") for position in (1000, 900, 500, 1): changed, touched = rewrite(T, position) k = resyncing(COPIES, position, T) print(f"{position:17d} {changed:17d} {touched:15d} {k:17d} {k * touched:15d}") print() print("local commits copies wrote on the old tip:", [x["local"] for x in K], "total", sum(x["local"] for x in K)) print() print(f"{'sharing status':<16s} {'copies':>6s} {'resyncing':>10s}" f" {'stranded (last commit)':>24s} {'stranded (first commit)':>25s}") for name, count in (("unshared", 0), ("three copies", 3), ("seven copies", 7)): stranded = sum(x["local"] for x in K[:count]) print(f"{name:<16s} {count:6d} {resyncing(count, 1000, T):10d}" f" {stranded:24d} {stranded:25d}") ``` ``` changed position identity changed writer objects resyncing copies copies objects 1000 1 2 7 14 900 101 602 7 4214 500 501 3002 7 21014 1 1000 5640 7 39480 local commits copies wrote on the old tip: [0, 0, 3, 2, 2, 2, 0] total 9 sharing status copies resyncing stranded (last commit) stranded (first commit) unshared 0 0 0 0 three copies 3 3 3 3 seven copies 7 7 9 9 ``` #### A Thousandfold Difference, a Zerofold Difference The top table's left half gives the expected result. Changing the last commit changes **1** identity and costs the writer **2** objects; changing the first commit changes **1000** identities and means **5640** objects. The gap is a thousandfold. Between fixing a typo and transforming the entire repository, in terms of the cost the writer sees, there is a chasm. The right half does not see this chasm. **The number of copies that have to resync is 7 in all four rows.** All seven of seven copies, no matter how small the change, have to pick up the branch tip's new identity and adjust their own state to it. The gap is not a thousandfold, it is **zerofold**. The chasm the writer sees does not exist at all in the table the copies see. The copies' object column gives the absolute counterpart of this: the cost of changing the last commit is **14** objects, of changing the first is **39480**. Both numbers are **seven times** the writer's column figure — and seven is not a number the writer decides. This is the lesson's third claim: **the cost scales with the number of copies, not the size of the change, and that number is not in the rewriter's hands.** The person doing the rewrite can choose the cheapest line in their own column; the copies' column is not subject to that choice. The bottom table states this in its starkest form. The local work the copies wrote on top of the old tip totals **9** commits — three of the seven copies have no local work at all, four do. The work stranded after force push is **9**, and this number is the same in both columns: whether you change the **last** commit or the **first**, nine commits lose their foundation. Whether the tip changed or not is the entire question; how much it changed concerns no one. There is exactly one safe row in the table: **unshared**. When the copy count is zero, resyncing is zero and stranded work is zero. This is the one criterion the measurement produces, and it is the only one. The middle rows show a detail as well. Rewriting from the nine hundredth commit changes **101** identities but touches **602** objects, not two hundred two. The difference comes from the binary assets that fall in that range — counting identities and counting objects do not give the same answer, and this lesson's unit is the object. This is why the copies' column multiplies the object count by seven, not the identity count. #### What the Copy Has to Do "Resyncing" is not a single operation, it is a decision, and the decision is the copy owner's. There are two paths and neither is free. **The first path** is resetting the local branch to the new tip. If there is local work, this path abandons it; the commits stay unreachable and can only be found from that copy's own reflog. If there is no local work, the path is clean, and that is the case for three of the copies in the measurement. **The second path** is rebasing the local work onto the new chain. Nothing is lost, but the local commits also get new identities, and this time the cost from this course's first lesson is paid on the copy's side. For four copies, this is what is paid: all nine of nine commits are rewritten. There is no free third path. Trying to write the local work to the other side as it is brings the same rejection, this time facing the copy — and now the copy's owner is looking at the force push button too. One force push inviting a second shows why this lesson's discipline belongs to a team, not a single person. #### Without Anyone Knowing The most irritating part of the operation does not show up in the numbers. Force push **is not written to history.** In the new history, no commit and no field says someone overwrote something. The previous course's sixth question showed that the conflict decision was not recorded; here, what is not recorded is that the record itself was changed. The copies do not learn instantly either. As long as a copy does not look at the other side, it keeps assuming its old chain is valid, writes commits on top of it, and grows its stranded work. The moment of learning is the next fetch; every commit made before that is a commit added onto an overwritten tip. That moment can itself be misleading. If a pull is called after the fetch, the tool tries to merge the two chains and usually succeeds: because the old and new chains carry the same changes, the merge completes silently. The result is a history in which the same changes are found twice, under two separate identities, and nothing warns of this — only looking at the history reveals it. The overwritten branch being overwritten again often starts this way too: a copy brings the old chain back, the rewriter sees this, and repeats the operation. Together, these three facts turn the operation from a tool problem into a **notification problem.** The tool warns no one, history tells no one; the one who has to say something is the rewriter. The notification's content is fixed too: which branch, which moment, and which path the copies are expected to take. #### Discipline The criteria come out of the measurement and all four can be written briefly. **First:** force push is applied only to **unshared** branches. A branch being unshared is not the assumption "no one is working from it," it is a condition that can be checked. **Second:** the chains the main branch, the release branch, and tags point at are not rewritten. When a change needs to be undone, the path is not to rewrite but to write a new commit that does the reverse — established in the Introduction to Version Control course under the name revert — and because it never changes history, it concerns no copy. **Third:** if a shared review branch needs to be rewritten, the operation is announced beforehand and again afterward. The option that adds the expected value to the request is used; that option protects unseen work, not seen work. **Fourth:** the protection is placed not on the tool but on the other side. Repository servers have branch protection abilities and can be set to reject updates other than fast-forward. A rule sitting on the server side rather than the tool side stops it from being something that can be forgotten. Where the four criteria collapse in practice also has to be said. Most of the time the rule is known and applied; the accident comes from a branch being **assumed unshared**. A branch's name is personal, no one is assumed to be working from it, yet someone has fetched it during a review. The way to test the assumption is to look at the other side, not your own memory; if the branch's status on the other side is fetched, the assumption turns into a measurement. Every lost path has a recovery path, and three can be listed here. When the old chain on the other side is left without a reference, recovery is **in the copies**: a copy that still points at the old tip can write it back as a new branch; the overwritten side's backup is the side that pays the cost. A copy's stranded local work can be found from that copy's own reflog and moved onto the new chain. The old tip the person doing the rewrite lost is read from a backup branch or from the local reflog. Two of the three paths rest on the same mechanism, and that mechanism has a lifespan. #### Summary - The ordinary push accepts only a chain that **contains** the other side's tip; rewriting breaks this condition and is rejected, and the rejection is the repository's one built-in protection. - Force push removes the rejection: it moves the reference without asking its old value. The line is not given in full form here; on a shared branch it cannot be undone. - In a thousand-commit history, changing the last commit means **1** identity and **2** objects, changing the first means **1000** identities and **5640** objects — but **all seven of seven copies** have to resync in every case. - What the copies pay ranges between **14** and **39480** objects and is always **seven times** what the writer pays; the **9** stranded commits are independent of where the change was made. **The writer does not pay the cost.** - The one safe row the measurement gives is **unshared** branches: zero copies, zero stranded work. - The operation is not written to history and the copies do not learn until the next fetch; notifying is the rewriter's job, not the tool's. #### Next Step Two of this lesson's recovery paths looked at the same place: the local record holding the branch tip's old value. That record is not unlimited. In a rewritten history, the old chain's commits stay unreachable, stay recoverable for a while, then get cleaned up. The next lesson measures that window: how many commits stay reachable after a rewrite, how many can only be found from the record, and at what moment the recovery window closes. ### Reflog Source: https://academia.sh/en/courses/advanced-git/reflog Rewriting does not shorten history, it doubles it: in a thousand-commit repository, a transformation from the first commit leaves 1000 commits reachable and 1000 unreachable; in a six-entry log, when the retention horizon is 7, 30, 60, and 90 days, the recoverable commits are 3, 50, 500, and 1000. Two of the previous lesson's recovery paths looked at the same place: the local record holding the branch tip's old value. Bringing back an overwritten branch and recovering stranded local work both started from that record. The lesson ended with a warning there — that record has a lifespan. This lesson measures that lifespan. Two questions are asked, and they are separate. First: after a rewrite, what stays inside the repository, how many commits are reachable and how many are not. Second: how long do the unreachable ones stay recoverable, at what moment does the window close, and what is lost when it does. #### Being Unreachable Is Not Ceasing to Exist This course's first lesson said that rewriting is not a deletion but a duplication. The measurement turns that sentence into a number. When a thousand-commit history is rewritten, the new chain is also a thousand commits; the old chain's commits go nowhere. The repository now holds two chains: one reachable from the branch tip, the other pointed to by no reference at all. For an object to be **reachable** means it can be found by starting from a branch, a tag, or `HEAD` and following parent links. An unreachable object cannot be found by this walk; but it stays on disk, is read if its identity is given, and stays there until **garbage collection** runs. When rewriting finishes, the repository does not shrink, it grows. #### The Log Holds Movements, Not Commits The way to recover the unreachable chain is to know the old tip's identity. The tool records this on its own: every move of every reference is written to a local log. This is the **reflog**. The unit of the record is this lesson's most important distinction. The log holds not **commits** but **movements**. A rewrite that leaves five hundred commits unreachable does not write five hundred lines to the log; it writes one line. That line holds the tip at that moment, and walking backward from the tip reaches all five hundred commits. The number of entries and the number of recoverable commits are therefore not the same thing. The log has three limits. **It is local:** it does not travel to the other side, is not cloned, and one copy's log does not recover another copy. **It is per reference:** every branch has its own log, and `HEAD` additionally keeps a general one. **It is assumed to be indefinite but is not:** entries are cleaned up according to a retention horizon, and that horizon is a setting. ```text # taught command and example dump — not executed $ git reflog metrics 5b6c7d8 metrics@{0}: rebase (finish): refs/heads/metrics onto 9c4d7e1 2f8a1c3 metrics@{1}: commit: metrics: print threshold value 7e1b4d2 metrics@{2}: commit: metrics: split config file $ git branch recovery/metrics metrics@{1} ``` The second line is the entire recovery: a branch name is given to the old tip, and that chain becomes reachable again. The operation deletes nothing and moves no reference; this is why it can be written here in full form. Recovery paths are not destructive — what is destructive is the operation that made them necessary. There is a second tool for a case the log cannot show: the subcommand that checks the repository's integrity lists objects pointed to by no reference as **dangling**. If an entry has been cleaned but garbage collection has not run yet, recovery is done from there. That, too, is a read operation. The third limit's most common practical cost is this: **when a branch is deleted, its log is deleted with it.** Looking at a mistakenly deleted branch's own record is no longer possible at that point. In this case the remedy is `HEAD`'s log — the tips the working area stood on are additionally kept there, and the deleted branch's last tip usually appears in that list. Keeping the two logs separate is not a detail, it can be the only recovery path left. When objects are actually cleaned up also has to be written down. **Garbage collection** runs on its own; it is triggered during maintenance operations and can also be called manually. But for an unreachable object to actually be cleaned up, two conditions have to be met at once: the log entry pointing to it has to have fallen outside the horizon **and** collection has to have run. Until both conditions are met, the object stays in place. The horizon itself is not a single number either: the entries for reachable tips and the entries for unreachable tips can be set separately, and the latter is usually kept shorter. The measurement's assumptions: - **RH25** — The setup is the previous lessons' setup and is unchanged; the measurement is done on a **1000**-commit history. - **RH26** — Rewriting does not shorten history: the new chain is the same length, and the old chain's commits stay in the repository, unreachable. - **RH27** — The reflog records the branch tip's movements, not commits. An entry holds the tip at that moment; it represents the whole chain reachable from that tip. - **RH28** — The old tips the entries point to are nested suffixes; the recoverable set is determined by the entry that goes **furthest back**, not by the sum of the entries. - **RH29** — The retention horizon is a setting, not a fixed number. The measurement tries four values: **7, 30, 60, 90** days. - **RH30** — An entry outside the horizon is deleted, and only the commits reachable from that entry become collectible. The measurement does not count when garbage collection runs, it counts which set is **collectible**. - **RH31** — Two separate things are counted in this lesson and are not mixed: **reachability** is counted in commits, **cost** is counted in touched objects. The step unit is not used. - **RH32** — The resolution is **1/1000** in commits, **1/6** in log entries. #### Measurement ```python """Reflog: reachability and the recovery window. Part 1 - what stays in the repository after a rewrite. Part 2 - log entries: entry count and recoverable commits are separate things. Part 3 - when the retention horizon closes the window. """ SEED = 20260814 FILES = ("metrics.py", "report.py", "identity.py", "config.py", "document.md") POSITIONS = (1, 300, 501, 800, 951, 998) HORIZONS = (7, 30, 60, 90) def rng(seed): state = seed % 2147483646 + 1 def draw(n): nonlocal state state = (state * 48271) % 2147483647 return state % n return draw def history(n, seed=SEED): """n-commit linear history; each commit touches one file.""" draw, commits = rng(seed), [] for i in range(n): file = FILES[draw(5)] binary = draw(11) == 0 commits.append({"no": i + 1, "file": file, "binary": binary, "objects": 2 + (40 if binary else 0)}) return commits def rewrite(t, position): after = [x for x in t if x["no"] >= position] return len(after), sum(x["objects"] for x in after) def log(seed=SEED): """Every move of the branch tip writes an entry; an entry holds the day and the tip.""" draw, entries, day = rng(seed), [], 0 for i, position in enumerate(POSITIONS): day += 1 + draw(30) entries.append({"no": i + 1, "day": day, "position": position}) return entries def recoverable(t, entries): """The union of the tips the entries point to: the one going furthest back decides.""" if not entries: return 0, 0 return rewrite(t, min(e["position"] for e in entries)) T = history(1000) G = log() TODAY = G[-1]["day"] print(f"{'rewritten at':>20s} {'reachable':>13s} {'unreachable':>11s}" f" {'total in repo':>16s} {'unreachable objects':>17s}") for position in (998, 501, 300, 1): changed, touched = rewrite(T, position) print(f"{position:20d} {len(T):13d} {changed:11d} {len(T) + changed:16d} {touched:17d}") print() print(f"{'entry':>6s} {'day':>5s} {'old tip position':>17s}" f" {'reachable from entry':>20s}") for e in G: changed, _ = rewrite(T, e["position"]) print(f"{e['no']:6d} {e['day']:5d} {e['position']:17d} {changed:20d}") print(f"entry count {len(G)}, today day {TODAY}") print() print(f"{'retention horizon':>18s} {'entries left':>13s} {'recoverable commits':>21s}" f" {'recoverable objects':>20s} {'permanent loss':>14s}") total_changed, _ = recoverable(T, G) for horizon in HORIZONS: remaining = [e for e in G if TODAY - e["day"] <= horizon] changed, touched = recoverable(T, remaining) print(f"{horizon:18d} {len(remaining):13d} {changed:21d} {touched:20d}" f" {total_changed - changed:14d}") ``` ``` rewritten at reachable unreachable total in repo unreachable objects 998 1000 3 1003 6 501 1000 500 1500 3000 300 1000 701 1701 4162 1 1000 1000 2000 5640 entry day old tip position reachable from entry 1 11 1 1000 2 20 300 701 3 40 501 500 4 49 800 201 5 58 951 50 6 81 998 3 entry count 6, today day 81 retention horizon entries left recoverable commits recoverable objects permanent loss 7 1 3 6 997 30 2 50 260 950 60 4 500 3000 500 90 6 1000 5640 0 ``` #### The Repository Does Not Shrink The top table's second column is the same across all four rows: **1000**. No matter where the rewrite starts, the history reachable from the branch tip is a thousand commits. The third column varies: **3**, **500**, **701**, and **1000** stay unreachable. The fourth column is the sum of the two and carries the lesson's headline — after a transformation from the first commit, **2000** commits stand in the repository, half reachable, half not. This explains why the previous lesson's bulk transformation did not bring an immediate gain. Removing a binary asset from history does not lower the repository's size; it only moves that asset's objects into the unreachable set. The unreachable-objects column is **5640** for a transformation from the first commit, and that number is only freed once garbage collection runs. There is a delay between the gain and the operation, and that delay is at the same time **the recovery window.** The two are the same thing: the duration objects are kept is the duration a mistake can be undone. #### Entry Count vs. Recoverable Commits The middle table shows the log's six entries. One entry recovers three commits, another recovers a thousand. A log of **6** entries can recover **1000** commits; the ratio is one hundred sixty-seven commits per entry. This is why the log is a cheap mechanism: it keeps tips, not chains. There is a second piece of information in reading the entries. Every line also records the movement's **type**: it reports separately whether a commit was written, a rebase finished, or a branch tip was reset. This is the field to look at when searching for the tip to recover — what is being sought is usually the tip one entry before the one reporting the rewrite. The log holding identity and type together turns finding the right entry from a guessing game into something else. The same structure creates a trap too. Because the sets the entries cover are nested suffixes, the recoverable set is determined not by the **number** of entries but by which one **goes furthest back** among them. Even if five entries remain, if the deepest one is gone, the recoverable set drops to that deepest one among the remaining five. Seeing many lines in the log says nothing about the scope of recovery. #### As the Window Closes The bottom table shows how the window closes. When the retention horizon is **90** days, all six entries stand, all **1000** commits are recoverable, and the permanent loss is **0**. When the horizon drops to **60** days, four entries remain and recoverable commits fall to **500**: half of history can no longer be found from the log. At **30** days, two entries and **50** commits remain. At **7** days, a single entry is left, recoverable commits are **3**, permanent loss is **997**. The numbers' uneven jump is not the horizon's result, it is the result of the entries' distribution: the moment the deep entry from day eleven falls outside horizon sixty, five hundred commits become unreachable at once. The window does not close smoothly, it closes **in steps**. This is why the sentence "there is still a week left on the log" is not a guarantee — without knowing which entry drops on which day, how much stays recoverable cannot be known. The steps' locations are random too: the entries' days come from the work schedule, while the depth they cover comes from what work was done that day. There is no link between the two, which is why the deepest entry being the oldest entry is common but not guaranteed. The horizon itself is a setting and varies from repository to repository. What the measurement says is not to defend a particular number but that **the horizon directly determines the scope of recovery.** A longer horizon means more recovery, more objects kept; a shorter one means the opposite. It is a two-way trade-off, and both directions show in this table: the recoverable-objects column climbing from **6** to **5640** is matched by an equal drop in the objects that could be freed. #### Loss Paths and Recovery Paths Every loss path counted in this topic has a recovery path, and all four can be written together. When **the old form of a rewritten chain** is lost, recovery is in the log: a branch name is given to the old tip. If the window is closed, the dangling-object list is the second path. If **an asset removed in a bulk transformation** was removed by mistake, recovery is again in the log — but only in the repository that did the transformation. The separate-clone habit pays off here: if the actual repository stays untouched, the question of recovery never comes up. For **an overwritten remote branch**, recovery is not in the local log but **in the copies**. The overwritten side's backup is the side that pays the cost, and that backup has no retention horizon; recovery stays open as long as a copy still points at the old tip. For **an entry that has fallen outside the horizon**, there is no recovery path. The only remedy is prevention, and prevention is the backup branch opened before rewriting: a backup branch is a reference, references are not subject to the horizon, and it keeps reachability standing until deleted. The four paths share a common feature, and it is the lesson's conclusion: **recovery always happens on the side that holds the objects.** There is no central recovery location. Your own log recovers only your own repository; your log does not show a copy's lost work, and your log does not bring back a branch you overwrote. This is the mirror image of the previous lesson's cost distribution — the cost was distributed to the copies, and so is the ability to recover. From here a one-line habit follows: **before rewriting, where the tip is gets recorded.** A backup branch is the cheapest way to do this, and its cost is as much as the previous course measured — a single reference. Not needing the window at all, rather than asking how long it will stay open, is the shortest of all the paths counted here. #### Summary - Rewriting does not shorten history. At all four positions, reachable commits stay at **1000**; unreachable commits become **3 / 500 / 701 / 1000**, and the total in the repository climbs as high as **2000**. - Unreachable objects stay on disk until garbage collection runs; a transformation from the first commit leaves **5640** objects waiting to be freed. The delay in the gain and the recovery window are the same duration. - The reflog records **branch tip movements**, not commits: **6** entries can recover **1000** commits. Entry count does not indicate the scope of recovery. - The recoverable set is determined by the entry that goes furthest back; at retention horizons of **7 / 30 / 60 / 90** days, recoverable commits are **3 / 50 / 500 / 1000** and permanent loss is **997 / 950 / 500 / 0**. - The log is local, per reference, and subject to the horizon; a backup branch is outside all three and is the only permanent safeguard. - Recovery operations are read operations: giving a branch name to an old tip deletes nothing. #### Next Step Up to this point, history was **changed**, and every change's cost was counted in touched objects: 3002 in interactive mode, 5640 in bulk transformation, 39480 through the copies in force push, between 6 and 5640 in recovery depending on the window's width. The cost of changing is now known. What is not known is the opposite's cost: how much it costs to **ask a question** of the same history. The next topic measures that — in a thousand-commit history, how many steps does finding which commit introduced a defect take, and how many does that step count climb to when history quadruples in size. The unit of cost changes there too: not the object, but the **step**. ## Diagnostic Tools Finding the commit a regression was introduced in with binary search, tracing a line's history, and when client-side and server-side hooks fire. ### Bug Hunting with Bisect Source: https://academia.sh/en/courses/advanced-git/bug-hunting-with-bisect The cost of asking history a question is measured in steps: to find the same bug, linear search spends 37, 150, 750, and 3000 steps while binary search finishes in 6, 7, 9, and 11, and the grouping that cheapens the search coarsens the resolution of the answer. The previous lesson counted what remains after a rewrite: lost commits stay reachable in the reflog for a while, until garbage collection closes that window. Up to this point, history was an object that got **changed**, and each change's cost was measured in touched objects. This lesson turns the direction around. We do not touch history; we **ask it a question**: which commit introduced this bug? The unit of cost changes too: what gets counted is not the touched object but the **step** — the number of times the oracle is consulted. The answer already sits inside history; the matter is how many tries it takes to extract it. #### The One Assumption the Search Relies On Across history, a single question can be asked of every commit: is the bug present here? If the answer is "no" up to a point and "yes" from there on, history is **monotonic** with respect to this question, and what is being searched for is the first commit where the answer turns. This monotonicity is not a gift; it is an assumption, and it can break. If a bug is introduced, then accidentally masked by a later commit, then exposed again, the answer turns twice and the search points to the wrong commit. The question to ask before setting up a search: does this condition change exactly once across history? The algorithm itself, which halves a range over a monotonic predicate, was built in the Searching and Sorting topic of the Algorithms course, where the **algorithm's complexity** was measured. What is measured here is different: how many **steps over history** the same method spends when applied to a sequence of commits. The tool's subcommand ties this method to history and moves the working area to a commit at every step. #### The Loop the Tool Runs The search is set up with three declarations: starting the search, a commit where the bug is present, and a commit where it is not. The tool runs everything after that; at each step it moves to the commit in the middle of the range and waits for the answer. ```text # taught commands and example dump — not executed git bisect start git bisect bad HEAD git bisect good v-previous-release # the tool jumps to the middle of the range and reports the remaining candidate count: Bisecting: revisions left to test after this [a1b2c3d] metrics: read threshold from config # a response is given at each step git bisect good # or: git bisect bad # when the search ends, the tool's working area is restored git bisect reset ``` Choosing the two endpoints is the search's one free parameter, and it can be chosen wrong either way. Placed further back than necessary, the bug-free endpoint grows the candidate set and costs one extra doubling step. Placed further forward is worse: the bug ends up behind that endpoint, the search runs over a set that fails monotonicity, and it still points to a commit. A wrong choice carries no sign; the commit returned is reported the same way in both cases. Giving the answer by hand is not required. If a script tests for the bug, the search can be left to the tool; the script's exit code stands in for the oracle — zero means "no bug," nonzero means "bug present." This is decisive in practice: if the oracle cannot be automated, step count converts directly into human labor. The search also has an incomplete answer. If the commit in the middle of the range does not build, or testing makes no sense there, that commit cannot answer and the search has to **skip** it. A skipped commit does not halve the range; it only moves to its neighbor. An unbuildable window in the middle of history costs as many steps as it is wide, without halving. The measurement's assumptions: - **DT1** — History is generated from the shared fixture: a linear commit sequence at four scales, each commit touching one of five files. - **DT2** — The oracle is known because we generated the fixture ourselves, and it says the bug was introduced three-quarters of the way through history. The oracle is the same rule at every scale and is not changed across the measurement. - **DT3** — Monotonicity is assumed: the bug never closes after the commit that introduced it. The measurement does not count the case where monotonicity breaks. - **DT4** — A step is one consultation of the oracle. Building, testing, and moving the working area are all counted inside one step; steps are not distinguished from one another. - **DT5** — Linear search walks history from start to end and stops at the first commit where it sees the bug. This measures the search under the case the fixture produces, not under the best case. - **DT6** — In the second part, the only thing that changes is the rate at which commits are grouped: the same changes, each recorded in fewer commits. The fixture and the oracle stay the same. - **DT7** — The fixture is linear: the candidate set is the range between two numbers. How the candidate set is built in a forked history is outside the measurement and is discussed in the section after it. #### Measurement ```python """Cost of finding the buggy commit in history: number of steps. Part 1 - two search forms at four scales. Part 2 - the search getting cheaper coarsens the resolution of the answer. """ SEED = 20260814 FILES = ("metrics.py", "report.py", "identity.py", "config.py", "document.md") def rng(seed): d = seed % 2147483646 + 1 def r(n): nonlocal d d = (d * 48271) % 2147483647 return d % n return r def history(n, seed=SEED): """n-commit linear history; each commit touches one file.""" r, log = rng(seed), [] for i in range(n): file = FILES[r(5)] binary = r(11) == 0 log.append({"no": i + 1, "file": file, "binary": binary, "object": 2 + (40 if binary else 0)}) return log def buggy_commit(t): """Oracle: the commit that introduced the bug, known because we wrote the fixture.""" return len(t) * 3 // 4 def linear_search(t): target, step = buggy_commit(t), 0 for x in t: step += 1 if x["no"] >= target: return step return step def binary_search(t): target, low, high, step = buggy_commit(t), 1, len(t), 0 while low < high: mid = (low + high) // 2 step += 1 if mid >= target: high = mid else: low = mid + 1 return step def candidate_trace(t): """Remaining candidate commits after each step.""" target, low, high, trace = buggy_commit(t), 1, len(t), [] while low < high: mid = (low + high) // 2 if mid >= target: high = mid else: low = mid + 1 trace.append(high - low + 1) return trace def grouped(t, k): """Same changes, k of them recorded in a single commit.""" return [{"no": i + 1, "object": sum(x["object"] for x in t[i * k:(i + 1) * k])} for i in range(len(t) // k)] SCALES = (50, 200, 1000, 4000) print(f"{'commits':>7s} {'buggy':>8s} {'linear step':>12s} {'binary step':>11s}") for n in SCALES: t = history(n) print(f"{n:7d} {buggy_commit(t):8d} {linear_search(t):12d} " f"{binary_search(t):11d}") big = history(4000) print() print("remaining candidates after each step at 4000 commits:", candidate_trace(big)) print() print(f"{'commits':>7s} {'group':>7s} {'binary step':>11s} {'answer scope':>13s}") for k in (1, 2, 5, 10, 40): g = grouped(big, k) print(f"{len(g):7d} {k:7d} {binary_search(g):11d} {k:13d}") ``` ``` commits buggy linear step binary step 50 37 37 6 200 150 150 7 1000 750 750 9 4000 3000 3000 11 remaining candidates after each step at 4000 commits: [2000, 1000, 500, 250, 125, 62, 31, 15, 7, 3, 1] commits group binary step answer scope 4000 1 11 1 2000 2 10 2 800 5 9 5 400 10 8 10 100 40 6 40 ``` #### Where the Two Costs Diverge The top table scans the same history in two forms. As history grows **eightyfold**, from **50** to **4000**, linear search rises from **37** steps to **3000**; binary search rises from **6** steps to **11**. **In a history that grows eightyfold, binary search does not even double.** The linear search column is identical to the second column, and this is not a coincidence: since linear search stops at the first commit where it sees the bug, its step count equals the bug's **position in history**. Cost depends not on the length of history but on where the bug falls — early is cheap, late is expensive. This is not an advantage, since the position is unknown before the search; if it were, no search would be needed. The measurement fixes the bug at a constant position and compares the two forms under the same condition. The binary search column has no such dependence. The trace line shows why: the candidate count starts at **2000** and halves at every step down to **1**. Every answer eliminates half the range, and the eliminated half is never tested again. In a 4000-commit history, eleven answers bring the candidate set to a resolution of **1/4000**; at the four scales the resolution is **1/50**, **1/200**, **1/1000**, and **1/4000** — the measurement cannot show a smaller distinction, since history's own resolution is one commit. This distinction is the search-side counterpart of the course's measurement axis: cost does not come from the length of history but from which set a step touches. Linear search touches one commit per step; binary search touches half of the remaining candidates per step. #### The Candidate Set in a Forked History The measurement's fixture is linear: commits sit in a sequence and the range is between two numbers. A real history is most often not linear; the merge commit built in the Branching and Collaboration course takes history out of being a single line. The search still works here, but the meaning of the range changes. The candidate set is no longer the commits between two numbers; it is the commits reachable from the commit reported buggy and not reachable from the one reported bug-free. At every step the tool moves to the commit that splits this set most evenly — halving is preserved, only what gets halved changes. This has two consequences. First, a bug on a side branch does not show up on the main line until that branch merges, but the search finds it in the actual side-branch commit, not the merge commit, since that commit is also inside the candidate set. Second, step count is determined by the size of the reachable set, not the distance between the endpoints — close endpoints with a large merged-in set still cost that set's steps. #### The Real Cost of a Step Step count alone is not a measure of labor. Every step includes moving the working area to that commit, rebuilding it if needed, and testing it. Eleven steps in a component with a long build can cost more than three thousand steps in one that builds instantly. This is why the search's practical measure is step count × the duration of one step. Binary search lowers the first factor; the second is lowered by the test script's speed, not the search itself. If the script answers within seconds in an already-built environment, the search can be left entirely to the tool and eleven steps pass inside a single command. Done by hand, eleven steps mean eleven instances of human attention. Skipped commits are the one case that breaks this product. A commit that does not build produces no answer; the search shifts to its neighbor, and that shift is paid without halving. Unbuildable windows in a wide history push the search closer to linear. The remedy is not on the search side: staying buildable at every point is a matter of integration discipline, not the search tool. #### The Search's Record and Verifying the Answer The search consists of a series of decisions, and decisions can be given wrong. If the bug is missed at one step and marked "bug-free," the search proceeds in the direction that answer opens and ends up pointing to the wrong commit. The tool cannot notice this, because the only information it has is the answers it was given. Two habits guard against this. The first is keeping a record: the sequence of answers given is written to a log and can be replayed when needed. A step suspected wrong can be removed from the log and the search run again from that point; going all the way back to the start is not required. ```text # taught commands and example dump — not executed git bisect log > search-log.txt git bisect reset git bisect replay search-log.txt git bisect skip # on a commit that cannot be tested git bisect run ./test.sh # the oracle is the script's exit code ``` The second is that the commit found is a **claim**, not an answer. When the search ends, one commit is in hand, and whether it actually introduced the bug still needs testing separately: does the bug disappear when the change is reverted? This verification adds a twelfth step. Uncounted in the measurement, it is not skipped in practice, because the search's entire assurance rests on the correctness of the oracle's answers. The oracle's precision ties back to the same place. A criterion like "running slow" leaves the threshold undefined, and the same commit can receive two different answers on two attempts. If the oracle cannot reduce to a binary decision, monotonicity becomes untestable and the search loses its ground. The real work in setting up a search is therefore not the commands but the script that reduces the bug to an exit code. #### The Resolution of the Answer The bottom table redoes the search by grouping the same changes into commits of different sizes. When four thousand changes are grouped into batches of forty, history comes down to **100** commits and the search finishes in **6** steps instead of **11** — a gain of five steps. The cost is in the right column: the answer's scope rises from **1** to **40**. The search no longer says "it was introduced in this commit" but "in one of these forty changes," and the remaining work moves outside the search, into the commit, done by hand. The trade is not equal — step count falls logarithmically while the answer's coarseness grows linearly: five steps are gained for an answer forty times coarser. This is the squash measurement from the Branching and Collaboration course, seen in a different unit: there, the cleaned-up history's ability to answer questions dropped; here, the searched history's ability to give an answer grows coarser. The shared conclusion: a search's resolution is decided not at search time but at **commit time**. Making atomic commits is not a matter of style; it sets the resolution of the question asked later. #### Summary - The search rests on a single assumption: the presence of the bug changes exactly once across history. If monotonicity breaks, the search points to the wrong commit and gives no sign of it. - As history grows eightyfold, linear search rises from **37** steps to **3000**, binary search from **6** to **11**; cost comes not from the length of history but from the set a step touches. - Linear search's step count equals the bug's position in history; binary search's does not depend on position, only on length, and it brings the candidate set down to a resolution of **1/4000**. - Step count is half of the labor; the other half is the duration of one step, and what determines it is whether the oracle can be automated. - In a forked history, the candidate set is not a numeric range but a set defined by reachability from the two endpoints; the step count is determined by the size of that set. - When changes are grouped into batches of forty, the step count falls from **11** to **6**, but the answer's scope rises from **1** to **40**; the resolution of the search is decided at commit time. #### Next Step Binary search asks history a single question and takes the answer as a commit: the bug was introduced here. But this is often not the question being asked. There is no commit in hand, only a single **line**, and what is asked is why it was written there. The next lesson looks at this: how many steps it costs to find a line's last writer in the same history; why that cost does not grow the way binary search's step count does; and where the answer shifts when the last writer is not the person actually sought. ### Blame and Line History Source: https://academia.sh/en/courses/advanced-git/blame-and-line-history The cost of blame is the number of commits that touch a file, and in the same history it climbs to 8, 36, 209, and 787 steps; next to binary search's 11 steps this is a second growth form, and however long the scan runs, the answer stays bounded by the file's line count. The previous lesson asked history a single question and took the answer as a commit: the bug was introduced here. The search's cost was counted in steps, and as history grew eightyfold it rose from six to eleven. This lesson's question comes from somewhere else. There is no sign of a bug; there is a file being read and a **line** inside it that is not understood. What is asked is why that line was written there. **Blame** does not answer this question directly, but it gives the nearest proxy: which commit **last** wrote that line. Same history, same repository — but this time the cost grows in an entirely different form. #### The Question Blame Answers Blame produces one commit name for every line of a file. The name it produces is the **last** commit to write that line's current state. This distinction carries the whole lesson: the answer is not "who wrote this line" but "who last touched this line." The two are usually the same, but when they diverge, blame gives no sign of it. To produce this answer, the tool walks the file's history from end to start. At each step it looks at what that commit changed in the file, marks whichever still-unowned lines that commit wrote, and moves to the previous commit with the rest. The scan ends once every line is owned — or once commits touching the file run out. The earliest lines get owned late, near the start of history, and the most recently changed lines in the first few steps; the scan's order is therefore history's order, not the file's. #### Why It Cannot Be Halved What lowered cost in the previous lesson's search was monotonicity: a single predicate changed once across history, and every answer eliminated half the candidates. Blame has no such predicate. What is asked is not where a single threshold is crossed but where each of forty separate lines was last written. Forty separate questions have forty separate answers, and none narrows another's candidate set. Halving is closed off for a second reason too: the answer is required to be **complete**. Search could find one commit and stop, because what it looked for was a single point. Blame cannot stop; as long as a single line remains unowned, it has to keep scanning. Blame therefore falls into the class of questions asked of history that require a **full scan**. ```text # taught commands and example dump — not executed git blame config.py git blame -L 12,18 config.py # only a specific line range git blame -w config.py # ignore whitespace changes git blame --ignore-rev a1b2c3d config.py # format of the output: commit ID, role, date field, line number, the line itself a1b2c3d (author 12) threshold = config.read("threshold") 7d4e5f6 (reviewer 13) if threshold is None: a1b2c3d (author 14) threshold = DEFAULT ``` The line-range option does not lower cost, only shortens the output: the tool still has to walk the file's full history. The only thing that genuinely lowers cost is the file's share of history being small. Every line of the output carries three things: the ID of the commit that last wrote the line, the role and timestamp field from that commit's metadata, and the line itself. What the answer points to is a **commit**, not a person; the person's identity is read from the commit's metadata and is not a finding blame itself produces. The team-side counterpart of this distinction — to whom and how a finding gets communicated — is the subject of the Code Review and Team Process course; here, blame is a measurement tool, not an accountability tool. The measurement's assumptions: - **DT17** — History is generated from the shared fixture; every commit touches one of five files, and the file choice is the fixture itself. The fixture and the oracle are not changed in this lesson either. - **DT18** — The tracked file is `config.py`; the measurement tracks the same file at four scales. - **DT19** — A step is one commit that touches the tracked file. Commits that do not touch it are not counted. - **DT20** — The file is **40** lines, and the line count is fixed across scales; moving, splitting, and renaming the file are outside the measurement. - **DT21** — Every content-changing touch rewrites one of the forty lines. A touch affecting more than one line is not counted in the measurement; this is an assumption that **underestimates** the number of hidden commits. - **DT22** — In the second part, the only thing that changes is the addition of a **format transition**: the commit at the exact midpoint of history rewrites all forty of the file's lines without changing their content. - **DT23** — Blame's answer is one commit per line. Whether the tool tracks the line back through its earlier form is not part of the measurement; what is measured is the answer itself. #### Measurement ```python """The cost of blame: one step is one commit that touched the file. Part 1 - step counts of two searches over the same history. Part 2 - effect of a format transition on a line's history. """ SEED = 20260814 FILES = ("metrics.py", "report.py", "identity.py", "config.py", "document.md") TRACKED, LINES, LINE_SEED = "config.py", 40, 20260815 def rng(seed): d = seed % 2147483646 + 1 def r(n): nonlocal d d = (d * 48271) % 2147483647 return d % n return r def history(n, seed=SEED): """n-commit linear history; each commit touches one file.""" r, log = rng(seed), [] for i in range(n): file = FILES[r(5)] binary = r(11) == 0 log.append({"no": i + 1, "file": file, "binary": binary, "object": 2 + (40 if binary else 0)}) return log def buggy_commit(t): """Oracle: the commit that introduced the bug, known because we wrote the fixture.""" return len(t) * 3 // 4 def binary_search(t): target, low, high, step = buggy_commit(t), 1, len(t), 0 while low < high: mid = (low + high) // 2 step += 1 if mid >= target: high = mid else: low = mid + 1 return step def blame(t, file): """Line history of a file: commits touching that file are scanned.""" return sum(1 for x in t if x["file"] == file) def last_writers(t, file, format_position=None): """Each line's last writer; a format transition rewrites every line.""" r, writer = rng(LINE_SEED), [0] * LINES for x in t: if x["no"] == format_position: writer = [x["no"]] * LINES if x["file"] == file: writer[r(LINES)] = x["no"] return writer SCALES = (50, 200, 1000, 4000) print(f"{'commits':>7s} {'binary step':>11s} {'touching commits':>17s} " f"{'shown':>6s} {'hidden':>6s}") for n in SCALES: t = history(n) scan = blame(t, TRACKED) distinct = len(set(last_writers(t, TRACKED)) - {0}) print(f"{n:7d} {binary_search(t):11d} {scan:17d} {distinct:6d} " f"{scan - distinct:6d}") print() print(f"{'commits':>7s} {'touches after transition':>26s} " f"{'lines showing transition':>26s} {'distinct commits':>17s}") for n in SCALES: t = history(n) y = last_writers(t, TRACKED, format_position=n // 2) after = sum(1 for x in t if x["file"] == TRACKED and x["no"] > n // 2) print(f"{n:7d} {after:26d} {sum(v == n // 2 for v in y):26d} " f"{len(set(y) - {0}):17d}") ``` ``` commits binary step touching commits shown hidden 50 6 8 8 0 200 7 36 25 11 1000 9 209 40 169 4000 11 787 40 747 commits touches after transition lines showing transition distinct commits 50 3 37 4 200 16 25 16 1000 120 3 38 4000 375 0 40 ``` #### Same Repository, Two Different Growth Forms The first two columns place two questions about the same history side by side. As history grows from **50** to **4000**, binary search's step count rises from **6** to **11**; in the same history, blaming a single file rises from **8** steps to **787**. In a history that grows eightyfold, the first question does not even double in cost; the second grows **ninety-eight times** more expensive. The difference does not come from the tools' speed; it comes from the **shape of the question asked**. Binary search looks for a single threshold, and every answer eliminates half the candidates. Blame looks for the owner of forty separate lines, and no answer narrows another's candidate set, while the answer is also required to be complete. Same repository, same history, two different growth forms — and what separates them is not how the tool is implemented but the question itself. This is the second place where the course's measurement axis gets paid: cost comes not from the length of history but from **the set an operation touches**. Binary search eliminates half of history in one step; blame has to touch the file's entire share of history. That share is one-fifth in the fixture, and the ratio **787 ÷ 4000** shows exactly this. The same thing sets the measurement's resolution. At the four scales, the smallest distinguishable difference is one commit — **1/50**, **1/200**, **1/1000**, and **1/4000**, respectively. Blame's rise from eight steps to seven hundred eighty-seven is far above this band and the measurement carries it comfortably; a few-step gap between two files' shares — in the fixture, the five files' shares sit close together — cannot be defended with this resolution. What the measurement says is not which file is more expensive but that **cost grows in the same proportion as the share does**. The practical consequence is this: the way to speed up blame is not to change the tool but to **shrink the file**. If a file touched five hundred times is split in two, each half's blame finishes in roughly two hundred fifty steps on its own. File size is as much a diagnostic-cost problem as it is a readability problem. #### Shown and Hidden The last two columns say something more unsettling. In the four-thousand-commit history, blame scans **787** commits, but only **40** distinct commits show up in its answer; **747** commits touched the file and show up nowhere in the answer. At the thousand-commit scale, the hidden count is **169**; at two hundred, **11**; at fifty, **0**. The shown column stopping at **40** is a ceiling, and where it comes from is clear: the file is forty lines, blame gives one name per line, so it can produce at most forty distinct names. As the scan grows longer, cost grows but the answer's size does not. At fifty commits, the scan is eight and the answer is eight — one to one. At four thousand, the scan is seven hundred eighty-seven and the answer is forty — one name for every twenty steps. This means blame is a **summary**. It is not the file's complete line history but that history's **top layer**. How many times a line changed, how many separate commits touched it, what its earlier forms were — none of this is in blame's answer, and its absence is not marked either. Whoever reads the answer has no way of knowing about the seven hundred forty-seven hidden commits. The full line history is a separate question and is asked with a separate subcommand: a query for a range's complete history returns every commit that touched that range, in order. Its cost is the same scan as blame's, but the answer's size no longer hits a ceiling. This is the criterion for the right tool choice: is the top layer wanted, or all of them? #### The Moved Line The measurement assumes the file's forty lines stay in place (**DT20**). In a real history, lines move: a function is pulled into another file, a block shifts to another spot in the same file. To blame, this counts as the line being **rewritten**. The commit that moved it becomes the last writer of every moved line, and the line's real history stays on the far side of a file boundary. The tool can partly undo this. One setting tracks lines that moved within the same commit; another tracks movement between files. Both widen the scan, because now not just the tracked file's history but the history of the files a line might have come from also has to be walked. Cost therefore rises above what we counted in the measurement: the scan no longer stays within a single file's share. A practical rule follows from this. If a move and a change are combined in the same commit, the line's history is **cut off** there — the tool can only reliably tell how much of a moved line survived at its new location if the two are separated. Recording a move in its own commit is the cheapest way to leave an answer for a question that will be asked later: a pure move commit can be written into the skip list; a commit that combines a move with a change cannot. #### What a Format Transition Costs The bottom table places a single format transition at the midpoint of history: a commit that does not change content but rewrites all forty of the file's lines. The result changes sharply with scale. At the fifty-commit scale, the file is touched only **3** times after the transition; blame's answer shows the format transition on **37** lines, leaving **4** distinct commits. The file's visible authorship has collapsed onto a single commit. At four thousand commits, there are **375** touches after the transition and **0** lines still showing it; the answer again has **40** distinct commits. The pattern is this: the damage from a format transition is set not by scale but by **how many touches follow it**. On a file that has not had enough work pass through it since, the transition pulls blame's entire answer onto itself. And the damage lands hardest exactly where the file is touched least — the lines blame will be consulted about the most and understood the least. There are two counters to this, and both stay inside blame itself. The first is ignoring whitespace changes: if only indentation or alignment changed, the line counts as unowned and the scan moves to the previous commit. The second is marking specific commits to be skipped during the scan; bulk format transitions get written to a list, and blame behaves as if it never saw those commits. Both are settings of the tool and do not change history — the first problem in this course solved without touching history. #### Summary - Blame produces one name per line, and that name is the commit that **last wrote** the line; when "who wrote it" and "who last touched it" diverge, the tool gives no sign of it. - The unit of cost is a step, and a step is a commit that touches the tracked file; as history grows from **50** to **4000**, cost rises from **8** to **787**. - In the same history, binary search rises from **6** steps to **11**; the shape of the question asked alone produces two different growth forms. - As the scan grows longer, the answer does not grow: at four thousand commits, **787** are scanned, **40** distinct commits appear in the answer, and **747** are hidden. Blame is a summary; the full line history is a separate query. - To blame, a moved line counts as rewritten; recording a move in its own commit is the cheapest way to preserve the line's history. - A single format transition at the midpoint of history pulls the answer for **37** lines onto itself in a file with little work since; ignoring whitespace and a commit-skip list undo this without touching history. #### Next Step Both of these lessons asked history a question **after the fact**. The bug had already been introduced, the line had already been written; the tools only read a record that had already happened and paid the cost on the reading side. But every commit passes through several separate moments before it enters history: from the staging area to a commit, from a commit to a push, from a push to the other side's acceptance. The next lesson looks at these moments — at which moment a check can step in, which commit can still be blocked at that moment, and who ends up paying the cost when the block comes late. ### Hooks Source: https://academia.sh/en/courses/advanced-git/hooks A hook is an event point, and the moment the block is set determines the cost: a change stopped before commit touches zero objects, before push ten commits and 140 objects, six hundred commits after acceptance 601 commits, 3522 objects, and all seven of the seven copies. The previous two lessons asked history a question after the fact: the bug had already been introduced, the line had already been written. The tools read a record that had already happened and paid the cost on the reading side. A change, however, does not enter history in one step. It first sits in the working area, then moves to the staging area, then becomes a commit, then gets pushed, then the other side accepts it. Each of these moments is a **gate**, and a check can be placed at any gate. A **hook** is the program the tool runs at one of these moments; if its exit code is nonzero, the operation stops at some gates. This lesson's question is not what a hook does: it is how cost changes depending on **which gate** a block is placed at, and who pays that cost. #### A Hook Is an Event Point A hook is nothing more than an executable file sitting under a fixed name inside the repository's own directory. When the tool reaches a specific moment of a specific operation, it looks for a file at that name, runs it if present, and reads the exit code. There is no plugin mechanism, no registry, no configuration language behind this; the contract is only the file's name, its being executable, and its exit code. The use of hooks to trigger a continuous integration pipeline was built in the Continuous Integration and Delivery course, where a hook was a **trigger** and what was discussed was the pipeline itself. That discussion is not repeated here. In this lesson, a hook is a **local event point**, and the only thing measured is which commit can still be stopped at that point. There are two classes, and the difference between them determines the lesson's conclusion. **Client-side hooks** run in the developer's own repository, at the moments before a commit and before a push. **Server-side hooks** run on the other side, before pushed changes are accepted. The first does not travel in the repository's history — cloning does not bring hooks along — the second always runs, independent of the repository's history. #### The Order of the Gates The moments and the hooks that run at them follow the order below. Only some of them carry the authority to block; the rest report after the fact. ```text # taught file names and example dump — not executed pre-commit before the commit is written — can block commit-msg after the message is written — can block post-commit after the commit is written — cannot block pre-push before the push starts — can block pre-receive before the other side accepts — can block update separately for each ref — can block post-receive after acceptance completes — cannot block # .git/hooks/pre-commit #!/bin/sh if git diff --cached --name-only | grep -q '\.bin$' then echo "binary asset in staging area; commit blocked" >&2 exit 1 fi exit 0 # example dump $ git commit -m "presentation refreshed" binary asset in staging area; commit blocked ``` Two distinctions follow. `post-` prefixed hooks cannot change a decision; the work is already done and the hook only reports. And every blocking hook on the client side can be skipped with an option given to the command — this is not a flaw but a design choice: a local hook exists for the developer's own convenience, not as an enforcement mechanism. The measurement's assumptions: - **DT33** — History is generated from the shared fixture and is a thousand commits; the fixture and the oracle are not changed. - **DT34** — The change meant to be blocked is at commit **400**. A single change is tracked; how many changes are blocked is not counted. - **DT35** — The unit of cost is a **touched object**. Removing a commit from history changes the ID of every commit after it; touched objects are the sum of the objects carried by the commits whose IDs change. - **DT36** — Pushes happen in **batches of ten**. The before-push and server-side hooks therefore fire nine commits later. - **DT37** — Being noticed after acceptance is measured at two points: **100** commits later and **600** commits later. - **DT38** — The repository has **seven** copies. An unpublished change concerns no copy; undoing a published change concerns all of them. - **DT39** — In the second part, the only thing that changes is scale. The number of times the hook runs and the number of commits it inspects are counted separately; the hook's own duration is not measured. #### Measurement ```python """The moment of the hook: which commit can still be blocked at that moment. Part 1 - the cost of undoing when the gate is placed at five separate moments. Part 2 - the hook's own cost: how many times it runs, how many commits it inspects. """ SEED = 20260814 FILES = ("metrics.py", "report.py", "identity.py", "config.py", "document.md") COPIES, POSITION, BATCH = 7, 400, 10 def rng(seed): d = seed % 2147483646 + 1 def r(n): nonlocal d d = (d * 48271) % 2147483647 return d % n return r def history(n, seed=SEED): """n-commit linear history; each commit touches one file.""" r, log = rng(seed), [] for i in range(n): file = FILES[r(5)] binary = r(11) == 0 log.append({"no": i + 1, "file": file, "binary": binary, "object": 2 + (40 if binary else 0)}) return log def rewrite(t, position, noticed_at): """The set to be rewritten if the commit at position is noticed at noticed_at.""" if noticed_at is None: return 0, 0 between = [x for x in t if position <= x["no"] <= noticed_at] return len(between), sum(x["object"] for x in between) def resyncing(copy_count, position, t, noticed_at): """Who has to resync the rewritten history.""" changed, _ = rewrite(t, position, noticed_at) return copy_count if changed else 0 EVENTS = ( ("before commit", "yes", None, 0), ("before push", "yes", POSITION + BATCH - 1, 0), ("at server acceptance", "yes", POSITION + BATCH - 1, 0), ("100 commits after acceptance", "no", POSITION + 100, COPIES), ("600 commits after acceptance", "no", POSITION + 600, COPIES), ) t = history(1000) print(f"{'point the block is set':>28s} {'can block':>10s} " f"{'rewritten':>10s} {'objects touched':>16s} {'resyncing copies':>17s}") for name, blocks, noticed_at, copies in EVENTS: commits, objects = rewrite(t, POSITION, noticed_at) print(f"{name:>28s} {blocks:>10s} {commits:10d} {objects:16d} " f"{resyncing(copies, POSITION, t, noticed_at):17d}") print() print(f"{'commits':>7s} {'before commit: runs':>20s} {'before push: runs':>18s} " f"{'commits inspected':>18s}") for n in (50, 200, 1000, 4000): print(f"{n:7d} {n:20d} {n // BATCH:18d} {n:18d}") ``` ``` point the block is set can block rewritten objects touched resyncing copies before commit yes 0 0 0 before push yes 10 140 0 at server acceptance yes 10 140 0 100 commits after acceptance no 101 522 7 600 commits after acceptance no 601 3522 7 commits before commit: runs before push: runs commits inspected 50 50 5 50 200 200 20 200 1000 1000 100 1000 4000 4000 400 4000 ``` #### The Moment the Block Is Set The first row of the top table is the measurement's anchor. A change stopped before the commit is written never enters history at all: rewritten commits **0**, touched objects **0**, copies that must resync **0**. The change is still in the working area, and fixing it is nothing more than editing a file. In the second and third rows, the commit has been written and nine more commits have piled on top. Undoing it costs **10** commits and **140** touched objects. Since it is not yet published, resyncing copies stays at **0**; the cost sits entirely on the writer, and history can be rewritten on their own machine. In the fourth and fifth rows, the gate has already closed. Noticed a hundred commits later, undoing changes the ID of **101** commits and touches **522** objects; six hundred commits later, **601** commits and **3522** objects. And resyncing copies is no longer **0** but **7**. The jump here is not in the objects column but in the rightmost one: as touched objects grow **twenty-five times**, who does the work shifts from one person to all seven copies. The measurement's resolution in the thousand-commit set is **1/1000**; the smallest distinguishable difference is a single commit. The **10**-commit gap between the first and second rows is ten times this band and is carried comfortably. The **500**-commit gap between the fourth and fifth rows is half the scale; the one thing this resolution cannot defend is drawing fine distinctions of a few commits between gates. The pattern is this: the cost of a block grows not with **which** moment it is placed at but with **how long it was not placed**. The gap between gates is a few seconds or a few hours; the gap between costs runs from zero to three thousand five hundred twenty-two. And past a certain point, this is no longer a cost the writer alone can pay. #### Why After Acceptance Is a Separate Class Removing an accepted commit from history is rewriting history, and that operation was established with its own cost in this course's first topic: it is not reversible, it requires every copy to resync, and its command is never written in full. The only thing added here is **why** that cost was incurred: the block was set at a gate too late. A published error has an answer without rewriting history too: adding a new commit that **reverts** the change. Revert does not break history, does not change IDs, does not force copies to resync — it is the safe path and is tried first. But revert fixes the record; it does not delete the object. A leaked secret or a large asset added by mistake keeps standing among history's objects even after a revert, and keeps going to everyone who clones. This is where the server-side hook's real justification comes from. A leaked secret is not recovered by fixing the repository but by **invalidating the value itself**; cleaning up history is only a second task that comes after that. The acceptance gate is the last moment at which **neither** of these two tasks is yet required. #### The Hook's Own Cost The bottom table shows that placing the block early is not free. In a four-thousand-commit history, the before-commit hook runs **4000** times; the before-push hook, because of batches of ten, runs **400** times. The number of commits inspected is **4000** in both cases — the same work, split across a tenfold-different number of runs. The cost of this split is the **fixed cost of every run**: starting the process, loading the check tool, scanning the directory. Fixed cost multiplies by the number of runs, not the number of commits inspected; the early gate therefore pays ten times the fixed cost. What it earns in return is **feedback distance**: the before-commit hook stops a change the moment it is produced, the before-push hook at worst nine commits later. Where the two gates stand in the measurement is now clear. An early gate means cheap fixes and expensive operation; a late gate means the reverse. The common arrangement splits the two: cheap checks that take seconds go on the before-commit gate, expensive checks that run long go on the before-push gate, and checks that require enforcement are repeated at the acceptance gate in either case. #### What a Hook Can See Which moment a hook runs at also determines **what it can see**. The before-commit hook sees the staging area's contents, not the working area's uncommitted changes. When this distinction is missed, a hook checks the wrong thing: a check that reads files from the working area also looks at unstaged lines in a partially staged change, and the staged content passes through without ever being checked while the commit goes through. The correct hook reads the staging area, not the file system. The two hooks at the acceptance gate also differ in **resolution**. One runs once for the whole pushed batch and gives its decision to the entire batch: either all of it goes in or none of it does. The other runs separately for each ref and can reject one branch while accepting another. The measurement's batches-of-ten assumption pays off here — a gate that rejects the whole batch turns away nine bug-free commits along with the one buggy commit. The nine rejected commits are not lost; they stay in the writer's own history — but the batch has to be reorganized, and this is the second cost the acceptance gate charges beyond the early gates. Two rules for the hook itself follow from this. A hook must be **fast**: a check that takes seconds on every run turns into hours across four thousand runs and eventually starts getting skipped. And a hook must be **deterministic**: a check that gives two different answers to the same content lets through as much as it blocks and carries no assurance at all. #### Whose Hook Runs The final distinction is the lesson's conclusion. Client-side hooks do not travel in the repository's history; cloning does not bring them, no commit can publish them. Every team member has to install them separately in their own repository, and whoever installs one can also skip it with an option. A client-side hook is therefore a check of **unknown installation rate** — it is not known in how many of the seven copies it is even set up. There is a partial answer to this non-travel. Hooks can be written to an ordinary tracked directory inside the repository, and the tool's hook directory can be pointed there with a setting; the hooks themselves then travel with history and the team shares the same check. But setting up that pointer is still a separate step on every copy, and skippability does not change. What travels is the hook's **content**, not its **enforcement** — how many of the seven copies have it active is still unknown. A server-side hook is the reverse: it sits in one place, runs the same way for everyone, and cannot be skipped. Every rule that requires enforcement eventually has to be written there. But the acceptance gate is also the **last** gate that can still block, and by the time it is reached, the commit has already been written, has already entered the writer's own history, and perhaps nine more commits have already piled on top. The measurement's third row shows exactly this: the acceptance gate can block, but even when it does, undoing it costs **10** commits and **140** objects. The conclusion is that both of these are true at once: the one gate with enforcement is not the gate that zeroes the cost, and the gate that zeroes the cost has no enforcement. A client-side hook is therefore not an assurance but a **convenience** — a convenience that tells the writer, five hundred twenty-two objects early, something the acceptance gate would have rejected anyway. #### Summary - A hook is an event point: a program the tool runs at a specific moment and an exit code it reads. `post-` prefixed hooks report; they do not change a decision. - If the block is set before commit, the change never enters history: **0** commits, **0** touched objects. Set before push, it is **10** commits and **140** objects, with **0** copies since it is not yet published. - Noticed six hundred commits after acceptance, undoing it touches **601** commits and **3522** objects, and all **seven of seven** copies must resync; the cost leaves the writer alone and spreads to everyone. - Revert answers a published error without breaking history but does not delete the object; the answer to a leaked secret is first invalidating the value itself. - The early gate runs **4000** times across four thousand commits, the push gate **400** times; commits inspected are **4000** in both. The early gate pays ten times the fixed cost, and in return brings feedback distance down from nine commits to zero. - A client-side hook does not travel and can be skipped; a server-side hook cannot be skipped but is the latest gate; the gate with enforcement is not the gate that zeroes the cost. #### Next Step Throughout this topic, the object count always stayed in the background of the measurement: counting the search's steps, counting blame's scan, counting the cost of a hook set too late — in every case history sat inside a single repository, and the objects were that repository's own. The next topic brings this background to the front: what if object count is not the measurement's ground but **the work itself**? When a repository absorbs another repository's history, or when every version of a single file accumulates as a separate object, the background we have counted as fixed up to now turns into the thing that has to be measured. ## Large Repositories Embedding dependent repositories and pinning versions, the weight binary assets bring to history, working in parallel on the same repository, line-ending and merge-driver settings, and the scale trade-off between a monorepo and a polyrepo. ### Submodules Source: https://academia.sh/en/courses/advanced-git/submodules A submodule writes an identity to the main repository, not a history: three submodules bind one commit to four separate histories, 76 objects written by 38 commits bring in 378 submodule commits, and a recursive clone climbs from 920 objects to 4540. Up to this point, history was inside **a single repository**. Bisect, blame, and rewriting were measured; object count stood behind these measurements — the variable that determined cost, not the thing being measured itself. In this topic, the background moves to the foreground. When a repository grows and pieces too large to fit in a single history start being carried, the question asked changes: **what happens when the object count itself becomes the work?** The first case is one repository containing another repository's history inside it. This lesson's question is not how a submodule is set up — setup is two commands. The question is: how many separate histories does a single identity written to the main repository bind a commit in that repository to, and how does clone cost respond to that? #### What the Main Repository Records A **submodule** is another repository that stands in a repository's working tree and has its own history. The main repository does not copy its files into its own object store; all it records is **which commit** the submodule stands at. This record is called the **pinned version** and it stands in the main repository's tree as a single directory entry. Two files are the whole job. `.gitmodules` declares the submodule's name and where it is fetched from; the tree entry holds the pinned version's identity. The first is a tracked text file, the second is a field rewritten at every commit. ```text # taught commands and file content — example dump, not executed git submodule add components/metrics-core git submodule update --init --recursive git clone --recurse-submodules # .gitmodules [submodule "components/metrics-core"] path = components/metrics-core url = ``` The `--recurse-submodules` option does not finish cloning with the main repository: it clones each submodule separately as well. A clone without the option leaves submodule directories **empty** — the main repository's history arrives complete, but the working tree does not work. `update --init` does the same job afterward. #### The Pinned Version's Two Faces The pinned version is **cheap** inside the main repository and **expensive** outside it, and the lesson's entire tension is here. The cheap face: advancing the submodule is a single commit in the main repository. The identity in the tree entry changes, and a new tree plus a blob carrying the message are written — **two objects**. However many commits have piled up in the submodule during that interval, the object count written to the main repository is independent of it. The expensive face: those two objects **bind** a commit in the submodule's history to the main repository's history. Reproducing a commit in the main repository now requires reaching not one history, but the main repository plus every submodule's history at once. In a repository with three submodules, building a single commit's tree requires reading **four** identities from **four** separate histories. This second face is something measurable, and the measurement separates two questions: how many objects are **written** in the main repository, and how much history that write **binds**. #### It Tracks a Commit, Not a Branch The pinned version is a **commit identity**, not a branch name. This has a direct consequence: once a submodule directory is cloned, it does not stand on any branch — it is in a detached-HEAD state. Detached HEAD was established in the Branching and Collaboration course in the context of a commit falling outside history; here it is not an accident, it is the submodule's **normal** state. The main repository cannot pin a branch, because a branch moves, and a moving reference cannot be a pinned version. ```text # taught commands — example dump, not executed git -C components/metrics-core checkout main git -C components/metrics-core pull git add components/metrics-core git commit -m "Advance metrics-core pinned version" git submodule update --remote components/metrics-core git submodule status ``` Advancing is four steps, and all four are done by hand: a branch is switched to inside the submodule, new commits are fetched, and **in the main repository** the submodule directory is staged and committed. If the third step is skipped, the submodule directory shows a new commit but the main repository still records the old one; `status` output reports this, history does not. The `--remote` option automates the first two steps according to the branch written in `.gitmodules`, but the third step is **still done by whoever writes it**: there is no advance that is not written to the main repository. This is where a submodule differs from a dependency manager. A dependency manager declares a range and resolves it itself; a submodule declares a single identity and there is no such thing as resolution. In exchange, a submodule brings the full history: every commit up to the pinned version is there, readable, and searchable. #### The Measurement's Assumptions - **LR1** — The main repository is the shared setup's **200**-commit scale; each of the three submodules is an independent history produced from the same setup with a separate seed. Submodule histories share no object with the main repository. - **LR2** — The main-repository commits that advance the pinned version are picked from a separate generator; each advance consumes **between 1 and 20** submodule commits and does not go past the submodule's end. - **LR3** — The commit that advances the pinned version writes **2 objects** to the main repository: the new tree and the blob carrying the message. This number is independent of how many commits were passed in the submodule. - **LR4** — "Covered commits" is the number of submodule-history commits consumed up to the main repository's tip; it is not visible in the main repository's history, but is required to reproduce the state at the main repository's tip. - **LR5** — "Ids needed" is the number of identities that must be read to build a single main-repository commit's tree: the main repository's own identity plus every submodule's pinned version. - **LR6** — A recursive clone fetches every submodule's **full history**. Cost is counted in the **touched object** unit; network, wall-clock time, and disk are outside the measurement. #### Measurement ```python """Submodules: the history the pinned version binds, and the clone cost. Part 1 - each submodule's advance count and clone objects. Part 2 - as submodule count grows: bound history and recursive clone. """ SEED = 20260814 FILES = ("metrics.py", "report.py", "identity.py", "config.py", "document.md") MAIN_SCALE = 200 SUBMODULES = (("metrics-core", 300, SEED + 1), ("report-format", 150, SEED + 2), ("identity-library", 80, SEED + 3)) def rng(seed): d = seed % 2147483646 + 1 def draw(n): nonlocal d d = (d * 48271) % 2147483647 return d % n return draw def history(n, seed=SEED): """n-commit linear history; each commit touches one file.""" draw, commits = rng(seed), [] for i in range(n): file = FILES[draw(5)] binary = draw(11) == 0 commits.append({"no": i + 1, "file": file, "binary": binary, "objects": 2 + (40 if binary else 0)}) return commits def clone_cost(t, include_binary=True): return sum(x["objects"] for x in t if include_binary or not x["binary"]) def advances(main, sub, seed): """Which main-repository commits advance the pinned version.""" draw, record, position = rng(seed), [], 0 for x in main: advance = draw(12) == 0 step = 1 + draw(20) if advance and position < len(sub): position = min(position + step, len(sub)) record.append({"main_no": x["no"], "pinned": position}) return record main = history(MAIN_SCALE) print(f"main repository {len(main)} commits, clone objects {clone_cost(main)}, " f"without binaries {clone_cost(main, False)}") print() print("submodule commits advances covered commits objects in main" " clone objects") records = [] for name, n, seed in SUBMODULES: sub = history(n, seed) a = advances(main, sub, seed + 100) records.append((name, sub, a)) print(f" {name:18s} {n:6d} {len(a):10d} {a[-1]['pinned']:16d} " f"{len(a) * 2:17d} {clone_cost(sub):13d}") print() print("submodule count bound history ids needed main clone objects" " recursive clone objects ratio") for k in range(len(SUBMODULES) + 1): recursive = clone_cost(main) + sum(clone_cost(a) for _, a, _ in records[:k]) print(f"{k:16d} {k + 1:14d} {k + 1:11d} {clone_cost(main):18d} " f"{recursive:25d} {recursive / clone_cost(main):5.2f}") print() total_advances = sum(len(a) for _, _, a in records) total_covered = sum(a[-1]["pinned"] for _, _, a in records) print(f"main repository commits that advance a pinned version {total_advances}, " f"objects written to main {total_advances * 2}, " f"submodule commits bound {total_covered}") print(f"main repository history shows {len(main)} commits; " f"commits covered {len(main) + total_covered}") ``` ``` main repository 200 commits, clone objects 920, without binaries 374 submodule commits advances covered commits objects in main clone objects metrics-core 300 18 161 36 2200 report-format 150 14 137 28 980 identity-library 80 6 80 12 440 submodule count bound history ids needed main clone objects recursive clone objects ratio 0 1 1 920 920 1.00 1 2 2 920 3120 3.39 2 3 3 920 4100 4.46 3 4 4 920 4540 4.93 main repository commits that advance a pinned version 38, objects written to main 76, submodule commits bound 378 main repository history shows 200 commits; commits covered 578 ``` #### The Bound History The last two lines give the lesson's core. In the main repository, **38** commits advance a pinned version, and these commits write **76** objects total to the main repository — a small share of a two-hundred-commit history's **920** objects. The same 38 commits bind **378** commits from the submodule histories to the main repository. The ratio reads directly: every **2** objects written to the main repository bring in an average of **ten** submodule commits. This is the asymmetry the lesson measures. Someone looking at the main repository's history sees **200** commits; the number of commits needed to reproduce the state at the main repository's tip is **578**. The visible history and the covered history are not the same thing, and the gap is not written anywhere in the main repository. This asymmetry has a diagnostic consequence. A search run in the main repository scans only the main repository's **200** commits; the submodules' **378** covered commits are outside that search. If a defect went in through a submodule, a search in the main repository **cannot find it** — the narrowest range it can find is the pinning commit that brought the defect in. The search's second round has to be rebuilt in the submodule's own history. Even if the range narrowing in the main repository is two commits, the number of submodule commits packed between those two commits can climb as high as **twenty**. The second table's first two columns say the same thing in terms of identity. With no submodules, building a commit's tree requires **1** identity and looks at **1** history. With three submodules, this number climbs to **4**. All four have to be found: if one of the submodules is unreachable, the commit in the main repository stands complete but **cannot be reproduced**. The main repository's history is not corrupted; the history that is missing is a different one. #### Adding Up the Clone Cost The last three columns of the lower table give the clone cost, and the unit is **touched object**. The main repository alone is **920** objects. The submodules bring **2200**, **980**, and **440** objects respectively. Once all three are added, the recursive clone climbs to **4540** objects — **4.93** times the main-repository clone. The main repository's history did not grow by even one commit in the process. The largest contribution comes from the first submodule, and the reason is written in the table: its **300**-commit history is larger than the main repository's. The submodule's length has nothing to do with the main repository's length; the main repository does not look at the submodule's size when it writes the identity that pins it. Nothing stands in the way of a setup where the main repository is small and the clone cost is large. The ratio column on the right does not climb linearly: **3.39**, then **4.46**, then **4.93**. The increase shrinks at each step because the submodules being added are progressively shorter. If the order were reversed, the numbers would look different, but the last row would stay the same — the total is independent of order. The ratio itself does not say anything by itself; what says something is that the sum of the submodule histories is **more than three times** the main repository's history. This ratio does not show up in any measurement that looks at the main repository alone. Recursion is not limited to one layer either. If a submodule has its own submodule, the `--recursive` option fetches that too, and the number of bound histories and the number of identities needed climb to **five**. The measurement does not build this, but the scheme's rule does not change: every layer writes **two objects** to the main repository and binds the whole of its own history behind it. Two practical conclusions follow from this. **First:** a non-recursive clone fetches the main repository's history complete and fetches **none** of the submodules' objects; this is the right choice for someone who wants to read the history. **Second:** someone who needs the working tree has to do a recursive clone and pays the ratio read from the table. Two separate costs exist in the same repository, and which one is paid is determined by the option used. #### The Broken Link and Recovery A pinned version is an identity, and an identity is valid as long as the commit it points to is reachable. If history is rewritten in the submodule's upstream repository — the operation measured in this topic's first lesson — old identities no longer point to any commit. How many of the main repository's **38** advance commits break depends on which point the rewrite starts from. This is not a data loss, but it makes **past points in the main repository's history unreproducible**, and the record of that loss is not kept in the main repository. Recovery has three paths. If the submodule's old commits still stand somewhere — in a copy, in a backup branch, or if the reflog window has not closed — those commits can be brought back from a separate reference and identities resolved again. If they still stand. The second path is writing a fix-up commit in the main repository that moves pinned versions to the new identities; this does not recover the history, it only repairs the tip. The third and cheapest path is never incurring the loss at all: before history is rewritten in a repository connected as a submodule, the main repositories that pin it need to be known. The link's direction is one-way — a submodule does not know who pins it. #### Summary - A submodule writes an **identity** to the main repository, not a history; the record consists of the `.gitmodules` file and the pinned version in the tree entry. - Advancing the pinned version writes **2 objects** to the main repository, and this number is independent of how many commits were passed in the submodule; in the measurement, **76** objects bind **378** submodule commits. - The main repository's history shows **200** commits; the number of commits needed to reproduce the state at its tip is **578**, and this gap is not written in the main repository. - With three submodules, building a single commit's tree requires **4** identities from **4** separate histories; if one is unreachable, the commit does not corrupt but cannot be reproduced. - A recursive clone climbs from **920** objects to **4540** — a **4.93** ratio; a non-recursive clone fetches the main repository's history complete and takes **0** objects from a submodule. #### Next Step In submodules, the thing that grew cost was **commit count**: a three-hundred-commit history brought in five times more objects than an eighty-commit one. This is a case where object count follows commit count, and it matches intuition. The next lesson looks at the case where intuition breaks: in the same history, when **fewer than a tenth** of the commits produce more than two-thirds of the total objects, the question of what weighs the history down has to be asked again — and the answer is not commit count. In that case, splitting the repository is no remedy, because the weight does not come from how commits are distributed but from the **type** of object they carry; what needs to come out of history is not a repository, but a class of file. ### Large File Storage Source: https://academia.sh/en/courses/advanced-git/large-file-storage What weighs down history is not commit count but object type: fewer than a tenth of a thousand commits — 91 binary commits — produce 0.6777 of the total objects; the clone is 5640 objects and drops to 1818 without binaries. In the previous lesson, object count tracked commit count: a three-hundred-commit submodule brought in roughly five times more objects than an eighty-commit one. The ratio was nearly one-to-one because every commit left a record of the same weight. This lesson looks at the case where that intuition breaks. In the same history, a small minority of commits can produce the majority of objects, and then the question "how big is the history?" gets the wrong answer if it is answered with commit count. Our question is: is the thing that weighs down a history how many commits were written, or **what type of object** those commits produce? Storage cost itself is not discussed here — that axis was established in the Relational Database Management and Cloud Storage and Data courses. The only thing measured here is **object share in history**. #### Why a Binary Asset Is a Separate Class Version control can store text files as a **diff**: the lines that change between two versions are much smaller than the whole file. This is what keeps a hundred versions of a file from being a hundred copies. This mechanism does not work for binary assets. An image, an audio recording, or a compiled output is a compressed byte sequence; when a single field in it changes, the entire byte sequence changes. There is no line structure, so there is no line diff either. The result: **every version of a binary asset is a new, whole object in history.** The shared setup models this directly — a commit carrying a binary asset writes **42** objects against a text commit's **2**. This accumulation has an irreversible side. Once a binary asset is committed, it enters history, and deleting it in a later commit does not remove it from history; it only removes it from the working tree. Everyone who clones keeps getting that object, because a clone fetches not the working tree but **history**. #### The Pointer Scheme **Large file storage** is a filter scheme that moves a binary body outside of history. A file pattern in the repository attributes is bound to a filter name; the filter works in both directions. The **clean filter** engages while a file is being committed, writes the body to a separate store, and puts a small **pointer file** in history in the body's place. The **smudge filter** engages while a file is being written to the working tree, reads the pointer, and fetches the body from the separate store. ```text # attributes file and pointer content — example dump, not executed # .gitattributes *.bin filter=large-file -text *.png filter=large-file -text # pointer file that enters history (not the body) version https:///v1 oid sha256: size 41943040 ``` What stands in history is this three-line text. The `-text` attribute marks the file as binary and stops line-ending conversion from being applied to it — that distinction is measured in this topic's fourth lesson. The filter's name is not a product name, it is a name defined in configuration; the same pattern could be bound to another filter. The scheme's one real gain is this: **the clone now fetches not every version, but only the pointers.** The body is requested for, and only for, the version that actually needs to be written to the working tree. #### The Measurement's Assumptions - **LR7** — The measurement uses the shared setup's four scales as they are: **50, 200, 1000, 4000** commits. Commits carrying a binary asset come from the setup's own distribution; they are not selected. - **LR8** — A text commit writes **2** objects (tree and blob). A commit carrying a binary asset adds **40** objects to these; this is that version of the body's counterpart in history. - **LR9** — In the pointer scheme, the binary body moves outside of history and a **1** object pointer blob is left in its place; a binary commit writes **3** objects. - **LR10** — "Separate store" is outside the measurement. The only thing counted is **the object in history**; how much room the bodies take up outside is not this measure's subject. - **LR11** — A single binary version's body is required to be able to work after a clone; the measurement adds this on top of the clone as **40** objects. - **LR12** — The "without binaries" column gives the state where commits carrying a binary asset were **never written**; this is not a cleanup result, it is the comparison baseline. - **LR13** — Cost is counted in the **touched object** unit. Bytes, disk, and network duration are outside the measurement. #### Measurement ```python """Large file storage: is what weighs down history commit count or object type. Part 1 - binary-commit share and object share at four scales. Part 2 - the same history with the pointer scheme: where does clone cost land. """ SEED = 20260814 FILES = ("metrics.py", "report.py", "identity.py", "config.py", "document.md") SCALES = (50, 200, 1000, 4000) POINTER = 1 # pointer blob: the single object that replaces the binary body BINARY_BODY = 40 # a binary version's objects in the shared setup def rng(seed): d = seed % 2147483646 + 1 def draw(n): nonlocal d d = (d * 48271) % 2147483647 return d % n return draw def history(n, seed=SEED): """n-commit linear history; each commit touches one file.""" draw, commits = rng(seed), [] for i in range(n): file = FILES[draw(5)] binary = draw(11) == 0 commits.append({"no": i + 1, "file": file, "binary": binary, "objects": 2 + (40 if binary else 0)}) return commits def clone_cost(t, include_binary=True): return sum(x["objects"] for x in t if include_binary or not x["binary"]) def pointer_cost(t): """The binary body moves outside; a single pointer blob is left in history.""" return sum(2 + (POINTER if x["binary"] else 0) for x in t) def working_copy(t): """The single binary version needed to work after cloning.""" return BINARY_BODY if any(x["binary"] for x in t) else 0 print("commits binary commits binary share clone objects no binaries binary objects object share") for n in SCALES: t = history(n) binary = sum(1 for x in t if x["binary"]) full, without = clone_cost(t), clone_cost(t, False) print(f"{n:6d} {binary:14d} {binary / n:12.4f} {full:14d} {without:11d} " f"{full - without:14d} {(full - without) / full:12.4f}") print() print("commits clone in history pointer scheme + one version gained objects ratio") for n in SCALES: t = history(n) full, pointer = clone_cost(t), pointer_cost(t) works = pointer + working_copy(t) print(f"{n:6d} {full:16d} {pointer:14d} {works:13d} {full - works:16d} " f"{full / works:5.2f}") print() n = 1000 t = history(n) binary = sum(1 for x in t if x["binary"]) full, without = clone_cost(t), clone_cost(t, False) print(f"history {n} commits, carrying a binary asset {binary}, total objects {full}, " f"binary share {(full - without) / full:.4f}") print(f"if {n - binary} text commits were deleted, objects gained {without}; " f"if {binary} binary commits were deleted, objects gained {full - without}") ``` ``` commits binary commits binary share clone objects no binaries binary objects object share 50 7 0.1400 380 86 294 0.7737 200 13 0.0650 920 374 546 0.5935 1000 91 0.0910 5640 1818 3822 0.6777 4000 365 0.0912 22600 7270 15330 0.6783 commits clone in history pointer scheme + one version gained objects ratio 50 380 107 147 233 2.59 200 920 413 453 467 2.03 1000 5640 2091 2131 3509 2.65 4000 22600 8365 8405 14195 2.69 history 1000 commits, carrying a binary asset 91, total objects 5640, binary share 0.6777 if 909 text commits were deleted, objects gained 1818; if 91 binary commits were deleted, objects gained 3822 ``` #### Object Count Does Not Track Commit Count The third row pays off the topic's fourth reading. In a thousand-commit history, the number of commits carrying a binary asset is **91** — **0.0910** of commits, fewer than a tenth. The same 91 commits produce **0.6777** of the total objects. The remaining **909** commits — close to nine-tenths of history — write only a third of the objects. The last line sharpens this with a thought experiment. If all nine hundred nine text commits were deleted, **1818** objects would drop from history. If the ninety-one binary commits were deleted, **3822** objects would drop — more than twice as many, with fewer than a tenth of the commits. **What weighs down history is not commit count, it is object type.** This is the exact opposite of the situation in this topic's first lesson: there, what determined cost was history's length; here, it is its composition. The top table also carries a warning. Object share comes out to **0.7737**, **0.5935**, **0.6777**, and **0.6783** at the four scales; there is no orderly trend. The reason is on the left side of the table, not the right: in a fifty-commit history, the number of binary commits is **7**, and the smallest measurable difference at this scale is **1/50 = 0.0200**. A single commit's position moving swings the share by two points. At a thousand and four thousand commits, the share settles between **0.6777** and **0.6783**; this is the real value, and the spread at small scale does not lend itself to drawing a conclusion from it. #### What the Pointer Scheme Pays The lower table's unit is again **touched object**. In a thousand-commit history, the clone drops from **5640** objects to **2091**; once the single binary version needed to work is added, **2131**. The gained objects are **3509**, the ratio **2.65**. At four thousand commits, the gain climbs to **14195** objects, and the ratio stays at **2.69** — the scheme's gain grows with scale, but its **ratio** stays fixed, because the gain is directly tied to binary commit share. The two-hundred-commit row comes out below the ratio: **2.03**. The reason is written in the top table — at that scale, binary commit share is at its lowest value, **0.0650**. The ratio is directly a function of this share and is not an independent metric. The same scheme gains **nothing** in a repository carrying no binary assets at all; every number that looks like a gain is, in fact, measuring that repository's binary share. Where the gain does not come from needs attention. The pointer scheme removes no commit; history is still a thousand commits. What it removes is **every binary version's body**, and these bodies do not vanish, they only move outside of history, into a separate store. The scheme's cost is there too: the cloning side now has to reach two sources at once, and if it cannot reach the second source, its history stands complete but it cannot build the working tree. This is another form of the dependency measured in submodules — there, what was missing was a history; here, a body store. There is an axis the measure does not see, and it needs to be named. The object unit does not ask how many **bytes** a body is; in the setup, a binary version is **40** objects, and this number is independent of the body's size. In a real repository, a large body and a small body produce the same object count but do not cost the same. This lesson measures object share; the byte axis is outside the measure, and no disk conclusion can be drawn from the table. A second limit: the scheme delivers its full gain only when set up **from the start**. If the filter is defined today, today's commits onward have their bodies moved out; the **3822** objects already standing in history stay in place, and every clone keeps fetching them. #### The Binary Cannot Be Merged Alongside object count, there is a second consequence, and it is paid not in history but in work. When two branches write separate versions of the same binary asset, a merge conflict is born and the conflict **cannot be resolved**: a merge works by blending two texts' lines, and a binary body has no lines to blend. The only decision left is choosing one of the two versions. This is why binary patterns are also given the `-merge` attribute; the attribute stops the tool from trying to blend and asks for a direct choice instead. The `-diff` attribute is the display side of the same reasoning: comparing two binary bodies line by line has no meaning, the output is unreadable. How much unnecessary work is born when these two attributes are not set is measured in this topic's fourth lesson. #### Narrowing the Clone Itself The pointer scheme moves the object outside of history. Two more schemes enter the same cost from a different angle, both leaving the object in place and changing **when it gets fetched**. ```text # taught commands — example dump, not executed git clone --filter=blob:none git clone --depth 1 git sparse-checkout set components/metrics-core ``` A **partial clone** does not fetch blobs at clone time; commit and tree objects arrive, and blobs are only requested once a file is actually read. A **shallow clone** cuts history at a given depth and never fetches older commits at all. **Sparse checkout** looks not at the object store but at the working tree: history arrives complete, and only the selected paths are written to the working tree. What all three share is that they are a deferral: the object is still in history and is requested when needed. The pointer scheme is different from this — it **removes** the body from history. This lesson measures the second; the first three could be measured with the same unit but need a different setup, because their gain is tied not to the repository but to the access pattern. A shallow clone has a separate limit of its own too: bisect and blame run over the truncated history never see the cut range at all. #### Retroactive Cleanup and Recovery Removing binary bodies from an existing history has only one way: **rewriting history.** This is the operation measured in this topic's first lesson, and its rules apply here too. The operation changes every binary commit's identity and the identity of every commit after it; in a thousand-commit history, if the first binary commit is at an early point, **almost every identity** changes. **It cannot be undone:** old identities no longer point to anything; every tag, every pinned version, and every external reference relying on those identities breaks. Every copy that has cloned the repository has to be **re-cloned**; a single branch pushed from the old copy brings the deleted bodies back, and the operation has to be redone from scratch. The command's full form is not given here, because it gives no warning of loss when run with the wrong range. Recovery paths are set up beforehand, not afterward. **First:** before the operation is run, a complete copy of the repository is kept in a separate place; as long as this copy is not synchronized with the new history, it is the sole source of the old identities. **Second:** the transformation is tried in this separate copy first, and the result is counted — how many commits' identities changed, how many objects dropped. **Third:** after a locally mistaken transformation, the reflog returns old tips before the garbage-collection window closes. Once the window closes, this path closes too. **Fourth, and cheapest:** write the pattern into the attributes file on the repository's first day. The gain is read in the table; its cost is only that it was not paid from the start. #### Summary - A binary asset does not lend itself to a line diff; every version accumulates in history as a new, whole object, and deleting it after it is committed does not remove it from history. - In a thousand-commit history, **91** commits carry a binary asset — **0.0910** of commits — and produce **0.6777** of total objects; the clone is **5640** objects, **1818** without binaries. - If **909** text commits were deleted, **1818** objects would drop; if **91** binary commits were deleted, **3822** would: what weighs down history is not commit count, it is object type. - The pointer scheme drops the clone from **5640** to **2131** objects at a thousand commits (a **2.65** ratio) but removes no commit; bodies move to a separate store and a second access condition is born. - Retroactive cleanup rewrites history, **cannot be undone**, and requires every copy to be re-cloned; the safe path is trying it in a separate copy and writing the pattern on the repository's first day. #### Next Step The pointer scheme dropped clone cost to a third, but the table still says **clone**: the cost is paid again every time. When it becomes necessary to work on two branches of the same repository together, the usual solution is a second clone, and that clone copies every object all over again. The next lesson questions this assumption: can a second working area be set up that shares the same repository's object store, and if so, how many objects go uncopied compared to a clone per branch? ### Multiple Working Trees Source: https://academia.sh/en/courses/advanced-git/multiple-working-trees An extra working tree shares the object store and copies 0 objects: in a thousand-commit history, the clone scheme writes 22560 objects for four branches while the tree scheme stays at 5640; 11466 of the 16920 gained objects are binary. The previous lesson dropped clone cost to a third but did not eliminate the cost: the table still said **clone**, and the clone was paid again every time it was done. This carries an assumption with it too — the assumption that working on two branches of the same repository together requires a second copy. The assumption is not true. A repository has two components, and they can be duplicated separately: an **object store** and a **working tree**. A clone copies both at once. This lesson's question is: if the object store is left single and the working tree is duplicated, how many objects go uncopied compared to a clone per branch? #### One Object Store, Multiple Trees A **worktree** is a repository commit written to the file system. **Multiple working trees** is the scheme that lets more than one working tree be set up bound to the same object store. An extra tree is opened as a directory; inside it, a small file pointing not to a repository directory but to the main repository is placed, and from that moment the tree reads its objects from the main repository. ```text # taught commands — example dump, not executed git worktree add ../tree-report report git worktree add --detach ../tree-review git worktree list git worktree remove ../tree-report git worktree prune ``` `add` opens the new directory and checks out the given branch there. `--detach` checks out not a branch but a commit directly; the tree stands in a detached-HEAD state, and this is the state wanted for a temporary inspection. `list` reports every tree and which branch each one is on. `remove` removes a tree, and `prune` cleans the administrative record of trees whose directory was deleted by hand. One constraint has to be known from the start: **a branch can be checked out in only one tree at a time.** Trying to check the same branch out in a second tree is refused. The constraint is not arbitrary — if two trees wrote commits to the same branch, which end the branch showed would depend on the tree, and it would lose reference uniqueness. #### What Gets Shared The distinction is built in one sentence: **what belongs to history is shared, what belongs to work is specific to the tree.** The shared side is the object store and the references. A commit written in one tree drops into the store as an object and becomes **reachable** from the other trees from that moment — no transfer is required. Branches, tags, and the reflog are shared too. The tree-specific side is the working state: the `HEAD` reference, the staging area, and the files in the working area. Every tree keeps its own `HEAD`, so it can stand on a separate branch; every tree keeps its own staging area, so a change staged in one does not show up in another. An uncommitted change stays inside the tree and is not shared. Configuration stands between the two. The repository's configuration file is shared, and a change made in one tree affects them all; making a setting specific to just one tree is an option that has to be separately turned on. Repository attributes are shared too — that file, the subject of the next lesson, is a tracked file and belongs to history, so it applies the same rule across all trees. This distinction directly gives the measurement's setup: an extra tree **copies no object**, it only writes an administrative record and checks files out to the working area. A clone, by contrast, copies the entire object store. #### The Measurement's Assumptions - **LR14** — The measurement uses the shared setup's four scales. The same work, on the same number of branches, is done under two separate schemes; the produced commit and object set is the same in both. - **LR15** — In the clone scheme, a separate clone is opened per branch and each clone copies the **entire** object store; partial clone and shallow clone are not used in this measurement. - **LR16** — In the tree scheme, the object store is single; an extra tree copies **0 objects**. The only thing it writes is **3** administrative records per tree, and these records are not objects, they are counted separately. - **LR17** — The files checked out to the working area are the same in both schemes; four branches materialize four file sets in both schemes. This is why it is outside the measurement. - **LR18** — "Binary uncopied" is the share of the gained objects coming from binary bodies; this is the same count as the previous lesson's object-type distinction. - **LR19** — The unit of the first and second parts is **touched object**. The unit of the third part is **step**, and a step is a single fetch that moves one branch's tip from one copy to another. #### Measurement ```python """Multiple working trees: what object sharing gains over a clone per branch. Part 1 - four trees, four scales: clone-scheme and tree-scheme objects. Part 2 - gain as tree count grows (1000-commit history). Part 3 - access from one tree to a commit written in another: unit step. """ SEED = 20260814 FILES = ("metrics.py", "report.py", "identity.py", "config.py", "document.md") SCALES = (50, 200, 1000, 4000) TREES = (1, 2, 4, 8) ADMIN_RECORD = 3 # admin record written per extra tree; copies no object def rng(seed): d = seed % 2147483646 + 1 def draw(n): nonlocal d d = (d * 48271) % 2147483647 return d % n return draw def history(n, seed=SEED): """n-commit linear history; each commit touches one file.""" draw, commits = rng(seed), [] for i in range(n): file = FILES[draw(5)] binary = draw(11) == 0 commits.append({"no": i + 1, "file": file, "binary": binary, "objects": 2 + (40 if binary else 0)}) return commits def clone_cost(t, include_binary=True): return sum(x["objects"] for x in t if include_binary or not x["binary"]) def clone_scheme(t, trees): """One clone per branch: each clone copies its own object store.""" return trees * clone_cost(t) def tree_scheme(t, trees): """Multiple working trees: one object store, extra trees copy no object.""" return clone_cost(t) def access_step(trees, scheme): """Access from a commit written in one tree to the other trees: fetch step.""" return 0 if scheme == "tree" else trees - 1 print("commits clone objects 4-clone objects 4-tree objects gained objects binary uncopied") for n in SCALES: t = history(n) k, a = clone_scheme(t, 4), tree_scheme(t, 4) binary = 3 * (clone_cost(t) - clone_cost(t, False)) print(f"{n:6d} {clone_cost(t):13d} {k:15d} {a:14d} {k - a:15d} {binary:16d}") print() t = history(1000) print("trees clone-scheme objects tree-scheme objects gained objects objects per extra tree admin records") for trees in TREES: k, a = clone_scheme(t, trees), tree_scheme(t, trees) extra = (k - clone_cost(t)) // (trees - 1) if trees > 1 else 0 print(f"{trees:4d} {k:20d} {a:20d} {k - a:15d} {extra:22d} " f"{(trees - 1) * ADMIN_RECORD:13d}") print() print("trees fetch steps in clone scheme fetch steps in tree scheme") for trees in TREES: print(f"{trees:4d} {access_step(trees, 'clone'):27d} " f"{access_step(trees, 'tree'):27d}") print() print(f"single object store {clone_cost(t)} objects; four trees share the same store, " f"objects copied by the extra trees {tree_scheme(t, 4) - clone_cost(t)}") print(f"if the same work were done with four clones, objects copied " f"{clone_scheme(t, 4) - clone_cost(t)}") ``` ``` commits clone objects 4-clone objects 4-tree objects gained objects binary uncopied 50 380 1520 380 1140 882 200 920 3680 920 2760 1638 1000 5640 22560 5640 16920 11466 4000 22600 90400 22600 67800 45990 trees clone-scheme objects tree-scheme objects gained objects objects per extra tree admin records 1 5640 5640 0 0 0 2 11280 5640 5640 5640 3 4 22560 5640 16920 5640 9 8 45120 5640 39480 5640 21 trees fetch steps in clone scheme fetch steps in tree scheme 1 0 0 2 1 0 4 3 0 8 7 0 single object store 5640 objects; four trees share the same store, objects copied by the extra trees 0 if the same work were done with four clones, objects copied 16920 ``` #### What Sharing Gains The first table's unit is **touched object**. In a thousand-commit history, working on four branches writes **22560** objects under the clone scheme; under the tree scheme, the total stays at **5640**. Gained objects: **16920** — the equivalent of three full clones. At four thousand commits, the same computation gives **67800** objects. Where the gain comes from is written in the last column. Of the **16920** objects gained at a thousand commits, **11466** come from binary bodies; the previous lesson's **0.6777** share drops in here exactly the same way. The more binary assets a repository has, the more opening a second clone costs — and that entire cost is **never paid** with a shared object store. The pattern is the same at small scales too, but the magnitudes can mislead. At fifty commits, the gain is **1140** objects; next to four thousand commits' **67800**, this looks small, but the ratio between the two schemes is identical in every row. The gain is directly the product `(trees − 1) × clone`; scale only grows the multiplier, it does not change the relationship. The smallest measurable difference at this scale is **1/50**, and no number in the table rests on a claim below that resolution. The second table gives the same thing by tree count, and the middle column stays fixed: two trees, four trees, eight trees — all **5640**. The object copied per extra tree is **0** in every row; the **5640** on the right is what the clone scheme pays per extra copy. The only thing paid under the tree scheme is the administrative record at the far right: **21** records for eight trees. These are not objects and do not enter history. The metric that follows from this is one sentence: **working-tree count does not grow the object store.** Under the clone scheme, cost grows linearly with tree count; under the tree scheme, it is fixed. This is the one pattern among the costs measured in this course that is **entirely** independent of scale: bisect grew slowly with scale, the rewrite ratio was fixed but its total grew; here, an extra copy's object cost is **zero** at every scale and every tree count. #### Visibility and the Fetch Step The third table's unit is **step**, and what it measures is not objects but access. Under the clone scheme, a commit written to a branch stands in that clone's object store. For the other copies to access it, a fetch is required; in a four-copy scheme, a commit reaching all of them is **3** steps, in an eight-copy one, **7**. Under the tree scheme, this number is **0** in every row — the commit is already in the shared object store, and the other tree sees it the moment it is written. This is the scheme's second gain, independent of object count, and it matters more in practice. A fix written in one tree can be tested immediately in another; cherry-pick, comparison, and merge operations happen between trees **without going over the network**. Under the clone scheme, the same work requires a round trip to a remote repository. The limit comes from the same place. A shared object store means a shared failure surface: a garbage collection done in one tree concerns every tree, and a repository corrupted in one tree affects all of them at once. Under the clone scheme, copies are isolated from each other, and this isolation is the one thing bought in exchange for the **16920** objects paid. #### A Third Scheme: Switching in a Single Tree The measurement compared two schemes, but the third one is the most common in practice: a single working tree and switching between branches. This scheme copies **0** extra objects — same as the tree scheme — and requires **0** fetch steps. Looked at by the object unit, it is the cheapest scheme. Its cost is paid somewhere else. Every switch rewrites the files in the working area; if there is an uncommitted change, the switch is either blocked or the change is carried along. The Branching and Collaboration course counted this case in detail: which change gets carried, which is blocked, and which is left behind was measured there and is not repeated here. The stash is the standard way to get past this obstacle — it sets an uncommitted change aside and reopens it after the switch. What multiple working trees solves is exactly this: **there is no switch.** Two branches are open at the same time, in two separate directories; a test can run in one while an edit is made in the other. The stash is not needed, because an uncommitted change does not need to give up its place. In exchange, every tree materializes its own working state, and this is not limited to files. Dependency directories, build outputs, and caches are recreated per tree; these are outside the measurement because they do not belong to history, but they really do take up room on disk, and in a four-tree scheme they are produced four times. The **16920** gained in the object unit does not cover this cost. The choice between the two schemes therefore does not come down to a single number: the measured object gain is real and large, the unmeasured working state grows with tree count. #### Removing a Tree and Recovery When a working tree is removed, objects are not removed — the objects did not belong to the tree. The only thing that can be lost is what was **uncommitted in that tree**: the changes in the working area and the staging area's contents. The `remove` command refuses if there is an uncommitted change in the tree; deleting the directory by hand asks nothing. The difference between these two paths is whether there is a warning. The second loss path is quieter. A commit written in a tree in a detached-HEAD state is not bound to any branch; once the tree is removed, that commit becomes unreachable and drops once the garbage-collection window closes. **The recovery path:** the reflog is shared, so the commit's tip can be read from another tree and bound to a branch before the window closes. The cheaper path is opening that tree not with `--detach` but with a branch name — a branch is a reference, and a reference is independent of the tree's lifetime. For a tree standing on a removable medium or a temporarily mounted directory, there is a separate safeguard: the tree is **locked**. A locked tree's administrative record is not cleaned up even when the directory is unreachable; otherwise the cleanup operation would ignore the tree, and the branch in that tree would become checkable out somewhere else. The lock does not prevent loss, it prevents a **misdiagnosis**. The third is the administrative record itself: the record of a tree whose directory was deleted by hand stays in the repository and keeps preventing that branch from being checked out in another tree. `prune` cleans these records and touches no object. #### Summary - A repository has two components: an **object store** and a **working tree**. A clone copies both; multiple working trees duplicates only the second. - The object store, branches, tags, and reflog are shared; `HEAD`, the staging area, and the working area are tree-specific. A branch can be checked out in only one tree at a time. - In a thousand-commit history, the clone scheme writes **22560** objects for four branches, the tree scheme stays at **5640**; **11466** of the **16920** gained objects come from binary bodies. - The object copied per extra tree is **0** at every scale; the only thing paid under the tree scheme is **3** administrative records per tree, and the object store does not grow with tree count. - In the step unit: a commit reaching every copy requires **3** and **7** fetches under the clone scheme, **0** under the tree scheme; what is lost in exchange is the isolation between copies. #### Next Step The measurement left one assumption out of scope: the assumption that the files checked out to the working area are the same under both schemes. This said that **how** files are written is not a choice — but there is a transformation layer applied to a file during checkout, and the blob in the object store and the file in the working area do not have to be identical to each other. The next lesson measures that layer: when line-ending conversion is not configured, which commit, in which file, writes a new object while actually changing nothing at all? ### Repository Attributes Source: https://academia.sh/en/courses/advanced-git/repository-attributes The attributes file determines which operation behaves how on which file: an unconfigured thousand-commit history writes 5888 objects, line endings produce 162 unnecessary objects, and 43 of fifty merges conflict in the generated file, adding 86 more objects. The previous lesson's measurement left one assumption outside: the assumption that the files checked out to the working area are the same under both schemes. This said that **how** a file is written is not a choice. It is not. The blob in the object store and the file in the working area do not have to be identical to each other; there is a transformation layer between them, and that layer is configurable. Left unconfigured, the layer works silently, and what it does gets written to history. This lesson's question is: which attribute changes which operation on which file, and in the unconfigured state, how many objects are written **even though content did not change**? #### What the Attributes File Determines **Repository attributes** are rules that bind behavior to file patterns, and they stand in the `.gitattributes` file. The file is a tracked file: it enters history, can differ between branches, and carries the same rule to everyone who clones. This is what separates it from a configuration file — configuration is personal and copy-specific, an attribute belongs to the repository. ```text # attributes file — example dump, not executed # .gitattributes * text=auto *.py text eol=lf *.md text eol=lf config.py text merge=ours presentation.bin -text -diff document.md diff=text ``` Each line is a pattern and the attributes bound to it. `text` declares that the file is considered text and can go through line-ending conversion; `text=auto` asks the tool to make that decision by looking at content. `eol=lf` fixes which **line ending** is used when writing to the working area. The leading minus sign removes an attribute: `-text` declares the file will go through no conversion at all, `-diff` declares comparison will not be attempted as text. `merge=ours` and `diff=text` name a **merge driver** or a comparison definition. #### Four Operations, Four Separate Moments Attributes engage at four separate moments, and if which one runs when gets confused, the measurement gets confused too. **Checkout moment:** line-ending conversion is applied while a blob is being written to the working area. **Commit moment:** the reverse conversion is applied while the file in the working area is converted to a blob — the form written to the repository is single. **Merge moment:** which driver is used is decided while the two sides' changes are being blended. **Diff moment:** whether the file is treated as text is decided while the diff between two blobs is produced. There is a fifth moment too, and it does not concern history at all: which paths are left out is also declared by an attribute while a version of the repository is packaged as an archive. The rule looks at packaging, not committing; the file stays in history, it just does not go into the archive. This lesson does not measure it because it has no counterpart in the object unit. The first two write objects to history; of the last two, only merge writes one. Diff affects readability, not history — this is why the `-diff` row gives zero in the measurement, and that is not a gap, it is the correct result. What happens when line-ending conversion is unconfigured? Whatever is in the writer's environment goes into the repository as is. When two environments using two different line-ending styles touch the same file in sequence, **even though the content did not change at all**, the blob is rewritten from scratch: the byte sequence is different, so the object is different. #### The Measurement's Assumptions - **LR20** — The measurement uses the shared setup's four scales. Two fields are added to each commit from a separate generator: the writer's line-ending style and whether the content actually changed. - **LR21** — **A third** of commits come from an environment using a different line-ending style; **a fifth** do not actually change content, they only touch the file. - **LR22** — With no line-ending setting, a commit whose content did not change writes a new blob only if the file's **last recorded style** differs; every such write is **2 objects**. When the setting is assumed, these commits write **0 objects**. - **LR23** — A merge happens every **20** commits; if both halves of the window touched the generated file, a merge conflict is born. Manual resolution writes one commit, **2 objects**. - **LR24** — When `merge=ours` is defined, no conflict is born and no resolution commit is written; the driver does not take the other side's version in the generated file. - **LR25** — `-text` and `-diff` add no object to the binary asset and remove none; their effects are at checkout and diff moments, not in history. - **LR26** — Cost is counted in the **touched object** unit. The "unconfigured" column is the baseline, the "fully configured" column is the shared setup's clone cost; the gap between them is the object the attribute **stops from being written**. #### Measurement ```python """Repository attributes: the unnecessary object born when unconfigured. Part 1 - which attribute changes which operation on which file (1000 commits). Part 2 - unconfigured, line-ending configured, and fully configured objects at four scales. """ SEED = 20260814 FILES = ("metrics.py", "report.py", "identity.py", "config.py", "document.md") BINARY_ASSET = "presentation.bin" GENERATED = "config.py" # generated file: the merge driver is bound to this one SCALES = (50, 200, 1000, 4000) MERGE_INTERVAL = 20 # one merge every 20 commits def rng(seed): d = seed % 2147483646 + 1 def draw(n): nonlocal d d = (d * 48271) % 2147483647 return d % n return draw def history(n, seed=SEED): """n-commit linear history; each commit touches one file.""" draw, commits = rng(seed), [] for i in range(n): file = FILES[draw(5)] binary = draw(11) == 0 commits.append({"no": i + 1, "file": file, "binary": binary, "objects": 2 + (40 if binary else 0)}) return commits def clone_cost(t, include_binary=True): return sum(x["objects"] for x in t if include_binary or not x["binary"]) def with_environment(t, seed): """Each commit gets the author's line-ending style and whether the content actually changed.""" draw, k = rng(seed), [] for x in t: k.append({**x, "crlf": draw(3) == 0, "real": draw(5) != 0}) return k def line_ending_objects(k, file=None): """Unconfigured: a blob rewritten because the line-ending style changed even though the content is the same. The last style is tracked per file.""" last, wasted = {}, 0 for x in k: if x["file"] == file or file is None: if not x["real"] and last.get(x["file"]) not in (None, x["crlf"]): wasted += 2 last[x["file"]] = x["crlf"] return wasted def conflicting_merges(k): """One merge per interval; if both halves touched the generated file, a conflict is born and the manual resolution writes one commit.""" count = 0 for start in range(0, len(k) - MERGE_INTERVAL + 1, MERGE_INTERVAL): window = k[start:start + MERGE_INTERVAL] half = MERGE_INTERVAL // 2 left = any(x["file"] == GENERATED for x in window[:half]) right = any(x["file"] == GENERATED for x in window[half:]) count += left and right return count def driver_objects(k): return 2 * conflicting_merges(k) K = with_environment(history(1000), SEED + 7) print("file attribute operation changed line-ending objects" " driver objects") ATTRIBUTES = (("metrics.py", "text eol=lf", "checkout and commit"), ("report.py", "text eol=lf", "checkout and commit"), ("identity.py", "text eol=lf", "checkout and commit"), ("config.py", "text merge=ours", "checkout, commit, merge"), ("document.md", "text eol=lf", "checkout and commit"), (BINARY_ASSET, "-text -diff", "checkout and diff")) for file, attr, op in ATTRIBUTES: s = 0 if file == BINARY_ASSET else line_ending_objects(K, file) d = driver_objects(K) if file == GENERATED else 0 print(f" {file:17s} {attr:16s} {op:26s} {s:16d} {d:15d}") print() print("commits unconfigured objects line-ending configured fully configured wasted objects share") for n in SCALES: k = with_environment(history(n), SEED + 7) full, s, d = clone_cost(k), line_ending_objects(k), driver_objects(k) print(f"{n:6d} {full + s + d:14d} {full + d:18d} {full:11d} {s + d:15d} " f"{(s + d) / (full + s + d):6.4f}") print() n = 1000 k = with_environment(history(n), SEED + 7) print(f"history {n} commits; blobs rewritten because of line ending " f"{line_ending_objects(k) // 2}, objects written {line_ending_objects(k)}") print(f"merges {n // MERGE_INTERVAL}, conflicting in the generated file " f"{conflicting_merges(k)}, objects written by manual resolution {driver_objects(k)}") ``` ``` file attribute operation changed line-ending objects driver objects metrics.py text eol=lf checkout and commit 38 0 report.py text eol=lf checkout and commit 34 0 identity.py text eol=lf checkout and commit 22 0 config.py text merge=ours checkout, commit, merge 36 86 document.md text eol=lf checkout and commit 32 0 presentation.bin -text -diff checkout and diff 0 0 commits unconfigured objects line-ending configured fully configured wasted objects share 50 390 384 380 10 0.0256 200 972 938 920 52 0.0535 1000 5888 5726 5640 248 0.0421 4000 23578 22912 22600 978 0.0415 history 1000 commits; blobs rewritten because of line ending 81, objects written 162 merges 50, conflicting in the generated file 43, objects written by manual resolution 86 ``` #### Which Attribute, Which File The top table answers the lesson's first question, and what stands out is that the columns are **independent of each other**. The line-ending column spreads across five text files: **38**, **34**, **22**, **36**, and **32** — **162** objects total. The distribution is not even, but the reason is not the attribute itself, it is the setup's file distribution. From the attribute's point of view, all five lines carry the same rule; the cost paid changes with how often the file is touched. The driver column collects onto a single row: **86** objects, only in the `config.py` row. This, by the setup's design, is a generated file — one not written by hand, but produced by a tool and placed in the repository. In the other four files the driver column is **0**, because in them a conflict has to be resolved by hand, and resolution is a decision, not something to be changed by a setting. The binary asset row gives **0** in both columns. The `-text` and `-diff` attributes do real work — they stop conversion, they prevent an unreadable comparison — but they **add and remove not a single object from history**. Because the measure's unit is objects, those effects do not show up here. This row is the declaration of the measure's scope boundary. #### What the Absence of Settings Costs The lower table's unit is **touched object**, and the three columns give three configuration levels of the same history. An unconfigured thousand-commit history writes **5888** objects; once line endings are configured, this drops to **5726**, and once the merge driver is also defined, to **5640** — that is, the shared setup's clone cost. Wasted objects: **248**, share **0.0421**. The share is small, and this is the measurement's honest result: line-ending irregularity does not bloat a repository. Two observations are still worth noting. **First**, the wasted object share comes out to **0.0256**, **0.0535**, **0.0421**, and **0.0415** at the four scales; it settles around **0.042** at the two large scales and does not decrease as scale grows. This is a cost that keeps being paid across a history's whole lifetime. **Second**, the **10** objects in the fifty-commit row correspond to five commits at that scale's **1/50** resolution; no conclusion can be drawn from this cost in a small repository. The real number stands in the third line: **43** of **50** merges conflict in the generated file. This is more than **four-fifths** of merges, and each one means a person sitting down and manually clearing conflict markers. The **86** objects it writes to history are the smallest part of this work; the unmeasured part is the decision given forty-three times, none of which carries any information. The Branching and Collaboration course showed that conflict resolution is not written to history — what is seen here is that **forty-three of** those unwritten decisions **were unnecessary from the start.** #### Which Line Wins The measurement assumed a single rule fell on each file. In a real attributes file, more than one pattern matches a path, and without knowing which one wins, no setting's effect can be predicted. Rule order has three layers. **Within the same file, the last matching line wins** — it is read top to bottom, and the one below overrides the one above; this is why general patterns go first and specific patterns go last. **An attributes file in a subdirectory overrides the one above it**, because it is closer. And inside the repository there is one more attributes file that never enters history, a local one; it **overrides all of them** and applies only in that copy. The third layer is double-edged. It lets a copy's behavior be tuned without changing the repository; for the same reason, when an unexpected conversion is seen in one copy, that is the first place to look. We said attributes belong to the repository — that is true outside this local layer, and if it is not known where the difference is, why a file comes out differently is never understood at all. An attribute can take three values: given (`text`), removed (`-text`), and **unspecified**. The third is not the same thing as the second: a removed attribute finalizes a decision, an unspecified attribute leaves it to a lower layer or the tool's own default. `text=auto` manages exactly this third case — it makes the decision by looking at content and applies no conversion to a file it judges binary. #### The Limit of Configuration and Recovery The attributes file works forward. A `text=auto` line written today does not fix blobs written yesterday; the **162** objects in history stay in place. The setting's effect starts after the first checkout following it, and if the repository is already checked out in people's hands, the files need to be re-normalized once — this shows up as a new commit in history and does not rewrite history. There is a rewriting path too, and it falls under the same rule as this topic's second lesson: normalizing every past blob **is rewriting history**, it changes identities, **cannot be undone**, and requires every copy to be re-cloned. Its gain is read in the measurement — **162** objects. In a thousand-commit repository, this is a thirty-fifth of the clone cost, and rewriting history for that gain is justified in almost no case. The cheap and reversible path is this: the attributes file is written on the repository's first day; if it was written later, it is corrected going forward with a single normalization commit and the past is left as is. A wrong attribute line is itself harmless — because the file is a tracked file, the line can be reverted, and the conversion returns to its old form on the next checkout. #### Summary - Repository attributes stand in a tracked file and belong to the repository; this is what separates them from configuration. They engage at four separate moments: checkout, commit, merge, and diff. - Diff moment writes no object to history; the binary-asset row gives **0** in both columns of the measurement, and this is the declaration of the measure's scope boundary. - With line endings unconfigured, **81** commits whose content did not change write a new blob and produce **162** objects; these spread across five text files as **38 / 34 / 22 / 36 / 32**. - With no merge driver defined for the generated file, **43** of **50** merges conflict and manual resolution writes **86** objects; `merge=ours` eliminates all of these conflicts. - At a thousand commits, the unconfigured history is **5888**, the fully configured one is **5640** objects; the wasted **248** objects have a share of **0.0421**, and it does not decrease as scale grows. #### Next Step This topic's four lessons made four separate interventions on the same repository: they embedded a dependent repository, moved a binary body outside, duplicated the working tree, and configured the transformation layer. All four assumed one thing — where the repository's **boundary** is. The next lesson measures that assumption: if the same work is done in a single repository and in four separate repositories, how do clone cost and search steps change, and how many of the questions asked across components go unanswered under the second scheme? ### Monorepo and Polyrepo Source: https://academia.sh/en/courses/advanced-git/monorepo-and-polyrepo The same work under two schemes: total objects are 22600 in both, but the cost per repository is 22600 versus 5650, bisect is 11 versus 36 steps, and of four cross-component questions the monorepo answers 4 while the polyrepo answers 1. This topic's four lessons made four separate interventions on the same repository: they embedded a dependent repository, moved a binary body outside, duplicated the working tree, and configured the transformation layer. All four held one thing fixed — the repository's **boundary**. This lesson makes that boundary a variable. The same work, the same commits, and the same objects; the only difference is whether all of it stands inside one repository or inside four separate ones. The institutional discussion of scale and ownership trade-offs was established in Enterprise Context and Integration in the Software Architecture course and in the DevOps and Platform Engineering course, and it is not repeated here. What is measured here is only this: **objects, steps, and answered questions.** #### Two Schemes, the Same Work **Monorepo** is the scheme where more than one component stands in a single repository and a single history. **Polyrepo** is the scheme where each component stands in its own repository and its own history. The side of the distinction that enters the measurement is built in one sentence: **in a monorepo, components share a common order; in a polyrepo, they do not.** In a monorepo, two components can be touched in the same commit, and history records this; in a polyrepo, every repository has its own order, and there is no comparable common point between two orders. The measurement therefore models splitting not as a rewrite but as the same commit sequence distributed across separate histories. The objects are the same, the commits are the same; the only thing that changes is the boundary. #### The Measurement's Assumptions - **LR27** — The same work is the shared setup's **4000**-commit scale. The polyrepo scheme splits this sequence into **4** equal parts and renumbers each part on its own; no commit is added, removed, or changed. - **LR28** — The object count in each part is unaffected by the split; the **total** objects of the two schemes are equal by definition. What is measured is not the total, it is the cost paid **per repository**. - **LR29** — The oracle works with its own rule in every history: the commit that introduced the defect is at the three-quarters point of that history. The split scheme has four separate oracles. - **LR30** — The "repository unknown" row is the case where which component the defect is in is not known beforehand; the search runs separately in each repository and the steps are summed. - **LR31** — A question is answered if all the components it requires stand in **a single history**. In a monorepo, four components stand in one history; in a polyrepo, one. - **LR32** — The first part's unit is **touched object**, the second part's unit is **step**. The third part does not measure cost, it counts answerability. - **LR33** — The measurement does not include axes like team size, ownership, permissions, or release scheme; those are the subject of other courses, and no organizational conclusion can be drawn from this table. #### Measurement ```python """Monorepo and polyrepo: the same work under two schemes. Part 1 - clone cost, unit touched object. Part 2 - bisect and blame, unit step. Part 3 - how many cross-component questions get answered. """ SEED = 20260814 FILES = ("metrics.py", "report.py", "identity.py", "config.py", "document.md") TOTAL = 4000 COMPONENTS = 4 def rng(seed): d = seed % 2147483646 + 1 def draw(n): nonlocal d d = (d * 48271) % 2147483647 return d % n return draw def history(n, seed=SEED): """n-commit linear history; each commit touches one file.""" draw, commits = rng(seed), [] for i in range(n): file = FILES[draw(5)] binary = draw(11) == 0 commits.append({"no": i + 1, "file": file, "binary": binary, "objects": 2 + (40 if binary else 0)}) return commits def faulty_commit(t): """Oracle: the commit that introduced the defect is known because we wrote the setup.""" return len(t) * 3 // 4 def linear_search(t): target, step = faulty_commit(t), 0 for x in t: step += 1 if x["no"] >= target: return step return step def bisect(t): target, low, high, step = faulty_commit(t), 1, len(t), 0 while low < high: mid = (low + high) // 2 step += 1 if mid >= target: high = mid else: low = mid + 1 return step def blame(t, file): return sum(1 for x in t if x["file"] == file) def clone_cost(t, include_binary=True): return sum(x["objects"] for x in t if include_binary or not x["binary"]) def split(t, count): """The same work, split into `count` separate repositories: objects are the same, histories are separate.""" size = len(t) // count return [[{**x, "no": i + 1} for i, x in enumerate(t[p * size:(p + 1) * size])] for p in range(count)] QUESTIONS = (("isolating a defect in one component", 1), ("two components changing together", 2), ("a change spreading to three components", 3), ("the shared state of all components", 4)) def answerable(needed, components_in_repo): """A question is answerable if all the components it needs stand in a single history.""" return needed <= components_in_repo mono = history(TOTAL) poly = split(mono, COMPONENTS) print("scheme repos objects per repo 1 component 2 components all components") print(f" monorepo 1 {clone_cost(mono):18d} {clone_cost(mono):11d} " f"{clone_cost(mono):13d} {clone_cost(mono):14d}") one = clone_cost(poly[0]) two = sum(clone_cost(p) for p in poly[:2]) total = sum(clone_cost(p) for p in poly) print(f" polyrepo {COMPONENTS:3d} {total // COMPONENTS:18d} {one:11d} " f"{two:13d} {total:14d}") print() print("scheme bisect linear search" " blame (config.py)") SEARCH = (("monorepo, 4000 commits", bisect(mono), linear_search(mono), blame(mono, "config.py")), ("polyrepo, defect's repo known", bisect(poly[0]), linear_search(poly[0]), blame(poly[0], "config.py")), ("polyrepo, repo unknown", sum(bisect(p) for p in poly), sum(linear_search(p) for p in poly), sum(blame(p, "config.py") for p in poly))) for name, bi, li, bl in SEARCH: print(f" {name:34s} {bi:7d} {li:14d} {bl:18d}") print() print("question needed components monorepo polyrepo") for name, needed in QUESTIONS: m = "yes" if answerable(needed, COMPONENTS) else "no" p = "yes" if answerable(needed, 1) else "no" print(f" {name:39s} {needed:16d} {m:>9s} {p:>9s}") print() print(f"monorepo answered {sum(answerable(n, COMPONENTS) for _, n in QUESTIONS)}/" f"{len(QUESTIONS)}, polyrepo answered " f"{sum(answerable(n, 1) for _, n in QUESTIONS)}/{len(QUESTIONS)}") print(f"total objects the same in both schemes {total}; what changes is per-repository " f"cost: {clone_cost(mono)} and {total // COMPONENTS}") ``` ``` scheme repos objects per repo 1 component 2 components all components monorepo 1 22600 22600 22600 22600 polyrepo 4 5650 5640 11280 22600 scheme bisect linear search blame (config.py) monorepo, 4000 commits 11 3000 787 polyrepo, defect's repo known 9 750 209 polyrepo, repo unknown 36 3000 787 question needed components monorepo polyrepo isolating a defect in one component 1 yes yes two components changing together 2 yes no a change spreading to three components 3 yes no the shared state of all components 4 yes no monorepo answered 4/4, polyrepo answered 1/4 total objects the same in both schemes 22600; what changes is per-repository cost: 22600 and 5650 ``` #### Same Total, Different per Repository The first table's unit is **touched object**, and the right column gives **22600** under both schemes. This is not a coincidence, it is the setup's rule: splitting produces no object and destroys none. The difference is in the columns on the left. In a monorepo, someone working on a single component also clones **22600** objects, because a clone has no smaller unit. In a polyrepo, the same person clones **5640** objects — a quarter. If two components are needed, **11280**; if all four, **22600** again. The crossover point reads directly from the table: **a polyrepo is only cheap if most workers touch a single component.** For someone touching all four, the two schemes pay the same cost, and under a polyrepo that cost is paid through four separate operations. Partial and sparse clone options achieve the same narrowing in a monorepo without splitting the repository; this measurement does not use them. #### Splitting Breaks Search The second table's unit is **step**, and the difference between the rows is the lesson's sharpest result. If the component the defect is in is **known**, the polyrepo wins: bisect drops from **11** steps to **9**, linear search from **3000** to **750**, blame from **787** to **209**. The search runs over a smaller history, and the cost shrinks. If it is not known, the table flips. Bisect climbs to **36** steps — more than three times the monorepo's. The reason is clear: bisect operates on a single ordered set, four separate histories are four separate sets, and each one runs its own **9**-step search. Splitting destroys bisect's one strength — growing slowly as the set grows — by fragmenting it. The same row's other two columns give **3000** and **787** — identical to the monorepo's values. Linear scan and blame are unaffected by the split, because both already touched every commit, and total commit count did not change. **The only search form the split damages is the one that works by halving the set.** #### The Question Left Unanswered The third table counts not cost but **answerability**. A monorepo answers **4** of four questions, a polyrepo answers **1**. The one question answered is the one that stays within a single component. The remaining three look at two, three, and four components at once, and there is no history in a polyrepo that can do that. This is not a tooling gap, it is the definition of the boundary: if no commit records two components changing together, there is no query that can later ask that question. There is a way to close the gap, and its cost was measured in this topic's first lesson: setting up a record repository that pins the four components together. This binds the four components as submodules and makes "the shared state of all components" answerable again. In exchange, the identities needed to build a single commit's tree climb from **1** to **5**, and a recursive clone returns to **22600** objects. When a polyrepo scheme wants to take back the question it lost, it starts paying back the monorepo's cost. #### Summary - Splitting produces no object: total objects are **22600** in both schemes. What changes is the cost paid **per repository** — **22600** versus **5650**. - Someone touching a single component clones **5640** objects in a polyrepo, someone touching all four clones **22600**; a polyrepo is cheap only if most workers stay within a single component. - If the defect's repository is known, bisect drops from **11** to **9** steps; if it is not, it climbs to **36**. Splitting breaks the search that works by halving the set. - Linear scan stays at **3000** steps and blame at **787**: splitting does not affect either form, because both already touched every commit. - Of four cross-component questions, a monorepo answers **4**, a polyrepo **1**; the loss is not a tooling gap but the definition of the boundary, and taking it back recovers the cost of a submodule. #### Course Wrap-Up The course ran on a single measure: the cost of an operation is not the length of history but the number of objects it touches — and cost is counted in two units, **step** or **touched object**. Twelve lessons applied this measure to separate places. | lesson | operation measured | how cost grows | |---|---|---| | Interactive Rebase | objects touched by rewriting from the middle | 252 / 362 / 3002 / 11322 objects; ratio from 0.520 to 0.500 — the ratio stays fixed while scale grows eightyfold | | Bulk History Transformation | objects touched by a transformation from the first commit | 380 / 920 / 5640 / 22600 objects, ratio 1.000 at every scale; the shared prefix drops to 0 and the cost is the entire history | | Force Push Discipline | what rewriting makes the copies pay | the writer pays between 2 and 5640 objects, copies between 14 and 39480 — always seven times as much; scales with copy count | | Reflog | what stays reachable after a rewrite | reachable fixed at 1000, unreachable 3 / 500 / 701 / 1000; recovery depends on the horizon and closes at 3 / 50 / 500 / 1000 | | Bug Hunting with Bisect | the step count of finding a defect | linear 37 / 150 / 750 / 3000 steps, bisect 6 / 7 / 9 / 11; while history grows eightyfold, bisect does not even double | | Blame and Line History | the step count of a file's line history | 8 / 36 / 209 / 787 steps; grows with the number of commits touching the file and the answer settles at 40 commits | | Hooks | the cost at the moment the block is placed | 0 objects before commit, 140 before push, 3522 objects and seven copies six hundred commits after acceptance; a late gate distributes the cost | | Submodules | the history the pinned version binds | 76 objects bind 378 commits; the clone climbs from 920 to 4540 (a 4.93 ratio) and totals with submodule count | | Large File Storage | object type's share of the total | 91 commits (0.0910) produce 0.6777 of objects; the clone drops from 5640 to 2131 — cost does not track commit count | | Multiple Working Trees | the objects an extra working area copies | an extra tree is 0 objects at every scale; 5640 instead of 22560 for four branches — cost never grows with tree count | | Repository Attributes | the unnecessary object the absence of settings writes | line ending 162, merge driver 86, total 248 objects and a ratio of 0.0421; does not decrease as scale grows | | Monorepo and Polyrepo | the cost of the same work under two schemes | total 22600 objects unchanged, per repository 22600 and 5650; bisect 11 and 36 steps, answered questions 4/4 and 1/4 | Three readings come out of the table. **First: costs diverge within the same history.** While scale grows eightyfold, linear search grows eightyfold, blame grows a hundredfold, cloning grows sixtyfold; bisect does not even double, and an extra working tree does not grow at all. A repository being "large" does not by itself say anything — what says something is which operation touches which set. **Second: the number that determines cost is, most of the time, not commit count.** Object type produces two-thirds of objects with fewer than a tenth of the commits; copy count forces resynchronization independent of the size of the change; where the boundary is drawn triples the search's step count. In all three cases, looking at the wrong variable gives the wrong result. **Third: most of this course's tools cannot be undone, and the one who writes does not pay the cost.** Bulk transformation zeroes the shared prefix, force push deletes without anyone's knowledge, retroactive cleanup forces every copy to be re-cloned. A recovery path was counted alongside each one — a backup branch, the reflog, a separate clone, a dry run — and what they all share is that they have to be set up **before** the operation. From here, the next course — Code Review and Team Process — takes up the same tools with a different question. Everything measured across this course was the work the tool did: how many objects it touched, how many steps it spent, which question it left unanswered. No measurement said whether the operation should have been done. Knowing that rewriting history touches 5640 objects does not decide whether that history should be rewritten; knowing at which moment a hook places a block does not say which change should be blocked. The distinction has to be set from the start: **the tool changes history, the human makes the decision.** The next course's subject is how that decision gets made — who looks, what they look at, and where the decision gets recorded. The numbers this course leaves behind are the unit that will be used there to compute the cost of the decisions to be made.