---
title: 'Advanced Algorithms and Problem Solving'
source: 'https://academia.sh/en/courses/advanced-algorithms'
language: en
updated: '2026-08-17T04:39:22+00:00'
license: 'CC BY-SA 4.0'
lessons: 23
outcomes:
- 'Distinguishes divide and conquer, greedy and dynamic programming together with their preconditions'
- "Reduces a problem to a recognized pattern and tests the pattern's precondition against an oracle"
- 'Justifies the optimal substructure and overlapping subproblem conditions with counts'
---
# Advanced Algorithms and Problem Solving
Measuring design approaches and problem patterns together with their preconditions; counting, beside the step a pattern gains, the wrong answers it gives once its precondition breaks.
## Algorithm Design Approaches
Brute force's role as an oracle, the divide-and-conquer recurrence, the systems where greedy choice is wrong, memoization's overlap condition, the search space pruning cuts, and randomization's expected performance.
### Brute Force and Its Limits
Source: https://academia.sh/en/courses/advanced-algorithms/brute-force-and-its-limits
Brute force's role in this course is not a slow option but an oracle: exhaustive counting spends 2640 steps on 40 inputs, early-exit pruning gives the same answer in 866 steps, and sampling that looks only at the first six positions diverges from the oracle on 14 inputs in 406 steps.
The Algorithms course closed by establishing a measure: operation counting, asymptotic
notation, lower bound. In that course the only thing measured was cost, because every
algorithm was **assumed to be correct**. This course opens by removing that assumption.
Choosing a design approach is not merely speeding up; it is accepting a
**precondition**. When the precondition breaks, the approach does not slow down — **it
gives a wrong answer**, and the wrongness cannot be seen from the output, because it
still returns a number. This is why **brute force** stands at the center of the
course; but here not as a "slow option," rather as an **oracle**. An approach's number
is not the steps it saves but the number of wrong answers it gives once its
precondition breaks; an approach whose precondition is not tested against the oracle
is considered unmeasured.
- **DA1.** The measure is **steps**, not time. No lesson measures wall-clock time.
- **DA2.** Seed **20260218**. The same seed gives the same corpus; every number here
is reproducible.
- **DA3.** The **corpus** consists of 40 arrays; each array holds 12 values, and the
values range from -9 to 20.
- **DA4.** A **step** is adding two values and comparing them with the target. The
counter counts every step as one.
- **DA5.** The **oracle is always brute force**: a procedure that sees every
possibility and never exits early. The oracle's answer is correct by definition.
- **DA6.** A candidate's correctness is claimed only by comparison with the oracle.
**No correctness claim is written without the oracle.**
- **DA7.** A **diverging input** is one where the oracle and the candidate give
different answers; it is counted out of 40.
- **DA8.** Resolution: 1 divergence out of 40 inputs is considered unmeasured; 3 or
more is meaningful.
- **DA9.** Randomness is modeled with a **deterministic generator**; the standard
library's random number generator is not used.
- **DA10.** Every measurement is also run on a second corpus with seed **20260219**.
#### Why Brute Force Can Be an Oracle
Brute force is the procedure that enumerates the entire solution space: every pair,
every subset, every ordering. In the text algorithms topic of the Algorithms course,
brute-force pattern matching was established as a **baseline**; the question there was
how many comparisons smarter procedures saved over that baseline. Here the role
changes.
Brute force's privilege in this course has nothing to do with speed but with **having
no precondition**. Two pointers requires the array to be sorted, a sliding window
requires values to be non-negative, greedy selection requires the local choice to fit
the global solution. Exhaustive counting makes no such demand: because it sees every
possibility, none of its assumptions can break. The one procedure with no assumption
to break is the only procedure that can measure everyone else's broken assumption.
This cost is paid in every lesson. The oracle is expensive, and its expense is not a
flaw but the price of being an oracle. An approach being faster than the oracle is not
an achievement; being fast **while giving the same answer** as the oracle is. What
turns this distinction into a number is the count of diverging inputs.
#### Measurement Framework
Every measurement in the course is done on the same corpus and the same counter. The
corpus is built from inputs that deliberately **break** preconditions; batches that
satisfy a precondition are derived from it.
```python
# Measurement framework: corpus and step counter. STEPS are counted, not time.
SEED = 20260218
LENGTH = 12
CORPUS_SIZE = 40
def generator(seed):
"""Deterministic generator. The same seed gives the same corpus."""
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def corpus(seed=SEED, n=CORPUS_SIZE, length=LENGTH):
r = generator(seed)
items = []
for i in range(n):
array = [r(30) - 9 for _ in range(length)] # -9 .. 20
items.append({"no": i + 1, "array": array,
"negative": any(x < 0 for x in array),
"sorted": array == sorted(array)})
return items
class Counter:
def __init__(self):
self.steps = 0
def add(self, n=1):
self.steps += n
items = corpus()
print("corpus:", len(items), "arrays x", LENGTH, "values")
print("sorted:", sum(1 for k in items if k["sorted"]),
"| containing negative:", sum(1 for k in items if k["negative"]))
print("first array:", items[0]["array"])
```
```
corpus: 40 arrays x 12 values
sorted: 0 | containing negative: 40
first array: [-8, -5, 2, -1, 2, 5, 4, 1, 16, 17, 6, -1]
```
**Zero** of the forty arrays are sorted, **forty** contain a negative value. This is
not an accident but a design choice: the base corpus violates most of the
preconditions the course will address. Batches that satisfy a precondition are
produced by sorting these arrays or taking their absolute values, so that the two
batches come from the same generator and the only difference between them is the
precondition itself.
#### The Oracle's Step and Two Shortcuts
The measured problem is this: in an array, do **two distinct positions** exist whose
sum equals the target. The target is taken as 11. Three procedures are written. The
first is the oracle: it sees every pair and does not stop even after finding a match.
The second applies an **early exit**, stopping at the first match. The third applies
**sampling**, looking only at the first six positions.
```python
# Continuing from the previous block: corpus, Counter and items come from there.
TARGET = 11
def oracle_full(array, target, s):
"""Sees every pair, never exits early. This is the oracle."""
found = False
for i in range(len(array)):
for j in range(i + 1, len(array)):
s.add()
if array[i] + array[j] == target:
found = True
return found
def early_exit(array, target, s):
"""EARLY EXIT: stops at the first match. Skipped pairs cannot change the answer."""
for i in range(len(array)):
for j in range(i + 1, len(array)):
s.add()
if array[i] + array[j] == target:
return True
return False
def sampled(array, target, s, look=6):
"""SAMPLING: looks only at the first 'look' positions. An unseen pair is unknown."""
for i in range(min(look, len(array))):
for j in range(i + 1, min(look, len(array))):
s.add()
if array[i] + array[j] == target:
return True
return False
def measure(candidate, oracle, items, target):
"""Compares the candidate with the oracle; counts diverging inputs."""
diverging, candidate_total, oracle_total = [], 0, 0
for k in items:
s1, s2 = Counter(), Counter()
if candidate(k["array"], target, s1) != oracle(k["array"], target, s2):
diverging.append(k["no"])
candidate_total += s1.steps
oracle_total += s2.steps
return {"input": len(items), "diverging": len(diverging), "diverging_no": diverging[:6],
"candidate_steps": candidate_total, "oracle_steps": oracle_total,
"ratio": round(oracle_total / candidate_total, 2) if candidate_total else 0.0}
s = Counter()
for k in items:
oracle_full(k["array"], TARGET, s)
print("oracle (exhaustive count) total steps:", s.steps)
print("early exit", measure(early_exit, oracle_full, items, TARGET))
print("sampling ", measure(sampled, oracle_full, items, TARGET))
```
```
oracle (exhaustive count) total steps: 2640
early exit {'input': 40, 'diverging': 0, 'diverging_no': [], 'candidate_steps': 866, 'oracle_steps': 2640, 'ratio': 3.05}
sampling {'input': 40, 'diverging': 14, 'diverging_no': [1, 2, 3, 4, 8, 9], 'candidate_steps': 406, 'oracle_steps': 2640, 'ratio': 6.5}
```
Three numbers stand side by side. The **oracle** spends **2640 steps** on 40 inputs;
this is exactly 66 pairs per array and does not change with the input. **Early exit**
spends 866 steps — **3.05 times less** than the oracle — and diverges from the oracle
on **none** of the 40 inputs. **Sampling** spends 406 steps — **6.50 times less** than
the oracle — and diverges from the oracle on **14 of the 40** inputs. The
diverging-input ratio is 0.3500, far above the resolution.
The lesson carried by these three rows is the frame for the rest of the course.
**There are two kinds of shortcuts, and the two cannot be told apart from the output
alone.** Early exit preserves correctness because it knows the pairs it skips cannot
change the answer: once a match is already found, the remaining pairs cannot turn the
answer away from `True`. Sampling does not rest on such knowledge; it has no guarantee
about the pairs it never sees, and it genuinely errs on 14 inputs.
The numbers of the diverging inputs also carry a pattern. The first six in the output
are 1, 2, 3, 4, 8, and 9; what they share is that for **every** pair that hits the
target, at least one end falls **outside the first six positions**. Sampling never
sees those pairs and answers "no". The cause of the error is not a computational
mistake but **an assumption about an unseen region**; this is the shared shape of
every approach's error in this course.
The order of the numbers is also meaningful. **The wrong one is faster than the
correct one:** sampling takes 406 steps, early exit 866. If speedup is measured alone,
sampling wins. The moment diverging inputs are added to the measure, the ranking
reverses. This is the first form of the course's second claim: **speeding up moves
you closer to being wrong**, and being wrong is attractive because it is cheap.
#### The Limit of Exhaustive Search
The question of when brute force is acceptable is answered with a step budget: the
number of steps allowed in a run is fixed, and for each search family, the input size
that fits within that budget is computed. The budget here is taken as one hundred
million steps.
```python
from math import comb, factorial
BUDGET = 10**8 # upper limit of steps allowed in one run
def fitting_n(count_fn, limit=20000):
"""The largest n that fits within the step budget."""
best = 0
for n in range(1, limit + 1):
if count_fn(n) > BUDGET:
break
best = n
return best
FAMILIES = (("pair selection", lambda n: comb(n, 2)),
("triple selection", lambda n: comb(n, 3)),
("subsets", lambda n: 2 ** n),
("permutations", factorial))
print("exhaustive search family n=12 steps n=20 steps n fitting budget")
for name, f in FAMILIES:
print(f" {name:16s} {f(12):10d} {f(20):19d} {fitting_n(f):15d}")
print("step budget:", BUDGET)
```
```
exhaustive search family n=12 steps n=20 steps n fitting budget
pair selection 66 190 14142
triple selection 220 1140 844
subsets 4096 1048576 26
permutations 479001600 2432902008176640000 11
step budget: 100000000
```
Four families correspond to four separate worlds under the same budget. Pair
selection tolerates an input above fourteen thousand; triple selection stops at 844;
the search that counts every subset stops at **26**, the one that counts every
permutation at **11**. The gap between them is why this course teaches not just
speedup but **approach selection**: a solution left in the permutation family blows
its budget again the moment the input grows by one element.
It should be noted that this bound does not remove the oracle's role. Even if a
procedure will be called in production with n=1000, comparison against the oracle is
done **on a corpus with n=12**; there, exhaustive counting is 2640 steps and strains
no budget. Brute force ends at small inputs **as a solution**, but stays usable in
every lesson **as a measurement tool**.
#### The Corpus Size Is the Oracle's Budget
The previous table also explains why the corpus is built from twelve-element arrays.
Throughout the course, the oracle will not stay confined to the pair-selection
family; some problems will require scanning every subset. That scan depends directly
on the corpus's element count.
```python
# Continuing from the previous blocks: corpus, Counter and LENGTH come from there.
LIMIT = 30
def oracle_subset(array, limit, s):
"""The largest subset sum not exceeding the limit. Every subset is scanned."""
best = 0
for mask in range(1 << len(array)):
s.add()
total = sum(array[i] for i in range(len(array)) if mask >> i & 1)
if total <= limit and total > best:
best = total
return best
items = corpus()
s = Counter()
for k in items:
oracle_subset(k["array"], LIMIT, s)
print("subsets per array:", 2 ** LENGTH, "| oracle steps on 40 arrays:", s.steps)
for length in (12, 16, 20, 26):
print(f" length {length:2d} -> 40 arrays {40 * 2 ** length:12d} steps",
"(within budget)" if 40 * 2 ** length <= 10**8 else "(outside budget)")
```
```
subsets per array: 4096 | oracle steps on 40 arrays: 163840
length 12 -> 40 arrays 163840 steps (within budget)
length 16 -> 40 arrays 2621440 steps (within budget)
length 20 -> 40 arrays 41943040 steps (within budget)
length 26 -> 40 arrays 2684354560 steps (outside budget)
```
A twelve-element array has 4096 subsets; on forty arrays the oracle spends
**163,840 steps**, using two-thousandths of the budget. The same oracle rises to
41,943,040 steps on twenty-element arrays — it still fits, but eats forty percent of
the budget. At twenty-six elements, the corpus itself exceeds the budget by
**twenty-seven times**.
The rule that follows is the foundation of the course's measurement scheme: **the
corpus's element count is chosen relative to the most expensive oracle to be
measured.** Twelve is the largest comfortable number that still carries even a
subset scan on a corpus of forty inputs. Growing the corpus does not give richer
input; it only makes the oracle unreachable and removes the measurement. Just as
there is a limit to speeding up an approach, there is a limit to **testing** it, and
both are paid from the same budget.
#### The Second Corpus
Whether the ratio measured on a single corpus depends on that corpus is tested
separately.
```python
# Continuing from the previous blocks: corpus, measure and the three procedures come from there.
for label, seed in (("first corpus (20260218)", 20260218),
("second corpus (20260219)", 20260219)):
items = corpus(seed)
e = measure(early_exit, oracle_full, items, TARGET)
o = measure(sampled, oracle_full, items, TARGET)
print(label)
print(" early exit: diverging", e["diverging"], "/ 40 | ratio",
round(e["diverging"] / 40, 4), "| candidate steps", e["candidate_steps"])
print(" sampling : diverging", o["diverging"], "/ 40 | ratio",
round(o["diverging"] / 40, 4), "| candidate steps", o["candidate_steps"])
```
```
first corpus (20260218)
early exit: diverging 0 / 40 | ratio 0.0 | candidate steps 866
sampling : diverging 14 / 40 | ratio 0.35 | candidate steps 406
second corpus (20260219)
early exit: diverging 0 / 40 | ratio 0.0 | candidate steps 970
sampling : diverging 21 / 40 | ratio 0.525 | candidate steps 512
```
Early exit's diverging count is **zero** in both corpora; this is the expected
result, because early exit's correctness rests not on the input but on the
observation that skipped branches cannot change the answer. Sampling's ratio rises
from **0.3500 to 0.5250**. The two ratios are of the same order of magnitude — both
between a third and a half of the inputs — but the difference between them is far
above the resolution. The reading is this: there **are** inputs where sampling is
wrong, and they are **countable**; how many inputs it is wrong on **depends on the
corpus** and cannot be reported with a single number.
#### Summary
- In this course, brute force is not a slow option but an **oracle**; its privilege
comes not from speed but from having no precondition to break.
- On the same problem, the oracle spends **2640 steps** on 40 inputs, early exit
**866 steps** (3.05 times less, **0 diverging inputs**), sampling **406 steps**
(6.50 times less, **14 diverging inputs**).
- The two kinds of shortcut cannot be told apart from the output: early exit knows
the branches it skips cannot change the answer, sampling does not.
- The wrong one is faster than the correct one; until diverging inputs are added to
the measure, sampling looks like the winner.
- With a hundred-million-step budget, pair selection fits up to 14,142 elements,
triple selection up to 844, subset search up to 26, permutation search up to 11;
the oracle's role is unaffected by this bound because measurement is done on a
12-element corpus.
- On the second corpus, early exit's diverging count is still 0, sampling's is 21
instead of 14; the ratio stays the same order of magnitude, but its exact value
**depends on the corpus**.
#### Next Step
In this lesson, brute force ran undivided, as a single piece. The first design
approach splits the problem: it reduces the same problem to two smaller copies of
itself, solves each one, and combines the results. The next lesson establishes this
approach's recurrence and counts two questions — on how many inputs a wrong answer
appears when the combine step is left incomplete, and where the input size ends at
which splitting saves no steps.
### Divide and Conquer
Source: https://academia.sh/en/courses/advanced-algorithms/divide-and-conquer
Splitting and combining are separate calculations: on the same split, linear combining spends 1.16 times fewer steps than the oracle and stays correct on all 40 inputs, while quadratic combining is 1.71 times slower than the oracle, and incomplete combining, which skips the boundary-crossing solution, gives a wrong answer on 40 of 40 inputs with 920 steps.
In the previous lesson, brute force ran as a single piece and saw every possibility.
The first design approach breaks this wholeness: it reduces the problem to two
smaller copies of the same kind, solves each recursively, and **combines** the two
solutions. This approach is called **divide and conquer**.
The Algorithms course already measured three examples of this approach: merge sort,
quicksort, and binary search. This lesson does not repeat those procedures. What it
measures is **the recurrence itself**, and the question it asks is this: splitting
and combining are separate calculations; on how many inputs does a wrong answer
appear when the combine step is left incomplete, and where does the input size end
at which splitting saves no steps.
- **DA11.** The measured problem: the **contiguous subarray** with the largest sum in
an array. The subarray cannot be empty, so in the worst case the answer is a single
element.
- **DA12.** The oracle sees every contiguous subarray; a 12-element array has 78
subarrays.
- **DA13.** A **step** is adding an element to a sum and comparing it. **The
recursive call itself is also a step** and is counted separately.
- **DA14.** The split rule is the same in every variant: the array is divided in
half, so $a = 2$ and $b = 2$ in the recurrence.
- **DA15.** Combining is measured in three variants — **linear**, **quadratic**,
**incomplete**. All three use the same split; the only difference between them is
the combine step.
- **DA16.** The approach's **precondition**: the combine step must also cover the
solution's form that **crosses the boundary**. Incomplete combining breaks only
this precondition.
- **DA17.** The length sweep doubles from 2 to 128; each length still has 40 arrays.
- **DA18.** The sorting and search procedures from M01/K04 are **not repeated**; the
measure here is the recurrence itself.
- **DA19.** Every measurement is also run on a second corpus with seed **20260219**.
#### The Recurrence of Splitting
A divide-and-conquer procedure consists of three tasks: splitting the problem into
$a$ parts, solving each part recursively, and combining the parts' solutions. If the
parts are $1/b$ the size of the original problem, the cost satisfies this recurrence:
$$
T(n) = a \cdot T(n/b) + f(n)
$$
Here $f(n)$ is the cost of **combining**, and it also includes the splitting itself.
The **master theorem** gives the solution of this recurrence by looking at the race
between $f(n)$ and $n^{\log_b a}$. There are three cases: if combining is cheaper
than the recursion, cost is determined by the recursion and the result is
$\Theta(n^{\log_b a})$; if the two are of the same magnitude, a $\log n$ factor
enters and the result is $\Theta(n^{\log_b a} \log n)$; if combining dominates, the
result is directly $\Theta(f(n))$.
In this lesson's split, $a = 2$, $b = 2$, and $\log_b a = 1$. So the race is over
whether combining is linear. Linear combining falls into the second case and gives
$\Theta(n \log n)$; quadratic combining falls into the third case and gives
$\Theta(n^2)$ — that is, **the same magnitude as brute force**. The split does not
change, yet the result does; this is the formal expression of splitting and
combining being separate calculations. A limit of the master theorem should also be
noted here: if the parts are not of equal size, if $f(n)$ falls into none of the three
cases — the gap between the cases has to be **polynomial** — or if in the third case
$f(n)$ does not satisfy the regularity condition ($a \cdot f(n/b) \le c \cdot f(n)$ for
some $c < 1$), the theorem gives no answer, and the recurrence must be solved by
direct expansion. Because this lesson's split is equal, that case is not
encountered.
#### The Oracle and the Problem
The measured problem is the plainest one where a boundary-crossing solution exists:
the contiguous subarray with the largest sum. The oracle counts every subarray.
```python
# Shared framework: corpus and step counter (same as the previous lesson).
SEED = 20260218
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def corpus(seed=SEED, n=40, length=12):
r = generator(seed)
return [{"no": i + 1, "array": [r(30) - 9 for _ in range(length)]}
for i in range(n)]
class Counter:
def __init__(self):
self.steps = 0
def add(self, n=1):
self.steps += n
def oracle_max(array, s):
"""Sees every contiguous subarray. This is the oracle."""
best = array[0]
for i in range(len(array)):
total = 0
for j in range(i, len(array)):
s.add()
total += array[j]
if total > best:
best = total
return best
items = corpus()
s = Counter()
answers = [oracle_max(k["array"], s) for k in items]
print("corpus:", len(items), "arrays x 12 values | oracle total steps:", s.steps)
print("steps per array:", s.steps // len(items), "= 12 x 13 / 2")
print("first three answers:", answers[:3])
```
```
corpus: 40 arrays x 12 values | oracle total steps: 3120
steps per array: 78 = 12 x 13 / 2
first three answers: [52, 55, 84]
```
The oracle spends **78 steps** per array, **3120 steps** on forty arrays, and this
does not change with the input. The size of the answers is also meaningful: 52, 55,
and 84 for the first three arrays. Since the corpus's values are at most 20, these
answers must span **more than one element**; the solution is almost never a single
element.
#### Three Combine Steps, One Split
The procedure below carries three variants in a single body. The split is the same
in all three; the only thing that changes is how the boundary-crossing solution is
computed.
```python
# Continuing from the previous block: corpus, Counter and items come from there.
def divide_conquer(array, s, combine="linear"):
"""T(n) = 2 T(n/2) + combine(n). The combine step finds the BOUNDARY-CROSSING solution."""
def solve(start, end):
s.add() # the recursive call is also a step
if end - start == 1:
return array[start]
mid = (start + end) // 2
left = solve(start, mid)
right = solve(mid, end)
if combine == "incomplete": # BOUNDARY-CROSSING SOLUTION IS SKIPPED
return max(left, right)
if combine == "quadratic": # correct but expensive combining
left_sums, t = [], 0
for i in range(mid - 1, start - 1, -1):
s.add()
t += array[i]
left_sums.append(t)
right_sums, t = [], 0
for j in range(mid, end):
s.add()
t += array[j]
right_sums.append(t)
best = None
for a in left_sums: # every boundary pair is tried separately
for b in right_sums:
s.add()
if best is None or a + b > best:
best = a + b
return max(left, right, best)
best_left, t = None, 0 # linear combining
for i in range(mid - 1, start - 1, -1):
s.add()
t += array[i]
if best_left is None or t > best_left:
best_left = t
best_right, t = None, 0
for j in range(mid, end):
s.add()
t += array[j]
if best_right is None or t > best_right:
best_right = t
return max(left, right, best_left + best_right)
return solve(0, len(array))
def measure(combine, items):
diverging, approach_total, oracle_total = [], 0, 0
for k in items:
s1, s2 = Counter(), Counter()
if divide_conquer(k["array"], s1, combine) != oracle_max(k["array"], s2):
diverging.append(k["no"])
approach_total += s1.steps
oracle_total += s2.steps
return {"diverging": len(diverging), "diverging_no": diverging[:6],
"approach_steps": approach_total, "oracle_steps": oracle_total,
"ratio": round(oracle_total / approach_total, 2)}
for name in ("linear", "quadratic", "incomplete"):
print(f"{name:10s} combining:", measure(name, items))
```
```
linear combining: {'diverging': 0, 'diverging_no': [], 'approach_steps': 2680, 'oracle_steps': 3120, 'ratio': 1.16}
quadratic combining: {'diverging': 0, 'diverging_no': [], 'approach_steps': 5320, 'oracle_steps': 3120, 'ratio': 0.59}
incomplete combining: {'diverging': 40, 'diverging_no': [1, 2, 3, 4, 5, 6], 'approach_steps': 920, 'oracle_steps': 3120, 'ratio': 3.39}
```
Three numbers side by side. The **oracle**: 3120 steps. **Linear combining** spends
2680 steps — 1.16 times fewer than the oracle — and diverges on **none** of the 40
inputs. **Quadratic combining** spends 5320 steps, that is, **1.71 times slower**
than the oracle, and it too does not diverge on any of the 40 inputs. **Incomplete
combining** spends 920 steps, 3.39 times fewer than the oracle, and diverges on
**40 of the 40** inputs.
Two readings stand together. First: **being correct does not require being fast.**
Quadratic combining always gives the correct answer but is slower than brute force;
here splitting gains nothing, and on top of that it adds the recursive calls' own
steps. Second: **being fast does not require being correct.** Incomplete combining
is the cheapest and looks tempting for exactly that reason; yet its answer is wrong
on all 40 inputs.
Looking at what the incomplete variant returns shows the shape of the error. Once
the boundary-crossing solution is skipped, recursion descends to the leaves and, at
every level, the better of the two halves is taken; the only candidate left is the
array's **largest single element**. While the oracle's first three answers are 52,
55, and 84, incomplete combining's answer can be at most 20. The error is not a
computational mistake but **a region of the solution space that is never visited**
— the same shape as the previous lesson's sampling error, this time at the
recursion boundary.
#### As the Recurrence Turns into Numbers
The master theorem's two cases become visible in the measured step count as the
length grows.
```python
# Continuing from the previous blocks: corpus, Counter, oracle_max, divide_conquer come from there.
print("length oracle steps linear steps ratio quadratic steps ratio")
for length in (2, 4, 8, 16, 32, 64, 128):
items = corpus(length=length)
s1, s2, s3 = Counter(), Counter(), Counter()
for k in items:
oracle_max(k["array"], s1)
divide_conquer(k["array"], s2, "linear")
divide_conquer(k["array"], s3, "quadratic")
print(f"{length:6d} {s1.steps:13d} {s2.steps:13d} {s1.steps / s2.steps:6.2f}"
f" {s3.steps:16d} {s1.steps / s3.steps:6.2f}")
```
```
length oracle steps linear steps ratio quadratic steps ratio
2 120 200 0.60 240 0.50
4 400 600 0.67 840 0.48
8 1440 1560 0.92 2680 0.54
16 5440 3800 1.43 8600 0.63
32 21120 8920 2.37 28760 0.73
64 83200 20440 4.07 101080 0.82
128 330240 46040 7.17 371160 0.89
```
Linear combining's ratio rises from 0.60 to 7.17 and **crosses 1 between 8 and 16**.
The meaning is clear: **splitting gains nothing up to 8 elements — on the contrary,
it costs.** At length 2, divide and conquer spends almost twice as many steps as the
oracle; the gain only starts at 16. The loss comes from the recursive calls — an
$n$-element array makes $2n - 1$ calls, and none of these exist in brute force. This
is the measured reason real implementations cut off recursion on small segments and
switch to a direct solution.
Quadratic combining's ratio, in contrast, stays between 0.50 and 0.89 and **never
crosses 1 at any length**. The ratio slowly approaches 1, because both procedures
grow as $\Theta(n^2)$ and the difference between them is a constant factor. This is
exactly the master theorem's third case: when combining dominates, the $\log n$ gain
from splitting disappears, and only the recursion's overhead remains. **Splitting is
not a gain by itself; it is a gain only if combining is cheap.**
Splitting the total step count into two components makes the master theorem's
second case directly visible.
```python
# Continuing from the previous blocks: corpus, Counter and divide_conquer come from there.
from math import log2
print("length calls combining total per array n x log2(n)")
for length in (8, 16, 32, 64, 128):
items = corpus(length=length)
s = Counter()
for k in items:
divide_conquer(k["array"], s, "linear")
calls = 2 * length - 1
total = s.steps // len(items)
print(f"{length:6d} {calls:6d} {total - calls:11d} {total:18d}"
f" {int(length * log2(length)):13d}")
```
```
length calls combining total per array n x log2(n)
8 15 24 39 24
16 31 64 95 64
32 63 160 223 160
64 127 384 511 384
128 255 896 1151 896
```
The combining column and the last column are **exactly the same**: 24, 64, 160, 384,
896. The recurrence's solution here is not an approximation but a counted equality —
at every level, a total of $n$ steps is spent, and there are $\log_2 n$ levels. The
calls column, by contrast, grows separately as $2n - 1$ and is linear; this is the
component that dominates at small $n$ and becomes negligible at large $n$. At
length 8, calls take up 38% of the total; at 128, 22%. This ratio is exactly the
numerical source of splitting's loss on small inputs.
#### The Second Corpus
```python
# Continuing from the previous blocks: corpus, measure come from there.
SECOND = 20260219
for label, seed in (("first corpus (20260218)", SEED),
("second corpus (20260219)", SECOND)):
items = corpus(seed)
print(label)
for name in ("linear", "incomplete"):
o = measure(name, items)
print(f" {name:10s} diverging {o['diverging']:2d} / 40 | ratio",
round(o["diverging"] / 40, 4), "| approach steps", o["approach_steps"])
```
```
first corpus (20260218)
linear diverging 0 / 40 | ratio 0.0 | approach steps 2680
incomplete diverging 40 / 40 | ratio 1.0 | approach steps 920
second corpus (20260219)
linear diverging 0 / 40 | ratio 0.0 | approach steps 2680
incomplete diverging 39 / 40 | ratio 0.975 | approach steps 920
```
Linear combining's step count is **exactly the same** in both corpora: 2680. This is
not surprising, because this procedure's step count depends only on length, not on
the values — neither brute force nor divide and conquer has an early exit.
Incomplete combining's diverging count drops from 40 to 39; the ratio from 1.0000 to
0.9750. The difference is one input and **below the resolution**, so it is
considered unmeasured. In the second corpus, the one array that does not diverge is
the ninth: ten of its twelve values are zero or negative, and its single large
value, 20, is both the largest element and the largest subarray sum. There,
incomplete combining gives the correct answer **for the wrong reason** — it still
never searched for the boundary-crossing solution; what it looked for this time
happened, this time, to be enough. An approach agreeing with the oracle on a single input
does not mean its precondition holds; this is why the measure is taken over 40
inputs.
#### Summary
- The divide-and-conquer recurrence is $T(n) = a \cdot T(n/b) + f(n)$; in this
lesson $a = 2$, $b = 2$, and $f(n)$ is the only thing that determines the result.
- On the same split, linear combining spends 2680 steps — 1.16 times fewer than the
oracle — and is correct on all 40 inputs; quadratic combining spends 5320 steps,
**1.71 times slower** than the oracle, but is still correct.
- Incomplete combining, which skips the boundary-crossing solution, is the cheapest
at 920 steps and is wrong on **40 of the 40** inputs; what it returns is the
array's largest single element.
- Splitting gains nothing up to 8 elements; the ratio crosses 1 between 8 and 16,
and the loss comes from $2n - 1$ recursive calls.
- With quadratic combining, the ratio never crosses 1 at any length; in the master
theorem's third case, splitting's $\log n$ gain disappears.
- On the second corpus, linear combining's step count is exactly the same;
incomplete combining's diverging count is 39 instead of 40; the one-input
difference is below the resolution.
#### Next Step
Divide and conquer split the problem and solved **every part**. The next approach
does something bolder: it makes a single choice at every step and **never goes
back**. This boldness sometimes has a proof — as with the three greedy procedures in
the Algorithms course — and sometimes does not. The next lesson counts when greedy
choice is wrong: on how many four-value coin systems greedy gives more coins than
necessary.
### Greedy Algorithms
Source: https://academia.sh/en/courses/advanced-algorithms/greedy-algorithms
Greedy choice is not a procedure until it is proven: on 827 of 969 four-value coin systems, greedy gives more coins than necessary, a ratio of 0.8535, with the largest excess being 16 coins; when the same error is tested only up to amount 10 instead of 50, it appears in just 85 of the systems.
Divide and conquer split the problem and solved **every part**. This lesson's
approach behaves more boldly: at every step it makes the choice that looks best at
that moment, never questions that choice again, and never goes back. The approach is
called **greedy**, and its appeal is obvious — because there is no backtracking, the
step count is small.
Boldness has a cost. In divide and conquer, the precondition concerned combining and
was visible to the eye. Here the precondition is far more hidden: **the locally best
choice must be part of the globally best solution.** This cannot be seen by looking
at the input; it is either proven or wrong. This lesson establishes what the proof is
and counts how wrong things go when there is none.
- **DA20.** The measured problem: paying a given amount with the fewest coins in a
coin system.
- **DA21.** A coin system has the form $(1, a, b, c)$. Having 1 guarantees every
amount is payable, so an "unpayable" case does not contaminate the measurement.
- **DA22.** The greedy procedure starts from the largest coin and never goes back. A
step is subtracting one coin from the remainder.
- **DA23.** The oracle is brute force: it counts every combination of coins. The
bottom-up solution is tested against the oracle before being used in place of it.
- **DA24.** The sweep covers every triple between 2 and 20 — **969 systems** — and
tries every amount from 1 to 50 in each system.
- **DA25.** A **diverging input** here is a system: one where greedy gives too many
coins for at least one amount is counted.
- **DA26.** **Excess** is greedy's coin count minus the minimum.
- **DA27.** The greedy procedures from M01/K04 — Dijkstra's algorithm, minimum
spanning tree algorithms, and Huffman coding — are **not repeated**; only why they
do not go wrong is written.
- **DA28.** The second corpus is five-value systems generated with seed
**20260219**.
#### Greedy Choice and the Burden of Proof
A greedy procedure builds a solution step by step, and at each step it takes
whichever candidate looks best by some measure. The measure is fixed, and the
choice is never undone. For such a procedure to be correct, it suffices to show a
single thing: **the choice made at every step is present in at least one of the
optimal solutions.** This is called the greedy choice property, and it is shown in
almost the same way every time — an **exchange** argument.
The Algorithms course established and proved three examples of this reasoning. In
the minimum spanning tree lesson, the cut property showed that the lightest edge
crossing a cut is present in some optimal tree: adding that edge to a tree that
lacks it creates a cycle, from which a heavier edge crossing the same cut can be
removed. In the Huffman Coding lesson, the same structure showed that the two
rarest symbols can be made siblings. In Dijkstra's algorithm, the choice was that
the unsettled node with the smallest estimate becomes settled, and this rested on
edge weights being non-negative.
What the three share is that the choice's **safety was shown**. This lesson does
not repeat those procedures; the question it asks is what happens when there is no
proof. The answer is something countable.
#### Verifying the Oracle
The procedure that gives the fewest coins for the change-making problem is the
bottom-up solution. But this course's rule requires the oracle to be **brute
force**. This is why the bottom-up solution is tested against the oracle before
being used in the broad sweep.
```python
# Shared framework: step counter (same as the previous lessons).
from itertools import combinations
class Counter:
def __init__(self):
self.steps = 0
def add(self, n=1):
self.steps += n
def greedy_coins(system, amount, s):
"""Starts from the largest coin, never goes back."""
remaining, count = amount, 0
for p in sorted(system, reverse=True):
while remaining >= p:
s.add()
remaining -= p
count += 1
return count if remaining == 0 else None
def dp_coins(system, amount, s):
"""Exact bottom-up solution."""
best = [0] + [10**9] * amount
for t in range(1, amount + 1):
for p in system:
if p <= t:
s.add()
if best[t - p] + 1 < best[t]:
best[t] = best[t - p] + 1
return best[amount] if best[amount] < 10**9 else None
def oracle_brute_coins(system, amount, s):
"""Counts every combination of coins. This is the oracle."""
best = None
def visit(i, remaining, count):
nonlocal best
s.add()
if remaining == 0:
if best is None or count < best:
best = count
return
if i == len(system):
return
p = system[i]
for k in range(remaining // p + 1):
visit(i + 1, remaining - k * p, count + k)
visit(0, amount, 0)
return best
EXAMPLES = [(1, 3, 7, 12), (1, 4, 9, 16), (1, 5, 10, 25), (1, 18, 19, 20)]
diverging, oc, dc = 0, Counter(), Counter()
for system in EXAMPLES:
for amount in range(1, 26):
if oracle_brute_coins(system, amount, oc) != dp_coins(system, amount, dc):
diverging += 1
print("verification: ", len(EXAMPLES), "systems x 25 amounts =", len(EXAMPLES) * 25, "inputs")
print("oracle (brute force) steps:", oc.steps, "| bottom-up steps:", dc.steps,
"| ratio:", round(oc.steps / dc.steps, 2))
print("diverging from oracle:", diverging)
```
```
verification: 4 systems x 25 amounts = 100 inputs
oracle (brute force) steps: 13584 | bottom-up steps: 2785 | ratio: 4.88
diverging from oracle: 0
```
There is **no divergence** in any of the hundred inputs; brute force spends 13,584
steps, the bottom-up solution 2785. This does not **prove** the bottom-up solution
correct — testing is not proof — but it puts using it in place of the oracle in the
969-system sweep on measured ground. The reason brute force is not used directly is
also visible: the 4.88-times gap is harmless over 100 inputs, but for 969 systems
and 50 amounts it would needlessly inflate the sweep.
#### Nine Hundred Sixty-Nine Systems
The main measurement sweeps every four-value system.
```python
# Continuing from the previous block: Counter, greedy_coins, dp_coins and combinations come from there.
def greedy_sweep(limit=20, amount_limit=50):
"""Every {1,a,b,c} system: in how many systems, and at how many amounts, does greedy give too much."""
broken, total_systems, largest_excess, example = 0, 0, 0, None
for a, b, c in combinations(range(2, limit + 1), 3):
system = (1, a, b, c)
total_systems += 1
s = Counter()
bad = 0
excess_max = 0
for t in range(1, amount_limit + 1):
greedy = greedy_coins(system, t, s)
dp = dp_coins(system, t, s)
if greedy is not None and dp is not None and greedy > dp:
bad += 1
if greedy - dp > excess_max:
excess_max = greedy - dp
if bad:
broken += 1
if excess_max > largest_excess:
largest_excess = excess_max
example = (system, bad, excess_max)
return {"systems": total_systems, "broken": broken,
"ratio": round(broken / total_systems, 4),
"largest_excess": largest_excess, "example": example}
result = greedy_sweep()
print("systems swept:", result["systems"], "| greedy gives too much in:", result["broken"],
"| ratio:", result["ratio"])
print("largest excess:", result["largest_excess"], "| example system:", result["example"])
```
```
systems swept: 969 | greedy gives too much in: 827 | ratio: 0.8535
largest excess: 16 | example system: ((1, 18, 19, 20), 2, 16)
```
On **827 of 969 systems**, greedy gives more coins than necessary. The ratio is
**0.8535**, meaning roughly five-sixths of four-value coin systems are not suited
to greedy. The largest excess is **16 coins**, seen in the system $(1, 18, 19, 20)$.
This number deserves its due. Greedy choice fits everyone's intuition in the
change-making problem: give the largest coin, shrink the remainder. The intuition
is not wrong — it genuinely gives the correct answer on 142 systems. What is wrong
is the intuition **standing in for a procedure**. Greedy choice is a heuristic; it
is not a procedure until it is proven.
#### Where and How Much It Is Wrong
Where the largest excess appears shows the shape of the error.
```python
# Continuing from the previous blocks: Counter, greedy_coins, dp_coins come from there.
SYSTEM = (1, 18, 19, 20)
s = Counter()
print("system", SYSTEM, "- amounts where greedy gives too much")
for amount in range(1, 51):
greedy = greedy_coins(SYSTEM, amount, s)
dp = dp_coins(SYSTEM, amount, s)
if greedy > dp:
print(f" amount {amount:2d}: greedy {greedy:2d} coins, minimum {dp:2d} coins,"
f" excess {greedy - dp:2d}")
print()
print("sweep limit broken systems ratio")
for amount_limit in (10, 20, 50, 100):
result = greedy_sweep(amount_limit=amount_limit)
print(f"{amount_limit:12d} {result['broken']:15d} {result['ratio']:6.4f}")
```
```
system (1, 18, 19, 20) - amounts where greedy gives too much
amount 36: greedy 17 coins, minimum 2 coins, excess 15
amount 37: greedy 18 coins, minimum 2 coins, excess 16
sweep limit broken systems ratio
10 85 0.0877
20 466 0.4809
50 827 0.8535
100 827 0.8535
```
At amount 36, the minimum solution is two coins: 18 plus 18. Greedy instead takes
20, and can only pay the remaining 16 with ones, using **17 coins**. The same thing
happens at 37: the minimum solution is 18 plus 19, greedy's solution is 20 plus
seventeen ones. The source of the error is visible — **taking the largest coin
leaves behind a remainder that is expensive to pay.** Greedy does not see this
remainder, because when making its choice it looks only at the largest value at
that moment.
The bottom table says something more unsettling. When the same 969 systems are
tested only up to 10, **85 systems** look broken, a ratio of 0.0877. Tested up to
20, 466 systems, a ratio of 0.4809. Tested up to 50, 827, a ratio of 0.8535. Going
up to 100 changes nothing. So in this system family, every error appears below 50,
yet **someone testing only up to 10 would think 91 percent of the systems are
clean**. The scope of testing determines the result itself; calling an approach
"tested" means nothing unless it also states **tested up to what**.
#### A Testable Sufficient Condition
If a heuristic cannot stand in for a proof, at least a **testable condition** can
be sought. For coin systems, the first candidate that comes to mind is that the
values form a divisibility chain: each value divides the next.
```python
# Continuing from the previous block: Counter, greedy_coins, dp_coins, combinations come from there.
chain_clean, chain_broken, other_clean, other_broken = 0, 0, 0, 0
clean_examples = []
for a, b, c in combinations(range(2, 21), 3):
system = (1, a, b, c)
s = Counter()
broken = any(greedy_coins(system, t, s) > dp_coins(system, t, s)
for t in range(1, 51))
chain = b % a == 0 and c % b == 0 # does each value divide the next
if chain and broken:
chain_broken += 1
elif chain:
chain_clean += 1
elif broken:
other_broken += 1
else:
other_clean += 1
if len(clean_examples) < 4:
clean_examples.append(system)
print("has divisibility chain : clean", chain_clean, "| broken", chain_broken)
print("no divisibility chain : clean", other_clean, "| broken", other_broken)
print("examples of clean systems without a chain:", clean_examples)
```
```
has divisibility chain : clean 13 | broken 0
no divisibility chain : clean 129 | broken 827
examples of clean systems without a chain: [(1, 2, 3, 4), (1, 2, 3, 5), (1, 2, 3, 6), (1, 2, 3, 7)]
```
Of the **13 systems carrying a divisibility chain, all 13 are clean**; under this
condition, greedy never gives too much on any amount. But **129** of the systems
without a chain are also clean. The condition is **sufficient, not necessary**: it
gives a guarantee when it holds, and says nothing when it does not. The system
$(1, 2, 3, 4)$ carries no chain — 3 does not divide 2 — but greedy is not wrong
there either.
This distinction explains why this course never gives up the oracle. The
sufficient condition removes the need for measurement on 13 of the 969 systems. For
the remaining 956, the only source of information is the oracle; the condition not
holding neither clears an approach nor condemns it. **A testable condition comes
before measurement, not instead of it.**
#### The Second Corpus
The measurement moves beyond four-value systems: five-value systems over a wider
range are generated with two seeds.
```python
# Continuing from the previous blocks: Counter, greedy_coins, dp_coins come from there.
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def five_value_corpus(seed, n=40):
"""Five-value {1,a,b,c,d} systems; values between 2..40 and increasing."""
r = generator(seed)
items = []
while len(items) < n:
d = sorted({2 + r(39) for _ in range(4)})
if len(d) == 4:
items.append((1, *d))
return items
for label, seed in (("first corpus (20260218)", 20260218),
("second corpus (20260219)", 20260219)):
systems = five_value_corpus(seed)
broken, largest_diff = 0, 0
for system in systems:
s = Counter()
bad = max((greedy_coins(system, t, s) - dp_coins(system, t, s))
for t in range(1, 61))
if bad > 0:
broken += 1
largest_diff = max(largest_diff, bad)
print(label, "| broken systems", broken, "/ 40 | ratio", round(broken / 40, 4),
"| largest excess", largest_diff)
```
```
first corpus (20260218) | broken systems 39 / 40 | ratio 0.975 | largest excess 21
second corpus (20260219) | broken systems 40 / 40 | ratio 1.0 | largest excess 21
```
For five-value systems, the ratio is 0.9750 and 1.0000; the one-system difference
between the two corpora is below the resolution. Adding more values **does not fix
greedy, it makes it worse** — more values mean more ways to pay the remainder after
taking the largest coin, and therefore more solutions greedy can miss. The largest
excess also rises from 16 to 21.
This completes the contrast with K04's three greedy procedures. There, the
choice's safety was proven, and the proof rested on a property of the input: the
lightest edge crossing a cut, the two rarest symbols, non-negative edge weight. In
a coin system, such a property depends on the system itself and **is absent in
most systems**. Choosing the greedy approach means assuming that property exists;
when the assumption is not written down, the procedure silently gives too much
change.
#### Summary
- A greedy procedure takes whichever choice looks best at every step and never
goes back; for it to be correct, the choice must be shown, by an **exchange
argument**, to be present in one of the optimal solutions.
- The bottom-up solution was tested against brute force before being used in the
broad sweep: **0 divergence** on 100 inputs, brute force 13,584 steps, bottom-up
2785 steps.
- On four-value **827 of 969 systems**, greedy gives too much change; the ratio is
**0.8535** and the largest excess is **16 coins**, at amount 37 in the system
$(1, 18, 19, 20)$; at amount 36 in the same system the excess is **15**.
- The source of the error is that taking the largest coin leaves a remainder that
is expensive to pay back; greedy does not see that remainder at the moment of
choice.
- Had the same sweep been done only up to 10, the number of broken systems would
have been **85**, not 827; the scope of testing determines the result, and
saying "tested" means nothing without stating **tested up to what**.
- On five-value systems, the ratio rises to 0.9750 and 1.0000; adding values does
not fix greedy, it carries the largest excess from 16 to 21.
#### Next Step
Greedy was wrong because it did not see the remainder. There is a cure for this:
solve the remainder too, that is, compute and store the optimal solution of every
subproblem. The bottom-up solution in this lesson already did this and was never
wrong. The next lesson names that procedure and measures its two conditions — the
**overlapping** of subproblems, and whether the optimal solution can be built from
the optimal solutions of subproblems. It will show, with numbers, that when there
is no overlap, storing gains nothing but still takes up space.
### Dynamic Programming
Source: https://academia.sh/en/courses/advanced-algorithms/dynamic-programming
Memoization's two conditions, both counted: in overlapping subproblems, calls drop from 21,891 to 39, while in non-overlapping ones both stay at 39 and the memo table holds 19 entries for nothing; when the memo's key does not carry the whole state, the approach diverges from the oracle on 39 of 40 inputs.
The greedy procedure left behind an expensive remainder because it never questioned
the largest coin it took. The cure was clear and was already used in the previous
lesson: solve the remainder too, that is, compute and store the optimal solution of
every subproblem. That procedure's name is **dynamic programming**.
The approach's appeal is that it brings an exponential search down to a countable
table. Its cost is two separate preconditions, and either can break silently. The
first is **optimal substructure**: the optimal solution can be built from the
optimal solutions of subproblems. The second is **overlapping subproblems**: the
same subproblem is asked more than once. When the first breaks, the answer comes out
wrong; when the second breaks, the answer stays correct but storing gains nothing.
This lesson counts both.
- **DA29.** The measured problem: the largest sum obtainable by selecting at most
three non-adjacent elements in an array. An empty selection is allowed, so the
answer is never below zero.
- **DA30.** The oracle sees every subset: a 12-element array has **4096 subsets**.
- **DA31.** A **step** is a recursive call in memoization, computing a cell in the
bottom-up table, or trying a subset in the oracle.
- **DA32.** **Memoization** adds a memo table to the recursive solution; the
**bottom-up** solution builds the same recurrence without recursion.
- **DA33.** The approach's **precondition**: the memo's key must carry the entire
state that uniquely determines the subproblem. An incomplete key breaks only this
precondition.
- **DA34.** The second precondition is optimal substructure, and it is kept
unbroken in this lesson; the case where it breaks belongs to the classical
problems topic.
- **DA35.** The overlap measurement is taken directly from the shared definition
and is not redefined.
- **DA36.** Memo size is measured by entry count, table size by cell count.
- **DA37.** Every measurement is also run on a second corpus with seed **20260219**.
#### Overlap Is Counted
Memoization's gain equals how many times subproblems repeat. This is not something
to be guessed but something to be counted. The two procedures below run for the
same $n$; in one, every subproblem is asked twice, in the other, every call produces
a new subproblem.
```python
# Overlap measurement from the shared definition: same n, two different subproblem structures.
class Counter:
def __init__(self):
self.steps = 0
def add(self, n=1):
self.steps += n
def overlapping(n, memoized=True):
"""Overlapping subproblems: every value is called twice."""
s = Counter()
memo = {}
def f(k):
s.add()
if k < 2:
return k
if memoized and k in memo:
return memo[k]
d = f(k - 1) + f(k - 2)
memo[k] = d
return d
f(n)
return {"calls": s.steps, "entries": len(memo)}
def non_overlapping(n, memoized=True):
"""Non-overlapping subproblems: every call is a different subproblem."""
s = Counter()
memo = {}
def f(start, end):
s.add()
if end - start <= 1:
return end - start
if memoized and (start, end) in memo:
return memo[(start, end)]
mid = (start + end) // 2
d = f(start, mid) + f(mid, end)
memo[(start, end)] = d
return d
f(0, n)
return {"calls": s.steps, "entries": len(memo)}
print(" n overlap memoized overlap unmemoized no-overlap memoized no-overlap unmemoized memo")
for n in (10, 15, 20):
a, b = overlapping(n, True), overlapping(n, False)
c, d = non_overlapping(n, True), non_overlapping(n, False)
print(f"{n:2d} {a['calls']:16d} {b['calls']:19d} {c['calls']:21d}"
f" {d['calls']:22d} {c['entries']:6d}")
```
```
n overlap memoized overlap unmemoized no-overlap memoized no-overlap unmemoized memo
10 19 177 19 19 9
15 29 1973 29 29 14
20 39 21891 39 39 19
```
In the overlapping structure, for $n = 20$, the call count drops from **21,891 to
39**; a 561-times gain. In the non-overlapping structure, for the same $n$, the
memoized and unmemoized versions **both make 39** calls — the memo never hits even
once — yet it still holds **19 entries**. What the second row says is clear:
**memoization's condition is overlap.** Without overlap, the memo table only takes
up space; the answer stays correct, the gain is zero, and memory goes to waste.
The difference between the two structures lies in the shape of the recurrence. In
the overlapping structure, two branches descend to the same values, and the
subproblem space is of size $n$; in the non-overlapping one, every branch descends
into its own range, and the subproblem space is as large as the call count. This is
the question to ask before choosing an approach: **is the subproblem space smaller
than the call count.**
#### The Memo's Key Must Carry the Entire State
Memoization's second and quieter precondition concerns the memo table. The problem
below has a two-variable state: which position we are at, and how many choices we
have left. The same procedure is run with two different keys.
```python
# Continuing from the previous block: Counter comes from there.
SEED = 20260218
PICKS = 3 # maximum number of elements that can be selected
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def corpus(seed=SEED, n=40, length=12):
r = generator(seed)
return [{"no": i + 1, "array": [r(30) - 9 for _ in range(length)]}
for i in range(n)]
def oracle_select(array, k, s):
"""Sees every subset; values those that are non-adjacent and at most k elements."""
best = 0
for mask in range(1 << len(array)):
s.add()
if mask & (mask << 1): # two adjacent picks exist
continue
if bin(mask).count("1") > k:
continue
best = max(best, sum(array[i] for i in range(len(array))
if mask >> i & 1))
return best
def memoized(array, k, s, key="full"):
"""key='full' -> memo keyed by (position, remaining); 'partial' -> position only."""
memo = {}
def f(i, remaining):
s.add()
if i >= len(array) or remaining == 0:
return 0
memo_key = (i, remaining) if key == "full" else i
if memo_key in memo:
return memo[memo_key]
d = max(f(i + 1, remaining), array[i] + f(i + 2, remaining - 1))
memo[memo_key] = d
return d
result = f(0, k)
return result, len(memo)
def measure(key, items):
diverging, approach_total, oracle_total, entries = [], 0, 0, 0
for item in items:
s1, s2 = Counter(), Counter()
value, size = memoized(item["array"], PICKS, s1, key)
if value != oracle_select(item["array"], PICKS, s2):
diverging.append(item["no"])
approach_total += s1.steps
oracle_total += s2.steps
entries += size
return {"diverging": len(diverging), "diverging_no": diverging[:6], "approach_steps": approach_total,
"oracle_steps": oracle_total, "ratio": round(oracle_total / approach_total, 2), "memo_entries": entries}
items = corpus()
print("oracle: per array,", 2 ** 12, "subsets")
for key in ("full", "partial"):
print(f"memo key {key:8s}:", measure(key, items))
```
```
oracle: per array, 4096 subsets
memo key full : {'diverging': 0, 'diverging_no': [], 'approach_steps': 2440, 'oracle_steps': 163840, 'ratio': 67.15, 'memo_entries': 1200}
memo key partial : {'diverging': 39, 'diverging_no': [1, 2, 3, 5, 6, 7], 'approach_steps': 1000, 'oracle_steps': 163840, 'ratio': 163.84, 'memo_entries': 480}
```
Three numbers side by side. The **oracle** spends **163,840 steps** on 40 inputs.
**Full-key memoization** spends 2440 steps — **67.15 times fewer** than the oracle —
and diverges on none of the 40 inputs. **Partial-key memoization** spends 1000
steps, **163.84 times fewer** than the oracle, and diverges from the oracle on
**39 of the 40** inputs.
The mechanism of this error differs from the previous two lessons' and is more
dangerous. Nothing is skipped here; the procedure visits every branch. The problem
is that the value written to the memo **belongs to the wrong question.** The best
value computed at the fifth position with three picks remaining is returned when
that same position is reached with only one pick remaining. The memo hits, the
computation is skipped, the step count drops — and the answer breaks. The drop in
step count here is not a speedup but **a symptom of the error**: 1000 steps instead
of 2440, the memo holding 480 entries instead of 1200, means that **more than half**
of the subproblems that should be computed are never computed at all.
#### Memoization Versus Bottom-Up
The same recurrence can also be built without recursion. This is dynamic
programming's second form, and the difference is measurable.
```python
# Continuing from the previous blocks: corpus, Counter, memoized, oracle_select and items come from there.
def bottom_up(array, k, s):
"""Builds the same recurrence without recursion; fills every (position, remaining) cell."""
n = len(array)
table = [[0] * (k + 1) for _ in range(n + 2)]
for i in range(n - 1, -1, -1):
for remaining in range(1, k + 1):
s.add()
table[i][remaining] = max(table[i + 1][remaining],
array[i] + table[i + 2][remaining - 1])
return table[0][k], n * k
s1, s2, s3 = Counter(), Counter(), Counter()
diverging, memo_entries, table_cells = 0, 0, 0
for item in items:
a, size = memoized(item["array"], PICKS, s1, "full")
b, cells = bottom_up(item["array"], PICKS, s2)
oracle_select(item["array"], PICKS, s3)
if a != b:
diverging += 1
memo_entries += size
table_cells += cells
print("oracle steps :", s3.steps)
print("memoization steps :", s1.steps, "| memo entries:", memo_entries)
print("bottom-up steps :", s2.steps, "| table cells:", table_cells)
print("inputs where they diverge:", diverging, "/ 40")
```
```
oracle steps : 163840
memoization steps : 2440 | memo entries: 1200
bottom-up steps : 1440 | table cells: 1440
inputs where they diverge: 0 / 40
```
The two forms give the same answer on all 40 inputs, but their numbers differ. The
bottom-up solution spends **1440 steps**, memoization **2440**; the difference is
the recursive calls themselves. In exchange, memoization holds **1200 entries**,
bottom-up **1440 cells**. That is, per array, memoization holds 30 entries, the
table 36 cells.
The trade-off lies exactly here. The bottom-up solution fills every cell —
including the ones that are never reachable — and in exchange pays no call
overhead. Memoization computes only the subproblems that are actually asked, 30 of
36 cells in this problem, and in exchange pays one call per subproblem. Which one
wins depends on **how much of the subproblem space is reachable**; in this problem,
five-sixths of the space is reachable, so bottom-up comes out ahead. As the
reachable fraction drops, memoization comes out ahead.
#### Optimal Substructure Is a Testable Claim
Both forms above use the same recurrence: the best value at a position is whichever
is better between skipping that position and moving to the next, or taking that
position and moving two ahead. This recurrence's correctness rests on the
**optimal substructure** assumption, and the assumption is testable — every cell of
the recurrence can be asked of the oracle separately.
```python
# Continuing from the previous blocks: corpus, Counter, oracle_select, PICKS and items come from there.
def oracle_suffix(array, start, remaining, s):
"""Finds the suffix's best value by brute force."""
return oracle_select(array[start:], remaining, s)
cells, matching, s = 0, 0, Counter()
for item in items:
array = item["array"]
for i in range(len(array)):
for remaining in range(1, PICKS + 1):
cells += 1
skip = oracle_suffix(array, i + 1, remaining, s) # i is not selected
take = array[i] + oracle_suffix(array, i + 2, remaining - 1, s) # i is selected
if max(skip, take, 0) == oracle_suffix(array, i, remaining, s):
matching += 1
print("cells tested:", cells, "| cells where the recurrence matches the oracle:", matching)
print("oracle steps:", s.steps)
```
```
cells tested: 1440 | cells where the recurrence matches the oracle: 1440
oracle steps: 1719960
```
The recurrence matches the oracle on **1440 of 1440 cells**. The cost of this test
should be noted: 1,719,960 steps, more than a thousand times the solution itself.
Testing optimal substructure is more expensive than solving the problem, and this
is not surprising — every cell requires solving the suffix from scratch by brute
force.
Two things follow from this. First, **optimal substructure is not a heuristic but
a testable claim**; once a recurrence is written, its validity can be asked of the
oracle cell by cell. Second, this test is done not in production but **during
design, on small input**. Once the recurrence is validated once, what remains is a
solution that runs at 1440 steps on 40 inputs; the 1.72-million-step test is
a one-time cost for that solution's correctness.
#### The Second Corpus
```python
# Continuing from the previous blocks: corpus, measure come from there.
for label, seed in (("first corpus (20260218)", 20260218),
("second corpus (20260219)", 20260219)):
items = corpus(seed)
print(label)
for key in ("full", "partial"):
o = measure(key, items)
print(f" key {key:8s} diverging {o['diverging']:2d} / 40 | ratio",
round(o["diverging"] / 40, 4), "| approach steps", o["approach_steps"],
"| memo entries", o["memo_entries"])
```
```
first corpus (20260218)
key full diverging 0 / 40 | ratio 0.0 | approach steps 2440 | memo entries 1200
key partial diverging 39 / 40 | ratio 0.975 | approach steps 1000 | memo entries 480
second corpus (20260219)
key full diverging 0 / 40 | ratio 0.0 | approach steps 2440 | memo entries 1200
key partial diverging 37 / 40 | ratio 0.925 | approach steps 1000 | memo entries 480
```
The step and memo counts are **exactly the same** in both corpora: 2440 with 1200,
1000 with 480. This is expected, because these procedures' steps depend only on the
array's length and the pick limit, not on its values. Partial key's diverging count
drops from 39 to 37; the ratio from 0.9750 to 0.9250. The two-input difference is
below the three-input threshold the resolution considers meaningful, and is
**considered unmeasured**. The reading is the same in both corpora: a partial key
gives a wrong answer on almost every input.
#### Summary
- Dynamic programming has two preconditions: optimal substructure and overlapping
subproblems; when the first breaks the answer is wrong, when the second breaks
the gain disappears.
- In the overlapping structure, calls for n=20 drop from 21,891 to 39; in the
non-overlapping structure, the memoized and unmemoized versions both make 39
calls, and the memo holds 19 entries for nothing.
- Full-key memoization spends 67.15 times fewer steps than the oracle and
diverges on 0 inputs; the partial-key version spends 163.84 times fewer and
diverges on 39 inputs.
- The drop in step count with a partial key is not a speedup but a symptom of
error; the memo holds 480 entries instead of 1200, meaning more than half the
subproblems are never computed.
- The bottom-up solution spends 1440 steps, fewer than memoization's 2440, but
holds 1440 cells; memoization holds 1200 entries. The winner is decided by how
much of the subproblem space is reachable.
- On the second corpus, the step and memo counts are exactly the same; the
partial key's diverging count is 37 instead of 39, and this two-input
difference is considered unmeasured.
#### Next Step
Dynamic programming walked and stored the subproblem space in its entirety. In some
problems, the space is so large that walking all of it is not an option; there, the
only path is to prove that most of the space contains no solution and cut it. The
next lesson measures that cut: pruned search visits 552 nodes on a seven-queens
board while unpruned search visits 960,800, and the ratio between the two numbers
grows as $n$ grows.
### Backtracking
Source: https://academia.sh/en/courses/advanced-algorithms/backtracking
Pruning is the one shortcut that preserves correctness, and what happens when its criterion breaks: on a seven-queens board, pruned search visits 552 nodes and unpruned search 960,800, but over-pruning, which also cuts the neighboring column, finishes in 82 nodes and loses every solution.
Dynamic programming walked and stored the subproblem space in its **entirety**, and
it could do this because the space fit into a table. In some problems, the space
does not fit into a table: an eight-queens board has 16,777,216 placements, and
only 92 of them are solutions. In such a space, the only path is to **prove that
most of it contains no solution and cut it**.
The approach is called **backtracking**: the solution is built piece by piece, and
the moment the piece built so far violates a constraint, that branch is abandoned
and the previous decision is returned to. The cutting operation is called
**pruning**. Pruning holds a special place among the shortcuts seen in this course
— like early exit in the first lesson, it **preserves** correctness, but only if
its criterion is correct. This lesson counts that "only."
- **DA38.** The measured problem: the **number** of placements of $n$ queens on an
$n \times n$ board that do not threaten each other.
- **DA39.** A **step** is a node in the search tree; leaves are counted as nodes
too.
- **DA40.** The **oracle is unpruned search**: it generates every placement and
tests validity at the end. Because it cuts no branch, it cannot miss any
solution.
- **DA41.** The approach's **precondition**: the pruning criterion must cut only
branches that are certainly unsolvable.
- **DA42.** Three criteria are measured — **correct**, **over-pruning** (also cuts
a branch that contains a solution), **under-pruning** (does not cut an
unsolvable branch).
- **DA43.** Here the **corpus is boards**: on a six-queens board, three squares are
forbidden, over 40 boards.
- **DA44.** A **diverging input** is a board where the oracle's and the approach's
solution counts differ.
- **DA45.** The 16,777,216 placements unpruned search would scan on an
eight-queens board are **computed, not run**; the step budget from the first
lesson requires this.
- **DA46.** Every measurement is also run on a second corpus with seed
**20260219**.
#### The Partial Solution Tree
Backtracking sees the solution as a sequence of decisions. Every decision produces
a node, and every node's children are the options that follow that decision. This
structure is called the **partial solution tree**, and its leaves are candidate
complete solutions.
Unpruned search walks this whole tree and tests validity only at the leaves.
Pruned search tests **at every node**: if the partial solution already violates a
constraint, every leaf beneath that node will violate it too, and the branch can
be cut. The legitimacy of cutting rests exactly on this inference — **it must be
proven that the subtree contains no solution.**
When the proof weakens, an error is made in one of two directions. If the
criterion cuts **too much**, branches containing solutions go with it and the
answer comes out short. If it cuts **too little**, unsolvable branches keep being
walked; this alone does not give a wrong answer, but if the test done at the leaf
is also weak, invalid placements get counted as solutions.
There is no way to tell the two forms of error apart from the output. A short
list of placements and a long one both keep returning as a list; the only thing
that can say which is correct is the number produced by the oracle that cuts no
branches at all.
#### The Space Pruning Cuts
```python
# Backtracking measurement from the shared definition: the same search, pruned and unpruned.
class Counter:
def __init__(self):
self.steps = 0
def add(self, n=1):
self.steps += n
def queens(n, prune=True):
"""The unpruned version tries every placement; the pruned version stops on conflict."""
s = Counter()
solutions = []
def visit(row, placement):
s.add()
if row == n:
solutions.append(tuple(placement))
return
for column in range(n):
if prune and any(column == y or abs(column - y) == row - i
for i, y in enumerate(placement)):
continue
placement.append(column)
visit(row + 1, placement)
placement.pop()
visit(0, [])
if not prune:
solutions = [c for c in solutions
if all(c[i] != c[j] and abs(c[i] - c[j]) != j - i
for i in range(n) for j in range(i + 1, n))]
return {"nodes": s.steps, "solutions": len(solutions)}
for n in (5, 6, 7):
pruned, unpruned = queens(n, True), queens(n, False)
print(f"n={n} pruned nodes {pruned['nodes']:5d} solutions {pruned['solutions']:3d}"
f" | unpruned nodes {unpruned['nodes']:7d} solutions {unpruned['solutions']:3d}"
f" | ratio {unpruned['nodes'] / pruned['nodes']:7.1f}")
pruned8 = queens(8, True)
print(f"n=8 pruned nodes {pruned8['nodes']:5d} solutions {pruned8['solutions']:3d}"
f" | unpruned would scan {8 ** 8} placements (ratio {8 ** 8 / pruned8['nodes']:.1f})")
```
```
n=5 pruned nodes 54 solutions 10 | unpruned nodes 3906 solutions 10 | ratio 72.3
n=6 pruned nodes 153 solutions 4 | unpruned nodes 55987 solutions 4 | ratio 365.9
n=7 pruned nodes 552 solutions 40 | unpruned nodes 960800 solutions 40 | ratio 1740.6
n=8 pruned nodes 2057 solutions 92 | unpruned would scan 16777216 placements (ratio 8156.2)
```
The solutions columns are the same across all three rows: 10, 4, 40. Pruning
misses no solution, so the **diverging count is zero**. The nodes columns,
however, show the gap: **552 versus 960,800 nodes** on a seven-queens board, a
ratio of **1740.6**. On the eight-queens board, pruned search finishes in 2057
nodes; unpruned search would scan 16,777,216 placements, and the ratio would be
**8156.2**.
The ratio grows with $n$: 72.3, then 365.9, then 1740.6, then 8156.2 — roughly
fivefold at every step. **Pruning's gain grows as scale grows.** But a second
reading balances this: the pruned node count itself also climbs — 54, 153, 552,
2057 — the step ratios being 2.83, 3.61, and 3.73. **Pruning cuts the exponential space but
does not eliminate the exponent.** A board twice as large stays out of reach even
with pruned search; pruning defers, it does not solve.
#### Which Constraint the Cut Comes From
The queens problem has two constraints — same column and same diagonal — and
pruning was using both together. The constraints can also be applied separately;
because each cuts only branches that are **certainly unsolvable**, all three
preserve correctness.
```python
# Continuing from the previous block: Counter comes from there.
def partial_pruning(n, criterion):
"""criterion: none | column | diagonal | both. All variants fully validate at the leaf."""
s = Counter()
solutions = 0
def visit(row, placement):
nonlocal solutions
s.add()
if row == n:
if all(placement[i] != placement[j] and abs(placement[i] - placement[j]) != j - i
for i in range(n) for j in range(i + 1, n)):
solutions += 1
return
for column in range(n):
if criterion in ("column", "both") and column in placement:
continue
if criterion in ("diagonal", "both") and any(
abs(column - y) == row - i for i, y in enumerate(placement)):
continue
placement.append(column)
visit(row + 1, placement)
placement.pop()
visit(0, [])
return {"nodes": s.steps, "solutions": solutions}
print("n=7 criterion nodes solutions")
for criterion in ("none", "column", "diagonal", "both"):
r = partial_pruning(7, criterion)
print(f" {criterion:9s} {r['nodes']:7d} {r['solutions']:7d}")
```
```
n=7 criterion nodes solutions
none 960800 40
column 13700 40
diagonal 10736 40
both 552 40
```
The solutions column is **40** in all four rows; all four are correct. The nodes
column shows what pruning's power depends on. The column constraint alone brings
960,800 nodes down to 13,700 — 70.1 times. The diagonal constraint alone brings it
down to 10,736 — 89.5 times. Both together give **552**, that is, 1740.6 times.
Combined cutting gains far more than the sum of the two single cuts; because most
of the branches one constraint leaves behind are cut by the other.
The design rule that follows is measured: **the earlier a constraint can be
proven, the more it cuts.** Testing constraints at the leaf is validation; testing
them at the node is pruning; the difference between them is, in this problem, the
gap between 960,800 and 552.
#### When the Pruning Criterion Is Wrong
The same search is run with three different criteria. Two of them break the
precondition.
```python
# Continuing from the previous block: Counter and queens come from there.
def queens_mode(n, mode):
"""mode: correct | over (also cuts the neighboring column) | under (only the previous row)"""
s = Counter()
solutions = []
def conflict(column, row, placement):
if mode == "under":
i = row - 1
return i >= 0 and (column == placement[i] or abs(column - placement[i]) == 1)
base = any(column == y or abs(column - y) == row - i
for i, y in enumerate(placement))
if mode == "over": # "queens must not be in neighboring columns"
return base or any(abs(column - y) == 1 for y in placement)
return base
def visit(row, placement):
s.add()
if row == n:
solutions.append(tuple(placement))
return
for column in range(n):
if conflict(column, row, placement):
continue
placement.append(column)
visit(row + 1, placement)
placement.pop()
visit(0, [])
return {"nodes": s.steps, "solutions": len(solutions)}
print(" n oracle solutions correct pruning over-pruning under-pruning")
diverging = {"correct": 0, "over": 0, "under": 0}
for n in (5, 6, 7):
oracle = queens(n, False)
row = f"{n:2d} {oracle['solutions']:15d}"
for mode in ("correct", "over", "under"):
r = queens_mode(n, mode)
if r["solutions"] != oracle["solutions"]:
diverging[mode] += 1
row += f" {r['solutions']:4d} ({r['nodes']:5d})"
print(row)
print("boards diverging from oracle (out of 3):", diverging)
```
```
n oracle solutions correct pruning over-pruning under-pruning
5 10 10 ( 54) 0 ( 20) 184 ( 306)
6 4 4 ( 153) 0 ( 39) 2642 ( 3747)
7 40 40 ( 552) 0 ( 82) 45514 (59196)
boards diverging from oracle (out of 3): {'correct': 0, 'over': 3, 'under': 3}
```
Three numbers side by side, and this time both are bad. **Correct pruning** gives
the oracle's solution count on all three boards: 10, 4, 40. **Over-pruning** finds
**zero solutions** on all three — the rule "queens must not sit in neighboring
columns" looks reasonable, but because it cuts branches that are not certainly
unsolvable, it leaves nothing behind. **Under-pruning** only looks at the
previous row, so it cannot see diagonal conflicts in distant rows, and on the
seven-queens board it counts **45,514** placements as solutions instead of 40.
The nodes columns are a trap here. Over-pruning finishes in **82 nodes** on seven
queens; 6.7 times faster than correct pruning's 552 nodes. If a measurement
counts only nodes, over-pruning looks like the **best procedure**. Under-pruning,
walking 59,196 nodes, is the slowest of all — so **being wrong is not always
fast.** Wrongness has two forms, and node count gives away neither of them; the
only thing that gives it away is the oracle's solution count.
#### A Corpus of Forbidden Squares
A single family of boards is not a corpus. The measurement moves to forty
separate examples by adding three forbidden squares to a six-queens board, and it
is run with two seeds.
```python
# Continuing from the previous block: Counter and queens_mode come from there.
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
N = 6
FORBIDDEN_COUNT = 3
def board_corpus(seed, count=40):
"""Every board has 3 forbidden squares. Forbidden squares come from the seed."""
r = generator(seed)
return [{"no": i + 1, "forbidden": {(r(N), r(N)) for _ in range(FORBIDDEN_COUNT)}}
for i in range(count)]
def forbidden_search(n, forbidden, mode, s):
"""mode: oracle (unpruned, validate at the end) | correct | over"""
solutions = []
def valid(c):
if any((i, c[i]) in forbidden for i in range(n)):
return False
return all(c[i] != c[j] and abs(c[i] - c[j]) != j - i
for i in range(n) for j in range(i + 1, n))
def visit(row, placement):
s.add()
if row == n:
if mode != "oracle" or valid(tuple(placement)):
solutions.append(tuple(placement))
return
for column in range(n):
if mode != "oracle":
if (row, column) in forbidden:
continue
if any(column == y or abs(column - y) == row - i
for i, y in enumerate(placement)):
continue
if mode == "over" and any(abs(column - y) == 1 for y in placement):
continue
placement.append(column)
visit(row + 1, placement)
placement.pop()
visit(0, [])
return len(solutions)
for label, seed in (("first corpus (20260218)", 20260218),
("second corpus (20260219)", 20260219)):
boards = board_corpus(seed)
oc, cc, ac = Counter(), Counter(), Counter()
div_correct, div_over, lost = 0, 0, 0
for t in boards:
h = forbidden_search(N, t["forbidden"], "oracle", oc)
d = forbidden_search(N, t["forbidden"], "correct", cc)
a = forbidden_search(N, t["forbidden"], "over", ac)
div_correct += h != d
div_over += h != a
lost += h - a
print(label)
print(" oracle nodes", oc.steps, "| correct pruning nodes", cc.steps,
"| ratio", round(oc.steps / cc.steps, 2))
print(" diverging inputs: correct pruning", div_correct, "/ 40 | over-pruning",
div_over, "/ 40 | lost solutions", lost)
```
```
first corpus (20260218)
oracle nodes 2239480 | correct pruning nodes 4736 | ratio 472.86
diverging inputs: correct pruning 0 / 40 | over-pruning 40 / 40 | lost solutions 92
second corpus (20260219)
oracle nodes 2239480 | correct pruning nodes 4452 | ratio 503.03
diverging inputs: correct pruning 0 / 40 | over-pruning 40 / 40 | lost solutions 90
```
The oracle's node count is **exactly the same** in both corpora: 2,239,480. This
is expected — unpruned search generates every placement without looking at the
forbidden squares, so the node count is independent of the board. Correct
pruning's node count varies between 4736 and 4452, because forbidden squares cut
branches early; the ratio is 472.86 and 503.03.
The diverging counts point the same way in both corpora: correct pruning
**0/40**, over-pruning **40/40**. The lost solution count is 92 and 90; the
two-solution difference between them comes from the placement of the forbidden
squares and is below the threshold the measure considers meaningful. The reading
does not change in either corpus: **correct pruning matches the oracle on all
forty boards, over-pruning differs on all forty as well.**
#### Summary
- Backtracking cuts a branch when it has been proven that no solution exists
beneath it in the partial solution tree; the legitimacy of the cut depends on
that proof.
- Correct pruning visits 552 nodes on a seven-queens board, unpruned search
960,800 (a ratio of 1740.6), and the solution counts stay the same: 10, 4, 40.
- The ratio grows roughly fivefold with $n$ (72.3 · 365.9 · 1740.6 · 8156.2), but
the pruned node count also grows, by a factor rising from 2.83 to 3.73 per step:
pruning does not eliminate the
exponent, it defers it.
- Over-pruning finishes in 82 nodes on seven queens and finds zero solutions; in
a measurement that counts only nodes, it looks like the best procedure.
- Under-pruning walks 59,196 nodes and counts 45,514 placements as solutions
instead of 40; being wrong is not always fast.
- On forty forbidden-square boards, correct pruning diverges on 0/40,
over-pruning on 40/40; the second corpus gives the same result, with 90 lost
solutions instead of 92.
#### Next Step
Every procedure so far has been deterministic: the same input always gave the
same steps and the same answer. The final design approach loosens this guarantee
and, in exchange, asks for something — sometimes for the step count, sometimes for
the answer itself, to be random. The next lesson measures two kinds of
randomization on the same corpus and answers a single question: does the number of
inputs a random procedure is wrong on change when the seed changes.
### Randomized Algorithms
Source: https://academia.sh/en/courses/advanced-algorithms/randomized-algorithms
Randomness can be used in two separate places, and the two are measured separately: unverified sampling returns a wrong value on all 40 of the 40 arrays with no majority, while verified sampling never errs on the same arrays, and on a sixteen-queens board, random column order brings a 10,053-node deterministic search down to an average of 360 nodes.
Every procedure up to this point has been deterministic: the same input always gave
the same steps and the same answer. The final design approach loosens this
guarantee. A random choice is placed inside the procedure, and something is hoped
for in exchange — either not getting stuck in bad regions of the search space, or
reaching an answer without doing an exhaustive count.
**Which** guarantee is loosened is decisive, and there are two options. In the
first, the answer stays certain and only the **step count** becomes random; the
procedure always gives the correct answer, but how many steps it takes is unknown.
In the second, the step count stays bounded and the **answer** becomes
probabilistic; the procedure is fast but sometimes wrong. This lesson counts both
with the same measure and models randomness according to this course's rule: with a
**deterministic generator**.
- **DA47.** Randomness is modeled with a **deterministic generator**; the standard
library's random number generator is not used. Every run is reproducible with
its seed.
- **DA48.** Two seeds are used: **20260218** and **20260219**. A random
procedure's result is **not reported with a single seed**.
- **DA49.** The first measured problem: the **majority element** in a 15-value
array — the value occupying more than half the length. If there is none, the
correct answer is "none."
- **DA50.** The oracle counts every value one by one; it does no sampling at all.
- **DA51.** The corpus is two batches: 40 arrays **with** a majority and 40
**without**.
- **DA52.** A **step** is drawing a sample or comparing a value.
- **DA53.** The approach's **precondition**: the majority element exists. The
second batch breaks this precondition.
- **DA54.** The second measured problem: finding the **first solution** in a
queens placement; column order is randomized.
- **DA55.** The random step count is reported as **minimum, average, and
maximum**; twenty runs are taken, and a single number is not considered
sufficient.
- **DA56.** **Expected performance is the average of the steps**, not time.
#### How Randomness Is Modeled
In this course, randomness is not a library call but a seeded generator. The
reason is measurement: a random procedure's diverging-input count can only be
reported if the run is reproducible.
```python
# Randomness is modeled with a deterministic generator; the standard library's generator is not used.
LENGTH = 15
class Counter:
def __init__(self):
self.steps = 0
def add(self, n=1):
self.steps += n
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def majority_corpus(seed, count=40, has_majority=True):
"""has_majority=True -> one value takes up more than half the length."""
r = generator(seed)
items = []
for i in range(count):
dominant = r(6)
share = 8 + r(3) if has_majority else 5 + r(3) # 8..10 or 5..7
array = [dominant] * share + [6 + r(6) for _ in range(LENGTH - share)]
for j in range(len(array) - 1, 0, -1): # deterministic shuffle
k = r(j + 1)
array[j], array[k] = array[k], array[j]
items.append({"no": i + 1, "array": array})
return items
def oracle_majority(array, s):
"""Counts every value one by one. This is the oracle."""
for i in range(len(array)):
count = 0
for j in range(len(array)):
s.add()
if array[j] == array[i]:
count += 1
if count * 2 > len(array):
return array[i]
return None
WITH_MAJORITY = majority_corpus(20260218)
WITHOUT_MAJORITY = majority_corpus(20260218, has_majority=False)
s1, s2 = Counter(), Counter()
v = sum(1 for item in WITH_MAJORITY if oracle_majority(item["array"], s1) is not None)
y = sum(1 for item in WITHOUT_MAJORITY if oracle_majority(item["array"], s2) is not None)
print("corpus with majority: 40 arrays x", LENGTH, "values | have a majority:", v)
print("corpus without majority:", 40, "arrays | have a majority:", y)
print("oracle steps:", s1.steps, "(with majority) |", s2.steps, "(without majority)")
```
```
corpus with majority: 40 arrays x 15 values | have a majority: 40
corpus without majority: 40 arrays | have a majority: 0
oracle steps: 780 (with majority) | 9000 (without majority)
```
Both batches are built as intended: one has a majority in 40 of 40 arrays, the
other in none. The difference in the oracle's step count is also meaningful —
**780 versus 9000**. When a majority exists, the oracle usually finds it at the
first value it tries and stops; when there is none, it has to count all 15 values
through to the end, that is, exactly 225 steps per array. **A procedure's most
expensive case is the one where the answer is "none."**
#### The Probabilistic Answer and the Role of Verification
A random procedure for the majority element is built in one sentence: pick a
random position, take the value there as a candidate. If a majority exists, the
candidate is correct with probability more than half. The two variants below share
this idea and diverge at a single point — one **verifies** the candidate, the
other does not.
```python
# Continuing from the previous block: Counter, generator, majority_corpus, oracle_majority, WITH_MAJORITY come from there.
def sampled_majority(array, trials, r, s, verify=True):
"""Picks 'trials' random positions. If verify=True, tests the candidate by counting."""
if verify:
for _ in range(trials):
candidate = array[r(len(array))]
s.add()
count = 0
for x in array:
s.add()
if x == candidate:
count += 1
if count * 2 > len(array):
return candidate
return None
sample = [] # no verification: the sample's mode
for _ in range(trials):
s.add()
sample.append(array[r(len(array))])
best, best_count = None, -1
for a in sample:
c = sample.count(a)
if c > best_count:
best, best_count = a, c
return best
def measure(items, trials, seed, verify):
r = generator(seed)
diverging, ac, oc = 0, Counter(), Counter()
for item in items:
a = sampled_majority(item["array"], trials, r, ac, verify)
if a != oracle_majority(item["array"], oc):
diverging += 1
return {"diverging": diverging, "approach_steps": ac.steps, "oracle_steps": oc.steps}
print("trials seed verified diverging/steps unverified diverging/steps")
for trials in (1, 3, 5):
for seed in (20260218, 20260219):
a = measure(WITH_MAJORITY, trials, seed, True)
b = measure(WITH_MAJORITY, trials, seed, False)
print(f"{trials:6d} {seed} {a['diverging']:12d} / {a['approach_steps']:5d}"
f" {b['diverging']:16d} / {b['approach_steps']:4d}")
```
```
trials seed verified diverging/steps unverified diverging/steps
1 20260218 10 / 640 10 / 40
1 20260219 17 / 640 17 / 40
3 20260218 0 / 944 6 / 120
3 20260219 1 / 928 13 / 120
5 20260218 0 / 944 6 / 200
5 20260219 0 / 1056 7 / 200
```
Three numbers side by side. The **oracle** spends 780 steps. **Verified sampling**
spends 944 to 1056 steps at five trials and diverges on **0 inputs** in both
seeds. **Unverified sampling** spends 200 steps — almost four times fewer than the
oracle — and diverges on **6 and 7** inputs across the two seeds.
The effect of trial count differs across the two columns. In the verified column,
the diverging count drops from 10 and 17 to 0 and 1 at three trials, and to 0 and
0 at five trials: every new trial cuts the probability of all previous trials
failing by more than half. In the unverified column, the diverging count drops
from 10 and 17 to 6 and 7 and **stays there**; it improves as the sample grows but
never reaches zero.
The real difference is not the size of the numbers but **the direction of the
error**. The verified version **cannot return** a wrong value; because it never
accepts a candidate without counting it, it either gives the correct value or
says "none." The 10 divergences seen at one trial do not mean it gave a wrong
value on ten inputs — they mean it **failed to find one** on ten inputs. The
unverified version really does return a wrong value. **The verification step
turns a two-sided error into a one-sided one**, and that is what 944 steps buy
over 200.
#### When the Precondition Breaks
The importance of this distinction becomes visible when the majority element
**does not exist**.
```python
# Continuing from the previous blocks: measure, WITH_MAJORITY and WITHOUT_MAJORITY come from there.
print("precondition seed verified diverging unverified diverging")
for label, items in (("holds ", WITH_MAJORITY), ("broken ", WITHOUT_MAJORITY)):
for seed in (20260218, 20260219):
a = measure(items, 5, seed, True)
b = measure(items, 5, seed, False)
print(f"{label} {seed} {a['diverging']:16d} {b['diverging']:20d}")
```
```
precondition seed verified diverging unverified diverging
holds 20260218 0 6
holds 20260219 0 7
broken 20260218 0 40
broken 20260219 0 40
```
On the 40 arrays without a majority, unverified sampling diverges from the oracle
on **all 40 of the 40** inputs; verified sampling diverges on **none**. The
result is the same across both seeds, so it does not depend on the corpus.
The reason is clear and fits the course's general shape. The unverified
procedure has buried the "a majority exists" assumption inside its code: it
returns the sample's most frequent value and never asks whether that value is
actually a majority. When the precondition breaks, it does not slow down — it
still finishes in 200 steps — but every time, it reports an answer that does not
exist. Verification takes the precondition out of the code and turns it into **a
condition tested at run time**; its cost is one exhaustive count per array, its
gain is not staying silent when the precondition breaks.
#### Certain Answer, Random Steps
The second use of randomness never touches the answer. The previous lesson's
pruned queens search always tried columns left to right; once the order is
randomized, the solution found is still valid, and the only thing that changes is
how many nodes it takes to reach it.
```python
# Continuing from the previous block: Counter and generator come from there.
def queens_first_solution(n, r=None):
"""Stops at the first solution. If column order is random, steps change, the answer does not."""
s = Counter()
found = None
def visit(row, placement):
nonlocal found
s.add()
if row == n:
found = tuple(placement)
return
columns = list(range(n))
if r:
for j in range(n - 1, 0, -1):
k = r(j + 1)
columns[j], columns[k] = columns[k], columns[j]
for column in columns:
if any(column == y or abs(column - y) == row - i
for i, y in enumerate(placement)):
continue
placement.append(column)
visit(row + 1, placement)
placement.pop()
if found is not None:
return
visit(0, [])
return {"nodes": s.steps, "valid": found is not None}
print(" n deterministic seed min average max valid solution")
for n in (8, 12, 16):
deterministic = queens_first_solution(n)
for seed in (20260218, 20260219):
r = generator(seed)
runs = [queens_first_solution(n, r) for _ in range(20)]
nodes = [run["nodes"] for run in runs]
print(f"{n:2d} {deterministic['nodes']:13d} {seed} {min(nodes):6d} {sum(nodes) / 20:9.1f}"
f" {max(nodes):7d} {sum(run['valid'] for run in runs):9d} / 20")
```
```
n deterministic seed min average max valid solution
8 114 20260218 9 35.7 89 20 / 20
8 114 20260219 16 35.0 101 20 / 20
12 262 20260218 15 98.7 491 20 / 20
12 262 20260219 17 90.1 356 20 / 20
16 10053 20260218 19 359.6 3379 20 / 20
16 10053 20260219 19 355.8 1237 20 / 20
```
The last column is **20 / 20** in every row: all forty runs find a valid
placement. The answer is not random; only the steps are. On the sixteen-queens
board, the deterministic order visits **10,053 nodes**, the random order an
average of **359.6 and 355.8 nodes** — roughly 28 times fewer. The averages for
the two seeds are very close to each other, meaning **expected performance does
not depend on the corpus**.
The extreme values complete this table and show why a single average is not
enough. The minimum is 19 nodes, the maximum 3379; the gap between them is 178
times, and the two seeds' maximums are also far apart, 3379 versus 1237. A random
procedure's performance cannot be reported with a single number; **the average is
not a guarantee, it is an expectation.** The deterministic order's 10,053 nodes is
not an accident: a fixed order walks the same solution-free region to the same
depth on every run, and a bad order's cost never shrinks on any run. What
randomization buys is **never trying the same bad order twice**.
#### Cutting the Long Tail
The 3379-node outlier is the random procedure's real problem: most runs are
short, a few are very long. This tail can be cut with a budget — if search
exceeds a set node count, it is abandoned and restarted from scratch with a new
random order.
```python
# Continuing from the previous block: Counter, generator and queens_first_solution come from there.
def budgeted_search(n, r, budget):
"""Gives up if the budget is exceeded. Giving up is not a wrong answer - there is no answer."""
s = Counter()
found = None
def visit(row, placement):
nonlocal found
s.add()
if s.steps > budget or found is not None:
return
if row == n:
found = tuple(placement)
return
columns = list(range(n))
for j in range(n - 1, 0, -1):
k = r(j + 1)
columns[j], columns[k] = columns[k], columns[j]
for column in columns:
if any(column == y or abs(column - y) == row - i
for i, y in enumerate(placement)):
continue
placement.append(column)
visit(row + 1, placement)
placement.pop()
if found is not None or s.steps > budget:
return
visit(0, [])
return s.steps, found
print("budget seed restarts total nodes max valid")
for budget in (100, 400, 100000):
for seed in (20260218, 20260219):
r = generator(seed)
total, attempts, worst, valid = 0, 0, 0, 0
for _ in range(20):
run_nodes = 0
while True:
steps, result = budgeted_search(16, r, budget)
run_nodes += steps
attempts += 1
if result is not None:
valid += 1
break
total += run_nodes
worst = max(worst, run_nodes)
print(f"{budget:6d} {seed} {attempts:16d} {total / 20:13.1f} {worst:7d}"
f" {valid:8d} / 20")
```
```
budget seed restarts total nodes max valid
100 20260218 54 225.8 940 20 / 20
100 20260219 57 240.8 1229 20 / 20
400 20260218 27 228.8 1731 20 / 20
400 20260219 29 276.1 835 20 / 20
100000 20260218 20 359.6 3379 20 / 20
100000 20260219 20 355.8 1237 20 / 20
```
With a budget of a hundred nodes, **54 and 57 attempts** are made for twenty
solutions, meaning between 2.70 and 2.85 restarts per solution. In exchange, the
total node average drops from 359.6 to **225.8**, and the worst run from 3379 to
**940**. The last rows repeat the unbudgeted run and confirm the difference.
The answer still does not break: the last column is 20 / 20 in all six rows.
Giving up is not the same as giving a wrong answer — when the budget is exceeded,
the procedure **says nothing**, and because it says nothing, it does not diverge
from the oracle. The distinction from the course's first lesson takes its final
form here: **a procedure that reports what it does not know can be measured; a
procedure that fabricates what it does not know cannot.**
#### Summary
- Randomization can loosen two separate guarantees: the answer can stay certain
while the step count becomes random, or the step count can stay bounded while
the answer becomes probabilistic. The two are measured separately.
- On the majority element, the oracle spends 780 steps; verified sampling spends
944 to 1056 steps at five trials with 0 diverging inputs, unverified sampling
200 steps with 6 to 7 diverging inputs.
- The verification step makes error one-sided: the verified version cannot
return a wrong value, it can only say "none"; the unverified version really
does return a wrong value.
- When the precondition breaks, unverified sampling diverges on 40 of 40 inputs,
verified sampling on none; the result is the same across both seeds.
- On a sixteen-queens board, the deterministic order visits 10,053 nodes, the
random order an average of 359.6 and 355.8 nodes, and all forty runs find a
valid solution; the extremes range from 19 to 3379, so the average is an
expectation, not a guarantee.
- With a hundred-node budget, restarting brings the average down to 225.8, the
worst run from 3379 to 940, and the answer still stays 20 / 20 valid.
#### Next Step
This topic ran six design approaches through the same measure, and the same thing
came out every time: an approach's value is visible not in the steps it saves but
in the wrong answers it gives once its precondition breaks. The next topic carries
this measure to problem patterns. There, the patterns are narrower and more
recognizable — scanning from both ends in a sorted array, incremental computation
on a contiguous subarray, connected-component search on a grid — and each one's
precondition can be written in a single sentence. The first lesson opens with the
two-pointer pattern, which requires sorted input, and its question is the same as
here: on how many inputs does a wrong answer appear once the ordering breaks.
## Problem-Solving Patterns
Measuring the two pointers, sliding window, fast and slow pointer, interval merging, cyclic sort, two heaps, selection, and grid traversal patterns by their preconditions and the wrong answers they give when those preconditions break.
### Two Pointers
Source: https://academia.sh/en/courses/advanced-algorithms/two-pointers
Scanning from both ends of a sorted array; the 25 wrong answers that appear once the precondition breaks and the step cost of establishing that precondition.
The previous topic showed that an algorithm's correctness can be stated **probabilistically**:
expected performance is a distribution, a single run is one sample. This topic opens with a
different source of uncertainty, and this source is not probabilistic but **structural**.
Choosing a problem-solving pattern is not buying a speedup — it is **accepting a
precondition**.
As long as the precondition holds, the pattern is both correct and cheap. When it does not,
the pattern does not stop, does not warn, does not slow down — it **gives a wrong answer**,
and the wrongness cannot be read from the output. That is why all eight lessons of this
topic's eight build the same frame: an **oracle** (brute force, always correct, always
expensive), a **pattern**, and the **number of inputs where the two diverge**. The first
pattern is two pointers; its precondition is one sentence: **the array must be sorted**.
#### Problem, Oracle, and Pattern
The problem is this: does a **pair of distinct positions summing to the target** exist in an
array. The oracle tries every pair; for an array of n values there are $n(n-1)/2$ pairs, and
when it must, the oracle looks at all of them. The pattern places two pointers at the array's
two ends, moves the left one right if the sum is below target, moves the right one left if it
is above.
**PP1.** The metric is **steps**, not time. The counter counts every comparison as one step,
and real time is measured nowhere.
**PP2.** The corpus comes from a deterministic generator; seed `20260218`. The same seed
gives the same 40 arrays.
**PP3.** Each array carries 12 values, ranging from −9 to 20.
**PP4.** The oracle is brute force and is counted as **always correct**. The pattern's
correctness is claimed only by comparison against the oracle.
**PP5.** The corpus satisfying the precondition is the **sorted** form of the same 40 arrays;
it comes from no other generator. The only difference between the two corpora is order.
```python
SEED, LENGTH, CORPUS_SIZE = 20260218, 12, 40
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def corpus(seed=SEED, n=CORPUS_SIZE, length=LENGTH):
r = generator(seed)
return [{"no": i + 1, "array": [r(30) - 9 for _ in range(length)]}
for i in range(n)]
class Counter:
def __init__(self):
self.step = 0
def count(self, n=1):
self.step += n
def oracle_pairs(array, target, s):
"""Tries every pair. Always correct, always expensive."""
for i in range(len(array)):
for j in range(i + 1, len(array)):
s.count()
if array[i] + array[j] == target:
return True
return False
def pattern_two_pointers(array, target, s):
"""PRECONDITION: array must be sorted."""
left, right = 0, len(array) - 1
while left < right:
s.count()
t = array[left] + array[right]
if t == target:
return True
if t < target:
left += 1
else:
right -= 1
return False
def measure(items, target):
diverging, pk, ok = [], 0, 0
for k in items:
s1, s2 = Counter(), Counter()
a = pattern_two_pointers(k["array"], target, s1)
b = oracle_pairs(k["array"], target, s2)
pk, ok = pk + s1.step, ok + s2.step
if a != b:
diverging.append(k["no"])
return {"diverging": len(diverging), "first_diverging": diverging[:6],
"pattern_step": pk, "oracle_step": ok, "ratio": round(ok / pk, 2)}
K = corpus()
S = [dict(k, array=sorted(k["array"])) for k in K]
print("corpus:", len(K), "arrays x", LENGTH, "values | already sorted:",
sum(1 for k in K if k["array"] == sorted(k["array"])))
for ad, items in (("precondition holds", S), ("precondition broken", K)):
print(f" {ad}", measure(items, 11))
```
```
corpus: 40 arrays x 12 values | already sorted: 0
precondition holds {'diverging': 0, 'first_diverging': [], 'pattern_step': 154, 'oracle_step': 972, 'ratio': 6.31}
precondition broken {'diverging': 25, 'first_diverging': [1, 2, 3, 6, 8, 9], 'pattern_step': 372, 'oracle_step': 866, 'ratio': 2.33}
```
Three numbers sit side by side. When the precondition holds, the pattern gives the same
answer as the oracle on **40 of 40 inputs** and spends **154** steps; the oracle spends
**972**, a ratio of **6.31**. When the precondition breaks, diverging inputs come to **25**
and the ratio drops to **2.33**.
The most important part of the second row is not the diverging-input count, it is that the
two arrive together. **Wrongness is not cheap either:** the pattern's step count climbs from
154 to 372, because in an unsorted array the pointers meet at the ends before finding the
right pair and the pattern cannot exit early. As the speedup falls from 6.31 to 2.33 times,
correctness goes with it.
#### What Happens on a Diverging Input
The divergence has a single cause. While applying the rule "if the sum is small, grow the
left value," the pattern assumes that **the left value grows moving rightward**. In an
unsorted array this assumption is false; once a pointer moves in the wrong direction, the
skipped positions are never revisited.
```python
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def corpus(seed):
r = generator(seed)
return [[r(30) - 9 for _ in range(12)] for _ in range(40)]
def oracle_pairs(array, target):
for i in range(len(array)):
for j in range(i + 1, len(array)):
if array[i] + array[j] == target:
return True, (array[i], array[j])
return False, None
def pattern_two_pointers(array, target):
left, right = 0, len(array) - 1
while left < right:
t = array[left] + array[right]
if t == target:
return True
if t < target:
left += 1
else:
right -= 1
return False
first = corpus(20260218)[0]
print("input 1 :", first)
print(" pattern:", pattern_two_pointers(first, 11))
print(" oracle :", oracle_pairs(first, 11))
print(" sorted :", sorted(first), "-> pattern", pattern_two_pointers(sorted(first), 11))
print()
print("seed target precondition diverging/40")
for seed in (20260218, 20260219):
for target in (11, 25):
K = corpus(seed)
for ad, items in (("holds ", [sorted(d) for d in K]), ("broken ", K)):
diverging = sum(1 for d in items
if pattern_two_pointers(d, target) != oracle_pairs(d, target)[0])
print(f"{seed} {target:5d} {ad} {diverging:8d}"
f" ratio {diverging / 40:.4f}")
```
```
input 1 : [-8, -5, 2, -1, 2, 5, 4, 1, 16, 17, 6, -1]
pattern: False
oracle : (True, (-5, 16))
sorted : [-8, -5, -1, -1, 1, 2, 2, 4, 5, 6, 16, 17] -> pattern True
seed target precondition diverging/40
20260218 11 holds 0 ratio 0.0000
20260218 11 broken 25 ratio 0.6250
20260218 25 holds 0 ratio 0.0000
20260218 25 broken 22 ratio 0.5500
20260219 11 holds 0 ratio 0.0000
20260219 11 broken 24 ratio 0.6000
20260219 25 holds 0 ratio 0.0000
20260219 25 broken 16 ratio 0.4000
```
On the first input, the oracle finds the pair `(-5, 16)`; the pattern returns `False`. Once
the same array is sorted, the pattern also returns `True`. **The input did not change, only
its order did** — what changes the pattern's answer is not the data's content but whether the
precondition holds.
**PP6.** The diverging-input count is out of 40. One divergence in 40 inputs is 0.0250; a
1-input gap **counts as unmeasured**, 3 and above is meaningful.
**PP7.** The second corpus comes from seed `20260219` and is used only to test whether the
ratio holds its **order of magnitude**.
In the second corpus, diverging inputs for target 11 are **24**, versus **25** in the first.
For target 25 they are **16** and **22**. In all four measurements the ratio falls between
0.40 and 0.63 — **the same order of magnitude**; the result does not depend on the corpus.
On all four rows where the precondition holds, diverging inputs are **zero**, and that too is
corpus-independent.
#### The Cost of Establishing the Precondition
The measurement so far leaves a question open: if the array is not sorted, it **can be
sorted**. Then the pattern can still be used. But sorting itself spends steps, and those steps
must be charged to the pattern's account.
**PP8.** When measuring the cost of establishing the precondition, sorting is performed with
a **comparison-based** procedure and comparisons are counted as steps. The sorting procedures
themselves were measured in the Algorithms course; they are not repeated here, only their
step count is counted.
```python
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def corpus(seed=20260218):
r = generator(seed)
return [[r(30) - 9 for _ in range(12)] for _ in range(40)]
class Counter:
def __init__(self):
self.step = 0
def count(self):
self.step += 1
def oracle_pairs(array, target, s):
for i in range(len(array)):
for j in range(i + 1, len(array)):
s.count()
if array[i] + array[j] == target:
return True
return False
def pattern_two_pointers(array, target, s):
left, right = 0, len(array) - 1
while left < right:
s.count()
t = array[left] + array[right]
if t == target:
return True
if t < target:
left += 1
else:
right -= 1
return False
def sort_counting(array, s):
"""The cost of establishing the precondition. Comparisons are counted as steps."""
a = list(array)
for i in range(1, len(a)):
j = i
while j > 0:
s.count()
if a[j - 1] <= a[j]:
break
a[j - 1], a[j] = a[j], a[j - 1]
j -= 1
return a
print("target correct pattern only sorting included oracle ratio")
for target in (11, 25):
only, included, oracle, correct = 0, 0, 0, 0
for array in corpus():
sa, sk, sh = Counter(), Counter(), Counter()
y = pattern_two_pointers(sort_counting(array, sa), target, sk)
h = oracle_pairs(array, target, sh)
only += sk.step
included += sa.step + sk.step
oracle += sh.step
correct += (y == h)
print(f"{target:5d} {correct:2d}/40 {only:12d} {included:14d} {oracle:5d}"
f" {oracle / included:5.2f}")
```
```
target correct pattern only sorting included oracle ratio
11 40/40 154 1737 866 0.50
25 40/40 352 1935 1564 0.81
```
Correctness comes back: **40/40**. But the ratio **drops below 1**. For target 11, total
steps including sorting are **1737**, the oracle spends **866**; the pattern does **twice**
the oracle's work. For target 25 it is 1935 against 1564, a ratio of 0.81.
This is the topic's second claim making its first payment: **a speedup sometimes does not
speed anything up.** At this input size, two pointers forced to establish its own
precondition is more expensive than brute force. The pattern wins where sorting is done
**once** and queried **many times**, or where the data already arrives sorted. As the array
grows this balance shifts — sorting grows as $n \log n$, the oracle as $n^2$ — but in this
corpus 12 values is small enough to leave the balance in brute force's favor.
#### Step Count Says Nothing About Correctness
In the two measurements above, divergence arrived together with a step change: from 154 to
372. That is not a rule, and relying on it is dangerous. A counting form of the same pattern
shows this. The problem is now "how **many pairs** sum below the target"; the pattern uses
the fact that once a pair is counted at the right end, every pair in between is also counted.
**PP9.** In the counting form the pattern cannot exit early; in every run the pointers take
exactly n−1 steps. The step count is therefore independent of the input's content.
```python
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def corpus(seed=20260218):
r = generator(seed)
return [[r(30) - 9 for _ in range(12)] for _ in range(40)]
class Counter:
def __init__(self):
self.step = 0
def count(self):
self.step += 1
def oracle_small_pairs(array, target, s):
"""Number of pairs summing below target. Every pair is tried."""
count = 0
for i in range(len(array)):
for j in range(i + 1, len(array)):
s.count()
if array[i] + array[j] < target:
count += 1
return count
def pattern_small_pairs(array, target, s):
"""PRECONDITION: array must be sorted. Counting a pair at the right end counts the ones between it too."""
left, right, count = 0, len(array) - 1, 0
while left < right:
s.count()
if array[left] + array[right] < target:
count += right - left
left += 1
else:
right -= 1
return count
print("precondition diverging/40 pattern oracle ratio")
for ad, prepare in (("holds ", sorted), ("broken ", list)):
diverging, pk, ok = 0, 0, 0
for array in corpus():
d = prepare(array)
s1, s2 = Counter(), Counter()
a = pattern_small_pairs(d, 6, s1)
b = oracle_small_pairs(d, 6, s2)
pk, ok = pk + s1.step, ok + s2.step
diverging += (a != b)
print(f"{ad} {diverging:8d} {pk:5d} {ok:5d} {ok / pk:5.2f}")
```
```
precondition diverging/40 pattern oracle ratio
holds 0 440 2640 6.00
broken 39 440 2640 6.00
```
The step columns in both rows are **identical**: pattern 440, oracle 2640, ratio 6.00. The
diverging-input column climbs from 0 to **39** — on 39 of 40 inputs the pattern returns a
wrong number. The pattern still returns an integer, still six times faster, still produces no
warning.
This shows why the topic's third claim is necessary. Step count is a performance metric and
**says nothing about correctness**. Brute force here is not a "slow alternative," it is the
**only** tool that makes the 39 wrong answers visible. Without the oracle, these two rows
would be indistinguishable.
#### Three Numbers
| Metric | Oracle | Pattern | Diverging input |
|---|---|---|---|
| Precondition holds (target 11) | 972 steps | 154 steps | **0/40** |
| Precondition broken (target 11) | 866 steps | 372 steps | **25/40** |
| Establishing the precondition (target 11) | 866 steps | 1737 steps | **0/40** |
| Counting form, precondition broken | 2640 steps | 440 steps | **39/40** |
Three rows are three separate decision points. The first row is the pattern's promise. The
second shows what happens when the precondition is not checked: sixty-two percent of the
answers are broken and the speedup has eroded by two thirds. The third row is the bill for
establishing the precondition by hand.
One might think the pattern can be made safe by adding a check: look at whether the array is
sorted, and fall back to the oracle if it is not. That check is n−1 comparisons, that is, 440
steps for 40 arrays. The check **rescues correctness** but does **not rescue the speedup**:
on unsorted input, the work still falls to the oracle. What needs to be measured is not the
check's cost but what share of inputs satisfy the precondition. In this corpus that number is
**zero**.
#### Summary
- Choosing a pattern is accepting a precondition; two pointers' precondition is that the
array be sorted.
- When the precondition holds, the pattern matches the oracle on 40 of 40 inputs and spends
154 steps instead of 972; the ratio is 6.31.
- When the precondition breaks, the pattern diverges from the oracle on **25 inputs** and the
speedup drops to 2.33 times; wrongness is not cheap.
- In the second corpus, diverging inputs are 24; because the ratio holds its order of
magnitude, the result does not depend on the corpus.
- Establishing the precondition by sorting brings correctness back but raises total steps to
1737, making the pattern more expensive than the oracle.
#### Next Step
Two pointers squeezed the array from both ends and its precondition was order. The next
pattern keeps the pointers moving in the **same direction** and grows and shrinks the region
between them like a **window**; its gain comes from updating the sum incrementally instead of
recomputing it from zero each time it slides. Its precondition is also different and never
looks at order at all: **no value may be negative**. The next lesson will count the 10 inputs
where that precondition breaks, using the oracle.
### Sliding Window
Source: https://academia.sh/en/courses/advanced-algorithms/sliding-window
Incremental computation over a contiguous subarray; the 10 inputs where a negative value breaks the shrink rule and why the precondition belongs to the window, not the rule.
The previous lesson placed two pointers at the array's two ends and met them in the middle;
its precondition was order. This lesson's pattern keeps the pointers moving in the **same
direction**. The region between them is a **window**: as the right pointer advances the
window grows, and once a condition is met the left pointer advances, shrinking it.
The gain comes from the sum never being recomputed from zero as the window slides; the
entering value is added, the leaving value is subtracted. Its precondition never looks at
order at all: **no value may be negative**. This lesson counts the inputs where that
precondition breaks and shows that the precondition actually belongs not to the window but to
the **shrink rule**.
#### Problem, Oracle, and Pattern
The problem is this: what is the length of the **shortest contiguous subarray** whose sum is
not below the target. The oracle starts from every possible starting position, expands
rightward, and stops the moment it first exceeds the target; it tries every starting point.
The pattern advances in a single pass: it adds the right-end value to the sum, and as long as
the sum exceeds the target, it narrows the window by subtracting from the left end.
**PP10.** The corpus is the previous lesson's corpus: seed `20260218`, 40 arrays, 12 values
each, values between −9 and 20. **All forty** of the forty arrays contain a negative value.
**PP11.** The corpus satisfying the precondition is the **absolute value** of the same
arrays' values. The only difference between the two corpora is sign; length, position, and
generator are the same.
**PP12.** The target is 25, and a subarray does not have to hit the target exactly;
"not below the target" is sufficient.
**PP13.** The oracle is brute force and is counted as always correct. The answer is a length;
if no subarray hits the target, the answer is undefined.
```python
SEED = 20260218
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def corpus(seed=SEED, n=40, length=12):
r = generator(seed)
return [{"no": i + 1, "array": [r(30) - 9 for _ in range(length)]}
for i in range(n)]
class Counter:
def __init__(self):
self.step = 0
def count(self):
self.step += 1
def oracle_shortest(array, target, s):
"""Tries every contiguous subarray. Always correct."""
best = None
for i in range(len(array)):
total = 0
for j in range(i, len(array)):
s.count()
total += array[j]
if total >= target and (best is None or j - i + 1 < best):
best = j - i + 1
break
return best
def pattern_sliding_window(array, target, s):
"""PRECONDITION: no value may be negative."""
left, total, best = 0, 0, None
for right in range(len(array)):
s.count()
total += array[right]
while total >= target:
if best is None or right - left + 1 < best:
best = right - left + 1
total -= array[left]
left += 1
s.count()
return best
def measure(items, target):
diverging, pk, ok = [], 0, 0
for k in items:
s1, s2 = Counter(), Counter()
a = pattern_sliding_window(k["array"], target, s1)
b = oracle_shortest(k["array"], target, s2)
pk, ok = pk + s1.step, ok + s2.step
if a != b:
diverging.append(k["no"])
return {"diverging": len(diverging), "first_diverging": diverging[:6],
"pattern_step": pk, "oracle_step": ok, "ratio": round(ok / pk, 2)}
K = corpus()
P = [dict(k, array=[abs(x) for x in k["array"]]) for k in K]
print("corpus:", len(K), "arrays | contain a negative value:",
sum(1 for k in K if any(x < 0 for x in k["array"])))
for ad, items in (("precondition holds", P), ("precondition broken", K)):
print(f" {ad}", measure(items, 25))
```
```
corpus: 40 arrays | contain a negative value: 40
precondition holds {'diverging': 0, 'first_diverging': [], 'pattern_step': 867, 'oracle_step': 2396, 'ratio': 2.76}
precondition broken {'diverging': 10, 'first_diverging': [2, 12, 13, 15, 24, 26], 'pattern_step': 761, 'oracle_step': 2462, 'ratio': 3.24}
```
When the precondition holds, the pattern spends **867** steps and matches the oracle on 40 of
40 inputs; the oracle spends **2396** steps, a ratio of **2.76**. On the corpus containing
negative values, diverging inputs come to **10**.
This number is lower than the previous lesson's 25, and that low count is this lesson's most
dangerous feature. The pattern still gives the correct answer on three quarters of the
inputs. If a test set were chosen at random, the odds of a clean result would be high; the
defect surfaces not through observation but through the **oracle**.
The second row's ratio is also worth noting: **3.24**, **higher** than the 2.76 seen when the
precondition holds. The pattern spends fewer steps on the broken input (761 versus 867),
because negative values pull the sum down and the shrink loop runs less often. **Fewer steps
does not mean a better answer.**
#### What the Shrink Rule Rests On
The pattern's one risky line is the `while total >= target` loop. This loop assumes that
subtracting a value from the left will **decrease** the sum. That is true for non-negative
values; the sum decreases monotonically as the left end advances, and the moment the loop
first breaks its condition, the shortest window for that starting point has been found.
When a negative value is subtracted, the sum **increases**. At that moment the window has
narrowed but the sum has grown; the pattern counts this as progress and never takes the left
end back. If a shorter solution exists among the skipped starting points, it is never seen.
**PP14.** The pattern never **backtracks** the left end; each position leaves the left end at
most once. The pattern's linear step count comes from this no-backtracking rule, and the rule
cannot be relaxed.
```python
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def corpus(seed):
r = generator(seed)
return [[r(30) - 9 for _ in range(12)] for _ in range(40)]
def oracle_shortest(array, target):
best = None
for i in range(len(array)):
total = 0
for j in range(i, len(array)):
total += array[j]
if total >= target and (best is None or j - i + 1 < best):
best = j - i + 1
break
return best
def pattern_sliding_window(array, target):
left, total, best = 0, 0, None
for right in range(len(array)):
total += array[right]
while total >= target:
if best is None or right - left + 1 < best:
best = right - left + 1
total -= array[left]
left += 1
return best
second = corpus(20260218)[1]
print("input 2:", second)
print(" pattern:", pattern_sliding_window(second, 25),
"| oracle:", oracle_shortest(second, 25))
print(" absolute value:", [abs(x) for x in second])
print(" pattern:", pattern_sliding_window([abs(x) for x in second], 25),
"| oracle:", oracle_shortest([abs(x) for x in second], 25))
print()
print("seed target precondition diverging/40 ratio")
for seed in (20260218, 20260219):
for target in (25, 15):
K = corpus(seed)
pool = (("holds ", [[abs(x) for x in d] for d in K]), ("broken ", K))
for ad, items in pool:
diverging = sum(1 for d in items
if pattern_sliding_window(d, target) != oracle_shortest(d, target))
print(f"{seed} {target:5d} {ad} {diverging:8d} {diverging / 40:.4f}")
```
```
input 2: [-6, 15, 10, 11, 6, -5, -4, 3, 2, 17, -8, 5]
pattern: 3 | oracle: 2
absolute value: [6, 15, 10, 11, 6, 5, 4, 3, 2, 17, 8, 5]
pattern: 2 | oracle: 2
seed target precondition diverging/40 ratio
20260218 25 holds 0 0.0000
20260218 25 broken 10 0.2500
20260218 15 holds 0 0.0000
20260218 15 broken 6 0.1500
20260219 25 holds 0 0.0000
20260219 25 broken 8 0.2000
20260219 15 holds 0 0.0000
20260219 15 broken 6 0.1500
```
On the second input, the oracle says **2**, the pattern says **3**. The correct answer is
the pair `15 + 10`; the pattern cannot see this pair, because once the left end dropped the
value `-6`, the sum grew and the window locked in the wrong place. The pattern's output is
still a length, still plausible, still wrong.
**PP15.** The second corpus comes from seed `20260219`. If the diverging-input ratio does not
hold its order of magnitude, the result depends on the corpus and is written up that way.
In the second corpus, diverging inputs for target 25 are **8**, versus **10** in the first;
for target 15 both give **6**. All four measurements fall between 0.15 and 0.25, that is,
**the same order of magnitude**. The result does not depend on the corpus. On the four rows
where the precondition holds, diverging inputs are **zero**.
#### The Precondition Belongs to the Rule, Not the Window
The name sliding window covers two separate patterns at once, and their preconditions are not
the same. The form above is **variable-size**: the window grows until a condition is met, and
shrinks once it is. There is also a **fixed-size** form: the window is always k wide, one
value enters from the right, one value leaves from the left.
In the fixed-size form there is no shrink rule; the window does not narrow by checking a
condition, it only slides. So the sum does not need to decrease monotonically either.
**PP16.** The fixed-size window uses only an **incremental sum**: one addition, one
subtraction. The pattern's step count is independent of k; the oracle's step count grows
with k.
```python
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def corpus(seed=20260218):
r = generator(seed)
return [[r(30) - 9 for _ in range(12)] for _ in range(40)]
class Counter:
def __init__(self):
self.step = 0
def count(self):
self.step += 1
def oracle_fixed(array, k, s):
"""Recomputes every window's sum from zero."""
best = None
for i in range(len(array) - k + 1):
total = 0
for j in range(i, i + k):
s.count()
total += array[j]
if best is None or total > best:
best = total
return best
def pattern_fixed(array, k, s):
"""Fixed-size window: one value enters, one leaves. No sign precondition."""
total, best = 0, None
for right in range(len(array)):
s.count()
total += array[right]
if right >= k:
total -= array[right - k]
if right >= k - 1 and (best is None or total > best):
best = total
return best
print("k input diverging/40 pattern oracle ratio")
for k in (3, 4, 6):
for ad, prepare in (("no negatives", lambda d: [abs(x) for x in d]),
("negatives ", list)):
diverging, pk, ok = 0, 0, 0
for array in corpus():
d = prepare(array)
s1, s2 = Counter(), Counter()
diverging += (pattern_fixed(d, k, s1) != oracle_fixed(d, k, s2))
pk, ok = pk + s1.step, ok + s2.step
print(f"{k} {ad} {diverging:12d} {pk:5d} {ok:5d} {ok / pk:5.2f}")
```
```
k input diverging/40 pattern oracle ratio
3 no negatives 0 480 1200 2.50
3 negatives 0 480 1200 2.50
4 no negatives 0 480 1440 3.00
4 negatives 0 480 1440 3.00
6 no negatives 0 480 1680 3.50
6 negatives 0 480 1680 3.50
```
All six rows give zero diverging inputs. Negative values do not affect the fixed-size window
at all; the pattern's step count is **480** at all three k values, the oracle's step count
grows with k, and the ratio climbs from 2.50 to 3.50.
This is the lesson's structural result: **the precondition belongs not to the pattern's name,
but to a single rule inside it.** The sentence "sliding window does not work with negative
values" is false; the correct sentence is "the shrink rule, which rests on the assumption
that the sum decreases monotonically, does not work with negative values." Learning a pattern
together with its precondition means knowing that rule, not knowing the pattern's name.
#### Three Numbers
| Metric | Oracle | Pattern | Diverging input |
|---|---|---|---|
| Variable window, precondition holds | 2396 steps | 867 steps | **0/40** |
| Variable window, precondition broken | 2462 steps | 761 steps | **10/40** |
| Fixed window (k=4), input with negatives | 1440 steps | 480 steps | **0/40** |
The second row is where the pattern spends the fewest steps, and it is the only broken row.
The third row gives zero divergence on the same input, because the pattern changed.
#### A Check Restores Correctness, Not the Speedup
Checking the precondition is cheap: looking for a negative value in an array is at most n
comparisons and stops at the first negative found. If the check fails, the work falls to the
oracle. This arrangement's correctness is complete; what needs asking is how much speedup is
left.
**PP17.** The checked pattern charges the check's steps to its own account too; when the work
is handed off to the oracle, the oracle's steps are added to the pattern's step count as well.
```python
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def corpus(seed=20260218):
r = generator(seed)
return [[r(30) - 9 for _ in range(12)] for _ in range(40)]
class Counter:
def __init__(self):
self.step = 0
def count(self):
self.step += 1
def oracle_shortest(array, target, s):
best = None
for i in range(len(array)):
total = 0
for j in range(i, len(array)):
s.count()
total += array[j]
if total >= target and (best is None or j - i + 1 < best):
best = j - i + 1
break
return best
def pattern_sliding_window(array, target, s):
left, total, best = 0, 0, None
for right in range(len(array)):
s.count()
total += array[right]
while total >= target:
if best is None or right - left + 1 < best:
best = right - left + 1
total -= array[left]
left += 1
s.count()
return best
def checked(array, target, s):
"""Tests the precondition; falls back to the oracle if it fails."""
for x in array:
s.count()
if x < 0:
return oracle_shortest(array, target, s)
return pattern_sliding_window(array, target, s)
for ad, prepare in (("no negatives", lambda d: [abs(x) for x in d]),
("negatives ", list)):
diverging, checked_step, oracle_step = 0, 0, 0
for array in corpus():
d = prepare(array)
s1, s2 = Counter(), Counter()
diverging += (checked(d, 25, s1) != oracle_shortest(d, 25, s2))
checked_step += s1.step
oracle_step += s2.step
print(f"{ad} diverging {diverging}/40 checked pattern {checked_step:5d}"
f" oracle {oracle_step:5d} ratio {oracle_step / checked_step:.2f}")
```
```
no negatives diverging 0/40 checked pattern 1347 oracle 2396 ratio 1.78
negatives diverging 0/40 checked pattern 2568 oracle 2462 ratio 0.96
```
Diverging inputs are **zero** in both rows. But the second row's ratio is **0.96**: on the
corpus with negative values, the checked pattern spends **more** steps than the oracle. The
reason is plain — all forty of the forty arrays fail the check, the work falls to the oracle
all forty times, and the check's 106 steps pile on top. On the corpus where the precondition
holds, the ratio stays at 1.78 — the check writes off a loss there too, from 2.76 down to
1.78.
The reading that follows is this: **a precondition check is a correctness tool, not a
performance tool.** Without knowing what share of inputs satisfy the precondition, nothing
can be said about what the checked pattern will gain; in this corpus that share is zero, and
the pattern has turned entirely into the oracle.
#### Summary
- Sliding window keeps two pointers moving in the same direction and updates the sum
incrementally instead of recomputing it from zero.
- The variable-size form's shrink rule assumes the sum decreases when a value is subtracted
from the left; that assumption holds only for non-negative values.
- On the corpus with negative values, the pattern diverges from the oracle on **10 inputs**
and does so with **fewer steps**; fewer steps is not a sign of correctness.
- In the second corpus, diverging inputs are 8; because the ratio holds its order of
magnitude, the result does not depend on the corpus.
- The fixed-size window carries no shrink rule and is unaffected by negative values: in all
six measurements, diverging inputs are zero.
#### Next Step
Both patterns worked on a single array, with the position count known. The next pattern also
advances its pointers in the same direction, but at **different speeds** and on a structure
of unknown length: every node has a successor, and whether an end exists is not known in
advance. The next lesson measures cycle detection and the middle element on this structure;
its precondition is that progress really is one-directional, and when that precondition
breaks, the pattern does not merely give a wrong answer — it may never stop at all.
### Fast and Slow Pointer
Source: https://academia.sh/en/courses/advanced-algorithms/fast-and-slow-pointer
Cycle detection and the middle element using two pointers moving at different speeds; the 14 cycles a second edge makes invisible and the cost of never stopping.
The previous two patterns worked on an array: the length was known, every position was
reachable through an index. This lesson's structure is different. There is a **starting
node** and every node has a **successor**; the length is unknown, there is no going back, and
whether the structure has an end is not known in advance.
The pattern advances two pointers in the same direction but at **different speeds**: the slow
one takes one step, the fast one takes two. Its precondition is one sentence: **progress must
be one-directional**, that is, every node must have exactly one successor. This lesson counts
the inputs where that precondition breaks and shows that the pattern's real gain is not in
steps but in the **held node count**.
#### Cycle Detection
The first problem is this: does a traversal starting from the beginning run forever. The
oracle searches for a cycle with the Data Structures course's depth-first search; the
"visited" versus "still open" distinction there is **not repeated, it is used directly**.
The pattern holds no set at all: two pointers advance at different speeds and, if a cycle
exists, sooner or later meet at the same node.
**PP18.** The structure has 12 nodes; the nodes are laid out along a path, and the path's end
either terminates or loops back. Seed `20260218`.
**PP19.** The only thing that breaks the precondition is a **second back edge** added to some
nodes. The corpus satisfying the precondition is the same structures with their second edges
dropped.
**PP20.** The pattern always follows the **first successor**; it cannot see whether a second
edge exists, because the only thing it looks at is `successor[d][0]`.
**PP21.** Alongside step count, a second metric is kept: the **number of nodes held at
once**. For the oracle this is the largest size the visited set reaches; for the pattern it is
two pointers.
```python
SEED, NODES, CORPUS_SIZE = 20260218, 12, 40
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def chain_corpus(seed=SEED, n=CORPUS_SIZE, nodes=NODES):
"""Every structure has a path. The path's end either terminates or loops back; a
SECOND back edge is also added to some nodes."""
r = generator(seed)
items = []
for i in range(n):
path = list(range(nodes))
for j in range(nodes - 1, 0, -1):
k = r(j + 1)
path[j], path[k] = path[k], path[j]
successor = {path[j]: [path[j + 1]] for j in range(nodes - 1)}
successor[path[-1]] = [path[r(nodes)]] if r(100) < 40 else [-1]
if r(100) < 60:
p = 2 + r(nodes - 3)
successor[path[p]].append(path[r(p)])
items.append({"no": i + 1, "start": path[0], "successor": successor})
return items
def one_directional(successor):
"""Second edges are dropped: every node keeps exactly one successor."""
return {d: [a[0]] for d, a in successor.items()}
class Counter:
def __init__(self):
self.step, self.held = 0, 0
def count(self, held=0):
self.step += 1
self.held = max(self.held, held)
def oracle_cycle(start, successor, s):
"""Walks every edge. Depth-first search's cycle detection is used directly."""
visited, open_set = set(), set()
def visit(d):
visited.add(d)
open_set.add(d)
s.count(len(visited))
for k in successor.get(d, []):
if k == -1:
continue
if k in open_set:
return True
if k not in visited and visit(k):
return True
open_set.discard(d)
return False
return visit(start)
def pattern_fast_slow(start, successor, s):
"""PRECONDITION: every node must have EXACTLY ONE successor. The pattern follows the first."""
slow = fast = start
while True:
s.count(2)
for _ in range(2):
fast = successor[fast][0]
if fast == -1:
return False
slow = successor[slow][0]
if slow == fast:
return True
def measure(items, one_way):
diverging, pk, ok, hk, ho = [], 0, 0, 0, 0
for k in items:
successor = one_directional(k["successor"]) if one_way else k["successor"]
s1, s2 = Counter(), Counter()
a = pattern_fast_slow(k["start"], one_directional(k["successor"]), s1)
b = oracle_cycle(k["start"], successor, s2)
pk, ok = pk + s1.step, ok + s2.step
hk, ho = max(hk, s1.held), max(ho, s2.held)
if a != b:
diverging.append(k["no"])
return {"diverging": len(diverging), "first_diverging": diverging[:6],
"pattern_step": pk, "oracle_step": ok, "ratio": round(ok / pk, 2),
"pattern_held": hk, "oracle_held": ho}
K = chain_corpus()
print("corpus:", len(K), "structures x", NODES, "nodes | with a second edge:",
sum(1 for k in K if any(len(a) > 1 for a in k["successor"].values())))
for ad, one_way in (("precondition holds", True), ("precondition broken", False)):
print(f" {ad}", measure(K, one_way))
```
```
corpus: 40 structures x 12 nodes | with a second edge: 23
precondition holds {'diverging': 0, 'first_diverging': [], 'pattern_step': 299, 'oracle_step': 480, 'ratio': 1.61, 'pattern_held': 2, 'oracle_held': 12}
precondition broken {'diverging': 14, 'first_diverging': [2, 8, 11, 12, 16, 19], 'pattern_step': 299, 'oracle_step': 480, 'ratio': 1.61, 'pattern_held': 2, 'oracle_held': 12}
```
The step columns of both rows are **identical**: pattern 299, oracle 480, ratio 1.61. The
held-node columns are the same too: pattern **2**, oracle **12**. The only column that
changes is diverging input — from 0 to **14**.
The step ratio of 1.61 is already modest, and it is not the pattern's selling point either.
The pattern wins on **held node count**: the oracle must hold every node of the structure in
a set, the pattern holds nothing but two pointers. As the node count grows, what the oracle
holds grows, while the pattern's **stays at 2**. That is the measured gain, and it **does not
change at all** when the precondition breaks — only the answer breaks.
Why the meeting is guaranteed also rests on the one-successor assumption. Once the two
pointers enter a cycle, the distance between them shrinks by **exactly one** every turn,
because the fast one takes two steps and the slow one takes one. The distance is an integer
and takes one of a number of values equal to the cycle length; because it shrinks by one every
turn, it reaches zero in a finite number of turns. A distance that shrinks by one means
**exactly one path** leaves every node. With a second successor present, there is no such
single number as "the distance between them," and the argument collapses.
#### Why the Second Edge Stays Invisible
The pattern's only source of information is the `successor[d][0]` value. If a node has a
second successor, the pattern never queries it — it cannot, because the pattern's definition
is built on the "single successor" assumption. If a structure's only closing cycle runs
through the second edge, the pattern never enters that cycle and returns `False`.
```python
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def chain_corpus(seed, n=40, nodes=12):
r = generator(seed)
items = []
for i in range(n):
path = list(range(nodes))
for j in range(nodes - 1, 0, -1):
k = r(j + 1)
path[j], path[k] = path[k], path[j]
successor = {path[j]: [path[j + 1]] for j in range(nodes - 1)}
successor[path[-1]] = [path[r(nodes)]] if r(100) < 40 else [-1]
if r(100) < 60:
p = 2 + r(nodes - 3)
successor[path[p]].append(path[r(p)])
items.append({"no": i + 1, "start": path[0], "successor": successor})
return items
def one_directional(successor):
return {d: [a[0]] for d, a in successor.items()}
def oracle_cycle(start, successor):
visited, open_set = set(), set()
def visit(d):
visited.add(d)
open_set.add(d)
for k in successor.get(d, []):
if k == -1:
continue
if k in open_set:
return True
if k not in visited and visit(k):
return True
open_set.discard(d)
return False
return visit(start)
def pattern_fast_slow(start, successor):
slow = fast = start
while True:
for _ in range(2):
fast = successor[fast][0]
if fast == -1:
return False
slow = successor[slow][0]
if slow == fast:
return True
K = chain_corpus(20260218)
example = next(k for k in K
if pattern_fast_slow(k["start"], one_directional(k["successor"]))
!= oracle_cycle(k["start"], k["successor"]))
print("first diverging input no:", example["no"], "| start:", example["start"])
print(" successor:", {d: a for d, a in sorted(example["successor"].items())})
print(" pattern (first successor):", pattern_fast_slow(example["start"], one_directional(example["successor"])))
print(" oracle (all edges):", oracle_cycle(example["start"], example["successor"]))
print()
print("seed precondition cyclic (oracle) diverging/40 ratio")
for seed in (20260218, 20260219):
K = chain_corpus(seed)
for ad, one_way in (("holds ", True), ("broken ", False)):
cyclic = diverging = 0
for k in K:
successor = one_directional(k["successor"]) if one_way else k["successor"]
b = oracle_cycle(k["start"], successor)
cyclic += b
diverging += (pattern_fast_slow(k["start"], one_directional(k["successor"])) != b)
print(f"{seed} {ad} {cyclic:15d} {diverging:8d} {diverging / 40:.4f}")
```
```
first diverging input no: 2 | start: 0
successor: {0: [8], 1: [2], 2: [7], 3: [-1], 4: [1, 8], 5: [10], 6: [5], 7: [6], 8: [9], 9: [11], 10: [3], 11: [4]}
pattern (first successor): False
oracle (all edges): True
seed precondition cyclic (oracle) diverging/40 ratio
20260218 holds 16 0 0.0000
20260218 broken 30 14 0.3500
20260219 holds 18 0 0.0000
20260219 broken 32 14 0.3500
```
In the second input, node 4 has two successors: `1` and `8`. The pattern sees only `1` and
follows the path `0 → 8 → 9 → 11 → 4 → 1 → 2 → 7 → 6 → 5 → 10 → 3 → end`, saying `False`. The
oracle also tries the `4 → 8` edge and finds the cycle `8 → 9 → 11 → 4`.
**PP22.** The second corpus comes from seed `20260219`. In both corpora, diverging inputs are
**14/40**, ratio **0.3500**; the result does not depend on the corpus.
#### The Middle Element and Not Stopping
The second problem is the same pattern's most commonly used second form: finding the
structure's **middle node**. The oracle collects the nodes in order, then picks the middle
one — two passes and a collection. The pattern finishes in a single pass: when the fast
pointer reaches the end, the slow pointer is in the middle.
Here the precondition changes. The middle element is only defined if the structure has an
**end**; if there is a cycle, there is no such node as "the middle." The oracle sees this and
returns `undefined`. There is nothing for the pattern to see: the fast pointer spins forever
inside the cycle.
**PP23.** In this measurement there are no second edges; the structures are single-successor.
The precondition that breaks is **acyclicity**.
**PP24.** The pattern is given a **step cap** (48 steps). A run that hits the cap returns
`did not stop` and is counted as diverging from the oracle. Without the cap the measurement
would never finish.
```python
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def path_corpus(seed=20260218, n=40, nodes=12):
"""`end` is the path's last node; a back link from there means the structure has a cycle."""
r = generator(seed)
items = []
for i in range(n):
path = list(range(nodes))
for j in range(nodes - 1, 0, -1):
k = r(j + 1)
path[j], path[k] = path[k], path[j]
successor = {path[j]: path[j + 1] for j in range(nodes - 1)}
successor[path[-1]] = path[r(nodes)] if r(100) < 40 else -1
r(100)
items.append({"no": i + 1, "start": path[0], "end": path[-1], "successor": successor})
return items
class Counter:
def __init__(self):
self.step, self.held = 0, 0
def count(self, held=0):
self.step += 1
self.held = max(self.held, held)
def oracle_middle(start, successor, s):
"""First collects every node, then picks the middle one. Holds a visited set."""
visited, order, d = set(), [], start
while d != -1 and d not in visited:
visited.add(d)
order.append(d)
s.count(len(visited))
d = successor[d]
if d != -1:
return "undefined"
for _ in range(len(order) // 2 + 1):
s.count(len(visited))
return order[len(order) // 2]
def pattern_middle(start, successor, s, cap=48):
"""PRECONDITION: structure must not contain a cycle. Single pass, holds only two pointers."""
slow = fast = start
while True:
s.count(2)
if s.step > cap:
return "did not stop"
if fast == -1 or successor[fast] == -1:
return slow
fast = successor[successor[fast]]
slow = successor[slow]
K = path_corpus()
A = [dict(k, successor={**k["successor"], k["end"]: -1}) for k in K]
print("corpus:", len(K), "structures | with a cycle:",
sum(1 for k in K if k["successor"][k["end"]] != -1))
print("precondition diverging/40 pattern oracle ratio pattern held oracle held")
for ad, items in (("holds ", A), ("broken ", K)):
diverging, pk, ok, hk, ho = 0, 0, 0, 0, 0
for k in items:
s1, s2 = Counter(), Counter()
a = pattern_middle(k["start"], k["successor"], s1)
b = oracle_middle(k["start"], k["successor"], s2)
pk, ok = pk + s1.step, ok + s2.step
hk, ho = max(hk, s1.held), max(ho, s2.held)
diverging += (a != b)
print(f"{ad} {diverging:8d} {pk:5d} {ok:5d} {ok / pk:5.2f}"
f" {hk:13d} {ho:13d}")
```
```
corpus: 40 structures | with a cycle: 14
precondition diverging/40 pattern oracle ratio pattern held oracle held
holds 0 280 760 2.71 2 12
broken 14 868 662 0.76 2 12
```
When the precondition holds, the pattern finds the same node as the oracle on 40 of 40 inputs
at **280** steps; the oracle spends **760**, a ratio of **2.71**. Once cyclic structures
arrive, diverging inputs come to **14** and the ratio drops to **0.76** — the pattern becomes
**more expensive** than the oracle.
This row shows something the previous two lessons did not. There, when the precondition
broke, the pattern gave a wrong answer and finished in a normal step count. Here the pattern
**does not finish**: most of the 868 steps are pointers spinning until they hit the cap in
each of the fourteen structures. Without a cap, the measurement would never have completed.
**The precondition's cost is not always a wrong answer; sometimes it is no answer at all.**
#### Three Numbers
| Metric | Oracle | Pattern | Diverging input |
|---|---|---|---|
| Cycle detection, precondition holds | 480 steps / 12 nodes | 299 steps / 2 nodes | **0/40** |
| Cycle detection, with a second edge | 480 steps / 12 nodes | 299 steps / 2 nodes | **14/40** |
| Middle element, precondition holds | 760 steps / 12 nodes | 280 steps / 2 nodes | **0/40** |
| Middle element, cyclic structure | 662 steps / 12 nodes | 868 steps / 2 nodes | **14/40** |
In all four rows the pattern's held node count is **2**, the oracle's is **12**. This column
explains why the pattern exists, and it never changes across the four rows. The diverging
input column is zero twice and fourteen twice — reading the other columns without reading
that one means missing that the pattern gives a wrong answer on fourteen inputs.
#### Summary
- Fast and slow pointer finds a cycle and the middle node in a one-directional structure of
unknown length, holding no set at all.
- The precondition is that every node have **exactly one successor**; once a second edge is
added, the pattern never queries it and diverges from the oracle on **14 inputs**.
- The divergence is not reflected in step count: on both corpora the pattern spends 299 steps
and the oracle spends 480.
- The pattern's measured gain is not in steps but in held node count: **2 versus 12**, and
this ratio stays fixed even when the precondition breaks.
- In the middle-element problem the precondition is acyclicity; on a cyclic structure the
pattern does not just give a wrong answer, it hits the step cap and the ratio drops from
2.71 to 0.76.
#### Next Step
The three patterns so far worked in a single pass over a single structure. The next pattern
first **rearranges** the input: merging overlapping intervals begins by sorting the intervals
by some criterion. The precondition is no longer a property of the data but **the criterion
itself** — and sorting by the wrong criterion breaks the pattern. The next lesson compares
three separate sorting criteria against the same oracle and counts how many inputs each merges
wrongly.
### Interval Merging
Source: https://academia.sh/en/courses/advanced-algorithms/interval-merging
Reducing overlapping intervals in a single pass; the 29 inputs a wrong sort key breaks the merge on, and the 8 inputs an event-ordering rule changes the answer for.
The previous three patterns took the input as given: two pointers expected sorted order,
sliding window looked at signs, fast and slow pointer followed a successor link. This
lesson's pattern **rearranges the input first**. Merging overlapping intervals begins by
sorting the intervals by some criterion, and once sorting is done, a single pass suffices.
The precondition is no longer a property of the data; it is **the criterion chosen itself**.
When the criterion is chosen wrongly, the pattern still finishes in a single pass, still
returns a list of intervals, and still gives no warning. This lesson compares three wrong
criteria against the same oracle and counts how many inputs each one merges incorrectly.
#### Problem, Oracle, and Pattern
The problem is this: given a set of intervals, **merge the overlapping ones** to produce the
simplest list. The oracle does no sorting at all; whenever it finds an overlapping pair, it
merges it and repeats until no pair remains. This makes the result correct by definition, but
it restarts the scan from the beginning on every merge.
The pattern sorts once, then scans the list left to right: each interval either extends the
last merged interval's right end, or opens a new merged interval.
**PP25.** The corpus is 40 sets; each set carries 12 intervals. Start 0–60, length 1–9;
seed `20260218`.
**PP26.** Intervals are **closed**: interval `(4, 20)` contains every point between 4 and 20
and overlaps `(20, 25)`.
**PP27.** The oracle's and the pattern's answers are **sorted** before comparison; a
divergence arises not from an order difference but only from a content difference.
**PP28.** Sorting's step count is charged to the pattern's account. A comparison-based
procedure is used and every comparison is counted as one step; the sorting procedures
themselves were measured in the Algorithms course and are not repeated here.
```python
SEED, INTERVAL_COUNT, CORPUS_SIZE = 20260218, 12, 40
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def interval_corpus(seed=SEED, n=CORPUS_SIZE, count=INTERVAL_COUNT):
r = generator(seed)
items = []
for i in range(n):
row = []
for _ in range(count):
start = r(61)
row.append((start, start + 1 + r(9)))
items.append({"no": i + 1, "interval": row})
return items
class Counter:
def __init__(self):
self.step = 0
def count(self):
self.step += 1
def oracle_merge(intervals, s):
"""Merges an overlapping pair whenever found, repeats until no change."""
remaining = [list(a) for a in intervals]
changed = True
while changed:
changed = False
for i in range(len(remaining)):
for j in range(i + 1, len(remaining)):
s.count()
if remaining[i][0] <= remaining[j][1] and remaining[j][0] <= remaining[i][1]:
remaining[i] = [min(remaining[i][0], remaining[j][0]),
max(remaining[i][1], remaining[j][1])]
remaining.pop(j)
changed = True
break
if changed:
break
return sorted(tuple(a) for a in remaining)
def sort_counting(items, key, s):
a = list(items)
for i in range(1, len(a)):
j = i
while j > 0:
s.count()
if key(a[j - 1]) <= key(a[j]):
break
a[j - 1], a[j] = a[j], a[j - 1]
j -= 1
return a
def pattern_merge(intervals, key, s):
"""PRECONDITION: sort key must be START. A single pass suffices."""
result = []
for start, end in sort_counting(intervals, key, s):
s.count()
if result and start <= result[-1][1]:
result[-1] = (result[-1][0], max(result[-1][1], end))
else:
result.append((start, end))
return sorted(result)
CRITERIA = (("by start ", lambda a: a[0]),
("by end ", lambda a: a[1]),
("by length ", lambda a: a[1] - a[0]),
("unsorted ", lambda a: 0))
K = interval_corpus()
print("corpus:", len(K), "sets x", INTERVAL_COUNT, "intervals")
print("criterion diverging/40 pattern oracle ratio first diverging")
for ad, key in CRITERIA:
diverging, pk, ok = [], 0, 0
for k in K:
s1, s2 = Counter(), Counter()
a = pattern_merge(k["interval"], key, s1)
b = oracle_merge(k["interval"], s2)
pk, ok = pk + s1.step, ok + s2.step
if a != b:
diverging.append(k["no"])
print(f"{ad} {len(diverging):8d} {pk:5d} {ok:5d} {ok / pk:5.2f}"
f" {diverging[:5]}")
```
```
corpus: 40 sets x 12 intervals
criterion diverging/40 pattern oracle ratio first diverging
by start 0 2154 2974 1.38 []
by end 29 2145 2974 1.39 [2, 3, 4, 6, 7]
by length 40 2041 2974 1.46 [1, 2, 3, 4, 5]
unsorted 40 920 2974 3.23 [1, 2, 3, 4, 5]
```
Four rows, four separate criteria, the same oracle, the same corpus. Only the first row is
correct: the pattern sorted by start produces the same list as the oracle on 40 of 40 inputs.
The last row is this lesson's sharpest number. **The unsorted pattern is the fastest** — 920
steps, ratio **3.23** — and **wrong on 40 of 40 inputs**. Skipping sorting makes the pattern
three times faster and breaks it entirely. Sorting's step count can be read directly: the
**1234 steps** between 2154 and 920 are the cost of establishing the precondition, and they
eat the largest share of the pattern's gain.
The oracle's 2974 steps also need explaining. The oracle restarts the scan **from the
beginning** after every merge, because the newly formed wide interval might overlap a pair
already checked. This is a deliberately expensive choice, and its purpose is not speed but
**never missing an overlap**. That the pattern gives the same result in a single pass comes
from sorting making this restart unnecessary: in a list sorted by start, no later interval can
touch a merged interval that has already closed.
The gap between the two middle rows is also instructive. Sorting by length is wrong on **40
of 40** inputs; sorting by end is wrong on **29**. Both are the wrong criterion, but one
produces the correct answer on **11 of the 40** inputs — and those eleven are enough for
the criterion to be mistaken for correct.
#### Why Sorting by End Breaks the Merge
The pattern's only decision rule is the comparison `start <= result[-1][1]`: merge if the new
interval's start does not go past the open merged interval's right end. This rule assumes
that every subsequent interval's start is **not smaller than the previous one's**. Sorting by
start guarantees exactly this.
When sorted by end, an interval that starts late but ends early can move ahead; the interval
following it, which **starts earlier**, no longer satisfies the merge condition and opens a
new merged interval. Two records that contain one another remain in the result list.
```python
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def interval_corpus(seed, n=40, count=12):
r = generator(seed)
items = []
for i in range(n):
row = []
for _ in range(count):
start = r(61)
row.append((start, start + 1 + r(9)))
items.append({"no": i + 1, "interval": row})
return items
def oracle_merge(intervals):
remaining = [list(a) for a in intervals]
changed = True
while changed:
changed = False
for i in range(len(remaining)):
for j in range(i + 1, len(remaining)):
if remaining[i][0] <= remaining[j][1] and remaining[j][0] <= remaining[i][1]:
remaining[i] = [min(remaining[i][0], remaining[j][0]),
max(remaining[i][1], remaining[j][1])]
remaining.pop(j)
changed = True
break
if changed:
break
return sorted(tuple(a) for a in remaining)
def pattern_merge(intervals, key):
result = []
for start, end in sorted(intervals, key=key):
if result and start <= result[-1][1]:
result[-1] = (result[-1][0], max(result[-1][1], end))
else:
result.append((start, end))
return sorted(result)
small = [(4, 20), (6, 8), (10, 12)]
print("small example :", small)
print(" oracle :", oracle_merge(small))
print(" by start :", pattern_merge(small, lambda a: a[0]))
print(" by end :", pattern_merge(small, lambda a: a[1]),
" <- sorted by end:", sorted(small, key=lambda a: a[1]))
print()
print("seed criterion diverging/40 ratio")
for seed in (20260218, 20260219):
K = interval_corpus(seed)
for ad, key in (("by start ", lambda a: a[0]),
("by end ", lambda a: a[1]),
("by length ", lambda a: a[1] - a[0]),
("unsorted ", lambda a: 0)):
diverging = sum(1 for k in K
if pattern_merge(k["interval"], key)
!= oracle_merge(k["interval"]))
print(f"{seed} {ad} {diverging:8d} {diverging / 40:.4f}")
```
```
small example : [(4, 20), (6, 8), (10, 12)]
oracle : [(4, 20)]
by start : [(4, 20)]
by end : [(6, 8), (10, 20)] <- sorted by end: [(6, 8), (10, 12), (4, 20)]
seed criterion diverging/40 ratio
20260218 by start 0 0.0000
20260218 by end 29 0.7250
20260218 by length 40 1.0000
20260218 unsorted 40 1.0000
20260219 by start 0 0.0000
20260219 by end 32 0.8000
20260219 by length 40 1.0000
20260219 unsorted 40 1.0000
```
The three-interval small example shows the whole mechanism at once. The correct answer is a
single interval: `(6, 8)` and `(10, 12)` are entirely contained in `(4, 20)`. Sorted by end,
`(4, 20)` falls to the **end** of the list; the pattern starts with `(6, 8)`, `(10, 12)` does
not merge and opens a new record, then `(4, 20)` arrives and turns it into `(10, 20)`. What
remains is `(6, 8)`, standing apart **even though it is contained**.
**PP29.** The second corpus comes from seed `20260219`. Diverging inputs sorted by end are
**32/40**, versus **29/40** in the first; ratios 0.7250 and 0.8000, the same order of
magnitude. The result does not depend on the corpus. Sorting by length and not sorting give
**40/40** in both corpora.
#### The Criterion Is Not Just the Key, It Is the Tie Rule
Once a sort criterion is chosen, one more question remains and is often left unasked:
**which order do two records with the same value come in.** A second interval problem makes
this question measurable: at a given point, **how many intervals are open at once, at most**.
The pattern turns intervals into events — every start an opening, every end a closing — and
sorts the events by coordinate to count them in a single pass. If one interval ends at the
same coordinate where another begins, the closing must be processed **before** the opening;
counting an already-closed end twice inflates the open-interval count.
**PP30.** In this measurement intervals are **half-open**: an interval contains its start but
not its end. The oracle counts, at every start point, exactly how many intervals are open.
**PP31.** In **32 of the corpus's sets**, one interval's end coincides with another's start;
the tie rule's effect can only be seen in these sets.
```python
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def interval_corpus(seed=20260218, n=40, count=12):
r = generator(seed)
items = []
for i in range(n):
row = []
for _ in range(count):
start = r(61)
row.append((start, start + 1 + r(9)))
items.append({"no": i + 1, "interval": row})
return items
class Counter:
def __init__(self):
self.step = 0
def count(self):
self.step += 1
def oracle_overlap(intervals, s):
"""How many intervals are open at every start point. Every pair is tried."""
most = 0
for start, _ in intervals:
open_count = 0
for b, e in intervals:
s.count()
if b <= start < e:
open_count += 1
most = max(most, open_count)
return most
def pattern_overlap(intervals, end_first, s):
"""PRECONDITION: an END event at the same coordinate must be processed before a START."""
event = []
for start, end in intervals:
event.append((start, 1 if end_first else 0, +1))
event.append((end, 0 if end_first else 1, -1))
most = open_count = 0
for _, _, delta in sorted(event):
s.count()
open_count += delta
most = max(most, open_count)
return most
K = interval_corpus()
tied = sum(1 for k in K
if {b for b, _ in k["interval"]} & {e for _, e in k["interval"]})
print("corpus: 40 sets x 12 intervals | an end tied to a start in:", tied)
print("event order diverging/40 pattern oracle ratio first diverging")
for ad, end_first in (("end first ", True), ("start first ", False)):
diverging, pk, ok = [], 0, 0
for k in K:
s1, s2 = Counter(), Counter()
a = pattern_overlap(k["interval"], end_first, s1)
b = oracle_overlap(k["interval"], s2)
pk, ok = pk + s1.step, ok + s2.step
if a != b:
diverging.append(k["no"])
print(f"{ad} {len(diverging):10d} {pk:5d} {ok:5d} {ok / pk:5.2f}"
f" {diverging[:5]}")
```
```
corpus: 40 sets x 12 intervals | an end tied to a start in: 32
event order diverging/40 pattern oracle ratio first diverging
end first 0 960 5760 6.00 []
start first 8 960 5760 6.00 [5, 7, 8, 15, 31]
```
The step columns of both rows are identical: pattern **960**, oracle **5760**, ratio
**6.00**. The only thing that changes is which of two events at the same coordinate is
processed first, and that single decision changes the answer on **8 inputs**.
The number eight is less than a quarter of 32, and this explains why the criterion goes
unnoticed. In the 8 sets with no coincidence, the tie rule has no effect at all; in most of
the 32 sets with a coincidence, the answer does not change because the widest overlap occurs
somewhere else. The rule only shows up on 8 inputs — but on those 8 it is **wrong**, and
which 8 they are is knowable only through the oracle.
#### Three Numbers
| Metric | Oracle | Pattern | Diverging input |
|---|---|---|---|
| Sorted by start | 2974 steps | 2154 steps | **0/40** |
| Sorted by end | 2974 steps | 2145 steps | **29/40** |
| Unsorted | 2974 steps | 920 steps | **40/40** |
| Overlap count, end first | 5760 steps | 960 steps | **0/40** |
| Overlap count, start first | 5760 steps | 960 steps | **8/40** |
In none of the five rows does the step column carry information about correctness; in the
third row the lowest step count and the highest wrong-answer count sit together. The
pattern's precondition can be stated in one sentence: **the sort key must guarantee exactly
what the pattern's single decision rule assumes** — no more, no less. Choosing the criterion
by looking at a field that "seems natural" is choosing without looking at the decision rule.
#### Summary
- Interval merging is the first pattern that sorts the input beforehand; its precondition is
a property not of the data but of the **chosen criterion**.
- Sorting by start is correct on 40 of 40 inputs; sorting by end diverges from the oracle on
**29**, sorting by length and not sorting diverge on **40**.
- Skipping sorting drops the pattern from 2154 to 920 steps and breaks it on **all inputs**;
the fastest row is the most broken row.
- In the second corpus, diverging inputs sorted by end are 32; because the ratio holds its
order of magnitude, the result does not depend on the corpus.
- In the overlap count, event order at the same coordinate alone changes the answer on **8
inputs**; step count stays exactly the same in both cases.
#### Next Step
This pattern paid the cost of sorting and got a single pass in return. The next pattern skips
sorting entirely: in a range where values **know their own position**, each value is placed
directly at its destination, with no comparison at all. Its precondition is heavy — the
values must lie between 1 and n and contain no duplicates — and when it breaks, the pattern
does not just give a wrong answer, it spins without stopping. The next lesson counts these two
defects separately.
### Cyclic Sort
Source: https://academia.sh/en/courses/advanced-algorithms/cyclic-sort
In-place placement without comparison over a bounded value range; the pattern that never stops on duplicate values, 40 wrong answers on out-of-range values, and what the guard does not fix.
The previous pattern paid the cost of sorting and got a single pass in return. This lesson's
pattern skips sorting entirely. Its idea is one sentence: **if a value tells you its own
destination, no comparison is needed.** When values lie between 1 and n, value `v`'s place is
position `v-1`; every value is sent straight home, and the value that comes out of that home
determines the next placement.
What is paid in exchange is a heavy precondition, and it is in fact **two separate**
preconditions: values must lie **between 1 and n**, and values must be **without
duplicates**. This lesson breaks each separately, because the consequence of breaking each is
also separate: one produces a wrong answer, the other produces **no answer at all**.
#### The Pattern's Idea and Its Two Preconditions
The pattern stops at a position and looks at the value there. If the value is not at its
home, it sends the value home; the new value that comes out of that home lands at the same
position and the same operation repeats. If the value is already home, it moves to the next
position. Because every swap puts **at least one value permanently home**, the total swap
count cannot exceed n.
This count's guarantee comes directly from the precondition. When a value is sent home, if
another value already sits there, that value must be **different**; otherwise the swap
advances nothing and the same two values swap places forever.
**PP32.** The array has 12 values and the target range is 1..12; seed `20260218`.
**PP33.** There are three corpora and all three come from the same generator: a **1..n
arrangement** (no duplicates, within range), **duplicate values**, and arrays with two values
moved **out of range**.
**PP34.** The oracle uses one of the comparison-based procedures measured in the Algorithms
course and counts every comparison as one step. The procedure itself is not repeated here.
**PP35.** The pattern is given a **step cap** (400 steps). A run that hits the cap returns
`did not stop` and is counted as diverging from the oracle.
```python
SEED, LENGTH, CORPUS_SIZE = 20260218, 12, 40
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def value_corpus(seed=SEED, n=CORPUS_SIZE, length=LENGTH):
"""Three corpora: 1..n arrangement; duplicate values; out-of-range values."""
r = generator(seed)
items = []
for i in range(n):
arrangement = list(range(1, length + 1))
for j in range(length, 1, -1):
k = r(j)
arrangement[j - 1], arrangement[k] = arrangement[k], arrangement[j - 1]
duplicates = [r(length) + 1 for _ in range(length)]
out_of_range = list(arrangement)
for _ in range(2):
out_of_range[r(length)] = length + 2 + r(length)
items.append({"no": i + 1, "arrangement": arrangement,
"duplicates": duplicates, "out_of_range": out_of_range})
return items
class Counter:
def __init__(self):
self.step = 0
def count(self):
self.step += 1
def oracle_arrangement(array, s):
"""Comparison-based selection sort. Always correct, always expensive."""
a = list(array)
for i in range(len(a)):
least = i
for j in range(i + 1, len(a)):
s.count()
if a[j] < a[least]:
least = j
a[i], a[least] = a[least], a[i]
return a
def pattern_plain(array, s, cap=400):
"""PRECONDITION: values must be within 1..n and WITHOUT DUPLICATES."""
a, n = list(array), len(array)
i = 0
while i < n:
s.count()
if s.step > cap:
return "did not stop"
home = a[i] - 1
if 0 <= home < n and home != i:
a[i], a[home] = a[home], a[i]
else:
i += 1
return a
def pattern_guarded(array, s, cap=400):
"""Duplicate guard added: no swap if the value at home is already the same."""
a, n = list(array), len(array)
i = 0
while i < n:
s.count()
if s.step > cap:
return "did not stop"
home = a[i] - 1
if 0 <= home < n and a[i] != a[home]:
a[i], a[home] = a[home], a[i]
else:
i += 1
return a
POOL = (("1..n arrangement ", "arrangement"),
("duplicate values ", "duplicates"),
("out-of-range values", "out_of_range"))
K = value_corpus()
print("corpus:", len(K), "arrays x", LENGTH, "values | examples:")
print(" arrangement :", K[0]["arrangement"])
print(" duplicates :", K[0]["duplicates"])
print(" out_of_range:", K[0]["out_of_range"])
print()
print("pattern pool diverging/40 pattern oracle ratio")
for kad, pattern in (("plain ", pattern_plain), ("guarded ", pattern_guarded)):
for ad, key in POOL:
diverging, pk, ok = 0, 0, 0
for k in K:
s1, s2 = Counter(), Counter()
a = pattern(k[key], s1)
b = oracle_arrangement(k[key], s2)
pk, ok = pk + s1.step, ok + s2.step
diverging += (a != b)
print(f"{kad} {ad} {diverging:8d} {pk:5d} {ok:5d} {ok / pk:5.2f}")
```
```
corpus: 40 arrays x 12 values | examples:
arrangement : [5, 9, 3, 4, 7, 10, 1, 12, 11, 2, 6, 8]
duplicates : [3, 4, 1, 2, 3, 4, 5, 6, 7, 12, 9, 2]
out_of_range: [5, 9, 3, 4, 7, 18, 1, 12, 11, 2, 6, 16]
pattern pool diverging/40 pattern oracle ratio
plain 1..n arrangement 0 826 2640 3.20
plain duplicate values 40 16040 2640 0.16
plain out-of-range values 40 801 2640 3.30
guarded 1..n arrangement 0 826 2640 3.20
guarded duplicate values 39 750 2640 3.52
guarded out-of-range values 40 801 2640 3.30
```
The first row is the pattern's promise: the same arrangement as the oracle on 40 of 40
inputs, **826** steps against **2640**, a ratio of **3.20**. The result is correct even
though no comparison is ever performed.
The second row carries this topic's most extreme result. On duplicate values, the plain
pattern spends **16,040** steps and the ratio drops to **0.16** — **six times more expensive**
than the oracle. This number is not a slowdown, it is a measure of **not stopping**: all forty
of the forty inputs hit the 400-step cap. Without the cap, the measurement would have finished
on the first input and no result would ever have come out.
The third row shows an entirely different defect. On out-of-range values, the pattern
finishes in **801** steps, ratio **3.30** — even **faster** than the first row — and **wrong
on 40 of 40 inputs**. An out-of-range value has no home; the pattern leaves it where it is and
moves on, and the resulting arrangement differs from the oracle's. There is no warning, no
delay, no sign at all.
#### The Guard Fixes Not Stopping, Not the Wrongness
The change that stops the infinite swap is a single comparison: before sending a value home,
check whether the **same value** is already there. If it is, sending it home is pointless and
the position advances instead.
```python
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def value_corpus(seed, n=40, length=12):
r = generator(seed)
items = []
for i in range(n):
arrangement = list(range(1, length + 1))
for j in range(length, 1, -1):
k = r(j)
arrangement[j - 1], arrangement[k] = arrangement[k], arrangement[j - 1]
duplicates = [r(length) + 1 for _ in range(length)]
out_of_range = list(arrangement)
for _ in range(2):
out_of_range[r(length)] = length + 2 + r(length)
items.append({"arrangement": arrangement, "duplicates": duplicates,
"out_of_range": out_of_range})
return items
def oracle_arrangement(array):
a = list(array)
for i in range(len(a)):
least = i
for j in range(i + 1, len(a)):
if a[j] < a[least]:
least = j
a[i], a[least] = a[least], a[i]
return a
def pattern(array, guard, cap=400):
a, n, i, step = list(array), len(array), 0, 0
while i < n:
step += 1
if step > cap:
return "did not stop"
home = a[i] - 1
if 0 <= home < n and ((a[i] != a[home]) if guard else (home != i)):
a[i], a[home] = a[home], a[i]
else:
i += 1
return a
example = value_corpus(20260218)[0]
print("duplicate input:", example["duplicates"])
print(" plain pattern :", pattern(example["duplicates"], False))
print(" guarded pattern:", pattern(example["duplicates"], True))
print(" oracle :", oracle_arrangement(example["duplicates"]))
print()
print("seed pattern pool diverging/40 ratio")
for seed in (20260218, 20260219):
K = value_corpus(seed)
for kad, guard in (("plain ", False), ("guarded ", True)):
for key in ("arrangement", "duplicates", "out_of_range"):
diverging = sum(1 for k in K
if pattern(k[key], guard) != oracle_arrangement(k[key]))
print(f"{seed} {kad} {key:12s} {diverging:8d} {diverging / 40:.4f}")
```
```
duplicate input: [3, 4, 1, 2, 3, 4, 5, 6, 7, 12, 9, 2]
plain pattern : did not stop
guarded pattern: [1, 2, 3, 4, 5, 6, 7, 4, 9, 2, 3, 12]
oracle : [1, 2, 2, 3, 3, 4, 4, 5, 6, 7, 9, 12]
seed pattern pool diverging/40 ratio
20260218 plain arrangement 0 0.0000
20260218 plain duplicates 40 1.0000
20260218 plain out_of_range 40 1.0000
20260218 guarded arrangement 0 0.0000
20260218 guarded duplicates 39 0.9750
20260218 guarded out_of_range 40 1.0000
20260219 plain arrangement 0 0.0000
20260219 plain duplicates 40 1.0000
20260219 plain out_of_range 39 0.9750
20260219 guarded arrangement 0 0.0000
20260219 guarded duplicates 40 1.0000
20260219 guarded out_of_range 39 0.9750
```
The example input shows the mechanism plainly. The plain pattern returns `did not stop`. The
guarded pattern stops and produces `[1, 2, 3, 4, 5, 6, 7, 4, 9, 2, 3, 12]`; the oracle's
answer is `[1, 2, 2, 3, 3, 4, 4, 5, 6, 7, 9, 12]`. They are not the same — **the guard fixed
not stopping, it did not fix the wrongness.** Diverging inputs dropped from 40 to **39**,
which by the resolution rule is not even a **measured** difference, because it stays at the
same order of magnitude.
**PP36.** The second corpus comes from seed `20260219`. In twelve of twelve rows, diverging
inputs are either 0 or 39–40; the result does not depend on the corpus.
**PP37.** A gap between 39 and 40 in 40 inputs **counts as unmeasured**; the swaps between the
two corpora (39/40 on duplicates, 40/39 on out-of-range) are therefore not a trend.
#### The Precondition Belongs Not to the Pattern but to the Pattern-Question Pair
So far only one question has been asked: **what will the arrangement itself be.** The same
pattern, with the same broken inputs, can also be used to answer a different question:
**which values from 1..n are missing.** Here the pattern uses the arrangement not as a goal
but as an **intermediate product**; positions that are not home directly give the missing
values.
**PP38.** In this measurement the pattern is the **guarded** form, and the answer is the list
of positions not at home. The oracle searches for each value by scanning the array.
**PP39.** The question changed; the pattern, oracle, corpus, and seed did not.
```python
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def value_corpus(seed, n=40, length=12):
r = generator(seed)
items = []
for i in range(n):
arrangement = list(range(1, length + 1))
for j in range(length, 1, -1):
k = r(j)
arrangement[j - 1], arrangement[k] = arrangement[k], arrangement[j - 1]
duplicates = [r(length) + 1 for _ in range(length)]
out_of_range = list(arrangement)
for _ in range(2):
out_of_range[r(length)] = length + 2 + r(length)
items.append({"arrangement": arrangement, "duplicates": duplicates,
"out_of_range": out_of_range})
return items
class Counter:
def __init__(self):
self.step = 0
def count(self):
self.step += 1
def oracle_missing(array, s):
"""Searches for every value in 1..n by scanning the array."""
missing = []
for v in range(1, len(array) + 1):
found = False
for x in array:
s.count()
if x == v:
found = True
break
if not found:
missing.append(v)
return missing
def pattern_missing(array, s, cap=400):
"""First places, then collects the positions not at home."""
a, n = list(array), len(array)
i = 0
while i < n:
s.count()
if s.step > cap:
return "did not stop"
home = a[i] - 1
if 0 <= home < n and a[i] != a[home]:
a[i], a[home] = a[home], a[i]
else:
i += 1
missing = []
for j in range(n):
s.count()
if a[j] != j + 1:
missing.append(j + 1)
return missing
print("seed pool diverging/40 pattern oracle ratio")
for seed in (20260218, 20260219):
K = value_corpus(seed)
for key in ("arrangement", "duplicates", "out_of_range"):
diverging, pk, ok = 0, 0, 0
for k in K:
s1, s2 = Counter(), Counter()
a = pattern_missing(k[key], s1)
b = oracle_missing(k[key], s2)
pk, ok = pk + s1.step, ok + s2.step
diverging += (a != b)
print(f"{seed} {key:12s} {diverging:8d} {pk:5d} {ok:5d}"
f" {ok / pk:5.2f}")
print()
K = value_corpus(20260218)
print("out-of-range example array:", K[0]["out_of_range"])
print(" pattern missing:", pattern_missing(K[0]["out_of_range"], Counter()))
print(" oracle missing :", oracle_missing(K[0]["out_of_range"], Counter()))
```
```
seed pool diverging/40 pattern oracle ratio
20260218 arrangement 0 1306 3120 2.39
20260218 duplicates 0 1230 3531 2.87
20260218 out_of_range 0 1281 3536 2.76
20260219 arrangement 0 1313 3120 2.38
20260219 duplicates 0 1233 3525 2.86
20260219 out_of_range 0 1294 3548 2.74
out-of-range example array: [5, 9, 3, 4, 7, 18, 1, 12, 11, 2, 6, 16]
pattern missing: [8, 10]
oracle missing : [8, 10]
```
Six of six rows give diverging inputs of **zero**. The same pattern, the same duplicate and
out-of-range inputs, two corpora — and no divergence at all. The rows that read 39 and 40 in
the previous table read 0 here.
The only thing that changed is the **question**. "What will the arrangement be" requires
every value in the array to have a home; "which values are missing" does not, because a
homeless value's position does not affect the answer. The pattern's precondition lies not in
the pattern's own code, but in **which question the pattern is answering**.
The rule that follows from this holds for the entire topic: **a pattern's precondition is not
defined when the pattern's name is asked, it is defined once the question asked is fixed.**
The sentence "cyclic sort does not work with duplicate values" is false in this table; the
correct sentence is "cyclic sort does not give the correct answer to the **arrangement
question** with duplicate values."
#### Three Numbers
| Metric | Oracle | Pattern | Diverging input |
|---|---|---|---|
| Arrangement, 1..n values | 2640 steps | 826 steps | **0/40** |
| Arrangement, duplicates, plain pattern | 2640 steps | 16,040 steps | **40/40** |
| Arrangement, duplicates, guarded pattern | 2640 steps | 750 steps | **39/40** |
| Arrangement, out-of-range | 2640 steps | 801 steps | **40/40** |
| Missing value, out-of-range | 3536 steps | 1281 steps | **0/40** |
The second row is a measure of not stopping, the third row of silent wrongness, the fifth row
of the question changing. In none of the five rows can the correct rows be picked out by
looking at the step column.
#### Summary
- Cyclic sort performs no comparison at all; it sends every value straight to position `v-1`,
and total swaps never exceed n.
- The pattern has two separate preconditions, and their breakdowns lead to separate
consequences: duplicate values lead to **not stopping**, out-of-range values lead to
**silent wrongness**.
- On duplicate input, the plain pattern hits the step cap on 40 of 40 inputs and the ratio
drops to 0.16; on out-of-range input it finishes in 801 steps and is wrong on 40 of 40.
- The duplicate guard removes not stopping but drops diverging inputs on the arrangement
question only from 40 to 39.
- When the same pattern answers "which values are missing," diverging inputs are zero on six
of six measurements; the precondition belongs not to the pattern but to the
pattern-question pair.
#### Next Step
The five patterns so far held the entire input in hand: the array could be seen start to end,
revisited at any position. The next pattern loses that ability. Values **flow one at a time**,
and after each new value a question must be answered: what is the median of what has been
seen so far. The pattern holds two heaps, and its precondition is that these two heaps stay
**balanced**. The next lesson will count how many wrong medians appear at how many steps once
that balance breaks.
### Two Heaps
Source: https://academia.sh/en/courses/advanced-algorithms/two-heaps
Median tracking in a stream; how removing the rebalancing step corrupts 295 of 480 medians, and how the pattern's gain grows as the stream lengthens.
The previous five patterns held the entire input in hand: the array could be seen start to
end, revisited at any position. This lesson's pattern loses that ability. Values **flow one
at a time**, and after every new value a question must be answered: what is the **median** of
what has been seen so far.
The direct way to find the median is to keep a sorted list: insert every new value at the
right place and read the middle. The pattern instead keeps **two heaps**: the lower half in a
**max-heap**, the upper half in a **min-heap**. The median is read from the tops of the two
heaps. The heap invariant, the sift operations, and the array representation were established
in the Data Structures course's Heaps lesson; they are **not repeated** here, they are used
directly.
The pattern's precondition is that the two heaps stay **balanced**. This lesson breaks that
balance and counts how many medians turn out wrong.
#### Pattern and the Balance Precondition
Every new value is placed first: if it is not greater than the low half's top, it goes to the
max-heap; otherwise it goes to the min-heap. That alone is not enough, because values can pile
up on one side. The second step is the **rebalancing step**: whenever the heaps' sizes differ
by more than one, the top of the larger one is moved to the other.
The median can only be read once the two heaps' sizes are correct. If the total is odd, the
top of the larger heap is the median; if even, the average of the two tops. If the balance is
off, both rules point to the wrong place — the pattern still returns a number.
**PP40.** The corpus is 40 streams; each stream carries 12 values, ranging from −9 to 20.
Seed `20260218`. Every stream asks for a median after every value; **480** questions total.
**PP41.** The oracle inserts every value into a sorted list by **linear scan** and reads the
middle. Every comparison is counted as one step.
**PP42.** The pattern's step count is the heap operations' **real comparisons**; values are
placed in a wrapper that counts comparisons, the step count is not estimated.
**PP43.** The only thing that breaks the precondition is **removing the rebalancing step**.
The value-placement rule, the median-reading rule, the corpus, and the seed are the same.
```python
import heapq
SEED, LENGTH, CORPUS_SIZE = 20260218, 12, 40
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def stream_corpus(seed=SEED, n=CORPUS_SIZE, length=LENGTH):
r = generator(seed)
return [{"no": i + 1, "stream": [r(30) - 9 for _ in range(length)]}
for i in range(n)]
class Counter:
def __init__(self):
self.step = 0
def count(self, n=1):
self.step += n
class Counted:
"""Wraps a value and counts every heap comparison. sign=-1 gives a max-heap."""
__slots__ = ("d", "sign", "s")
def __init__(self, d, sign, s):
self.d, self.sign, self.s = d, sign, s
def __lt__(self, o):
self.s.count()
return self.d * self.sign < o.d * o.sign
def oracle_median(stream, s):
"""Inserts every value into a sorted list by linear scan, reads the middle."""
sorted_list, answer = [], []
for x in stream:
i = 0
while i < len(sorted_list):
s.count()
if sorted_list[i] >= x:
break
i += 1
sorted_list.insert(i, x)
n = len(sorted_list)
answer.append(sorted_list[n // 2] if n % 2
else (sorted_list[n // 2 - 1] + sorted_list[n // 2]) / 2)
return answer
def pattern_two_heaps(stream, rebalance, s):
"""PRECONDITION: the two heaps must stay balanced. low=max-heap, high=min-heap."""
low, high, answer = [], [], []
max_diff = 0
for x in stream:
if not low or x <= low[0].d:
heapq.heappush(low, Counted(x, -1, s))
else:
heapq.heappush(high, Counted(x, 1, s))
if rebalance:
if len(low) > len(high) + 1:
heapq.heappush(high, Counted(heapq.heappop(low).d, 1, s))
elif len(high) > len(low):
heapq.heappush(low, Counted(heapq.heappop(high).d, -1, s))
max_diff = max(max_diff, abs(len(low) - len(high)))
if not low:
answer.append(high[0].d)
elif not high or len(low) > len(high):
answer.append(low[0].d)
elif len(high) > len(low):
answer.append(high[0].d)
else:
answer.append((low[0].d + high[0].d) / 2)
return answer, max_diff
K = stream_corpus()
print("corpus:", len(K), "streams x", LENGTH, "values =", len(K) * LENGTH, "medians")
print("setup wrong medians diverging streams/40 pattern oracle ratio max diff")
for ad, rebalance in (("balanced ", True), ("unbalanced", False)):
wrong, diverging, pk, ok, diff = 0, 0, 0, 0, 0
for k in K:
s1, s2 = Counter(), Counter()
a, f = pattern_two_heaps(k["stream"], rebalance, s1)
b = oracle_median(k["stream"], s2)
pk, ok = pk + s1.step, ok + s2.step
diff = max(diff, f)
broken = sum(1 for x, y in zip(a, b) if x != y)
wrong += broken
diverging += broken > 0
print(f"{ad} {wrong:14d} {diverging:15d} {pk:5d} {ok:5d} {ok / pk:5.2f}"
f" {diff:13d}")
```
```
corpus: 40 streams x 12 values = 480 medians
setup wrong medians diverging streams/40 pattern oracle ratio max diff
balanced 0 0 1261 1693 1.34 1
unbalanced 295 39 550 1693 3.08 12
```
In the balanced setup, **480 of 480 medians** match the oracle; the largest size difference
is **1**, that is, the balance never breaks. Once the rebalancing step is removed, **295
medians** come out wrong and **39 of 40** streams are corrupted.
The second row's step column carries the lesson's central conflict: the unbalanced pattern
spends **550** steps, **less than half** the balanced one. The rebalancing step makes up the
larger share of the pattern's total work; removing it speeds the pattern up against the
oracle from a ratio of 1.34 to **3.08**. The fastest setup is again the most broken one. The
largest size difference also climbs from 1 to **12**: every value piles into a single heap.
#### How Imbalance Produces a Wrong Median
Without the rebalancing step, the placement rule feeds on itself. When a small value arrives,
it goes into the low half; the low half's top does not shrink further, but the half grows.
The next value is compared against the top of that grown half and mostly lands there too. One
heap swells while the other stays empty, and the median-reading rule keeps looking at the
**same** top.
```python
import heapq
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def stream_corpus(seed, n=40, length=12):
r = generator(seed)
return [[r(30) - 9 for _ in range(length)] for _ in range(n)]
def oracle_median(stream):
sorted_list, answer = [], []
for x in stream:
i = 0
while i < len(sorted_list) and sorted_list[i] < x:
i += 1
sorted_list.insert(i, x)
n = len(sorted_list)
answer.append(sorted_list[n // 2] if n % 2
else (sorted_list[n // 2 - 1] + sorted_list[n // 2]) / 2)
return answer
def pattern_two_heaps(stream, rebalance):
"""low=max-heap (sign flipped), high=min-heap."""
low, high, answer = [], [], []
for x in stream:
if not low or x <= -low[0]:
heapq.heappush(low, -x)
else:
heapq.heappush(high, x)
if rebalance:
if len(low) > len(high) + 1:
heapq.heappush(high, -heapq.heappop(low))
elif len(high) > len(low):
heapq.heappush(low, -heapq.heappop(high))
if not low:
answer.append(high[0])
elif not high or len(low) > len(high):
answer.append(-low[0])
elif len(high) > len(low):
answer.append(high[0])
else:
answer.append((-low[0] + high[0]) / 2)
return answer
first = stream_corpus(20260218)[0]
print("stream 1 :", first)
print(" oracle :", oracle_median(first))
print(" balanced :", pattern_two_heaps(first, True))
print(" unbalanced:", pattern_two_heaps(first, False))
print()
print("seed setup wrong medians/480 diverging streams/40 first wrong position")
for seed in (20260218, 20260219):
K = stream_corpus(seed)
for ad, rebalance in (("balanced ", True), ("unbalanced", False)):
wrong, diverging, first_position = 0, 0, []
for stream in K:
a, b = pattern_two_heaps(stream, rebalance), oracle_median(stream)
broken = [i + 1 for i, (x, y) in enumerate(zip(a, b)) if x != y]
wrong += len(broken)
diverging += bool(broken)
if broken:
first_position.append(broken[0])
average = round(sum(first_position) / len(first_position), 2) if first_position else 0
print(f"{seed} {ad} {wrong:18d} {diverging:15d} {average:16}")
```
```
stream 1 : [-8, -5, 2, -1, 2, 5, 4, 1, 16, 17, 6, -1]
oracle : [-8, -6.5, -5, -3.0, -1, 0.5, 2, 1.5, 2, 2.0, 2, 2.0]
balanced : [-8, -6.5, -5, -3.0, -1, 0.5, 2, 1.5, 2, 2.0, 2, 2.0]
unbalanced: [-8, -6.5, -5, -5, -5, -5, -5, -5, -5, -5, -5, -5]
seed setup wrong medians/480 diverging streams/40 first wrong position
20260218 balanced 0 0 0
20260218 unbalanced 295 39 3.44
20260219 balanced 0 0 0
20260219 unbalanced 268 38 3.89
```
The first stream's trace shows the mechanism in one line. The balanced setup produces every
one of the oracle's answers. The unbalanced setup gives the first three medians correctly,
then **locks onto −5** and writes the same number for the rest of the stream. Even as the
values climb to 16 and 17, the answer never changes, because every large value has piled
into a single heap and the top being read has stayed fixed.
**PP44.** The second corpus comes from seed `20260219`. Wrong medians are **268/480**, versus
**295/480** in the first; diverging streams are **38/40** and **39/40**. Ratios fall between
0.56 and 0.61, the same order of magnitude — the result does not depend on the corpus.
**PP45.** The first wrong median arrives on average at the **3.44th** value. The balance
breaks by at latest the fourth value, meaning the defect forms not at the stream's end but at
its **start**.
This last number explains not why the defect goes unnoticed but why it **can**. The first
three answers are correct; a short test stream that checks only these three sees nothing. The
defect only accumulates as the stream lengthens, and no single wrong median can be picked out
by looking at the output — the unbalanced pattern still returns a plausible number.
#### The Pattern's Gain Grows With Stream Length
On a twelve-value stream, the balanced pattern's ratio is **1.34**. That is too small a gain
to be worth building the pattern for, and it raises a question: what is the point of two
heaps. The answer lies in stream length. The oracle's every insertion scans half the list, so
its cost is directly proportional to the value count seen so far; a heap insertion instead
makes as many comparisons as the heap's **depth**.
**PP46.** In this measurement stream count drops to 10 and stream length climbs from 12 to
200. The corpus comes from the same generator, the same seed; the only thing that changes is
length.
```python
import heapq
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def stream_corpus(seed, n, length):
r = generator(seed)
return [[r(30) - 9 for _ in range(length)] for _ in range(n)]
class Counter:
def __init__(self):
self.step = 0
def count(self, n=1):
self.step += n
class Counted:
"""Counts every heap comparison. sign=-1 gives a max-heap."""
__slots__ = ("d", "sign", "s")
def __init__(self, d, sign, s):
self.d, self.sign, self.s = d, sign, s
def __lt__(self, o):
self.s.count()
return self.d * self.sign < o.d * o.sign
def oracle_median(stream, s):
sorted_list, answer = [], []
for x in stream:
i = 0
while i < len(sorted_list):
s.count()
if sorted_list[i] >= x:
break
i += 1
sorted_list.insert(i, x)
n = len(sorted_list)
answer.append(sorted_list[n // 2] if n % 2
else (sorted_list[n // 2 - 1] + sorted_list[n // 2]) / 2)
return answer
def pattern_two_heaps(stream, s):
low, high, answer = [], [], []
for x in stream:
if not low or x <= low[0].d:
heapq.heappush(low, Counted(x, -1, s))
else:
heapq.heappush(high, Counted(x, 1, s))
if len(low) > len(high) + 1:
heapq.heappush(high, Counted(heapq.heappop(low).d, 1, s))
elif len(high) > len(low):
heapq.heappush(low, Counted(heapq.heappop(high).d, -1, s))
answer.append(low[0].d if len(low) > len(high)
else (low[0].d + high[0].d) / 2)
return answer
print("stream length diverging streams/10 pattern oracle ratio")
for length in (12, 25, 50, 100, 200):
K = stream_corpus(20260218, 10, length)
diverging, pk, ok = 0, 0, 0
for stream in K:
s1, s2 = Counter(), Counter()
a = pattern_two_heaps(stream, s1)
b = oracle_median(stream, s2)
pk, ok = pk + s1.step, ok + s2.step
diverging += (a != b)
print(f"{length:13d} {diverging:15d} {pk:5d} {ok:6d} {ok / pk:6.2f}")
```
```
stream length diverging streams/10 pattern oracle ratio
12 0 303 431 1.42
25 0 949 1602 1.69
50 0 2471 6440 2.61
100 0 5971 25292 4.24
200 0 13686 98651 7.21
```
Diverging streams are **zero** at all five lengths: as long as balance is preserved, the
pattern is correct regardless of length. The ratio climbs from 1.42 to **7.21**. The oracle's
step count grows from 431 to 98,651, that is, **229 times**, while the pattern's step count
grows from 303 to 13,686, **45 times**.
This column is the reason for building the pattern. Keeping two heaps for a twelve-value
stream is unnecessary; for a two-hundred-value stream it pays off seven times over, and it
pays off more on longer streams still. **A pattern's gain is not a number, it is a trend**,
and measuring it at a single input size reads it wrong.
#### Three Numbers
| Metric | Oracle | Pattern | Diverging input |
|---|---|---|---|
| Balanced, 12-value stream | 1693 steps | 1261 steps | **0/480 medians** |
| Unbalanced, 12-value stream | 1693 steps | 550 steps | **295/480 medians** |
| Balanced, 200-value stream | 98,651 steps | 13,686 steps | **0/10 streams** |
The only difference between the first and second row is a two-line rebalancing step; those
two lines spend more than half the pattern's steps and prevent **295 wrong answers**. The
third row shows why that cost is paid.
The diverging-input column of the table carries two separate units, and this must be noted.
In the first two rows the unit is a **median**, because in a data stream every value produces
a question and what needs measuring is how many questions were answered wrongly. In the third
row the unit is a **stream**, because there the question asked is whether the pattern breaks
with length. Mixing the two units makes the ratios impossible to compare: 295/480 and 0/10
cannot be read on the same scale. Every pattern that measures streaming data must state this
distinction explicitly.
#### Summary
- The two heaps pattern gives the median of a data stream by holding the lower half in a
max-heap and the upper half in a min-heap, and reads two heap tops.
- The precondition is that the two heaps stay balanced; once the rebalancing step is removed,
**295 of 480** medians come out wrong and 39 of 40 streams are corrupted.
- The rebalancing step spends more than half the pattern's total steps: without it the
pattern spends 550 steps instead of 1261, and the ratio climbs from 1.34 to 3.08.
- The first wrong median arrives on average at the 3.44th value; the defect forms at the
stream's start but stays invisible in a short test because the first few answers are
correct.
- The pattern's gain grows with stream length: the ratio is 1.42 at 12 values, 7.21 at 200,
and diverging streams are zero at every length.
#### Next Step
Two heaps gave the median, that is, the **middle value in sorted order**. The median is a
special case; the general question is what the **k-th** value is in sorted order. The next
lesson answers this question with two separate patterns — a k-sized heap and partitioning —
and tests both against the same oracle. There the precondition touches not the data but the
**question itself**: in duplicate values, "the k-th element" and "the k-th distinct value"
are not the same thing, and that difference will be counted.
### K-th Element
Source: https://academia.sh/en/courses/advanced-algorithms/kth-element
Heap-based and partition-based solutions to selection problems; the 27 diverging inputs that appear once the question shifts, and the 15-fold step increase from pivot choice.
The previous pattern gave the median, the value **in the middle** of sorted order. The median
is a special case of a general question: what is the **k-th** value in sorted order. If k is
in the middle, that is the median; if k is one, the smallest value is asked; if k is n, the
largest.
This lesson answers the question with two separate patterns. The first keeps a k-sized
**heap**; the second **partitions** the array and searches only the relevant half. Both are
tested against the same oracle, and two different kinds of precondition emerge: one touches
**the question's definition**, the other touches **the input's arrangement**.
#### Two Patterns, One Oracle
The oracle places the array front to back by comparison and reads the k-th position; every
comparison is counted as one step. The heap pattern holds only k elements: every new value
goes into a max-heap, and if the heap exceeds k, its top is popped. The value left on top at
the end is the k-th smallest among all values seen. The heap invariant and sift operations
were established in the Data Structures course's Heaps lesson; they are not repeated here.
The same structure appears to answer one more question: **the k-th smallest distinct
value.** The only difference is that a value already seen is not put back into the heap. The
code is nearly identical, the output is a number, the pattern is still cheap.
**PP47.** The corpus is the previous lessons' corpus: 40 arrays, 12 values each, ranging from
−9 to 20; seed `20260218`.
**PP48.** The oracle's answer is the **k-th element**: the value at position k in sorted
order, duplicates included.
**PP49.** The heap's steps are real comparisons; values are placed into a wrapper that counts
comparisons.
**PP50.** Four k values are swept: 3, 4, 6, and 8.
```python
import heapq
SEED, LENGTH, CORPUS_SIZE = 20260218, 12, 40
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def corpus(seed=SEED, n=CORPUS_SIZE, length=LENGTH):
r = generator(seed)
return [{"no": i + 1, "array": [r(30) - 9 for _ in range(length)]}
for i in range(n)]
class Counter:
def __init__(self):
self.step = 0
def count(self):
self.step += 1
class Counted:
"""Counts every heap comparison. sign=-1 gives a max-heap."""
__slots__ = ("d", "sign", "s")
def __init__(self, d, sign, s):
self.d, self.sign, self.s = d, sign, s
def __lt__(self, o):
self.s.count()
return self.d * self.sign < o.d * o.sign
def oracle_kth(array, k, s):
"""Places the whole array by comparison, reads the k-th position."""
a = list(array)
for i in range(len(a)):
least = i
for j in range(i + 1, len(a)):
s.count()
if a[j] < a[least]:
least = j
a[i], a[least] = a[least], a[i]
return a[k - 1]
def pattern_heap(array, k, s):
"""Keeps a k-sized max-heap; the top is the k-th smallest value."""
heap = []
for x in array:
heapq.heappush(heap, Counted(x, -1, s))
if len(heap) > k:
heapq.heappop(heap)
return heap[0].d
def pattern_distinct_value(array, k, s):
"""SAME structure, DIFFERENT question: the k-th smallest DISTINCT value."""
heap, seen = [], set()
for x in array:
s.count()
if x in seen:
continue
seen.add(x)
heapq.heappush(heap, Counted(x, -1, s))
if len(heap) > k:
heapq.heappop(heap)
return heap[0].d if len(heap) == k else None
K40 = corpus()
print("corpus: 40 arrays x 12 values | arrays with a duplicate value:",
sum(1 for k in K40 if len(set(k["array"])) < LENGTH))
print(" k pattern diverging/40 pattern oracle ratio")
for k in (3, 4, 6, 8):
for ad, pattern in (("k-th element ", pattern_heap),
("k-th distinct val", pattern_distinct_value)):
diverging, pk, ok = 0, 0, 0
for record in K40:
s1, s2 = Counter(), Counter()
a = pattern(record["array"], k, s1)
b = oracle_kth(record["array"], k, s2)
pk, ok = pk + s1.step, ok + s2.step
diverging += (a != b)
print(f"{k:2d} {ad} {diverging:10d} {pk:5d} {ok:5d} {ok / pk:5.2f}")
print()
first = K40[0]["array"]
print("input 1 :", first)
print(" sorted order :", sorted(first))
print(" oracle k=4 :", oracle_kth(first, 4, Counter()))
print(" heap k=4 :", pattern_heap(first, 4, Counter()))
print(" distinct k=4 :", pattern_distinct_value(first, 4, Counter()))
```
```
corpus: 40 arrays x 12 values | arrays with a duplicate value: 37
k pattern diverging/40 pattern oracle ratio
3 k-th element 0 1440 2640 1.83
3 k-th distinct val 9 1640 2640 1.61
4 k-th element 0 1559 2640 1.69
4 k-th distinct val 17 1723 2640 1.53
6 k-th element 0 1348 2640 1.96
6 k-th distinct val 27 1503 2640 1.76
8 k-th element 0 1292 2640 2.04
8 k-th distinct val 30 1381 2640 1.91
input 1 : [-8, -5, 2, -1, 2, 5, 4, 1, 16, 17, 6, -1]
sorted order : [-8, -5, -1, -1, 1, 2, 2, 4, 5, 6, 16, 17]
oracle k=4 : -1
heap k=4 : -1
distinct k=4 : 1
```
The heap pattern matches the oracle on **40 of 40 inputs** at all four k values; the ratio
falls between 1.69 and 2.04. The distinct-value form diverges more as k grows: **9, 17, 27,
30**.
The first input's difference shows this at a glance. Sorted order is `[-8, -5, -1, -1, 1,
...]`; the fourth element is **−1**, because −1 appears twice. The fourth **distinct** value
is **1**. Both answers are correct — but to two **different questions**. This is not a flaw
in the pattern, it is the question shifting.
#### The Question Looks the Same, But Is Not
To show that the divergence really comes from duplicates, a control is needed: the same
measurement, repeated on a corpus with duplicates stripped out. If there are no duplicates,
the two questions should become **identical** and the divergence should be zero.
**PP51.** The control corpus consists of the same arrays' distinct values; it comes from no
other generator. Because the arrays shrink, only k=3 and k=6 are measured.
**PP52.** The second corpus comes from seed `20260219` and is swept with the same four k
values.
```python
import heapq
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def corpus(seed, n=40, length=12):
r = generator(seed)
return [[r(30) - 9 for _ in range(length)] for _ in range(n)]
def oracle_kth(array, k):
a = list(array)
for i in range(len(a)):
least = i
for j in range(i + 1, len(a)):
if a[j] < a[least]:
least = j
a[i], a[least] = a[least], a[i]
return a[k - 1]
def pattern_heap(array, k):
heap = []
for x in array:
heapq.heappush(heap, -x)
if len(heap) > k:
heapq.heappop(heap)
return -heap[0]
def pattern_distinct_value(array, k):
heap, seen = [], set()
for x in array:
if x in seen:
continue
seen.add(x)
heapq.heappush(heap, -x)
if len(heap) > k:
heapq.heappop(heap)
return -heap[0] if len(heap) == k else None
print("seed k k-th element k-th distinct value distinct-value ratio")
for seed in (20260218, 20260219):
K = corpus(seed)
for k in (3, 4, 6, 8):
e = sum(1 for d in K if pattern_heap(d, k) != oracle_kth(d, k))
f = sum(1 for d in K if pattern_distinct_value(d, k) != oracle_kth(d, k))
print(f"{seed} {k:2d} {e:13d} {f:17d} {f / 40:16.4f}")
print()
print("same measurement on a duplicate-free corpus (each array's distinct values):")
for seed in (20260218, 20260219):
K = [sorted(set(d)) for d in corpus(seed)]
for k in (3, 6):
f = sum(1 for d in K if pattern_distinct_value(d, k) != oracle_kth(d, k))
print(f" {seed} k={k} diverging {f}/40")
```
```
seed k k-th element k-th distinct value distinct-value ratio
20260218 3 0 9 0.2250
20260218 4 0 17 0.4250
20260218 6 0 27 0.6750
20260218 8 0 30 0.7500
20260219 3 0 12 0.3000
20260219 4 0 24 0.6000
20260219 6 0 28 0.7000
20260219 8 0 32 0.8000
same measurement on a duplicate-free corpus (each array's distinct values):
20260218 k=3 diverging 0/40
20260218 k=6 diverging 0/40
20260219 k=3 diverging 0/40
20260219 k=6 diverging 0/40
```
The control is decisive: on a corpus with duplicates stripped out, diverging inputs are
**zero** on all four measurements. The source of divergence is not the pattern, it is the
duplicates in the corpus and **the question's definition**.
In the second corpus, ratios fall between 0.30 and 0.80; in the first, between 0.2250 and
0.7500 — the **same order of magnitude**, rising in the same direction. The result does not
depend on the corpus. The heap pattern's k-th-element column is **zero** on all eight of
eight rows.
The warning that follows from this is the topic's most practical one. The difference between
the two questions is **a single line in code**, and that line looks like an improvement: not
putting the same value into the heap twice shrinks the heap and lowers the step count. On
none of the eight rows is there any sign that this line is wrong; the only sign is in the
oracle's column.
#### Partitioning's Precondition Is Pivot Choice
The second pattern **partitions** the array around a **pivot**: values smaller than the pivot
move left, larger ones move right, and the pivot settles into its final position. If that
position is k, the answer is found; if not, the search continues on one side only. The
partitioning procedure was established inside quicksort in the Algorithms course; it is not
repeated here.
This pattern's precondition touches not correctness but **step count**: the pivot must
actually split the array in two. If the pivot always lands on the array's edge value, the
split advances by one element at a time and the search space shrinks slowly.
**PP53.** Two pivot choices are compared: the section's **first** value and its **middle**
value.
**PP54.** k is the array's middle (`length // 2`), that is, the position that demands the most
partitioning.
**PP55.** The measurement is repeated at two lengths: 12 and 60 values. The sorted pool is the
same arrays in sorted form.
```python
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def corpus(seed, n=40, length=12):
r = generator(seed)
return [[r(30) - 9 for _ in range(length)] for _ in range(n)]
class Counter:
def __init__(self):
self.step = 0
def count(self):
self.step += 1
def oracle_kth(array, k, s):
a = list(array)
for i in range(len(a)):
least = i
for j in range(i + 1, len(a)):
s.count()
if a[j] < a[least]:
least = j
a[i], a[least] = a[least], a[i]
return a[k - 1]
def pattern_partition(array, k, pivot_first, s):
"""Selection by partitioning. PRECONDITION: pivot choice must not coincide with input order."""
a, left, right = list(array), 0, len(array) - 1
while left < right:
p = left if pivot_first else (left + right) // 2
a[p], a[right] = a[right], a[p]
pivot, i = a[right], left
for j in range(left, right):
s.count()
if a[j] < pivot:
a[i], a[j] = a[j], a[i]
i += 1
a[i], a[right] = a[right], a[i]
if k - 1 == i:
return a[i]
if k - 1 < i:
right = i - 1
else:
left = i + 1
return a[left]
print("length input pivot diverging/40 pattern oracle ratio")
for length in (12, 60):
K = corpus(20260218, 40, length)
for gad, prepare in (("unsorted", list), ("sorted ", sorted)):
for pad, pivot_first in (("first value", True), ("middle val ", False)):
diverging, pk, ok = 0, 0, 0
for array in K:
d = prepare(array)
s1, s2 = Counter(), Counter()
a = pattern_partition(d, length // 2, pivot_first, s1)
b = oracle_kth(d, length // 2, s2)
pk, ok = pk + s1.step, ok + s2.step
diverging += (a != b)
print(f"{length:6d} {gad} {pad} {diverging:10d} {pk:5d}"
f" {ok:6d} {ok / pk:6.2f}")
```
```
length input pivot diverging/40 pattern oracle ratio
12 unsorted first value 0 943 2640 2.80
12 unsorted middle val 0 1073 2640 2.46
12 sorted first value 0 2040 2640 1.29
12 sorted middle val 0 473 2640 5.58
60 unsorted first value 0 6726 70800 10.53
60 unsorted middle val 0 7346 70800 9.64
60 sorted first value 0 53400 70800 1.33
60 sorted middle val 0 3407 70800 20.78
```
Eight of eight rows give zero diverging inputs. Selection by partitioning gives the correct
answer regardless of pivot; this pattern's precondition has nothing to do with correctness.
The step column is striking on 60-value sorted input: choosing the first value as pivot
spends **53,400** steps, choosing the middle value spends **3407**. A single-line choice
multiplies the step count by **15.7 times** and drops the pattern's ratio from 20.78 to
**1.33** — the pattern becomes nearly as expensive as the oracle.
This row is another face of the topic's second claim: **a broken precondition does not always
mean a wrong answer.** Sometimes it is only the step count that explodes, and it does not
require broken data to explode — here the input is **sorted**, that is, in its most orderly
state. What breaks the pattern is not the data's disorder, it is the pivot rule
**coinciding** with that order.
#### Three Numbers
| Metric | Oracle | Pattern | Diverging input |
|---|---|---|---|
| Heap, k-th element (k=6) | 2640 steps | 1348 steps | **0/40** |
| Heap, k-th distinct value (k=6) | 2640 steps | 1503 steps | **27/40** |
| Partitioning, 60 values, unsorted, middle pivot | 70,800 steps | 7346 steps | **0/40** |
| Partitioning, 60 values, sorted, first pivot | 70,800 steps | 53,400 steps | **0/40** |
The first two rows show that correctness depends on the question's definition; the last two
show that performance depends on the pivot rule. In none of the four rows does the step
column signal the diverging-input column: the row with the lowest step count is correct, the
row with the second-lowest step count is wrong on 27 inputs.
#### Summary
- The k-th element question generalizes the median; keeping a k-sized heap gives the answer
without placing the whole array.
- The heap pattern matches the oracle on 40 of 40 inputs at all four k values; the ratio
falls between 1.69 and 2.04.
- On duplicate values, "the k-th element" and "the k-th distinct value" are separate
questions; the second diverges from the oracle on **27 inputs** for k=6, and the difference
is a single line of code.
- On a duplicate-stripped control corpus, diverging inputs are zero on all four measurements;
the source is not the pattern, it is the question's definition.
- Selection by partitioning is correct with any pivot, but on 60-value sorted input, choosing
the first value as pivot raises the step count from 3407 to 53,400 and drops the ratio from
20.78 to 1.33.
#### Next Step
All seven of this topic's patterns so far worked on one-dimensional data: an array, a stream,
a successor chain. The last pattern handles data in **two dimensions**. In a grid, connected
groups of cells are searched for, and the traversal is done with the Data Structures course's
breadth-first and depth-first search — those two procedures are not repeated, they are called
directly. The precondition is again a choice of definition: **what connectivity means.** The
next lesson counts how many different answers two connectivity definitions produce on the
same grid.
### Grid Traversal
Source: https://academia.sh/en/courses/advanced-algorithms/grid-traversal
Connected-component search on a matrix; how the connectivity definition changes the answer on 38 grids, and what marking and traversal choice do to held cell count.
All seven patterns in this topic so far worked on one-dimensional data: an array, a stream, a
successor chain. The last pattern handles data in **two dimensions**. A grid has filled
cells, and the question is: how many sets of cells connected to one another are there.
A grid is not really a new structure. Every filled cell is a node, every pair of neighboring
cells is an edge; what is asked is the number of **connected components**, and the procedure
was established in the Data Structures course. Breadth-first and depth-first search
themselves, why marking a visit is required, and the $O(V + E)$ cost were measured there;
they are **not repeated** here, they are used directly.
What the pattern adds is the modeling, and that is where its precondition sits: **what
connectivity means.** This lesson counts how many different answers two connectivity
definitions produce on the same grid.
#### Problem, Oracle, and Pattern
The oracle does no traversal at all. It scans every pair of filled cells, groups adjacent ones
together, and repeats until no merge remains. The pattern starts a breadth-first search from
every unvisited filled cell, and every search counts as one component.
**PP56.** The corpus is 40 grids; each grid is 8×8, and every cell is filled with a
probability of 30 in 100. Seed `20260218`.
**PP57.** Two connectivity definitions are measured: **4-connectivity** (edge-adjacent cells
only) and **8-connectivity** (corner-adjacent cells included too).
**PP58.** The oracle and the pattern are run **separately** with each connectivity
definition; all four combinations are measured. This shows whether a divergence comes from
the pattern or from a difference in definition.
**PP59.** The visit is marked **on enqueue**; this is the rule established in the
Breadth-First Search lesson and its rationale is not repeated here.
```python
from collections import deque
SEED, SIZE, CORPUS_SIZE = 20260218, 8, 40
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def grid_corpus(seed=SEED, n=CORPUS_SIZE, size=SIZE):
r = generator(seed)
return [{"no": i + 1,
"grid": [[1 if r(101) < 30 else 0 for _ in range(size)]
for _ in range(size)]}
for i in range(n)]
class Counter:
def __init__(self):
self.step = 0
def count(self):
self.step += 1
def adjacent(a, b, connectivity):
di, dj = abs(a[0] - b[0]), abs(a[1] - b[1])
return di + dj == 1 if connectivity == 4 else max(di, dj) == 1 and (di or dj)
def oracle_component(grid, connectivity, s):
"""Scans every cell pair, groups adjacent ones, repeats until no change.
Always correct, always expensive."""
cell = [(i, j) for i, row in enumerate(grid)
for j, v in enumerate(row) if v]
group = {h: i for i, h in enumerate(cell)}
changed = True
while changed:
changed = False
for a in range(len(cell)):
for b in range(a + 1, len(cell)):
s.count()
if adjacent(cell[a], cell[b], connectivity) and \
group[cell[a]] != group[cell[b]]:
old, new = group[cell[b]], group[cell[a]]
for h in cell:
if group[h] == old:
group[h] = new
changed = True
return len(set(group.values()))
DIR4 = ((-1, 0), (1, 0), (0, -1), (0, 1))
DIR8 = DIR4 + ((-1, -1), (-1, 1), (1, -1), (1, 1))
def pattern_bfs(grid, connectivity, s):
"""The Data Structures course's breadth-first search is used directly: the
visit is marked ON ENQUEUE."""
direction = DIR4 if connectivity == 4 else DIR8
n, m = len(grid), len(grid[0])
visited, component, widest = set(), 0, 0
for i in range(n):
for j in range(m):
if not grid[i][j] or (i, j) in visited:
continue
component += 1
visited.add((i, j))
queue = deque([(i, j)])
while queue:
widest = max(widest, len(queue))
x, y = queue.popleft()
s.count()
for dx, dy in direction:
a, b = x + dx, y + dy
if 0 <= a < n and 0 <= b < m and grid[a][b] \
and (a, b) not in visited:
visited.add((a, b))
queue.append((a, b))
return component, widest
K = grid_corpus()
filled = sum(sum(sum(row) for row in k["grid"]) for k in K)
print("corpus:", len(K), "grids x", SIZE, "x", SIZE, "| filled cells:", filled,
"| average", round(filled / len(K), 2))
print("pattern connectivity oracle connectivity diverging/40 pattern oracle ratio")
for pc in (4, 8):
for oc in (4, 8):
diverging, pk, ok = 0, 0, 0
for record in K:
s1, s2 = Counter(), Counter()
a, _ = pattern_bfs(record["grid"], pc, s1)
b = oracle_component(record["grid"], oc, s2)
pk, ok = pk + s1.step, ok + s2.step
diverging += (a != b)
print(f"{pc:19d} {oc:19d} {diverging:10d} {pk:5d} {ok:6d}"
f" {ok / pk:6.1f}")
```
```
corpus: 40 grids x 8 x 8 | filled cells: 723 | average 18.07
pattern connectivity oracle connectivity diverging/40 pattern oracle ratio
4 4 0 723 12768 17.7
4 8 38 723 12834 17.8
8 4 38 723 12768 17.7
8 8 0 723 12834 17.8
```
The diagonal rows are zero, the other two are **38**. When the pattern uses the **same
connectivity definition** as the oracle, it gives the same answer on 40 of 40 grids; once the
definitions diverge, 38 of 40 grids diverge.
This table makes one thing certain: **the divergence does not come from the pattern.**
Breadth-first search works correctly with either definition; what is broken is not the
pattern itself, it is the **connectivity definition** given to it. The pattern's step count is
**723** in both definitions, the oracle's is 12,768 to 12,834; the ratio is **17.7**. The step
column is nearly identical across all four rows and gives no signal at all about the
diverging-input column.
#### Connectivity Is Not a Data Property, It Is a Decision
In the previous seven patterns, the precondition was a property of the input: sorted or not,
negative or not, duplicate or not. Here the precondition is not in the input; it is **a
decision made by whoever modeled the problem**, and cannot be verified by looking at the
data. The small grid below shows this in a glance.
**PP60.** The small example is built by hand; it comes from no generator and exists only to
show the divergence between the two definitions.
**PP61.** The second corpus comes from seed `20260219`. What is measured is **how many
grids** the two definitions give a different component count on.
```python
from collections import deque
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def grid_corpus(seed, n=40, size=8):
r = generator(seed)
return [[[1 if r(101) < 30 else 0 for _ in range(size)] for _ in range(size)]
for _ in range(n)]
DIR4 = ((-1, 0), (1, 0), (0, -1), (0, 1))
DIR8 = DIR4 + ((-1, -1), (-1, 1), (1, -1), (1, 1))
def component(grid, connectivity):
"""Component count by breadth-first search; the visit is marked on enqueue."""
direction = DIR4 if connectivity == 4 else DIR8
n, m = len(grid), len(grid[0])
visited, count = set(), 0
for i in range(n):
for j in range(m):
if not grid[i][j] or (i, j) in visited:
continue
count += 1
visited.add((i, j))
queue = deque([(i, j)])
while queue:
x, y = queue.popleft()
for dx, dy in direction:
a, b = x + dx, y + dy
if 0 <= a < n and 0 <= b < m and grid[a][b] \
and (a, b) not in visited:
visited.add((a, b))
queue.append((a, b))
return count
small = [[1, 0, 0, 0, 1],
[0, 1, 0, 1, 0],
[0, 0, 1, 0, 0],
[0, 1, 0, 1, 0],
[1, 0, 0, 0, 1]]
for row in small:
print(" ", "".join("#" if v else "." for v in row))
print(" 4-connectivity:", component(small, 4), "components")
print(" 8-connectivity:", component(small, 8), "components")
print()
print("seed 4-connectivity total 8-connectivity total differing grid/40")
for seed in (20260218, 20260219):
K = grid_corpus(seed)
d4 = [component(g, 4) for g in K]
d8 = [component(g, 8) for g in K]
differing = sum(1 for a, b in zip(d4, d8) if a != b)
print(f"{seed} {sum(d4):17d} {sum(d8):17d} {differing:16d}")
```
```
#...#
.#.#.
..#..
.#.#.
#...#
4-connectivity: 9 components
8-connectivity: 1 components
seed 4-connectivity total 8-connectivity total differing grid/40
20260218 387 233 38
20260219 377 227 38
```
The five-row grid has nine cells and none of them is edge-adjacent; all of them sit on a
diagonal. With 4-connectivity there are **9 separate components**; with 8-connectivity, **a
single component**. Same data, same procedure, a ninefold difference.
In the forty-grid corpus, total component count drops from **387 to 233**, and the two
definitions give a different answer on **38 grids**. In the second corpus the numbers are 377
and 227, differing grids again **38**; the result does not depend on the corpus.
The practical conclusion of this measurement is this: **the first question to ask when
reading a grid problem is what connectivity means.** If the definition is not written in the
problem statement, the correctness of the pattern's answer cannot be tested — not even the
oracle can be built, because the oracle needs the same definition too. This is the topic's
only precondition that **cannot be determined by looking at the data**.
#### Marking and Traversal Change What Is Held, Not the Answer
Once connectivity is fixed, two implementation decisions remain: whether marking happens on
enqueue or on dequeue, and whether the traversal is breadth-first or depth-first. Neither
changes the component count. What they change is the **number of cells held at once**.
**PP62.** This measurement is done on a larger grid: 10 grids, each 20×20, cells filled with
probability 60 in 100. In a small, sparse grid the difference between these three procedures
falls below the resolution.
**PP63.** What is measured is, for breadth-first search, the **queue's widest point**, and
for depth-first search, the **largest value of recursion depth**.
```python
from collections import deque
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def grid_corpus(seed=20260218, n=10, size=20):
r = generator(seed)
return [[[1 if r(101) < 60 else 0 for _ in range(size)] for _ in range(size)]
for _ in range(n)]
DIR4 = ((-1, 0), (1, 0), (0, -1), (0, 1))
def bfs(grid, on_enqueue):
"""Is marking done ON ENQUEUE or ON DEQUEUE."""
n, m = len(grid), len(grid[0])
visited, component, step, widest = set(), 0, 0, 0
for i in range(n):
for j in range(m):
if not grid[i][j] or (i, j) in visited:
continue
component += 1
queue = deque([(i, j)])
if on_enqueue:
visited.add((i, j))
while queue:
widest = max(widest, len(queue))
x, y = queue.popleft()
step += 1
if not on_enqueue:
if (x, y) in visited:
continue
visited.add((x, y))
for dx, dy in DIR4:
a, b = x + dx, y + dy
if 0 <= a < n and 0 <= b < m and grid[a][b] \
and (a, b) not in visited:
if on_enqueue:
visited.add((a, b))
queue.append((a, b))
return component, step, widest
def dfs(grid):
"""Recursive depth-first search; the deepest stack size is measured."""
n, m = len(grid), len(grid[0])
visited, component, step, deepest = set(), 0, 0, 0
def visit(x, y, depth):
nonlocal step, deepest
step += 1
deepest = max(deepest, depth)
visited.add((x, y))
for dx, dy in DIR4:
a, b = x + dx, y + dy
if 0 <= a < n and 0 <= b < m and grid[a][b] \
and (a, b) not in visited:
visit(a, b, depth + 1)
for i in range(n):
for j in range(m):
if grid[i][j] and (i, j) not in visited:
component += 1
visit(i, j, 1)
return component, step, deepest
K = grid_corpus()
base = [bfs(g, True) for g in K]
print("corpus: 10 grids x 20 x 20 | filled cells:",
sum(sum(sum(row) for row in g) for g in K))
print("traversal same component steps most held cells")
for ad, result in (("bfs, mark on enqueue ", base),
("bfs, mark on dequeue ", [bfs(g, False) for g in K]),
("dfs (recursive) ", [dfs(g) for g in K])):
same = sum(1 for a, b in zip(result, base) if a[0] == b[0])
print(f"{ad} {same:12d} {sum(x[1] for x in result):4d}"
f" {max(x[2] for x in result):20d}")
```
```
corpus: 10 grids x 20 x 20 | filled cells: 2378
traversal same component steps most held cells
bfs, mark on enqueue 10 2378 18
bfs, mark on dequeue 10 2887 24
dfs (recursive) 10 2378 103
```
All three rows give the same component count: 10 of 10 grids match. Diverging inputs are
**zero**, and these rows are not a correctness measurement.
The difference measured sits in two columns instead. Marking on dequeue raises steps from
2378 to **2887** and the widest queue from 18 to **24**: the same cell enters the queue more
than once before being visited. Depth-first search's step count is **identical** to breadth-
first search's (2378), but its deepest stack holds **103** cells — roughly **five and a half
times** breadth-first search's widest queue.
This last number is a practical limit. On a dense, large grid, recursive depth-first search
demands a stack depth proportional to the grid's size; breadth-first search demands a queue
proportional to the widest layer. Which is cheaper depends on the grid's **shape**, and this
distinction was already established in the Data Structures course's comparison table; here
the same distinction is confirmed numerically.
#### Three Numbers
| Metric | Oracle | Pattern | Diverging input |
|---|---|---|---|
| 4-connectivity, 4-connectivity | 12,768 steps | 723 steps | **0/40** |
| 8-connectivity, 4-connectivity | 12,768 steps | 723 steps | **38/40** |
| Breadth-first search, 20×20 grid | — | 2378 steps / 18 cells | **0/10** |
| Depth-first search, 20×20 grid | — | 2378 steps / 103 cells | **0/10** |
In the first two rows the pattern's step count is **identical**, and diverging inputs range
from 0 to 38; in the last two rows diverging inputs are identical and held cell count
changes by five and a half times. Two kinds of decision show up in two separate columns —
and neither leaves a trace in the other's column.
#### Summary
- A grid is a graph: filled cells are nodes, adjacent cell pairs are edges; connected-
component counting is done with the Data Structures course's breadth-first and depth-first
search.
- When the pattern uses the same connectivity definition as the oracle, it is correct on 40 of
40 grids; once the definitions diverge, it diverges on **38 grids**, and the pattern's step
count is 723 in both cases.
- Connectivity is a decision made by whoever models the problem, not a property of the data;
it cannot be determined by looking at the data, and the oracle cannot even be built without
it.
- A nine-cell diagonal grid gives 9 components under 4-connectivity and 1 under
8-connectivity; the total over the 40-grid corpus drops from 387 to 233.
- Marking on dequeue does not change the answer but raises steps from 2378 to 2887 and the
queue from 18 to 24; depth-first search holds a stack of 103 cells at the same step count.
#### Next Step
This topic measured eight patterns under the same frame: an oracle, a pattern, and the number
of inputs where the two diverge. In all eight lessons, the source of divergence was not a bug
in the pattern — it was **an accepted but unchecked precondition**: order, sign, single
successor, sort key, value range, balance, the question's definition, and connectivity. The
next topic moves from patterns to **named problems**: the knapsack, the traveling salesman,
the longest path, the n-queens placement. The question asked there changes — not whether a
pattern's precondition has broken, but why a problem is **hard**; and the oracle remains the
tool by which that hardness is measured.
## Classical Problems
Comparing solutions to the knapsack, travelling salesman, longest path, n-queens, knight's tour, and Hamiltonian path problems against an oracle, and measuring the deviation of approximate solutions.
### The Knapsack Problem
Source: https://academia.sh/en/courses/advanced-algorithms/knapsack-problem
The same greedy order giving two different results in two variants: in the fractional knapsack, the same answer as the oracle on 40 of 40 inputs, at 579.3 times fewer steps; in the 0/1 variant, the same order splits from the oracle on 5 of 40 inputs, with a worst loss of 7 value units. Dynamic programming matches the oracle on 40/40 in 4544 steps, against the oracle's 15,360. On the second input pool, the split count is 6, keeping the ratio at the same order of magnitude.
The Problem-Solving Patterns topic measured eight patterns together with their
preconditions. There, the problem's name was the pattern's name: two pointers,
sliding window, grid traversal. This topic turns that around. It takes up six
classical problems whose names come not from a pattern but from the **problem
itself**, and in each one the question is: which pattern does this problem fit,
and where does the pattern it seems to fit break down.
The first problem is the knapsack, and it has two faces. Items with weight and
value, and a bag of limited capacity. In the **fractional** variant, part of an
item can be taken; in the **0/1** variant, an item is taken whole or not at all.
This one-word difference is enough to make the same greedy order match the oracle
**on every input** in one variant and **be wrong** in the other.
- **CP1.** The shared definition's generator and input pool are used unchanged:
seed **20260218**, second pool **20260219**, 40 sequences with 12 values each.
- **CP2.** In every sequence, the **first six values are weight**, the **last six
are value**. Both are made absolute and incremented by one, so weight and value
stay between 1 and 21.
- **CP3.** Capacity is half the total weight, rounded down. It is derived from
the input, not chosen by hand; otherwise the desired result could be produced
just by picking the capacity.
- **CP4.** The oracle is brute force in both variants: **every fill order** in
the fractional variant, **all 64 subsets** in the 0/1 variant. The oracle is
expensive, and it stays that way.
- **CP5.** The measure is **steps**; time is not measured. A step is the one
unit the oracle and the pattern can be compared in.
- **CP6.** The split-input count is out of 40. Per the shared definition's
Section 5 resolution, **a difference below 3 counts as unmeasured**.
- **CP7.** The greedy order is the same in both variants: decreasing
**value-to-weight** ratio. The only thing that changes is whether an item can
be split.
- **CP8.** The dynamic programming procedure was built in the Algorithm Design
Approaches topic; it is **not rebuilt**, only compared against the oracle.
- **CP9.** Capacity is also swept as a **setting**. A split count measured at a
single capacity and generalized is a consequence of the chosen setting, and it
is written that way.
#### The Fractional Variant and Its Oracle
In the fractional variant, any desired fraction of an item can be taken. The
intuition is straightforward: start with the item giving the most value per unit
weight, continue in order until the bag is full, and take a fraction of the last
item equal to the remaining space.
This intuition is not **claimed** to be correct — it is **tested**. The oracle
tries every fill order of the six items and keeps the best. A fill order fills
the bag in sequence; whichever order is taken, the value obtained cannot exceed
the best possible value, and the order giving the best value is among these 720
orders. So the oracle is genuinely an upper bound.
```python
from itertools import permutations
SEED = 20260218
def generator(seed):
d = seed
def next_val(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_val
def item_pool(seed=SEED, n=40, length=12):
r = generator(seed)
return [{"no": i + 1, "values": [r(30) - 9 for _ in range(length)]}
for i in range(n)]
def items(values):
"""Six items from the shared definition's 12 values: weight first six, value last six."""
return [(abs(values[i]) + 1, abs(values[i + 6]) + 1) for i in range(6)]
class Counter:
def __init__(self):
self.steps = 0
def count(self):
self.steps += 1
def oracle_fractional(its, capacity, c):
"""Every fill order is tried, the best is kept."""
best = 0.0
for p in permutations(range(len(its))):
remaining, v = capacity, 0.0
for i in p:
c.count()
w, val = its[i]
if w <= remaining:
remaining, v = remaining - w, v + val
else:
v += val * remaining / w
break
best = max(best, v)
return round(best, 4)
def pattern_fractional(its, capacity, c):
"""PRECONDITION: items must be divisible. Greedy by value-to-weight ratio."""
remaining, v = capacity, 0.0
for w, val in sorted(its, key=lambda t: -t[1] / t[0]):
c.count()
if w <= remaining:
remaining, v = remaining - w, v + val
else:
v += val * remaining / w
break
return round(v, 4)
split, pk, ok = 0, 0, 0
for k in item_pool():
its = items(k["values"])
capacity = sum(w for w, _ in its) // 2
c1, c2 = Counter(), Counter()
if pattern_fractional(its, capacity, c1) != oracle_fractional(its, capacity, c2):
split += 1
pk, ok = pk + c1.steps, ok + c2.steps
print("first example (weight, value):", items(item_pool()[0]["values"]))
print("fractional | inputs 40 | split", split, "| pattern steps", pk,
"| oracle steps", ok, "| ratio", round(ok / pk, 1))
```
```
first example (weight, value): [(9, 5), (6, 2), (3, 17), (2, 18), (3, 7), (6, 2)]
fractional | inputs 40 | split 0 | pattern steps 174 | oracle steps 100800 | ratio 579.3
```
Three numbers sit side by side: **the oracle at 100,800 steps**, **the pattern at
174 steps**, **0 split inputs**. The pattern spends 579.3 times fewer steps and
matches the oracle on 40 of 40 inputs.
The reason greedy never goes wrong in the fractional variant is that the freedom
to take a fraction allows an **exchange**. If a unit of weight in the bag belongs
to a low-ratio item, removing that unit and putting in a unit from a high-ratio
item instead never lowers the value. Because this exchange argument always holds,
no solution that deviates from ratio order can beat ratio order. The precondition
is exactly this: **a unit of weight must be exchangeable.**
#### The Same Order, an Item That Cannot Be Split
In the 0/1 variant, this exchange disappears. Removing a unit of weight requires
removing the item's **whole** weight, and the replacement item comes in whole
too. The greedy order does not change, and the code is almost identical, but the
precondition is no longer there.
```python
SEED = 20260218
def generator(seed):
d = seed
def next_val(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_val
def item_pool(seed=SEED, n=40):
"""The shared definition's pool: 40 sequences x 12 values, six items per sequence."""
r = generator(seed)
pool = []
for i in range(n):
z = [r(30) - 9 for _ in range(12)]
pool.append((i + 1, [(abs(z[j]) + 1, abs(z[j + 6]) + 1) for j in range(6)]))
return pool
class Counter:
def __init__(self):
self.steps = 0
def count(self):
self.steps += 1
def oracle_01(its, capacity, c):
"""All 64 subsets of the six items are tried."""
best = 0
for mask in range(1 << len(its)):
w = v = 0
for i, (wi, vi) in enumerate(its):
c.count()
if mask >> i & 1:
w, v = w + wi, v + vi
if w <= capacity and v > best:
best = v
return best
def pattern_01(its, capacity, c):
"""Same greedy order, but items cannot be split."""
remaining, v = capacity, 0
for w, val in sorted(its, key=lambda t: -t[1] / t[0]):
c.count()
if w <= remaining:
remaining, v = remaining - w, v + val
return v
def dp_01(its, capacity, c):
best = [0] * (capacity + 1)
for w, val in its:
for cap in range(capacity, w - 1, -1):
c.count()
best[cap] = max(best[cap], best[cap - w] + val)
return best[capacity]
def measure(solver, seed):
split, pk, ok, loss = [], 0, 0, 0
for no, its in item_pool(seed):
capacity = sum(w for w, _ in its) // 2
c1, c2 = Counter(), Counter()
a, b = solver(its, capacity, c1), oracle_01(its, capacity, c2)
pk, ok = pk + c1.steps, ok + c2.steps
if a != b:
split.append(no)
loss = max(loss, b - a)
return split, pk, ok, loss
for name, solver, seed in (("greedy 20260218", pattern_01, 20260218),
("greedy 20260219", pattern_01, 20260219),
("dynamic 20260218", dp_01, 20260218)):
sp, pk, ok, f = measure(solver, seed)
print(name, "| split", len(sp), "/40", sp[:6], "| pattern steps", pk,
"| oracle steps", ok, "| worst loss", f)
```
```
greedy 20260218 | split 5 /40 [4, 21, 22, 32, 40] | pattern steps 240 | oracle steps 15360 | worst loss 7
greedy 20260219 | split 6 /40 [8, 11, 20, 28, 36, 39] | pattern steps 240 | oracle steps 15360 | worst loss 3
dynamic 20260218 | split 0 /40 [] | pattern steps 4544 | oracle steps 15360 | worst loss 0
```
#### Reading the Split Inputs
In 0/1, greedy splits from the oracle on **5 of 40 inputs**. The split ratio is
0.1250, above the resolution limit (3 inputs), so it counts as measured. The
numbers of the inputs it splits on are known: 4, 21, 22, 32, and 40. The worst
loss is **7 value units** — seven units short of the best value the bag could
carry.
This loss cannot be noticed by looking at the output. Greedy still returns a
number, still stays within capacity, still produces a valid selection. **Being
valid does not mean being optimal**, and only the oracle can reveal the
difference. There is no way to look at the pattern's answer alone and say it is
wrong.
The second input pool is mandatory and is applied here too: with seed
`20260219`, the split count is **6**, ratio 0.1500. The difference between five
and six is below the resolution, but both ratios sit at the same order of
magnitude. The result **does not depend on the pool**: in the 0/1 variant, greedy
is wrong on roughly one input in seven.
#### Inside One Split Input
The split-input count is, by itself, just a number; it does not show what
breaks. Opening up input 4 and placing the two selections side by side makes the
mechanism visible. The same block also sweeps capacity as a setting.
```python
SEED = 20260218
def generator(seed):
d = seed
def next_val(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_val
def item_pool(seed=SEED, n=40):
r = generator(seed)
pool = []
for i in range(n):
z = [r(30) - 9 for _ in range(12)]
pool.append((i + 1, [(abs(z[j]) + 1, abs(z[j + 6]) + 1) for j in range(6)]))
return pool
def oracle_01(its, capacity):
best, choice = 0, ()
for mask in range(1 << len(its)):
w = v = 0
for i, (wi, vi) in enumerate(its):
if mask >> i & 1:
w, v = w + wi, v + vi
if w <= capacity and v > best:
best, choice = v, tuple(i for i in range(len(its)) if mask >> i & 1)
return best, choice
def pattern_01(its, capacity):
remaining, v, choice = capacity, 0, []
for i in sorted(range(len(its)), key=lambda j: -its[j][1] / its[j][0]):
w, val = its[i]
if w <= remaining:
remaining, v = remaining - w, v + val
choice.append(i)
return v, tuple(sorted(choice))
its = dict(item_pool())[4]
capacity = sum(w for w, _ in its) // 2
print("input 4 |", its, "| capacity", capacity)
print(" greedy:", pattern_01(its, capacity), " oracle:", oracle_01(its, capacity))
print("capacity share -> split inputs (out of 40)")
for share in (2, 3, 4, 5, 6):
split = sum(1 for _, e in item_pool()
if pattern_01(e, sum(w for w, _ in e) * share // 8)[0]
!= oracle_01(e, sum(w for w, _ in e) * share // 8)[0])
print(f" {share}/8 -> {split}")
```
```
input 4 | [(5, 21), (6, 8), (9, 5), (2, 4), (19, 13), (18, 16)] | capacity 29
greedy: (38, (0, 1, 2, 3)) oracle: (45, (0, 1, 5))
capacity share -> split inputs (out of 40)
2/8 -> 3
3/8 -> 13
4/8 -> 5
5/8 -> 12
6/8 -> 8
```
Greedy takes four items and collects 38 value; the weight it uses is 22, leaving
7 units of empty space. The oracle takes three items, fills the capacity
**exactly**, and collects 45 value. The source of the difference is that greedy
takes the small, high-ratio items early and, as a result, leaves no room for the
sixth item, which has a low ratio but is **large**. The sixth item's ratio,
0.889, ranks fourth on the list, but it carries 16 value on its own. Greedy can
never see it, because it builds its order by ratio alone and **never looks
back**.
The capacity sweep gives a second, more unsettling result. The split count does
not change regularly with capacity: 3 at two-eighths of the total weight, 13 at
three-eighths, 5 at four-eighths, 12 at five-eighths, 8 at six-eighths. **The
pattern's wrong-answer count is not a constant; it is a function of the
setting.** If the measurement had been taken at a single capacity and
generalized, the 3 at the two-eighths setting would have given the impression
that greedy is almost always right; the 13 at the three-eighths setting would say
the exact opposite. Unless the setting the measurement was taken at is written
down, the number cannot be read.
#### The Cost of Dynamic Programming
The third row shows dynamic programming: matches the oracle on **40 of 40
inputs**, at **4544 steps**. The oracle spends 15,360 steps, so the correct
answer can be obtained with roughly a third of brute force's steps. But compared
with greedy's 240 steps, it is **18.9 times** more expensive.
These three rows are exactly what the course's reading of measurement is about.
Greedy is cheap and wrong; dynamic programming is expensive and correct; the
oracle is the most expensive, and its correctness is known **by definition**. The
choice between them is not a speed preference; it is the question of **how many
inputs a wrong answer is acceptable on**.
It should be noted that dynamic programming's step count grows with capacity.
Here capacities are small, because weights fall between 1 and 21. If weights
were multiplied by a thousand with the item count unchanged, dynamic
programming's steps would also multiply by a thousand; brute force's steps, by
contrast, would **not change at all**, since it only looks at the number of
subsets. Which procedure is expensive depends on which dimension of the input
grows.
#### Summary
- The knapsack's two variants share the same greedy order; the one point where
they differ is whether an item can be split, and that single point changes the
answer.
- In the fractional variant, greedy matches the oracle on 40 of 40 inputs in
**174 steps**; the oracle does the same work in **100,800 steps**.
- In the 0/1 variant, the same order splits from the oracle on **5 of 40
inputs**, with a worst loss of **7 value units**; on the second pool, the
split count is 6, at the same order of magnitude.
- The answer greedy produces is always valid; being valid does not mean being
optimal, and only the oracle reveals the difference.
- When capacity is swept as a setting, the split count moves between 3 and 13;
the pattern's wrong-answer count is not a constant, it is a function of the
setting.
- Dynamic programming gives a correct answer on 40/40 in 4544 steps: 3.4 times
cheaper than the oracle, 18.9 times more expensive than greedy.
#### Next Step
In the knapsack, the brute-force oracle scanned 64 subsets, and that scan stayed
small. The next lesson takes up the travelling salesman problem; there, the
number of candidate solutions is not subsets but **orderings**, and going from
six cities to ten makes the search space explode in a way that can be shown
numerically. The question will be: once the oracle grows too large to run at
all, how is an approximate solution's percentage deviation from the oracle
measured.
### The Travelling Salesman Problem
Source: https://academia.sh/en/courses/advanced-algorithms/travelling-salesman-problem
Making the best tour found by exhaustive enumeration on eight cities the oracle, and measuring how far two approximate solutions deviate from it in percent: nearest neighbor splits from the oracle on 17 of 20 samples, with an average deviation of 9.3 percent and a worst case of 33.33 percent; the two-opt improvement brings the split count down to 2 and the average deviation down to 0.49 percent, in 819 steps. The oracle spends 352,800 steps on the same 20 samples. The growth of the search space is shown by counting, not by running it: at eighteen cities the tour count is 177,843,714,048,000. The question of why it is hard is referred to the Theory of Computation course.
In the knapsack, the oracle scanned 64 subsets of six items, and that scan was
cheap. The candidate-solution count was an exponential function of the item
count, but the base of the exponent was two, and for six items 64 stayed a small
number. In the travelling salesman problem, the candidate count is not subsets
but **orderings**, and that number grows far faster.
The problem is this: cities and the distances between them are given, and the
**shortest closed tour** that visits every city exactly once and returns to the
start is sought. This lesson answers two questions separately. The first is
measurable: how far, in percent, does an approximate solution deviate from the
oracle. The second is not answered in this course: **why** this problem is hard.
The second question is a classification question and belongs to the Theory of
Computation course; here only the question itself is set up.
- **CP10.** City positions come from the shared definition's generator: a random
point on a 30-by-30 grid. Seed **20260218**, second pool **20260219**.
- **CP11.** Each pool has **20 samples**, **8 cities** per sample. The city
count is kept small so the oracle can be run; large city counts are conveyed
**by counting**.
- **CP12.** Distance is an integer: the integer square root of the sum of the
squared coordinate differences. Floating-point numbers are not used, so
equality between two tours can be tested exactly.
- **CP13.** The oracle is **exhaustive enumeration**: city zero is fixed, and
every ordering of the remaining seven cities is tried; since a tour's reverse
is the same tour, half are eliminated, leaving 2520 tours.
- **CP14.** The measure is **steps**. A step is computing the length of one
edge.
- **CP15.** Deviation is written as a **percentage**: the pattern's tour minus
the best tour, divided by the best tour. **Worst-case deviation** is placed
alongside the average; the average alone is misleading.
- **CP16.** The two-opt improvement removes two edges from the tour and
reconnects it by reversing the segment between them. It continues until no
improvement remains.
- **CP17.** Large city counts are **not run**, they are counted. The
exhaustive-enumeration table is produced by computing factorials; no tour is
actually scanned.
- **CP18.** The **name** of the difficulty is not written in this lesson.
Complexity classes, reductions, and approximation ratios belong to Theory of
Computation.
- **CP19.** How deviation changes with city count is also swept. A separate
pool is generated for each city count; the rows are not extensions of one
another.
#### The Oracle and Two Approximate Solutions
The nearest-neighbor procedure follows intuition directly: from the current
city, go to the nearest unvisited city, and return to the start once done.
Two-opt takes the resulting tour and fixes it locally; if two edges in the tour
cross, removing them and reversing the segment between them shortens the tour.
Neither is **claimed** to be correct. The oracle finds the best tour on every
sample by exhaustive enumeration, and both patterns are measured against it.
```python
from itertools import permutations
from math import isqrt
SEED = 20260218
def generator(seed):
d = seed
def next_val(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_val
def cities(seed=SEED, samples=20, n=8):
"""The shared definition's generator; each sample is n cities' (x, y) positions."""
r = generator(seed)
return [[(r(30), r(30)) for _ in range(n)] for _ in range(samples)]
def distance(p, q):
return isqrt((p[0] - q[0]) ** 2 + (p[1] - q[1]) ** 2)
class Counter:
def __init__(self):
self.steps = 0
def count(self):
self.steps += 1
def oracle_exhaustive(k, c):
"""City zero is fixed; every ordering of the remaining cities is tried."""
best = None
for p in permutations(range(1, len(k))):
if p[0] > p[-1]:
continue
tour, previous = 0, 0
for i in p:
c.count()
tour += distance(k[previous], k[i])
previous = i
tour += distance(k[previous], k[0])
if best is None or tour < best:
best = tour
return best
def nearest_neighbor(k, c):
remaining, path, current, tour = set(range(1, len(k))), [0], 0, 0
while remaining:
best_i, best_d = None, None
for i in remaining:
c.count()
d = distance(k[current], k[i])
if best_d is None or d < best_d:
best_i, best_d = i, d
remaining.remove(best_i)
path.append(best_i)
tour, current = tour + best_d, best_i
return tour + distance(k[current], k[0]), path
def two_opt(k, path, c):
n = len(path)
tour = sum(distance(k[path[i]], k[path[(i + 1) % n]]) for i in range(n))
improved = True
while improved:
improved = False
for i in range(1, n - 1):
for j in range(i + 1, n):
c.count()
a, b, cc, d = path[i - 1], path[i], path[j], path[(j + 1) % n]
delta = (distance(k[a], k[cc]) + distance(k[b], k[d])
- distance(k[a], k[b]) - distance(k[cc], k[d]))
if delta < 0:
path[i:j + 1] = reversed(path[i:j + 1])
tour, improved = tour + delta, True
return tour
for seed in (20260218, 20260219):
ha, totals = 0, {"nearest neighbor": [0, 0, 0.0, 0.0], "two-opt ": [0, 0, 0.0, 0.0]}
for k in cities(seed):
sh = Counter()
best = oracle_exhaustive(k, sh)
ha += sh.steps
s1, s2 = Counter(), Counter()
t1, path = nearest_neighbor(k, s1)
t2 = two_opt(k, list(path), s2)
for name, t, s in (("nearest neighbor", t1, s1), ("two-opt ", t2, s2)):
r = totals[name]
r[0] += 0 if t == best else 1
r[1] += s.steps
dev = 100.0 * (t - best) / best
r[2], r[3] = r[2] + dev, max(r[3], dev)
print("seed", seed, "| oracle steps", ha)
for name, r in totals.items():
print(f" {name} split {r[0]:2d}/20 | pattern steps {r[1]:4d}"
f" | avg deviation percent {round(r[2] / 20, 2)}"
f" | worst percent {round(r[3], 2)}")
```
```
seed 20260218 | oracle steps 352800
nearest neighbor split 17/20 | pattern steps 560 | avg deviation percent 9.3 | worst percent 33.33
two-opt split 2/20 | pattern steps 819 | avg deviation percent 0.49 | worst percent 7.59
seed 20260219 | oracle steps 352800
nearest neighbor split 18/20 | pattern steps 560 | avg deviation percent 10.65 | worst percent 26.67
two-opt split 6/20 | pattern steps 924 | avg deviation percent 1.13 | worst percent 6.59
```
#### Reading the Deviation
Three numbers side by side: **the oracle at 352,800 steps**, **nearest neighbor
at 560 steps**, **17 split samples**. The pattern spends 630 times fewer steps
and fails to find the best tour on 17 of 20 samples. But the split-sample count
alone is not enough here, because the question is not "is it correct" but **"how
far off"**. Average deviation is 9.3 percent and **worst-case deviation is 33.33
percent**. Writing the average alone would hide the existence of a tour a third
longer than optimal.
Two-opt takes the same tour and applies local fixes. The split count drops from
17 to **2**, average deviation from 9.3 percent to **0.49 percent**, worst-case
deviation from 33.33 percent to **7.59 percent**. The cost of this is 819 steps
instead of 560, that is, 46 percent more work. The procedure is still not the
oracle: it fails to find the best tour on two samples, and it **cannot know**
that it has failed.
The second pool is mandatory. With seed `20260219`, nearest neighbor splits on
18 samples, average deviation 10.65 percent; two-opt splits on 6 samples,
average deviation 1.13 percent. Nearest neighbor's result is the same on both
pools: it misses the best tour on almost every sample. Two-opt's split count
rises from 2 to 6, and this difference is above the resolution; that is, **how
many samples two-opt finds the best tour on depends on the pool** and cannot be
generalized from a single one. The order of magnitude of the average deviation,
however, does not change: it stays around one percent.
#### Where the Oracle Runs Out
The oracle spending 352,800 steps on eight cities is not the problem. The
problem is how this number grows with city count. The tour count is half of
every ordering of the remaining cities once city zero is fixed. This number is
shown not by running it but by **counting**.
```python
from math import factorial
def group(x):
return f"{x:,}".replace(",", ".")
print(" n | tours | oracle steps | nearest neighbor steps | ratio")
for n in (8, 10, 12, 14, 16, 18):
tours = factorial(n - 1) // 2
oracle = tours * (n - 1)
pattern = n * (n - 1) // 2
print(f"{n:2d} | {group(tours):>22s} | {group(oracle):>25s} |"
f" {pattern:4d} | {group(oracle // pattern)}")
```
```
n | tours | oracle steps | nearest neighbor steps | ratio
8 | 2.520 | 17.640 | 28 | 630
10 | 181.440 | 1.632.960 | 45 | 36.288
12 | 19.958.400 | 219.542.400 | 66 | 3.326.400
14 | 3.113.510.400 | 40.475.635.200 | 91 | 444.787.200
16 | 653.837.184.000 | 9.807.557.760.000 | 120 | 81.729.648.000
18 | 177.843.714.048.000 | 3.023.343.138.816.000 | 153 | 19.760.412.672.000
```
Going from eight cities to eighteen, the oracle's steps rise from 17,640 to
3,023,343,138,816,000; nearest neighbor's steps rise from 28 to 153. The ratio
climbs from 630 to over nineteen trillion. This table is not a speed claim; it
is a **count of where the oracle becomes impossible to run**.
The conclusion that follows is the harshest form of the course's third claim.
Brute force is an oracle, but **it cannot be an oracle at every scale**. The 9.3
percent deviation measured at eight cities cannot be measured at eighteen,
because the best tour to compare against is unknown. Every sentence written
about an approximate solution's quality on large inputs either generalizes from
a deviation **measured on small input** or uses a **lower bound**. Neither is
proof, and this lesson does not present either as proof.
#### What Deviation Does With City Count
The previous section's last sentence leaves a question: what does a deviation
measured on small input say about large input. This question cannot be fully
answered, but which direction the deviation moves **in the range where the
oracle can still be run** can be measured. City count is swept from five to
nine.
```python
from itertools import permutations
from math import isqrt
SEED = 20260218
def generator(seed):
d = seed
def next_val(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_val
def cities(seed=SEED, samples=20, n=8):
r = generator(seed)
return [[(r(30), r(30)) for _ in range(n)] for _ in range(samples)]
def distance(p, q):
return isqrt((p[0] - q[0]) ** 2 + (p[1] - q[1]) ** 2)
def oracle_exhaustive(k):
best = None
for p in permutations(range(1, len(k))):
if p[0] > p[-1]:
continue
tour, previous = 0, 0
for i in p:
tour += distance(k[previous], k[i])
previous = i
tour += distance(k[previous], k[0])
if best is None or tour < best:
best = tour
return best
def nearest_neighbor(k):
remaining, current, tour = set(range(1, len(k))), 0, 0
while remaining:
best_i = min(remaining, key=lambda i: distance(k[current], k[i]))
tour += distance(k[current], k[best_i])
remaining.remove(best_i)
current = best_i
return tour + distance(k[current], k[0])
print(" n | split / 20 | avg deviation | worst deviation")
for n in (5, 6, 7, 8, 9):
split, total, worst = 0, 0.0, 0.0
for k in cities(n=n):
e, t = oracle_exhaustive(k), nearest_neighbor(k)
if t != e:
split += 1
dev = 100.0 * (t - e) / e
total, worst = total + dev, max(worst, dev)
print(f"{n:2d} | {split:11d} | percent {round(total / 20, 2):5} "
f" | percent {round(worst, 2)}")
```
```
n | split / 20 | avg deviation | worst deviation
5 | 12 | percent 5.03 | percent 16.67
6 | 14 | percent 6.29 | percent 17.57
7 | 15 | percent 7.17 | percent 17.86
8 | 17 | percent 9.3 | percent 33.33
9 | 14 | percent 9.77 | percent 22.99
```
Average deviation is 5.03 percent at five cities, 9.77 percent at nine. It rises
in all five rows and roughly doubles. The split-sample count, however, is not
regular: 12, 14, 15, 17, then 14. This irregularity is expected, because each
row is a **separate pool** generated for its own city count; the nine-city
samples are not extensions of the eight-city samples.
The only thing that can be drawn from the table is the **direction** itself:
deviation rises with city count. This suggests that the 9.3 percent measured at
eight cities is an **underestimate** for large inputs. It suggests — it does
not prove. Saying something about eighteen cities from five data points spanning
five to nine means stepping outside a measured range, and this lesson does not
take that step.
#### Where the Question of Why It Is Hard Belongs
Everything measured so far is concrete: steps, deviation, split samples. One
thing remains unmeasured. Is there a procedure **fundamentally** cheaper than
exhaustive enumeration, or is there a difficulty inherent to this problem's
nature.
This question is not answered in this course, and no attempt is made to answer
it. Answering it requires **classifying** problems by degree of difficulty,
transforming one problem into another, and establishing proven bounds on how far
an approximate solution can deviate from the best. All of these tools belong to
the Theory of Computation course. All that is done here is to **separate the
question from its measurable part**: deviation is measured, the name of the
difficulty is not.
This distinction has a practical consequence. Knowing that a problem is hard
does not substitute for knowing an approximate solution's deviation. Saying
"it is already a hard problem" without measuring the deviation leaves the pattern
without an oracle.
#### Summary
- In the travelling salesman problem, the candidate-solution count is not
subsets but orderings; 2520 tours at eight cities, 177,843,714,048,000 at
eighteen.
- The oracle spends 352,800 steps on 20 samples; nearest neighbor finishes in
560 steps but misses the best tour on **17 of 20 samples**.
- Deviation averages 9.3 percent, worst case 33.33 percent; writing the average
alone hides the worst case.
- Two-opt brings the split count down to 2 and average deviation down to 0.49
percent, at a cost of 819 steps instead of 560; since the split count rises to
6 on the second pool, this number depends on the pool.
- The oracle cannot be run at every scale; there is no deviation measured on
large input, only a generalization made from small input.
- The question of **why** the problem is hard belongs to the Theory of
Computation course and is not answered here.
#### Next Step
In the travelling salesman problem, the oracle was exhaustive enumeration and
the pattern was an approximate tour. The next lesson takes up a more unsettling
situation: one where the pattern is a **proven** procedure, but the problem it
is applied to changes. Shortest-path algorithms were proven in the Algorithms
course; when the same procedure is turned toward the longest path, on how many
graphs does it give a wrong answer, and which assumption falling away is the
source of that wrongness.
### The Longest Path Problem
Source: https://academia.sh/en/courses/advanced-algorithms/longest-path-problem
Turning the shortest-path relaxation toward the longest path and measuring it on two graph families: the same answer as the oracle on 40 of 40 inputs on acyclic graphs, splitting on 38 of 40 on cyclic graphs with a worst overshoot of 228 weight units; 39 on the second pool. The reason for the split is not that the procedure slows down but that what it is looking for changes: the relaxation finds the heaviest walk, while the problem asks for the heaviest simple path. The oracle is cheaper than the pattern at eight nodes; at thirteen nodes it becomes 86.37 times more expensive.
In the travelling salesman problem, the pattern was an approximate solution, and
being wrong was expected of it. This lesson looks at a more unsettling
situation: the pattern is a **proven** procedure, its code is correct, it
contains no bug, and it still gives a wrong answer. The only thing that changes
is the problem it is applied to.
The shortest-path problem was built in the Algorithms course's Graph Algorithms
topic. The Dijkstra's Algorithm and Bellman–Ford Algorithm lessons measured the
relaxation operation, the algorithm's invariant, and the negative-weight
constraint; **this lesson does not rebuild those procedures**. Here, a single
change is made: the comparison in the relaxation is turned from **less than** to
**greater than**, and the same procedure is set to search for the longest path.
What is asked is how many graphs this one-character change produces a wrong
answer on.
- **CP20.** Graphs come from the shared definition's generator: 8 nodes, each
directed edge present with a 45 percent share, weight between 1 and 9. Seed
**20260218**, second pool **20260219**.
- **CP21.** Two graph families are used. In the **acyclic** family, an edge only
goes from a lower-numbered node to a higher one, so no cycle can form. In the
**cyclic** family, this restriction does not exist.
- **CP22.** The question asked is the same in both families: the **heaviest
simple path** from node zero to node seven. A simple path is a path that
repeats no node.
- **CP23.** The oracle scans every simple path with depth-first search and
takes the heaviest. This is the direct counterpart of the problem's
definition.
- **CP24.** The pattern is the shortest-path relaxation turned toward the
longest: every edge is passed over node-count-minus-one times, and at each
pass the larger value is accepted.
- **CP25.** The measure is **steps**. In the oracle a step is a node visit; in
the pattern, an edge relaxation. The two numbers are not in the same unit, and
are read that way.
- **CP26.** The split-input count is out of 40; a difference below 3 counts as
unmeasured.
- **CP27.** If there is no path to the target node at all, both procedures
return no answer; this does not count as a split, since both give the same
answer.
- **CP28.** Node count is also swept. That the oracle is expensive is not an
assumption but a measured threshold.
- **CP29.** The six-node graph that shows the loss of optimal substructure is
**built by hand**; it does not come from the generator. It is chosen to show a
single mechanism in its smallest form.
#### The Same Procedure, Two Graph Families
The block below generates both families, runs the oracle and the pattern side
by side on every graph, and prints the split count separately for both pools.
```python
SEED = 20260218
N = 8
def generator(seed):
d = seed
def next_val(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_val
def graphs(seed=SEED, samples=40, cyclic=False, n=N):
"""Directed graph: in the acyclic version, edges only go from a lower number to a higher one."""
r = generator(seed)
pool = []
for _ in range(samples):
edges = []
for i in range(n):
for j in range(n):
if i == j or (not cyclic and j < i):
continue
if r(100) < 45:
edges.append((i, j, r(9) + 1))
pool.append(edges)
return pool
class Counter:
def __init__(self):
self.steps = 0
def count(self):
self.steps += 1
def oracle_longest(edges, c, n=N):
"""Every simple path is scanned: from 0 to n-1, with no node repeated."""
neighbors = {}
for i, j, w in edges:
neighbors.setdefault(i, []).append((j, w))
best = None
def walk(v, seen, weight):
nonlocal best
c.count()
if v == n - 1:
if best is None or weight > best:
best = weight
return
for j, w in neighbors.get(v, []):
if j not in seen:
walk(j, seen | {j}, weight + w)
walk(0, {0}, 0)
return best
def pattern_relaxation(edges, c, n=N):
"""The shortest-path relaxation turned toward the longest: greater instead of smaller."""
best = [None] * n
best[0] = 0
for _ in range(n - 1):
for i, j, w in edges:
c.count()
if best[i] is not None and (best[j] is None or best[i] + w > best[j]):
best[j] = best[i] + w
return best[n - 1]
for name, cyclic in (("acyclic", False), ("cyclic ", True)):
for seed in (20260218, 20260219):
split, pk, ok, overshoot = [], 0, 0, 0
for no, edges in enumerate(graphs(seed, cyclic=cyclic), 1):
s1, s2 = Counter(), Counter()
a = pattern_relaxation(edges, s1)
b = oracle_longest(edges, s2)
pk, ok = pk + s1.steps, ok + s2.steps
if a != b:
split.append(no)
if a is not None and b is not None:
overshoot = max(overshoot, a - b)
print(name, seed, "| split", len(split), "/40", split[:6],
"| pattern steps", pk, "| oracle steps", ok, "| worst overshoot", overshoot)
```
```
acyclic 20260218 | split 0 /40 [] | pattern steps 3626 | oracle steps 562 | worst overshoot 0
acyclic 20260219 | split 0 /40 [] | pattern steps 3696 | oracle steps 536 | worst overshoot 0
cyclic 20260218 | split 38 /40 [1, 2, 3, 4, 5, 6] | pattern steps 7210 | oracle steps 3235 | worst overshoot 228
cyclic 20260219 | split 39 /40 [1, 2, 3, 4, 5, 6] | pattern steps 7203 | oracle steps 3272 | worst overshoot 226
```
#### Correct on Acyclic Graphs, Not on Cyclic Ones
On the acyclic family, the split count is **0**. The same procedure, the same
code, matches the oracle on 40 of 40 graphs, and stays at 0 on the second pool
too. On the cyclic family, the split count is **38**, and **39** on the second
pool. The procedure did not break; the graph changed.
The **direction** of the split is measured too: the pattern's value is never
smaller than the oracle's, and the largest difference is **228 weight units**.
The pattern overestimates — it reports the weight of a path that does not
exist. This is direct proof that the two procedures are searching for
**different objects**.
What the relaxation procedure computes is the heaviest **walk**: going from one
node to another by following edges, allowing nodes to repeat. What is asked for
is the heaviest **simple path**: repeating a node is forbidden. On an acyclic
graph, these two objects coincide, because without a cycle no walk can visit a
node twice. Once a cycle exists, the coincidence ends, and the relaxation
starts circling the cycle, accumulating weight.
The only reason this accumulation does not run forever is that the number of
passes is limited. But that limit is not as narrow as it seems: because edges
are processed in a fixed order within a single pass, the value one edge
produces can chain into later edges in the same pass. The resulting walk can
therefore be far longer than the pass count, and this is why the overestimate
reaches as high as 228. If the order edges are processed in changes, the wrong
value returned changes too — **the wrong answer is not even stable**.
#### The Loss of Optimal Substructure
The proof of shortest path rests on a single property: every segment of a
shortest path is itself a shortest path. Thanks to this property, a node's best
value can be built from its neighbors' best values, and relaxation works.
For the longest simple path, this property fails. The longest simple path from
node zero to node five, and the longest simple path from node five to node
seven, can each be found separately; but joining the two end to end **may not
give a simple path**, because the two segments may pass through a shared node.
If the join is forbidden, the subproblems' solutions do not carry over into the
main problem's solution, and the ground the relaxation stands on is gone.
This loss can be shown clearly on a single six-node graph. The block below
prints three numbers side by side: the sum of the two subproblems' separately
best solutions, the actual best solution, and the value the relaxation gives.
```python
EDGES = [(0, 1, 5), (1, 2, 5), (2, 3, 5), (0, 3, 1),
(3, 2, 5), (2, 4, 5), (4, 5, 5), (3, 5, 1)]
N = 6
def oracle_path(edges, start, end):
"""Every simple path is scanned; the heaviest is returned along with its path."""
neighbors = {}
for i, j, w in edges:
neighbors.setdefault(i, []).append((j, w))
best = (None, None)
def walk(v, path, weight):
nonlocal best
if v == end:
if best[0] is None or weight > best[0]:
best = (weight, tuple(path))
return
for j, w in neighbors.get(v, []):
if j not in path:
walk(j, path + [j], weight + w)
walk(start, [start], 0)
return best
def pattern_relaxation(edges, end, n=N):
best = [None] * n
best[0] = 0
for _ in range(n - 1):
for i, j, w in edges:
if best[i] is not None and (best[j] is None or best[i] + w > best[j]):
best[j] = best[i] + w
return best[end]
a, b, c = oracle_path(EDGES, 0, 3), oracle_path(EDGES, 3, 5), oracle_path(EDGES, 0, 5)
print("longest simple path from 0 to 3:", a)
print("longest simple path from 3 to 5:", b)
print("sum of the two segments :", a[0] + b[0],
"| shared node:", sorted(set(a[1]) & set(b[1])))
print("longest simple path from 0 to 5:", c)
print("what the relaxation gives :", pattern_relaxation(EDGES, 5))
```
```
longest simple path from 0 to 3: (15, (0, 1, 2, 3))
longest simple path from 3 to 5: (15, (3, 2, 4, 5))
sum of the two segments : 30 | shared node: [2, 3]
longest simple path from 0 to 5: (20, (0, 1, 2, 4, 5))
what the relaxation gives : 70
```
The three numbers must be read separately. The subproblems' best solutions are
15 and 15; their sum is **30**. This 30 is not the weight of any path, because
the two segments share nodes 2 and 3. The actual answer is **20**, and the path
it corresponds to contains neither of the two sub-solutions. The value the
relaxation gives is **70**: the weight of a walk accumulated by circling the
cycle multiple times.
The gap between thirty and seventy matters. When optimal substructure fails, it
is not just that "the sub-solutions cannot be joined"; the relaxation goes
further, past the unjoinable segments, and starts measuring an entirely
different object. Someone looking only at the output cannot tell this: the
value returned is an integer, unmarked, carrying no warning. The course's rule
holds here too — **wrongness cannot be picked out by looking at the output**,
only seen with the oracle.
Reading the same observation the other way around gives a useful rule: when
carrying a procedure over to a new problem, the question to ask is not "does
the code run" but **"does the property the proof rests on still hold"**. In
this lesson, the acyclicity constraint is exactly the condition that keeps that
property standing.
#### When the Oracle Is Expensive
There is a striking anomaly in the output above: at eight nodes the oracle
spends **562 steps**, the pattern **3626 steps**. The oracle is cheaper than
the pattern. The shared definition's reading that "the oracle is always
expensive" does not match the number measured here, and the number is not
hidden. Being expensive is not a definition but a **function of scale**.
Sweeping node count makes the threshold visible.
```python
SEED = 20260218
def generator(seed):
d = seed
def next_val(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_val
def graphs(n, seed=SEED, samples=10):
r = generator(seed)
return [[(i, j, r(9) + 1) for i in range(n) for j in range(n)
if i != j and r(100) < 45] for _ in range(samples)]
class Counter:
def __init__(self):
self.steps = 0
def count(self):
self.steps += 1
def oracle_longest(edges, n, c):
neighbors = {}
for i, j, w in edges:
neighbors.setdefault(i, []).append((j, w))
best = None
def walk(v, seen, weight):
nonlocal best
c.count()
if v == n - 1:
best = weight if best is None or weight > best else best
return
for j, w in neighbors.get(v, []):
if j not in seen:
walk(j, seen | {j}, weight + w)
walk(0, {0}, 0)
return best
def pattern_relaxation(edges, n, c):
best = [None] * n
best[0] = 0
for _ in range(n - 1):
for i, j, w in edges:
c.count()
if best[i] is not None and (best[j] is None or best[i] + w > best[j]):
best[j] = best[i] + w
return best[n - 1]
print(" n | oracle steps | pattern steps | ratio")
for n in (6, 8, 10, 12, 13):
ah = ak = 0
for edges in graphs(n):
s1, s2 = Counter(), Counter()
pattern_relaxation(edges, n, s1)
oracle_longest(edges, n, s2)
ak, ah = ak + s1.steps, ah + s2.steps
print(f"{n:2d} | {ah:10d} | {ak:10d} | {round(ah / ak, 2)}")
```
```
n | oracle steps | pattern steps | ratio
6 | 102 | 655 | 0.16
8 | 746 | 1785 | 0.42
10 | 10210 | 3762 | 2.71
12 | 196705 | 6677 | 29.46
13 | 747243 | 8652 | 86.37
```
The threshold is at ten nodes. At six nodes, the oracle spends about a sixth of
the pattern's steps; at thirteen nodes, it spends **86.37 times** more. The
pattern's steps rise from 655 to 8652, a factor of thirteen; the oracle's steps
rise from 102 to 747,243, a factor of roughly seven thousand. This gap does not
change the fact that the pattern **is still wrong** on cyclic graphs: what
becomes cheap is not the correct answer but the wrong one.
This table should be read together with the previous lesson. In the travelling
salesman problem, the oracle could be run on small input and could not be run
on large input; here, the oracle is **cheaper than the pattern** on small
input. In both cases, the oracle's cost has been measured, not assumed.
#### Summary
- The longest simple path is not the shortest path reversed; the relaxation
procedure finds the heaviest **walk**, while the problem asks for the
heaviest **simple path**.
- On an acyclic graph the two objects coincide, and the pattern matches the
oracle on 40 of 40 inputs; the split count is also 0 on the second pool.
- On a cyclic graph, the pattern splits on **38 of 40 inputs**, 39 on the
second pool, with the worst overestimate at **228 weight units**.
- The source of the split is the loss of optimal substructure: joining two
sub-paths may not give a simple path, since they can share a node.
- The oracle's expense comes with scale; at eight nodes the oracle is cheaper
than the pattern, at thirteen nodes it is 86.37 times more expensive. A
procedure cannot be called expensive without measuring it.
#### Next Step
In this lesson, the pattern gave a wrong answer, the oracle gave the correct
one, and the oracle's cost was measured. The next lesson takes up speeding up
the oracle **itself**. In the n-queens problem, backtracking cuts conflicting
branches instead of scanning every placement; the shared definition already
gave the numbers for this. The new question to ask is: how much does
eliminating the board's symmetries shrink the search space, and does that
shrinkage change the exponential growth.
### The N-Queens Problem
Source: https://academia.sh/en/courses/advanced-algorithms/n-queens-problem
Taking the shared definition's backtracking counts as input and measuring symmetry breaking on top of them: restricting the first queen to half the board brings the node count at eight queens from 2057 down to 1029, a ratio of 2.00; 1.69 at five queens, 1.78 at nine. The mirrored solutions restore the oracle's solution set exactly, on all five board sizes. The fundamental-solution count at eight queens is 12 instead of 92, but the smallest equivalence class is 4, not 8. Pruning's gain grows with n; symmetry breaking's gain stays fixed at 2 and buys nothing when finding the first solution.
In the previous lesson, the pattern gave a wrong answer and the oracle gave the
correct one. This lesson looks at the oracle itself. In the n-queens problem,
the oracle is brute force trying every placement of queens on the board;
backtracking does the same search by cutting conflicting branches. The shared
definition already gave the numbers for both, and **this lesson does not
remeasure them — it takes them as input**.
The problem is this: n queens are to be placed on an n-by-n board so that no
two share a row, a column, or a diagonal. The new question is: can the board's
own symmetries be used to shrink the search space, and if so, is that gain
**the same kind** as what pruning contributes.
- **CP30.** The shared definition's backtracking counts are **input**: at n=5,
54 pruned nodes and 3906 unpruned; at n=6, 153 and 55,987; at n=7, 552 and
960,800; at n=8, 2057 pruned nodes and 16,777,216 unpruned placements. These
numbers are not remeasured.
- **CP31.** The pruned traversal is exactly the shared definition's procedure:
a queen is placed on every row, and a branch is cut on the spot if there is a
conflict. Symmetry breaking is added **on top of** this.
- **CP32.** Symmetry breaking is built with a single constraint: the queen in
the **first row can only be placed in the first half of the board**. With an
odd number of columns, the middle column belongs to that half.
- **CP33.** The oracle is the **solution set** produced by the unrestricted
pruned traversal. Solutions coming from the half board are mirrored and
compared against this set; set equality is tested exactly.
- **CP34.** The board's symmetry group has eight elements: four quarter-turns
and the mirror of each. A solution's equivalence class consists of the
distinct solutions these eight transformations produce.
- **CP35.** The measure is **steps**, and a step is a node visited in the
search tree.
- **CP36.** Board size is kept between 5 and 9; large n is conveyed **by
counting**, not by running it. **This lesson's input does not come from the
generator:** the only input is board size, and every number is exhaustive
enumeration. In this lesson, the second-pool rule is replaced by a
**board-size sweep**; every row is an independent run, and no value depends
on a seed.
- **CP37.** Finding all solutions and finding **one** solution are measured
separately. They are two different questions about the same problem, and
their costs are not comparable.
- **CP38.** Verifying a candidate is counted separately: every pair of queens
is compared.
- **CP39.** The second constraint, forward checking, is independent of
symmetry breaking. What the two contribute together is compared against the
product of what they contribute separately; equality is not assumed, it is
measured.
#### Measuring Symmetry Breaking
Once a solution is found, its mirror image is also a solution. The same holds
for quarter-turns. This means part of the search is **redundant**: every
solution found by placing the first queen in the right half is the mirror of a
solution found by placing it in the left half.
The block below adds the constraint, places the two traversals' node counts
side by side, compares the set restored by mirroring against the oracle's set,
and counts the equivalence classes.
```python
class Counter:
def __init__(self):
self.steps = 0
def count(self):
self.steps += 1
def queens(n, half_first=False):
"""The shared definition's pruned backtracking; half_first adds symmetry breaking."""
s = Counter()
solutions = []
def walk(row, placed):
s.count()
if row == n:
solutions.append(tuple(placed))
return
top = (n + 1) // 2 if (half_first and row == 0) else n
for col in range(top):
if any(col == y or abs(col - y) == row - i
for i, y in enumerate(placed)):
continue
placed.append(col)
walk(row + 1, placed)
placed.pop()
walk(0, [])
return s.steps, solutions
def mirror(c, n):
return tuple(n - 1 - x for x in c)
def rotate(c, n):
new = [0] * n
for i, x in enumerate(c):
new[x] = n - 1 - i
return tuple(new)
def orbit(c, n):
"""All images of a solution under the eight transformations."""
group, d = set(), c
for _ in range(4):
group.add(d)
group.add(mirror(d, n))
d = rotate(d, n)
return group
print(" n | full nodes | full solutions | half nodes | half solutions | node ratio"
" | mirrored | fundamental solutions | smallest class")
for n in (5, 6, 7, 8, 9):
d1, c1 = queens(n)
d2, c2 = queens(n, half_first=True)
restored = set(c2) | {mirror(c, n) for c in c2}
full = set(c1)
fundamental, seen, smallest = 0, set(), 99
for c in sorted(full):
if c in seen:
continue
k = orbit(c, n)
seen |= k
fundamental += 1
smallest = min(smallest, len(k))
print(f"{n:2d} | {d1:9d} | {len(c1):9d} | {d2:11d} | {len(c2):11d} |"
f" {round(d1 / d2, 2):11} | {'same' if restored == full else 'DIFFERENT':12s}"
f" | {fundamental:11d} | {smallest:14d}")
```
```
n | full nodes | full solutions | half nodes | half solutions | node ratio | mirrored | fundamental solutions | smallest class
5 | 54 | 10 | 32 | 6 | 1.69 | same | 2 | 2
6 | 153 | 4 | 77 | 2 | 1.99 | same | 1 | 4
7 | 552 | 40 | 316 | 23 | 1.75 | same | 6 | 4
8 | 2057 | 92 | 1029 | 46 | 2.0 | same | 12 | 4
9 | 8394 | 352 | 4704 | 203 | 1.78 | same | 46 | 4
```
#### The Size and Kind of the Gain
The first column confirms the shared definition's numbers: 54, 153, 552, 2057.
Once the symmetry constraint is added, the node count at eight queens drops
from 2057 to **1029**, a ratio of **2.00**. At five queens, 1.69; at six,
1.99; at seven, 1.75; at nine, 1.78. In none of the five rows does the ratio
exceed 2.
The mirroring column is the oracle comparison. Adding the mirror images to the
solutions coming from the half board gives a set that is **the same on all
five board sizes** as the unrestricted search's solution set. Symmetry
breaking is not an approximate solution: no solution is lost, only the way it
is found changes. This is a rare case in this course's measurements — the
pattern splits from the oracle on no input at all.
This gain should now be placed alongside the shared definition's pruning gain.
| n | unpruned (shared definition) | pruned | pruning ratio | half board | symmetry ratio |
|---|---|---|---|---|---|
| 5 | 3,906 | 54 | 72.3 | 32 | 1.69 |
| 6 | 55,987 | 153 | 365.9 | 77 | 1.99 |
| 7 | 960,800 | 552 | 1740.6 | 316 | 1.75 |
| 8 | 16,777,216 placements | 2057 | 8156.2 | 1029 | 2.00 |
The two columns are in the same unit but are **not the same kind**. The
pruning ratio rises from 72.3 to 8156.2, meaning pruning's contribution grows
as n grows. The symmetry ratio stays between 1.69 and 2.00 and does not grow;
it cannot grow, because what is eliminated is **a fixed number of symmetries
of the board**, and that number does not change with n.
This distinction is part of the course's reading of measurement. A single
number for an improvement is not enough; **what that number does as n grows**
must be asked. A constraint that saves a thousand nodes at eight queens still
saves only half at sixteen, and that half is still exponential. The shared
definition's fourth reading said this about pruning: pruning's gain grows with
scale, but **the result is still exponential**. In symmetry breaking, the gain
does not even grow with scale.
The fundamental-solution column gives a third number. At eight queens there
are 92 solutions, but the number of equivalence classes is **12**. This is not
92 divided by 8: 92 divided by 8 is 11.5. The difference shows up in the
smallest-class column. Some solutions map to **themselves** under a
transformation, so their classes have four or two elements instead of eight.
At five queens, the smallest class has 2 elements. Saying "divide by eight"
when eliminating by symmetry is therefore wrong.
#### Do Two Improvements Multiply
Symmetry breaking restricts the **start** of the search. A second, independent
constraint can be added on top: once a queen is placed, if one of the
remaining rows has no open column left at all, that branch is cut
immediately. This is called forward checking, and it does **not wait for the
conflict to appear** — it looks ahead and cuts. The two improvements are
independent of each other; what is to be asked is whether their gains
multiply when used together.
```python
class Counter:
def __init__(self):
self.steps = 0
def count(self):
self.steps += 1
def conflicts(col, row, placed):
return any(col == y or abs(col - y) == row - i
for i, y in enumerate(placed))
def queens(n, half_first=False, look_ahead=False):
"""Two independent improvements: symmetry breaking and forward checking."""
s = Counter()
count = 0
def walk(row, placed):
nonlocal count
s.count()
if row == n:
count += 1
return
top = (n + 1) // 2 if (half_first and row == 0) else n
for col in range(top):
if conflicts(col, row, placed):
continue
placed.append(col)
if not look_ahead or all(any(not conflicts(c, r, placed) for c in range(n))
for r in range(row + 1, n)):
walk(row + 1, placed)
placed.pop()
walk(0, [])
return s.steps, count
print(" n | plain | symmetry | forward checking | both"
" | product of separate | measured ratio")
for n in (6, 7, 8, 9, 10):
d0, _ = queens(n)
d1, _ = queens(n, half_first=True)
d2, _ = queens(n, look_ahead=True)
d3, _ = queens(n, half_first=True, look_ahead=True)
product = (d0 / d1) * (d0 / d2)
print(f"{n:2d} | {d0:5d} | {d1:7d} | {d2:11d} | {d3:12d} |"
f" {round(product, 2):16} | {round(d0 / d3, 2)}")
```
```
n | plain | symmetry | forward checking | both | product of separate | measured ratio
6 | 153 | 77 | 87 | 44 | 3.49 | 3.48
7 | 552 | 316 | 334 | 187 | 2.89 | 2.95
8 | 2057 | 1029 | 1165 | 583 | 3.53 | 3.53
9 | 8394 | 4704 | 4720 | 2658 | 3.17 | 3.16
10 | 35539 | 17770 | 17489 | 8745 | 4.06 | 4.06
```
The last two columns agree on all five rows: 3.49 against 3.48, 2.89 against
2.95, 3.53 against 3.53, 3.17 against 3.16, 4.06 against 4.06. The product of
the separately measured gains comes out equal to the gain measured jointly.
Because the two constraints touch **different parts** of the search, they do
not repeat each other's work.
This agreement is not a rule, it is a measured result; if the two constraints
cut the same branches, the product would come out larger than the measured
ratio, and the gap would be a **double-counted gain**. The measurement exists
precisely to test this.
Still, the combined ratio stays at 4.06. Three improvements were stacked, and
the search space still grows exponentially with n; 8745 nodes at ten queens,
and this number will multiply many times over again at twelve. **Even when
constant factors are combined, the kind of growth does not change.**
#### The Question Where Symmetry Buys Nothing
The question asked so far was "all solutions." The same problem can also be
asked with a different question: finding **one** solution. The two questions
look at the same board, but their costs cannot be compared.
```python
class Counter:
def __init__(self):
self.steps = 0
def count(self):
self.steps += 1
def search(n, half_first=False, stop=False):
"""Pruned backtracking. stop=True halts at the first solution."""
s = Counter()
found = []
def walk(row, placed):
s.count()
if stop and found:
return
if row == n:
found.append(tuple(placed))
return
top = (n + 1) // 2 if (half_first and row == 0) else n
for col in range(top):
if stop and found:
return
if any(col == y or abs(col - y) == row - i
for i, y in enumerate(placed)):
continue
placed.append(col)
walk(row + 1, placed)
placed.pop()
walk(0, [])
return s.steps, found
def verify(c, n):
"""A candidate solution is checked by comparing every pair of queens."""
steps = 0
for i in range(n):
for j in range(i + 1, n):
steps += 1
if c[i] == c[j] or abs(c[i] - c[j]) == j - i:
return False, steps
return True, steps
print(" n | first solution | first solution half | all solutions | ratio | verification")
for n in (5, 6, 7, 8, 9, 10):
d1, b1 = search(n, stop=True)
d2, _ = search(n, half_first=True, stop=True)
d3, everything = search(n)
ok, steps = verify(b1[0], n)
print(f"{n:2d} | {d1:9d} | {d2:15d} | {d3:14d} | {d3 // d1:4d} |"
f" {steps:3d} steps, correct {ok}")
```
```
n | first solution | first solution half | all solutions | ratio | verification
5 | 6 | 6 | 54 | 9 | 10 steps, correct True
6 | 32 | 32 | 153 | 4 | 15 steps, correct True
7 | 10 | 10 | 552 | 55 | 21 steps, correct True
8 | 114 | 114 | 2057 | 18 | 28 steps, correct True
9 | 42 | 42 | 8394 | 199 | 36 steps, correct True
10 | 103 | 103 | 35539 | 345 | 45 steps, correct True
```
The second and third columns are **the same on all six of the six rows**.
Symmetry breaking buys nothing when finding the first solution, because in the
first solution the search finds, the first queen already sits in column zero,
and that column is inside the constraint. The constraint closes off a region
that was never going to be entered.
This is the measured demonstration that improvements are **question-dependent**.
The same constraint halves the node count exactly when all solutions are
asked for, and buys zero when one solution is asked for. An improvement's
value comes not from the procedure but from **the question asked**.
The last two columns show a sharper difference. At ten queens, finding all
solutions is 35,539 nodes, finding one solution is **103 nodes**, a ratio of
345. **Verifying** that a found candidate is correct, however, is only 45
steps: comparing every pair of queens. Searching, finding, and verifying are
three separate costs, and the gap between them widens with board size.
#### Summary
- Symmetry breaking brings the node count at eight queens from 2057 down to
1029; the ratio stays between 1.69 and 2.00 across five board sizes.
- The solution set restored by mirroring matches the oracle's set on all five
sizes; no solution is lost.
- Pruning's ratio rises from 72.3 to 8156.2, symmetry's ratio stays at 2: one
is a gain that grows with n, the other a **constant-factor** gain.
- The fundamental-solution count at eight queens is 12; dividing 92 by eight
is wrong, because some solutions map to their own images and the smallest
equivalence class has 4 elements.
- Symmetry breaking buys nothing when a single solution is sought; the same
constraint buys half in one question and zero in the other.
#### Next Step
In n-queens, the search space could be read directly off the board: one queen
per row, n choices per queen. The next lesson moves to problems where the
search space is **not given**. The knight's tour and the maze are defined on
the same grid, but how many objects are scanned changes by a factor of
thousands depending on how they are modeled. The question to ask will be: is
the search space a property of the problem, or of the model.
### The Knight's Tour and Maze Problems
Source: https://academia.sh/en/courses/advanced-algorithms/knights-tour-and-maze-problems
Representing the same grid with two different models and watching the search space change with the model: on a five-by-five grid, when the candidate solution is counted as a path, 8512 simple paths and 90,111 steps; when counted as a cell, 25 steps — a ratio of 3604, and both models give the same shortest length. In the knight's tour, trying the least-option square first finds the tour in 25 nodes on all four starting squares of a five-by-five board; natural order visits between 182 and 101,718 nodes. On a four-by-four board there is no tour at all, and both orderings visit exactly 29,976 nodes: the move-ordering heuristic buys nothing on a no answer.
In n-queens, the search space could be read directly off the board: one queen
per row, n choices per queen. The size of the space followed from the
problem's definition, with nothing to argue about. This lesson removes that
comfort. The knight's tour and the maze are both defined on the same grid, but
what decides what object gets scanned is not the problem but the **model**,
and the gap between two models can reach a factor of thousands.
The maze problem is this: get from a grid's top-left cell to its bottom-right
cell, moving between adjacent cells. The knight's tour problem is this:
starting from a knight on a five-by-five board, visit every square exactly
once. Both are search problems, and in both the first question to ask is not
"which algorithm" but **"what is being searched for."**
- **CP40.** Two models are compared for the maze. In the **first model**, the
candidate solution is a **path**; what is scanned is every simple path from
top-left to bottom-right. In the **second model**, the candidate solution is
a **cell**; every cell is opened at most once.
- **CP41.** The second model is **breadth-first search**, built in the Data
Structures course's Graphs topic. The procedure is **not rebuilt**; it is
used directly.
- **CP42.** The oracle is the first model: it scans every simple path and
finds the shortest. The length the second model gives is compared against
this.
- **CP43.** The grid has no obstacles. Adding obstacles changes the path
count but does **not change the ratio** between the two models; what is
measured is the ratio's order of magnitude. **This lesson's input does not
come from the generator either**: grid size and board size are the only
inputs, and every number is exhaustive enumeration. The second-pool rule is
replaced by a **size sweep**, and no value depends on a seed.
- **CP44.** The measure is **steps**. In the first model, a step is a
partial-path node; in the second, a cell being opened.
- **CP45.** In the knight's tour, two orderings are compared: trying squares
in **natural order**, and trying the square that **leaves the fewest
options** first. The second ordering is not pruning; no branch is cut, only
the order of trials changes.
- **CP46.** The second ordering is a heuristic, and its correctness is **not
assumed**. Every tour it finds is verified against the oracle: was every
square visited once, is every step a legal knight move.
- **CP47.** Four starting squares are measured on the five-by-five board.
Measuring every square would require scanning the entire search space on
squares where no tour exists, and would exceed the run budget.
- **CP48.** The no-tour case is measured on a **four-by-four board**; there,
no square has a tour, and the entire search space can be scanned.
- **CP49.** The limit of the cheap model is tested by changing the question.
The same count yields both the shortest path and the count of paths
visiting every cell; a model's adequacy is never written independently of
the question.
#### The Same Grid, Two Models
The first model sets up the question like this: generate every path from
top-left to bottom-right, pick the shortest. The second sets it up like this:
starting from top-left, open cells in order of distance, and stop at the
first arrival at bottom-right. Both give the correct answer; what is measured
is what it costs.
```python
from collections import deque
class Counter:
def __init__(self):
self.steps = 0
def count(self):
self.steps += 1
def neighbors(h, r, c):
for dr, dc in ((1, 0), (-1, 0), (0, 1), (0, -1)):
i, j = h[0] + dr, h[1] + dc
if 0 <= i < r and 0 <= j < c:
yield (i, j)
def oracle_all_paths(r, c, s):
"""Model 1: the candidate solution is a PATH. Every simple path is scanned."""
target, count, shortest = (r - 1, c - 1), 0, None
def walk(h, seen):
nonlocal count, shortest
s.count()
if h == target:
count += 1
if shortest is None or len(seen) < shortest:
shortest = len(seen)
return
for k in neighbors(h, r, c):
if k not in seen:
walk(k, seen | {k})
walk((0, 0), {(0, 0)})
return count, shortest
def pattern_bfs(r, c, s):
"""Model 2: the candidate solution is a CELL. Every cell is opened at most once."""
target = (r - 1, c - 1)
queue, seen = deque([((0, 0), 1)]), {(0, 0)}
while queue:
h, u = queue.popleft()
s.count()
if h == target:
return u
for k in neighbors(h, r, c):
if k not in seen:
seen.add(k)
queue.append((k, u + 1))
return None
print(" grid | simple path count | oracle steps | bfs steps | ratio | shortest")
for n in (3, 4, 5):
s1, s2 = Counter(), Counter()
count, shortest = oracle_all_paths(n, n, s1)
u = pattern_bfs(n, n, s2)
print(f" {n}x{n:2d} | {count:16d} | {s1.steps:10d} | {s2.steps:10d} |"
f" {s1.steps // s2.steps:5d} | {shortest} and {u}")
```
```
grid | simple path count | oracle steps | bfs steps | ratio | shortest
3x 3 | 12 | 51 | 9 | 5 | 5 and 5
4x 4 | 184 | 1271 | 16 | 79 | 7 and 7
5x 5 | 8512 | 90111 | 25 | 3604 | 9 and 9
```
#### What Determines the Space Is the Model
The last column shows the two models give **the same answer**: shortest
length is the same on all three grids, split count 0. The second model is not
an approximate solution; it is a solution that agrees exactly with the
oracle.
But their costs are not the same. On the five-by-five grid, the first model
spends **90,111 steps** to produce **8512 simple paths**; the second spends
**25 steps**, one per cell. The ratio is **3604**. On the three-by-three grid
the ratio is 5, on four-by-four it is 79, on five-by-five it is 3604. The
ratio widens rapidly as the grid grows.
The source of the difference is not the algorithm but **the object being
counted**. In the first model, the candidate is a path, and the path count
grows exponentially with cell count. In the second model, the candidate is a
cell, and the cell count equals the grid's area. The second model finds the
shortest of the 8512 paths the first model produces **without producing
them**, because it uses the fact that arriving at the same cell by different
paths does not change what comes after.
This gives a sentence that ties back to the course's reading of measurement:
**the size of the search space is a property of the model, not the
problem.** Saying "the search space is too large" for a problem carries
meaning only once it is said which model is meant. The same maze has 8512
candidates in one model and 25 in another.
#### The Question the Cheap Model Cannot Answer
The cheap model has a cost, and it has not shown up yet. Breadth-first search
used the fact that arriving at the same cell by different paths does not
change what comes after. This assumption is true for the shortest-path
question, but it is not true for every question. The block below performs
the same count and sorts the paths **by their length**.
```python
from collections import Counter as Tally
class Counter:
def __init__(self):
self.steps = 0
def count(self):
self.steps += 1
def paths(n, s):
"""Every simple path; tallied by length."""
target, tally = (n - 1, n - 1), Tally()
def walk(h, seen):
s.count()
if h == target:
tally[len(seen)] += 1
return
for dr, dc in ((1, 0), (-1, 0), (0, 1), (0, -1)):
k = (h[0] + dr, h[1] + dc)
if 0 <= k[0] < n and 0 <= k[1] < n and k not in seen:
walk(k, seen | {k})
walk((0, 0), {(0, 0)})
return tally
for n in (4, 5):
s = Counter()
tally = paths(n, s)
full = tally[n * n]
print(f"{n}x{n} | paths {sum(tally.values()):5d} | steps {s.steps:6d}"
f" | shortest {min(tally)} long, {tally[min(tally)]:3d} paths"
f" | visiting every cell {full:3d} paths")
print(" by length:", dict(sorted(tally.items())))
```
```
4x4 | paths 184 | steps 1271 | shortest 7 long, 20 paths | visiting every cell 0 paths
by length: {7: 20, 9: 36, 11: 48, 13: 48, 15: 32}
5x5 | paths 8512 | steps 90111 | shortest 9 long, 70 paths | visiting every cell 104 paths
by length: {9: 70, 11: 224, 13: 510, 15: 956, 17: 1586, 19: 2224, 21: 2106, 23: 732, 25: 104}
```
On the five-by-five grid, **70 of the 8512 paths** are of the shortest
length, and **104 pass through all 25 cells**. Breadth-first search gives the
first number in 25 steps. It can say nothing about the second: since it keeps
only a single value per cell, it forgets which cells were used to arrive at a
cell, and the question "is there a path that visits every cell" asks for
exactly that information. Once the question changes, the 25-step model falls
away, leaving only the 90,111-step count.
The four-by-four row shows the same question's **no** answer: none of the
184 paths pass through all 16 cells, the longest reaching 15. This no answer
also cannot be obtained without spending all 1271 steps. Seeing one question
of the search become cheap while the other does not is proof that the model
is chosen **together with the question**.
#### What Ordering Contributes in the Knight's Tour
In the knight's tour, the second model cannot be built directly. Which path
arrives at a square **matters**, because which squares have been used
determines what comes next. The candidate is not a cell but a **partial
tour**, and the search space is exponential. The only tool left is the
**order in which branches are tried**.
The intuition is this: if the square with the fewest options is not visited
early, it can become unreachable later, and the search wanders in vain. The
block below compares this ordering against natural order and verifies every
tour found against the oracle.
```python
MOVES = ((1, 2), (2, 1), (-1, 2), (-2, 1), (1, -2), (2, -1), (-1, -2), (-2, -1))
class Counter:
def __init__(self):
self.steps = 0
def count(self):
self.steps += 1
def candidates(h, visited, n):
r, c = h
return [(r + a, c + b) for a, b in MOVES
if 0 <= r + a < n and 0 <= c + b < n and (r + a, c + b) not in visited]
def tour(start, n, ordered, s):
"""Halts at the first complete tour. ordered=True: least-option square first."""
def walk(h, visited):
s.count()
if len(visited) == n * n:
return [h]
options = candidates(h, visited, n)
if ordered:
options.sort(key=lambda k: len(candidates(k, visited | {k}, n)))
for k in options:
y = walk(k, visited | {k})
if y:
return [h] + y
return None
return walk(start, {start})
def verify(path, n):
"""Oracle: is every square visited once, is every step a legal knight move."""
if path is None or len(set(path)) != n * n:
return False
return all((abs(a[0] - b[0]), abs(a[1] - b[1])) in ((1, 2), (2, 1))
for a, b in zip(path, path[1:]))
print("5x5 | start | natural order nodes | least-option nodes | verified tour")
for start in ((0, 0), (0, 2), (2, 2), (4, 0)):
s1, s2 = Counter(), Counter()
y1, y2 = tour(start, 5, False, s1), tour(start, 5, True, s2)
print(f" | {str(start):9s} | {s1.steps:16d} | {s2.steps:19d} |"
f" {verify(y1, 5)} and {verify(y2, 5)}")
t1 = t2 = found = 0
for i in range(4):
for j in range(4):
s1, s2 = Counter(), Counter()
y1, y2 = tour((i, j), 4, False, s1), tour((i, j), 4, True, s2)
t1, t2 = t1 + s1.steps, t2 + s2.steps
found += 1 if verify(y1, 4) else 0
print(f"4x4 | 16 squares | {t1:16d} | {t2:19d} | tours found {found}")
```
```
5x5 | start | natural order nodes | least-option nodes | verified tour
| (0, 0) | 70716 | 25 | True and True
| (0, 2) | 101718 | 25 | True and True
| (2, 2) | 25542 | 25 | True and True
| (4, 0) | 182 | 25 | True and True
4x4 | 16 squares | 29976 | 29976 | tours found 0
```
#### Cheap on Yes, Not on No
On the five-by-five board, the least-option order finds the tour in **25
nodes** on all four starting squares. Twenty-five is the number of squares on
the board: the search never backtracks, choosing the right move on the first
try at every square. Natural order finds the same tours between 182 and
**101,718** nodes. Because the starting squares differ, the rows must be
compared within themselves, not against each other; the ratio in each row
ranges from at least 7 to as much as 4068.
The verification column is `True` on every one of these rows. The tours the
heuristic finds are genuinely tours: all twenty-five squares are visited
once, and every step is a legal knight move. It is not enough to say the
heuristic speeds things up — **it also does not go wrong**, and this has been
measured separately.
The last row is the lesson's harshest number. On the four-by-four board, no
square has a tour. Both orderings visit **exactly 29,976 nodes** to find this
out. The numbers are equal, the difference zero. The move-ordering heuristic
buys nothing here, because what it bought was **finding the right branch
early**; when there is no right branch, every branch is tried, and the order
of trying does not change the total.
This is the measured form of the difference between move-ordering heuristics
and pruning. Pruning **removes pieces** from the search space, so it pays off
whether the answer is yes or no. Ordering removes nothing from the space; it
only changes the direction of traversal, so it pays off only on a **yes**
answer. A heuristic's number cannot be read without saying which answer it
was measured on.
#### Summary
- The same maze produces two different search spaces in two models: with the
candidate solution counted as a path, 8512 candidates and 90,111 steps on
the five-by-five grid; counted as a cell, 25 steps; a ratio of 3604.
- The two models give the same shortest length on all three grids; split
count 0, meaning the cheap model is an exact solution, not an
approximation.
- The size of the search space is a property of the model, not the problem;
the sentence "the space is too large" carries no meaning without saying
which model is meant.
- The cheap model's limit is set by the question: 104 of the 8512 paths
visit every cell, but breadth-first search cannot give this number, and
once the question changes, it returns to the 90,111-step count.
- In the knight's tour, trying the square with the fewest options first
finds the tour in 25 nodes on all four starting squares of the five-by-five
board; natural order ranges between 182 and 101,718 nodes.
- On the four-by-four board there is no tour, and both orderings visit
29,976 nodes: the move-ordering heuristic buys nothing on a no answer,
because it removes nothing from the space.
#### Next Step
This lesson's last row opened a distinction: a search's yes answer and its no
answer do not cost the same. The next lesson makes this distinction the
topic's central question. The Hamiltonian path problem is an **existence**
question and is framed as a decision problem; what will be measured is the
step gap between deciding, finding the path, and verifying a given path.
Class names will again be left to Theory of Computation, but the gap will be
counted here.
### Hamiltonian Paths
Source: https://academia.sh/en/courses/advanced-algorithms/hamiltonian-paths
Framing an existence question as a decision problem and counting decision, search, counting, and verification separately: a path exists on 25 of 40 graphs, and deciding spends 1526 steps while counting spends 29,270, a ratio of 19.18. On all 15 graphs where no path exists, the decision step count is exactly equal to the counting step count, both 2538. The second pool gives the same structure, with 23 and 17 graphs. Verifying a found path takes 7 steps per graph, searching for one takes an average of 61. The names of complexity classes are left to the Theory of Computation course.
The last row of the knight's tour opened a distinction: on the four-by-four
board there was no tour, and both orderings visited exactly the same number
of nodes. A yes answer could become cheap; a no answer could not. This lesson
makes that distinction the topic's central question.
A Hamiltonian path is a path that **passes through every node of a graph
exactly once**. The knight's tour is a special case of this: squares are
nodes, knight moves are edges. The question here is no longer "which is best"
but **"does one exist."** This is a **decision problem**, and its answer is a
single word. Two more questions stand alongside it: the **search** problem
asks for a path, the **counting** problem asks how many there are. All three
questions look at the same input, and the cost of all three is measured
separately.
- **CP50.** Graphs come from the shared definition's generator: 8 nodes, each
pair of nodes connected with a 38 percent share, undirected. Seed
**20260218**, second pool **20260219**, 40 graphs per pool.
- **CP51.** **The decision problem:** does a path passing through every node
exist. The procedure halts as soon as it finds the first path; if none
exists, it scans the entire search space.
- **CP52.** **The search problem:** if such a path exists, **which one**. In
this lesson, the decision procedure already produces the path; the step
count of search and decision are **the same** for this problem, and this is
a measured result, not a general rule.
- **CP53.** **The counting problem:** how many are there. The procedure never
halts; it scans the entire space. Since every path is traversed from both
ends, the count includes every path twice; what is measured is **steps**,
not the count itself.
- **CP54.** **Verification:** whether a given candidate path is genuinely a
Hamiltonian path is tested. A step is checking whether two consecutive
nodes are adjacent.
- **CP55.** The oracle is the counting procedure: it scans the space
completely, so its answer is correct by definition. The decision's answer
is compared against the oracle's count on every graph.
- **CP56.** The measure is **steps**; a step is a node visit in the search
tree.
- **CP57.** Yes graphs and no graphs are **totaled separately**. Combining
them into a single average would erase the difference this lesson measures.
- **CP58.** **The names of complexity classes are not written in this
lesson.** How the difference between verification and search is named
belongs to the Theory of Computation course.
- **CP59.** The cheap test checks only **necessary conditions**. Its
wrong-answer count is measured against the oracle; coming out zero is not
assumed, it is counted.
#### Three Questions, One Input
The block below runs decision and counting separately on every graph,
verifies any path found, and totals the results separately for yes and no
graphs.
```python
SEED = 20260218
N = 8
def generator(seed):
d = seed
def next_val(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_val
def graphs(seed=SEED, samples=40, share=38, n=N):
"""Undirected graph: each pair connected with probability `share` percent."""
r = generator(seed)
pool = []
for _ in range(samples):
neighbors = {i: set() for i in range(n)}
for i in range(n):
for j in range(i + 1, n):
if r(100) < share:
neighbors[i].add(j)
neighbors[j].add(i)
pool.append(neighbors)
return pool
class Counter:
def __init__(self):
self.steps = 0
def count(self):
self.steps += 1
def search(neighbors, s, stop, n=N):
"""stop=True: halts at the first Hamiltonian path. stop=False: counts them all."""
found, count = [], 0
def walk(v, path):
nonlocal count
s.count()
if len(path) == n:
count += 1
if not found:
found.append(tuple(path))
return True
for k in sorted(neighbors[v]):
if k not in path:
path.append(k)
done = walk(k, path)
path.pop()
if done and stop:
return True
return False
for start in range(n):
if walk(start, [start]) and stop:
break
return (found[0] if found else None), count
def verify(neighbors, path, n=N):
"""Oracle: does the candidate path have n nodes, are consecutive pairs adjacent."""
steps = 0
if path is None or len(set(path)) != n:
return False, steps
for a, b in zip(path, path[1:]):
steps += 1
if b not in neighbors[a]:
return False, steps
return True, steps
for seed in (20260218, 20260219):
yes = {"n": 0, "decision": 0, "counting": 0, "verification": 0}
no = {"n": 0, "decision": 0, "counting": 0}
equal = 0
for neighbors in graphs(seed):
s1, s2 = Counter(), Counter()
path, _ = search(neighbors, s1, True)
_, count = search(neighbors, s2, False)
if path is not None:
ok, d = verify(neighbors, path)
yes["n"] += 1
yes["decision"] += s1.steps
yes["counting"] += s2.steps
yes["verification"] += d
else:
no["n"] += 1
no["decision"] += s1.steps
no["counting"] += s2.steps
if s1.steps == s2.steps:
equal += 1
print("seed", seed)
print(" yes ", yes, "ratio", round(yes["counting"] / yes["decision"], 2))
print(" no ", no, "graphs with decision=counting:", equal, "/", no["n"])
```
```
seed 20260218
yes {'n': 25, 'decision': 1526, 'counting': 29270, 'verification': 175} ratio 19.18
no {'n': 15, 'decision': 2538, 'counting': 2538} graphs with decision=counting: 15 / 15
seed 20260219
yes {'n': 23, 'decision': 915, 'counting': 26792, 'verification': 161} ratio 29.28
no {'n': 17, 'decision': 4390, 'counting': 4390} graphs with decision=counting: 17 / 17
```
#### Why a No Answer Accepts No Discount
On the first pool, a path exists on **25 of 40 graphs**, and none on 15. On
the yes graphs, decision spends a total of **1526 steps**, counting **29,270
steps**; a ratio of **19.18**. The decision procedure halts the moment it
finds the first path and never enters the rest of the space.
On no graphs, the two numbers are **identical**: 2538 and 2538. Moreover,
this equality does not just hold in total — it holds **individually on all 15
of the 15 graphs**. The reason is not procedural but logical. To say "no,"
the decision procedure must see that not a single branch gives a path; if
even one branch remains unseen, its answer is a guess. When there is no
path, no early-stopping point ever arises, so the decision procedure
traverses exactly the tree the counting procedure traverses.
The second pool gives the same structure: 23 yes, 17 no, ratio 29.28, and the
decision step count equals the counting step count on **17 of the 17** no
graphs. The ratio on yes graphs rises from 19.18 to 29.28, meaning **the size
of the ratio depends on the pool**; but the exact equality on no graphs holds
**without exception** on both pools. One is a measured value; the other is a
consequence of the procedure's structure, and the two are not written in the
same sentence.
This lets the previous lesson's knight's-tour row be reread. Both orderings
visiting 29,976 nodes on the four-by-four board was not a coincidence:
because the answer was no, there was no early-stopping point, and ordering
could only have paid off by pulling that stop earlier.
#### A Cheap Test That Works One Way
A no answer being expensive does not mean **every** no answer is expensive.
On some graphs, the absence of a path can be seen without entering the
search at all: if the graph is disconnected, no path can reach every node,
and if more than two nodes have only a single neighbor, they cannot all be an
endpoint of the path. These are **necessary conditions**: if unmet, there is
no path; if met, they say nothing.
```python
from collections import deque
SEED, N = 20260218, 8
def generator(seed):
d = seed
def next_val(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_val
def graphs(seed=SEED, samples=40, share=38, n=N):
r = generator(seed)
pool = []
for _ in range(samples):
neighbors = {i: set() for i in range(n)}
for i in range(n):
for j in range(i + 1, n):
if r(100) < share:
neighbors[i].add(j)
neighbors[j].add(i)
pool.append(neighbors)
return pool
class Counter:
def __init__(self):
self.steps = 0
def count(self):
self.steps += 1
def cheap_test(neighbors, s, n=N):
"""Tests necessary conditions. Can say 'no'; cannot say 'yes'."""
queue, seen = deque([0]), {0}
while queue:
v = queue.popleft()
for k in neighbors[v]:
s.count()
if k not in seen:
seen.add(k)
queue.append(k)
if len(seen) < n:
return "no"
lone = 0
for v in range(n):
s.count()
if len(neighbors[v]) <= 1:
lone += 1
return "no" if lone > 2 else "unknown"
def decide(neighbors, s, n=N):
def walk(v, path):
s.count()
if len(path) == n:
return True
for k in sorted(neighbors[v]):
if k not in path:
path.append(k)
if walk(k, path):
return True
path.pop()
return False
return any(walk(start, [start]) for start in range(n))
for seed in (20260218, 20260219):
cheap_no, cheap_steps, wrong = 0, 0, 0
remaining_no, remaining_steps = 0, 0
for neighbors in graphs(seed):
s1, s2 = Counter(), Counter()
y = cheap_test(neighbors, s1)
exists = decide(neighbors, s2)
cheap_steps += s1.steps
if y == "no":
cheap_no += 1
if exists:
wrong += 1
elif not exists:
remaining_no += 1
remaining_steps += s2.steps
print("seed", seed, "| cheap test said 'no':", cheap_no,
"| wrong:", wrong, "| cheap total steps:", cheap_steps)
print(" no graphs the cheap test could not resolve:", remaining_no,
"| full-scan steps:", remaining_steps)
```
```
seed 20260218 | cheap test said 'no': 11 | wrong: 0 | cheap total steps: 1092
no graphs the cheap test could not resolve: 4 | full-scan steps: 676
seed 20260219 | cheap test said 'no': 13 | wrong: 0 | cheap total steps: 1102
no graphs the cheap test could not resolve: 4 | full-scan steps: 1638
```
On the first pool, **11 of the 15** no graphs are resolved by the cheap
test, with **zero wrong answers**: the test never says "no" to a graph that
has a path. 4 graphs remain, and for those a full scan is unavoidable, 676
steps. On the second pool, 13 of the 17 no graphs are resolved, again zero
wrong, again 4 graphs remaining, this time at 1638 steps.
The 1092 steps the test spends across all forty graphs is below the 2538
steps a full scan spends on no graphs. But the entire saving is
**one-directional**. The test can never say "yes" on any graph; of the 29
graphs it calls "unknown," 25 have a path and 4 do not, and telling the two
apart again requires a full scan. A cheap proof always works in one
direction, and its number cannot be read without saying which direction that
is.
#### Searching Versus Verifying
The fourth number is verification. On yes graphs, the total is **175
steps**, that is, 7 steps per graph: seven consecutive pairs in an
eight-node path. On the same graphs, **finding** the path spends an average
of 61 steps, 8.7 times verification.
The gap between these two numbers is small on a single graph in this lesson,
but **what the two numbers do with scale** is not the same. Verification is
node-count-minus-one steps and grows linearly with node count. Search's step
count, on the other hand, depends on the size of the tree traversed. The
block below sweeps node count and places the two growths side by side.
```python
SEED = 20260218
def generator(seed):
d = seed
def next_val(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_val
def graphs(n, share, seed=SEED, samples=20):
r = generator(seed)
pool = []
for _ in range(samples):
neighbors = {i: set() for i in range(n)}
for i in range(n):
for j in range(i + 1, n):
if r(100) < share:
neighbors[i].add(j)
neighbors[j].add(i)
pool.append(neighbors)
return pool
class Counter:
def __init__(self):
self.steps = 0
def count(self):
self.steps += 1
def decide(neighbors, n, s):
"""Halts at the first Hamiltonian path; if none exists, the whole space is scanned."""
found = []
def walk(v, path):
s.count()
if len(path) == n:
found.append(tuple(path))
return True
for k in sorted(neighbors[v]):
if k not in path:
path.append(k)
done = walk(k, path)
path.pop()
if done:
return True
return False
for start in range(n):
if walk(start, [start]):
return found[0]
return None
print(" n | yes | yes avg steps | no avg steps | verification steps")
for n, share in ((6, 30), (8, 30), (10, 30), (12, 30)):
e = h = ea = ha = 0
for neighbors in graphs(n, share):
s = Counter()
y = decide(neighbors, n, s)
if y is None:
h, ha = h + 1, ha + s.steps
else:
e, ea = e + 1, ea + s.steps
print(f"{n:2d} | {e:4d} | {(ea // e if e else 0):18d} |"
f" {(ha // h if h else 0):19d} | {n - 1:15d}")
```
```
n | yes | yes avg steps | no avg steps | verification steps
6 | 2 | 8 | 23 | 5
8 | 6 | 121 | 92 | 7
10 | 9 | 164 | 757 | 9
12 | 10 | 1054 | 11601 | 11
```
The three columns show three different kinds of growth. Verification is 5,
7, 9, 11: increasing by one with node count. On yes graphs, decision is 8,
121, 164, 1054. On no graphs, decision is 23, 92, 757, **11,601**; going from
six nodes to twelve, it grows by more than five hundred times. At the
eight-node row, the no average falls below the yes average; this single-row
reversal comes from the scarcity of no graphs in that pool and does not
generalize.
What is measured is this: **verifying an answer and finding it are not the
same order of magnitude**, and the gap widens with node count. At twelve
nodes, testing a candidate path is 11 steps; showing that no such path
exists averages 11,601 steps. The same graph, the same question, a
thousandfold difference.
#### Where the Naming Belongs
Four numbers have been measured so far: decision, search, counting,
verification. One thing remains unmeasured, and it is left out on purpose.
Problems that appear cheap to verify but expensive to find have a **shared
name**; there is a procedure for transforming these problems into one
another, and a theory of the difference between "appears expensive" and "is
expensive."
None of this is built in this course. In the travelling salesman lesson, the
"why is it hard" question was referred to the Theory of Computation course;
the naming question here goes to the same place. What this course does is
measure the **difference that course will speak about** and put it on the
table: 11 steps against 11,601 steps, and that this is an observation, not a
proof.
This distinction leaves behind a useful habit. Before calling a problem
hard, it must be written **which question** of that problem is expensive.
For the same set of graphs, decision spent 1526 steps, counting 29,270,
verification 175. The sentence "the Hamiltonian path is expensive" says
nothing unless it states which of these three numbers it means.
#### Summary
- An existence question is framed as a decision problem; search and counting
problems stand alongside it, and all three costs are measured separately
on the same input.
- On the 25 graphs where a path is found, decision spends 1526 steps,
counting 29,270; the ratio is 19.18, and 29.28 on the second pool, meaning
the size of the ratio depends on the pool.
- On **every** graph where no path is found, decision steps exactly equal
counting steps: 15 of 15 on the first pool, 17 of 17 on the second. A no
answer leaves no early-stopping point.
- Verification is 7 steps per graph, search averages 61; when node count is
swept, verification rises one at a time while the cost of a no answer
climbs from 23 to 11,601.
- This measured difference is an observation; naming and proving it belongs
to the Theory of Computation course and is not done in this lesson.
#### Next Step
The Classical Problems topic followed the same procedure across six
problems: build the oracle, measure the pattern, count the split inputs. The
next topic makes this procedure independent of the problem. Practice
Discipline turns inferring which pattern fits a problem's step budget from
its constraints, and comparing a solution against an oracle, into a habit.
The first question is: given an input size, how is which pattern fits the
budget found **by counting**, rather than guessed.
## Practice Discipline
Deriving a step budget from input size, verifying a solution by comparing it against an oracle, and counting repeated practice's progress metric through pattern diversity.
### Problem Reading and Constraint Analysis
Source: https://academia.sh/en/courses/advanced-algorithms/problem-reading-and-constraint-analysis
Deriving a step budget from input size, and measuring that the budget alone cannot select the pattern: at n=12 the sorting pattern is 1.61 times costlier than the oracle, and a wrong constraint reading gives a wrong answer on 18 of 40 inputs.
The previous topic worked with named problems: the knapsack, the traveling salesman, n
queens. The shape of these problems was given; the question asked was which approach
solved them. In practice the situation is reversed. There is a piece of text in hand, a
few numbers appear inside it, and which pattern is suitable is never stated. What
selects the pattern is not the problem's name but its **constraints**.
This lesson turns constraint reading into a procedure and measures where that procedure
works and where it turns into guessing. The result runs two ways: input size genuinely
eliminates large families of solutions, but it does **not** choose among the ones that
remain — and when a constraint is misread, the solution that emerges can be the
cheapest of all of them, and still be wrong.
#### From Input Size to a Step Budget
Constraint reading is a single calculation: the largest input size in the problem text
is taken, an upper bound is placed on the number of steps that can be spent at that
size, and each candidate solution's fit within that bound is computed. The bound is
called the **step budget**. Asymptotic notation itself was built in the Algorithm
Analysis topic of the Algorithms course and is not repeated here; what happens here is
turning the notation into a **decision criterion**.
- **AD1** — The step budget is the most steps a solution can spend and still count as
acceptable. In this lesson the budget is **108 steps** and is taken as a
constant.
- **AD2** — The budget is computed over the **worst case**. An average input does not
enter the calculation; the purpose of the calculation is to eliminate a solution, not
to praise one.
- **AD3** — A lookup in a hash table is one step, an insertion is one step. Both are
constant.
- **AD4** — A sort's step count is the actual number of **comparisons made**, and it is
measured by counting them. The sorting algorithms themselves were built in the
Algorithms course and are used here.
- **AD5** — The worst-case input is produced like this: every value is even, the target
is odd. Then no pair can give the target, and every solution is forced to run to the
end.
The problem is this: does an array contain **two distinct positions whose sum equals
the target**. There are three candidates. Brute force tries every pair; it is this
course's oracle. The second candidate sorts the array and scans it with two pointers.
The third keeps a hash table in a single pass.
```python
"""How the step count of three solutions to the same problem behaves on the worst-case input."""
from functools import cmp_to_key
SEED = 20260218
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
class Counter:
def __init__(self):
self.step = 0
def count(self, n=1):
self.step += n
def worst_case_input(n, seed=SEED):
"""All values even, target odd: no pair can give the target."""
array = [2 * i for i in range(n)]
r = generator(seed)
for i in range(n - 1, 0, -1):
j = r(i + 1)
array[i], array[j] = array[j], array[i]
return array
def oracle_pairs(array, target, s):
for i in range(len(array)):
for j in range(i + 1, len(array)):
s.count()
if array[i] + array[j] == target:
return True
return False
def pattern_sort_two_pointer(array, target, s):
def compare(a, b):
s.count()
return -1 if a < b else (1 if a > b else 0)
sorted_array = sorted(array, key=cmp_to_key(compare))
left, right = 0, len(sorted_array) - 1
while left < right:
s.count()
total = sorted_array[left] + sorted_array[right]
if total == target:
return True
left, right = (left + 1, right) if total < target else (left, right - 1)
return False
def pattern_hash_table(array, target, s):
seen = set()
for x in array:
s.count()
if target - x in seen:
return True
s.count()
seen.add(x)
return False
print("n brute force sort+two pointer hash table")
for n in (4, 8, 32, 128, 512, 2048):
d = worst_case_input(n)
s1, s2, s3 = Counter(), Counter(), Counter()
oracle_pairs(d, 7, s1)
pattern_sort_two_pointer(d, 7, s2)
pattern_hash_table(d, 7, s3)
print(f"{n:5d} {s1.step:11d} {s2.step:19d} {s3.step:11d}")
print()
for ad, pattern in (("sort+two pointer", pattern_sort_two_pointer),
("hash table ", pattern_hash_table)):
for n in range(2, 40):
d = worst_case_input(n)
s1, s2 = Counter(), Counter()
oracle_pairs(d, 7, s1)
pattern(d, 7, s2)
if s2.step < s1.step:
print(f"{ad} first n where it beats brute force: {n}"
f" (pattern {s2.step} , oracle {s1.step})")
break
```
```
n brute force sort+two pointer hash table
4 6 10 8
8 28 22 16
32 496 152 64
128 8128 861 256
512 130816 4483 1024
2048 2096128 22015 4096
sort+two pointer first n where it beats brute force: 8 (pattern 22 , oracle 28)
hash table first n where it beats brute force: 6 (pattern 12 , oracle 15)
```
The table's first row is constraint reading's most overlooked result: at n=4, **brute
force is the cheapest of the three solutions** — 6 steps, against the hash table's 8 and
the sorting pattern's 10. The sorting pattern only becomes cheaper than the oracle at
**n=8**, the hash table at **n=6**. Brute force is not a poor choice at small input
sizes; what is poor is choosing it without knowing at which size it stops being one.
#### Budget Calculation at Four Input Sizes
The measurement above goes up to 2048, but constraint reading has to make a decision up
to 108, and actually running brute force at that size is not an option. For
this reason the budget calculation is done **with a formula validated against the
measured count**: first it is checked whether the formula holds at small n, then the
formula is carried to large n.
- **AD6** — Brute force's worst-case step count is estimated as $n(n-1)/2$, the hash
table's as $2n$, the sorting pattern's as $n\log_2 n + n$.
- **AD7** — A formula can be used in the budget calculation only **as long as it does
not fall below** the measured step count. A formula that understates is not accepted.
```python
"""Does the estimate formula hold against the measured step count, and the budget
calculation at four input sizes."""
from math import log2
MEASURED = {8: (28, 22, 16), 32: (496, 152, 64), 128: (8128, 861, 256),
512: (130816, 4483, 1024), 2048: (2096128, 22015, 4096)}
BUDGET = 10 ** 8
def estimate(n):
return (n * (n - 1) // 2, int(n * log2(n)) + n, 2 * n)
print("n brute force sort+two pointer hash table")
print(" measured estimate m/e measured estimate m/e measured estimate")
for n, (a, b, c) in MEASURED.items():
ta, tb, tc = estimate(n)
print(f"{n:5d} {a:7d} {ta:6d} {a / ta:.2f} {b:8d} {tb:6d} {b / tb:.2f}"
f" {c:8d} {tc:6d}")
print()
print(f"step budget: {BUDGET}")
print("n brute force sort+two pointer hash table")
for n in (10 ** 3, 10 ** 5, 10 ** 6, 10 ** 8):
ta, tb, tc = estimate(n)
d = [("fits" if t <= BUDGET else "no fit") for t in (ta, tb, tc)]
print(f"{n:<11d} {ta:.2e} {d[0]:6s} {tb:.2e} {d[1]:6s} {tc:.2e} {d[2]:6s}")
```
```
n brute force sort+two pointer hash table
measured estimate m/e measured estimate m/e measured estimate
8 28 28 1.00 22 32 0.69 16 16
32 496 496 1.00 152 192 0.79 64 64
128 8128 8128 1.00 861 1024 0.84 256 256
512 130816 130816 1.00 4483 5120 0.88 1024 1024
2048 2096128 2096128 1.00 22015 24576 0.90 4096 4096
step budget: 100000000
n brute force sort+two pointer hash table
1000 5.00e+05 fits 1.10e+04 fits 2.00e+03 fits
100000 5.00e+09 no fit 1.76e+06 fits 2.00e+05 fits
1000000 5.00e+11 no fit 2.09e+07 fits 2.00e+06 fits
100000000 5.00e+15 no fit 2.76e+09 no fit 2.00e+08 no fit
```
The top table confirms the formula. For brute force, measured and estimated are
**identical** (ratio 1.00); the same for the hash table. For the sorting pattern the
ratio **rises** from 0.69 to 0.90: the formula overstates the true comparison count at
every n, that is, it errs on the safe side, satisfying AD7. The amount of overstatement
is not constant — at small n the estimate is 1.45 times the measurement, at n=2048 it is
1.12 times. **This is the point where constraint reading turns into guessing:** the
formula used carries a coefficient, and until that coefficient is measured, the budget
calculation is a guess.
The bottom table is the four sizes this lesson asked for. At n=1,000, **all three
solutions** fit the budget; constraint reading eliminates nothing here, and the choice
is left to another criterion. At n=100,000, brute force is eliminated at 5.00·109
steps, leaving two candidates. At n=1,000,000 the table stays the same. At
n=108, **none of the three fit** — even the hash table, the cheapest, asks
for 2.00·108 steps, twice the budget. This last row is constraint reading's
most useful output: if not even a single-pass solution fits at that size, the problem
text must be giving **another constraint**, and it has not been read.
#### When a Constraint Is Misread
The entire calculation above assumed one thing: that all three candidates give the
correct answer. Until that assumption is tested, the budget calculation cannot select a
solution. The course's rule applies here as well — **a pattern whose precondition is
not tested against the oracle is not considered measured.**
A fourth candidate is added. Someone who thinks the problem text says "values do not
exceed 20" can keep an array marking the range 0..20 and answer in a single pass. What
was missed is that the text does not state a lower bound, and values can be negative.
- **AD8** — The corpus is the shared reference's corpus: 40 inputs, each with 12 values,
values ranging from −9 to 20. The target is 11.
- **AD9** — The wrong constraint reading **silently skips** an out-of-range value; it
does not crash, it returns an answer.
- **AD10** — The second corpus is produced with seed `20260219`, and whether the
diverging-input ratio stays in the same order of magnitude is written down.
```python
"""The budget calculation does not choose the pattern, the oracle does. Two corpora."""
from functools import cmp_to_key
SEED, SECOND, LENGTH, CORPUS_SIZE = 20260218, 20260219, 12, 40
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
class Counter:
def __init__(self):
self.step = 0
def count(self, n=1):
self.step += n
def corpus(seed=SEED, n=CORPUS_SIZE, length=LENGTH):
r = generator(seed)
return [{"no": i + 1, "array": [r(30) - 9 for _ in range(length)]}
for i in range(n)]
def oracle_pairs(array, target, s):
for i in range(len(array)):
for j in range(i + 1, len(array)):
s.count()
if array[i] + array[j] == target:
return True
return False
def pattern_sort_two_pointer(array, target, s):
def compare(a, b):
s.count()
return -1 if a < b else (1 if a > b else 0)
sorted_array = sorted(array, key=cmp_to_key(compare))
left, right = 0, len(sorted_array) - 1
while left < right:
s.count()
total = sorted_array[left] + sorted_array[right]
if total == target:
return True
left, right = (left + 1, right) if total < target else (left, right - 1)
return False
def pattern_hash_table(array, target, s):
seen = set()
for x in array:
s.count()
if target - x in seen:
return True
s.count()
seen.add(x)
return False
def pattern_counting_array(array, target, s, upper=20):
"""WRONG CONSTRAINT READING: values are assumed to be in the range 0..upper."""
present = [False] * (upper + 1)
for x in array:
s.count()
if 0 <= x <= upper:
complement = target - x
if 0 <= complement <= upper and present[complement]:
return True
present[x] = True
return False
def measure(pattern, items, target):
diverging, pattern_steps, oracle_steps = [], 0, 0
for k in items:
s1, s2 = Counter(), Counter()
a = pattern(k["array"], target, s1)
b = oracle_pairs(k["array"], target, s2)
pattern_steps, oracle_steps = pattern_steps + s1.step, oracle_steps + s2.step
if a != b:
diverging.append(k["no"])
return {"diverging": len(diverging), "first_diverging": diverging[:5],
"pattern_step": pattern_steps, "oracle_step": oracle_steps}
for seed in (SEED, SECOND):
print(f"corpus seed {seed} , 40 inputs x 12 values , target 11")
K = corpus(seed)
for ad, pattern in (("sort+two pointer", pattern_sort_two_pointer),
("hash table ", pattern_hash_table),
("counting (wrong constraint)", pattern_counting_array)):
print(f" {ad}", measure(pattern, K, 11))
print()
```
```
corpus seed 20260218 , 40 inputs x 12 values , target 11
sort+two pointer {'diverging': 0, 'first_diverging': [], 'pattern_step': 1398, 'oracle_step': 866}
hash table {'diverging': 0, 'first_diverging': [], 'pattern_step': 486, 'oracle_step': 866}
counting (wrong constraint) {'diverging': 18, 'first_diverging': [4, 10, 13, 14, 15], 'pattern_step': 396, 'oracle_step': 866}
corpus seed 20260219 , 40 inputs x 12 values , target 11
sort+two pointer {'diverging': 0, 'first_diverging': [], 'pattern_step': 1441, 'oracle_step': 970}
hash table {'diverging': 0, 'first_diverging': [], 'pattern_step': 566, 'oracle_step': 970}
counting (wrong constraint) {'diverging': 17, 'first_diverging': [3, 4, 6, 7, 8], 'pattern_step': 391, 'oracle_step': 970}
```
Three numbers side by side. **Oracle:** 866 steps, correct on 40 of 40 inputs.
**Patterns:** the hash table 486 steps, the sorting pattern 1398 steps, the wrong
constraint reading 396 steps. **Diverging inputs:** **0** for the first two, **18** for
the wrong reading — that is, a different answer than the oracle on 18 of 40 inputs, a
ratio of **0.4500**.
The result says two things. First, **step-count ranking is not correctness ranking**:
the candidate that spends the fewest steps (396) is exactly the one that gives the wrong
answer. Second, at n=12 the sorting pattern is **1.61 times costlier** than the oracle
(1398 against 866). The budget calculation had selected this pattern for large n; at the
actual input size of 12, that same choice is worse than the oracle. The budget
calculation is a tool for **elimination**, not a tool for selection.
The second corpus confirms the result: diverging inputs drop from 18 to 17 (ratio
0.4250), staying in the same order of magnitude; the two correct patterns give 0
diverging inputs there as well. The shared reference's first reading had already shown
two pointers to be wrong on 25 of 40 inputs on an unsorted input; the 18 here is the
**constraint-reading** side of the same rule — the pattern is correct, the reading is
not.
#### Summary
- Constraint reading is the operation of deriving a **step budget** from input size and
eliminating candidates with it; in this lesson the budget is 108 steps and
the calculation is always done over the worst case.
- Measurement vindicates brute force at small input: at n=4, the cheapest of the three
solutions is brute force with 6 steps; the hash table only beats it at n=6, the
sorting pattern at n=8.
- The budget leaves a different number of candidates at four sizes: at n=1,000 all
three fit, at n=100,000 and n=1,000,000 brute force is eliminated, at
n=108 **none** fit — even the hash table asks for 2.00·108 steps.
- Until a formula is validated against measurement, the budget calculation is a guess:
for the sorting pattern the estimate is 1.45 times the measurement at n=8, 1.12 times
at n=2048; for brute force and the hash table the ratio is exactly 1.00.
- Comparison against the oracle says what the budget calculation cannot: the candidate
spending the fewest steps (396 steps) gives the wrong answer on **18 of 40** inputs,
17 on the second corpus; the two correct patterns give 0 diverging inputs on both
corpora.
- At the real input size, budget ranking can reverse: at n=12 the sorting pattern is
1.61 times costlier than the oracle. The budget calculation eliminates, it does not
select.
#### Next Step
In this lesson the oracle was used once, and it caught a wrong constraint reading on 18
inputs. But those 18 inputs came from a corpus already in hand; a different corpus
would have caught a different number. The next lesson takes up exactly this dependency:
**which inputs** should a solution be compared against the oracle on, how many times do
edge cases show up in a randomly generated corpus, and once a diverging input is found,
how is it shrunk down to its smallest form. The measure remains a count: how many of
the deliberately broken versions are caught by which corpus.
### Solution Verification
Source: https://academia.sh/en/courses/advanced-algorithms/solution-verification
Turning comparison against an oracle into a procedure: 40 random inputs catch four of five defects, 12 edge cases catch all five, and a diverging input shrinks from 12 values to 1 in 23 attempts.
The previous lesson showed that a solution fitting the step budget does not make it
correct: the candidate spending the fewest steps diverged from the oracle on 18 of 40
inputs. That number 18 came from a corpus already in hand and was never questioned.
This lesson does that questioning — **which inputs a comparison is run against is as
decisive as the comparison itself.**
Test-writing discipline, edge-case thinking, and review habits were built in other
courses and are not repeated here. What this lesson adds is a single thing: the
**oracle** — counting correctness not against an opinion but against a second
solution's answer.
#### The Verification Procedure
The procedure has three steps, and their order does not change. First, a set of inputs
is generated. Second, both the solution under test and the oracle are run on the same
inputs. Third, the inputs on which the two answers differ are **counted**. The output is
a number: **diverging inputs**. If it is zero, no proof was found in that set; if it is
above zero, the solution is wrong and a concrete example is in hand.
- **AD11** — The solution under test is two pointers on a sorted array. The
precondition holds on every input; this lesson tests not the precondition but the
**implementation**.
- **AD12** — Five defective versions are produced by hand, and each defect is a
**one-line** change: the end condition, the starting position, the ending position,
the direction of movement, an empty input.
- **AD13** — If a version crashes, that too counts as a divergence; a crash is less
dangerous than a silent wrong answer, but it is still a divergence.
- **AD14** — The random corpus is 40 inputs, each carrying **exactly 12 values**; the
target also comes from the same generator and ranges from $-18$ to $22$.
- **AD15** — The edge-case set is 12 inputs and is built by hand: an empty array, one
element, two elements, only repeated values, values at the extremes, an unreachable
target.
```python
"""Five defective versions, two separate input sets, against the oracle."""
SEED, SECOND = 20260218, 20260219
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
class Counter:
def __init__(self):
self.step = 0
def count(self, n=1):
self.step += n
def random_corpus(seed=SEED, n=40, length=12):
r = generator(seed)
return [(sorted(r(30) - 9 for _ in range(length)), r(41) - 18)
for _ in range(n)]
EDGE_CASES = [([], 11), ([5], 10), ([5], 5), ([5, 6], 11), ([5, 6], 10),
([7, 7], 14), ([-9, -9], -18), ([0, 0, 0], 0), ([-9, 20], 11),
([1, 2, 3, 4], 3), ([1, 2, 3, 4], 7), ([2, 2, 2, 2], 4)]
def oracle(array, target, s):
for i in range(len(array)):
for j in range(i + 1, len(array)):
s.count()
if array[i] + array[j] == target:
return True
return False
def pattern_two_pointers(array, target, s, defect=None):
"""defect=None is the correct version. Each defect is one line."""
left = 1 if defect == "K2" else 0
right = (len(array) - 2) if defect == "K3" else (len(array) - 1)
if defect == "K5":
array[0] # does not account for an empty array
while (left <= right) if defect == "K1" else (left < right):
s.count()
total = array[left] + array[right]
if total == target:
return True
forward = (total > target) if defect == "K4" else (total < target)
left, right = (left + 1, right) if forward else (left, right - 1)
return False
DEFECTS = [(None, "correct version"), ("K1", "left <= right"), ("K2", "first element skipped"),
("K3", "last element skipped"), ("K4", "directions reversed"),
("K5", "empty array uncounted")]
def measure(defect, items):
diverging, pattern_steps, oracle_steps = 0, 0, 0
for array, target in items:
s1, s2 = Counter(), Counter()
try:
a = pattern_two_pointers(list(array), target, s1, defect)
except IndexError:
a = "crashed"
b = oracle(list(array), target, s2)
pattern_steps, oracle_steps = pattern_steps + s1.step, oracle_steps + s2.step
if a != b:
diverging += 1
return diverging, pattern_steps, oracle_steps
R = random_corpus()
print(f"random corpus {len(R)} inputs x 12 values | "
f"edge case set {len(EDGE_CASES)} inputs")
print("defect random edge pattern steps oracle steps")
for k, ad in DEFECTS:
ar, akr, ahr = measure(k, R)
ak, akk, ahk = measure(k, EDGE_CASES)
print(f"{ad:22s} {ar:8d} {ak:5d} {akr + akk:10d} {ahr + ahk:10d}")
print()
Y = random_corpus(SECOND)
print("second corpus 20260219 , random set:",
{ad: measure(k, Y)[0] for k, ad in DEFECTS})
```
```
random corpus 40 inputs x 12 values | edge case set 12 inputs
defect random edge pattern steps oracle steps
correct version 0 0 329 1596
left <= right 3 2 344 1596
first element skipped 4 5 297 1596
last element skipped 3 5 317 1596
directions reversed 25 2 423 1596
empty array uncounted 0 1 329 1596
second corpus 20260219 , random set: {'correct version': 0, 'left <= right': 2, 'first element skipped': 2, 'last element skipped': 2, 'directions reversed': 27, 'empty array uncounted': 0}
```
Three numbers side by side. **Oracle:** **1596 steps** across all 52 inputs, correct on
every one of them. **Pattern:** the correct version, **329 steps**, that is, **4.85
times fewer** than the oracle. **Diverging inputs:** 0 for the correct version in both
sets; between 0 and 25 for the five defective versions.
The table's real finding is in the last row. With 40 inputs, the random corpus catches
**four of the five defects** and **never catches** the fifth — the version that does not
account for an empty input. The reason is procedural, not bad luck: every input in the
corpus carries exactly 12 values, so an empty or single-element input **never exists**
in that set. One defect class is structurally unreachable because of the shape of the
input generator, and raising the corpus from 40 to 400 does not change this.
The edge-case set does the opposite: **12 inputs catch all five** of the five defects.
Yield per input is 4 out of 40 versus 5 out of 12 — **0.1000 against 0.4167**. On the
other hand, the edge-case set alone is not enough either: it catches the
direction-reversed version on only 2 inputs, while the random corpus catches it on 25.
The two sets see different defect classes, and neither substitutes for the other.
The second corpus confirms the result: with seed `20260219`, the numbers come out 2, 2,
2, 27, 0 — four defects are caught again, the empty-input defect slips through again,
and the direction-reversed version's dominance continues. The diverging-input ratio
stays in the same order of magnitude; the result does not depend on the corpus.
#### Shrinking a Diverging Input
Once a divergence is found, a 12-value array is in hand, and which property of that
array triggers the defect is not clear. Shrinking resolves this: a value is removed from
the input; if the divergence persists, the smaller form is kept; if not, the value is
put back. The operation repeats until no removal preserves the divergence.
- **AD16** — Shrinking only **removes values**; it never adds or changes a value. The
result is a subsequence of the starting input.
- **AD17** — One attempt means running both the defective version and the oracle once
on the candidate input.
- **AD18** — Shrinking's result is **locally smallest**: it cannot be shrunk further by
removing one value. It is not claimed to be the smallest among all possible
subsequences.
```python
"""Shrinking a diverging input down to its smallest form."""
from math import comb
SEED = 20260218
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def random_corpus(seed=SEED, n=40, length=12):
r = generator(seed)
return [(sorted(r(30) - 9 for _ in range(length)), r(41) - 18)
for _ in range(n)]
def oracle(array, target):
return any(array[i] + array[j] == target
for i in range(len(array)) for j in range(i + 1, len(array)))
def pattern_two_pointers(array, target, defect=None):
left = 1 if defect == "K2" else 0
right = (len(array) - 2) if defect == "K3" else (len(array) - 1)
if defect == "K5":
array[0]
while (left <= right) if defect == "K1" else (left < right):
total = array[left] + array[right]
if total == target:
return True
forward = (total > target) if defect == "K4" else (total < target)
left, right = (left + 1, right) if forward else (left, right - 1)
return False
def diverges(array, target, defect):
try:
return pattern_two_pointers(list(array), target, defect) != oracle(array, target)
except IndexError:
return True
def shrink(array, target, defect):
"""One value is removed; if divergence persists, the removed form is kept."""
attempts = 0
progressed = True
while progressed:
progressed = False
for i in range(len(array)):
candidate = array[:i] + array[i + 1:]
attempts += 1
if diverges(candidate, target, defect):
array, progressed = candidate, True
break
return array, attempts
R = random_corpus()
print("defect first diverging input no length shrunk attempts smallest input")
for defect in ("K1", "K2", "K3", "K4"):
for no, (array, target) in enumerate(R, 1):
if diverges(array, target, defect):
shrunk, attempts = shrink(array, target, defect)
print(f"{defect:5s} {no:20d} {len(array):7d} {len(shrunk):8d}"
f" {attempts:6d} {shrunk} target {target}")
break
print()
print("number of sorted length-12 inputs (values -9..20):", comb(41, 12))
print("the 52-input set sees", f"{52 / comb(41, 12):.1e}", "of this space")
```
```
defect first diverging input no length shrunk attempts smallest input
K1 1 12 1 23 [-8] target -16
K2 17 12 2 31 [-8, -5] target -13
K3 7 12 2 21 [-5, 16] target 11
K4 4 12 3 20 [-3, 18, 20] target 15
number of sorted length-12 inputs (values -9..20): 7898654920
the 52-input set sees 6.6e-09 of this space
```
All four diverging inputs drop from **12 values to 1, 2, and 3 values**, and this
happens in 20 to 31 attempts. What shrinking earns is not fewer steps but
**readability**: someone looking at `[-3, 18, 20]` with target 15 can see directly why
the direction-reversed version goes wrong; the same information is hidden inside a
12-value array.
The first row ties the two sections together. The defective version with a broken end
condition has a smallest diverging input that is a **single-element** array: `[-8]`,
target $-16$. That single-element input was written **by hand** in the edge-case set in
the previous section. In other words, shrinking **spontaneously produces** some of the
hand-built edge cases — starting from a random 12-value input and arriving at the same
place. An edge-case list does not have to be a product of intuition; it can be derived
from a diverging input.
#### Oracle Independence
The whole procedure rests on the oracle's correctness. If the oracle is wrong, the
diverging-input count comes out wrong, and it comes out wrong in the most dangerous
direction: **zero.** If the oracle shares the same blind spot as the solution under
test, both give the same wrong answer, and the comparison finds nothing.
- **AD19** — The oracle must **not be derived from the same idea** as the solution
under test. Brute force carries this property, because it uses no pattern assumption.
- **AD20** — An oracle sharing the same blind spot produces **0 diverging inputs**
against the solution under test; that zero is not a proof of correctness, it is a sign
of dependence.
```python
"""Oracle independence: two solutions sharing the same blind spot give 0 diverging inputs."""
SEED = 20260218
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def random_corpus(seed=SEED, n=40, length=12):
r = generator(seed)
return [(sorted(r(30) - 9 for _ in range(length)), r(41) - 18)
for _ in range(n)]
def scan(array, target):
return any(array[i] + array[j] == target
for i in range(len(array)) for j in range(i + 1, len(array)))
def oracle_correct(array, target):
return scan(array, target)
def oracle_defective(array, target):
"""Same blind spot: the last element is never looked at."""
return scan(array[:-1], target)
def pattern_defective(array, target):
"""Two pointers never sees the last element."""
left, right = 0, len(array) - 2
while left < right:
total = array[left] + array[right]
if total == target:
return True
left, right = (left + 1, right) if total < target else (left, right - 1)
return False
def diverging_count(a, b, items):
return sum(1 for array, target in items if a(array, target) != b(array, target))
R = random_corpus()
print("two solutions compared diverging inputs / 40")
print("defective pattern - correct oracle ", diverging_count(pattern_defective, oracle_correct, R))
print("defective pattern - defective oracle ", diverging_count(pattern_defective, oracle_defective, R))
print("defective oracle - correct oracle ", diverging_count(oracle_defective, oracle_correct, R))
```
```
two solutions compared diverging inputs / 40
defective pattern - correct oracle 3
defective pattern - defective oracle 0
defective oracle - correct oracle 3
```
The same defective pattern, on the same 40 inputs, gives **3** against one oracle and
**0** against another. The zero in the middle row reads like good news, and it is not:
neither the pattern nor the oracle sees the last element, both give the same wrong
answer on the same inputs, and the difference the procedure measures goes blind exactly
where there is no procedural difference to find. The third row shows that the defective
oracle itself is wrong on **3 inputs** — the error was there, the measurement could not
see it.
This is the measured justification for why brute force remains this course's oracle. It
carries no pattern assumption: it does not assume order, sign, range, or uniqueness.
Turning a sped-up version of a pattern into the oracle means carrying the assumption the
speedup rests on straight into the measurement.
#### What Zero Diverging Inputs Proves
The correct version gave 0 diverging inputs in both sets. This does **not prove** the
version is correct, and it cannot be written that way. The last two lines say why not:
the number of sorted, length-12 inputs with values between $-9$ and $20$ is
**7,898,654,920**, and the 52-input set sees **6.6·10-9** of that space.
Considering that the target also takes 41 distinct values, the ratio shrinks further
still.
For this reason, verification's output is two different sentences, and they must not
be conflated. **If diverging inputs is greater than zero:** the solution is wrong, a
proof is in hand, and the proof can be made readable by shrinking. **If diverging
inputs is zero:** no defect was found in this set — the defect is not absent, it was
not found. The strength of the second sentence grows not with the size of the set but
with **the defect classes it covers**; 40 random inputs could not have caught the
empty-input defect even raised to 400, while 12 hand-built inputs caught it in one line.
#### Summary
- The verification procedure has three steps: generate a corpus, run the oracle on the
same inputs, count the inputs where the two answers diverge. The output is a number,
not an opinion.
- A 40-input random corpus catches **four of five** defects; it cannot catch the defect
that does not account for an empty input, because every input in the corpus carries
exactly 12 values and that defect class is structurally unreachable.
- A 12-input edge-case set catches **all five** of the five defects; yield per input is
0.1000 against 0.4167. But it catches the direction-reversed version on only 2
inputs, while the random corpus catches it on 25 — the two sets do not substitute for
each other.
- The second corpus gives the same result (2, 2, 2, 27, 0). A diverging input drops
from 12 values to 1–3 values through single-value removal attempts, taking 20–31
attempts; the broken-end-condition version's smallest diverging input is `[-8]`, one
of the hand-written edge cases.
- If the oracle shares the same blind spot as the solution under test, diverging inputs
drops from 3 to **0**; that zero is not correctness, it is a sign of dependence.
Brute force remains the oracle because it carries no pattern assumption.
- Zero diverging inputs is not proof of correctness: the 52-input set sees
6.6·10-9 of the 7,898,654,920-input space.
#### Next Step
Two lessons tied how a solution is chosen and how it is tested to a number. One
question remains, and it does not concern a single solution: how does someone doing
this work over and over know they are making progress. The next lesson separates the
types of practice environments and counts repeated practice's progress metric over the
shared reference's pattern set. What gets measured is surprising: the number of
problems solved is not a progress metric, and why not is counted separately across
three environment types.
### Using Practice Environments
Source: https://academia.sh/en/courses/advanced-algorithms/using-practice-environments
Counting repeated practice's progress metric over the shared reference's five patterns: an environment that solves 144 problems sees all five patterns too, but its balance ratio is 0.2431, against 0.6667 in an environment that solves 30.
The previous two lessons concerned a single solution: which pattern is chosen, how it
is tested. This lesson carries the question outside a single solution. Practice is
repeated work, and repeated work has to have a progress metric, or the work continues
without anyone knowing where it is going.
Environments are referred to by **type**, because what is measured is not the
environment itself but the **shape of the problem stream** it produces. The result
comes in two steps: neither the number of problems solved nor how many distinct
patterns were seen is a progress metric.
#### Three Environment Types
In a **pool-based practice environment**, a large number of problems sit available and
the person practicing chooses. Because the choice is free, it drifts toward what is
familiar; the pattern distribution reflects the practitioner's habit. In a **timed
contest environment**, the environment determines the set and the practitioner has to
see all of it; the set is small. In an **in-house assessment**, the number of problems
is smallest, the distribution is again not chosen by the practitioner, and the inputs
are not prepared in advance.
- **AD21** — The pattern set is the shared reference's five patterns: two pointers,
sliding window, greedy, backtracking, memoization. These are the patterns measured
throughout the course.
- **AD22** — A round is one practice session spent in an environment. Problems per
round are fixed by environment type: pool 24, contest 5, assessment 2. Six rounds are
measured.
- **AD23** — In the pool-based environment, choice weights are 50, 30, 10, 6, 4; in the
other two environments the distribution is even. The weight models the
practitioner's tendency to return to a familiar pattern.
- **AD24** — Choice is made with the shared reference's deterministic generator;
`random` is not used.
- **AD25** — The **balance ratio** is the count of the least-seen pattern divided by
its share under an even distribution. If all five patterns were seen equally, the
ratio would be 1.0000.
```python
"""Pattern diversity across three environment rounds, over the shared reference's five patterns."""
SEED, SECOND = 20260218, 20260219
PATTERNS = ["two pointers", "sliding window", "greedy", "backtracking", "memoization"]
ENVIRONMENTS = [("pool-based practice ", 24, [50, 30, 10, 6, 4]),
("timed contest ", 5, [20, 20, 20, 20, 20]),
("in-house assessment ", 2, [20, 20, 20, 20, 20])]
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
def choose(r, weights):
x = r(sum(weights))
for i, w in enumerate(weights):
if x < w:
return i
x -= w
return len(weights) - 1
def metrics(counts):
total = sum(counts)
fair_share = total / len(counts)
return {"problems": total, "covered": sum(1 for c in counts if c),
"balance": round(min(counts) / fair_share, 4),
"top_two": round(sum(sorted(counts)[-2:]) / total, 4)}
def run(seed, rounds=6):
print(f"seed {seed} , {rounds} rounds")
print("environment problems covered balance top two distribution")
for ad, per_round, weights in ENVIRONMENTS:
r = generator(seed)
counts = [0] * len(PATTERNS)
for _ in range(rounds * per_round):
counts[choose(r, weights)] += 1
o = metrics(counts)
print(f"{ad} {o['problems']:7d} {o['covered']:8d} {o['balance']:6.4f}"
f" {o['top_two']:7.4f} {counts}")
run(SEED)
print()
run(SECOND)
print()
print("pool-based environment , cumulative round by round (seed 20260218)")
print("round problems covered balance top two")
r = generator(SEED)
counts = [0] * len(PATTERNS)
for t in range(1, 7):
for _ in range(24):
counts[choose(r, ENVIRONMENTS[0][2])] += 1
o = metrics(counts)
print(f"{t:3d} {o['problems']:7d} {o['covered']:8d} {o['balance']:6.4f}"
f" {o['top_two']:7.4f}")
```
```
seed 20260218 , 6 rounds
environment problems covered balance top two distribution
pool-based practice 144 5 0.2431 0.7986 [64, 51, 11, 11, 7]
timed contest 30 5 0.6667 0.5000 [4, 5, 9, 6, 6]
in-house assessment 12 5 0.4167 0.5833 [1, 4, 3, 2, 2]
seed 20260219 , 6 rounds
environment problems covered balance top two distribution
pool-based practice 144 5 0.1389 0.8194 [76, 42, 16, 6, 4]
timed contest 30 5 0.6667 0.5667 [8, 4, 9, 4, 5]
in-house assessment 12 5 0.4167 0.6667 [1, 1, 5, 2, 3]
pool-based environment , cumulative round by round (seed 20260218)
round problems covered balance top two
1 24 4 0.0000 0.8333
2 48 5 0.2083 0.7917
3 72 5 0.2778 0.7639
4 96 5 0.2604 0.7500
5 120 5 0.2500 0.7750
6 144 5 0.2431 0.7986
```
The first table refutes two metrics at once. The pool-based environment solves **144**
problems, the timed contest **30**, the in-house assessment **12** — a 12-times
difference. Yet **the count of covered patterns is 5 in all three**: "how many
problems did I solve" swings 12 times over; "how many distinct patterns did I see"
does not move at all.
The metric that distinguishes them is the distribution. In the pool the balance ratio
is **0.2431**: the least-seen pattern gets under a quarter of its even-distribution
share, and the two most-seen patterns cover **0.7986** of the problems. The timed
contest gives **0.6667** with 30 problems — **2.74 times** more balanced on five times
fewer. The in-house assessment's 0.4167 comes not from the distribution but from the
smallness of the sample.
The third table is the lesson's most contrarian result. In the pool-based environment,
coverage rises to 5 in the **second round** and never changes again: the 72 problems
solved from round three to round six **add nothing** to it. The balance ratio peaks at
0.2778 in the third round and then **falls** — 0.2604 in the fourth, 0.2431 in the
sixth. As problem count rises, the progress metric declines, because every added
problem goes to the two dominant patterns.
The second corpus confirms the ranking: at seed `20260219` the pool gives 0.1389, the
contest 0.6667, the assessment 0.4167. The pool's value drops from 0.2431 to 0.1389 —
**the absolute value depends on the corpus** — but the ordering of the three
environments and the size of the gap between pool and contest run the same way on
both.
#### Does the Environment Show the Error
Distribution alone is not enough either. Applying a pattern fifty times does not show
**what it does when its precondition breaks** — the shared reference's first reading
already proved this: two pointers matches the oracle on 40 of 40 inputs when the
precondition holds, so someone practicing on inputs that satisfy it **never sees** the
pattern's wrongness.
What is measured here is the environment's mix: what percentage of the inputs satisfy
the precondition, and how that ratio changes the number of wrong answers seen. Two
selection policies are compared.
- **AD26** — In the pool-based environment, **90 percent** of inputs satisfy the
precondition; in the timed contest, **75 percent**; in the in-house assessment, **50
percent**. In the pool, problems are prepared according to the pattern; in the
assessment, the input arrives as it is.
- **AD27** — **Habitual selection** applies the most-practiced pattern to every problem
and never tests the precondition.
- **AD28** — **Precondition-checked selection** first checks whether the input is
sorted; if it is not, it does not apply the pattern and falls back to the oracle. The
check is at most $n-1$ steps and stops at the first out-of-order pair.
- **AD29** — 40 problems are solved in each environment, and the target is 11.
```python
"""Two selection policies, against the oracle, across a mix of three environments."""
SEED, SECOND = 20260218, 20260219
def generator(seed):
d = seed
def next_value(n):
nonlocal d
d = (d * 1103515245 + 12345) % 2147483648
return d % n
return next_value
class Counter:
def __init__(self):
self.step = 0
def count(self, n=1):
self.step += n
def mix(sorted_ratio, seed=SEED, n=40, length=12):
"""sorted_ratio: what percentage of inputs satisfy the precondition."""
r = generator(seed)
items = []
for _ in range(n):
array = [r(30) - 9 for _ in range(length)]
items.append(sorted(array) if r(100) < sorted_ratio else array)
return items
def oracle_pairs(array, target, s):
for i in range(len(array)):
for j in range(i + 1, len(array)):
s.count()
if array[i] + array[j] == target:
return True
return False
def pattern_two_pointers(array, target, s):
left, right = 0, len(array) - 1
while left < right:
s.count()
total = array[left] + array[right]
if total == target:
return True
left, right = (left + 1, right) if total < target else (left, right - 1)
return False
def habitual_selection(array, target, s):
"""Applies the most-practiced pattern to every problem."""
return pattern_two_pointers(array, target, s)
def precondition_checked_selection(array, target, s):
"""Tests the precondition first; falls back to the oracle if it does not hold."""
for i in range(len(array) - 1):
s.count()
if array[i] > array[i + 1]:
return oracle_pairs(array, target, s)
return pattern_two_pointers(array, target, s)
def measure(policy, items, target=11):
diverging, policy_steps, oracle_steps = 0, 0, 0
for array in items:
s1, s2 = Counter(), Counter()
a = policy(list(array), target, s1)
b = oracle_pairs(list(array), target, s2)
policy_steps, oracle_steps = policy_steps + s1.step, oracle_steps + s2.step
if a != b:
diverging += 1
return diverging, policy_steps, oracle_steps
ENVIRONMENTS = [("pool-based practice ", 90), ("timed contest ", 75),
("in-house assessment ", 50)]
for seed in (SEED, SECOND):
print(f"seed {seed} , 40 problems per environment , target 11")
print("environment sorted habitual precondition check oracle")
print(" inputs diverging step diverging step step")
for ad, ratio in ENVIRONMENTS:
K = mix(ratio, seed)
s = sum(1 for d in K if d == sorted(d))
a1, k1, h1 = measure(habitual_selection, K)
a2, k2, _ = measure(precondition_checked_selection, K)
print(f"{ad} {s:6d} {a1:7d} {k1:4d} {a2:7d} {k2:4d} {h1:4d}")
print()
```
```
seed 20260218 , 40 problems per environment , target 11
environment sorted habitual precondition check oracle
inputs diverging step diverging step step
pool-based practice 34 5 165 0 600 834
timed contest 29 8 199 0 681 914
in-house assessment 17 15 255 0 754 856
seed 20260219 , 40 problems per environment , target 11
environment sorted habitual precondition check oracle
inputs diverging step diverging step step
pool-based practice 36 2 213 0 609 1055
timed contest 31 4 238 0 691 1079
in-house assessment 22 10 296 0 823 1161
```
Three numbers side by side. **Oracle:** 834 steps in the pool mix, correct on 40 of 40
inputs. **Pattern:** habitual selection 165 steps, precondition-checked selection 600
steps. **Diverging inputs:** habitual selection gives **5** in the pool, **8** in the
contest, **15** in the assessment; the precondition check gives **0** in all three.
The first reading concerns the environment. Same policy, same pattern, same problem
count — yet the wrong answers it sees rise from 5 to 15, **three times over**. The
cause is the environment's input mix: 34 of 40 pool inputs satisfy the precondition,
against 17 in the assessment. Working with prepared inputs means seeing a pattern's
wrongness less, and seeing it less is not the same as fixing it.
The second reading concerns the policy. The precondition check gives **0 diverging
inputs** in all three environments; the cost is rising from 165 to **600 steps** in the
pool mix, that is, **3.64 times**. It still stays below the oracle's 834 steps:
correctness is bought not at brute force's price, but at **72 percent** of it. The
second corpus gives the same ranking (2, 4, 10) and the same 0 diverging inputs; the
absolute numbers depend on the corpus, the ranking does not.
What follows is repeated practice's progress metric, made of three numbers: **how many
patterns were seen**, **the distribution's balance ratio**, and **how many were seen
with their precondition broken**. Problems solved gives none of these three.
#### Summary
- Environments separate by type: in pool-based practice the practitioner chooses the
distribution; in the timed contest and in-house assessment the environment chooses
and the set is small.
- The number of problems solved separates the environments by a factor of 12 (144, 30,
12), but the covered-pattern count is 5 in all three; neither metric shows progress.
- The distinguishing metric is the distribution: in the pool the balance ratio is
0.2431 and the two dominant patterns cover 0.7986 of the problems; the timed contest
gives 0.6667 with five times fewer problems.
- Coverage rises to 5 in the second round and freezes; the balance ratio peaks at
0.2778 in the third round and then **declines** to 0.2431 — adding problems can make
the metric worse.
- The environment's input mix determines how much wrongness is seen: the same policy
produces 5, 8, and 15 diverging inputs in the pool, contest, and assessment.
- Precondition-checked selection gives 0 diverging inputs in all three environments;
its cost rises from 165 to 600 steps, 72 percent of the oracle's 834.
#### Course Wrap-Up
Twenty-three lessons asked a single question: what does choosing a pattern mean
accepting. Every lesson answered with three numbers. The **oracle** was always brute
force and always correct — but **not always expensive**: the Longest Path Problem
lesson measured the oracle at 562 steps against the pattern's 3626 on eight vertices,
and showed the threshold sits at ten. The **pattern** usually spent fewer steps.
**Diverging inputs** counted how many inputs the pattern answered differently than the
oracle. The course's rule: a pattern's number is not the steps it saves, but **how many
wrong answers it gives once its precondition breaks**.
| Lesson | Oracle (steps) | Pattern (steps) | Diverging inputs / cost |
|---|---|---|---|
| Brute Force | exhaustive count **2640** | early exit 866 | early exit **0**; sampling **14** at 406 |
| Divide and Conquer | **3120** | linear combine step **2680** | **0**; incomplete combine step **40/40** at 920 |
| Greedy Algorithms | bottom-up | greedy | **827 of 969 systems** (0.8535); largest excess **16 coins** |
| Dynamic Programming | **163,840** | full-key memoization **2440** (67.15 times fewer) | **0**; missing key **39/40**; with no overlap, 19 memo entries wasted |
| Backtracking | without pruning **960,800** nodes, n=7 | with pruning **552** nodes | **1740.6 times**, ratio grows with n (72.3 → 8156.2); over-pruning gives 0 solutions |
| Randomization | **780** | verified sampling **944–1056** | **0**; unverified, at 200, **40/40** once the precondition breaks |
| Two Pointers | **972** | **154** (6.31 times fewer) | **25/40** unsorted, speedup drops to **2.33 times** |
| Sliding Window | **2396** | **867** (2.76 times fewer) | **10/40** with negatives, in **fewer steps** |
| Fast and Slow Pointer | **480** | **299** | **14/40** on the second edge case; steps **never change** |
| Interval Merging | sorting by start: **40/40** correct | **2154** | **29/40** sorting by end, **40/40** by length and unsorted |
| Cyclic Sort | 1..n, no duplicates | comparison-free | with duplicates, plain **40/40**; guarded **39/40**; missing case **0/40** |
| Two Heaps | **480** median | with rebalancing step **1261** | unbalanced **295/480** wrong; rebalancing is **more than half** the pattern (1261/550) |
| K-th Element | four k values x 40 inputs | k-sized heap | **0/40**; "k-th distinct value" is a separate question: **27/40** |
| Grid Traversal | same connectivity **40/40** | connected component | **38/40** once the definitions diverge |
| Knapsack | fractional variant **100,800** | greedy **174** | fractional **0/40**; the same order in 0/1 gives **5/40**, loss 7 |
| Travelling Salesman | **352,800** (20 samples) | nearest neighbor **560** | **17/20**; average deviation 9.3, **worst 33.33**; two-opt 2/20 |
| Longest Path | acyclic **40/40** | relaxation procedure | with a cycle **38/40**; largest overestimate **228** |
| N Queens | with pruning **2057** nodes, n=8 | with symmetry breaking **1029** | ratio **2.00**, constant across five sizes; pruning's grows, symmetry's **does not** |
| Knight's Tour and Maze | path model **90,111** | cell model **25** | ratio **3604**, diverging **0** |
| Hamiltonian Paths | counting **29,270** | decision **1526** | ratio **19.18**; in every case with no path, decision = counting |
| Problem Reading and Constraint | **866** | hash table **486**, sorting pattern **1398**, counting **396** | the wrong constraint reading diverges on **18 of 40** inputs (17 on the second corpus); at n=108 none of the three solutions fit the budget |
| Solution Verification | brute force, **1596** across 52 inputs | correct two pointers **329** | 40 random inputs catch **4 of 5** defects, 12 edge cases catch **all 5**; against an oracle sharing the same defect, diverging drops from **3 to 0** |
| Practice Environments | **834** | habitual selection **165**, precondition check **600** | **5** diverging in the pool, **8** in the contest, **15** in the assessment; precondition check gives 0 diverging but **3.64 times** the steps |
The table's second reading matters more than its first: **speedup brings a pattern
closer to being wrong, and sometimes it does not even speed things up.** Two pointers'
speedup drops to 2.33 times on an unsorted input while it is wrong on 25 inputs; greedy
over-answers on 827 of 969 systems; memoization wastes 19 entries with no overlap. None
of this was hidden: not knowing when a pattern goes wrong means not knowing when to
choose it.
The third and lasting result concerns the oracle itself: **brute force is not a
baseline, it is an oracle.** Every number in the course was written because of it. In
the final lesson this became a habit: counting correctness against a second solution's
answer, not an opinion.
The next course, **Theory of Computation**, takes up the question left in the oracle's
shadow. The traveling salesman and Hamiltonian paths lessons explicitly deferred "why
is this problem hard." What will be asked there is whether not knowing anything better
than brute force is, for some problems, a fact about our knowledge or about the
problems themselves. The answer comes through decidability, reduction, and complexity
classes.