Score Tuning

The previous lesson broke the relevance score into its components: how many times a term appears in a document, how many documents it appears in, and how long the document is. The decomposition was done, but the weight of the components relative to one another was taken as given; every field counted the same, and that was never questioned as a decision. In the catalog, the question is concrete: for a reader searching “ocean ship,” is a word appearing in a book’s title the same as appearing in its summary? What does appearing in the author’s name mean?

This lesson measures two adjustments. Field weight is the coefficient that enters the score depending on which field the same term is found in. Function-based scoring is the inclusion in the score of a quantity unrelated to the text — publication year, loan count. Both are defended with the justification of “a more relevant result”; here that justification is converted into a number.

Corpus and Four Fields

The entire measurement runs on a single corpus: 1,200 book records generated from a library catalog, seed 20260801. Each record is indexed in four fields.

Code	Assumption	Value	Rationale
QR10	analyzer chain	established in the previous topic	tokens arrive normalized
QR11	indexed fields	title, summary, subject, author	each with its own posting list
QR12	subject tag	differs from the text’s theme in a quarter of records	the tag is a publisher decision
QR13	loan count and year	sits in the index as a document field	input to function-based scoring

QR12 is this lesson’s distinguishing assumption: if the fields were copies of one another, weight would move nothing.

// corpus.mjs — library catalog: seeded corpus, inverted index, and per-field scorer.
export const N = 1200, SEED = 20260801, FIELDS = ["title", "summary", "subject", "author"];
let d = SEED;                                             // 32-bit generator, no overflow
export const random = () => { d = (d + 0x6D2B79F5) | 0; let t = Math.imul(d ^ (d >>> 15), 1 | d);
  t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t; return ((t ^ (t >>> 14)) >>> 0) / 2 ** 32; };
const pick = (a) => a[Math.floor(random() * a.length)];
const THEMES = [["story|tale", "literature", "short selection collection narrative fiction rural"],
  ["fable", "children's literature", "child illustrated school youth forest sleep"],
  ["history", "history", "ottoman republic archive document chronicle foundation"],
  ["ocean", "travel", "coast fisherman harbor ship island lighthouse"],
  ["mathematics", "science", "geometry number proof theory solution probability"],
  ["poetry", "poetry", "verse collected divan translation selection meter"]]
  .map(([k, subject, a]) => ({ k: k.split("|"), subject, core: a.split(" ") }));
const COMMON = "book volume edition publication review notes introduction glossary".split(" ");
const FIRST_NAMES = "James Emma Simon Nora Kevin Ocean Grace Owen Meredith Blake".split(" ");
const SURNAMES = "Stone Bishop Miller Hayes Reed Frost".split(" ");

export const documents = Array.from({ length: N }, (_, id) => {
  const t = pick(THEMES), draw = t.k[t.k.length > 1 && random() < 0.5 ? 1 : 0], sm = [draw, draw];
  const title = [...new Set([draw, pick(t.core), pick(t.core)])];
  t.core.forEach((s, j) => { if (random() < 1 / (1 + j * 0.42)) sm.push(s); });
  COMMON.forEach((s, j) => { if (random() < 0.55 / (1 + j * 0.28)) sm.push(s); });
  if (random() < 0.5) sm.push(pick(pick(THEMES).core));   // word leaking from another theme
  for (let k = sm.length - 1; k > 0; k -= 1) { const j = Math.floor(random() * (k + 1));
    [sm[k], sm[j]] = [sm[j], sm[k]]; }
  // the subject tag is a publisher decision: a quarter of documents do not match the text's theme
  return { id, title: title.join(" "), summary: sm.join(" "),
    subject: (random() < 0.25 ? pick(THEMES) : t).subject,
    author: `${pick(FIRST_NAMES)} ${pick(SURNAMES)}`, year: 1975 + Math.floor(random() ** 0.6 * 50),
    loans: Math.floor(random() ** 3 * 400) };
});

export const tokenize = (s) => s.toLocaleLowerCase("en-US").match(/[\p{L}\p{N}]+/gu) ?? [];
export function buildIndex(bs) {                          // per-field inverted index
  const idx = {};
  for (const a of FIELDS) {
    const postings = new Map(), lengths = new Float64Array(bs.length);
    for (const b of bs) {
      const ts = tokenize(b[a]), count = new Map();
      lengths[b.id] = ts.length;
      for (const t of ts) count.set(t, (count.get(t) ?? 0) + 1);
      for (const [t, tf] of count) {
        if (!postings.has(t)) postings.set(t, []);
        postings.get(t).push({ id: b.id, tf });
      }
    }
    idx[a] = { postings, lengths, avgLength: lengths.reduce((x, y) => x + y, 0) / bs.length };
  }
  return idx;
}
export function fieldScore(idx, a, terms) {               // tf, inverse document frequency, length
  const k1 = 1.2, b = 0.75, p = new Map();
  for (const t of terms) {
    const g = idx[a].postings.get(t);
    if (!g) continue;
    const idf = Math.log(1 + (N - g.length + 0.5) / (g.length + 0.5));
    for (const e of g) {
      const norm = 1 - b + (b * idx[a].lengths[e.id]) / idx[a].avgLength;
      p.set(e.id, (p.get(e.id) ?? 0) + (idf * e.tf * (k1 + 1)) / (e.tf + k1 * norm));
    }
  }
  return p;
}
export function search(idx, query, weight) {              // total score with field weights
  const terms = tokenize(query), totals = new Map();
  for (const a of FIELDS) {
    if (!weight[a]) continue;
    for (const [id, v] of fieldScore(idx, a, terms))
      totals.set(id, (totals.get(id) ?? 0) + weight[a] * v);
  }
  return [...totals].sort((x, y) => y[1] - x[1] || x[0] - y[0]);
}

The scorer runs separately per field: each field has its own posting list, its own average length, and its own inverse document frequency; the total score is the weighted sum of the field scores. If the weight is zero, that field is not scanned at all.

Weight Changes Order, Field Selection Changes the Set

Six configurations run the same query. Four only change weight; two narrow the set of fields scanned.

// score.mjs — same query, changing field weights: set, top ten, rank shift, and cost.
import { documents, buildIndex, search, fieldScore, tokenize, FIELDS, N, SEED } from "./corpus.mjs";
const idx = buildIndex(documents), QUERY = "ocean ship", terms = tokenize(QUERY);
const touched = (a) => terms.reduce((s, t) => s + (idx[a].postings.get(t)?.length ?? 0), 0);
const CONFIGS = { "equal": { title: 1, summary: 1, subject: 1, author: 1 },
  "title 5x": { title: 5, summary: 1, subject: 1, author: 1 },
  "title 12x": { title: 12, summary: 1, subject: 1, author: 1 },
  "author 5x": { title: 1, summary: 1, subject: 1, author: 5 },
  "title only": { title: 1 }, "summary only": { summary: 1 } };

const base = search(idx, QUERY, CONFIGS["equal"]);
const rank = (r) => new Map(r.map(([id], i) => [id, i + 1])), baseRank = rank(base);
const baseTop = new Set(base.slice(0, 10).map(([id]) => id));
console.log(`corpus ${N} documents, seed ${SEED}; query "${QUERY}"`);
console.log("field term count / average length: " +
  FIELDS.map((a) => `${a} ${idx[a].postings.size}/${idx[a].avgLength.toFixed(1)}`).join(", "));
console.log("posting entries touched: " + FIELDS.map((a) => `${a} ${touched(a)}`).join(", ") + "\n");
console.log("configuration".padEnd(14) + "matched".padStart(9) + "entries".padStart(9) +
  "top10 changed".padStart(15) + "largest jump".padStart(21) + "  top result");
for (const [label, w] of Object.entries(CONFIGS)) {
  const r = search(idx, QUERY, w), y = rank(r);
  const changed = r.slice(0, 10).filter(([id]) => !baseTop.has(id)).length;
  let maxJump = 0, jumpDoc = -1;
  for (const [id] of base.slice(0, 50))
    if (y.has(id) && Math.abs(baseRank.get(id) - y.get(id)) > maxJump) {
      maxJump = Math.abs(baseRank.get(id) - y.get(id)); jumpDoc = id;
    }
  const b = documents[r[0][0]];
  console.log(label.padEnd(14) + `${r.length}`.padStart(9) +
    `${Object.keys(w).reduce((s, a) => s + touched(a), 0)}`.padStart(9) +
    `${changed}`.padStart(15) + (jumpDoc < 0 ? "base" : `${maxJump} (document ${jumpDoc})`).padStart(21) +
    `  ${b.title} / ${b.author}`);
}

const components = (id) => FIELDS
  .map((a) => `${a} ${(fieldScore(idx, a, terms).get(id) ?? 0).toFixed(2)}`).join(", ");
const authorTop = search(idx, QUERY, CONFIGS["author 5x"])[0][0];
console.log(`\nwith author 5x, document ${authorTop} reaches the top: ${documents[authorTop].title} / ` +
  `${documents[authorTop].author}\n  ranked ${baseRank.get(authorTop)} under equal weight; ` +
  `components: ${components(authorTop)}`);
console.log(`top document under equal weight ${base[0][0]}: ${documents[base[0][0]].title} / ` +
  `${documents[base[0][0]].author}\n  components: ${components(base[0][0])}`);

corpus 1200 documents, seed 20260801; query "ocean ship"
field term count / average length: title 42/2.9, summary 50/8.4, subject 7/1.3, author 16/2.0
posting entries touched: title 265, summary 302, subject 0, author 131

configuration   matched  entries  top10 changed         largest jump  top result
equal               325      698              0                 base  ocean coast ship / Owen Miller
title 5x            325      698              4    44 (document 365)  ocean ship / Blake Reed
title 12x           325      698              4    44 (document 365)  ocean ship / Blake Reed
author 5x           325      698             10   131 (document 143)  ocean ship coast / Ocean Bishop
title only          199      265              9  144 (document 1122)  ocean ship / Simon Miller
summary only        211      302              8   159 (document 852)  ocean lighthouse fisherman / Owen Reed

with author 5x, document 622 reaches the top: ocean ship coast / Ocean Bishop
  ranked 17 under equal weight; components: title 4.59, summary 2.69, subject 0.00, author 2.21
top document under equal weight 143: ocean coast ship / Owen Miller
  components: title 4.59, summary 5.46, subject 0.00, author 0.00

Document contents depend on the seed; the number of posting entries touched does not — that number comes directly from the length of the posting lists.

The first column is the same across four rows: 325. Changing weight does not touch the matching set, because weight is a multiplier that enters the score, not a matching condition. A document is in the set if it carries at least one query term; weight only decides where it lands. In the last two rows the set drops to 199 and 211, because there what changes is not weight but which fields are scanned. Searching only the title field drops 126 documents whose title carries none of the query terms. Weight and field selection are talked about as if they were the same adjustment, but one changes order and the other changes the set.

On the order side, the first surprise is that the title 5x and title 12x rows are identical: four documents change in the top ten, the largest jump is forty-four ranks, and raising the coefficient two and a half times further adds nothing. As weight grows, the ranking converges on the title field’s own internal order and stops there; past a certain point, an instruction to raise it further has no effect.

The second surprise is in the author 5x row: all ten of the top ten change. The reason appears in the output’s lower section. In the author field, “Ocean” is a name; document 622 sits seventeenth under equal weight, but multiplying the author component by five carries it to first place, and document 143 — first under equal weight — has an author component of zero, so it drops to rank 132. A single coefficient, a hundred thirty-one ranks. Field weight is not increasing relevance here; it is amplifying field collision.

The subject field stays silent throughout the row: posting entries touched is zero, because “ocean” and “ship” appear in no subject tag. Raising the subject weight to a hundred changes nothing in this query; a weight’s effect depends on the query term being present in that field.

Function-Based Scoring and the Suppression Point

The second adjustment comes from outside the text. The library wants “new and heavily borrowed books to surface higher.” This request can enter the score in two forms: multiplied with the text score or added to the text score. Both are implementations of the same request, and their results are not the same.

// function-score.mjs — function-based score: two combination modes and relevance's suppression point.
import { documents, buildIndex, search } from "./corpus.mjs";
const idx = buildIndex(documents), QUERY = "ocean ship";
const textScore = search(idx, QUERY, { title: 1, summary: 1, subject: 1, author: 1 });
const PEAK = textScore[0][1];
const textRank = new Map(textScore.map(([id], i) => [id, i + 1]));
const textTop = new Set(textScore.slice(0, 10).map(([id]) => id));
const factor = (b) => 0.6 * (Math.log1p(b.loans) / Math.log1p(400)) + 0.4 * ((b.year - 1975) / 49);
const topTen = (f) => textScore.map(([id, p]) => [id, f(p, documents[id])])
  .sort((x, y) => y[1] - x[1] || x[0] - y[0]).slice(0, 10);
const counts = (r) => `${r.filter(([id]) => !textTop.has(id)).length}/` +
  `${r.filter(([id]) => textRank.get(id) > 50).length}`;
const avg = (r, f) => (r.reduce((s, [id]) => s + f(documents[id]), 0) / r.length).toFixed(0);

console.log(`query "${QUERY}", ${textScore.length} matching documents; set is the same at every weight`);
console.log(`highest text score ${PEAK.toFixed(2)}; top ten average loan count ` +
  `${avg(textScore.slice(0, 10), (b) => b.loans)}, average year ` +
  `${avg(textScore.slice(0, 10), (b) => b.year)}\n`);
console.log("w".padEnd(6) + "multiplicative".padStart(16) + "additive".padStart(11) +
  "avg loans".padStart(11) + "avg year".padStart(9) + "  top result in additive mode");
for (const w of [0.25, 0.5, 1, 2, 4, 8]) {
  const c = topTen((p, b) => p * (1 + w * factor(b))), t = topTen((p, b) => p + w * PEAK * factor(b));
  const b = documents[t[0][0]];
  console.log(`${w}`.padEnd(6) + counts(c).padStart(16) + counts(t).padStart(11) +
    avg(t, (x) => x.loans).padStart(11) + avg(t, (x) => x.year).padStart(9) +
    `  ${b.title} (${b.year}, ${b.loans} loans, text rank ${textRank.get(b.id)})`);
}

query "ocean ship", 325 matching documents; set is the same at every weight
highest text score 10.05; top ten average loan count 100, average year 2006

w       multiplicative   additive  avg loans avg year  top result in additive mode
0.25               4/0        5/0        185     2013  ocean coast ship (2008, 235 loans, text rank 1)
0.5                5/0        6/0        191     2013  ocean ship fisherman (2013, 398 loans, text rank 14)
1                  6/0        6/0        191     2013  ocean ship fisherman (2013, 398 loans, text rank 14)
2                  6/0        7/2        198     2017  ocean ship fisherman (2013, 398 loans, text rank 14)
4                  6/0        8/5        257     2018  ocean ship fisherman (2013, 398 loans, text rank 14)
8                  6/0       10/8        324     2021  ocean coast fisherman (2024, 301 loans, text rank 123)

The two numbers in the columns read as follows: how many of the top ten results were not in the text ranking’s top ten, and how many of those came after rank fifty in the text ranking.

The multiplicative mode freezes once w passes one: four, five, six, and it stays there; the second number never leaves zero at any weight. The reason lies in the multiplier’s structure: because the text score remains a factor, the ranking converges on the product of the text score and the coefficient, and a document that is very weak in the text cannot win that product. The multiplicative mode reorders the top ten; it does not refill it.

In additive mode the contribution is added as a magnitude independent of the text score, and as w grows the text score dissolves into the sum. At w four, five of the top ten come from beyond rank fifty in the text ranking; if a countable criterion is needed, this is the suppression point — half the top ten is now selected not by the query but by loan count. At w eight the top ten changes completely, the top-ranked document sits at rank one hundred twenty-three in the text ranking, the top ten’s average loan count rises from 100 to 324, and the average year rises from 2006 to 2021. The list is still the answer to the “ocean ship” query, and every one of its documents carries one of the query terms; but the order is now the order of the library’s most heavily borrowed new books.

The set stays fixed at 325 across all these weights: function-based scoring adds or removes no document. Once one considers that what the user actually sees is the top ten, the distinction thins — even though the set does not change, the seen set has changed completely.

Summary

• Field weight does not touch the matching set: all four weight configurations returned 325 documents. What changes the set is field selection, not weight; searching only the title field dropped the set to 199.
• Weight’s effect saturates: raising title weight from 5x to 12x left the change in the top ten at four documents and the largest jump at forty-four ranks. Weight does not change the scan cost either; the only decision that drops 698 posting entries to 265 is closing a field.
• Weight given to the wrong field amplifies field collision, not relevance: when author weight rose to five, all ten of the top ten changed, and the document that was first under equal weight dropped to rank 132. The subject field, in contrast, does no work at all in this query, because the posting entries touched is zero.
• In function-based scoring, multiplicative mode only reorders the top ten; additive mode fills half the top ten from beyond rank fifty in the text ranking at w four, and at w eight the top-ranked document sits at rank one hundred twenty-three in the text ranking.

Next Step

Every question asked so far wanted a document list: which books, in what order. What the loan desk asks is often not a document but a number: how many of this search’s results were published after 2015, how many books fall under each subject heading, what is the matching set’s average publication year. These questions run on the same inverted index, but their output is not a sorted list, and whether the answer is computed over the matching set or the whole corpus changes the result entirely. The next lesson takes up queries that produce numbers instead of documents, and their memory cost.