Skip to content
academia.sh
tr
Log in

Unit 01 of 02

Embeddings and Vector Search

Measuring which chunk of the corpus a query retrieves and what each layer of the search scaffolding gains and loses: the retrieval precision and retrieval recall of keyword matching against embedding similarity on the same query set, the difference in retrieved chunks between an in-domain and an out-of-domain embedding at the same dimension, in how many queries cosine similarity and the dot product separate the first three results and in which one the length bias appears, the recall of an indexed search against the exhaustive search baseline read side by side with its comparison count, the neighborhood breaking down in a space that narrows as filter selectivity rises and the candidate count dropping, and the comparison of keyword-only, embedding-only and combined setups together with the query class where the combined setup stays worse than both.

Published Updated

Lessons

  1. 01 What Is an Embedding Deciding which document in the corpus a query resembles by meaning rather than by words, and counting that decision with two separate measures. Forty queries are run on a fictional corpus of two hundred documents, 693 paragraphs and twenty concepts: keyword matching leaves a retrieval precision of 0.0800 and a retrieval recall of 0.0363, while embedding similarity gives 0.4700 and 0.2258; the first result's concept hit rate rises from 0.0250 to 0.5000 and its class hit rate from 0.1000 to 1.0000. The ceiling on recall for five chunks is 0.4904, and reading the class hit rate alone hides the half share in the concept hit rate. The cost is 8000 comparisons, 240 embedding calls and a stored value count of 6400. 11 min read
  2. 02 Embedding Models Placing two of an embedding model's three settings side by side on the same workload: dimension and domain fit. At the same thirty-two dimensions, the same 6400 stored values and the same forty queries, the in-domain embedding gives a retrieval precision of 0.4700 and a concept hit rate of 0.5000, while the out-of-domain embedding falls to 0.0650 and 0.0500; on concept and class hit rate it rises only a step above the keyword search baseline, while on the retrieved set it drops below the baseline. With a second embedding model of the same family the difference stays at 0.0250 in concept hit rate while the loss from falling out of domain is 0.4500; the first is unmeasured, the second is measured. In the dimension sweep the two models do not agree between 64 and 8 on which dimension is good, and the cliff appears only between 8 and 4. 12 min read
  3. 03 Similarity Metrics Ranking the same vectors with two separate measures and how much the choice changes the retrieved chunk. The dot product is the cosine similarity multiplied by the norms of the two vectors; for one query the query norm is constant, so the entire ranking difference comes from the norm of the document vector, and this norm is 4.6737 in the shortest quarter and 5.4708 in the longest. In thirty of the forty queries the first three results diverge; cosine similarity gives a retrieval precision of 0.4700 and a concept hit rate of 0.5000 while the dot product stays at 0.4100 and 0.4250, and the same direction appears in a second embedding model. The length bias is in the dot product: the mean of the retrieved set is 66.2 tokens, 57.7 under cosine similarity and 64.1 in the corpus; since the mean of the relevant documents is also 64.7, length carries no relevance. 10 min read
  4. 04 Vector Databases Removing the assumption that a query is compared with the whole corpus and measuring the price of approximation. On two hundred fictional documents, exhaustive search needs 200 comparisons per query; the bucket-based index holds recall at 0.9500 with 16 buckets and 2 probes and brings the comparisons down to 47.2, and with 4 probes recall is 1.0000 and comparisons are 69.2. With 1 probe recall falls to 0.7200 and the concept precision of the first five falls from 0.4700 to 0.3800: the index damages not only the candidate set but also the answer. The neighborhood graph at degree 8 can reach only 27 documents from the entry node and its recall is 0.0800; at degree 48 it stores 9600 edges and, with 84.9 comparisons, stays at 0.7650, that is, both more expensive and worse than the bucket-based index. The difference between two index builds is 0.0100, and this is the threshold of this workload. 14 min read
  5. 05 Metadata Filtering Combining vector search with a metadata constraint and measuring how the neighborhood breaks down as the filter narrows. Two hundred fictional documents are distributed over sixteen regions; the unfiltered bucket-based index gives a recall of 0.9500 with 47.2 comparisons. When the filter is applied after the search, recall falls to 0.4450 and to 0.0900, and forty of the forty queries return fewer than five results; retrieval raised to fifty pulls this up to 0.8900 and 0.4750 but cannot exceed the ceiling of in-index filtering. In in-index filtering, as the filter width goes from 16 down to 1, the candidate count falls from 31.2 to 2.8 and recall from 0.9500 to 0.4750; the comparisons fall from 47.2 to 18.8 while the exhaustive search of the filtered set falls from 200 to 18. At width 1 the index gives 0.4750 with 18.8 comparisons and exhaustive search gives 1.0000 with 18 comparisons. The difference between two index builds is 0.0100 without a filter and between 0.0850 and 0.1300 with one. 2 min read
  6. 06 Hybrid Search Measuring keyword matching and embedding search separately and together. Three setups run side by side on three fictional query classes: in subscriber language keyword search is 0.0250 and embedding is 0.5000; in document language both are 0.5000; on the document code query keyword search is 1.0000 and embedding is 0.0000. Rank fusion gives 0.4000 in the document language class, that is, worse than both; this holds at constants of 5, 20 and 60. In the code class fusion drops the share of keeping the target document in the first five from 1.0000 to 0.0500. The reason is the number of documents shared by the first five of the two lists: 0.12 in subscriber language, 1.05 in document language. A document that stands second in both lists passes a document that is first in one and absent from the other. Fusion raises recall up to rank twenty from 0.7122 to 0.7214; the place it gains is depth, the place it loses is the first rank. 15 min read

Start typing to search.

↑↓ ↵ Esc navigate · open · close