N+1 Problem and Batch Loading

The Resolvers lesson left a number unaddressed: three member calls were made for three loan records, yet there were two distinct members. Fragments and Variables did not touch this number; field merging only removes collisions at the same response key, it does not remove different records asking for the same member.

This lesson shows the problem at scale and solves it. The data source holds fifty loan records, eight members, and twelve books.

// data.mjs — loan data generated for scale; every data source round trip is counted
export const roundTrips = { book: 0, member: 0, loan: 0 };
export const recordsFetched = { book: 0, member: 0 };
export const reset = () => { roundTrips.book = roundTrips.member = roundTrips.loan = 0; recordsFetched.book = recordsFetched.member = 0; };

const AUTHORS = ["Cormen", "Knuth", "Kernighan", "Dijkstra", "Hoare", "Lamport"];
export const BOOKS = new Map(Array.from({ length: 12 }, (_, i) => {
  const isbn = `978-000000${String(i).padStart(2, "0")}`;
  return [isbn, { isbn, title: `Book ${i}`, author: AUTHORS[i % AUTHORS.length], copies: 1 + (i % 3) }];
}));
export const MEMBERS = new Map(Array.from({ length: 8 }, (_, i) => {
  const id = `U-${1001 + i}`;
  return [id, { id, createdAt: "2025-01-01", name: `Member ${i}` }];
}));
export const LOANS = new Map(Array.from({ length: 50 }, (_, i) => {
  const id = `O-${i + 1}`;
  return [id, { id, createdAt: "2026-01-01", memberId: `U-${1001 + (i % 8)}`,
    status: i % 3 === 0 ? "CLOSED" : "OPEN", returnDate: "2026-04-01",
    items: [{ isbn: `978-000000${String(i % 12).padStart(2, "0")}`, branch: "central" }] }];
}));

// Single-key accesses: every call is one round trip.
export const getBook = (isbn) => { roundTrips.book++; recordsFetched.book++; return BOOKS.get(isbn); };
export const getMember = (id) => { roundTrips.member++; recordsFetched.member++; return MEMBERS.get(id); };
export const listLoans = () => { roundTrips.loan++; return [...LOANS.values()]; };

// Batch accesses: more than one key in a single round trip.
export const getBooks = (isbns) => { roundTrips.book++; recordsFetched.book += isbns.length; return isbns.map((i) => BOOKS.get(i)); };
export const getMembers = (ids) => { roundTrips.member++; recordsFetched.member += ids.length; return ids.map((i) => MEMBERS.get(i)); };

The parser, schema, flattener, and engine are used exactly as they were written in the previous lessons; what is new is the data source and the loader layer.

The Name of the Problem

A query that asks for fifty loan records together with their members reaches the data source fifty-one times: once to fetch the list, fifty times to fetch each record’s member. When the book field is added, fifty more round trips. This is called the N+1 problem: one list query plus as many single-record queries as the list is long.

The problem’s source is the resolver model. The Loan.member resolver sees a single loan record and fetches that record’s member; it does not know it has forty-nine siblings. Every resolver seeing only its own source is the resolver model’s strength — and it is also the cause of this problem.

The solution is not to change the resolvers, but to put a layer between them. A loader does not fetch the keys handed to it right away; it collects the keys requested within the same tick and reduces them to a single batch call. Its second job is deduplication: if the same key is requested twice, it is fetched once.

// loader.mjs — reduces keys requested in the same tick to a single batch call
// Its second job is deduplication: the same key requested twice is fetched once.
export function createLoader(batchFetch) {
  let pending = new Map();          // key -> [resolve, ...]
  let scheduled = false;

  const flush = async () => {
    const batch = pending; pending = new Map(); scheduled = false;
    const keys = [...batch.keys()];
    const values = await batchFetch(keys);
    keys.forEach((k, i) => batch.get(k).forEach((resolve) => resolve(values[i])));
  };

  return (key) => new Promise((resolve) => {
    if (!pending.has(key)) pending.set(key, []);
    pending.get(key).push(resolve);
    if (!scheduled) { scheduled = true; queueMicrotask(flush); }
  });
}

The batching window is set by the queueMicrotask call: every key that arrives before the currently running tick finishes falls into the same batch. The level-ordered execution measured in the Resolvers lesson pays off here — the fifty Loan.member calls at the same level were already started together, so all of them enter the same window.

Measurement

// run.mjs — runs the same query with single-key and batch resolvers, compares round trips
import { parse } from "./parser.mjs";
import { execute } from "./executor.mjs";
import { SCHEMA } from "./schema.mjs";
import { createLoader } from "./loader.mjs";
import { listLoans, getBook, getMember, getBooks, getMembers, roundTrips, recordsFetched, reset } from "./data.mjs";

const QUERY = `{ loans { id status member { name } items { branch book { title author } } } }`;

const SINGLE = {
  Query: { loans: () => listLoans() },
  Loan: { member: (o) => getMember(o.memberId) },
  Item: { book: (k) => getBook(k.isbn) },
};

// Batch resolvers pull their loaders from the per-request context.
const BATCH = {
  Query: { loans: () => listLoans() },
  Loan: { member: (o, _, c) => c.loadMember(o.memberId) },
  Item: { book: (k, _, c) => c.loadBook(k.isbn) },
};

async function measure(title, resolvers, makeContext) {
  reset();
  const calls = [];
  const s = await execute(SCHEMA, resolvers, parse(QUERY), { context: makeContext(), tracer: (b) => calls.push(b) });
  const total = roundTrips.book + roundTrips.member + roundTrips.loan;
  console.log(`${title.padEnd(9)} resolver calls=${String(calls.length).padStart(3)}  ` +
    `data source round trips: loan=${roundTrips.loan} member=${String(roundTrips.member).padStart(2)} book=${String(roundTrips.book).padStart(2)} total=${String(total).padStart(3)}  ` +
    `records fetched: member=${recordsFetched.member} book=${recordsFetched.book}`);
  return s.data.loans.length;
}

const n = await measure("single", SINGLE, () => ({}));
await measure("batch", BATCH, () => ({
  loadMember: createLoader(getMembers),      // a new loader per request
  loadBook: createLoader(getBooks),
}));
console.log(`\nloan records in the response: ${n}   distinct members: 8   distinct books: 12`);

single    resolver calls=451  data source round trips: loan=1 member=50 book=50 total=101  records fetched: member=50 book=50
batch     resolver calls=451  data source round trips: loan=1 member= 1 book= 1 total=  3  records fetched: member=8 book=12

loan records in the response: 50   distinct members: 8   distinct books: 12

Three numbers need reading.

The round-trip count dropped from 101 to 3. The query did not change, the resolver count did not change, the response did not change; the only thing that changed is how the resolvers reach the data source.

The resolver-call count held steady at 451. The loader does not reduce resolvers; every loan record’s member field is still resolved separately. What it reduces is the round trips those resolvers make to the data source. This distinction matters: the shape of execution comes from the query and cannot be changed, the shape of data access comes from the resolver and can be.

The records-fetched count dropped from 50 to 8. This is deduplication’s share. Because fifty loan records are spread across eight members, the same member was requested six times on average; the loader fetched each one once.

The Batching Window Depends on Execution Order

The loader’s gain depends on keys falling into the same window. The window is as wide as one tick, and it closes if there is a wait in between.

// window.mjs — a loader's batching window is as wide as a single tick
import { createLoader } from "./loader.mjs";

const watch = (label) => (keys) => {
  console.log(`  ${label}: ${keys.length} key(s) in one round trip -> ${keys.join(", ")}`);
  return keys.map((a) => ({ id: a }));
};

console.log("keys requested in the same tick:");
const l1 = createLoader(watch("together"));
await Promise.all(["U-1001", "U-1002", "U-1003", "U-1001"].map(l1));

console.log("keys requested one after another, each awaited:");
const l2 = createLoader(watch("in turn "));
for (const a of ["U-1001", "U-1002", "U-1003"]) await l2(a);

keys requested in the same tick:
  together: 3 key(s) in one round trip -> U-1001, U-1002, U-1003
keys requested one after another, each awaited:
  in turn : 1 key(s) in one round trip -> U-1001
  in turn : 1 key(s) in one round trip -> U-1002
  in turn : 1 key(s) in one round trip -> U-1003

Four keys were requested, three were fetched: U-1001 was requested twice and fetched once. When the same keys were requested one after another, each awaited, three separate round trips resulted; batching did not work at all.

Two rules follow from this. Using an unnecessary await inside a resolver breaks batching: if a resolver awaits something else first and only then calls the loader, it cannot enter the same window as its siblings. And because mutation root fields run in order, there is no batching at the root level; batching still works at the levels below.

The Loader’s Lifetime

In the measurement script, the loaders are rebuilt on every run. This is not a detail, it is a requirement: a loader is created per request.

The reason is that deduplication is also a cache. Across a single request’s lifetime, the loader returns the same value for the same key; this keeps a single response internally consistent. A loader shared between requests, on the other hand, turns into a cache that holds stale data indefinitely, and two separate users’ requests start sharing the same records. A loader created per request goes away when the request’s lifetime ends.

This also explains why context exists. Loaders are placed on the context, because context is the one structure created per request and reachable by every resolver.

Summary

• The N+1 problem is one list query plus as many single-record queries as the list is long; its source is every resolver seeing only its own source.
• A loader collects the keys requested within the same tick, reduces them to a single batch call, and fetches the same key once.
• On a fifty-record query, the data-source round-trip count drops from 101 to 3 and the records-fetched count from 50 to 8; the resolver-call count holds steady at 451.
• The shape of execution comes from the query and cannot be changed; the shape of data access comes from the resolver and can be.
• The batching window is as wide as one tick; calls awaited one after another form separate round trips, and unnecessary waits inside a resolver break batching.
• A loader is created per request; a shared loader turns into an indefinite cache and mixes together separate requests’ data.

Next Step

All fifty loan records came back in a single response. The library’s real loan count is not fifty but hundreds of thousands, and no client wants all of them. Pagination was handled in three ways in the Resource and Contract Design topic — offset-based, cursor-based, and key-based — but the design there was built on address and query parameters. In the query-based approach, pagination is a property not of an endpoint but of a field, and it recurs in every nested list. The next lesson builds the cursor-based connection model, decides where page information gets written, and shows how pagination behaves in nested lists.