---
title: 'N+1 Problem and Batch Loading'
source: 'https://academia.sh/en/courses/api-design/n-plus-1-problem-and-batch-loading'
course: 'Web API Design'
language: en
updated: '2026-08-19T05:19:29+00:00'
license: 'CC BY-SA 4.0'
---

# N+1 Problem and Batch Loading

The data-source round-trip count growing together with list length, a loader layer that merges calls from the same tick into a single round trip, and a measurement dropping from 101 round trips to 3.

The Resolvers lesson left a number unaddressed: three member calls were made for three
loan records, yet there were two distinct members. Fragments and Variables did not touch
this number; field merging only removes collisions at the same response key, it does not
remove different records asking for the same member.

This lesson shows the problem at scale and solves it. The data source holds fifty loan
records, eight members, and twelve books.

```js
// data.mjs — loan data generated for scale; every data source round trip is counted
export const roundTrips = { book: 0, member: 0, loan: 0 };
export const recordsFetched = { book: 0, member: 0 };
export const reset = () => { roundTrips.book = roundTrips.member = roundTrips.loan = 0; recordsFetched.book = recordsFetched.member = 0; };

const AUTHORS = ["Cormen", "Knuth", "Kernighan", "Dijkstra", "Hoare", "Lamport"];
export const BOOKS = new Map(Array.from({ length: 12 }, (_, i) => {
  const isbn = `978-000000${String(i).padStart(2, "0")}`;
  return [isbn, { isbn, title: `Book ${i}`, author: AUTHORS[i % AUTHORS.length], copies: 1 + (i % 3) }];
}));
export const MEMBERS = new Map(Array.from({ length: 8 }, (_, i) => {
  const id = `U-${1001 + i}`;
  return [id, { id, createdAt: "2025-01-01", name: `Member ${i}` }];
}));
export const LOANS = new Map(Array.from({ length: 50 }, (_, i) => {
  const id = `O-${i + 1}`;
  return [id, { id, createdAt: "2026-01-01", memberId: `U-${1001 + (i % 8)}`,
    status: i % 3 === 0 ? "CLOSED" : "OPEN", returnDate: "2026-04-01",
    items: [{ isbn: `978-000000${String(i % 12).padStart(2, "0")}`, branch: "central" }] }];
}));

// Single-key accesses: every call is one round trip.
export const getBook = (isbn) => { roundTrips.book++; recordsFetched.book++; return BOOKS.get(isbn); };
export const getMember = (id) => { roundTrips.member++; recordsFetched.member++; return MEMBERS.get(id); };
export const listLoans = () => { roundTrips.loan++; return [...LOANS.values()]; };

// Batch accesses: more than one key in a single round trip.
export const getBooks = (isbns) => { roundTrips.book++; recordsFetched.book += isbns.length; return isbns.map((i) => BOOKS.get(i)); };
export const getMembers = (ids) => { roundTrips.member++; recordsFetched.member += ids.length; return ids.map((i) => MEMBERS.get(i)); };
```

The parser, schema, flattener, and engine are used exactly as they were written in the
previous lessons; what is new is the data source and the loader layer.

## The Name of the Problem

A query that asks for fifty loan records together with their members reaches the data
source fifty-one times: once to fetch the list, fifty times to fetch each record's
member. When the book field is added, fifty more round trips. This is called the **N+1
problem**: one list query plus as many single-record queries as the list is long.

The problem's source is the resolver model. The `Loan.member` resolver sees a single
loan record and fetches that record's member; it does not know it has forty-nine
siblings. Every resolver seeing only its own source is the resolver model's strength —
and it is also the cause of this problem.

The solution is not to change the resolvers, but to put a layer between them. A
**loader** does not fetch the keys handed to it right away; it collects the keys
requested within the same tick and reduces them to a single batch call. Its second job
is deduplication: if the same key is requested twice, it is fetched once.

```js
// loader.mjs — reduces keys requested in the same tick to a single batch call
// Its second job is deduplication: the same key requested twice is fetched once.
export function createLoader(batchFetch) {
  let pending = new Map();          // key -> [resolve, ...]
  let scheduled = false;

  const flush = async () => {
    const batch = pending; pending = new Map(); scheduled = false;
    const keys = [...batch.keys()];
    const values = await batchFetch(keys);
    keys.forEach((k, i) => batch.get(k).forEach((resolve) => resolve(values[i])));
  };

  return (key) => new Promise((resolve) => {
    if (!pending.has(key)) pending.set(key, []);
    pending.get(key).push(resolve);
    if (!scheduled) { scheduled = true; queueMicrotask(flush); }
  });
}
```

The batching window is set by the `queueMicrotask` call: every key that arrives before
the currently running tick finishes falls into the same batch. The level-ordered
execution measured in the Resolvers lesson pays off here — the fifty `Loan.member` calls
at the same level were already started together, so all of them enter the same window.

## Measurement

```js
// run.mjs — runs the same query with single-key and batch resolvers, compares round trips
import { parse } from "./parser.mjs";
import { execute } from "./executor.mjs";
import { SCHEMA } from "./schema.mjs";
import { createLoader } from "./loader.mjs";
import { listLoans, getBook, getMember, getBooks, getMembers, roundTrips, recordsFetched, reset } from "./data.mjs";

const QUERY = `{ loans { id status member { name } items { branch book { title author } } } }`;

const SINGLE = {
  Query: { loans: () => listLoans() },
  Loan: { member: (o) => getMember(o.memberId) },
  Item: { book: (k) => getBook(k.isbn) },
};

// Batch resolvers pull their loaders from the per-request context.
const BATCH = {
  Query: { loans: () => listLoans() },
  Loan: { member: (o, _, c) => c.loadMember(o.memberId) },
  Item: { book: (k, _, c) => c.loadBook(k.isbn) },
};

async function measure(title, resolvers, makeContext) {
  reset();
  const calls = [];
  const s = await execute(SCHEMA, resolvers, parse(QUERY), { context: makeContext(), tracer: (b) => calls.push(b) });
  const total = roundTrips.book + roundTrips.member + roundTrips.loan;
  console.log(`${title.padEnd(9)} resolver calls=${String(calls.length).padStart(3)}  ` +
    `data source round trips: loan=${roundTrips.loan} member=${String(roundTrips.member).padStart(2)} book=${String(roundTrips.book).padStart(2)} total=${String(total).padStart(3)}  ` +
    `records fetched: member=${recordsFetched.member} book=${recordsFetched.book}`);
  return s.data.loans.length;
}

const n = await measure("single", SINGLE, () => ({}));
await measure("batch", BATCH, () => ({
  loadMember: createLoader(getMembers),      // a new loader per request
  loadBook: createLoader(getBooks),
}));
console.log(`\nloan records in the response: ${n}   distinct members: 8   distinct books: 12`);
```

```
single    resolver calls=451  data source round trips: loan=1 member=50 book=50 total=101  records fetched: member=50 book=50
batch     resolver calls=451  data source round trips: loan=1 member= 1 book= 1 total=  3  records fetched: member=8 book=12

loan records in the response: 50   distinct members: 8   distinct books: 12
```

Three numbers need reading.

**The round-trip count dropped from 101 to 3.** The query did not change, the resolver
count did not change, the response did not change; the only thing that changed is how
the resolvers reach the data source.

**The resolver-call count held steady at 451.** The loader does not reduce resolvers;
every loan record's `member` field is still resolved separately. What it reduces is the
**round trips those resolvers make to the data source**. This distinction matters: the
shape of execution comes from the query and cannot be changed, the shape of data access
comes from the resolver and can be.

**The records-fetched count dropped from 50 to 8.** This is deduplication's share.
Because fifty loan records are spread across eight members, the same member was
requested six times on average; the loader fetched each one once.

## The Batching Window Depends on Execution Order

The loader's gain depends on keys falling into the same window. The window is as wide
as one tick, and it closes if there is a wait in between.

```js
// window.mjs — a loader's batching window is as wide as a single tick
import { createLoader } from "./loader.mjs";

const watch = (label) => (keys) => {
  console.log(`  ${label}: ${keys.length} key(s) in one round trip -> ${keys.join(", ")}`);
  return keys.map((a) => ({ id: a }));
};

console.log("keys requested in the same tick:");
const l1 = createLoader(watch("together"));
await Promise.all(["U-1001", "U-1002", "U-1003", "U-1001"].map(l1));

console.log("keys requested one after another, each awaited:");
const l2 = createLoader(watch("in turn "));
for (const a of ["U-1001", "U-1002", "U-1003"]) await l2(a);
```

```
keys requested in the same tick:
  together: 3 key(s) in one round trip -> U-1001, U-1002, U-1003
keys requested one after another, each awaited:
  in turn : 1 key(s) in one round trip -> U-1001
  in turn : 1 key(s) in one round trip -> U-1002
  in turn : 1 key(s) in one round trip -> U-1003
```

Four keys were requested, three were fetched: `U-1001` was requested twice and fetched
once. When the same keys were requested one after another, each awaited, three separate
round trips resulted; batching did not work at all.

Two rules follow from this. Using an unnecessary `await` inside a resolver breaks
batching: if a resolver awaits something else first and only then calls the loader, it
cannot enter the same window as its siblings. And because mutation root fields run in
order, there is no batching at the root level; batching still works at the levels below.

## The Loader's Lifetime

In the measurement script, the loaders are rebuilt on every run. This is not a detail,
it is a requirement: **a loader is created per request.**

The reason is that deduplication is also a cache. Across a single request's lifetime,
the loader returns the same value for the same key; this keeps a single response
internally consistent. A loader shared between requests, on the other hand, turns into a
cache that holds stale data indefinitely, and two separate users' requests start sharing
the same records. A loader created per request goes away when the request's lifetime
ends.

This also explains why context exists. Loaders are placed on the context, because
context is the one structure created per request and reachable by every resolver.

## Summary

- The N+1 problem is one list query plus as many single-record queries as the list is
  long; its source is every resolver seeing only its own source.
- A loader collects the keys requested within the same tick, reduces them to a single
  batch call, and fetches the same key once.
- On a fifty-record query, the data-source round-trip count drops from 101 to 3 and the
  records-fetched count from 50 to 8; the resolver-call count holds steady at 451.
- The shape of execution comes from the query and cannot be changed; the shape of data
  access comes from the resolver and can be.
- The batching window is as wide as one tick; calls awaited one after another form
  separate round trips, and unnecessary waits inside a resolver break batching.
- A loader is created per request; a shared loader turns into an indefinite cache and
  mixes together separate requests' data.

## Next Step

All fifty loan records came back in a single response. The library's real loan count is
not fifty but hundreds of thousands, and no client wants all of them. Pagination was
handled in three ways in the Resource and Contract Design topic — offset-based,
cursor-based, and key-based — but the design there was built on address and query
parameters. In the query-based approach, pagination is a property not of an endpoint but
of a **field**, and it recurs in every nested list. The next lesson builds the
cursor-based connection model, decides where page information gets written, and shows
how pagination behaves in nested lists.