---
title: 'Asset Processing'
source: 'https://academia.sh/en/courses/rendering-strategies/asset-processing'
course: 'Rendering Strategies and Infrastructure'
language: en
updated: '2026-08-17T18:11:06+00:00'
license: 'CC BY-SA 4.0'
---

# Asset Processing

Images, fonts, and static files entering the build output; extracting asset links into the graph as nodes, the embed-threshold decision, rewriting references, the asset manifest, and the directly copied directory.

For three lessons, the graph was built only from script files. The North Slope
Measurement Station interface's output is not limited to that: the measurement list's
style file, the icon in front of every row in the list, the drawing of the station's
elevation curve, the interface's font, and a handful of fixed-name files search engines
read are also published.

These files enter the output through two separate routes. Some are linked from code or
a style file; the link is an edge in the graph, and the asset becomes a node. Some are
linked from nowhere but must still be published. This lesson separates the two routes
and runs the pipeline linked assets pass through.

## An Asset Is a Graph Node

For an asset to enter the graph, its link must be statically readable. Two standard
syntaxes provide this.

A `url()` declaration in a style file gives a relative path as its address. The path in
the source is resolved against the directory the style file sits in.

A `new URL("./x.svg", import.meta.url)` expression in code does the same job on the
script side. The second argument is the module's own address; the expression produces
an address resolved against the module's location. This syntax is part of the standard
and is not tied to any tool; it works with no tool present, and can also be caught at
build time when a tool is present.

A syntax that cannot be caught is one where the address is assembled at runtime. A
concatenation in the form `"/asset/" + name + ".svg"` does not say which file it points
to during the build. Such a link does not enter the pipeline: the file is not copied,
its name is not rewritten, and a request goes to an address absent from the output. The
rule is clear — asset addresses computed at runtime are served from the directly copied
directory.

## Two Flows

The **processed flow** covers linked assets. Depending on their size, these assets are
either written to the output as a separate file or embedded into the body of the file
that links them; either way, the link in the source is translated into an address in
the output.

The **directly copied flow** is for files that must be published with their name and
location unchanged: the file carrying crawler rules, domain verification files, data
files downloaded from a fixed address. These files are not scanned, their names are
preserved, and their content does not change. The distinguishing criterion is a single
question: is the build free to determine the address, or is it pinned from outside?

## The Asset Tree

The block below produces assets deterministically. A filler file of the same size
stands in for a font body; because the pipeline's decisions depend on a file's size, not
its content, this does not change the result.

The block below adds asset links to the `source/` tree set up in the previous lessons;
those lessons' setup blocks must run in this directory first.

```bash
#!/usr/bin/env bash
mkdir -p asset public style

cat > asset/icon.svg <<'EOF'
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 16 16" fill="none" stroke="#243">
<path d="M2 13 L6 5 L9 10 L11 7 L14 13 Z"/><circle cx="12" cy="3" r="2"/>
</svg>
EOF

node --input-type=module -e '
import { writeFileSync } from "node:fs";
// Station map: a 60-point elevation curve, deterministic generation.
const point = Array.from({ length: 60 }, (_, i) =>
  `${(i * 8).toFixed(0)} ${(120 - 40 * Math.sin(i / 6) - i / 3).toFixed(2)}`);
writeFileSync("asset/station.svg",
  `<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 480 160">\n` +
  `<path fill="none" stroke="#1b3a2f" stroke-width="2" d="M${point.join(" L")}"/>\n` +
  point.filter((_, i) => i % 6 === 0).map((n) => `<circle cx="${n.split(" ")[0]}" cy="${n.split(" ")[1]}" r="3"/>`).join("\n") +
  `\n</svg>\n`);
// A deterministic binary file of the same size stands in for a font body.
writeFileSync("asset/measurement.woff2", Buffer.alloc(4096, 0x77));
'

cat > style/station.css <<'EOF'
@font-face {
  font-family: "Measurement";
  src: url("../asset/measurement.woff2") format("woff2");
  font-display: swap;
}
.measurement li::before {
  content: "";
  background-image: url("../asset/icon.svg");
  width: 16px;
  height: 16px;
}
.station-map {
  background-image: url("../asset/station.svg");
  min-height: 160px;
}
EOF

cat > source/asset-links.js <<'EOF'
export const iconUrl = new URL("../asset/icon.svg", import.meta.url);
export const mapUrl = new URL("../asset/station.svg", import.meta.url);
EOF

printf 'User-agent: *\nDisallow: /panel/\n' > public/robots.txt
printf '{ "station": "north-slope", "elevation_m": 1840 }\n' > public/station.json
```

## The Pipeline

The pipeline is four steps: scan for links, decide embedding for each asset, replace
the reference with its output address, and write the mapping between source and output
into a manifest.

```js
// asset.mjs — asset pipeline: scanning, embed decision, rewriting, manifest generation.
import { readFileSync, writeFileSync, mkdirSync, cpSync, statSync, rmSync } from "node:fs";
import { basename, dirname, join, relative } from "node:path";

const EMBED_THRESHOLD = 1024;                    // bytes
const OUTPUT = "output";
const TYPE = { ".svg": ["image/svg+xml", "text"], ".css": ["text/css", "text"],
              ".woff2": ["font/woff2", "binary"], ".png": ["image/png", "binary"] };
const extension = (p) => p.slice(p.lastIndexOf("."));

const SOURCES = ["style/station.css", "source/asset-links.js"];
const LINK = [
  /url\(\s*["']?([^"')]+)["']?\s*\)/g,                          // link inside a style file
  /new URL\(\s*["']([^"']+)["']\s*,\s*import\.meta\.url\s*\)/g, // link inside code
];

function dataUrl(filePath) {                     // percent-encoding for text assets, base64 for binary
  const [mime, kind] = TYPE[extension(filePath)], body = readFileSync(filePath);
  return kind === "text"
    ? "data:" + mime + "," + encodeURIComponent(body.toString("utf8")).replace(/%20/g, " ")
    : "data:" + mime + ";base64," + body.toString("base64");
}

rmSync(OUTPUT, { recursive: true, force: true });
mkdirSync(join(OUTPUT, "asset"), { recursive: true });

const assets = new Map();                        // path -> { size, refs, embedded, target, outputSize }
const manifest = {};

for (const sourcePath of SOURCES) {
  let text = readFileSync(sourcePath, "utf8");
  for (const pattern of LINK) {
    text = text.replace(pattern, (full, link) => {
      const filePath = relative(".", join(dirname(sourcePath), link));
      if (!assets.has(filePath)) {
        const size = statSync(filePath).size;
        const embedded = size <= EMBED_THRESHOLD;
        const target = embedded ? dataUrl(filePath) : "/asset/" + basename(filePath);
        if (!embedded) cpSync(filePath, join(OUTPUT, target));
        assets.set(filePath, { size, refs: 0, embedded, target,
          outputSize: embedded ? Buffer.byteLength(target) : size });
      }
      const a = assets.get(filePath);
      a.refs++;
      return full.startsWith("url") ? 'url("' + a.target + '")'
        : 'new URL("' + a.target + '", import.meta.url)';
    });
  }
  const outputPath = join(OUTPUT, basename(sourcePath));
  writeFileSync(outputPath, text);
  manifest["/" + basename(sourcePath)] = { source: sourcePath, size: statSync(outputPath).size };
}

for (const [filePath, a] of assets)
  if (!a.embedded) manifest[a.target] = { source: filePath, size: a.size };
cpSync("public", OUTPUT, { recursive: true });   // directly copied directory
writeFileSync(join(OUTPUT, "asset-manifest.json"), JSON.stringify(manifest, null, 2) + "\n");

const base64Length = (filePath) => Math.ceil(statSync(filePath).size / 3) * 4;
console.log("asset".padEnd(20) + "size".padStart(6) + "  ref  decision    output  if embedded");
for (const [filePath, a] of assets)
  console.log(basename(filePath).padEnd(20) + String(a.size).padStart(6) + "   " + a.refs +
    "   " + (a.embedded ? "embedded  " : "separate  ") + String(a.outputSize).padStart(7) +
    String(base64Length(filePath)).padStart(12));

console.log("-- rewritten style file (long values truncated) --");
for (const s of readFileSync(join(OUTPUT, "station.css"), "utf8").trimEnd().split("\n"))
  console.log("  " + (s.length > 64 ? s.slice(0, 64) + "[...]" : s));

console.log("-- manifest keys --");
for (const [url, k] of Object.entries(manifest))
  console.log("  " + url.padEnd(24) + String(k.size).padStart(5) + " B  <- " + k.source);

const separate = [...assets.values()].filter((a) => !a.embedded);
console.log("-- total --");
console.log("  style file      : " + manifest["/station.css"].size + " B  (source was " +
  statSync("style/station.css").size + " B)");
console.log("  separate asset  : " + separate.length + " file, " +
  separate.reduce((t, a) => t + a.size, 0) + " B");
console.log("  direct copy     : robots.txt, station.json (name and content unchanged)");
```

```
$ node asset.mjs
asset                 size  ref  decision    output  if embedded
measurement.woff2     4096   1   separate     4096        5464
icon.svg               168   2   embedded      275         224
station.svg           1162   2   separate     1162        1552
-- rewritten style file (long values truncated) --
  @font-face {
    font-family: "Measurement";
    src: url("/asset/measurement.woff2") format("woff2");
    font-display: swap;
  }
  .measurement li::before {
    content: "";
    background-image: url("data:image/svg+xml,%3Csvg xmlns%3D%22ht[...]
    width: 16px;
    height: 16px;
  }
  .station-map {
    background-image: url("/asset/station.svg");
    min-height: 160px;
  }
-- manifest keys --
  /station.css              586 B  <- style/station.css
  /asset-links.js           398 B  <- source/asset-links.js
  /asset/measurement.woff2 4096 B  <- asset/measurement.woff2
  /asset/station.svg       1162 B  <- asset/station.svg
-- total --
  style file      : 586 B  (source was 332 B)
  separate asset  : 2 file, 5258 B
  direct copy     : robots.txt, station.json (name and content unchanged)
```

In the rewritten style file, all three links have changed: two into a root-relative
address, one into the body itself. The `../asset/` relative path in the source never
appears in the output — output layout is independent of source layout.

The ref column exposes a detail: the icon and the map are linked from both the style
file and the code. The map, written as a separate file, is copied once, and both
references point at the same address. The embedded icon, on the other hand, is
rewritten at every reference; the 168-byte file totals 550 bytes across two outputs.

## The Embed Threshold

The embed decision is made with a threshold, and the threshold balances two opposing
costs.

**Embedding's gain is one request.** Every asset written as a separate file brings a
request's latency alongside the byte it carries. For small assets, this latency is more
expensive than the file itself.

**Embedding's cost has three items.** First is encoding expansion: binary data grows by
roughly a third when converted to base64 — a 4096-byte font body becomes 5464
characters. Text assets can use percent-encoding, but it is not always cheaper: the
example's icon is 275 bytes with percent-encoding and 224 with base64, because the
proportion of characters that need escaping is high. Second is duplication: an embedded
asset sits separately in every output that links it. Third is caching — an embedded
asset has no separate address, so it cannot be cached separately; it downloads again
with the file that links it every time that file changes.

This third item explains why the threshold is kept small. Embedding a rarely changing
image inside a frequently changing style file makes that image download again with
every release. The next lesson will make this connection measurable.

## Decisions Specific to Asset Classes

The threshold alone is not a sufficient policy; each asset class has its own
constraints.

**Fonts** are written as separate files regardless of size. A font body is shared
across multiple pages and cached long-lived; embedding it destroys that sharing. The two
settings established in the Font Loading lesson of the Visual Presentation with CSS
course apply here as well: display behavior is declared with `font-display`, and the
body can be requested with an early loading hint inside the document. A third thing the
build side can add is subsetting — keeping only the used character ranges from the body
shrinks the file noticeably, and the decision depends on the targeted language set.

**Images** carry two extra pieces of metadata. First are dimensions: when width and
height are read during the build and written into the markup, the content shift defined
in the Web Fundamentals and HTML course is prevented. Second are derivatives: the
candidate set in the Layout Systems and Responsive Design course's Responsive Images
lesson asks for several widths of the same image; the pipeline is what produces those
versions. Format choice is made by class, not by product name — drawings that need
lossless compression and photographs suited to lossy compression are handled
separately.

**Style files** are both asset and source: the links inside them are scanned, and they
themselves are written to the output. This is why they appear twice in the pipeline.

## The Asset Manifest

The pipeline's final output is a mapping: which source file corresponds to which
address in the output. This file is called the **asset manifest**, and it is the
contract between the build and everything outside it.

At least three parties read the manifest. The document template takes the address of
the style file and the entry chunk from it. A runtime rendering on the server writes
the same addresses into the markup it produces. A service worker running offline
generates the list of files it will precache from it — the Caching Strategies lesson in
the Browser and the Web Platform course showed how fragile writing that list by hand is;
the manifest lets the build produce the list instead.

Name similarity invites confusing three separate files. The package manifest carries
dependencies and package metadata; the application manifest carries installability
information; the asset manifest is only the source-to-output mapping. The three are
read at different times, by different parties.

## Summary

- Assets enter the graph through two standard links: a `url()` declaration in a style
  file and a `new URL(...)` expression in code; addresses assembled at runtime cannot
  enter the pipeline.
- Linked assets pass through the processed flow; files with a fixed name pass through
  the directly copied flow — the criterion is whether the build is free to determine
  the address.
- The pipeline is four steps: scan for links, decide embedding, replace the reference
  with the output address, write the mapping into the manifest.
- Embedding buys one request; in exchange it brings encoding expansion, duplication at
  every reference, and the loss of separate caching — a 4096-byte body becomes 5464
  characters in base64.
- Fonts are written as separate files regardless of size; images are produced together
  with dimension metadata and width derivatives.
- The asset manifest is the shared contract for the document template, the server
  runtime, and the service worker.

## Next Step

The files in the output still carry their source names for now. This leaves both of two
questions unresolved. If a published file is cached long-lived, the user keeps getting
the old version once its content changes; if it is cached short-lived, revalidation
happens on every visit and the cache's value is lost. The way to avoid both extremes is
binding the name to the content: when a file's name carries a digest of its content, the
name does not change unless the content changes, and the cache stays valid as long as
the name does not change. The next lesson produces these names, measures how names
change in a chain across interdependent chunks, and builds the mechanism that breaks
that chain.