---
title: Buffers
source: 'https://academia.sh/en/courses/nodejs/buffers'
course: 'The Node.js Runtime'
language: en
updated: '2026-08-17T18:09:50+00:00'
license: 'CC BY-SA 4.0'
---

# Buffers

How a raw byte array differs from a string, zeroed versus unzeroed allocation, the view-versus-copy distinction, the problem of splitting at a character boundary, and binary record encoding.

In the previous lesson, chunks coming from a stream showed up with type `Buffer`, and
the encoding name was given by hand when converting to text. What happens if a chunk
splits in the middle of a multi-byte character was left unasked.

This lesson answers that question. The subject is a direct continuation of the
character encoding and byte order lessons from the How Computers Work course: the
concepts established there turn into working code here.

## A Buffer as a Byte Array

A **buffer** is an object representing a fixed-length raw byte array. It derives
from the language's `Uint8Array` type, so its indexed access, length, and iteration
behavior are the same as a typed array. What is added on top is encoding conversions
and methods for reading and writing numeric fields.

The difference between a string and a buffer shows up in the unit of measure:

```js
// buffers.mjs
const text = 'sıcaklık';

const buf = Buffer.from(text, 'utf8');
console.log('character count :', text.length);
console.log('byte count      :', buf.length, '=', Buffer.byteLength(text, 'utf8'));
console.log('hex             :', buf.toString('hex'));
console.log('is Uint8Array   :', buf instanceof Uint8Array);
console.log('first three bytes:', [...buf.subarray(0, 3)]);
```

```sh
node buffers.mjs
```

```
character count : 8
byte count      : 10 = 10
hex             : 73c4b163616b6cc4b16b
is Uint8Array   : true
first three bytes: [ 115, 196, 177 ]
```

An eight-character string held ten bytes; the letter `ı` is encoded in UTF-8 with two
bytes (`c4 b1`) and appears twice in the word. This distinction matters when writing
a `Content-Length` header on an HTTP response: the number that has to go in the
header is the byte count, not the character count. `Buffer.byteLength` gives this
number without producing a buffer.

## Allocation, View, and Copy

There are three ways to produce a buffer, and the difference between them is in
memory behavior.

```js
// allocation.mjs
const zeroed = Buffer.alloc(8);
console.log('alloc(8)          :', zeroed.toString('hex'));
console.log('allocUnsafe(8) len:', Buffer.allocUnsafe(8).length);

// subarray is not a copy, it is a view opened onto the same memory
const full = Buffer.from('edge-01', 'utf8');
const view = full.subarray(0, 5);
view[0] = 'E'.charCodeAt(0);
console.log('view changed ->', full.toString('utf8'));

// A copy is requested explicitly when one is needed
const copy = Buffer.from(full.subarray(0, 5));
copy[0] = 'x'.charCodeAt(0);
console.log('copy changed ->', full.toString('utf8'), '|', copy.toString('utf8'));

// Joining pieces into a single buffer
const joined = Buffer.concat([Buffer.from('edge'), Buffer.from('-'), Buffer.from('01')]);
console.log('concat        ->', joined.toString('utf8'), `(${joined.length} bytes)`);
```

```sh
node allocation.mjs
```

```
alloc(8)          : 0000000000000000
allocUnsafe(8) len: 8
view changed -> Edge-01
copy changed -> Edge-01 | xdge-
concat        -> edge-01 (7 bytes)
```

`Buffer.alloc` gives zeroed memory of the requested size. `Buffer.allocUnsafe` skips
the zeroing step and is faster; in exchange, the buffer contains whatever was
previously in that memory. Using this buffer without writing over its content start
to finish can leak data found elsewhere in the process. Rule: **use zeroed
allocation unless you are going to fill the buffer immediately and completely.**

The third line of the output shows view behavior: `subarray` does not allocate new
memory, it returns a window opened onto the same memory. Writing to the window
changes the original buffer. This is valuable in stream code because it allows
slicing without copying; unnoticed, it produces hard-to-explain bugs. When an
independent copy is needed, it is requested explicitly with `Buffer.from`.

`Buffer.concat` joins a list of pieces into a single buffer. This is the standard way
to gather chunks coming from a stream and convert them to a single piece of text at
the end; converting pieces to strings and joining them leads to the problem below.

## Splitting at a Character Boundary

A chunk can end in the middle of a multi-byte character. If that chunk is converted
to text on its own, a replacement character is put in place of the incomplete
character, and information is lost irreversibly.

```js
// boundary.mjs
import { StringDecoder } from 'node:string_decoder';

const buf = Buffer.from('sıcaklık', 'utf8');   // 10 bytes
const chunkA = buf.subarray(0, 2);              // cuts through the middle of 'ı'
const chunkB = buf.subarray(2);

console.log('direct decode :', chunkA.toString('utf8') + chunkB.toString('utf8'));

const decoder = new StringDecoder('utf8');
console.log('with decoder  :', decoder.write(chunkA) + decoder.write(chunkB) + decoder.end());
```

```sh
node boundary.mjs
```

```
direct decode : s��caklık
with decoder  : sıcaklık
```

The two replacement characters on the first line came from the two bytes of the
split `ı` being decoded separately. The decoder in the `node:string_decoder` module
keeps an incomplete byte sequence in itself and joins it with the next write; the
`end` call reports any leftover incomplete sequence.

This exposes a hidden assumption in the previous lesson's `LineSplitter` class.
There, every chunk was converted directly with `toString('utf8')`; because the
measurement file only contains ASCII characters, no problem was visible. The same
code breaks on a dataset with non-ASCII characters in node names. The correct fix is
keeping a decoder in the transform and writing `decoder.write(chunk)` instead of
`chunk.toString('utf8')`.

## Binary Record Format

A buffer offers methods that read and write numeric fields at a given width and byte
order. This is how you parse a binary protocol coming over the network, and how you
define your own format to save space.

A measurement record is fit into six bytes below: one byte for the node number, one
byte for the metric number, four bytes for a single-precision floating-point value.

```js
// binary.mjs
// Binary format fitting a measurement into 6 bytes:
// [0] node no (uint8)  [1] metric no (uint8)  [2..5] value (float32, big-endian)
function encode(nodeNo, metricNo, value) {
  const buf = Buffer.alloc(6);          // zeroed memory
  buf.writeUInt8(nodeNo, 0);
  buf.writeUInt8(metricNo, 1);
  buf.writeFloatBE(value, 2);
  return buf;
}

function decode(buf) {
  return {
    nodeNo: buf.readUInt8(0),
    metricNo: buf.readUInt8(1),
    value: Number(buf.readFloatBE(2).toFixed(2)),
  };
}

const record = encode(1, 0, 21.4);
console.log('binary form   :', record.toString('hex'));
console.log('decoded record:', decode(record));

const littleEndian = Buffer.alloc(4);
littleEndian.writeFloatLE(21.4, 0);
console.log('little-endian :', littleEndian.toString('hex'));
console.log('big-endian    :', record.subarray(2).toString('hex'));

const textSize = Buffer.byteLength(JSON.stringify({ node: 'edge-01', metric: 'temperature', value: 21.4 }));
console.log(`text form ${textSize} bytes, binary form ${record.length} bytes`);
```

```sh
node binary.mjs
```

```
binary form   : 010041ab3333
decoded record: { nodeNo: 1, metricNo: 0, value: 21.4 }
little-endian : 3333ab41
big-endian    : 41ab3333
text form 54 bytes, binary form 6 bytes
```

The `BE` and `LE` suffixes in the method names select the big-endian and
little-endian layouts introduced in the How Computers Work course. The third and
fourth lines of the output show the same number's two layouts side by side: the byte
sequence is reversed. The conventional choice in network protocols is big-endian
layout; when defining a format, which endianness is used has to be written down, or
the two ends read each other's data wrong.

The last line shows why binary formats exist: the same measurement takes 54 bytes as
text, 6 bytes as binary. In exchange, the binary form is unreadable by a human, does
not describe itself, and breaks old decoders when a field is added. This trade-off
was discussed in the How the Internet Works course while justifying why HTTP is
text-based; the choice is between size and inspectability.

Squeezing the value into four bytes has one more cost. Single-precision
floating-point representation carries about seven significant decimal digits; the
value `21.4` cannot be represented exactly in this form. This is why the example
rounds it with `toFixed(2)`. If measurement precision needs to exceed this limit,
double-precision read and write methods are used.

Buffer methods do bounds checking: a call that tries to write outside the array's
range throws `RangeError`. This blocks, at the runtime level, the class of
unbounded-memory-access errors covered in the How Computers Work course.

## Summary

- A buffer is a raw byte array built on `Uint8Array`; character count and byte count
  diverge in multi-byte encodings, and it is the byte count that goes into protocol
  headers.
- `Buffer.alloc` gives zeroed memory; allocation that skips zeroing leaks old memory
  content when not filled completely.
- `subarray` returns a view, not a copy; an independent copy is requested explicitly
  with `Buffer.from`.
- When a multi-byte character splits at a chunk boundary, direct decoding corrupts
  the data; `node:string_decoder` prevents this by holding onto partial bytes.
- Numeric fields are read and written by choosing a width and an endianness; a
  binary format saves space at the cost of inspectability and extensibility.

## Next Step

Up to here, streams were said to emit events, but the event mechanism itself was
never opened up. The measurement collector's next step needs a structure that
notifies interested parties when data arrives: a mechanism that produces a warning
when a threshold is crossed, reports a broken line, announces that the source has
closed. The next lesson covers this mechanism's base class — the event emitter.