---
title: 'Floating-Point Numbers'
source: 'https://academia.sh/en/courses/how-computers-work/floating-point-numbers'
course: 'How Computers Work'
language: en
updated: '2026-08-17T18:08:10+00:00'
license: 'CC BY-SA 4.0'
---

# Floating-Point Numbers

Representing real values in finite bits, the IEEE 754 field layout, and the source of precision loss.

For integers, representation is all or nothing: $200$ can be written in eight bits,
$300$ cannot. Fractional numbers are more troubling still. There are infinitely many
real numbers between $0$ and $1$; a finite set of bits can hold only a finite subset
of them. The rest cannot be represented and are **rounded to the nearest
representable value.**

This lesson asks how that subset is chosen. The choice feeds directly into computation
results: by the end of this lesson, the fact that $0.1 + 0.2$ does not come out to
exactly $0.3$ will look like expected behavior rather than a quirk.

## Fixed Point and Its Limit

The first solution that comes to mind is placing the decimal separator at a fixed
position. The upper bits of an eight-bit value could be assigned to the integer part
and the lower six to the fractional part; the positional values of the fractional
bits then run $2^{-1}$, $2^{-2}$, $2^{-3}$, and so on:

$$
\texttt{0101{.}1010}_2 = 4 + 1 + 0.5 + 0.125 = 5.625
$$

This is **fixed-point** representation, and it is still used in specific domains. Its
problem is that range and resolution are locked together: every bit assigned to the
fraction halves the largest representable number. If a single computation needs to
carry both interplanetary distances and atomic radii, a fixed separator position
cannot serve both needs at once.

## The Binary Counterpart of Scientific Notation

The solution is to store the separator's position alongside the value itself. In
scientific notation, writing $6.022 \times 10^{23}$ carries two pieces of information:
significant digits and order of magnitude. The binary counterpart has the same
structure:

$$
(-1)^{s} \times 1.m \times 2^{e}
$$

Here $s$ is the sign, $m$ is the fractional part (the **mantissa**), and $e$ is the
exponent. The separator "floats" as the exponent changes — which is where the name of
the representation comes from.

One detail saves a bit: every nonzero number can be written in the form $1.\dots$ by
choosing the exponent appropriately. Since the leading $1$ is always present, it does
not need to be stored; hardware treats it as **implicit**. The entire mantissa field
is then given over to the fractional part, and one bit is gained for free.

## The IEEE 754 Field Layout

The bit-level details of floating-point representation are defined in the IEEE 754
standard. Two common widths:

| Format | Total | Sign | Exponent | Mantissa | Exponent bias |
|---|---|---|---|---|---|
| binary32 (single precision) | 32 bit | 1 | 8 | 23 | 127 |
| binary64 (double precision) | 64 bit | 1 | 11 | 52 | 1023 |

The exponent field is not stored as a signed number but **biased**: the true exponent
is found by subtracting a fixed bias from the value stored in the field. In binary32,
if the field holds $130$, the true exponent is $130 - 127 = 3$. The reason for this
arrangement is ordering: a biased exponent lets two positive floating-point numbers be
sorted by comparing their bit patterns as if they were unsigned integers.

The two extreme values of the exponent field are reserved for special meanings:

- All bits $0$: the value is zero (if the mantissa is also zero) or a **subnormal**
  number. Subnormal numbers sustain resolution very close to zero by abandoning the
  implicit-$1$ rule.
- All bits $1$: the value is **infinity** if the mantissa is zero, and **not a number
  (NaN)** if it is nonzero.

Infinity and NaN carry the result of an overflowing or undefined operation without
halting the program. $1/0$ gives infinity, $0/0$ gives NaN. NaN's distinguishing
property is that it is not even equal to itself; this is the standard way to test
whether a value is NaN.

## The Floating-Point Interpretation of the Shared Example

The course's shared example, `0x41424344`, splits into the following fields when
interpreted as binary32:

$$
\underbrace{0}_{s}\ \underbrace{1000\,0010}_{\text{exponent field}}\ \underbrace{100\,0010\,0100\,0011\,0100\,0100}_{\text{mantissa}}
$$

- The sign bit is $0$, so the number is positive.
- The exponent field is $1000\,0010_2 = 130$; the true exponent is $130 - 127 = 3$.
- The mantissa field is $4{,}342{,}596$; its fractional value is
  $4{,}342{,}596 / 2^{23} = 0.517677783966064453125$. Adding the implicit $1$ gives a
  significand of $1.517677783966064453125$.

The value:

$$
1.517677783966064453125 \times 2^{3} = 12.141422271728515625
$$

The same 32 bits read as $1{,}094{,}861{,}636$ as an unsigned integer and as
$12.141422271728515625$ as a floating-point number. The bit pattern has not changed;
what changes is the interpretation rule applied to it. A variable's **type** in a
programming language exists precisely to select that rule.

## Which Numbers Are Exactly Represented

The mantissa is a binary fraction. The values represented exactly are therefore
fractions whose denominator is a power of two: $0.5$, $0.25$, $0.75$, $3.125$, and the
like. Such numbers are called **dyadic rationals**.

$0.1$ is not one of them. This number, written with a single digit in the decimal
system, has a repeating expansion in the binary system:

$$
0.1_{10} = 0.0001100110011\overline{0011}\dots_2
$$

A repeating expansion does not fit a finite mantissa; it is rounded to the nearest
representable value. What binary64 stores is not $0.1$ but another number very close
to it. The same holds for $0.2$ and $0.3$.

This is not a flaw of the language or the hardware: it is exactly as natural that
$1/10$ cannot be written with finitely many binary digits as it is that $1/3$ cannot
be written with finitely many decimal digits. What changes is which denominators are
"lucky."

## The Result of Addition

When two rounded values are added, the result is rounded as well, and errors
accumulate:

```python
print(0.1 + 0.2)              # 0.30000000000000004
print(0.1 + 0.2 == 0.3)       # False
print(f"{0.1:.20f}")          # 0.10000000000000000555

import math
print(math.isclose(0.1 + 0.2, 0.3))   # True

import struct
(value,) = struct.unpack(">f", bytes.fromhex("41424344"))
print(value)                  # 12.141422271728516
```

The comparison `0.1 + 0.2 == 0.3` giving a false result is not a bug — it follows
directly from the representation. Floating-point values are therefore never
**compared for equality**; instead, one checks whether the difference falls below an
acceptable threshold. Functions such as `isclose` perform this comparison using a
combination of relative and absolute tolerance.

The `struct` module interprets four bytes of raw data according to a given format;
the format string `">f"` means "big-endian, single-precision floating point." What the
byte-order marker here means is the subject of this topic's final lesson.

## The Limit of Precision

Mantissa width determines how many significant digits can be carried. binary32 carries
about 7 significant decimal digits, binary64 about 15 to 17. This limit has two
practical consequences.

First: when a very large and a very small value are added, the smaller one can vanish.
If a value below the mantissa's resolution is added to a large number, the result does
not change.

Second: computation order affects the result. Floating-point addition does not carry
the associative property — $(a + b) + c$ and $a + (b + c)$ can give different results.
This means algebraic properties familiar from integers do not carry over here, and it
is one of the reasons numerical methods form a separate discipline.

In domains where decimal precision is a matter of contract, such as monetary amounts,
floating-point types are not used; fixed-precision decimal types, or integers storing
the smallest unit (for example, cents), are preferred instead.

## Summary

- Finite bits hold only a finite subset of the real numbers; the rest are rounded to
  the nearest representable value.
- Floating-point representation stores a value as $(-1)^s \times 1.m \times 2^{e}$;
  the leading $1$ is implicit and gains one bit.
- In IEEE 754 binary32 the fields are 1 / 8 / 23 bits, and the exponent is stored with
  a bias of $127$; the extreme values of the exponent field are reserved for zero,
  subnormal numbers, infinity, and NaN.
- Exactly represented fractions are those whose denominator is a power of two; $0.1$
  is not among them.
- Floating-point values are compared with tolerance, not equality; addition does not
  carry the associative property.
- The same `0x41424344` pattern reads as $1{,}094{,}861{,}636$ as an integer and as
  $12.141422271728515625$ as binary32.

## Next Step

Up to this point, bit patterns have always been interpreted as a whole. Programs,
however, frequently work with individual bits: turning on a flag, extracting a field,
doubling a value. The next lesson defines the bit-level operators that do this; the
`>>` and `&` operators used in previous lessons will also be given their rationale
there.