---
title: 'Syntax and Indentation'
source: 'https://academia.sh/en/courses/python-fundamentals/syntax-and-indentation'
course: 'Python Fundamentals'
language: en
updated: '2026-08-17T18:10:28+00:00'
license: 'CC BY-SA 4.0'
---

# Syntax and Indentation

Indentation is not a style preference but part of the grammar: the same 21 content tokens produce two different trees and two different results under two different indentations, four different widths give a single tree, and only 5 of ten parts are expressions.

The previous lesson showed that the source is compiled as a whole before anything runs,
and said that the first step of compilation is splitting the text into **tokens**. That
step left a question open: what happens to the whitespace at the start of a line?

In most languages that build a block with braces, nothing — the whitespace is discarded,
braces say where the body starts and ends, and indentation is there only for the reader.
In Python, the whitespace is not discarded. **Indentation** adds its own tokens to the
token sequence, and it is exactly those tokens that build the block structure. The
observable consequence is this: the same token sequence turns into a different tree and
a different result just by changing its indentation.

## Whitespace Builds the Block, Not Braces

The tokenizing step measures the whitespace at the start of every line and keeps it on a
stack. If a line sits deeper than the one before it, an `INDENT` token is placed in the
stream; if it sits shallower, a `DEDENT` token is placed for every level that closes. If
several levels close at once, that many `DEDENT` tokens are produced, so the number of
`DEDENT` tokens in the sequence gives the number of blocks that closed. These tokens have
no character in the source text that corresponds to them — they are produced from the
whitespace itself.

The two sources below are indistinguishable if you only look at the tokens' names. Only
one line's indentation differs: the `print` call sits outside the loop body in one,
inside it in the other.

The measurement's assumptions:

- **LF8** — The two sources differ only in the last line's indentation; no character was
  added, removed, or changed.
- **LF9** — The comparison is made through the tokens' **names**; `INDENT`, `DEDENT`,
  `NEWLINE`, `NL`, and the end-of-file token are counted as "layout tokens" and set
  aside, the rest count as "content tokens."
- **LF10** — The tree is written with the class names of the statements at the parsed
  structure's top level; the loop statement's body is given in square brackets. What is
  read is not the tree's full dump but the shell that is enough for this measurement.
- **LF11** — Execution output is captured into a separate buffer and written as a list of
  printed lines; no environment-dependent value is read.

```python
"""Same token sequence, two indentations: three views."""
import ast
import contextlib
import io
import tokenize

OUTSIDE = "total = 0\nfor x in (1, 2, 3):\n    total += x\nprint(total)\n"
INSIDE = "total = 0\nfor x in (1, 2, 3):\n    total += x\n    print(total)\n"
SOURCE = {"outside the body": OUTSIDE, "inside the body": INSIDE}
LAYOUT = {"INDENT", "DEDENT", "NEWLINE", "NL", "ENDMARKER"}


def tokens(source):
    reader = io.StringIO(source).readline
    return [tokenize.tok_name[t.type] for t in tokenize.generate_tokens(reader)]


def tree(source):
    parts = []
    for d in ast.parse(source).body:
        name = type(d).__name__
        if isinstance(d, ast.For):
            name += "[" + ",".join(type(g).__name__ for g in d.body) + "]"
        parts.append(name)
    return " ".join(parts)


def run(source):
    buffer = io.StringIO()
    with contextlib.redirect_stdout(buffer):
        exec(compile(source, "<lesson>", "exec"), {})
    return buffer.getvalue().split()


for name, source in SOURCE.items():
    all_tokens = tokens(source)
    content = [t for t in all_tokens if t not in LAYOUT]
    print(f"{name:17s} tokens {len(all_tokens)}  non-layout {len(content)}  "
          f"INDENT/DEDENT {[i for i, t in enumerate(all_tokens) if t in ('INDENT', 'DEDENT')]}")

a, b = (tokens(k) for k in SOURCE.values())
print(f"equal once layout tokens are dropped: "
      f"{[x for x in a if x not in LAYOUT] == [x for x in b if x not in LAYOUT]}")
print()
for name, source in SOURCE.items():
    print(f"{name:17s} tree: {tree(source)}")
    print(f"{'':17s} printed: {run(source)}")
```

```
outside the body  tokens 28  non-layout 21  INDENT/DEDENT [16, 21]
inside the body   tokens 28  non-layout 21  INDENT/DEDENT [16, 26]
equal once layout tokens are dropped: True

outside the body  tree: Assign For[AugAssign] Expr
                  printed: ['6']
inside the body   tree: Assign For[AugAssign,Expr]
                  printed: ['1', '3', '6']
```

Both sources produce **28** tokens, and **21** of them are content tokens. Once layout
tokens are dropped, the two remaining sequences are **exactly equal** — same names, same
order. The difference sits at a single token's position: `DEDENT` stands at position
**21** in one, position **26** in the other. The only thing that changed in the source is
four spaces, and those four spaces moved one token five positions over.

That single difference changes the tree. In the first source, the call is the loop's
**sibling**: in the tree, `For[AugAssign]` finishes and a separate `Expr` follows. In the
second, the call is the loop's **child**: `For[AugAssign,Expr]`. The result splits
accordingly — one prints a single line, the other prints three.

The rule that follows from this closes one of the most expensive misunderstandings when
approaching the language: **breaking indentation is not a style defect, it is changing
the program.** In a brace-based language, these same two sources would be formatted
versions of each other and both would do the same thing. Here indentation is the sole
determinant; no other marker states the body's boundary.

## Width Is Free, Consistency Is Mandatory

Indentation being grammatical does not mean a particular **width** is grammatical. The
language does not impose a number; what it imposes is that a block be internally
consistent. The boundary is observable: the same structure can be written with four
different widths and their trees compared, then four inconsistent sources can be tried
for compilation.

- **LF12** — In valid sources, width changes but stays consistent within a block; in
  broken sources, consistency breaks in four different ways. The sources are compiled,
  **not executed** — what is measured is whether the compile phase accepts them.

```python
"""The rule of indentation: width is free, consistency is mandatory."""
import ast

WIDTH = {"two spaces": "if 1:\n  a = 1\n  b = 2\n",
         "four spaces": "if 1:\n    a = 1\n    b = 2\n",
         "eight spaces": "if 1:\n        a = 1\n        b = 2\n",
         "tab": "if 1:\n\ta = 1\n\tb = 2\n"}

BROKEN = {"no indentation at all": "for x in (1, 2):\nb = 1\n",
          "unexpected indent": "a = 0\n    b = 1\n",
          "misaligned dedent": "if 1:\n        a = 1\n    b = 2\n",
          "tab mixed with spaces": "if 1:\n\ta = 1\n        b = 2\n"}


def tree(source):
    d = ast.parse(source).body[0]
    return f"{type(d).__name__}[{','.join(type(g).__name__ for g in d.body)}]"


print("same structure, different width:")
for name, source in WIDTH.items():
    print(f"  {name:12s} tree: {tree(source)}")
print(f"  are all four trees the same: "
      f"{len({tree(k) for k in WIDTH.values()}) == 1}")

print("\ninconsistent indentation:")
for name, source in BROKEN.items():
    try:
        compile(source, "<lesson>", "exec")
        print(f"  {name:24s} compiled")
    except SyntaxError as e:
        print(f"  {name:24s} {type(e).__name__}")
print(f"  is IndentationError a SyntaxError: "
      f"{issubclass(IndentationError, SyntaxError)}")
print(f"  is TabError an IndentationError: "
      f"{issubclass(TabError, IndentationError)}")
```

```
same structure, different width:
  two spaces   tree: If[Assign,Assign]
  four spaces  tree: If[Assign,Assign]
  eight spaces tree: If[Assign,Assign]
  tab          tree: If[Assign,Assign]
  are all four trees the same: True

inconsistent indentation:
  no indentation at all    IndentationError
  unexpected indent        IndentationError
  misaligned dedent        IndentationError
  tab mixed with spaces    TabError
  is IndentationError a SyntaxError: True
  is TabError an IndentationError: True
```

Four widths give a single tree. What is grammatical is not the **amount** of
indentation but its **structure**: a level opens and continues at the same alignment,
and when it closes it lands on an alignment that was already opened. A program written
with two spaces and one written with eight spaces are the same program; the choice
between them is a matter of team convention, not the language.

When consistency breaks, compilation stops in all four cases. Three give
`IndentationError`: a block whose body is never opened, a line that slides inward
without opening anything, and a dedent that lands on no previously opened alignment.
The fourth stands apart — a source mixing tabs and spaces gives `TabError`, because the
width of these two characters cannot be resolved just by looking at the source. The
output's last two lines show all three come from a single lineage: `TabError` is an
`IndentationError`, which is in turn a `SyntaxError`.

This lineage also settles the previous lesson's boundary. Indentation defects are found
at **compile time**; none of them make it to execution. A broken indentation on the
file's last line keeps even the assignment on the first line from running.

## Logical Line vs. Physical Line

Indentation being grammatical stops somewhere, and that limit is worth knowing. What
builds the block structure is the whitespace at the start of a **logical line**; not
every line start on screen begins a logical line. While an expression sits inside an
open bracket, a line ending does not finish the statement — an `NL` is placed in the
stream instead of a `NEWLINE`, and **the indentation of those lines produces no token at
all**. The same continuation can also be built with a backslash.

- **LF13** — Four forms write the same sum and differ only in how they split across
  lines; what is measured is the physical line count, the logical line ending
  (`NEWLINE`), the continuation line ending (`NL`), and the `INDENT` count. The value
  column is the sum the source itself produces.

```python
"""Logical line vs physical line: indentation is not grammatical inside parens."""
import ast
import io
import tokenize

FORM = {
    "single line": "total = 1 + 2 + 3\n",
    "aligned inside parens": "total = (1 +\n          2 +\n          3)\n",
    "ragged inside parens": "total = (1 +\n2 +\n              3)\n",
    "with backslash": "total = 1 + \\\n    2 + \\\n    3\n",
}


def count(source):
    reader = io.StringIO(source).readline
    names = [tokenize.tok_name[t.type] for t in tokenize.generate_tokens(reader)]
    return names.count("NEWLINE"), names.count("NL"), names.count("INDENT")


print(f"{'form':<22s} {'physical':>8s} {'NEWLINE':>7s} {'NL':>3s} {'INDENT':>6s}"
      f"  tree     value")
for name, source in FORM.items():
    logical, nl, indent = count(source)
    env = {}
    exec(compile(source, "<lesson>", "exec"), env)
    tree = " ".join(type(d).__name__ for d in ast.parse(source).body)
    print(f"{name:<22s} {len(source.splitlines()):8d} {logical:7d} {nl:3d} {indent:6d}"
          f"  {tree:8s} {env['total']}")
```

```
form                   physical NEWLINE  NL INDENT  tree     value
single line                   1       1   0      0  Assign   6
aligned inside parens         3       1   2      0  Assign   6
ragged inside parens          3       1   2      0  Assign   6
with backslash                3       1   0      0  Assign   6
```

All four forms give a single `Assign` node and a single value. Even though three of
them spread across **3** physical lines, the `NEWLINE` count is **1** in every one:
there is exactly one logical line. In the two forms inside parentheses, the line
endings in between pass through as `NL`, meaning "no statement ended here." In the
backslash form, the line ending does not turn into a token at all; the continuation is
resolved before the text is even split into tokens.

The most telling column is `INDENT`: **0** in all four. The "ragged" form's second line
starts at column zero, its third starts deep in, and neither changes anything. So
indentation stops being grammatical **inside brackets**; the alignment done there is
entirely for the reader, and inconsistency there produces no defect. The rule can be
narrowed like this: indentation only carries meaning at the start of a logical line.

## Statement and Expression

Indentation builds a block; a block, in turn, is made of **statements**. This requires
the language's second fundamental distinction. An **expression** produces a value: it is
computed and leaves an object behind. A **statement** does a job: it binds a name,
enters a block, directs the flow; it leaves no value behind.

The distinction does not have to be left to intuition, because the compile mode states
it directly. `eval` mode accepts only an expression; `exec` mode accepts a sequence of
statements.

- **LF14** — Every part is attempted in both modes; what is measured is whether it is
  accepted. Parts whose value is printed are evaluated in an environment where a single
  name is bound, so the value column depends on that binding, not on the environment.

```python
"""Statement or expression: 'eval' mode only compiles an expression, 'exec' compiles both."""

PARTS = (
    "2 + 3",
    "total",
    "len('value')",
    "[k for k in (1, 2, 3)]",
    "total if total else 0",
    "total = 5",
    "total += 1",
    "if total: pass",
    "for k in (1, 2): pass",
    "pass",
)

print(f"{'part':<26s} {'expr':>5s} {'stmt':>5s}  value")
expr_count = stmt_count = 0
for p in PARTS:
    try:
        code_obj = compile(p, "<lesson>", "eval")
        is_expr = True
    except SyntaxError:
        is_expr = False
    try:
        compile(p, "<lesson>", "exec")
        is_stmt = True
    except SyntaxError:
        is_stmt = False
    expr_count += is_expr
    stmt_count += is_stmt
    value = repr(eval(code_obj, {"total": 3})) if is_expr else "—"
    print(f"{p:<26s} {str(is_expr):>5s} {str(is_stmt):>5s}  {value}")

print(f"\n{len(PARTS)} parts: expression {expr_count}, statement {stmt_count}; "
      f"statement-only {stmt_count - expr_count}")
```

```
part                        expr  stmt  value
2 + 3                       True  True  5
total                       True  True  3
len('value')                True  True  5
[k for k in (1, 2, 3)]      True  True  [1, 2, 3]
total if total else 0       True  True  3
total = 5                  False  True  —
total += 1                 False  True  —
if total: pass             False  True  —
for k in (1, 2): pass      False  True  —
pass                       False  True  —

10 parts: expression 5, statement 10; statement-only 5
```

**5** of ten parts are expressions, **10** are statements. The statement column being
entirely filled gives the direction of the distinction: **every expression is also a
statement** — placed alone on a line, its value is computed and discarded. The reverse
does not hold; the **5** statement-only parts cannot be placed anywhere an expression
could stand.

The distinction has a direct consequence for the course. All eleven forms the previous
lesson measured **consumed** an object, and what they consumed was an expression's
value; special methods operate on expressions. This is also why it matters that the row
`total += 1` does not come out as an expression: the Programming Fundamentals course
said that a compound assignment is a statement; what is measured here is **which
method** that statement calls and what it does with the returned value. The result of
the `__iadd__` call does not sit around as a value, it is bound straight back to the
name — the detail is paid off in the operators lesson.

One last observation hides in the table: `if total: pass`, even though written on a
single line, is a valid statement and uses no indentation at all. Indentation is not
the **only** way to write a block; what makes a block mandatory is the statement itself,
and indentation is the way to write a body with more than one statement.

## Summary

- Indentation is not discarded; the whitespace at the start of a line is turned into
  `INDENT` and `DEDENT` tokens, and it is these tokens that build the block structure.
- Two sources differing only in indentation carry the same **21** content tokens in the
  same order; the position of a single `DEDENT` token changes the tree and the result.
- Indentation's width is free — four different widths give a single tree — but it must
  be consistent within a block.
- Inconsistent indentation stops at compile time; `TabError` is an `IndentationError`,
  which is in turn a `SyntaxError`.
- Indentation is only grammatical at the start of a logical line: **3** physical lines
  inside an open bracket produce a single `NEWLINE`, and the `INDENT` count stays **0**.
- **5** of ten parts are expressions, **10** are statements: every expression can be
  written as a statement, no statement can stand in for an expression.

## Next Step

In this lesson, assignment — the most commonly written statement — showed up only as
"not an expression" in the table; what it does was never examined. The Programming
Fundamentals course established that assignment is a binding — a name is not a box, it
is a label attached to an object. The next lesson measures this bond from the object's
side: what does it mean when two names are bound to the same object, under what
condition do two equal values turn out to be the same object, and how does a type being
"immutable" affect binding?