---
title: 'Cutting, Pasting, and Splitting'
source: 'https://academia.sh/en/courses/shell-programming/cutting-pasting-and-splitting'
course: 'Shell Programming'
language: en
updated: '2026-08-17T18:10:03+00:00'
license: 'CC BY-SA 4.0'
---

# Cutting, Pasting, and Splitting

Column cutting and its limits, pasting files side by side, character-level transformation, splitting a file into pieces, and the tool-selection criterion.

The tools of the previous lessons worked at the line level: they selected lines,
transformed lines, sorted lines. Column-level cutting, bringing two files side by side,
character-level mapping, and splitting a large file into pieces requires a separate
family of tools.

This lesson introduces that family and closes out the text processing topic with a
criterion that decides which job belongs to which tool.

## Column Cutting

`cut` has been used in this course since the first lesson. It has two modes: field
mode (`-d` and `-f`) and character mode (`-c`).

```sh
head -2 access.log | cut -d' ' -f1,7,9
```

```
10.0.0.12 /index.html 200
10.0.0.31 /static/style.css 200
```

```sh
head -2 access.log | cut -c1-11
```

```
10.0.0.12 -
10.0.0.31 -
```

Character mode is used on fixed-width output — aligned tables, fixed-field records. It
is useless on data whose field count varies.

`cut` is fast and short to write, but it has three firm limits.

**The delimiter is a single character.**

```sh
printf 'a::b::c\n' | cut -d'::' -f2
```

```
cut: bad delimiter
```

The field-based tool does the same job with a regular-expression delimiter:

```sh
printf 'a::b::c\n' | awk -F'::' '{ print $2 }'
```

```
b
```

**Consecutive delimiters are not merged.** Every delimiter starts a new field:

```sh
printf 'a  b  c\n' | cut -d' ' -f2
printf 'a  b  c\n' | awk '{ print $2 }'
```

```

b
```

`cut` found the second field empty, because there is an empty field between the two
spaces. The field-based tool's default delimiter is a run of whitespace, so it
returned `b`. In aligned or hand-written text, this difference is decisive.

**Field order cannot be changed.** Whatever order the requested field numbers are
written in, the output follows the order in the file:

```sh
head -1 access.log | cut -d' ' -f9,1
```

```
10.0.0.12 200
```

Reordering requires the field-based tool:

```sh
head -1 access.log | awk '{ print $9, $1 }'
```

```
200 10.0.0.12
```

These three limits leave `cut` a clear domain: **data separated by a single
character, with no consecutive delimiters, whose field order will not change.** The
log file fit this definition, which is why it was used in the course's earliest
topics.

## Pasting Side by Side

`paste` merges the corresponding lines of files. It is the reverse of the `cut`
operation.

```sh
cut -d' ' -f7 access.log | head -3 > path.txt
cut -d' ' -f9 access.log | head -3 > code.txt
paste path.txt code.txt
```

```
/index.html	200
/static/style.css	200
/product/12	200
```

The default separator is a tab; `-d` changes it:

```sh
paste -d: path.txt code.txt
```

```
/index.html:200
/static/style.css:200
/product/12:200
```

The `-s` option merges each file's lines into a single line; it is the short way to
turn a list into comma-separated text:

```sh
paste -sd, code.txt
```

```
200,200,200
```

`paste` assumes lines match up **positionally**. If a line is missing in one file, the
whole alignment shifts and the result is silently wrong. If matching by key is needed,
`join` from the previous lesson is used; `paste` is suited only to data whose order can
be trusted.

## Character-Level Transformation

`tr` maps input characters one by one. It recognizes no pattern, no field, no line —
it works only on character sets, and it only reads from standard input.

```sh
printf 'hello\n' | tr 'a-z' 'A-Z'
```

```
HELLO
```

Three options are the most frequently used:

```sh
printf 'aaa  bbb\n' | tr -s ' '     # collapse consecutive repeats to one
printf 'a1b2c3\n'   | tr -d '0-9'   # delete characters in the set
printf 'a1b2c3\n'   | tr -cd '0-9'  # delete characters outside the set
```

```
aaa bbb
abc
123
```

Collapsing consecutive spaces to one with `tr -s ' '` is a common preparation step for
making a file usable with `cut`.

Its most frequent use is turning a delimiter into a line break to spill every field
onto its own line:

```sh
tr ' ' '\n' < access.log | grep -c '^"GET$'
```

```
27
```

`tr` is far cheaper than pattern-based tools because it parses nothing. If a character
mapping is enough, there is no need to call the stream editor.

## Splitting a File into Pieces

`split` divides a large file into pieces of fixed size or a fixed number of lines.

```sh
mkdir -p parts
split -l 10 access.log parts/log-
ls parts
```

```
log-aa
log-ab
log-ac
```

```sh
wc -l parts/*
```

```
      10 parts/log-aa
      10 parts/log-ab
      10 parts/log-ac
      30 total
```

`-l N` splits by line count, `-b N` by byte count. The suffix length is set with `-a`;
the default of two letters is enough for up to 676 pieces.

There are two uses. First, processing piece by piece a file that does not fit within a
memory limit. Second, distributing the pieces to separate processes for parallel
processing — if the pieces can be processed independently, the total time is divided
by the piece count.

It should be kept in mind that splitting by byte boundary can cut a line in half; for
line-based data, `-l` is used instead of `-b`.

## Choosing a Tool

This topic's six lessons built a family of tools looking at the same data from
different angles. The criterion for choosing among them is the shape of the question:

| Question | Tool |
|---|---|
| Which lines fit a pattern | filter (`grep`) |
| Change the text inside a line | stream editor (`sed`) |
| Compute over fields, accumulate per key | field-based tool (`awk`) |
| Take a column from fixed-delimiter data | `cut` |
| Merge columns | `paste`, `join` |
| Map, delete, compress characters | `tr` |
| Sort, deduplicate, do set operations | `sort`, `uniq`, `comm` |

Two general rules stand above this table.

**Choose the narrowest tool.** The field-based tool can do nearly every job in the
table; writing everything with it is still wrong. A narrow tool is shorter to write,
runs faster, and tells the reader its intent. The form `awk '/404/ { print }'` does the
same job as `grep 404`, but it makes the reader wonder "is there a computation here."

**If a problem takes more than three tools to solve, switch tools.** Five-stage
pipelines can be written, and were written in this course; but if every line in a
script looks like that, the job should be considered to have crossed the shell's
boundary. The shell is unrivaled at connecting processes; it is not designed for
complex data structures and multi-step business logic.

## Applying to the Script

At the end of the text processing topic, `report.sh` produces all three of its
sections with their proper tools:

```sh
summary() {                     # summary LOG
  awk '
    { requests++; bytes += $10; class[substr($9, 1, 1)]++ }
    END {
      printf "requests: %d\n", requests
      printf "bytes   : %d\n", bytes
      printf "average : %.1f bytes\n", (requests ? bytes / requests : 0)
      for (s = 2; s <= 5; s++)
        printf "  %sxx %4d  %%%.1f\n", s, class[s], (requests ? class[s] * 100 / requests : 0)
    }
  ' "$1"
}

top_paths() {                   # top_paths LOG COUNT
  cut -d' ' -f7 "$1" \
    | sed -E 's#/[0-9]+#/:id#g' \
    | sort | uniq -c | sort -k1,1nr -k2,2 | head -"$2"
}

error_paths() {                 # error_paths LOG
  awk '$9 ~ /^[45]/ { n[$7]++ } END { for (y in n) printf "%3d %s\n", n[y], y }' "$1" \
    | sort -k1,1nr -k2,2
}
```

Every function answers a single question, follows a single data path, and its output
is deterministic. The script's skeleton, however, is still incomplete: it produces a
half report on error, leaves temporary files behind, and expects to be run by hand.

## Summary

- `cut` wants a single-character delimiter, does not merge consecutive delimiters,
  and cannot reorder fields; outside these three limits it is the cheapest column
  tool.
- `paste` brings files together positionally; if key matching is needed, `join` is
  used.
- `tr` works on character sets; it recognizes no pattern and reads only from
  standard input.
- `split` divides a file by line or byte boundary; `-l` is used for line-based data.
- The narrowest tool is preferred when choosing; if a problem needs more than three
  tools, the shell's boundary should be considered crossed.

## Next Step

The script produces correct output, but only when everything goes right. When a
command fails, the report is left half-finished and the exit code does not report it;
an undefined variable silently expands to an empty string. The next topic closes these
gaps and starts with strict-mode options: which errors should stop the script, to what
extent each option provides that, and the traps the options carry of their own.