---
title: 'Line Filtering'
source: 'https://academia.sh/en/courses/shell-programming/line-filtering'
course: 'Shell Programming'
language: en
updated: '2026-08-17T18:10:03+00:00'
license: 'CC BY-SA 4.0'
---

# Line Filtering

Selecting, counting, and inverting lines by pattern, word and full-line matching, fixed string mode, multi-file search, and using the exit code as a condition.

The pattern language is built. The first tool that applies it is the filter that
selects, from a stream, the lines that fit a pattern. This tool is the most frequently
called of shell text processing, and most of its options turn the behavior "write the
matching lines" into a different question.

This lesson takes up those questions and the correct option for each.

## Selecting, Counting, Locating

The default behavior is to write the matching lines. The `-n` option adds the line
number:

```sh
grep -n '500' access.log
```

```
8:192.168.1.5 - - [07/Feb/2024:09:17:22 +0000] "GET /api/data HTTP/1.1" 500 180
13:192.168.1.5 - - [07/Feb/2024:09:21:38 +0000] "GET /api/data HTTP/1.1" 500 180
27:10.0.2.7 - - [07/Feb/2024:09:35:44 +0000] "POST /api/data HTTP/1.1" 500 180
```

`-c` gives only the number. What is counted is the **number of matching lines**, not
the number of matches:

```sh
grep -Ec  '\.' access.log
grep -Eo '\.' access.log | wc -l
```

```
30
     131
```

All thirty lines have at least one dot; the total dot count is 131. The distinction
separates the questions "how many requests got a 404" from "how many times does the
string 404 occur." If a match count is wanted, `-o` spills each match onto its own line
and it is counted.

The `grep -c ''` form gives a line count, because the empty pattern matches every line,
and it does not produce `wc -l`'s alignment padding:

```sh
grep -c '' access.log
```

```
30
```

## Inverting

The `-v` option selects lines that do **not** match. It is the most-used option in
filtering jobs; it is the direct way to drop noise lines.

```sh
grep -vc '" 200 ' access.log
```

```
13
```

Thirteen of thirty requests got a code other than `200`. `-v` can be combined with
`-c`; combining it with `-o` is meaningless, because a non-matching line has no match to
write.

## Narrowing the Boundaries of a Match

Three options restrict which part of the line a pattern can match against.

`-i` removes case sensitivity. `-w` requires the pattern to start and end at **word
boundaries**. `-x` requires the pattern to match the **entire line**.

```sh
printf '%s\n' 'GET /a' 'get /b' 'GETTER /c' > case.txt
grep -c  'GET' case.txt
grep -ic 'GET' case.txt
grep -wc 'GET' case.txt
```

```
2
3
1
```

The file contains the lines `GET /a`, `get /b`, and `GETTER /c`. The default search
selected two, the case-insensitive search three, the word search only one — `GET`
inside `GETTER` is not a word.

`-x` is a shortcut for writing anchors:

```sh
printf '%s\n' '404' '4040' > code.txt
grep -c  '404' code.txt
grep -xc '404' code.txt
```

```
2
1
```

The line `4040` in the file fit the unanchored pattern. In checks that want an exact
match, forgetting `-x` silently produces an inflated result.

## Fixed String Mode

The `-F` option interprets the pattern not as a regular expression but as a literal
string.

```sh
printf '%s\n' 'a.b' 'axb' > dot.txt
grep -c  'a.b' dot.txt
grep -Fc 'a.b' dot.txt
```

```
2
1
```

`-F` matters for two reasons. First, correctness: if the pattern comes from a variable
— user input, a file name, a configuration value — the `.`, `*`, `[` characters inside
it are interpreted as pattern syntax, and the search gives a wider result than
expected. Second, using user input as a pattern is a security boundary: a crafted
pattern can deliberately blow up the matching cost.

Rule: **if you did not write the pattern yourself, use `-F`.**

Patterns can be kept in a file; the `-f` option counts each line as a separate pattern:

```sh
printf '%s\n' '403' '500' > patterns.txt
grep -Fcf patterns.txt access.log
```

```
5
```

## Multi-File Search

When more than one file is given, the file name is prefixed to the output:

```sh
mkdir -p log-dir
head -10 access.log > log-dir/monday.log
tail -10 access.log > log-dir/tuesday.log
grep -c '404' log-dir/*.log
```

```
log-dir/monday.log:1
log-dir/tuesday.log:2
```

`-h` suppresses this prefix, `-H` adds it even for a single file. If the output is
going to be parsed in a script, the option must be written explicitly; an output whose
prefix changes with the file count is a fragile foundation.

`-l` gives the names of files where a match was found, `-L` the names where it was not:

```sh
grep -l '500' log-dir/*.log
```

```
log-dir/monday.log
log-dir/tuesday.log
```

`-r` searches a directory tree recursively. The order files are visited is whatever
order the file system gives, and it varies from implementation to implementation; if
sorted output is needed, the result must be sorted separately with `sort`.

`-m N` stops after the first `N` matches in each file. For the question "does it exist"
on large files, this avoids reading the whole file.

## Context Lines

`-A N` writes the `N` lines after a match, `-B N` before, `-C N` on both sides. A `--`
separator is placed between blocks.

```sh
grep -A 1 '" 500 ' access.log | head -4
```

```
192.168.1.5 - - [07/Feb/2024:09:17:22 +0000] "GET /api/data HTTP/1.1" 500 180
10.0.0.12 - - [07/Feb/2024:09:18:00 +0000] "GET /api/data HTTP/1.1" 200 3300
--
192.168.1.5 - - [07/Feb/2024:09:21:38 +0000] "GET /api/data HTTP/1.1" 500 180
```

This is used in debugging to look at the requests before and after a fault. Because
the separator line breaks the output's format, context options are not used on lines
whose output will flow into another tool.

## Using the Exit Code as a Condition

The `-q` option writes nothing; it stops at the first match and reports the result only
through its exit code. This is the correct form of an "does it exist" question in a
script:

```sh
if grep -q '" 500 ' access.log; then
  echo "server error found"
fi
```

```
server error found
```

The form `grep -c ... > /dev/null` does not do the same job: `-c` writes `0` even
without a match, and the exit code still depends on whether there was a match, but the
entire file is read regardless. `-q` stops at the first match.

The three-code contract built in the Exit Codes lesson applies here: `0` found, `1` not
found, `2` file error. In scripts running under strict mode, the "not found" case must
be used as a condition, or `|| true` must be added, so it does not stop the script.

## Applying to the Script

`report.sh` gains two new sections. The first counts the log's broken lines; the
second lists the paths that returned an error.

```sh
log_file=access.log

LINE_PATTERN='^[0-9.]+ - - \[[^]]+\] "[A-Z]+ [^ ]+ HTTP/[0-9.]+" [0-9]{3} [0-9]+$'

malformed=$(grep -Evc "$LINE_PATTERN" "$log_file")
if [ "$malformed" -gt 0 ]; then
  warn "$malformed lines are not in the expected format and are included in the counts"
fi

section "Paths returning errors"
grep -E '" [45][0-9]{2} ' "$log_file" | cut -d' ' -f7 | sort | uniq -c | sort -rn
```

When run with a log that has a broken line:

```sh
./report.sh all.log
```

The warning falls to standard error, the report continues to standard output. On a
clean log, the error-returning-paths section gives this:

```
   4 /missing.html
   3 /api/data
   2 /secret
```

The quotes and spaces in the `grep -E '" [45][0-9]{2} '` pattern separate the status
code field from another number in the log. This is imitating the concept of a field
with a pattern; in the following lessons, field-based tools will do the same job by
position, and the pattern will no longer be needed.

## Filtering's Place in the Pipeline

Concurrency was established in the pipeline lesson. Its practical consequence is an
ordering rule: **put the step that shrinks the data at the start of the pipeline.**

```sh
grep -E '" [45][0-9]{2} ' "$log_file" | cut -d' ' -f7 | sort | uniq -c
```

In this pipeline, `grep` brings thirty lines down to nine; `sort` sorts nine lines. Had
sorting come last, thirty lines would have been sorted. That sorting cost depends on
input size — as shown in the Algorithms course — is the justification for this rule.

The rule carries one exception: filtering must not drop lines the next step needs. In
transformations that require context, filtering is deferred to later.

## Summary

- `-c` counts matching lines, not matches; a match count is spilled into lines with
  `-o` and counted.
- `-v` inverts the selection; `-w` requires a word match, `-x` a full-line match.
- `-F` interprets the pattern literally and is required for patterns coming from
  outside.
- In multi-file search, the output prefix changes with the file count; in a script,
  `-h` or `-H` is written explicitly.
- `-q` writes nothing and stops at the first match; it is the correct form of an
  existence check in a script.
- The filter that shrinks the data is placed at the start of the pipeline.

## Next Step

Filtering selects lines but does not change them. Reformatting the timestamp in the
log, simplifying field delimiters, or masking a field requires transforming the line
itself. The next lesson takes up the stream editor: substitution, address and range
selection, rewriting with backreferences, and the mismatch between implementations in
the in-place editing option.