Static Analysis

The previous lesson’s tools find an error after it has occurred. Most errors in shell scripts, though, are determined before they occur: an unquoted expansion, an unchecked exit code, building a file list from a command’s output, using a feature the interpreter declaration does not permit.

These errors can be seen by looking at the code itself, without running it. Static analysis is the method of parsing source text and searching for known faulty patterns. For the shell, this job is delegated to a tool that parses the script and applies hundreds of pattern rules.

The tool’s value is that it remembers, all at once, every trap this course has explained one by one.

An Inspection

The script below carries most of the errors covered in previous lessons at once:

#!/bin/sh
log=$1
for f in $(ls logs); do
  echo "processing: $f"
done
if [ $log = "access.log" ]; then
  count=`wc -l < $log`
  echo "lines: $count"
fi
cd /tmp
rm -rf $log_backup

The analyzer’s long-form output marks the finding beneath the source line and suggests a corrected form. The same findings read more compactly in single-line form:

shellcheck -f gcc flawed.sh

flawed.sh:3:10: error: Iterating over ls output is fragile. Use globs. [SC2045]
flawed.sh:6:6: note: Double quote to prevent globbing and word splitting. [SC2086]
flawed.sh:7:9: note: Use $(...) notation instead of legacy backticks `...`. [SC2006]
flawed.sh:7:18: note: Double quote to prevent globbing and word splitting. [SC2086]
flawed.sh:10:1: warning: Use 'cd ... || exit' or 'cd ... || return' in case cd fails. [SC2164]
flawed.sh:11:8: warning: log_backup is referenced but not assigned. [SC2154]

Five separate error classes, in an eleven-line script. The last one’s consequence is rm -rf running with no argument at all in a script without set -u.

The default format is more detailed: it writes the corrected line under every finding with the heading Did you mean:. This format suits working with a single finding; for a batch inspection, the single-line format above is appropriate.

Error Classes Caught

Warnings fall into four groups, and each was the subject of a separate lesson in this course.

Expansion and quoting. Unquoted variable expansion, using $* instead of "$@", the wrong form of array expansion. The entire Variables and Quoting Rules lesson belongs to this class. This is the most frequently seen warning.

Exit code. An unchecked cd, unchecked command substitution, assignment combined with local, a pipeline code getting lost. The traps from the Error Handling lesson are caught here.

Shell idioms. Building a file list from a command’s output, a missing -r in a read call, the old backtick notation, the wrong operator inside [ ]. Warnings from the Loops and Conditionals lessons.

Undefined name. Reading a variable that was never assigned, variables that turn into a different name through a typo. This class can go unnoticed even at run time in an interpreted language.

Every warning has an identifier (SC and four digits). The identifier serves two purposes: reaching the warning’s detailed explanation, and — when needed — suppressing that warning in a targeted way.

Severity Levels

Warnings fall into four levels: error, warning, info, style. The -S option hides everything below a given level:

shellcheck -S warning -f gcc flawed.sh

flawed.sh:3:10: error: Iterating over ls output is fragile. Use globs. [SC2045]
flawed.sh:10:1: warning: Use 'cd ... || exit' or 'cd ... || return' in case cd fails. [SC2164]
flawed.sh:11:8: warning: log_backup is referenced but not assigned. [SC2154]

Three findings remain; the quoting and backtick warnings at the info and style levels were hidden.

This is useful when adding analysis to an existing codebase: the error level is cleaned up first, then the threshold is lowered in stages. Keeping the threshold permanently high, on the other hand, means giving up most of the tool’s value — info-level quoting warnings are the ones that most often turn into real errors.

Portability Checking

The analyzer reads the interpreter declaration and applies its rules accordingly. In a script declaring #!/bin/sh, bash extensions count as errors:

#!/bin/sh
codes=(200 404)
if [[ ${#codes[@]} -gt 1 ]]; then
  echo "multiple codes"
fi

In portable.sh line 2:
codes=(200 404)
      ^-------^ SC3030 (warning): In POSIX sh, arrays are undefined.


In portable.sh line 3:
if [[ ${#codes[@]} -gt 1 ]]; then
   ^----------------------^ SC3010 (warning): In POSIX sh, [[ ]] is undefined.

This is the rule from the Script Anatomy lesson enforced by a tool: declaring POSIX while using a bash feature causes the script to break silently on another system. When the same script declares bash, these two warnings are not produced.

The limit of this check can be seen from here too: the tool decides based on the declared shell. If the declaration is wrong, the check is done wrong. This is why the interpreter declaration must be written to match the features the script actually uses.

Warning Suppression

When a warning needs to be deliberately ignored, a directive is written into the source code:

#!/usr/bin/env bash
# shellcheck disable=SC2086
log="a b.log"
echo $log

The directive applies to the command after it; placed at the top of the file, before the first command, it covers the whole file.

Suppression is bound by three rules.

An identifier is given. The disable directive can be written without an identifier, but then it turns off every warning and also hides real errors that will occur in the future.

The scope is narrowed. File-level suppression takes every line in that file out of the check. Writing it directly above the line where the warning arises is preferred.

A justification is written. Why the suppressed warning is a false alarm is stated with a comment. A suppression with no justification tells the next person reading the code, “I do not know what is going on here.”

In cases where word splitting is genuinely wanted, there is a better solution: putting the value into an array. The notation "${array[@]}" produces no warning and also makes the intent visible in the code. Suppression is reserved for cases where rewriting is not possible.

Putting the Script Through Inspection

report.sh has been written throughout this course avoiding the patterns the analyzer would warn about. The small script below follows the same rules:

#!/usr/bin/env bash
set -Eeuo pipefail
log="${1:-access.log}"
[ -r "$log" ] || { echo "cannot read: $log" >&2; exit 3; }
count=$(grep -c '' "$log")
printf 'lines: %d\n' "$count"

shellcheck clean.sh; echo "code=$?"

code=0

The exit code is 0 and there is no output. The tool returns a nonzero code if there is a finding; this makes it possible to turn the check into a validation step:

shellcheck report.sh && bash -n report.sh && echo "check passed"

Running this command before a submission to version control, or as a continuous integration step, is the cheapest way to keep the quality of shell scripts intact.

The Limit of Static Analysis

The tool sees the code’s form, not its meaning. What it cannot catch:

• A wrong field number (writing $8 instead of $7).
• A wrong regular expression (the pattern is valid but does not describe the intended set).
• Wrong business logic (a percentage divided by the wrong total).
• Errors that depend on the runtime environment (a nonexistent command, insufficient permission).

These classes require testing: running the script with a known input and comparing its output against the expected output. This course’s access.log file being fixed and reproducible exists precisely to make this kind of testing possible.

Static analysis does not substitute for testing; it reduces the number of errors testing has to find.

Summary

• Static analysis parses a script without running it and searches for known faulty patterns.
• The main classes caught are quoting, exit codes, shell idioms, and undefined names; all were the subject of separate lessons in this course.
• The check is done according to the interpreter declaration; bash extensions produce warnings in a script that declares POSIX.
• Suppression is done with an identifier, a narrow scope, and a justification; if word splitting is wanted, the correct solution is using an array.
• The tool returns a nonzero code if there is a finding; this turns the check into a validation step.
• Static analysis sees form, not meaning; a wrong field number and wrong business logic can only be found through testing.

Next Step

The script is correct, robust, and checked — but it is still run by hand. A report’s value lies in its being produced regularly. The final lesson takes up scheduled tasks: reading a cron entry, at which points the scheduler’s environment departs from an interactive shell, where output should go, and preventing overlapping runs.