Special Variables and Arguments

The log’s name now comes from a variable, but this is not the natural path for whoever calls the script. Instead of writing LOG_FILE=access.log ./report.sh, writing ./report.sh access.log is expected. Values coming from the command line are handled in the shell by a separate mechanism — positional parameters.

In the Programming Fundamentals course, function parameters were defined by name. There is no naming in the shell: parameters are numbered and read by their numbers. This lesson establishes those numbers, the two ways of treating the list as a whole, and option parsing.

Positional Parameters

$1, $2, … give the arguments the script received, in order. $0 holds the name the script was called with, $# the argument count.

#!/usr/bin/env bash
echo "script name    : $0"
echo "argument count : $#"
echo "first          : $1"
echo "second         : $2"
echo "tenth          : ${10}"

./args.sh a b c d e f g h i j

script name    : ./args.sh
argument count : 10
first          : a
second         : b
tenth          : j

Numbers greater than nine require braces. If $10 had been written, the shell would read it as $1 and 0, producing a0 by appending a zero after the first argument. Using braces is therefore mandatory.

$0 is the name the script was invoked with; it does not have to be the file’s real name, and it may not contain an absolute path. Writing $0 in usage messages is common, but hardcoding the script’s name gives more readable output.

The Argument List: "$@" and "$*"

There are two representations for passing all arguments together, and the difference between them shows up only inside quotes.

• "$@" expands every argument as a separate word.
• "$*" joins all arguments with IFS’s first character and produces a single word.
• Unquoted $@ and $* both undergo word splitting, and the distinction disappears.

#!/usr/bin/env bash
show_args() { printf 'count=%d\n' "$#"; printf '  <%s>\n' "$@"; }
echo '--- "$@" ---'; show_args "$@"
echo '--- "$*" ---'; show_args "$*"
echo '--- $@   ---'; show_args $@

./list.sh "access log.log" 404

--- "$@" ---
count=2
  <access log.log>
  <404>
--- "$*" ---
count=1
  <access log.log 404>
--- $@   ---
count=3
  <access>
  <log.log>
  <404>

The three results carry three different meanings. The first passes the list while preserving the boundaries the caller gave. The second turns the list into a single piece of text — useful for nothing but printing a message. The third splits the space-containing argument in two and breaks the caller’s intent.

Rule: always use "$@" when passing arguments on to another command. This is the quoting rule from the previous lesson extended to positional parameters.

Consuming the List

The shift command shifts positional parameters one to the left: $2 becomes the new $1, $# decreases by one. It is the foundation of loops that process arguments in order.

#!/usr/bin/env bash
while [ "$#" -gt 0 ]; do
  echo "processing: $1 (remaining: $#)"
  shift
done

./shift.sh one two three

processing: one (remaining: 3)
processing: two (remaining: 2)
processing: three (remaining: 1)

The form shift N discards N parameters at once. Trying to discard more than the remaining parameter count returns a nonzero code and leaves the list unchanged.

Script-Related Special Variables

The shell holds several values about the running script under special names.

Name	Content
$?	the last command’s exit code
$$	the running shell’s process number
$!	the number of the last process started in the background
$-	active shell options
$0	the name the script was called with
$#	positional parameter count

$$ is often used to generate temporary file names; but because process numbers can be reused, it is not safe on its own. The Traps and Cleanup lesson will show the correct way with mktemp.

$! is needed for waiting on and stopping background processes; in the Exit Codes lesson, signal codes were produced using this variable.

Option Parsing

Parsing arguments by hand — if the first argument is an option, if the second is its value, and so on — quickly becomes unreadable. The getopts builtin parses short options in a way that follows the convention.

getopts takes two arguments: the string of recognized options, and the name of the variable the found option is written to. If a letter in the string is followed by :, that option expects a value; the value is placed in the OPTARG variable. A : at the start of the string declares that error messages will be produced by the script, not by getopts.

#!/usr/bin/env bash
count=5
verbose=0
while getopts ':n:va' option; do
  case "$option" in
    n)  count="$OPTARG" ;;
    v)  verbose=1 ;;
    a)  echo "usage: opts.sh [-n NUMBER] [-v] LOG_FILE"; exit 0 ;;
    :)  echo "error: -$OPTARG expects a value" >&2; exit 2 ;;
    \?) echo "error: unknown option -$OPTARG" >&2; exit 2 ;;
  esac
done
shift $((OPTIND - 1))
echo "count=$count verbose=$verbose remaining=$# first=[${1-none}]"

Four calls show four behaviors:

./opts.sh -n 3 -v access.log
./opts.sh access.log
./opts.sh -n
./opts.sh -z

count=3 verbose=1 remaining=1 first=[access.log]
count=5 verbose=0 remaining=1 first=[access.log]
error: -n expects a value
error: unknown option -z

The last two calls ended with code 2. Reporting a usage error with a code separate from another error class is what makes the script usable in a chain.

OPTIND holds the index of the last argument getopts processed. The shift $((OPTIND - 1)) line after the loop discards the options from the list; only the non-option arguments remain. If this line is forgotten, $1 still points at the first option.

getopts recognizes only single-letter options; there is no support for long options like --verbose. If a long option is needed, hand-written parsing with while–case is written instead. Also, the backslash in the \? written in the case branches is required: ? is a special symbol in case patterns that matches a single character.

The End-of-Options Marker

A value starting with a dash can be mistaken for an option. For example, a file named -old.log is interpreted as an option by most commands. The convention is that once -- is seen in the argument list, everything after it is accepted as not being an option:

grep -- '-404-' access.log

In your own scripts, getopts also recognizes this marker and stops there. Writing -- when passing file names coming from a variable to another command prevents user input from being confused with an option.

Applying It to the Script

report.sh now gains an interface: how many lines to list is given with -n, the log name with the first positional parameter.

#!/usr/bin/env bash
# report.sh — produces a summary report from the access log.
# Usage: report.sh [-n NUMBER] [LOG_FILE]

count=5

while getopts ':n:' option; do
  case "$option" in
    n)  count="$OPTARG" ;;
    :)  echo "error: -$OPTARG expects a value" >&2; exit 2 ;;
    \?) echo "error: unknown option -$OPTARG" >&2; exit 2 ;;
  esac
done
shift $((OPTIND - 1))

log_file="${1:-access.log}"

echo "== Log: $log_file (top $count paths) =="
cut -d' ' -f7 "$log_file" | sort | uniq -c | sort -rn | head -"$count"

./report.sh -n 3

== Log: access.log (top 3 paths) ==
   6 /api/data
   4 /missing.html
   4 /index.html

The script is still missing something: if the given log does not exist, cut errors out and the report header has been printed for nothing. This gap is the next lesson’s subject.

Summary

• Positional parameters start at $1; numbers greater than nine require braces. $# gives the count, $0 the name it was called with.
• "$@" expands arguments as separate words, "$*" as a single word; unquoted, both undergo word splitting. "$@" is used for passing them on.
• shift shifts the list one to the left and makes a loop that consumes arguments in order possible.
• getopts parses short options; OPTARG holds the value, OPTIND the processed index, and shift $((OPTIND - 1)) is required after the loop.
• The -- marker declares that what follows is not an option, and protects values that begin with a dash.

Next Step

The script assumes that the log given to it exists and is readable. Testing this assumption requires a condition; in the shell, a condition is a command’s exit code. The next lesson takes up file, string, and number tests, the difference between [ ] and [[ ]], and why this difference is a matter of correctness rather than syntax preference.