--- title: 'Scheduled Tasks' source: 'https://academia.sh/en/courses/shell-programming/scheduled-tasks' course: 'Shell Programming' language: en updated: '2026-08-17T18:10:00+00:00' license: 'CC BY-SA 4.0' --- # Scheduled Tasks Reading a cron entry, how the scheduler's environment differs from an interactive shell, managing output, and a wrapper that prevents overlap. The script is correct, robust, and checked — but it is still run by hand. A report's value lies in its being produced regularly: a tool that analyzes the previous day's log every morning and drops the result somewhere is a different thing from the same tool run by hand. This lesson takes up preparing a script for scheduled execution. The subject is not just the scheduling definition; the real issue is that the script will be run by a **non-human caller**, in a **narrow environment**, **without anyone watching**. ## The Cron Entry On Unix systems, scheduled tasks are defined to a scheduler service. The definition table is kept per user, and every line consists of five time fields and a command: ``` minute hour day of month month day of week command ``` | Field | Range | |---|---| | minute | 0–59 | | hour | 0–23 | | day of month | 1–31 | | month | 1–12 | | day of week | 0–7 (0 and 7 are Sunday) | Every field takes one of four forms: a fixed number, `*` (every value), `a-b` (a range), `a,b,c` (a list). Appending `/n` to a range selects every `n`th step. ``` 30 6 * * * every day at 06:30 0 */4 * * * every four hours, on the hour 15 2 * * 1 every Monday at 02:15 0 0 1 * * midnight on the first of every month */10 * * * * every ten minutes ``` Two fields together narrow the constraint, but the **day-of-month and day-of-week** pair is the exception: if both take a value other than `*`, the conditions combine with "or," not "and." The entry `0 0 13 * 5` runs on the 13th of the month **or** on Friday. This is the most commonly misread point in these entries. The definition table is edited with `crontab -e`, listed with `crontab -l`. `crontab -r` deletes the table **without asking for confirmation**; since it is a single letter away from `-e`, the habit of listing should be built around `-l`. Some schedulers recognize shorthands such as `@daily`, `@hourly`, `@reboot`. These are not defined in POSIX; portable entries use the five-field form. Another class of scheduler service is the service manager's **timer units**. They offer the ability to define dependencies, make up for missed runs, and merge run logs with the service infrastructure. The service manager model is the subject of the System Administration course. ## The Scheduler's Environment The most common reason scheduled tasks fail is not the script, it is the environment. The scheduler runs the command **without a login shell**: shell startup files are not read, there are no shell functions or aliases, and most environment variables are undefined. The most visible difference is the search path. The same script, run in a narrow environment: ```sh env -i PATH=/usr/bin:/bin bash env.sh ``` ``` PATH=/usr/bin:/bin is shellcheck found: no ``` A tool present in an interactive session may not be found in a scheduled run. The result shows up as "the script works by hand but not under the scheduler," and since the scheduler is silent, noticing it is delayed. Three precautions are taken. **State the search path in the script.** Writing `export PATH=...` at the top of the script gives independence from the calling environment. If the paths of the tools used are known, this is the most robust solution. **Write file paths as absolute.** The scheduler's starting directory is not guaranteed; relative paths resolve in a different directory. Every path the script reads or writes must be absolute. **Define the environment variables needed inside the script.** Locale, language, and time zone variables are not inherited from the interactive session. This is also why the line `export LC_ALL=C` was placed at the top of the script in this course: sort order would otherwise stay ambiguous in the scheduler's environment. There is one more detail specific to the definition table: the `%` character in the command field means end of line, and cannot be written without escaping. Commands containing `%`, such as date formatting, are not written directly into the table — they go into a wrapper script. ## Where Output Goes The scheduler tries to collect the standard output and standard error the task produces and deliver them to the user. If delivery is not configured, the output is lost or piles up in a local mail queue. This behavior produces two consequences. First, a task that produces output on every run generates constant notifications; the notifications become unread after a while. Second, error messages get lost the same way. The correct arrangement is to manage output explicitly: ``` 30 6 * * * /opt/report/report-job.sh ``` and doing the redirection **inside** the script. Done inside the script, the paths stay readable, the `%` problem does not arise, and the redirection logic enters version control. The stream separation set up in this topic's first lesson pays off here: the report itself is written to one file, diagnostic messages to a separate one. If the two are mixed, the next step reading the report mistakes the warning text for data. ## The Wrapper Script The scheduled task does not call the working script directly; it wraps it in a **wrapper**. The wrapper sets up the environment, acquires the lock, redirects the output, and calls the working script. ```sh #!/usr/bin/env bash set -Eeuo pipefail export PATH=/usr/local/bin:/usr/bin:/bin export LC_ALL=C ROOT="/opt/report" LOCK="$ROOT/lock" LOG_DIR="$ROOT/log" mkdir -p "$LOG_DIR" if ! mkdir "$LOCK" 2>/dev/null; then echo "previous run still in progress" >&2 exit 75 fi trap 'rmdir "$LOCK"' EXIT "$ROOT/report.sh" -n 3 "$ROOT/access.log" \ >> "$LOG_DIR/report.out" 2>> "$LOG_DIR/report.err" ``` Five decisions stand together here. **Lock.** The `mkdir` pattern set up in the Traps and Cleanup lesson finds its real purpose here: a task that runs every ten minutes and sometimes takes longer than ten minutes collides with itself without a lock. The code **75** reports "temporary error." **Environment.** `PATH` and `LC_ALL` are fixed inside the script; the environment the scheduler provides is not trusted. **Absolute paths.** There is no relative path at all; whatever the starting directory is, the script finds the same files. **Separate output.** The report is appended to `report.out`, diagnostic messages to `report.err`. The redirections use `>>`; each run does not overwrite the previous one. **Strict mode.** The wrapper is itself a script and is subject to the same rules. Log files growing forever is a separate problem, solved with log rotation tools; these tools are taken up in the System Administration course. ## Testing the Task A scheduled task must be tested under the scheduler's conditions before it is defined. The testing order is as follows: 1. Run the wrapper by hand; verify the output files are created in the right place. 2. Repeat the same call in a narrow environment: `env -i PATH=/usr/bin:/bin ./report-job.sh`. This is a close approximation of the scheduler's environment. 3. Test the lock: start a second instance while one is running and verify the code 75. 4. Test the error path: give a nonexistent log name and see a meaningful message land in `report.err`. 5. Write the entry into the table a few minutes ahead of the actual time, and observe the first run. The exit code is the last link in this chain. What reports a task's failure to a monitoring system is the exit code; this is why the codes a script produces must be chosen deliberately. The code scheme used in this course was as follows: | Code | Meaning | |---|---| | 0 | report produced | | 2 | usage error (invalid option or value) | | 3 | input error (log missing, unreadable) | | 75 | temporary error, retry later (lock not acquired) | | 128 + n | terminated by signal | ## The Finished Script The final form of the script developed throughout the course: ```sh #!/usr/bin/env bash # report.sh — produces a summary report from the access log. # Usage: report.sh [-n COUNT] [LOG] set -Eeuo pipefail export LC_ALL=C SCRIPT_NAME="report.sh" LINE_PATTERN='^[0-9.]+ - - \[[^]]+\] "[A-Z]+ [^ ]+ HTTP/[0-9.]+" [0-9]{3} [0-9]+$' temp="" error() { printf '%s: error: %s\n' "$SCRIPT_NAME" "$*" >&2; } warn() { printf '%s: warning: %s\n' "$SCRIPT_NAME" "$*" >&2; } die() { error "$1"; exit "${2:-1}"; } section() { printf '\n== %s ==\n' "$*"; } cleanup() { local code=$? [ -n "$temp" ] && rm -rf "$temp" return "$code" } trap cleanup EXIT trap 'cleanup; trap - INT; kill -INT $$' INT trap 'cleanup; trap - TERM; kill -TERM $$' TERM trap 'error "$LINENO. line unexpected error"' ERR summary() { awk ' { requests++; bytes += $10; class[substr($9, 1, 1)]++ } END { printf " requests: %d\n", requests printf " bytes : %d\n", bytes printf " average : %.1f bytes\n", (requests ? bytes / requests : 0) for (s = 2; s <= 5; s++) printf " %sxx %4d %%%.1f\n", s, class[s], (requests ? class[s] * 100 / requests : 0) } ' "$1" } top_paths() { cut -d' ' -f7 "$1" \ | sed -E 's#/[0-9]+#/:id#g' \ | sort | uniq -c | sort -k1,1nr -k2,2 | head -"$2" } error_paths() { awk '$9 ~ /^[45]/ { n[$7]++ } END { for (y in n) printf "%3d %s\n", n[y], y }' "$1" \ | sort -k1,1nr -k2,2 } count=5 while getopts ':n:' option; do case "$option" in n) count="$OPTARG" ;; :) die "-$OPTARG expects a value" 2 ;; \?) die "unknown option -$OPTARG" 2 ;; esac done shift $((OPTIND - 1)) log="${1:-access.log}" [[ "$count" =~ ^[1-9][0-9]*$ ]] || die "-n expects a positive integer: $count" 2 [ -f "$log" ] || die "not a regular file or does not exist: $log" 3 [ -r "$log" ] || die "no read permission: $log" 3 [ -s "$log" ] || warn "log is empty: $log" temp=$(mktemp -d) valid="$temp/valid.log" grep -E "$LINE_PATTERN" "$log" | awk 'NF == 10' > "$valid" || true invalid=$(( $(grep -c '' "$log") - $(grep -c '' "$valid") )) [ "$invalid" -eq 0 ] || warn "$invalid lines not in the expected format, excluded from analysis" section "Log: $log" summary "$valid" section "Top $count requested paths" top_paths "$valid" "$count" section "Paths returning errors" error_paths "$valid" ``` The script's last addition is the step that separates valid lines before analysis. The line pattern from the Regular Expressions lesson and the field-count check from the Field-Based Processing lesson are applied together: neither was sufficient alone. Discarded lines are counted and reported to standard error, while counts are computed only over the valid lines — so a corrupted input does not silently distort the report. With a clean log: ```sh ./report.sh -n 3 ``` ``` == Log: access.log == requests: 30 bytes : 117780 average : 3926.0 bytes 2xx 18 %60.0 3xx 3 %10.0 4xx 6 %20.0 5xx 3 %10.0 == Top 3 requested paths == 6 /api/data 6 /product/:id 4 /index.html == Paths returning errors == 4 /missing.html 3 /api/data 2 /secret ``` With a log that has two malformed lines added: ```sh ./report.sh -n 2 all.log ``` ``` report.sh: warning: 2 lines not in the expected format, excluded from analysis == Log: all.log == requests: 30 bytes : 117780 average : 3926.0 bytes 2xx 18 %60.0 ... ``` The warning went to standard error, the report to standard output, and the counts were unaffected by the malformed lines. The script has a known behavior: when its output is piped into a command that closes early, like `head`, the `ERR` hook is triggered by `SIGPIPE` because `pipefail` is active. This is a deliberate trade-off of strict mode; shortening the output must be done with the `-n` option. ## Summary - A cron entry consists of five time fields and a command; when day of month and day of week are both given, the conditions combine with "or." - The scheduler runs the command without a login shell; the search path is narrow, startup files are not read, and most environment variables are undefined. - The search path and locale are fixed inside the script, and all file paths are written as absolute. - Output is managed explicitly: the report and diagnostic messages are appended to separate files. - The wrapper script sets up the environment, acquires the lock, and redirects output; the lock prevents overlapping runs. - Exit codes are the monitoring system's only input; usage, input, and temporary-error classes are reported with separate codes. ## Course Wrap-Up This course started with a single question: exactly where does a command write the data it writes? The answer was three streams and file descriptors. Everything that follows from there is an expansion of the same idea. **Streams and Pipelines** established connecting commands together: redirection being applied before the command, the pipeline's processes running concurrently, variables lost in a subshell, and the exit code convention. **Script Structure** turned one-line commands into a program. At the center of this section stood the thing that sets the shell apart from other languages: an expansion's result is **reparsed**. Word splitting, pathname expansion, and quoting rules are this course's academic core; most errors in shell scripts are born here. **Text Processing** handed the work to its dedicated tools: the pattern language, line filtering, the stream editor, field-based processing, and the sort family. The selection criterion that closed the section — choose the narrowest tool, switch tools when a problem needs more than three of them — is this course's practical inheritance. **Writing Robust Scripts** prepared the script for unattended operation: strict mode, cleanup hooks, tracing tools, static analysis, and scheduled runs. The same script was developed throughout the course. `report.sh` began as a fixed pipeline; it gained variables, acquired an argument interface, learned to validate its inputs, was split into functions, reduced its counting to a single pass, and learned to stop on errors and clean up after itself. Its final form is a tool that passes inspection and can be handed to a scheduler. The shell's limit also showed itself on this journey. The shell has no equal at connecting processes together and working with the file system. It was not designed for complex data structures, decimal arithmetic, structured data formats, or extensive error handling. A good shell script is a script that knows its own limit. The next course — **System Administration** — takes up the system this script runs on. `report.sh` was handed to a scheduler; but how is that scheduler itself defined and inspected as a service? The script created a process; how are processes monitored, prioritized, and managed with signals? Log files are growing; how is disk and file system configuration changed? In this course, signals, exit codes, and processes were seen from **inside** a script. The next course looks at them from **outside** the system.