Signals

The previous lesson passed by the edge of a signal: when the terminal closed, the hangup signal was delivered to the head of the session, the shell propagated it to its own jobs, and some of the jobs ignored it. Ignoring itself was not measured. This lesson takes up signals directly.

What it measures is a single sentence: a termination request is a request. A command returning without error does not mean the target is gone, and the difference between the two can be counted.

What a Signal Is, and Is Not

A signal is an interrupt notification the kernel delivers to a process. It carries no data inside it; it has only an identity. When a process receives a signal, one of three things happens: the handler the process has defined for that signal runs, the signal is ignored, or the signal’s default action is applied. The default action varies from signal to signal — terminate, stop, continue, or do nothing at all.

trap in the Shell Programming course sets up exactly the first option: it defines the script’s signal handler. The difference here is the direction of view — there, the receiving side was written; here, the sending side is measured.

Signal	Default action	Catchable	For what
SIGHUP	terminate	yes	the controlling terminal is gone; many services treat this as a reload request
SIGINT	terminate	yes	interrupt from the terminal (Ctrl+C)
SIGQUIT	terminate and dump core	yes	quit request from the terminal
SIGTERM	terminate	yes	the termination request — the polite form, the default
SIGKILL	terminate	no	the forced signal — the process has no say
SIGSTOP	stop	no	freezes the process
SIGCONT	continue	yes	resumes a stopped process
SIGUSR1, SIGUSR2	terminate	yes	its meaning is the program’s own contract
SIGPIPE	terminate	yes	writing to a pipe whose reading end has closed
SIGCHLD	ignore	yes	a child process ended or stopped

Two rows stand apart from the others. SIGKILL and SIGSTOP cannot be caught, ignored, or blocked; the kernel makes the decision, and the process is not even informed. This is exactly the cost: a process killed by the forced signal can never run its own shutdown procedure.

Signals also have numbers, but the numbers are not the same on every architecture. Portable code is written by name: kill -TERM is written, not the number.

Sending a Signal

The command that sends a signal is named kill, and the name is misleading: the command does not kill a process, it delivers a signal. Which signal is delivered depends on the option; if no option is given, the termination request is sent.

$ kill -TERM 1128
$ echo $?
0
$ kill -0 1128           # no signal sent, only check whether it is reachable
$ echo $?
0
$ pkill -TERM -u app queue-worker
$ kill -TERM -1014       # a negative number: the signal goes to the PROCESS GROUP

This transcript is illustrative and has not been run. It carries four details. kill -0 sends no signal at all; it only tests whether the target exists and whether there is authority to send it a signal — this is the tool of verification. pkill selects the target by name and user; the measure from the first lesson applies here too, a name is a label, not an identity. A negative number turns the target into a process group: it sends to the whole of a pipeline, not to a single member. killall also works by name, and because its behavior can vary between systems, pkill is preferred in portable scripts.

The most critical detail is in the second line. kill’s exit code reports whether the signal was delivered, not anything else. How the target met the signal, what its handler did, whether the process finished, none of this is ever reflected in this code. An operator seeing a code of 0 and saying “the job is done” is an unmeasured diagnosis.

• PM15. The target set is the processes on the synthetic server with the continuously heavy pattern.
• PM16. Some of the processes ignore the termination request; the ignores_signal field in the common definition carries this. The forced signal cannot be ignored.
• PM17. A process that accepts the request does not finish immediately: it does its own cleanup first. The cleanup time is between 0 and 39 seconds per process.
• PM18. kill’s exit code is 0 on every attempt; it means the signal was delivered.
• PM19. The diagnosis given without waiting is this: “the command returned, the targets are gone.” The false diagnosis count is the number of targets that, at that moment, have not actually finished.
• PM20. The forced signal finishes every target; the cleanup of processes that accepted the request and were still cleaning up gets cut short.
• PM21. The second seed is 20260219.

# --- common definition, in the form used in the first lesson
SEED, SECOND_SEED = 20260218, 20260219
USER = ("root", "app", "backup", "monitor")
COMMAND = ("data-receiver", "report-generator", "backup-job", "metric-collector",
           "queue-worker", "cache-cleaner")


def generator(seed):
    d = seed

    def next_value(n):
        nonlocal d
        d = (d * 1103515245 + 12345) % 2147483648
        return d % n
    return next_value


def processes(seed=SEED, count=24):
    r = generator(seed)
    result = []
    for i in range(count):
        kind = r(10)
        pattern = "flat" if kind < 5 else ("spike" if kind < 8 else "heavy")
        result.append({"pid": 1000 + i * 7 + r(5),
                       "user": USER[r(len(USER))],
                       "command": COMMAND[r(len(COMMAND))], "pattern": pattern,
                       "priority": r(11) - 5, "memory": 20 + r(400),
                       "ignores_signal": r(10) < 2})
    return result


# --- this lesson's layer: signal and cleanup
def send_signal(processes_, signal, targets):
    """Common definition: `terminate` can be ignored, `force` cannot."""
    finished, resisting = [], []
    for s in processes_:
        if s["pid"] not in targets:
            continue
        if signal == "force" or not s["ignores_signal"]:
            finished.append(s["pid"])
        else:
            resisting.append(s["pid"])
    return {"finished": len(finished), "resisting": len(resisting),
            "resisting_pid": sorted(resisting)}


S = processes()
TARGET = {s["pid"] for s in S if s["pattern"] == "heavy"}
print("target:", len(TARGET), "heavy processes")
print("terminate:", send_signal(S, "terminate", TARGET))
print("force    :", send_signal(S, "force", TARGET))

target: 7 heavy processes
terminate: {'finished': 2, 'resisting': 5, 'resisting_pid': [1014, 1043, 1092, 1108, 1128]}
force    : {'finished': 7, 'resisting': 0, 'resisting_pid': []}

Only 2 of the seven heavy processes accept the termination request, 5 resist. With the forced signal, 7 of the 7 finish. The list of resisting processes does not appear in real output — the kill command does not give this information, and cannot. This is the second binding reading of the course’s common definition.

The Measure of Waiting

The two rows above are the signal’s final result. The operator, however, sees not the result but the state at one instant. A process that accepts the termination request does not finish immediately; it closes its open files, flushes its buffer, releases its lock. During this, the process still sits in the list.

# On top of the previous block: S, TARGET, generator, SEED come from there.
def assign_cleanup(processes_, seed=SEED):
    """A process that accepts the termination request does not finish immediately: it cleans up first."""
    r = generator(seed)
    for s in processes_:
        s["cleanup"] = r(40)              # seconds
    return processes_


def termination_attempt(processes_, targets, wait):
    """The request was sent, checked again after `wait` seconds. `kill`'s exit code is
    always 0: it means the signal was DELIVERED, not that the target FINISHED."""
    finished, cleaning_up, ignoring = [], [], []
    for s in processes_:
        if s["pid"] not in targets:
            continue
        if s["ignores_signal"]:
            ignoring.append(s["pid"])
        elif s["cleanup"] <= wait:
            finished.append(s["pid"])
        else:
            cleaning_up.append(s["pid"])
    return len(finished), len(cleaning_up), len(ignoring)


assign_cleanup(S)
print("target:", len(TARGET), "processes | `kill` exit code is 0 on every line")
print("wait     finished  cleaning up  ignoring  alive  false diagnosis")
for b in (0, 1, 5, 10, 30, 60):
    finished, cln, ign = termination_attempt(S, TARGET, b)
    print(f"{b:7d}  {finished:5d}  {cln:12d}  {ign:9d}  {cln + ign:7d}"
          f"  {len(TARGET) - finished:11d}")

target: 7 processes | `kill` exit code is 0 on every line
wait     finished  cleaning up  ignoring  alive  false diagnosis
      0      0             2          5        7            7
      1      0             2          5        7            7
      5      0             2          5        7            7
     10      1             1          5        6            6
     30      2             0          5        5            5
     60      2             0          5        5            5

The first row is the lesson’s harshest number. At the instant the command returns, none of the targets have finished; false diagnosis 7/7. Five processes ignored the request, and the other two have not finished their cleanup yet. The exit code is still 0 — the command has done its job, the signal was delivered.

The second reading is in the wait column. Waiting five seconds changes nothing; at the tenth second 1 process finishes, at the thirtieth 2. After thirty, nothing changes: the remaining 5 processes ignore the request and will not finish no matter how long the wait. Waiting completes the cleanup, it does not resolve resistance. This distinction determines the correct procedure.

The correct order is four steps: send the request, wait, verify with kill -0, and force only what is still standing. Service managers do this too, and the wait time is a value set in the unit definition; that side is the subject of this course’s next topic.

The choice of wait time is made by looking at this table, and it has a cost in both directions. If chosen short, processes still cleaning up are forced needlessly; forcing at five seconds in the table means cutting off two processes that would have finished on their own at the tenth and thirtieth seconds. If chosen long, resisting processes are waited for pointlessly: no column in the table changes after the thirtieth second, and waiting sixty seconds only prolongs the malfunction. Without a measured threshold, the chosen duration remains a guess, and that guess produces false diagnoses in both directions.

The Signal That Does Not Arrive, and the One That Waits

Resistance is not the only reason a signal fails to work. There are three more situations, and all three produce the same picture: the command returns 0, the target stands.

The first is a pending signal. A termination request sent to a stopped process is not processed immediately; because the process runs no code, the signal waits. The stopped jobs from the previous lesson are in exactly this state: the request is delivered, written to the queue, and stays there until the process is resumed with SIGCONT. The forced signal is an exception to this rule; the kernel applies it and does not wait for the process to run code. The portable way to truly stop a job is therefore to resume it first and then send the request.

The second is a blocked signal. A process can temporarily block certain signals while entering a critical section; the signal waits until the block is lifted. An operator looking from outside cannot tell a blocked signal apart from an ignored one — in both cases, nothing happens.

The third is the wrong target. Process numbers are reused: after a process ends, the same number can be assigned to a different process. Scripts that read a number from a file and send to it should therefore never use the number without verifying it; testing existence with kill -0 is not enough, what needs to be tested is whom the number belongs to.

The common conclusion of these three situations is this: sending a signal is not an event, it is a request, and the result of the request is learned only by looking at the target. Whether a shutdown procedure is written correctly is also the sending side’s concern — this is why SIGHUP is commonly used to reconfigure services: the service rereads its configuration without restarting, and does not drop open connections. In another program, the default action of the same signal is to terminate; the meaning belongs not to the signal, but to the program’s contract.

The Cost of Forcing

Skipping the wait step and sending the forced signal directly is an attractive shortcut: the result is certain, all the targets finish. Certainty has a cost, and it too can be counted.

# On top of the previous blocks: S, TARGET, termination_attempt, send_signal,
# assign_cleanup, processes, SECOND_SEED.
forced = send_signal(S, "force", TARGET)
interrupted = [s["pid"] for s in S if s["pid"] in TARGET
              and not s["ignores_signal"] and s["cleanup"] > 0]
print("direct forced signal:", forced["finished"], "/", len(TARGET), "finished")
print("  cleanup interrupted midway:", len(interrupted), "processes")
print("correct order (request, wait, verify, force):")
finished, cln, ign = termination_attempt(S, TARGET, 30)
print("  finished on its own after 30 seconds:", finished,
      "| needing to be forced:", cln + ign)
print()
for seed in (SEED, SECOND_SEED):
    S2 = assign_cleanup(processes(seed), seed)
    H2 = {s["pid"] for s in S2 if s["pattern"] == "heavy"}
    g = send_signal(S2, "terminate", H2)
    b0 = termination_attempt(S2, H2, 0)[0]
    b30 = termination_attempt(S2, H2, 30)[0]
    print(f"seed {seed}: target {len(H2)} | accepted the request {g['finished']}"
          f" | finished after 0 s {b0} | finished after 30 s {b30}"
          f" | false diagnosis {len(H2) - b0}")

direct forced signal: 7 / 7 finished
  cleanup interrupted midway: 2 processes
correct order (request, wait, verify, force):
  finished on its own after 30 seconds: 2 | needing to be forced: 5

seed 20260218: target 7 | accepted the request 2 | finished after 0 s 0 | finished after 30 s 2 | false diagnosis 7
seed 20260219: target 5 | accepted the request 5 | finished after 0 s 0 | finished after 30 s 4 | false diagnosis 5

Forcing directly gives a 7/7 result, and the cleanup of 2 processes is cut short. The interrupted cleanup is not an abstract loss: a process that has not flushed its buffer loses data it thought it had written, a process that has not released its lock leaves a lock file behind, a process that has not deleted its temporary directory leaves it on disk. None of these three outcomes appears in kill’s output, and all three surface as a different malfunction on the next run.

In the sequence done by waiting, the same two processes finish on their own, and only 5 targets need to be forced. The difference is not two processes; it is two clean shutdowns.

With the second seed, the target count drops to 5, and the number accepting the request becomes 5 instead of 2 — the composition changes completely. The one thing that does not change is the first column: in both seeds, the number of targets finished at the instant the command returns is 0, and the diagnosis given without waiting is wrong for every target. How many processes will resist depends on the fiction; that the command returning says nothing is not.

A limit outside the model should also be noted: the forced signal finishes every target here, but on a real system, a process waiting in uninterruptible sleep inside a kernel call does not respond even to that. In that case the process appears in state D, and the only remedy is removing the cause of the wait.

Summary

• A signal is an interrupt notification that carries no data; a process can catch it, ignore it, or leave it to the default action. SIGKILL and SIGSTOP cannot be caught or blocked.
• kill’s exit code reports that the signal was delivered, not that the target has finished. The tool of verification is kill -0.
• Of seven heavy targets, only 2 accept the termination request, 5 ignore it; with the forced signal, 7/7 finish.
• The number of targets finished at the instant the command returns is 0: false diagnosis 7/7. At the tenth second 1 process finishes, at the thirtieth, 2; after thirty, nothing changes.
• Forcing directly cuts short the cleanup of 2 processes; in the wait-based sequence, the same two processes shut down cleanly and only 5 targets are forced. The correct order is request, wait, verify, force.

Next Step

This lesson measured stopping a process. Stopping is not always the right answer: a heavy process may be doing necessary work, and what is wanted may not be to end it but to slow it down. The next lesson takes up the priority value and counts a single question: how much of a genuinely visible effect does changing a process’s priority have, and is a changed column the same thing as changed behavior.