--- title: 'Introduction to Linux' source: 'https://academia.sh/en/courses/introduction-to-linux' language: en updated: '2026-08-17T04:41:45+00:00' license: 'CC BY-SA 4.0' lessons: 20 outcomes: - 'Navigates the file system hierarchy from the command line and carries out file operations' - 'Reads and changes file permissions and ownership' - 'Installs, updates and removes software with a package manager' --- # Introduction to Linux The shell, the file system hierarchy, file and directory operations, the permission model and package management; an introduction that explains why the command line behaves as it does. ## Getting to Know the System The distinction between kernel, distribution, and shell; command structure, the directory hierarchy, manual pages, and the executable search order. ### The Unix Tradition and Linux Source: https://academia.sh/en/courses/introduction-to-linux/unix-tradition-and-linux The distinction between the kernel, distribution, and shell layers; the assumptions carried by the Unix design tradition and POSIX's promise of portability. The phrase "learning Linux" covers not one thing but three layers stacked on top of each other: the kernel that manages hardware, the set of programs gathered around it, and the command interpreter used to invoke them. The three were built by different organizations, at different times, for different reasons, and each can be replaced independently. This lesson separates the layers and establishes which one the rest of the course lives in. The distinction can look like needless rigor, but most "the command is not working" problems come from not knowing which layer the trouble is in. #### The Assumptions of the Unix Tradition Linux carries on the design tradition of the **Unix** operating system, developed at Bell Laboratories in the 1970s. This tradition is built on a handful of assumptions, and those assumptions directly explain how commands behave today. **Everything is a file.** Not only data on disk; devices, terminals, and inter-process communication channels are also represented within the file system under a name. Programs access all of them through a single interface — open, read, write, close. **Small tools, one job.** Every program does a narrow job. Complex tasks are built by combining programs. The grammar of that combination is the subject of the Shell Programming course; this course introduces the tools themselves. **Text is the common interface.** Data flowing between programs is mostly plain text split into lines. Text was chosen as the common denominator because it can be read by both humans and programs. **Configuration lives in files.** System behavior is kept in readable text files, not in a hidden binary-format registry. Examining a setting means reading the relevant file. These assumptions are a design choice, not a law of nature. They have costs too: a text-based interface distributes the burden of parsing structured data across every program. Later lessons in the course will show where this cost is paid. #### The Kernel The **kernel** is the one software layer that talks directly to hardware. Its responsibilities overlap with concepts from the How Computers Work course: - **Process scheduling:** decides which program uses the processor when. Enforcing this decision rests on the timer interrupt described in the interrupts lesson. - **Memory management:** gives every process its own address space; prevents one process from reading another's memory. - **File system:** dresses blocks on disk with names and a directory structure. - **Device drivers:** hide the detail of hardware such as the keyboard, disk, and network interface. A program reaches the kernel's services through a **system call**, not an ordinary function call: it changes the processor's privilege level, control passes to the kernel, and returns once the work is done. Opening a file, requesting memory, and creating a process are each system calls. This boundary matters because it is also **the boundary of portability**: the same program runs on any kernel offering the same system call interface. Which kernel a system is running can be asked directly: ``` $ uname -s Linux ``` The `uname` command prints the kernel's name. The same command gives a different answer on another system that shares the Unix tradition — `Darwin` on macOS, for example. The command itself exists on both; the answer differs. This is the shortest demonstration of what portability means. #### The Distribution The kernel by itself is not a usable system. Programs that copy files, display text, and establish network connections are not part of the kernel; they are separate projects. A **distribution** is the packaging effort that combines a kernel with a chosen set of programs and turns them into an installable whole. A distribution typically determines the following components: - Choice of kernel and drivers - The standard C library - Basic command tools (file, text, and process tools) - The system that starts the first process and manages services - The package manager and repository configuration - The default shell and default editor These choices vary by distribution. This course stays with the **common ground**: most commands covered are defined in POSIX or exist under the same name across every common distribution. Where distribution-specific behavior comes up, it is stated explicitly. **POSIX** is a standard defining the interface of Unix-like systems: a command's inclusion in POSIX means its name and basic options carry the same meaning on every conforming system. The standard does not cover every option: systems add options on top of it, and these additions diverge from one another. The most common example of this divergence in the course — the differences between the GNU and BSD tool families — is shown wherever it comes up. #### The Shell The **shell** is a program that takes a command line, interprets it, and runs the corresponding programs. It is not part of the kernel, not privileged; it runs like any other program and can be replaced. Which shell is running can be read from the process list: ``` $ ps -p $$ -o pid=,comm= 1076 bash $ echo $$ 1076 ``` `$$` is a variable holding the shell's own process number. The `ps` command prints the name of the process with that number. The answer here is the shell's name — it varies from system to system and from user to user. The shell's replaceability is plainly visible in one place on the system: ``` $ ls -l /bin/sh lrwxrwxrwx 1 root root 4 Mar 31 2024 /bin/sh -> dash ``` `/bin/sh` is not a program itself but a name given to a **role**: "the POSIX shell." The `l` at the start of the line says this is a link; the arrow shows its target. On this system, the role is given to the shell named `dash`; on another, to a different program. How links work is covered in detail in the File and Directory Operations topic. The distinction arrives at this: if a command "is not working," there are three separate possibilities. The command is not installed on the system (the distribution layer), the shell cannot find the command (the shell layer), or the kernel is rejecting the operation (the permission layer). The order of this course's topics follows these three layers. #### Everything Is a File — A Concrete Example The most abstract of the assumptions, "everything is a file," can be confirmed by looking at devices: ``` $ ls -l /dev/null /dev/zero crw-rw-rw- 1 root root 1, 3 Jul 26 18:45 /dev/null crw-rw-rw- 1 root root 1, 5 Jul 26 18:45 /dev/zero ``` The `c` at the start of the line says these entries are not ordinary files but **character devices**. In the column where an ordinary file's size would sit, there are two numbers here: the device's major and minor numbers. The kernel uses these numbers to decide which driver to call. `/dev/null` swallows everything written to it and signals end of file immediately when read; `/dev/zero` produces zero bytes for as long as it is read. Neither holds data on disk. Yet a program opens, reads, and writes them like an ordinary file — because the interface is shared. #### The Course's Axis The course will work with a single example throughout: a measurement project called `project/`, set up under the home directory. This directory tree advances one step in every lesson; raw measurement files are added, processed, linked, archived, have their permissions arranged, and are finally shared with a group. The goal is not to list commands but to show **why** each behaves as it does, on that same tree. Before the tree is built, the next lesson examines the layer that interprets the commands themselves — the shell. #### Summary - The Unix tradition rests on four assumptions: everything is a file, tools are small and single-purpose, the common interface is text, and configuration lives in readable files. - The kernel manages hardware and offers its services through system calls; this boundary is also the boundary of portability. - A distribution is the packaging effort that combines a kernel with chosen programs; it is the part that varies. - POSIX defines the common meaning of command names and basic options; the options systems add on top of the standard diverge from one another. - The shell is not a privileged component but a replaceable program; `/bin/sh` is the name of a role, not a program. #### Next Step Now that the layers are separated, it is time for the topmost one. When a command line is typed, what exactly does the shell do with it: how does it split it into words, which part does it count as the program name, what do quotation marks change, and how is it known whether a command succeeded? The next lesson unpacks the shell's parsing work step by step. ### What Is a Shell Source: https://academia.sh/en/courses/introduction-to-linux/what-is-a-shell The prompt, splitting the command line into words, performing expansions before the command runs, option forms, builtins, and exit status. The previous lesson defined the shell as a replaceable program that interprets commands. That definition does not yet explain anything: what exactly does the shell do with a typed line? The answer also explains the command line's most common mistakes. Why a filename with a space splits into two separate arguments, who actually interprets the asterisk, how it is known whether a command succeeded — all of these are the result of a single parsing process. #### The Prompt The shell signals that it is waiting for a command with a **prompt**. The prompt's content is configurable; the username, machine name, and current directory are commonly shown. In this course's examples, the prompt is shown as a single character: ``` $ echo Hello Hello ``` Lines starting with `$` are the typed command; the lines below them are the command's output. Documentation carries an established convention: the ordinary user prompt is shown as `$`, the superuser prompt as `#`. If a command starting with `#` is seen in a document, it is understood that the command requires administrative privilege. #### The Line Is Split Into Words The shell first splits the line into **words**. The split is done on whitespace characters, and consecutive spaces count as a single separator. The first word is the name of the program to run; the remaining words are passed to it as **arguments**. ``` $ echo Hello world Hello world ``` The output has one space instead of four. This is not because the `echo` command compresses spaces: `echo` received two arguments — `Hello` and `world` — and printed them joined by a single space. The spaces between them were gone before `echo` even ran. The argument count can be counted directly: ``` $ file="measurement 01.csv" $ set -- $file; echo $# 2 $ set -- "$file"; echo $# 1 ``` `$#` gives the argument count. When the variable's value is written without quotes, the shell split it into two words; written inside quotes, it left it as a single word. This is why filenames with spaces must be written in quotes; otherwise the command looks for two files that do not exist. The same difference can be shown with a print format that makes word boundaries visible: ``` $ printf '[%s]\n' $file [measurement] [01.csv] $ printf '[%s]\n' "$file" [measurement 01.csv] ``` #### Expansions Happen Before the Command Splitting into words is not the only operation. Before the program runs, the shell applies several **expansions** to the words. Three are used in this course: variable expansion (`$name`), filename expansion (`*`, `?`), and home directory expansion (`~`). The decisive point is this: the shell performs the expansion, not the program that runs. The program sees only the result of the expansion. ``` $ echo /usr/bin/zc* /usr/bin/zcat /usr/bin/zcmp $ ls /usr/bin/zc* /usr/bin/zcat /usr/bin/zcmp ``` Both commands produced the same two filenames. The `*` character never reached the `ls` command; the shell replaced the pattern with matching filenames, and `ls` received two ready-made arguments. The only difference between `echo`'s output and `ls`'s is that `ls` performs column alignment. This has two consequences. First, if the pattern matches no file, the shell leaves it unchanged and the program receives a meaningless name containing `*`. Second, and more important, what a command will do can be **seen in advance**: putting `echo` in front of a command shows the argument list the shell will produce, without running it. This will become a safety habit in the deletion lesson. Quoting also stops expansion: ``` $ echo "/usr/bin/zc*" /usr/bin/zc* ``` Double quotes block filename expansion and word splitting, but not variable expansion. Single quotes block all of them: text inside single quotes is preserved literally. #### Options and Arguments A command line's arguments are of two kinds. An **option** changes the command's behavior and, by convention, starts with a dash; the rest are the objects the command will operate on. Options have three written forms, and all three give the same result: ``` $ ls -l -a /home total 16 drwxr-xr-x 1 root root 4096 Jul 26 18:53 . drwxr-xr-x 1 root root 4096 Jul 26 18:46 .. drwxr-x--- 3 student student 4096 Jul 26 18:49 student $ ls -la /home total 16 drwxr-xr-x 1 root root 4096 Jul 26 18:53 . drwxr-xr-x 1 root root 4096 Jul 26 18:46 .. drwxr-x--- 3 student student 4096 Jul 26 18:49 student ``` Single-letter options are written with a single dash and can be combined: `-l -a` is the same as `-la`. Long options are written with two dashes (`--all`) and cannot be combined; each is a separate word. The long-option form is not defined in POSIX; it is a GNU tool family addition, so the single-letter form is preferred wherever portability matters. If a **filename** starts with a dash, trouble follows: the command mistakes it for an option. This is what the `--` separator is for. Everything after the word `--` is not interpreted as an option. Why this separator is a security matter will be seen in the deletion lesson. #### Builtins and External Programs Not every command is a separate program. Some commands are implemented inside the shell itself; these are called **builtins**. Which one a name is can be asked directly: ``` $ type -a cd pwd echo cd is a shell builtin pwd is a shell builtin pwd is /usr/bin/pwd pwd is /bin/pwd echo is a shell builtin echo is /usr/bin/echo echo is /bin/echo ``` Three different situations appear. `cd` is only a builtin. `pwd` and `echo` are both a builtin and exist as external programs; in that case, the builtin wins. `cd` being a builtin is not a preference but a necessity. A program's working directory belongs to its own process; a `cd` running as a separate program would change its own directory and exit, leaving the shell's directory where it was. To change directory, the command must run **inside the shell's own process**. `echo` being both a builtin and an external program is for portability: it needs to be callable even in environments without a shell. The behavior of the two implementations is not exactly identical; this is why `printf` is preferred over `echo` for formatted output. #### Exit Status Every command leaves behind an integer when it ends: the **exit status**. The convention runs in reverse — zero reports success, and every nonzero value reports a failure. This direction was chosen because there is one way to succeed and countless ways to fail. The last command's status is held in the `$?` variable: ``` $ true; echo $? 0 $ false; echo $? 1 $ ls /missing; echo $? ls: cannot access '/missing': No such file or directory 2 ``` `ls` reports its failure both with an error message and with a nonzero status. The message is for humans; the status code is for the scripts that chain commands together. Two values come from the shell itself and need to be told apart: ``` $ lss -l bash: lss: command not found $ echo $? 127 ``` **127** reports that the command was not found; **126** that it was found but could not be run. The first is a search problem, the second a permission problem. The difference between these two numbers shows exactly what the next lesson and the permissions topic each solve. Where error messages are written to is a separate matter: standard output and standard error are two separate streams and can be redirected independently. Using this distinction belongs to the Shell Programming course. #### Summary - The shell first splits the line into words; the first word is the program name, the rest are arguments. - Variable, filename, and home directory expansions happen in the shell before the program runs; the program sees only the result. - Double quotes stop word splitting and filename expansion but not variable expansion; single quotes stop all of them. - Single-letter options can be combined, long options cannot; the `--` separator ends option interpretation. - Builtins run in the shell's own process; this is why `cd` has to be a builtin. - In exit status, zero shows success; 127 reports the command was not found, 126 that it could not be run. #### Next Step Commands now need files to operate on. But files are not placed at random: every branch of the hierarchy starting from the root directory has a specific meaning. The next lesson introduces this layout, establishes the distinction between absolute and relative paths, and builds the project tree that will be developed throughout the course. ### The Directory Hierarchy Source: https://academia.sh/en/courses/introduction-to-linux/directory-hierarchy The single-rooted file tree, mount points, the standard directory layout, and step-by-step path resolution; building the project tree used throughout the course. Commands operate on files, and files do not sit in random places. On Unix systems, every file's address is a path starting from the root of a single tree, and every branch of that tree has a specific meaning. This lesson establishes the logic of that layout, explains how the kernel resolves a path, and builds the `project/` tree the course will work on throughout. #### A Single Root The file system is a single tree, and its root is shown as `/`. There are no drive letters; a second disk does not open a second tree. Instead, disks are **mounted** onto a branch of the tree. Mounting makes a storage device's contents visible over an existing directory. Once mounted, a program entering that directory does not notice it is reaching a different device; the path is still part of the same tree. Where the device boundary lies matters only when an operation crosses devices — the Hard Links and Move Operations lesson will show what this boundary prevents. The root has no level above it. In the root directory, `..` still points to the root: ``` $ cd / $ pwd / $ cd .. $ pwd / ``` This is a definition, not a bug: there is nothing above the top of the tree, so `..` refers back to itself. This is what keeps path resolution from ever leaving the root. #### The Standard Layout The names and functions of the directories under the root are set by a layout standard. The standard says what kind of file goes where, so an administrator knows where to look even on an unfamiliar system. | Directory | Contents | |---|---| | `/bin`, `/usr/bin` | Basic commands run by all users | | `/sbin`, `/usr/sbin` | System administration commands | | `/lib`, `/usr/lib` | Shared libraries | | `/etc` | System-specific configuration files | | `/home` | User home directories | | `/root` | The superuser's home directory | | `/var` | Data of changing size: logs, queues, caches | | `/tmp` | Temporary files; may be cleared on reboot | | `/opt` | Add-on application software packages | | `/dev` | Device nodes | | `/proc`, `/sys` | Virtual file systems presenting kernel state as files | | `/boot` | Kernel image and bootloader files | | `/mnt`, `/media` | Temporary mount points | Two distinctions sit behind the layout. The first is **static versus variable**: content under `/usr` changes only when software is installed, while content under `/var` changes continuously while the system runs. The second is **shareable versus local**: `/usr` can be shared across multiple machines, while `/etc` is specific to that machine. These two distinctions make it possible to place directories on separate partitions and apply different backup policies. A standard layout merges some branches that were split for historical reasons back into a single point: ``` $ ls -l /bin /lib lrwxrwxrwx 1 root root 7 Apr 22 2024 /bin -> usr/bin lrwxrwxrwx 1 root root 7 Apr 22 2024 /lib -> usr/lib ``` On this system, `/bin` is not a separate directory but a link pointing to `/usr/bin`. `/bin/ls` and `/usr/bin/ls` are the same file. Old paths continuing to work is for backward compatibility. #### Absolute and Relative Paths An **absolute path** starts from the root and is a string beginning with `/`: `/home/student/project/data`. It points to the same file no matter where you are. A **relative path** starts from the current directory and does not begin with `/`: `data/raw`. Which file it points to depends on the current **working directory**. The working directory is a property of the process, not the shell; every process has its own working directory, and child processes inherit it. The `pwd` command prints this value. Three special names are valid in every directory: - `.` — the current directory itself - `..` — the parent directory - `~` — the home directory; this is not a directory name but an expansion the shell performs `.` and `..` are real directory entries; the kernel keeps them in every directory. `~` belongs to the shell and is converted to the home directory's absolute path before it reaches the command. #### How a Path Is Resolved A path is resolved **component by component** by the kernel. For the path `/home/student/project`, the order is: the root directory is opened, `home` is looked up inside it, the directory found is opened, `student` is looked up inside it, that directory is opened, `project` is looked up inside it. This step-by-step walk has three consequences, and all three come up again in later lessons of the course: **Every step requires a lookup.** The deeper the path, the more directories are read. **Every intermediate directory is checked for traverse permission.** Reaching the file at the end of the path requires execute permission on every directory along the way. The Permission Bits lesson will explain why the `x` bit means "the right to pass through" for directories. **Intermediate components can be links.** If a symbolic link is encountered along the path, resolution continues from its target. This is why the path `/bin/ls` ends up at the file `/usr/bin/ls`. A relative path is resolved the same way; the only difference is that it starts from the working directory, not the root: ``` $ cd /home/student/project/data $ pwd /home/student/project/data $ cd - / $ pwd / ``` `cd -` returns to the previous directory and prints the directory it returns to. This is a shortcut used when switching frequently between two directories. #### Hidden Files Files whose name starts with a dot are not shown by default in a listing: ``` $ ls project $ ls -a . .. .bash_history .bash_logout .bashrc .profile project ``` This is not a security feature but a display convention. As far as the kernel is concerned, a name starting with a dot has no special status; only listing tools skip them by default. The convention keeps user configuration files from cluttering the home directory. The first two entries in `ls -a`'s output are `.` and `..`; here they appear as the real entries every directory has. #### Building the Project Tree The tree used throughout the course is built under the home directory. Measurement data, processed output, scripts, documentation, and archives are placed in separate branches — a small-scale repetition of the layout under the root: ``` $ mkdir -p project/data/raw project/data/processed $ mkdir -p project/scripts project/docs project/archive $ find project -type d | sort project project/archive project/data project/data/processed project/data/raw project/docs project/scripts ``` `mkdir` normally creates only a single level of directory and errors if the parent does not exist. The `-p` option also creates missing intermediate directories and does not error if the directory already exists. The second property lets the same command be rerun safely. The `find` command walks a tree recursively; `-type d` selects only directories. The output is not sorted — the file system can return entries in no particular order — so it is piped through `sort`. The tree's meaning is this: `data/raw` holds measurement files that come from outside and will **not be modified**; `data/processed` holds output derived from them. `scripts` holds the programs that do the processing, `docs` holds the descriptions, `archive` holds packaged versions. Keeping raw data separate and untouched will sit at the center of the irreversibility discussion in the deletion lesson. #### Summary - The file system has a single root; extra disks do not open a new tree, they mount onto an existing directory. - The layout under the root is standardized and rests on two distinctions: static/variable and shareable/local. - An absolute path starts from the root, a relative path from the working directory; the working directory is a property of the process. - A path is resolved component by component; every intermediate directory is checked for traverse permission, and intermediate components can be links. - The hiddenness of names starting with a dot is a listing convention, not a kernel-level privilege. - `mkdir -p` creates missing intermediate directories and does not error on an existing directory. #### Next Step The tree is built, but a command's options and behavior still have to be learned by rote for now. The system itself makes that unnecessary: every command's own documentation is installed with it, and the documentation follows a consistent structure divided into numbered sections. The next lesson shows how to read this documentation, and why the same name can describe different things in different sections. ### Help Resources Source: https://academia.sh/en/courses/introduction-to-linux/help-resources Manual page section numbers, page structure, and synopsis notation; help for builtins and determining a command's type. The previous lesson used the `mkdir -p` option and explained what it does in the text. That is not a sustainable way to learn: dozens of options across hundreds of commands cannot be memorized. The system does not require it. Every command's documentation is installed along with the command itself, and the documentation follows a consistent structure. This lesson introduces that structure; for the rest of the course, some options will be left for the reader to look up rather than explained in the text. #### Manual Pages A **manual page** is the reference document for a command, file format, or system call. It is displayed with the `man` command: ``` $ man 1 ls | sed -n '1,13p' LS(1) User Commands LS(1) NAME ls - list directory contents SYNOPSIS ls [OPTION]... [FILE]... DESCRIPTION List information about the FILEs (the current directory by default). Sort entries alphabetically if none of -cftuvSUX nor --sort is speci- fied. ``` Only the page's first lines are taken in this example; typing `man ls` directly opens the page in a pager program, navigated with the arrow keys and exited with `q`. Searching inside the pager is done with `/`. Manual pages carry standard subheadings. The most commonly used, in order: - **NAME** — the command's name and a one-line description - **SYNOPSIS** — the usage summary - **DESCRIPTION** — detailed behavior and the option list - **EXIT STATUS** — which exit code is returned under which condition - **FILES** — the files the command reads or writes - **SEE ALSO** — related pages To quickly confirm a command's behavior, NAME and SYNOPSIS are read in that order; for detail, DESCRIPTION. When investigating the cause of an error, the EXIT STATUS and FILES sections often answer directly. #### Reading the Usage Synopsis The SYNOPSIS line uses a notation, and that notation is the same across every manual page: | Notation | Meaning | |---|---| | `plain text` | Written exactly as shown | | `[...]` | Contents in brackets are optional | | `...` | The preceding item can repeat | | `a\|b` | One of the alternatives | | `UPPERCASE` | A placeholder to be replaced with a value | Accordingly, the line `ls [OPTION]... [FILE]...` says: `ls` takes zero or more options and zero or more filenames; neither is required. `ls` can run with no arguments at all; this is consistent with the phrase "the current directory by default" in DESCRIPTION. For comparison, another command's synopsis: ``` $ mkdir --help | head -9 Usage: mkdir [OPTION]... DIRECTORY... Create the DIRECTORY(ies), if they do not already exist. Mandatory arguments to long options are mandatory for short options too. -m, --mode=MODE set file mode (as in chmod), not a=rwx - umask -p, --parents no error if existing, make parent directories as needed, with their file modes unaffected by any -m option. -v, --verbose print a message for each created directory -Z set SELinux security context of each created directory ``` `DIRECTORY...` is not in brackets: at least one directory name is required. The two separate functions of the `-p` option used in the previous lesson are both written here — creating missing parent directories and not erroring on an existing one. The `--help` option does not substitute for the manual page; on most commands it gives only the option list. On the other hand, it is always fast and is the only source on systems where manual pages are not installed. #### Section Numbers The manual is not a single set but is divided into numbered **sections**. The same name can appear in more than one section and describe different things. | Section | Contents | |---|---| | 1 | User commands | | 2 | System calls | | 3 | Library functions | | 4 | Special files and devices | | 5 | File formats and configuration files | | 6 | Games | | 7 | Miscellaneous: conventions, notations, concepts | | 8 | System administration commands | What the section division is for can be shown with a single name: ``` $ whatis passwd passwd (1) - change user password passwd (1ssl) - OpenSSL application commands passwd (5) - the password file ``` `passwd` is both a command and a file format. The entries in the list depend on the packages installed on the system; a page for software that is not installed does not appear. Suffixed section names like `1ssl` in the middle line come from a piece of software keeping its own pages in a separate subsection. `man passwd` shows only the first one found — section 1. To see the file format, the section is given explicitly: ``` $ man 5 passwd | sed -n '1,10p' PASSWD(5) File Formats and Configuration PASSWD(5) NAME passwd - the password file DESCRIPTION /etc/passwd contains one line for each user account, with seven fields delimited by colons (":"). These fields are: o login name ``` The `PASSWD(5)` notation in the page header repeats which section you are in, on every line. Documentation carries notations like `printf(3)` or `open(2)`; the number in parentheses is the section number, and it identifies which `printf` is meant — a shell command or a library function. This section's definition of the user account file will be used directly in the Users and Permissions topic. #### Searching When the Name Is Unknown The manual is useful once a command's name is known. If the name is not known, the descriptions are searched: ``` $ man -k 'list directory' | head -3 dir (1) - list directory contents ls (1) - list directory contents vdir (1) - list directory contents ``` `man -k` (equivalent to `apropos`) searches the NAME lines of every manual page. `man -f` (equivalent to `whatis`) searches for an exact name match. Both look at a prebuilt index; if the index hasn't been built, the result comes back empty. The search scans only the one-line descriptions, not entire pages. This is why the result set is narrow, and the search term should be kept general. #### Documentation for Builtins The previous lesson showed that builtins are not separate programs. This has a consequence for documentation: builtins have no separate manual page, because there is no separate program. Their documentation is in the shell's own page and in the shell's `help` builtin. ``` $ help type | sed -n '1,6p' type: type [-afptP] name [name ...] Display information about command type. For each NAME, indicate how it would be interpreted if used as a command name. ``` To choose the right source, the name's type is determined first: ``` $ type -t ls cd if file builtin keyword ``` The `-t` option prints only the type. The three answers correspond to three separate help sources: `man` for `file`, `help` for `builtin`, the shell's manual page for `keyword`. `if` is not a command but a **keyword** belonging to the shell's grammar; control structures are the subject of the Shell Programming course. #### The Limits of Documentation Manual pages are reference documents, not tutorials. They enumerate a command's every option completely but mostly do not say when to use which one. The documentation also belongs to the version on that system: if an option is not in the manual, it genuinely is not on that system — its existing on another system changes nothing. These two limits determine what the manual is used for. The manual answers "what does this command do"; it usually does not answer "which command do I use for this job." The second question is this course's subject. #### Summary - Manual pages carry standard subheadings such as NAME, SYNOPSIS, DESCRIPTION, EXIT STATUS, FILES, and SEE ALSO. - In a usage synopsis, brackets show optionality, ellipsis shows repeatability, and uppercase names show placeholders. - The manual is divided into numbered sections; the same name describes different things in different sections, and the `name(section)` notation states which one is meant. - `man -k` searches description lines; it is the way to find a command whose name is unknown. - Builtins have no separate manual page; they are documented with `help`, and the right source is determined with `type -t`. #### Next Step The `type` command performs a specific search to find what a name resolves to, and in the previous lesson, that same search failed when a command gave a "not found" error. The next lesson unpacks the rule behind that search: which directories the shell searches for an executable, in what order, how that order is changed, and why the current directory is not added to that list. ### The Command Path Source: https://academia.sh/en/courses/introduction-to-linux/command-path The executable search order, how environment variables pass to processes, the command hash table, and why the current directory is not added to the search list. In the previous lesson, a name's type was determined with `type`, and when the answer was `file`, it meant the name resolved to a file. But when `ls` is typed, no full path is given; the shell **finds** the file somewhere. This lesson unpacks the rule behind that search. The same rule explains, all at once, why a command is not found, why the same name can resolve to two different programs, and why a program in the current directory does not run directly. #### Environment Variables Every process has an **environment** table mapping names to values. When a process creates a new process, this table is copied over; the child process inherits the environment, but any change it makes never propagates back to the parent. ``` $ env | grep -E '^(HOME|PATH|PWD|SHELL|LANG|LC_ALL)=' PWD=/home/student HOME=/home/student LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin ``` Not every variable defined in the shell enters the environment. An ordinary assignment only creates a **shell variable**; this variable is not passed to programs that are run. For a variable to enter the environment, it must be marked with `export`. The distinction is this: a shell variable lives in the shell's own memory, an environment variable is copied to child processes. Three environment variables are used directly in this course: `HOME` holds the path to the home directory, `PWD` the working directory, and `PATH` the command search list. Localization variables such as `LC_ALL` affect the language of command output and sort order; they are kept at the standard setting for this course's output. #### Search Order When the shell sees a command name, it decides in a fixed order: 1. Is the name a **keyword**? (`if`, `for`, `while`) 2. Is the name an **alias**? 3. Is the name a **shell function**? 4. Is the name a **builtin**? 5. If not, it is searched for in the directories in the `PATH` list. The first four live in the shell's own memory, and the file system is never consulted. The fifth step searches on disk. The `PATH` variable consists of directory paths separated by colons. The shell scans these directories **left to right** and picks the first file that carries the name and has execute permission. It does not look at the remaining directories. If the name is found in none of them, it gives "command not found" with exit status 127. The order being left to right means position in the search list carries a meaning of priority. #### Shadowing The same name can exist in more than one directory. In that case, whichever comes first in the list wins and **shadows** the other. The effect can be shown by setting up a separate directory: ``` $ mkdir -p /tmp/demo $ printf '#!/bin/sh\necho fake version\n' > /tmp/demo/date $ chmod +x /tmp/demo/date $ type -a date date is /usr/bin/date date is /bin/date $ PATH="/tmp/demo:$PATH" $ type -a date date is /tmp/demo/date date is /usr/bin/date date is /bin/date $ date fake version ``` `type -a` lists **every** match carrying the name, in order; the first line is the one that will run. Adding the new directory to the front of the list did not make the system's `date` command unreachable — it merely fell later in the search order. Shadowing is both a tool and a risk. A tool: a command's own version can be moved ahead without touching the system version. A risk: if a writable directory sits at the front of the search list, anyone who can write a file to that directory can hijack command names. This is why the write permissions of directories added to the search list must be judged by the criteria covered in the permissions topic. The demo directory is removed once its job is done; leaving such a directory in place permanently causes unexpected behavior in later sessions. #### The Current Directory Is Not in the List The current directory (`.`) does not appear in `PATH`. This has a direct consequence: running a program in a directory requires specifying a path. ``` $ pwd /home/student/project/scripts $ summary.sh bash: summary.sh: command not found $ echo $? 127 $ ./summary.sh raw measurement files: 0 ``` The difference between `summary.sh` and `./summary.sh` is a matter of the shell's perspective. The first is a name with no slash in it; the shell searches for it in the `PATH` list and does not find it. The second contains a slash; the shell treats it as a path, does no search, and runs that file directly. Leaving the current directory out of the list is a deliberate security decision. If it were included, leaving a file named `ls` in a directory everyone can write to would be enough to make everyone entering that directory run that file. This is why the search list consists of fixed, controlled directories. If the current directory is to be added to the list at all, it belongs at the **end**, at least; that is the one position from which it cannot shadow system commands. #### Execute Permission It is not enough for a file to sit in a directory on `PATH`; it must also be marked executable. ``` $ ls -l summary.sh -rw-r--r-- 1 student student 142 Jul 26 18:49 summary.sh $ ./summary.sh bash: ./summary.sh: Permission denied $ echo $? 126 $ chmod +x summary.sh $ ls -l summary.sh -rwxr-xr-x 1 student student 142 Jul 26 18:49 summary.sh $ ./summary.sh raw measurement files: 0 ``` Two exit statuses report separate problems: **127** the file was not found, **126** it was found but could not be run. The `chmod +x` command adds execute permission; what the permission bits mean and how to read the string `-rwxr-xr-x` will be established in the first lesson of the permissions topic. The `#!/bin/sh` notation on the script's first line tells the kernel which interpreter to run the file with. The kernel reads this line, calls the named program, and hands it the file as an argument. This is why a shell script can be run directly even though it is not an executable binary. The `0` seen in the output reports that the raw measurement directory is still empty; the next topic will begin by filling it. #### The Hash Table If `PATH` were scanned from the start for every command, frequently used commands would cause repeated directory reads. The shell avoids this by storing the paths it finds in a **hash table**. ``` $ PATH="$HOME/project/scripts:$PATH" $ type summary.sh summary.sh is /home/student/project/scripts/summary.sh $ summary.sh raw measurement files: 0 $ type summary.sh summary.sh is hashed (/home/student/project/scripts/summary.sh) $ hash hits command 2 /home/student/project/scripts/summary.sh $ hash -r $ type summary.sh summary.sh is /home/student/project/scripts/summary.sh ``` After the first call, `type`'s answer changes: the path is now recorded in the table. This has a side effect — when a program is moved or deleted, the shell can keep using the old path and give a "no such file or directory" error. `hash -r` clears the table and forces a fresh search. This behavior is a typical use of the hash table from the Data Structures course: the result of a repeated, expensive search is cached, in exchange for accepting the risk that the cached information can go stale. #### Permanently Extending the Search List A `PATH` assignment made in the shell lasts only for that session. For it to be permanent, it must be written to a configuration file that is read when the session starts: ``` $ tail -2 ~/.profile PATH="$HOME/project/scripts:$PATH" export PATH ``` `~/.profile` is the user configuration file read at session start. The effect is visible in a new session: ``` $ echo "$PATH" /home/student/project/scripts:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin $ summary.sh raw measurement files: 0 ``` Preserving the old value with `$PATH` in the assignment matters. An assignment of the form `PATH="$HOME/project/scripts"` would overwrite the list, and none of the system commands would be found. This is a mistake that is a hassle to reverse: until it is fixed, most commands, including `ls`, do not run, and the fix requires typing the full path. #### Summary - The shell searches for a name in order as a keyword, alias, function, and builtin; if it finds none, it scans the `PATH` directories left to right. - The first matching file wins; the other matches for the same name are shadowed and can be seen with `type -a`. - The current directory is not in `PATH`; running a program there requires the `./` prefix, and this is a safeguard against command hijacking in writable directories. - Finding the file is not enough; without execute permission, the exit status is 126. - The shell caches the paths it finds in a hash table; `hash -r` is needed when a program is moved. - A permanent `PATH` change is written to a configuration file read at session start, and the old value must be preserved. #### Next Step The system has been introduced, the tree has been built, the script has run — but the count of measurements the script reports is still zero. The next topic descends to the files themselves: how to navigate between directories, what each column of a listing's output reports, and how the `project/data/raw` directory will be filled. ## File and Directory Operations Navigation, creation, copying, moving, and deletion; hard and symbolic links, archiving, compression, and file transfer. ### Navigation Commands Source: https://academia.sh/en/courses/introduction-to-linux/navigation-commands Changing the working directory, reading the listing output column by column, sorting criteria, and accessing a file's complete metadata. The previous topic built the `project/` tree and placed a script inside it. Before working on the tree, it must be possible to **read** it: which files exist, how much space they take, when they last changed, who owns them. This lesson decodes the listing output column by column. Reading this output is the verification tool for every operation done in the rest of the course; the result of copying, moving, linking, and changing permissions will always be seen in this same output. #### Changing the Working Directory The `cd` command changes the working directory and is used in four common forms: | Command | Effect | |---|---| | `cd path` | Switches to the given directory | | `cd` | Returns to the home directory (no argument) | | `cd ..` | Goes up one directory | | `cd -` | Returns to the previous directory and prints its name | `cd` with no argument reads the home directory from the `HOME` environment variable. This is a direct use of the environment concept established in the command path lesson: the home directory is not a value built into the shell, but a setting that comes from the environment. #### Default Listing The `ls` command, called with no argument, lists the current directory: ``` $ cd project $ ls archive data docs scripts ``` Three behaviors are default: names are sorted, those starting with a dot are not shown, and the output is split into columns according to terminal width. The third depends on where the command's output goes: ``` $ ls archive data docs scripts $ ls | cat archive data docs scripts ``` The same command writes in columns if its output goes to a terminal, and one name per line if it goes to another program. The reason is that column width is meaningless outside a terminal. This behavior matters for scripts that process the output: trying to parse the column format is unnecessary, because the script already gets one name per line. To also see names starting with a dot, `-a` is given: ``` $ ls -a scripts . .. summary.sh ``` #### Long Listing The `-l` option gives a one-line metadata summary for every file: ``` $ ls -l total 16 drwxr-xr-x 2 student student 4096 Jul 26 18:47 archive drwxr-xr-x 4 student student 4096 Jul 26 18:47 data drwxr-xr-x 2 student student 4096 Jul 26 18:47 docs drwxr-xr-x 2 student student 4096 Jul 26 18:49 scripts ``` The lines consist of seven fields: **1. Type and permissions.** The first character is the file type: `-` regular file, `d` directory, `l` symbolic link, `c` character device, `b` block device. The remaining nine characters are permission bits and will be decoded in detail in the permissions topic. **2. Link count.** The number of directory entries pointing to this file. For directories this number starts at 2 and each subdirectory adds one: the `data` directory's value of `4` comes from the two subdirectories inside it. What this number means will be fully explained in the links lesson. **3. Owner user.** The file's owner. **4. Owner group.** The group the file belongs to. **5. Size.** For regular files, the content size in bytes; for directories, the space taken by the directory entry table — **not** the total size of the files inside it. **6. Modification time.** The moment the content was last changed. If older than six months, the year is shown instead of the time. **7. Name.** The `total 16` value in the first line is the number of **blocks** the listed files occupy; it is not a file count or a byte total. Block size varies by tool and system, so this number is not comparable across systems. The size column is in bytes, and it becomes hard to read for large files: ``` $ ls -l /usr/bin/bash -rwxr-xr-x 1 root root 1543048 Mar 31 2024 /usr/bin/bash $ ls -lh /usr/bin/bash -rwxr-xr-x 1 root root 1.5M Mar 31 2024 /usr/bin/bash ``` The `-h` option converts the size into human-readable units. The conversion rounds; `-h` is not used when the exact byte value is needed. #### Listing the Directory Itself When given a directory name, `ls` lists the directory's **contents**. To see the directory's own line, `-d` is needed: ``` $ ls -ld data drwxr-xr-x 4 student student 4096 Jul 26 18:47 data ``` This distinction is used constantly to verify results in permission and ownership commands: when a directory's permissions are changed, the line to check is the directory's own, not those of the files inside it. When more than one path is given, the output is divided by headers: ``` $ ls -li scripts data scripts: total 4 21931 -rwxr-xr-x 1 student student 142 Jul 26 18:49 summary.sh data: total 8 21908 drwxr-xr-x 2 student student 4096 Jul 26 18:47 raw 21909 drwxr-xr-x 2 student student 4096 Jul 26 18:47 processed ``` The `-i` option adds a number on the far left: the **inode number**. This number is the identity, within the file system, of the structure that holds the file's metadata. It is the identity of the file, not of the name — this distinction is the foundation of the entire links lesson. #### Sorting The default sort is by name. To sort by modification time, `-t` is used: ``` $ ls -lt total 16 drwxr-xr-x 2 student student 4096 Jul 26 18:49 scripts drwxr-xr-x 2 student student 4096 Jul 26 18:47 archive drwxr-xr-x 4 student student 4096 Jul 26 18:47 data drwxr-xr-x 2 student student 4096 Jul 26 18:47 docs $ ls -ltr total 16 drwxr-xr-x 2 student student 4096 Jul 26 18:47 docs drwxr-xr-x 4 student student 4096 Jul 26 18:47 data drwxr-xr-x 2 student student 4096 Jul 26 18:47 archive drwxr-xr-x 2 student student 4096 Jul 26 18:49 scripts ``` `-t` puts the newest first; `-r` reverses the order. The `-ltr` combination puts the newest **last**, and it is the most commonly used form for long listings: the most recently changed file appears at the bottom without needing to scroll. For files with the same time, the secondary criterion is name; this keeps the output stable for the same input. Other commonly used sorts are the `-S` (by size, largest first) and `-X` (by extension) options. #### The Full Metadata `ls -l` is a summary of the metadata. The full set is read with `stat`: ``` $ stat scripts/summary.sh File: scripts/summary.sh Size: 142 Blocks: 8 IO Block: 4096 regular file Device: 0,71 Inode: 21931 Links: 1 Access: (0755/-rwxr-xr-x) Uid: ( 1001/ student) Gid: ( 1001/ student) Access: 2026-07-26 18:49:26.461777000 +0000 Modify: 2026-07-26 18:49:26.458777000 +0000 Change: 2026-07-26 18:49:26.460777000 +0000 Birth: 2026-07-26 18:49:26.458777000 +0000 ``` The output reveals three pieces of information hidden in `ls -l`. **Permissions are given in two forms at once:** `0755` and `-rwxr-xr-x`. These are the octal and letter representations of the same value; the conversion between them is the subject of the permission bits lesson. **User and group are shown together with their numeric IDs:** `1001/student`. The value the system stores is the number; the name is resolved from a mapping file. **There are three separate timestamps:** - **Access** — the moment the content was last read - **Modify** — the moment the content was last changed; `ls -l` shows this value - **Change** — the moment the metadata last changed; a permission or ownership change updates this, not the content The third is often confused with the others. When a file's permissions are changed, the `ls -lt` ordering does not change, because the time that changes is `Change`, not `Modify`. This distinction changes the answer to the question "has the file been touched." The `Blocks` field gives the number of allocated blocks and does not have to be proportional to `Size`: a 142-byte file takes up more space on disk because of block size. `stat` is not defined in POSIX, and its output format varies by tool family. The formatting option is `-c` in GNU tools and `-f` in BSD tools; in scripts that need to be portable, `ls` output or `find` criteria are preferred over `stat`. #### Summary - By default, `ls` sorts names, skips hidden files, and splits its output into columns if it goes to a terminal. - Long listing gives seven fields: type and permissions, link count, owner, group, size, modification time, name. - For directories, the size column is the size of the directory entry table; it is not the total of the contents. - `-d` shows the directory's own line, `-i` shows the inode number; the inode number is the identity of the file, not of the name. - `-t` sorts by modification time, `-r` reverses it; `-ltr` puts the newest last. - Metadata has three timestamps: access, content modification, and metadata change. #### Next Step The tree can be read now, but it is still empty: there is not a single measurement file in the `raw` directory. The next lesson takes up file creation, copying, and moving; it shows why these three operations have entirely different costs at the file system level, and under what conditions they silently lose data. ### Creating, Copying, and Moving Source: https://academia.sh/en/courses/introduction-to-linux/creating-copying-and-moving Creating files and setting timestamps, copying's overwrite behavior, the case where the target is a directory, and what moving means at the file system level. The previous lesson made the tree readable, but the `data/raw` directory was empty. This lesson fills the tree: measurement files are created, processed copies are produced, and a backup directory is set up. Three commands will be used — `touch`, `cp`, `mv` — and all three have situations where they silently lose data. Which rules give rise to these situations matters more than the commands themselves. #### Creating Files and Timestamps The `touch` command's real job is setting timestamps; creating an empty file when given a name that does not exist is a secondary behavior. ``` $ touch data/raw/measurement-01.csv $ ls -l data/raw total 0 -rw-r--r-- 1 student student 0 Jul 26 19:02 measurement-01.csv ``` The size is zero: the file exists but has no content. A file existing and a file having content are separate things; a directory entry and an inode have been created, but no data block has been allocated. Output redirection is used to write content: ``` $ printf 'zaman,sicaklik,nem\n08:00,21.4,55\n12:00,24.9,49\n' > data/raw/measurement-01.csv $ cat data/raw/measurement-01.csv zaman,sicaklik,nem 08:00,21.4,55 12:00,24.9,49 ``` The `>` sign writes the command's output to a file instead of the terminal. The grammar of redirection is the subject of the Shell Programming course; here it is used only to produce a file. The point to note is this: `>` **empties the target file without warning**. Writing to an existing file with `>` deletes the old content irreversibly. Two more measurement files are created the same way: ``` $ printf 'zaman,sicaklik,nem\n08:00,19.8,61\n12:00,23.1,52\n' > data/raw/measurement-02.csv $ printf 'zaman,sicaklik,nem\n08:00,22.6,58\n12:00,26.3,44\n' > data/raw/measurement-03.csv $ ls -l data/raw total 12 -rw-r--r-- 1 student student 47 Jul 26 19:02 measurement-01.csv -rw-r--r-- 1 student student 47 Jul 26 19:03 measurement-02.csv -rw-r--r-- 1 student student 47 Jul 26 19:03 measurement-03.csv ``` On an existing file, `touch` does not touch the content, it only sets the timestamps to that moment. To set a specific time, the `-t` option is used: ``` $ touch /tmp/sample.txt $ ls -l /tmp/sample.txt -rw-r--r-- 1 student student 0 Jul 26 19:03 /tmp/sample.txt $ touch -t 202603110930 /tmp/sample.txt $ ls -l /tmp/sample.txt -rw-r--r-- 1 student student 0 Mar 11 09:30 /tmp/sample.txt ``` The `-t` value is in `YYYYMMDDhhmm` format and is defined in POSIX. The `-d` option in GNU tools accepts human-readable date strings but is not portable; `-t` is preferred in scripts. Setting a timestamp is not just a matter of tidiness: most backup and build tools do an "is the source newer than the target" comparison, and this comparison relies on the modification time. #### Copying The `cp` command takes two arguments: source and target. Two different rules apply depending on what the target is. **If the target is an existing directory**, the source is copied into it, **under the same name**: ``` $ cp data/raw/measurement-01.csv data/processed/ $ ls -l data/processed total 4 -rw-r--r-- 1 student student 47 Jul 26 19:03 measurement-01.csv ``` **If the target is not a directory**, the source is copied under that name: ``` $ cp data/raw/measurement-01.csv data/processed/summary-01.csv $ ls -l data/processed total 8 -rw-r--r-- 1 student student 47 Jul 26 19:03 measurement-01.csv -rw-r--r-- 1 student student 47 Jul 26 19:03 summary-01.csv ``` These two rules are the source of the most common mistake: if the target directory's name is misspelled, the directory is not found, and the command creates the copy **as a file under that wrong name**. No error message appears. Putting a slash at the end of the target catches this mistake: when `targetdir/` is written and no such directory exists, the command gives an error. #### Overwriting Is Silent By default, `cp` writes over the target without asking: ``` $ cat data/processed/summary-01.csv zaman,sicaklik,nem 08:00,21.4,55 12:00,24.9,49 $ cp data/raw/measurement-02.csv data/processed/summary-01.csv $ cat data/processed/summary-01.csv zaman,sicaklik,nem 08:00,19.8,61 12:00,23.1,52 ``` The old content is gone and cannot be brought back. There is no warning, and the exit status is zero. The `-i` option asks before overwriting: ``` $ cp -i data/raw/measurement-03.csv data/processed/summary-01.csv cp: overwrite 'data/processed/summary-01.csv'? n ``` On a negative answer, the file does not change, and the command returns a non-zero exit status. `-i` is defined in POSIX and is the recommended habit for interactive use. There is also an option to protect an existing target without asking at all, but it is not portable; in GNU tools `-n` is deprecated, and when it skips a file it prints a diagnostic naming that file and returns a nonzero exit status. In scripts, the target's existence is tested first; the option is not relied on. #### Copying Directories Directories are not copied by default; `-r` is needed for recursive copying. ``` $ cp -r data/raw archive/raw-backup $ ls -l archive/raw-backup total 12 -rw-r--r-- 1 student student 47 Jul 26 19:04 measurement-01.csv -rw-r--r-- 1 student student 47 Jul 26 19:04 measurement-02.csv -rw-r--r-- 1 student student 47 Jul 26 19:04 measurement-03.csv ``` Copies are new files: a new inode, a new timestamp. If the timestamps should be preserved, `-p` is added: ``` $ cp -rp data/raw archive/raw-p $ ls -l archive/raw-p total 12 -rw-r--r-- 1 student student 47 Jul 26 19:02 measurement-01.csv -rw-r--r-- 1 student student 47 Jul 26 19:03 measurement-02.csv -rw-r--r-- 1 student student 47 Jul 26 19:03 measurement-03.csv ``` Comparing the two listings, the difference is clear: without `-p` all files carry the moment of copying, with `-p` they keep their original times. `-p` also tries to preserve permissions and ownership. This detail is decisive in backups; a backup that has lost its timestamps cannot answer the question "which version is newer." In directory copying, the target rule turns into a trap: ``` $ cp -r /tmp/demo/source /tmp/demo/target $ find /tmp/demo | sort /tmp/demo /tmp/demo/target /tmp/demo/target/source /tmp/demo/target/source/a.txt /tmp/demo/source /tmp/demo/source/a.txt ``` If the target directory **already exists**, the source is placed inside it. When the same command is run a second time, the result does not change, because `target/source` now exists too and the content is overwritten. But if it is run while the target does not exist, the target is created as a copy of the source. The same command giving two different results depends on whether the target existed beforehand. #### Moving `mv` looks like copying on the surface, but at the file system level it is an entirely different operation. ``` $ ls -li data/processed total 8 22212 -rw-r--r-- 1 student student 47 Jul 26 19:03 measurement-01.csv 22545 -rw-r--r-- 1 student student 47 Jul 26 19:03 summary-01.csv $ mv data/processed/summary-01.csv data/processed/daily-summary.csv $ ls -li data/processed total 8 22545 -rw-r--r-- 1 student student 47 Jul 26 19:03 daily-summary.csv 22212 -rw-r--r-- 1 student student 47 Jul 26 19:03 measurement-01.csv ``` The inode number has not changed: `22545` is the same under both the old and new name. The only thing that changed is **the directory entry**. The name is not a property of the file; it is a mapping kept in the directory. Changing the name does not touch the file. The measurable consequence of this is that moving within the same file system has constant cost, independent of file size. Renaming a one-byte file and a one-gigabyte file takes the same time, because no data is read or written. This does not hold when the same operation is done between different file systems. Inode numbers are meaningful only within their own file system; an entry cannot be carried from one device to another. In this case `mv` copies the content and deletes the source — the time is proportional to file size, and if the operation is interrupted midway, the target can be left incomplete. If the target is a directory, `mv` follows the same naming rule too: ``` $ mv data/processed/measurement-01.csv docs/ $ ls docs measurement-01.csv $ ls data/processed daily-summary.csv ``` `mv` also overwrites silently, and the `-i` option makes it ask. Unlike copying, here two files are lost at once: the target's old content is deleted, and the source disappears too. #### Summary - `touch` sets timestamps; creating a nonexistent file as empty is a secondary behavior, and a specific time can be given with `-t`. - In copying, if the target is an existing directory, the source is copied into it under the same name; otherwise it is copied under the given name. A misspelled directory name silently creates a file. - `cp` and `mv` write over the target without asking; `-i` makes them ask. - Copying a directory requires `-r`; `-p` preserves timestamps, permissions, and ownership. - Moving within the same file system changes only the directory entry: the inode number is preserved, and the cost is independent of file size. - Moving between different file systems turns into a copy-and-delete operation; the cost is proportional to size and carries the risk of being interrupted midway. #### Next Step All three commands in this lesson can lose data, but each does it by overwriting something. The deletion command, on the other hand, destroys directly, and the file system has no "undo" operation. The next lesson takes up what deletion means at the kernel level, under what condition a deleted file is still alive, and how to make verification a habit before running destructive commands. ### Deletion and Irreversibility Source: https://academia.sh/en/courses/introduction-to-linux/deletion-and-irreversibility Defining deletion as unlinking, the effect of open file descriptors, verification habits for destructive commands, and the danger of filenames starting with a dash. The previous lesson's commands could lose data too, but all of them did it by overwriting something. Deletion destroys directly, and the file system has no "undo" operation: a file deleted from the command line does not go to a trash can. This lesson first defines what deletion is at the kernel level — the definition explains why a deleted file can sometimes still be read — and then establishes the verification habits to apply with destructive commands. #### Deleting Is Unlinking What the `rm` command does is narrower than its name suggests: it **removes the directory entry**. The name of this operation at the system call level is `unlink`. Every file's metadata keeps a **link count**: the number of directory entries pointing to the file. `rm` decreases this count by one. The data blocks are freed only when both conditions are met: 1. The link count has dropped to zero. 2. No process is left holding the file open. The second condition can be shown directly: ``` $ printf 'temporary data\n' > /tmp/temp.txt $ exec 3< /tmp/temp.txt $ rm /tmp/temp.txt $ ls -l /tmp/temp.txt ls: cannot access '/tmp/temp.txt': No such file or directory $ cat <&3 temporary data $ exec 3<&- ``` The file has been deleted: its name is not in the directory, and listing gives an error. Yet its content could still be read, because a **file descriptor** is holding it open. The moment the descriptor closes, the data actually becomes free. This behavior has two practical consequences. First, when a running program's log file is deleted, the program keeps writing and the disk space is not reclaimed; the space is freed only when the program restarts. Second, deleting a file a program is using does not crash it — the program uses the file by its descriptor, not by its name. If the link count is greater than one, deletion does not touch the data at all; only one of the names goes. How this is possible is the subject of the next lesson. #### Deleting Directories Directories are deleted with two commands, and the difference between them is a safety criterion. `rmdir` deletes only **empty** directories: ``` $ ls archive raw-p raw-backup $ rmdir archive/raw-p rmdir: failed to remove 'archive/raw-p': Directory not empty ``` `rm -r` deletes a directory together with its contents, recursively: ``` $ rm -r archive/raw-p $ ls archive raw-backup ``` This removes the second backup that was created in the previous lesson to demonstrate the effect of the `-p` option; the `raw-backup` directory continues to exist. `rmdir` failing is not a flaw, it is a feature: when it reports that the directory is not empty, it announces the presence of content that was not intended to be deleted. Using `rmdir` to verify that a directory has been emptied is safer than using `rm -r`. #### What `-f` Does and Does Not Do `rm` gives an error for a nonexistent file and returns a non-zero status: ``` $ rm /tmp/demo2/missing.txt rm: cannot remove '/tmp/demo2/missing.txt': No such file or directory $ echo $? 1 $ rm -f /tmp/demo2/missing.txt $ echo $? 0 ``` The `-f` option does two things: it gives no error for nonexistent files, and it does not ask for confirmation. It does not increase the power to delete — it does not delete a file that cannot be deleted; permissions still apply. `-f` is used often in scripts because the "fine if it is already gone" behavior is needed. But because it also turns off confirmation questions, combined with `-r` it becomes the most dangerous form. #### Verification Habits A fact established in the shell lesson turns into a safety tool here: expansion is done by the shell, the command only sees the result. So the result can be seen before the command is run. ``` $ echo data/raw/*.csv data/raw/measurement-01.csv data/raw/measurement-02.csv data/raw/measurement-03.csv $ echo rm data/raw/*.csv rm data/raw/measurement-01.csv data/raw/measurement-02.csv data/raw/measurement-03.csv ``` The second line is the exact form of the command that will run. Putting `echo` before a deletion command and reading the output lets you see a misspelled pattern before paying its price. Running a destructive command this way once is a one-step check that should become a habit. Testing the pattern with `ls` does the same job and also verifies that the files actually exist. The shell's word-splitting rule becomes critical here. If `rm data/raw/*.csv` is mistakenly written as `rm data/raw/* .csv` — a space between the star and the dot — the pattern matches every file and the directory is emptied. The difference between the two commands is a single space; the `echo` pre-check shows this difference before it produces a result. `rm` offers two confirmation options for interactive use: ``` $ rm -i normal.txt rm: remove regular empty file 'normal.txt'? n $ rm -I *.txt rm: remove 5 arguments? n ``` `-i` asks separately for every file and is defined in POSIX. `-I` asks once, only when more than three files are given or `-r` is used; it is specific to GNU tools and is not found in BSD tools. `-I` is more usable in practice: being asked about every single file trains the user to answer "yes" automatically, while a single question is actually read. #### Names Starting with a Dash Why the `--` separator introduced in the shell lesson is necessary becomes visible here: ``` $ ls -log.txt normal.txt $ rm -log.txt rm: invalid option -- 'l' Try 'rm ./-log.txt' to remove the file '-log.txt'. Try 'rm --help' for more information. $ rm -- -log.txt $ ls normal.txt ``` The command has taken the argument starting with a dash for an option. Everything after the `--` separator is always interpreted as a filename; the `./` prefix does the same job, because the name no longer starts with a dash. This is not a theoretical problem. A pattern matching such a name can cause a command to run with unexpected options. In scripts, filenames coming from the user should always be given after a `--` separator. #### The Most Destructive Form The `rm -rf` combination deletes everything under the given path, without asking and without reporting errors. Run on the root directory, it targets the entire system. This command has **not been run** in this lesson and must not be tried. Common tool implementations have a guard for the root directory and the command is refused, but the guard can be disabled with an option and is not present on every system. The real danger is not even the root itself: with an empty variable, a line like `rm -rf "$dir"/` targets the root directory. In practice, three habits prevent most mistakes of this class. **Working with a relative path inside the directory instead of an absolute path.** In the wrong directory, the command either gives an error or affects only a narrow area. **Running `echo` before `rm` on lines with variable paths.** If the variable is empty, this shows up immediately in the output. **Moving instead of deleting.** Moving content aside before deleting it keeps the decision reversible; the moved directory is deleted deliberately after a while. The only reliable system-level way to recover deleted data is a backup. Backup and snapshot strategies are the subject of the System Administration course; there is no undo within the scope of this course. #### Summary - Deletion removes the directory entry and decreases the link count; data is not freed until the count reaches zero and no process still holds the file open. - `rmdir` deletes only empty directories, and this failure is a safety criterion; `rm -r` deletes it together with its contents. - `-f` turns off confirmation and the error for a nonexistent file; it does not grant the authority to delete. - Putting `echo` before a destructive command shows the argument list the shell will produce, without producing the result. - `-i` asks once per file, `-I` asks once for a batch operation; `-I` is not portable. - Filenames starting with a dash are mistaken for options; the `--` separator or the `./` prefix removes this. #### Next Step The definition of deletion left a question open: how can the link count be greater than one, that is, how can a file have more than one name? The next lesson answers this question and compares two different kinds of link — the hard link, which shares the same inode, and the symbolic link, which holds a path string. ### Links Source: https://academia.sh/en/courses/introduction-to-linux/links The distinction between a directory entry and an inode, a hard link sharing the same inode, a symbolic link holding a path string, and a comparison of the two kinds' constraints. The previous lesson defined deletion as "unlinking" and ended by saying a file can have more than one name. This lesson builds that possibility. The foundation is the distinction introduced in the navigation lesson: a **directory entry** maps a name to an inode, while an **inode** holds the file's metadata and the location of its data blocks. The name is not stored inside the inode, so nothing prevents more than one entry from pointing to the same inode. #### Hard Link A **hard link** is adding a new directory entry to an existing inode. ``` $ ln data/raw/measurement-03.csv data/latest-measurement.csv $ ls -li data/raw/measurement-03.csv data/latest-measurement.csv 22143 -rw-r--r-- 2 student student 47 Jul 26 19:03 data/raw/measurement-03.csv 22143 -rw-r--r-- 2 student student 47 Jul 26 19:03 data/latest-measurement.csv ``` Both lines show inode number `22143`, and the link count column has risen from `1` to `2`. This is not a copy: there is a single file with two names. No extra disk space has been used; only a directory entry has been added. There is **no hierarchy** between the two names. Which one is "real" is meaningless; both are equally close to the same inode. This shows up in deletion: ``` $ rm data/raw/measurement-03.csv $ ls -li data/latest-measurement.csv 22143 -rw-r--r-- 1 student student 47 Jul 26 19:03 data/latest-measurement.csv $ cat data/latest-measurement.csv zaman,sicaklik,nem 08:00,22.6,58 12:00,26.3,44 ``` One of the names has been deleted, the link count has dropped to `1`, and the data remains in place. The rule from the previous lesson is confirmed here: data blocks are freed only when the link count reaches zero. The deleted name can be put back, because the inode is still reachable: ``` $ ln data/latest-measurement.csv data/raw/measurement-03.csv $ ls -li data/raw/measurement-03.csv data/latest-measurement.csv 22143 -rw-r--r-- 2 student student 47 Jul 26 19:03 data/raw/measurement-03.csv 22143 -rw-r--r-- 2 student student 47 Jul 26 19:03 data/latest-measurement.csv ``` #### Two Constraints of the Hard Link Hard links have two constraints, and both are a direct consequence of the concept of an inode. **It cannot cross a file system boundary.** An inode number is meaningful only within its own file system; an entry cannot be written to an inode on another file system. ``` $ ln data/raw/measurement-01.csv /dev/shm/measurement-01.csv ln: failed to create hard link '/dev/shm/measurement-01.csv' => 'data/raw/measurement-01.csv': Invalid cross-device link ``` This is the twin, from the same root, of the observation in the moving lesson: moving within the same file system is cheap, moving outside it turns into copying — because a directory entry can only point to inodes within its own file system. **It cannot be placed on directories.** ``` $ ln data/raw data/raw-link ln: data/raw: hard link not allowed for directory ``` The prohibition exists to keep the file system a tree. If a hard link could be placed on directories, cycles could form, and any tool traversing the tree could fall into infinite recursion. The only exception to this prohibition is two entries the system itself sets up: ``` $ ls -lid data/. data/raw/.. 21906 drwxr-xr-x 4 student student 4096 Jul 26 19:08 data/. 21906 drwxr-xr-x 4 student student 4096 Jul 26 19:08 data/raw/.. ``` `data/.` and `data/raw/..` point to the same inode: both are the `data` directory itself. This also explains a number left unexplained in the navigation lesson. A directory's link count consists of: its name in the parent directory, its own `.` entry, and the `..` entry of every subdirectory. Because the `data` directory has two subdirectories, the number is $1 + 1 + 2 = 4$. #### Symbolic Link A **symbolic link** is an entirely different object: a small file with its own inode, whose content consists of nothing but a **path string**. When the kernel reads this file, it goes on to resolve the path inside it. A symbolic link suits accessing the most recent file in the measurements directory under a fixed name: ``` $ ln -s raw/measurement-03.csv data/current.csv $ ls -li data total 8 28322 lrwxrwxrwx 1 student student 22 Jul 26 19:08 current.csv -> raw/measurement-03.csv 21908 drwxr-xr-x 2 student student 4096 Jul 26 19:07 raw 21909 drwxr-xr-x 2 student student 4096 Jul 26 19:03 processed ``` The output has four details worth noting. The type character is `l`. The inode number differs from the target's — it is a separate file. The size is `22`; this is not the content of the target, but the length of the `raw/measurement-03.csv` string. The permission bits appear as `rwxrwxrwx` and are meaningless: access control is done according to the target's permissions. The target is read with `readlink`, while the content is accessed directly: ``` $ readlink data/current.csv raw/measurement-03.csv $ cat data/current.csv zaman,sicaklik,nem 08:00,22.6,58 12:00,26.3,44 ``` Most commands **follow** a symbolic link: `cat` reads the target, not the link. Commands that deal with the link itself (`rm`, `mv`, `ls -l`) do not follow it. `rm current.csv` deletes the link, and does not touch the target. #### Which Directory a Relative Target Is Resolved Against The path inside a symbolic link can be relative. In that case, resolution is done not against the directory the command is run from, but against **the directory the link is located in**. The link above sits inside `data/` and its target is `raw/measurement-03.csv`; resolution gives `data/raw/measurement-03.csv`. If the same link were created in a different directory, the target could not be found: ``` $ ln -s data/raw/measurement-01.csv /dev/shm/measurement-01.csv $ ls -l /dev/shm/measurement-01.csv lrwxrwxrwx 1 student student 27 Jul 26 19:08 /dev/shm/measurement-01.csv -> data/raw/measurement-01.csv $ cat /dev/shm/measurement-01.csv cat: /dev/shm/measurement-01.csv: No such file or directory ``` The link has been created — whether the target exists is not checked — but it cannot be resolved. There is no `data/raw` path inside `/dev/shm`. This situation is called a **dangling link**. A dangling link is not an error but the natural consequence of the definition: a symbolic link is a path string, not a handle to the target. If the target is deleted, moved, or the link is copied somewhere else, the link breaks. The selection criterion follows from this. A relative target makes the link movable with its target: when the whole tree is copied elsewhere, the link keeps working. An absolute target points to the same file wherever the link goes, but breaks when the tree is moved. Inside a tree, a relative target is preferred; for fixed, system-wide locations, an absolute target is preferred. #### Comparing the Two Kinds | Criterion | Hard link | Symbolic link | |---|---|---| | What it holds | A second entry to the same inode | The target's path string | | Its own inode | No | Yes | | File system boundary | Cannot cross | Can cross | | Can be placed on a directory | No | Yes | | When the target is deleted | Data survives, count decreases | Link breaks | | How to tell them apart | Same number with `ls -i` | `l` and `->` in `ls -l` output | The choice reduces to a simple question: is a second name needed for the same data, or a fixed pointer to a target that can change? The first case uses a hard link, the second a symbolic link. `current.csv` in the project tree is an example of the second: when a new measurement file arrives, the link's target changes, its name stays fixed. The same approach appears in system layout too. The `/bin -> usr/bin` link from the directory hierarchy lesson keeps the old path working — it could not be a hard link, since it is placed on a directory. #### Summary - A name is not a property of a file; a directory entry maps a name to an inode. - A hard link adds a second entry to the same inode; the link count rises and there is no hierarchy among the names. - A hard link cannot cross a file system boundary and cannot be placed on directories; `.` and `..` are the system's own exceptions to this prohibition. - A symbolic link is a file with its own inode, whose content is a path string; its size is the length of the target string. - A relative-target symbolic link is resolved against the directory the link is located in; if the target does not exist, the link is dangling, and this is a consequence of the definition, not an error. - Inside a tree, a relative target is preferred; for fixed, system-wide locations, an absolute target is preferred. #### Next Step The project tree now contains files, copies, and links. Packaging this whole tree into a single file — to send it to another machine or to preserve a point in time — raises a separate set of problems: how are links stored, how is the directory structure converted into a flat byte sequence, and where does compression fit into this? The next lesson begins by separating archiving from compression. ### Archiving and Compression Source: https://academia.sh/en/courses/introduction-to-linux/archiving-and-compression Separating archiving from compression, the archive format carrying metadata, the ratio–cost trade-off of compression algorithms, and verifying content before extracting. The question of packaging the tree into a single file starts where the previous lesson left off, and immediately splits into three sub-questions: how is a directory structure converted into a flat byte sequence, are permissions and timestamps preserved, are symbolic links stored with their targets' content or with their path strings? This lesson first separates the two operations — they are confused because most tools do both together — and then establishes how archives are verified. #### Two Separate Operations **Archiving** is converting multiple files, together with their metadata, into a single byte sequence. It does not shrink size; it even grows it a little. **Compression** is re-encoding a single byte sequence with a shorter representation. It does not change the number of files. The separation is a direct consequence of the Unix tradition's "small tools, one job" assumption. Every compression algorithm can be paired with every archive format; if the two jobs were merged into a single program, this freedom would be lost. The archive format places a **header block** in front of each file's content. The header holds the name, size, permissions, ownership, timestamp, and file type. Because of this, files extracted from an archive recover not only their content but also their metadata — this is something an ordinary concatenation operation cannot do. ``` $ tar -cf archive/data.tar data $ ls -l archive total 56 drwxr-xr-x 2 student student 4096 Jul 26 19:04 raw-backup -rw-r--r-- 1 student student 51200 Jul 26 19:10 data.tar ``` The options determine the operation: `-c` creates an archive, `-f` gives the archive file's name. The `data` directory given as the argument is taken recursively. The archive size being larger than the sum of the content is an expected result. The format uses fixed-length blocks and adds padding at the end of the archive; for small files, this padding can be larger than the data. #### Looking Inside the Archive Listing an archive's contents before extracting it is a habit that should be applied: ``` $ tar -tvf archive/data.tar drwxr-xr-x student/student 0 2026-07-26 19:08 data/ drwxr-xr-x student/student 0 2026-07-26 19:03 data/processed/ -rw-r--r-- student/student 47 2026-07-26 19:03 data/processed/measurement-01.csv -rw-r--r-- student/student 47 2026-07-26 19:03 data/processed/daily-summary.csv drwxr-xr-x student/student 0 2026-07-26 19:10 data/raw/ -rw-r--r-- student/student 47 2026-07-26 19:02 data/raw/measurement-01.csv -rw-r--r-- student/student 47 2026-07-26 19:03 data/raw/measurement-02.csv -rw-r--r-- student/student 42019 2026-07-26 19:10 data/raw/measurement-04.csv -rw-r--r-- student/student 47 2026-07-26 19:03 data/raw/measurement-03.csv lrwxrwxrwx student/student 0 2026-07-26 19:08 data/current.csv -> raw/measurement-03.csv ``` `-t` lists, `-v` adds detail. The output confirms everything established in the previous lessons. Permissions and ownership are preserved. Timestamps are preserved. The symbolic link in the last line has been stored **as a link, not as a copy** of its target: the type character `l`, size `0`, and the target path string are visible. If following symbolic links is wanted, the `-h` option is given; in that case the target's content, not the link, goes into the archive, and the archive grows. If the tree is being moved as a whole, the default behavior is correct — both the link and its target are in the archive, and the relative target continues to resolve. The reason for the listing habit is that archives **carry their own directory structure**. A well-formed archive contains a single top-level directory and, when extracted, creates only that. An archive that is not like this dumps dozens of files directly into the current directory and overwrites files with the same names. Listing before extracting shows this in advance. #### Removal of Absolute Paths If a file is added to the archive with an absolute path, the tool removes the leading slash and warns: ``` $ tar -cf /tmp/absolute.tar /home/student/project/docs tar: Removing leading `/' from member names $ tar -tf /tmp/absolute.tar home/student/project/docs/ ``` This behavior is a security measure. If absolute paths were preserved, a user extracting an archive could receive one that overwrites system files. When paths are kept relative, the archive can affect at most the directory it is extracted into. The practical consequence of the warning is that archives are created **standing above the tree, with a relative path**: the command `tar -cf archive/data.tar data` produces paths starting with `data/`, and wherever it is extracted, it creates a `data` directory there. #### Extracting an Archive The `-x` option extracts, `-C` specifies the target directory: ``` $ mkdir -p /tmp/restore $ tar -xf archive/data.tar -C /tmp/restore $ find /tmp/restore | sort /tmp/restore /tmp/restore/data /tmp/restore/data/current.csv /tmp/restore/data/raw /tmp/restore/data/raw/measurement-01.csv /tmp/restore/data/raw/measurement-02.csv /tmp/restore/data/raw/measurement-03.csv /tmp/restore/data/raw/measurement-04.csv /tmp/restore/data/processed /tmp/restore/data/processed/daily-summary.csv /tmp/restore/data/processed/measurement-01.csv $ ls -l /tmp/restore/data/current.csv lrwxrwxrwx 1 student student 22 Jul 26 19:08 /tmp/restore/data/current.csv -> raw/measurement-03.csv ``` The tree has come back complete, and the symbolic link has been rebuilt as a link. Thanks to its relative target, it resolves in its new location too — the payoff of the relative-target preference from the links lesson is collected here. The `-C` option removes the risk of dumping the archive into the current directory: extraction is always done into an empty directory or one set aside for this purpose. #### Compression Algorithms Compression tools work on a single file and usually replace the original file with the compressed one; the `-k` option preserves the original file. The result of three common algorithms on the same data: ``` $ gzip -k /tmp/m4.csv $ bzip2 -k /tmp/m4.csv $ xz -k /tmp/m4.csv $ ls -l /tmp/m4.csv* -rw-r--r-- 1 student student 42019 Jul 26 19:10 /tmp/m4.csv -rw-r--r-- 1 student student 5579 Jul 26 19:10 /tmp/m4.csv.bz2 -rw-r--r-- 1 student student 10776 Jul 26 19:10 /tmp/m4.csv.gz -rw-r--r-- 1 student student 2444 Jul 26 19:10 /tmp/m4.csv.xz ``` The same 42019-byte measurement file has dropped to 10776, 5579, and 2444 bytes respectively. The ordering is not a coincidence; it follows from the algorithms' design choices. | Tool | Approach | Trade-off | |---|---|---| | `gzip` | Dictionary-based coding, small window | Fastest, lowest ratio, most widely supported | | `bzip2` | Block-sorting transform | Medium speed, medium ratio | | `xz` | Dictionary-based coding, very large window | Slowest and most memory, highest ratio | The common principle is this: a higher compression ratio requires searching for patterns further back in history, which means more memory and more processing. None of the algorithms is "better" than another; they sit at different points. The selection criterion is how many times the data will be compressed and how many times decompressed. In an archive produced once and downloaded many times, the compression cost is paid once, and a high ratio pays off. In daily backups produced frequently, the compression time is paid every time. There is also data that cannot be compressed: image and audio formats that are already compressed, encrypted files, and random data. Compressing these does not shrink the size, it grows it a little — because the output must carry at least as much information as the input. #### Combining Archiving and Compression The tool can chain archiving and compression in a single command: ``` $ tar -czf archive/data.tar.gz data $ tar -cJf archive/data.tar.xz data $ ls -l archive total 76 drwxr-xr-x 2 student student 4096 Jul 26 19:04 raw-backup -rw-r--r-- 1 student student 51200 Jul 26 19:10 data.tar -rw-r--r-- 1 student student 11330 Jul 26 19:10 data.tar.gz -rw-r--r-- 1 student student 4416 Jul 26 19:10 data.tar.xz ``` `-z` applies gzip, `-j` bzip2, `-J` xz. The extension convention reflects the order: `.tar.gz` means archived first, then compressed. The order matters. Archiving first and compressing second lets repetitions between files be found too; because the measurement files share the same header line and similar number patterns, this gain is real. In exchange, reading a single file requires extracting from the start of the archive. Compressing each file separately preserves random access but misses repetition between files. The padding in archives also misleadingly raises the compression ratio: part of the 51200-byte archive dropping to 4416 bytes is the padding being compressed, not the data. When evaluating a compression ratio, the criterion is the size of the original data, not of the archive. Now that the archives are ready, the directory copy left over from the previous lesson is unnecessary: ``` $ rm -r archive/raw-backup $ ls -l archive total 72 -rw-r--r-- 1 student student 51200 Jul 26 19:10 data.tar -rw-r--r-- 1 student student 11330 Jul 26 19:10 data.tar.gz -rw-r--r-- 1 student student 4416 Jul 26 19:10 data.tar.xz ``` The archive's advantage over a directory copy is not that it is a single file, but that it freezes the metadata and link structure at a point in time. #### A Note on Portability The archiving tool has two common implementations, and their option sets are not identical. The `-c`, `-t`, `-x`, `-f`, `-v`, `-C` options above mean the same thing in both. The compression shortcuts (`-z`, `-j`, `-J`) are common but not standard; in portable scripts, compression is applied as a separate step through a pipeline. In older implementations, options can also be written without a dash (`tar cf`). This form continues to work; the dashed form is preferred in written documentation. #### Summary - Archiving converts many files, together with their metadata, into a single byte sequence; compression shortens a single byte sequence. The two operations are separate and can be combined independently. - Archive headers carry permissions, ownership, timestamp, and file type; symbolic links are stored with their target strings. - An archive is listed before extracting; absolute paths are removed, and extraction is done into a directory set aside with `-C`. - As compression ratio increases, memory and processing cost rise; the choice depends on how many times the data will be compressed and decompressed. - Archiving first and compressing second catches repetition between files but loses random access. - Data that is already compressed or random cannot be compressed; the output grows a little. #### Next Step Now that the archive has become a single file, it can be sent to another machine. But archiving and sending the entire tree every time is wasteful when only a small part of the tree has changed. The next lesson takes up remote copying and synchronization tools: how a connection is established, which tool sends only what changed, and why a trailing slash on a path changes the result. ### File Transfer Source: https://academia.sh/en/courses/introduction-to-linux/file-transfer Remote copying over an encrypted shell, verifying the server's identity, incremental synchronization sending only what changed, and the meaning of the slash at the end of the source path. The previous lesson reduced the project data to a single archive file. This file can now be sent to another machine — but archiving and sending the whole tree every time is wasteful when only a small part has changed. This lesson compares two approaches: a tool that copies the file as is, and a tool that sends only the difference. Before either, the transport layer both are built on is introduced. In the examples below, the remote machine is referred to by the name `server`, and the remote account by the name `backup`. #### Encrypted Shell The standard way to run a command on a remote machine is the **encrypted shell (ssh)**. It provides three things: the connection is encrypted, the user's identity is verified, and **the server's identity** is verified. The third is often skipped but matters most; encryption means nothing if who is on the other side is unknown. On first connecting to a server, its identity is asked about: ``` $ ssh backup@server pwd The authenticity of host 'server (127.0.0.1)' can't be established. ED25519 key fingerprint is SHA256:BXO0kZ7GJ+yv/Idqx3h4fOdWRWCgAH6QbY4y0M/YuHk. This key is not known by any other names. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added 'server' (ED25519) to the list of known hosts. /home/backup ``` The server has its own key pair, and the digest of its public key is shown as a **fingerprint**. Once approved, this fingerprint is saved to a local file. No question is asked on later connections: ``` $ ssh backup@server pwd /home/backup ``` If the saved fingerprint does not match on a later connection, the tool refuses the connection and warns. This is the mechanism that catches a man-in-the-middle attempt. The warning also appears when the server has been reinstalled; in both cases, the correct response is not to suppress the warning but to verify the fingerprint through an independent channel. The `/home/backup` path in the output shows that the command ran on the remote machine, in the home directory of the `backup` account. The user's identity is verified in two ways: a password or a key pair. With a key pair, the private key stays on the local machine and the public key is authorized on the server account; the password never goes onto the network. Key management and server configuration are the subject of the Linux Network Administration and Troubleshooting course. #### Remote Copying `scp` is the remote counterpart of local copying, and it uses the same transport layer. A remote path is written in the form `user@machine:path`: ``` $ scp archive/data.tar.xz backup@server:incoming/ $ ssh backup@server ls -l incoming total 8 -rw-r--r-- 1 backup backup 4416 Jul 26 19:16 data.tar.xz ``` The colon separator is decisive: `backup@server:incoming/` specifies a remote path, while `backup@server` alone specifies a local file of that name. When the separator is forgotten, the command gives no error — it creates a local file. In remote paths, a relative path is relative to the home directory of the logging-in user. The path `incoming/` means `/home/backup/incoming/`. `scp` sends the file as a whole; it retransmits even if the same content already exists at the target. Sending a directory requires `-r`, just as in local copying. #### Verifying the Transfer Transfer completeness is tested with a digest of the content: ``` $ sha256sum archive/data.tar.xz 6608712cb484f9629dcdf1cbaa3bced3cb87ff00b24d6d32646775ccff3b5443 archive/data.tar.xz $ ssh backup@server sha256sum incoming/data.tar.xz 6608712cb484f9629dcdf1cbaa3bced3cb87ff00b24d6d32646775ccff3b5443 incoming/data.tar.xz ``` If the two digests are the same, the two files' content is the same. A size comparison does not give this guarantee: two different contents can have the same size, but two different contents with the same digest cannot be found in practice. Digest verification is not only against transfer errors; it also tests whether the archive was changed at the source. This is why digest files are published alongside packages in software distributions — the same principle will be taken up together with the chain of signatures in the repository configuration lesson. #### Synchronizing `rsync` does not copy, it **synchronizes**: it compares the source with the target and sends only the difference. The comparison is done, by default, on size and modification time. It can be asked to show what it would do before actually running: ``` $ rsync -avn data/ backup@server:incoming/data/ sending incremental file list created directory incoming/data ./ current.csv -> raw/measurement-03.csv raw/ raw/measurement-01.csv raw/measurement-02.csv raw/measurement-03.csv raw/measurement-04.csv processed/ processed/daily-summary.csv processed/measurement-01.csv sent 351 bytes received 81 bytes 864.00 bytes/sec total size is 42,270 speedup is 97.85 (DRY RUN) ``` `-n` performs a **dry run**: nothing is transferred, only what would be done is listed — the `(DRY RUN)` in the last line confirms this. `-a` is an option set that preserves metadata — permissions, timestamps, and symbolic links; the `current.csv -> raw/measurement-03.csv` line appearing as a link in the list is a result of this. `-v` prints the files being processed. After the dry run is approved, the same command is run without `-n`. The real difference is visible on the second run: ``` $ rsync -av data/ backup@server:incoming/data/ sending incremental file list raw/measurement-01.csv sent 430 bytes received 43 bytes 946.00 bytes/sec total size is 42,284 speedup is 89.40 ``` Only one measurement file had changed in the meantime. Only that file is in the list; the data sent is 430 bytes, while the tree's total size exceeds 42 kilobytes. Doing the same job with `scp -r` would mean sending the entire tree every time. Unchanged files not being opened at all is what makes synchronization the standard tool for regular backups: the cost is proportional not to the size of the tree, but to the size of the change. #### The Slash Changes the Meaning of the Source The most often misused detail of the synchronization tool is **the slash at the end of the source path**. In the commands above, the source was written as `data/`: the trailing slash means "the **contents** of this directory," and the content is placed directly into the target. When the slash is removed, the meaning changes: ``` $ rsync -avn data backup@server:incoming/trial/ sending incremental file list created directory incoming/trial data/ data/current.csv -> raw/measurement-03.csv data/raw/ data/raw/measurement-01.csv data/raw/measurement-02.csv data/raw/measurement-03.csv data/raw/measurement-04.csv data/processed/ data/processed/daily-summary.csv data/processed/measurement-01.csv sent 370 bytes received 84 bytes 908.00 bytes/sec total size is 42,284 speedup is 93.14 (DRY RUN) ``` The paths are now listed with a `data/` prefix: the directory itself is placed inside the target, and the result becomes `incoming/trial/data/...`. The two forms of the same command produce two separate trees, one level apart. A slash on the target path has no such effect. The rule applies only to the source, and a dry run makes this difference visible every time. #### Leftovers at the Target By default, synchronization does not delete extra files at the target. If a file not present in the source is sitting at the target, it stays there: ``` $ rsync -av data/ backup@server:incoming/data/ sending incremental file list ./ sent 329 bytes received 21 bytes 700.00 bytes/sec total size is 42,284 speedup is 120.81 $ ssh backup@server ls incoming/data old-measurement.csv current.csv raw processed ``` Making the target an exact mirror of the source requires `--delete`: ``` $ rsync -avn --delete data/ backup@server:incoming/data/ sending incremental file list deleting old-measurement.csv sent 326 bytes received 39 bytes 730.00 bytes/sec total size is 42,284 speedup is 115.85 (DRY RUN) ``` This option is destructive, and the warning from the deletion lesson applies here exactly: a misspelled source path, or one pointing to an empty directory, deletes everything at the target. Every command using `--delete` must be run with `-n` first; the `deleting` lines in the dry-run list are the exact list of files that will be deleted. #### Choosing a Tool | Criterion | `scp` | `rsync` | |---|---|---| | Data sent | The entire file | Only the difference | | Repeated run | Same cost | Cost proportional to change | | Metadata preservation | Limited | Full, with `-a` | | Dry run | No | Yes, with `-n` | | Deletion at target | No | Yes, with `--delete` | | Requirement on the other side | None | The tool must be installed | Copying is enough for one-time, single-file transfers; synchronization is preferred for repeated tree transfers. If interactive browsing and selective downloading are needed, the `sftp` tool provides a file-system-like session over the same transport layer. #### Summary - The encrypted shell verifies both the user and the server; the server's fingerprint is approved on the first connection and compared on later ones. - Remote paths are written in the form `user@machine:path`; when the colon is forgotten, the command silently creates a local file. - A transfer is verified by comparing content digests; equal size gives no guarantee. - Synchronization sends only the files that changed; the cost is proportional to the size of the change, not of the tree. - A trailing slash on the source path means "the contents of the directory," the form without a slash means "the directory itself." - `--delete` makes the target an exact mirror of the source and is destructive; it must be checked with a dry run first. #### Next Step The files on the remote machine belong to the `backup` account; the local files to another. How this distinction is built — what a user's identity is stored as in the system, what groups are for, what it means for a file to belong to someone — has not been explained yet. The next topic builds the account model and moves from there to permission bits. ## Users and Permissions The account model, identity files, octal notation for permission bits, ownership management, privilege escalation, and special permission bits. ### Users and Groups Source: https://academia.sh/en/courses/introduction-to-linux/users-and-groups Identity held as a number at the kernel level, the field structure of the account and group files, the distinction between primary and supplementary groups, and name resolution. The previous lesson's output belonged to the `backup` account for files on the remote machine, and to a different account for local files. The owner and group columns in listing output had been visible from the start, but were never explained. This lesson closes that gap. What identity is stored as on the system is the input to the next lesson's permission check: when the kernel answers "can this user read this file," it compares exactly these values. #### Identity Is a Number The kernel does not know usernames. What every process is associated with is two integers: the **user ID (UID)** and the **group ID (GID)**. Names exist only for humans and are resolved by programs in user space. ``` $ id uid=1001(student) gid=1001(student) groups=1001(student),2001(project) ``` The output gives three pieces of information: the user ID, the primary group ID, and every group the user belongs to. The names in parentheses are the resolved counterparts of the numbers; the actual value is the number. The fields can also be requested separately: ``` $ id -u 1001 $ id -g 1001 $ id -un student ``` The `-n` suffix gives the name instead of the number. The `whoami` command produces the same result as `id -un`. The practical consequence: a file's owner is not a name, it is a number. If a user is deleted and a new account opens with the same number, the old files appear to belong to the new account. Likewise, files extracted from an archive can land on a different person's ownership on the target system — what an archive stores is also the number. **Root** is the account whose UID value is zero. Its privilege comes not from its name but from this number; the kernel skips most checks for UID zero. #### The Account File Account records are kept in a text file. Each line is one account, and fields are separated by colons: ``` $ grep -E '^(root|student|backup):' /etc/passwd root:x:0:0:root:/root:/bin/bash student:x:1001:1001::/home/student:/bin/bash backup:x:1002:1002::/home/backup:/bin/bash ``` The seven fields, in order, are: | Order | Field | Value in the example | |---|---|---| | 1 | Username | `student` | | 2 | Password placeholder | `x` | | 3 | User ID | `1001` | | 4 | Primary group ID | `1001` | | 5 | Comment field | empty | | 6 | Home directory | `/home/student` | | 7 | Login shell | `/bin/bash` | This file's format was already read from section five of the manual in the Help Resources lesson; this is the concrete example of what section 5 is for. The `x` in the second field is not a password. Password hashes are kept in a separate file that only root can read: ``` $ ls -l /etc/passwd /etc/shadow -rw-r--r-- 1 root root 1236 Jul 26 19:14 /etc/passwd -rw-r----- 1 root shadow 668 Jul 26 19:14 /etc/shadow $ cat /etc/shadow cat: /etc/shadow: Permission denied ``` The reason for the separation shows up in the permission columns: the account file is readable by everyone (`rw-r--r--`), because every program needs access to the username-to-ID mapping. If password hashes had the same accessibility, anyone could take the hashes and attempt offline cracking. Hashes are therefore separated and are readable only by their owner and the `shadow` group. System accounts and human accounts are separated by ID ranges. Low numbers are reserved for services; human accounts start above a threshold that depends on the system's configuration, but the separation itself is general, and account-creation tools pick the number from within this range. The shell in the seventh field runs when a session opens. For service accounts where login is not wanted, this field holds a program that terminates immediately when run — the standard way to disable an account. #### The Account's Life Cycle Account files are not edited by hand; account-management commands update these files consistently. A command that opens a new account writes not to one file but to three at once: a line in the account file, a password record in the shadow file, the primary group in the group file. It also creates the home directory and copies starting configuration files into it from the skeleton directory. Supplementary group membership can be granted afterward. A common misconception: membership added at that moment is not reflected in the running session. A process's group list is determined when the session opens and stays fixed for its duration; new membership only takes effect in a new session. Closing an account and deleting it are separate operations. **Closing** invalidates the password, or replaces the login shell with a program that terminates the session; identity and file ownership stay in place. **Deletion** removes the record from the files — but does not clean up the files the user left on the system. The result is a direct, visible consequence of identity being a number: ``` $ ls -l /data/report.txt -rw-r--r-- 1 student student 0 Jul 26 19:54 /data/report.txt $ sudo userdel student $ ls -l /data/report.txt -rw-r--r-- 1 1001 1001 0 Jul 26 19:54 /data/report.txt ``` A number appears in the owner column instead of a name: ownership has not changed, only the record that would translate the number into a name is gone. Such files silently belong to whoever gets that same number next — what needs handing over or cleaning up when an account is deleted. #### The Group File Groups are kept in a separate file, with four fields: ``` $ grep -E '^(root|student|backup|project):' /etc/group root:x:0: student:x:1001: project:x:2001:student backup:x:1002: ``` The fields, in order, are the group name, the password placeholder, the group ID, and the list of **supplementary members**. The last field deserves attention. The `student` group's member list is empty, yet the `student` user is a member of this group. That is not a contradiction: membership comes from two separate places. The **primary group** is in the account file's fourth field. New files a user creates take this group by default. **Supplementary groups** come from the group file's member lists. The line `project:x:2001:student` makes the `student` user a supplementary member of the `project` group. Total membership is the union of both sources: ``` $ groups student project ``` Giving every user a primary group named after themselves is a common configuration. The reason will be seen in the permission bits lesson: if the primary group were a shared group, every file a user creates would by default be open to that group. A separate `project` group has been defined for this course's project tree. It will be used to open the tree up for sharing in the ownership, group, and special permission bits lessons. #### Name Resolution Identity information does not always come from local files. At organizational scale, accounts can be kept in a directory service on the network, so programs call a resolution layer instead of reading the files directly. ``` $ getent passwd student student:x:1001:1001::/home/student:/bin/bash $ getent group project project:x:2001:student ``` The `getent` command queries every configured source on the system and gives the answer in a standard format. Even when a record sits on a network source rather than in a local file, the output has the same shape; the program calling the command does not need to know where the source is. In practice this means whether an account exists is tested not by grepping a file but by asking with `getent`. Not being found in the file does not show that the account does not exist. #### Account Management Commands The commands for creating, modifying, and deleting accounts and groups require root privilege and are documented in the manual's eighth section. No account will be opened in this course; the related tool family is the subject of the System Administration course. The one common operation that does not require root privilege is a user changing their own password. How this is possible is an interesting question: the file holding password hashes cannot even be read by an ordinary user, so how is writing to it made possible? ``` $ ls -l /usr/bin/passwd -rwsr-xr-x 1 root root 72056 May 30 2024 /usr/bin/passwd ``` There is an unusual character in the permission string: an `s` instead of an `x` at the third position of the owner permissions. This is the subject of the special permission bits lesson, and it will be answered at that topic's close. #### Summary - The kernel stores identity as a number; names are resolved in user space, and file ownership is bound to the number. - Root's privilege comes not from its name but from the user ID being zero. - The account file carries seven fields: name, password placeholder, user ID, primary group ID, comment, home directory, and login shell. - Password hashes sit in a separate, privilege-only-readable file to prevent offline cracking attacks. - Membership comes from two sources: the primary group in the account file and the supplementary member lists in the group file. - Name resolution is not limited to local files; queries go through a resolution layer. #### Next Step Now that identity is defined, what this identity can do to a file can be asked. The next lesson resolves the nine characters in the first column of listing output: three access types, three user classes, octal notation, and why the same bits mean something entirely different in directories. ### Permission Bits Source: https://academia.sh/en/courses/introduction-to-linux/permission-bits Splitting the nine permission bits into three classes and three access types, octal notation, the exclusivity of class selection, what bits mean differently in directories, and the mask that sets default permissions. The previous lesson defined identity. Now what this identity can do to a file can be asked. The answer has been sitting in the first column of listing output from the start: ``` $ ls -l scripts/summary.sh -rwxr-xr-x 1 student student 142 Jul 26 18:49 scripts/summary.sh $ ls -l data/raw/measurement-01.csv -rw-r--r-- 1 student student 61 Jul 26 19:17 data/raw/measurement-01.csv ``` The first character is the file type and is unrelated to permission. The remaining nine characters are the table the kernel consults on every access request. #### Three Classes, Three Access Types The nine characters split into three groups of three. The groups correspond to **user classes**: - First three: the file's **owner** - Middle three: the file's **group** - Last three: **others** Within each group, three access types stand in the same order: read (`r`), write (`w`), execute (`x`). A dash sits in place of a permission that is not granted. So `-rwxr-xr-x` means: the owner can read, write, and execute; group members can read and execute but not write; others the same. #### Class Selection Is Exclusive The most common wrong assumption is that permissions are cumulative. They are not. The kernel selects a single class in order and applies **only that one**: 1. If the requesting process's user ID is the file's owner: the owner bits apply. 2. If not, and one of the user's groups is the file's group: the group bits apply. 3. If neither: the other bits apply. Once a decision is made, the other classes are never looked at. The surprising consequence is that an owner can have less access to their own file than others do: ``` $ chmod 044 /tmp/class.csv $ ls -l /tmp/class.csv ----r--r-- 1 student student 47 Jul 26 19:20 /tmp/class.csv $ id -un student $ cat /tmp/class.csv cat: /tmp/class.csv: Permission denied ``` The file's owner is `student`, and the read attempt is denied. The group and other classes have read permission, but because the owner class was selected, they were never consulted. The owner does retain one advantage: the right to change permissions is not held in the bits, it comes from ownership. An owner can lock themselves out, but can always unlock the file again. ``` $ chmod 644 /tmp/class.csv $ cat /tmp/class.csv time,temperature,humidity 08:00,19.8,61 12:00,23.1,52 ``` #### Octal Notation Nine bits, read in groups of three, give three octal digits. The octal base from the How Computers Work course fits this job exactly: each digit encodes exactly three bits. Weights are fixed by access type: | Access | Letter | Value | |---|---|---| | Read | `r` | 4 | | Write | `w` | 2 | | Execute | `x` | 1 | A class's digit is the sum of the bits open in that class: | Octal | Bits | Meaning | |---|---|---| | 7 | `rwx` | $4+2+1$ | | 6 | `rw-` | $4+2$ | | 5 | `r-x` | $4+1$ | | 4 | `r--` | $4$ | | 0 | `---` | $0$ | So `-rwxr-xr-x` becomes `755`, `-rw-r--r--` becomes `644`, and the `----r--r--` above becomes `044`. Octal notation's advantage is exactness: `chmod 644` sets all nine bits, regardless of the previous state. The letter notation instead makes a selective change. #### Symbolic Notation `chmod`'s letter form has three parts: which class (`u` owner, `g` group, `o` others, `a` all), which operation (`+` add, `-` remove, `=` set exactly), and which bits. ``` $ chmod 644 /tmp/s.txt $ ls -l /tmp/s.txt -rw-r--r-- 1 student student 0 Jul 26 19:20 /tmp/s.txt $ chmod u+x /tmp/s.txt $ ls -l /tmp/s.txt -rwxr--r-- 1 student student 0 Jul 26 19:20 /tmp/s.txt $ chmod go-r /tmp/s.txt $ ls -l /tmp/s.txt -rwx------ 1 student student 0 Jul 26 19:20 /tmp/s.txt $ chmod a=r /tmp/s.txt $ ls -l /tmp/s.txt -r--r--r-- 1 student student 0 Jul 26 19:20 /tmp/s.txt ``` `+` and `-` touch only the named bits; the others are preserved. `=` sets a class's entire bit set to the given value, that is, it turns off any bit not listed. The choice is clear: use octal to reach a known state, symbolic to add to or remove from an existing one. The `chmod +x` usage in the command path lesson was of the second kind — the file's read permissions were kept and only execute was added. #### Meaning Changes in Directories The same three letters mean something entirely different in directories. This is the most commonly misunderstood side of the permission model. **Read (`r`)** allows the **names in a directory to be listed**. **Write (`w`)** allows **entries to be added and removed**. **Execute (`x`)** allows **passing through** a directory, that is, reaching a name inside it. The directory hierarchy lesson said the path is resolved component by component; this is the permission checked at every component. The two bits are independent and can be tested separately. A directory with only pass-through permission cannot have its contents listed, but a file whose name is known can still be reached: ``` $ chmod 100 /tmp/dir $ ls -ld /tmp/dir d--x------ 3 student student 4096 Jul 26 19:20 /tmp/dir $ ls /tmp/dir ls: cannot open directory '/tmp/dir': Permission denied $ cat /tmp/dir/data.txt content ``` Listing was denied, but the file whose name was given directly could be read. This configuration is used in practice: directories whose contents stay hidden but grant access to specific files are set up this way. The reverse is more striking — read without pass-through: ``` $ chmod 400 /tmp/dir $ ls -ld /tmp/dir dr-------- 3 student student 4096 Jul 26 19:20 /tmp/dir $ ls /tmp/dir data.txt sub $ ls -l /tmp/dir ls: cannot access '/tmp/dir/data.txt': Permission denied ls: cannot access '/tmp/dir/sub': Permission denied total 0 -????????? ? ? ? ? ? data.txt d????????? ? ? ? ? ? sub ``` The names could be read because names are the directory's **content**. But getting each name's metadata requires passing through the directory, and pass-through permission is missing; that is why every column shows a question mark. The output makes visible the distinction that names live in the directory while metadata lives in the inode. The write bit's effect is also unexpected: a user with write permission on a directory can **delete the files inside it** — even without any permission on the file itself. Because deleting changes the directory entry, not the file, the permission checked is the directory's. How this gap is closed is the subject of the special permission bits lesson. #### Default Permissions New files' permissions are not given by a command; they come from the **mask (umask)**. The mask marks the bits to be turned off: the mask's bits are cleared from the requested permissions. ``` $ umask 0022 $ touch /tmp/u1.txt $ mkdir -p /tmp/u1dir $ ls -l /tmp/u1.txt -rw-r--r-- 1 student student 0 Jul 26 19:20 /tmp/u1.txt $ ls -ld /tmp/u1dir drwxr-xr-x 2 student student 4096 Jul 26 19:20 /tmp/u1dir ``` The calculation: the starting value is `666` for files, `777` for directories. Clearing mask `022` leaves `644` for files and `755` for directories. Files starting at `666` instead of `777` is deliberate: execute permission should never be granted unintentionally, since a data file being executable is never necessary. In directories, though, a directory cannot be used at all without pass-through permission. The mask can be changed: ``` $ umask 077 $ touch /tmp/u2.txt $ mkdir -p /tmp/u2dir $ ls -l /tmp/u2.txt -rw------- 1 student student 0 Jul 26 19:20 /tmp/u2.txt $ ls -ld /tmp/u2dir drwx------ 2 student student 4096 Jul 26 19:20 /tmp/u2dir ``` A `077` mask turns off all bits for the group and other classes; new files are open only to the owner. On multi-user systems this is the preferred default. The mask is a process property inherited by child processes; to make it persist, it is written into the configuration file read at login — the same place as the `PATH` setting from the command path lesson. #### Summary - The nine permission bits split into three classes (owner, group, others) and three access types (read, write, execute). - Class selection is exclusive: the first matching class applies and the others are never consulted; an owner can have less access to their own file than others do. - In octal notation, read is 4, write is 2, execute is 1; `chmod` sets exact state with octal, selective change with symbolic notation. - In directories, read allows listing names, write allows adding and removing entries, execute allows passing through. - A user with write permission on a directory can delete the files inside it even with no permission on the files themselves. - New files' permissions are set by the mask: mask bits are subtracted from a starting value of `666` for files and `777` for directories. #### Next Step If permissions apply based on a file's owner and group, how are ownership and the group itself changed? The next lesson covers the ownership-changing commands, who can change what, and why symbolic links cause trouble when changing ownership of an entire tree. ### Changing Ownership Source: https://academia.sh/en/courses/introduction-to-linux/changing-ownership Commands for changing owner and group, why transferring ownership requires privilege, symbolic link behavior in recursive changes, and opening a tree to a group. If permissions apply based on a file's owner and group, how are ownership and the group itself set? The previous lesson's table was incomplete: the nine bits said who could do what, but the answer to "who" was two numbers sitting in the file's metadata, and how those numbers change was never explained. This lesson covers the commands that change those two fields and — more importantly — who can change them. The restriction is not arbitrary; lift it, and the permission model itself becomes meaningless. #### Transferring Ownership Requires Privilege An ordinary user cannot hand off even a file they own: ``` $ ls -l data/raw/measurement-01.csv -rw-r--r-- 1 student student 61 Jul 26 19:17 data/raw/measurement-01.csv $ chown backup data/raw/measurement-01.csv chown: changing ownership of 'data/raw/measurement-01.csv': Operation not permitted ``` The restriction has two reasons. **Disk quota.** Space used is charged to the file's owner. If transferring ownership were free, a user hitting their quota could reset it by handing their files off to someone else. Transfer is therefore left to root alone. **Privilege transfer.** As will be seen in the special permission bits lesson, a file can be marked to run with its owner's privileges when executed. If ownership transfer were free, preparing such a file and handing its ownership to a privileged account would be the shortest path to privilege escalation. The restriction is one-directional: taking a file for yourself and giving it to someone else are both forbidden. The rule reads as "the owner does not change"; the only account that can change it is root. #### Changing the Group The rule for the group is different. A file's owner can turn its group into any group they are **a member of**: ``` $ chgrp project data/raw/measurement-01.csv $ ls -l data/raw/measurement-01.csv -rw-r--r-- 1 student project 61 Jul 26 19:17 data/raw/measurement-01.csv ``` Turning it into a group they are not a member of is forbidden: ``` $ chgrp backup data/raw/measurement-01.csv chgrp: changing group of 'data/raw/measurement-01.csv': Operation not permitted ``` This too is a consistent restriction: a user can only hand off a privilege they already hold. Handing a file to a group they do not belong to would mean granting that group's members a claim on the user's own data — or polluting that group's namespace. The memberships seen in the `groups` output in the previous lesson are used directly here: that list determines which groups a user can open their files to. #### Two Fields, One Command `chown` can take owner and group together; the two fields are separated by a colon. ``` $ chown student:student data/raw/measurement-01.csv $ ls -l data/raw/measurement-01.csv -rw-r--r-- 1 student student 61 Jul 26 19:17 data/raw/measurement-01.csv $ chown :project data/raw/measurement-01.csv $ ls -l data/raw/measurement-01.csv -rw-r--r-- 1 student project 61 Jul 26 19:17 data/raw/measurement-01.csv ``` Leaving the owner part empty changes only the group; this form is equivalent to `chgrp`. The first command did not attempt to change the owner — the given value was already the current owner — so it required no privilege. A numeric ID can be written instead of a name. In environments where name resolution does not work — rescue shells or bare container environments — this is the only option. There is also a `--reference` option to match one file's ownership to another's; it is specific to the GNU tools and is used to align an entire tree to a sample file. #### Opening a Tree to a Group The project tree is about to be prepared for sharing with a group. The `-R` option applies the change recursively: ``` $ chgrp -R project data scripts docs archive $ ls -l total 16 drwxr-xr-x 2 student project 4096 Jul 26 19:11 archive drwxr-xr-x 2 student project 4096 Jul 26 19:03 docs drwxr-xr-x 2 student project 4096 Jul 26 18:49 scripts drwxr-xr-x 4 student project 4096 Jul 26 19:08 data $ ls -l data total 8 lrwxrwxrwx 1 student project 22 Jul 26 19:08 current.csv -> raw/measurement-03.csv drwxr-xr-x 2 student project 4096 Jul 26 19:10 raw drwxr-xr-x 2 student project 4096 Jul 26 19:03 processed ``` The whole tree now belongs to the `project` group. But changing the group column alone does not deliver sharing: the group permission bits are still at `r-x`, meaning group members can read but not write. Write access requires the group bits to be opened too — that the two settings are independent is a commonly overlooked side of the permission model. Whether newly created files also take on this group is a separate problem and is not solved by `chgrp`; its solution is in the special permission bits lesson. #### Symbolic Links in Recursion Recursive changes walk the tree, and the tree may contain a symbolic link. If the link's target lies outside the tree, a "change the tree's group" command can spill outside it. The default behavior does not spill — the change applies to the link itself: ``` $ ls -l /tmp/lt/outside.txt /tmp/lt/dir/link -rw-r--r-- 1 student student 2 Jul 26 19:23 /tmp/lt/outside.txt lrwxrwxrwx 1 student student 14 Jul 26 19:23 /tmp/lt/dir/link -> ../outside.txt $ chgrp -R project /tmp/lt/dir $ ls -l /tmp/lt/outside.txt /tmp/lt/dir/link -rw-r--r-- 1 student student 2 Jul 26 19:23 /tmp/lt/outside.txt lrwxrwxrwx 1 student project 14 Jul 26 19:23 /tmp/lt/dir/link -> ../outside.txt ``` The link's group has changed, the target's has not. The previous lesson said links' own permission bits are not used; that is why this change here has no practical effect either — what matters is that the target was left untouched. If following links is explicitly requested, the result changes: ``` $ chgrp -R -L project /tmp/lt/dir $ ls -l /tmp/lt/outside.txt /tmp/lt/dir/link -rw-r--r-- 1 student project 2 Jul 26 19:23 /tmp/lt/outside.txt lrwxrwxrwx 1 student project 14 Jul 26 19:23 /tmp/lt/dir/link -> ../outside.txt ``` With `-L`, the command followed the link and changed the group of the file **outside** the tree. If a directory writable by someone else exists inside the tree, a link placed there can steer a recursive command toward unintended targets. The three-option distinction is defined in POSIX, and the rule of thumb is: `-P` does not follow links and is the default for recursive operations; `-L` follows every link; `-H` follows only links given directly on the command line. The general rule for recursive permission and ownership changes is not to change the default. Without recursion, behavior on a link given directly as an argument is the reverse: the command follows the link and changes the target's ownership. To target the link itself, the `-h` option is used. #### Verification Ownership changes are silent; no error means no output. So the result is verified with a separate command. The `ls -ld` form from the navigation lesson is required here: when a directory's ownership is asked about, its own line must be read, not its contents. Whether unexpected ownership remains anywhere in the tree can be tested with `find`; the criterion options carry the names `-user` and `-group` and list files that do not match. One more verification point is metadata change's effect on timestamps. Of the three times separated in the navigation lesson, only the last — metadata change time — is updated. Because content modification time is preserved, `ls -lt` ordering is unaffected. #### Summary - Transferring ownership is open only to root; the restriction prevents abuse of disk quotas and privilege transfer. - A file's owner can turn its group only into a group they are a member of. - `chown` takes owner and group together separated by a colon; leaving the owner part empty changes only the group. - Changing the group column does not deliver sharing by itself; the group permission bits must also be opened. - Symbolic links are not followed by default in recursive changes; with `-L` they are, and the operation can spill outside the tree. - An ownership change updates only the metadata change timestamp; content time is preserved. #### Next Step Some operations, like transferring ownership, require root privilege. But logging in and working as root all day means a single typo can affect the whole system. The next lesson covers privilege escalation: what the superuser is, how privilege is delegated temporarily and narrowly, and what the principle of least privilege means in practice. ### Privilege Escalation Source: https://academia.sh/en/courses/introduction-to-linux/privilege-escalation The superuser's privilege of bypassing checks, the difference between switching users and delegating privilege per command, the structure of a privilege-delegation policy, and the principle of least privilege. The previous lesson rejected an ownership transfer with "Operation not permitted." Some operations require root privilege — but logging in as root and working all day means every typo can affect the whole system. This lesson covers how privilege is taken **temporarily and narrowly**. The subject is less about learning a command and more about applying a principle: the narrowest privilege needed for an operation, taken for the shortest time needed. #### The Superuser The account whose user ID is zero is the **superuser**. Its privilege comes from the kernel bypassing most permission checks for this identity: file permissions are not consulted, ownership can be changed, privileged network ports can be opened, file systems can be mounted. The privilege is not unlimited. Even root cannot write to a filesystem mounted read-only, or to a hardware write-protected device. These are not permission checks, so they cannot be bypassed. Working continuously as root has three concrete downsides. **Blast radius.** A destructive command run in the wrong directory stays confined to an ordinary user's own files; for root there is no boundary. **Audit trail.** An action taken in a root session does not record which person did it. On systems with multiple administrators, this makes finding the source of a problem impossible. **Accidental permanent effect.** Files created under root belong to root. A file created in a home directory while running as root leaves behind a remnant an ordinary user cannot access. Most setups have no password defined for the root account and cannot be logged into directly. Privilege is taken by one of the two methods below. #### Switching Users The `su` command switches to a **target user** within the current session and opens a new shell. Authentication asks for the **target user's password**. ``` $ su - Password: su: Authentication failure ``` The leading dash matters: when given, the target user's login environment is also set up — home directory, `PATH`, and other environment variables are set to the target account's. Without it, the user changes but the environment stays the old user's, which is a common source of confusion. The approach has two weaknesses. First, it requires sharing the root password; anyone who knows it can do anything, and it must be redistributed to everyone whenever it changes. Second, actions taken in the opened shell are not logged individually. #### Delegating Privilege Per Command `sudo` is a different model: the user is authenticated with **their own password** and runs commands defined in a policy file with the authority of a defined account. ``` $ sudo id [sudo] password for student: uid=0(root) gid=0(root) groups=0(root) ``` The command ran with root identity. The password asked for belongs to `student`; the root password is unknown and was not needed. Authentication is cached for a while: a second command given shortly after, in the same session, does not ask for the password again. The cache is cleared with `sudo -k`; in the examples above this was done before each attempt. Comparing the two models: | Criterion | `su` | `sudo` | |---|---|---| | Password asked | Target account's | Own account's | | Scope | Entire shell session | Single command | | Restrictable | No, full privilege | Command-level, by policy | | Logging | Session-level | Command-level | | Password sharing | Required | Not required | #### Reading the Policy Which commands a user can run can be asked directly: ``` $ sudo -l [sudo] password for student: Matching Defaults entries for student on host: env_reset, mail_badpass, secure_path=/usr/local/sbin\:/usr/local/bin\:/usr/sbin\:/usr/bin\:/sbin\:/bin\:/snap/bin, use_pty User student may run the following commands on host: (root) /usr/bin/id, /usr/bin/chown ``` The last line is the policy's core. The `root` in parentheses shows which account's authority the command will run with; what follows are the full paths of the permitted commands. This user can run only two programs with root authority. Policy lines have a general four-part shape: which user, on which machine, as which account, which commands. A group name can be written instead of a user; administrative privilege is most often granted through membership in a specific group. There is also a flag that allows running without a password prompt, used for automatically run jobs. The policy file is not edited directly. A policy with a syntax error can leave no one on the system able to get privilege, so the file is edited with a dedicated tool that checks syntax before saving. #### The Principle of Least Privilege Keeping the policy narrow shows up directly on a command that is not permitted: ``` $ sudo cat /etc/shadow [sudo] password for student: Sorry, user student is not allowed to execute '/usr/bin/cat /etc/shadow' as root on host. ``` The **principle of least privilege** says every actor should be given the narrowest privilege needed to do its job. In practice this means listing specific commands in the policy instead of "all commands." The principle is harder to apply than it looks, because some commands look narrow but carry broad privilege. **Commands that can open a shell.** Most text editors and pagers carry a feature for running commands from inside them. An editor allowed to run with root privilege is equivalent to handing over a root shell. **Commands with unrestricted arguments.** In the policy above, `chown`'s argument is not restricted; the user can change the owner of any file they choose, and so can transfer ownership of system files to themselves. The policy looks narrow, but its effect is broad. **Commands that write files.** A command that can write to any file with root privilege can add a line to the account file and create a new privileged user. The question to ask when writing a policy is not "is this command harmful," it is **"can a root shell be obtained through this command."** #### Cleaning the Environment The `env_reset` and `secure_path` settings in the policy output are a quiet but critical part of privilege delegation. The shadowing from the command path lesson would turn into an attack right here: ``` $ printf '#!/bin/sh\necho fake identity\n' > /tmp/demo/id $ chmod +x /tmp/demo/id $ PATH="/tmp/demo:$PATH" $ id fake identity $ sudo id [sudo] password for student: uid=0(root) gid=0(root) groups=0(root) ``` In the user's own session, the name `id` resolved to the fake program. Under privileged execution, the real program ran: the delegation tool ignored the user's `PATH` value and used the safe list defined in the policy. Without this, a policy of the form "may only run `id`" would mean nothing; the user would run their own program named `id` with root privilege. Resetting environment variables rests on the same reasoning: variables like a library search path can change a running program's behavior from the outside. The same reasoning produces a good habit for ordinary users too: commands to be run with privilege are written with their full paths. #### Summary - The superuser bypasses most of the kernel's permission checks; it does not bypass non-permission restrictions like a read-only mount. - Working continuously as root enlarges the blast radius, destroys the audit trail, and leaves inaccessible remnants. - `su` asks for the target account's password and delegates the whole session; `sudo` asks for its own password and grants privilege per command, restricted by policy. - A policy has four parts: user, machine, the account acted as, and permitted commands. - Under the principle of least privilege, the criterion is not whether a command looks harmful, but whether a root shell can be obtained through it. - The privilege-delegation tool resets `PATH` and environment variables; otherwise the command name could be shadowed and the policy rendered ineffective. #### Next Step Privilege delegation was an externally configured way to run a command under a different identity. The same effect has a form embedded directly in a file's own metadata: the `s` in the `-rwsr-xr-x` string seen in the users lesson. The next lesson closes the topic by covering these three extra bits — two that change the identity a program runs as, and one that narrows delete permission in directories. ### Special Permission Bits Source: https://academia.sh/en/courses/introduction-to-linux/special-permission-bits The three bits above the nine permission bits; setuid and setgid, which change who a program runs as, group inheritance in directories, and the sticky bit that narrows delete permission. The users lesson left one detail open: in the permission string of the password-change program, there was an `s` instead of an `x` at the third position of the owner class. ``` $ ls -l /usr/bin/passwd -rwsr-xr-x 1 root root 72056 May 30 2024 /usr/bin/passwd ``` This lesson explains that letter and its two siblings. Three extra bits solve three separate problems the nine permission bits cannot, and all three are forms of privilege escalation — this time embedded in the file's own metadata. #### Four-Digit Notation The permission field is actually **twelve bits**, not nine. Three extra bits form the fourth digit from the left in octal notation: | Bit | Octal | Name | |---|---|---| | setuid | 4000 | Set user ID | | setgid | 2000 | Set group ID | | sticky | 1000 | Sticky | The full value can be read with `stat`: ``` $ stat -c '%a %A %n' scripts/summary.sh share 4755 -rwsr-xr-x scripts/summary.sh 3770 drwxrws--T share ``` `4755` means setuid plus `755`; `3770` means setgid plus sticky plus `770`. `chmod` can take a four-digit value; the symbolic counterparts are `u+s`, `g+s`, and `+t`. The extra bits do not get their own column; they change the letter at the existing `x` positions: | Position | With `x` on | With `x` off | |---|---|---| | Owner (setuid) | `s` | `S` | | Group (setgid) | `s` | `S` | | Others (sticky) | `t` | `T` | An uppercase letter reports that the extra bit is on but the execute bit beneath it is off. This is usually a sign of misconfiguration; the `drwxrws--T` output above, though, is an intended state — the "others" class has been given no permission at all on this directory. #### setuid: Changing Who the Runner Is Normally a program runs with the identity of the user who runs it. If the **setuid** bit is on for an executable file, it runs with its **owner's** identity, not the runner's. The password-change program is marked this way for exactly this reason. The program is owned by root and has the setuid bit on; when an ordinary user runs it, the process runs with root privilege and can update the password file, which only root can write to. The user does not access the file directly; they access a controlled program that does. The model follows the same principle as the policy in the privilege escalation lesson, but keeps the privilege not in an externally defined policy but in the file's own metadata. Two consequences follow. **setuid programs are an attack surface.** Every bug in such a program can turn into privilege gain. For this reason setuid programs are kept few, do narrow work, and scrutinize their input closely. The list of setuid files on a system can be pulled with `find`; the criterion option carries the name `-perm` and is a standard step in routine audits. **It is ignored in shell scripts.** The kernel does not apply this bit to files run through an interpreter: ``` $ printf '#!/bin/sh\nid -un\n' > /tmp/identity.sh $ chmod 4755 /tmp/identity.sh $ ls -l /tmp/identity.sh -rwsr-xr-x 1 student student 17 Jul 26 19:31 /tmp/identity.sh ``` The result does not change when another user runs it: ``` $ id -un backup $ /tmp/identity.sh backup ``` Although the bit appears on, the process ran with the `backup` identity. The reason is a race condition: in the time between the kernel opening the file and running the interpreter, the file could be changed, and the privileged interpreter could end up running a different script. This door is closed by ignoring the bit. #### setgid: On Files and On Directories On executable files, the **setgid** bit is setuid's group counterpart: the process runs with the file's group, not the runner's. On directories it carries an entirely different and far more commonly used meaning: **new files created in the directory take the directory's group, not the creator's primary group.** This was exactly the problem left open in the ownership lesson. The tree's group had been changed, but new files still formed with the user's primary group. ``` $ mkdir -p share $ chgrp project share $ chmod 770 share $ ls -ld share drwxrwx--- 2 student project 4096 Jul 26 19:30 share $ touch share/test-1.csv $ ls -l share total 0 -rw-r--r-- 1 student student 0 Jul 26 19:30 test-1.csv ``` Even though the directory belongs to the `project` group, the file created inside it takes on the `student` group; group members cannot write to it. Once the setgid bit is added, the behavior changes: ``` $ chmod g+s share $ ls -ld share drwxrws--- 2 student project 4096 Jul 26 19:30 share $ touch share/test-2.csv $ ls -l share total 0 -rw-r--r-- 1 student student 0 Jul 26 19:30 test-1.csv -rw-r--r-- 1 student project 0 Jul 26 19:30 test-2.csv ``` The file created before the bit kept its old group; the one created after took the directory's group. The bit is also **copied to newly created subdirectories**, so the inheritance persists across the whole tree. For group sharing to work, three settings need to be made together: the directory's group, group write permission, and the setgid bit. If any one of the three is missing, sharing does not work, and the symptom looks different each time. For new files to also get group write permission, the mask needs adjusting too; the `022` mask from the permission bits lesson turns off group write. When working in shared directories, the mask is set to `002`. #### The Path Must Be Passable Sharing does not end with directory permissions. The rule from the path resolution lesson comes into play here: reaching a shared directory requires **pass-through permission at every directory along the path**. ``` $ chgrp project ~ $ chmod 710 ~ $ ls -ld ~ drwx--x--- 5 student project 4096 Jul 26 19:13 /home/student ``` A value of `710` gives the owner full access and the group only pass-through permission. Group members cannot **list** the home directory's contents, but can pass through it: ``` $ id -un backup $ ls /home/student ls: cannot open directory '/home/student': Permission denied $ ls /home/student/share test-1.csv test-2.csv ``` The reverse of the "read without pass-through" example from the permission bits lesson has become a real configuration here: the home directory's contents stay hidden while a subdirectory whose name is known can be shared. #### The Sticky Bit The shared directory now works, but the gap left open in the permission bits lesson is still open: a user with write permission on the directory can delete **any** file inside it. ``` $ id -un backup $ ls -l total 0 -rw-rw-r-- 1 student student 0 Jul 26 19:31 test-1.csv -rw-rw-r-- 1 student project 0 Jul 26 19:31 test-2.csv $ rm test-1.csv $ ls -l total 0 -rw-rw-r-- 1 student project 0 Jul 26 19:31 test-2.csv ``` The `backup` user deleted a file they did not own. The definition from the deletion lesson explains this: deleting removes a directory entry, and the permission checked is the directory's write permission. The **sticky bit** narrows this rule. In a directory with the bit on, only three parties can remove an entry: the file's owner, the directory's owner, and root. ``` $ chmod +t share $ ls -ld share drwxrws--T 2 student project 4096 Jul 26 19:31 share ``` The same user tries the same operation again: ``` $ rm test-2.csv rm: cannot remove 'test-2.csv': Operation not permitted $ touch backup-note.csv $ ls -l total 0 -rw-rw-r-- 1 student project 0 Jul 26 19:31 test-2.csv -rw-r--r-- 1 backup project 0 Jul 26 19:31 backup-note.csv $ rm backup-note.csv $ ls -l total 0 -rw-rw-r-- 1 student project 0 Jul 26 19:31 test-2.csv ``` Deleting someone else's file is blocked; deleting a file the user created is not blocked. Write permission has not disappeared either — new files can still be added to the directory. The bit narrows only **deletion and renaming**. This is the standard configuration for temporary directories on a system: ``` $ ls -ld /tmp drwxrwxrwt 1 root root 4096 Jul 26 19:24 /tmp ``` The `rwxrwxrwt` string says: everyone can write, but everyone can only delete their own file. Without the sticky bit, the temporary directory would be a space where everyone could delete everyone else's files. #### Summary - The permission field is twelve bits; three extra bits form the fourth octal digit. - The extra bits do not get their own column, they change the letter at the `x` positions; an uppercase letter reports the execute bit beneath it is off. - A setuid-marked executable runs with its owner's identity, not the runner's; it is ignored in shell scripts because of a race condition. - In directories, setgid makes new files take the directory's group and is copied to subdirectories; group sharing needs the group, write permission, and this bit together. - Reaching a shared directory requires pass-through permission at every directory along the path; a value of `710` grants pass-through without listing. - The sticky bit lets only the file's owner, the directory's owner, and root remove an entry in a directory; it is the standard configuration for temporary directories. #### Next Step The identity, permission, and privilege model is complete; the project tree can now be safely shared with a group. One last layer about the system itself remains: how did the tools used to work in this tree get onto the system, how are they updated, and how are they removed? The next topic opens with the concept of a package manager and closes the course with the text editors needed to work in the tree. ## Package Management and Editors Repository, dependency, and version resolution; installing, removing, and verifying packages; the repository trust model; and terminal text editors. ### The Package Manager Concept Source: https://academia.sh/en/courses/introduction-to-linux/package-manager-concept A package's structure as files plus metadata, the function of the installed-package database, dependency resolution as a constraint problem, and version comparison rules. Every tool used throughout the course was already present on the system. How it got there, how it is updated, and how it is removed was never asked. The question is harder than it looks. Installing a program is not just copying files: the executable goes into one directory, its libraries into another, its configuration under `/etc`, its documentation into the manual directory — exactly the distribution the layout standard from the directory hierarchy lesson defined. Done by hand, no record exists of which file belongs to which program, and removal becomes impossible. #### What a Package Is A **package** is an archive carrying the files to be installed together with metadata about those files. Its structure resembles the format from the archiving lesson: a file tree plus header information. The difference is that the header is read for installation decisions. The information metadata carries shows naming differences across package manager families, but the fields are common: | Field | Function | |---|---| | Name | The package's identity | | Version | Which release it is | | Architecture | Which processor family it was built for | | Dependencies | Other packages required for it to run | | Conflicts | Packages it cannot be installed alongside | | Provides | The abstract capability names it satisfies | | Checksums | A content digest of every file | | Install scripts | Steps to run before and after installation | The last two fields are often skipped over but are decisive. Checksums make it possible to test whether installed files have since been modified — the system-scale counterpart of the verification method from the file transfer lesson. Install scripts cover the cases file copying alone cannot: opening a user account, registering a service, adjusting a database — and because they run with root privilege, they sit at the center of package trust. #### The Installed-Package Database A package manager's second component is a local database holding a record of every package installed on the system. The record contains the package's metadata and the full list of files it installed. Without this list, three operations cannot be done. **Removal.** Which files to delete cannot be known; software installed by hand leaves remnants behind when removed. **Upgrading.** Which files of the old version are absent from the new one cannot be known; remnant files accumulate. **Ownership query.** Which package a file on the system belongs to cannot be asked; this query is used to determine which package's responsibility a fault should be charged to. Another consequence of the database is that **manually modified files** can be distinguished. Configuration files ship with the package but are edited by the administrator; on upgrade, these changes need to be preserved. Package managers compare a file's checksum against the recorded one to tell whether it has changed, and if it has, they ask the user instead of overwriting it on upgrade. #### Why Dependencies Exist A dependency is not an abstract concept; it can be seen in the executables themselves: ``` $ ldd /usr/bin/ls linux-vdso.so.1 (0x0000ffff88cce000) libselinux.so.1 => /lib/aarch64-linux-gnu/libselinux.so.1 (0x0000ffff88be0000) libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000ffff88a20000) /lib/ld-linux-aarch64.so.1 (0x0000ffff88c80000) libpcre2-8.so.0 => /lib/aarch64-linux-gnu/libpcre2-8.so.0 (0x0000ffff88970000) ``` The listing command does not run on its own: it depends on the standard C library, the security-labeling library, and the regular expression library. The numbers in parentheses are the libraries' load addresses for that particular run and change on every invocation; the library names do not change. These libraries come in separate packages. The linking and loading lesson in the How Computers Work course defined this relationship; package dependencies are that relationship's distribution-level counterpart. The reason for using a shared library is avoiding duplication: hundreds of programs do not each carry a separate copy of the same library. The cost is that installation turns into a dependency graph. The **provides** concept loosens this graph. A package can supply an abstract capability name alongside its concrete one; other packages depend not on the concrete name but on the capability, so different implementations doing the same job can substitute for one another. This is the packaging-level counterpart of the `/bin/sh` example from the shell lesson: a dependency is built on a role, and which program fills that role is the system's decision. #### Resolution Is a Constraint Problem When a package's installation is requested, the question that must be solved is: which packages, added to the currently installed set, satisfy every dependency and produce no conflict? This is a search problem. A small example shows its shape: | Package | Dependency | Conflict | |---|---|---| | `plot` 2.0 | `graphics` ≥ 3 | — | | `report` 1.4 | `graphics` ≤ 2 | — | | `graphics` 1.8 | — | — | | `graphics` 2.9 | — | — | | `graphics` 3.1 | — | — | `plot` and `report` cannot be installed on the same system: one wants the third release of the `graphics` library, the other wants at most the second. The resolver reports this and waits for the user to choose. On real systems this graph carries tens of thousands of nodes, and constraints grow complex with intersections, alternatives, and version ranges. Resolution ends in one of three states: a solution is found and a list of actions is presented; no solution exists and a conflict is reported; or multiple solutions exist and the manager's preference rules — which repository takes priority, which version is preferred — take over. For this reason, installation commands **list what they will do and ask for confirmation** before acting. The list carries not just the requested package but every dependency it pulls in and everything that needs removing; it should not be approved without being read. #### Version Comparison Since dependencies set constraints of the form "at least this version," versions must be comparable. Comparison cannot be done lexicographically: - Version `1.10` is **greater than** `1.9`, but compared as text it looks smaller. - `2.0` and `2.0.0` are treated as equal under most rules. - Version `1.0-beta` is **less than** `1.0`; letter-bearing suffixes signal pre-release. For this reason every package manager family has a defined comparison algorithm: the version string is split into numeric and alphabetic parts, numeric parts are compared as numbers, alphabetic parts are compared in order. There is also a need to override ordering by hand. When a piece of software's version numbering scheme changes, the new version can look smaller than the old one. To handle this, a number consulted before anything else during comparison is prepended to the version string; when this number increases, every subsequent comparison favors the new scheme. #### Package Manager Classes Tools operate at two levels, and the split exists in every family. The **low-level tool** deals with a single package file: it unpacks it, places the files, records it in the database. It does **not** resolve dependencies; it errors out if one is missing. Querying installed packages is also at this level. The **high-level tool** knows about repositories, resolves dependencies, downloads, and calls the low-level tool. This is the tool a user works with day to day. The split has a practical consequence: if a package file is in hand and installed with the low-level tool, dependencies can be left unsatisfied. The correct approach is handing the file to the high-level tool, so missing dependencies are filled in from the repository. There is also a domain outside the system package manager: programming languages' own package managers. The two keep separate databases and are unaware of each other. If the same library is installed through both, which copy gets used becomes ambiguous. Common practice is installing language-level packages not into system directories but into a project- or user-specific area. #### Summary - A package is an archive carrying the files to install and metadata; the metadata includes dependency, conflict, checksum, and install-script information. - The installed-package database is the record that makes removal, upgrading, and file ownership queries possible. - Dependencies are the concrete result of shared libraries; abstract capability names let different implementations substitute for one another. - Installation is a constraint problem; a solution may not exist or may not be unique, so tools present the action list for approval. - Version comparison cannot be done lexicographically; a version string is split into numeric and alphabetic parts for comparison. - The low-level tool deals with a single package and does not resolve dependencies; the high-level tool knows about repositories and performs resolution. #### Next Step With the concepts established, operations can be covered. The next lesson introduces the set of operations every package manager has a counterpart for, whatever its family: updating the index, installing, removing, purging along with configuration, querying, and verifying — and which of these operations can be undone. ### Installing and Removing Packages Source: https://academia.sh/en/courses/introduction-to-linux/installing-and-removing-packages The operation classes common to package manager families, reading the action list, the difference between removal and purging, and installation areas outside the package manager. The previous lesson established the package concept and dependency resolution. This lesson descends to everyday operations. Command names vary by package manager family; this course does not teach a specific tool's syntax. What is taught is **which operation does what and in what order** — this knowledge travels, command names do not. To find their counterparts on a given system, the manual pages from the help resources lesson are enough. #### Operation Classes Most package managers have a counterpart for these operations: | Operation | What it does | |---|---| | Index update | Refreshes the local copy of the repositories' package list | | Search | Searches for a package by name and description | | Show | Prints a package's metadata | | Install | Installs a package and its dependencies | | Remove | Deletes the package's files, leaves configuration | | Purge | Also deletes configuration files; not a separate operation in every family | | Autoremove | Removes packages installed as dependencies that are no longer needed | | Upgrade | Moves installed packages to their new versions | | File list | Lists the files a package installed | | Ownership query | Finds which package a file belongs to | | Verify | Compares installed files against the recorded checksums | The first row is the operation that should come first in sequence. The local index is a copy of the repository's contents and can go stale. An installation done with a stale index tries to download a version no longer in the repository and fails. This is why installation follows an index update. The last row is where the checksums covered in the previous lesson get used: if an installed file's content differs from the recorded one, it was either edited by hand or has been corrupted. This is one of the first questions to ask in a fault investigation. #### Reading the Action List Install and remove commands list what will happen before acting and ask for confirmation. The list has three sections: what will be installed, what will be removed, what will be upgraded. Three points need checking before confirming. **Is the removal list empty?** If installing a package requires removing others, a conflicting dependency exists. If the list contains unexpected names, the operation should be canceled. **Is the number of pulled-in dependencies at the expected level?** A small tool pulling in hundreds of packages can mean the wrong package was chosen, or that a far too broad meta-package was requested. **Are the packages coming from the expected repository?** If more than one repository is configured, the same package can come from different sources. Source is the subject of the next lesson. Installation operations require root privilege; they write to system directories and run install scripts with privilege. The principle from the privilege escalation lesson applies directly here: granting package-installation privilege is close to granting root privilege itself, because install scripts run as root. #### Removal and Purging The difference between the two removal operations lies in the configuration files. **Removal** deletes the files the package brought but leaves the configuration under `/etc`. The reasoning: configuration files have been edited by the administrator, and that effort is expected to be preserved if the package is removed and reinstalled. **Purging** deletes the configuration too. This is chosen when the package is certain never to be used again and a clean state is wanted. Neither deletes user data. Settings in home directories and data under `/var` are not in the package manager's record; the package did not create them, and it does not delete them either. If a service's data should also go after it is removed, that is a separate, deliberate step. Packages installed as dependencies are marked separately. When a package is removed, only the dependencies pulled in for it remain on the system, but they are marked "no longer needed"; the autoremove operation collects them. Without this distinction, the system would fill over time with packages nothing uses. #### Upgrading Upgrading has two scopes, and confusing them is risky. **Single-package upgrade** renews only the named package. If its dependencies also require a new version, they are upgraded too. **System upgrade** moves every installed package to the repository's current versions. Because the repository is published as a consistent whole, this is the recommended path. The dangerous zone in between is a **partial upgrade**: when some packages are left new and some old, the versions of interdependent libraries can stop matching. Repository maintainers test packages together; a set upgraded separately is an untested combination. If a version needs to be deliberately held back, package managers offer a **pinning** mechanism for this: a package or version is flagged and left out of upgrades. Pinning is knowingly taking on the risk of a partial upgrade, and the reason should be documented. Downgrading is not supported to the same degree in every family; one family has a dedicated command for it while another requires the version to be named by hand. Install scripts are written to run forward; going back to an old version after an upgrade that converted a data format can leave the data unreadable. #### Outside the Package Manager Not all software is in a repository. The layout standard defines an area set aside for this case: `/usr/local` is where locally installed software goes, untouched by the package manager. The user-level counterpart of the same distinction is a local directory under the home directory, and it requires no root privilege. ``` $ mkdir -p ~/.local/bin $ chmod 755 ~/.local/bin/project-backup $ cat ~/.local/bin/project-backup #!/bin/sh # Archives the project tree with a dated name. tar -czf "$HOME/project/archive/project-$(date +%Y%m%d).tar.gz" -C "$HOME" project/data project/scripts project/docs ``` This directory is added to the search list by the configuration read at login: ``` $ echo "$PATH" /home/student/project/scripts:/home/student/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin $ command -v project-backup /home/student/.local/bin/project-backup ``` The tool can now be called from anywhere: ``` $ project-backup $ ls -l project/archive total 84 -rw-r--r-- 1 student student 11665 Jul 26 19:35 project-20260726.tar.gz -rw-r--r-- 1 student project 51200 Jul 26 19:10 data.tar -rw-r--r-- 1 student project 11330 Jul 26 19:10 data.tar.gz -rw-r--r-- 1 student project 4416 Jul 26 19:10 data.tar.xz ``` The command from the archiving lesson has turned into a tool that runs with a dated name in a single call. This approach's limit is the database discussion from the previous lesson. For a single-file tool, removal is one `rm` command. For software that spreads dozens of files across six directories, removal cannot be done reliably because no record exists of which file belongs to what. That is exactly the problem the package manager solves. The criterion is this: single-file, dependency-free tools can go into the local directory; dependent software that spreads across the system should go through the package manager. For the middle ground, a layout that confines the software to its own directory and links it to the search list with a single link is preferred; removal then reduces to deleting that directory. #### Summary - Package managers offer the same operation classes under different command names; what travels is the operation knowledge, not the command name. - Installation happens after the local package index is updated; a stale index tries to download versions no longer available. - The action list is checked for what will be removed, how many dependencies are pulled in, and where the packages come from. - Removal leaves configuration, purging deletes it too; neither touches user data. - A partial upgrade produces an untested combination of versions; pinning is knowingly taking on that risk. - Installation outside the package manager suits single-file, dependency-free tools; it is not reliable for software that spreads across the system, since no removal record exists. #### Next Step Where packages come from was called the "repository" in this lesson but never defined. A repository is not just a file server: an index file, that file's signature, and package checksums together build a chain of trust. The next lesson opens that chain and shows why adding a new repository to the system is a trust decision. ### Repository Configuration Source: https://academia.sh/en/courses/introduction-to-linux/repository-configuration The components of a repository definition, the chain of trust formed by a signed index and package checksums, verifying a key's fingerprint, and the consequences of repository priority. The previous two lessons said packages come "from the repository" but never defined what a repository is. A repository is not just a file server; if it were, nothing would say the downloaded package is the right package. This lesson covers the repository's structure and the chain of trust built on it. The chain is the combination of the checksum verification from the file transfer lesson and the fingerprint confirmation set up in that lesson's encrypted shell section; the two meet here in a single mechanism. #### Repository Definition Introducing a repository to the system means giving it at least this information: | Field | Meaning | |---|---| | Address | The base location the files are downloaded from | | Release | Which version branch the repository offers | | Component | A subsection within the repository (for example, the free-software split) | | Architecture | Which processor family packages will be requested for | | Key | The public key that will verify the repository's signatures | Definitions are kept under `/etc`, following the "configuration lives in text files" principle from the directory hierarchy lesson. Naming and syntax vary by package manager family; the fields do not. A consequence of the address field is that the same content can be served from more than one server: **mirrors**. A mirror is an independent server carrying an exact copy of the repository, used to speed up downloads. Verifying a mirror's content is the reason the chain below exists; otherwise each mirror would have to be trusted separately. #### The Chain of Trust A repository serves two kinds of file: the packages themselves, and an **index file**. The index carries every package's name, version, dependencies, and **content checksum**. The index file itself is signed. The chain has three links: 1. The public key represents the identity of the party running the repository, and is introduced to the system once. 2. The index file's signature is verified with this key; if the signature is valid, the index's content has not been altered. 3. Every downloaded package's checksum is compared against the value in the index; if they match, the package has not been altered. A single signature indirectly covers thousands of packages. Packages do not need to be signed individually; it is enough that their checksums sit in a signed file. The chain can be built and verified at small scale. The repository below consists of two packages, an index file, the index's signature, and the repository's public key: ``` $ ls INDEX INDEX.asc plot-2.0.pkg repo-key.asc report-1.4.pkg ``` #### Introducing the Key The first step is examining the key. A key's identity is represented by its **fingerprint**, a fixed-length checksum: ``` $ gpg --show-keys --with-fingerprint repo-key.asc gpg: directory '/home/student/.gnupg' created gpg: keybox '/home/student/.gnupg/pubring.kbx' created pub ed25519 2026-07-26 [SC] E31E 9629 AEBE 63E9 0EEB EDEE A258 E130 F14C ACAA uid Repo Maintainer ``` There is a critical step here, and skipping it renders the whole chain meaningless: the fingerprint must be compared against a source **independent** of the channel the key was downloaded through. Downloading the key from the repository itself and verifying it against the fingerprint the same repository states verifies nothing. The same problem existed with the server fingerprint in the file transfer lesson's encrypted shell section, and the solution is the same: identity is confirmed through a different channel on first contact, and any change afterward is caught automatically. Once confirmed, the key is imported into the system: ``` $ gpg --import repo-key.asc gpg: /home/student/.gnupg/trustdb.gpg: trustdb created gpg: key A258E130F14CACAA: public key "Repo Maintainer " imported gpg: Total number processed: 1 gpg: imported: 1 ``` #### Verifying the Chain The index's signature is tested with the key: ``` $ gpg --verify INDEX.asc INDEX gpg: Signature made Sun Jul 26 19:37:53 2026 UTC gpg: using EDDSA key E31E9629AEBE63E90EEBEDEEA258E130F14CACAA gpg: Good signature from "Repo Maintainer " [unknown] gpg: WARNING: This key is not certified with a trusted signature! gpg: There is no indication that the signature belongs to the owner. Primary key fingerprint: E31E 9629 AEBE 63E9 0EEB EDEE A258 E130 F14C ACAA ``` The two lines say separate things and should not be confused. **"Good signature"** reports that the file was signed with this key and has not changed since — a mathematical verification. The **warning** says there is no proof the key's owner is really who they claim to be; this means the independent-channel confirmation above was not done. The signature is valid, but who it belongs to is a separate question. Once the index is verified, package checksums are tested: ``` $ cat INDEX 3375dab7d8aa59a185e9e8f5e0f4095517d89bdb75245ef22a25966e2e348756 plot-2.0.pkg 3fe6fac9c36f99b284199580aead74c6c2d33f3a60449954c90dac700ca48e08 report-1.4.pkg $ sha256sum -c INDEX plot-2.0.pkg: OK report-1.4.pkg: OK ``` The chain is complete: the signature guarantees the index, the index guarantees the packages. #### What the Chain Catches If the package file is changed in transit, the checksum stops matching: ``` $ printf 'malicious content\n' > plot-2.0.pkg $ sha256sum -c INDEX plot-2.0.pkg: FAILED report-1.4.pkg: OK sha256sum: WARNING: 1 computed checksum did NOT match ``` To get past this, an attacker must also change the index. Once checksums are recomputed, the index becomes internally consistent again: ``` $ sha256sum *.pkg > INDEX $ sha256sum -c INDEX plot-2.0.pkg: OK report-1.4.pkg: OK ``` But the signature no longer matches: ``` $ gpg --verify INDEX.asc INDEX gpg: Signature made Sun Jul 26 19:37:53 2026 UTC gpg: using EDDSA key E31E9629AEBE63E90EEBEDEEA258E130F14CACAA gpg: BAD signature from "Repo Maintainer " [unknown] ``` This is where the chain's strength lies: changing the index requires the signing key, and that key sits not in the repository but with the maintainer. So compromising a mirror is not enough to alter the packages. This is a guarantee transport-layer encryption cannot provide. An encrypted download shows the data did not change **in transit**; it does not show the content on the server was the correct content. A signature verifies the content's origin, and having passed through mirrors does not change that. The two protections do not substitute for each other. #### Adding a Repository Is a Trust Decision Adding a new repository to the system means granting its maintainer the following: the ability to install packages on the system, the ability to run install scripts with root privilege, and the ability to replace existing packages with their own versions. The last item is the least noticed. If a new repository offers a package with the same name as one in the system repository but a higher version number, that package takes the system package's place on upgrade. For this reason package managers offer a **priority** mechanism: each repository is given a weight, and which source the same package is taken from is decided by that weight. In practice, three criteria are followed. **The number of repositories is kept low.** Every extra repository adds a party that must be trusted. **Extra repositories' scope is narrowed.** Only the packages actually needed being taken from that repository is ensured through priority rules or package-name restrictions. **Keys are kept separate per repository.** Treating a single key as valid for all repositories turns compromising one key into compromising every repository. If only a few packages are needed from a repository, downloading the package files and verifying them against their checksums leaves a narrower trust surface than adding the whole repository. The cost is that updates do not arrive automatically. #### Summary - A repository definition consists of an address, release, component, architecture, and verification key. - Alongside packages, a repository serves an index file carrying package checksums, and this file is signed. - The chain of trust has three links: the key verifies the maintainer, the signature verifies the index, the checksums in the index verify the packages. - A key's fingerprint must be confirmed through a source independent of the channel it came from; otherwise the chain verifies itself. - A valid signature shows a file has not changed; who the key belongs to is a separate question, and the warning line reports that. - Transport encryption shows data did not change in transit; a signature verifies the content's origin; the two do not substitute for each other. #### Next Step The system has been surveyed, files organized, permissions established, and a path to installing software opened. One gap remains: how are the script and configuration files created in this course edited without leaving the terminal? The course's last lesson covers terminal text editors — modal and modeless designs — and shows which one to choose under which conditions. ### Terminal Text Editors Source: https://academia.sh/en/courses/introduction-to-linux/terminal-text-editors The design line from line editor to screen editor, comparing modal and modeless editors, basic editing commands, and declaring an editor choice through an environment variable. Files throughout the course were created with `printf` and redirection. That works for one-line files; it is not enough for changing a line in the middle of a script or adding two lines to a configuration file. On a server with no graphical interface — the case for most remote sessions — editing is done in the terminal. This lesson covers the two design families of terminal editors and the choice between them. #### The Line Editor Terminal editors' design comes from the era when printing terminals were used. There was no screen back then; output was printed on paper, and moving a cursor was not a thing. Editing was done with commands over line numbers. This design's representative, `ed`, is still defined in POSIX and still works. A complete session to add a line to the project script: ``` $ ed scripts/summary.sh 142 $ echo "raw measurement file: $(ls -1 "$HOME/project/data/raw" | wc -l)" a echo "current measurement: $(readlink "$HOME/project/data/current.csv")" . %p #!/bin/sh # Reports the count of measurement files in project/data/raw echo "raw measurement file: $(ls -1 "$HOME/project/data/raw" | wc -l)" echo "current measurement: $(readlink "$HOME/project/data/current.csv")" w 215 q ``` The session should be read line by line. The `142` printed at open is the file's byte count. The `$` command prints the last line. `a` switches to append mode; the following lines are added to the file, and a lone period ends append mode. `%p` prints the entire file. `w` writes and reports the new byte count, `q` quits. `ed` shows no prompt and prints only a question mark on error. This is inconvenient for everyday use, but matters for two reasons. First, its command language has been inherited in full by later editors. Second, `ed` can be driven from scripts through redirection, and this is sometimes the only option in rescue environments. #### The Modal Screen Editor Once screen terminals became widespread, a **visual mode** was added on top of the line editor: part of the file is shown on screen, and the cursor moves over it. This design was standardized in POSIX under the name `vi` and is expected to be present on every standard-conforming system. The editor's distinguishing feature is its **modes**. The same key does different things depending on the current mode: | Mode | Function | How to enter | |---|---|---| | Normal | Navigation and editing commands | `Esc` | | Insert | Typed characters go into the text | `i`, `a`, `o` | | Command line | Line editor commands | `:` | The design's rationale is that keys become a **language**. Because letter keys do not enter characters in normal mode, all of them can be used as commands; commands themselves are built by combining an action with a motion. `d` is the delete action, `w` is a word-forward motion; `dw` deletes a word. `3dw` deletes three words. Every learned motion can combine with every learned action. The approach it is contrasted with is binding commands to modifier-key combinations. In that approach the number of commands is limited by the number of combinations, and combinations cannot be composed. The modal design's second gain is that it uses only basic characters: it works on terminals that do not send arrow keys or modifier combinations. This is a practical advantage for remote sessions and rescue consoles. Its cost is that a command typed in insert mode gets mixed into the text. Mode confusion is the problem new users run into most, and its fix is fixed: when unsure, press `Esc` and return to normal mode. #### The Smallest Survival Set The commands needed to open, change, and save a file are few: | Command | Function | |---|---| | `i` | Enters insert mode before the cursor | | `a` | Enters insert mode after the cursor | | `o` | Opens a new line below and enters insert mode | | `Esc` | Returns to normal mode | | `h` `j` `k` `l` | Left, down, up, right | | `0` / `$` | Start of line / end of line | | `gg` / `G` | Start of file / end of file | | `x` | Deletes the character under the cursor | | `dd` | Deletes the current line | | `u` | Undoes the last change | | `/pattern` | Search forward; `n` for next match | | `:w` | Saves | | `:q` | Quits | | `:q!` | Quits, discarding changes | | `:wq` | Saves and quits | Commands beginning with a colon in the last four rows are the line editor's commands. The inheritance shows here: `:w` and `:q` are the same commands as `w` and `q` in an `ed` session. The same command language can also be run directly; the editor's line mode can be driven from scripts: ``` $ printf '%s/file/count/\nwq\n' | ex -s /tmp/edit.sh $ cat /tmp/edit.sh #!/bin/sh # Reports the count of measurement files in project/data/raw echo "raw measurement count: $(ls -1 "$HOME/project/data/raw" | wc -l)" echo "current measurement: $(readlink "$HOME/project/data/current.csv")" ``` The `%s/old/new/` command makes the substitution on every line; `wq` saves and quits. Typing a colon in the screen editor and giving the same command produces the same result. A screen-based session cannot be shown as text, but the command language can, and that is what is shown here. #### Modeless Editors The second design family uses no modes: typed characters go straight into the text, commands are given with control-key combinations, and the available commands are listed continuously at the bottom of the screen. The comparison runs along two axes: | Criterion | Modal | Modeless | |---|---|---| | Learning threshold | High; the mode concept must be learned | Low; commands are written on screen | | Command composition | Action and motion combine | One command per combination | | Guaranteed presence | Defined in POSIX, expected on every system | Separate package, may not be installed | | Limited terminal | Works with basic characters | Needs control combinations | The third row is decisive in practice. A modeless editor may not be present on a bare system or in a rescue environment: ``` $ command -v vi ed nano /usr/bin/vi /usr/bin/ed /usr/bin/nano ``` All three exist on this system; but only the first two can be assumed present. For this reason, the modal editor's survival set should be learned regardless of which editor is preferred. #### Declaring an Editor Preference Many programs launch an editor when editing is needed: scheduled-task definitions, version control tools' message screens, privileged file-editing tools. Which editor opens is read from environment variables. ``` $ echo "EDITOR=${EDITOR:-undefined}" EDITOR=undefined $ EDITOR=vi $ export EDITOR $ echo "EDITOR=$EDITOR" EDITOR=vi ``` An assignment made without `export` becomes only the shell's own variable and would not be passed to programs it runs; the environment distinction from the command path lesson applies directly here. For the setting to persist, it is written to the same place as the `PATH` change — the configuration file read at login. Two variables are used: `VISUAL` for a full-screen editor, `EDITOR` for line-based environments. If both are defined, most programs prefer `VISUAL`; if only one will be defined, `EDITOR` is enough. #### Summary - Terminal editors' command language is inherited from the line editor designed for screenless terminals, and it is still visible in commands beginning with `:`. - In the modal editor, letter keys can be used as commands, so action and motion combine into a command language. - The command set needed to open, change, and save a file is small, and pressing `Esc` returns to normal mode when unsure. - Modeless editors have a low learning threshold but cannot be assumed present on every system. - Editor preference is declared through the `VISUAL` and `EDITOR` environment variables and is not passed to run programs unless `export`ed. #### Course Wrap-Up The course progressed over a single directory tree. The tree was empty when set up; by the close it carries every operation performed on it: ``` $ scripts/summary.sh raw measurement file: 4 current measurement: raw/measurement-03.csv ``` The whole course stands behind these two lines. For the script to run, its execute bit had to be on; because the path contains a slash, the shell never consulted the search list. The files it counts were created, copied, and moved; the directory they were counted in was found through path resolution. The current measurement it reports is a symbolic link's target. The whole tree was archived, compressed, synced to a remote machine; opened to a group and its shared directory protected with special permission bits. The course's takeaways sum up in retrospect as follows: - **Navigation and file operations:** The hierarchy's logic, path resolution, creating, copying, moving, deleting, links, archiving, and transfer. - **Permissions and ownership:** Identity's numeric representation, the nine permission bits and octal notation, changing ownership, privilege escalation, and the three special bits. - **Package management:** A package's structure, dependency resolution, operation classes, and the repository's chain of trust. Alongside commands, a set of habits was built through the course, and these outlive the commands themselves: seeing a destructive command's expansion before running it, listing an archive before opening it, trying a sync with a dry run first, verifying a transfer against its checksum, and taking a privilege in the narrowest scope possible. The next course covers the layer this course deliberately left out: chaining commands together. The Shell Programming course builds input and output redirection, pipelines, variables, conditionals, and loops; commands run one at a time in this course turn there into scripts that can be run again. The word-splitting and expansion rules introduced in the shell lesson will form the foundation of that course's grammar.