What Is a Database

The Data Structures course established how data is organized in memory and how that organization determines processing cost: arrays, hash tables, binary search trees. Introduction to Linux and Shell Programming taught filtering, cutting, and counting that same data while it sat on disk in a text file. The Node.js Runtime course read a file, parsed it, and used it inside a program. All three courses shared one assumption: data is either in memory or in a single file, and exactly one program reaches it.

This curriculum removes that assumption. Its question is: what problems appear once data is kept in a file, and which of these does a database solve? A single domain runs through the whole course — a library’s loan records: books, members, branches, and loan transactions. However well a library catalog’s interface is designed, the interface answers wrongly if the data behind it is modeled wrongly.

The Record in the File

Suppose the library keeps its loan transactions in a comma-separated text file. Shell tools are enough to work with that file:

cat > loan.csv <<'CSV'
loan_no,member_name,member_email,book_title,book_author,branch,pickup_date,return_date
1001,Alice Kane,alice.kane@example.test,Lost Time,Elena Marsh,Central,2025-03-02,2025-03-16
1002,Marcus Reyes,marcus.reyes@example.test,Lost Time,Elena Marsh,Central,2025-03-04,
1003,Alice Kane,alice.kane@example.test,Sea Lighthouses,Kevin Ashford,Bahcelievler,2025-03-05,2025-03-19
1004,Alice Kane,alice.kane@example.test,Lost Time,Elena Marsh,central,2025-03-20,
1005,Sylvia Renner,sylvia.renner@example.test,Sea Lighthouses,Kevin Ashford,Central,2025-03-21,
1006,Marcus Reyes,marcus.reyes@example.test,Silent Garden,Elena Marsh,Bahcelievler,2025-03-22,2025-04-01
CSV

awk -F, 'NR > 1 && $8 == "" { print $2, $4 }' loan.csv

Marcus Reyes Lost Time
Alice Kane Lost Time
Sylvia Renner Sea Lighthouses

The list of unreturned books came out. While the file is small, this is enough. Problems begin not when the file grows, but when the question changes and a second program starts reaching the file.

Writing the Question

Consider the question “how many times was each book loaned, and by how many distinct members?” With shell tools this means several passes, a sort, and a counter held by hand. Once the same data sits in a database, the question is written without saying how the answer is computed:

cat > loan.csv <<'CSV'
loan_no,member_name,member_email,book_title,book_author,branch,pickup_date,return_date
1001,Alice Kane,alice.kane@example.test,Lost Time,Elena Marsh,Central,2025-03-02,2025-03-16
1002,Marcus Reyes,marcus.reyes@example.test,Lost Time,Elena Marsh,Central,2025-03-04,
1003,Alice Kane,alice.kane@example.test,Sea Lighthouses,Kevin Ashford,Bahcelievler,2025-03-05,2025-03-19
1004,Alice Kane,alice.kane@example.test,Lost Time,Elena Marsh,central,2025-03-20,
1005,Sylvia Renner,sylvia.renner@example.test,Sea Lighthouses,Kevin Ashford,Central,2025-03-21,
1006,Marcus Reyes,marcus.reyes@example.test,Silent Garden,Elena Marsh,Bahcelievler,2025-03-22,2025-04-01
CSV

sqlite3 :memory: <<'SQL'
.mode csv
.import loan.csv loan_raw
.headers on
.mode box
SELECT book_title,
       COUNT(*)                     AS loan_count,
       COUNT(DISTINCT member_email) AS member_count
FROM loan_raw
GROUP BY book_title
ORDER BY loan_count DESC;
SQL

┌─────────────────┬────────────┬──────────────┐
│   book_title    │ loan_count │ member_count │
├─────────────────┼────────────┼──────────────┤
│ Lost Time       │ 3          │ 2            │
│ Sea Lighthouses │ 2          │ 2            │
│ Silent Garden   │ 1          │ 1            │
└─────────────────┴────────────┴──────────────┘

The difference is not one of syntax alone. The shell solution writes how the answer is computed; the second writes what is wanted. This is called declarative querying: the desired result is defined, and the system decides in what order and by what method the result is reached. The same question is written the same way whether the table holds a thousand rows or ten million; what changes is the execution plan the system chooses.

Two Writers at Once

The real break appears when two programs write to the file at the same time. Lending a copy takes three steps: read the state, confirm the copy is free, add the record. What happens when two counters hand out the same copy at once?

cat > checkout.mjs <<'JS'
import { readFile, writeFile } from "node:fs/promises";
import { setTimeout as delay } from "node:timers/promises";

const file = "state.json";
const member = process.argv[2];

const text = await readFile(file, "utf8");
const state = JSON.parse(text);

if (state.on_loan.includes(7)) {
  console.log(`${member}: copy 7 already on loan`);
} else {
  await delay(50);                       // stand-in for disk and network latency
  state.on_loan.push(7);
  state.records.push({ copy_no: 7, member });
  await writeFile(file, JSON.stringify(state));
  console.log(`${member}: copy 7 given out`);
}
JS

echo '{"on_loan": [], "records": []}' > state.json
node checkout.mjs alice & node checkout.mjs marcus & wait
cat state.json

marcus: copy 7 given out
alice: copy 7 given out
{"on_loan":[7],"records":[{"copy_no":7,"member":"marcus"}]}

The order of the two lines and which record remains in the file change from run to run; two things stay fixed. First: both processes report that the operation succeeded, meaning one copy was given to two members. Second: exactly one record remains in the file — the one that wrote last has also erased the record the earlier write had added. This is called a lost update.

The wait between the read and the write was inserted to make the delay deterministic. In a real system that delay comes from disk access, a network round trip, or the operating system switching processes; removing the wait does not remove the problem, it only makes it appear more rarely.

The Constraint That Was Never Written

There are other things the file does not say. Record 1004 writes the branch name as central, while the others write Central; shell tools count these as two separate branches. The same member name repeats across three rows — if the member’s email address changes, all three rows need correcting, and forgetting one leaves the file contradicting itself. No row carries the rule “a copy can be on loan to only one member at a time”; that rule lives only in the minds of the programs that write to the file.

In a database management system, the rule is stored together with the data, and the engine rejects any violation:

sqlite3 :memory: <<'SQL'
CREATE TABLE loan (
  loan_no     INTEGER PRIMARY KEY,
  copy_no     INTEGER NOT NULL,
  member_no   INTEGER NOT NULL,
  pickup_date TEXT NOT NULL,
  return_date TEXT
);
CREATE UNIQUE INDEX loan_open ON loan (copy_no) WHERE return_date IS NULL;
INSERT INTO loan VALUES (1001, 7, 41, '2025-03-02', NULL);
INSERT INTO loan VALUES (1002, 7, 52, '2025-03-04', NULL);
SQL

Runtime error near line 10: UNIQUE constraint failed: loan.copy_no (19)

The shape of the error message varies by engine; what does not vary is that the second insert never takes effect. The rule is now enforced not by application code, but by the system that stores the data itself. This distinction is the axis of the course: the difference between expressing a rule in the schema and repeating it in every program that writes.

What a Database Management System Provides

A database management system is software that stores data and governs access to it. The guarantees it provides fall under four headings:

• Declarative access. The query states what is wanted; the system chooses the access path, index use, and join order.
• Integrity. Constraints written into the schema are checked on every write attempt, regardless of which program is writing.
• Concurrency control. Transactions running at the same time do not see each other’s half-finished work; the lost update is the engine’s responsibility to prevent.
• Durability. Work reported as complete stays in the data even if the system crashes at that instant; work left unfinished is rolled back as if it never happened.

The last two guarantees together give the concept of a transaction, defined in the course’s final topic. Integrity is the subject of the lessons immediately ahead.

Where a File Is the Right Choice

A database is not the answer to every problem. Configuration files, logs with a single writer, raw data downloaded for a one-time analysis, text that belongs under version control — these are right to leave in a file. The distinguishing measure is not size but three questions: are there multiple simultaneous writers to the data, are there rules that must be enforced on the data, and is a half-written record acceptable? If the answer to all three is “no,” a file is enough.

Summary

• File-based storage falls short not when data grows, but when the question diversifies and a second writer appears.
• Declarative querying separates the desired result from the method of computing it; the execution decision belongs to the system.
• Two processes performing a concurrent read–modify–write on the same file produce a lost update, and both report success.
• A constraint written into the schema applies to every program that reaches the data; a check repeated in application code applies only to the program that wrote it.
• A database management system’s four guarantees are declarative access, integrity, concurrency control, and durability.

Next Step

This lesson showed that rules need to be stored together with the data, but it did not define what the rules are written on. The rows in loan.csv and the rows in a database differ by more than comma usage: under one of them stands a mathematical structure, under the other only a formatting convention. The next lesson defines that structure — the concepts of relation, row, column, and domain — and shows why a table is not an ordered list.