Migrations and Seeding

Relationships rest on tables, and the tables have to come from somewhere. If the schema is built by hand, it comes out different in every environment; migrations exist to build the schema step by step and in a repeatable way. But if the same migration is run twice, is the table built twice?

The measurement axis of this lesson is whether the migration file ran in this run. The decision order is the classify() function itself: ran, deferred, skipped, not wired. The dominant outcome is skipped: the migration file was found and placed on the path, but it was not run because it is in the log.

VD16. The schema is kept in the file schema.json in the working directory as an array from table names to column lists; there is no real database. VD17. The migration log keeps the names of the migrations that have run, in order, in migration_log.json. VD18. Migration files run sorted by name under migrations/, and each returns the up and down functions. VD19. The seeder writes data only when the table is empty; the starting data is in data.json.

The measurement setup comes unchanged.

<?php
// ran.php — did an expected part run in this request?
declare(strict_types=1);

final class Ledger {
    public static array $path = [];        // parts placed on the request's path
    public static array $ran = [];         // parts called within the request
    public static array $deferred = [];    // parts left outside the request
    public static function newRequest(): void { self::$path = self::$ran = self::$deferred = []; }
    public static function onPath(string $part): void { self::$path[] = $part; }
    public static function ran(string $part): void { self::$ran[] = $part; }
    public static function defer(string $part): void { self::$deferred[] = $part; }
}

function classify(string $part): string {
    return match (true) {
        in_array($part, Ledger::$ran, true) => 'ran',
        in_array($part, Ledger::$deferred, true) => 'deferred',
        in_array($part, Ledger::$path, true) => 'skipped',
        default => 'not wired',
    };
}

The Migration Runner

A migration is a file that moves the schema from one version to another. The up function applies the change, and the down function reverses it. The migrator goes through the migration files in order, does not run the ones in the log, and runs the ones not in the log and adds them to the log. A rollback calls the down function of the last migration added to the log and deletes the log entry. Seeding writes starting data on top of the schema. Adding a column in a migration file goes through addColumn(), and each addition records itself under the name column:<table>.<column>.

<?php
// migration.php — a migration runner, a migration log and a seeder
declare(strict_types=1);
require_once 'ran.php';

function readJson(string $file): array { return is_file($file) ? json_decode(file_get_contents($file), true) : []; }
function writeJson(string $file, array $data): void { file_put_contents($file, json_encode($data)); }

function addColumn(array $schema, string $table, string $column): array {
    Ledger::ran("column:$table.$column");
    $schema[$table][] = $column;
    return $schema;
}
function dropColumn(array $schema, string $table, string $column): array {
    $schema[$table] = array_values(array_diff($schema[$table], [$column]));
    return $schema;
}

final class Migrator {
    public function up(): void {
        $log = readJson('migration_log.json');
        $schema = readJson('schema.json');
        $files = glob('migrations/*.php');
        sort($files);
        foreach ($files as $file) {
            $name = basename($file, '.php');
            Ledger::onPath("migration:$name");
            if (in_array($name, $log, true)) continue;
            Ledger::ran("migration:$name");
            $migration = require $file;
            $schema = $migration['up']($schema);
            $log[] = $name;
        }
        writeJson('schema.json', $schema);
        writeJson('migration_log.json', $log);
    }

    public function rollBack(): void {
        $log = readJson('migration_log.json');
        $schema = readJson('schema.json');
        $name = array_pop($log);
        $migration = require "migrations/$name.php";
        Ledger::ran("down:$name");
        writeJson('schema.json', $migration['down']($schema));
        writeJson('migration_log.json', $log);
    }
}

final class Seeder {
    public function run(): void {
        Ledger::onPath('seed:posts');
        $data = readJson('data.json');
        if (!empty($data['posts'])) return;
        Ledger::ran('seed:posts');
        $data['posts'] = [['id' => 1, 'title' => 'First Post'], ['id' => 2, 'title' => 'Second Post']];
        writeJson('data.json', $data);
    }
}

function resetState(): void {
    foreach (['schema.json', 'migration_log.json', 'data.json'] as $file) if (is_file($file)) unlink($file);
    if (!is_dir('migrations')) mkdir('migrations');
    foreach (glob('migrations/*.php') as $file) unlink($file);
}
function report(array $parts): void {
    foreach ($parts as $part) printf("  %-24s %s\n", $part, classify($part));
}

Three migration files are written by an author script. The first builds the posts table, the second adds the slug column, and the third builds the tags table; the third is added later in the opposing measurement.

<?php
// migrations.php — writes the example migration files
function writeMigration(string $name): void {
    $source = match ($name) {
        '001_posts' => <<<'MIGRATION'
<?php
return [
    'up' => function (array $schema): array {
        $schema['posts'] = [];
        return addColumn(addColumn($schema, 'posts', 'id'), 'posts', 'title');
    },
    'down' => fn(array $schema): array => array_diff_key($schema, ['posts' => 1]),
];
MIGRATION,
        '002_slug' => <<<'MIGRATION'
<?php
return [
    'up' => fn(array $schema): array => addColumn($schema, 'posts', 'slug'),
    'down' => fn(array $schema): array => dropColumn($schema, 'posts', 'slug'),
];
MIGRATION,
        '003_tags' => <<<'MIGRATION'
<?php
return [
    'up' => function (array $schema): array {
        $schema['tags'] = [];
        return addColumn(addColumn($schema, 'tags', 'id'), 'tags', 'name');
    },
    'down' => fn(array $schema): array => array_diff_key($schema, ['tags' => 1]),
];
MIGRATION,
    };
    file_put_contents("migrations/$name.php", $source);
}

Skipping a Recorded Migration

The measurement runs the two migrations twice, and calls the seeder after each run. Expected: in the first run the two migrations and the seeding are ran, in the second all three are skipped; the schema does not change in the second run.

<?php
// observe.php — on the second run the recorded migrations are skipped
require_once 'migration.php';
require_once 'migrations.php';

resetState();
writeMigration('001_posts');
writeMigration('002_slug');
foreach ([1, 2] as $n) {
    Ledger::newRequest();
    (new Migrator())->up();
    (new Seeder())->run();
    echo "run $n\n";
    report(['migration:001_posts', 'migration:002_slug', 'seed:posts']);
}
echo "schema: " . json_encode(readJson('schema.json')) . "\n";

run 1
  migration:001_posts      ran
  migration:002_slug       ran
  seed:posts               ran
run 2
  migration:001_posts      skipped
  migration:002_slug       skipped
  seed:posts               skipped
schema: {"posts":["id","title","slug"]}

In the second run the migration files were found and placed on the path, and they were not called because they are in the log: skipped. This is not an error; it is the mechanism that makes a migration resistant to repetition. The seeder was likewise skipped because the table was full; otherwise two posts would have become four.

The Opposing Measurement: A New Migration and a Rollback

A fourth migration is added to a schema on which three migrations have run. Expected: the new migration is ran, and the earlier ones are skipped. Then the last migration is rolled back and run again; because the rolled-back migration has left the log, it should be ran again.

<?php
// new_migration.php — a new migration runs, a rolled-back migration runs again
require_once 'migration.php';
require_once 'migrations.php';

resetState();
writeMigration('001_posts');
writeMigration('002_slug');
(new Migrator())->up();

writeMigration('003_tags');
Ledger::newRequest();
(new Migrator())->up();
echo "new migration added\n";
report(['migration:001_posts', 'migration:002_slug', 'migration:003_tags']);

Ledger::newRequest();
(new Migrator())->rollBack();
echo "rolled back: " . json_encode(array_keys(readJson('schema.json'))) . "\n";
report(['down:003_tags']);

Ledger::newRequest();
(new Migrator())->up();
echo "ran again: " . json_encode(array_keys(readJson('schema.json'))) . "\n";
report(['migration:002_slug', 'migration:003_tags']);

new migration added
  migration:001_posts      skipped
  migration:002_slug       skipped
  migration:003_tags       ran
rolled back: ["posts"]
  down:003_tags            ran
ran again: ["posts","tags"]
  migration:002_slug       skipped
  migration:003_tags       ran

The migration runner found the new file on its own; nobody called it by name. The rollback reversed the log and the schema together, which is why the migration ran again. The distinction between what is skipped and what runs is in the log, not in the file.

The Bounding Measurement

If the file of a migration is edited after it has run, does the runner notice? The measurement changes the 002_slug file while the migration is in its already-run state, and adds a second column, summary. The runner is run again. Expected: the migration is skipped, because it is in the log; the edit is not wired, because it left no run record. summary will not be in the schema. The way to correct this is to make the change with a new migration file; this is tried at the end of the measurement.

<?php
// edit.php — editing a migration file that has run is not wired
require_once 'migration.php';
require_once 'migrations.php';

resetState();
writeMigration('001_posts');
writeMigration('002_slug');
(new Migrator())->up();

file_put_contents('migrations/002_slug.php', <<<'MIGRATION'
<?php
return [
    'up' => fn(array $schema): array => addColumn(addColumn($schema, 'posts', 'slug'), 'posts', 'summary'),
    'down' => fn(array $schema): array => dropColumn(dropColumn($schema, 'posts', 'slug'), 'posts', 'summary'),
];
MIGRATION);
Ledger::newRequest();
(new Migrator())->up();
echo "002 edited\n";
report(['migration:002_slug', 'column:posts.summary']);
echo "schema: " . json_encode(readJson('schema.json')) . "\n";

file_put_contents('migrations/004_summary.php', <<<'MIGRATION'
<?php
return [
    'up' => fn(array $schema): array => addColumn($schema, 'posts', 'summary'),
    'down' => fn(array $schema): array => dropColumn($schema, 'posts', 'summary'),
];
MIGRATION);
Ledger::newRequest();
(new Migrator())->up();
echo "004 added\n";
report(['migration:002_slug', 'migration:004_summary', 'column:posts.summary']);
echo "schema: " . json_encode(readJson('schema.json')) . "\n";

002 edited
  migration:002_slug       skipped
  column:posts.summary     not wired
schema: {"posts":["id","title","slug"]}
004 added
  migration:002_slug       skipped
  migration:004_summary    ran
  column:posts.summary     ran
schema: {"posts":["id","title","slug","summary"]}

The edited file is skipped: it was found, but it was not called because it is in the log. The summary column written into the file is not wired; it was neither placed on the path nor called, and it is not in the schema. That no error is given is the danger of this outcome: the developer thinks the column was added, and the environment stays on a different schema. A new migration file made the same column ran and brought it to the schema. For this reason a migration that has run is not changed; a schema change is always made with a new migration.

Summary

• A migration versions the schema; the runner runs the migrations not in the log and updates the log.
• This lesson’s outcome is skipped: the recorded migration was found and placed on the path on the second run, but it was not called; the schema did not change.
• A newly added migration is ran; a rolled-back migration is ran again because it left the log.
• An edit in a migration file that has run is not wired: the file was skipped, the added column did not reach the schema, and no error occurred.
• The seeder is also skipped when the table is full; a schema change is always made with a new migration.

Next Step

The schema and the data are ready. When a new column is added to a model, does that column reach the API response on its own? The next lesson, Pagination and Resource Transformation, measures pagination and resource transformation, and shows that a column that is in the model but not in the response definition is not wired.