Static Analysis

The Web Application Framework course held the code and the configuration fixed and measured whether a part ran in the request. In this course the code and the request are fixed; what changes is the place where the code runs. The same script can show one behavior on a developer’s machine and another in production. Most of the time this difference is not a defect but a design: PHP ships two separate configuration files, and the two differ on purpose. Every lesson in the course asks a single question: does this behavior reach production?

The first lesson opens with tools that read code without running it. Static analysis examines a program’s source for defects without executing it. For that reason it sits in a place of its own on the axis: defects that never appear inside the run can only be revealed by a tool outside the run.

Two Configuration Files, Seven Differences

The PHP source distribution contains two sample configuration files: php.ini-development and php.ini-production. Both carry most of the same directives with the same values. They differ on purpose in seven directives: error display (display_errors), display of startup errors (display_startup_errors), error level (error_reporting), assertions (zend.assertions), arguments in the exception trace (zend.exception_ignore_args), the length of an argument string (zend.exception_string_param_max_len), and memory statistics (mysqlnd.collect_memory_statistics). The production profile adds the code cache on top of these.

This difference is not a mistake. Production does not print error messages into the response, does not compile assertions, does not put arguments into the exception trace, and removes deprecation notices from the error level. The lesson does not try to close this gap; it names the side on which each behavior stands. In the DevOps and Platform Engineering curriculum every deviation between development and production was treated as a hidden defect and an attempt was made to close it. Here most of the difference is intentional and is shipped by PHP itself.

The Measurement Core

The course measures every behavior with the same two files. The first is side.php, and it stays exactly the same in every lesson of the course.

<?php
// side.php — does this behavior reach production?
declare(strict_types=1);

// The two configuration files PHP ships (php.ini-development, php.ini-production) differ
// in seven directives; all seven are copied here verbatim. The last two production lines
// are comments in the shipped file: code cache on for the command line too, file checks off.
const PROFILES = [
    'development' => [
        'display_errors' => '1',
        'display_startup_errors' => '1',
        'error_reporting' => '32767',
        'mysqlnd.collect_memory_statistics' => '1',
        'zend.assertions' => '1',
        'zend.exception_ignore_args' => '0',
        'zend.exception_string_param_max_len' => '15',
    ],
    'production' => [
        'display_errors' => '0',
        'display_startup_errors' => '0',
        'error_reporting' => '24575',
        'mysqlnd.collect_memory_statistics' => '0',
        'zend.assertions' => '-1',
        'zend.exception_ignore_args' => '1',
        'zend.exception_string_param_max_len' => '0',
        'opcache.enable_cli' => '1',
        'opcache.validate_timestamps' => '0',
    ],
];

/** Runs the script in a separate process with no configuration file, only the profile's directives. */
function run(string $script, array $directives): bool {
    $command = ['php', '-n'];
    foreach ($directives as $name => $value) {
        array_push($command, '-d', "$name=$value");
    }
    $command[] = $script;
    $process = proc_open($command, [1 => ['pipe', 'w'], 2 => ['file', '/dev/null', 'w']], $pipes);
    $output = stream_get_contents($pipes[1]);
    fclose($pipes[1]);
    proc_close($process);
    $lines = array_values(array_filter(array_map('trim', explode("\n", $output))));
    return end($lines) === 'yes';
}

/** Derives the side from whether the behavior was observed in the two runs. */
function side(bool $inDevelopment, bool $inProduction): string {
    return match (true) {
        $inDevelopment && $inProduction => 'both sides',
        $inDevelopment => 'development only',
        $inProduction => 'production only',
        default => 'not in the run',
    };
}

function sideReport(string $name, string $script): string {
    $inDevelopment = run($script, PROFILES['development']);
    $inProduction = run($script, PROFILES['production']);
    $finalSide = side($inDevelopment, $inProduction);
    echo "$name → $finalSide\n    development ", $inDevelopment ? 'yes' : 'no',
        ' · production ', $inProduction ? 'yes' : 'no', "\n";
    return $finalSide;
}

The file has four parts. The PROFILES constant carries the seven differences of the two configuration files verbatim. The last two production lines are comments in the shipped file; the course sets them up explicitly, because the code cache is off by default on the command line. run() runs a probe script in a separate process and checks whether the last line is yes. side() derives one of the four sides from two boolean values. sideReport() prints this as one line and one detail line.

Two rules are born in this file. The first is the -n flag: the child process never loads the machine’s configuration file and runs only with the profile’s directives. This way the measurement does not depend on which configuration file happens to be on that machine. The second is that the child process is started by the name 'php', through PATH. PHP_BINARY is not used, because the path of the machine’s binary cannot be trusted: that path may point to a place that does not sit next to the libraries the binary depends on, and the process may not start at all.

The four sides do not change throughout the course: if a behavior is observed in both profiles, it is both sides; if it is observed only in development, development only; if it is observed only in production, production only; if it is observed in neither, not in the run. The side is given as a rule, not as a label. Each probe prints yes when it observes the behavior and no when it does not. run() runs in both profiles, and side() derives the name from the two results.

Twelve Behaviors, Four Sides

The second file, behaviors.php, passes twelve small probes through this core. The probe texts sit inside nowdoc strings, so variables and escapes do not interfere.

<?php
// behaviors.php — twelve behaviors, four sides
declare(strict_types=1);
require 'side.php';

// Each probe prints "yes" on its last line if the behavior was observed, "no" if not.
$behaviors = [
    'a request that exceeds the memory limit stops' => <<<'PHP'
        <?php
        ini_set('memory_limit', '32M');
        register_shutdown_function(function () {
            $last = error_get_last();
            echo ($last['type'] ?? 0) === E_ERROR ? "yes\n" : "no\n";
        });
        $big = str_repeat('x', 64 * 1024 * 1024);
        PHP,
    'the destructor runs when the request ends' => <<<'PHP'
        <?php
        $connection = new class {
            public function __destruct() { echo "yes\n"; }
        };
        PHP,
    'a warning reaches the log' => <<<'PHP'
        <?php
        $log = __DIR__ . '/log.txt';
        @unlink($log);
        ini_set('log_errors', '1');
        ini_set('error_log', $log);
        ob_start();
        echo $undefined;
        ob_end_clean();
        echo str_contains((string) @file_get_contents($log), 'Warning') ? "yes\n" : "no\n";
        PHP,
    'a wrong-typed argument stops execution' => <<<'PHP'
        <?php
        declare(strict_types=1);
        function quantity(int $n): int { return $n; }
        try { quantity('3'); echo "no\n"; } catch (TypeError) { echo "yes\n"; }
        PHP,
    'a failing assertion stops the request' => <<<'PHP'
        <?php
        try { assert(1 === 2); echo "no\n"; } catch (AssertionError) { echo "yes\n"; }
        PHP,
    'a warning is printed into the response' => <<<'PHP'
        <?php
        ob_start();
        echo $undefined;
        $response = ob_get_clean();
        echo str_contains($response, 'Warning') ? "yes\n" : "no\n";
        PHP,
    'a deprecated call reaches the log' => <<<'PHP'
        <?php
        $log = __DIR__ . '/log.txt';
        @unlink($log);
        ini_set('log_errors', '1');
        ini_set('error_log', $log);
        ob_start();
        strlen(null);
        ob_end_clean();
        echo str_contains((string) @file_get_contents($log), 'Deprecated') ? "yes\n" : "no\n";
        PHP,
    'the exception trace carries the argument value' => <<<'PHP'
        <?php
        function pay(string $card) { throw new RuntimeException('declined'); }
        try { pay('card-0042'); } catch (RuntimeException $e) {
            echo count($e->getTrace()[0]['args'] ?? []) > 0 ? "yes\n" : "no\n";
        }
        PHP,
    'the old version of a file changed in place runs' => <<<'PHP'
        <?php
        $version = __DIR__ . '/version.php';
        file_put_contents($version, "<?php return 1;\n");
        touch($version, time() - 60);
        $before = include $version;
        file_put_contents($version, "<?php return 2;\n");
        touch($version, time() - 30);
        $after = include $version;
        echo $after === $before ? "yes\n" : "no\n";
        PHP,
    'the old version runs after the symbolic link switch' => <<<'PHP'
        <?php
        foreach ([1, 2] as $n) {
            @mkdir("release-$n");
            file_put_contents("release-$n/hello.php", "<?php return $n;\n");
            touch("release-$n/hello.php", time() - 60);
        }
        @unlink('current');
        symlink('release-1', 'current');
        $before = include 'current/hello.php';
        @unlink('current-new');
        symlink('release-2', 'current-new');
        rename('current-new', 'current');
        $after = include 'current/hello.php';
        echo $after === $before ? "yes\n" : "no\n";
        PHP,
    'an undefined function in an uncalled branch is reported' => <<<'PHP'
        <?php
        function report(bool $summary): string { return $summary ? summarize() : 'full'; }
        report(false);
        echo error_get_last() !== null ? "yes\n" : "no\n";
        PHP,
    'unformatted code is reported' => <<<'PHP'
        <?php
        file_put_contents(__DIR__ . '/unformatted.php', "<?php\nfunction  add(\$a,\$b){return \$a+\$b;}   \n");
        include __DIR__ . '/unformatted.php';
        add(1, 2);
        echo error_get_last() !== null ? "yes\n" : "no\n";
        PHP,
];

$tally = array_fill_keys(['both sides', 'development only', 'production only', 'not in the run'], 0);
foreach ($behaviors as $name => $code) {
    file_put_contents('probe.php', $code);
    $tally[sideReport($name, 'probe.php')]++;
}
echo "\n";
foreach ($tally as $finalSide => $n) {
    echo "$finalSide: $n\n";
}

The full output is:

a request that exceeds the memory limit stops → both sides
    development yes · production yes
the destructor runs when the request ends → both sides
    development yes · production yes
a warning reaches the log → both sides
    development yes · production yes
a wrong-typed argument stops execution → both sides
    development yes · production yes
a failing assertion stops the request → development only
    development yes · production no
a warning is printed into the response → development only
    development yes · production no
a deprecated call reaches the log → development only
    development yes · production no
the exception trace carries the argument value → development only
    development yes · production no
the old version of a file changed in place runs → production only
    development no · production yes
the old version runs after the symbolic link switch → production only
    development no · production yes
an undefined function in an uncalled branch is reported → not in the run
    development no · production no
unformatted code is reported → not in the run
    development no · production no

both sides: 4
development only: 4
production only: 2
not in the run: 2

The numbers are binding, and none of the ten lessons of the course changes them: of the twelve behaviors, four are on both sides, four are in development only, two are in production only, and two are not in the run. The four groups read as follows.

The four behaviors on both sides (the memory limit, the destructor running at the end of the request, the open log, and the TypeError under strict_types) are independent of the configuration file. Because printing to the screen is off in production, the main way to see an error is the log, and the log lands in the same place on both sides.

The four behaviors in development only are the places where production deliberately stays silent: a failing assertion, a warning printed into the response, a deprecation notice reaching the log, and an exception trace that carries arguments. In production, zend.assertions=-1 does not compile the assertion at all.

The two behaviors in production only are behaviors the developer never sees: the code cache compiles a file once and never looks at the disk again, so both an overwritten file and a switched symbolic link run the old version.

The two behaviors that are not in the run are defects the run cannot catch. The rest of the lesson examines the first of them.

The Defect the Run Cannot Find

The eleventh of the twelve probes is the undefined function in an uncalled branch. The excerpt is as follows.

<?php
// report.php — a function that calls an undefined function in one branch
declare(strict_types=1);

function report(bool $summary): string {
    return $summary ? summarize() : 'full';
}

summarize() is not defined anywhere, but when report(false) is called, that branch is never entered. PHP resolves a function name at call time; for the uncalled branch it produces no notice at compile time or at run time. This behavior is the same in both profiles, and error_get_last() stays null. The question of the Web Application Framework course was whether a part was called in this request. Here the part is always called; what is asked is whether the behavior of the called part is the same in the two profiles. For this defect the answer is the same in both profiles: silence.

PHP’s own check, php -l, cannot see this defect either. The command checks only the syntax; that is, it looks at whether the file can be parsed as PHP, and it resolves no function.

# check.sh — php -l looks at syntax only
printf '<?php\nfunction report( {\n' > broken.php
php -n -l broken.php >/dev/null 2>&1
echo "broken file: $?"
printf '<?php\nfunction report() { return 1; }\n' > valid.php
php -n -l valid.php >/dev/null 2>&1
echo "valid file: $?"
php -n -l report.php >/dev/null 2>&1
echo "file with an undefined function: $?"

broken file: 255
valid file: 0
file with an undefined function: 0

The broken file gives 255. The valid file and the file that calls an undefined function give the same code, 0. The lesson prints only the exit code.

The tokenizer extension goes one step further. PhpToken::tokenize() splits the source into tokens: keywords, names, variables, operators. The checker below compares the function names that are defined with the names that are called. A name is defined if it comes right after the function keyword in the source. A name is called if it is followed by ( and is not preceded by ->, ::, or new. A call counts as undefined if it is neither defined in the source nor passes PHP’s own function_exists() check.

<?php
// check.php — a small checker that compares called names with defined names
declare(strict_types=1);

/** Splits the source into tokens; discards whitespace and comments. */
function tokens(string $source): array {
    $list = PhpToken::tokenize($source);
    return array_values(array_filter($list, fn(PhpToken $b) => !$b->isIgnorable()));
}

/** Collects the function names that are defined and called. */
function collectNames(string $source): array {
    $b = tokens($source);
    $defined = [];
    $called = [];
    foreach ($b as $i => $token) {
        if (!$token->is(T_STRING)) {
            continue;
        }
        $previous = $b[$i - 1] ?? null;
        $next = $b[$i + 1] ?? null;
        if ($previous !== null && $previous->is(T_FUNCTION)) {
            $defined[] = $token->text;
        } elseif ($next !== null && $next->text === '('
            && ($previous === null || !$previous->is([T_OBJECT_OPERATOR, T_DOUBLE_COLON, T_NEW]))) {
            $called[] = $token->text;
        }
    }
    return ['defined' => $defined, 'called' => $called];
}

/** Returns the names of the calls whose definition cannot be found. */
function undefinedCalls(string $source): array {
    $names = collectNames($source);
    $missing = [];
    foreach (array_unique($names['called']) as $name) {
        if (!in_array($name, $names['defined'], true) && !function_exists($name)) {
            $missing[] = $name;
        }
    }
    return $missing;
}

The driver first measures with a probe that the defect stays silent in both profiles, then applies the checker to the same file, and finally actually runs the branch.

<?php
// analysis.php — run or read
declare(strict_types=1);
require 'side.php';
require 'check.php';

file_put_contents('probe.php', <<<'PHP'
    <?php
    include 'report.php';
    report(false);
    echo error_get_last() !== null ? "yes\n" : "no\n";
    PHP);
sideReport('an undefined function in an uncalled branch is reported', 'probe.php');

echo 'checker found: ', implode(', ', undefinedCalls(file_get_contents('report.php'))), "\n";

include 'report.php';
try {
    report(true);
} catch (Error $e) {
    echo 'when the branch is entered: ', get_class($e), "\n";
}

an undefined function in an uncalled branch is reported → not in the run
    development no · production no
checker found: summarize
when the branch is entered: Error

The run is silent in both profiles: the side is not in the run. The checker finds the name summarize. When the branch is entered, PHP throws an Error; that is, the defect is real, and only the run hides it until that request.

The Bounding Measurement

Static analysis sees what the run cannot see, but it has a limit too: the name must be readable in the source. The code below calls the same undefined function, but takes the name from a variable.

<?php
// limit.php — does the checker see a call whose name is carried in a variable
declare(strict_types=1);
require 'check.php';

file_put_contents('dynamic.php', <<<'PHP'
    <?php
    $name = 'summarize';
    $name();
    PHP);

echo 'undefined calls the checker found: ', count(undefinedCalls(file_get_contents('dynamic.php'))), "\n";
try {
    include 'dynamic.php';
} catch (Error $e) {
    echo 'direct run: ', get_class($e), "\n";
}

undefined calls the checker found: 0
direct run: Error

The checker finds nothing, while the direct run throws an Error. At the token level, the name of the call is not even the word summarize; it is a string constant (T_CONSTANT_ENCAPSED_STRING), and $name is a variable. At this point every tool that reads the source either needs an inference layer that tries to resolve the name or has to accept its limit. What the measurement teaches is that the sentence “the checker gave clean output” does not mean “there is no defect”: it means only “this tool, with this way of reading, saw no defect”.

Summary

• Static analysis reads code without running it; for that reason its side on the measurement axis is not in the run.
• PHP’s development and production configuration files differ on purpose in seven directives: error display, display of startup errors, error level, assertions, arguments in the exception trace, argument string length, and memory statistics.
• The measuring question is a single one, does this behavior reach production; the four answers (both sides, development only, production only, not in the run) do not change throughout the course, and the distribution of the twelve behaviors is 4, 4, 2, 2.
• php -n -l checks only the syntax: the broken file gives 255, while the valid file and the file with an undefined function give 0.
• A small checker that reads tokens compares called names with defined names and finds the name summarize, but it cannot see a name carried as a string.

Next Step

In this lesson a tool that reads the code showed the defect the run could not see. The second behavior of the same class is not in the run either: unformatted code runs without trouble in both profiles. The next lesson asks whether the format of the code changes the run, and uses the same token reading for whitespace, line endings, and indentation.