Data Formats

The number of queries and their text were in our hands. But an application does not talk only to its own database; it also receives data from a partner outside, and this data arrives as JSON, XML or CSV. The partner decides the format. The question is this: what happens if the partner’s data changes format?

The measuring axis of this lesson is again the owner of the result. The decision order does not change: client, third party, environment; if none of them can change the result, the server. The dominant party is the third party: the partner the application connects to is the only party outside the code that it connects to. The text from the partner is not fetched over a network; in this lesson it is written into the code as a fixed string and no network call is made. The core file is given verbatim.

<?php
// party.php — who can change this result?
declare(strict_types=1);

// The parties outside the code, in decision order: every request, every external call, every deployment.
const EXTERNAL_PARTIES = ['client', 'third party', 'environment'];

/** Computes the result once without deviation, then once with each external party's deviation. */
function probe(callable $result, array $deviations): array {
    $baseline = $result([]);
    $changed = [];
    foreach (EXTERNAL_PARTIES as $party) {
        if (array_key_exists($party, $deviations)) {
            $changed[$party] = $result([$party => $deviations[$party]]) !== $baseline;
        }
    }
    return $changed;
}

/** The first external party that can change the result; the server if none can. */
function party(array $changed): string {
    foreach (EXTERNAL_PARTIES as $p) {
        if ($changed[$p] ?? false) return $p;
    }
    return 'server';
}

function partyReport(string $name, callable $result, array $deviations): string {
    $changed = probe($result, $deviations);
    $lines = [];
    foreach ($changed as $p => $c) $lines[] = $p . ($c ? ' changed' : ' could not change');
    $owner = party($changed);
    echo "$name → $owner\n    ", implode(' · ', $lines), "\n";
    return $owner;
}

DA17. The text the partner sends is a fixed string produced by a function; it describes the same two products in three formats: the codes k1 and k2, the prices 12.5 and 8. DA18. The partner’s deviation is writing the same field differently: it sends the price as a string instead of a number, or it writes the field name cost instead of price. DA19. The code reads the partner’s data as it is without validating it; validation is the job of the Security topic. DA20. The result is the JSON representation of the parsed price list.

Three Formats

JSON (JavaScript Object Notation) is a text format that carries nested arrays and objects. In PHP, json_decode() turns it into an array. On invalid text the default behavior is to return null silently; the JSON_THROW_ON_ERROR flag turns this into a JsonException so that the error cannot go unnoticed. XML is a tree built from tags; simplexml_load_string() turns it into an object. CSV rests on lines and fields separated by commas; str_getcsv() splits one line into an array. The limit is the same in all three: the code has to know the shape of what is parsed itself.

The file below produces the partner’s text in the three formats and defines a parser that reads the prices for each. It has no output and is only written to disk.

<?php
// partner.php — the partner's three formats and parsers that read the prices
declare(strict_types=1);

function partnerJson(?string $deviation = null): string {
    $first = $deviation === 'string' ? '"12.5"' : '12.5';
    $second = $deviation === 'string' ? '"8"' : '8';
    $field = $deviation === 'field' ? 'cost' : 'price';
    return '{"products":[{"code":"k1","' . $field . '":' . $first . '},{"code":"k2","' . $field . '":' . $second . '}]}';
}

function partnerXml(): string {
    return '<products><product code="k1"><price>12.5</price></product><product code="k2"><price>8</price></product></products>';
}

function partnerCsv(): string {
    return "code,price\nk1,12.5\nk2,8";
}

function jsonPrices(string $text): array {
    $data = json_decode($text, true, flags: JSON_THROW_ON_ERROR);
    return array_map(fn(array $p) => $p['price'] ?? null, $data['products']);
}

function xmlPrices(string $text): array {
    $root = simplexml_load_string($text);
    $prices = [];
    foreach ($root->product as $product) $prices[] = (string) $product->price;
    return $prices;
}

function csvPrices(string $text): array {
    $lines = explode("\n", $text);
    array_shift($lines);
    return array_map(fn(string $line) => str_getcsv($line, escape: '')[1], $lines);
}

The escape: '' argument in the str_getcsv() call turns off treating the backslash as an escape character; the only rule of CSV itself is the quotation mark. Run the three parsers and look at invalid JSON and invalid XML.

<?php
// formats.php — three parsers and invalid input
declare(strict_types=1);
require 'partner.php';

echo 'json: ', json_encode(jsonPrices(partnerJson())), "\n";
echo 'xml: ', json_encode(xmlPrices(partnerXml())), "\n";
echo 'csv: ', json_encode(csvPrices(partnerCsv())), "\n";

try {
    jsonPrices('{"products":[');
} catch (JsonException $e) {
    echo 'invalid json: ', get_class($e), ', message: ', $e->getMessage(), "\n";
}

libxml_use_internal_errors(true);
echo 'invalid xml: ', var_export(simplexml_load_string('<products><product>'), true), "\n";

json: [12.5,8]
xml: ["12.5","8"]
csv: ["12.5","8"]
invalid json: JsonException, message: Syntax error
invalid xml: false

JSON brought the numbers as numbers; XML and CSV brought every value as text. This difference comes from the parser and is not the partner’s decision: the 12.5 inside an XML tag is always text by the nature of the format. Invalid JSON threw a JsonException; invalid XML returned false instead of an error. The code itself has to check both.

When the Partner Writes the Same Field Differently

The partner’s deviation is a small departure from the format the code expects. It writes the price as "12.5" instead of 12.5, or it names the field cost instead of price. In the first case the code gives no error, only the type of the returned value changes. In the second case the field cannot be found and the code reads null. Both are in the partner’s hands.

<?php
// partner-result.php — who can change the prices?
declare(strict_types=1);
require 'party.php';
require 'partner.php';

function prices(array $d): string {
    return json_encode(jsonPrices(partnerJson($d['third party'] ?? null)));
}

partyReport('prices, string', 'prices', ['third party' => 'string']);
partyReport('prices, field name', 'prices', ['third party' => 'field']);

prices, string → third party
    third party changed
prices, field name → third party
    third party changed

In both deviations the result is in the third party’s hands. In the string case the prices stopped being numbers; their appearance inside quotation marks in the output shows this, and passing them to a function that expects a number under strict_types would now be a type mismatch. In the field name case none of the prices could be read. The code stayed the same; the partner’s text changed. For this reason the result of parsing is correct as long as the partner’s format is followed; the agreement depends on the partner’s word.

What the Parser Guarantees and What It Does Not

In CSV, a field that contains a comma is put in double quotes. If the partner applies this rule, the parser keeps the field whole; if it does not, the line is split into more columns than expected. Read the same line in two forms.

<?php
// quote.php — a CSV field that contains a comma
declare(strict_types=1);

$quoted = 'k3,"Pen, blue",4.5';
$unquoted = 'k3,Pen, blue,4.5';
echo 'quoted: ', json_encode(str_getcsv($quoted, escape: ''), JSON_UNESCAPED_UNICODE), "\n";
echo 'unquoted: ', json_encode(str_getcsv($unquoted, escape: ''), JSON_UNESCAPED_UNICODE), "\n";

quoted: ["k3","Pen, blue","4.5"]
unquoted: ["k3","Pen"," blue","4.5"]

The quoted line gave three fields, the unquoted line four. The parser worked correctly in both cases; the difference came from the text the partner wrote. The same distinction holds for JSON and XML: the parser applies the rules of the format and does not decide whether the partner followed those rules. For this reason none of the three parsers can know in advance that a price will be a number; only the partner’s contract knows this, and the contract is outside the code.

The Bounding Measurement

Now the direction is reversed: we give a value we received from the partner back in our own response as JSON. The discount the partner sends may be 1.0 or 1; json_decode() brings one as a float and the other as an integer. When the response is produced with json_encode(), PHP writes the value 1.0 as 1 by default; the JSON_PRESERVE_ZERO_FRACTION flag keeps the fractional part and writes 1.0. We are the ones who choose this notation.

<?php
// response.php — the number notation in our own response
declare(strict_types=1);
require 'party.php';

echo 'no flag: ', json_encode(1.0), "\n";
echo 'with flag: ', json_encode(1.0, JSON_PRESERVE_ZERO_FRACTION), "\n";

function response(array $d, int $flag, bool $force): string {
    $text = $d['third party'] ?? '{"discount":1.0}';
    $discount = json_decode($text, true, flags: JSON_THROW_ON_ERROR)['discount'];
    return json_encode(['discount' => $force ? (float) $discount : $discount], $flag);
}

$deviation = ['third party' => '{"discount":1}'];
partyReport('default flag', fn(array $d) => response($d, 0, false), $deviation);
partyReport('flag, partner notation', fn(array $d) => response($d, JSON_PRESERVE_ZERO_FRACTION, false), $deviation);
partyReport('flag and float cast', fn(array $d) => response($d, JSON_PRESERVE_ZERO_FRACTION, true), $deviation);

no flag: 1
with flag: 1.0
default flag → server
    third party could not change
flag, partner notation → third party
    third party changed
flag and float cast → server
    third party could not change

With the default flag the response is the same output whether the partner writes 1 or 1.0: the result is on the server. Only when the flag is added does the response start to carry the notation the partner wrote, and the result returns to the third party. When the value is also converted to float in addition to the flag, the notation of the response is again entirely in our hands. How a number looks in the response is the server’s choice; the partner determines only as much as we leave to it.

Summary

• JSON, XML and CSV are read with separate parsers; JSON_THROW_ON_ERROR turns invalid JSON into a JsonException, and escape: '' is given in the str_getcsv() call.
• XML and CSV values arrive as text; JSON numbers arrive as numbers. The parser determines the type, not the partner.
• If the partner writes the same field as a string or under another name, the price list changes; in this lesson the party is the third party.
• The number notation in our own response is determined by the json_encode() flag and the type conversion, and it is the server’s choice.
• The same partner data can belong to the third party in one result and to the server in another.

Next Step

We parsed the partner’s data once; but to reach the partner a request has to be sent first. Between the request and the response there is also a timeout. The next lesson handles what the client does when the service does not respond.