Service and Version Detection

The previous lesson definitively distinguished that a port is open: 21 open ports were marked at the evidence level. A port being open, though, does not say what is running there. For now, each of the twenty-one ports is only a location; what software sits behind it, which version it is running, and whether that version is associated with a known vulnerability have not been asked at all.

The step that asks this question is fingerprinting and it brings a new measure: precision. The signatures collected from an endpoint point to a version set. The set sometimes has a single member, sometimes contains dozens of versions — and sometimes has a single member but carries the wrong one. The third case is this lesson’s main subject, because a wrong answer that looks certain is more expensive than a correct answer that stays ambiguous.

Precision Is the Size of the Candidate Set

RE13: every service carries three things — the fictional true version, the banner string’s status (correct, misleading, none), and the size of the candidate version set per probe level. RE14: a service is confirmed after N probes (candidate set has a single member); a service with a misleading banner reaches the true version only at the third level, and is confirmed but assigned the wrong one at earlier levels. RE15: the trace comes from the probe level — N requests per service, each request a network and an application record; from the third level on, unusual probes produce alerts.

// recon/version.mjs — fictional fingerprinting results for 04's 21 evidence-level open ports
// (a model): no real product, no real version, and no real vulnerability identifier; the version
// labels are fictional too.

// RE13: every service carries three things — the fictional true version, the banner string's
// status (correct, misleading, none), and the candidate version set size for probe levels 1..4.
// For a service with a misleading banner, "wrongVersion" is the version the string points to.
export const SERVICE = `
  subscriber/n1      A4 correct    -   1 1 1 1
  subscriber/n2      B2 correct    -   2 1 1 1
  subscriber/n3      C7 none       -  12 5 2 1
  field/n1           A4 correct    -   1 1 1 1
  field/n5           D1 misleading D0   1 1 3 1
  field/n6           E3 correct    -   1 1 1 1
  field/b1           F2 none       -   9 4 2 2
  vpn/n6             E3 correct    -   1 1 1 1
  collector/n6       E3 correct    -   1 1 1 1
  collector/n7       G5 none       -  14 6 3 2
  collector/n8       H1 misleading H9   1 1 4 2
  collector/b1       F2 none       -   9 4 2 2
  report/n1          A4 correct    -   1 1 1 1
  report/n6          E3 correct    -   1 1 1 1
  shared/n1          A4 correct    -   2 1 1 1
  shared/n3          C7 none       -  12 5 2 1
  monitoring/n1      A4 correct    -   1 1 1 1
  monitoring/n7      I2 misleading I5   1 2 3 1
  monitoring/b1      F2 none       -   9 4 2 2
  log/n1             A4 correct    -   1 1 1 1
  log/n5             J8 misleading J1   1 1 2 1`
  .trim().split("\n").map((r) => {
    const [id, trueVersion, banner, wrongVersion, ...d] = r.trim().split(/\s+/);
    return { id, trueVersion, banner, wrongVersion, set: d.map(Number) };
  });

// Fictional version-to-finding mapping: how many lines a version label carries into the finding list.
export const FINDINGS = { A4: 1, B2: 0, C7: 2, D1: 3, E3: 2, F2: 1, G5: 2, H1: 4, I2: 1, J8: 2,
  D0: 0, H9: 1, I5: 3, J1: 5 };

// RE14: a service is confirmed after N probes (candidate set has a single member). A service with
// a misleading banner reaches the true version only at the third level; at earlier levels it is
// confirmed but assigned the wrong one.
export const confirmed = (s, n) => s.set[n - 1] === 1;
export const correct = (s, n) => (s.banner === "misleading" ? n >= 3 : true);
export const assigned = (s, n) => (correct(s, n) ? s.trueVersion : s.wrongVersion);

// RE15: the trace comes from the probe level — N requests per service, each request a network and
// an application record; from the third level on, unusual probes produce alerts.
export const trace = (n) => ({ requests: SERVICE.length * n, records: SERVICE.length * n * 2,
  alerts: n >= 3 ? (n - 2) * 3 : 0 });

// recon/precision.mjs — fingerprinting in terms of precision: how many candidate versions narrow
// down to a single version.
import { SERVICE, FINDINGS, confirmed, correct, assigned, trace } from "./version.mjs";

const write = (g, ...s) => console.log(s.map((v, i) =>
  (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join(""));
const LEVEL = [1, 2, 3, 4];
const K = (n) => SERVICE.filter((s) => confirmed(s, n));

console.log(`${SERVICE.length} services: ${SERVICE.filter((s) => s.banner === "correct").length} ` +
  `with correct banner, ${SERVICE.filter((s) => s.banner === "misleading").length} misleading, ` +
  `${SERVICE.filter((s) => s.banner === "none").length} bannerless`);

const A = [-8, 8, 8, 10, 9, 8, 11];
console.log("\n1. precision per probe level");
write(A, "level", "requests", "conf", "ambiguous", "falseP", "evid", "prec %");
for (const n of LEVEL) {
  const k = K(n), fp = k.filter((s) => !correct(s, n)).length;
  write(A, n, trace(n).requests, k.length, SERVICE.length - k.length, fp, k.length - fp,
    (((k.length - fp) / SERVICE.length) * 100).toFixed(1));
}

const B = [-8, 8, 8, 12, 12, 15];
console.log("\n2. the trace cost of precision");
write(B, "level", "records", "alerts", "rec/evid", "alert/evid", "avg set size");
for (const n of LEVEL) {
  const k = K(n), evid = k.length - k.filter((s) => !correct(s, n)).length;
  const avg = SERVICE.reduce((t, s) => t + s.set[n - 1], 0) / SERVICE.length;
  write(B, n, trace(n).records, trace(n).alerts, (trace(n).records / evid).toFixed(2),
    (trace(n).alerts / evid).toFixed(3), avg.toFixed(2));
}

const C = [-8, 10, 8, 8, 8, 10];
console.log("\n3. lines carried into the finding list");
write(C, "level", "reported", "correct", "wrong", "missed", "wrong %");
for (const n of LEVEL) {
  const k = K(n);
  const rep = k.reduce((t, s) => t + FINDINGS[assigned(s, n)], 0);
  const cor = k.filter((s) => correct(s, n)).reduce((t, s) => t + FINDINGS[s.trueVersion], 0);
  const tru = k.reduce((t, s) => t + FINDINGS[s.trueVersion], 0);
  write(C, n, rep, cor, rep - cor, tru - cor, ((rep - cor) / rep * 100).toFixed(1));
}

const D = [-10, 7, 7, 7, 7, 7];
console.log("\n4. confirmed count by banner status");
write(D, "banner", "count", "level1", "level2", "level3", "level4");
for (const t of ["correct", "misleading", "none"]) {
  const l = SERVICE.filter((s) => s.banner === t);
  write(D, t, l.length, ...LEVEL.map((n) => l.filter((s) => confirmed(s, n)).length));
}
const b = SERVICE.filter((s) => s.banner === "misleading");
console.log(`misleading banner: at level 1, ${b.filter((s) => confirmed(s, 1)).length} services ` +
  `are confirmed but assigned the wrong version; their true finding count is ` +
  `${b.reduce((t, s) => t + FINDINGS[s.trueVersion], 0)}, reported is ` +
  `${b.reduce((t, s) => t + FINDINGS[s.wrongVersion], 0)}`);

21 services: 11 with correct banner, 4 misleading, 6 bannerless

1. precision per probe level
level   requests    conf ambiguous   falseP    evid     prec %
1             21      13         8        4       9       42.9
2             42      14         7        3      11       52.4
3             63      11        10        0      11       52.4
4             84      16         5        0      16       76.2

2. the trace cost of precision
level    records  alerts    rec/evid  alert/evid   avg set size
1             42       0        4.67       0.000           3.90
2             84       0        7.64       0.000           2.10
3            126       3       11.45       0.273           1.71
4            168       6       10.50       0.375           1.24

3. lines carried into the finding list
level     reported correct   wrong  missed   wrong %
1               22      13       9      10      40.9
2               20      14       6       9      30.0
3               14      14       0       0       0.0
4               24      24       0       0       0.0

4. confirmed count by banner status
banner      count level1 level2 level3 level4
correct        11      9     11     11     11
misleading      4      4      3      0      3
none            6      0      0      0      2
misleading banner: at level 1, 4 services are confirmed but assigned the wrong version; their true finding count is 10, reported is 9

The numbers belong to the measurement class; their inputs are RE13–RE15.

Thirteen Confirmed, Nine Pieces of Evidence

The first row of the first table sets up this lesson’s question all on its own. At the first probe level, 13 of 21 services narrow to a single version — by the precision measure, 61.9% are confirmed. Yet 4 of those thirteen are assigned the wrong version. The evidence count is not 13, it is 9, and the real precision is 42.9%.

The gap is not a measurement error; it has to do with the measure itself. A candidate set having a single member only shows that ambiguity has been eliminated; it does not show that the member is correct. A step that assigns a version by reading a banner string reads a piece of text the target wrote and counts that text as evidence. Nothing checks whether the text is true.

The fourth table breaks this down by banner status. Of the 11 services with a correct banner, 9 are confirmed at the first level, two remain stuck between two versions and get resolved at the second level. Of the 4 services with a misleading banner, all four are confirmed at the first level — at a higher rate than the correctly-bannered services, in fact. Of the 6 bannerless services, none is confirmed in the first three levels.

This is this lesson’s most counterintuitive finding: a misleading banner produces the highest apparent precision. A claim of “I am this version” from an endpoint narrows the candidate set to one, regardless of whether the claim is true or false. The side that sees ambiguity is the bannerless endpoint; the side that gets a wrong answer is the misleading one.

The Level Where the Confirmed Count Drops

The third row of the first table looks, at first glance, like a regression: the confirmed count drops from 14 at the second level to 11. In the same row, false positives fall from 3 to 0, and the evidence count stays at 11.

Here is what is happening. The behavioral probes at the third level produce signatures that contradict the banner string; three misleading services now point to three or four candidates instead of a single version. The candidate set grows, because the measurement accounts for the first time for the banner possibly being a lie. Apparent precision falls, real precision holds steady, and wrong assignments drop to zero.

In an assessment, this row reads as follows: a drop in the confirmed count is not a loss, it is the previous precision turning out to have been fake. Adding a probe does not have to increase precision; what it increases is accuracy, and when accuracy increases, some items rightly turn back into ambiguous.

At the fourth level, the confirmed count rises to 16 and all of them are correct — precision is 76.2%. The remaining five services still sit at two-candidate sets even after four probes; four of them are bannerless endpoints. Fingerprinting’s ceiling in this run is 16 of 21.

The Wrong Lines Carried Into the Finding List

The third table measures the cost of incorrect version information in the report itself. Every version label carries a certain number of lines into the finding list; a wrong version carries wrong lines.

At the first level, the finding list has 22 lines. 13 of them are correct, 9 are wrong — 40.9% of the list. At the same level, 10 real lines never show up at all. So on one side there are nine items that waste the remediation budget, on the other side ten items that never get looked at. The closing line gives this via the four misleading services: the true finding count totals 10 while the reported count is 9; the two numbers are close to each other and share not a single line.

At the second level, the wrong ratio falls to 30%. At the third level, the finding list drops to 14 lines and all of them are correct; missed lines are zero, because all eleven services counted as confirmed are correctly assigned. At the fourth level, the list rises to 24 lines and, again, all of them are correct.

The comparison between the first and fourth levels is this lesson’s practical takeaway: both produce a list of roughly the same size (22 versus 24), but in one, 40.9% of the lines are wrong, in the other, 0%. The finding count does not tell the two apart; the only thing that does is verification.

This table has a counterpart on the defense side, and it shows the asymmetry clearly. In the organization’s own software inventory, the version of the same 21 services can be read in a single step and with zero trace; installation records say which version is installed, which patch is applied, and which component is not used at all, without looking at a single misleading banner string. The measurement done from outside spends 84 requests and 168 records to reach 16 services, while the same measurement done from inside covers all 21 services. The assessment’s finding at this step is an expensive and incomplete copy of information the organization could read from its own inventory; its real value shows up in the places where that inventory does not match what is visible from outside.

The Trace Cost of Precision

The second table opens up, for the last time in this topic, the column that has been carried throughout. The average candidate set size falls from 3.90 to 1.24; in return, the records accumulating at the target rise from 42 to 168, and alerts from 0 to 6.

The cost is not distributed evenly. The first two levels leave 84 records and produce no alerts at all: these probes are indistinguishable from ordinary requests. The unusual probes used from the third level onward produce a protocol-level anomaly and turn into an alert. In other words, the step that completes accuracy is, at the same time, the first step the defense notices.

The rec/evid column reduces this to a single number: 4.67, 7.64, 11.45, and 10.50. The most expensive level is the third; the fourth level leaves more records but brings five new pieces of evidence, so its cost per piece of evidence falls. Read from the blue team’s side, it comes down to this: most of this topic’s steps are seen but not noticed. Across the total of five lessons, the records accumulating at the target are measured in the hundreds; the items that turn into an alert do not exceed the fingers of two hands, and every one of them comes from either density or a protocol anomaly — none of them is a record in the shape of “reconnaissance is under way.”

Summary

• Precision is the size of the candidate version set: at the first probe level, 13 of 21 services narrow to a single version but 4 are assigned the wrong version; evidence is 9, real precision is 42.9%.
• A misleading banner produces the highest apparent precision: all four services with a misleading banner are confirmed at the first level, none of the six bannerless services are confirmed in the first three levels.
• At the third level, the confirmed count drops from 14 to 11 while wrong assignments fall from 3 to 0: a drop in precision is the previous precision turning out to have been fake.
• At the first level, wrong version information carries 9 wrong lines into the finding list (40.9%) and misses 10 real lines; the two sets share not a single line.
• At the fourth level, all 24 lines are correct; it is a list of similar size to the 22 lines at the first level, and the only difference between them is verification.
• The cost of raising precision from 42.9% to 76.2% is rising from 42 records to 168 and from 0 alerts to 6; the first two levels produce no alerts at all, and the third level, which completes accuracy, produces the first alert.

Next Step

At the end of five lessons, the externally visible inventory has been drawn up, and candidates were counted at every step: 20 candidates from public items, 8 verified ownerships from registration lookup, 21 records and 6 dangling items from the name space, 21 open ports from scanning, 16 confirmed versions from fingerprinting. The trace each number leaves behind was written down too.

Yet all of these measurements share a common limit. No step in this topic can say whether a candidate is actually exploitable. A version label being associated with a known vulnerability does not show that the installation at that endpoint is actually affected: a patch may have been applied, the affected component may not be in use at all, or a control in front of it may never let the relevant request through. The fourth level’s 24 lines, even with full precision, are still, each one, a candidate.

The only way to close this gap is to look at the traffic coming from and going to the endpoint: which packets flow, how the response changes when a request is modified, what a file does when it is run, what independent sources say about the same item. The next topic — Analysis Tools — takes up each of these tools with the same two questions: what does it produce, and what cannot it prove.