Application Allowlists

The previous lesson’s most expensive item was the software-source restriction: it narrowed the write authority of 96 points and dropped seven legitimate tasks to a manual step. Behind the item is a decision mentioned in a single line — deciding in advance what software can run on the endpoint. An application allowlist turns that decision from a configuration item into a decision made at every execution attempt.

A configuration item is applied once and its compliance is asked about. An allowlist is re-evaluated on every execution, and its accuracy can only be measured against a real execution set: the items that ran on the endpoints over a period are checked against the list one by one, and two kinds of error are counted — a blocked legitimate item and an illegitimate item that gets through.

Two Kinds of Rule

Allowlist rules are written along two axes. A path-based rule looks at which directory class the executed file sits in: the system directory and the program directory are allowed; the user directory, the temp directory, a network share, and removable media are not. A signature-based rule looks at who signed the file: the operating system vendor, the business software vendor, and an internal signature are allowed; unsigned and unknown-signature items are not.

The two rules can be combined. In an or form, an item passing a single criterion is enough; in an and form, it must pass both. The same two rules produce very different results with two different combinations, and the difference is visible only once measured against a set.

• SH19. The twenty-five-item run set, the period, and the run counts are a model; no execution is blocked on a real endpoint.
• SH20. Illegitimate items are identified only by type, path class, and signature class; what they do and how they arrived are not written.
• SH21. The path class has six values, the signature class five. In reality, subdirectory and certificate detail grow the item count without changing the rules’ behavior relative to each other.
• SH22. The decision is binary: pass or block. An alert-only, non-blocking mode is not modeled.
• SH23. For script items, the decision is made on the interpreter that runs them; with script inspection on, the script itself is evaluated by the same rule.
• SH24. Script inspection being inapplicable in two classes comes from the previous lesson: in those two classes, the work itself runs through a script.

// allowlist/allowlist.mjs — evaluating an application allowlist against a run set IS A MODEL.
// No real endpoint execution is blocked; items are written at a type level.
export const PATH_ALLOW = ['system', 'program'];
export const SIGNATURE_ALLOW = ['os-vendor', 'business-vendor', 'internal'];
// row: name | type | path class | signature class | legitimate? | machines it ran on | runs in the period
export const ITEM = `
  metric-agent                    binary system       os-vendor       e  102  91800
  log-forwarder                   binary system       os-vendor       e  102   3060
  patch-agent                     binary program       business-vendor e  102    408
  driver-updater                  binary program       os-vendor       e  102    204
  archive-tool                    binary program       business-vendor e  102   1224
  document-editor                 binary program       business-vendor e   84   5040
  field-measurement-application   binary program       business-vendor e   60   7200
  database-client                 binary program       business-vendor e   18    540
  internal-report-tool            binary program       unsigned        e   24    480
  maintenance-script               script system       internal        e  102   3060
  diagnostic-tool                 binary temp          internal        e   30     90
  new-version-installer           binary temp          business-vendor e   40     40
  internal-helper-tool            binary user          unsigned        e   18    360
  meter-reading-tool              binary removable     business-vendor e   12    720
  field-setup-script               script user         unsigned        e   60    180
  quarterly-report-script          script network-share internal        e   24     96
  unknown-binary-1                binary temp          unsigned        h    4     11
  unknown-binary-2                binary user          unsigned        h    7     23
  unknown-binary-3                binary network-share unknown         h    2      4
  unknown-binary-4                binary removable     unknown         h    1      1
  interpreted-item-1               script user         unsigned        h    6     19
  interpreted-item-2               script temp         unsigned        h    3      8
  signed-tool-misuse              binary program       business-vendor h    2      5
  unsigned-in-allowed-path        binary program       unsigned        h    5     14
  signed-item-in-user-path        binary user          business-vendor h    3      9`
  .trim().split('\n').map((r) => {
    const [name, type, path, signature, m, machines, runs] = r.trim().split(/\s+/);
    return { name, type, path, signature, legitimate: m === 'e', machines: Number(machines), runs: Number(runs) };
  });

export const RULE = {
  path: { rows: PATH_ALLOW.length, passes: (k) => PATH_ALLOW.includes(k.path) },
  signature: { rows: SIGNATURE_ALLOW.length, passes: (k) => SIGNATURE_ALLOW.includes(k.signature) },
  'path-or-signature': { rows: PATH_ALLOW.length + SIGNATURE_ALLOW.length,
    passes: (k) => PATH_ALLOW.includes(k.path) || SIGNATURE_ALLOW.includes(k.signature) },
  'path-and-signature': { rows: PATH_ALLOW.length + SIGNATURE_ALLOW.length,
    passes: (k) => PATH_ALLOW.includes(k.path) && SIGNATURE_ALLOW.includes(k.signature) },
};
// Script items: the decision is made on the interpreter that runs them, not the script itself.
// With script inspection on, the script itself is evaluated by the same rule.
export const passes = (k, rule, scriptInspection) =>
  (k.type === 'script' && !scriptInspection ? true : RULE[rule].passes(k));

export function measure(rule, scriptInspection) {
  const blocked = ITEM.filter((k) => k.legitimate && !passes(k, rule, scriptInspection));
  const evaded = ITEM.filter((k) => !k.legitimate && passes(k, rule, scriptInspection));
  const sum = (l, field) => l.reduce((a, k) => a + k[field], 0);
  return { blockedItems: blocked.length, blockedMachines: sum(blocked, 'machines'),
    blockedRuns: sum(blocked, 'runs'), evadedItems: evaded.length,
    evadedMachines: sum(evaded, 'machines'), evadedRuns: sum(evaded, 'runs'),
    rows: RULE[rule].rows + (scriptInspection ? 1 : 0) + blocked.length };
}

// How many of the fleet's foothold points become unusable with the allowlist.
// A point that can carry a script does not close, because it can call an allowed interpreter.
export const CLASS = [
  { name: 'field-laptop', machines: 60, remaining: 9, scriptable: 3, scriptInspection: true },
  { name: 'branch-endpoint', machines: 24, remaining: 12, scriptable: 3, scriptInspection: true },
  { name: 'collector', machines: 12, remaining: 12, scriptable: 4, scriptInspection: false },
  { name: 'processing-server', machines: 6, remaining: 11, scriptable: 3, scriptInspection: false },
];
export const pointMeasure = (s) => ({
  points: s.remaining * s.machines,
  closed: (s.scriptInspection ? s.remaining : s.remaining - s.scriptable) * s.machines,
  open: (s.scriptInspection ? 0 : s.scriptable) * s.machines,
});

Four Rule Sets, the Same Set

// allowlist/measure.mjs — four rule sets evaluated against the same run set
import { ITEM, RULE, CLASS, passes, measure, pointMeasure } from './allowlist.mjs';

const row = (g, ...s) => console.log(s.map((v, i) =>
  (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join('').trimEnd());
const sum = (f, field) => ITEM.filter(f).reduce((a, k) => a + k[field], 0);

console.log(`${ITEM.length} run items, ${sum(() => true, 'runs')} runs; ` +
  `${ITEM.filter((k) => !k.legitimate).length} items not legitimate, ` +
  `${ITEM.filter((k) => k.type === 'script').length} items are scripts`);

const A = [-20, 7, 12, 18, 11, 17, 13, 8];
console.log('\n1. rule sets (script inspection off / on)');
row(A, 'rule', 'script', 'blocked i.', 'blocked machines', 'evaded i.', 'evaded machines', 'evaded runs', 'rows');
for (const rule of Object.keys(RULE))
  for (const si of [false, true]) {
    const o = measure(rule, si);
    row(A, rule, si ? 'yes' : 'no', o.blockedItems, o.blockedMachines, o.evadedItems,
      o.evadedMachines, o.evadedRuns, o.rows);
  }

const B = [-32, 8, 13, 17, 12, 9];
console.log('\n2. items given a wrong decision under the path-or-signature rule (script inspection on)');
row(B, 'item', 'type', 'path', 'signature', 'legitimate', 'result');
for (const k of ITEM) {
  const g = passes(k, 'path-or-signature', true);
  if (g === k.legitimate) continue;                    // correctly decided items are skipped
  row(B, k.name, k.type, k.path, k.signature, k.legitimate ? 'yes' : 'no', g ? 'passed' : 'blocked');
}

const C = [-18, 8, 10, 8, 19];
console.log('\n3. the allowlist\'s effect on foothold points');
row(C, 'class', 'points', 'closed', 'open', 'script inspection');
let t = { points: 0, closed: 0, open: 0 };
for (const s of CLASS) {
  const o = pointMeasure(s);
  for (const a of Object.keys(t)) t[a] += o[a];
  row(C, s.name, o.points, o.closed, o.open, s.scriptInspection ? 'yes' : 'no');
}
row(C, 'total', t.points, t.closed, t.open, '');
console.log('\nselected rule set: path-or-signature');
for (const si of [false, true]) {
  const v = measure('path-or-signature', si);
  console.log(`  script inspection ${si ? 'yes' : 'no'}: blocked ${v.blockedItems} items / ` +
    `${v.blockedMachines} machines / ${v.blockedRuns} runs; evaded ${v.evadedItems} items / ` +
    `${v.evadedMachines} machines; ${v.rows} rows`);
}
console.log(`  classes where script inspection applies: ` +
  `${CLASS.filter((s) => s.scriptInspection).length}/${CLASS.length}`);

25 run items, 114596 runs; 9 items not legitimate, 5 items are scripts

1. rule sets (script inspection off / on)
rule                 script  blocked i.  blocked machines  evaded i.  evaded machines  evaded runs    rows
path                     no           4               100          4               16           46       6
path                    yes           6               184          2                7           19       9
signature                no           2                42          4               14           41       5
signature               yes           3               102          2                5           14       7
path-or-signature        no           1                18          5               19           55       6
path-or-signature       yes           2                78          3               10           28       8
path-and-signature       no           5               124          3               11           32      10
path-and-signature      yes           7               208          1                2            5      13

2. items given a wrong decision under the path-or-signature rule (script inspection on)
item                                type         path        signature  legitimate   result
internal-helper-tool              binary         user         unsigned         yes  blocked
field-setup-script                script         user         unsigned         yes  blocked
signed-tool-misuse                binary      program  business-vendor          no   passed
unsigned-in-allowed-path          binary      program         unsigned          no   passed
signed-item-in-user-path          binary         user  business-vendor          no   passed

3. the allowlist's effect on foothold points
class               points    closed    open  script inspection
field-laptop           540       540       0                yes
branch-endpoint        288       288       0                yes
collector              144        96      48                 no
processing-server       66        48      18                 no
total                 1038       972      66

selected rule set: path-or-signature
  script inspection no: blocked 1 items / 18 machines / 360 runs; evaded 5 items / 19 machines; 6 rows
  script inspection yes: blocked 2 items / 78 machines / 540 runs; evaded 3 items / 10 machines; 8 rows
  classes where script inspection applies: 2/4

Same Number, Different Items

With script inspection off, the path-based rule misses 4 illegitimate items; the signature-based rule also misses 4. The two numbers are equal, the items are not. The path rule lets through an unsigned item sitting in an allowed directory — the moment a writable subdirectory exists inside an allowed path class, the rule covers that subdirectory too. The signature rule lets through an item signed with a recognized signature sitting in the user directory — a signature says who produced the file, not where it was placed.

This means the question “which rule type is better” has no answer. The question to ask is which items they miss, and the two lists do not cover each other. Combining the two rules with and brings the evaded items down from 4 to 3; because it requires passing both criteria, items that pass only one of them are stopped.

The cost of the combination form is readable in the same rows. The or form brings the wrong block down to 1 item and 18 machines but raises evaded items to 5. The and form brings evaded down to 3 while raising the wrong block to 5 items and 124 machines. The same two rules, the same five rows, produce a fivefold difference in wrong blocks with two different combinations. The rule count never tells this decision; the combination form does.

An Item Both Rules Let Through

The second table lists the five items given a wrong decision under the selected rule set. The third row is this lesson’s boundary: the misuse of a tool that sits in an allowed directory and is signed with a recognized signature. This item passes every rule set including path-and-signature, and script inspection does not stop it either.

The reason is not a gap in the rule; it is the question itself. The allowlist answers “can this binary run”; it does not look at “what is this binary doing right now.” An approved tool being invoked for an unapproved task is an item the allowlist cannot see by definition. Its count is small — two machines, five runs — but it stays the same across every rule set, and a stricter list does not reduce it. Measuring this item belongs to the next topic, controls that watch behavior.

Blocked Legitimate Work

The strictest configuration — path-and-signature plus script inspection — brings evaded items down to 1 and evaded runs to 5. In the same row, the wrong block rises to 7 items and 208 machines. On a fleet of one hundred two machines, 208 machine-items means some machines run into more than one block.

Reading the blocked items by name, each has a justification. The diagnostic tool runs from the temp directory because it is downloaded there during an incident. The new-version installer is in the temp directory by definition. The meter-reading tool runs from removable media because there is no network in the field. The internal report tool is unsigned because no signing process was ever set up. The field-setup script sits in the user directory and is unsigned. None of these is a mistake; all of them are real work that falls outside the order the list assumes.

Every blocked item produces either an exception line or a manual approval on every run. The rows column counts this: 6 rows in the loosest configuration, 13 in the strictest. Maintenance cost does not come from the list itself but from the exceptions, and every new tool, every new version, and every new field procedure reopens this list.

The Interpreted Item and the Foothold Point

Five items are scripts, and their decision is made on the interpreter that runs them. With script inspection off, every rule set lets these five items through, because the interpreter is an allowed binary and that is what the list is asked about. When script inspection is turned on, evaded items drop under every rule set, but not by the same ratio: by half (4 to 2) under the path and signature rules, by a third (3 to 1) under the and rule.

The third table gives this a counterpart in foothold points, and for the first time since the first lesson, a large number lands in the closed column. 972 of the fleet’s 1038 points become unable to run unapproved code: a startup entry, a service definition, or a scheduled task stays in place, but what it calls does not run if it does not pass the list.

The 66 points that stay open come from two classes. Script inspection cannot be applied on the collector machine and the processing server, because — as measured in the previous lesson — the work itself runs through a script in those two classes. In these classes, a point that can carry a script can bypass the list by calling an allowed interpreter. The number is small (6 percent of 1038) and lands exactly where the list cannot be applied. The rule for a point that cannot be closed applies here too: the 66 points are at least written to the baseline, and whether they change is compared.

Summary

• An allowlist is not a configuration item but a decision made on every execution; its accuracy is measured only against a real execution set. 9 of twenty-five items are not legitimate, 5 are scripts.
• The path-based and signature-based rules miss the same number of items (4) but not the same items: one lets through an unsigned item in an allowed directory, the other a signed item in the user directory.
• The combination form carries more of the decision than the rule count: the same five rows wrongly block 1 item and miss 5 in the or form, and wrongly block 5 items and miss 3 in the and form.
• The strictest configuration brings evaded items down to 1, at the cost of blocking 7 legitimate items on 208 machines and raising the exception rows from 6 to 13. Every blocked item has a valid justification.
• The misuse of a tool sitting in an allowed directory and signed with a recognized signature passes every rule set; the allowlist looks at what can run, not at what it is doing.
• The allowlist makes 972 of the fleet’s 1038 foothold points unable to run unapproved code; 66 stay open in the two classes where script inspection does not apply and can only be written to the baseline.

Next Step

This lesson decided whether an item could run by looking at path and signature. Neither criterion ever asks what the item does, and that left exactly the item that evades both. The only way to learn what an item does is to run it — but running it on the endpoint defeats the purpose of the decision itself. The next lesson’s topic is the answer to this bind: the sandbox is the decision to run a suspicious item somewhere isolated from the production environment and observe its behavior. What gets measured is not the strength of the isolation — that was measured in a previous course and is taken here as an input — it is the run decision itself: how many suspicious items actually get run in isolation, how many behaviors show up in the observation window, how many behaviors never surface, and how many items the decision falls to manual review for.