Sandbox Usage

The previous lesson decided whether an item could run by looking at path and signature; neither criterion ever asked what the item does, and that left exactly the items it missed. The only way to learn what an item does is to run it — but running it on the endpoint defeats the purpose of the decision itself.

The sandbox is the answer to this bind: a suspicious item is run somewhere isolated from the production environment and its behavior is observed. How strong the isolation is was measured in the Information Technology and Systems Fundamentals course and is taken as an input here. What is measured is not the isolation, it is the decision itself.

Not a Control, a Decision

This lesson’s result can be written in the first line: the sandbox closes no foothold point. A startup entry, a service definition, or a scheduled task is in the same state after the sandbox is set up as before. The only thing the sandbox produces is a decision, and that decision becomes input to another control — the allowlist. It is written to only the second of the course’s two columns, the visibility column, and this lesson is that column’s purest example.

The decision has three values. Block: a decisive behavior was seen in observation. Allow: every behavior of the item was seen and none was decisive. Uncertain: the item ran but at least one behavior was not seen, so its absence does not mean it does not exist. The third value is this lesson’s real subject; a two-valued decision table folds uncertain into allow and the measurement breaks.

When a suspicious item is run, it may try to reach out over the network; the sandbox’s controlled network must never reach a real target, and examining an item belonging to another organization without written authorization and a defined scope is both illegal and unprofessional.

• SH25. The eighteen-item queue, eight behavior classes, and four configurations are a model; no real item is run. Suspicious items are identified only by type and observed behavior class; what they are and how they arrived is not written.
• SH26. The strength of the isolation is not measured here; it is taken as an input.
• SH27. A behavior becoming visible is treated as depending on a single environment condition. In reality a behavior may need more than one condition; this grows the unseen column.
• SH28. Whether an item can run at all depends on a single required condition; if the condition is not provided, the item is never run.
• SH29. The observation-window and manual-review durations are written by hand; there is no randomness.
• SH30. The decisive flag belongs to the behavior, not the item. The same behavior can also be seen in a legitimate item, and the model marked it that way.

// sandbox/queue.mjs — the run decision in a sandbox IS A MODEL. No real item is run;
// suspicious items are identified only by type and observed behavior class.
// A behavior becoming visible depends on the sandbox providing its environment condition.
export const BEHAVIOR = {
  'file-write': '-', 'process-creation': '-', 'persistence-definition': '-',
  'network-connection': 'network', 'credential-read': 'domain-identity',
  'scheduled-trigger': 'long-duration', 'user-action': 'user-interaction',
  'device-access': 'native-hardware',
};
// Configuration: environment conditions it provides, observation window (min), whether it needs an operator.
export const CONFIG = {
  isolated: { provides: [], window: 5, operator: false },
  'controlled-network': { provides: ['network'], window: 5, operator: false },
  'long-window': { provides: ['network', 'long-duration'], window: 60, operator: false },
  interactive: { provides: ['network', 'long-duration', 'user-interaction'], window: 60, operator: true },
};
export const MANUAL_MIN = 25;                 // manual review time per uncertain item
// row: name | type | condition required to run | actual | behaviors ('*' = decisive)
export const ITEM = `
  diagnostic-tool             binary -               legitimate  file-write,process-creation
  new-version-installer       binary -               legitimate  file-write,network-connection,*persistence-definition
  internal-helper-tool        binary -               legitimate  file-write,process-creation,network-connection
  meter-reading-tool          binary native-hardware  legitimate  device-access,file-write
  field-setup-script          script -               legitimate  file-write,*persistence-definition,user-action
  quarterly-report-script     script -               legitimate  file-write,network-connection
  internal-report-tool        binary -               legitimate  file-write,network-connection
  new-document-template       document -             legitimate  file-write,user-action
  vendor-archive               archive -             legitimate  file-write,process-creation
  domain-management-script    script domain-identity  legitimate  network-connection,*credential-read
  suspicious-item-1           binary -               malicious   file-write,process-creation,*persistence-definition
  suspicious-item-2           script -               malicious   file-write,network-connection,*credential-read
  suspicious-item-3           binary -               malicious   file-write,*scheduled-trigger
  suspicious-item-4           binary -               malicious   process-creation,*credential-read
  suspicious-item-5           document -             malicious   file-write,*user-action
  suspicious-item-6           binary -               malicious   file-write,process-creation
  suspicious-item-7           archive -              malicious   file-write,*network-connection
  suspicious-item-8           binary native-hardware  malicious   file-write,*device-access`
  .trim().split('\n').map((r) => {
    const [name, type, required, actual, beh] = r.trim().split(/\s+/);
    return { name, type, required, malicious: actual === 'malicious',
      behavior: beh.split(',').map((d) => ({ name: d.replace('*', ''), decisive: d.startsWith('*') })) };
  });

export const condition = (d) => BEHAVIOR[d.name];
export const visible = (d, c) => condition(d) === '-' || CONFIG[c].provides.includes(condition(d));
export const runs = (k, c) => k.required === '-' || CONFIG[c].provides.includes(k.required);

// A single item's decision in a configuration: block | allow | uncertain | no-run
export function decide(k, c) {
  if (!runs(k, c)) return 'no-run';
  const seen = k.behavior.filter((d) => visible(d, c));
  if (seen.some((d) => d.decisive)) return 'block';
  return seen.length === k.behavior.length ? 'allow' : 'uncertain';
}

export function scan(c) {
  const s = { ran: 0, didNotRun: 0, seen: 0, unseen: 0,
    block: 0, allow: 0, uncertain: 0, wrongAllow: 0, wrongBlock: 0 };
  for (const k of ITEM) {
    const d = decide(k, c);
    if (d === 'no-run') { s.didNotRun += 1; continue; }
    s.ran += 1;
    s.seen += k.behavior.filter((b) => visible(b, c)).length;
    s.unseen += k.behavior.filter((b) => !visible(b, c)).length;
    s[d] += 1;
    if (d === 'allow' && k.malicious) s.wrongAllow += 1;
    if (d === 'block' && !k.malicious) s.wrongBlock += 1;
  }
  s.observeMin = s.ran * CONFIG[c].window;
  s.operatorMin = CONFIG[c].operator ? s.observeMin : 0;
  s.manualMin = (s.uncertain + s.didNotRun) * MANUAL_MIN;
  return s;
}

Four Configurations, the Same Queue

// sandbox/measure.mjs — four sandbox configurations evaluated against the same suspicious-item queue
import { BEHAVIOR, CONFIG, ITEM, MANUAL_MIN, condition, visible, runs, decide, scan } from './queue.mjs';

const row = (g, ...s) => console.log(s.map((v, i) =>
  (g[i] < 0 ? String(v).padEnd(-g[i]) : String(v).padStart(g[i]))).join('').trimEnd());
const neverProvided = [...new Set(Object.values(BEHAVIOR))]
  .filter((c) => c !== '-' && !Object.values(CONFIG).some((v) => v.provides.includes(c)));

console.log(`${ITEM.length} suspicious items, ${ITEM.filter((k) => k.malicious).length} malicious; ` +
  `${Object.keys(BEHAVIOR).length} behavior classes, ${Object.keys(CONFIG).length} configurations`);
console.log(`condition never provided by any configuration: ${neverProvided.join(', ')}`);

const A = [-21, 9, 12, 10, 12, 9, 7, 11, 13, 14];
console.log('\n1. result per configuration');
row(A, 'configuration', 'ran', 'no-run', 'seen', 'unseen', 'block', 'allow',
  'uncertain', 'wrong allow', 'wrong block');
for (const c of Object.keys(CONFIG)) {
  const s = scan(c);
  row(A, c, s.ran, s.didNotRun, s.seen, s.unseen, s.block, s.allow, s.uncertain,
    s.wrongAllow, s.wrongBlock);
}

const B = [-27, 12, 14, 21, 14, 14];
console.log('\n2. decision per item');
row(B, 'item', 'actual', ...Object.keys(CONFIG));
for (const k of ITEM)
  row(B, k.name, k.malicious ? 'malicious' : 'legitimate', ...Object.keys(CONFIG).map((c) => decide(k, c)));

const C = [-21, 9, 13, 14, 12, 11, 15];
console.log('\n3. decision time (min) and closed foothold points');
row(C, 'configuration', 'window', 'observe min', 'operator min', 'manual min', 'total min', 'closed points');
for (const c of Object.keys(CONFIG)) {
  const s = scan(c);
  row(C, c, CONFIG[c].window, s.observeMin, s.operatorMin, s.manualMin,
    s.observeMin + s.operatorMin + s.manualMin, 0);
}
const u = scan('long-window'), e = scan('interactive');
console.log(`\nitems falling to manual review (long-window)   ${u.uncertain + u.didNotRun} / ${ITEM.length}`);
console.log(`undecided in every configuration                ${e.uncertain + e.didNotRun} / ${ITEM.length}`);
console.log(`  malicious items unstoppable in the best config  ` +
  `${ITEM.filter((k) => k.malicious && decide(k, 'interactive') !== 'block').length}`);
console.log(`wrong allow / wrong block in every configuration ${e.wrongAllow} / ${e.wrongBlock}`);
console.log(`manual review time                               ${MANUAL_MIN} min/item`);

18 suspicious items, 8 malicious; 8 behavior classes, 4 configurations
condition never provided by any configuration: domain-identity, native-hardware

1. result per configuration
configuration              ran      no-run      seen      unseen    block  allow  uncertain  wrong allow   wrong block
isolated                    15           3        23          12        3      3          9            1             2
controlled-network          15           3        29           6        4      6          5            1             2
long-window                 15           3        30           5        5      6          4            1             2
interactive                 15           3        33           2        6      7          2            1             2

2. decision per item
item                             actual      isolated   controlled-network   long-window   interactive
diagnostic-tool              legitimate         allow                allow         allow         allow
new-version-installer        legitimate         block                block         block         block
internal-helper-tool         legitimate     uncertain                allow         allow         allow
meter-reading-tool           legitimate        no-run               no-run        no-run        no-run
field-setup-script           legitimate         block                block         block         block
quarterly-report-script      legitimate     uncertain                allow         allow         allow
internal-report-tool         legitimate     uncertain                allow         allow         allow
new-document-template        legitimate     uncertain            uncertain     uncertain         allow
vendor-archive               legitimate         allow                allow         allow         allow
domain-management-script     legitimate        no-run               no-run        no-run        no-run
suspicious-item-1             malicious         block                block         block         block
suspicious-item-2             malicious     uncertain            uncertain     uncertain     uncertain
suspicious-item-3             malicious     uncertain            uncertain         block         block
suspicious-item-4             malicious     uncertain            uncertain     uncertain     uncertain
suspicious-item-5             malicious     uncertain            uncertain     uncertain         block
suspicious-item-6             malicious         allow                allow         allow         allow
suspicious-item-7             malicious     uncertain                block         block         block
suspicious-item-8             malicious        no-run               no-run        no-run        no-run

3. decision time (min) and closed foothold points
configuration           window  observe min  operator min  manual min  total min  closed points
isolated                     5           75             0         300        375              0
controlled-network           5           75             0         200        275              0
long-window                 60          900             0         175       1075              0
interactive                 60          900           900         125       1925              0

items falling to manual review (long-window)   7 / 18
undecided in every configuration                5 / 18
  malicious items unstoppable in the best config  4
wrong allow / wrong block in every configuration 1 / 2
manual review time                               25 min/item

Observed and Unobserved Behavior

15 of eighteen items can be run, 3 cannot be run at all. Two of the ones that cannot run are items requiring native hardware — the meter-reading tool and a suspicious binary — and one is a management script requiring a domain identity. The sandbox says nothing about these three items; a sixth of the queue falls to manual review before any observation happens at all.

Loosening the isolation increases observed behavior: from 23 to 33, unobserved from 12 to 2. But what is being loosened is exactly the isolation itself. Adding a controlled network makes six more behaviors visible; extending the observation window from five minutes to sixty adds only one behavior; operator interaction adds three behaviors. The gains are not equal to each other, and neither are the costs.

Two conditions are never provided in any configuration: domain identity and native hardware. The first cannot be provided because giving the sandbox a real domain identity eliminates the isolation — everything reachable with that identity enters the sandbox. The second cannot be provided because the hardware is not there. These two conditions are not a configuration gap; they are the method’s limit: as long as the sandbox is not the production environment itself, behavior tied to the production environment stays invisible, and the moment it is the production environment, it stops being a sandbox.

The Item Left Undecided

The second table shows how decisions change from configuration to configuration. In the isolated configuration, 9 of the 15 running items stay uncertain; in the most capable configuration that number drops to 2. But three values stay the same: wrong allow at 1, wrong block at 2, and items that cannot run at 3.

Wrong allow is a single item, and it does not get fixed in any configuration. All of that item’s behaviors are visible in observation — file write and process creation — and neither is decisive. Extending the observation window, adding the network, or seating an operator changes nothing for this item, because what is missing is not observation, it is the distinction itself. If an item’s sandbox behavior cannot be told apart from a legitimate item’s, the sandbox cannot produce a decision about it.

Wrong block is two items, and the reason is the same, reversed. The new-version installer and the field setup script both write a persistence definition — the very thing this course measures, a foothold-point definition. A decisive behavior is seen and the decision comes out “block.” Both are legitimate work. The sandbox sees the behavior, not who did it under what authority; writing a persistence definition during installation is an installer’s job.

Five items cannot be decided in any configuration, and three are malicious. Three come from items that cannot be run, two from items with a behavior requiring domain identity. Of the queue’s eight malicious items, only four are stopped by the sandbox; of the remaining four, three get no answer from the sandbox at all — the question is handed to manual review — and the fourth passes with a wrong allow decision in all four configurations and never falls to review.

The Cost of the Decision

The third table gives the cost, and it is not one-directional. Observation time is 75 minutes in the isolated configuration, 900 in the long window. Manual review time goes the other way: from 300 minutes down to 125. Added together, the cheapest configuration turns out to be controlled network — 275 minutes — and the most expensive is interactive, 1925 minutes. The gap is exactly sevenfold, and in return uncertain items drop from 5 to 2 while wrong decisions do not change at all.

This is the number that determines where the decision stops. Adding a controlled network to a five-minute window raises observed behavior from 23 to 29 and lowers total time from 375 to 275, because fewer items fall to manual review. Extending the window to sixty minutes, on the other hand, gains a single behavior and quadruples the time. The cheap step is adding the network; the expensive one is extending the duration.

There is another cost the table does not show: the sandbox itself needs maintenance. For observation to work, the environment has to resemble production endpoints — the same software set, the same versions, the same configuration — and the more it resembles them, the less isolation there is. Every time the endpoints’ baseline changes, the sandbox image has to be updated too; when it is not, an item may show no behavior at all because it does not recognize the environment it is running in, and the decision silently drifts to “allow.” Add to this the inventory of the machines that run the sandbox itself: they are endpoints too, and they go into this course’s first-lesson count.

The last column is zero in every row. No matter how many more behaviors one configuration reveals over another, the number of closed foothold points does not change. The decision the sandbox produces either adds a row to the allowlist or it does not; what closes the point is that row, not the sandbox.

Summary

• The sandbox is not a control but a run decision: closed foothold points are zero across all four configurations. The decision it produces becomes input to the allowlist, and the point closes there.
• The decision is three-valued; if uncertain is not kept separate, the measurement breaks. In the isolated configuration, 9 of the 15 running items are uncertain; in the most capable configuration, 2.
• Two environment conditions can never be provided in any configuration — domain identity and native hardware. Providing the first eliminates the isolation; this is not a gap, it is the method’s limit.
• Wrong allow (1) and wrong block (2) never change in any configuration. Wrong block happens because a legitimate installer also writes a persistence definition: the sandbox sees the behavior, not the authority.
• Only four of the eight malicious items are stopped by the sandbox; three cannot be decided and go to manual review, and one passes with a wrong allow decision in all four configurations.
• Time runs in two directions: adding a controlled network lowers total time from 375 minutes to 275; extending the window to sixty minutes gains one behavior and pushes time to 1075 minutes.

Next Step

Everything counted up to here held while the machine was on and an operating system was running. The inventory was pulled from inside that system, the allowlist audited that system’s run decisions, the sandbox ran an item on a copy of that system. Every one of them shares the same assumption: the only way to reach the machine is through the running system.

The next lesson removes that assumption. When the disk is pulled from the machine and mounted somewhere else, the running system drops out of the picture, and every control drops with it: the allowlist performs no evaluation, the hardening baseline is not read, no log is kept. What gets measured is this: how many items can be read with physical access on an unencrypted and an encrypted disk, which foothold-point class full disk encryption closes, and, item by item, what encryption does not protect while the machine is on, a session is open, or the screen is locked. The topic closes with this measurement.