Unit 01 of 02
Detection and Response
Measuring how well application telemetry can answer the questions asked after an incident: matching the question set against the log schema and counting the unanswered questions, the volume and retention cost of each extra field, what a threshold misses and falsely wakes up in detection scenarios based on business logic, the step–record dependency in application breach response and the minute cost of a missing record, the window of items that can be written out in short-lived workloads, and the verification and fix times in the external report intake and bug bounty program.
Published Updated
Lessons
- 01 Audit Log Design Matched against three log sources, a set of sixteen questions leaves six answered and ten unanswered: four because the field is not written, five because no join key links the sources, one because of the retention window. Three added fields raise the answered count to 13, but once all five are added the daily volume grows 1.45 times, retention under the fixed budget falls from 65 days to 45, and the five questions that reach into the 60-day audit period go unanswered again, so the answered count drops to 10. 14 min read
- 02 Application-Specific Detection Scenarios In an event stream of 900 sessions, 18 abuse sessions are searched for with four detection scenarios. Scanning thresholds one at a time gives the same trade-off on every metric: a denied-authorization threshold of 3 yields 364 false positives and 211 off-hours pages, while 25 yields zero false positives but zero catches. When the four scenarios are chosen together, an alert budget of 25 catches 8 of the 18 sessions and a budget of 180 catches all 18 — the latter means 101 night pages in 30 days. When the out-of-region read field is not written, the number caught at the same budget falls from 15 to 13. 13 min read
- 03 Application Breach Response In an unauthorized-access breach, twelve response steps are run against four log schemas. In the base schema seven steps are left without a record, the total takes 630 minutes and two steps cannot be completed at all; in the widest schema the same response takes 151 minutes, 4.17 times shorter. The absence of a record prolongs steps by between 4 and 8.3 times. The containment decision is made at minute 10 in every schema — the difference is not in containment but in scoping: when the affected subscriber set cannot be derived, the notification widens from 380 subscribers to an upper bound of 41,200 subscribers. 15 min read
- 04 Trace in Short-Lived Workloads 2,000 short-lived workloads produce 671,501 log items and 642,504 of them are written out: the loss is only 4.3%. In the same data, the last item of the workload can be collected in only 115 of 2,000 workloads, and not a single item comes out of the 92 workloads of the failed exit class. When the level of detail is raised eight times, the items written out rise to 4,272,000, but the number of workloads that answer the authorization decision question falls from 1,218 to zero. The cost of the setting that collects all items is 91,084 extra flush requests and 33 core-minutes of extra lifetime. 13 min read
- 05 External Report Intake Of the 340 reports arriving in 180 days, 91 are unique; 190 are duplicates and 59 are out of scope, so 73% of the queue carries no new vulnerability. At a triage capacity of two items per day the queue rises to 104 items and the mean wait is 31.2 days. Run against four log schemas, verification falls from 28.6 days to 25.8 days but the time from report to fix rises from 85.7 days to 89.9 days. The definite answer to the exploitation question rises from 15 to 20 and then falls to 13 in the widest schema; what raises the same number to 24 is not an extra field but raising triage capacity from 2 to 5. 15 min read