Unit 01 of 02
Automated Checks
Measuring the checks that run on every change with three numbers: how true findings, false positives, and unseen items move together as the threshold shifts in static analysis, and the alert fatigue that follows; the surface a tool scanning a running application can and cannot reach; matching the dependency graph against a known-vulnerability catalog; secret scanning in repository history and the cost of its history; separating vulnerabilities that come from the base image from those the application adds across image layers; and the items a template scan catches before deployment.
Published Updated
Lessons
- 01 Dynamic Application Security Testing The area a tool scanning a running application can and cannot reach, across a 96-endpoint surface: anonymous browsing reaches 13 endpoints and finds 2 vulnerabilities; adding a session and a recorded flow brings that to 84 endpoints and 10 vulnerabilities, while 12 admin endpoints stay out of reach in every configuration. 18 of the 28 vulnerabilities go unseen — 5 because they are unreachable, 13 despite being reached. The same scan is placed in the pipeline three different ways: a full scan on every build costs 5,316 minutes with zero delay, a full nightly scan costs 443 minutes with a 6-build delay, and scanning only changed endpoints costs 126 minutes but never sees 3 of the vulnerabilities at all. 14 min read
- 02 Dependency Component Analysis A 210-component inventory is matched against the known-vulnerability catalog: 102 of the 139 raw name matches are eliminated at the version-range check, 6 at the backported-fix check, leaving 31 real matches; 12 unlisted vulnerabilities are never seen at all, 6 of them in a component the catalog does not recognize. 25 of the 30 affected components are transitive, 12 are never invoked. Blocking on name alone stops 78 of 80 builds; the version-range check brings that to 70, the invocation filter to 57, and the remaining blocked builds trace to a single transitive component with no fixed upstream version. 14 min read
- 03 Secret Scanning in Repository History Four patterns working together across 940 string constants: as the entropy threshold is pulled from 0.60 to off, matches drop from 855 to 519 while the false-positive rate falls only from 68.0 to 52.8, because the source of the noise is not entropy but the keyword pattern that marks 270 of its 442 matches wrong; 50 of 310 secrets match no pattern at all, and 26 of them sit in a single class. Three placements carry the same pattern set: working tree at every build scans for 1,057 minutes and never looks at 18 secrets, incremental diff drops to 9 minutes but never sees the remaining 164, and a one-time full history reads the entire past in 48 minutes and shows that 14 secrets exist only there. 13 min read
- 04 Container Image Scanning A 348-component image's 144 findings split by source: 75 come from the base layer, 69 from the application layer, and moving the base image to a new version closes 64 findings — 85.3 percent of the base-sourced ones — in a single operation while never touching the application layer; 18 vulnerabilities sit in 31 components absent from the package registry, so they show up in no run. The three images' finding-row total is 269, unique vulnerabilities 144. The policy that blocks every severe finding stops 49 of 60 builds, and in 12 of them every blocking finding is entirely in the base layer, which the team cannot fix; narrowing the block leaves severe base exposure at 940 build-findings, and refreshing the base every twenty builds brings it down to 382. 14 min read
- 05 Infrastructure as Code Scanning Across 420 template items, the decision a rule can make before deployment is counted: of 100 real violations, 55 are caught with certainty, 26 stay undeterminable because their value is computed at deployment time, and 19 fall in a field class with no rule written at all. Passing the undeterminable through leaves 26 violations unseen, stopping them blocks 65 compliant items for nothing. The same rule set runs across 60 builds in four placements: a strict gate stops 34 builds and leaves 1,339 build-item exposure, a check that only scans after deployment drops to 979 without stopping a single build, and the two together drop to 697; the same 19 violations never close under any placement. 13 min read