Unit 03 of 03
Testing Process
The process of an authorized assessment: turning the scope definition into a programmatic audit and catching out-of-scope items, the candidate gap produced by black-, gray- and white-box assessments, running the candidate list through validation rules and eliminating false positives, scoring the same finding in two business contexts, measuring the report in terms of reproducibility, and calculating responsible disclosure as time and steps.
Published Updated
Lessons
- 01 Rules of Engagement A written scope definition for a 64-item fictional inventory is turned into a programmatic audit: 20 items come out in scope, 11 on the border, 33 out of scope. Through five shared front ends, 29 out-of-scope and borderline items can be reached by mistake; 11 of them are banned by name in the rules of engagement. The raw window exceeds the planned work by 4 hours, while subtracting the meter-reading interval leaves it 6 hours short; two of the four thresholds carry no on-call coverage, and for one of them the actual notification time comes to one and a half times what the contract states. 14 min read
- 02 Test Types Black-, gray- and white-box assessments are run against the same 20 items and the same 41 fictional flaws. Black box produces 78 candidates and yields 6 evidenced findings (7.7%), gray box turns 63 candidates into 20, white box turns 60 candidates into 31; 10 flaws cannot be evidenced by any type because proving them requires an action the rules of engagement forbid. Time per evidenced finding drops from 8.82 hours to 1.87 hours, and the defense-side trace per evidenced finding drops from 78 to 2.9; 17 of white box's 31 evidenced findings are meaningful only from the inside. 8 min read
- 03 Vulnerability Validation 240 raw candidates are run through four validation rules: deduplication removes 101 candidates without dropping a single finding, the version support rule removes 23, the impact boundary removes 9 candidates, validation eliminates 46 false positives, and 61 evidenced findings remain. 43 percent of the candidates come from version info, but only 16.3 percent of that class is real. When the validation order is tied to the chain, the validated candidates drop from 107 to 74, risk from 17.8 to 11.9, and the defense-side trace from 461 to 313 records; the cost is that 16 real flaws whose precondition is treated as closed stay unevidenced. 13 min read
- 04 Impact Assessment 61 evidenced findings are first scored by technical weight, then in two separate business contexts. Not one finding stays in the same position between the two business rankings, the mean shift is 22.23 positions, and the top-ten lists share no finding in common; 41.6 percent of pairwise comparisons invert. Under a 40-person-day budget, the technical ranking remediates 12 findings and closes 24.6 percent of the field context's impact while its own ranking closes 28.8 percent, and the ranking that divides business score by remediation day closes 39.4 percent. 13 min read
- 05 Reporting 61 report records are audited in terms of reproducibility. Even though each of the five mandatory fields is individually filled at a high rate, the rate at which all five are filled together is 31.1 percent; a second reader can reproduce 8 findings, and 3 can be called closed after a retest round. Writing the missing fields takes 9.4 hours and raises the closeable count to 49; 12 findings carry an environment variable and cannot be reproduced step by step under any circumstance. The remediation suggestion is directly actionable for 29 findings, requires a separate decision for 22, and is entirely absent for 10. 13 min read
- 06 Responsible Disclosure The time from notification to publication is measured across six stages: single-party findings close in an average of 12.7 days, four-party findings in 96.0 days, and the longest stage is remediation. Comparing three publication policies, the fixed 90-day window publishes 13 findings unfixed, the fix-linked window publishes none unfixed but pushes the latest publication to day 226, and the extendable window stays at 3. The lesson closes the course and gathers the candidate, evidence, and trace counts of eighteen lessons into a single table. 12 min read