Unit 01 of 02
Fundamental Concepts
Counting what a procedure that learns its decision from data does and how much it gains, together with the baseline: the difference between a hand-written rule set and a learner that derives its rules from data on the same problem, the number of labels asked for by supervised, unsupervised, semi-supervised and reinforcement setups, what a pretext task whose label is derived from the data adds to the actual task, the four separate baselines of classification, regression, clustering and ranking problems, the number of returns in the workflow from problem definition to monitoring, and the same problem turning into a different output in three roles.
Published Updated
Lessons
- 01 What Is Machine Learning The same illicit-use suspicion problem is solved with three procedures and three numbers are printed side by side: the non-learning baseline scores 0.7533 on the held-out set, the hand-written rule set written by the field expert scores 0.7507, and the three-question decision tree that derives its rules from data scores 0.8594. The hand-written rule falls 0.0027 below the baseline, the tree rises 0.1061 above it. The difference does not come from the thresholds: when the same thresholds are moved into the hand-written rule, the number drops to 0.7480. Of the 377 subscribers in the held-out set, 93 are suspicious; the hand-written rule flags 151 subscribers and 76 turn out empty, the tree flags 66 subscribers and 13 turn out empty. The dataset is reduced from a readings table of 3,199 rows to a subscriber table of 1,260 rows, seed 20260218. 12 min read
- 02 Types of Learning The same illicit-use suspicion problem is set up in four setups and the number of labels each setup asks for is counted: supervised 0.8594 on the held-out set with 883 labels, unsupervised 0.8117 with 6 labels, semi-supervised 0.8090 with 60 labels, reinforcement 0.8090 with 120 visits. The baseline is 0.7533. As the label budget falls from 883 to 30, the held-out figure drops from 0.8594 to 0.7745, while the training figure rises from 0.8709 to 0.9400. The two-cluster split sits exactly on the baseline, the six-cluster split gains 0.0584. The visit policy finds 73 illicit cases in 120 visits; if the same budget were spent at random, 25 would be found. 14 min read
- 03 Self-Supervised Learning The label is derived from the data: the last reading of each subscriber is hidden and predicted from the earlier readings. On the pretext task the baseline gives a mean absolute error of 6.3776 m³ on the held-out set and the least squares model 5.9661 m³ — a gain of 0.4115 m³, obtained with zero labels. When the three numbers produced by the pretext task are carried to the main task, the 0.7851 measured with the record features alone rises to 0.8037. When the same representation is built from the baseline of the pretext task, it gives 0.8064, so the contribution of the learned part is minus 0.0027: the difference comes not from what the pretext task learned but from the raw reading that was carried along. The nine hand-derived features still give 0.8594. 14 min read
- 04 Application Areas Four questions are asked of the same fictional subscriber table and four separate baselines are chosen: in classification the most frequent class gives 0.7533 on the held-out set and the tree 0.8594; in regression the overall mean gives a mean absolute error of 5.2255 m³ and least squares 4.1275 m³; in clustering a random four-way grouping gives a within-cluster distance of 2.7683 and the four learned centers 2.1551; in ranking a random order finds 20 illicit cases in the first 100 visits, an order starting from the highest consumption finds 30, and an ensemble of nine trees finds 71. Of the four measures, a large number is good in two and a small number is good in two; the numbers can be compared not with one another but only with their own baselines. 15 min read
- 05 Machine Learning Workflow The seven-step flow produces three returns, and the number each return moved is printed. The gate at the fifth step tests the gap over the baseline with a threshold of 0.0500: the model built with three columns gives 0.0318 and stays at the gate, with nine columns it gives 0.1061 and passes. At the sixth step the model flags 240 of 1,260 subscribers, the daily capacity is 40 visits and the list spreads over six days; the constraint turns the problem from classification into ranking, and the ranking finds 30 illicit cases in the first 40 visits, the random order 14. At the seventh step the measurement policy changes, the label does not change, yet the same model falls to 23 and returns to 30 when rebuilt on the drifted data. The seven-step flow is completed with a total of 22 step visits. 15 min read
- 06 Roles The same illicit-use problem turns into three separate outputs and three separate baselines in three job definitions. The baseline of the data scientist is the field's current procedure and finds 13 illicit cases in the first 40 visits, the model order delivered finds 30, the gap is 17. The baseline of the machine learning engineer is the model running today and finds 23 on the drifted data, the rebuilt procedure finds 30, the gap is 7. The baseline of the AI engineer is the system without a model and finds 21 in the worst case, the system with a signal finds 30, the gap is 9. The same delivered count produces 17, 7 and 9 in three roles; what determines the number is not the model but the chosen baseline. The system that falls back gives 21 on the drifted data, the one that never falls back 23. 14 min read