Skip to content
academia.sh
tr
Log in

Course Intermediate

Data Preparation and Feature Engineering

Published Updated

By the end of this course

Start course

01

Preparation Steps

Counting how much each decision taken on the existing columns moves the number on the held-out set while the model stays fixed: the contribution of the cleaning decision and its opposite signs in two model families, in which model family standardization and normalization produce a contribution and why the scale is fitted only on the training set, the column count and leakage of one-hot, ordinal and target encoding, the three separate numbers of imputation, the missingness indicator and dropping the row, the result that oversampling, undersampling and class weight give on an imbalanced set, the opposite effect of the same outlier on a gradient-based and a tree-based method.

  1. 01 Data Cleaning Measuring on the held-out set what a cleaning decision adds to the model: the three defects in the raw record are fixed one at a time, the model is held fixed, and two numbers on the test set are printed side by side. Fixing the unit mix-up gains the distance-based method twelve subscribers and costs the decision tree two; when all three defects are fixed together, the tree's test accuracy rises from 0.7659 to 0.7778 and the distance-based method's from 0.7540 to 0.7857, and the baseline is 0.7579 for both.
  2. 02 Feature Scaling Measuring the standardization and normalization decisions separately in three model families: standardization adds +0.0437 to the gradient-based linear model, gives −0.0159 on the distance-based method, and its contribution on the decision tree is exactly zero. The scale is an object fitted on the training set; applied to the test set, two values fall outside the [0, 1] range and standard scores rise as high as 5.11. When all eleven columns are scaled, the distance-based method loses one more subscriber.
  3. 03 Categorical Encoding Measuring three encodings as column count and contribution when a categorical field is turned into numbers: one-hot encoding the region raises test accuracy from 0.7302 to 0.7778, and the 240-code reading route field gives +0.0119 for 231 extra columns when one-hot encoded and the same +0.0119 with a single column when ordinal encoded. Target encoding inflates training accuracy to 0.9114 while dropping the test to 0.7421; computed in-fold, 0.0595 of the inflation on the training set recedes and the test returns to 0.7540.
  4. 04 Missing Value Strategies Measuring three strategies for a column with missing values separately on the held-out set: median imputation gives 0.7619, adding a missingness indicator next to the imputation gives 0.8333, and dropping the missing rows from the training set gives 0.7659; the contribution of the indicator is +0.0714, that is, eighteen subscribers. The indicator also exceeds by 0.0555 the upper bound of 0.7778 given by the column with nothing missing, because the missingness itself carries information that the imputed value does not: the suspicious rate is 0.5276 among subscribers who did not answer the survey and 0.1383 among those who did.
  5. 05 Imbalanced Classes Reading three answers to class imbalance with two measures at once: the baseline gives 0.7579 in accuracy while the number of suspicious subscribers it catches is zero; the model on the raw training set gives 0.7778 accuracy and catches 21 of 61 suspicious subscribers. Oversampling the minority twice lowers accuracy to 0.7619 and raises the caught count to 33, undersampling the majority gives 0.7143 and 36, and class weight 3.5 gives 0.6349 and 39. Oversampling the minority twice and class weight 2.0 produce the same prediction for all 252 subscribers in the test set.
  6. 06 Outlier Management Measuring that a single clipping decision gives opposite signs in two model families: clipping with a fence learned from the training set adds +0.0040 to the gradient-based linear model on real extremes only and +0.0317 when the stuck meter records are also present; the same decision gives −0.0278 and −0.0357 on the decision tree. The suspicious rate of the 34 subscribers outside the fence is 0.6765, so clipping also erases real information, and 41 values are clipped without deleting a single row.

02

Feature Generation and Selection

Counting what generating new columns, dropping columns and binding the preparation to an object add on the held-out set: the individual contribution of ratio, difference, interaction and threshold indicators and those whose contribution comes out zero, the number of numbers that come out of a date and a short text, the number of features chosen by filter, wrapper and embedded methods and the inflation produced when the selection is made from the whole set, the relation between the number of components and the held-out number and what is lost from interpretability, the number of steps learned from the whole set without a pipeline, the two cross-validation numbers given by preparing outside the folds and inside the folds.

  1. 01 Feature Generation Counting one by one what new columns derived from existing columns add on the held-out set: the ratio feature carries the test number from 0.7778 to 0.7897 while the difference and the indicator learned from the training set add 0.0040, the hand-set 25 m³ threshold loses 0.0159 and the product of two columns loses 0.0238, and the 40 m³ threshold produces exactly zero contribution. When the six columns are added cumulatively the training number rises from 0.8466 to 0.8677 while the held-out number peaks at 0.7897 and falls to 0.7698; the three columns that each add a positive contribution on their own add a total contribution of 0.0000 when added together.
  2. 02 Time and Text Features Extracting five numbers each from a date and from a short text and counting their total contributions on the held-out set: while the line before the decision is 0.7897, the five numbers from the date lose 0.0198, the five numbers from the note add 0.0317, and the ten numbers together add 0.0198. The whole contribution comes from a single indicator; the seal trace indicator alone adds 0.0357 while the text length loses 0.0357 and the word count loses 0.0516. How many numbers a field yields does not determine the contribution; the relation of the field to the label does.
  3. 03 Feature Selection Counting how many features the filter, wrapper and embedded methods select from the same 25 candidate columns, and the inflation produced by the selection itself: while the 25 columns without selection give 0.8135 on the test set, the filter gives 0.8056 with 10 columns, the wrapper 0.7579 with 4 columns and the embedded method 0.8254 with 14 columns. The wrapper reports 0.8413 on the validation set it reads its score from and drops to 0.7579 on the test set; the inflation is 0.0834. When the selection from 1,200 noise columns is made from the whole set, the held-out number inflates by 0.0198, 0.0040 and 0.0714, with the same sign in all three measurements.
  4. 04 Dimensionality Reduction as a Feature Tool Measuring what happens to the held-out number and to interpretability when a component-based representation replaces thirteen raw features: while the raw columns give 0.8254 on the test set, two components give 0.8175, three components 0.8056, five components 0.7778 and eight components 0.7460, and no number of components catches up with the raw representation. The component carrying the largest component share falls below the baseline on its own with 0.7381, while the fifth component with a share of 0.093 gives 0.7659. In the raw representation 0.35 of the splits are read from the mean consumption, while in the component representation the same share falls on a mixture with no name.
  5. 05 Pipelines Binding the preparation chain to one object and counting how many steps leak without a pipeline: when the four-step chain is applied by hand, all four are learned from the whole set and the test number rises from 0.7817 to 0.8294, an inflation of 0.0476. When imputation and scaling leak, the number does not change at all; when target encoding leaks it inflates by 0.0397, and when selection leaks as well it inflates by 0.0476. The pipeline learns 131 numbers, and all of them come from the training set; the contribution of the scaling step in the tree model is exactly zero, while the contribution of the target encoding step is minus 0.0317.
  6. 06 Fitting with Cross-Validation Comparing the two cross-validation numbers that come from learning the preparation chain outside the folds and inside the folds with the actual number on the held-out set: while the actual number on the test set is 0.7817, preparation inside the folds gives 0.7966 and preparation outside the folds gives 0.8105, and the deviation of preparation inside the folds is 0.0149 against 0.0288 for the one outside. The moment the second step of the chain, target encoding, moves outside the folds, the number jumps from 0.7966 to 0.8115. The 0.8254 given by the single split equals the highest of the four folds and is 0.0437 away from the actual number. In the course wrap-up, the before-decision and after-decision numbers of the twelve lessons are gathered in a single table.

Final exam

Data Preparation and Feature Engineering — Final Exam

In preparation

This course's question bank is not complete yet. The exam opens here automatically once enough questions are added.

Start typing to search.

↑↓ ↵ Esc navigate · open · close