IB Computer Science Topic A4.2.1: Significance of Data Cleaning Notes

Data Cleaning: A Model Is Capped by the Quality of Its Data

1. It runs after the data has been gathered and before any algorithm sees it, so every later stage inherits whatever is left uncorrected.
2. A fault that survives cleaning is not caught later: the model learns it as though it were a real pattern in the world.

The Impact of Data Quality on Model Performance

1. Accuracy and reliability: clean data lets a model fit the real relationship, while errors in the training data are learned as if they were genuine signal.
2. Generalization: a model trained on clean, representative data performs on unseen data, rather than only on the records it was fitted to.
3. Bias and fairness: a data set that under-represents a group produces a model that performs worse for that group, so cleaning includes checking who is in the data and who is missing from it.
4. Cost of failure: a fault found after deployment means retraining the whole model, which costs far more than correcting the data first.

Where Cleaning Sits in the Machine Learning Lifecycle

Cleaning is step three of a pipeline, and the seven steps that follow all depend on it.

1. Define the problem: identify the task and the desired outcome.
2. Gather data: collect relevant records from the available sources.
3. Data preprocessing: clean and prepare the data by handling outliers, missing values and formatting.
4. Exploratory data analysis: visualise the data to understand its patterns and relationships.
5. Feature engineering and selection: create and keep the attributes that carry the most information.
6. Choose a model: select an algorithm suited to the task and the data.
7. Split the data: divide the records into training and testing or validation sets.
8. Train the model: fit the model to the training data.
9. Model evaluation: measure performance on the test or validation data.
10. Hyperparameter tuning: adjust the settings that control learning to improve performance.
11. Deployment: put the model into a real-world environment.
12. Monitor and maintain: track performance over time and retrain when the data shifts.

Techniques for Data Cleaning

1. The syllabus names six problems a cleaning stage has to deal with, and each has its own set of fixes.
2. Choosing between the fixes is a judgement about how much data you can afford to lose.

Fixing Bad Values: Outliers, Duplicates and Incorrect Entries

1. Handling Outliers
   1. Outliers are data points that sit far away from the rest of the observations, and they pull a model's fit toward values that are not typical.
   2. Trim: remove the outlying record from the data set.
   3. Cap: replace the outlier with the nearest acceptable value, so the record is kept but its influence is limited.
   4. Transform: apply a log or square root transformation, which compresses extreme values toward the rest of the distribution.
   5. Not every outlier is an error, so check whether the value is a genuine rare case before deleting it.
2. Removing or Consolidating Duplicate Data
   1. Duplicates inflate the weight of whatever they record, so the model treats a repeated case as more common than it really is.
   2. Identify and remove: use a database query or dedicated software to find exact matches and delete the extra copies.
   3. Consolidate: merge partially varying duplicates by averaging the numerical fields or taking the most frequent category.
3. Identifying Incorrect Data
   1. Data validation: apply rules based on known ranges and formats, so an age of 200 or a date in 2099 is flagged automatically.
   2. Cross-referencing: check the values against an external source that is known to be correct.
   3. Anomaly detection: use a machine learning technique to surface records that do not fit the pattern of the rest.

Fixing Bad Structure: Irrelevant, Malformed and Missing Data

1. Filtering Irrelevant Data
   1. Columns that carry no information about the target add computation and noise without improving predictions.
   2. Domain expertise: ask someone who knows the field which fields cannot plausibly affect the outcome.
   3. Statistical ways of deciding which attributes to keep belong to feature selection, covered in A4.2.2.
2. Transforming Improperly Formatted Data
   1. Parsing: rewrite values into one consistent structure, such as storing every date as YYYY-MM-DD.
   2. Regular expressions: match a pattern in text and rewrite it, for example stripping currency symbols and commas from a price column so it can be read as a number.
3. Handling Missing Data
   1. Imputation: replace the gap with the mean, median or mode of that column.
   2. The median is safer than the mean when the column contains outliers, because the mean is dragged toward them.
   3. Deletion: drop the affected records, either listwise, which removes the whole record, or pairwise, which removes it only from the calculations that need the missing field.
   4. Predictive modelling: train a regression or decision tree on the complete records and use it to predict what each missing value should be.
   5. Every option costs something: deletion throws away real data, and imputation shrinks the natural variation in the column.

Normalization and Standardization: Putting Features on the Same Scale

1. Many algorithms compare features by their raw size, so a salary column measured in tens of thousands will drown out an age column measured in tens.
2. Rescaling removes that accident of units so each feature contributes according to its information rather than its magnitude.
3. Normalization
   1. Definition: rescales a feature into a fixed range, usually [0, 1] or [-1, 1].
   2. The min-max formula is x' = (x - min) / (max - min).
   3. Use cases: algorithms sensitive to input scale, such as gradient descent based methods, k-nearest neighbours and neural networks.
   4. It is vulnerable to outliers, because one extreme value sets the maximum and squashes every other record toward zero.
4. Standardization
   1. Definition: rescales a feature to have a mean of 0 and a standard deviation of 1.
   2. The z-score formula is z = (x - mean) / standard deviation.
   3. Use cases: data whose minimum and maximum are unknown, or data that still contains outliers, since the mean and standard deviation are less distorted by a single extreme value than the range is.

One Word, Two Unrelated Processes

1. Normalization in A4.2.1 means rescaling numeric features so they share a common range.
2. Normalization in A3.2.5 means restructuring a relational database into 1NF, 2NF and 3NF to remove partial and transitive dependencies.
3. The two share a name and nothing else, and an answer that mixes them loses the mark.

What Examiners Expect on Data Cleaning

1. A4.2.1 uses the command term Describe, so naming a technique earns little on its own; say what it does to the data.
2. Questions are set in a scenario, so tie the technique to it: "impute the missing blood pressure readings with the median" earns more than "handle missing data".
3. Where a technique has a cost, say so, because the trade between losing records and distorting a column is what a Describe answer at the top of the range shows.