Feature selection is the process of choosing the input variables that are most useful for a model and excluding irrelevant or redundant variables. It matters because suitable features can improve model performance, reduce computational cost, limit overfitting, and make results easier to interpret.
In IB Computer Science A4.2, feature selection is part of data preprocessing. A feature is an individual measurable input, usually represented by a column in a dataset. For example, a house-price model might use floor area, location, number of bedrooms, and property age as features.
Feature selection examines whether each variable contributes useful information. A unique property identification number is unlikely to help predict price and may introduce noise. Similarly, floor area measured in square metres and floor area measured in square feet are redundant because they encode the same information.
| Effect | Why it matters |
|---|---|
| Fewer irrelevant features | Reduces noise that may hide meaningful patterns |
| Fewer redundant features | Prevents duplicated information from dominating the model |
| Lower dimensionality | Reduces memory use and training time |
| Simpler model | Makes outputs easier to explain and evaluate |
| Reduced overfitting | Makes the model less likely to learn accidental patterns in training data |
A common misconception is that feature selection deletes unsuitable records. It does not: it removes or excludes features (columns), whereas data cleaning may remove incomplete, duplicated, or invalid records (rows).
Feature selection must also be performed carefully. Removing a genuinely predictive feature can reduce accuracy. Selection decisions should therefore be based on the training data and checked using validation data; using the test data to select features can cause data leakage and produce misleading performance results.
For an IB exam response, define feature selection, distinguish it from record cleaning, and explain at least one consequence. For an explain command term, link cause and effect: removing irrelevant inputs reduces noise, which can improve the model's ability to generalize to unseen data.
