Loading subject…
Curse of dimensionality
The set of problems that appear as the number of features grows, because the space the data describes expands exponentially while the number of records does not.
Example
Splitting each of four features into ten bands gives 10,000 cells, so 3,200 sales average less than one sale per cell.
Dimensionality
The number of features, or columns, in a data set. It is counted separately from the number of records the data set holds.
Example
A table of 3,200 property sales recording bedrooms, bathrooms, floor area and postcode district has a dimensionality of four.
Dimensionality reduction
The process of reducing the number of variables in a data set while preserving the relevant aspects of the data.
Example
Cutting a property data set from 214 columns to the 12 that carry information about price is a dimensionality reduction.
Feature extraction
A route to fewer variables that builds new features by combining several original ones, so the data set narrows but the new features no longer correspond to a named original attribute.
Example
Blending floor area, room count and plot size into one new column reduces three features to one, and that column has no name or unit of its own.
Overfitting
A model that fits the data it was trained on so closely that it captures accidental patterns in that data and performs poorly on records it has not seen.
Example
A price model given 214 columns may split the training sales on wall colour, which says nothing about the price of the next house.