Clustering groups unlabeled data by identifying data points with similar features and placing them into the same cluster. It is an unsupervised learning approach because the training data has no predefined class labels.
The clustering mechanism
First, each data item is represented using selected features. For example, customers might be represented by age, annual spending, and number of purchases.
A clustering algorithm then applies a distance measure or similarity measure to compare data points. With numerical features, Euclidean distance is commonly used: points with a smaller distance are treated as more similar.
In k-means clustering, the process is:
- Select the required number of clusters, .
- Initialize points as cluster centres, called centroids.
- Assign each data point to its nearest centroid.
- Recalculate each centroid as the mean position of the points assigned to it.
- Repeat assignment and recalculation until the clusters no longer change significantly.
For example, if , customer data could be divided into a high-spending cluster and a low-spending cluster. These meanings are interpreted after clustering; the algorithm itself does not receive those labels.
| Factor | Effect on clustering |
|---|---|
| Feature selection | Irrelevant features may produce misleading groups. |
| Feature scaling | A feature with larger values may dominate the distance calculation. |
| Choice of | Too few clusters merge distinct groups; too many divide meaningful groups. |
| Initial centroids | Different starting positions can produce different final clusters. |
A common misconception is that clustering predicts known categories. It does not: classification learns from labeled examples, whereas clustering discovers possible structures within unlabeled data.
IB exam technique
For A4.3 Machine learning approaches, define clustering as unsupervised learning, explain how similarity determines membership, and describe an algorithmic process such as k-means. When asked to evaluate clustering, discuss feature choice, scaling, the selection of , and whether the resulting clusters are meaningful in context.
