Linear regression predicts a continuous outcome by learning a linear relationship between one or more input features and a numerical target. It substitutes new input values into the learned equation to generate a predicted value.
In supervised learning, the model is trained using labelled examples containing input features and known outcomes. For one input feature, the linear regression model is:
Here, is the input feature, is the intercept, is the coefficient or slope, and is the predicted continuous outcome. With several features, the model becomes:
During training, the algorithm adjusts the coefficients to reduce a loss function. A common choice is mean squared error (MSE):
This penalizes large differences between actual values, , and predictions, . The fitted model is the line or hyperplane that minimizes this error for the training data.
| Stage | What happens |
|---|---|
| Training | Labelled data are used to estimate the intercept and coefficients. |
| Prediction | New feature values are substituted into the learned equation. |
| Evaluation | Predictions are compared with actual outcomes using measures such as MSE. |
For example, suppose study time predicts a test score using . For hours, the prediction is .
The difference between an actual value and its prediction is a residual. If the actual score were , the residual would be , and its squared error would be . Training combines these errors across all examples, while evaluation should use separate validation or test data to assess performance on unseen cases.
A common misconception is that linear regression predicts categories. It predicts continuous numerical values; classification models predict discrete classes. Predictions also become less reliable when extrapolating far beyond the training-data range.
In an IB Computer Science HL response on A4.3, identify linear regression as supervised learning, explain training with labelled data, interpret the coefficients, and show how the model produces and evaluates a prediction.
