IB Computer Science Topic A4.3.1: Linear Regression in Predicting Continuous Outcomes Notes

Linear Regression: Fitting a Straight Line to Predict a Number

1. Linear regression predicts a continuous outcome, a number that can take any value in a range, such as a house price, a temperature or an exam score.
2. This is what separates it from classification, which predicts a discrete label such as "spam" or "not spam".
3. The model works by finding the straight line that comes closest to the data points, then reading predictions off that line.

The Two Kinds of Variable

1. The independent variable ($X$) is the predictor, the input you already know and feed into the model.
2. The dependent variable ($Y$) is the response, the outcome you want the model to produce.
3. The names carry an assumption: $X$ is treated as influencing $Y$, and $Y$ is not treated as influencing $X$.
4. In the house example, floor area is the independent variable and selling price is the dependent variable.
5. A model can take several independent variables at once, such as floor area, number of bedrooms and distance to a station, all predicting one price.

The Regression Equation and What Each Term Does

1. The simple form of the equation is: $$ Y = \beta_0 + \beta_1X + \epsilon $$
   1. $Y$ is the dependent variable the model predicts.
   2. $X$ is the independent variable supplied as input.
   3. $\beta_0$ is the intercept, the value of $Y$ when $X$ is zero.
   4. $\beta_1$ is the slope, the change in $Y$ for a one-unit increase in $X$.
   5. $\epsilon$ is the error term, the part of $Y$ that the straight line cannot account for.
2. Training the model means choosing values of $\beta_0$ and $\beta_1$ that make the errors as small as possible across the whole data set.

Reading the Slope and the Intercept

1. The slope $\beta_1$ tells you both the direction and the steepness of the relationship.
2. A positive slope means $Y$ rises as $X$ rises, and a negative slope means $Y$ falls as $X$ rises.
3. A slope of 1200 in a house-price model means each extra square metre adds 1200 to the predicted price, holding everything else fixed.
4. The intercept $\beta_0$ is where the regression line crosses the y-axis.
5. It is the model's prediction when every independent variable is zero.

Judging How Well the Line Fits: $r^2$

1. Model fit is how closely the predicted values sit to the actual observed values.
2. The gap between an observed value and the value the line predicts for it is called a residual.
3. The standard measure of fit is the coefficient of determination, written $r^2$, which reports the share of the variation in $Y$ that the model accounts for.
4. The formula is: $$ r^2 = 1 - \frac{SS_{res}}{SS_{tot}} $$
   1. $SS_{res}$ is the sum of squared residuals, which measures how far the observations fall from the regression line.
   2. $SS_{tot}$ is the total sum of squares, which measures how far the observations fall from the mean of $Y$.
5. An $r^2$ of 0 means the line explains none of the variation, so it is no better than always predicting the mean.
6. An $r^2$ of 1 means the line passes exactly through every data point.
7. An $r^2$ of 0.85 in the house-price model means floor area accounts for 85% of the variation in price, leaving 15% to other factors.

Where Linear Regression Is Used

1. Sales forecasting uses past advertising spend to predict next quarter's revenue.
2. Energy modelling uses outdoor temperature to predict a building's heating demand.
3. Sports analytics uses training load to predict an athlete's finishing time.
4. In each case the target is a number on a scale, which is the signal that regression rather than classification is the right tool.