What Is the Sum of Square Residuals and How Does It Measure Model Fit?

The sum of square residuals, written $SS_{\text{res}}$, measures the total squared difference between observed values and the values predicted by a regression model. For models fitted to the same data, a smaller $SS_{\text{res}}$ generally indicates a closer fit.

How Is It Calculated?

A residual is the vertical difference between an observed value $y_i$ and its predicted value $\hat{y}_i$:

$$e_i=y_i-\hat{y}_i$$

The residuals are squared and added:

$$SS_{\text{res}}=\sum_{i=1}^{n}(y_i-\hat{y}_i)^2$$

Squaring prevents positive and negative residuals from cancelling. It also gives greater weight to large prediction errors.

Suppose a model produces residuals $2$, $-1$, and $3$. Then

$$SS_{\text{res}}=2^2+(-1)^2+3^2=14.$$

If another model fitted to the same observations has residuals $1$, $-1$, and $1$, then its sum is $3$. The second model fits these data more closely because $3<14$.

However, the total alone does not show where the model fails. A residual plot helps identify structure: residuals scattered randomly around zero support the chosen model, while a curve or trend suggests that the model form is unsuitable. Because each residual is squared, one unusually large error can strongly increase the total. Therefore, inspect both the numerical value and the residual pattern before making a final judgement.

A common misconception is that the model with the smallest $SS_{\text{res}}$ is always the best model. Comparisons are meaningful only for the same response data and scale; model shape, context, extrapolation, and possible overfitting must also be considered. Unlike $R^2$, $SS_{\text{res}}$ is not standardized and has squared units.

Exam Technique

This is AHL 4.13 Non-linear regression, so it is HL-only and may appear on any GDC-active HL paper. Use technology to obtain $SS_{\text{res}}$, compare candidate models, and finish with a contextual judgement rather than merely stating which value is smaller.