How to read this
Stack the observations into one vector . Every fit is a combination of the columns of , so all fits form a plane: the subspace generated by the columns of . Least squares picks the point of the plane closest to , the foot of the perpendicular: , with the residual at a right angle to the plane. All these vectors lie in the 3D slice through , and , so the lengths and angles in the figure are the true ones.
- Look down on the plane: lands exactly on . That is the projection.
- Try another fit : every other is further from , by Pythagoras.
- Drag a point in the scatter plot: moves, the right angle stays.
- Remove the intercept: leaves the plane and no longer holds.
Model
Data
Simulated from , , uniform on .
Layers
Geometry: it is a simple projection
The same regression as a scatter plot
Drag a point. fitted line, residuals , , the line of . The squared length of in the figure is the sum of the squared red distances.