Matrix Intuition for OLS

An interactive guide to understanding how matrix operations power linear regression

1. Matrix Multiplication Basics

Click any cell in the result matrix to see how it is computed. The animation has two steps: first you see which numbers pair up, then you see the computation.

A (2x3)
×
B (3x2)
=
C (2x2)
How it works: To get cell C[i,j], take row i from A and column j from B. Multiply each pair of numbers, then add all the products together. A (2×3) times B (3×2) gives C (2×2) because the inner dimensions (3) match and the outer dimensions (2×2) remain.

2. From Model to Formula: How OLS Works

A step-by-step visual derivation of the OLS formula. Use the buttons to walk through the algebra.

1
2
3
4
5
6

3. The OLS Building Blocks: X'X, X'y, and β

Walk through each step of computing OLS coefficients from real data. Click the step numbers to navigate.

1
2
3
4
5
6

4. Why Does Controlling for Correlation Matter?

Adjust the correlation between x1 and x2 and see how it affects the estimated coefficient on x1.

Imagine you want to measure the effect of advertising (x1) on sales. But advertising spending is correlated with store size (x2), and bigger stores also have more sales.

  • Simple regression (only x1): The coefficient on advertising captures BOTH its own effect AND part of the store-size effect, because the two are tangled together.
  • Multiple regression (x1 and x2): The coefficient on advertising captures ONLY its own effect, because we control for store size.

The slider below lets you see this in action. As the correlation between x1 and x2 increases, the simple regression coefficient becomes more biased — it picks up effects that actually belong to x2.


r = 0.00

Simple Regression (ignoring x2)

β1 = 2.00
True value: 2.00

Multiple Regression (controlling for x2)

β1 = 2.00
True value: 2.00
When x1 and x2 are uncorrelated (r = 0), the simple and multiple regression coefficients are identical. There is no confounding to adjust for.