Skip to content

Regression and least squares

Updated 11 Oct 2026

•
Regression
- •
  Regression -> models a numeric outcome using predictors and residual error.
- •
  Simple linear regression -> one predictor.
- •
  Multiple regression -> multiple predictors.
- •
  Use -> prediction or association analysis under assumptions.
- •
  Causality caution -> a fitted line alone does not establish causation.
•
Least-squares example
- •
  Data points -> $(1,2)$ , $(2,3)$ , $(3,5)$ , $(4,6)$ , $(5,8)$ .
- •
  Least-squares line -> $\hat{y}=0.3+1.5x$ .
- •
  Prediction at $x=6$ -> $9.3$ .
- •
  Residual -> observed value minus predicted value.
- •
  RMSE -> summarizes residual magnitude in outcome units.
•
Estimation
- •
  the numerical least-squares method minimizes the sum of squared residuals
- •
  Maximum likelihood -> chooses parameters that make observed data most likely under a specified model.
- •
  Gradient descent -> updates parameters iteratively to reduce an objective.
  - •
    Step size affects speed and stability.

Linked references 2

◌ Explore connections in Graph view