What Does “Closest Line” Mean?

In simple linear regression, we fit a line of the form:

Here, $\beta_0$ is the intercept, $\beta_1$ is the slope, and $\varepsilon$ is the errors.
Given data points $(x_i, y_i)$, the goal of least squares is to find the line that is best “close” to all these points—in other words, the sum of squared vertical distances from each point to the line is minimized.

Defining the Objective Function

We measure quality by the sum of squared distances (errors):

Our task: find $\beta_0, \beta_1$ that minimize $S$.

Derivation: Formula for Slope and Intercept

Step 1: Partial Derivatives for Minimization

Take partial derivatives with respect to $\beta_0$ and $\beta_1$, and set them to zero.

With respect to $\beta_0$:

With respect to $\beta_1$:

Step 2: Normal Equations

From these two conditions we get:

1. $\sum y_i = n \beta_0 + \beta_1 \sum x_i$
2. $\sum x_i y_i = \beta_0 \sum x_i + \beta_1 \sum x_i^2$

Solving these, we obtain:

where $\bar{x} = \tfrac{1}{n} \sum x_i$ and $\bar{y} = \tfrac{1}{n} \sum y_i$.

Visual Demo of the Intuition

In the visualization below, you can drag the points around and see:

• The regression line updating in real time
• Vertical red lines showing distances from each point to the line
• As you adjust the line, notice that these red distances—and particularly their sum of squares—change

When the sum of squared red distances becomes minimized, that line is the least squares solution.

Summary

Least squares regression finds the line that is as close as possible to all data points, in the sense of minimizing the sum of squared vertical distances. The derivation may be algebraic, but the underlying concept is beautifully intuitive and easy to visualize with the demo above.