Eigenvalues and Eigenvectors

August 11, 2025

An intuitive introduction to eigenvalues and eigenvectors, explained with the idea of direction and length change, plus real-world applications like PCA.

Linear AlgebraEigenvaluesEigenvectorsPCA

What Are Eigenvalues and Eigenvectors?

A matrix transforms vectors — it can rotate, stretch, squash, or flip them.
Most vectors change both their direction and length when multiplied by a matrix.

But there are special vectors that behave differently:
Their direction stays the same, and only their length changes.

These special vectors are called eigenvectors, and
the factor by which their length changes is called the eigenvalue.

The Mathematical Definition

An eigenvalue–eigenvector pair satisfies:

A\mathbf{v} = \lambda \mathbf{v}

$A$ : the matrix (transformation)
$\mathbf{v}$ : eigenvector
$\lambda$ : eigenvalue

Here, $\mathbf{v} \neq \mathbf{0}$ .
The equation says: Applying $A$ to $\mathbf{v}$ simply scales it by $\lambda$ .

How to Find Them

Start with the equation:

(A - \lambda I)\mathbf{v} = 0

For a non-trivial solution ( $\mathbf{v} \neq \mathbf{0}$ ), we require:

\det(A - \lambda I) = 0

This is called the characteristic equation.
Solving it gives the eigenvalues $\lambda$ .

Substitute each eigenvalue back into $(A - \lambda I)\mathbf{v} = 0$ to find the corresponding eigenvectors.

Example: A 2×2 Matrix

Let

A = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}

Characteristic equation:

\det\left( \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix} - \lambda \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \right) = 0

That is:

\det\begin{pmatrix} 2-\lambda & 1 \\ 1 & 2-\lambda \end{pmatrix} = 0

Expanding:

(2-\lambda)^2 - 1 = 0 \\ (2-\lambda)^2 = 1

So:

\lambda = 3 \quad \text{or} \quad \lambda = 1

For $\lambda = 3$ , solving $(A - 3I)\mathbf{v} = 0$ gives:

\mathbf{v} = \begin{pmatrix} 1 \\ 1 \end{pmatrix}

For $\lambda = 1$ , we get:

\mathbf{v} = \begin{pmatrix} 1 \\ -1 \end{pmatrix}

Geometric Meaning

$\lambda = 3$ with eigenvector $(1, 1)$ : the transformation triples the vector’s length but keeps its direction.
$\lambda = 1$ with eigenvector $(1, -1)$ : the length stays the same, direction unchanged.

Real-World Application: PCA

Eigenvalues and eigenvectors are not just theoretical — they are key tools in data science and machine learning.
A famous example is Principal Component Analysis (PCA).

PCA analyzes high-dimensional data to find the directions where the data varies the most.
These directions are:

Eigenvectors → directions of maximum variance (principal components)
Eigenvalues → amount of variance along those directions

In PCA, we compute the eigenvectors and eigenvalues of the covariance matrix of the data,
and choose the top eigenvectors (with the largest eigenvalues) to reduce dimensionality while keeping most of the information.

Interactive Demo

Eigenvalues and Eigenvectors Demo

Watch how regular vectors change both direction and length, while eigenvectors only change length!

Legend:

Regular vectors (direction and length change)

Eigenvectors (only length changes)

Transformation Matrix:

[3, 1]

[1, 3]

Eigenvalues:

λ₁ = 4(for eigenvector (1,1))

λ₂ = 2(for eigenvector (1,-1))

🧮 Try calculating by hand:

Matrix A × vector v₁ = [3,1; 1,3] × [1,0]

= [3×1+1×0, 1×1+3×0] = [3,1]

Matrix A × eigenvector e₁ = [3,1; 1,3] × [1,1]

= [3×1+1×1, 1×1+3×1] = [4,4] = 4×[1,1]

→ Direction unchanged, scaled by λ₁=4!

🔑 Key Insight:

When you apply the transformation, notice that the red eigenvectorskeep pointing in the same direction but change length by their eigenvalue factor. The gray regular vectors change both direction and length.

Demo idea:

Show multiple arrows (vectors) on a plane.
When a matrix is applied, all arrows move — except eigenvectors, which only change in length.
PCA mode: show a scatter plot of data points, highlight the principal components (eigenvectors), and scale them according to their eigenvalues.