Eigenvalues and Eigenvectors

Definition

In simple words

Imagine stretching a rubber sheet with an arrow drawn on it. Most arrows change direction as the sheet stretches. A few special arrows stay pointing the same way and only get longer or shorter. Those are eigenvectors, and how much they stretch is the eigenvalue.

Why it matters

PCA finds the main directions in data, and those are eigenvectors of the covariance matrix. Google's original PageRank is an eigenvector problem. They also explain how optimisation behaves and whether a repeated process settles or blows up.

How it works

1. Rewrite Av = λv as (A − λI)v = 0.
2. A non-zero v exists only when det(A − λI) = 0. Solve this for λ.
3. Put each λ back and solve (A − λI)v = 0 for v.

The maths

Find the eigenvalues of A = [[2, 1], [1, 2]].

det(A − λI) = (2−λ)(2−λ) − 1×1 = λ² − 4λ + 3 = (λ−1)(λ−3) = 0
So λ = 3 and λ = 1.

For λ = 3: (A − 3I)v = 0 gives v = [1, 1]
Check: A[1, 1] = [3, 3] = 3 × [1, 1]  ✓

In code

import numpy as np

A = np.array([[2, 1], [1, 2]])
values, vectors = np.linalg.eigh(A)

print("Eigenvalues:", values.round(3))
v = vectors[:, 1]
print("Eigenvector for 3:", v.round(3))
print("A times v:", (A @ v).round(3))
print("3 times v:", (3 * v).round(3))

Eigenvalues: [1. 3.]
Eigenvector for 3: [0.707 0.707]
A times v: [2.121 2.121]
3 times v: [2.121 2.121]

Common mistakes

• Trying this on a non-square matrix. Use singular value decomposition there.
• Expecting one exact eigenvector. Any non-zero multiple of it is also an eigenvector.

Check yourself

Summary

• Eigenvectors keep their direction under a matrix. Eigenvalues say how much they stretch.
• Find λ from det(A − λI) = 0, then v from (A − λI)v = 0.
• They power PCA, and help explain optimisation.