What is a Linear Transformation?

By the end of this section you will see a linear transformation as a geometric reshaping of all of space that keeps the origin fixed, keeps grid lines straight and evenly spaced, and is fully determined by what it does to two basis vectors. No matrices required.

A Little Idea That Ate Geometry

Through the first half of the nineteenth century, people who worked with equations kept bumping into the same kind of object. A change of coordinates in astronomy would look like $x' = ax + by$, $y' = cx + dy$. A rotation in mechanics would look like that too. A projection onto a plane in surveying would look like that too. Different motivations, same four-number object. Nobody had a name for it yet.

In 1858, Arthur Cayley published a short paper called A Memoir on the Theory of Matrices and did something quietly radical. He stopped thinking of $x' = ax + by,\; y' = cx + dy$ as a pair of equations to be solved, and started thinking of it as a single object that takes a point in the plane and returns a new point. Two such objects can be composed to get a third. Some of them have inverses. Some collapse the plane onto a line. They have an algebra of their own.

That shift — from “a system of equations to solve” to “a transformation of space” — is the entire reason linear algebra exists as a field. The matrices came later, as bookkeeping. The object came first. That object is what this section is about, and we are going to meet it without writing down a single matrix.

The Mental Picture First

Before we define anything, play with the widget below. It applies various functions to the plane and shows what happens to grid lines.

Something striking is going on. The functions marked linear all do the same three things, no matter how different they look numerically.

One. The origin stays put. Whatever else the transformation does, the point $(0, 0)$ lands on $(0, 0)$.

Two. Grid lines stay straight. A straight line in the input plane maps to a straight line in the output plane. No curves, no kinks, no bending.

Three. Grid lines that were parallel and evenly spaced stay parallel and evenly spaced. The grid might be stretched, rotated, sheared, or even flattened onto a line — but it is still a regular grid of parallelograms.

The functions marked not linear break at least one of those rules. Translation slides the origin somewhere else. Squaring bends the grid into curves. Absolute value folds the plane along an axis. Whatever informal notion of “linear” the reader arrived with, those three visual invariants are the one that pays off.

The visual definition. A transformation of the plane is linear when it keeps the origin fixed, keeps lines straight, and keeps parallel lines parallel and evenly spaced. Everything else in this chapter is extracting that picture into symbols.

The Formal Definition, Earned

Here is how a mathematician writes down the pictures you just played with. Let $T: \mathbb{R}^2 \to \mathbb{R}^2$ be a function from the plane to itself. We call $T$ a linear transformation when, for every pair of vectors $\mathbf{u}, \mathbf{v} \in \mathbb{R}^2$ and every scalar $c \in \mathbb{R}$, both of the following hold.

Additivity. $T(\mathbf{u} + \mathbf{v}) = T(\mathbf{u}) + T(\mathbf{v})$. If you add two vectors first and then apply $T$, you get the same answer as applying $T$ to each one separately and then adding.

Homogeneity. $T(c\,\mathbf{u}) = c\,T(\mathbf{u})$. If you scale a vector first and then apply $T$, you get the same answer as applying $T$ first and then scaling.

You can fold both axioms into one statement: $T(a\mathbf{u} + b\mathbf{v}) = a\,T(\mathbf{u}) + b\,T(\mathbf{v})$ for all scalars $a, b$ and all vectors $\mathbf{u}, \mathbf{v}$. The slogan: T commutes with addition and scalar multiplication. Those two operations are the entire structure of a vector space, so another way to say it is that $T$ preserves the structure.

Consequences you get for free

The origin is fixed. Plug $c = 0$ into homogeneity: $T(0 \cdot \mathbf{u}) = 0 \cdot T(\mathbf{u})$, which says $T(\mathbf{0}) = \mathbf{0}$. There is no choice about this. Any transformation that moves the origin is not linear.

Two vectors tell you everything. Every vector in the plane can be written as $\mathbf{v} = x\,\mathbf{e}_1 + y\,\mathbf{e}_2$ where $\mathbf{e}_1 = (1, 0)$ and $\mathbf{e}_2 = (0, 1)$ are the standard basis vectors. Apply linearity:

$T(\mathbf{v}) = T(x\,\mathbf{e}_1 + y\,\mathbf{e}_2) = x\,T(\mathbf{e}_1) + y\,T(\mathbf{e}_2).$

Read that slowly. On the left, we do not know what $T$ does to an arbitrary $\mathbf{v}$. On the right, we only need two things: $T(\mathbf{e}_1)$ and $T(\mathbf{e}_2)$. The whole transformation on an infinite plane is compressed into two output vectors.

The two-vector theorem. A linear transformation $T: \mathbb{R}^2 \to \mathbb{R}^2$ is completely determined by the pair $(T(\mathbf{e}_1), T(\mathbf{e}_2))$. In Chapter 4 we will give that pair a name and learn to compute with it efficiently. For now, the pair itself is $T$.

Grid lines stay straight because the image of the line $\mathbf{a} + t\,\mathbf{d}$ is $T(\mathbf{a}) + t\,T(\mathbf{d})$ — a straight line through $T(\mathbf{a})$ in the direction $T(\mathbf{d})$. Parallel lines stay parallel because two lines in direction $\mathbf{d}$ both map to lines in direction $T(\mathbf{d})$. Equal spacing survives because homogeneity preserves the scalar $t$. Every picture is now a theorem.

Worked Example — By Hand

Let us fix a specific $T$ and compute with it the way a human would on paper. Choose $T(\mathbf{e}_1) = (2, 1)$ and $T(\mathbf{e}_2) = (-1, 1)$.

Those two outputs are all we have committed to. No formula, no matrix. Yet we can now compute $T(\mathbf{v})$ for any $\mathbf{v}$, just by using linearity.

Take $\mathbf{v} = (3, 2)$. Decompose it in terms of the basis: $\mathbf{v} = 3\,\mathbf{e}_1 + 2\,\mathbf{e}_2$. Apply linearity:

$T(\mathbf{v}) = 3\,T(\mathbf{e}_1) + 2\,T(\mathbf{e}_2) = 3(2, 1) + 2(-1, 1).$

Distribute the scalars: $3(2, 1) = (6, 3)$ and $2(-1, 1) = (-2, 2)$. Add the two: $(6, 3) + (-2, 2) = (4, 5)$. So $T(3, 2) = (4, 5)$. No matrix multiplication, no determinants. Just linearity, twice.

Drive the widget below to the state $T(\mathbf{e}_1) = (2, 1)$ (set $a = 2, c = 1$) and $T(\mathbf{e}_2) = (-1, 1)$ (set $b = -1, d = 1$). The blue arrow lands where our by-hand calculation says it should, and the grid shows the full reshape.

Notice: the origin stays at the origin, the light-colored grid lines have all become straight lines in the darker image grid, and every cell of the image grid is the same parallelogram. That is the visual fingerprint of linearity.

Python From Scratch

We now write $T$ as a Python function that takes a vector and returns its image. No NumPy, no matrices — just the two output vectors $T(\mathbf{e}_1)$ and $T(\mathbf{e}_2)$ and the linearity rule, exactly as you computed by hand.

1# T is defined by where it sends the two basis vectors.
2Te1 = (2.0, 1.0)
3Te2 = (-1.0, 1.0)
4
5def T(v):
6    x, y = v
7    out_x = x * Te1[0] + y * Te2[0]
8    out_y = x * Te1[1] + y * Te2[1]
9    return (out_x, out_y)
10
11v = (3.0, 2.0)
12Tv = T(v)
13print("T(v) =", Tv)
14
15u = (1.0, 1.0)
16w = (2.0, -1.0)
17c = 3.0
18
19u_plus_w = (u[0] + w[0], u[1] + w[1])
20lhs_add = T(u_plus_w)
21rhs_add = (T(u)[0] + T(w)[0], T(u)[1] + T(w)[1])
22print("Additivity:  T(u+w) =", lhs_add, " |  T(u)+T(w) =", rhs_add)
23
24cu = (c * u[0], c * u[1])
25lhs_hom = T(cu)
26rhs_hom = (c * T(u)[0], c * T(u)[1])
27print("Homogeneity: T(c*u) =", lhs_hom, " |  c*T(u) =", rhs_hom)

Click any line on the right to see the values flowing through it. Lines 7–8 are the linearity rule in pure arithmetic. The additivity and homogeneity checks at the bottom are numerical sanity — if either printed pair had mismatched, the function would not be linear.

The PyTorch Way

In production, almost nobody writes the above. Hand-written loops produce the right answer, but once you have a thousand vectors to transform, or a million, Python loops are the wrong tool. PyTorch lets us bundle $T(\mathbf{e}_1)$ and $T(\mathbf{e}_2)$ into a single $2 \times 2$ tensor whose columns are those images, and then apply $T$ to any vector with one multiplication.

That $2 \times 2$ tensor is what Chapter 4 will formally name the matrix of T. We are not there yet, so we treat the tensor purely as bookkeeping that packs the two output vectors into one object. The $@$ operator does the same two arithmetic lines the previous code did, but as one fused operation.

        col0  col1
row0:  [ 2.,  -1.]
row1:  [ 1.,   1.]
shape (2, 2)

A (10, 2) tensor. Values differ per run because randn is random, but the shape is always (10, 2).

Transpose of A — swaps rows and columns. A was [[2,-1],[1,1]], so A.T is [[2,1],[-1,1]].

A (10, 2) tensor. Row i of TV is T applied to row i of V. Ten applications of T in one instruction.

1import torch
2
3# Where the two basis vectors get sent.
4Te1 = torch.tensor([2.0, 1.0])
5Te2 = torch.tensor([-1.0, 1.0])
6
7# Stack the images as COLUMNS. In Ch. 4 we will call this the matrix of T.
8A = torch.stack([Te1, Te2], dim=1)
9print("A =")
10print(A)
11
12# Apply T to a vector with one multiplication: A @ v
13v = torch.tensor([3.0, 2.0])
14Tv = A @ v
15print("T(v) =", Tv)
16
17# Batched: send ten vectors at once
18V = torch.randn(10, 2)
19TV = V @ A.T
20print("TV shape:", TV.shape)

What PyTorch hid, and why that hiding matters. The two dot products are still happening on line 14 — PyTorch just computes them inside a C++ kernel, which on a GPU runs thousands of these side by side. When we scale up to layers of a neural network with thousands of neurons, the difference between a Python loop and a fused kernel is roughly four orders of magnitude. That is why the matrix bookkeeping is not a notational convenience. It is the thing that makes linear algebra computationally real.

Where Engineers Actually Use This: Italic Fonts

One thing your eyes are doing right now, whether you notice it or not, is reading the output of a linear transformation. Every vector font — TrueType, OpenType, PostScript Type 1, the fonts Apple and Microsoft ship with their operating systems — stores each letter as a set of outline points. When the rendering engine needs to draw that letter on your screen, it applies a $2 \times 2$ transformation to every one of those outline points before rasterizing. The PostScript language specification calls the four numbers the font matrix. They are exactly the two images $T(\mathbf{e}_1)$ and $T(\mathbf{e}_2)$ of our two-vector theorem.

The most recognizable use of the font matrix is synthetic italic. When a typeface does not ship with its own hand-drawn italic, graphics libraries synthesize one by shearing the upright glyphs: leave the horizontal direction alone, and tilt the vertical direction to the right by a fixed angle. The italic transformation sends $\mathbf{e}_1$ to $(1, 0)$ — horizontal stays horizontal — and sends $\mathbf{e}_2$ to $(\tan\theta, 1)$ where $\theta$ is the italic angle. For $\theta = 12^\circ$ this gives $T(\mathbf{e}_2) \approx (0.213, 1)$. That one pair of output vectors defines the italic transformation for the entire alphabet.

The demo is the same math as the worked example, with different numbers. Pull up the shear angle, and because $T$ is linear, every glyph outline point transforms consistently: the overall letter leans, but it does not bend, because straight lines stay straight. If the transformation were not linear — say, if the engine used $T(x, y) = (x + y^2, y)$ — the letter would warp instead of tilt, and typography would be ruined. Linear is a surprisingly strong word.

The same idea, expanded from $2 \times 2$ to $4 \times 4$ and promoted to three dimensions, is how every polygon in every 3D game gets positioned on your screen, how every photo gets scaled and rotated in an image editor, and how every robot arm figures out where its gripper is. Synthetic italic is the simplest honest example of a linear transformation doing real work, but it is not a toy — it ships on every font rendering stack in the world.

Pitfalls & What to Watch For

“Linear” does not mean $y = mx + b$

This is the single most common point of confusion coming from high-school algebra. The function $f(x) = mx + b$, whose graph is a straight line, is not a linear transformation unless $b = 0$. Plug $x = 0$ into it: $f(0) = b$, which is the origin only when $b = 0$. A genuine linear transformation must fix the origin. Functions that “look linear on a graph” but shift the origin are called affine. Neural network layers are affine (they add a bias), not linear, which matters the moment you start proving things about them.

Translation is not linear

Moving every point by a fixed vector $\mathbf{t}$ — the transformation $T(\mathbf{v}) = \mathbf{v} + \mathbf{t}$ — is the most useful non-linear operation in graphics. It feels linear. It is not. It moves the origin to $\mathbf{t}$. In Chapter 6 we will see how computer graphics gets around this with homogeneous coordinates, but for now the rule is hard: a shift is never linear.

The determinant can be zero

A linear transformation is free to collapse the plane onto a line, or even to a single point. The projection $T(x, y) = (x, 0)$ that flattens everything onto the x-axis is perfectly linear — it satisfies both axioms — but it is not invertible, because once you have flattened the plane you cannot recover the lost information. We will call such transformations singular, and the number that detects them is the determinant. Not every linear transformation can be undone.

Why the image of $(1,1)$ is not a third independent fact

Students sometimes ask: if I tell you $T(\mathbf{e}_1)$, $T(\mathbf{e}_2)$, and $T(1, 1)$, is that more information than giving the first two? It is not. By linearity $T(1, 1) = T(\mathbf{e}_1) + T(\mathbf{e}_2)$, so the third value is forced. Give any basis-pair's images and the rest of $T$ is pinned down.

What This Unlocks Next

You now own the object. In the next section, we catalogue the 2D linear transformations that matter in practice — rotations, scalings, shears, reflections, projections — and learn to recognize each one from the pair $(T(\mathbf{e}_1), T(\mathbf{e}_2))$ alone. Then Chapter 4 finally gives that pair the name it has been earning: the matrix of the transformation.

• It preserves vector addition: $T(\vec{u} + \vec{v}) = T(\vec{u}) + T(\vec{v})$.
• It preserves scalar multiplication: $T(c\,\vec{v}) = c\,T(\vec{v})$.

Geometrically, this means three things: the origin never moves, every straight line stays a straight line, and the spacing between parallel lines stays uniform. Under a linear transformation, the whole space gets stretched, squashed, rotated, sheared, or flipped — but never curved.

A Matrix IS a Transformation

Every linear transformation of $\mathbb{R}^n$ can be written as a matrix. The key idea is deceptively simple:

The columns of a matrix are the images of the standard basis vectors.

In 2D, the standard basis is $\hat{\imath} = (1, 0)$ and $\hat{\jmath} = (0, 1)$. If a matrix

represents a linear transformation, then its first column $(a, c)$ is where $\hat{\imath}$ lands, and its second column $(b, d)$ is where $\hat{\jmath}$ lands. That's the entire picture — once you know what happens to the two basis vectors, the rest of space follows because every vector is a linear combination of $\hat{\imath}$ and $\hat{\jmath}$.

2D: Play with a 2×2 Matrix

Type any $2 \times 2$ matrix below. The grid you see is the image of the integer lattice under that matrix — the original square grid is not drawn, so you're seeing pure output. The red arrow is where $\hat{\imath}$ lands (column 1); the green arrow is where $\hat{\jmath}$ lands (column 2). Try each preset in turn, then mix them.

Try the “Collapse” preset. It uses a matrix whose columns are parallel (one is a scalar multiple of the other), so the whole 2D plane gets squashed onto a single line. This is what it looks like when the determinant is zero: the transformation destroys a dimension.

3D: Play with a 3×3 Matrix

In three dimensions the story is identical — just with one more basis vector. A $3 \times 3$ matrix has three columns, one for each transformed basis vector:

Below you can set all nine entries. The purple wireframe is the image of the unit cube $[0,1]^3$ under the matrix — its volume equals $|\det A|$, and when the cube flattens to a plane or a line, you're watching $\det A = 0$ in action.

Ten Numbers That Tell the Story

A matrix is a machine that bends, stretches, rotates, squashes, or collapses space. People extract special numbers from that machine to understand what it does. No single number tells the full story — each one answers a different question. For each of the ten most-used invariants below you get: why we compute it, how it is computed with a concrete numerical walk-through, and an interactive viewer pre-loaded with a matrix that shows the behaviour at a glance.

Our running example for the non-degenerate cases will be $A = \begin{bmatrix} 3 & 1 \\ 0 & 2 \end{bmatrix}$. It shears, it stretches, and it keeps its two dimensions, so it is a good workhorse. For degenerate cases we'll switch to $B = \begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}$, whose second column is two times its first — it crushes the plane onto a single line.

Rank & Nullity — Dimensions Surviving vs. Lost

Why we compute it. Rank tells you how many independent output directions the matrix can still reach, and nullity tells you how many input directions it throws away. If rank is full, the matrix is invertible and nothing is lost. If rank drops, some entire subspace of inputs collapses to a single point — and that subspace is the null space. Rank is the first number you should ever look at: it is the sentence “does this transformation lose information?” in numeric form.

How it's computed. Reduce the matrix to row-echelon form and count the non-zero rows. For $B = \begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}$ subtract 2×row 1 from row 2:

One non-zero row ⇒ $\text{rank}(B) = 1$, so by the rank-nullity theorem $\text{nullity}(B) = 2 - 1 = 1$.

See it in action. The red dashed line in the viewer below is the null space — the entire line $y = -x/2$ of inputs getting squashed to the origin. Every amber grid line has collapsed onto a single line instead of covering the plane.

Determinant — Volume Scale Factor

Why we compute it. The determinant is the signed factor by which the transformation scales $n$-dimensional volume. Its sign tells you whether the space got flipped (a left hand becomes a right hand). Its magnitude tells you whether the transformation is expanding, compressing, or (at zero) collapsing. Along with rank, this is the single most useful scalar summary of a matrix.

How it's computed. For a $2 \times 2$ matrix,

Plug in $A = \begin{bmatrix} 3 & 1 \\ 0 & 2 \end{bmatrix}$:

So every area gets multiplied by 6. For the singular case $B = \begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}$: $\det B = (1)(4) - (2)(2) = 0$, so area becomes zero — exactly the collapse we saw under rank.

See it in action. The purple parallelogram is the image of the unit square. With A its area is exactly 6; with Flip X (next viewer) its area is 1 but orientation is reversed (you can see the parallelogram “vertices” go counterclockwise instead of clockwise).

Sign matters. Try the Flip X preset ($\begin{bmatrix} -1 & 0 \\ 0 & 1 \end{bmatrix}$). Det is $-1$: area scale factor 1 (unchanged), but the amber $\hat\imath'$ arrow now points left, flipping orientation.

Trace — Sum of Eigenvalues

Why we compute it. Trace is a cheap algebraic summary that turns out to equal the sum of eigenvalues, counted with multiplicity. It shows up everywhere: in probability as the expected value of a quadratic form, in mechanics as a conserved moment, in ML as the sum of diagonal variances. If you only have a second to look at a matrix, trace + determinant already narrows down a 2×2 matrix's behaviour to a small number of cases.

How it's computed. $\text{tr}(A) = a_{11} + a_{22} + \cdots + a_{nn}$. For our $A$,

We'll see in a moment that A's eigenvalues are 3 and 2, so the sum of eigenvalues = trace identity is instantly verified: $3 + 2 = 5 = \text{tr}(A)$. This is a cheap “sanity check” that your eigenvalue computation didn't go sideways.

Eigenvalues & Eigenvectors — Invariant Directions

Definition. Given a square matrix $A$, a non-zero vector $\vec{v}$ is called an eigenvector if applying $A$ keeps it on its own line — i.e. the output is a scalar multiple of $\vec{v}$ itself. The scalar is the eigenvalue $\lambda$. The defining equation is deceptively short:

Three things make this the most important pair of quantities derived from a matrix. First, an eigenvector is a direction the transformation does not bend — it only stretches or flips it. Second, any matrix that has enough independent eigenvectors can be rewritten in the eigenbasis, where it acts as a pure diagonal scaling — the cleanest possible description of a linear map. Third, when you apply $A$ repeatedly — to model a dynamical system, a Markov chain, or a deep network — the eigenvalues directly control growth, decay, oscillation, and stability.

Intuition — four ways to see it.

• Invariant lines. Most input directions get rotated by the matrix. Eigenvectors span the handful of lines that don't rotate — they only scale. A real $n \times n$ matrix can have up to $n$ of them (though sometimes fewer, or complex ones).
• Natural modes. Think of $A$ acting on a vibrating drum or a coupled spring system. The eigenvectors are the pure modes of vibration; the eigenvalues are their frequencies (or growth rates). Any motion decomposes into a sum of these independent modes.
• Diagonalisation. If $A$ has $n$ linearly independent eigenvectors, then $A = P D P^{-1}$, where $P$ has the eigenvectors as columns and $D$ is diagonal with the eigenvalues on it. That's the algebraic punchline: every diagonalisable matrix is secretly a diagonal matrix in disguise — and the eigenvectors are the rotation you apply to uncover it.
• Long-term behaviour. Applying $A$ repeatedly gives $A^k \vec{v}$. In the eigenbasis this becomes $D^k$, which is trivial: $\lambda_i^k$ on the diagonal. So whichever eigenvalue has the largest absolute value dominates; the trajectory pulls onto the corresponding eigenvector. (This is the iteration animation further down.)

How eigenvalues are computed. The equation $A\vec{v} = \lambda\vec{v}$ rearranges to $(A - \lambda I)\vec{v} = \vec{0}$. A non-zero $\vec{v}$ can satisfy this only when the matrix $A - \lambda I$ is singular — i.e. its determinant is zero. So:

For our running matrix $A = \begin{bmatrix} 3 & 1 \\ 0 & 2 \end{bmatrix}$,

Two roots:

How eigenvectors are computed. For each eigenvalue, solve $(A - \lambda I)\,\vec{v} = 0$. For $\lambda_1 = 3$:

For $\lambda_2 = 2$:

Sanity check: $A\vec{v}_1 = (3, 0) = 3\vec{v}_1$ ✓ and $A\vec{v}_2 = (2, -2) = 2\vec{v}_2$ ✓. A second cheap cross-check: $\lambda_1 + \lambda_2 = 5 = \text{tr}(A)$ and $\lambda_1\,\lambda_2 = 6 = \det(A)$. Those two identities always hold for a $2 \times 2$ matrix.

See it in action. The amber dashed lines in the viewer below are exactly those two eigenvector directions. The red $\hat\imath'$ arrow (image of $\hat\imath$) sits on the first amber line and is three times as long — that's $\lambda_1 = 3$. The direction $\vec{v}_2 = (1, -1)$ sits on the second amber line and gets scaled by 2. The whole transformation decomposes into “stretch by 3 along the first amber line and by 2 along the second” — diagonal in the eigenbasis.

Complex eigenvalues mean rotation. The 90° rotation matrix $R = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}$ has no real eigenvectors. Its characteristic polynomial is $\lambda^2 + 1 = 0$, giving $\lambda = \pm i$. The viewer below shows those complex eigenvalues in the Matrix-invariants panel, and no amber eigenvector lines appear — because every real direction gets rotated somewhere else. The magnitude $|\lambda| = 1$ tells you this is a pure rotation (no stretching); the argument $\arg\lambda = 90°$ tells you the rotation angle.

Watch eigenvectors emerge from iteration. The deepest reason eigenvectors matter is this: pick any starting vector and apply $A$ to it repeatedly. The result $A^k \vec{v}$ rotates toward the eigenvector whose eigenvalue has the largest absolute value, while its length grows like $|\lambda_1|^k$. This is called power iteration, and it's the engine behind PageRank (find the dominant eigenvector of the web's link matrix), PCA (find the dominant eigenvector of the data's covariance), and countless numerical algorithms.

The viewer below turns on iteration mode: click step → to apply $A$ once, and watch the pink trajectory $\vec{v} \to A\vec{v} \to A^2\vec{v} \to \ldots$ line up with the amber eigenvector line whose eigenvalue is biggest. The live ratio $\|A^{k+1}\vec{v}\| / \|A^k\vec{v}\|$ converges to $|\lambda_1|$.

Algebraic vs. geometric multiplicity. A matrix can have a repeated eigenvalue but still only one eigenvector direction — the classic example is the shear $\begin{bmatrix} 2 & 1 \\ 0 & 2 \end{bmatrix}$, whose only eigenvalue is $\lambda = 2$ (algebraic multiplicity 2) but whose eigenspace is the single line along $\hat\imath$ (geometric multiplicity 1). Such a matrix is called defective: you cannot diagonalise it, and it instead has a Jordan block structure. The viewer correctly shows only one amber line in that case — a useful signal that the transformation is hiding coupling beyond its eigenvalues.

When you'll reach for eigenvalues and eigenvectors.

What eigenvalues tell you about the transformation.

• $|\lambda| > 1$ on some eigenvector — that direction is amplified under repeated application.
• $|\lambda| < 1$ on all eigenvectors — the matrix is a contraction; iterations converge to the origin.
• $\lambda < 0$ — the direction is flipped at each application.
• Complex $\lambda = r e^{i\theta}$ — that pair of directions is rotated by angle $\theta$ and scaled by $r$. $r = 1$ is a pure rotation; $r > 1$ is a spiral out; $r < 1$ is a spiral in.
• $\lambda = 0$ — that eigenvector is the null direction: the matrix kills it. If present, the matrix is singular.

Together, the full spectrum of eigenvalues answers the deepest question you can ask about a linear map: over time, where does this transformation send things?

Singular Values — True Stretching Strengths

Why we compute them. Eigenvalues work best for well-behaved square matrices. Singular values work for every matrix — square, rectangular, defective — and they are always real and non-negative. They tell you the actual stretching strengths along orthogonal principal directions, which is what you need for PCA, for SVD-based compression, for solving least squares, and for defining matrix norms that behave properly.

How they're computed. Singular values are the square roots of the eigenvalues of $A^\top A$. For our A,

Solve $\det(A^\top A - \mu I) = 0$: $\mu^2 - 14\mu + 36 = 0 \implies \mu = 7 \pm \sqrt{13}$. So the singular values are

Sanity checks: (i) the product $\sigma_1 \sigma_2 = \sqrt{36} = 6 = |\det A|$ ✓; (ii) the sum of squares $\sigma_1^2 + \sigma_2^2 = 14 = \|A\|_F^2$ ✓. These identities link SVD back to the determinant and Frobenius norm.

See it in action. The viewer below turns on the singular ellipse: the dashed white unit circle is the set of all input unit vectors, and the solid blue ellipse is its image under A. The ellipse's long semi-axis has length $\sigma_1$, the short one $\sigma_2$ — every unit input produces an output whose length lies between those two numbers.

Spectral Norm — Maximum Amplification

Definition. The spectral norm (also called the operator 2-norm) of a matrix $A$ is the largest factor by which it can stretch the length of any vector:

Equivalently, it is the largest singular value of $A$, or the square root of the largest eigenvalue of $A^\top A$:

All three definitions give exactly the same number. The first is the honest meaning (“how much can this matrix amplify?”). The second and third are how you compute it from SVD or from eigenvalues of $A^\top A$.

Three ways to see it. 1. Worst-case amplifier. Feed every unit vector into$A$ and measure the output's length. The biggest number you find is $\|A\|_2$. Every output length lies between $\sigma_{\min}$ and $\sigma_{\max}$, so the spectral norm caps the amplification.

2. Geometric — the ellipse's longest reach. The unit sphere maps to an ellipsoid whose semi-axes are the singular values. The spectral norm is literally the length of the longest semi-axis. In the viewer below, it is the distance from the origin to the blue arrowtip.

3. Lipschitz constant. For any two inputs,

The matrix cannot separate two points by more than $\|A\|_2$ times their original distance. If $\|A\|_2 < 1$ the matrix is a contraction (everything gets closer). If $\|A\|_2 = 1$ it is an isometry or non-expansive map (rotations and reflections live here). If $\|A\|_2 > 1$ some direction gets amplified.

How it's computed. Take the largest singular value we already computed for $A = \begin{bmatrix} 3 & 1 \\ 0 & 2 \end{bmatrix}$:

Concrete sanity check: feed in $\hat\imath = (1, 0)$, get $(3, 0)$ — length 3, below the 3.26 bound. Try $\hat\jmath = (0, 1)$, get $(1, 2)$ — length $\sqrt{5} \approx 2.24$, also below. The only unit input that reaches the bound is the first right singular vector of A, aligned with the ellipse's long axis.

When you'll reach for it.

• Dynamical systems and stability. If a system evolves via $\vec{x}_{t+1} = A\vec{x}_t$, then $\|\vec{x}_t\| \leq \|A\|_2^t \cdot \|\vec{x}_0\|$. So $\|A\|_2 < 1$ guarantees the trajectory decays to zero (stable); $\|A\|_2 > 1$ can blow up. This is the linear version of Lyapunov stability.
• Gradient flow / exploding gradients in deep learning. Each layer of a neural network multiplies the gradient by some $W_i^\top$. If $\prod_i \|W_i\|_2 \gg 1$, gradients explode; if $\ll 1$, they vanish. Spectral normalization in GANs (Miyato et al., 2018) divides each weight matrix by its spectral norm to pin it to 1, ensuring 1-Lipschitz layers.
• Numerical error bounds. Solving $A\vec{x} = \vec{b}$ with a perturbed $\vec{b} + \Delta\vec{b}$ gives a relative output error bounded by $\kappa(A) \cdot \|\Delta\vec{b}\|/\|\vec{b}\|$ — and the condition number $\kappa(A) = \sigma_{\max}/\sigma_{\min}$ puts the spectral norm on top of the ratio.
• Robust control and $H^\infty$ design. In control theory the worst-case input-to-output amplification of a linear transfer function is its spectral norm; designing a controller that minimises $\|G\|_\infty$ directly bounds disturbance amplification.
• Approximation and compression. The best rank-k approximation of $A$ (the Eckart-Young theorem) has error exactly $\sigma_{k+1}$ in spectral norm — the (k+1)-th singular value. This is why SVD- based low-rank compression uses the spectral norm as its natural error measure.

What it tells you about the transformation. The spectral norm isolates one specific question: what is the worst thing this matrix does? If it is < 1, the matrix is shrinking space everywhere (and repeated application will drive everything to the origin). If it equals 1, the matrix at best preserves lengths — rotations, reflections, and orthogonal projections all have spectral norm exactly 1. If it is > 1, at least one direction is being actively amplified, and that amplification compounds under iteration. Knowing just this one number lets you reason about stability, convergence of iterative solvers, robustness of ML models, and the propagation of measurement error — without ever computing the full transformation.

See it in action. The viewer below reproduces the singular ellipse for $A = \begin{bmatrix} 3 & 1 \\ 0 & 2 \end{bmatrix}$. The solid blue arrow points from the origin to the farthest point on the ellipse — its length is the spectral norm ≈ 3.26. Every unit input vector lands somewhere on that blue ellipse, so no output ever exceeds that length.

Feel the stretch on a single vector. The viewer above shows the whole ellipse of possible outputs. The one below isolates one vector at a time: drag anywhere on the canvas (or type coordinates) to set an input v, and watch the amber arrow Av land wherever the matrix carries it. The panel reports $\|v\|$, $\|Av\|$, and the stretch ratio $\|Av\|/\|v\|$ — try spinning $v$ around the origin and notice the ratio bobbing between $\sigma_{\min}$ and $\sigma_{\max}$, never exceeding the spectral norm.

Frobenius Norm — Total Energy

Why we compute it. It is the simplest possible matrix norm: just the Euclidean length of the matrix treated as a vector of its entries. It's what you get when you compute an “L2 loss” over a matrix in ML, and it bounds the spectral norm from above, so if $\|A\|_F$ is small, the matrix can't do anything too dramatic in any direction.

How it's computed. $\|A\|_F = \sqrt{\sum_{i,j} a_{ij}^2}$. For A,

It equals $\sqrt{\sigma_1^2 + \sigma_2^2}$, so it pools the two stretching strengths. For our A, $\sqrt{3.26^2 + 1.84^2} \approx 3.74$ ✓.

Condition Number — Numerical Fragility

Why we compute it. When you solve $A\vec{x} = \vec{b}$, a small change in $\vec{b}$ can produce a change in $\vec{x}$ up to $\kappa(A)$ times larger. A condition number close to 1 means inputs and outputs change in lockstep. A huge condition number means small measurement errors can ruin the answer. In numerical linear algebra, this is the single most important diagnostic for “can I trust this computation?”

How it's computed. Just a ratio of singular values:

For our A:

That's well-conditioned — inputs and outputs stay roughly proportional. The viewer below shows an ill-conditioned matrix instead: one direction stretches enormously while the other barely moves, so $\sigma_1 / \sigma_2$ is huge. Watch the amber grid: it gets badly flattened — that squeezing is the geometric face of “ill-conditioned.”

Summary Table — Which Number Answers Which Question

Why so many numbers? Because each one answers a different question, and the matrix's behaviour is rich enough that no single summary captures it all. Rank tells you what's lost; the determinant tells you how volume scales; eigenvalues tell you the natural modes; singular values tell you real stretching strength; the condition number tells you whether you can trust the computation. Together they form a toolbox — and once you've played with enough matrices in the viewers above, you start to feel which one the situation calls for.