Chapter 08

Linear Transformations

00 — Symbol Glossary

01 — Definition of a Linear Transformation

Definition

A function $T:V\to W$ between vector spaces is a linear transformation (or linear map) if for all $\mathbf{u},\mathbf{v}\in V$ and all scalars $c$ :

Additivity: $T(\mathbf{u}+\mathbf{v})=T(\mathbf{u})+T(\mathbf{v})$
Homogeneity: $T(c\mathbf{v})=cT(\mathbf{v})$

Equivalently, both conditions together: $T(c\mathbf{u}+d\mathbf{v})=cT(\mathbf{u})+dT(\mathbf{v})$ for all scalars $c,d$ .

Note

Every $T(\mathbf{x})=A\mathbf{x}$ for a matrix $A$ is a linear transformation. The converse is also true for finite-dimensional spaces: every linear transformation between finite-dimensional spaces has a matrix representation.

Immediate consequences of linearity:

$T(\mathbf{0})=\mathbf{0}$ (the zero vector maps to zero).
$T(-\mathbf{v})=-T(\mathbf{v})$ .
$T(c_1\mathbf{v}_1+\cdots+c_k\mathbf{v}_k)=c_1T(\mathbf{v}_1)+\cdots+c_kT(\mathbf{v}_k)$ .

Example

Scaling: $T(\mathbf{x})=c\mathbf{x}$ for fixed scalar $c$ .
Rotation by $\theta$ : $T\begin{pmatrix}x\\y\end{pmatrix}=\begin{pmatrix}\cos\theta&-\sin\theta\\\sin\theta&\cos\theta\end{pmatrix}\begin{pmatrix}x\\y\end{pmatrix}$ .
Projection onto $x$ -axis: $T\begin{pmatrix}x\\y\end{pmatrix}=\begin{pmatrix}x\\0\end{pmatrix}$ .
Differentiation: $T(f)=f'$ is linear on the space of differentiable functions.

Common mistake

Wrong: $T\begin{pmatrix}x\\y\end{pmatrix}=\begin{pmatrix}x+1\\y\end{pmatrix}$ (translation) is linear because it "looks simple."
Why it happens: translations feel like "scaling by 1" but they violate additivity.
Correct: $T(\mathbf{0})=\begin{pmatrix}1\\0\end{pmatrix}\neq\mathbf{0}$ , so $T$ fails $T(\mathbf{0})=\mathbf{0}$ — not linear. Translations are affine maps, not linear.
Check: always verify $T(\mathbf{0})=\mathbf{0}$ as a quick necessary condition.

02 — Kernel and Image

Definition

Let $T:V\to W$ be a linear transformation.

Kernel (null space): $\ker(T)=\{\mathbf{v}\in V : T(\mathbf{v})=\mathbf{0}_W\}$ This is a subspace of the domain $V$ .

Image (range): $\text{Im}(T)=\{T(\mathbf{v}) : \mathbf{v}\in V\}$ This is a subspace of the codomain $W$ .

Example

$T:\mathbb{R}^3\to\mathbb{R}^2$ defined by $T(\mathbf{x})=A\mathbf{x}$ with $A=\begin{pmatrix}1&2&3\\4&5&6\end{pmatrix}$ .

Kernel: Solve $A\mathbf{x}=\mathbf{0}$ . Row reduce: $\begin{pmatrix}1&0&-1\\0&1&2\end{pmatrix}$ . Free variable $x_3=t$ ; then $x_1=t$ , $x_2=-2t$ . $\ker(T)=\text{span}\!\left\{\begin{pmatrix}1\\-2\\1\end{pmatrix}\right\}$ .

Image: $\text{Col}(A)=\text{span}\!\left\{\begin{pmatrix}1\\4\end{pmatrix},\begin{pmatrix}2\\5\end{pmatrix}\right\}=\mathbb{R}^2$ (since the two pivot columns span $\mathbb{R}^2$ ).

03 — Rank–Nullity Theorem

Definition

Let $T:V\to W$ be a linear transformation with $\dim(V)=n$ (finite). Then:

$\dim(\ker(T))+\dim(\text{Im}(T))=n$

$\text{nullity}(T)+\text{rank}(T)=n$

Example

For $T:\mathbb{R}^5\to\mathbb{R}^3$ with $\text{rank}(T)=3$ : $\text{nullity}(T)=5-3=2$ The kernel is 2-dimensional — there are 2 independent "input directions" that collapse to zero.

Note

The rank-nullity theorem is a conservation law for dimensions. It places hard limits on what a transformation can do: a map from $\mathbb{R}^5$ to $\mathbb{R}^3$ must collapse at least a 2-dimensional subspace.

Common mistake

Wrong: for $T:\mathbb{R}^5\to\mathbb{R}^3$ with $\text{rank}(T)=2$ , the image is $\mathbb{R}^3$ .
Why it happens: the codomain is $\mathbb{R}^3$ , so the image "should be" $\mathbb{R}^3$ .
Correct: the image is a 2-dimensional subspace of $\mathbb{R}^3$ (a plane through the origin), not all of $\mathbb{R}^3$ .
Check: $\dim(\text{Im}(T))=\text{rank}(T)$ , not $\dim(W)$ . Surjectivity ( $\text{Im}(T)=W$ ) is a separate condition.

04 — Matrix Representation of a Linear Transformation

Every linear $T:\mathbb{R}^n\to\mathbb{R}^m$ has a unique standard matrix $A\in\mathbb{R}^{m\times n}$ such that $T(\mathbf{x})=A\mathbf{x}$ .

How to find $A$ : the $j$ -th column of $A$ is $T(\mathbf{e}_j)$ , where $\mathbf{e}_j$ is the $j$ -th standard basis vector.

A = \begin{pmatrix} T(\mathbf{e}_1) & T(\mathbf{e}_2) & \cdots & T(\mathbf{e}_n) \end{pmatrix}

Find the matrix for $T:\mathbb{R}^2\to\mathbb{R}^2$: rotation by $90°$ counter-clockwise

Rotating $\begin{pmatrix}1\\0\end{pmatrix}$ by $90°$ counter-clockwise: the point $(1,0)$ maps to $(0,1)$ .
$T(\mathbf{e}_1)=\begin{pmatrix}0\\1\end{pmatrix}$ — this becomes column 1 of $A$ .

Rotating $\begin{pmatrix}0\\1\end{pmatrix}$ by $90°$ counter-clockwise: the point $(0,1)$ maps to $(-1,0)$ .
$T(\mathbf{e}_2)=\begin{pmatrix}-1\\0\end{pmatrix}$ — this becomes column 2 of $A$ .

$A=\begin{pmatrix}T(\mathbf{e}_1)&T(\mathbf{e}_2)\end{pmatrix}=\begin{pmatrix}0&-1\\1&0\end{pmatrix}$ — the general formula for rotation by $\theta$ is $\begin{pmatrix}\cos\theta&-\sin\theta\\\sin\theta&\cos\theta\end{pmatrix}$ ; at $\theta=90°$ : $\cos90°=0$ , $\sin90°=1$ .

$A\begin{pmatrix}1\\0\end{pmatrix}=\begin{pmatrix}0\\1\end{pmatrix}\,\checkmark$ ; $A\begin{pmatrix}0\\1\end{pmatrix}=\begin{pmatrix}-1\\0\end{pmatrix}\,\checkmark$ .

05 — Injective, Surjective, and Bijective Transformations

Definition

Let $T:V\to W$ .

Injective (one-to-one): $T(\mathbf{u})=T(\mathbf{v})\Rightarrow\mathbf{u}=\mathbf{v}$ . Equivalently, $\ker(T)=\{\mathbf{0}\}$ .
Surjective (onto): $\text{Im}(T)=W$ .
Bijective: both injective and surjective. For $T:\mathbb{R}^n\to\mathbb{R}^n$ , bijective $\iff$ $A$ is invertible $\iff$ $\det(A)\neq0$ .

$T:\mathbb{R}^n\to\mathbb{R}^m$	Injective condition	Surjective condition
$n>m$	Impossible (nullity $\geq n-m>0$ )	Possible
$n<m$	Possible	Impossible ( $\text{rank}\leq n<m$ )
$n=m$	$\det(A)\neq0$	$\det(A)\neq0$ (same condition)

06 — Composition and Invertibility

Definition

If $T:U\to V$ has matrix $A$ and $S:V\to W$ has matrix $B$ , then the composition $S\circ T:U\to W$ has matrix $BA$ :

$(S\circ T)(\mathbf{x})=S(T(\mathbf{x}))=B(A\mathbf{x})=(BA)\mathbf{x}$

Example

Rotate by $90°$ : $A=\begin{pmatrix}0&-1\\1&0\end{pmatrix}$ . Reflect across $x$ -axis: $B=\begin{pmatrix}1&0\\0&-1\end{pmatrix}$ .

Composition (rotate, then reflect): $BA=\begin{pmatrix}1&0\\0&-1\end{pmatrix}\begin{pmatrix}0&-1\\1&0\end{pmatrix}=\begin{pmatrix}0&-1\\-1&0\end{pmatrix}$ Note that $AB\neq BA$ — the order of composition matters.

07 — Quant Application — Factor Models as Linear Maps

A linear factor model for asset returns is exactly a linear transformation:

$\mathbf{r}=B\mathbf{f}+\boldsymbol{\epsilon}$

where $\mathbf{r}\in\mathbb{R}^n$ is the vector of $n$ asset returns, $\mathbf{f}\in\mathbb{R}^k$ is the vector of $k$ factor returns ( $k\ll n$ ), and $B\in\mathbb{R}^{n\times k}$ is the loading matrix.

The transformation $T:\mathbb{R}^k\to\mathbb{R}^n$ defined by $T(\mathbf{f})=B\mathbf{f}$ is linear. Its image $\text{Im}(T)=\text{Col}(B)$ is the factor subspace — the $k$ -dimensional slice of return space explained by the factors.

The kernel $\ker(T^\top)=\ker(B^\top)$ identifies directions in return space orthogonal to all factors — pure idiosyncratic risk.

Rank-nullity in practice: if $B$ has rank $k$ (full column rank), then the factor map is injective — each factor profile $\mathbf{f}$ maps to a distinct return vector. If two columns of $B$ are nearly collinear (rank deficient), two "different" factors have nearly the same effect — a sign of a misspecified model.

In the Barra / APT framework, PCA (eigendecomposition of $\Sigma$ ) chooses $B$ so that $\text{Col}(B)$ captures the maximum variance in $\mathbf{r}$ with only $k$ factors.

Exercises

EXERCISE 8.1

Check the two linearity conditions: $T(\mathbf{u}+\mathbf{v})=T(\mathbf{u})+T(\mathbf{v})$ and $T(c\mathbf{v})=cT(\mathbf{v})$ . If either fails for even one example, the map is not linear. Also check $T(\mathbf{0})=\mathbf{0}$ as a quick filter.

(a) $T\begin{pmatrix}x\\y\end{pmatrix}=\begin{pmatrix}2x-y\\x+3y\end{pmatrix}$ . This equals $\begin{pmatrix}2&-1\\1&3\end{pmatrix}\begin{pmatrix}x\\y\end{pmatrix}$ — a matrix multiplication. Linear.

(b) $T\begin{pmatrix}x\\y\end{pmatrix}=\begin{pmatrix}x^2\\y\end{pmatrix}$ . Check homogeneity: $T\begin{pmatrix}cx\\cy\end{pmatrix}=\begin{pmatrix}c^2x^2\\cy\end{pmatrix}\neq c\begin{pmatrix}x^2\\y\end{pmatrix}$ for $c\neq0,1$ . Not linear.

(c) $T\begin{pmatrix}x\\y\end{pmatrix}=\begin{pmatrix}x+2\\y-1\end{pmatrix}$ . $T(\mathbf{0})=\begin{pmatrix}2\\-1\end{pmatrix}\neq\mathbf{0}$ . Not linear (translation).

Determine which of the following maps $T:\mathbb{R}^2\to\mathbb{R}^2$ are linear:
(a) $T\begin{pmatrix}x\\y\end{pmatrix}=\begin{pmatrix}2x-y\\x+3y\end{pmatrix}$ , (b) $T\begin{pmatrix}x\\y\end{pmatrix}=\begin{pmatrix}x^2\\y\end{pmatrix}$ , (c) $T\begin{pmatrix}x\\y\end{pmatrix}=\begin{pmatrix}x+2\\y-1\end{pmatrix}$ .

EXERCISE 8.2

Evaluate $T$ on each standard basis vector $\mathbf{e}_1,\mathbf{e}_2,\mathbf{e}_3$ and place the results as columns of the matrix. Then use the matrix to compute $T(\mathbf{v})$ .

$T\begin{pmatrix}x\\y\\z\end{pmatrix}=\begin{pmatrix}x+2y\\3z-x\end{pmatrix}$ .

$T(\mathbf{e}_1)=T\begin{pmatrix}1\\0\\0\end{pmatrix}=\begin{pmatrix}1\\-1\end{pmatrix}$ ; $T(\mathbf{e}_2)=T\begin{pmatrix}0\\1\\0\end{pmatrix}=\begin{pmatrix}2\\0\end{pmatrix}$ ; $T(\mathbf{e}_3)=T\begin{pmatrix}0\\0\\1\end{pmatrix}=\begin{pmatrix}0\\3\end{pmatrix}$ .

Standard matrix: $A=\begin{pmatrix}1&2&0\\-1&0&3\end{pmatrix}$ .

$T\begin{pmatrix}2\\-1\\3\end{pmatrix}=A\begin{pmatrix}2\\-1\\3\end{pmatrix}=\begin{pmatrix}2-2+0\\-2+0+9\end{pmatrix}=\begin{pmatrix}0\\7\end{pmatrix}$ .

Find the standard matrix for $T:\mathbb{R}^3\to\mathbb{R}^2$ defined by $T\begin{pmatrix}x\\y\\z\end{pmatrix}=\begin{pmatrix}x+2y\\3z-x\end{pmatrix}$ , then compute $T\begin{pmatrix}2\\-1\\3\end{pmatrix}$ .

EXERCISE 8.3

For $T(\mathbf{x})=A\mathbf{x}$ : the kernel is $\text{Nul}(A)$ (row reduce $[A|\mathbf{0}]$ ); the image is $\text{Col}(A)$ (columns corresponding to pivots). Use rank-nullity to verify.

$A=\begin{pmatrix}1&-1&2\\2&-2&4\end{pmatrix}$ .

Row reduce: $R_2\leftarrow R_2-2R_1$ : $\begin{pmatrix}1&-1&2\\0&0&0\end{pmatrix}$ . One pivot (rank $=1$ ).

Kernel: Free variables $x_2=s$ , $x_3=t$ ; $x_1=x_2-2x_3=s-2t$ . $\ker(T)=\text{span}\!\left\{\begin{pmatrix}1\\1\\0\end{pmatrix},\begin{pmatrix}-2\\0\\1\end{pmatrix}\right\}$ . Nullity $=2$ .

Image: $\text{Col}(A)=\text{span}\!\left\{\begin{pmatrix}1\\2\end{pmatrix}\right\}$ . Rank $=1$ .

Rank-nullity check: $1+2=3=\dim(\mathbb{R}^3)\,\checkmark$ .

For $T:\mathbb{R}^3\to\mathbb{R}^2$ with matrix $A=\begin{pmatrix}1&-1&2\\2&-2&4\end{pmatrix}$ , find $\ker(T)$ and $\text{Im}(T)$ . Verify the rank-nullity theorem.

EXERCISE 8.4

$T$ is injective iff $\ker(T)=\{\mathbf{0}\}$ ; surjective iff $\text{rank}(A)=m$ (number of rows). Use the rank-nullity theorem and the dimensions to decide. A map from $\mathbb{R}^n$ to $\mathbb{R}^m$ with $n < m$ cannot be surjective.

$T:\mathbb{R}^2\to\mathbb{R}^3$ with $A=\begin{pmatrix}1&0\\0&1\\1&1\end{pmatrix}$ .

Rank: column 1 and column 2 are linearly independent (neither is a multiple of the other). Rank $=2$ .

Injective: nullity $=2-2=0$ , so $\ker(T)=\{\mathbf{0}\}$ . Yes, injective.

Surjective: $\text{rank}=2 < 3=\dim(\mathbb{R}^3)$ . Image is a 2-D plane in $\mathbb{R}^3$ , not all of $\mathbb{R}^3$ . Not surjective.

Geometric interpretation: $T$ embeds $\mathbb{R}^2$ as a plane $\{(x,y,x+y): x,y\in\mathbb{R}\}$ inside $\mathbb{R}^3$ .

Let $T:\mathbb{R}^2\to\mathbb{R}^3$ have matrix $A=\begin{pmatrix}1&0\\0&1\\1&1\end{pmatrix}$ . Is $T$ injective? Surjective? Justify.

EXERCISE 8.5

The composition $S\circ T$ has matrix $BA$ (apply $T$ first with matrix $A$ , then $S$ with matrix $B$ ). Compute the product, then find its kernel by row reduction.

$T:\mathbb{R}^2\to\mathbb{R}^2$ : $A=\begin{pmatrix}1&1\\0&1\end{pmatrix}$ (shear). $S:\mathbb{R}^2\to\mathbb{R}^2$ : $B=\begin{pmatrix}2&0\\0&2\end{pmatrix}$ (scaling by 2).

Matrix of $S\circ T$ : $BA=\begin{pmatrix}2&0\\0&2\end{pmatrix}\begin{pmatrix}1&1\\0&1\end{pmatrix}=\begin{pmatrix}2&2\\0&2\end{pmatrix}$ .

$\det(BA)=4\neq0$ , so $BA$ is invertible. $\ker(S\circ T)=\{\mathbf{0}\}$ . The composition is injective (and bijective since it's square).

Let $T:\mathbb{R}^2\to\mathbb{R}^2$ have matrix $\begin{pmatrix}1&1\\0&1\end{pmatrix}$ and $S:\mathbb{R}^2\to\mathbb{R}^2$ have matrix $\begin{pmatrix}2&0\\0&2\end{pmatrix}$ . Find the matrix of $S\circ T$ and the kernel of the composition.

EXERCISE 8.6

In a factor model $\mathbf{r}=B\mathbf{f}$ , the image of $B$ is the factor subspace. Rank of $B$ = number of independent factors. Use the rank-nullity theorem to find the idiosyncratic dimension. Then interpret: if two columns of $B$ are collinear, the model is misspecified.

$B=\begin{pmatrix}1&2&3\\0&1&2\\1&0&1\\2&1&1\end{pmatrix}$ ( $4$ assets, $3$ factors).

Row reduce $B$ : $R_3\leftarrow R_3-R_1$ , $R_4\leftarrow R_4-2R_1$ :

$\begin{pmatrix}1&2&3\\0&1&2\\0&-2&-2\\0&-3&-5\end{pmatrix}$ . $R_3\leftarrow R_3+2R_2$ , $R_4\leftarrow R_4+3R_2$ : $\begin{pmatrix}1&2&3\\0&1&2\\0&0&2\\0&0&1\end{pmatrix}$ . $R_4\leftarrow R_4-\tfrac{1}{2}R_3$ : $\begin{pmatrix}1&2&3\\0&1&2\\0&0&2\\0&0&0\end{pmatrix}$ .

Rank $=3$ . $B$ has full column rank — the 3 factors are independent.

The factor-space image $\text{Col}(B)$ is a 3-D subspace of $\mathbb{R}^4$ . Nullity of $B^\top$ (idiosyncratic dimension) $=4-3=1$ : one direction in asset return space is orthogonal to all factors — pure idiosyncratic risk.

If rank were 2 (e.g. two factors nearly collinear), the model would be over-parameterised: two "different" factors would explain the same variance, leading to unstable loading estimates.

A 3-factor model for 4 assets has loading matrix $B=\begin{pmatrix}1&2&3\\0&1&2\\1&0&1\\2&1&1\end{pmatrix}$ . Find the rank of $B$ , the dimension of the factor subspace, and the dimension of the idiosyncratic (factor-orthogonal) subspace. Interpret the result.

Chapter Summary

Concept	Formula / Rule
Linearity	$T(c\mathbf{u}+d\mathbf{v})=cT(\mathbf{u})+dT(\mathbf{v})$
Always true	$T(\mathbf{0})=\mathbf{0}$
Standard matrix	$A=\begin{pmatrix}T(\mathbf{e}_1)&\cdots&T(\mathbf{e}_n)\end{pmatrix}$
Kernel	$\ker(T)=\text{Nul}(A)$ ; subspace of domain
Image	$\text{Im}(T)=\text{Col}(A)$ ; subspace of codomain
Rank-nullity	$\text{rank}(T)+\text{nullity}(T)=\dim(V)$
Injective	$\ker(T)=\{\mathbf{0}\}$
Surjective	$\text{Im}(T)=W$
Bijective	Injective + surjective; $A$ invertible when square
Composition	$(S\circ T)(\mathbf{x})=(BA)\mathbf{x}$

Up next: Chapter 09 — Change of Basis, where we see how the matrix representation $[T]_\mathcal{B}^\mathcal{C}$ changes when we switch coordinate systems, and how diagonalisation is a special case of this.