Chapter 10

Inner Products & Orthogonality

00 · Symbol Glossary

$\langle \mathbf{u}, \mathbf{v} \rangle$angle brackets u v — inner product

The inner product of vectors $\mathbf{u}$ and $\mathbf{v}$ . Angle-bracket notation is standard for abstract inner products, distinguishing them from the specific dot product $\mathbf{u}\cdot\mathbf{v}$ . The dot product is one particular inner product, but not the only one.

$\mathbf{u} \perp \mathbf{v}$u perp v — orthogonal

$\mathbf{u}$ and $\mathbf{v}$ are orthogonal — their inner product is zero: $\langle \mathbf{u}, \mathbf{v} \rangle = 0$ . Orthogonality is the precise definition of "perpendicular" in any inner product space, including high-dimensional spaces where geometric angles are not directly visible.

$W^\perp$W perp — orthogonal complement

The set of all vectors orthogonal to every vector in subspace $W$ : $W^\perp = \{\mathbf{v} \mid \langle \mathbf{v}, \mathbf{w} \rangle = 0 \text{ for all } \mathbf{w} \in W\}$ . Together $W$ and $W^\perp$ partition $\mathbb{R}^n$ : every vector decomposes uniquely as $\mathbf{v} = \mathbf{w} + \mathbf{w}^\perp$ with $\mathbf{w} \in W$ , $\mathbf{w}^\perp \in W^\perp$ .

$\|\mathbf{v}\|_{A}$v norm A — A-weighted norm

The norm induced by a positive definite matrix $A$ : $\|\mathbf{v}\|_A = \sqrt{\mathbf{v}^T A \mathbf{v}}$ . In finance, $A = \Sigma^{-1}$ (precision matrix) gives the Mahalanobis norm, which accounts for correlations and scales units.

01 · From Dot Product to Inner Product

The dot product $\mathbf{u}\cdot\mathbf{v} = \sum u_i v_i$ works in $\mathbb{R}^n$ but fails to generalise to function spaces or weighted geometries. The inner product abstracts the properties that make the dot product useful — linearity, symmetry, positivity — into axioms satisfied by many different operations.

Definition — Inner Product

An inner product on a real vector space $V$ is a function $\langle \cdot, \cdot \rangle: V \times V \to \mathbb{R}$ satisfying:

IP1. Linearity: $\langle c\mathbf{u} + \mathbf{w}, \mathbf{v} \rangle = c\langle \mathbf{u}, \mathbf{v} \rangle + \langle \mathbf{w}, \mathbf{v} \rangle$ for all $c \in \mathbb{R}$ .

IP2. Symmetry: $\langle \mathbf{u}, \mathbf{v} \rangle = \langle \mathbf{v}, \mathbf{u} \rangle$ for all $\mathbf{u}, \mathbf{v}$ .

IP3. Positive definiteness: $\langle \mathbf{v}, \mathbf{v} \rangle \geq 0$ for all $\mathbf{v}$ , with equality $\iff \mathbf{v} = \mathbf{0}$ .

A vector space equipped with an inner product is an inner product space.

✓ Example — The Standard Dot Product

The dot product $\langle \mathbf{u}, \mathbf{v} \rangle = \mathbf{u}^T\mathbf{v} = \sum_{i=1}^n u_iv_i$ on $\mathbb{R}^n$ satisfies all three axioms:

IP1: $\langle c\mathbf{u}+\mathbf{w}, \mathbf{v} \rangle = (c\mathbf{u}+\mathbf{w})^T\mathbf{v} = c\mathbf{u}^T\mathbf{v} + \mathbf{w}^T\mathbf{v}$ — linear in first argument.

IP2: $\mathbf{u}^T\mathbf{v} = \mathbf{v}^T\mathbf{u}$ — symmetric (both equal $\sum u_iv_i$ ).

IP3: $\mathbf{v}^T\mathbf{v} = \sum v_i^2 \geq 0$ , and equals zero only when all $v_i=0$ — positive definite.

✓ Example — Weighted Inner Product

For a positive definite matrix $W \in \mathbb{R}^{n\times n}$ , the weighted dot product:

\langle \mathbf{u}, \mathbf{v} \rangle_W = \mathbf{u}^T W \mathbf{v}

is also a valid inner product. Taking $W = \Sigma^{-1}$ (inverse of a covariance matrix) gives the Mahalanobis inner product — used in portfolio optimisation and anomaly detection to measure distances that account for correlations between assets.

❌ What Breaks — Not Every Bilinear Form Is an Inner Product

Consider $f(\mathbf{u},\mathbf{v}) = u_1v_1 - u_2v_2$ on $\mathbb{R}^2$ . This is bilinear and symmetric, but fails IP3: take $\mathbf{v}=\begin{pmatrix}0\\1\end{pmatrix}$ , then $f(\mathbf{v},\mathbf{v}) = 0-1=-1 < 0$ . Negative value for a nonzero vector violates positive definiteness. This is the Minkowski metric from special relativity — a pseudo-inner-product, not an inner product.

02 · The Induced Norm and Cauchy-Schwarz

Every inner product induces a norm — a way to measure the "length" of a vector.

Definition — Induced Norm

Given an inner product $\langle \cdot, \cdot \rangle$ , the induced norm is:

\|\mathbf{v}\| = \sqrt{\langle \mathbf{v}, \mathbf{v} \rangle}

For the standard dot product, this recovers $\|\mathbf{v}\| = \sqrt{\sum v_i^2}$ — the Euclidean length from Chapter 1.

Definition — Cauchy-Schwarz Inequality

For any inner product space:

|\langle \mathbf{u}, \mathbf{v} \rangle| \leq \|\mathbf{u}\|\,\|\mathbf{v}\|

Equality holds $\iff \mathbf{u}$ and $\mathbf{v}$ are parallel ( $\mathbf{u} = c\mathbf{v}$ for some scalar $c$ ). Cauchy-Schwarz guarantees that $\frac{\langle \mathbf{u},\mathbf{v}\rangle}{\|\mathbf{u}\|\|\mathbf{v}\|} \in [-1, 1]$ , so the cosine angle formula $\cos\theta = \frac{\langle \mathbf{u},\mathbf{v}\rangle}{\|\mathbf{u}\|\|\mathbf{v}\|}$ is always valid.

Step-by-step — Verifying Cauchy-Schwarz for $\mathbf{u}=\begin{pmatrix}2\\-1\end{pmatrix}$, $\mathbf{v}=\begin{pmatrix}3\\4\end{pmatrix}$

Compute the inner product: $\langle \mathbf{u},\mathbf{v}\rangle = 2(3)+(-1)(4) = 6-4 = 2$ .

Compute the norms: $\|\mathbf{u}\| = \sqrt{4+1} = \sqrt{5}$ . $\|\mathbf{v}\| = \sqrt{9+16} = \sqrt{25} = 5$ .

$\|\mathbf{u}\|\|\mathbf{v}\| = 5\sqrt{5} \approx 11.18$ .

Check the inequality: $|\langle\mathbf{u},\mathbf{v}\rangle| = 2 \leq 5\sqrt{5} \approx 11.18$ ✓. The inequality holds with room to spare — the vectors are not parallel. Cosine of angle: $\cos\theta = \frac{2}{5\sqrt{5}} \approx 0.179$ , so $\theta \approx 79.7°$ .

03 · Orthogonality

Orthogonality — zero inner product — is the most important structural property in linear algebra. Orthogonal vectors contain no "shared direction," which makes computations separable.

Definition — Orthogonal Vectors

Vectors $\mathbf{u}, \mathbf{v}$ in an inner product space are orthogonal if:

\langle \mathbf{u}, \mathbf{v} \rangle = 0

Written $\mathbf{u} \perp \mathbf{v}$ .

Pythagorean theorem: if $\mathbf{u} \perp \mathbf{v}$ , then $\|\mathbf{u}+\mathbf{v}\|^2 = \|\mathbf{u}\|^2 + \|\mathbf{v}\|^2$ .

Proof: $\|\mathbf{u}+\mathbf{v}\|^2 = \langle\mathbf{u}+\mathbf{v},\mathbf{u}+\mathbf{v}\rangle = \|\mathbf{u}\|^2 + 2\langle\mathbf{u},\mathbf{v}\rangle + \|\mathbf{v}\|^2 = \|\mathbf{u}\|^2 + \|\mathbf{v}\|^2$ since $\langle\mathbf{u},\mathbf{v}\rangle = 0$ .

Definition — Orthogonal Set and Orthonormal Set

A set $\{\mathbf{v}_1, \ldots, \mathbf{v}_k\}$ is:

Orthogonal if $\langle \mathbf{v}_i, \mathbf{v}_j \rangle = 0$ for all $i \neq j$ .

Orthonormal if additionally $\|\mathbf{v}_i\| = 1$ for all $i$ :

\langle \mathbf{v}_i, \mathbf{v}_j \rangle = \delta_{ij} = \begin{cases}1 & i=j \\ 0 & i\neq j\end{cases}

$\delta_{ij}$ is the Kronecker delta — 1 when indices match, 0 otherwise.

✓ Example — Standard Basis Is Orthonormal

In $\mathbb{R}^3$ , the standard basis $\mathbf{e}_1=\begin{pmatrix}1\\0\\0\end{pmatrix}$ , $\mathbf{e}_2=\begin{pmatrix}0\\1\\0\end{pmatrix}$ , $\mathbf{e}_3=\begin{pmatrix}0\\0\\1\end{pmatrix}$ satisfies $\mathbf{e}_i\cdot\mathbf{e}_j = \delta_{ij}$ . Any two distinct standard basis vectors are orthogonal (dot product = 0); each has norm 1. It is the prototype orthonormal basis.

Orthogonal Sets Are Automatically Independent

Every orthogonal set of nonzero vectors is linearly independent. Suppose $c_1\mathbf{v}_1 + \cdots + c_k\mathbf{v}_k = \mathbf{0}$ . Take the inner product with $\mathbf{v}_j$ : all cross terms vanish by orthogonality, leaving $c_j\|\mathbf{v}_j\|^2 = 0$ . Since $\|\mathbf{v}_j\| \neq 0$ , we get $c_j = 0$ for each $j$ . This is why orthonormal bases are so powerful — independence is free.

04 · Orthogonal Complement

Definition — Orthogonal Complement

The orthogonal complement of a subspace $W \subseteq \mathbb{R}^n$ is:

W^\perp = \{ \mathbf{v} \in \mathbb{R}^n \mid \langle \mathbf{v}, \mathbf{w} \rangle = 0 \text{ for all } \mathbf{w} \in W \}

Key facts:

$W^\perp$ is itself a subspace.
$\dim(W) + \dim(W^\perp) = n$ .
Every $\mathbf{v} \in \mathbb{R}^n$ decomposes uniquely as $\mathbf{v} = \mathbf{w} + \mathbf{w}^\perp$ with $\mathbf{w} \in W$ , $\mathbf{w}^\perp \in W^\perp$ .
$(W^\perp)^\perp = W$ .

Step-by-step — Finding $W^\perp$ where $W = \text{span}\left\{\begin{pmatrix}1\\2\\3\end{pmatrix}\right\}$ in $\mathbb{R}^3$

Set up the orthogonality condition: $\mathbf{v} = \begin{pmatrix}x\\y\\z\end{pmatrix} \in W^\perp$ requires $\mathbf{v} \cdot \begin{pmatrix}1\\2\\3\end{pmatrix} = 0$ .

$x + 2y + 3z = 0$ .

Solve the linear equation: one equation, three unknowns — a two-dimensional solution space. Set $y=s$ , $z=t$ (free variables). Then $x = -2s - 3t$ .

General solution: $\mathbf{v} = s\begin{pmatrix}-2\\1\\0\end{pmatrix} + t\begin{pmatrix}-3\\0\\1\end{pmatrix}$ .

Identify the orthogonal complement:

W^\perp = \text{span}\left\{\begin{pmatrix}-2\\1\\0\end{pmatrix},\ \begin{pmatrix}-3\\0\\1\end{pmatrix}\right\}

$\dim(W)=1$ and $\dim(W^\perp)=2$ , summing to $n=3$ ✓. $W$ is a line through the origin; $W^\perp$ is the plane perpendicular to that line.

❌ What Breaks — The Complement of a Complement

A common error: "the complement of the $xy$ -plane in $\mathbb{R}^3$ is the $z$ -axis, so the complement of the $z$ -axis is not the $xy$ -plane." In fact $(W^\perp)^\perp = W$ exactly — complements are involutive. The error usually arises from confusing the orthogonal complement with a geometric "remainder" in the wrong sense. The identity $\dim(W)+\dim(W^\perp)=n$ with $(W^\perp)^\perp=W$ is exact.

05 · Orthonormal Bases and the Expansion Formula

When a basis is orthonormal, coordinates are computed by inner products — no matrix inversion required.

Definition — Orthonormal Basis Expansion

If $\{\mathbf{q}_1, \ldots, \mathbf{q}_n\}$ is an orthonormal basis for $\mathbb{R}^n$ , then every $\mathbf{v} \in \mathbb{R}^n$ has expansion:

\mathbf{v} = \langle \mathbf{v}, \mathbf{q}_1 \rangle \mathbf{q}_1 + \langle \mathbf{v}, \mathbf{q}_2 \rangle \mathbf{q}_2 + \cdots + \langle \mathbf{v}, \mathbf{q}_n \rangle \mathbf{q}_n = \sum_{i=1}^n \langle \mathbf{v}, \mathbf{q}_i \rangle \mathbf{q}_i

The coefficient of $\mathbf{q}_i$ is simply $\langle \mathbf{v}, \mathbf{q}_i \rangle$ — computed by a single inner product, no linear system needed. This is why orthonormal bases are computationally ideal.

If $Q$ is the matrix with columns $\mathbf{q}_1, \ldots, \mathbf{q}_n$ , then $Q^TQ = I_n$ (orthonormality) and $QQ^T = I_n$ (since $Q$ is square and full rank). A square matrix with orthonormal columns is called orthogonal and satisfies $Q^{-1} = Q^T$ — inversion is free.

06 · Quant Application — Orthogonal Risk Factors in PCA

In portfolio management, the covariance matrix $\Sigma$ mixes all assets' risks together. PCA changes to the eigenvector basis, producing orthogonal principal components (PCs).

Given $\Sigma = Q\Lambda Q^T$ with $\Lambda = \text{diag}(\lambda_1, \ldots, \lambda_p)$ , the PC scores of a return vector $\mathbf{r}$ are $\mathbf{f} = Q^T\mathbf{r}$ . The covariance of PC scores:

\text{Cov}(\mathbf{f}) = Q^T \Sigma Q = \Lambda

is diagonal. The PCs are orthogonal risk factors — knowing the value of one PC tells you nothing about the others (zero covariance). This makes risk attribution clean: the variance explained by PC $_k$ is exactly $\lambda_k$ , and the total variance is $\text{tr}(\Sigma) = \sum \lambda_k$ .

In practice, keeping only the top $k$ PCs (those with the largest $\lambda_i$ ) explains most variance with far fewer dimensions — dimensionality reduction without losing much information.

07 · Practice Exercises

EXERCISE 10.1

Compute $\langle \mathbf{u},\mathbf{v}\rangle_W = \mathbf{u}^T W \mathbf{v}$ by first multiplying $W\mathbf{v}$ , then dotting with $\mathbf{u}$ .

$W\mathbf{v} = \begin{pmatrix}2&1\\1&3\end{pmatrix}\begin{pmatrix}2\\1\end{pmatrix} = \begin{pmatrix}5\\5\end{pmatrix}$ .

$\langle\mathbf{u},\mathbf{v}\rangle_W = \mathbf{u}^T(W\mathbf{v}) = \begin{pmatrix}1&2\end{pmatrix}\begin{pmatrix}5\\5\end{pmatrix} = 5+10 = 15$ .

Verify symmetry: $W\mathbf{u} = \begin{pmatrix}2&1\\1&3\end{pmatrix}\begin{pmatrix}1\\2\end{pmatrix} = \begin{pmatrix}4\\7\end{pmatrix}$ . $\langle\mathbf{v},\mathbf{u}\rangle_W = \mathbf{v}^T(W\mathbf{u}) = \begin{pmatrix}2&1\end{pmatrix}\begin{pmatrix}4\\7\end{pmatrix} = 8+7=15$ ✓.

Let $W = \begin{pmatrix}2&1\\1&3\end{pmatrix}$ , $\mathbf{u}=\begin{pmatrix}1\\2\end{pmatrix}$ , $\mathbf{v}=\begin{pmatrix}2\\1\end{pmatrix}$ . Compute $\langle\mathbf{u},\mathbf{v}\rangle_W = \mathbf{u}^TW\mathbf{v}$ and verify it equals $\langle\mathbf{v},\mathbf{u}\rangle_W$ .

EXERCISE 10.2

Verify IP1 (linearity), IP2 (symmetry), and IP3 (positive definiteness) for $\langle f, g\rangle = \int_0^1 f(x)g(x)\,dx$ . For IP3, use the fact that $f^2(x) \geq 0$ .

IP1 (Linearity): $\langle cf+h, g\rangle = \int_0^1(cf+h)g\,dx = c\int_0^1 fg\,dx + \int_0^1 hg\,dx = c\langle f,g\rangle + \langle h,g\rangle$ ✓ (by linearity of integration).

IP2 (Symmetry): $\langle f,g\rangle = \int_0^1 f(x)g(x)\,dx = \int_0^1 g(x)f(x)\,dx = \langle g,f\rangle$ ✓ (multiplication of reals is commutative).

IP3 (Positive definiteness): $\langle f,f\rangle = \int_0^1 f(x)^2\,dx \geq 0$ since $f(x)^2 \geq 0$ for all $x$ . Equality holds $\iff f(x)=0$ for all $x \in [0,1]$ , i.e. $f$ is the zero function ✓.

Therefore $\langle f,g\rangle = \int_0^1 fg\,dx$ is an inner product on $C[0,1]$ .

Verify that $\langle f, g\rangle = \int_0^1 f(x)g(x)\,dx$ defines an inner product on the space $C[0,1]$ of continuous functions on $[0,1]$ by checking all three axioms IP1–IP3.

EXERCISE 10.3

Find $W^\perp$ by solving $\mathbf{v}\cdot\mathbf{w}_1 = 0$ and $\mathbf{v}\cdot\mathbf{w}_2 = 0$ simultaneously. The solution space is $W^\perp$ .

$W = \text{span}\left\{\begin{pmatrix}1\\0\\1\end{pmatrix},\begin{pmatrix}0\\1\\1\end{pmatrix}\right\}$ . Need $\mathbf{v}=\begin{pmatrix}x\\y\\z\end{pmatrix}$ satisfying:

Eq 1: $x+z=0 \Rightarrow z=-x$ .

Eq 2: $y+z=0 \Rightarrow y=-z=x$ .

Free variable: $x=t$ . Then $y=t$ , $z=-t$ .

W^\perp = \text{span}\left\{\begin{pmatrix}1\\1\\-1\end{pmatrix}\right\}

Check dimensions: $\dim(W)=2$ , $\dim(W^\perp)=1$ , sum $=3=n$ ✓.

Verify: $\begin{pmatrix}1\\1\\-1\end{pmatrix}\cdot\begin{pmatrix}1\\0\\1\end{pmatrix}=1+0-1=0$ ✓ and $\begin{pmatrix}1\\1\\-1\end{pmatrix}\cdot\begin{pmatrix}0\\1\\1\end{pmatrix}=0+1-1=0$ ✓.

Find $W^\perp$ where $W = \text{span}\left\{\begin{pmatrix}1\\0\\1\end{pmatrix}, \begin{pmatrix}0\\1\\1\end{pmatrix}\right\} \subseteq \mathbb{R}^3$ . Verify using the dimension formula $\dim(W)+\dim(W^\perp)=3$ .

EXERCISE 10.4

$\{\mathbf{q}_1,\mathbf{q}_2\}$ is orthonormal iff $\mathbf{q}_1\cdot\mathbf{q}_2=0$ and $\|\mathbf{q}_1\|=\|\mathbf{q}_2\|=1$ . Once orthonormal, the expansion coefficients are just dot products.

$\mathbf{q}_1 = \frac{1}{\sqrt{2}}\begin{pmatrix}1\\1\end{pmatrix}$ , $\mathbf{q}_2 = \frac{1}{\sqrt{2}}\begin{pmatrix}1\\-1\end{pmatrix}$ .

Orthogonality: $\mathbf{q}_1\cdot\mathbf{q}_2 = \frac{1}{2}(1\cdot1 + 1\cdot(-1)) = \frac{1}{2}(0)=0$ ✓.

Unit norms: $\|\mathbf{q}_1\|^2 = \frac{1}{2}(1+1)=1$ ✓. $\|\mathbf{q}_2\|^2 = \frac{1}{2}(1+1)=1$ ✓.

Expansion of $\mathbf{v}=\begin{pmatrix}3\\1\end{pmatrix}$ :

$c_1 = \langle\mathbf{v},\mathbf{q}_1\rangle = \frac{1}{\sqrt{2}}(3+1) = \frac{4}{\sqrt{2}} = 2\sqrt{2}$ .

$c_2 = \langle\mathbf{v},\mathbf{q}_2\rangle = \frac{1}{\sqrt{2}}(3-1) = \frac{2}{\sqrt{2}} = \sqrt{2}$ .

Verify: $c_1\mathbf{q}_1 + c_2\mathbf{q}_2 = 2\sqrt{2}\cdot\frac{1}{\sqrt{2}}\begin{pmatrix}1\\1\end{pmatrix} + \sqrt{2}\cdot\frac{1}{\sqrt{2}}\begin{pmatrix}1\\-1\end{pmatrix} = \begin{pmatrix}2\\2\end{pmatrix}+\begin{pmatrix}1\\-1\end{pmatrix}=\begin{pmatrix}3\\1\end{pmatrix}$ ✓.

Let $\mathbf{q}_1 = \frac{1}{\sqrt{2}}\begin{pmatrix}1\\1\end{pmatrix}$ , $\mathbf{q}_2 = \frac{1}{\sqrt{2}}\begin{pmatrix}1\\-1\end{pmatrix}$ . Verify this is an orthonormal basis of $\mathbb{R}^2$ , then expand $\mathbf{v}=\begin{pmatrix}3\\1\end{pmatrix}$ as $\mathbf{v}=c_1\mathbf{q}_1+c_2\mathbf{q}_2$ using the inner product formula for the coefficients.

EXERCISE 10.5

The matrix $Q$ has orthonormal columns iff $Q^TQ=I$ . Check this condition. Then verify $Q^{-1}=Q^T$ by computing both products.

$Q = \frac{1}{\sqrt{2}}\begin{pmatrix}1&1\\1&-1\end{pmatrix}$ .

$Q^TQ = \frac{1}{2}\begin{pmatrix}1&1\\1&-1\end{pmatrix}\begin{pmatrix}1&1\\1&-1\end{pmatrix} = \frac{1}{2}\begin{pmatrix}1+1&1-1\\1-1&1+1\end{pmatrix} = \frac{1}{2}\begin{pmatrix}2&0\\0&2\end{pmatrix} = I_2$ ✓.

$QQ^T = \frac{1}{2}\begin{pmatrix}1&1\\1&-1\end{pmatrix}\begin{pmatrix}1&1\\1&-1\end{pmatrix} = I_2$ ✓ (same calculation by symmetry).

Therefore $Q^{-1} = Q^T$ — inversion is just transposition. This makes orthogonal matrices computationally valuable: no Gaussian elimination needed to invert.

$\det(Q) = \frac{1}{\sqrt{2}}\cdot\frac{1}{\sqrt{2}}\cdot\det\begin{pmatrix}1&1\\1&-1\end{pmatrix} = \frac{1}{2}(-1-1) = -1$ . $|\det(Q)|=1$ — orthogonal matrices preserve volumes.

Let $Q = \frac{1}{\sqrt{2}}\begin{pmatrix}1&1\\1&-1\end{pmatrix}$ . Verify $Q^TQ = I$ and $QQ^T = I$ , confirming $Q$ is an orthogonal matrix with $Q^{-1}=Q^T$ . Also compute $\det(Q)$ and state what it means geometrically.

EXERCISE 10.6

In PC coordinates, the covariance is diagonal: $\text{Cov}(\mathbf{f})=\Lambda$ . The fraction of variance in PC $_k$ is $\lambda_k / \sum_i \lambda_i$ . Reconstruct the original returns approximately using only the top PCs.

$\Sigma = \begin{pmatrix}5&4\\4&5\end{pmatrix}$ .

Eigenvalues: $p(\lambda)=(5-\lambda)^2-16=\lambda^2-10\lambda+9=(\lambda-9)(\lambda-1)$ . So $\lambda_1=9$ , $\lambda_2=1$ .

Total variance: $\text{tr}(\Sigma) = 10$ .

PC1 explains $\frac{9}{10}=90\%$ of variance; PC2 explains $\frac{1}{10}=10\%$ .

Eigenvectors: For $\lambda_1=9$ : $(Σ-9I)=\begin{pmatrix}-4&4\\4&-4\end{pmatrix}$ ; $\mathbf{q}_1=\frac{1}{\sqrt{2}}\begin{pmatrix}1\\1\end{pmatrix}$ .

For $\lambda_2=1$ : $\mathbf{q}_2=\frac{1}{\sqrt{2}}\begin{pmatrix}1\\-1\end{pmatrix}$ (orthogonal to $\mathbf{q}_1$ ).

PC1 direction $\mathbf{q}_1 = \frac{1}{\sqrt{2}}\begin{pmatrix}1\\1\end{pmatrix}$ corresponds to the equally-weighted portfolio — the "market" factor. PC2 direction $\mathbf{q}_2=\frac{1}{\sqrt{2}}\begin{pmatrix}1\\-1\end{pmatrix}$ is long asset 1, short asset 2 — a "spread" factor. 90% of the portfolio's variance is in the market direction; only 10% is in the spread.

A two-asset portfolio has covariance matrix $\Sigma = \begin{pmatrix}5&4\\4&5\end{pmatrix}$ . Find the eigenvalues of $\Sigma$ , interpret each as a fraction of total variance, and identify which eigenvector direction corresponds to the "market" factor and which to the "spread" factor.

08 · Chapter Summary

Concept	Key Fact
Inner product	Bilinear, symmetric, positive definite — generalises dot product
Standard dot product	$\langle\mathbf{u},\mathbf{v}\rangle = \mathbf{u}^T\mathbf{v}$ — prototype inner product on $\mathbb{R}^n$
Weighted inner product	$\langle\mathbf{u},\mathbf{v}\rangle_W = \mathbf{u}^TWv$ ; $W$ positive definite
Induced norm	$\\|\mathbf{v}\\|=\sqrt{\langle\mathbf{v},\mathbf{v}\rangle}$
Cauchy-Schwarz	$\lvert\langle\mathbf{u},\mathbf{v}\rangle\rvert\leq\\|\mathbf{u}\\|\\|\mathbf{v}\\|$ ; equality iff parallel
Orthogonal vectors	$\langle\mathbf{u},\mathbf{v}\rangle=0$ ; Pythagorean theorem holds
Orthonormal set	All pairwise inner products zero, all norms one; $\langle\mathbf{v}_i,\mathbf{v}_j\rangle=\delta_{ij}$
Orthogonal complement	$W^\perp$ ; $\dim(W)+\dim(W^\perp)=n$
Orthonormal basis expansion	$\mathbf{v}=\sum_i\langle\mathbf{v},\mathbf{q}_i\rangle\mathbf{q}_i$ — no linear system needed
Orthogonal matrix	$Q^TQ=I$ ; $Q^{-1}=Q^T$ ; $\lvert\det Q\rvert=1$
PCA	$\Sigma=Q\Lambda Q^T$ ; PC scores have diagonal covariance $\Lambda$

Next: Chapter 11 — Gram-Schmidt Process shows how to construct an orthonormal basis from any linearly independent set, the computational foundation behind QR decomposition and numerical stability in least squares.