Chapter 03

Systems of Linear Equations

00 · Symbol Glossary

$A\mathbf{x} = \mathbf{b}$A x equals b — matrix equation

The compact form of a system of linear equations. $A$ is the coefficient matrix, $\mathbf{x}$ is the unknown vector, $\mathbf{b}$ is the right-hand side vector. Solving this equation is the central problem of linear algebra.

$[A \mid \mathbf{b}]$augmented matrix — A bar b

The matrix formed by appending $\mathbf{b}$ as an extra column to $A$ , separated by a vertical bar. Encodes the entire system in one object. Row operations on $[A \mid \mathbf{b}]$ transform the system without changing its solution set.

$x_1, x_2, \ldots, x_n$x sub 1 through x sub n — unknowns

The unknown real numbers to be determined. Subscripts label which position in the solution vector $\mathbf{x}$ . $x_1$ is the first unknown, $x_n$ the last. These are variables, not fixed values.

$\iff$if and only if — biconditional

Logical equivalence. $P \iff Q$ means $P$ is true exactly when $Q$ is true — both directions hold simultaneously. Used to state conditions that are both necessary and sufficient.

$\emptyset$empty set

The set with no elements. A system with no solution has solution set $\emptyset$ . An inconsistent system's solution set is the empty set — there is no vector $\mathbf{x}$ satisfying $A\mathbf{x} = \mathbf{b}$ .

01 · What is a System of Linear Equations

A system of linear equations is a collection of equations, each expressing a linear constraint on the same set of unknowns. "Linear" means each unknown appears at most to the first power, with no products of unknowns like $x_1 x_2$ .

A portfolio construction problem is a system: given target return, target risk, and a budget constraint, find the weights. Each constraint is one equation; the weights are the unknowns.

Definition — System of Linear Equations

A system of $m$ equations in $n$ unknowns $x_1, x_2, \ldots, x_n$ :

\begin{aligned} a_{11}x_1 + a_{12}x_2 + \cdots + a_{1n}x_n &= b_1 \\ a_{21}x_1 + a_{22}x_2 + \cdots + a_{2n}x_n &= b_2 \\ \vdots \qquad\qquad &\vdots \\ a_{m1}x_1 + a_{m2}x_2 + \cdots + a_{mn}x_n &= b_m \end{aligned}

$a_{ij}$ — the coefficient of unknown $x_j$ in equation $i$ .

$b_i$ — the right-hand side constant of equation $i$ .

A solution is an assignment of real values to $x_1, \ldots, x_n$ that satisfies all $m$ equations simultaneously.

✓ Example — Solving a $2 \times 2$ System by Substitution

Find $x_1, x_2$ satisfying:

\begin{aligned} 2x_1 + x_2 &= 5 \\ x_1 - x_2 &= 1 \end{aligned}

From equation 2: $x_1 = 1 + x_2$ . Substitute into equation 1: $2(1+x_2) + x_2 = 5 \implies 2 + 3x_2 = 5 \implies x_2 = 1$ . Then $x_1 = 1 + 1 = 2$ . Solution: $(x_1, x_2) = (2, 1)$ .

Verify: Equation 1: $2(2)+1 = 5$ ✓. Equation 2: $2-1 = 1$ ✓.

02 · Matrix Form $A\mathbf{x} = \mathbf{b}$

Every system of linear equations can be written as a single matrix equation $A\mathbf{x} = \mathbf{b}$ . This compact form is not just notation — it connects the system to every matrix tool from Chapter 2.

Definition — Matrix Form of a System

The system of $m$ equations in $n$ unknowns is equivalent to:

A\mathbf{x} = \mathbf{b} \quad\text{where}\quad A = \begin{pmatrix}a_{11}&\cdots&a_{1n}\\\vdots&\ddots&\vdots\\a_{m1}&\cdots&a_{mn}\end{pmatrix},\quad \mathbf{x} = \begin{pmatrix}x_1\\\vdots\\x_n\end{pmatrix},\quad \mathbf{b} = \begin{pmatrix}b_1\\\vdots\\b_m\end{pmatrix}

$A \in \mathbb{R}^{m \times n}$ — coefficient matrix.

$\mathbf{x} \in \mathbb{R}^n$ — unknown vector, the object to be found.

$\mathbf{b} \in \mathbb{R}^m$ — right-hand side vector of constants.

Step-by-step — Writing the System $\{x_1+2x_2-x_3=4,\ 3x_1-x_2+2x_3=1,\ 2x_1+x_2+x_3=7\}$ in Matrix Form

Read off the coefficient matrix $A$ : the coefficient of $x_j$ in equation $i$ goes in position $(i,j)$ . Equation 1 contributes row $(1, 2, -1)$ . Equation 2 contributes row $(3, -1, 2)$ . Equation 3 contributes row $(2, 1, 1)$ .

$A = \begin{pmatrix}1&2&-1\\3&-1&2\\2&1&1\end{pmatrix}$ .

Read off the right-hand side $\mathbf{b}$ : the constant from each equation in order. $\mathbf{b} = \begin{pmatrix}4\\1\\7\end{pmatrix}$ .

Write $\mathbf{x}$ : the unknown vector in the same order as the columns of $A$ . $\mathbf{x} = \begin{pmatrix}x_1\\x_2\\x_3\end{pmatrix}$ .

Verify the multiplication recovers the original system:

\begin{pmatrix}1&2&-1\\3&-1&2\\2&1&1\end{pmatrix}\begin{pmatrix}x_1\\x_2\\x_3\end{pmatrix} = \begin{pmatrix}x_1+2x_2-x_3\\3x_1-x_2+2x_3\\2x_1+x_2+x_3\end{pmatrix} = \begin{pmatrix}4\\1\\7\end{pmatrix} \checkmark

❌ Failure — Missing a Term Becomes a Zero, Not a Blank

The system $\{2x_1 + 3x_3 = 5,\ x_2 - x_3 = 2\}$ has two equations and three unknowns. Equation 1 has no $x_2$ term. The coefficient of $x_2$ in row 1 is $0$ , not missing:

$A = \begin{pmatrix}2&0&3\\0&1&-1\end{pmatrix}$ .

Leaving a blank or writing $A = \begin{pmatrix}2&3\\1&-1\end{pmatrix}$ (a $2 \times 2$ matrix) misrepresents the system as having only 2 unknowns. Every unknown must have a column, even if its coefficient is zero in some equations.

03 · The Augmented Matrix

To solve $A\mathbf{x} = \mathbf{b}$ , we work simultaneously on $A$ and $\mathbf{b}$ . The augmented matrix $[A \mid \mathbf{b}]$ combines them into one object.

Definition — Augmented Matrix

For $A \in \mathbb{R}^{m \times n}$ and $\mathbf{b} \in \mathbb{R}^m$ , the augmented matrix is:

[A \mid \mathbf{b}] = \left(\begin{array}{cccc|c} a_{11}&a_{12}&\cdots&a_{1n}&b_1 \\ a_{21}&a_{22}&\cdots&a_{2n}&b_2 \\ \vdots&\vdots&\ddots&\vdots&\vdots \\ a_{m1}&a_{m2}&\cdots&a_{mn}&b_m \end{array}\right)

The vertical bar separates the coefficient columns from the right-hand side column. Row operations on $[A \mid \mathbf{b}]$ transform the system into an equivalent one with the same solution set.

Three elementary row operations preserve the solution set:

Definition — Elementary Row Operations

R_i \leftrightarrow R_j \quad\quad cR_i \to R_i\ (c \neq 0) \quad\quad R_i + cR_j \to R_i\ (i \neq j)

$R_i \leftrightarrow R_j$ — swap rows $i$ and $j$ .

$cR_i \to R_i$ — multiply row $i$ by a nonzero scalar $c$ . (Multiplying by zero would destroy information.)

$R_i + cR_j \to R_i$ — replace row $i$ with itself plus $c$ times row $j$ . (Row $j$ is unchanged.)

✓ Example — Row Operations on the Augmented Matrix

System: $x_1 + 2x_2 = 5$ , $2x_1 - x_2 = 0$ .

Augmented matrix: $\left(\begin{array}{cc|c}1&2&5\\2&-1&0\end{array}\right)$ .

Apply $R_2 \to R_2 - 2R_1$ : subtract 2 times row 1 from row 2.

Row 1 unchanged: $(1, 2, 5)$ . Row 2: $(2-2\cdot1,\ -1-2\cdot2,\ 0-2\cdot5) = (0, -5, -10)$ .

New matrix: $\left(\begin{array}{cc|c}1&2&5\\0&-5&-10\end{array}\right)$ .

Apply $R_2 \to -\frac{1}{5}R_2$ : multiply row 2 by $-\tfrac{1}{5}$ .

Row 2: $(0, 1, 2)$ . Matrix: $\left(\begin{array}{cc|c}1&2&5\\0&1&2\end{array}\right)$ .

From row 2: $x_2 = 2$ . Back-substitute into row 1: $x_1 + 2(2) = 5 \implies x_1 = 1$ . Solution: $(1, 2)$ .

04 · Types of Solutions

A system $A\mathbf{x} = \mathbf{b}$ has exactly one of three outcomes. There is no fourth possibility.

Definition — Solution Types

\text{System } A\mathbf{x} = \mathbf{b} \text{ has:}\begin{cases} \text{exactly one solution} & \text{(consistent, unique)} \\ \text{infinitely many solutions} & \text{(consistent, underdetermined)} \\ \text{no solution} & \text{(inconsistent)} \end{cases}

Consistent — there exists at least one solution.

Inconsistent — no solution exists; the equations contradict each other.

The geometry: in $\mathbb{R}^2$ , two equations define two lines. Unique solution = lines intersect at one point. Infinite solutions = lines are identical (same equation). No solution = lines are parallel but distinct.

Step-by-step — Classifying the Solution of $\{x_1 - x_2 = 2,\ 2x_1 - 2x_2 = 3\}$

Form the augmented matrix: $\left(\begin{array}{cc|c}1&-1&2\\2&-2&3\end{array}\right)$ .

Apply $R_2 \to R_2 - 2R_1$ : row 2 becomes $(2-2\cdot1,\ -2-2\cdot(-1),\ 3-2\cdot2) = (0,\ 0,\ -1)$ .

New matrix: $\left(\begin{array}{cc|c}1&-1&2\\0&0&-1\end{array}\right)$ .

Read row 2: $0 \cdot x_1 + 0 \cdot x_2 = -1$ , which simplifies to $0 = -1$ . This is a contradiction — no values of $x_1, x_2$ can make $0 = -1$ true.

Conclusion: The system is inconsistent. Solution set $= \emptyset$ . Geometrically, the two lines are parallel (same direction vector $(1,-1)$ ) but pass through different points, so they never intersect.

Step-by-step — Solving $\{x_1 + x_2 + x_3 = 6,\ x_1 + 2x_2 + 3x_3 = 10\}$

Form the augmented matrix: $\left(\begin{array}{ccc|c}1&1&1&6\\1&2&3&10\end{array}\right)$ . Two equations, three unknowns — expect infinitely many solutions.

Apply $R_2 \to R_2 - R_1$ : $(1-1,\ 2-1,\ 3-1,\ 10-6) = (0,\ 1,\ 2,\ 4)$ .

Matrix: $\left(\begin{array}{ccc|c}1&1&1&6\\0&1&2&4\end{array}\right)$ .

Apply $R_1 \to R_1 - R_2$ : $(1-0,\ 1-1,\ 1-2,\ 6-4) = (1,\ 0,\ -1,\ 2)$ .

Matrix: $\left(\begin{array}{ccc|c}1&0&-1&2\\0&1&2&4\end{array}\right)$ .

Identify free variable: $x_3$ has no leading 1 in its column — it is a free variable, assigned any real value $t \in \mathbb{R}$ .

Express solution: From row 1: $x_1 = 2 + t$ . From row 2: $x_2 = 4 - 2t$ . $x_3 = t$ .

\mathbf{x} = \begin{pmatrix}2\\4\\0\end{pmatrix} + t\begin{pmatrix}1\\-2\\1\end{pmatrix}, \quad t \in \mathbb{R}

Infinitely many solutions — one for each value of

t

❌ Failure — Treating a Free Variable as Determined

In the system above, $x_3 = t$ is free — it can be any real number. Arbitrarily setting $x_3 = 0$ gives one particular solution $(2, 4, 0)$ but misses the entire family. If asked "find all solutions," writing only $(2, 4, 0)$ is incomplete — the answer is an infinite set parameterised by $t$ .

05 · Homogeneous Systems

The system $A\mathbf{x} = \mathbf{0}$ (right-hand side all zeros) is called homogeneous. It always has at least one solution: $\mathbf{x} = \mathbf{0}$ (plug in zero — every equation gives $0 = 0$ ). The interesting question is whether there are other solutions.

Definition — Homogeneous System and Null Space

The system $A\mathbf{x} = \mathbf{0}$ is homogeneous. Its solution set:

\text{Null}(A) = \{\mathbf{x} \in \mathbb{R}^n : A\mathbf{x} = \mathbf{0}\}

is called the null space (or kernel) of $A$ . It is always a subspace of $\mathbb{R}^n$ — it contains $\mathbf{0}$ , is closed under addition, and closed under scalar multiplication.

$A\mathbf{x} = \mathbf{0}$ has only the trivial solution $\mathbf{x} = \mathbf{0}$ $\iff$ $\text{Null}(A) = \{\mathbf{0}\}$ .

✓ Example — Non-trivial Null Space

$A = \begin{pmatrix}1&2\\2&4\end{pmatrix}$ . Row 2 is exactly $2 \times$ row 1 — the rows are linearly dependent.

Augmented: $\left(\begin{array}{cc|c}1&2&0\\2&4&0\end{array}\right)$ . Apply $R_2 \to R_2 - 2R_1$ : row 2 becomes $(0,0,0)$ .

Matrix: $\left(\begin{array}{cc|c}1&2&0\\0&0&0\end{array}\right)$ . Free variable $x_2 = t$ . Row 1: $x_1 = -2t$ .

$\text{Null}(A) = \left\{t\begin{pmatrix}-2\\1\end{pmatrix} : t \in \mathbb{R}\right\}$ — a line through the origin. Every point on this line satisfies $A\mathbf{x} = \mathbf{0}$ .

06 · Practice Exercises

EXERCISE 3.1

Write the coefficient of each unknown ( $x_1$ , $x_2$ , $x_3$ ) in its column position. If a variable is absent in an equation, its coefficient is 0.

Equation 1: $2x_1 - x_2 + 3x_3 = 7$ gives row $(2, -1, 3)$ with right-hand side $7$ .

Equation 2: $x_1 + 4x_3 = -2$ — no $x_2$ term, so coefficient of $x_2$ is 0. Row $(1, 0, 4)$ with right-hand side $-2$ .

Equation 3: $-3x_1 + 2x_2 - x_3 = 5$ gives row $(-3, 2, -1)$ with right-hand side $5$ .

$A = \begin{pmatrix}2&-1&3\\1&0&4\\-3&2&-1\end{pmatrix}$ , $\mathbf{b} = \begin{pmatrix}7\\-2\\5\end{pmatrix}$ .

$[A|\mathbf{b}] = \left(\begin{array}{ccc|c}2&-1&3&7\\1&0&4&-2\\-3&2&-1&5\end{array}\right)$ .

Write the system in matrix form $A\mathbf{x} = \mathbf{b}$ and form the augmented matrix $[A|\mathbf{b}]$ : $2x_1 - x_2 + 3x_3 = 7$ ; $x_1 + 4x_3 = -2$ ; $-3x_1 + 2x_2 - x_3 = 5$ .

EXERCISE 3.2

Apply $R_2 \to R_2 - 2R_1$ to eliminate $x_1$ from equation 2. Then read off $x_2$ from the resulting row 2 and back-substitute.

Augmented matrix: $\left(\begin{array}{cc|c}1&-3&4\\2&1&3\end{array}\right)$ .

$R_2 \to R_2 - 2R_1$ : $(2-2, 1-(-6), 3-8) = (0, 7, -5)$ . Matrix: $\left(\begin{array}{cc|c}1&-3&4\\0&7&-5\end{array}\right)$ .

$R_2 \to \frac{1}{7}R_2$ : row 2 becomes $(0, 1, -5/7)$ . Matrix: $\left(\begin{array}{cc|c}1&-3&4\\0&1&-5/7\end{array}\right)$ .

$R_1 \to R_1 + 3R_2$ : row 1 becomes $(1, 0, 4 + 3(-5/7)) = (1, 0, 4 - 15/7) = (1, 0, 13/7)$ .

Solution: $x_1 = 13/7$ , $x_2 = -5/7$ .

Verify: eq 1: $13/7 - 3(-5/7) = 13/7 + 15/7 = 28/7 = 4$ ✓. Eq 2: $2(13/7) + (-5/7) = 26/7 - 5/7 = 21/7 = 3$ ✓.

Solve $\{x_1 - 3x_2 = 4,\ 2x_1 + x_2 = 3\}$ using row operations on the augmented matrix. Show every row operation applied. Verify your solution in both equations.

EXERCISE 3.3

After forming the augmented matrix, look at what happens in the last row after elimination. A row of the form $(0\ 0\ \cdots\ 0\ |\ c)$ with $c \neq 0$ signals inconsistency.

Augmented: $\left(\begin{array}{ccc|c}1&2&-1&3\\2&4&-2&5\\-1&-2&1&-3\end{array}\right)$ .

$R_2 \to R_2 - 2R_1$ : $(0, 0, 0, 5-6) = (0,0,0,-1)$ . Row 2: $(0,0,0,-1)$ .

Row 2 reads $0x_1+0x_2+0x_3=-1$ , i.e. $0 = -1$ . Contradiction.

The system is inconsistent — solution set is $\emptyset$ . No need to process row 3.

Geometric interpretation: all three equations define planes in $\mathbb{R}^3$ . Equations 1 and 2 define parallel planes (equation 2 is $2 \times$ equation 1 shifted), so they do not intersect.

Classify (consistent/inconsistent) and solve if consistent: $x_1 + 2x_2 - x_3 = 3$ ; $2x_1 + 4x_2 - 2x_3 = 5$ ; $-x_1 - 2x_2 + x_3 = -3$ . Show all row operations.

EXERCISE 3.4

A system with more unknowns than equations and at least one non-contradiction row typically has free variables. After row reduction, count how many columns lack a leading 1.

Augmented: $\left(\begin{array}{cccc|c}1&-1&2&0&3\\0&1&-1&2&1\\2&-1&3&2&7\end{array}\right)$ .

$R_3 \to R_3 - 2R_1$ : $(0, 1, -1, 2, 7-6) = (0,1,-1,2,1)$ . Row 3 equals row 2 — subtract: $R_3 \to R_3 - R_2 = (0,0,0,0,0)$ .

Reduced: $\left(\begin{array}{cccc|c}1&-1&2&0&3\\0&1&-1&2&1\end{array}\right)$ (ignoring zero row).

Pivots in columns 1 and 2. Free variables: $x_3 = s$ , $x_4 = t$ .

Row 2: $x_2 = 1 + s - 2t$ . Row 1: $x_1 = 3 + x_2 - 2s = 3 + (1+s-2t) - 2s = 4 - s - 2t$ .

$\mathbf{x} = \begin{pmatrix}4\\1\\0\\0\end{pmatrix} + s\begin{pmatrix}-1\\1\\1\\0\end{pmatrix} + t\begin{pmatrix}-2\\-2\\0\\1\end{pmatrix}$ , $s,t \in \mathbb{R}$ . Infinitely many solutions.

Find all solutions: $x_1 - x_2 + 2x_3 = 3$ ; $x_2 - x_3 + 2x_4 = 1$ ; $2x_1 - x_2 + 3x_3 + 2x_4 = 7$ . Express the answer in vector parametric form.

EXERCISE 3.5

Find the null space of $A$ by solving $A\mathbf{x} = \mathbf{0}$ . Then, if $\mathbf{x}_p$ is any particular solution of $A\mathbf{x} = \mathbf{b}$ , the complete solution set is $\mathbf{x}_p + \text{Null}(A)$ .

$A = \begin{pmatrix}1&2\\2&4\end{pmatrix}$ . Null space from Section 05: $\text{Null}(A) = \{t(-2,1)^T : t \in \mathbb{R}\}$ .

For $A\mathbf{x} = \mathbf{b}$ with $\mathbf{b} = \begin{pmatrix}3\\6\end{pmatrix}$ : augmented $\left(\begin{array}{cc|c}1&2&3\\2&4&6\end{array}\right)$ . $R_2 \to R_2-2R_1$ : $(0,0,0)$ . Matrix: $\left(\begin{array}{cc|c}1&2&3\\0&0&0\end{array}\right)$ . $x_2 = t$ free, $x_1 = 3-2t$ . Particular solution (at $t=0$ ): $\mathbf{x}_p = (3,0)^T$ .

Complete solution: $\mathbf{x} = \begin{pmatrix}3\\0\end{pmatrix} + t\begin{pmatrix}-2\\1\end{pmatrix}$ .

For $\mathbf{b} = \begin{pmatrix}3\\7\end{pmatrix}$ : $R_2 \to R_2-2R_1$ : $(0,0,7-6)=(0,0,1)$ . Row 2: $0=1$ . Inconsistent — no solution.

For $A = \begin{pmatrix}1&2\\2&4\end{pmatrix}$ : (a) Solve $A\mathbf{x} = \begin{pmatrix}3\\6\end{pmatrix}$ and express the complete solution set. (b) Explain why $A\mathbf{x} = \begin{pmatrix}3\\7\end{pmatrix}$ has no solution.

EXERCISE 3.6

Set up three equations: weights sum to 1 (budget), the expected return equation, and the volatility-contribution equation. Write as $A\mathbf{w} = \mathbf{b}$ and solve.

Three equations: (1) $w_1 + w_2 + w_3 = 1$ (budget). (2) $5w_1 + 8w_2 + 12w_3 = 8$ (return target). (3) $2w_1 + 5w_2 + 10w_3 = 5$ (risk target).

Augmented: $\left(\begin{array}{ccc|c}1&1&1&1\\5&8&12&8\\2&5&10&5\end{array}\right)$ .

$R_2 \to R_2 - 5R_1$ : $(0,3,7,3)$ . $R_3 \to R_3 - 2R_1$ : $(0,3,8,3)$ .

$R_3 \to R_3 - R_2$ : $(0,0,1,0)$ . So $w_3 = 0$ .

Back-sub into $R_2$ : $3w_2 + 7(0) = 3 \implies w_2 = 1$ . Row 1: $w_1 + 1 + 0 = 1 \implies w_1 = 0$ .

Solution: $w_1 = 0$ , $w_2 = 1$ , $w_3 = 0$ . The target return and risk are matched by holding 100% in asset 2 — a degenerate solution. This result tells the manager the targets are achievable only by concentrating entirely in asset 2.

A portfolio manager wants weights $w_1, w_2, w_3 \geq 0$ (summing to 1) for three assets with expected returns $5\%, 8\%, 12\%$ and risk contributions $2, 5, 10$ . Targets: portfolio return $= 8\%$ , portfolio risk contribution $= 5$ , weights sum to 1. Write the system $A\mathbf{w} = \mathbf{b}$ and solve.

07 · Summary

Term	Definition
System of equations	$m$ linear constraints on $n$ unknowns. Solution: values satisfying all $m$ simultaneously.
Matrix form	$A\mathbf{x} = \mathbf{b}$ . $A \in \mathbb{R}^{m \times n}$ coefficient matrix, $\mathbf{b} \in \mathbb{R}^m$ RHS.
Augmented matrix	$[A\mid\mathbf{b}]$ — combines $A$ and $\mathbf{b}$ for row operations.
Row operations	Swap rows; scale row; add multiple of one row to another. Preserve solution set.
Consistent	At least one solution exists.
Inconsistent	No solution. Row reduces to a row $(0\cdots0\mid c)$ , $c \neq 0$ .
Unique solution	Consistent, no free variables.
Infinite solutions	Consistent, at least one free variable. Parameterised by $t \in \mathbb{R}$ .
Null space	$\text{Null}(A) = \{\mathbf{x}: A\mathbf{x}=\mathbf{0}\}$ . Always a subspace. Always contains $\mathbf{0}$ .

Next: Gaussian Elimination & Row Reduction — the systematic algorithm for reducing any augmented matrix to a solved form, with formal definitions of pivot positions, rank, and free variables.