Matrix Multiplication and Ax as a Combination of Columns

A matrix times a vector

Write a matrix as a row of columns. Then $A\mathbf{x}$ is defined to be the linear combination of those columns whose coefficients are the entries of $\mathbf{x}$:

$$A\mathbf{x} = \begin{bmatrix} \mathbf{a}_1 & \mathbf{a}_2 & \cdots & \mathbf{a}_n \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix} = x_1\mathbf{a}_1 + x_2\mathbf{a}_2 + \cdots + x_n\mathbf{a}_n$$

For this to make sense, $\mathbf{x}$ must have one entry per column of $A$. With

$$A = \begin{bmatrix} 1 & 2\\ 0 & 1\\ 2 & 3 \end{bmatrix} \qquad \mathbf{x} = \begin{bmatrix} 2 \\ 3 \end{bmatrix}$$

the product is $2(1, 0, 2) + 3(2, 1, 3) = (8, 3, 13)$. Computing entry by entry, each component is a dot product of a row of $A$ with $\mathbf{x}$: $1(2) + 2(3) = 8$, then $0(2) + 1(3) = 3$, then $2(2) + 3(3) = 13$. The two recipes always agree.

This is why the two halves of the course fit together:

$$A\mathbf{x} = \mathbf{b} \quad\Longleftrightarrow\quad x_1\mathbf{a}_1 + \cdots + x_n\mathbf{a}_n = \mathbf{b}$$

So $A\mathbf{x} = \mathbf{b}$ has a solution exactly when $\mathbf{b}$ lies in $\text{Span}\{\mathbf{a}_1, \ldots, \mathbf{a}_n\}$. A system of equations, a combination of columns, and a matrix equation are three ways of writing one problem.

Multiplying two matrices

Since $A$ acts on vectors, it acts on each column of a second matrix. That is the definition of the product:

$$AB = A\begin{bmatrix} \mathbf{b}_1 & \cdots & \mathbf{b}_p \end{bmatrix} = \begin{bmatrix} A\mathbf{b}_1 & \cdots & A\mathbf{b}_p \end{bmatrix}$$

The sizes have to line up: $A$ must have as many columns as $B$ has rows. An $m \times n$ matrix times an $n \times p$ matrix is $m \times p$. Entry by entry, the number in row $i$, column $j$ of $AB$ is the dot product of row $i$ of $A$ with column $j$ of $B$.

Why the product is a composition

Applying $B$ to a vector and then $A$ to the result is doing two transformations in a row. Because $A(B\mathbf{x})$ is a linear combination of the columns of $A$ with the entries of $B\mathbf{x}$ as coefficients, unwinding the definitions gives $A(B\mathbf{x}) = (AB)\mathbf{x}$ for every $\mathbf{x}$. The product $AB$ is the single matrix that does $B$ first and then $A$, and the order of the letters matches the order of application, right to left.

Two rules follow. Composition of functions is associative, so $(AB)C = A(BC)$. And composition is not commutative: turning then stretching differs from stretching then turning, so $AB \ne BA$ in general.

The columns of a matrix have a meaning in this language. Since $\mathbf{e}_j$ picks out the $j$th column, $A\mathbf{e}_j = \mathbf{a}_j$: the columns are the images of the standard basis vectors. That is enough to pin down the whole transformation.

Worked examples

Common mistakes

Practice problems