Skip to main content

Section 2.3 Matrix Multiplication

While matrix addition and scalar multiplication are straightforward extensions of familiar arithmetic operations, matrix multiplication introduces a new layer of complexity and utility. This operation is not only fundamental to linear algebra but also underpins many applications in computer science, physics, economics, and beyond.
Understanding why matrix multiplication is defined the way it is reveals deep connections to composition of linear transformations, systems of equations, and data transformations. Unlike the element-wise operations we’ve seen with addition, matrix multiplication follows a row-by-column pattern that may initially seem arbitrary, but proves to be precisely what’s needed for practical applications.
A key insight in linear algebra is that expressing a matrix as a product of simpler matrices often reveals its fundamental properties. This factorization approach—decomposing complex matrices into products of elementary or special matrices—is a powerful analytical tool that simplifies computation, reveals structure, and provides deeper understanding of matrix behavior. We will explore this idea throughout this section and see how matrix multiplication enables us to build complex transformations from simpler building blocks.

Subsection 2.3.1 Matrix Multiplication

Definition 2.3.1.

If \(A=\left[a_{i j}\right]\) is an \(m \times \mathbf{n}\) matrix and \(B=\left[b_{i j}\right]\) is an \(\mathbf{n} \times p\) matrix, then the product \(A B\) is an \(m \times p\) matrix \(A B=\left[c_{i j}\right]\text{,}\) where
\begin{align*} c_{i j} \amp=\sum_{k=1}^{n} a_{i k} b_{k j} \\ \amp=a_{i 1} b_{1 j}+a_{i 2} b_{2 j}+a_{i 3} b_{3 j}+\cdots+a_{i n} b_{n j} \end{align*}

Proof.

Note that the \((i,j)\)-th entry of \(AB\) is given by
\begin{gather*} (AB)_{i j} =\sum_{k=1}^{n} a_{i k} b_{k j} = a_{i 1} b_{1 j}+a_{i 2} b_{2 j}+a_{i 3} b_{3 j}+\cdots+a_{i n} b_{n j}\text{.} \end{gather*}
On the other hand, the \((i,j)\)-th entry of \([A \beta_1 \ A \beta_2 \ \cdots \ \ A \beta_p]\) is given by
\begin{gather*} (A \beta_j)_{i} =\sum_{k=1}^{n} a_{i k} b_{k j} = a_{i 1} b_{1 j}+a_{i 2} b_{2 j}+a_{i 3} b_{3 j}+\cdots+a_{i n} b_{n j}. \end{gather*}
Thus we have \((AB)_{i j}=(A \beta_j)_{i}\) for all \(1 \leq i \leq m\) and \(1\leq j\leq p\text{.}\) This completes the proof.

Exercises Examples

1.
Let \(A = \left[\begin{array}{cc}3 \amp -4 \\ -4 \amp 3 \\ 3 \amp 1\end{array}\right]\) and \(B = \left[\begin{array}{cc}-1 \amp -3 \\ 2 \amp 1\end{array}\right]\text{.}\) Compute \(AB\text{.}\) Is it possible to compute \(BA\text{?}\) Why or why not?
2.
Let
\begin{equation*} A = \begin{pmatrix} 2 \amp -8 \amp 8 \\ -2 \amp 1 \amp 0 \\ 0 \amp -5 \amp 10 \end{pmatrix} \text{ and } B = \begin{pmatrix} 0 \amp 3 \amp -6 \amp 6 \amp 4 \amp -5 \\ 3 \amp -7 \amp 8 \amp -5 \amp 8 \amp 9 \\ 3 \amp -9 \amp 12 \amp -9 \amp 6 \amp 15 \end{pmatrix}\text{.} \end{equation*}
Compute \(AB_{14}, AB_{21}\) and \(AB_{36}\text{,}\) respectively.
3.
Using the matrices \(A\) and \(B\) from the previous exercise, verify your answers with the following computation, then find only \((AB)_{23}\text{.}\)
Hint.
To find only \((AB)_{23}\text{,}\) use a loop over the appropriate row and column:
4.
Exercise: Given the matrices:
\begin{equation*} A=\left[\begin{array}{cc} 4 \amp 4 \\ -3 \amp 4 \end{array}\right], \quad B=\left[\begin{array}{cc} 4 \amp -5 \\ -5 \amp -1 \\ -1 \amp 4 \end{array}\right], \quad C=\left[\begin{array}{ccc} -5 \amp -5 \amp -1 \\ -5 \amp 1 \amp 1 \end{array}\right] \end{equation*}
If possible, compute the following. If an answer does not exist, enter DNE.
  1. \(\displaystyle CB - A = ?\)
  2. \(\displaystyle CA = ?\)
5.
Let \(A\) be an \(m \times n\) matrix, and let \(\alpha_1, \alpha_2, \ldots, \alpha_n\) denote the columns of \(A\text{,}\) so that \(A = [\alpha_1 \ \alpha_2 \ \cdots \ \alpha_n]\text{.}\) Let \(e_1, e_2, \ldots, e_n\) be the standard basis vectors of \(\mathbb{R}^n\) (i.e., the columns of \(I_n\)).
  1. Show that \(Ae_i = \alpha_i\) for each \(i = 1, 2, \ldots, n\text{.}\) That is, multiplying \(A\) on the right by the \(i\)-th standard basis vector extracts the \(i\)-th column of \(A\text{.}\)
  2. Use part (1) to show that \(AI_n = A\text{.}\)
  3. Show that \(I_m A = A\text{.}\)
Hint.
For part (1), write out \(Ae_i\) using the definition of matrix multiplication. For part (2), use the column partition \(I_n = [e_1 \ e_2 \ \cdots \ e_n]\) and the result of part (1).
Solution.
Part (1). By the definition of matrix-vector multiplication,
\begin{equation*} Ae_i = \sum_{k=1}^{n} (e_i)_k \alpha_k = \alpha_i, \end{equation*}
since \((e_i)_k = 1\) if \(k = i\) and \(0\) otherwise. Thus multiplying \(A\) on the right by \(e_i\) picks out the \(i\)-th column of \(A\text{.}\)
Part (2). Using the column partition of \(I_n\text{,}\)
\begin{equation*} AI_n = A[e_1 \ e_2 \ \cdots \ e_n] = [Ae_1 \ Ae_2 \ \cdots \ Ae_n] = [\alpha_1 \ \alpha_2 \ \cdots \ \alpha_n] = A. \end{equation*}
Part (3). Let \(\beta_j^T\) denote the \(j\)-th row of \(A\text{.}\) By a symmetric row-extraction argument using the standard basis of \(\mathbb{R}^m\text{,}\) one checks that \(e_j^T A = \beta_j^T\text{,}\) and therefore
\begin{equation*} I_m A = \begin{bmatrix} e_1^T \\ \vdots \\ e_m^T \end{bmatrix} A = \begin{bmatrix} e_1^T A \\ \vdots \\ e_m^T A \end{bmatrix} = \begin{bmatrix} \beta_1^T \\ \vdots \\ \beta_m^T \end{bmatrix} = A. \end{equation*}
6. Challenge: National Collegiate Mathematics Competition.
Let \(A = [a_{ij}]\) be an \(n \times n\) matrix, and let
\begin{equation*} F = \begin{bmatrix} 0 & 0 & \cdots & 0 & -a_n \\ 1 & 0 & \cdots & 0 & -a_{n-1} \\ 0 & 1 & \cdots & 0 & -a_{n-2} \\ \vdots & \vdots & \ddots & \vdots & \vdots \\ 0 & 0 & \cdots & 1 & -a_1 \end{bmatrix} \end{equation*}
be the companion matrix associated with scalars \(a_1, a_2, \ldots, a_n\text{.}\) Suppose that \(AF = FA\text{.}\)
Show that
\begin{equation*} A = a_{n1}F^{n-1} + a_{n-1,1}F^{n-2} + \cdots + a_{21}F + a_{11}I, \end{equation*}
where \(a_{i1}\) denotes the entry in row \(i\text{,}\) column \(1\) of \(A\text{.}\)
Hint.
Let \(M = a_{n1}F^{n-1} + a_{n-1,1}F^{n-2} + \cdots + a_{21}F + a_{11}I\text{.}\) Show that the columns of \(A\) and \(M\) coincide, i.e., \(Me_i = Ae_i\) for all \(i = 1, \ldots, n\text{,}\) by mathematical induction. Start by observing that \(Fe_1 = e_2\text{,}\) \(Fe_2 = e_3\text{,}\) \(\ldots\text{,}\) \(Fe_{n-1} = e_n\text{,}\) so that \(F^k e_1 = e_{k+1}\text{.}\)
Solution.
Let \(M = a_{n1}F^{n-1} + a_{n-1,1}F^{n-2} + \cdots + a_{21}F + a_{11}I\text{.}\)
Key observation. Examining the columns of \(F\text{,}\) we have \(Fe_1 = e_2\text{,}\) \(Fe_2 = e_3\text{,}\) \(\ldots\text{,}\) \(Fe_{n-1} = e_n\text{.}\) By induction, \(F^k e_1 = e_{k+1}\) for \(k = 0, 1, \ldots, n-1\) (where \(F^0 = I\)).
Base case (\(i=1\)). Using the column-extraction result \(Ae_1 = \alpha_1\text{,}\)
\begin{align*} Me_1 &= a_{n1}F^{n-1}e_1 + a_{n-1,1}F^{n-2}e_1 + \cdots + a_{21}Fe_1 + a_{11}Ie_1\\ &= a_{n1}e_n + a_{n-1,1}e_{n-1} + \cdots + a_{21}e_2 + a_{11}e_1\\ &= \begin{bmatrix} a_{11} \\ a_{21} \\ \vdots \\ a_{n1} \end{bmatrix} = Ae_1. \end{align*}
Inductive step. Suppose \(Me_k = Ae_k\) for some \(1 \leq k < n\text{.}\) Since \(e_{k+1} = Fe_k\text{,}\)
\begin{align*} Me_{k+1} &= M(Fe_k) = (MF)e_k\\ &= (FM)e_k \qquad (MF = FM, \text{ because } M \text{ is a polynomial in } F \text{ and } AF=FA)\\ &= F(Me_k) = F(Ae_k)\\ &= (FA)e_k = (AF)e_k = A(Fe_k) = Ae_{k+1}. \end{align*}
By mathematical induction, \(Me_i = Ae_i\) for all \(i = 1, \ldots, n\text{.}\) Since two matrices are equal if and only if all their columns are equal, we conclude \(A = M\text{.}\)
7.
Let \(X = [x_{ij}]\) be a matrix of size \(n \times n\text{.}\) The trace of the matrix \(X\) is defined as the sum of the diagonal entries, that is,
\begin{equation*} \text{tr}(X) = x_{11} + x_{22} + \cdots + x_{nn} \end{equation*}
Show that \(\text{tr}(AB) = \text{tr}(BA)\) for any two matrices \(A\) and \(B\) of size \(n \times n\text{.}\)

Subsection 2.3.2 \(AB\neq BA\)

For linear algebra, the most inconvinence is noncommutativity, that is, \(AB\neq BA\) in general. Let check it!
Example Let
\begin{equation*} A = \begin{pmatrix} 3 \amp 4 \\ 7 \amp 8 \end{pmatrix}, B = \begin{pmatrix} 5 \amp 3\\ 2 \amp 1 \end{pmatrix}. \end{equation*}
Show that \(AB\neq BA\text{.}\)
For this moment, it looks bad to fail the commutativity of matrix multiplication. The failure of the commutative law is actually a good thing, because it enables matrices to represent other things that do not commute, such as rotations in space.

Subsection 2.3.3 The \(n\)-th Power of a matrix

Let \(A\) be a square matrix. In this course, computing \(A^{n}\) is important. We can use SageMath to find the pattern, then give a conjecture. If possible, you may provide a proof by mathematical induction. Let us begin.
Example Find the conjecture of \(A^{n}\text{,}\) where
\begin{equation*} A = \begin{pmatrix} 1 \amp 1\\ 0 \amp 1 \end{pmatrix} \end{equation*}

Proof.

This is a direct consequence of the commutative and associative properties of addition of real numbers. Regardless of the order in which we sum, we are adding all \(np\) terms \(a_{jk}\text{.}\) Since real number addition satisfies commutativity and associativity, the order of summation does not affect the final result.
More specifically, the left-hand side can be expanded as:
\begin{equation*} \sum_{j=1}^{n} \sum_{k=1}^{p} a_{jk} = \sum_{j=1}^{n} (a_{j1} + a_{j2} + \cdots + a_{jp}) = \sum_{j=1}^{n}\sum_{k=1}^{p} a_{jk} \end{equation*}
while the right-hand side can be expanded as:
\begin{equation*} \sum_{k=1}^{p} \sum_{j=1}^{n} a_{jk} = \sum_{k=1}^{p} (a_{1k} + a_{2k} + \cdots + a_{nk}) = \sum_{k=1}^{p}\sum_{j=1}^{n} a_{jk} \end{equation*}
Both are equal to the sum of all \(a_{jk}\text{,}\) and therefore they are equal.

Proof.

We prove each property by examining the entries of the matrices.
Property 1: Associative Property of Multiplication
Let \(A = [a_{ij}]\) be an \(m \times n\) matrix, \(B = [b_{jk}]\) be an \(n \times p\) matrix, and \(C = [c_{k\ell}]\) be a \(p \times q\) matrix.
For the left side \(A(BC)\text{,}\) we first compute \(BC = [d_{j\ell}]\) where
\begin{equation*} d_{j\ell} = \sum_{k=1}^{p} b_{jk}c_{k\ell} \end{equation*}
Then the \((i,\ell)\)-entry of \(A(BC)\) is:
\begin{align*} [A(BC)]_{i\ell} \amp= \sum_{j=1}^{n} a_{ij}d_{j\ell}\\ \amp= \sum_{j=1}^{n} a_{ij}\left(\sum_{k=1}^{p} b_{jk}c_{k\ell}\right)\\ \amp= \sum_{j=1}^{n} \sum_{k=1}^{p} a_{ij}b_{jk}c_{k\ell} \end{align*}
For the right side \((AB)C\text{,}\) we first compute \(AB = [e_{ik}]\) where
\begin{equation*} e_{ik} = \sum_{j=1}^{n} a_{ij}b_{jk} \end{equation*}
Then the \((i,\ell)\)-entry of \((AB)C\) is:
\begin{align*} [(AB)C]_{i\ell} \amp= \sum_{k=1}^{p} e_{ik}c_{k\ell}\\ \amp= \sum_{k=1}^{p} \left(\sum_{j=1}^{n} a_{ij}b_{jk}\right)c_{k\ell}\\ \amp= \sum_{k=1}^{p} \sum_{j=1}^{n} a_{ij}b_{jk}c_{k\ell} \end{align*}
By Lemma 2.3.4, we can exchange the order of summation:
\begin{equation*} \sum_{j=1}^{n} \sum_{k=1}^{p} a_{ij}b_{jk}c_{k\ell} = \sum_{k=1}^{p} \sum_{j=1}^{n} a_{ij}b_{jk}c_{k\ell} \end{equation*}
Therefore, \([A(BC)]_{i\ell} = [(AB)C]_{i\ell}\) for all \(i\) and \(\ell\text{,}\) which proves \(A(BC) = (AB)C\text{.}\)
Property 2: Left Distributive Property
Let \(A = [a_{ij}]\) be an \(m \times n\) matrix, and let \(B = [b_{jk}]\) and \(C = [c_{jk}]\) be \(n \times p\) matrices.
The \((i,k)\)-entry of \(A(B+C)\) is:
\begin{align*} [A(B+C)]_{ik} \amp= \sum_{j=1}^{n} a_{ij}(B+C)_{jk}\\ \amp= \sum_{j=1}^{n} a_{ij}(b_{jk} + c_{jk})\\ \amp= \sum_{j=1}^{n} (a_{ij}b_{jk} + a_{ij}c_{jk})\\ \amp= \sum_{j=1}^{n} a_{ij}b_{jk} + \sum_{j=1}^{n} a_{ij}c_{jk}\\ \amp= [AB]_{ik} + [AC]_{ik}\\ \amp= [AB + AC]_{ik} \end{align*}
Therefore, \(A(B+C) = AB + AC\text{.}\)
Property 3: Right Distributive Property
Let \(A = [a_{ik}]\) and \(B = [b_{ik}]\) be \(m \times n\) matrices, and let \(C = [c_{k\ell}]\) be an \(n \times p\) matrix.
The \((i,\ell)\)-entry of \((A+B)C\) is:
\begin{align*} [(A+B)C]_{i\ell} \amp= \sum_{k=1}^{n} (A+B)_{ik}c_{k\ell}\\ \amp= \sum_{k=1}^{n} (a_{ik} + b_{ik})c_{k\ell}\\ \amp= \sum_{k=1}^{n} (a_{ik}c_{k\ell} + b_{ik}c_{k\ell})\\ \amp= \sum_{k=1}^{n} a_{ik}c_{k\ell} + \sum_{k=1}^{n} b_{ik}c_{k\ell}\\ \amp= [AC]_{i\ell} + [BC]_{i\ell}\\ \amp= [AC + BC]_{i\ell} \end{align*}
Therefore, \((A+B)C = AC + BC\text{.}\)
Property 4: Scalar Multiplication Compatibility
Let \(A = [a_{ij}]\) be an \(m \times n\) matrix, \(B = [b_{jk}]\) be an \(n \times p\) matrix, and let \(c\) be a scalar.
For \(c(AB)\text{,}\) the \((i,k)\)-entry is:
\begin{align*} [c(AB)]_{ik} \amp= c \cdot [AB]_{ik}\\ \amp= c \cdot \sum_{j=1}^{n} a_{ij}b_{jk}\\ \amp= \sum_{j=1}^{n} c(a_{ij}b_{jk})\\ \amp= \sum_{j=1}^{n} (ca_{ij})b_{jk}\\ \amp= [(cA)B]_{ik} \end{align*}
This shows that \(c(AB) = (cA)B\text{.}\)
Similarly, we can also write:
\begin{align*} \sum_{j=1}^{n} c(a_{ij}b_{jk}) \amp= \sum_{j=1}^{n} a_{ij}(cb_{jk})\\ \amp= [A(cB)]_{ik} \end{align*}
This shows that \(c(AB) = A(cB)\text{.}\)
Combining these results, we have \(c(AB) = (cA)B = A(cB)\text{.}\)
This completes the proof of all four properties.

Exercises Exercises

1.
Find the conjecture of \(X^{n}\text{,}\) where \(X=\begin{pmatrix} 2 \amp 1\\ 0 \amp 2 \end{pmatrix}\text{.}\)
Solution.
\(\begin{pmatrix} 2 \amp 1\\ 0 \amp 2 \end{pmatrix}^{n}=\begin{pmatrix} 2^{n} \amp 2^{n-1}\cdot n\\ 0 \amp 2^{n} \end{pmatrix}\)
2.
Find the conjecture of \(X^{n}\text{,}\) where \(X=\begin{pmatrix} 3 \amp 5\\ 0 \amp 3 \end{pmatrix}\text{.}\)
3.
Compute \(X^{n}\) for \(n=2,3,4,5\) and \(n=2023\text{,}\) where \(X=\begin{pmatrix} 0 \amp 1 \amp 0 \amp 0 \amp 0 \\ 0 \amp 0 \amp 1 \amp 0 \amp 0 \\ 0 \amp 0 \amp 0 \amp 1 \amp 0 \\ 0 \amp 0 \amp 0 \amp 0 \amp 1 \\ 0 \amp 0 \amp 0 \amp 0 \amp 0 \end{pmatrix}\)
Solution.
4.
Exercise: Compute \(X^{n}\) for \(n=2,3,4,5,6\) and conjecture the formula for \(X^{n}, n\geq 6\text{,}\) where \(X=\begin{pmatrix} 0 \amp 0 \amp 1 \amp 0 \amp 0 \amp 0 \\ 0 \amp 0 \amp 0 \amp 1 \amp 0 \amp 0 \\ 0 \amp 0 \amp 0 \amp 0 \amp 1\amp 0 \\ 0 \amp 0 \amp 0 \amp 0 \amp 0 \amp 1 \\ 0 \amp 0 \amp 0 \amp 0 \amp 0 \amp 0 \\ 0 \amp 0 \amp 0 \amp 0 \amp 0 \amp 0 \end{pmatrix}\)