Skip to main content

Linear Algebra For Undergraduate Students

Yilan(Peter) Tan

Section 2.4 Inverse Matrix

In this section, we explore the concept of matrix inverses, a fundamental idea in linear algebra that extends the notion of multiplicative inverses from real numbers to square matrices. Just as the inverse of a nonzero real number allows us to solve simple equations, the inverse of a matrix provides a powerful tool for solving systems of linear equations and understanding linear transformations.
🔗
We will begin by examining the theoretical foundation of matrix inversion, followed by practical methods for computing inverses. We will also discuss important properties of invertible matrices and their applications in various fields such as computer science, engineering, and economics.
🔗

Subsection 2.4.1 Theoretical Foundation of Matrix Inversion

In elementary algebra, we learn to solve equations like \(ax = b\) where \(a\) and \(b\) are real numbers and \(x\) is the unknown. When \(a \neq 0\text{,}\) the solution is simply \(x = \frac{b}{a}\text{,}\) or equivalently, \(x = a^{-1}b\) where \(a^{-1} = \frac{1}{a}\) is the multiplicative inverse of \(a\text{.}\)
🔗
This familiar approach from elementary algebra naturally extends to matrix equation \(A\mathbf{x} = \mathbf{b}\text{,}\) where \(A\) is an \(n \times n\) square matrix. Just as we multiply both sides of \(ax = b\) by \(a^{-1}\) to obtain \(x = a^{-1}b\text{,}\) we might hope to "multiply" both sides of \(A\mathbf{x} = \mathbf{b}\) by some matrix \(A^{-1}\) to obtain:
\begin{equation*} \mathbf{x} = A^{-1}\mathbf{b} \end{equation*}
🔗
This motivates the following fundamental question: Does there exist a matrix \(A^{-1}\) such that when we multiply it by \(A\text{,}\) we get the identity matrix? If such a matrix exists, we call it the inverse of \(A\text{,}\) and it provides us with a powerful tool for solving matrix equations.
🔗

Definition 2.4.1. Inverse Matrix.

Let \(A\) be an \(n \times n\) square matrix. An \(n \times n\) matrix \(B\) is called the inverse of \(A\) if
\begin{equation*} AB = BA = I_n \end{equation*}
where \(I_n\) is the \(n \times n\) identity matrix. If such a matrix \(B\) exists, we write \(B = A^{-1}\) and say that \(A\) is invertible (or nonsingular). If no such matrix exists, we say that \(A\) is singular (or noninvertible).
🔗
🔗

Remark 2.4.2. Why Study Matrix Inverses?

The concept of matrix inverse is central to linear algebra for several reasons:
  • Solving Linear Systems: If \(A\) is invertible, then the matrix equation \(A\mathbf{x} = \mathbf{b}\) has the unique solution \(\mathbf{x} = A^{-1}\mathbf{b}\text{.}\) This provides a direct formula for solving systems of linear equations, analogous to solving \(ax = b\) in elementary algebra.
    🔗
    🔗
  • Understanding Invertibility: Not every square matrix has an inverse. Understanding when a matrix is invertible reveals important structural properties about linear transformations and systems of equations.
    🔗
    🔗
  • Practical Applications: Matrix inverses appear in numerous applications including computer graphics (undoing transformations), cryptography (decoding messages), economics (input-output analysis), and engineering (control systems).
    🔗
    🔗
  • Theoretical Foundation: The invertibility of a matrix is connected to many other fundamental concepts in linear algebra, including determinants, eigenvalues, and the rank of a matrix.
    🔗
    🔗
🔗
🔗

Example 2.4.3. From Elementary Equations to Matrix Equations.

Consider the system of linear equations:
\begin{align*} 2x + y \amp= 5\\ x + 3y \amp= 8 \end{align*}
🔗
This can be written as the matrix equation \(A\mathbf{x} = \mathbf{b}\) where:
\begin{equation*} A = \begin{pmatrix} 2 \amp 1 \\ 1 \amp 3 \end{pmatrix}, \quad \mathbf{x} = \begin{pmatrix} x \\ y \end{pmatrix}, \quad \mathbf{b} = \begin{pmatrix} 5 \\ 8 \end{pmatrix} \end{equation*}
🔗
If we can find \(A^{-1}\text{,}\) then the solution is simply:
\begin{equation*} \mathbf{x} = A^{-1}\mathbf{b} \end{equation*}
This is remarkably similar to how we solve \(ax = b\) by computing \(x = a^{-1}b\) in elementary algebra.
🔗
🔗

Proof.

Suppose \(B\) and \(C\) are both inverses of \(A\text{.}\) Then we have:
\begin{equation*} AB = BA = I_n \quad \text{and} \quad AC = CA = I_n \end{equation*}
To show that \(B = C\text{,}\) we can use the properties of matrix multiplication:
\begin{align*} B \amp= B I_n\\ \amp= B(AC)\\ \amp= (BA)C\\ \amp= I_n C\\ \amp= C \end{align*}
Thus, we have shown that \(B = C\text{,}\) proving the uniqueness of the inverse.
🔗
🔗

Insight 2.4.5. How to compute the inverse.

Suppose that \(A\) is invertible, that is, there exists another matrix \(B\) such that
\begin{equation*} AB=I_n. \end{equation*}
🔗
Let \(B=[\beta_1\,\beta_2\,\ldots\, \beta_n]\) and \(I_n=[\mathbf{e}_1\ \mathbf{e}_2\ \ldots\ \mathbf{e}_n]\text{.}\) \(AB=I_n\) can be read as
\begin{equation*} AB=A[\beta_1\,\beta_2\,\ldots\, \beta_n]=[A\beta_1\, A\beta_2\,\ldots\,A\beta_n]=[\mathbf{e}_1\,\mathbf{e}_2\,\ldots\, \mathbf{e}_n], \end{equation*}
🔗
which can be read as \(n\)-linear systems(in matrix equations): \(A\beta_i = \mathbf{e}_i\text{,}\) \(i=1,2,\ldots, n\text{.}\) How to solve the \(n\)-linear systems together?
🔗
🔗

Activity 2.4.1. Find the Inverse of a Matrix Using Elementary Row Operations.

We will find the inverse of matrix \(A\) from Subsection 2.4.2 using elementary row operations. The method is to transform the augmented matrix \([A \mid I]\) into \([I \mid A^{-1}]\) using only elementary row operations.
🔗
Recall that
\begin{equation*} A = \begin{pmatrix} 2 \amp -8 \amp 8 \\ -2 \amp 1 \amp 0 \\ 0 \amp -5 \amp 10 \end{pmatrix} \end{equation*}
🔗
(a)
First, create the augmented matrix \([A \mid I_3]\text{:}\)
🔗
🔗
(b)
Now we perform elementary row operations to transform \([A \mid I]\) into \([I \mid A^{-1}]\text{.}\) Execute the following steps:
🔗
Step 1: Make the first pivot equal to 1 by multiplying row 1 by \(\frac{1}{2}\)
🔗
Step 2: Eliminate below the first pivot: Add 2 times row 1 to row 2
🔗
Step 3: Make the second pivot equal to 1 by multiplying row 2 by \(-\frac{1}{7}\)
🔗
Step 4: Eliminate above the second pivot: Add 4 times row 2 to row 1
🔗
Step 5: Eliminate below the second pivot: Add 5 times row 2 to row 3
🔗
Step 6: Make the third pivot equal to 1 by multiplying row 3 by \(\frac{7}{30}\)
🔗
Step 7: Eliminate above the third pivot (row 2): \(\frac{8}{7} R_3 + R_2\)
🔗
Step 8: Eliminate row 1, column 2: \(\frac{32}{7} R_3 + R_1\)
🔗
Final Result: The augmented matrix is now \([I \mid A^{-1}]\)
🔗
🔗
(c)
Verify that the right half of the final augmented matrix is indeed \(A^{-1}\) by checking that \(AA^{-1} = I\text{:}\)
🔗
🔗
This activity demonstrates how to find the inverse of a matrix using elementary row operations. The key insight is that the same sequence of row operations that transforms \(A\) into \(I\) will simultaneously transform \(I\) into \(A^{-1}\text{.}\)
🔗
🔗
🔗

Subsection 2.4.2 Computing Inverse Matrices

You can easily find the inverse of a square matrix \(A\) by A.inverse().
🔗

Activity 2.4.2.

Find the inverse of the matrix
\begin{equation*} B = \left[\begin{array}{rrrr} 2 \amp 1 \amp -2 \amp -3 \\ 3 \amp 0 \amp -1 \amp -2 \\ -3 \amp 4 \amp 1 \amp 2\\ 1 \amp 3 \amp 3 \amp -1 \\ \end{array}\right]\text{.} \end{equation*}
🔗
🔗
🔗

Subsection 2.4.3 Application: Encoding Messages with Invertible Matrices

One interesting application of invertible matrices is in encoding and decoding messages. The basic idea is simple: we use an invertible matrix to transform a message (encoding), and use its inverse to recover the original message (decoding).
🔗
We’ll demonstrate this with a simple example using a \(2 \times 2\) invertible matrix. This illustrates the key concept without requiring advanced techniques.
🔗

Subsubsection 2.4.3.1 The Encoding Process

Step 1: Convert Message to Numbers
🔗
First, we assign each letter a number. We’ll use: A = 1, B = 2, C = 3, ..., Z = 26. We also use 0 to represent a space.
🔗
For example, the message "HELLO" converts to:
\begin{equation*} H=8, E=5, L=12, L=12, O=15 \end{equation*}
🔗
Step 2: Group into Vectors
🔗
Since we’re using a \(2 \times 2\) encoding matrix, we group the numbers into column vectors of length 2:
\begin{equation*} \mathbf{v}_1 = \begin{pmatrix} 8 \\ 5 \end{pmatrix}, \quad \mathbf{v}_2 = \begin{pmatrix} 12 \\ 12 \end{pmatrix}, \quad \mathbf{v}_3 = \begin{pmatrix} 15 \\ 0 \end{pmatrix} \end{equation*}
(We pad with 0 if the message length is odd.)
🔗
Step 3: Choose an Encoding Matrix
🔗
Let’s use the simple invertible matrix:
\begin{equation*} A = \begin{pmatrix} 1 \amp 2 \\ 1 \amp 3 \end{pmatrix} \end{equation*}
🔗
Step 4: Encode by Matrix Multiplication
🔗
To encode each vector \(\mathbf{v}_i\text{,}\) we compute \(\mathbf{w}_i = A\mathbf{v}_i\text{.}\)
🔗
Example 2.4.7. Encoding "HELLO".
Let’s encode the first vector \(\mathbf{v}_1 = \begin{pmatrix} 8 \\ 5 \end{pmatrix}\text{:}\)
\begin{align*} \mathbf{w}_1 \amp= A\mathbf{v}_1 = \begin{pmatrix} 1 \amp 2 \\ 1 \amp 3 \end{pmatrix} \begin{pmatrix} 8 \\ 5 \end{pmatrix}\\ \amp= \begin{pmatrix} 1(8) + 2(5) \\ 1(8) + 3(5) \end{pmatrix}\\ \amp= \begin{pmatrix} 18 \\ 23 \end{pmatrix} \end{align*}
🔗
Similarly:
\begin{align*} \mathbf{w}_2 \amp= A\mathbf{v}_2 = \begin{pmatrix} 1 \amp 2 \\ 1 \amp 3 \end{pmatrix} \begin{pmatrix} 12 \\ 12 \end{pmatrix} = \begin{pmatrix} 36 \\ 48 \end{pmatrix}\\ \mathbf{w}_3 \amp= A\mathbf{v}_3 = \begin{pmatrix} 1 \amp 2 \\ 1 \amp 3 \end{pmatrix} \begin{pmatrix} 15 \\ 0 \end{pmatrix} = \begin{pmatrix} 15 \\ 15 \end{pmatrix} \end{align*}
🔗
The encoded message is: \(18, 23, 36, 48, 15, 15\text{.}\)
🔗
🔗
🔗

Subsubsection 2.4.3.2 The Decoding Process

To decode a message, the receiver needs to know the inverse of the encoding matrix \(A\text{.}\) The decoding process reverses the encoding by multiplying with \(A^{-1}\text{.}\)
🔗
Step 1: Find the Inverse Matrix
🔗
For our encoding matrix \(A = \begin{pmatrix} 1 \amp 2 \\ 1 \amp 3 \end{pmatrix}\text{,}\) we find:
\begin{equation*} A^{-1} = \begin{pmatrix} 3 \amp -2 \\ -1 \amp 1 \end{pmatrix} \end{equation*}
🔗
Step 2: Decode by Multiplying with Inverse
🔗
To decode each encoded vector \(\mathbf{w}_i\text{,}\) we compute \(\mathbf{v}_i = A^{-1}\mathbf{w}_i\text{.}\)
🔗
Example 2.4.8. Decoding the Message.
Let’s decode the first encoded vector \(\mathbf{w}_1 = \begin{pmatrix} 18 \\ 23 \end{pmatrix}\text{:}\)
\begin{align*} \mathbf{v}_1 \amp= A^{-1}\mathbf{w}_1 = \begin{pmatrix} 3 \amp -2 \\ -1 \amp 1 \end{pmatrix} \begin{pmatrix} 18 \\ 23 \end{pmatrix}\\ \amp= \begin{pmatrix} 3(18) + (-2)(23) \\ -1(18) + 1(23) \end{pmatrix}\\ \amp= \begin{pmatrix} 8 \\ 5 \end{pmatrix} \end{align*}
This gives us H (8) and E (5), recovering "HE".
🔗
🔗
🔗

Subsubsection 2.4.3.3 Key Observations

Remark 2.4.9. Why This Works.
The encoding and decoding process works because of the fundamental property of invertible matrices:
\begin{equation*} A^{-1}(A\mathbf{v}) = (A^{-1}A)\mathbf{v} = I\mathbf{v} = \mathbf{v} \end{equation*}
🔗
In other words, applying the encoding matrix \(A\) and then the decoding matrix \(A^{-1}\) recovers the original vector.
🔗
🔗
Insight 2.4.10. Security and Practical Considerations.
  • The security of this encoding scheme relies on keeping the matrix \(A\) secret. Without knowing \(A\text{,}\) it’s difficult to decode the message.
    🔗
    🔗
  • The encoding matrix must be invertible. If it’s not invertible, we cannot decode the message!
    🔗
    🔗
  • In practice, more sophisticated encryption methods are used for secure communication, but this example illustrates how linear algebra provides powerful tools for encoding information.
    🔗
    🔗
🔗
🔗
Activity 2.4.3. Create Your Own Cipher.
  1. Choose your own \(2 \times 2\) invertible matrix. (Make sure it has an inverse!)
    🔗
    🔗
  2. Encode a short message using your matrix.
    🔗
    🔗
  3. Find the inverse of your matrix.
    🔗
    🔗
  4. Decode your encoded message to verify it works.
    🔗
    🔗
  5. Share your encoded message with a classmate (but not your matrix!) and see if they can decode it without knowing your encoding matrix.
    🔗
    🔗
🔗
🔗
🔗
🔗
🔗
PrevTopNext