Application II: spectral decomposition

Section 7.6 Application II: spectral decomposition

Spectrum Decomposition is Important.

Spectral decomposition is important because it provides a way to understand the structure of a matrix in terms of its eigenvalues and eigenvectors. This decomposition reveals the intrinsic geometric properties of the linear transformation represented by the matrix, such as its scaling and rotation effects on different directions in the vector space. Additionally, spectral decomposition is widely used in various applications, including principal component analysis (PCA), image compression, and solving differential equations.

🔗

A symmetric matrix is orthogonally diagonalizable. It means that if we have an \(n \times n\) symmetric matrix \(\mathbf{A}\text{,}\) we can decompose it as:

\begin{equation*} \mathbf{A} = \mathbf{PDP}^T \end{equation*}

where \(\mathbf{D}\) is an \(n \times n\) diagonal matrix comprised of the \(n\) eigenvalues of \(\mathbf{A}\text{.}\) \(\mathbf{P}\) is also an \(n \times n\) matrix, and the columns of \(\mathbf{P}\) are the \(n\) linearly independent eigenvectors of \(\mathbf{A}\) that correspond to those eigenvalues in \(\mathbf{D}\) respectively.

🔗

If \(\mathbf{u}_1, \mathbf{u}_2, \mathbf{u}_3, \ldots, \mathbf{u}_n\) are the eigenvectors of \(\mathbf{A}\text{,}\) and \(\lambda_1, \lambda_2, \ldots, \lambda_n\) are their corresponding eigenvalues respectively, then \(\mathbf{A}\) can be written as:

\begin{equation*} \mathbf{A} = \begin{bmatrix} \mathbf{u}_1 & \mathbf{u}_2 & \cdots & \mathbf{u}_n \end{bmatrix} \begin{bmatrix} \lambda_1 & 0 & \cdots & 0\\ 0 & \lambda_2 & \cdots & 0\\ \vdots & \vdots & \ddots & \vdots\\ 0 & 0 & \cdots & \lambda_n \end{bmatrix} \begin{bmatrix} \mathbf{u}_1^T\\ \mathbf{u}_2^T\\ \vdots\\ \mathbf{u}_n^T \end{bmatrix} \end{equation*}

🔗

This can also be written as:

\begin{equation*} \mathbf{A} = \lambda_1 \mathbf{u}_1\mathbf{u}_1^T + \lambda_2 \mathbf{u}_2\mathbf{u}_2^T + \cdots + \lambda_n \mathbf{u}_n\mathbf{u}_n^T \end{equation*}

This factorization of \(\mathbf{A}\) is called the spectrum decomposition (eigendecomposition) of \(\mathbf{A}\text{.}\)

🔗

Example 7.6.1. Eigendecomposition Example.

🔗

Example 7.6.2. Spectral Decomposition of a Symmetric Matrix (with Repeated Eigenvalues).

Consider the symmetric matrix

\begin{equation*} \mathbf{A} = \begin{bmatrix} 5 & -8 & 4 \\ -8 & 5 & -4 \\ 4 & -4 & -1 \end{bmatrix}\text{.} \end{equation*}

We will find its eigenvalues, an orthogonal set of eigenvectors, and write its spectral decomposition.

🔗

The characteristic polynomial (by determinant expansion or CAS) is \(\chi_A(\lambda) = (15-\lambda)(-3-\lambda)^2\text{.}\) Hence the eigenvalues are \(\lambda_1 = 15\) and the repeated eigenvalue \(\lambda_2 = -3\) with algebraic multiplicity 2.

🔗

For \(\lambda_1 = 15\text{,}\) solving \((\mathbf{A}-15\mathbf{I})\mathbf{x}=0\) yields an eigenvector which we take as

\begin{equation*} \mathbf{v}_1 = \begin{bmatrix} 2 \\ -2 \\ 1 \end{bmatrix} \end{equation*}

with norm \(\|\mathbf{v}_1\|=3\text{.}\)

🔗

For \(\lambda_2 = -3\text{,}\) solving \((\mathbf{A}+3\mathbf{I})\mathbf{x}=0\) gives the eigenspace

\begin{equation*} \{ (x,\, y,\, 2y-2x)^T : x,y\in \mathbb{R} \}\text{.} \end{equation*}

Choose two independent eigenvectors

\begin{equation*} \mathbf{w}_1 = \begin{bmatrix} 1 \\ 0 \\ -2 \end{bmatrix},\quad \mathbf{w}_2 = \begin{bmatrix} 4 \\ 5 \\ 2 \end{bmatrix}\text{.} \end{equation*}

🔗

They are mutually orthogonal: \(\mathbf{v}_1\cdot \mathbf{w}_1=0\text{,}\) \(\mathbf{v}_1\cdot \mathbf{w}_2=0\text{,}\) \(\mathbf{w}_1\cdot \mathbf{w}_2=0\text{,}\) so we already have an orthogonal eigenbasis.

🔗

Normalize them:

\begin{equation*} \mathbf{e}_1 = \frac{1}{\sqrt{5}}\begin{bmatrix} 1\\0\\-2 \end{bmatrix},\quad \mathbf{e}_2 = \frac{1}{3\sqrt{5}}\begin{bmatrix} 4\\5\\2 \end{bmatrix},\quad \mathbf{e}_3 = \frac{1}{3}\begin{bmatrix} 2\\-2\\1 \end{bmatrix} \end{equation*}

where \(\mathbf{e}_3\) corresponds to eigenvalue \(15\text{,}\) and \(\mathbf{e}_1, \mathbf{e}_2\) correspond to \(-3\text{.}\)

🔗

The spectral (orthogonal) decomposition is

\begin{equation*} \mathbf{A} = 15\,\mathbf{e}_3\mathbf{e}_3^T -3\,(\mathbf{e}_1\mathbf{e}_1^T + \mathbf{e}_2\mathbf{e}_2^T). \end{equation*}

Using the unnormalized \(\mathbf{v}_1\text{,}\) we obtain a concise rank-one update form:

\begin{equation*} \mathbf{A} = -3\mathbf{I} + 2\,\mathbf{v}_1\mathbf{v}_1^T, \quad \text{with } \mathbf{v}_1=(2,-2,1)^T,\; \mathbf{v}_1\mathbf{v}_1^T = \begin{bmatrix}4 & -4 & 2\\ -4 & 4 & -2\\ 2 & -2 & 1\end{bmatrix}. \end{equation*}

🔗