Skip to main content

Linear Algebra For Undergraduate Students

Yilan(Peter) Tan

Section 7.4 Quadratic Forms

One of the most important applications of orthogonal diagonalization is the study of quadratic forms. These expressions appear naturally in optimization, geometry, mechanics, and statistics. The spectral theorem lets us rewrite a complicated quadratic form in a coordinate system where all cross terms disappear.
🔗

Definition 7.4.1. Quadratic Form.

A quadratic form in \(n\) variables is a function of the form
\begin{equation*} Q(x_1, x_2, \ldots, x_n) = \sum_{i=1}^{n} a_{ii}x_i^2 + 2\sum_{1 \le i < j \le n} a_{ij}x_ix_j, \end{equation*}
where the coefficients \(a_{ij}\) are real numbers.
🔗
Equivalently, if \(\mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}\text{,}\) then every quadratic form can be written as
\begin{equation*} Q(\mathbf{x}) = \mathbf{x}^T A\mathbf{x} \end{equation*}
for a unique symmetric matrix \(A = [a_{ij}]\text{.}\)
🔗
A quadratic form is said to be in standard form if it has no cross terms \(x_ix_j\) with \(i \ne j\text{.}\)
🔗
🔗

Example 7.4.2. A First Example of a Quadratic Form.

Consider
\begin{equation*} Q(x_1, x_2, x_3) = 2x_1^2 + 6x_1x_2 - 4x_1x_3 + 5x_2^2 + 8x_2x_3 + 3x_3^2. \end{equation*}
🔗
Show that this is a quadratic form, identify its cross terms, and write it in the form \(Q(\mathbf{x})=\mathbf{x}^TA\mathbf{x}\text{.}\)
🔗
Solution.
This is a quadratic form because every term has total degree \(2\text{.}\) The square terms are \(2x_1^2\text{,}\) \(5x_2^2\text{,}\) and \(3x_3^2\text{.}\)
🔗
The cross terms are \(6x_1x_2\text{,}\) \(-4x_1x_3\text{,}\) and \(8x_2x_3\text{.}\) Since cross terms are present, this quadratic form is not yet in standard form.
🔗
To write it as \(Q(\mathbf{x})=\mathbf{x}^TA\mathbf{x}\text{,}\) we place the coefficients of the square terms on the diagonal, and we halve each cross-term coefficient to get the off-diagonal entries. Thus the associated symmetric matrix is
\begin{equation*} A=\begin{bmatrix}2 \amp 3 \amp -2 \\ 3 \amp 5 \amp 4 \\ -2 \amp 4 \amp 3\end{bmatrix}. \end{equation*}
🔗
Therefore, if \(\mathbf{x}=\begin{bmatrix}x_1 \\ x_2 \\ x_3\end{bmatrix}\text{,}\) then
\begin{equation*} Q(\mathbf{x})=\begin{bmatrix}x_1 \amp x_2 \amp x_3\end{bmatrix} \begin{bmatrix}2 \amp 3 \amp -2 \\ 3 \amp 5 \amp 4 \\ -2 \amp 4 \amp 3\end{bmatrix} \begin{bmatrix}x_1 \\ x_2 \\ x_3\end{bmatrix}. \end{equation*}
🔗
🔗
🔗

Remark 7.4.3.

In the matrix expression \(\mathbf{x}^TA\mathbf{x}\text{,}\) the coefficient of \(x_i^2\) is \(a_{ii}\text{,}\) while the coefficient of the cross term \(x_ix_j\) for \(i \ne j\) is \(2a_{ij}\text{.}\) This is why symmetric matrices are the natural matrices associated with quadratic forms.
🔗
🔗

Proof.

By the spectral theorem (Theorem 7.3.11), a real symmetric matrix can be written as \(A = PDP^T\) with \(P\) orthogonal. Therefore
\begin{align*} Q(\mathbf{x}) \amp= \mathbf{x}^TA\mathbf{x}\\ \amp= \mathbf{x}^T(PDP^T)\mathbf{x}\\ \amp= (P^T\mathbf{x})^T D(P^T\mathbf{x})\\ \amp= \mathbf{y}^TD\mathbf{y} \end{align*}
where \(\mathbf{y} = P^T\mathbf{x}\text{.}\) Since \(D\) is diagonal, this expands to
\begin{equation*} \lambda_1 y_1^2 + \lambda_2 y_2^2 + \cdots + \lambda_n y_n^2, \end{equation*}
which has no cross terms.
🔗
🔗

Proof.

By Theorem 7.4.4, we can write
\begin{equation*} Q(\mathbf{x}) = \lambda_1 y_1^2 + \lambda_2 y_2^2 + \cdots + \lambda_n y_n^2 \end{equation*}
where \(\mathbf{y} = P^T\mathbf{x}\) for an orthogonal matrix \(P\text{.}\) Since orthogonal matrices preserve length by Theorem 7.3.6, the condition \(\|\mathbf{x}\| = 1\) implies \(\|\mathbf{y}\| = 1\text{.}\) Hence
\begin{equation*} y_1^2 + y_2^2 + \cdots + y_n^2 = 1. \end{equation*}
🔗
Thus \(Q(\mathbf{x})\) is a weighted average of the eigenvalues with nonnegative weights \(y_1^2, \ldots, y_n^2\) summing to \(1\text{.}\) Therefore it must lie between the smallest and largest eigenvalues.
🔗
If \(\mathbf{y} = \mathbf{e}_k\text{,}\) then \(Q(\mathbf{x}) = \lambda_k\text{.}\) Since \(\mathbf{x} = P\mathbf{y}\text{,}\) these extreme values are attained when \(\mathbf{x}\) is a unit eigenvector of \(A\) corresponding to the largest or smallest eigenvalue.
🔗
🔗

Insight 7.4.6. The Key Idea.

The change of variables \(\mathbf{y} = P^T\mathbf{x}\) does not distort geometry, because \(P\) is orthogonal. It only rotates or reflects the coordinate system. In the new coordinates, the mixed terms disappear, so the quadratic form becomes much easier to analyze.
🔗
🔗

Example 7.4.7. Finding the Maximum and Minimum of a Quadratic Form.

Let
\begin{equation*} Q(x_1, x_2, x_3) = 3x_1^2 - 4x_1x_2 + 8x_1x_3 + 6x_2^2 + 4x_2x_3 + 3x_3^2. \end{equation*}
Find the maximum and minimum values of \(Q\) subject to the constraint
\begin{equation*} x_1^2 + x_2^2 + x_3^2 = 1. \end{equation*}
🔗
Solution.
Step 1: Write \(Q\) in matrix form. The associated symmetric matrix is
\begin{equation*} A = \begin{bmatrix} 3 \amp -2 \amp 4 \\ -2 \amp 6 \amp 2 \\ 4 \amp 2 \amp 3 \end{bmatrix}, \end{equation*}
so \(Q(\mathbf{x}) = \mathbf{x}^TA\mathbf{x}\text{.}\)
🔗
Step 2: Orthogonally diagonalize \(A\text{.}\) In Example 7.3.13, we already found an orthogonal matrix \(P\) and a diagonal matrix
\begin{equation*} D = \begin{bmatrix} 7 \amp 0 \amp 0 \\ 0 \amp 7 \amp 0 \\ 0 \amp 0 \amp -2 \end{bmatrix} \end{equation*}
such that \(A = PDP^T\text{.}\)
🔗
Step 3: Pass to the new coordinates \(\mathbf{y} = P^T\mathbf{x}\text{.}\) By Theorem 7.4.4,
\begin{equation*} Q(\mathbf{x}) = \mathbf{y}^T D\mathbf{y} = 7y_1^2 + 7y_2^2 - 2y_3^2. \end{equation*}
Since \(P\) is orthogonal and \(\|\mathbf{x}\| = 1\text{,}\) we also have \(\|\mathbf{y}\| = 1\text{,}\) so
\begin{equation*} y_1^2 + y_2^2 + y_3^2 = 1. \end{equation*}
🔗
Therefore
\begin{equation*} Q(\mathbf{x}) = 7y_1^2 + 7y_2^2 - 2y_3^2 \leq 7y_1^2 + 7y_2^2 +7y_3^2\leq 7. \end{equation*}
The inequality holds if \(y_3 = 0\text{,}\) giving
\begin{equation*} Q_{\max} = 7, \end{equation*}
\(Q(\mathbf{x}) = 7y_1^2 + 7y_2^2 - 2y_3^2\geq -2y_1^2 -2y_2^2 - 2y_3^2y_3^2 = -2\text{.}\) The inequality holds if \(y_1 = 0 = y_2\text{,}\) giving
\begin{equation*} Q_{\min} = -2. \end{equation*}
🔗
The maximum occurs for any unit vector in the eigenspace corresponding to \(\lambda = 7\text{,}\) and the minimum occurs for a unit eigenvector corresponding to \(\lambda = -2\text{.}\)
🔗
🔗
🔗

Example 7.4.8. Finding the Standard Form of a Quadratic Form.

Find the standard form of
\begin{equation*} f(x_1, x_2, x_3) = x_1^2 + 4x_1x_2 + 4x_1x_3 + x_2^2 + 4x_2x_3 + x_3^2. \end{equation*}
🔗
Solution.
The associated symmetric matrix is
\begin{equation*} A = \begin{bmatrix} 1 \amp 2 \amp 2 \\ 2 \amp 1 \amp 2 \\ 2 \amp 2 \amp 1 \end{bmatrix}. \end{equation*}
🔗
A convenient orthonormal eigenbasis is
\begin{gather*} \mathbf{u}_1 = \frac{1}{\sqrt{3}}\begin{bmatrix} 1 \\ 1 \\ 1 \end{bmatrix},\\ \mathbf{u}_2 = \frac{1}{\sqrt{2}}\begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix},\\ \mathbf{u}_3 = \frac{1}{\sqrt{6}}\begin{bmatrix} 1 \\ 1 \\ -2 \end{bmatrix}, \end{gather*}
with corresponding eigenvalues \(5, -1, -1\text{.}\)
🔗
If \(P = [\mathbf{u}_1 \mid \mathbf{u}_2 \mid \mathbf{u}_3]\) and \(\mathbf{y} = P^T\mathbf{x}\text{,}\) then
\begin{equation*} P^TAP = \operatorname{diag}(5, -1, -1), \end{equation*}
so the standard form is
\begin{equation*} f = 5y_1^2 - y_2^2 - y_3^2. \end{equation*}
🔗
🔗
🔗

Checkpoint 7.4.9. Another Standard-Form Computation.

Find the standard form of the quadratic form
\begin{equation*} f(x_1, x_2, x_3) = x_1^2 + x_2^2 + x_3^2 + 2x_1x_2. \end{equation*}
🔗
Hint.
First write \(f(\mathbf{x}) = \mathbf{x}^TA\mathbf{x}\) with \(A\) symmetric. Then find the eigenvalues and an orthonormal eigenbasis for \(A\text{,}\) and apply Theorem 7.4.4.
🔗
🔗
🔗
🔗
PrevTopNext