Quadratic Forms and Extremal Values

Section 7.4 Quadratic Forms and Extremal Values

A quadratic form in \(n\) real variables is a homogeneous polynomial of degree two. It can always be written using a symmetric matrix so that the form equals \(\mathbf{x}^T A \mathbf{x}\text{.}\) Orthogonal diagonalization transforms a quadratic form into a sum of squares without cross terms, revealing its geometric nature and allowing us to analyze extrema under constraints like \(\|\mathbf{x}\|=1\text{.}\)

🔗

Subsection 7.4.1 Definition and Standard Form

A quadratic form in variables \(x_1,\dots,x_n\) has the expanded expression

\begin{gather*} Q(x_1,\dots,x_n)= a_{11}x_1^2 + 2a_{12}x_1x_2 + 2a_{13}x_1x_3 + \cdots + 2a_{1n}x_1x_n\\ \quad + a_{22}x_2^2 + 2a_{23}x_2x_3 + \cdots + a_{nn}x_n^2\quad. \end{gather*}

Grouping terms gives a matrix representation \(Q(\mathbf{x})= \mathbf{x}^T A \mathbf{x}\text{,}\) where \(A=(a_{ij})\) is symmetric (\(a_{ij}=a_{ji}\)). If a form contains no cross terms \(x_i x_j\) for \(i\neq j\text{,}\) it is said to be in standard form.

🔗

Any quadratic form can be converted to standard form by an orthogonal change of variables. If \(A = PDP^T\) with \(P\) orthogonal and \(D=\operatorname{diag}(\lambda_1,\dots,\lambda_n)\text{,}\) then with \(\mathbf{y}=P^T\mathbf{x}\text{,}\) we have

\begin{equation*} Q(\mathbf{x}) = \mathbf{x}^T A \mathbf{x} = (P\mathbf{y})^T A (P\mathbf{y}) = \mathbf{y}^T D \mathbf{y} = \lambda_1 y_1^2 + \cdots + \lambda_n y_n^2. \end{equation*}

🔗

Subsection 7.4.2 Rayleigh Quotient and Extremal Values

The Rayleigh quotient of a symmetric matrix \(A\) is the scalar function

\begin{equation*} R_A(\mathbf{x}) = \frac{\mathbf{x}^T A \mathbf{x}}{\mathbf{x}^T \mathbf{x}}, \quad \mathbf{x}\neq \mathbf{0}. \end{equation*}

When \(\|\mathbf{x}\|=1\text{,}\) we have \(R_A(\mathbf{x}) = \mathbf{x}^T A \mathbf{x} = Q(\mathbf{x})\text{.}\)

🔗

Theorem 7.4.1. Extrema on the Unit Sphere.

Let \(A\) be real symmetric with eigenvalues \(\lambda_1 \geq \lambda_2 \geq \cdots \geq \lambda_n\text{.}\) Then for all unit vectors \(\mathbf{x}\text{,}\)

\begin{equation*} \lambda_n \leq \mathbf{x}^T A \mathbf{x} \leq \lambda_1. \end{equation*}

Moreover, the maximum value \(\lambda_1\) is attained exactly at the eigenvectors corresponding to \(\lambda_1\text{,}\) and the minimum value \(\lambda_n\) is attained exactly at the eigenvectors corresponding to \(\lambda_n\text{.}\)

🔗

This result follows immediately after orthogonal diagonalization: write \(A=PDP^T\text{,}\) set \(\mathbf{y}=P^T\mathbf{x}\) (so \(\|\mathbf{y}\|=1\)), then \(\mathbf{x}^T A \mathbf{x}=\sum_{i=1}^n \lambda_i y_i^2\) with \(\sum y_i^2=1\text{.}\) The sum is a convex combination of the eigenvalues, bounded between the largest and smallest.

🔗

Subsection 7.4.3 Example: Maximizing and Minimizing a Quadratic Form

Consider the quadratic form

\begin{equation*} Q(x_1,x_2,x_3)=3x_1^2 -4x_1x_2 + 8x_1x_3 + 6x_2^2 + 4x_2x_3 + 3x_3^2 \end{equation*}

subject to the constraint \(x_1^2+x_2^2+x_3^2=1\text{.}\)

🔗

1. Identify the symmetric matrix \(A\) so that \(Q=\mathbf{x}^T A \mathbf{x}\text{:}\)

\begin{equation*} A = \begin{bmatrix}3 & -2 & 4\\ -2 & 6 & 2 \\ 4 & 2 & 3\end{bmatrix}. \end{equation*}

2. Orthogonally diagonalize \(A=PDP^T\text{.}\) A suitable diagonalization (one possible choice) is

\begin{equation*} P = \begin{bmatrix} \frac{1}{\sqrt{2}} & -\tfrac{1}{\sqrt{18}} & -\tfrac{2}{3} \\ 0 & \tfrac{4}{\sqrt{18}} & -\tfrac{1}{3} \\ \frac{1}{\sqrt{2}} & \tfrac{1}{\sqrt{18}} & \tfrac{2}{3} \end{bmatrix}, \quad D=\begin{bmatrix}7 & 0 & 0\\ 0 & 7 & 0 \\ 0 & 0 & -2\end{bmatrix}. \end{equation*}

3. With \(\mathbf{y}=P^T\mathbf{x}\) and \(\|\mathbf{y}\|=1\text{,}\)

\begin{equation*} Q(\mathbf{x}) = 7y_1^2 + 7y_2^2 - 2 y_3^2. \end{equation*}

4. Since \(y_1^2+y_2^2+y_3^2=1\text{,}\) the maximum is \(7\) (take \(y_3=0\text{,}\) \(y_1=1\) or \(y_2=1\)) and the minimum is \(-2\) (take \(y_3=1\)). The maximizing vectors are those in the eigenspace of eigenvalue \(7\text{;}\) minimizing vectors lie in the eigenspace of eigenvalue \(-2\text{.}\)

🔗

Subsection 7.4.4 Definiteness

The sign pattern of the eigenvalues of a symmetric matrix \(A\) classifies the quadratic form \(Q(\mathbf{x})=\mathbf{x}^T A \mathbf{x}\text{:}\)

Positive definite: all eigenvalues > 0.

🔗
Positive semidefinite: all eigenvalues \geq 0 and at least one is 0.

🔗
Negative definite: all eigenvalues < 0.

🔗
Negative semidefinite: all eigenvalues \leq 0 and at least one is 0.

🔗
Indefinite: mixture of positive and negative eigenvalues.

🔗

🔗

In optimization, definiteness provides second-derivative tests for classifying critical points of multivariable real-valued functions via their Hessian matrices.

🔗

Subsection 7.4.5 Exercises

Checkpoint 7.4.2.

Express \(Q(x_1,x_2)=5x_1^2+8x_1x_2+3x_2^2\) as \(\mathbf{x}^T A \mathbf{x}\text{,}\) diagonalize \(A\text{,}\) and classify the form.

🔗

Checkpoint 7.4.3.

Find the maximum and minimum of \(Q(x,y)=4x^2-6xy+5y^2\) on the unit circle.

🔗

Checkpoint 7.4.4.

Give an example of an indefinite quadratic form in \(\mathbb{R}^3\) and justify the classification.

🔗

Prev Top Next