Quadratic Forms and Definiteness

$5x^2+4xy+5y^2$ is always positive for nonzero $(x,y)$. $x^2-y^2$ takes both signs. Neither fact is visible from the coefficients: both have positive squared terms, and the second has no cross term at all.

Writing the expression as $x^{\mathsf{T}}Ax$ with $A$ symmetric turns the question into one about eigenvalues, and the spectral theorem answers it. A symmetric matrix can be diagonalised by a rotation, so there is always a set of perpendicular directions in which the cross terms disappear and the form becomes a weighted sum of squares. The weights are the eigenvalues, and their signs settle everything.

That is why this classification appears wherever second-order behaviour matters: a critical point is a minimum, a maximum or a saddle according to whether the Hessian's form is positive definite, negative definite or indefinite.

Reading a form into a matrix. The coefficient of $x_i^2$ goes on the diagonal at $a_{ii}$. The coefficient of $x_i{x}_j$ is halved and placed at both $a_{ij}$ and $a_{ji}$, because the expansion of $x^{\mathsf{T}}Ax$ collects those two entries into one term:

Forgetting to halve is the most common error, and it changes the matrix without changing the intent: represents $5x^2+8xy+5y^2$, a different form with different eigenvalues.

Why the symmetric representative is canonical. For any square $M$, the number $x^{\mathsf{T}}Mx$ is a $1\times 1$ matrix and therefore equal to its own transpose, $x^{\mathsf{T}}{M}^{\mathsf{T}}x$. So $M$ and $M^{\mathsf{T}}$ define the same function, and so does their average $\frac{1}{2}(M+M^{\mathsf{T}})$, which is symmetric.

Concretely, gives the same form as : both evaluate to 5, 5, 14 and 89 at $(1,0)$, $(0,1)$, $(1,1)$ and $(2,3)$. The symmetric one is preferred because only it has the properties the classification uses, real eigenvalues and an orthonormal eigenbasis. The eigenvalues of the non-symmetric version are both 5, which says nothing about this form.

What the matrix is a property of. Given the form, the symmetric matrix is unique. Given a non-symmetric matrix, the form is still determined, but the matrix is not recoverable from it, information about the antisymmetric part is discarded, and it never affected the values.

Evaluation is a double sum. $Q(x)=\sum _{i,j}{a}_{ij}{x}_i{x}_j$, so a $3\times 3$ form has three squared terms and three cross terms. When checking a classification by evaluating at specific vectors, it is usually quicker to substitute into the original expression than into the matrix product.

Spectral theorem. Every real symmetric matrix $A$ has real eigenvalues and an orthonormal basis of eigenvectors, so $A=PD{P}^{\mathsf{T}}$ with $P$ orthogonal and $D$ diagonal.

Real eigenvalues. Suppose $Av=\lambda v$ with $v\ne 0$, allowing complex entries. Then

Taking the conjugate transpose of the left side and using $A^{\mathsf{T}}=A$ with $A$ real gives $\overline{{\overline{v}}^{\mathsf{T}}Av}={\overline{v}}^{\mathsf{T}}Av$, so that number is real. Since is real and positive, $\lambda$ is real.

Orthogonal eigenvectors for distinct eigenvalues. If $A{v}_1={\lambda }_1{v}_1$ and $A{v}_2={\lambda }_2{v}_2$ then

using symmetry in the middle step. So $({\lambda }_1-{\lambda }_2)v_1^{\mathsf{T}}{v}_2=0$, and distinct eigenvalues force $v_1⟂v_2$. Within a repeated eigenvalue's eigenspace an orthonormal basis is chosen by Gram–Schmidt, and the full argument that the eigenvectors span proceeds by induction on dimension. $◼$

This is the theorem the diagonalization and SVD units invoked without proof; here it is discharged.

Corollary (principal axes). Substituting $y=P^{\mathsf{T}}x$ into $Q(x)=x^{\mathsf{T}}Ax$ gives

Every real quadratic form is a weighted sum of squares in suitable perpendicular coordinates, with the eigenvalues as weights.

Corollary (classification). for all $x\ne 0$ exactly when every . The forward direction: take $x=P{e}_i$, giving $Q={\lambda }_i$, so each eigenvalue is a value of the form and must be positive. The converse: in principal coordinates $Q=\sum _i{\lambda }_i{y}_i^2$, which is positive whenever some $y_i\ne 0$, and $y=0$ only when $x=0$ since $P$ is invertible. The other classes follow identically.

Why orthogonality matters here. Completing the square also removes cross terms, and Sylvester's law of inertia guarantees it produces the same counts of positive, negative and zero coefficients. But its change of variables is not orthogonal, so it does not preserve lengths: the axes it produces are not perpendicular and the level set it describes is a sheared picture of the true one. The eigenvalues are recoverable only from the orthogonal route.

1. Build the symmetric matrix. Squared coefficients on the diagonal; each cross-term coefficient halved and placed symmetrically.
2. If given a matrix instead, check it is symmetric. If not, replace it by $\frac{1}{2}(M+M^{\mathsf{T}})$. The form is unchanged.
3. Find the eigenvalues.
4. Classify by their signs: all positive, all negative, mixed, or some zero with the rest of one sign.
5. For the axes, compute an orthonormal eigenbasis; these are the principal directions, and $Q=\sum _i{\lambda }_i{y}_i^2$ in those coordinates.
6. Confirm by evaluating $Q$ on a few vectors, including an eigenvector, $Q(v_i)={\lambda }_i$ for a unit eigenvector.

Shortcuts for $2\times 2$. With $t=\mathrm{tr}A$ and $d=detA$:

These follow from $d={\lambda }_1{\lambda }_2$ and $t={\lambda }_1+{\lambda }_2$: a negative product forces opposite signs, and a positive product with the trace's sign settles which.

The analogue for larger matrices is Sylvester's criterion: positive definite exactly when every leading principal minor is positive. It does not extend naively to semidefiniteness, where all principal minors, not only the leading ones, must be checked.

Checks worth running.

• $\sum _i{\lambda }_i=\mathrm{tr}A$ and $\prod _i{\lambda }_i=detA$, both free.
• A positive definite matrix has every diagonal entry positive, since $Q(e_i)=a_{ii}$. The converse fails, which is what the non-example block is about.
• For a semidefinite verdict, exhibit the nonzero vector on which $Q$ vanishes; it lies in the kernel.

A note on what not to do. Do not classify by inspecting the coefficients of the squared terms. $2x^2-4xy+5y^2$ has a negative cross term and is positive definite; $x^2+4xy+y^2$ has all-positive squared coefficients and is indefinite. The cross terms are not a perturbation. They are half the matrix.

Step 1: the symmetric matrix. Squared coefficients 5 and 5 on the diagonal; the cross coefficient 4 halved to 2 off it:

Check. $Q(1,0)=5$, $Q(0,1)=5$, $Q(1,1)=5+4+5=14$, and $x^{\mathsf{T}}Ax$ at $(1,1)$ gives $5+2+2+5=14$.

Step 2–3: eigenvalues. $\mathrm{tr}A=10$, $detA=25-4=21$, so $p(\lambda )={\lambda }^2-10\lambda +21=(\lambda -7)(\lambda -3)$:

Check. $7+3=10=\mathrm{tr}A$ and $7\times 3=21=detA$.

Step 4: classify. Both eigenvalues positive, so $Q$ is positive definite: for every $(x,y)\ne (0,0)$. The $2\times 2$ shortcut agrees, and .

Step 5: principal axes. For $\lambda =7$: , giving $v_1\propto (1,1)$. For $\lambda =3$: , giving $v_2\propto (1,-1)$. Normalising,

perpendicular as the spectral theorem promised. In these coordinates

Step 6: confirm. Evaluating the original form on the unit eigenvectors:

Each eigenvalue is literally the value of the form along its own axis.

The geometry. The level set $Q=1$ is an ellipse with axes along $(1,1)$ and $(1,-1)$, of semi-axis lengths $1/\sqrt{7}\approx 0.378$ and $1/\sqrt{3}\approx 0.577$. The larger eigenvalue gives the shorter axis: the form reaches 1 sooner in the direction where it grows faster.

Completing the square, for contrast.

Checking at $(2,-1)$: directly $20-8+5=17$, and $5(2-\frac{2}{5}{)}^2+\frac{21}{5}=5\cdot \frac{64}{25}+\frac{21}{5}=\frac{64}{5}+\frac{21}{5}=17$.

Both coefficients, 5 and $21/5$, are positive, confirming positive definiteness. But they are not the eigenvalues 7 and 3, and the variables $x+\frac{2}{5}y$ and $y$ are not perpendicular directions. Sylvester's law guarantees the signs agree; nothing guarantees the values do.

Positive definite, despite a negative cross term. $Q=2x^2-4xy+5y^2$ has matrix , trace 7 and determinant $10-4=6$. The characteristic polynomial ${\lambda }^2-7\lambda +6$ factors as $(\lambda -6)(\lambda -1)$, so the eigenvalues are 6 and 1: positive definite.

The sign of the cross term was irrelevant. Checking a few values: $Q(1,0)=2$, $Q(0,1)=5$, $Q(1,1)=2-4+5=3$, all positive, and the minimum over the unit circle is 1, the smaller eigenvalue.

Its principal axes are $\frac{1}{\sqrt{5}}(1,-2)$ for $\lambda =6$ and $\frac{1}{\sqrt{5}}(2,1)$ for $\lambda =1$, perpendicular as the theorem requires, and $Q$ evaluates to exactly 6 and 1 on them.

Indefinite. $Q=x^2-y^2$ has matrix $\mathrm{diag}(1,-1)$, eigenvalues 1 and $-1$. Already diagonal, so the standard axes are the principal ones. , , and $Q(1,1)=0$. The form vanishes on the lines $y=\pm x$ without being semidefinite, because it takes both signs elsewhere. Its level set $Q=1$ is a hyperbola.

Negative definite. $Q=-3x^2-2xy-3y^2$ has matrix , trace $-6$, determinant $9-1=8$, eigenvalues $-2$ and $-4$. Every value is negative: $Q(1,0)=-3$, $Q(1,1)=-8$, $Q(1,-1)=-4$. Note the determinant is positive here, for a $2\times 2$, a positive determinant means same-signed eigenvalues, and the diagonal entry settles which sign.

Positive semidefinite in three variables. $Q=x^2+y^2+z^2+2xy$ has matrix

which is block diagonal: the $2\times 2$ block has eigenvalues 2 and 0, and the isolated entry gives 1. So the eigenvalues are 2, 1, 0, semidefinite, not definite.

The zero eigenvalue's eigenvector is $(1,-1,0)$, and indeed $Q(1,-1,0)=1+1+0-2=0$ with the vector nonzero. Along the whole line through it the form vanishes; everywhere off that line it is strictly positive, as $Q(1,1,0)=4$, $Q(0,0,3)=9$ and $Q(1,-1,5)=25$ show.

Cross terms decide as much as squared ones; a positive determinant does not mean positive definite; and the gap between definite and semidefinite is a single nonzero vector on which the form vanishes. None of this is visible without the eigenvalues.

Reading definiteness off the squared coefficients. " $x^2+4xy+y^2$ has positive coefficients on $x^2$ and $y^2$, so it is positive definite." Its matrix is with , so the eigenvalues are $3$ and $-1$: indefinite. Evaluating confirms it, and .

The converse error is just as common: $2x^2-4xy+5y^2$ has a negative cross term and is positive definite, with eigenvalues $6$ and $1$.

Forgetting to halve the cross coefficient. Writing for $5x^2+4xy+5y^2$. That matrix represents $5x^2+8xy+5y^2$, whose eigenvalues are $9$ and $1$ rather than $7$ and $3$. The verdict happens to survive here, both are still positive, but the axes, the level set and any quantitative conclusion are wrong.

Using a non-symmetric matrix's eigenvalues. defines the same form as , as evaluating at any point confirms. Its eigenvalues are both 5, which are neither 7 nor 3 and say nothing about this form. Definiteness is a property of the symmetric representative; the eigenvalues of any other representative are an artefact of how it was written.

Calling a semidefinite form definite. $(x+y{)}^2$ has matrix with eigenvalues $2$ and $0$. It is never negative, but it is not positive definite: $Q(1,-1)=0$ with $(1,-1)\ne 0$. The distinction is the difference between and $\ge$, and it matters. A positive semidefinite Hessian does not establish a strict minimum.

Taking completing-the-square coefficients as eigenvalues. From $5(x+\frac{2}{5}y{)}^2+\frac{21}{5}{y}^2$, reporting the eigenvalues as $5$ and $21/5$. They are $7$ and $3$. Sylvester's law guarantees only that the signs match; the values differ because the change of variables was not orthogonal.

The first three mistake a representation for the object, coefficients, an unhalved matrix, an asymmetric one. The last two mistake a weaker conclusion for a stronger one. All five are caught by the same habit: build the symmetric matrix, take its eigenvalues, and test the verdict on a concrete vector.