Matrix Inverses and Elementary Matrices

The inverse is usually defined by what it does, $A{A}^{-1}=A^{-1}A=I$, and that says nothing about how to find one, or how to tell whether one exists.

Both answers come from the same place. Reducing $A$ to the identity is a sequence of reversible row operations, and each operation is itself a matrix. So a reduction is a product, the product is the inverse, and running the same operations on $I$ accumulates it. If the reduction stalls, the matrix has dependent columns and no inverse. The method reports failure as informatively as success.

Seeing row operations as matrices pays twice more: it gives the LU factorisation, which is what implementations actually store, and it explains why the inverse of a product reverses the order of its factors.

Building an elementary matrix. Perform the row operation on $I$; the result is the matrix that performs it. To add $-5\times$ row 1 to row 2 in a $2\times 2$ system,

and $EA$ is $A$ with that operation applied. The rule matters because it is easy to build the transpose by mistake: the multiplier $-5$ sits at position $(2,1)$, row 2, the row being changed, and column 1, the row being used.

Left multiplication acts on rows. $EA$ combines rows of $A$; $AE$ combines its columns. Elimination is a row process, so its matrices act from the left, and the order of a reduction is read right to left: $E_k\cdots E_{1}A$ applies $E_1$ first.

Every elementary matrix is invertible. Each operation is undone by another of the same type, swap again, scale by the reciprocal, subtract what was added, so the inverse is elementary too. That is what makes a reduction a product of invertible factors and therefore invertible itself.

Why one-sided invertibility suffices. For square $A$, if $BA=I$ then $A$ has trivial kernel: $Ax=0$ gives $x=BAx=0$. By rank–nullity the rank is full, so $A$ is onto, and a right inverse exists; the uniqueness argument then forces it to equal $B$. So checking one side is enough, and the two-sided definition is stated for clarity rather than necessity.

This fails outside the square finite-dimensional case. The shift on infinite sequences has a left inverse and no right inverse, which is the same phenomenon that breaks rank–nullity there.

Uniqueness. If $BA=I=AC$ then $B=B(AC)=(BA)C=C$, so the inverse is justified. The argument uses only associativity, which is why it holds in any setting where products associate.

Computing $A^{-1}$.

1. Write the augmented array $[A\mid I]$.
2. Reduce the left block to $I$ by row operations, applying each to the whole row.
3. The right block is $A^{-1}$.
4. Verify by computing $A{A}^{-1}$.

If at any point the left block acquires a zero row, stop: $A$ is singular and has no inverse. That is a proof, not a failure of the method.

Why step 3 works. The operations that reduce $A$ to $I$ multiply to $A^{-1}$, and applying them to $I$ accumulates exactly that product. The augmented array runs both reductions in parallel.

For $2\times 2$, a formula.

swap the diagonal, negate the off-diagonal, divide by the determinant. Worth memorising at this size and not worth generalising: the adjugate version for $n\times n$ needs $n^2$ cofactors and is impractical beyond $3\times 3$.

LU factorisation.

1. Eliminate downward using only "add a multiple of a row to a lower row".
2. Record each multiplier $m_{ji}$. The factor used to clear position $(j,i)$.
3. $U$ is the resulting upper triangular matrix.
4. $L$ is unit lower triangular with $m_{ji}$ at position $(j,i)$.

The multipliers go into $L$ with their own sign, not negated: if the step was $R_j\to R_j-2R_i$, then $L_{ji}=2$. This is the step most often reversed, and multiplying $LU$ back out catches it immediately.

What LU is for. Solving $Ax=b$ becomes $Ly=b$ by forward substitution then $Ux=y$ by back substitution, each $O(n^2)$. With several right-hand sides the factorisation is computed once and reused. And $detA$ is the product of $U$'s diagonal, since $L$ has unit diagonal and both are triangular.

When a swap is needed. If a pivot position holds zero, rows must be exchanged, and the factorisation becomes $PA=LU$ with $P$ a permutation. Implementations swap even for a small pivot, because dividing by it magnifies error.

Cost, and why inverses are rarely formed. Elimination to solve one system is about $n^3/3$ operations. Computing $A^{-1}$ is about $n^3$, and then $A^{-1}b$ is a further $n^2$, three times the work for a worse-conditioned result. Form an inverse when the matrix itself is the object of interest; otherwise factor.

Gauss–Jordan. Start from $[A\mid I]$ and record each operation.

So

Verify. , and $A^{-1}A=I$ likewise.

Cross-check by formula. $detA=6-5=1$, so the inverse is . The determinant being 1 is why the entries came out as integers.

---

The same reduction as elementary matrices. Each step was multiplication on the left:

Since $E_4{E}_3{E}_2{E}_{1}A=I$, the product is the inverse. Multiplying out:

Note the order: $E_1$ was applied first and stands rightmost. Reading the product left to right runs the reduction backwards.

And therefore $A$ is a product of elementary matrices, namely $A=E_1^{-1}{E}_2^{-1}{E}_3^{-1}{E}_4^{-1}$. The inverses in reversed order, each itself elementary. That is the general statement: a matrix is invertible exactly when it is such a product.

---

LU for a $3\times 3$.

Eliminate downward. $R_2\to R_2-2R_1$ with multiplier $2$; $R_3\to R_3-(-1)R_1$ with multiplier $-1$; then $R_3\to R_3-(-1)R_2$ with multiplier $-1$. The result is

Check. $LU$ reproduces $C$ exactly, row 2 of $LU$ is $2(2,1,1)+(0,-8,-2)=(4,-6,0)$, and row 3 is $-(2,1,1)-(0,-8,-2)+(0,0,1)=(-2,7,2)$.

A free determinant. $detC=detL\cdot detU=1\times (2)(-8)(1)=-16$, since both are triangular and $L$ has unit diagonal. No cofactor expansion was needed.

Theorem (invertible matrix equivalences). For a square $A$ of size $n$, the following are equivalent:

1. $A$ is invertible;
2. $A$ reduces to $I$ by row operations;
3. $A$ is a product of elementary matrices;
4. $\mathrm{rank}A=n$;
5. the columns of $A$ are independent;
6. $Ax=0$ has only the solution $x=0$;
7. $Ax=b$ has exactly one solution for every $b$;
8. $detA\ne 0$;
9. $0$ is not an eigenvalue of $A$.

A circuit of implications. $(1)\Rightarrow (6)$: $Ax=0$ gives $x=A^{-1}Ax=0$. $(6)\Rightarrow (5)$ is the definition of independence applied to the columns. $(5)\Rightarrow (4)$ is the definition of rank. $(4)\Rightarrow (2)$: full rank means a pivot in every column, and back-substitution clears above them. $(2)\Rightarrow (3)$: the reduction is $E_k\cdots E_{1}A=I$, so $A=E_1^{-1}\cdots E_k^{-1}$, a product of elementary matrices. $(3)\Rightarrow (1)$: each factor is invertible, so the product is.

The remaining three attach to the circuit: $(8)$ by multiplicativity, since $detA\cdot det{A}^{-1}=1$ forces $detA\ne 0$, and conversely a zero determinant means dependent columns; $(9)$ because $0$ being an eigenvalue means a nonzero kernel vector, which is $(6)$ failing; and $(7)$ because uniqueness for one $b$ is $(6)$ and existence for all $b$ is $(4)$. $◼$

The list functions as a whole. Each entry is cheap to check in some situations and expensive in others, and the theorem says any one settles all the rest. A determinant for a small matrix, a rank for a reduced one, an eigenvalue when the spectrum is already known.

Theorem (order reversal). If $A$ and $B$ are invertible then so is $AB$, with

Proof. $(AB)(B^{-1}{A}^{-1})=A(B{B}^{-1})A^{-1}=A{A}^{-1}=I$, and the other side is the same computation. Uniqueness of inverses does the rest. $◼$

The order must reverse because the inner factors have to meet: in $(AB)(B^{-1}{A}^{-1})$ it is $B$ and $B^{-1}$ that are adjacent. Writing $A^{-1}{B}^{-1}$ instead leaves $B$ next to $A^{-1}$, which cancels nothing.

Corollary. $(A_1{A}_2\cdots A_k{)}^{-1}=A_k^{-1}\cdots A_2^{-1}{A}_1^{-1}$, by induction, which is what turns a reduction $E_k\cdots E_{1}A=I$ into the factorisation $A=E_1^{-1}\cdots E_k^{-1}$.

The analogous rules. $(A^{\mathsf{T}}{)}^{-1}=(A^{-1}{)}^{\mathsf{T}}$, and $(AB{)}^{\mathsf{T}}=B^{\mathsf{T}}{A}^{\mathsf{T}}$. The transpose reverses order too. Both reversals have the same cause: these operations turn a composition around.

Diagonal. $\mathrm{diag}(d_1,\dots ,d_n{)}^{-1}=\mathrm{diag}(1/d_1,\dots ,1/d_n)$, provided every $d_i\ne 0$. A single zero on the diagonal makes the matrix singular, since that column is zero.

Elementary. Each is inverted by undoing its operation: , and a swap matrix is its own inverse.

Orthogonal. $Q^{-1}=Q^{\mathsf{T}}$, by definition of orthogonality. The cheapest inverse available, and the reason $U$ and $V$ in a singular value decomposition are convenient to work with.

Triangular. The inverse of an invertible triangular matrix is triangular of the same kind, and its diagonal entries are the reciprocals. For

the diagonal holds $1/2$ and $1/4$ as promised, and the matrix is still upper triangular.

A $2\times 2$ with determinant 1. , integer entries throughout, because dividing by $det=1$ introduces no fractions. Any integer matrix with determinant $\pm 1$ inverts to an integer matrix, which is why such matrices are the invertible ones over the integers.

Singular cases. has determinant 0 and its second row is twice the first, so reduction produces a zero row. has a zero row already, and has equal rows. Each fails every entry of the equivalence list at once, rank below $n$, zero determinant, nontrivial kernel, $0$ an eigenvalue.

What the pattern shows. For a structured matrix the inverse is usually structured the same way and readable without elimination. Recognising the structure first is what saves the computation, and it is also what tells you the answer is wrong when a triangular matrix produces a full inverse.

Preserving the order. Writing $(AB{)}^{-1}=A^{-1}{B}^{-1}$. With and , the product is with inverse . Computing $B^{-1}{A}^{-1}$ gives exactly that; computing $A^{-1}{B}^{-1}$ gives , a different matrix. The inner factors must be adjacent to cancel.

Inverting entrywise. Replacing each entry by its reciprocal. For that would give , whose product with $A$ is nothing like $I$. The inverse inverts the map, not the numbers; the entrywise reciprocal has no algebraic meaning here, and a zero entry makes it undefined for matrices that are perfectly invertible.

Negating the LU multipliers. After $R_2\to R_2-2R_1$, recording $-2$ in $L$ rather than $2$. Multiplying $LU$ back out then fails to reproduce $A$. The check exists precisely because this slip is silent otherwise. $L$ holds the multipliers used, with their own signs.

Treating a stalled reduction as a dead end. Reaching a zero row on the left of $[A\mid I]$ and concluding the arithmetic went wrong. It is a proof: a zero row means the rows are dependent, so the rank is below $n$ and no inverse exists. The correct response is to report singularity, not to restart.

Cancelling a singular matrix. From $AB=AC$, concluding $B=C$. That step multiplies by $A^{-1}$ and needs $A$ invertible. With , taking $B=I$ and gives , both products equal , while $B\ne C$. Cancellation is a theorem about invertible matrices, not a rule of algebra.

What unites them. The first and last forget that matrix multiplication is neither commutative nor cancellative without invertibility. The middle three treat a matrix as a container of numbers rather than as a map, which is the habit the elementary-matrix reading is meant to displace.