Matrices as Operators

A matrix is best understood by what it does: applied to a vector it returns a linear combination of its own columns, weighted by that vector's entries. Reading $A x$ that way turns a wall of arithmetic into one idea, and it is the reading every later result depends on. A constraint system, a basis, a pivot step. Multiplication of matrices is then composition of those actions, which is why order matters and why it is not commutative.

Definition

An $m \times n$ matrix $A$ has entries $a_{i j}$ and columns $A_{1}, \dots, A_{n} \in R^{m}$ . Applied to $x \in R^{n}$ it produces

A x = \sum_{j = 1}^{n} x_{j} A_{j} \in R^{m},

a linear combination of the columns weighted by the entries of $x$ ; equivalently $(A x)_{i} = \sum_{j} a_{i j} x_{j}$ . The product $A B$ is defined when $A$ is $m \times n$ and $B$ is $n \times p$ , giving $(A B) x = A (B x)$ . The transpose $A^{T}$ has $(A^{T})_{i j} = a_{j i}$ . The identity $I$ satisfies $I x = x$ , and $A$ is invertible when some $A^{- 1}$ satisfies $A A^{- 1} = A^{- 1} A = I$ .

Formal statement

$A x = \sum_{j} x_{j} A_{j}$ ; $(A B) x = A (B x)$ ; $(A^{T})_{i j} = a_{j i}$ ; $A A^{- 1} = A^{- 1} A = I$ .

Assumptions and scope

The product $A B$ requires the inner dimensions to agree: $A$ is $m \times n$ and $B$ is $n \times p$ . Mismatched shapes make the product undefined rather than zero.
Matrix multiplication is not commutative. $A B$ and $B A$ may differ, may have different shapes, and one may be defined while the other is not.
Only a square matrix can be invertible, and not every square matrix is. Whether an inverse exists is settled by the independence of the columns, which the independence unit establishes.
$A x$ lives in $R^{m}$ while $x$ lives in $R^{n}$ . The action moves between spaces unless the matrix is square.
Transposing reverses a product: $(A B)^{T} = B^{T} A^{T}$ . The order flips, which is a frequent source of error in derivations.

Worked material

Example

The same product, both readings

Take

A = (\begin{matrix} 2 & 1 & 0 \\ 1 & 3 & 4 \end{matrix}), x = (3, 2, 1) .

By rows. First entry: $(2) (3) + (1) (2) + (0) (1) = 8$ . Second entry: $(1) (3) + (3) (2) + (4) (1) = 13$ . So $A x = (8, 13)$ .

By columns. The columns are $A_{1} = (2, 1)$ , $A_{2} = (1, 3)$ , $A_{3} = (0, 4)$ , and

A x = 3 (\begin{matrix} 2 \\ 1 \end{matrix}) + 2 (\begin{matrix} 1 \\ 3 \end{matrix}) + 1 (\begin{matrix} 0 \\ 4 \end{matrix}) = (\begin{matrix} 6 \\ 3 \end{matrix}) + (\begin{matrix} 2 \\ 6 \end{matrix}) + (\begin{matrix} 0 \\ 4 \end{matrix}) = (\begin{matrix} 8 \\ 13 \end{matrix}) .

Same answer, as it must be.

What each shows. The row computation produces the number. The column computation shows that $(8, 13)$ is reachable by mixing $A$ 's columns with weights $3, 2, 1$ , and therefore that this particular $b$ has at least one solution to $A x = b$ . That is a fact about existence, and the row reading does not surface it.

Note the shapes. $A$ is $2 \times 3$ and $x$ is in $R^{3}$ , so $A x$ lands in $R^{2}$ . The action moved between spaces.

Non-example

Undefined products and rules that do not hold

Multiplying mismatched shapes. A $2 \times 3$ matrix times a $2 \times 2$ matrix is undefined: the inner dimensions, $3$ and $2$ , disagree. The product is not zero and not approximate. There is nothing to compute.

Assuming $A B = B A$ . It generally fails, and for non-square matrices one side may not even exist.

Transposing without reversing. $(A B)^{T} = B^{T} A^{T}$ , not $A^{T} B^{T}$ . The order flips, and derivations that forget it go quietly wrong.

Expecting every square matrix to be invertible. $(\begin{matrix} 1 & 2 \\ 2 & 4 \end{matrix})$ has no inverse: its second column is twice the first, so its columns cannot reach everything. Whether an inverse exists is settled by the independence of the columns.

Cancelling matrices. From $A B = A C$ it does not follow that $B = C$ , unless $A$ is invertible. Matrices are not numbers, and division is not an available move.

Reading $A x$ as a scaling of $x$ . It is a combination of $A$ 's columns, and it generally lives in a different space from $x$ altogether.

Contrast

Two readings of one product

	Row reading	Column reading
Computes	One output entry at a time	The whole output at once
$A x$ is	A list of dot products	A combination of $A$ 's columns
Natural question	What is the answer?	What answers are reachable?
Makes visible	The arithmetic	Existence, span, basis
Used by	Hand computation	Every structural argument later

Both are correct. They are the same sum, grouped differently. Neither is an approximation of the other.

Why the column reading earns its place. Solvability, bases, pivots and degeneracy are all questions about which vectors the columns can reach. Phrased in rows, each becomes a statement about a system of equations having a solution, true, but it hides the object doing the work.

A concrete case. If $A$ 's columns are $(1, 2)$ and $(2, 4)$ , the column reading shows immediately that everything reachable lies along a single line, so $A x = b$ is solvable only for $b$ on that line. The row reading arrives at the same conclusion after elimination, having said nothing about why.

When to reach for rows. When you need the number. The two readings cost the same arithmetic; only one of them tells you something while you do it.

Common errors

Common misconception

Matrix multiplication works like multiplication of numbers, so $A B$ and $B A$ give the same result.

Common misconception

Whether a matrix product is defined is something you find out by trying it: if the arithmetic runs out of entries you transpose something until the shapes agree.

Related units

Requires

Vectors and Linear Combinations

Connected

Linear Independence, Rank, and Bases (part of)
Solving and Characterising Linear Systems (suggested next)

Learn this topic

Matrices as Operators

Used in

Foundations for Operational Research

Sources

Introduction to Linear Algebra (2016)

Matrices as Operators

Assumptions and scope

Worked material

The same product, both readings

Undefined products and rules that do not hold

Two readings of one product

Common errors

Related units

Requires

Connected

Learn this topic

Used in

Sources

Settings

Focus timer

Today

Last seven days

Where the time went

Recent sessions

Settings

Notifications

Your history