Vectors and Linear Combinations

A vector is a list of numbers that behaves like a direction with a length. Adding two of them and scaling one are the only operations linear algebra is built from, and every later object, a constraint, a cost, a basis, is assembled out of those two moves. The dot product turns that geometry into arithmetic: it measures how much of one direction lies along another, which is what lets an optimiser decide where to move without drawing anything.

Definition

A vector $x \in R^{n}$ is an ordered list of $n$ real numbers. Two operations define the space: addition $(x + y)_{i} = x_{i} + y_{i}$ and scalar multiplication $(λ x)_{i} = λ x_{i}$ . A linear combination of $v_{1}, \dots, v_{k}$ is any vector $λ_{1} v_{1} + \dots + λ_{k} v_{k}$ . The dot product is $x^{T} y = \sum_{i = 1}^{n} x_{i} y_{i}$ , and the Euclidean norm is $‖ x ‖ = \sqrt{x^{T} x}$ . Two vectors are orthogonal when $x^{T} y = 0$ .

Formal statement

$x + y$ and $λ x$ entrywise; $λ_{1} v_{1} + \dots + λ_{k} v_{k}$ ; $x^{T} y = \sum_{i} x_{i} y_{i}$ ; $‖ x ‖ = \sqrt{x^{T} x}$ ; orthogonal when $x^{T} y = 0$ .

Assumptions and scope

Addition requires both vectors to have the same number of entries. Vectors of different dimensions cannot be added, and the operation is undefined rather than zero-padded.
The dot product is defined between two vectors of equal dimension and returns a scalar, not a vector. Its result carries the units of the two factors multiplied together.
A dot product of zero means orthogonality, not that either vector is zero. Two nonzero vectors can be perpendicular.
Scaling by zero yields the zero vector, which is a legitimate member of the space and is a linear combination of any set. The independence definition depends on that fact.
The norm is nonnegative and is zero only for the zero vector. It measures length, so a comparison of norms is a comparison of magnitudes and says nothing about direction.

Forms this is expressed in

The same content in several forms. Each makes something visible that the others leave implicit, so moving between them is part of understanding the topic rather than a presentation choice.

geometric

the tip-to-tail path and the right angle are two readings of one pair

A vector read as an arrow rather than a list. Addition becomes placing the second arrow's tail at the first's head; the sum is the arrow from the original tail to the final head.

Scalar multiplication stretches an arrow without turning it, and a negative scalar reverses it along the same line. The translation is mechanical in both directions, which is what later algorithms assume when they speak of moving in a direction.

What the arrow picture does not carry is the arithmetic itself: the dot product is a single number obtained from the lists, and it is the number, not the picture, that decides whether a direction improves an objective. Here it also reports the right angle between the two arrows, which the drawing shows and the arithmetic confirms.

Worked material

Example

Three quantities, three readings

Take $x = (2, - 1, 3)$ and $y = (4, 5, 1)$ in $R^{3}$ .

Addition and scaling.

x + y = (6, 4, 4), 3 x = (6, - 3, 9) .

Each is another vector in $R^{3}$ .

A linear combination. With coefficients $2$ and $- 1$ :

2 x - y = (4, - 2, 6) - (4, 5, 1) = (0, - 7, 5) .

The dot product.

x^{T} y = (2) (4) + (- 1) (5) + (3) (1) = 8 - 5 + 3 = 6.

One number, not a vector. Its sign says the two vectors broadly agree in direction; had it come out zero they would be perpendicular.

The norm.

‖ x ‖ = \sqrt{4 + 1 + 9} = \sqrt{14} \approx 3.74 .

A length, so it is never negative, and it says nothing about which way $x$ points.

Non-example

Undefined vector operations

Adding vectors of different dimensions. $(1, 2) + (1, 2, 3)$ is undefined. The shorter vector is not padded with a zero; the operation simply does not apply.

A dot product that returns a vector. $x^{T} y$ is a single number. Multiplying entry by entry to get $(x_{1} y_{1}, x_{2} y_{2}, \dots)$ is a different operation and is not what any later formula means by the notation.

Concluding that a zero dot product means a zero vector. $(1, 0)^{T} (0, 1) = 0$ with neither vector zero. Orthogonality is a relationship, not a statement that something vanished.

Reading a larger norm as a better direction. The norm measures length only. Scaling a worsening direction makes it longer and no less worsening.

Treating the zero vector as outside the space. It is a perfectly ordinary member, and it is a linear combination of every set, which is the fact the definition of independence is built on.

Contrast

Two products that are easy to confuse

	Dot product $x^{T} y$	Entrywise product
Result	One scalar	A vector
Formula	$\sum_{i} x_{i} y_{i}$	$(x_{1} y_{1}, \dots, x_{n} y_{n})$
Answers	How much of one lies along the other	Nothing geometric
Used for	Costs, constraint tests, orthogonality	Rescaling data entry by entry
Appears in this subject	Constantly	Almost never

Why the confusion is common. Both are written as a product of two vectors, and a calculator will happily do either. The notation $x^{T} y$ is what distinguishes them: the transpose is doing real work, turning a column into a row so the multiplication collapses to a single number.

The test. Ask what the answer is supposed to be. A cost applied to a plan is money, one number. A constraint applied to a point is a quantity to compare with a bound, one number. If the answer should be a single quantity, it is a dot product.

Where the entrywise product does appear. Rescaling: multiplying each entry of a plan by a per-unit factor. That is a legitimate operation and it is never written $x^{T} y$ .

Common errors

Common misconception

The dot product of two vectors is another vector, obtained by multiplying them entry by entry.

Related units

Connected

Linear Independence, Rank, and Bases (part of)
Half-Spaces and Hyperplanes (part of)
Matrices as Operators (suggested next)

Learn this topic

Vectors and Linear Combinations

Used in

Foundations for Operational Research

Sources

Introduction to Linear Algebra (2016)

Vectors and Linear Combinations

Assumptions and scope

Forms this is expressed in

Worked material

Three quantities, three readings

Undefined vector operations

Two products that are easy to confuse

Common errors

Related units

Connected

Learn this topic

Used in

Sources

Settings

Focus timer

Today

Last seven days

Where the time went

Recent sessions

Settings

Notifications

Your history