Practice: Orthogonality and Projection

(a) Gram–Schmidt. Take $u_1=(1,0,1)$, so $⟨u_1,u_1⟩=2$. For $v_2=(0,1,1)$: $⟨v_2,u_1⟩=0+0+1=1$, so

Check. $⟨u_1,u_2⟩=-\frac{1}{2}+0+\frac{1}{2}=0$ ✓. Also $⟨u_2,u_2⟩=\frac{1}{4}+1+\frac{1}{4}=\frac{3}{2}$. (b) The projection. With $b=(2,1,0)$:

So ${\mathrm{proj}}_W(b)=1\cdot (1,0,1)+0\cdot u_2=(1,0,1)$. Residual. $e=(2,1,0)-(1,0,1)=(1,1,-1)$. Orthogonality check. $⟨e,u_1⟩=1+0-1=0$ ✓ and $⟨e,u_2⟩=-\frac{1}{2}+1-\frac{1}{2}=0$ ✓. Since $e$ is orthogonal to a basis of $W$, it is orthogonal to all of $W$. Pythagoras. $\Vert b{\Vert }^2=4+1+0=5$; $\Vert {\mathrm{proj}}_W(b){\Vert }^2=1+0+1=2$ and $\Vert e{\Vert }^2=1+1+1=3$, and $2+3=5$ ✓. That $c_2=0$ is informative rather than degenerate: $b$ happens to have no component along $u_2$, so the nearest point of the plane lies on the $u_1$ line. (c) The least-squares fit. With $n=3$, $\sum x_i=6$, $\sum x_i^2=14$, $\sum y_i=5$, $\sum x_i{y}_i=11$, the normal equations are

The determinant is $42-36=6$, so $a={\displaystyle \frac{5(14)-6(11)}{6}}={\displaystyle \frac{4}{6}}={\displaystyle \frac{2}{3}}$ and $b={\displaystyle \frac{3(11)-6(5)}{6}}={\displaystyle \frac{3}{6}}={\displaystyle \frac{1}{2}}$. The fitted line is $y=\frac{2}{3}+\frac{1}{2}x$. Residuals. Fitted values are $\frac{7}{6}$, $\frac{5}{3}$, $\frac{13}{6}$, giving

The two conditions. Each row of $A^{\mathsf{T}}e=0$ is the residual against one column of $A$. Row 1, the column of ones: $\sum _i{e}_i=-\frac{1}{6}+\frac{1}{3}-\frac{1}{6}=0$ ✓. Row 2, the column of $x_i$: $\sum _i{x}_i{e}_i=-\frac{1}{6}+\frac{2}{3}-\frac{1}{2}=0$ ✓. The first holds because the model has an intercept; it is a consequence of the geometry, not of the data. (d) A different inner product.

so $\{1,x\}$ is not orthogonal under this inner product, even though the two functions look unrelated. One Gram–Schmidt step, with $u_1=1$ and $⟨u_1,u_1⟩={\int }_0^{1}1dx=1$:

Check. $⟨1,x-\frac{1}{2}⟩={\int }_0^1(x-\frac{1}{2})dx=\frac{1}{2}-\frac{1}{2}=0$ ✓. So $\{1,x-\frac{1}{2}\}$ is an orthogonal basis of $P_1$ under this inner product. (These are the first two Legendre polynomials on $[0,1]$, up to scaling.) (e) What the construction needed. In both parts it needed exactly three things: a way to pair two elements into a number, that pairing being symmetric and linear, and for $v\ne 0$. Everything used, subtracting a projection, checking the result pairs to zero, is written in those terms alone. What it did not need: coordinates, a finite dimension, or the elements being lists of numbers. In (a) the pairing was a sum over three components; in (d) it was an integral over an interval. The formulas were identical because the axioms are identical, which is the point of defining an inner product abstractly rather than defining the dot product and stopping. What is specific to the choice: which vectors count as orthogonal. Under the standard dot product $(1,1)$ and $(1,-1)$ are orthogonal; under $⟨u,v⟩=u_1{v}_1+2u_2{v}_2$ they are not. Orthogonality is a relation between two vectors and an inner product, and naming only the first two is incomplete.