Random Variables, Expectation and Variance

Outcomes and events. An experiment has a sample space $\mathrm{\Omega }$ of possible outcomes; an event is a subset of $\mathrm{\Omega }$. A probability assigns each event a number in $[0,1]$ with $P(\mathrm{\Omega })=1$ and $P(A\cup B)=P(A)+P(B)$ for disjoint $A,B$.

Random variables. A random variable $X$ is a function from $\mathrm{\Omega }$ to the reals: it attaches a number to each outcome. It is neither random nor a variable in the algebraic sense: the randomness is in which outcome occurs, and $X$ merely reports a number once one does. A discrete $X$ takes countably many values with probability mass function $p(x)=P(X=x)$.

Expectation. For discrete $X$,

the average of the values weighted by their probabilities. It is a property of the distribution, not of any sample, and need not be an attainable value: a fair die has $E[X]=7/2$.

Variance. The spread about the mean,

with standard deviation $\sigma =\sqrt{\mathrm{Var}(X)}$, in the units of $X$ itself.

Linear transformations. For constants $a,b$:

Expectation passes through a linear transformation unchanged. Variance ignores the shift $b$, because shifting every value moves the mean equally and alters no deviation, and picks up $a^2$ rather than $a$, because scaling multiplies every deviation by $a$ and hence every squared deviation by $a^2$.

For a non-linear $g$ there is no corresponding shortcut: $E[g(X)]=\sum _{x}g(x)p(x)$ must be computed term by term, and $E[g(X)]\ne g(E[X])$ in general. The gap in the quadratic case is exactly the variance, since $E[X^2]-E[X{]}^2=\mathrm{Var}(X)$.

Whether $\mathrm{Var}(X+Y)$ decomposes into $\mathrm{Var}(X)+\mathrm{Var}(Y)$ is a separate question, governed by a condition on the joint behaviour of $X$ and $Y$ rather than by either distribution alone.