Practice: Covariance, Independence and the Variance of a Sum

Classification

A fair die has $Var (X) = \frac{35}{12}$ .

Consider two quantities:

$S = X_{1} + X_{2}$ , the total from two independent rolls
$T = X + X = 2 X$ , a single roll counted twice

Which is true?

2 hints available, least help first.

Hint 1: Retrieval cue

Does the variance addition rule apply to any pair of variables, or does it carry a condition?

Hint 2: Concept cue

$T = 2 X$ is a scaling, not a sum of two separate things. Which rule governs $Var (a X)$ ?

Error diagnosis · Classification

An analyst reports: "The correlation between dose and response is $0.02$ , essentially zero, so dose and response are independent and the dose can be set on cost grounds alone."

The response is known to be harmful at low and high doses and beneficial in the middle.

Which statement identifies the error?

2 hints available, least help first.

Hint 1: Retrieval cue

What kind of association does a correlation coefficient measure?

Hint 2: Concept cue

Sketch a U shape. Do its two halves contribute deviations of the same sign?

Transfer · Evaluation

A researcher wants the average score of pupils in a city. Rather than sampling 400 pupils individually, they pick 20 classrooms at random and take all 20 pupils in each, 400 pupils in total.

They then report the standard error as $s / \sqrt{400}$ , reasoning that the sample size is 400.

Pupils in the same classroom share a teacher, a timetable and a catchment, so their scores tend to resemble one another.

What is wrong with the reported standard error?

2 hints available, least help first.

Hint 1: Retrieval cue

Which of the two combination rules, for expectation, or for variance, carries a condition?

Hint 2: Concept cue

Write $Var (\sum X_{i})$ out in full, including the covariance terms, and ask what the sign of those terms is here.

Construction · Direct application · Explanation

(a) Let $S = X_{1} + X_{2}$ be the total of two independent die rolls, and let $T = X + X$ be a single roll counted twice. Give $E$ and $Var$ for each, state which rule you use, and say which of the two rules carries a condition.

(b) Let $X$ take values $- 1, 0, 1$ each with probability $\frac{1}{3}$ and let $Y = X^{2}$ . Show $Cov (X, Y) = 0$ but that $X$ and $Y$ are not independent.

(c) For the pair in (b), decide whether $Var (X + Y) = Var (X) + Var (Y)$ holds, and say what that tells you about which condition the addition rule actually needs.

Write your answer

Write your answer, then compare it with the worked solution.

3 hints available, least help first.

Hint 1: Retrieval cue

In (a), ask of each rule whether it carries a condition before applying it.

Hint 2: Concept cue

In (b), compute $E [X Y]$ first; then test independence with a single well-chosen pair of values.

Hint 3: Strategy cue

In (c), do not assume the answer follows from (b)'s dependence. Check which condition the rule actually names.

Compare with the worked solution

Comparing does not record a result. Judging your own written answer cannot show that you can do this without help.

(a) Two rolls against one roll doubled. A fair die has $E [X] = 3.5$ and $Var (X) = \frac{35}{12}$ .

The independent total.

E [S] = E [X_{1}] + E [X_{2}] = 7,

using $E [X + Y] = E [X] + E [Y]$ , which holds unconditionally.

Var (S) = \frac{35}{12} + \frac{35}{12} = \frac{35}{6} \approx 5.833333,

using variance addition, which does carry a condition: the variables must be uncorrelated. Independence supplies it, confirmed by $E [X_{1} X_{2}] = \frac{49}{4} = E [X_{1}] E [X_{2}]$ , so the covariance is zero. Both results verified by summing over all 36 outcomes ✓.

The doubled roll. Here the two terms are the same roll, hence perfectly dependent, and the addition rule does not apply. Since $T = 2 X$ , the scaling rule gives

Var (2 X) = 4 \cdot \frac{35}{12} = \frac{35}{3} \approx 11.666667,

twice $Var (S)$ . Meanwhile $E [T] = 7$ , exactly as for $S$ .

Which rule carries the condition. Expectation is the same in both cases and cannot see the dependence; the variances differ by a factor of two. In general $Var (X + Y) = Var (X) + Var (Y) + 2 Cov (X, Y)$ , and the familiar rule is the special case of zero covariance.

(b) Uncorrelated but dependent.

E [X] = \frac{1}{3} (- 1 + 0 + 1) = 0, E [Y] = \frac{1}{3} (1 + 0 + 1) = \frac{2}{3},

E [X Y] = E [X^{3}] = \frac{1}{3} ((- 1) + 0 + 1) = 0,

so

Cov (X, Y) = E [X Y] - E [X] E [Y] = 0 - 0 \cdot \frac{2}{3} = 0 ✓

All values exact in rational arithmetic. The pair is uncorrelated.

Yet dependent. Independence requires the joint to factor. But

P (X = 0, Y = 0) = \frac{1}{3} while P (X = 0) P (Y = 0) = \frac{1}{3} \cdot \frac{1}{3} = \frac{1}{9} .

Not equal ✓. More starkly, $Y = X^{2}$ is a deterministic function of $X$ : knowing $X$ fixes $Y$ completely. $P (Y = 0) = \frac{1}{3}$ unconditionally, but $P (Y = 0 ∣ X = 0) = 1$ .

What covariance detects. Linear association only, meaning whether $Y$ tends to rise as $X$ rises. Here $Y$ falls on $[- 1, 0]$ and rises on $[0, 1]$ , symmetrically, so the two halves cancel exactly and the net linear tendency is zero. Covariance reports no net linear tendency, which a symmetric non-linear relationship produces just as readily as genuine unrelatedness.

(c) Does addition still hold? Yes. The addition rule requires only $Cov (X, Y) = 0$ , which (b) established, so $Var (X + Y) = Var (X) + Var (Y)$ holds for this dependent pair. Checking directly: $Var (X) = E [X^{2}] = \frac{2}{3}$ , and $Var (Y) = E [Y^{2}] - E [Y]^{2} = \frac{2}{3} - \frac{4}{9} = \frac{2}{9}$ . Then $X + Y$ takes values $0, 0, 2$ with probabilities $\frac{1}{3}, \frac{1}{3}, \frac{1}{3}$ , giving $E [X + Y] = \frac{2}{3}$ and $E [(X + Y)^{2}] = \frac{4}{3}$ , so $Var (X + Y) = \frac{4}{3} - \frac{4}{9} = \frac{8}{9} = \frac{2}{3} + \frac{2}{9}$ ✓.

The logical structure. Independence $\Rightarrow$ zero covariance $\Rightarrow$ variance addition, with neither implication reversible. The failure of independence is not always fatal, and which property a given argument requires is worth checking: an argument resting on independence where uncorrelatedness would serve is claiming more than it needs. Practically, a reported correlation near zero licenses "no linear relationship", never "no relationship", and a U-shaped dose–response can be strongly determinative while showing almost no correlation.

A complete answer does each of these:

applies addition conditions
distinguishes independence

Practice: Covariance, Independence and the Variance of a Sum

Classification

Error diagnosis · Classification

Transfer · Evaluation

Construction · Direct application · Explanation

Session complete

Settings

Focus timer

Today

Last seven days

Where the time went

Recent sessions

Settings

Notifications

Your history