Practice: Sampling Distributions and Standard Error

(a) $SE(\overline{X})=12/\sqrt{40}=12/6.32\approx 1.90$ units.

(b) Setting $12/\sqrt{n}=1.0$ gives $\sqrt{n}=12$ and $n=144$ measurements.

(c) Precision improves with the square root of the sample size, not with the sample size itself. Going from 1.90 to 1.00 is a factor of about 1.9, so the sample must grow by roughly ${1.9}^2\approx 3.6$ times, from 40 to 144. The general rule: each halving of the standard error costs four times the data already collected. This is why precision becomes expensive quickly, and why a modest tightening of a target can have a large consequence for cost.

A caveat. This calculation assumes the measurements are independent. If they are clustered, several per batch, per site or per operator, the required number is larger, and the design must account for the clustering rather than only the count.

(d) Rescaling. Multiplying every measurement by 10 gives $s^{\prime }=10\times 12=120$, since $\mathrm{Var}(aX)=a^2\mathrm{Var}(X)$ scales the variance by 100 and the standard deviation by $|a|=10$. So $SE(\overline{X})=120/\sqrt{40}\approx 18.97$, exactly ten times the original. Adding a constant. Adding 5 to every measurement shifts the mean by 5 and leaves $s$ and $SE$ untouched: $\mathrm{Var}(X+b)=\mathrm{Var}(X)$. Location moves, spread does not.

The error. The central limit theorem is being applied to the observations. It concerns the sampling distribution of the mean, and says nothing about the distribution of household incomes, which remain heavily right-skewed however many households are surveyed. Sample size gives a clearer picture of that skew; it does not remove it.

Why the reported range is wrong. Using mean ± 1.96 standard deviations to describe where 95% of households fall assumes the incomes themselves are normal. With strong right skew, the true distribution has a long upper tail and a compressed lower one, so a symmetric interval will misstate both ends, and the lower endpoint may even fall below plausible values. Empirical quantiles of the observed incomes would answer this question directly and require no distributional assumption at all.

What the theorem does provide. That the sampling distribution of the mean income is approximately normal, which is ample with $n=\mathrm{5,000}$ even under strong skew. So a confidence interval for mean household income, using $SE=s/\sqrt{5000}$, is well justified. That interval would be roughly £45,000 ± a few hundred pounds. A statement about the average, and a far narrower range than the one reported, because it answers a different question.

The distinction to keep. Questions about individual households are about the observations. Questions about the average are about the mean. Only the second is what the theorem addresses.

A related slip to avoid. Reporting figures after a change of units invites the same confusion in reverse: a shift of origin leaves every spread statistic alone, while a change of scale multiplies the standard deviation by $|a|$ and the variance by $a^2$. Neither operation makes a skewed distribution normal, which is the error above.

The problem. The 45,000 page views are not independent observations. They are nested within 3,200 users, and views by the same user are far more alike than views by different users, same device, same habits, same intent. The standard error appears to have been computed as $s/\sqrt{\mathrm{45,000}}$, which is only valid under independence.

Direction and size of the error. Positive within-user correlation means the true standard error is larger than reported, so the error runs toward overconfidence. The effective sample size lies between 3,200 and 45,000, and approaches 3,200 as within-user correlation rises. Since $\sqrt{\mathrm{45,000}}/\sqrt{\mathrm{3,200}}\approx 3.75$, the standard error could plausibly be several times the reported 0.02, which undermines the claim about detecting a few seconds.

What I would change. Work at the user level: compute one summary per user, mean session duration for that user, and analyse those 3,200 values, which are plausibly independent if users were sampled independently. Alternatively use a method that models the clustering directly and reports an effective sample size. Either way the unit of analysis should match the unit of independence.

What else to check. Whether users themselves are independent. If many share households, organisations or referral sources, even 3,200 may overstate the information available.

Converting to seconds. Multiplying every session duration by 60 multiplies the mean, the standard deviation and the standard error by 60 alike: $\mathrm{Var}(aX)=a^2\mathrm{Var}(X)$, so the standard deviation scales by $|a|$ and so does $SE=s/\sqrt{n}$. The conversion changes no conclusion, it is a change of units, not of information, which is worth saying explicitly, because a standard error of 1.2 seconds sounds more impressive than 0.02 minutes and is the same number.