Sampling Distributions and Standard Error

A statistic computed from a sample would have come out differently had the sample been different. The distribution of those hypothetical values is the sampling distribution, and the standard errors, intervals and tests of classical inference are built from it. Its most-used consequence, that the mean of enough observations is approximately normal, is a statement about the mean, not about the data.

Definition

For observations $X_{1}, \dots, X_{n}$ , the sample mean is $\bar{X} = \frac{1}{n} \sum_{i} X_{i}$ and the sample variance is $s^{2} = \frac{1}{n - 1} \sum_{i} (X_{i} - \bar{X})^{2}$ , the divisor $n - 1$ making $s^{2}$ unbiased for $σ^{2}$ under independent, identically distributed sampling. Applying the expectation and variance rules to that average gives the two identities behind the standard error of the mean. Expectation passes through the sum and the $\frac{1}{n}$ unconditionally, so if each $X_{i}$ has the same mean $μ$ then $E [\bar{X}] = μ$ however the observations relate to one another. Dependence does not disturb that step; the common-mean assumption is what it needs. The variance needs the scaling rule and then variance addition:

Var (\bar{X}) = \frac{1}{n^{2}} Var (\sum_{i} X_{i}) = \frac{1}{n^{2}} \cdot n σ^{2} = \frac{σ^{2}}{n},

where the middle step collapses only because the observations are pairwise uncorrelated with common variance $σ^{2}$ . Independence is the usual way that holds and is sufficient rather than necessary: zero pairwise covariance is all this identity requires. Its square root, $S E (\bar{X}) = σ / \sqrt{n} \approx s / \sqrt{n}$ , is the standard error of the mean. The $\sqrt{n}$ is not a convention: it is what is left of the $\frac{1}{n^{2}}$ after the square root, which is why halving a standard error takes four times the data. These identities give the standard error of the sample mean and illustrate the sampling-distribution logic used throughout frequentist inference; a different estimator or sampling scheme needs its own derivation. Under regularity conditions the central limit theorem gives $\frac{\bar{X} - μ}{σ / \sqrt{n}} \overset{d}{⟶} N (0, 1)$ .

Formal statement

$\bar{X} = \frac{1}{n} \sum_{i} X_{i}$ ; $s^{2} = \frac{1}{n - 1} \sum_{i} (X_{i} - \bar{X})^{2}$ ; $Var (\bar{X}) = σ^{2} / n$ ; $S E (\bar{X}) = σ / \sqrt{n}$ ; $\frac{\bar{X} - μ}{σ / \sqrt{n}} \overset{d}{⟶} N (0, 1)$ .

Assumptions and scope

The central limit theorem concerns the sampling distribution of the mean. It makes no claim that the raw observations become normal, and no sample size makes a skewed population symmetric.
$Var (\bar{X}) = σ^{2} / n$ requires the observations to be pairwise uncorrelated with common variance. Independence implies that and is the standard sampling assumption, but it is stronger than this identity needs. Clustered, paired or serially dependent observations generally have nonzero covariances, carry less information than their count suggests, and using $n$ regardless understates the standard error.
The divisor $n - 1$ makes $s^{2}$ unbiased under independent, identically distributed sampling. It is a correction for estimating $μ$ by $\bar{X}$ , not a small-sample fudge.
How large $n$ must be for the normal approximation to serve depends on the population's shape. Heavy skew or strong discreteness needs more observations than mild departures from symmetry.
$S E (\bar{X}) = σ / \sqrt{n}$ uses the population standard deviation, and replacing $σ$ by $s$ introduces further uncertainty. Under a normal model for the observations, $(\bar{X} - μ) / (s / \sqrt{n})$ follows $t_{n - 1}$ exactly. Outside that model, estimating a scale does not by itself produce a $t$ statistic: the usual large-sample justification is that the statistic is asymptotically normal while $t$ critical values approach the normal ones.

Worked material

Example

Spread of the data against spread of the estimate

A hospital records length of stay for 900 discharges. The distribution is heavily right-skewed: most stays are 2–4 days, a few run past 60. The sample mean is 4.8 days with $s = 6.0$ days.

Spread of the observations. $s = 6.0$ days. Individual stays vary enormously, and they always will. This is a real feature of the population, not noise to be averaged away.

Spread of the estimate.

S E (\bar{X}) = \frac{6.0}{\sqrt{900}} = \frac{6.0}{30} = 0.20 days .

The mean of 900 stays would move by only a fifth of a day or so across repetitions. These two numbers describe different things, and only the second is about the reliability of the estimate.

What the central limit theorem licenses here. That $\bar{X}$ is approximately normal, so the usual interval around 4.8 is reasonable. With $n = 900$ and skew this strong, that approximation is doing real work and is plausible at this sample size.

What it does not license. Any claim that lengths of stay are normal. Plot them and the skew is unmistakable. If the question were "what proportion of patients stay longer than 14 days?", the normal approximation would answer it badly, that question is about the observations, and no sample size makes their distribution symmetric.

The distinction in one line. The CLT is about $\bar{X}$ . Questions about individual units are not questions the CLT addresses.

Non-example

Situations the usual standard error does not describe

Pupils within classrooms. 600 pupils across 20 classrooms are not 600 independent observations. Pupils share a teacher, a room and a peer group. Dividing by $\sqrt{600}$ can badly overstate precision.

Repeated measures on one person. Twelve blood-pressure readings from each of 30 patients is not $n = 360$ . The readings within a patient are far more alike than readings across patients.

A time series. Daily sales for two years are serially dependent: today resembles yesterday. Treating 730 days as 730 independent draws understates the standard error.

A convenience sample. Volunteers who responded to an advertisement have a sampling distribution the formula does not describe, because they were not drawn by a known mechanism from the target population. The arithmetic still runs, which is exactly the danger.

Using the CLT for an individual prediction. "Roughly 95% of patients stay between 4.4 and 5.2 days" misapplies an interval for the mean to individual units. That interval describes where the average sits, not where a patient falls.

A sample standard deviation reported as a standard error. These differ by a factor of $\sqrt{n}$ . At $n = 900$ that is a factor of 30, so the mistake is not a small one.

Contrast

Two distributions people conflate

	Distribution of the observations	Sampling distribution of $\bar{X}$
What varies	Individual units	The estimate, across repetitions
Spread measured by	$s$	$S E = s / \sqrt{n}$
Shape as $n$ grows	Unchanged — a skewed population stays skewed	Approaches normal, by the CLT
Observable	Yes, plot the data	No, only one draw is ever seen
Answers	How do units differ?	How reliable is the estimate?

Where the confusion does damage. The claim " $n$ is large, so by the central limit theorem the data are approximately normal" moves a result about column two into column one. It is used to justify procedures that depend on the observations being normal, prediction intervals for individuals, reference ranges, tolerance limits, and no sample size supports them.

The clean test. Ask what the statement is about. If it is about individual units, the CLT is silent. If it is about an average or a total, the CLT applies.

Why $n$ still helps the mean. Not by reshaping the population, but by averaging: deviations partly cancel, the variance of the average shrinks as $σ^{2} / n$ , and the shape of that average's distribution tends to normal whatever the population looked like. Both effects are about $\bar{X}$ .

Common errors

Common misconception

The central limit theorem says that with a large enough sample the data themselves become normally distributed, so a big sample justifies treating skewed observations as normal.

Sampling Distributions and Standard Error

Assumptions and scope

Worked material

Spread of the data against spread of the estimate

Situations the usual standard error does not describe

Two distributions people conflate

Common errors

Related units

Requires

Connected

Learn this topic

Used in

Sources

Settings

Focus timer

Today

Last seven days

Where the time went

Recent sessions

Settings

Notifications

Your history