Blocked and Paired Randomized Experiments

Grouping similar units before assignment, then randomizing within each group, reduces variance by removing baseline variation the comparison would otherwise carry. What it costs is a commitment: the analysis must follow the design. A blocked estimator weights stratum effects by stratum size, a paired estimator works on within-pair differences, and treating either as a free split across all units discards the precision the design was built to gain.

Definition

In a blocked (stratified) design, units are partitioned into $K$ strata before assignment and randomized independently within each. With $N_{k}$ units and estimated effect ${\hat{τ}}_{k} = {\bar{Y}}_{1 k} - {\bar{Y}}_{0 k}$ in stratum $k$ , and $N = \sum_{k} N_{k}$ , the sample-average treatment effect is estimated by $\hat{τ} = \sum_{k = 1}^{K} \frac{N_{k}}{N} {\hat{τ}}_{k}$ . A matched-pair design is blocking with two units per block and one treated in each. With pair differences $D_{j} = Y_{j, T} - Y_{j, C}$ over $J$ pairs, $\hat{τ} = \frac{1}{J} \sum_{j} D_{j}$ , $s_{D}^{2} = \frac{1}{J - 1} \sum_{j} (D_{j} - \bar{D})^{2}$ , and $S E (\hat{τ}) = s_{D} / \sqrt{J}$ .

Formal statement

Blocked: $\hat{τ} = \sum_{k = 1}^{K} \frac{N_{k}}{N} {\hat{τ}}_{k}$ with ${\hat{τ}}_{k} = {\bar{Y}}_{1 k} - {\bar{Y}}_{0 k}$ . Paired: $D_{j} = Y_{j, T} - Y_{j, C}$ , $\hat{τ} = \bar{D}$ , $S E (\hat{τ}) = s_{D} / \sqrt{J}$ .

Assumptions and scope

The weights follow the estimand. Weighting strata by $N_{k} / N$ targets the sample-average effect; equal weights target an average of stratum effects, which is a different quantity whenever strata differ in size, and answers a different question.
Blocking must happen before assignment. Grouping units after treatment, or on a variable measured after treatment, is not blocking and can introduce the bias the design was meant to avoid.
Blocking on a variable unrelated to the outcome need not improve precision. It does no harm to unbiasedness provided the within-block randomization is valid and the analysis respects it.
A paired design permits $2^{J}$ allocations, one choice per pair, not $(\binom{2 J}{J})$ . Randomization tests and standard errors must enumerate the former; using the latter describes an experiment that was not run.
A paired $t$ -test can be applied to before-and-after measurements, naturally occurring pairs, or randomized matched pairs. The arithmetic is the same and the causal interpretation is not: only the last has treatment assigned by a known mechanism within pairs.

Worked material

Example

Weights that answer different questions

A trial blocks by site. Site A holds 40% of the sample with an estimated effect of 8; site B holds 60% with an estimated effect of 2.

Size-weighted.

\hat{τ} = 0.4 (8) + 0.6 (2) = 4.4 .

This estimates the average effect over the units in the trial, which is usually what is wanted.

Equally weighted.

\frac{8 + 2}{2} = 5.

This estimates the average of the two site effects, giving a small site and a large one the same influence.

Neither is wrong. They answer different questions, and 4.4 against 5 is not a rounding difference. It is a choice of estimand. Reporting one while describing the other is the error.

When they coincide. Equal strata sizes, or equal stratum effects. Neither is common enough to assume.

Non-example

Groupings that are not blocking

Grouping after assignment. Splitting the analysis by a variable recorded after treatment began, adherence, completion, side effects, is not blocking. Those variables can be affected by treatment, so the groups are defined partly by what the treatment did, and the comparison within them is no longer protected by randomization.

Matching in an observational study. Pairing treated and untreated units on covariates after the fact resembles a paired design and is not one. In a matched-pair experiment the pairs are formed first and treatment is randomized within them; in observational matching, nobody randomized anything, and comparability rests on an assumption rather than on a mechanism.

Post-hoc subgroup analysis. Reporting the effect among older participants, having chosen 'older' after seeing the results, is not a blocked analysis. The subgroup was not a design feature, and the multiplicity it introduces is unaccounted for.

Re-randomizing until balanced. Drawing repeatedly and keeping the allocation that looks most balanced restricts the mechanism, but not in a way any standard analysis accounts for. If balance on a variable matters, block on it in advance; that is the version whose analysis is known.

A before-and-after comparison. Measuring each unit before and after treatment produces paired numbers and a paired $t$ -test computes fine. But the second measurement is not a control outcome. It is the same unit later, with everything else that changed in the meantime. The algebra transfers; the causal interpretation does not.

Contrast

What the design permits, and what the analysis assumes

	Complete randomization	Blocked	Paired
Allocations permitted	$(\binom{N}{N_{1}})$	$\prod_{k} (\binom{N_{k}}{N_{1 k}})$	$2^{J}$
Estimator	${\bar{Y}}_{1} - {\bar{Y}}_{0}$	$\sum_{k} \frac{N_{k}}{N} {\hat{τ}}_{k}$	$\bar{D}$
Standard error	$\sqrt{s_{1}^{2} / N_{1} + s_{0}^{2} / N_{0}}$	Combined across strata	$s_{D} / \sqrt{J}$
Unit of analysis	Unit	Unit, within stratum	Pair
Balance on the blocking variable	By chance	By construction	By construction

The failure this table exists to prevent. Every row of the paired column differs from the completely randomized column. An analysis that uses the first column's estimator, standard error and reference distribution on data from the third is not approximately right. It describes a different experiment.

Which direction the error runs. With well-chosen blocks, ignoring them inflates the standard error, so the result looks weaker than the design earned. With badly chosen blocks the loss is small. Neither case makes the unpaired analysis correct; it makes the cost variable.

Balance, twice. Complete randomization leaves balance to the draw and gives valid inference anyway. Blocking removes the question for the variable blocked on. The temptation in between, re-randomizing after seeing imbalance, takes the commitment of blocking without its known analysis.

Common errors

Common misconception

Once a paired or blocked experiment has been run, the data are just treated and control observations, so a standard two-sample comparison across all units gives the same answer.

Blocked and Paired Randomized Experiments

Assumptions and scope

Worked material

Weights that answer different questions

Groupings that are not blocking

What the design permits, and what the analysis assumes

Common errors

Related units

Requires

Learn this topic

Used in

Sources

Settings

Focus timer

Today

Last seven days

Where the time went

Recent sessions

Settings

Notifications

Your history