Unconfoundedness and Overlap
When nobody assigned treatment, adjustment can still identify an effect, but only under two assumptions. Unconfoundedness says the measured covariates are enough to make assignment as good as random within their levels; overlap says both treatment states are actually possible at every covariate value that matters. They fail differently and, critically, they can be checked differently: overlap is visible in the data, and unconfoundedness is not.
Definition
Unconfoundedness (conditional ignorability, selection on observables) is
Formal statement
Assumptions and scope
Unconfoundedness is not a claim that treatment was marginally random. It claims that adjusting for
suffices, which is weaker in one sense and unverifiable in another. must consist of pretreatment variables. Conditioning on anything the treatment influenced can block part of the effect or open a non-causal path, so 'adjust for everything measured' is unsafe rather than cautious. Overlap can hold mathematically and fail practically. Propensity scores very near zero or one leave a handful of units carrying enormous weight, and estimates then rest on those few observations.
Trimming units to restore overlap changes the population the estimate describes. The estimand must be restated, not silently retained.
Overlap and covariate balance after adjustment are diagnosable from data. Unconfoundedness is not: it concerns missing potential outcomes and unmeasured variables, so no balance table can establish it.
Worked material
Example
Where each assumption fails
Unconfoundedness fails. A study compares patients who received a new surgical technique with those who received the standard one, adjusting for age, sex and comorbidity count. Surgeons chose the technique, and they chose partly on operative fitness. A judgement recorded nowhere in the data. Fit patients did better regardless of technique. Within every level of the measured covariates, assignment still depends on the potential outcomes, so the adjusted estimate mixes the technique's effect with the surgeons' selection.
Nothing in the data reveals this. The balance table looks fine, because the variable driving it was never measured.
Overlap fails. A study of an intensive tutoring programme finds that every student below the 20th percentile was enrolled. For those students
Both hold, plausibly. A workplace randomly audited by a regulator on a published rota, with the rota depending only on recorded sector and size. Assignment depends on
Non-example
Things that do not establish unconfoundedness
A balance table. Balance on measured covariates shows the adjustment worked on those covariates. The assumption is about whether they are sufficient, which the same data cannot address.
A large sample. Confounding is bias, not noise. A million observations give a precise estimate of a confounded quantity.
Adjusting for everything available. Including post-treatment variables can block part of the effect or open a non-causal path by conditioning on a collider. The rule is pretreatment common causes, not maximum coverage.
A good propensity model. High classification accuracy means treatment is predictable from covariates, which strains overlap. The score's purpose is balance, not prediction.
A statistical test for confounding. No test of the observed data can detect an unmeasured confounder, because the data carry no trace of a variable nobody recorded.
Similar point estimates from several methods. Matching, weighting and regression that agree have agreed about the same covariate set under the same assumption. They can be jointly wrong in the same direction, and usually would be.
Contrast
Which assumption the data can check
| Overlap | Unconfoundedness | |
|---|---|---|
| Statement | ||
| About | Measured covariates and assignment | Assignment and unobserved outcomes |
| Checkable from data | Yes | No |
| How | Propensity distributions, weights, effective sample size | Not available |
| Failure looks like | Extreme weights, extrapolation, estimates driven by few units | Nothing at all |
| Remedy | Trim, restrict the estimand, redesign | Measure more, or a sensitivity analysis |
Why the asymmetry matters. Analysts check what is checkable and then report as though both assumptions had been verified. The checkable one is the less consequential of the two.
Against randomization. A randomized experiment gets unconfoundedness from the mechanism and overlap by construction. Every unit had a genuine chance of either arm. An observational study must assume the first and demonstrate the second. That is the cost of not having assigned treatment, and it is why design occupies the first half of this subject.
Failure is not visible in the output. Poor overlap is detectable in diagnostics. Unmeasured confounding produces a clean table, a tight interval and a wrong answer.
Common errors
Common misconception
If the treated and control groups are balanced on the measured covariates after adjustment, then confounding has been removed and the comparison estimates a causal effect.
Related units
Requires
Connected
- Propensity Scores (suggested next)