Practice: Binary Outcome Models for Experimental Research

The proportions. ${\hat{p}}_1=312/400=0.78$; ${\hat{p}}_0=268/400=0.67$.

Risk difference. $0.78-0.67=0.11$, or 11 percentage points. Across 400 patients that is about 44 additional attendances.

Risk ratio. $0.78/0.67\approx 1.16$, a 16% relative increase.

Odds ratio. Odds treated $=0.78/0.22\approx 3.545$; odds control $=0.67/0.33\approx 2.030$; $OR\approx 1.75$.

Which to report. The risk difference. A capacity plan needs to know how many more patients will attend, and 11 percentage points converts directly into rooms, staff and appointment slots. The odds ratio of 1.75 cannot be turned into that number without the baseline rate, and would be actively misleading if read as '75% more likely'. The actual relative increase is 16%.

Why no model is needed. Treatment was randomized, so the difference in sample proportions already estimates the causal risk difference. Fitting a logistic model would answer the same question on a less useful scale and add nothing to the identification.

What to add. A confidence interval for the difference, since 11 points with an interval of 5 to 17 supports a firmer plan than one spanning 1 to 21.

The linear probability model here. Its treatment coefficient would report the risk difference directly, the same $78\mathrm{\%}-67\mathrm{\%}=11$ percentage points, which is the scale a decision needs, with no conversion required. Two qualifications: its errors are structurally heteroskedastic because the variance of a Bernoulli outcome depends on its mean, so robust standard errors are required rather than optional; and fitted values can fall outside $[0,1]$. With both arms near the middle of the range that second point does not touch this estimand, so it is a reason for care, not for rejecting the model.

The error. The second sentence converts an odds ratio into a risk ratio, which is not valid at a 22% baseline. 'Doubles the odds' is accurate; 'twice as likely' is a claim about probabilities and is not what was estimated.

The correct conversion. Control odds $=0.22/0.78\approx 0.282$. Multiplying by 2.1 gives $0.592$, so the treated probability is $0.592/1.592\approx 37.2\mathrm{\%}$.

• Risk ratio: $0.372/0.22\approx 1.69$. A 69% relative increase, not 100%.
• Risk difference: $37.2\mathrm{\%}-22\mathrm{\%}=15.2$ percentage points.

Why it matters. The result is genuinely good, fifteen additional quitters per hundred smokers is a substantial public-health effect. The error overstates the relative effect and, more importantly, buries the absolute one, which is what a reader needs to judge whether the app is worth using or funding.

What should have been reported. The quit rates in both arms, 22% and 37%, their difference of about 15 percentage points with a confidence interval, and the odds ratio only alongside the baseline rate so a reader can recover the absolute effect.

A simpler route. The trial was randomized, so the difference in sample proportions estimates the causal risk difference directly. The headline never needed a model at all.

What would have avoided the error. Reporting on the probability scale in the first place. A linear probability model would have returned the risk difference directly, with robust standard errors for the heteroskedasticity a binary outcome always carries. Its fitted values can stray outside $[0,1]$, which matters when predicting at the extremes and not when reporting an average difference between two arms.

The scale error. An odds ratio of 1.35 is not a 35% increase in the probability of claiming. Baseline odds $=0.08/0.92\approx 0.087$; multiplied by 1.35 gives $0.1174$, so the zone-3 probability is $0.1174/1.1174\approx 10.5\mathrm{\%}$.

• Risk ratio: $0.105/0.08\approx 1.31$, a 31% relative increase.
• Risk difference: about 2.5 percentage points.

At an 8% baseline the odds ratio and risk ratio are reasonably close, so the relative error here is modest, but the reasoning is still wrong, and the same habit applied to a common outcome would misstate the figure badly.

The larger error is the pricing logic. Even the correct 31% relative increase does not justify a 35% premium rise, and neither would a 31% rise. Premiums are set against expected claim cost, which depends on the absolute claim probability and the severity per claim. The absolute risk rose by 2.5 percentage points; whether that warrants a 35% premium increase depends on the loading, the expected claim size and the existing margin, none of which the odds ratio speaks to.

What I would report. Claim probabilities in each zone, 8% against 10.5%, with intervals; the absolute difference of 2.5 points; and an expected-cost calculation combining that with average claim severity. If an odds ratio appears at all, the baseline rate belongs beside it.

One further check. Whether flood zone is a cause of claiming or a marker for other correlated property characteristics, for pricing, prediction may suffice, but the note should not be read as establishing a causal effect of the zone designation.

The alternative the insurer should consider. Fitting the claim indicator linearly would give the change in claim probability directly, which is what a premium calculation consumes. The cost is that such a model can predict probabilities outside $[0,1]$ for extreme properties and that its errors are heteroskedastic by construction, so robust standard errors are mandatory. Whether that cost matters depends on whether the extremes or the average is being priced.