Choosing a Regression Form

Each case names a response and a predictor. The question is which forms remain available once the response is taken seriously.

Number of emergency admissions per day, against the day's mean temperature. A count, bounded below at zero, typically small. A straight line predicts negative admissions at some temperatures, and the spread around the fit is visibly wider on busy days than on quiet ones. Poisson regression, with $e^{\beta }$ read as the multiplicative change in the daily rate per degree.

Bacterial population, against hours since inoculation. Positive, and multiplied rather than incremented while growth is unconstrained. Successive ratios roughly constant is the check. Exponential regression on $\log y$, with the coefficient reported as a factor per hour or a doubling time. The form fails once the culture saturates, which is a statement about the biology rather than about the fit.

Braking distance, against speed. Continuous, positive, and curved for a reason that is known in advance: kinetic energy rises with the square of speed. Polynomial regression with an $x^2$ term, chosen from the mechanism rather than from the scatter plot. That distinction matters. The same curve chosen by trying degrees until one fits well is the selection problem of the previous unit.

Proportion of a cohort still subscribed, against months elapsed. Bounded in $[0,1]$. Every form in this unit will eventually predict outside that interval, the linear one soonest. This case needs logistic regression, which is not covered here; recognising the bound is what shows the available forms are inadequate.

Household electricity use, against outdoor temperature. Continuous and non-monotone: consumption falls as temperature rises toward a comfortable range, then climbs again as cooling starts. No single-turning-point form fits. A quadratic can represent one turning point and will place it somewhere between the two real ones; the available options are a higher-degree polynomial chosen on held-out error, or a nonparametric fit that lets the data set the shape.

---

In each case the response's nature narrowed the field before any curve was drawn, and in two of them, the proportion and the non-monotone case, it ruled out every form this unit offers. Knowing that a form is unavailable is as useful as knowing which to pick, and it is only visible by asking what the response can be.

$\log y=a+bx$ against $y={\beta }_0+{\beta }_{1}x$.

The third row is the one most often overlooked. Least squares on $\log y$ treats a factor-of-two error on a value of 3 as equal to a factor-of-two error on a value of 300. Fitting the same exponential curve directly on the original scale weights the large values far more heavily and gives different coefficients. Both are defensible; they are not the same fit.

Poisson regression against least squares on $\log (y+1)$.

Both handle a count response and produce a multiplicative reading. The second requires an arbitrary offset to cope with zero counts, and the choice of offset changes the estimate, adding 1 rather than 0.5 gives a different slope with no principle deciding between them. Poisson regression models the log of the mean, which is defined when an observed count is zero, so no offset is required.

A degree-2 fit against a degree-6 fit on the same eight points.

The more flexible fit is better by the criterion computed on the training data and disagrees by roughly a quarter two units beyond it, in the opposite direction. Training fit and extrapolation behaviour are not merely different questions; here they rank the two models oppositely.

Choosing a form from the mechanism against choosing it from the plot.

Braking distance takes an $x^2$ term because kinetic energy goes as the square of speed. A reason available before the data. The same term chosen by fitting degrees 1 through 6 and keeping the best is a selection, and the reported error from that search is optimistic by the amount the previous unit describes. The fitted curve can be identical; what differs is what the error figure means.

In each, two procedures produce comparable-looking output and commit to different claims: about scale, about what is defined at zero, about behaviour outside the data, or about what an error figure estimates. None of the differences is visible in the fitted line alone.

That adding $x^2$ or $x^3$ to a regression makes it a nonlinear model requiring a different fitting method. Linear regression means linear in the coefficients, not linear in the predictors. A polynomial model enters each power as an additional column of the design matrix and is fitted by the same least-squares solution, with the same closed form and the same standard errors. The curve in the picture comes from the predictor values, not from any change to the estimation. Genuinely nonlinear models are those where a coefficient appears inside a function, such as $y={\beta }_0{e}^{{\beta }_{1}x}$ fitted directly, which has no closed-form least-squares solution.

That a coefficient from a model fitted on the log scale reports how much the response changes per unit of the predictor. A slope of $0.7$ in $\log y=a+bx$ does not mean $y$ rises by $0.7$; it means $y$ is multiplied by $e^{0.7}\approx 2.01$ for each unit of $x$, roughly a doubling. The same applies to a Poisson regression coefficient, where $e^{\beta }$ is a rate ratio. Reading the log-scale value additively understates large effects and misstates the units, and the error is invisible because the reported number is plausible on its face.