Estimators and How They Are Judged

Each case fixes a model and an estimator, then asks the three questions separately.

The sample mean for a population mean. Unbiased at every $n$, since $\mathbb{E}[\overline{X}]=\mu$ whatever the distribution. Variance ${\sigma }^2/n$, which falls with $n$, so it is also consistent. This is the case where the properties agree, and it is why the sample mean is the default, not because it is obvious, but because it satisfies every criterion at once.

The sample variance with divisor $n-1$. Unbiased for ${\sigma }^2$; the $n-1$ exists precisely to make it so, compensating for estimating $\mu$ by $\overline{X}$. Consistent as well. Its square root is not unbiased for $\sigma$: an unbiased estimator of a quantity does not give an unbiased estimator of a nonlinear function of it, and $s$ understates $\sigma$ on average.

The sample variance with divisor $n$. This is the maximum likelihood estimator under normality, and it is biased at every sample size, with $\mathbb{E}[s_n^2]=\frac{n-1}{n}{\sigma }^2$:

The bias never reaches zero at any finite $n$, and it goes to zero in the limit. Biased at every sample size, consistent nonetheless.

The first observation, as an estimator of the mean. $\mathbb{E}[X_1]=\mu$, so it is unbiased at every $n$, perfectly, not approximately. Its variance is ${\sigma }^2$ regardless of how many observations were collected, so the estimator never improves and is not consistent. It is unbiased and useless, which is the sharpest available demonstration that unbiasedness alone is not a recommendation.

---

The first satisfies everything. The second and third differ only in a divisor and land on opposite sides of unbiasedness while both being consistent. The fourth is unbiased and inconsistent. So among these four, unbiasedness and consistency appear in all four combinations that matter, which is what it means for the two properties to be logically independent rather than one implying the other.

Unbiased against minimum mean squared error, on the uniform family.

The first and third are unbiased and their mean squared errors differ by a factor of $3.3$. So unbiasedness does not determine the ranking in either direction: the best and the worst of these three are both unbiased, and the biased one sits between them.

Divisor $n-1$ against divisor $n$ in the sample variance.

One change to a denominator moves the estimator across the unbiasedness line. Both are consistent, both converge to the same thing, and the one that is biased is the maximum likelihood estimator. The choice is conventional rather than forced, and the convention exists because unbiasedness is easy to state, not because it is the better criterion here.

Method of moments against maximum likelihood.

For the Poisson family they coincide exactly, both giving $\overline{X}$, because $\mathbb{E}[X]=\lambda$ makes the moment equation and the score equation the same equation. Agreement is common; the uniform case, where they give $12.0000$ and $9.4000$, is the one that teaches the difference.

Unbiased for ${\sigma }^2$ against unbiased for $\sigma$.

$s^2$ is unbiased for the variance and $s$ is not unbiased for the standard deviation. Unbiasedness is not preserved by nonlinear transformation, so "an unbiased estimate of the variance" and "an unbiased estimate of the spread" are different claims and only the first is true.

Consistency against unbiasedness, as claims.

The first says the estimator eventually concentrates on the truth and promises nothing about the data in hand. The second says the estimator is centred correctly for the data in hand and promises nothing about improvement. A study with $n=30$ is not helped by a limiting guarantee, and a study accumulating data indefinitely is not helped by correct centring around an unchanging spread.

That an unbiased estimator is always preferable to a biased one, since being centred on the truth is the point of estimating. Unbiasedness constrains where the sampling distribution is centred and says nothing about how far it spreads, and mean squared error, which combines both as ${\mathrm{bias}}^2+\mathrm{Var}$, can favour the biased competitor. For a uniform distribution on $(0,\theta )$ with eight observations, the unbiased method-of-moments estimator $2\overline{X}$ has mean squared error $0.0417{\theta }^2$ while the biased maximum likelihood estimator, the sample maximum, has $0.0222{\theta }^2$, smaller by a factor of $1.875$ despite understating $\theta$ at every sample size. Unbiasedness is one criterion among several, and imposing it is a restriction that has a cost.

That bias, variance and consistency describe the number a study reported. They describe the rule that produced it. An estimator is a function of the sample and therefore a random variable with a distribution; an estimate is one realised value of it, and a single number has no variance and no bias. Saying that a reported figure of $2.14$ is unbiased is a category error: what may be unbiased is the procedure that would have produced a different figure from a different sample. The confusion matters because it makes the properties sound checkable against the data in hand, when every one of them is a statement about samples that were not drawn.