Practice: Estimators and How They Are Judged

1. Two derivations. Method of moments. $\mathbb{E}[X]=\theta$, so the moment equation is $\overline{X}=\theta$ and ${\hat{\theta }}_{\mathrm{MoM}}=\overline{X}$. Maximum likelihood. The likelihood is $L(\theta )=\prod _i\frac{1}{\theta }{e}^{-x_i/\theta }={\theta }^{-n}{e}^{-\sum _i{x}_i/\theta }$, so

Setting ${\ell }^{\prime }(\theta )=0$ gives $\theta =\frac{1}{n}\sum _i{x}_i=\overline{X}$. The second derivative at that point is , so it is a maximum. The support does not depend on $\theta$, so there is no boundary case here. Both methods give $\overline{X}$. The lifetimes total $760$ over $n=6$, so

2. Bias and variance. $\mathbb{E}[\overline{X}]=\theta$, so the estimator is unbiased and its bias term is zero. $\mathrm{Var}(\overline{X})=\mathrm{Var}(X)/n={\theta }^2/6$. Therefore $\mathrm{MSE}(\hat{\theta })=0+{\theta }^2/6=0.1667{\theta }^2$. It is also consistent: the bias is zero at every $n$ and the variance ${\theta }^2/n$ goes to zero. 3. The transformed quantity. No. Unbiasedness is not preserved by nonlinear transformation. $\mathbb{E}[1/\overline{X}]\ne 1/\mathbb{E}[\overline{X}]$ in general, and since $1/x$ is convex on , Jensen's inequality gives , so $\hat{\lambda }$ overstates the failure rate on average. Here the size of the overstatement can be given exactly. For exponential lifetimes $\overline{X}$ has a gamma distribution with shape $n$ and scale $\theta /n$, whence

At $n=6$ that is $1.2000/\theta$: the estimator overstates $\lambda$ by 20% on average. The factor $n/(n-1)$ falls to $1.0526$ at $n=20$, $1.0101$ at $n=100$ and $1.0010$ at $n=1000$, so $\hat{\lambda }$ is consistent for $\lambda$ while not being unbiased at any finite $n$. It also shows the repair: $\frac{n-1}{n}\cdot \frac{1}{\overline{X}}$ is unbiased for $\lambda$. An unbiased estimate of a transformed parameter has to be derived for that parameter rather than inherited from an unbiased estimate of the original. 4. The competitor. ${\hat{\theta }}_1=X_1$ has $\mathbb{E}[X_1]=\theta$, so it is unbiased, exactly as $\overline{X}$ is. Its variance is $\mathrm{Var}(X)={\theta }^2$ regardless of $n$, so $\mathrm{MSE}({\hat{\theta }}_1)={\theta }^2$, six times that of $\overline{X}$ at this sample size. Because the variance does not fall as data accumulate, it is not consistent. So the two estimators are equally unbiased and differ entirely in the other two properties. This is the case that shows unbiasedness alone does not rank estimators: on this criterion they tie, and on every other criterion one is plainly better. 5. What the properties describe. The number $126.6667$ has no bias, no variance and no limiting behaviour. Those are properties of the rule $\overline{X}$, which describes what would happen across repeated samples that were never drawn. What can be claimed: the procedure that produced this figure is unbiased for $\theta$ and consistent, and its standard error is $\theta /\sqrt{6}$, estimated as $126.6667/\sqrt{6}\approx 51.71$ hours. What cannot: that $126.6667$ is close to $\theta$, or that it is "an unbiased value". The estimate could be far from the truth on this particular sample, and nothing in the data reveals whether it is. A defensible report states the estimate together with a measure of its uncertainty, and attributes the properties to the method rather than to the number.