S3.3 - Estimation, confidence intervals and tests

An estimator is a rule calculated from a random sample to estimate a population parameter. Its observed value is an estimate. An estimator $\hat\theta$ is unbiased when $E(\hat\theta)=\theta$; its bias is $E(\hat\theta)-\theta$.

The standard error is the standard deviation of an estimator's sampling distribution. It measures how much estimates would vary across repeated samples, rather than how much individual observations vary.

The sample mean $\bar x$ is an unbiased estimate of $\mu$. Dividing the squared deviations by $n-1$, not $n$, makes $s^2$ an unbiased estimate of $\sigma^2$. When $\sigma$ is unknown, $s/\sqrt n$ estimates the standard error of the sample mean.

For the sample 2, 4, 6, $\bar x=4$ and $\sum(x_i-\bar x)^2=8$, so $s^2=8/(3-1)=4$. The estimated standard error of $\bar x$ is $2/\sqrt3$.

For independent observations from a population with mean $\mu$ and variance $\sigma^2$, the sample mean $\bar X$ has the same mean $\mu$ but the smaller variance $\sigma^2/n$. Averaging therefore preserves the centre while reducing sampling variability.

If each observation is Normal, $X\sim N(\mu,\sigma^2)$, then the result is exact: $\bar X\sim N(\mu,\sigma^2/n)$. No proof is required.

If bag weights are $N(50,6^2)$ and $n=36$, then $\bar X\sim N(50,1^2)$. Thus $P(\bar X<48.5)=P(Z<-1.5)\approx0.0668$.

A confidence interval is a range produced by a sampling procedure for an unknown fixed parameter. A 95% procedure is designed so that, over many independent samples, about 95% of the intervals constructed in the same way contain the true parameter.

After one interval has been calculated, the parameter and its endpoints are fixed: the interval either contains the parameter or it does not. The confidence level describes the long-run success rate of the method, not a 95% probability assigned to this particular fixed interval.

For a two-sided test of $H_0:\mu=\mu_0$ at significance level $\alpha$, use the matching $100(1-\alpha)\%$ confidence interval. Reject $H_0$ exactly when $\mu_0$ lies outside the interval; if it lies inside, do not reject $H_0$.

If a 95% confidence interval for $\mu$ is $(22.24,22.56)$, the value 22.50 is compatible with the data at the 5% two-sided level, whereas 23.00 would be rejected.

For a random sample from a Normal population whose variance $\sigma^2$ is known, a two-sided $100(1-\alpha)\%$ confidence interval for $\mu$ is centred on the sample mean and extends by a critical value times its standard error.

Find $\bar x$, calculate $\sigma/\sqrt n$, select the standard Normal critical value for the required confidence level, then subtract and add the margin of error. State both limits with appropriate accuracy.

For $\bar x=22.4$, known $\sigma=0.4$ and $n=36$, a 98% interval uses $z_{0.99}=2.3263$. The margin is $2.3263(0.4/6)=0.1551$, giving $(22.245,22.555)$, or approximately $(22.25,22.56)$.

To test a Normal population mean when $\sigma^2$ is known, compare the observed sample mean with the null value $\mu_0$ in units of its standard error.

State $H_0$ and a one- or two-sided $H_1$ in terms of $\mu$. Calculate $z$, compare it with the appropriate standard Normal critical value (or use its $p$-value), then give a conclusion about $H_0$ and a contextual conclusion.

Suppose $H_0:\mu=1010$ against $H_1:\mu\ne1010$, with known $\sigma=8$, $n=100$ and $\bar x=1008.47$. Then $z=(1008.47-1010)/(8/10)=-1.9125$. Since $|z|<1.96$, do not reject $H_0$ at 5%.

The Central Limit theorem makes the sampling distribution of $\bar X$ approximately Normal for a sufficiently large independent random sample, even when the population distribution is not Normal. This extends Normal confidence intervals and hypothesis tests beyond Normal populations.

When $\sigma^2$ is unknown, use the unbiased sample variance $s^2$ and estimated standard error $s/\sqrt n$. Then use standard Normal critical values for the required large-sample confidence interval or test.

For a large sample with $n=100$, $\bar x=12.4$ and $s=3.0$, an approximate 95% interval is $12.4\pm1.96(3/10)$, giving $(11.812,12.988)$. The same standard error is used to standardise a hypothesised mean.

For two independent Normal populations with known variances, the difference of sample means is Normal. Test a claimed population-mean difference $\delta_0$ by comparing $\bar X-\bar Y$ with $\delta_0$ using the combined standard error.

Independence makes the variances of the two sample means add. The null difference is often 0, but use the value actually claimed and keep the subtraction order consistent in the hypotheses, numerator and conclusion.

With $\bar x=52$, $\bar y=48$, $\sigma_x=6$, $n_x=36$, $\sigma_y=8$, $n_y=64$ and $\delta_0=0$, the standard error is $\sqrt{1+1}=\sqrt2$. Hence $z=4/\sqrt2=2.83$, so reject equality at the 5% two-sided level.

For two independent large samples, the Central Limit theorem allows inference for $\mu_x-\mu_y$ even when the populations are not Normal. If the population variances are unknown, estimate each one from its own sample.

State hypotheses in terms of $\mu_x-\mu_y$, calculate the two-sample standard error from $s_x^2$ and $s_y^2$, standardise, then compare with the correct one- or two-tailed Normal critical value. Interpret the decision in context.

For $n_x=64$, $\bar x=75.2$, $s_x=8$ and $n_y=81$, $\bar y=71.8$, $s_y=9$, testing $H_0:\mu_x-\mu_y=0$ gives standard error $\sqrt{8^2/64+9^2/81}=\sqrt2$ and $z=3.4/\sqrt2=2.40$. This exceeds 1.6449 for a 5% upper-tail test.