4.2 Statistics and probability - AHL content

Data collection is the planned process of defining variables, selecting observations and recording measurements. Validity asks whether the method measures the intended construct; reliability asks whether it is consistent enough to repeat.

Sampling frame, wording, response options and recording conditions can introduce bias or random error. A questionnaire may be efficient, but leading questions and non-response can change who is represented and what is reported.

To estimate sleep in a school, define sleep time, choose a sampling method, pilot the question and record missing answers. A precise question about the previous night is more valid than an unbounded question about ‘usual’ sleep.

A reliable measurement can be consistently wrong, and a valid idea can be measured unreliably. Name the source of error and its direction before judging the data.

Question design should be unbiased, structured with consistent choices, precise and limited to relevant variables. For a $\chi^2$ table, justify category boundaries, keep expected frequencies above 5 and reduce degrees of freedom when parameters are estimated from data. Reliability checks include test–retest and parallel forms; validity checks include content and criterion-related validity.

Regression estimates a relationship between explanatory and response variables. If a straight line leaves a systematic curve in the residuals, a non-linear model or transformation may represent the pattern better.

Choose a model from theory, residual behaviour and the observed range. A transformed linear model can simplify calculation, but the fitted relationship must be converted back before interpreting predictions in the original units.

A growth dataset may follow y=ab^x; plotting ln y against x can make the pattern linear and estimate a and b. A good fit inside the measured range still says little about values far outside it.

A high R² does not prove the model is causal or valid everywhere. Check residuals, outliers, units and extrapolation before trusting a forecast.

Technology may fit linear, quadratic, cubic, exponential, power and sine regressions. Compare models using residual plots, $SS_{res}=\sum(y-\hat y)^2$ and $R^2$, the proportion of response variability accounted for by that model. Smaller $SS_{res}$ or larger $R^2$ helps only for the same response data; neither overrides mechanism, residual pattern or extrapolation limits.

A transformation maps a random variable to a new variable, such as Y=aX+b. For expectation and variance, E(Y)=aE(X)+b and Var(Y)=a²Var(X); a shift changes location while a scale changes spread.

For a discrete variable, transform each possible value and keep its probability. For a continuous variable, monotonic changes also alter the density and interval interpretation, so do not transform only the mean and ignore the distribution.

If E(X)=10 and Var(X)=4, then Y=3X−2 has E(Y)=28 and Var(Y)=36. The subtraction moves every value down by two; it does not reduce the variance.

Variance is squared in the scale factor and is not measured in X's units. Check whether the transformation reverses order or changes the support before interpreting probabilities.

For $Y=\sum a_iX_i+b$, $E(Y)=\sum a_iE(X_i)+b$; if the $X_i$ are independent, $Var(Y)=\sum a_i^2Var(X_i)$. The sample mean $\bar X$ is an unbiased estimator of $\mu$, and $s_{n-1}^2=\sum f_i(x_i-\bar x)^2/(n-1)$ is an unbiased estimator of $\sigma^2$. Independence is required for adding variances without covariance terms.

A sampling distribution is the probability distribution of a statistic, such as a sample mean, over repeated samples of the same size. It is not the distribution of individual observations.

For independent observations with population mean μ and standard deviation σ, the sample mean has mean μ and standard deviation σ/√n. The central limit theorem makes the shape approximately normal when n is sufficiently large under suitable conditions.

If μ=50, σ=12 and n=36, the standard error of the mean is 2. A sample mean of 54 is two standard errors above μ; that statement concerns sampling variation, not one student's score.

Increasing n reduces the standard error, not the population spread σ. The CLT does not erase dependence, extreme bias or a badly defined sampling process.

A linear combination of independent normal variables is exactly normal. If the population is normal, $\bar X\sim N(\mu,\sigma^2/n)$ for any $n$; for a general population the central limit theorem makes $\bar X$ approximately normal as $n$ grows, with $n>30$ treated as sufficient in examinations. Use variance $\sigma^2/n$ or standard error $\sigma/\sqrt n$ consistently.

A confidence interval combines a sample estimate with a margin of error to give a range of plausible values for a population parameter under a stated procedure and confidence level.

The interval widens when variability increases or the sample becomes smaller, and narrows when the sample is larger. Its interpretation is about the long-run performance of the method, not a probability assigned to a fixed parameter after the interval is calculated.

If a mean estimate is 72 with margin 4, a 95% interval is (68,76) under the model used. Repeating the sampling procedure would produce intervals that capture the true mean about 95% of the time in the long run.

A confidence level is not the chance that this already-fixed interval contains the parameter. Check assumptions, units and whether the target parameter matches the estimate.

For a normal population mean, use $\bar x\pm z^*\sigma/\sqrt n$ when population $\sigma$ is known and $\bar x\pm t^*s/\sqrt n$ with $n-1$ degrees of freedom when $\sigma$ is unknown, regardless of sample size. Report the confidence level, parameter, units and population in the contextual interpretation.

A Poisson random variable counts events in a time, length or area interval when events occur independently at a stable average rate λ. Its mean and variance are both λ, and disjoint intervals have independent counts under the model.

If the interval is multiplied by k, the mean rate becomes kλ. Use the model only when events are countable, the rate is approximately constant and one event does not make another more or less likely.

If calls arrive at an average rate of 3 per hour, the count in two hours is Poisson with mean 6. The probability of exactly four calls uses that interval's λ, not the original one-hour value.

Poisson is not any small count. Clustering, a changing rate or a maximum capacity can violate the assumptions even when the data look discrete.

If independent counts satisfy $X\sim Pois(\lambda_1)$ and $Y\sim Pois(\lambda_2)$, then $X+Y\sim Pois(\lambda_1+\lambda_2)$. Choose binomial for successes in fixed independent trials with constant $p$, Poisson for independent events at a uniform average rate, and normal for an appropriate continuous symmetric measurement model. Match $\lambda$ to the interval before using technology.

Choose the test from the parameter and information available: use a normal mean test when $\sigma$ is known, a t-test when $\sigma$ is unknown, a binomial test for a proportion, a Poisson test for a rate/mean, or a technology test of $H_0:\rho=0$ for bivariate normal data.

Samples may be paired or unpaired; matched pairs become one sample of differences. Normal, Poisson and binomial tests use the tail named by $H_1$; discrete critical regions maximize Type I error probability while keeping it below $\alpha$. t-test critical regions need not be calculated manually.

Error interpretation

A Type I error rejects a true $H_0$ and has probability controlled by the critical region. A Type II error fails to reject a false $H_0$; calculate its probability under the stated alternative parameter by finding the chance of landing outside the rejection region.

Do not select a test by distribution name alone: identify the population parameter, known or unknown variance, pairing, assumptions and direction. Poisson and binomial hypothesis tests are one-tailed in this syllabus.

A transition matrix stores probabilities of moving between states. With a column-vector convention, v_(n+1)=Pv_n; each column must sum to one. Powers of P give multi-step transition probabilities.

The orientation is part of the model: row-vector conventions transpose the multiplication order. A steady state is a distribution π with Pπ=π and entries summing to one, if the chain converges to one.

If 70% of users remain in state A and 30% move to B each month, the first column of P records those two destinations under the chosen convention. Multiply by the current state vector before interpreting the next month.

Matrix entries are not percentages to add across unrelated rows, and a stationary distribution is not guaranteed for every chain. State the convention, initial vector and convergence assumption.

A transition diagram labels the same probabilities encoded in $T$. A regular Markov chain has some power of $T$ with all positive entries and approaches a unique steady state independent of the initial state. Find long-run probabilities by repeated multiplication or solve $T\pi=\pi$ with entries summing to 1; $\pi$ is the eigenvector for eigenvalue 1 after normalization.