4. Further Probability & Statistics

$$A probability density function satisfies$$f(x)\ge0,\qquad \int_{-\infty}^{\infty}f(x),dx=1.$$Probabilities are areas:$$P(a<X<b)=\int_a^b f(x),dx.$$For a piecewise PDF, split every integral at each branch boundary.$$

$$Suppose$$f(x)=\begin{cases}kx,&0\le x<1,\k(2-x),&1\le x\le2,\0,&\text{otherwise}.\end{cases}$$Normalization gives$$1=k\int_0^1x,dx+k\int_1^2(2-x),dx=\frac{k}{2}+\frac{k}{2},so $k=1$.

$$Using the second branch,$$P(X>1.5)=\int_{1.5}^{2}(2-x),dx=0.125.If an interval crossed $x=1$, its probability would be the sum of two integrals, one from each branch.

If continuous $X$ has PDF $f$, then for a function $g$,E[g(X)]=\int_{-\infty}^{\infty}g(x)f(x),dx,$$integrated over the actual support. Important cases are$$E[X]=\int xf(x),dx,\qquad E[X^2]=\int x^2f(x),dx,and $\operatorname{Var}(X)=E[X^2]-E[X]^2$.

For $f(x)=2x$ on $0\le x\le1$,E[X]=\int_0^1 2x^2,dx=\frac23,$$and$$E[X^2]=\int_0^1 2x^3,dx=\frac12.$$Therefore$$\operatorname{Var}(X)=\frac12-\left(\frac23\right)^2=\frac1{18}. $$$$

For continuous $X$,F(x)=P(X\le x)=\int_{-\infty}^{x}f(t),dt.Hence, where differentiable, $f(x)=F'(x)$, andP(a<X\le b)=F(b)-F(a).A percentile $q_p$ satisfies $F(q_p)=p$.

If $f(x)=2x$ on $0\le x\le1$, then $F(x)=x^2$ on that interval. ThusP(0.2<X\le0.8)=0.8^2-0.2^2=0.60.The 75th percentile solves $q^2=0.75$, soq_{0.75}=\sqrt{0.75}. $$$$

For $Y=h(X)$, begin withF_Y(y)=P(Y\le y)=P(h(X)\le y).Translate this inequality into an event for $X$, using monotonicity and the support. Substitute into $F_X$, state the transformed support, then differentiate to obtain $f_Y(y)=F_Y'(y)$.

Let $f_X(x)=2x$ for $0\le x\le1$, so $F_X(x)=x^2$ there, and set $Y=X^3$. For $0\le y\le1$,F_Y(y)=P(X^3\le y)=P(X\le y^{1/3})=F_X(y^{1/3})=y^{2/3}. $$$$

$$Therefore$$F_Y(y)=\begin{cases}0,&y<0,\y^{2/3},&0\le y\le1,\1,&y>1,\end{cases}and on $0<y<1$,f_Y(y)=\frac{d}{dy}y^{2/3}=\frac23y^{-1/3}.Its improper integral over $(0,1)$ is one.

For a random sample of size $n$ from a normal population with unknown variance, test $H_0:\mu=\mu_0$ usingT=\frac{\bar X-\mu_0}{S/\sqrt n}\sim t_{n-1}\quad\text{under }H_0.The alternative $H_1$ determines whether one or both tails are critical.

A sample has $n=10$, $\bar x=52.0$ and $s=3.0$. Test $H_0:\mu=50$ against $H_1:\mu>50$ at 5%. The observed statistic ist=\frac{52-50}{3/\sqrt{10}}=2.108.With 9 degrees of freedom, the one-tail critical value is $1.833$, so reject $H_0$.

For independent samples of sizes $n_1,n_2$ from populations assumed to share variance $\sigma^2$, the pooled estimate iss_p^2=\frac{(n_1-1)s_1^2+(n_2-1)s_2^2}{n_1+n_2-2}.Equivalently, add the two within-sample sums $\sum(x-\bar x)^2$ and divide by the combined degrees of freedom.

$$From raw data, each within-sample sum can be calculated as$$\sum(x-\bar x)^2=\sum x^2-\frac{(\sum x)^2}{n}.$$Do this separately for each sample; differences between the two sample means do not enter the pooled variance numerator.$$

If $n_1=8,s_1^2=9$ and $n_2=12,s_2^2=16$, thens_p^2=\frac{7(9)+11(16)}{18}=\frac{239}{18}=13.28,so $s_p=3.64$ approximately.

For paired data define one signed difference consistently. Test $H_0:\mu_d=0$ usingT=\frac{\bar d}{s_d/\sqrt n}.If $n=9$, $\bar d=2.0$ and $s_d=2.4$, then $t=2.5$ with 8 df. Against $H_1:\mu_d>0$ at 5%, $2.5>1.860$, so reject $H_0$.

For a random sample from a normal population with unknown variance, a $100(1-\alpha)\%$ confidence interval for $\mu$ is\bar x\pm t_{n-1,,1-\alpha/2}\frac{s}{\sqrt n},where $s^2$ is the unbiased sample variance.

Suppose $n=13$, $\bar x=55.6$ and $s=13.07$. For a 90% interval, $t_{12,0.95}=1.782$. The margin is1.782\frac{13.07}{\sqrt{13}}=6.46,$$giving approximately$$(49.1,62.1).$$Report endpoints with suitable accuracy and units.$$

For independent equal-variance samples, let $n_1=10,n_2=8$, $\bar x_1-\bar x_2=5.7$ and $s_p^2=341.725$. A 95% interval uses $t_{16,0.975}=2.120$:5.7\pm2.120\sqrt{341.725\left(\frac1{10}+\frac18\right)},$$giving approximately$$(-12.9,24.3). $$$$

$$For a fitted Poisson model, estimate$$\hat\lambda=\bar x=\frac{\sum xf}{N},$$then$$P(X=r)=e^{-\hat\lambda}\frac{\hat\lambda^r}{r!},\qquad E_r=N P(X=r).$$A tail class has$$E_{\ge k}=N\left(1-\sum_{r=0}^{k-1}P(X=r)\right).$$$$

If $N=100$ and the fitted value is $\hat\lambda=1.2$, then the expected frequency for $X=0$ is100e^{-1.2}=30.12,and for $X=1$ it is100e^{-1.2}(1.2)=36.14.$$Continue with unrounded probabilities before combining any small tail classes.$$

For the final $k$ classes,X^2=\sum\frac{(O-E)^2}{E}.If $p$ distribution parameters were estimated from these data,\text{df}=k-1-p.$$Compare in the upper tail: a large statistic is evidence against the fitted model.$$

With final observed counts $(20,30,50)$ and expected counts $(25,25,50)$,X^2=\frac{25}{25}+\frac{25}{25}+0=2.00.If no parameter was estimated, df $=3-1=2$; since $2.00<5.991$ at 5%, do not reject H0.

For an $r\times c$ contingency table, $H_0$ states that the two categorical variables are independent. The expected count in cell $(i,j)$ isE_{ij}=\frac{(\text{row }i\text{ total})(\text{column }j\text{ total})}{\text{grand total}},and df $=(r-1)(c-1)$.

$$For observed counts$$\begin{pmatrix}30&20\10&40\end{pmatrix},$$row totals are 50,50 and column totals 40,60, so expected counts are$$\begin{pmatrix}20&30\20&30\end{pmatrix}.$$Thus$$X^2=\frac{100}{20}+\frac{100}{30}+\frac{100}{20}+\frac{100}{30}=16.67.$$$$

For $n$ signed ranks with no zero differences or ties, the positive-rank sum $W^+$ has, under $H_0$,E(W^+)=\frac{n(n+1)}4,\qquad \operatorname{Var}(W^+)=\frac{n(n+1)(2n+1)}{24}.$$These support a normal approximation when appropriate.$$

For independent samples of sizes $n_1,n_2$ and $N=n_1+n_2$, the rank sum $W_1$ hasE(W_1)=\frac{n_1(N+1)}2,\qquad \operatorname{Var}(W_1)=\frac{n_1n_2(N+1)}{12}.$$$$

Test $H_0:m= m_0$ against $H_1:m>m_0$. If 10 of 12 observations exceed $m_0$, then under $H_0$P(B\ge10)=\frac{\binom{12}{10}+\binom{12}{11}+\binom{12}{12}}{2^{12}}=\frac{79}{4096}=0.0193.$$At 5$$

For a large sign test, $B$ may be approximated byN\left(\frac n2,\frac n4\right),using a continuity correction. For signed-rank, use the null mean and variance from the previous card and correct the discrete rank-sum boundary by $0.5$ when using a normal approximation.

For independent samples $n_1=20,n_2=25$, let $W_1=560$ and $N=45$. Under identical populations,E(W_1)=20(46)/2=460,$$$$\operatorname{Var}(W_1)=20(25)(46)/12=1916.67.$$For an upper-tail normal approximation, continuity correction gives$$z=\frac{559.5-460}{\sqrt{1916.67}}=2.27.$$$$

For a non-negative integer-valued random variable $X$,G_X(s)=E[s^X]=\sum_{r=0}^{\infty}P(X=r)s^r.Thus $[s^r]G_X(s)=P(X=r)$, $G_X(0)=P(X=0)$ and $G_X(1)=1$.

For example, if $G(s)=0.2+0.5s+0.3s^2$, thenP(X=0)=0.2,\quad P(X=1)=0.5,\quad P(X=2)=0.3.$$The coefficient rule also lets a closed form be expanded to recover probabilities.$$

For $G(s)=E[s^X]$,E[X]=G'(1),\qquad E[X(X-1)]=G''(1).Since $X^2=X(X-1)+X$,\operatorname{Var}(X)=G''(1)+G'(1)-[G'(1)]^2.Differentiate before setting $s=1$.

For $X\sim\operatorname{Po}(\lambda)$,G(s)=e^{\lambda(s-1)},\quad G'(s)=\lambda e^{\lambda(s-1)},\quad G''(s)=\lambda^2e^{\lambda(s-1)}.Hence $G'(1)=\lambda$ and $G''(1)=\lambda^2$.

$$Therefore$$E[X]=\lambda,$$and$$\operatorname{Var}(X)=\lambda^2+\lambda-\lambda^2=\lambda.$$The cancellation checks the familiar Poisson equality of mean and variance.$$

If $X_1,\ldots,X_k$ are independent non-negative integer-valued variables and $S=\sum X_i$, thenG_S(s)=E\left[s^{\sum X_i}\right]=E\left[\prod s^{X_i}\right]=\prod G_{X_i}(s).$$Independence justifies the factorisation of the expectation.$$

If $X\sim\operatorname{Po}(\lambda)$ and $Y\sim\operatorname{Po}(\mu)$ independently,G_{X+Y}(s)=e^{\lambda(s-1)}e^{\mu(s-1)}=e^{(\lambda+\mu)(s-1)},$$so$$X+Y\sim\operatorname{Po}(\lambda+\mu).$$$$

If $X\sim\operatorname{Bin}(n_1,p)$ and $Y\sim\operatorname{Bin}(n_2,p)$ independently,G_{X+Y}(s)=(1-p+ps)^{n_1+n_2},hence $X+Y\sim\operatorname{Bin}(n_1+n_2,p)$. If the success probabilities differ, multiply the PGFs but do not label the result binomial without further justification.