S1.4 - Correlation and regression

A scatter diagram plots paired observations $(x,y)$ and reveals the direction, form and strength of their association. A linear regression line of $y$ on $x$ is the straight line used to model the average response $y$ for a given explanatory value $x$.

Calculate $S_{xy}$ and $S_{xx}$ from the data or supplied summaries, find $b$, then use the means to find $a$. The least-squares line passes through $(\bar x,\bar y)$; this gives a useful arithmetic check and a reliable point when drawing the line on the scatter diagram.

Suppose $\bar x=4$, $\bar y=11$, $S_{xx}=20$ and $S_{xy}=30$. Then $b=30/20=1.5$ and $a=11-1.5(4)=5$, so the regression line is$$y=5+1.5x.$$To draw it, calculate two fitted points within the plotted $x$-range, join them with a straight line, and check that it passes through $(4,11)$.

In a regression of $y$ on $x$, $x$ is the explanatory (independent) variable and $y$ is the response (dependent) variable. Substitute an observed-range value of $x$ to predict the corresponding $y$; use a regression of $x$ on $y$ only when predicting $x$ from $y$.

For $w=46.0+3.27h$, where height $h$ is in cm and weight $w$ is in kg, a 1 cm increase in height corresponds to an average predicted increase of $3.27$ kg in weight. A height of 153 cm may be substituted if it lies within the recorded height range; a prediction at 170 cm is extrapolation if 170 lies beyond that range.

A linear change of variable must be undone before interpreting or reporting the original quantity. For example, if $d=w/11.5$ and a fitted line is $w=-3.46t+30.9$, then dividing every term by 11.5 gives $d=-0.301t+2.69$. Keep the named variables and their units visible through the rearrangement.

The product moment correlation coefficient (PMCC), $r$, measures the direction and strength of a linear association between two quantitative variables.

If $S_{xy}=-91.55$, $S_{xx}=445.57$ and $S_{yy}=26.43$, then$$r=\frac{-91.55}{\sqrt{445.57(26.43)}}=-0.844\ \text{(approximately)}.$$In context, larger values of one variable tend to occur with smaller values of the other. Because $r$ is fairly close to $-1$, a negative linear model may be useful, but the scatter diagram should still be checked.

Adding a constant to either variable or multiplying it by a positive constant does not change $r$; multiplying one variable by a negative constant reverses the sign. This is why converting units by a positive linear coding preserves the strength and direction of correlation.