4.4 Data analysis

A research finding becomes useful evidence when the measure, comparison and context are clear. Interpretation should connect the observed pattern to the construct being studied without claiming more than the design can support.

Start by identifying what was measured, who was included and how the data were produced. Then ask whether the pattern supports a causal explanation, a change over time, a group difference or only an association; bias, sampling and measurement can alter each inference.

If a study reports higher memory scores after a strategy, check the control condition, timing and scoring rule before attributing the difference to the strategy. A short, standardised task may show an effect under those conditions while saying little about everyday learning.

A visible or statistically significant pattern is not automatically important, causal or generalisable. For each source, state the finding, its supporting value or excerpt, the design-based inference and the population or context boundary.

For quantitative findings, report direction and magnitude using the relevant values, spread, association, effect size or significance, then relate them to the design. For qualitative findings, identify a theme, explain the coding or interpretive pattern and support it with an excerpt or source detail. When both are provided, compare whether they converge, complement or contradict one another rather than forcing a single story.

A graph is a visual model of data, not decoration. The variable type and question determine the display: categories are separated, continuous intervals touch, time trends follow an ordered axis, and two measured variables can be compared as a scatterplot.

Bar graphs suit categorical or discrete conditions and leave gaps; histograms show continuous frequency distributions with touching bars; line graphs show change across ordered time points; scatterplots show the direction and strength of an association between two co-variables.

If three memory conditions produce mean scores, a bar graph compares the conditions. If temperature is recorded each day, a line graph shows the time trend. A scatterplot of stress and sleep can slope downward without proving that changing sleep would cause stress to fall.

A graph cannot repair poor sampling, misleading axes or confounded measures. Check units, scale, labels and spread before describing a pattern; correlation in a scatterplot is not causation.

Use an interpretation chain: identify variables, units, groups and sample sizes; read exact table values before summarising the pattern; inspect centre, spread, outliers and distribution; interpret an inferential result or effect size without treating significance as importance; then connect the result to the design, validity, bias and the conclusion. A result supports only the comparison or association actually measured.

Credibility is the trustworthiness and believability of a qualitative account; transferability is how far its detailed findings can inform another population or context. Neither means statistical proof that every case is identical.

A reader needs enough detail about sampling, procedure, recording and analysis to judge whether the interpretation fits the data. Triangulation can compare datasets, methods or researchers; reflexivity makes the researcher’s influence and limitations visible.

An interview study of one community may transfer to another setting when the experience and context are richly described and the reader can judge the fit. A second researcher checking the coding and comparing interview themes with another data source can strengthen credibility.

Small samples do not automatically make qualitative work worthless, and a familiar topic is not automatically transferable. Distinguish credibility from transferability, state what is described, and avoid generalising beyond the evidence.

A useful improvement targets a named threat: random or stratified sampling can address a biased sampling frame; standardised procedures or observer training can improve reliability; validated or culturally equivalent measures can improve construct validity; blinding can reduce expectancy bias; triangulation, member reflection and an audit trail can strengthen qualitative credibility; replication and a new context can test generalization or transferability. Larger samples alone do not repair confounding or invalid measurement.