Build your IB Psychology HL understanding of Data Analysis by evaluating how study design supports or limits generalization and transfer of findings.
To what extent can findings from this qualitative research study be generalized?
Refer to the paper 3 markbands when awarding marks.
The command term "to what extent" requires candidates to consider the degree to which findings from this qualitative study can be generalized.
Candidates may say that generalization could be important in qualitative research because it means that the results are relevant outside the context of the study itself. They could also argue that generalization may be difficult with a study such as this one because there is little previous research in the same area to compare it with.
Candidates could refer to ways of generalizing findings in qualitative research, for example, as outlined by Lewis and Richie (2003):
- Representational generalization: findings from qualitative research studies can be applied to populations outside the population of the study. With reference to the study in the stimulus material, candidates may say that even though the results may primarily apply to the seven clinics in the study, there is a similar pattern that could indicate a more general trend. If the sample is small the results cannot be statistically representative but if similar studies confirm the findings of this study, it could be argued that generalization is possible. The problem is that not many studies have been performed so far.
- Theoretical generalization: this refers to how far results could contribute to formulation of theory, for example, with reference to the study in the stimulus material that a shabby appearance negatively affects the treatment clients receive in most clinics for the poor. Such a theory could be put to test in other similar settings to test its generalizability and this could be used to develop further theory. For example, the findings from this study could lead to inferences about how to select and provide effective training of staff in healthcare clinics for the poor.
- Inferential generalization: the findings of the study can be applied to settings outside the study, ie transferability. Since the study is on poor, marginalized and homeless individuals, candidates may argue that the findings could be transferred to other similar settings (for example, shelters) but this will depend on the richness of the data.
Candidates may, in their discussion of generalization of findings from this qualitative research, briefly refer to statistical generalization (quantitative research) as part of their argument. This should be given credit as long as the main focus is on generalization from qualitative research.
Candidates may refer to the possibility of comparing findings from one qualitative study to other qualitative studies without using specific terminology relating to generalization, such as "representational", "inferential" and "theoretical".
The stimulus material below is based on a study on the influence of multitasking on student learning. Multitasking (doing more than one task at a time) and its consequences on learning has become a growing concern in education because students are increasingly engaged with their laptops or smartphones. In classrooms, students tend to switch between academic and non-academic tasks. Research indicates that this multitasking results in cognitive overload and weaker encoding of primary information into long-term memory. The aim of the study was to investigate if multitasking on a laptop would impair learning as measured by the number of correct scores on a comprehension test. The participants were forty undergraduate students from a university in North America (N=40). There were even numbers of males and females and the mean age was 18.9 years. A convenience sample of students enrolled in an introductory psychology course received course credit for participating. They were recruited from a psychology research website. It was only explained that the study involved listening to a class lecture and filling out a multiple-choice quiz. All participants attended a 45-minute lecture on meteorology in a traditional college classroom. Their primary task was to take notes using their laptops. The 20 participants in the multitasking condition were also asked to complete 12 online tasks during the lecture. The participants were randomly allocated a seat number as they entered the classroom. The researchers told participants that their individual instruction sheet and consent form were placed on their seat. After the lecture, all participants completed a 40-question multiple-choice quiz on the lecture content in order to check their comprehension. Finally, they were debriefed. The results showed that participants who multitasked during the lecture scored 11 % lower than participants who did not multitask. The result was significant and consistent with previous studies showing that multitasking during learning negatively affects encoding and transfer of information to long-term memory.
[Source 1: Faria Sana, Melody Wiseheart and Tina Weston (2014). ‘The direct and indirect effects of laptop multitasking in higher education.’ Pédagogie Collégiale, vol. 27, no. 2, Winter 2014; http://aqpc.qc.ca/sites/default/files/revue/Weston-Vol_27-2%2520%28A%29%2520.pdf Source 2: adapted from Computers & Education, Vol. 62, March 2013, Faria Sana, Tina Weston and Nicholas J. Cepeda, ‘Laptop multitasking hinders classroom learning for both users and nearby peers’, pp. 24–31, copyright 2012, with permission from Elsevier; https://www.sciencedirect.com/science/article/pii/S0360131512002254?via%3Dihub]
Discuss the possibility of generalizing/transferring the findings of the study.
Refer to the Paper 3 markbands when awarding marks.
Marks should be awarded according to the descriptors in the markbands. Each level of the markband corresponds to a range of marks to differentiate candidates' performance. A best-fit approach is used to ascertain which particular mark to use from the possible range for each level descriptor.
The command term "discuss" requires candidates to offer a considered review that includes a range of arguments. Conclusions should be presented clearly and supported by appropriate knowledge of generalizing the findings of the study in the stimulus material.
Generalization means drawing inferences from results of a study to something outside the study (external validity). The study in the stimulus is quantitative. The most appropriate model of generalization would be statistical generalization but that would require a random sample because this is typically representative of the target population.
Discussion points related to the possibility of generalizing/transferring the findings of the study in the stimulus material could include, but are not limited to:
- The sampling method (a convenience sample). In this study the population is university students in North America enrolled in an introductory psychology course. The study used convenience sampling and students could sign up for participation in the study or not (self-selection). This means that it is a non-probability sample rather than a random sample (probability sampling).
- The sample was relatively small with only 40 participants and therefore the sample is not considered statistically representative of the population even though the researchers had recruited an even number of males and females.
- The fact that students received credit for participation could also result in selection bias. When there is requirement to participate, students may be more likely to sign up for one study and not another on the basis of a convenient appointment time, rather than because they are making an informed choice about the kind of study they want to participate in.
- If the researchers added more participants to the sample it would enhance the possibility of generalization, as well as adding to statistical power. The more participants, the greater the chance that differences between participants will be balanced out, and therefore generalization is more likely to happen.
- If replications of this study arrived at the same conclusion (planned replication) the potential for generalization is enhanced. If the same theory of cause-effect relationship between multitasking and lowered performance found support in additional studies it would be more likely to confirm validity of the original findings. The result of this experiment has been supported by previous research and this indicates some external validity of the findings.
Candidates who use the terms generalization and transferability interchangeably should not be penalized.