Practise justifying blocking, matching, and direct control by showing how a design reduces irrelevant variation, avoids confounding, and sharpens treatment comparisons.
A dermatologist will conduct an experiment to investigate the effectiveness of a new drug to treat acne. The dermatologist has recruited 36 pairs of identical twins. Each person in the experiment has acne and each person in the experiment will receive either the new drug or a placebo. After each person in the experiment uses either the new drug or the placebo for 2 weeks, the dermatologist will evaluate the improvement in acne severity for each person on a scale from 0 (no improvement) to 100 (complete cure).
Each twin in the experiment has a severity of acne similar to that of the other twin. However, the severity of acne differs from one twin pair to another.
For the dermatologist's experiment, describe a statistical advantage of using a matched-pairs design where twins are paired rather than using a completely randomized design.
Improvement scores will vary due to many factors, including initial acne severity, what treatment is received, and other variables such as diet and genetics. Because the pairs of twins are similar in initial acne severity, pairing allows for the variation in improvement scores due to the treatment received to be distinguished from variation due to initial acne severity, unlike in a completely randomized design. Consequently, using the matched-pairs design will provide a more precise estimate of the mean difference in improvement in acne severity for the new drug compared to the placebo and make it easier to find convincing evidence that the new drug is better, if it really is better.
Scoring
Essentially correct (E) if the response describes a statistical advantage of a matched-pairs design AND satisfies the following three components:
The advantage pertains to an inference made after collecting the data (e.g., the ability to distinguish between the effects of the treatments or the precision of the estimate of the drug effect)
Indicates that the matched-pairs design is better by using a comparative word (e.g., easier, clearer, greater) or by making an explicit comparison to a completely randomized design
Includes context (e.g., "drug," "improvement," "acne," or "twins")
Partially correct (P) if the response describes a statistical advantage of a matched-pairs design AND satisfies one or two of the three components.
Incorrect (I) if the response does not satisfy the criteria for E or P.
Additional Notes:
- To be considered an advantage of a matched-pairs design, the advantage described must be true for a matched-pairs design and not be true for a completely randomized design. For example, saying that "random assignment allows us to conclude cause-and-effect" is true of both designs. Similarly, "this allows the dermatologist to make conclusions about people with differing acne severity" is true of both designs. Also, "reduces bias" and "reduces variability in the estimates of the individual treatment means" is true of neither design.
- Responses that describe only the set-up of a matched-pairs experiment do not satisfy the requirement to describe an advantage of a matched-pairs design. For example, the response "in a matched-pairs design, the members of each pair will be similar in terms of acne severity" does not describe an advantage. However, "in a matched-pairs design, we can compare two people with similar acne severity" does describe an advantage.
- Advantages of a matched-pairs design that satisfy component 1 include "makes it easier to determine if the drug is effective," "gives a better estimate of the effect of the new drug," "reduces variability in the estimate of the drug effect," "makes the difference between the drug and the placebo more easily distinguishable," and "gives a clearer picture of how well the drug works."
- Advantages of a matched-pairs design that don't satisfy component 1 include "accounts for a source of variability," "controls for potentially confounding variables," "allows you to distinguish variation due to severity from variation due to treatment," "each person can be compared to someone similar," "reduces variability," "more balanced treatment groups," and "more accurate results."
- It is acceptable to provide a disadvantage of a completely randomized design rather than an advantage of the matched-pairs design (e.g., "The completely randomized design will make it harder to find convincing evidence that the new drug is better").
- It is acceptable to use the term "blocking" as a synonym for "pairing."
- A response that states that a matched-pairs design requires a smaller sample size to get power or precision equal to that in a completely randomized design and describes this advantage in context should be scored E.