AP Statistics 1.13: Experimental Design

Part (b):
Yes. Because the 0 ml/L concentration contains no fungus, the containers that are sprayed with the 0 ml / L concentration form the control group.

Part (c):
Label each container with a unique integer from 1 to 20. Then use a random number generator to choose 15 integers from 1 to 20 without replacement. Use the first five of these numbers to identify the five containers that will receive the 0 ml / L treatment. Use the second five of these numbers to identify the five containers that will receive the 1.25 ml / L treatment. Use the third five of these numbers to identify the five containers that will receive the 2.5 ml / L treatment. The remaining five containers will receive the 3.75 ml / L treatment.
(Alternative solution) Using 20 equally sized slips of paper, label five slips with 0 ml/L, five slips with 1.25 ml/L, five slips with 2.5 ml/L, and five slips with 3.75 ml/L. Mix the slips of paper in a hat. For each container, select a slip of paper from the hat (without replacement) and spray that container with the treatment selected.

Scoring

Parts (a), (b), and (c) are each scored as essentially correct (E), partially correct (P), or incorrect (I).
Part (a) is scored as follows:
Essentially correct (E) if the response satisfies the following three components:

Identifies the 4 concentrations (or mixtures or sprays) as the treatments

Identifies the 20 containers as the experimental units

Identifies the number of insects that are still alive in each container as the response variable

Partially correct (P) if response satisfies only two of the three components.
Incorrect (I) if the response does not meet the criteria for E or P.
Notes:

- Listing the four treatments satisfies component 1 (including ml/L is not required). However, if the list does not include all four treatments, component 1 is not satisfied.

- To satisfy component 1, the response must refer to plural concentrations/mixtures/sprays (e.g., the mixtures, the levels of the concentration). Referring only to the explanatory variable (concentration) does not satisfy component 1.

- The following responses satisfy component 2: "the 20 containers"; "the containers"; "the 20 groups of insects"; or "the groups of insects in each container." References to only "groups of insects" do not satisfy component 2 because it is unclear if these groups are formed by treatment or by container.

- To satisfy component 3, it must be clear that the response variable is being measured separately for each experimental unit. A response that says only "number of insects alive" does not satisfy component 3 because it could be referring to the total number of insects alive.

- To satisfy component 3, the response must be stated as a variable by using "number of" or equivalent. For example, "insects alive in each container" is not a variable and would not satisfy component 3.

- If the response states that the insects are the experimental units, then component 3 can still be satisfied by providing a binary response variable for each insect (e.g., whether the insect lived or died, survival status).

Part (b) is scored as follows:
Essentially correct (E) if the response indicates that there is a control group and justifies this claim by identifying the control group or by explaining that there is a treatment which contains no fungus.

Partially correct (P) if the response indicates that there is no control group because every container is sprayed with some mixture

OR
if the response states that there is a control group but implies that 0 ml / L is not a treatment (e.g., "the containers with 0 ml/L form a control group because they don't receive a treatment"; "yes, there is a group that got no treatment").

Incorrect (I) if the response does not meet the criteria for E or P.
Notes:

- The response does not need to explain the purpose of a control group.

- The response does not need to explicitly say "yes"-it can be implied by stating that there is a control group or saying "the control group is...."

Part (c) is scored as follows:
Essentially correct (E) if the response satisfies the following three components:

Creates appropriate labels for the units/treatments (e.g., label the containers from 1 through 20, label 20 slips of paper with five for each treatment)

Describes how to correctly implement the random assignment process

The random assignment process results in an equal number of experimental units assigned to each treatment

Partially correct (P) if response satisfies only two of the three components.
Incorrect (I) if the response does not meet the criteria for E or P.
Notes:

- If the response states that insects are the experimental units in part (a), the response in part (c) can be in terms of insects or containers. In either case, the same three components are used to determine the score.

- If the response states that the containers are the experimental units in part (a), but only describes how to assign insects to treatments in part (c), component 1 is not satisfied.

- For responses that use slips of paper:

If the number of slips of paper is not equal to the number of experimental units, then component 1 is not satisfied. The slips of paper do not need to be specifically identified as equally-sized.

If the slips of paper are not mixed/shuffled or the slips are not "selected at random," component 2 is not satisfied. Sampling without replacement is implied when using slips of paper, unless the response specifies sampling with replacement.

- For responses that use random number generators (or a 20-sided die):

If the initial assignment of numbers to units does not give each unit the same probability of being assigned to each treatment (e.g., units are represented by different numbers of integers), then component 1 is not satisfied.

If the response does not indicate that the numbers are selected without replacement or that different numbers must be used, the response does not satisfy component 2. The response does not need to specify the interval of numbers from which they are selecting (e.g., randomly generate a number from 1 to 20 ).

- For responses that use a table of random digits:

If the initial assignment of numbers to units does not give each unit the same probability of being assigned to each treatment, component 1 is not satisfied. For example, responses that use the labels 1 to 20 (not 01 to 20) do not satisfy component 1 because label 1 has a $\frac{1}{10}$ probability of being selected but label 20 has a $\frac{1}{100}$ probability of being selected.

If the response does not indicate that the numbers are selected without replacement or that different numbers must be used, the response does not satisfy component 2. The response does not need to specify the interval of numbers from which they are selecting or state that the numbers corresponding to unused labels will be skipped (e.g., skip numbers 00 and 21 to 99).

- For responses that use a 4-sided die (or random integers from 1 to 4):

If the die is rolled for each experimental unit, then component 3 is not satisfied because an equal number of units per treatment is not guaranteed.

If the die is rolled for each experimental unit until treatments are "full," then component 1 is not satisfied because this setup doesn't allow for all possible random assignments to be equally likely (unless the order of the units is randomized initially).

- If a response groups the experimental units before any random assignment (e.g., forms five groups of four containers or four groups of five containers), and then randomly assigns treatments to the groups or randomly assigns treatments within each group, component 1 is not satisfied. However, if a response forms groups in the context of a randomized block design with a reasonable blocking variable, component 1 can be satisfied.

- If a response describes two different random assignment processes in detail (e.g., how to randomly assign insects to containers and how to assign containers to treatments), both descriptions are scored according to the three components and the lower score is used.

- Responses that assign experimental units only to groups and not to treatments (e.g., randomly select five containers and put them in group 1) do not satisfy component 3.

- If the response randomly assigns insects to containers, the containers must be assigned to a treatment to satisfy component 3. In this case, the assignment of treatment to container does not need to be at random to satisfy component 3.

Complete Response
Three parts essentially correct

Substantial Response
Two parts essentially correct and one part partially correct

Developing Response
Two parts essentially correct and no parts partially correct

OR
One part essentially correct and one or two parts partially correct

OR
Three parts partially correct

Minimal Response
One part essentially correct

OR

No parts essentially correct and two parts partially correct

Improvement scores will vary due to many factors, including initial acne severity, what treatment is received, and other variables such as diet and genetics. Because the pairs of twins are similar in initial acne severity, pairing allows for the variation in improvement scores due to the treatment received to be distinguished from variation due to initial acne severity, unlike in a completely randomized design. Consequently, using the matched-pairs design will provide a more precise estimate of the mean difference in improvement in acne severity for the new drug compared to the placebo and make it easier to find convincing evidence that the new drug is better, if it really is better.

Scoring

Essentially correct (E) if the response describes a statistical advantage of a matched-pairs design AND satisfies the following three components:

The advantage pertains to an inference made after collecting the data (e.g., the ability to distinguish between the effects of the treatments or the precision of the estimate of the drug effect)

Indicates that the matched-pairs design is better by using a comparative word (e.g., easier, clearer, greater) or by making an explicit comparison to a completely randomized design

Includes context (e.g., "drug," "improvement," "acne," or "twins")

Partially correct (P) if the response describes a statistical advantage of a matched-pairs design AND satisfies one or two of the three components.

Incorrect (I) if the response does not satisfy the criteria for E or P.

Additional Notes:

- To be considered an advantage of a matched-pairs design, the advantage described must be true for a matched-pairs design and not be true for a completely randomized design. For example, saying that "random assignment allows us to conclude cause-and-effect" is true of both designs. Similarly, "this allows the dermatologist to make conclusions about people with differing acne severity" is true of both designs. Also, "reduces bias" and "reduces variability in the estimates of the individual treatment means" is true of neither design.

- Responses that describe only the set-up of a matched-pairs experiment do not satisfy the requirement to describe an advantage of a matched-pairs design. For example, the response "in a matched-pairs design, the members of each pair will be similar in terms of acne severity" does not describe an advantage. However, "in a matched-pairs design, we can compare two people with similar acne severity" does describe an advantage.

- Advantages of a matched-pairs design that satisfy component 1 include "makes it easier to determine if the drug is effective," "gives a better estimate of the effect of the new drug," "reduces variability in the estimate of the drug effect," "makes the difference between the drug and the placebo more easily distinguishable," and "gives a clearer picture of how well the drug works."

- Advantages of a matched-pairs design that don't satisfy component 1 include "accounts for a source of variability," "controls for potentially confounding variables," "allows you to distinguish variation due to severity from variation due to treatment," "each person can be compared to someone similar," "reduces variability," "more balanced treatment groups," and "more accurate results."

- It is acceptable to provide a disadvantage of a completely randomized design rather than an advantage of the matched-pairs design (e.g., "The completely randomized design will make it harder to find convincing evidence that the new drug is better").

- It is acceptable to use the term "blocking" as a synonym for "pairing."

- A response that states that a matched-pairs design requires a smaller sample size to get power or precision equal to that in a completely randomized design and describes this advantage in context should be scored E.

Model Solution

For each pair of twins, label one person as twin A and label the other person as twin B. For each pair of twins, toss a coin. If the coin lands on heads, twin A gets the placebo and twin B gets the active drug. If the coin lands on tails, twin A gets the active drug and twin B gets the placebo.

OR

Label the members of each pair of twins as "Twin 1" and "Twin 2." Using a random number generator, generate an integer from 1 to 2. Give the drug to the twin whose number is selected and the placebo to the twin whose number is not selected. Repeat for all pairs of twins.

OR

Label 1 notecard "A" and another notecard "B." For each pair of twins, shuffle the cards and give one card to each twin. The twin who gets "A" receives the drug and the twin who gets "B" receives the placebo.

Scoring
Essentially correct (E) if the response randomly assigns the two treatments within pairs of twins AND satisfies the following three components:

Uses a random process (e.g., flipping a coin, using a random number generator, shuffling cards) that gives each twin in a pair a 50\% probability of getting the drug and a 50\% probability of getting the placebo

Describes how to use the random process to assign one specific twin in each pair to the drug and the other twin to the placebo

Indicates that the random assignment process will be completed for each pair of twins

Partially correct (P) if the response randomly assigns the two treatments within pairs of twins AND satisfies only two of the three components for E.

Incorrect (I) if the response does not satisfy the criteria for E or P.

Additional Notes:

- A response that does not randomly assign both treatments within pairs of twins should be scored incorrect (I). Examples include a response that describes a completely randomized design, describes a crossover design where each person receives both treatments, uses pairs other than twins, does not use random assignment, or indicates that both twins in a pair receive the same treatment.

- For responses that use slips of paper or selecting items from a hat, the slips must be shuffled (or blindly drawn) or the hat mixed or shaken to have a random process and satisfy component 1.

- To satisfy component 2, the response must describe what to do for each possible outcome of the random process and specify which treatment each twin receives. For example, none of the following descriptions satisfy component 2:

"Roll a die. If it is 1-3, give the first twin the drug and the second twin the placebo." (Response doesn't describe what to do if the die is 4-6.)

"Have one member of each pair flip a coin. If it is heads, that twin gets the drug. If it is tails, that twin gets the placebo." (Response doesn't indicate what treatment the other twin will receive.)

"Flip a coin. If it is heads, give one twin the drug and the other twin the placebo. If it is tails, do the reverse." (Response doesn't specify which twin is getting the drug.)

"Label one slip of paper "A" and a second slip of paper "B." Mix them in a hat and have each member of the pair choose one slip." (Response doesn't specify if A represents the new drug or the placebo.)

- Ignore any discussion about randomly selecting 36 pairs of twins to obtain subjects for the experiment. Likewise, ignore any discussion about how to perform the analysis for a paired design (e.g., "subtract the improvement scores for each pair of twins").

- It is acceptable to refer to each pair of twins as a block.

Scoring for Question 2

Score

Complete Response

Three parts essentially correct

Substantial Response

Two parts essentially correct and one part partially correct

Developing Response

Two parts essentially correct and no part partially correct
OR

One part essentially correct and one or two parts partially correct
OR

Minimal Response

One part essentially correct and no part partially correct
OR

No part essentially correct and two parts partially correct