3.8 Potential Errors When Performing Tests

Identify Type I and Type II errors.

• A Type I error occurs when there is convincing statistical evidence that the alternative hypothesis is true (due to the small p-value), but it is not.
• A Type II error occurs when there is not convincing statistical evidence that the alternative hypothesis is true (due to the large p-value), but it is.
• The power of a hypothesis test is the probability that a hypothesis test will correctly reject the false null hypothesis.

Calculate the probability of Type I and Type II errors.

• The probability of making a Type I error is defined as the significance level, α. For a given study and hypothesis test, the probability of making a Type I error is typically set to a small value (e.g., 0.01, 0.05, 0.10) prior to collecting the data.
• The probability of making a Type II error is 1 power.

Identify the factors that affect the probability of errors in hypothesis testing.

• For a given study and hypothesis test, the probability of a Type II error should ideally be small, and thus, the power will be large (e.g., P Type II error 02. 0 and power0 .80). The probability of a Type II error decreases and the power increases when any one of the following occurs, provided the others do not change:
  • i. Sample size(s) increases.
  • ii. Standard error decreases.
  • iii. True parameter value is farther from the null hypothesis.
  • iv. Significance level of a test increases.

Interpret Type I and Type II errors.

• In some studies, making a Type I error may have more serious consequences than making a Type II error. In other studies, making a Type II error may have more serious consequences than making a Type I error. The consequences of each error should be considered prior to conducting the study.
• Because the significance level, α, is the probability of making a Type I error, the consequences of a Type I error influence decisions about a significance level.
• Because sample size influences the probability of making a Type II error, the consequences of a Type II error influence decisions about how large the sample size should be. Inference for Categorical Data: Proportions UNIT 3 100