MA19.A2.33

Use data from a randomized experiment to compare two treatments; limit to informal use of simulations to decide if an observed difference in the responses of the two treatment groups is unlikely to have occurred due to randomization alone, thus implying that the difference between the treatment groups is meaningful.

COS Examples

Example: Fifteen students are randomly assigned to a treatment group that listens to music while completing mathematics homework and another 15 are assigned to a control group that does not, and their means on the next quiz are found to be different. To test whether the differences seem significant, all the scores from the two groups are placed on index cards and repeatedly shuffled into two new groups of 15 each, each time recording the difference in the means of the two groups. The differences in means of the treatment and control groups are then compared to the differences in means of the mixed groups to see how likely it is to occur.