Experiments

STAT 20: Introduction to Probability and Statistics

Agenda

• Announcements
• Concept Questions
• Lab 7.2

Announcements

• Lab 7.1 due Friday 4/19 at 11:59 PM. Aim to finish during class today.
• Lab 7.2 due Tuesday 4/23 at 9:00 AM.
• Experiment groups must be of size 3 or 4.
• 2 problem sets will be assigned next week (both due Tuesday 4/30).

Concept Questions

We run an experiment to see if informative phone calls encourage people to vote. We obtain phone numbers for 1000 registered voters and randomly assign half of them to receive calls from volunteers. The volunteers tell them the location of their polling place and the date of the election. Our outcome is whether or not the subjects vote in the next election.

Which of the following are possible covariates for our study?

1. The party affiliation of the voter (Republican or Democrat).
2. Whether the next election is a presidential election or not.
3. Whether the voter voted in the last election.
4. Whether the voter votes primarily for Republicans or Democrats in the next election.
5. Whether the treated voters hung up on the volunteer.

The point of this question is to get the students thinking about whether variables are measured before or after treatment assignment.A and C are correct answers, D and E are not (although E is measured before the primary outcome, it is still after treatment). B is technically a covariate but probably not a useful one; unless our experiments spans multiple election cycles, it will have the same value for all subjects.

01:00

Which of the following claims about covariate balance are true?

1. Randomized treatment assignment tends to produce covariate balance.
2. A standardized mean difference that is not equal to zero likely means that randomization was not conducted correctly.
3. The Love plot shows the difference in means or proportions for each covariate across treatment groups.
4. In a hypothesis test for balance, the null hypothesis is that the treatment assignment and the covariate are independent.

01:00

The Essential Question of Our Time

Which is the best flavor?

This purely for fun. Some people have strong opinions about this.

Lab 7.2

Please:

1. Conduct your experiment
2. Record your data
3. Complete and submit part II as a group

If you don’t finish in the allotted time, finish up after class.

60:00

Could your taster distinguish between flavors?

1. Yes, and it was statistically significant (p < .05).
   
2. Yes, but it was not statistically significant (p > .05).
   
3. Yes, but we haven’t conducted the test yet.
4. No.
   
5. Not sure, we haven’t computed the difference in proportions yet.
6. We tested a different claim.
   

The goal of this question is to allow all groups to concisely share their results with the class. It also opens up the conversation about how there were many different ways to carry out this experiment. Two groups might have come to different conclusions because their tasters really had different abilities, or because their claims and experiments were slightly different, or just due to random chance.

You may want to project 1 or 2 protocols and ask the groups to talk through how they uses the principles of experimental design.