Lab: A Matter of Taste

STAT 20: Introduction to Probability and Statistics

Suggested timeline - [10 min] Slides and making teams - [45 min] Working on the protocol

General Advice - The students have seen nothing about experimental design yet in class. The idea is to have them use their own mental models to solve this problem, then later review and revise them. That’s to say: coach them only to be precise in their protocol, not to be using best practices in experimental design. Errors made here are great learning opportunities later. - The first phase of the lab can all be done with paper and pencil, so you can ask that students stow their laptops. - It might be helpful to guide students to start at the end, what they expect their data and plots to look like, then work back towards the hypotheses and claims. - As written, most null hypotheses will suggest the use of a difference in proportions as the test statistic.

Making Claims with Data

• So far: Pose question -> observe data -> analyze data -> make claim.

• e.g. Iranian election:
  1. Did fraud occur?
  2. Observe vote counts by city.
  3. Test of hypothesis that votes follow Benford’s law.
  4. Votes don’t follow Benford’s law well; but we wouldn’t expect them to in a fair election.

Making Claims with Data

• This week:
  1. Identify possible claim
  2. Imagine analysis that would show it.
  3. Plan data collection to allow that analysis.
  4. Carry out data collection + analysis.
  5. Check claim.

This week students will work backwards: start with a claim that they think may be true, then decide how to collect data in a manner that will be most effective at determining whether or not the claim is true.

But first

Please get into groups of 3 (or 4). Take turns introducing yourselves.

A Matter of Taste

Your challenge: Determine whether you can affect one your teammates’ perceptions of bubble water by manipulating their experience of tasting.

Each team will have access to

• 50 minutes (half of next class)
• 2 cans of soda water, each one from a different flavor
• small paper cups
• saltine crackers
• other materials welcome

Group Members

The main reason for the roles here is to dedicate just one person to handling the materials for health safety.

Question and Hypotheses

Prompt them to think carefully about their wording here in the question. How ambitious and general do they want to be?

It is also very common for people to suggest non-causal questions. It will help to underline the idea of intervention on this slide.

Hypotheses

Protocol

Here they must be very precise. We’re spoken about reproducible science in class, so use that language to encourage careful work. They must be able to pass this protocol off to another group, and they should be able to carry out the intended experiment with no ambiguity.

Data

This is intended to be a blank table with n rows and at least 1 column. They may plan to record either several extraneous variables or not enough to draw the conclusions they intend. As before, don’t correct them. They may discover mid-experiment that they did this part wrong, and that’s great.

Graphics

Encourage them to check that their viz is possible given their data frame and their data frame is possible give their protocol, and that their viz bears directly on their hypotheses.

Things to remember

• Be precise in your protocol.
• Be sure your claim corresponds to your protocol corresponds to your data corresponds to your plots.
• Important: Part I must be fully completed and shown to instructor before beginning of part II. Finish today if possible, by Friday night at latest.

50:00

Part II

The Essential Question of Our Time

Which is the best flavor?

This purely for fun. Some people have strong opinions about this.

Please:

1. Conduct your experiment
2. Record your data
3. Complete and submit part II as a group

If you don’t finish in the allotted time, finish up after class.

60:00

Could your taster distinguish between flavors?

1. Yes, and it was statistically significant (p < .05).
   
2. Yes, but it was not statistically significant (p > .05).
   
3. Yes, but we haven’t conducted the test yet.
4. No.
   
5. Not sure, we haven’t computed the difference in proportions yet.
6. We tested a different claim.
   

The goal of this question is to allow all groups to concisely share their results with the class. It also opens up the conversation about how there were many different ways to carry out this experiment. Two groups might have come to different conclusions because their tasters really had different abilities, or because their claims and experiments were slightly different, or just due to random chance.

You may want to project 1 or 2 protocols and ask the groups to talk through how they uses the principles of experimental design.