From Samples to Populations

STAT 20: Introduction to Probability and Statistics

The top plots are population distributions; the bottom two are sampling distributions of the means from many samples of size 200. Match numbers to letters.

Correct answer: It depends on the sampling process.

These sampling distributions were actually made from taking samples of size 200 with replacement from populations of size 100. Most students will assume this when they first answer and choose 1-B, 2-A. This is a good opportunity to cue them to think very carefully about exactly how it is worded, then give them time to discuss and revote.

01:00

Scenario 1: Calling on the front row

This is the example from the notes, where the prof just takes a sample of size n = 18 from the first row. 1st year students are 10 times more likely to be drawn than 4th year students. The result is that most of the sample means are below the true population parameter (the blue vertical line).

How would the sampling distribution change if instead of calling on the front row, the Prof. put all 527 names on tickets in a box, mixed them up, then drew 18 names without replacement? Select the most dramatic change.

This describes a simple random sample, the sampling distribution will move up to be centered around the parameter.

A few follow up questions if you’d like:

1. What would happen if in the SRS we called on 8 instead of 18? Answer: the sampling distribution would still be centered about the parameter, but the standard deviation would increase.

Takehome: sampling variability for an average increases or decreases with changing sample size according to 1/sqrt(n).

2. Returning to the “calling on the front row” sampling method and plot: what would happen if, by the end of the semester, 90% of students sit in the same seat every class period? Answer: in that case there would be very little variability from one day / sample to the next and the SD of the sampling distribution would decrease dramatically.

Takehome: if there is little / no variability in the sampling process, there really isn’t much of a sampling distribution. It’s more like a constant and your only concern is bias.

01:00

Scenario 2: Drawing names from a box

Say we want to estimate the size of an average class at Berkeley.

Should we survey the students, and ask them how large their classes are? Should we ask the administration? Does it matter?

01:00

This one you may want to send straight to “discuss with your group”.

This is a good question to debrief with careful boardwork. You can go quite deep on it, so decide beforehand how far you’d like to go.

One way to start the debrief is to make a table of Pros and Cons for both the Administration and the Student Survey, then ask students for help filling it in. Things that might come up include sources of variability (small sample variability in the survey), non-response bias in the survey, and possibly selection bias in the survey (which could take several forms).

You can also debrief by sketching data frames. For the administrative data, draw a sketch of the data frame they have access to, where the unit is a class, and one of the columns records the class size. Then draw a sketch of the student survey data frame, where the unit is a student in a particular class, with a column that records the class size. If you summarize the class size column with the mean in the first dataframe and n = N, then you’ve just calculated the parameter - it’s a summary task, no generalization necessary.

If you take the mean of class size in the student survey data set, you’ll get a much bigger number, because you’ll be over-sampling students from the very large classes. This can be thought of as selection bias, leading to an overestimate of the parameter.

One wrinkle that you could end with: which of those two averages is actually better for understanding the effect of large class sizes? The first one is a property of classes. That might be useful for thinking through things like the size of classrooms on campus. The second average is a property of students. In a way, it better captures the student experience of being in large vs small classes.

This is going very far afield, but there’s a similar phenomenon when measuring population density of cities. If it’s measured as people / sq mile, that gives us a sense of how each sq mile of land feels. You could also measure it using the average number of people that you run into in a day per person. This gives a sense of how crowded things are for people.

These two measures diverge if you imagine a city on 2 square miles with a population of 100,000 people split evenly between those two squares. Then imagine a second city also on 2 sq miles, but 99,999 live in one sq mile and just 1 person lives in the other square mile. By the first metric, they have the same density, but the second city will feel much more dense to its residents.