Causation

  • review of claim types
  • program for the causation unit

At the beginning of this course, we considered four different types of claims. Let’s see an example:

  1. 15% of the students who responded to the class survey are Economics majors.
  2. 15% of Berkeley students are Economics majors.
  3. The next Berkeley student who’s gonna ask me for a recommendation will be an Economics major.
  4. If I drop Stat 20 from the requirements for the Economics major, I expect to receive fewer requests for recommendations.

Each one of these four claims illustrates a different type of claim made using data. As a brief recap of where we are in this course, let’s take them each in turn.

  1. 15% of the students who responded to the class survey are Economics majors.

This is a claim concerning the nature of the data that is on hand, a summary, descriptive claim. While these seem like the most straightforward type of claim, don’t underestimate their utility or the challenges involved in crafting them. They are virtually always necessary in order to make the other types of claims, and deciding which measure is most appropriate and the process of wrangling the data takes careful thought and time.

  1. 15% of Berkeley students are Economics majors.

This claim looks deceptively like the first but there is one important difference. The first claim concerns the sample from the class survey only. This second claim is about the broader Berkeley student population of which the survey is a subset. In other words, this is a generalization from a sample to a population.

For a generalization to be sound, we must take several considerations into account. First off: is the sample representative of the population or is it biased in some way? Secondly: what sources of variance are present? When working with a sample that originated from a chance method, it’s important to consider the degree to which sampling variance might be able to explain the structure you see in the data. Our primary tools in this area are confidence intervals, to assess the uncertainty in a parameter estimate, and hypothesis tests, to check whether the data is consistent with a proposed value of population parameters or distribution.

We need summaries to make this type of claims.

  1. The next student who’s gonna ask me for a recommendation for a Master’s in Business will be an Economics major.

This is a prediction, a claim that uses associations between variables in the data to predict the value of new observations about which we have only partial information. Whether the student wants to be recommended for a Master’s in Business is the partial information I have (the x-variable). But what will their y-variable be: their major? Now we recognize this as a classification problem. (Of course I could also try to predict how long it’ll take me to write the letters, yikes… That would be a regression problem.)

Prediction claims are always also generalization claims (generalized to new unseen data) and we need summaries to make this type of claims, as well.

  1. If I drop Stat 20 from the requirements for the Economics major, I expect to receive fewer requests for recommendations.

This bring us to the final claim, which is one concerning causation. The claim asserts that information in the data (the number of recommendations I get) can be attributed to specific cause (Stat 20 being a requirement for Economics majors). Causal claims are often the most challenging claims to craft but they are also some of the most useful. (In fact, quite frankly, most claims we ever make are implicitly, and someimes sneakily, dishonestly, causal.) Uncovering causes and effects is at the heart of many sciences from Economics to Biomedicine. They also help guide decision making for individuals (Is it worth my time to study for the final? Yeah, the answer is yes.), as well as for organizations (Will Twitter’s new option to pay for verification result in a net increase in revenue for the company?).

Causal claims are always also generalization claims (generalized to unseen counterfactual scenarios and to larger populations), they are sometimes also prediction claims (for example predicting recovery time for one specific patient, if a treatment is used vs a different one) and, as always, we need summaries to make this type of claims, as well.

For the remainder of Stat 20, we lay the foundation for causation in the social sciences, first by defining it precisely, then identifying a few methods for quantifying it from data.

Figure 1: Four types of claims made with data covered in this class.

P.S. Also remember that all of the three major claims (generalization, prediction, causation) are based on the Fundamental Assumption of Statistics™:

Statistics tries to make claims about new data using old data, by assuming that both old and new data come from probability distributions.

This is why the probability unit was the foundation of everything else.