Overfitting

STAT 20: Introduction to Probability and Statistics

Agenda

• Announcements
• Concept Questions
• Problem Set: Overfitting
• Lab: Cancer Diagnosis

Announcements

• Problem Sets:
  • PS 18: Overfitting releases Tuesday and due next Tuesday at 9am
  • Extra Practice: Logistic Regression releases Thursday (non-turn in)

• Lab 6:
  • Lab 6.1 releases Tuesday and due next Tuesday at 9am
  • Lab 6.2 releases Thursday and due next Tuesday at 9am

• Quiz 4:
  • next Monday in-class.
  • covers Wrong By Design through Logistic Regression (Thu/Fri)

Concept Questions

RQ 1

01:00

Which one of these (open pollev.com) is not an example of overfitting (either in real life or in statistics)?

Where is overfitting worse?

01:00

Suppose I overfit my model to the training data. In which scenario (for which training data) would I expect the test set performance to be significantly worse? Assume that the testing sets A and B look like their corresponding training sets.

If you overfit the data to training set A, you will likely have a complex curve (maybe a polynomial). This curve will fit poorly a testing set that looks similar to A. If you overfit to B, since the data has low variance you will have a positively sloping line. The testing set performance will be very similar to the training set performance (it will not be markedly worse).

Answer: Training set A

Problem Set

25:00

Break

05:00

Lab

25:00

End of Lecture