Overfitting

STAT 20: Introduction to Probability and Statistics

Concept Questions

Overfitting or not? (Taken from the reading questions)

01:00

Which one of these (open pollev.com) is not an example of overfitting (either in real life or in statistics)?

Note that the question is opposite of the reading questions. Options (b) and (d) are NOT examples of overfitting.

Are RSS and R square related?

01:00

If I decrease the RSS (e.g. by fitting a more accurate model) does the \(R^2\) value necessarily increase?

Yes, because \(R^2 = 1 - \dfrac{RSS}{TSS}\)

Overfit models

00:30

Fill in the blanks: overfit models tend to

• fit the training data (well/poorly)

• fit the testing data (well/poorly)

Fit the training data well, but testing data not as well. (Can choose option “poorly”)

Where is overfitting worse?

01:00

Suppose I overfit my model to the training data. In which scenario (for which training data) would I expect the test set performance to be significantly worse? Assume that the testing sets A and B look like their corresponding training sets.

If you overfit the data to training set A, you will likely have a complex curve (maybe a polynomial). This curve will fit poorly a testing set that looks similar to A. If you overfit to B, since the data has low variance you will have a positively sloping line. The testing set performance will be very similar to the training set performance (it will not be markedly worse).

Answer: Training set A