AIDAID202 · Week 02

Lecture 4 · Model complexity

Generalization and Regularization

Use several target functions and data regimes to see underfitting, overfitting, the effect of sample size, and how regularization controls effective model complexity.

AID202 · Fall 20263 target functions + variable data size
1 · Pick a target2 · Raise degree M3 · Add more data4 · Add λ
Prediction: As degree M increases, training error almost always improves. Does that guarantee better predictions on held-out data?
Reveal the generalization lesson
No. A flexible polynomial can fit signal and noise. The high-degree model may drive training error toward zero while test error rises.