Week 03 Interactive Labs

10-12 min · core idea

1. Why do we need three data splits?

Students choose polynomial degree using training or validation error, then lock the model and reveal the test set. A deliberately bad “choose using test error” path demonstrates why peeking contaminates the final estimate.

Open activity

10 min · diagnosis

2. Capacity Clinic

Change polynomial degree, sample size, and noise. Training and validation curves update together. Students diagnose underfitting / reasonable fit / overfitting and select a remedy.

Open activity

12-15 min · mechanics

3. Cross-Validation Microscope

Step through a holdout split, 3-fold, 5-fold, or LOOCV one fold at a time. Compare linear, quadratic, and join-the-dots models while tracking data usage and computational cost.

Open activity

10-12 min · classification

4. LOOCV for k-NN classification

Hold out one 2-D example, inspect the neighbors that vote, and count misclassifications. Compare k values under clean, overlapping, and noisy-label scenarios.

Open activity

10-12 min · model selection

5. Hyperparameter Search Game

The full CV-error curve is hidden and students have only eight evaluations. Probe k coarsely, refine locally, then reveal the global curve and fit the final model using all data.

Open activity