Predict first: Why is LOOCV error for 1-NN not automatically zero, even though 1-NN has zero training error when every training point can be its own nearest neighbor?
One leave-one-out step
Class 0Class 1Held-out pointlines = neighbors voting for prediction
Point 1 is held out. Ask the class to predict its label before revealing.
Which k would you choose?
| k | LOOCV mistakes | LOOCV error |
|---|