Watch the regression line and its parameter vector move together. Change the data geometry, learning rate, and update rule to see why optimization behavior changes.
AID202 · Fall 20263 optimization examples
1 · Inspect the fit2 · Take one step3 · Change α4 · Switch examples
Prediction: What will a very large learning rate do? What will a very small one do?
Reveal the interpretation
The negative gradient gives a downhill direction, but the learning rate controls step size. Too small is slow. Too large can overshoot, oscillate, or diverge.