Predict first: If every local derivative has magnitude about 0.2, what happens to a gradient after passing backward through 8 layers?
top bars = forward activation magnitudebottom bars = backward gradient magnitude
Lecture 12 · Gradient flow
Watch the forward signal and backward signal simultaneously through a deeper network.