Game 05 · Engine · measures: optimizers
Descent Golf
Four loss surfaces, each a specific pathology: a ravine, a curved valley, a saddle, and a bumpy basin. Pick an optimizer and a learning rate, then reach the target in as few steps as you can.
Start at the marker. Reach the target loss before the step budget runs out.
How it works
Each surface has an analytic gradient, so there is no autograd and no library. SGD takes x ← x − lr·g. Momentum accumulates v ← 0.9v + g. Adam keeps the two moment estimates with bias correction, which is why it walks straight down the ravine at a learning rate that makes plain SGD oscillate out of the frame. The point of the ravine level is the condition number: the two curvatures differ by 25×, and every optimizer's behaviour on it follows from that one number.