SGD, Adam, Lion and friends race downhill across a vintage topographic map.
Six optimizers (SGD, Momentum, Nesterov, RMSProp, Adam and Lion) run their real update rules on the real gradients of a loss surface, printed as an old survey quadrangle. The map is derived from a sampled grid: marching squares contours chained into polylines so index contours carry labels set along the line, hachures hung downhill from each contour and weighted by a north-west light, a hypsometric tint with hillshade, and a pond in every true minimum, found by descending from a lattice of seeds and confirmed with a Newton step and a positive definite Hessian. Their personalities are not scripted: momentum overshoots the banana valley, Adam and RMSProp travel diagonally because they rescale each axis, Lion strides in fixed sign steps, and on the Highlands map a noisy gradient sends each racer into a different basin. The legend charts every loss on a log scale.
Try it. Click or drag on the map to drop a new start point. Pick a landscape from the title box or keys 1 to 4 (Highlands, Rosenbrock, Beale, Saddle Pass). Drag a legend row left or right to scrub that optimizer's learning rate. Space pauses, R reruns, N jumps to the next scenario.
Paste this into Claude Code, Codex or any coding agent to get a simple version running, then take it wherever you like.
Build a race between gradient descent optimizers on a loss surface drawn as a topographic map, using JavaScript and the HTML canvas element. Put everything in a single index.html file with no libraries or build step, so I can open it directly in a browser.
Start simple:
- Make a canvas that fills the window, stays sharp on high-DPI screens (scale by devicePixelRatio), and resizes with the window.
- Use the Rosenbrock function f(x, y) = (1 - x)^2 + 100 (y - x^2)^2 with its analytic gradient, viewed over roughly x in [-2, 2] and y in [-0.5, 2.5].
- Sample log10(f + 0.001) on a grid every few pixels and draw contour lines with marching squares, in sepia on a cream paper background. Make every fourth line thicker.
- Implement SGD, Momentum, RMSProp and Adam as small update functions that take a position, a gradient and some state. Start them all at the same point and take a few steps per frame with requestAnimationFrame, drawing each one's path in its own ink color with a marker at its head.
- Click the map to restart the race from that point.
Once that works, make it beautiful:
- Draw the map once into an offscreen canvas and only redraw the trails each frame.
- Add a hypsometric tint (pale green low, tan high) and a soft hillshade from the north-west.
- Label the thick contours with their loss values, rotated along the line, and knock a gap in the line behind each label.
- Add a legend box with each optimizer's learning rate and current loss, and a small loss-over-time sparkline.
Explain the key ideas in short code comments, especially why each optimizer behaves differently. When you're done, tell me how to open it and suggest three directions I could take it next, such as adding Nesterov and Lion, a noisy multimodal landscape where optimizers end up in different basins, or hachure shading drawn downhill from the contours.
