A ReLU network's exact linear regions, drawn as a stained glass window.
A small ReLU network learns to separate two classes of points, and the plane is drawn as what the network really is: a mosaic of convex polygons, inside each of which it computes one exact affine function. The mosaic is not sampled on a grid. Each neuron's pre-activation is affine inside every polygon, so the polygon is clipped in two along that neuron's line, layer by layer, and the affine maps are composed through the units that stay on. Panes are tinted by the predicted class and grow pale near the decision boundary, which is drawn as a gold wire with exactly one straight segment per pane, and the lead is heavier for cuts made by earlier layers.
Try it. Click to add a point of one class and Shift-click (or right-click) for the other. Use the chips or the arrow keys to change depth and width, which shatters the window and grows a new one; press 1 to 4 for datasets and R to reinitialize. Hover any pane to see its activation pattern and its affine logit.
Paste this into Claude Code, Codex or any coding agent to get a simple version running, then take it wherever you like.
Build a "ReLU stained glass" visualization with JavaScript and the HTML canvas element: a tiny ReLU neural network that classifies 2D points, drawn as the mosaic of linear regions it really is. Put everything in a single index.html file with no libraries or build step, so I can open it directly in a browser.
Start simple:
- Make a canvas that fills the window, stays sharp on high-DPI screens (scale by devicePixelRatio), and resizes with the window.
- Generate a two-class dataset (two interleaved spirals) in the square from -1 to 1, mapped to the middle of the screen.
- Write a small network from scratch: 2 inputs, two hidden layers of 8 ReLU units, one output logit. Train it with full-batch Adam and binary cross-entropy, a few steps per frame.
- Compute the linear regions exactly. Start with one polygon, the visible rectangle, carrying the identity affine map from (x, y) to the inputs. For each hidden layer, every neuron's pre-activation is an affine function a x + b y + c inside each polygon, so split each polygon along that line. Then zero the units that are off in each piece and compose the affine maps for the next layer.
- Fill each final polygon with a color from the predicted class at its centroid, and stroke the edges dark. Show the region count.
Once that works, make it beautiful:
- Treat it as a stained glass window: vary each pane's hue slightly using a hash of its activation pattern, make panes near the decision boundary pale, add a faint mottled texture, and draw thicker lead for edges cut by the first layer.
- The output is affine in every pane, so draw the decision boundary as one straight gold segment per pane.
- Let clicks add points and buttons change depth and width.
Explain the key ideas in short code comments. When you're done, tell me how to open it and suggest three directions I could take it next, such as a hover card showing a pane's activation pattern, shattering the window into falling shards when the architecture changes, or plotting region count against depth.
