Paint sound in time and frequency, then watch the real spectrogram of what you hear.
Time slides right to left through a fixed playhead. To the right is the score you paint, a grid of 20 columns per second by 210 oscillators, 30 per octave from 55 Hz to 7 kHz, and as it crosses the playhead an additive oscillator bank turns lit cells into sines, with amplitudes interpolated between columns. Thin strokes are peak-picked and tracked from column to column as single gliding partials, so a line between two rows is one sine rather than two beating neighbors, while wide areas like pictures and noise stay on the per-row bank. To the left is the proof: a spectrogram of the sound actually produced, from a Hann-windowed FFT of the output (4096 or 8192 points, always about 186 ms), so every stroke comes out the other side slightly blurred by the FFT's time and frequency trade-off. The autopilot composes a slow ambient piece with a pen (harmonic pads, bell-like melody glides and wind) and every third phrase hides a drawn picture in the high register.
Try it. Click for sound. Paint on the right of the playhead: holding the brush still draws a sustained tone because the canvas keeps moving, and moving it draws glides. Pick Tone, Harmonic (overtone stacks), Noise or Erase (keys 1 to 4), change brush size with [ and ], switch between log and linear frequency axes (L), turn the loop on or off (O), clear (C) and toggle the autopilot (A).
Paste this into Claude Code, Codex or any coding agent to get a simple version running, then take it wherever you like.
Build a spectrogram painter, where you draw in time and frequency and hear the picture, with JavaScript, the HTML canvas element and the Web Audio API. Put everything in a single index.html file with no libraries or build step, so I can open it directly in a browser.
Start simple:
- Store the painting as a grid: 200 columns (a tenth of a second each) by 96 rows, where row r is a sine at 55 * 2^(r / 12) Hz, so the vertical axis is logarithmic in pitch. Each cell holds an amplitude from 0 to 1.
- Draw the grid on a full-window canvas through a small offscreen canvas and ImageData, scaled up. Dragging paints a soft dot (a Gaussian a couple of rows tall) under the pointer, interpolating between pointer positions so fast strokes have no gaps.
- On the first click, create an AudioContext and a ScriptProcessorNode (or an AudioWorklet) that resynthesizes the column under a moving playhead: an oscillator bank that sums one sine per lit row. Interpolate each amplitude between columns and skip silent rows, so it is smooth and cheap. Loop the playhead across the grid and draw it as a glowing line.
Once that works, make it beautiful:
- Add brushes: a tone, a harmonic brush that also paints rows at 2f, 3f and 4f with falling amplitude, a noise band whose cells flicker randomly, and an eraser.
- Add a toggle between log and linear frequency axes, and notice how a harmonic stack bunches up on one and spreads evenly on the other.
- Draw a hidden picture into the score in code: draw a shape into a small offscreen canvas and copy its alpha into the high rows.
Explain the key ideas in short code comments. When you're done, tell me how to open it and suggest three directions I could take it next, such as an FFT spectrogram of the actual output next to the score, merging thin strokes into single gliding partials, or an autopilot that composes an ambient piece.
