3D Gaussian splatting in a 2D canvas, trained from scratch by gradient descent.
Each scene, a bonsai or a pastel still life, is a signed distance field covered by about 20,000 flat, anisotropic 3D Gaussians. Every frame their covariances are pushed through the camera with the Jacobian of the perspective projection to become 2D ellipses, which are depth sorted with a 16 bit counting sort and alpha composited front to back, with 4x4 tiles that stop work as soon as they turn opaque. Training is the real thing: the SDF is ray marched from twenty cameras to make reference photos, and a sparse cloud of gray blobs is fitted to them with Adam through a hand-written backward pass that differentiates color, opacity, the 2D conic, a Cholesky factor of the 3D covariance and the projected mean. Splats with large screen-space gradients are cloned or split and faint ones are pruned, so the needles, glaze and shadows sharpen out of the fog while the inset compares the reference with the splats from the same camera.
Try it. Drag to orbit (arrow keys too). Pick a scene, press 'Train from scratch' (or T) to watch a new set of splats learn it, use the slider or [ and ] to shrink the splats to their raw blobs, and press E to outline every ellipse. Z zooms, Space toggles the turntable.
Paste this into Claude Code, Codex or any coding agent to get a simple version running, then take it wherever you like.
Build a tiny 3D Gaussian splatting renderer with JavaScript and the HTML canvas element (2D context only, no WebGL). Put everything in a single index.html file with no libraries or build step, so I can open it directly in a browser.
Start simple:
- Render into a small offscreen buffer (about 320x200) with ImageData and scale it up to a full-window canvas with drawImage.
- Create about 3,000 splats on a sphere and a ground disc, each with a position, RGB color, opacity and 3x3 covariance. Make them flat discs: wide along the surface, thin along the normal.
- Set up an orbiting camera. Project each splat's center to pixels and its covariance with the Jacobian of the perspective divide: S2 = J W S W^T J^T. Add 0.3 to the diagonal so tiny splats stay a pixel wide.
- Invert the 2x2 matrix to get the conic, and use 3 standard deviations to find the splat's bounding box on screen.
- Sort splats by depth, nearest first. For every pixel keep an accumulated color and a transmittance T that starts at 1. For each splat and each pixel in its box: alpha = opacity * exp(-0.5 * d^T conic d); add color * alpha * T, then multiply T by (1 - alpha). Skip pixels whose T is already below 0.01.
- Composite the leftover T over a dark background.
Once that works, make it beautiful:
- Light each splat once when you create it (a sun direction plus soft sky light), so the scene looks shaded.
- Add a slider that scales every covariance, so I can shrink the splats to dots and grow them into a painterly blur.
- Drag to orbit with a little inertia, and slowly turn the camera when idle.
Explain the key ideas in short code comments. When you're done, tell me how to open it and suggest three directions I could take it next, such as fitting the splats to a reference image with gradient descent, splitting splats where detail is missing, or loading a real .ply splat capture.
