Word2vec learns from scratch, and king minus man plus woman lands on queen.
A procedural corpus streams simple sentences about royalty, family, capitals, countries, languages and verbs in two tenses, and skip-gram with negative sampling learns a 20-dimensional vector for each word from nothing but the words around it, with frequent words subsampled and five negatives per pair. The overview is a t-SNE map run continuously on the live embeddings, so scattered words drift into glowing constellations as training proceeds. For an analogy the sky morphs to an exact linear projection onto the plane spanned by its two relations, stretched per axis to fill the view (an affine map, so parallel arrows stay parallel): the arrow from man to king is carried to woman, lands on the query vector, and a ring marks the word with the highest cosine similarity in the full 20 dimensions. Every other pair of the same relation (each country and its capital, each verb and its past tense) is drawn faintly too, and they run parallel.
Try it. Type three words such as 'king man woman' and press Enter to compute king minus man plus woman, or use the arrow keys to step through prepared analogies. Click a star to list its nearest neighbours, or drag one out of place and let go to watch t-SNE pull it home. Space pauses training and Shift+R retrains from new random vectors.
Paste this into Claude Code, Codex or any coding agent to get a simple version running, then take it wherever you like.
Build a word2vec demo that learns word embeddings live in the browser and draws them as a constellation map. Use JavaScript and the HTML canvas element, in a single index.html file with no libraries or build step, so I can open it directly in a browser.
Start simple:
- Make a canvas that fills the window, stays sharp on high-DPI screens (scale by devicePixelRatio), and resizes with the window. Paint it a deep night-sky color.
- Write a sentence generator from templates, such as 'the king sat on the throne and he ...', 'the queen sat on the throne and she ...', 'paris is the capital of france', 'people in france speak french', for a few pairs of royal and family words and a few countries.
- Train skip-gram with negative sampling: give every word an input and an output vector of 16 numbers, and for each word in a sentence and each neighbor within two positions, nudge the dot product of their vectors up through a sigmoid, and nudge it down for five random words. Run a few thousand pairs per animation frame.
- Project the normalized input vectors to 2D with PCA (power iteration on the covariance matrix is enough) and draw each word as a glowing dot with its label, colored by its group.
Once that works, make it beautiful:
- Add an analogy box: compute a - b + c, find the word with the highest cosine similarity (excluding the three inputs), and draw the arrow from b to a copied onto c.
- For an analogy, project onto the plane spanned by the two relation vectors instead of PCA, so the parallelogram is exact.
- Show a ticker with the current training sentence and its context window, and a small loss chart.
Explain the key ideas in short code comments. When you're done, tell me how to open it and suggest three directions I could take it next, such as a t-SNE overview map, drawing every pair of a relation as parallel arrows, or subsampling frequent words.
