Hey HN, I'm Abdur-Rahman Butler, a machine learning engineer at Isaacus.

I made an interactive write-up that walks you through the A to Z of vector embeddings, inspired by Brendan Bycroft's incredible interactive LLM visualisation guide (see here - https://bbycroft.net/llm).

Most of my work is on encoder models, and I kept noticing how few good interactive resources there are on embeddings. The ones that do exist tend to either plot some embeddings in 3D or only explain a narrow applied technique like semantic search or RAG.

So I wanted to make something that teaches how embeddings work by letting you play with them. In the write-up, there are interactive visualisations that allow you to explore the concepts of vectorisation, cosine similarity, semantic search, hierarchical retrieval, clustering, zero-shot classification and deduplication.

The centrepiece of the post is a game I've dubbed 'SemanticGeussr'. It's inspired by Wordle and GeoGuessr, and the goal is to draw a loop around a random target word by triangulating guesses on a 2D map of 43,339 common English words.

It's admittedly a bit intimidating at first, as semantic space does not perfectly match our intuition of meaning. But once you get the hang of it, the game becomes really addictive. I recently had the pleasure of demoing the game to some law students at the University of Melbourne, and we had a lot of fun throwing different vector equations at it to see where they'd land.

Some fun ones to get started with:

  london - english + french
  running - land + water
  avg(breakfast, lunch, dinner)
  0.4 * noodles + 0.6 * pasta

Although the post uses our embedding model, Kanon 2 Embedder, everything is pre-embedded so it all runs statically in your browser. We reduced the dimensions and quantised the vectors to shrink the whole vocabulary from about 311 MB down to 12.3 MB.

I've also included Python snippets alongside some of the visualisations if you want to follow along with the techniques later.

It was great fun to make and I hope you have fun playing around with it. Happy to take any questions, and if there's enough interest I might set up a dedicated games page for some of the other encoder-based games I've been working on.

Do let me know what highest score you get based on your target word and chosen path :)