LLM Concept Lab

Word2Vec: meaning as a position in space

Gives every word a dense vector so similar words sit close and relations become directions.

The idea in plain words

A map of meaning. Cities that are alike sit close on a map; words that are used alike sit close in vector space. And just as "go 500 km north" moves you the same way from any city, a relation like man → woman is a consistent direction.

Step by step

  1. Give every word a short list of numbers (its vector), learned from the company it keeps.
  2. Close vectors mean similar usage: measure with cosine similarity.
  3. Relations become arrows. king − man + woman lands near queen, as long as you leave the input words out of the answer list.

Play it on paper first

Plot five foods and five vehicles on a sheet of paper so that similar things are close. Then draw one arrow from "car" to "cars". Does the same arrow turn "dog" into "dogs"? That repeatable arrow is the whole analogy trick.

Watch it explained

  • Word Embedding and Word2Vec, Clearly Explained!!! (StatQuest with Josh Starmer): Uses two tiny movie reviews ("Troll 2 is great!", "Gymkata is great!") to show how a network learns that words used in the same places get similar numbers. Starts from zero and ends at Word2Vec.
  • Vectoring Words (Word Embeddings) (Computerphile): Rob Miles types analogies into a real trained model live (king − man + woman, cities and countries) and shows where they work and where they break.

Other labs