Semantic Phonons: Lattice Vibrations in AI Internals

Phonons refer to collective vibrational modes in crystals, propagating through the lattice as a wave.Introduction One of the most pressing questions in AI research is how language models represent meaning. How does a model know that "excellent" is better than "good"? How does a model represent time scales, e.g., hour vs. day? Where does the knowledge live that July comes after June? Language models are trained to predict text, and they are very good at doing so, but our understanding of their internal representations remains limited. Figuring this out is the holy grail of mechanistic interpretability: the quest of opening the black box and understanding what is going on inside.There is now growing evidence that semantics are not stored as arbitrary, unstructured patterns. Instead, we find that models encode meaning in predictable geometric structures. A recent example comes from Karkada et al. (2026), who showed that the twelve months of the year form a near-perfect circle in the activation space of a language model (Gemma 2B). The months are not just close to each other in some vague sense, but are arranged in an ordered loop, reflecting the cyclical structure of the year. The question is whether this result is a coincidence, or reflecting something deeper.Without noting it explicitly, what the authors did in their paper is actually to apply mathematical frameworks established decades ago in the field of solid-state physics, in particular for the study of phonon modes. Phonons describe how vibrations propagate through crystals, and the mathematics behind them provides a rich framework for understanding collective behavior on discrete lattices. The case of the months of the year is an example of one of the simplest structures one can deal with: periodic boundary conditions, where the first and last atom of the chain are coupled to each other, as if bending the chain into a ring. Crucially, physics offers an entire arsenal of further-reaching tools, covering different