A Geometric Account of Activation Steering through Angle–Norm Decomposition

This blog post provides an overview of our recent paper: A Geometric Account of Activation Steering through Angle–Norm Decomposition.TL;DR: We decompose linear activation steering into two distinct operations: one that changes the angle of the activation toward a concept direction, and one that changes its norm. Through controlled experiments, we analyze the role of each component. We find that concept information is indeed primarily encoded in the angular component of activations. However, the norm also plays an important role, which we interpret as reflecting the effective representational capacity of a token. Based on this, we argue that activation steering should be described by two independent parameters: an angular parameter and a radial parameter, rather than by a single steering-strength coefficient.Main hypotheses and how we test themThe hypothesis and prior workActivation steering in LLMs is most commonly implemented as a parallel shift of activations along a precomputed concept vector, often called a steering vector. This design is based on the hypothesis that the manifold of LLM activations is locally linear. However, several recent works have criticized this approach, arguing that linear steering can substantially change the activation norm, pushing activations out of distribution and thereby degrading the mdel.One of the alternatives is spherical steering, which preserves activation norms and only rotates activations toward the concept vector by some angle (Vu & Nguyen, 2025, You et al., 2026). This idea, together with a few additional tricks, does indeed lead to better steering quality on several benchmarks compared with linear steering and some other approaches.However, these works did not sufficiently analyze the core hypothesis on which they rely, i.e. that preserving the activation norm exactly is necessary for better steering quality.Our framework for testing the hypothesisIn our work, we decided to thoroughly test this hypothesis. We proposed a