Forecasting Side Effects of Activation Steering in Language Models
A recent study published on arXiv (ID: 2608.11227) investigates the predictability of side effects arising from a technique known as activation steering, used in the modification of language models. This method integrates learned directions into the hidden activations of these models, enabling behavioral modifications without the need for retraining. Researchers analyzed 67 distinct behaviors across three language models with open weights, revealing that side effects are prevalent, structured, and frequently asymmetric. Notably, they found that these side effects can often be predicted prior to steering, with their magnitude largely influenced by the intended behavior, contributing to enhanced safety in AI systems.
Key facts
- Paper arXiv:2608.11227 announced as new preprint
- Activation steering adds learned directions to hidden activations
- Technique enables targeted behavioral changes without retraining
- Side effects are common, structured, and often asymmetric
- Cross-effect matrix over 67 behaviors across three open-weight language models
- Side effects are largely predictable before steering
- Magnitude depends primarily on target behavior
- Direction can be forecasted from unsteered representations
Entities
Institutions
- arXiv