Activation Steering Enables Interpretable Attribute Control in Symbolic Music Generation

A new research paper from arXiv proposes a framework for fine-grained, interpretable control over symbolic music generation without retraining. The study focuses on the Multitrack Music Transformer (MMT) and uses inference-time activation steering to modulate discrete signal attributes like Pitch and Duration. The Difference-in-Means (DiffMean) methodology isolates latent directions in the residual stream, validating the Linear Representation Hypothesis with high correlation between steering magnitude and attribute shift. To handle feature entanglement in multi-attribute steering, the authors introduce a Dual Steering framework using Gram-Schmidt Orthogonalization. Experimental results demonstrate the effectiveness of this approach, bridging the gap between mechanistic interpretability and controllable music generation.