GLASS: Training-Free Personalized Text Generation via Activation Steering
A team of researchers has introduced GLASS, a framework for personalized text generation that does not require training. This innovative approach utilizes sparse autoencoders to derive global user-style priors and local contrastive style vectors. By integrating these vectors into various model layers during inference, GLASS facilitates context-aware personalization without the need for retrieval or updates to parameters. Testing on LaMP and LongLaMP benchmarks demonstrates that GLASS surpasses retrieval, fine-tuning, and steering-based methods in ROUGE metrics and evaluations by LLM-as-judge. This framework effectively tackles the issue of distinguishing stylistic elements from the semantic content in individual writing styles.
Key facts
- GLASS is a training-free framework for personalized text generation
- Uses sparse autoencoders to extract global user-style priors
- Constructs local contrastive style vectors over clustered interaction scenarios
- Injects global and local vectors into different model layers during inference
- No retrieval or parameter updates needed
- Evaluated on LaMP and LongLaMP benchmarks
- Outperforms retrieval-, fine-tuning-, and steering-based baselines
- Metrics include ROUGE and LLM-as-judge evaluation
Entities
—