Attention Rows as Compositional Data: Sink Token Choice Reverses Conclusions
A recent paper titled "Attention Rows as Compositional Data" (arXiv:2608.14712) examines the role of sink tokens in attention mechanisms, revealing that their inclusion or exclusion can significantly alter conclusions. The study highlights that attention rows represent probability distributions over tokens, with the sink token typically being the first. Utilizing standard metrics like cosine similarity and Jensen-Shannon divergence, the research found that verdicts can flip by 17-47% across ten models from five different families. Additionally, it identifies a prominent structure in BERT's head-clustering as an artifact, while Aitchison distance distinguishes between sink and content terms.
Key facts
- Paper arXiv:2608.14712
- Attention rows are probability distributions over tokens
- Sink token usually the first token
- Standard tools: cosine similarity, Jensen-Shannon divergence, Shannon entropy
- Choice of keeping or dropping sink token can reverse conclusions
- 17-47% of verdicts flip with convention across ten models from five families
- Most prominent structure in BERT head-clustering is an artifact
- Aitchison distance splits orthogonally into sink and content terms
Entities
Institutions
- arXiv