Theoretical Guarantees for Distinguishing LLMs via Dynamical Systems
A new study on arXiv (2607.28667) presents a theoretical approach to classifying large language models (LLMs) by viewing token embeddings as paths in a black-box dynamical system (DS). The authors describe this classification as a binary hypothesis test involving two stochastic linear DSs. They show that even with notable differences in dynamics, the total variation distance between the stationary marginal distributions can be minimized, establishing a limit on the accuracy of classifiers that ignore token dynamics. The researchers also find that the likelihood of misclassification decreases exponentially as the token sequence lengthens. Lastly, the paper addresses scalability and the transferability of embedding models, bridging gaps in past research while linking theory to empirical evidence in LLM fingerprinting and authorship attribution.
Key facts
- Paper arXiv:2607.28667
- Classifies LLMs by modeling token embeddings as dynamical system trajectories
- Formalizes classification as binary hypothesis test between two stochastic linear DSs
- Total variation distance between stationary marginal distributions can be arbitrarily small
- Provides fundamental accuracy floor for classifiers ignoring token dynamics
- Misclassification probability decays exponentially in token sequence length
- Addresses scalability and transferability across embedding models
- Published on arXiv
Entities
Institutions
- arXiv