Study Argues Symbolic Music Tokenization Needs Better Coordinate Systems, Not Larger Units

A recent study published on arXiv suggests that GPT-style models are not directly applicable to symbolic music, as the benefits of tokenization stem from compression rather than merely from reusable combinations. The researchers argue that effective compression necessitates a framework where repeating patterns create stable, predictable conditional distributions. Thus, the main challenge lies not in identifying larger musical combinations but in finding the coordinate system that allows musical elements to be predictively compressible. To tackle this issue, they introduce the Effectiveness–Losslessness Framework, which conceptualizes tokenization as a means to create a predictively effective and relationally lossless coordinate system. Furthermore, the Predictive Effectiveness Principle establishes the Fact–Token Boundary, distinguishing between musical facts and tokens. This paper critiques existing methods that equate recurring musical elements, such as chords and motifs, with linguistic tokens, advocating for a significant reevaluation of symbolic music tokenization.