BandTok: A 2D Mel-Spectrogram Tokenizer for Music Generation

A team of researchers has introduced BandTok, a 2D Mel-spectrogram tokenizer aimed at enhancing music generation for different generations. In contrast to current high-fidelity codecs that rely on residual multi-codebook quantization, BandTok utilizes Mel-frequency band tokens from a single shared codebook for each frame. This results in a time-frequency token grid that is physically interpretable and features a more independent token structure, minimizing sequential dependencies and reducing error accumulation. BandTok enhances reconstruction through a multi-scale PatchGAN objective and EMA codebook updates. Additionally, the authors present an autoregressive language model incorporating 2D Rotary Position Embedding (2D RoPE) to maintain the structure of temporal and frequency bands during generation. The full paper can be accessed on arXiv.