DLLM-TTS: Block Discrete Diffusion Language Model for Efficient Text-to-Speech
A novel text-to-speech (TTS) system known as DLLM-TTS has been unveiled, framing TTS as a conditional block discrete diffusion utilizing X-Codec2 neural audio codec tokens. This innovative method tackles the balance between autoregressive codec language models, which yield clear speech but necessitate extensive models and sequential decoding, and non-autoregressive techniques that enhance speed but sacrifice linguistic precision. The model breaks sequences into blocks and employs masked diffusion within these blocks, processing them in sequence to learn both local acoustic coherence and overall text-speech alignment. With a real-time factor (RTF) of 0.15, parallel token prediction during inference allows for efficient generation. A model with 0.6B parameters, trained on 20K hours, performs competitively on the Seed-TTS-eval benchmark, showcasing the practicality of block discrete diffusion language models for TTS synthesis.
Key facts
- DLLM-TTS is a framework for text-to-speech synthesis.
- It uses conditional block discrete diffusion over X-Codec2 neural audio codec tokens.
- The model decomposes sequences into blocks and applies masked diffusion within each block.
- Blocks are processed sequentially, learning local acoustic coherence and global text-speech alignment.
- Parallel token prediction within blocks enables efficient generation with a real-time factor (RTF) of 0.15.
- A 0.6B-parameter model trained on 20K hours achieves competitive performance on the Seed-TTS-eval benchmark.
- The approach balances intelligibility and speed, addressing the trade-off in existing TTS systems.
Entities
—