Text-Only Data Integration for Encoder-Dominated Speech Recognition

This study, found in the arXiv Computer Science > Computation and Language category, explores effective strategies for enhancing speech recognition through the use of text-only data, with an emphasis on encoder-centric models that facilitate quicker recognition. The authors present an extensive evaluation of various methods for incorporating text-only data, such as modality matching and dynamic downsampling, to create text-level representations in the encoder. Results from experiments using the LibriSpeech corpus indicate that employing a larger encoder paired with a smaller decoder can achieve performance levels comparable to or better than those of models with larger decoders. Furthermore, simpler setups, like random duration models, often outperform more intricate ones, greatly streamlining the training process. All related code and methodologies are publicly accessible.