Neural Sparse Retrieval System for Industrial Music Search

A recent study published on arXiv introduces an advanced neural sparse retrieval framework aimed at enhancing fuzzy matching for industrial music searches at Amazon Music. This innovative system tackles the issue of queries that stray from indexed metadata due to errors like misspellings, transpositions, and phonetic differences, all while maintaining millisecond-level latency. The current High Confidence Index (HCI) system learns from customer interactions and uses exploration to select candidates. Although traditional n-gram matching facilitates this exploration, it is hindered by inadequate semantic robustness and excessive noise, which restricts learning from less common queries. The new approach integrates a cutting-edge inference-free sparse retrieval model with a specialized subword tokenization method tailored for the music sector to optimize exploration efficiency.