OPERA: Data Pruning Framework Enhances Dense Retriever Finetuning

A new data pruning framework named OPERA has been developed by researchers to enhance the efficiency and effectiveness of domain-specific finetuning for dense retrievers. This framework tackles the uneven contribution of training data pairs to the learning process. The initial focus is on static pruning (SP), which keeps only the query-document pairs with high similarity, uncovering a fundamental tradeoff between quality and coverage: while ranking performance (NDCG) improves, retrieval recall may decline due to diminished query diversity. To address this issue, the authors suggest a two-stage dynamic pruning (DP) approach that adjusts sampling probabilities at both the query and document levels during training, ensuring high-quality examples are prioritized while still utilizing the complete training set. Evaluations across eight datasets from six domains confirm the effectiveness of both methods: SP enhances ranking compared to standard finetuning, and DP achieves a better balance between ranking and retrieval. The paper can be found on arXiv with the identifier 2603.17205, categorized as 'replace-cross'.