Neural Code Search System for MediaWiki Repositories
A new neural retrieval system named MediaWiki Code2Code Search has been created by researchers to facilitate semantic code-to-code discovery within MediaWiki's open-source framework. This system catalogs 1.29 million structural entities—such as functions, types, and templates—sourced from over 2,500 repositories, allowing retrieval based on computational intent instead of mere surface tokens. Its architecture separates GPU-heavy offline indexing from a CPU-only serving layer; the FAISS IVF-PQ index is compressed to 168.6 MB, achieving a 96.6% reduction from a flat float32 baseline. It provides a median query latency of 1.85 seconds on standard hardware, adhering to the 6 GiB RAM limit of Wikimedia Toolforge. Testing with a 27-query benchmark highlights its ability to connect user inquiries with implementation specifics, merging the speed of traditional Information Retrieval with the accuracy of Deep Learning.
Key facts
- System indexes 1.29 million structural entities across 2,500+ MediaWiki repositories
- FAISS IVF-PQ index occupies 168.6 MB, a 96.6% reduction from flat float32 baseline
- Median query latency of 1.85 seconds on commodity hardware
- Satisfies 6 GiB RAM constraint of Wikimedia Toolforge
- Split-build architecture decouples GPU offline indexing from CPU serving
- Enables retrieval based on computational intent rather than surface tokens
- Evaluated on a 27-query benchmark
- Addresses lexical gap between user queries and implementation details
Entities
Institutions
- Wikimedia Toolforge
- MediaWiki
- FAISS