CiteCheck: A New Framework for Detecting LLM Citation Hallucinations
A novel hybrid system known as CiteCheck has been introduced to identify citation hallucinations in scientific texts produced by large language models (LLMs). This framework assesses whether a citation is linked to an actual scholarly work and evaluates the accuracy of its metadata. It gathers potential publications from external academic sources, utilizes a structured LLM verifier to compare citations, and categorizes the results into three labels: Exact, Minor, and Major. Researchers created a benchmark of 982 citations in physics featuring controlled inaccuracies that highlight both slight metadata variations and entirely fabricated references. In tests, CiteCheck achieved a macro-F1 score of 88.7 and an accuracy of 88.9%, surpassing the performance of GPT, Claude, and Gemini, including their web-search enhanced versions. This framework is vital in tackling the increasing issue of LLMs producing believable yet incorrect references, which is essential for maintaining scientific integrity.
Key facts
- CiteCheck is a hybrid framework for citation hallucination detection.
- It verifies if a citation corresponds to a real scholarly work and if its metadata is faithful.
- The system retrieves candidate publications from external scholarly sources.
- It uses a structured LLM verifier to compare citations against candidates.
- Verifier scores are mapped into three labels: Exact, Minor, and Major.
- A 982-citation physics benchmark was constructed with controlled corruptions.
- CiteCheck achieved 88.7 macro-F1 and 88.9% accuracy on the test set.
- It outperformed GPT, Claude, and Gemini baselines, including web-search versions.
Entities
Institutions
- arXiv