RAG Models in Deception Detection: Theory-Guided AI Shows Minimal Accuracy Gains
A recent investigation published on arXiv (2608.08881) examines the application of Retrieval-Augmented Generation (RAG) models for detecting deception, informed by prominent theories on the subject. Researchers created seven distinct RAG models and evaluated their deception assessments against baseline models. Analyzing 700 statements from five established deception datasets, they tested four large language models (gpt-4o, claude-sonnet-4-6, ollama/llama3, deepseek-v4-flash) across two types of runs (RAG and baseline), yielding a total of 39,200 deception assessments. The accuracy rates were comparable to typical human performance, with RAG models at 54.5% and baseline models at 54.6%, revealing no significant difference. While RAG models exhibited a slightly lower truth bias (57.0%) than baseline models (59.7%), the difference was minimal. The theoretical framework influenced response bias significantly, varying from a strong lie bias (verifiability approach, 32.2%) to a strong truth bias (truth-default theory, 88.1%). The study also highlighted effects of content and model variations, although specifics were truncated. This research enhances the understanding of AI's role in deception detection, emphasizing the limited advantages of theory-driven RAG for accuracy improvement.
Key facts
- Seven RAG models based on deception theories were developed.
- 700 statements from five published deception datasets were used.
- Four LLMs: gpt-4o, claude-sonnet-4-6, ollama/llama3, deepseek-v4-flash.
- Two run-types: RAG and baseline.
- 39,200 deception judgments were rendered.
- RAG accuracy: 54.5%; baseline accuracy: 54.6%.
- RAG truth-bias: 57.0%; baseline truth-bias: 59.7%.
- Response bias ranged from 32.2% (verifiability) to 88.1% (truth-default theory).
Entities
Institutions
- arXiv