Epistemic Transfer: A New Framework for AI-Assisted Verification
A recent paper published on arXiv (2608.08882v2) presents the idea of 'epistemic transfer' to assess AI tools that aid users in evaluating online assertions. The author contends that existing assessments primarily concentrate on tool performance during use, neglecting to gauge users' independent capabilities afterward. The study characterizes epistemic transfer as the influence of previous AI-supported verification on subsequent unaided performance with new claims. It differentiates this from correction effects, trust, reliance, and human-AI collaboration. The paper introduces two quantitative metrics: the Epistemic Transfer Effect (ETE), which compares unassisted performance over time across different conditions, and Tool-Removal Cost (TRC), which evaluates the immediate decline in performance upon tool removal. These metrics are incorporated into a practical evaluation framework designed for online experiments and field studies, merging answer-first and other approaches. This research fills a significant gap in AI evaluation by highlighting the importance of long-term learning and user independence.
Key facts
- Paper arXiv:2608.08882v2 introduces 'epistemic transfer'.
- Epistemic transfer measures AI tool impact on later unassisted performance.
- Distinguishes from correction effects, trust, reliance, and human-AI team performance.
- Introduces Epistemic Transfer Effect (ETE) and Tool-Removal Cost (TRC).
- Provides a practical evaluation protocol for online experiments and field studies.
- Focuses on AI tools for judging online claims.
- The protocol combines answer-first and other methods.
- The paper is available on arXiv.
Entities
Institutions
- arXiv