KnowledgeGain: A New Metric for Science News Learning
A new metric called KnowledgeGain has been developed by researchers to assess the knowledge acquired by readers from science news. The effectiveness of this metric was confirmed through a controlled human study, which demonstrated its capability to measure varying levels of knowledge gain across different media formats. The findings from this study were utilized to fine-tune a prompt-only LLM reader simulator, which is employed to evaluate and select articles. Additionally, a follow-up human study revealed that articles chosen by the simulator enhanced post-reading accuracy and standardized KnowledgeGain compared to a robust generation baseline.
Key facts
- KnowledgeGain measures reader knowledge gain from science news.
- Controlled human study validated the metric.
- LLM reader simulator calibrated using study data.
- Simulator ranks and filters candidate articles.
- Second human study showed improved accuracy and KnowledgeGain.
- arXiv:2605.31099v1
- Announce Type: cross
- Abstract from arXiv
Entities
Institutions
- arXiv