Framework for Evidence-Based Scientific Question Discovery with Historical Backtesting
A novel framework aimed at identifying evidence-based scientific questions has been unveiled to tackle a significant challenge in the scientific field: determining which inquiries merit investigation rather than simply providing answers. This framework converts a reproducible, traceable, and scope-limited research corpus into a list of ranked, falsifiable questions. It represents evidence through provenance-rich claims, identifies and categorizes tensions across papers, and involves human judgment for resolution. The refined signals are then transformed into questions, prioritized through a two-step process that distinguishes between scientific and execution priorities. This framework was applied to exoplanet atmospheres, integrating literature, structured catalogs, and space telescope data. A historical backtest showed that all questions derived from pre-2021 evidence were significantly addressed by literature from 2021 to 2026, including two answered questions, one of which was later rejected by the community. The paper can be found on arXiv with the identifier 2608.09968.
Key facts
- Framework turns research corpus into ranked, falsifiable research questions.
- Evidence is represented as provenance-carrying claims.
- Cross-paper tensions are detected, typed, and human adjudicated.
- Two-stage ranking protocol separates scientific priority from execution priority.
- Instantiated on exoplanet atmospheres.
- Historical backtest: all questions from pre-2021 evidence engaged by 2021–2026 literature.
- Two questions answered, including one premise later explicitly rejected.
- Paper available on arXiv (2608.09968).
Entities
Institutions
- arXiv