LLMs and Information Theory Quantify Reconstructability of Astrophysical Methods
A recent study published on arXiv (2605.11154) presents a framework grounded in information theory to evaluate the effectiveness of reconstructing astrophysical techniques from written accounts. The researchers employed Shannon entropy and Jensen-Shannon divergence, viewing algorithmic reconstruction as a probability distribution produced by Large Language Models (LLMs). In their case study focusing on the spectral reconstruction of Trans-Neptunian Objects (TNO) from limited photometric data, they tested advanced LLMs with different amounts of manuscript content (Title, Abstract, Methods). Findings indicate that while increased text enhances the clarity of the algorithmic framework, it does not reduce variance at the implementation level, underscoring the challenges of reproducibility with current LLMs.
Key facts
- arXiv paper 2605.11154 proposes an information-theoretic framework for reconstructability
- Uses Shannon entropy and Jensen-Shannon divergence to measure text constraints on algorithmic hypothesis space
- Case study focuses on Trans-Neptunian Object (TNO) spectral reconstruction from sparse photometry
- Frontier LLMs were prompted with Title, Abstract, and Methods sections
- Increasing text clarifies algorithmic structure but does not eliminate implementation variance
- Published descriptions often lack detail for computational reproducibility
- Study treats algorithmic reconstruction as a probability distribution generated by LLMs
- Work demonstrates limitations of LLMs in reproducing complex astrophysical methods
Entities
Institutions
- arXiv