ReaComp: Compiling LLM Reasoning into Symbolic Solvers for Efficient Program Synthesis
ReaComp is an innovative approach that transforms LLM reasoning traces into reusable symbolic program synthesizers tailored for constrained domain-specific languages (DSLs). These solvers do not require LLM calls during testing and achieve an impressive accuracy of 91.3% on PBEBench-Lite and 84.7% on PBEBench-Hard, surpassing LLMs by +16.3 percentage points on the hard set without incurring inference costs. In a neuro-symbolic hybrid context, ReaComp boosts PBEBench-Hard accuracy from 68.4% to 85.8% while cutting token usage by 78%, and enhances SLR-Bench hard-tier accuracy from 34.4% to 58.0%. The solvers generated are more Pareto-efficient compared to coding agents as per-instance solvers, incurring only a minimal one-time cost.
Key facts
- ReaComp compiles LLM reasoning traces into symbolic program synthesizers over constrained DSLs.
- Symbolic solver ensembles reach 91.3% accuracy on PBEBench-Lite and 84.7% on PBEBench-Hard.
- Outperforms LLMs with test-time scaling on PBEBench-Hard by +16.3 percentage points at zero LLM inference cost.
- Improves PBEBench-Hard accuracy from 68.4% to 85.8% while reducing token usage by 78%.
- Raises SLR-Bench hard-tier accuracy from 34.4% to 58.0% in a neuro-symbolic hybrid setting.
- Induced solvers are more Pareto-efficient than per-instance coding agents.
Entities
—