Re-FORC: Adaptive Reward Prediction for Efficient Chain-of-Thought Reasoning

Researchers have introduced Re-FORC, a method for adaptive reward prediction that estimates future rewards by analyzing the quantity of thinking tokens. Utilizing a lightweight adapter trained on reasoning models enhances predictions, particularly with extended reasoning and larger models. Re-FORC facilitates the early termination of ineffective reasoning chains, leading to a compute reduction of as much as 26% while preserving accuracy. Additionally, it fine-tunes the selection of model and reasoning length, resulting in a peak accuracy increase of 1.7 percentage points with up to 12% less compute than the largest model alone. Furthermore, adaptive test-time scaling boosts accuracy by 9.9 percentage points compared to confidence-based baselines, allowing for dynamic reasoning controlled by cost-per-token thresholds.