CURE: New Method to Improve Speculative Decoding in LLMs

A recent paper on arXiv (2608.00531) presents CURE, a dynamic repair tree designed for budget-conscious block-parallel speculative decoding in autoregressive Large Language Models (LLMs). This technique tackles the challenge of significant accuracy loss over extended sequences in parallel drafting systems, which results in elevated rejection rates and less-than-ideal speed enhancements. The researchers note that drafting mistakes are concentrated around specific high-uncertainty tokens that disrupt subsequent generation. CURE identifies these error-prone tokens using predictive confidence margins, selectively expands repair paths at these vulnerable points, and implements a budget-aware approach to minimize costly tree-verification processes. The abstract highlights the motivation and methodology, aiming to enhance wall-clock speedups by addressing errors at critical uncertainty areas without incurring high computational expenses.