LACE Framework Enables Cross-Thread Attention for Improved LLM Reasoning

A novel research framework named LACE revolutionizes the reasoning capabilities of large language models by facilitating interactions among parallel reasoning paths. Traditionally, these models operate by sampling various reasoning trajectories independently, often leading to repetitive failures due to a lack of shared insights. LACE modifies the model architecture to incorporate cross-thread attention, enabling simultaneous reasoning threads to exchange intermediate results and correct one another during inference. This method tackles a significant issue: the absence of natural training data showcasing such collaborative dynamics. To address this, researchers created a synthetic data pipeline that trains models to communicate and rectify errors across threads. Experiments reveal that this integrated approach surpasses conventional parallel search techniques, enhancing reasoning accuracy by more than 7 percentage points. The findings imply that large language models can be improved through synchronized parallel processing instead of isolated reasoning efforts. This research was published on arXiv under the identifier 2604.15529v1.