Gelina: Unified Speech and Gesture Synthesis via Interleaved Token Prediction

A team of researchers has introduced Gelina, a comprehensive framework that simultaneously generates speech and accompanying gestures from text by utilizing interleaved token sequences within a discrete autoregressive architecture featuring modality-specific decoders. In contrast to traditional approaches that produce speech and gestures one after the other, Gelina ensures alignment in both synchrony and prosody. The system is capable of cloning multiple speakers and styles and allows for gesture-only synthesis derived from speech inputs. Evaluation results indicate that Gelina achieves competitive speech quality and enhances gesture generation when compared to unimodal baselines. This research was published on arXiv (2510.12834v4) and marks progress toward more intuitive human-computer interactions.