Aero Realtime: A 4B Streaming Multimodal Model with Duplex Architecture
A recent study published on arXiv (2608.08469) presents Aero Realtime, a 4B multimodal streaming model aimed at facilitating low-latency, duplex generation. Unlike traditional streaming models that utilize a turn-based prefill-then-decode approach, Aero Realtime synchronizes video, audio, and text outputs on a unified temporal grid, with each audio segment of about 80 milliseconds predicting either a lexical or silence token. This design enables simultaneous progression of input and output, allowing a single autoregressive objective to determine both response timing and content generation. During inference, the model only adds the latest multimodal slot while retaining prior context, enhancing efficiency and reducing latency. The paper critiques the limitations of proactive methods such as micro-turn polling and external response gates, which disrupt seamless interaction and complicate KV-cache-friendly serving. This research is significant for advancing realtime interactive AI systems, especially in multimodal contexts.
Key facts
- Aero Realtime is a 4B streaming multimodal model.
- It uses a duplex architecture for realtime generation.
- It aligns video, audio, and textual output on a shared temporal grid.
- Each approximately 80-ms audio slot predicts either a lexical token or a silence token.
- The model allows input and output to advance together.
- It uses a single autoregressive objective to learn when to respond and what to generate.
- During inference, it appends only the newest multimodal slot.
- The paper is available on arXiv with ID 2608.08469.
Entities
Institutions
- arXiv