OpenAI's New WebRTC Architecture for Low-Latency Voice AI

OpenAI engineers Yi Zhang and William McDonald detail a rearchitected WebRTC stack for real-time voice AI, addressing scalability challenges for over 900 million weekly active users. The system uses a split relay-plus-transceiver model: a lightweight relay layer handles packet routing via ICE username fragments, while transceivers terminate WebRTC sessions. This design reduces the public UDP footprint to a small number of ports, enabling deployment on Kubernetes without exposing large port ranges. Global Relay ingress points shorten first-hop latency, and geo-steered signaling directs clients to nearby clusters. The relay, written in Go, uses SO_REUSEPORT and thread pinning for efficiency. Key results include lower latency, reduced jitter, and simplified infrastructure scaling. The architecture preserves standard WebRTC behavior for clients, ensuring interoperability with browsers and mobile apps. OpenAI's approach avoids kernel bypass, relying on a narrow implementation that handles global real-time media traffic with a small relay footprint.