Faster-WAM: Lightweight Action Heads for World Action Models

A recent publication on arXiv presents Faster-WAM, a technique aimed at minimizing computational demands in World Action Models (WAMs) by separating the action module's depth from the video backbone. The authors introduce the Dock of Transformer (DoT) concept, which utilizes a pretrained video Transformer as a representation hub, linking lightweight output heads via docking interfaces. Faster-WAM integrates a single-layer action head with a 30-layer video backbone, employing a docking interface that merges keys and values from all video layers and implements RoPE realignment. This method removes the necessity for further embodied pretraining, resulting in reduced inference latency while preserving performance. The paper can be found on arXiv under identifier 2608.02365.