Auto-JEPA: Latent World Model for End-to-End Autonomous Driving
A recent study titled Auto-JEPA introduces a latent world model aimed at autonomous driving, emphasizing the prediction of future driving intentions instead of reconstructing the entire future environment. Detailed in arXiv:2607.29031, this model prioritizes scene characteristics that influence future ego actions, utilizing joint-embedding prediction to synchronize an intent embedding with the latent future ego trajectory representation. It retrieves actionable trajectories from a fixed memory, organized by a scene-conditioned selection module. Notably, the visual encoder remains unchanged, and the model does not require explicit perception annotations or a learned trajectory generator, focusing solely on task-specific goals. This method differs from traditional world models that engage in dense video predictions, occupancy states, or agent movements. The paper asserts that effective planning should concentrate on relevant features rather than reconstructing the entire future world.
Key facts
- Auto-JEPA is a latent world model for autonomous driving.
- It predicts future driving intent via joint-embedding prediction.
- The model does not reconstruct the complete future world.
- It uses a frozen visual encoder and no explicit perception annotations.
- Trajectories are retrieved from a fixed memory and ranked by a scene-conditioned module.
- The paper is available on arXiv with ID 2607.29031.
- The approach focuses on scene features affecting future ego action.
- It contrasts with dense prediction methods like video or occupancy prediction.
Entities
Institutions
- arXiv