PhyMotion: Physics-Grounded 3D Motion Reward for Human Video Generation
Researchers have introduced PhyMotion, a systematic 3D motion reward system that anchors restored human trajectories within a physics simulator to assess the realism of motion in video production. Current video rewards depend on 2D perceptual cues and do not accurately represent 3D body states, contact, and dynamics, often giving high scores to movements that are physically unrealistic. PhyMotion retrieves SMPL body meshes from the generated videos, adapts them to a humanoid model in the MuJoCo physics simulator, and measures motion quality across various aspects of physical plausibility. This development tackles a significant obstacle in the post-training phase of reinforcement learning for generating human motion.
Key facts
- PhyMotion is a structured 3D motion reward for human video generation.
- It grounds recovered 3D human trajectories in a physics simulator.
- Existing video rewards rely on 2D perceptual signals without modeling 3D body state.
- Current rewards often assign high scores to physically implausible movements.
- PhyMotion recovers SMPL body meshes from generated videos.
- Meshes are retargeted onto a humanoid in the MuJoCo physics simulator.
- The reward evaluates motion quality along multiple dimensions of physical feasibility.
- The work targets a bottleneck in RL-based post-training for human motion.
Entities
Institutions
- arXiv