Reinforcement Learning Deployed on Humanoid Ballbot for the First Time
A team of researchers has successfully implemented reinforcement learning (RL) on a humanoid ballbot known as asRoBallet. Ballbots serve as a traditional benchmark for nonholonomic and underactuated control, yet the application of RL from simulation to physical hardware has faced obstacles due to issues with contact modeling, actuator delays, and safe exploration. To tackle these challenges, the researchers designed a detailed MuJoCo simulation that accurately represents the discrete roller mechanics of ETH-type omni-wheels, accounting for previously overlooked parasitic vibrations and contact discontinuities. Additionally, they developed a friction-aware RL strategy to help close the sim-to-real gap, pushing forward advancements in robotics control for real humanoid ballbots.
Key facts
- First successful deployment of RL on a humanoid ballbot hardware.
- Ballbots are a canonical benchmark for underactuated and nonholonomic control.
- Previous work used LQR and MPC for 3D balancing, not RL on hardware.
- High-fidelity MuJoCo simulation models discrete roller mechanics of ETH-type omni-wheels.
- Friction-aware reinforcement learning is proposed to close the sim-to-real gap.
- Challenges addressed: contact modeling, actuator latency & jitter, safe hardware exploration.
- The system is named asRoBallet.
- The research is published on arXiv (2604.24916).
Entities
Institutions
- arXiv