Deep Learning Framework Enhances Grasping in Legged Robots
A novel deep learning technique is enhancing the grasping capabilities of quadrupedal robots equipped with arms, emphasizing both accuracy and flexibility. This method employs a simulation-based approach to reduce the need for extensive real-world data. It involves the Genesis simulation platform, which creates synthetic datasets to imitate grasp attempts on everyday objects. By analyzing thousands of interactions from multiple angles, it generates detailed grasp-quality maps. Utilizing a tailored convolutional neural network (CNN) inspired by U-Net architecture, the system processes various input types from onboard RGB and depth cameras, ultimately producing a heatmap to pinpoint optimal grasp locations, successfully tested on a four-legged robot.
Key facts
- Framework enhances grasping in quadrupeds with arms
- Uses sim-to-real methodology to reduce physical data collection
- Pipeline in Genesis simulation environment generates synthetic dataset
- Dataset includes thousands of simulated interactions from various perspectives
- Pixel-wise annotated grasp-quality maps serve as ground truth
- Custom CNN with U-Net-like architecture processes multi-modal input
- Input includes RGB images, depth maps, segmentation masks, and surface normal maps
- Model outputs grasp-quality heatmap for optimal grasp point identification
Entities
—