Causal Reasoning Framework Enables Creative Robot Tool Use

A new causal reasoning framework enables robots to creatively use tools beyond their primary functions. The approach first discovers causal relationships between tools and tasks through simulated experiments in a dynamics model. It decouples causal discovery into two components: VLM-based feature suggestion and counterfactual tool generation via geometric and physical feature perturbations. Novel objects are then classified based on identified causal features, and tool use skills are transferred via keypoint matching conditioned on those features. By reconstructing tasks in a dynamics model, the framework grounds tool use in physics. The method is illustrated with reaching a distant object using different sticks, scooping candies from a bowl with diverse items, and other tasks. The research is detailed in arXiv paper 2605.05411.