Pro$^2$Assist: Step-Aware Proactive Assistant for Long-Horizon Tasks
Pro$^2$Assist is an innovative proactive assistant designed by researchers to monitor detailed task progress and assess the user's changing state, offering timely support during extended procedural tasks. In contrast to current systems that mainly provide reactive assistance or minimal proactive help for brief events, Pro$^2$Assist utilizes multimodal data from augmented reality (AR) glasses to enhance motion perception. It gathers step-focused procedural context from various temporal scales and expert knowledge specific to tasks, enabling it to continuously reason and determine user requirements.
Key facts
- Pro$^2$Assist is a step-aware proactive assistant for long-horizon procedural tasks.
- It uses multimodal data from AR glasses for motion-based perception.
- It extracts step-oriented procedural context from multi-scale temporal dynamics and task-specific expert knowledge.
- It performs continuous reasoning to infer user needs.
- Existing systems provide reactive guidance or limited proactive assistance for isolated short-term events.
- Pro$^2$Assist tracks fine-grained task progress and reasons over the user's evolving state.
Entities
—