Asynchronous Human-AI Collaboration Framework for HPC Environments

A novel workflow framework facilitates asynchronous collaboration between humans and AI within high-performance computing (HPC) environments, tackling the issue of real-time interaction in resource-heavy situations. Created for defense and security applications, this framework permits workflows to pause at designated checkpoints for human contributions while allowing ongoing compute jobs to continue, thus avoiding resource idleness. It is compatible with SLURM-based scheduling, as well as both containerized and native tasks, functioning seamlessly across hybrid infrastructures, including HPC clusters, local systems, and cloud services. Showcased on MareNostrum 5 for model training, the framework enhances portability, efficiency, and oversight in AI operational workflows.