AI Assistant Extracts Laboratory Know-How from Video for Safer Experimental Support

An innovative AI proof-of-concept has been created to capture essential laboratory insights often overlooked by conventional manuals. This assistant, which involves human input, integrates first-person experimental footage with multimodal AI and retrieval-augmented generation (RAG) methods. By utilizing powder X-ray diffraction experiments alongside videos recorded by students, the system gathers specific operational information and verbal confirmations from the documented processes. This research highlights the progress of Self-Driving Laboratories (SDLs) through Materials Informatics, while traditional human-led experiments continue to dominate educational and exploratory research. Essential practical skills and localized guidelines are crucial for ensuring safe laboratory practices. To address potential risks of unreliable outputs, a two-layer safety mechanism has been established. The findings were shared on arXiv with identifier 2604.16345v1.