MedSkillAudit: A Framework for Medical Research Agent Skills
MedSkillAudit is a domain-specific audit framework for medical research agent skills, developed to assess reliability against expert review. The framework evaluates skill release readiness before deployment across five categories. 75 skills were tested, with two experts assigning quality scores, release dispositions, and high-risk failure flags. Agreement was quantified using ICC(2,1) and Cohen's kappa.
Key facts
- MedSkillAudit is a domain-specific audit framework for medical research agent skills.
- The framework assesses skill release readiness before deployment.
- 75 skills were evaluated across five medical research categories.
- Two experts independently assigned quality scores, release dispositions, and high-risk failure flags.
- Agreement was quantified using ICC(2,1) and linearly weighted Cohen's kappa.
- The study focuses on reliability against expert review.
- Agent skills are deployed as modular, reusable capability units in AI agent systems.
- Medical research agent skills require safeguards including scientific integrity and methodological validity.
Entities
Institutions
- arXiv