SkillSentry: Runtime Assurance Framework for Reliable LLM Agent Skill Execution

SkillSentry, a novel framework, seeks to enhance the dependability of LLM agents when performing skills. Although these agents possess procedural knowledge, they frequently struggle to consistently apply skills across similar tasks or during repeated executions, often due to deviations from established procedures or errors in step execution. To address this, SkillSentry employs a domain-specific language (DSL) for runtime guidance, which is initialized by merging skill specifications from documentation with execution data derived from historical traces. This framework aims to provide runtime assurance, ensuring that agents adhere to skill protocols accurately. The research paper can be found on arXiv with the identifier 2608.09253.