SkillGenBench: New Benchmark for LLM Agent Skill Generation

SkillGenBench has been launched by researchers as a benchmark aimed at assessing skill generation pipelines for LLM agents. In contrast to current benchmarks that focus on the application of given skills or performance on downstream tasks, SkillGenBench centers on skill generation itself. This benchmark follows a standardized protocol in which a generator takes raw corpora to create uniform skill artifacts, which are then executed in fixed environments and assessed through consistent methods. It encompasses two scenarios: task-conditioned generation, where a skill is developed post-task revelation, and task-agnostic generation, which involves creating reusable skills without a specific task. This initiative tackles the issue of producing accurate, reusable, and executable skills from various sources.