SkillShapley: A New Framework for Step-Level Attribution in AI Agent Skills
A recent study presents SkillShapley, a novel framework designed to assess the impact of individual steps within agent skills utilized by language models. This research, which can be found on arXiv (2608.13173), tackles the difficulty of discerning how each step influences overall task performance, such as in coding and document processing. The authors approach skill-step attribution through a Shapley value-based estimation method, introducing a two-phase strategy that incorporates empirical data on benchmark rewards and step interactions. SkillShapley reveals meaningful coalitions for calculating step-level attributions, providing a means to enhance and evaluate agent skills. This work is crucial for advancing the transparency and efficiency of AI agents, especially in intricate procedural tasks.
Key facts
- SkillShapley is a step-level attribution framework for agent skills.
- It models skill-step attribution as a Shapley value-based contribution estimation problem.
- The framework operates in two phases.
- It is motivated by empirical insights: discretized benchmark rewards and additive step interactions.
- The paper is available on arXiv with ID 2608.13173.
- Agent skills are external instructions for language agents to execute long procedural tasks.
- Existing agent skills are created through human manual crafting or agent execution traces.
- The problem of quantifying individual step contributions is open.
Entities
Institutions
- arXiv