TIER: New Reward Framework for Multi-Step Tool Use in LLMs

A new framework for reinforcement learning in large language models (LLMs), named TIER (Trajectory-Invariant Execution Rewards), has been introduced by researchers to tackle scalability challenges in multi-step tool composition. Unlike traditional outcome-based rewards that offer limited feedback or trajectory-supervised rewards reliant on annotated references, TIER generates supervision from function schemas and actual runtime execution. The reward system is broken down into several components: format validity, adherence to schemas, successful execution, and correctness of answers, providing comprehensive and interpretable feedback at the sequence level through detailed verification of each tool-use step. This approach allows for recognition of any valid execution path, accommodating various solution methods and adapting to changing tool interfaces. The study is available on arXiv under ID 2605.16790.