FACTOR: New Credit Assignment Method for Multi-Turn Agent Reinforcement Learning

A recent research article presents FACTOR, a novel approach for credit assignment in multi-turn agent reinforcement learning. This paper, which can be found on arXiv (2608.07118), distinguishes between trajectory-level credit assignment and token-level credit distribution. By utilizing checkpoint-calibrated TD residuals, FACTOR allocates credits per action that contribute to the trajectory advantage, while feedback-conditioned teacher-student likelihood gaps help distribute each credit among the executed action tokens. The normalization at the action level maintains the action-average coefficient and avoids sign flips at the token level. Additionally, this method incorporates an action-mean reduction, eliminating the implicit relationship between an action's scalar surrogate weight and its token length. FACTOR demonstrates consistent improvements over established baselines in ALFWorld, WebShop, and ScienceWorld environments. The research team announced this submission on arXiv.