CWAC: A Unified Framework to Mitigate Overestimation in Off-Policy Reinforcement Learning
Researchers propose Collaborative Weighting Actor-Critic (CWAC), a framework addressing overestimation bias in deep off-policy reinforcement learning for continuous control. CWAC uses a distributional critic to model return uncertainty and a pessimistic critic to downweight high-uncertainty transitions, countering bias amplification from temporal-difference learning and greedy policy updates. The method is detailed in arXiv preprint 2607.26509.
Key facts
- CWAC stands for Collaborative Weighting Actor-Critic
- It targets overestimation bias in deep off-policy RL for continuous control
- Uses distributional critic to model return uncertainty
- Employs pessimistic critic to downweight high-uncertainty transitions
- Addresses bias from TD learning and greedy policy updates
- Published as arXiv:2607.26509
- Focuses on actor-critic methods
- Proposes unified framework rather than separate modifications
Entities
Institutions
- arXiv