Dual-Force: New Offline RL Algorithm Enhances Diversity Maximization
A groundbreaking algorithm named Dual-Force is set for release, aimed at improving offline diversity maximization while adhering to imitation constraints. This innovative method, outlined in a recent arXiv paper (arXiv:2501.04426v2), tackles the stability challenges faced by current offline approaches relying on mutual-information and skill discrimination. By utilizing an off-policy estimator centered on a Van der Waals force goal linked to successor features, it removes the requirement for a skill discriminator. The Dual-Force also enhances training consistency amidst changing intrinsic rewards by leveraging a pre-trained Functional Reward Encoding, which facilitates zero-shot skill recall. The announcement was made on January 26, 2025.
Key facts
- Dual-Force is an offline algorithm for diversity maximization under imitation constraints.
- It uses an off-policy estimator of a Van der Waals (VdW) force objective computed from successor features.
- It eliminates the need for a skill discriminator.
- It stabilizes training by conditioning on a pre-trained Functional Reward Encoding (FRE).
- The FRE code enables zero-shot recall of encountered skills.
- The method aims to improve robustness to distribution shift without additional environment interaction.
- The paper is available on arXiv with ID 2501.04426v2.
- The announcement type is replace-cross.
Entities
Institutions
- arXiv