ALaM: Augmented Lagrangian Multiplier Network for Safe RL
A new framework called Augmented Lagrangian Multiplier Network (ALaM) addresses training instability in reinforcement learning with state-wise safety constraints. Standard Lagrangian methods require a distinct multiplier per state, approximated by a neural network, but dual gradient ascent causes severe oscillations due to network generalization. ALaM stabilizes learning of state-dependent multipliers, enabling safer RL in real-world applications.
Key facts
- Safety is a primary challenge in real-world reinforcement learning.
- State-wise constraints require a distinct multiplier for every state.
- Multiplier networks approximate these multipliers.
- Standard dual gradient ascent causes severe training oscillations.
- Instability is exacerbated by network generalization.
- Existing stabilization techniques are designed for scalar multipliers.
- ALaM framework is proposed for stable learning of state-wise multipliers.
- The work is from arXiv:2605.00667.
Entities
—