PINN-Based Policy Iteration Solves Nonconvex HJI Equations
A new preprint on arXiv, numbered 2507.15455, presents an innovative method for policy iteration that combines classic dynamic programming with physics-informed neural networks. This technique tackles complex, high-dimensional Hamilton-Jacobi-Isaacs (HJI) equations, which are important in stochastic differential games and robust control. The process involves alternating between solving linear second-order PDEs with set feedback policies and refining controls using minimax strategies through automatic differentiation. The researchers prove that, under typical Lipschitz and uniform ellipticity conditions, the value-function iterations converge uniformly to a unique viscosity solution. Their findings also show the iterations maintain equi-Lipschitz regularity, ensuring stability without requiring convexity in the Hamiltonian. Additionally, numerical tests demonstrate the method's effectiveness in a two-dimensional stochastic path-planning game.
Key facts
- Proposes a mesh-free policy iteration framework combining dynamic programming with physics-informed neural networks.
- Targets high-dimensional, nonconvex Hamilton–Jacobi–Isaacs equations.
- Equations arise in stochastic differential games and robust control.
- Method alternates between solving linear second-order PDEs and updating controls via pointwise minimax optimization.
- Convergence to the unique viscosity solution is proven under standard Lipschitz and uniform ellipticity assumptions.
- Analysis establishes equi-Lipschitz regularity of the iterates.
- Convergence and stability do not require convexity of the Hamiltonian.
- Numerical experiments demonstrate accuracy and scalability, including a 2D stochastic path-planning game.
Entities
Institutions
- arXiv