HERO: Enhancing Long-Horizon Autoregressive Neural Operators with History-Enriched Rollout Training

A novel training approach known as history-enriched rollout training (HERO) has been introduced to enhance the long-term performance of neural operators acting as surrogates for time-dependent partial differential equations (PDEs). While neural operators recursively apply a learned evolution operator to their predictions, this autoregressive process can lead to the buildup of local errors due to the feedback of prediction inaccuracies. Current rollout-training methods focus on minimizing the difference between training inputs and self-generated states, but they only assess the absolute deviation from the true trajectory, failing to indicate whether the operator has resolved earlier long-horizon failures. HERO improves upon this by incorporating relative supervision based on the model's optimization history. It evaluates candidate rollouts from a periodically updated lagged operator, the current model, and a modified version, offering more comprehensive feedback. This method is elaborated in a paper on arXiv (ID 2607.29135), categorized as a cross-type submission, and aims to tackle the issue of error accumulation in autoregressive neural operators, potentially enhancing their dependability for long-term simulations in scientific computing.