ORBIT Training Paradigm for Time Series Foundation Models

A new training method called ORBIT (Omni-Range Bootstrap Incremental Training) has been introduced to improve how we train time series foundation models (TSFMs), an area that hasn't received much attention. This framework helps clarify and control the distribution used during pre-training, tackling issues such as domain imbalance, context requirements, prediction timelines, and data gaps. ORBIT combines Bootstrap Multi-Level Sampling to manage exposure to datasets with Omni-Range Incremental Training, which adjusts context lengths and prediction timelines in one training session. Researchers applied this method to train Falcon-2.0, a simple univariate encoder-only Transformer that uses innovative techniques for handling missing data. You can find the study on arXiv with the identifier 2608.13262.