Research Examines Non-Stationarity in Time Series Foundation Model Embeddings

A new study investigates how non-stationarity manifests within the embedding spaces of time series foundation models (TSFMs), which are commonly employed as generic feature extractors. The research clarifies that non-stationarity, a concept from classical time-series analysis and statistical process control (SPC), is often incorrectly equated with distribution shift. In SPC methodology, non-stationarity indicates a process exiting a stable regime through changes in mean, variance, or the emergence of trends, with detection being crucial for quality monitoring and change-point analysis. The paper, identified as arXiv:2604.16428v1, explores how specific forms of distributional non-stationarity—including mean shifts, variance changes, and linear trends—become linearly separable in TSFM embeddings under controlled experimental conditions. Additionally, the work examines temporal non-stationarity related to persistence, which involves violations of weak stationarity assumptions. This analysis is motivated by the diagnostic traditions of SPC and aims to address the current poor understanding of non-stationarity in these model spaces.