StarEmbed Benchmark Tests Time Series Foundation Models on Astronomical Data

A novel dataset named StarEmbed has been launched to assess time series foundation models (TSFMs) using astronomical data from variable stars. This dataset features authentic light curves from around 40,000 stars, categorized by experts into seven distinct classes, and offers evaluations for clustering, classification, and out-of-distribution (OOD) source detection. The findings, presented in arXiv paper 2510.06200, tackle the limitations in TSFM training datasets that frequently overlook complexities such as irregular temporal sampling. The irregularities in astronomical time series of stellar fluxes present a significant challenge. The research evaluates various TSFM architectures and training methods, revealing that the Chronos family, although pre-trained on regularly sampled non-astronomical data, excels in light curve clustering and OOD detection. Nonetheless, no TSFM outperforms the domain-specific transformer in classification tasks, indicating that while TSFMs are promising, specialized models retain advantages in specific areas. This benchmark is the first public dataset for light curves, serving as an essential tool for the community to enhance TSFMs on real-world irregular datasets.