Synthetic Task Distributions Key to Tabular Foundation Model Quality

A new study from arXiv (2605.18971) investigates what determines the quality of tabular foundation models, finding that synthetic pretraining distributions are the primary source of inductive biases, unlike in language or vision models. The authors argue that standard synthetic priors are overly idealized, omitting irregularities and failure modes crucial for deployment robustness. They introduce O'Prior, a compositional realism prior with four components: a hierarchical SCM meta-generator for diverse functional families, a modular realism engine for heterogeneous marginals and missingness, an explicit stress module for confounding and support-query mismatch, and a curriculum-governed, leakage-safe generation protocol. By holding architecture, optimizer, and compute budget constant, the study isolates prior design as the key variable. The research highlights the need for more realistic synthetic data to improve tabular model performance.