LUCoS: New Method for Context Selection in Tabular Foundation Models
The cold-start challenge in tabular foundation models (TFMs) such as TabPFN is tackled by a novel technique known as LUCoS (Latent Unsupervised Context Selection). This method utilizes geometric selection within a learned latent space, avoiding the complications of the original tabular space, which is affected by diverse types, varying scales, and nonlinear relationships. Experiments with supervised oracles indicate that strategically selected labeled context sets can significantly exceed the performance of random selections within the same labeling budget. This research underscores the intrinsic geometric aspects of the issue and offers a resolution for low-label learning in tabular contexts.
Key facts
- LUCoS stands for Latent Unsupervised Context Selection
- Method targets cold-start setting in tabular foundation models
- TabPFN is an example of a tabular foundation model
- Context selection directly determines predictive performance in TFMs
- Supervised oracle experiments show carefully chosen sets outperform random selection
- Original tabular space lacks a natural metric for context construction
- Raw-space distances are unreliable due to heterogeneous types, mixed scales, nonlinear interactions
- Paper is available on arXiv with ID 2605.27254
Entities
Institutions
- arXiv