TabPFN-MT: Multitask In-Context Learner for Tabular Data

TabPFN-MT represents an innovative approach for handling tabular data, enhancing Prior-Data Fitted Networks (PFNs) to perform multiple prediction tasks concurrently within a unified framework. Unlike traditional PFNs, which necessitate distinct forward passes for each target variable and hinder information exchange between tasks, TabPFN-MT incorporates a broadened y-encoder along with a common decoder head. This design facilitates multitask in-context learning and allows for simultaneous inference. The model is trained on a synthetic multi-target dataset to capture dependencies among tasks and is tailored for small to medium datasets, typically under 1,000 samples, utilizing in-context learning instead of gradient-based methods. Evaluations across 344 datasets demonstrate that TabPFN-MT sets a new benchmark for deep tabular multitask learning. The research is available on arXiv with ID 2605.20234.