TabPFN-MT slashes multi-target inference costs from O(T) to O(1)
New model handles multiple targets in one pass, beating ensembles on 344 datasets.
Tabular data is everywhere, but most deep learning models handle one target at a time—requiring repeated forward passes for multiple predictions. TabPFN-MT, built on Prior-Data Fitted networks (PFNs), solves this by training on a multi-target synthetic prior. It uses an expanded y-encoder and a shared decoder head to capture inter-task dependencies in context, enabling simultaneous inference across T tasks. This design cuts the computational cost from O(T) to a constant O(1), a leap that makes multitask predictions as cheap as single-task ones.
Evaluated on 344 datasets averaging fewer than 1,000 samples, TabPFN-MT achieves an overall Accuracy rank of 4.89—the highest average rank among all tested models, including state-of-the-art single-task ensembles. It sets a new benchmark for deep tabular multitask learning, particularly in the small-to-medium data regime where gradient-based training struggles. The model's in-context learning approach bypasses traditional gradient updates, making it ideal for scenarios with limited data but multiple prediction targets.
- Inference cost drops from O(T) to O(1) forward passes for T simultaneous prediction tasks.
- Achieves top average accuracy rank (4.89) across 344 datasets, beating single-task ensembles.
- Specialized for small-to-medium datasets (<1,000 samples) via in-context learning, not gradient updates.
Why It Matters
Multitask tabular predictions now cost the same as single-task, unlocking efficiency for data scientists.