When Data Is Scarce: The Strength of the Prior in Tabular Foundation Models
Published in 2nd ICML Workshop on Foundation Models for Structured Data, 2026
In this paper we evaluate prior-data fitted networks (PFNs) on small-sample prediction tasks with fewer than 500 examples. In synthetic experiments, we assess how informative the priors of a correctly specified parametric model must be to match the performance of PFNs, and find that PFNs remain competitive even against strong, well-calibrated priors. On subsamples of the TabArena benchmark, PFNs outperform traditional regression and tree-based methods in small-sample settings.
Recommended citation: van Leeuwen, F. D., & van Erp, S. (2026). "When Data Is Scarce: The Strength of the Prior in Tabular Foundation Models." 2nd ICML Workshop on Foundation Models for Structured Data.
Download Paper