This might be the first time after 10 years that boosted trees are not the best default choice when working with data in tables. Instead a pre-trained neural network is, the new TabPFN, as we just published in Nature 🎉 - ThreadSky

sammuller.bsky.social • 89 days ago

This might be the first time after 10 years that boosted trees are not the best default choice when working with data in tables.
Instead a pre-trained neural network is, the new TabPFN, as we just published in Nature 🎉

Comments

sammuller.bsky.social•89 days ago

The new TabPFN is clearly the best single method for all regression and classification problems up to 10,000 data points and 500 features in our eval, no other limits.
Not only that, it requires only a few seconds to train on new data, while the baselines are tuned for 4 hours.

sammuller.bsky.social•89 days ago

How did we do this?
We train a neural network that natively handles tables, using attention across rows and columns, on millions of artificial tabular datasets from a meticulously designed data generator. It then performs in-context learning to make predictions on unseen data.

sammuller.bsky.social•89 days ago

The new TabPFN even outperforms Autogluon, which is a tool that mixes the best already existing methods (e.g. boosted trees and random forests). See plot (c) for classification results and (d) for regression results.

sammuller.bsky.social•89 days ago

Thanks to all of my collaborators, without you this would not have been possible. 🙏🏼🙏🏼 If you want to use our model, check out the following:

Nature article: https://nature.com/articles/s41586-024-08328-6
Try on free cloud: https://github.com/PriorLabs/tabpfn-client/
Try locally (gpu recommended): https://github.com/PriorLabs/TabPFN

viv-data.bsky.social•87 days ago

Is there any possibility that tabpfn will be available in R, perhaps through tidymodels?

sammuller.bsky.social•86 days ago

We have an r implementation under development currently. See here https://github.com/robintibor/R-tabpfn

viv-data.bsky.social•86 days ago

This is amazing! I would be happy to test it and contribute if necessary.

lottameijerink.bsky.social•89 days ago

Very interesting work! I am especially intrigued by the synthetic datasets. I saw the code is not available , but is there more detailed information about this than in the article ? Like, how often you used certain connection types, how you generated missing data, etc. Would love to know more!

sammuller.bsky.social•89 days ago

Thank you :) So far, we only open source the model itself and how to use it. We do not open source how to train it exactly, sorry for that :| there is a company starting based on the model, thus it is kinda its mode

lottameijerink.bsky.social•89 days ago

Ah, too bad! It would be interesting to know more about how you designed those data and how you made those choices.. but anyway, eager to try it so already very nice that’s possible!

rbly.bsky.social•89 days ago

Pretty interesting approach. Maybe I'm misreading, but did you give the other methods the chance to train on the synthetic data as well? I couldn't tell from the manuscript

Comments

Posting Rules

Reply