Some caveats: All DL models are trained with a batch size of 1024, while we recommend using 256 for RealMLP on medium-sized datasets. Other choices (selection of datasets, not using bagging, choice of metrics, search spaces for baselines) can of course also influence results. 2/ - ThreadSky

About ThreadSky

dholzmueller.bsky.social • 43 days ago

Some caveats: All DL models are trained with a batch size of 1024, while we recommend using 256 for RealMLP on medium-sized datasets. Other choices (selection of datasets, not using bagging, choice of metrics, search spaces for baselines) can of course also influence results. 2/

Comments

dholzmueller.bsky.social•43 days ago

When including more baselines, RealMLP’s average rank slightly improves to make it the top-performing method overall, with a fifth place on binary classification, first place on multi-class, and second place on regression. 3/

dholzmueller.bsky.social•43 days ago

It is surprising how many DL methods perform worse than the simple MLP baseline by Gorishniy, @puhsu.bsky.social
et al. This highlights the benchmarking problems in the field (and potentially the difficulty in using many of these models correctly). The situation is slowly improving. 4/

dholzmueller.bsky.social•43 days ago

The benchmark: https://arxiv.org/abs/2407.00956
RealMLP: https://github.com/dholzmueller/pytabkit
5/
https://bsky.app/profile/dholzmueller.bsky.social/post/3lba4alreok23

dholzmueller.bsky.social•43 days ago

In case anyone is wondering about the name RealMLP, it is motivated by the “Real MVP” meme (which probably also inspired the RealNVP method). 6/6

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply