🤖🧠New paper!🧠🤖
A common assumption: In neural networks, "inductive bias" = "model architecture"
But in fact initial weights also greatly influence inductive bias! This makes it possible for the same bias to be realized in very different architectures - albeit w/ limits
https://arxiv.org/abs/2502.20237
A common assumption: In neural networks, "inductive bias" = "model architecture"
But in fact initial weights also greatly influence inductive bias! This makes it possible for the same bias to be realized in very different architectures - albeit w/ limits
https://arxiv.org/abs/2502.20237
Comments
https://arxiv.org/abs/2403.02241
"Neural Redshift: Random Networks are not Random Functions"
(Fig. 14 from the appendix ⬇️)