ThreadSky
About ThreadSky
Log In
howard.fm
•
72 days ago
It's stacked matrix multiplies with ReLUs (or similar). Without the ReLU it doesn't work.
Comments
Log in
with your Bluesky account to leave a comment
[–]
rdgao.bsky.social
•
72 days ago
but you can say the same about many other architectural details in practice? Without depth, skip connections, tied convolutions, etc. it doesn't work.
As far as I know, nobody has gotten a 1 layer relu network to do AI stuff? What's the "right" abstraction level to make this argument sensible?
1
2
reply
[–]
howard.fm
•
72 days ago
You *can* get deep neural nets to work without convs or skip connections etc -- mainly you have to be super careful about init and normalization. (I don't recommend it though! Using skip connections makes life much easier. :) )
2
reply
[–]
tmoldwin.bsky.social
•
72 days ago
Logistic regression/perceptrons are great at solving many simple problems. More complex problems require moar layers.
1
reply
Posting Rules
Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service
×
Reply
Post Reply
Comments
As far as I know, nobody has gotten a 1 layer relu network to do AI stuff? What's the "right" abstraction level to make this argument sensible?