My mailbox is literally flooded with discussions of whether “Deep learning is hitting a wall”, almost none of which cite my much-ridiculed-at-the-time 2022 paper called “Deep learning is hitting a wall”. nautil.us/deep-learnin... - ThreadSky

garymarcus.bsky.social • 86 days ago

My mailbox is literally flooded with discussions of whether “Deep learning is hitting a wall”, almost none of which cite my much-ridiculed-at-the-time 2022 paper called “Deep learning is hitting a wall”.

https://nautil.us/deep-learning-is-hitting-a-wall-238440/

Comments

raulizahi.bsky.social•86 days ago

Because people’s minds are dazed with the potential of tens of billions of investment dollars going down the drain and hundreds of billions of dollars to be lost in stock market value.

gaiamarcus.bsky.social•85 days ago

📌

plopseven.bsky.social•86 days ago

I mean, I filed a complaint with like 8 government agencies about corporate web-scraping ML algorithms stealing private college portfolios for training purposes, but nobody has written me back for 9 months.

They’re stealing data from college students who haven’t even graduated yet. That’s a wall.

plopseven.bsky.social•86 days ago

Let me know the best method to send you that report, @garymarcus.bsky.social

baykenney.bsky.social•86 days ago

That’s because it was from 2022

davidmanheim.alter.org.il•85 days ago

Yes. I think it's important to differentiate between saying "this will end eventually," and saying "GPT-3 is already at the end point," which this article seems to say.

garymarcus.bsky.social•85 days ago

It really doesn’t say that. The word “possibly” raised the question.

garymarcus.bsky.social•85 days ago

The article did NOT say that. Read it carefully - it says it could be and it says that scaling likely won’t last and that hallucinations and boneheaded errors will be obstacles that are hard to overcome.

davidmanheim.alter.org.il•85 days ago

Do you still stand behind the claim that neuro-symbolic methods are needed, and LLMs can't learn symbols on their own - even after we see the newer models able to both output and interpret symbolic data?

garymarcus.bsky.social•85 days ago

Yes to the first, and what do you mean specifically by newer models? (And are they hybrids? Pure LLMs?)

davidmanheim.alter.org.il•85 days ago

Even GPT3.5 can reason about symbols and manipulate them with various tools - while newer models like 4o and Claude have tool usage that enhances their capabilities built in essentially from the (post-pretraining) ground up.

gserasset.bsky.social•86 days ago

Well LLMs are indeed very bad in citing their sources 🧐

xyzmayb.bsky.social•86 days ago

Bounced

justreading108.bsky.social•86 days ago

I was using Grok, and it gave you a shout-out (without me
asking about you). Though it didn't cite a specific article.

denloginoff.com•86 days ago

I've just heard the same sentiment from a colleague who's using AI at a medical company. While they found it to be very useful, he was saying it's become apparent in the industry there's a hard limit now

ihorgowda.bsky.social•86 days ago

Mark 6:4

'But Jesus said to them, “A prophet is not without honor except in his own country, among his own relatives, and in his own house.” '

mastercrumb.bsky.social•86 days ago

My friend in the know keeps pounding this message

mastercrumb.bsky.social•86 days ago

Is kind of a confusing thing cause like ASCII codes != Symbolic logic (which I think is what he generally means). But to zoom out I generally agree that we tend to (at least in our conscious perception) reason using pretty meaningful and explicit abstractions …

mastercrumb.bsky.social•86 days ago

"The “embeddings” that are popular in neural networks also look remarkably like symbols, though nobody seems to acknowledge this. Often, for example, any given word will be assigned a unique vector, in a one-to-one fashion that is quite analogous to the ASCII code.

garymarcus.bsky.social•86 days ago

I have actually made the point a bunch of times, back I think to 2001/Algebraic Mind.

mastercrumb.bsky.social•86 days ago

From text chain sparked: I do def agree with the general skepticism that the current architectures will ever reach anything we colloquially understand as "intelligence"

Comments

Posting Rules

Reply