The most underreported and important story in AI right now: Pure scaling has failed to produce AGI, despite a half-trillion dollar investment. New essay at Fortune: fortune.com/2025/02/19/g... - ThreadSky

garymarcus.bsky.social • 3 days ago

The most underreported and important story in AI right now: Pure scaling has failed to produce AGI, despite a half-trillion dollar investment.

New essay at Fortune:

https://fortune.com/2025/02/19/generative-ai-scaling-agi-deep-learning/

Comments

t3rrybowman.bsky.social•3 days ago

This is good news. The longer we go without AGI, the longer we have to mitigate the possible negative effects of AGI and prepare the workforce to adapt.

matttkk.bsky.social•3 days ago

Gosh, text pattern mimicry has failed to produce abstract higher order intelligence even if you do it a lot? What a surprise.
Hoping this is the apotheosis of the big-data fallacy. So much good research passed over for moon-shot white elephant national prestige projects, so much energy wasted.

mlanalysis.bsky.social•3 days ago

I kind of disagree, pure scaling has resulted in continued improvements in performance when stagnation could be expected. Just throwing pure compute at Grok resulted in a SOTA LLM, its problems seem to stem from core flaws of the transformer architecture.

Maybe Titans and LLaDA are on the horizon!

garymarcus.bsky.social•3 days ago

Did you read the article and links?

mlanalysis.bsky.social•3 days ago

Yeah, my position is that pure scaling has worked up till now at least. Yeah hallucinations still happen, but significant progress is being made nevertheless, for instance for reasoning. I do agree that new ideas need to be implemented, TTC alone isn't enough to address fundamental problems

on-ai.bsky.social•3 days ago

And this is surprising? Isn't this what most OS-leaning researchers like LeCun have been saying for years. Pure scaling + pretrained deterministic architectures != AGI and never will. AGI will require many things, not least of which is test-time out-of-distribution generalization.

garymarcus.bsky.social•3 days ago

You are telling ME this? I have been pushing re out of distribution since 1998! And lecun used to mock me for saying this stuff, though I am glad he came around. Read some history, please:

https://open.substack.com/pub/garymarcus/p/open-letter-responding-to-yann-lecun?r=8tdk6&utm_campaign=post&utm_medium=web&showWelcomeOnShare=false

poddraws.bsky.social•3 days ago

Thank you for the article. This is very much in tune with what I would have expected to happen so far. We're going to have to do something other than just grind up large quantities of human made works. AI has yet to genuinely help me personally with anything.

vicklien.bsky.social•3 days ago

Hope this is OK Gary, but https://msn.com has your same Fortune article without the Fortune paywall (adverts instead :):
https://www.msn.com/en-us/technology/artificial-intelligence/the-most-underreported-and-important-story-in-ai-right-now-is-that-pure-scaling-has-failed-to-produce-agi/ar-AA1zo0YP?ocid=BingNewsSerp

manlius.bsky.social•3 days ago

I could not agree more, since 2023

https://open.substack.com/pub/manlius/p/navigating-the-transformative-potential?utm_source=share&utm_medium=android&r=n4iyo

tdverstynen.bsky.social•3 days ago

If only someone had shown that scaling was a computationally intractable way of producing AGI.

https://link.springer.com/article/10.1007/s42113-024-00217-5

stevehuntsman.bsky.social•3 days ago

…by learning. There may be other ways:

tdverstynen.bsky.social•3 days ago

I guess more technically by "supervised learning". But you are correct.

This was more a critique of the "get all the data in the world and make our models as big as possible and AGI will magically appear" approach that dominated Silicon Valley for a while.

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply