This paper’s up on arxiv now so here’s an explainer thread from my perspective (I’m just one contributor and this is my own take, take it with a grain of salt) 🧵 - ThreadSky

eryk.bsky.social • 21 days ago

This paper’s up on arxiv now so here’s an explainer thread from my perspective (I’m just one contributor and this is my own take, take it with a grain of salt) 🧵

Reposted from Borhane Blili-Hamelin

arxiv.org/abs/2502.03689

Comments

oddletters.bsky.social•20 days ago

im very psyched about this paper

omarsultan.me•21 days ago

Good paper--will be curious as to the response. IMO AI is following the recent SV goal of building things that will make you rich vs building things that actually solve problems. AGI robots are sexy but better cancer screening is not.

niall-mcauley.bsky.social•20 days ago

Response will be: give me a bazillion dollars and I'll prove you wrong.

eryk.bsky.social•21 days ago

AI research keeps doing weird shit because it’s aiming at weird shit. It’s trying to replicate human thought (“AGI”) instead of trying to solve problems that matter to normal people. The paper argues that AGI is a bad goal because it’s a distraction from the things technology should be focused on.

eryk.bsky.social•21 days ago

There’s six reasons “AGI” is a bad goal. First, nobody agrees on what the hell “AGI” is. Using the term anyway creates a false sense of agreement. So AI research gets pulled in vague and often useless directions instead of developing specific, useful things.

eryk.bsky.social•21 days ago

Second, “AGI” being badly defined means that measuring success is anybody’s guess. Bad science blooms: you got a result? Call it AGI! Whatever it does, call it a benchmark and shift your definition of AGI to whatever your thing just did. But! That’s *bad science!*

eryk.bsky.social•21 days ago

If nobody knows what AGI is you can just say anything is AGI. Other underdefined terms are up for grabs too, go wild! You can call it “reasoning” or “a world model” or anything else and nobody can prove you wrong because nobody has a definition to measure those things.

caravellin.bsky.social•21 days ago

ISTM engineers come at AGI from an SF/mystical perspective of "the workings of the human soul are unknowable but materialism means a computer could replicate it, therefore we can try and replicate it and it's fine if we're never quite able to tell we've succeeded because that's how it works".

caravellin.bsky.social•20 days ago

Hence the lack of definitions, the hapless benchmarking, the breathless hype. Except it's been a long time since Alan Turing and "the workings of the human soul are unknowable" is increasingly unsupported (entirely so if one is a materialist, IMO).

eryk.bsky.social•21 days ago

Then, *engineering* questions are answered if the thing works, but *scientific* questions require evidence that it works the way you expected for the reasons you’re *saying* it works. So AGI as an engineering goal allows you say “looks like thinking” but you don’t have to prove it *is* thinking.

eryk.bsky.social•21 days ago

Third. AGI as a goal focuses on the engineering question — “can researchers do it?” — rather than “what should we be doing?” In centering AGI you aim for a technical achievement and move toward that, rather than thinking about real problems in the world that some other technology might solve.

verbalobe.bsky.social•21 days ago

I expect you touch on this, but the trap I see with this AGI focus is that implementations that SEEM to get closer to someone’s IDEA of AGI get press and investment, when a) it’s not really closer, and b) it’s not really AGI. Not just diverting from broader goals, but actively wrong for AGI.

Comments

Posting Rules

Reply