I’m an AI optimist but think the concept of AGI (as used in that sort of talk) is incoherent and oversold. The slide from chatbots to fully autonomous agents involves assuming a solution to social and legal problems that I just don’t see getting resolved easily. bsky.app/profile/tedu... - ThreadSky

tedunderwood.me • 173 days ago

I’m an AI optimist but think the concept of AGI (as used in that sort of talk) is incoherent and oversold. The slide from chatbots to fully autonomous agents involves assuming a solution to social and legal problems that I just don’t see getting resolved easily. https://bsky.app/profile/tedunderwood.me/post/3letzpsgchc22

Reposted from Ted Underwood

This is also, btw, one reason why I think extreme panic / triumphalism about AI is misplaced.

IRL agency matters; legal accountability matters; reciprocity matters. Being smart does not automatically confer any of those properties on a piece of software that can be retuned or deleted at will.

Comments

conjugateprior.org•173 days ago

Perhaps of interest, should you not have run into it already, also for the comparison with other, non-AI related concepts of legal person.

kyrick.art•173 days ago

Not to mention technical hurdles! Setting aside the ML models for a moment, chatbot to full agent is also a non linear increase in engineering complexity

tedunderwood.me•173 days ago

I think it’s a real failure of imagination. The success of Gen AI ought to have revealed that there are a lot of forms language and reasoning can take other than “autonomous robot subjectivities.” But nope: people are still determined to pour technology into that mold.

keytryer.bsky.social•173 days ago

I don't think it matters much that there's a developed legal framework for virtual persons if you get to have virtual persons, and tech companies probably won't be too concerned about the former.

keytryer.bsky.social•173 days ago

I don't know if superhuman intelligences are possible at all, but it seems pretty likely to me that we'll have convincing virtual people circa ~5 years from now (Whether this is "real" or not is another matter), and the conundrum about their status wouldn't change that.

tedunderwood.me•173 days ago

I agree with you, but I feel like Joi in Blade Runner 2049 is a fairly plausible picture of the social consequences of virtual personalities. It's not nothing, but it's not an existential challenge leaving people with little to do.

keytryer.bsky.social•173 days ago

Virtual personalities are more impactful than that imo, in the sense that they could replace humans and do any task, be duplicated any number of times and collaborate in any way in real time. It's more impactful imo than any arbitrarily smarter text predictor anyone could develop.

keytryer.bsky.social•173 days ago

https://bsky.app/profile/keytryer.bsky.social/post/3levezkx5es2d

adverb.bsky.social•173 days ago

Particularly looking at the near & midterm: it would seem like a hot-take if ML-twitter were the only vantage, but I actually get the sense that most researchers in the field think that "AI agents" (at least as they exist now) are not particularly useful or interesting.

adverb.bsky.social•173 days ago

And I don't think it's *just* because models aren't capable enough.

Many of the use cases involve:
- Making the models better at a task - with agents this comes out as just scaling inference naively in practice
- Personalizing a task like "schedule me a trip" - which has nothing to do with agents.

tedunderwood.me•173 days ago

Right. Personalizing is tough.

tedunderwood.me•173 days ago

I think agents of a sort will happen eventually. By "of a sort" I mean, agents in the sense that they're always-on and can initiate e.g. reminding the user about emails needing a response.

But as you say, even that's hard and not easy to pull off well. And real autonomy is another thing entirely.

adverb.bsky.social•173 days ago

haha yeah, getting a system that doesn't bug the hell out of users with reminders would be the big one 😄

henrythe5thelement.bsky.social•16 days ago

It’s an impoverished imagination: AI makers are following a simple formula— automation to replace wage labor. That’s why AI is getting built, funded and disseminated. It’s not mysterious or complex. Independent researchers with expertise should start from that place and never lose sight of it

tedunderwood.me•16 days ago

Replacing labor is also the only reason we built water wheels, so this should not be shocking or surprising. We’ve been on that kick for a while.

henrythe5thelement.bsky.social•16 days ago

Yes, but the entire educational system wasn't leveraged to ensure that students come out qualified to operate water wheels. And besides, it still needs to be the first premise of critically engaged thought and discussion. I see you making some bad logical and rhetorical analogies around AI.

henrythe5thelement.bsky.social•16 days ago

I don't mean you have to oppose it and denigrate it at every turn, and I know you are providing helpful critique, but you're an authority so we need your help pushing back against the totalizing of these systems and their bad-faith (or just bad) operators.

tedunderwood.me•16 days ago

Scott, we're just on opposite sides of this issue. I'm not a person on your side of this issue who has become temporarily confused and could be rescued by helpful guidance.

Comments

Posting Rules

Reply