Every week we get more “demos” that, if next-token prediction is insufficient to make powerful autonomous AGIs that can accomplish long-term goals via out-of-the-box strategies, then people will keep searching for other approaches that CAN do that. 10/16 - ThreadSky

stevebyrnes.bsky.social • 73 days ago

Every week we get more “demos” that, if next-token prediction is insufficient to make powerful autonomous AGIs that can accomplish long-term goals via out-of-the-box strategies, then people will keep searching for other approaches that CAN do that. 10/16

Comments

stevebyrnes.bsky.social•73 days ago

Here is a “demo” that it’s possible for companies to ignore or suppress obvious future problems when they would interfere with immediate profits. 11/16

stevebyrnes.bsky.social•73 days ago

Here is a “demo” that it’s possible for there to be a global catastrophe causing millions of deaths and trillions of dollars of damage, and then immediately afterwards everyone goes back to not even taking trivial measures to prevent similar or worse catastrophes from recurring. 12/16

stevebyrnes.bsky.social•73 days ago

Here is a “demo” that the arrival of a new kind of highly competent agent with the capacity to invent technology, coordinate at scale, self-reproduce, etc., is a big friggin’ deal. https://en.wikipedia.org/wiki/Human_impact_on_the_environment 13/16

stevebyrnes.bsky.social•73 days ago

And here is a “demo” that it’s possible for a small number of such highly competent agents to maneuver their way into dictatorial control over a much much larger population of humans. https://www.lesswrong.com/posts/ivpKSjM4D6FbqF4pZ/cortes-pizarro-and-afonso-as-precedents-for-takeover 14/16

stevebyrnes.bsky.social•73 days ago

Here is a “demo” that it’s possible for a future technology to feel impossibly, laughably far away from being invented, when it’s actually mere months away. https://en.wikipedia.org/w/index.php?title=Flying_Machines_Which_Do_Not_Fly&oldid=1263288714 15/16

stevebyrnes.bsky.social•73 days ago

I could go on. None of these is PROOF that AGI will cause human extinction, obviously! And I could make a long list of optimistic “demos” too. But I do think there are some people simply forgetting about possible problems, and maybe one or two of those “demos” would be helpful brainstorming. 16/16

Comments

Posting Rules

Reply