One of the things about "superintelligence" paranoia is that in the real world, allowing that the span of human intelligence is more narrow in practice than we imagine it to be (it is simultaneously as wide as we can imagine and as narrow as a bunch of organisms with similar brains would be)... - ThreadSky

mall.bsky.social • 48 days ago

One of the things about "superintelligence" paranoia is that in the real world, allowing that the span of human intelligence is more narrow in practice than we imagine it to be (it is simultaneously as wide as we can imagine and as narrow as a bunch of organisms with similar brains would be)...

Comments

mall.bsky.social•48 days ago

... supposedly extremely intelligent individuals do not go around necessarily exploiting that high intellect to turn the bottom X% of the population by IQ into slaves.

mall.bsky.social•48 days ago

People with exceptional intellectual capacities are often marked as disabled or defective for it in education (because they operate outside of the parameters of social norms for intellectual capacity and this is stigmatised). They don't automatically turn evil and turning evil is often punished.

quantumjared.bsky.social•48 days ago

This is all very true, but I feel this still imagines AGI as something similar to humans, that decides its own goals. But it could also be an agent with advanced reasoning but a fixed goal (like making paperclips): the real fear then is that such an agent turns the Earth into paperclips

quantumjared.bsky.social•48 days ago

And once we turn it on, if it’s more intelligent than a human, we can’t turn it off or change its goals. Both of those would get in the way of making paperclips, and it would immediately take action to prevent that (e.g. distribute itself) and then acquire more computing power

mall.bsky.social•48 days ago

Capitalism is already doing that.

But our general response to such systems is really what I was talking about.

mall.bsky.social•48 days ago

I might be optimistic but I think in the unlikely event a super intelligent machine appeared there's a good chance we'd shoot it or blow it up before it did us much harm because punishing smart asses is a thing we already kind of do.

brandonjc.bsky.social•48 days ago

Trying to predict the choices of a super intelligent machine seems akin to an intelligent bird trying to predict human choices. It might be able to in very specific immediate circumstances, but we often act according to structures and abstracts it has no awareness of.

mall.bsky.social•48 days ago

For sure. But that goes as well for the paranoia about super intelligent AI gods as much as anything else. And either way we regularly get eaten by wild animals outside of environments where we specifically engineered our own safety

brandonjc.bsky.social•48 days ago

True. In general I’d expect higher order decisions to seem alien, rather than obviously malicious.

I think we could situate a machine such that it could never be a threat to us. But IDK about the ethics of constraining such a mind to the degree required for complete safety.

lookarose.bsky.social•48 days ago

I think hyper advanced machine intelligence that displayed a greater level of empathy would terrify capitalists far more than one with ruthless efficiency

Comments

Posting Rules

Reply