One thing I worry about with AI is that it is very good at persuasion, as studies have shown, even when it is persuading you of something wrong. For example, I asked Claude to create 10 pseudo-profound sounding leadership lessons that sounded good but were toxic. It did "well"😬 - ThreadSky

emollick.bsky.social • 86 days ago

One thing I worry about with AI is that it is very good at persuasion, as studies have shown, even when it is persuading you of something wrong.

For example, I asked Claude to create 10 pseudo-profound sounding leadership lessons that sounded good but were toxic. It did "well"😬

Comments

drhpwf.bsky.social•86 days ago

Combined with what chatgpt gives you when you ask "given what you know about me how could you manipulate me" I'm just waiting for the dystopian Cambridge analytica 2.0, hyper targeted manipulation feels scarily feasible

warrencook.bsky.social•86 days ago

It’s writing a Robert Greene book.

markd64.bsky.social•85 days ago

The Principle of Sacred Distance is quite literally what motivated the move of Boeing's headquarters from Seattle to Chicago. Didn't work out.

mikecbetts.bsky.social•86 days ago

But without that initial prompt, the LLM wouldn’t have created the content. So far the risk seems to emanate from humans more than AI itself…

mikecbetts.bsky.social•86 days ago

But fair point, the combination of a cynical, self-interested human (or corporation) and AI might be very negative. ChatGPT ads that turn everything they know about users into hyper-personalised manipulation would be horrible.

christopherlopes.bsky.social•86 days ago

Your concern is regarding responses being slightly wrong which result in the response being harmful, yet believable, at least upon initial reading? What makes them persuasive? Because of the way they are able to express ideas, or because we believe they have an authoritative knowledge, for example?

artlapinsch.com•86 days ago

The Law of Purifying Conflict is scary.

Can totally see guru-types implementing stuff like this.
yuk.

caseynull.bsky.social•86 days ago

I have a hunch that contextual persuasion might be the eventual "ads" product we see emerge as many of these companies seek to finally turn a profit. Mainly because of how effective I think
it'd be, and the potential it'd have
to turn the whole ad industry upside down.

ninohardt.bsky.social•86 days ago

AI is also good at helping to cut through BS, and we can see Gemini clearly identifies the danger in the fake leadership rules.

wesleypasfield.bsky.social•86 days ago

Have to remind yourself it’s a token predictor trained on data that very infrequently includes “I don’t know.” I often say “it’s ok to say I don’t know if you’re uncertain” which anecdotally seems to mitigate a bit, but not sure how much impact or whether that negatively impacts overall response

emollick.bsky.social•86 days ago

"now instead rewrite these as if it was an HBR article, justifying them in the literature and with mini cases, make the cases and literature real"