People quoting these AI summaries for fact checks drive me wild. I sussed out a core factual conflation it made between two areas of a technical standard last week. It's good I didn't trust it. Glad I grew up with Wikipedia being drilled as untrustworthy. Wikipedia has issues even when sourced! - ThreadSky

swiftonsecurity.com • 58 days ago

People quoting these AI summaries for fact checks drive me wild. I sussed out a core factual conflation it made between two areas of a technical standard last week. It's good I didn't trust it. Glad I grew up with Wikipedia being drilled as untrustworthy. Wikipedia has issues even when sourced!

Reposted from Jason Schreier

I was excited to tell my kids that there's a sequel to Encanto, only to scroll down and learn that Google's AI just completely made this up

Comments

oggie.bsky.social•58 days ago

Wikipedia tends to have a number of people at least trying to present information close to honestly, and thus in my book ranks higher than a number of sources.

As long as you stay clear of politics, race, gender, sexual orientation, tech origins ...

AI is just like an API with half the docs gone.

engineering-nerd.bsky.social•58 days ago

But it wasn’t made up. There’s an actual wiki linked where the model pulled that information from: https://ideas.fandom.com. The scarier implication here is that all these gen AI models are open to easy manipulation by malicious misinformation ON TOP of their already established propensity to hallucination

swiftonsecurity.com•58 days ago

That's a fair point but they hide the URL and they're speaking in the voice of Google

dragostech.bsky.social•57 days ago

And in the end, this looks like a little too detailed for a "hallucination" - it looks like a pretty real project.

That url wasn't, uh, very obscured. This tea cup isn't big enough for this tempest.

dragostech.bsky.social•57 days ago

Wouldn't it be ironic if this collective hallucination of the internet, becomes a reality, due to to spotlight from the gullible Google LLM?

(And to be fair to Google, with all the noise on "Encanto 2" it's tough enough for humans looking at searches to figure out, never mind LLM training sets.)

nicolas.grumpycoder.net•58 days ago

To be fair, this would've been an issue even without ai summaries. It's just that the summary makes it look more official.

the-ballmer-peak.bsky.social•57 days ago

I enjoy https://phind.com, which is an AI that cites its sources. I haven’t seen another like that yet, not that I’ve been looking.

tehlaser.bsky.social•58 days ago

Oh fuck. I just realized “don’t use AI” is gonna be the new “don’t use Wikipedia.” In ten years people are going to think we were idiots.

notapreppie.bsky.social•58 days ago

It's already a common problem in r/Chemistry, r/Labrats, and r/Cartalk on Reddit.

People have come in with questions saying, "ChatGPT said to do it this way but it didn't work and I ended up in coughing and my eyes burn. What went wrong?"

Well, for starters, you used ChatGPT.

downtym.bsky.social•58 days ago

It's hard to blame the AI. It just really wants to dive deeper into the Encanto lore with a sequel.

earlydeparture.bsky.social•57 days ago

This came up in a Scrabble argument. Two people around the table googled the word and just instantly said "yup it's a word"

futureretroism.bsky.social•58 days ago

sydney58.bsky.social•57 days ago

Research SEEMS easy now but it's too easy to get it wrong. Need to fact check everything, fact check the fact checkers, check for original, peer reviewed, scrupulously researched publications, but a teensy tiny percent of us even understand how to tell flawless research from sloppy crap!

notapreppie.bsky.social•58 days ago

My boss and I were looking for alternatives to our usual sulfur-bearing material (dimethyldisilfide) for our pilot plant. I had a bunch dibutylsulfide and he used Google to find the wt% S in it while I just calculated it.

Google's AI showed all the correct arithmetic steps but did the math wrong.

Comments

Posting Rules

Reply