AI medical note taking in practice pulitzercenter.org/stories/rese... (it is worth asking “is the AI backlash justified?” before vilifying criticism as anti-innovation) - ThreadSky

abeba.bsky.social • 48 days ago

AI medical note taking in practice

https://pulitzercenter.org/stories/researchers-say-ai-powered-transcription-tool-used-hospitals-invents-things-no-one-ever

(it is worth asking “is the AI backlash justified?” before vilifying criticism as anti-innovation)

Comments

michaelakellar.bsky.social•48 days ago

Rich's job isn't to promote discussion; it's to quash it. Took about 30 seconds reading his feed to know he was a Block.

tracingvrl.bsky.social•48 days ago

OpenAI's Whisper appears to be an interesting embellishment technology: https://pulitzercenter.org/stories/researchers-say-ai-powered-transcription-tool-used-hospitals-invents-things-no-one-ever

tracingvrl.bsky.social•48 days ago

The parents of OpenAI whistleblower Suchir Balaji say forensics and autopsy show signs of blunt-force trauma, that it was murder not suicide https://bsky.app/profile/tracingvrl.bsky.social/post/3ldbhr5o6uc2y

bereza.cz•48 days ago

What is the human error rate?

martinoberg.bsky.social•48 days ago

Do you seriously think it’s even close?

bereza.cz•48 days ago

I have no idea. It's probably not, but I would like to know anyway. Humans are not flawless either.

justinbuist.bsky.social•48 days ago

From what I gather it's acceptable to have a critical error 2% of the time in human transcription.

With clear audio (like a Dr's office) that 187 / 13,000 has an error rate of 1.4% total, which may or may not be critical in nature.

So... the AI is likely better. And can be improved.

bereza.cz•48 days ago

Thanks! Would you know why a 2% is an acceptable threshold?

martinoberg.bsky.social•48 days ago

That’s certainly one way to interpret those stats.

mcwyrm.bsky.social•48 days ago

Lol.

Those "short clear" audio samples were each *10 seconds* long. That's hardly representative of any actual use case.

https://www.healthcare-brew.com/stories/2024/11/18/openai-transcription-tool-whisper-hallucinations

martinoberg.bsky.social•48 days ago

The human error rate for 10 second clips of clear audio would also be way lower than the average since that was the comparison.

mcwyrm.bsky.social•48 days ago

...and the rate at which human transcriptionists straight up fabricated dialog would be very very close to zero in any case.

mcwyrm.bsky.social•48 days ago

I do believe that's what they call "cherry-picking." Error rates of 80% and 100% were also reported, but you ignore those because they don't suit your narrative.

martinoberg.bsky.social•48 days ago

Medical records are almost never ”short clips”. He just ignored that part.

justinbuist.bsky.social•48 days ago

I ignored those due to character limits. Whisper has a known issue with transcribing silence into words so if you feed it a public meeting, which will contain silence, and don't preprocess it to remove that silence you're going to get made up stuff. It does much better with speech.

martinoberg.bsky.social•48 days ago

Anything that is not ”short clips” will contain silence in varying degrees. That also sounds like a pretty fundamental flaw that should be fixed BEFORE even thinking about using it in any sort of critical application.

It’s always about some future version that will be better.

mcwyrm.bsky.social•48 days ago

It very much sounds, then, that it does *not* perform better than humans, who have solved the "make up stuff" problem for the most part.

webrant.bsky.social•48 days ago

A reminder again that the models also do not improve as they scrape data already tainted by their hallucinations once adopted in a regressive loop of degeneration.

https://bsky.app/profile/abeba.bsky.social/post/3lf3d2fspo22a

micheil.bsky.social•48 days ago

Reality distortion field. 😵‍💫

Hallucinations cannot be removed from LLMs. It is a feature. So when you use it in medicine, legal, tax, etc. - you absolutely have to verify the output.

What, then, is the ROI? The human still needs: expertise; must review the doc; etc.

Bonkers.

justinbuist.bsky.social•48 days ago

Whisper isn't an LLM.

micheil.bsky.social•48 days ago

Thanks for the clarification.

puellaludens.bsky.social•48 days ago

Even setting aside error rates in transcription, current clinical documentation systems are heavily optimized for billing needs, rather than patient/clinician needs. I suspect AI tools likewise will be designed and implemented not to improve care or usability for providers, but to optimize payment.

altiam.bsky.social•48 days ago

Exploring AI's role in healthcare is crucial!

miketonge.bsky.social•48 days ago

This is meaningless, unless you can show that humans do not have a similar or worse failure rate.

ahundt.bsky.social•48 days ago

It is not meaningless, there are many metrics and yes end to end outcomes can be one, but also AI free systems that make calculations directly are another baseline.

Then AI has ridiculous bias and distribution shift issues where who you evaluate changes the accuracy and so on.

abeba.bsky.social•48 days ago

since the person detached the post I'm quoting, I'm including a screenshot here.

if you can't reason with empirical evidence that challenges your "AI backlash is bad" argument, the problem is you

tumulte.bsky.social•48 days ago

that baby must go down the drain indeed.

And even if it was wrong 1 out of a 100 recordings (but you don’t know when or where) it’s as good as 0% accuracy

Comments

Posting Rules

Reply