The AI transcription tool Whisper hallucinates in a large fraction of its transcripts.
It's been used for 7 million medical transcripts. People want to use similar text to transcribe bodycam audio for police reports.
Both are among the WORST applications of the tech.
https://apnews.com/article/ai-artificial-intelligence-health-business-90020cdf5fa16c79ca2e5b6c4c9bbb14
It's been used for 7 million medical transcripts. People want to use similar text to transcribe bodycam audio for police reports.
Both are among the WORST applications of the tech.
https://apnews.com/article/ai-artificial-intelligence-health-business-90020cdf5fa16c79ca2e5b6c4c9bbb14
Comments
All it can do is generate statistically probable patterns of text, with no understanding of what they mean.
"AI promoters" are scammers and grifters.
The implementation described in the article seems responsible for deleting the audio, not the library.
Pay people wage double plus ungood.
Dead patients... meh.
1) The people selling the magic beans that (they claim) will get rid of that pesky 'paying wages' problem
2) The people buying the magic beans
3) The people investing in #1, who don't give a fuck what happens to #2 when the beans don't work
A way for Microsoft et al. to crowdsource free labor to shape up their half-baked product
Good example of how evolution only has to be good enough to pass on information into the next generation.
My only thought is the push for AI here is to link to other applications, eg diagnosis AI?
Imagine hallucinations for diagnoses.
And _that_ might be why we're seeing it get deployed. It's cheap.
I might actually look into shoving this into a toy app of mine.
Transcription on Windows was something we played around with on like Windows ME. What the fuck are we doing?
The Whisper model is free to download and tiny enough to run on a tablet or a cellphone.
There's no API calls to OpenAI here. It's free. Runs local.
financial and environmental costs of using inaccurate software programs….
https://dl.acm.org/doi/pdf/10.1145/3630106.3658996
https://www.science.org/content/article/ai-transcription-tools-hallucinate-too
(Note that standard auto transcription tools in medical environments like Dragon or even MS Word tool also make mistakes, but do not hallucinate)
But also, yeah, just writing the notes works better.
I would be very interested to see the research that says we have found a way to consistently identify and flag or correct hallucinations.
the models are always hallucinating, whether they are doing so constructively is the problem.
how do you figure that out with audio?
[0]: at which point why not just use more reliable transcription models that aren’t generative in the sense that LLMs are…?
“Let's use a tool known for hallucinating as the backbone of our customer support system”
A bit later: “What do you mean, 'the tool known for hallucinating has hallucinated a non-existent refund policy'?”
2) sending raw audio PHI to software explicitly precluded from accepting that data
either of these can and should bankrupt a hospital. doing both at once... that takes talent
That, well, is bad.