If I had a coworker who gave me incorrect answers to questions more than half the time I talked to him, I would simply stop talking to him. That guy would get fired. - ThreadSky

About ThreadSky

cooperlund.online • 48 days ago

If I had a coworker who gave me incorrect answers to questions more than half the time I talked to him, I would simply stop talking to him. That guy would get fired.

Reposted from Noah Shachtman

OpenAI's "most powerful system" makes shit up more than half of the time. For its "o4-mini" model, the rate is ***79 percent***

Comments

pfohl.bsky.social•48 days ago

also read the entire corpus of the English language and still batted <50%

ulrichw.bsky.social•48 days ago

IMO this is not the right way to think about it. LLMs aren't humans and won't replace humans. LLMs are tools that are great (in certain cases) at making humans more productive. For example, I use them all the time to learn stuff in a technical area that I already have some expertise in...

ulrichw.bsky.social•48 days ago

I can usually spot "hallucinations" easily, and the "big picture" view that the LLM can quickly give me of a topic beats using Google to search for three or four peripherally relevant pieces of content to stitch together the same view myself.

ulrichw.bsky.social•48 days ago

The hype has led people to positioning these tools as more than they are, but that doesn't mean that they're without value.

cooperlund.online•48 days ago

I guess in a way thais kind of helps us see what's going on with AI-as-coworker. It's the hire that management sees huge potential in but who sucks to work with. AI is the C-Suite's nepo baby.

felledman.bsky.social•48 days ago

I have more than once now had the following interaction with colleagues

them: does our product do this
me: now
them: but ChatGPT said it did??

maximumeffort433.bsky.social•48 days ago

Am I wrong to be a little pleased that consumer facing LLMs are having these problems?

A year ago I was terrified that every writer and artist on Earth was going to be out of a job, today I'm *marginally* less terrified.

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply