I've run out of my first DeepResearch monthly limit, and I'm not really impressed by it. It seems good at compiling easy-to-find information that I don't have time to compile myself, although I'd have to fact-check them anyway. - ThreadSky

scientificdiscovery.dev • 45 days ago

I've run out of my first DeepResearch monthly limit, and I'm not really impressed by it.

It seems good at compiling easy-to-find information that I don't have time to compile myself, although I'd have to fact-check them anyway.

Comments

bread314.bsky.social•44 days ago

That's the entire point. It's not there to deliver truth but to find relevance. There are 5000+ new medical articles published daily. Here's the 6 discussing your subject. It's still up to you to determine BS vs gold standard and that's easier on 6 vs 5000.

essaywells.bsky.social•45 days ago

It's an LLM with a browser; if they sold it to you as being capable of some kind of thought or judgement, they lied to you.

scientificdiscovery.dev•45 days ago

I'm starting to think you didn't read the thread!

bendulon.bsky.social•45 days ago

LLMs - they trivialize the trivial :)

scientificdiscovery.dev•45 days ago

Beyond that, if I ask it a difficult question that would take me >10 mins to find myself, it frequently makes silly mistakes and cites things that have nothing to do with the claim. Plus, it takes ages to run.

scientificdiscovery.dev•45 days ago

It told me that an HIV drug was developed by a totally unrelated but famous pharmacologist, who has a set of principles named after him that were mentioned along with the drug.

scientificdiscovery.dev•45 days ago

It seems worse than 4o in some ways because it focuses on giving answers from specific (often wrong) citations, instead of synthesising an answer from many sources. Maybe the next version will be better.

scientificdiscovery.dev•45 days ago

My main use for 4o, which I think it's great for, is translating jargon into plain language; asking it questions to check my understanding (and then sometimes spotting errors in how it has explained it to me); also, for literally learning a language and asking it to write test questions for me.

nickup.bsky.social•45 days ago

It's also great for reverse search when I remember a vague scramble of facts about something and it can help me find the source

nilwhu.bsky.social•45 days ago

So this what I mainly use 4o for. Translating into easy to ready/understand format for me and also simplifying what I write. E.g rather than a paragraph into 1 or 2 short sentences. I would still then modify it but has made me think a bit sharper in this respect.

scientificdiscovery.dev•45 days ago

Maybe worth adding that I try to follow up each name, date, number, etc. that's mentioned, so it really annoys me when things are made up or untraceable!

This isn't just true of AI either. Many papers make citation-free claims about scientific events in history, for example.

scientificdiscovery.dev•45 days ago

Other times people include a citation but then you trace it back and it's something like this, below. Happens way too often.

Then I spend half an hour or more down a rabbit hole trying to find where the claim came from or if anyone at all has made a thoughtful estimate.

Comments

Posting Rules

Reply