« Systematic reviews’ that aim to extract broad conclusions from many studies are in peril: Fake papers are “poisoning the well” for these gold-standard syntheses, researchers say » @science.org - ThreadSky

floriannaudet.bsky.social • 75 days ago

« Systematic reviews’ that aim to extract broad conclusions from many studies are in peril:
Fake papers are “poisoning the well” for these gold-standard syntheses, researchers say » @science.org

Comments

sofiev.bsky.social•75 days ago

Who are the people or organizations who pay for making those fraudulent articles? The ones who promote the investigated ‘treatment ‘?

alicecann.bsky.social•75 days ago

I know this isn’t ideal re the extra work it creates, but shouldn’t these papers be weeded out (& subsequently reported) thru critical appraisal. The work to ensure that fake papers are excluded will be part of the value of the systematic review. It’s not quick, but it’s not a quick research method.

bragrman.bsky.social•75 days ago

The bigger problem is that it exposes the inherent flaws in the publishing sector that isn't doing its job properly regarding reviewing, editing and quality control.

It's almost as if having a primarily for-profit sector as an essential part of the public research process isn't viable.

alicecann.bsky.social•75 days ago

Yeah - that makes sense. I suppose that for me this is baked in already - not a surprise.

bragrman.bsky.social•75 days ago

Yeah, but before LLM bullshit generation flooding the sector, the failures of the system were likely to let a poor quality, or even disingenuous data analysis slip through.

Now it's failures are allowing fabricated nonsense to be published.

alicecann.bsky.social•75 days ago

I’m thinking a bit of all the published systematic reviews that aren’t systematic reviews out there that won’t do this, but they’re likely to have other weaknesses too.

alicecann.bsky.social•75 days ago

But am going to try and include the fake papers issue in my systematic review teaching next term. As a way to encourage people to do proper critical appraisal (not a section that I do more than highlight issues and sources, since I don’t have the experience yet in this part of the review!)

ginettelafit.bsky.social•75 days ago

What do you think about the use of NLP to extract data from papers in systematic reviews. Could this also jeopardised assessments into what is fake or not? I see more and more interest in using NLP for coding, but I’m worried about risk

drmattg.bsky.social•74 days ago

This might help you (focused on medicine but widely applicable)- https://www.nice.org.uk/about/what-we-do/our-research-work/use-of-ai-in-evidence-generation--nice-position-statement

spottingthespot.bsky.social•75 days ago

The problem is that publishers seldomly retract! So fake stuff stays.

egonw.mastodon.social.ap.brid.gy•75 days ago

@floriannaudet.bsky.social so, that will give an interesting new source of information, articles that get deliberately left out from systematic reviews. they will be overrepresented with fake papers. Will this help us crowdsource the quality of scientific literature?

Comments

Posting Rules

Reply