When news reports about scientific papers make causal claims that are not justified by the original study, the mistake often traces to a university press release. Here, observational data reveals an association. The press release asserts causality. today.ucsd.edu/story/walkab... - ThreadSky

carlbergstrom.com • 174 days ago

When news reports about scientific papers make causal claims that are not justified by the original study, the mistake often traces to a university press release.

Here, observational data reveals an association. The press release asserts causality.

https://today.ucsd.edu/story/walkable-neighborhoods-help-adults-socialize-increase-community

Comments

jessthegray.bsky.social•174 days ago

How do most science reporters enter the field?

It seems like once you get away from science focused publications many haven't really been trained on how to read formal research publication. I'll give it is very much a skill and one nearly no undergrads learn even in STEM fields.

evornithology.bsky.social•174 days ago

Yeesh

phillyphile215.bsky.social•173 days ago

📌

kevinleecaster.bsky.social•174 days ago

I don't know if a university could run without an administration, but I'm not convinced they're running well with them.

onkelbob.bsky.social•174 days ago

Needs more deans!

davidralin.bsky.social•174 days ago

“Six-cent test for cancer” was my favorite Harvard press release theme.

carlbergstrom.com•174 days ago

And there’s clearly not even an association with community.

This is the highlights section right at the top of the paper.It’s not like they had to actually read it.

(The paper explains that once you adjust for self selection, you lose any association between sense of community and walkability.)

cynlynn3.bsky.social•174 days ago

Seems like the interpretation of a scientific study is being handed to the marketing department for the best spin “news” reporting. 🤦‍♀️

carlbergstrom.com•174 days ago

Researchers: collaborate with your university press office on these things. Rewrite them repeatedly if you need to. Most press offices will be tolerant if not outright grateful.

My awesome press office is outright grateful—or at least pretends so effectively that they’ve got me fooled.

clio.bsky.social•174 days ago

As a volunteer Master Gardener, I cannot tell you how many times we struggle with such issues, specifically with panic set off by pest spread or other information that is distorted and we have to walk people off the ledge of panic. Usually some sensational adjective is added.

bill-n1vux.bsky.social•174 days ago

👍

carlbergstrom.com•174 days ago

We wrote about this at some length in our paper "Misinformation in and about science".

https://www.pnas.org/doi/10.1073/pnas.1912444117

loverunandpray.bsky.social•174 days ago

“How can thoughtful analysis of minimum wage rates and unemployment trends possibly compete with yet another […] nine cats who look like characters from The Office?” I felt delight at even just the idea of an article showing a cat that looked like Dwight lmao. Let out a bit of a chuckle.

carlbergstrom.com•174 days ago

😹

socialistpsych.bsky.social•174 days ago

You assume some researchers aren't on board with the causal claims in press releases where they won't get push back from editors or peer reviewers, and where they stand to boost their profiles with higher ups who are more interested in university branding than scientific progress.

carlbergstrom.com•174 days ago

I’m not assuming anything. I’m giving the benefit of the doubt.

socialistpsych.bsky.social•174 days ago

That's a good idea. My point is that we can extend that to the communications staff also. Not coming at you dude.

steamtraen.eu•174 days ago

You might both be interested in this (from 10 years ago!), plus associated links:

https://www.theguardian.com/science/blog/2014/dec/10/science-health-news-hype-press-releases-universities

nih-llamas.bsky.social•174 days ago

This probably varies by disease, but is there still significant evidence that communicable diseases don't have as much variation as one might expect vs. population density because people do whatever traveling is needed to satisfy needed & desired amounts of connection?

nih-llamas.bsky.social•174 days ago

If so, seems like a parallel.

outeast.bsky.social•174 days ago

I'm confused by your objection. Doesn't that just say that people who don't care about feeling part of a community (ie don't factor it in as a reason for where they live), don't feel part of a community? But if you want to feel part of a community, a walkable neighbourhood is more likely to deliver?

outeast.bsky.social•174 days ago

(I read the whole paper, I'm just not very bright and so a lot of the actual analysis is a bit blurry. That seemed to be what this "correction" is saying but I'm entirely prepared to accept I may be utterly wrong.)

rsek.bsky.social•174 days ago

Many lessons to take away from this. One of them is for PIs to INSIST on right to vet and OK or VETO any release about their work.

aliettejonkers.nl•174 days ago

A problem with a history:

The association between exaggeration in health related science news and academic press releases: retrospective observational study
BMJ 2014; 349 doi: https://doi.org/10.1136/bmj.g7015 (Published 10 December 2014)

https://www.bmj.com/content/349/bmj.g7015

calocitta.bsky.social•174 days ago

A researcher can always blame the university press release for inaccuracies and exaggerations, but university press offices usually provide the researcher an opportunity for corrections and clarifications before the release is distributed.

danny-rose.bsky.social•174 days ago

100%

oacarah.bsky.social•174 days ago

In this one, the elements of the causal claims can also be traced to the original article (even in the abstract).

twitskeptic.sirvin.com•174 days ago

It seems odd that they omitted the middle four income deciles, if I'm interpreting this correctly.

carlbergstrom.com•174 days ago

Dude, do you know how hard it is to get statistical significance on these things?

dbartram.bsky.social•174 days ago

But why publish a paper offering an "association" in the first place? What good is a finding like that, what does it tell us if we don't have reason to believe it's causal?

carlbergstrom.com•174 days ago

I suppose the hope is that it is causal. When you have a strong enough mechanistic understanding, you don't need direct causal evidence to be pretty sure of what is going on if there's an association.

If you DON'T find an association, that would be suggestive in the other direction, after all.

barrygoldman1.bsky.social•174 days ago

y r uni press releases SO bad??? they often don't even link to the original papers.

WHY?

grouchytommy.bsky.social•174 days ago

Never gets old https://www.phdcomics.com/comics/archive/phd051809s.gif

jovanevery.bsky.social•174 days ago

I wonder if comms people (who write press releases) think a causal claim is what is going to attract attention? And an association is too boring?

carlbergstrom.com•174 days ago

Well, in our book, we put it like this:

“Correlation doesn’t imply causation, but it doesn’t sell newspapers either.”

carlbergstrom.com•174 days ago

I suppose that phrasing is a bit archaic. Maybe

“…but it doesn’t incentivize monetizeable user behaviors toward microtargeted informational opportunities.“

gerardharbison.bsky.social•174 days ago

I'm sure you've had to do this, but dealing with University PR departments is a struggle. They want to talk up the research, but the researcher wants to make sure they say things that aren't true, and since they don't understand it anyway, it's often a dumpster fire.

carriewhet.bsky.social•174 days ago

Such an important thread. I often rant to media organizations who misrepresent research because inevitably I will hear the same “facts” confidently repeated by people within that industry or discipline.

drmaximvoronov.bsky.social•174 days ago

True. But we also have to be careful and remember that the vast majority of world's problems cannot be studied via RCTs, so observational methods are as good as we can do. And yes, we do need to be careful how we report our findings regardless of the methods used.

carlbergstrom.com•174 days ago

I spent 2020-2022 explaining that we don’t need RCTs for everything.

That’s no excuse for misrepresenting findings.

drmaximvoronov.bsky.social•174 days ago

And while we are at it, a much needed cautionary tale about excessive trust in inferential statustics:

https://www.nature.com/articles/d41586-024-04096-5

katalogeur.bsky.social•174 days ago

It's one 'scientific way' to suggest factors of a phenomenon everyone experiences, but which science can't 'explain'. (In cases where advanced calculus doesn't suffice.)

joannekenen.bsky.social•174 days ago

Ditto for overhyped “breakthroughs”. It’s often from the institutution’s press office

tamarhaspel.bsky.social•174 days ago

The paper itself implies causality.

sanjaysrivastava.com•174 days ago

Aaand empirical evidence backs that up https://pmc.ncbi.nlm.nih.gov/articles/PMC6521363/

carlbergstrom.com•174 days ago

Yep. We teach that paper in Calling Bullshit.

carlbergstrom.com•174 days ago

Which, as I was telling my students a couple of weeks ago, was inspired in part by your seminar “everything is fucked”.

pennypriest.bsky.social•174 days ago

I too like to call out bullshit. There's much bullshit calling here: https://midpsy.uk/publications and more in my novel, Team Of One, as sometimes stories about the evidence are more powerful than the evidence itself. Available here: https://www.thegreatbritishbookshop.co.uk/products/team-of-one

gombeisos.bsky.social•174 days ago

Great name. I experimented with explicitly teaching scientific skepticism this semester as a part of info/data literacy. Some students just could not get over how distorted reporting on even 'trustworthy' sites could be.
@theskepticsguide.bsky.social

sanjaysrivastava.com•174 days ago

I like the back story (which you may also know). Chris Chambers was yelling at journalists on twitter about exaggerating science stories. They said a lot of times it originates with the scientists. His collaborators and him looked into it and ended up with a series of papers documenting the problem

carlbergstrom.com•174 days ago

I didn’t know that backstory. That’s fascinating. It’s a nice series of papers and one that Jevin and I cite often.

Comments

Posting Rules

Reply