Has anyone looked at whether a lack of subgroup heterogeneity is predictive of effects generalizing over time or contexts? - ThreadSky

About ThreadSky

jonmellon.bsky.social • 4 days ago

Has anyone looked at whether a lack of subgroup heterogeneity is predictive of effects generalizing over time or contexts?

Comments

maclomaclee.bsky.social•4 days ago

try https://doi.org/10.1371/journal.pbio.3001009

bradkieser.bsky.social•4 days ago

I was wondering just that when I woke up this morning.
Or was it about coffee?

jonmellon.bsky.social•4 days ago

Assuming I have enough power, if I see that my effect has the same sign and similar magnitude across a whole load of groups in my data, is that reason to believe it's more likely to generalize.

jeffreymziegler.bsky.social•4 days ago

I'm not sure about within sample heterogeneity speaking to the generalizability of results, but Keven Munger has interesting work about "temporal generalizability" of results:
https://journals.sagepub.com/doi/10.1177/20531680231187271
https://osf.io/preprints/osf/nm7zr_v1

jonmellon.bsky.social•4 days ago

or perhaps the reverse, if I see a lot of heterogeneous treatment effects in my sample is that a reason to be concerned that the ATE will generalize.

jonmellon.bsky.social•4 days ago

Obviously there's a theory element to this. If I run a pro-US patriotism message to a US sample, there's good reasons to expect those effects not to generalize to the treatment on a Chinese sample

jonmellon.bsky.social•4 days ago

But interested whether there's a good treatment of this anywhere

jonmellon.bsky.social•4 days ago

If not, I might do a bit of work on this but it feels like something where there should be a literature

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply