Profile avatar
matthewbjane.bsky.social
PhD student | Quantitative Psychology, UConn | Applied Statistics & Meta-Analysis | Methodological reviewer at Psychological Bulletin | Advisor: Blair T. Johnson | matthewbjane.com
322 posts 5,143 followers 321 following
Regular Contributor
Active Commenter

"Meta-meta-analyses trade precision & rigor for expediency. If your aim is rigor, there’s no substitute for doing the hard work." Preach! I once read a #meta-analysis that mistook SE for SD. Since then I am suspicious of all such studies. Do the hard work! matthewbjane.github.io/blog-posts/b...

This is very nice from @matthewbjane.bsky.social as to why umbrella reviews are basically useless (sorry!), essentially because they compound up the issues with the meta analyses they include. I have the same concerns but articulate them less eloquently. matthewbjane.github.io/blog-posts/b...

I've moaned about meta-analyses of meta-analyses for a while and had several interesting experiences reviewing them... Now I can save time and just send people to Matt's post to explain why not to do them.

Happy Friday everyone! I just posted what I think is an important blog post on my website. It is a critique of meta-meta-analyses: meta-analyses of meta-analyses. Link: matthewbjane.github.io/blog-posts/b... #stats #metascience

Happy Friday everyone! I just posted what I think is an important blog post on my website. It is a critique of meta-meta-analyses: meta-analyses of meta-analyses. Link: matthewbjane.github.io/blog-posts/b... #stats #metascience

I'm calling for a total and complete shutdown of JAMA Pediatrics until we can figure out what is going on goodscience.substack.com/p/the-resear... I mean, come ON

(#metascience) A good article by @stuartbuck.bsky.social on the Fluoride and IQ meta-analysis: goodscience.substack.com/p/the-resear.... Summarizes some of the points in our forensic review of it (see the forensic review as a pre-print here: osf.io/preprints/os...). Very nice excerpt below:

When a STALT error messes up your pagination.

Error Report #4: matthewbjane.quarto.pub/meta-analysi... The meta-analysis on Fluoride and IQ by Taylor et al (2025) is methodologically flawed and has data integrity issues. The full report by me, @jamesheathers.bsky.social, and @drg.bsky.social is on OSF: osf.io/preprints/os... #metascience

Error Report #4: matthewbjane.quarto.pub/meta-analysi... The meta-analysis on Fluoride and IQ by Taylor et al (2025) is methodologically flawed and has data integrity issues. The full report by me, @jamesheathers.bsky.social, and @drg.bsky.social is on OSF: osf.io/preprints/os... #metascience

Spent the past couple days looking at meta-analyses in body comp literature to get reasonable parameter estimates to use in simulations for a study we're planning and really trying to avoid giving myself a to-do list of meta-analyses that I need to correct/redo 🙃 cc: @matthewbjane.bsky.social

Coding RCTs for a meta-analysis…

The meta-analysis on the relationship between Fluoride and IQ by Taylor et al. (2025) that RFK Jr. has cited contains major methodological flaws and data integrity issues that we describe in the following pre-print (submitted to meta-psychology): osf.io/preprints/os...

Another great meta-science contribution by @matthewbjane.bsky.social and others. This and his re-analysis of the “social media causes harm” meta-analysis are great examples of how we can use science to serve society without necessarily having to be “activists”

The notorious MBJ and his crew are at it again! 🔥 🔥 #RStats #PyData #Stats #EconSky #DataBS

Most studies in the meta-analysis are no more detailed than the following: “Area A has low fluoride water levels, Area B has high fluoride water levels, and area A has a higher mean IQ” No matching or balancing of covariates. Distinct geographic areas differ more than just their fluoride levels.

They chose to use the standardized mean difference in IQ between a reference and exposure group however the reference in many studies has more fluoride exposure than the exposure group in other studies! There is no consistent definition for the groupings.

One major finding in our pre-print is that the journal ‘Fluoride’ publishes much larger effects than any other academic journal. Fluoride is published by the International Society for Fluoride Research which is run by anti-fluoride activists and hosts water defluoridation workshops. #metascience

I saw @epiellie.bsky.social comment on this so I am tagging her here #metascience

The meta-analysis on the relationship between Fluoride and IQ by Taylor et al. (2025) that RFK Jr. has cited contains major methodological flaws and data integrity issues that we describe in the following pre-print (submitted to meta-psychology): osf.io/preprints/os...

In probability the following is true by definition Pr(sample space) = 1 Pr(empty set) = 0

👀

We just published a paper on the performance of various tests and confidence intervals for the scale coefficients in location-scale models in meta-analysis: doi.org/10.3758/s134... Overall, REML-based LRTs and profile-likelihood CIs performed best, but the number of studies typically must be large.

Error Report #2 (#metascience): t.co/9xyqIVUele The meta-analysis by Stanyer et al. (2022) contains severe data extraction errors. Due to these errors, the meta-analytic results for memory retention scores are incorrect. Reported effects for WPAL outcomes are 5x larger than the corrected effects.