If you want to do ambulatory assessment (e.g., EMA) in applied settings, say with patients, how many observations do you need to get a reliable estimate of a person's mean, variability, or correlation? We (@jzpsych.bsky.social & Florian Scharf) ran some simulations to get the answer...a🧵 - ThreadSky

aidangcw.bsky.social • 132 days ago

If you want to do ambulatory assessment (e.g., EMA) in applied settings, say with patients, how many observations do you need to get a reliable estimate of a person's mean, variability, or correlation?

We (@jzpsych.bsky.social & Florian Scharf) ran some simulations to get the answer...a🧵

Reposted from PsyArXivBot

Minimum Sampling Recommendations for Applied Ambulatory Assessments: http://osf.io/3tme5/

Comments

tervoclemmensb.bsky.social•132 days ago

This looks great!

stephenbenning.bsky.social•131 days ago

Really nifty work! For some reason, I couldn't see Figure 1, which seemed like it would aid mightily in understanding the ARMA model. Is my browser just not rendering that figure?

aidangcw.bsky.social•131 days ago

Thanks! Maybe if you download?

stephenbenning.bsky.social•131 days ago

Alas, it comes up as a 6.30" x 4.72" blank image with the title "Grafik 2" when I download the manuscript and click on Figure 1's location. I'm also an odd bat in using LibreOffice on Linux, but it's at least consistently missing for me.

aidangcw.bsky.social•132 days ago

Why do simulations? Well, EMA data is autocorrelated and often skewed, which means it may require more observations than cross-sectional between-person data.

So we estimated simulations based on parameters from real data from a large EMA study.

aidangcw.bsky.social•132 days ago

We estimated two types of reliability, precision and stability (i.e., retest-reliability). Precision was estimated for each statistic (mean, var, cor) and for changes in the statistic, like pre-post an intervention or different contexts.

aidangcw.bsky.social•132 days ago

This plot summarizes our findings in the aggregate. So, for a precise mean N>10, for Sd N>30, and for cor N>100. These numbers go up for testing changes...needless to say, the findings for the cor are quite sobering.

aidangcw.bsky.social•132 days ago

Interestingly skewness didn't matter much at all, but autocorrelation really did. But there are a lot more details in the manuscript. If you are doing work that involves using EMA as a clinical tool, this might be worth checking out.

If you have thoughts, we'd love to hear them. Under review now.

drrodebaugh.bsky.social•131 days ago

Ouch that's worse than I had hoped on the correlation but not necessarily surprising.

Autocorrelation worse or better than correlation? (Better under usual circumstances but depends is my guess.)

My intuition is that linear trends are easier than correlation but highly dependent on how much change

aidangcw.bsky.social•131 days ago

Higher autocorrelation means you need more observations to get a precise effect

If you think about it, the more autocorrelation the less informative each observation is, it shares a lot of info with neighboring observations

But may not be a big prob for realistic beh data bc AC is low

dingdingpeng.the100.ci•132 days ago

Now this is a cooperation I somehow didn’t anticipate. Florian and I studied together and worked together as student teaching assistants 🤓 looks like a super useful project!

aidangcw.bsky.social•132 days ago

Thanks. Obviously @jzpsych.bsky.social was the link here…the collider? But I think it was a fruitful collaboration overall

dingdingpeng.the100.ci•132 days ago

Maybe more like a confounder? Aidan <- Johannes -> Florian. Depending on who acted upon whom 🧐

aidangcw.bsky.social•132 days ago

I think collider was right (Aidan -> JZ <- Florian) right?

dingdingpeng.the100.ci•132 days ago

We will need to conduct a mediation analysis to find out 😆

aidangcw.bsky.social•132 days ago

Mediation models for all!

Comments

Posting Rules

Reply