A naïve way to do this might be to divide X up into some number of bins. Then weight each sample in your data by 1/(number of samples where X falls in the same bin as this one). That will give you a distribution where X "looks flat" in that it has a flat histogram. - ThreadSky

About ThreadSky

nathanielvirgo.bsky.social • 97 days ago

A naïve way to do this might be to divide X up into some number of bins. Then weight each sample in your data by 1/(number of samples where X falls in the same bin as this one). That will give you a distribution where X "looks flat" in that it has a flat histogram.

Comments

nathanielvirgo.bsky.social•97 days ago

But how do you calculate the mutual information from that data - wouldn't you need to bin it anyway? In that case you might as well bin all the data first, then calculate the effective information using the binned distribution. Or is there another way to calculate the MI in the continuous case?

thosvarley.bsky.social•96 days ago

Nope, continuous estimators exist. If you're willing to make parametric assumptions that your data is drawn from a multivariate Gaussian, mutual information is a deterministic function of the Pearson correlation coefficient.

If you want a non-parametric estimator, there are approaches...

thosvarley.bsky.social•96 days ago

...based on K-nearest neighbors graphs (the "Kraskov" estimators) that work for continuous data sampled from arbitrary point clouds.
https://journals.aps.org/pre/abstract/10.1103/PhysRevE.69.066138

nathanielvirgo.bsky.social•96 days ago

I hadn't come across Krashov estimators before, that's interesting. For Gaussians it seems like there should be an analytic estimator for the effective correlation - except that I'm not sure it's even defined in that case, because then the variables are unbounded and there's no uniform distribution.

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply