Do LLMs understand probability distributions? Can they serve as effective simulators of probability? No! However, in our latest paper that via in-context learning, LLMs update their broken priors in a manner akin to Bayseian updating. 📝 arxiv.org/abs/2503.04722 - ThreadSky

About ThreadSky

ritwikgupta.bsky.social • 12 days ago

Do LLMs understand probability distributions? Can they serve as effective simulators of probability? No!
However, in our latest paper that via in-context learning, LLMs update their broken priors in a manner akin to Bayseian updating.

📝 https://arxiv.org/abs/2503.04722

Comments

ritwikgupta.bsky.social•12 days ago

Biased coin flips follow a simple probability distribution that LLMs should be able to simulate explicitly. In fact, when prompted to flip a fair coin, most LLMs predict heads 70-85% of the time! This holds true even if you prompt the model to flip a biased coin 🪙

ritwikgupta.bsky.social•12 days ago

Can we control this behavior? We tried many things before settling on in-context learning as a working mechanism. If we prompt an LLM to flip a biased coin, and then show increasing rollouts of flips from such a distribution, models converge to the right underlying parameter.

ritwikgupta.bsky.social•12 days ago

Interestingly, models follow a very similar trajectory to what the true Bayesian posterior should look like with the same amount of evidence! When we prompt for coin flips from a 60% heads-biased coin but give it evidence the follows 70% heads, models converge to the latter.

ritwikgupta.bsky.social•12 days ago

This behavior has very interesting quirks. LLMs implicitly demonstrate time-discounting over the ICL examples. That is, recent evidence matters more!

ritwikgupta.bsky.social•12 days ago

Oh, and I must mention the BAIR espresso machine! It was only huddled around freshly ground coffee machines did we come up with this idea (initially wondering if content length matters for statistical behaviors). If you want good research, provide your students with coffee.

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply