Every living system must adapt, or die.
In my first blog post, I show how this fundamental principle can be mathematized:
✅ Brains adapt, and adaptation is about KL divergence minimization.
Let's unpack the main insights 🧵[1/n]
Link: https://mysterioustune.com/2025/01/13/world-models-adaptation-and-surprise-part-1/
🧠🤖🧠📈 #AI
In my first blog post, I show how this fundamental principle can be mathematized:
✅ Brains adapt, and adaptation is about KL divergence minimization.
Let's unpack the main insights 🧵[1/n]
Link: https://mysterioustune.com/2025/01/13/world-models-adaptation-and-surprise-part-1/
🧠🤖🧠📈 #AI
Comments
✅ Adaptation is so central to all biological intelligence, that it may very well define it.
But can we mathematize this fuzzy English term❓
🧵[2/n]
🧠🤖🧠📈 #AI
Here's the table of contents to give you a sense of what's there👇
⏩ Let's continue with summarizing the most important results.
🧵[3/n]
This magical equation connects two seemingly distinct concepts:
1⃣ P_obs: the subjective probability your brain assigns to its observations
2⃣ KL divergence between the true world and your internal beliefs
🧵[4/n]
you adapt to the world
↔️
you accurately predict the world
↔️
fewer nasty surprises that could get you killed
↔️
KL ( world || your beliefs ) drops
🧵[5/n]
I didn't put it in by hand, or make crazy assumptions.
✅ There is something truly fundamental, unique, and privileged about KL divergence.
🧵[6/n]
✅ In section 4, I apply these foundational principles to "derive" (almost) all of machine learning from KL minimization.
(first pointed out by @alexalemi.bsky.social : https://blog.alexalemi.com/kl-is-all-you-need.html)
🧵[7/n]