Here is XKCD describing machine learning as a pile of algebra (no fault here, it's a 40-word webcomic). The serious version of this description (it's all statistics) conjures in peoples' minds the Bag of Words approach to machine learning. 2/n xkcd.com/1838 - ThreadSky

rkeelan.bsky.social • 88 days ago

Here is XKCD describing machine learning as a pile of algebra (no fault here, it's a 40-word webcomic). The serious version of this description (it's all statistics) conjures in peoples' minds the Bag of Words approach to machine learning. 2/n
https://xkcd.com/1838

Comments

rkeelan.bsky.social•88 days ago

You put all the words in a bag and count how often they show up. Maybe you count pairs of words (bigrams) or triplets (trigrams) or some other sequence length (n-grams). I have seen the results from those kinds of systems and they were not good. 3/n

rkeelan.bsky.social•88 days ago

When I saw ChatGPT, it was obvious there was more going on. This is the kind of thing that was going on. Training to predict the next word resulted in the LLMs building increasingly detailed, comprehensive, and accurate models of the world. 4/n
https://transformer-circuits.pub/2025/attribution-graphs/biology.html

rkeelan.bsky.social•88 days ago

If you throw a ball in the air you can calculate how long it will take to hit the ground using simple math. But not just *any* math. You need the equations of motion. These aren't just random calculations. They are a model of the world encoding facts about reality. 5/n

rkeelan.bsky.social•88 days ago

Here's another intuition pump: how well would you have to know someone in order to predict what they'd say in certain situations? This is possible—maybe you can do this for your spouse or children or siblings—but you need to know them *really* well. 6/n

rkeelan.bsky.social•88 days ago

For LLMs to write as coherently as they do on such a broad range of topics requires more than just knowledge of language, because language isn't precise enough.

"I saw a man in a park with a telescope."

Is the telescope in the park, or with the speaker? It's ambiguous. 7/n

rkeelan.bsky.social•88 days ago

Modern LLMs have hundreds of billions of parameters (maybe even trillions by now). That's alot of space to represent a lot of concepts. No one should be confident that they know when LLM performance and abilities will plateau. 8/8

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply