LLMs as "the 4. humiliation of humanity". 1. Kopernikus: "humans are not at the center" 2. Darwin: "humans are descendants from animals" 3. Freud: "humans are not 'masters in their own mind'" 4. ChatGPT: "human language imitable by simple statistics" - ThreadSky

About ThreadSky

dirque.bsky.social • 594 days ago

LLMs as "the 4. humiliation of humanity".

1. Kopernikus: "humans are not at the center"
2. Darwin: "humans are descendants from animals"
3. Freud: "humans are not 'masters in their own mind'"
4. ChatGPT: "human language imitable by simple statistics"

Reposted from Dirk Lorenz

The great Misha Belkin on Large Language Models:

mishabelkin.substack.com/p/copernicus...

Comments

gubrud.bsky.social•541 days ago

Agree and have said the same many times. But GPT is NOT "simple statistics." It is a neural model that is trained to predict language as successfully as it can, and this success entails building an internal semantic model, a model of meaning, not of words.

gubrud.bsky.social•541 days ago

Belkin says "the model estimates the conditional probability... for each next word." OK, that's formally true, but that "estimate" is just the output of a softmax function, whose input is the output of a neural computation which is just tuned to produce the most accurate predictions.
+

gubrud.bsky.social•541 days ago

It is not accurate, as many people think, that what LLMs have done is compile statistics of text and that it is generating the next word with no understanding, with no semantic representation. That would actually be impossible (intractable).

pekka.bsky.social•539 days ago

Agreed, the humiliation is real but the reasoning is based on a very common misunderstanding, likely resulting from all the talk about next word predictors.

I have been promoting this excellent article, which seems to strike a very good balance between sufficient detail and understandability.

dirque.bsky.social•539 days ago

The article actually explains that ChatGPT is really predicts the next word by basic statistics (in quite some convolved and ingenious way to learn the probabilities)

pekka.bsky.social•539 days ago

Statistics are of course part of the story but the article you linked to for example claims that the models "are not aware of any linguistic rules" while the Ars article explains in some detail how they are, thanks to those neural networks.

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply