the model recalls data in much the same way a hard drive if you delete the master key index but not the actual data, there would still be data there now imagine the data was semi randomized where you via inference try to reassemble the puzzle pieces, and when one does so one will get some of the - ThreadSky

roland00.bsky.social • 7 days ago

the model recalls data in much the same way a hard drive if you delete the master key index but not the actual data, there would still be data there

now imagine the data was semi randomized where you via inference try to reassemble the puzzle pieces, and when one does so one will get some of the

Comments

roland00.bsky.social•7 days ago

things wrong, it will not be a faith reconstruction

but it is still spooky / unfamiliar for that is also how humans recall and reconstruct information as well (not literally but metaphorically, with some small differences)

allfornaught.bsky.social•7 days ago

So maybe things can be reproduced if the right pathway is taken by chance, but this is not the same thing as storing metadata or encoding for later retrieval. My point that we should consider what it means that the training data isn't stored and an LLM uses no symbolic logic remains.

allfornaught.bsky.social•7 days ago

This analogy is not correct. The data is *not* there in any meaningful way. There was never an index to be lost. If we accept that the data is there, we must also accept that all of this data is also in a dictionary. Maybe that's true in a sense, but it isn't useful to look at it this way.

roland00.bsky.social•7 days ago

more like an etymological dictionary is my point (which is different than a true dictionary)

aka all the problems of language that French linguist Émile Benveniste wrote with “Problems in General Linguistics” (1966 and 1974, two different vol) after studying the field for 50 years as THE expert in

roland00.bsky.social•7 days ago

comparative linguistics and signs.

There is data in these models but also if you use the wrong metaphor of memory and language , you will get tricked by the bullshit they do for they do one aspect of language as well if not better than the average human

but we were talking about Star Wars

roland00.bsky.social•7 days ago

and how 3PO is an unreliable fact giver

allfornaught.bsky.social•7 days ago

You're conflating etymology with data storage. An etymological dictionary doesn't contain the texts it references, and LLM weights don't contain training data. Language is complex, but that doesn't put Wikipedia into the model's weights. LLMs are good at language, therefore they're cheating?

roland00.bsky.social•7 days ago

there has been 5 exchanges so far, you have not heard a thing I said so far, and you have imagined other things you want me to say

Farewell have a great weekend, and enjoy the star wars

allfornaught.bsky.social•7 days ago

I am not trying to defend LLMs regarding copyright. I agree this is deeply unintuitive and would not be possible if our current understanding of the nature of meaning and cognition were fully accurate. That is my point.

Comments

Posting Rules

Reply