Perhaps there is text encoded into the static decimal weights somehow, but there is *no logic* to do this or retrieve it. This thing turns your words into numbers (tokens) runs them through a math function that "predicts" output tokens (one at a time, in order!) to create understanding and meaning.
Comments
For the humans supervising it, laziness masquerading as the creation of life.
That's actually not entirely true. People have been layer-probing models for a couple of years now to figure out how different parameter subsets interact and encode heuristics.
Google went heavy on this; qv GemmaScope.
I just get annoyed at the oft-repeated assertion that "nobody knows how it works!" which is IMO misleading.
There's a lot which is not known, but in some contexts what we *do* know matters more than what we don't.
I apologize for jumping down your throat a bit.
I never claimed there was explicit "logic to retrieve it", just that it is encoded in the weights and shows up in generated text
the human brain stories information in multiple type of nerve cells, these nerve cells are of different types and then reconstructs it
or you need to be in a similar emotional state to recall specific details that you half forgotten
now imagine the data was semi randomized where you via inference try to reassemble the puzzle pieces, and when one does so one will get some of the