Yes, good way to put it. Looking at the architecture of a LLM bot there is a "foundation model" and an "inference engine". The inference engine can be developed to incrementally improve the "reasoning process" - break down problem to steps and write language programs in turn operate on those steps. - ThreadSky

ebben.bsky.social • 55 days ago

Yes, good way to put it. Looking at the architecture of a LLM bot there is a "foundation model" and an "inference engine". The inference engine can be developed to incrementally improve the "reasoning process" - break down problem to steps and write language programs in turn operate on those steps.

Comments

ebben.bsky.social•55 days ago

Secondly, note that it is the foundation model that captures the statistics. But the new thing is a level of abstraction above that. Turns out the model captures a semantic space in many dimensions where distances in this large space between words can be easily calculated. So, more than statistics.

ebben.bsky.social•55 days ago

About the novel concept of high-dimensional semantic space in millions, sometimes billions of dimensions. We don't have concepts for this. Language models compress all they have read into a neat vector-based space where semantic distances between elements can be calculated along all the dimensions.

annamillsoer.bsky.social•55 days ago

I don't know if that is technically still statistics, but there's got to be a mathy term that still encompasses it... it's still numbers-based prediction, no?
I am fascinated by the explainability research Anthropic is doing. If I were a philosopher I would focus on that...

ebben.bsky.social•53 days ago

Regarding a mathy term for what LLM's are doing, I'd say 2 things: The foundation model is based on "auto-regression"
and the inference engine is based on matrix algebra that does a sort of multidimensional optimization search ("attention") over the input prompt+foundation model. (Loosely speaking).

ebben.bsky.social•55 days ago

Good question. Yes it's numbers based because the model was implemented by a computer. But, from a Marr levels perspective, you can look at it from a higher level of abstraction. In a very loose sense LLMs develop a sort of semantic model of language.

ebben.bsky.social•55 days ago

We can say the brain is probability based too at the implementation level, but we don't talk like that. Maybe that's the conceptual confusion with the term statistics in a computer vs "thinking" in humans. I know this is unclear, I'm just brainstorming how to describe this possibly new phenomenon.

annamillsoer.bsky.social•55 days ago

I'm thinking of the "features" the Anthropic engineers are finding in the models for things like sycophancy.

swepalm.bsky.social•55 days ago

I like to seperate the question about how a model "represent knowledge" from training, and how you generate output during inference.
I think it is pretty easy to understand that a model could generate different outputs, based on different generation strategies (https://huggingface.co/docs/transformers/generation_strategies#decoding-strategies)

swepalm.bsky.social•55 days ago

But how a model represent knowledge is still an area of research, I accept that they do not represent it in a way we could fully understand 😀 Different models have different numbers of dimensions, layers and nodes... some nodes we can say what they do, and some nodes we can remove...

annamillsoer.bsky.social•55 days ago

Yes; that's part of why it's fascinating that Anthropic created this concept of a "feature" that involves multiple nodes. They're only at the beginning of explainability research...

Comments

Posting Rules

Reply