LLMs have shown impressive performance in some reasoning tasks, but what internal mechanisms do they use to solve these tasks? In a new preprint, we find evidence that abstract reasoning in LLMs depends on an emergent form of symbol processing arxiv.org/abs/2502.20332 (1/N) - ThreadSky

taylorwwebb.bsky.social • 18 days ago

LLMs have shown impressive performance in some reasoning tasks, but what internal mechanisms do they use to solve these tasks? In a new preprint, we find evidence that abstract reasoning in LLMs depends on an emergent form of symbol processing https://arxiv.org/abs/2502.20332 (1/N)

Comments

wiringthebrain.bsky.social•17 days ago

Really impressive work and thanks for the great thread on it!

professorwcohen.bsky.social•17 days ago

Bookmarking, looks really interesting

taylorwwebb.bsky.social•18 days ago

Cognitive scientists have long argued that human-like reasoning requires some form of symbol processing, and this has often been contrasted with neural networks, which are thought to lack key properties of symbol systems. (2/N)

taylorwwebb.bsky.social•18 days ago

The success of LLMs on abstract reasoning tasks (e.g. https://www.nature.com/articles/s41562-023-01659-w) therefore raises the question: do LLMs solve these tasks using structured, human-like reasoning mechanisms, or do they merely mimic this approach via other mechanisms (e.g. approximate retrieval)? (3/N)

taylorwwebb.bsky.social•18 days ago

We looked at the internal mechanisms that support abstract reasoning in an open-source LLM (Llama3-70B), focusing on an algebraic rule induction task. We found evidence for an emergent 3-stage architecture that solves this task via a form of symbol-processing. (4/N)

jknorrland.bsky.social•17 days ago

📌

taylorwwebb.bsky.social•18 days ago

In the first stage, 'symbol abstraction heads' convert to input tokens to abstract variables, based on their relations with other tokens. Interestingly, this implements an emergent form of the 'Abstractor' architecture that we recently proposed for relational reasoning https://arxiv.org/abs/2304.00195 (5/N)

taylorwwebb.bsky.social•18 days ago

In the second stage, 'symbolic induction heads' predict the abstract variable associated with the next token, implementing a symbolic variant of the 'induction head' mechanism that has been tied to in-context learning https://arxiv.org/abs/2209.11895 (6/N)

taylorwwebb.bsky.social•18 days ago

In the third stage, 'retrieval heads' retrieve the token associated with the predicted variable (effectively inverting the symbol abstraction heads). (7/N)

rivb.bsky.social•18 days ago

📌

Comments

Posting Rules

Reply