The success of LLMs on abstract reasoning tasks (e.g. www.nature.com/articles/s41...) therefore raises the question: do LLMs solve these tasks using structured, human-like reasoning mechanisms, or do they merely mimic this approach via other mechanisms (e.g. approximate retrieval)? (3/N) - ThreadSky

taylorwwebb.bsky.social • 20 days ago

The success of LLMs on abstract reasoning tasks (e.g. https://www.nature.com/articles/s41562-023-01659-w) therefore raises the question: do LLMs solve these tasks using structured, human-like reasoning mechanisms, or do they merely mimic this approach via other mechanisms (e.g. approximate retrieval)? (3/N)

Comments

taylorwwebb.bsky.social•20 days ago

We looked at the internal mechanisms that support abstract reasoning in an open-source LLM (Llama3-70B), focusing on an algebraic rule induction task. We found evidence for an emergent 3-stage architecture that solves this task via a form of symbol-processing. (4/N)

jknorrland.bsky.social•20 days ago

📌

taylorwwebb.bsky.social•20 days ago

In the first stage, 'symbol abstraction heads' convert to input tokens to abstract variables, based on their relations with other tokens. Interestingly, this implements an emergent form of the 'Abstractor' architecture that we recently proposed for relational reasoning https://arxiv.org/abs/2304.00195 (5/N)

taylorwwebb.bsky.social•20 days ago

In the second stage, 'symbolic induction heads' predict the abstract variable associated with the next token, implementing a symbolic variant of the 'induction head' mechanism that has been tied to in-context learning https://arxiv.org/abs/2209.11895 (6/N)

taylorwwebb.bsky.social•20 days ago

In the third stage, 'retrieval heads' retrieve the token associated with the predicted variable (effectively inverting the symbol abstraction heads). (7/N)

taylorwwebb.bsky.social•20 days ago

Using causal mediation analyses, we confirmed the presence of these heads in a series of three stages: abstraction heads were present in early layers, symbolic induction heads were present in middle layers, and retrieval heads were present in later layers. (8/N)

taylorwwebb.bsky.social•20 days ago

Using attention analyses, we confirmed that these heads' attention patterns were consistent with the hypothesized mechanisms. For instance, for symbolic induction heads, attention was primarily directed to tokens that instantiated the same abstract variable as the next token. (9/N)

Comments

Posting Rules

Reply