How do LLMs learn to reason from data? Are they ~retrieving the answers from parametric knowledge🦜? In our new preprint, we look at the pretraining data and find evidence against this:
Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢
🧵⬇️
Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢
🧵⬇️
Comments
Or is the preprint suggesting reasoning is more nuanced than originally thought ?
When LLMs are reasoning, they are doing some form of approximate retrieval where they “retrieve” the answer to intermediate reasoning steps from parametric knowledge, as opposed to doing “genuine” reasoning.
The approach to reasoning LLMs use looks unlike retrieval, and more like a generalisable strategy synthesising procedural knowledge from many documents doing a similar form of reasoning.
do you pointers for evidence of this statement? Your proof relies on it and I don't think it's true
It's tough to navigate this discussion without predefining the terms tho. Here https://arxiv.org/abs/2403.17125 authors show that multiple shot prompting works not because you "teach llm to reason" but because you prime it with regards to data it already seen. Reasoning =\= Reasoning
am trying to develop options for probabilistic firewalls
Q: what is/are the best security measure(s) that you are aware of to help stop or mitigate probabilistic injection ?
the simplest form of probabilistic injection is a ‘prompt injection’
an iterative flip around and find out !😎
appreciate the chuckle
☮️ peace
https://medium.com/@john_94579/probability-injection-some-coin-flips-are-more-equal-than-others-a-feature-not-a-bug-dda9b77f2f54
Join for more: http://epic-quotes.org