Disagree. Modern LMs should have a knowledge store for facts (don’t rely on the parameters) and they’re moving in the direction of being able to do reasoning, it just doesn’t look like human reasoning. - ThreadSky

natolambert.bsky.social • 102 days ago

Disagree.

Modern LMs should have a knowledge store for facts (don’t rely on the parameters) and they’re moving in the direction of being able to do reasoning, it just doesn’t look like human reasoning.

Reposted from Katie Mack

I honestly believe all LLM results should come with a disclaimer reminding people that the thing doesn’t (and absolutely cannot) know any facts or do any reasoning; it is simply designed to *sound* like it knows facts and frequently sweeps some up accidentally in the process.

Comments

swalton.ai•102 days ago

I believe @astrokatie.com‘s point is that they can’t distinguish truth from fiction. Which would make them schizophrenic.

The reasoning question is more nuanced and personally I’m unconvinced. In fact, I see strong evidence against it. Though who knew highly overfit models could be this useful?

besonders-kreativ.bsky.social•102 days ago

What does their reasoning look like instead then?

natolambert.bsky.social•102 days ago

The mess of o1 outputs :)

https://open.substack.com/pub/robotic/p/openais-o1-using-search-was-a-psyop?r=68gy5&utm_medium=ios

besonders-kreativ.bsky.social•102 days ago

Thank you, that was very insightful. Dumb question, maybe - but it seems like you would know, what's the difference to perplexitys pro search, which really seems to work well too? Which one do you like better?

miror-ergo-sum.bsky.social•102 days ago

I thought I would be able to search public domain philosophy books and get proper citations for quotes from whatever edition the LLM trained on. But I ran into this absence of a knowledge store thing (I call it amnesia). It's a serious limitation and highly frustrating.

natolambert.bsky.social•102 days ago

People are working on it. I discussed it at length here https://www.interconnects.ai/p/interviewing-andrew-trask

miror-ergo-sum.bsky.social•102 days ago

Thanks so much for sharing! I watched the relevant section. Much terminology I don't understand (yet), but am comforted to know that people are working on providing LLMs with a memory/database & looking at retrieval methods and recognizing all this as imp. (basic) functionality for some end users. 👍

miror-ergo-sum.bsky.social•102 days ago

No chance of this being available by the weekend though? :)

natolambert.bsky.social•102 days ago

Lolol no

nima292.bsky.social•102 days ago

Do you envision a knowledge graph of sort for the knowledge store?

natolambert.bsky.social•102 days ago

Ehhh no graphs are too hard. Is retrieval based

natolambert.bsky.social•102 days ago

To be fair, most LLMs today don’t have the knowledge store so the disclaimer is good business practice in some cases

knutjaegersberg.bsky.social•101 days ago

reasoning is a word that implies cognition, contemporary AI has none of that. AI borrows such words to make it easier to understand what it does, but it's only meant metaphorically. Despite some similarities, it doesn't make the cut imo.
The structural dissimilarities weight heavier.

knutjaegersberg.bsky.social•101 days ago

all there is is tools that mimic certain aspects of cognition, and when you stuff human cognition into models, it's gonna look even more similar.
the category error must be avoided, even if a system outperforms humans.

knutjaegersberg.bsky.social•101 days ago

Golems don't think.

natolambert.bsky.social•101 days ago

Reasoning is a process of finding an answer imo. Cognition also I’m fine with computers having right now. Consciousness a lot more blurry.

Tbf I don’t believe in free will and the likes

knutjaegersberg.bsky.social•101 days ago

I think cognition requires a form of self-organization which currently viable algos don't have. free will is imo both an illusion and reality, but not magical.

sanwu.bsky.social•102 days ago

What is your definition of reasoning?

sanwu.bsky.social•101 days ago

Context: when someone asserts that LLMs is/can be capable of reasoning, they need to provide its definition so that we know what goal to drive towards and how to measure success at the end. Else it's a meaningless statement.

Comments

Posting Rules

Reply