we do not know anything conclusive about the computational basis of the human facility for language. it would be reasonable to say that it does not likely correspond in any direct sense to what is happening in a LLM but it is impossible to say that we even know what it consists of. - ThreadSky

theophite.bsky.social • 7 days ago

we do not know anything conclusive about the computational basis of the human facility for language. it would be reasonable to say that it does not likely correspond in any direct sense to what is happening in a LLM but it is impossible to say that we even know what it consists of.

Comments

parkerkosticak.bsky.social•7 days ago

I think her conclusion is reasonable though. An LLM only takes in the written word, devoid of the intent and audience of that written word, and people interpret that as thought and human cognition itself. Language isn’t that as we don’t need language for that function.

jkfecke.bsky.social•7 days ago

Yes, but the written word has been a primary means of transmitting thoughts to others for thousands of years; it's simply not tenable to argue that it's impossible for someone to understand thought based on text; that's the point of text.

parkerkosticak.bsky.social•7 days ago

Yes, but text is only the intermediary between thoughts. To another human being, who interprets the text based on what they do know or believe to be true, including extratextual knowledge and similarities in encoding, and then writes about to another of a possible different culture knowledge set.

dferrer.bsky.social•7 days ago

The relationship between language, meaning, and thought is philosophically... controversial. You could easily spend a whole PhD doing nothing but arguing for *or* against every clause you have written here. That's without even mentioning LLMs.

ubermitch.bsky.social•7 days ago

ugh, that sounds boring; can I just watch Arrival again instead?

ankurgos.bsky.social•7 days ago

It also is out of date commentary on pre-rlhf LMs. We literally do elicit feedback from people for model outputs and incorporate that feedback explicitly into model training.

gothedistance49.bsky.social•7 days ago

As my intro to linguistics prof loved to say. "The brain is a black box", "we can't see inside and pull information off of it like a flight recorder, we don't fully understand it and how it works and what makes everything magically give us language and other species different types of communication"

gothedistance49.bsky.social•7 days ago

The second part is kinda paraphrased. I'm not sure he was referring to the flight recorder black box style thing originally. Could be later through my studies I filled that logical leap in somehow lol

daringsteel.bsky.social•7 days ago

"they don't have qualia"-ass argument

look at my philosophers, dawg, we're never getting a coherent theory of personhood (everyone who could construct one is possessed by the engram of Commander Bruce Maddox and compelled to hack out an exception that lets him dismantle Data)

jkfecke.bsky.social•7 days ago

What if P-zombies, but cyber?

michaelander45.bsky.social•7 days ago

The larger point though that language coherence cannot be directly equated with understanding and intention seems correct though even if you cannot conclusively disprove it either

jacob.barss-bailey.org•7 days ago

I don't know we can even claim that with any confidence

gigercat.bsky.social•7 days ago

Coherence, on the other hand, strikes me as a more interesting question. This paper (linked below) shows that LLMs are remarkably bad at multi-turn conversations.

https://arxiv.org/pdf/2505.06120

gigercat.bsky.social•7 days ago

This is talked about as reliability (correctly) in the paper but it also strikes me as a coherence problem - it seems like some internal mechanism of the model isn't able to be as rigorously logically consistent as one would want in any kind of really complex conversation.

gigercat.bsky.social•7 days ago

Consider for example back-and-forth with an LLM over a piece of generated code; at least personally I've noticed that it can get stuck on its own mistakes and struggles to maintain what was a right versus a wrong choice. Some models also people please themselves into incoherency.

michaelander45.bsky.social•7 days ago

That seems obvious to me given they are structured with a complete lack of internalized time dependent updating (eg outside of the input prompt)

gigercat.bsky.social•7 days ago

I think I agree, but even then I would expect them to be better. Conversations are sequences.

That said, there's also a mismatch of training and usage, I think. A PhD on my team thinks that multi-turn conversations are a significant enough departure from the training process to cause these issues.

michaelander45.bsky.social•7 days ago

I have always wondered how these systems would perform with some kind of self updating layers. For all I know thats been tried

gigercat.bsky.social•7 days ago

Sure, but I think there's a bunch of studies that show that LLMs (and actually neural networks trained on English text largely) do have some understanding of the language. It may not be the same as the one we have, but it is structurally reproducible (assuming the same objective and modality).

erisianrite.com•7 days ago

Small point: LLMs are neural networks.

gigercat.bsky.social•7 days ago

Yes - I think I was unclear, I meant "LLMS and neural networks more generally" but didn't have space.

michaelander45.bsky.social•7 days ago

That depends on how you define a NN, LLMS are a mix of attention blocks and FFL (eg classic NN)

erisianrite.com•7 days ago

There aren't several dozen definitions of 'neural network' floating around or anything. Transformer architectures are still neuron-based deep learning networks. Encoders and/or decoders (depending on specific architecture) are neural networks and are an essential building block of a transformer.

isaiahbishop.bsky.social•7 days ago

Probably referring to something like MLP rather than neural nets tbh

michaelander45.bsky.social•7 days ago

Right I was more questioning whether it is ontologically correct to say something that includes NN as a building block but also other stuff is a NN. This is kind of not that important though TBH.

michaelander45.bsky.social•7 days ago

Oh obviously the models have an "understanding" of the language but I was taking coherence and understanding there to imply something different (the ongoing problem with all of this TBF)

Comments

Posting Rules

Reply