Technology

59201 readers

2829 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related content.
Be excellent to each another!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, to ask if your bot can be added please contact us.
Check for duplicates before posting, duplicates may be removed

Approved Bots

founded 1 year ago

MODERATORS

LLMs develop their own understanding of reality as their language abilities improve (news.mit.edu)

submitted 2 months ago* (last edited 2 months ago) by Hackworth@lemmy.world to c/technology@lemmy.world

31 comments fedilink hide all child comments

“An intriguing open question is whether the LLM is actually using its internal model of reality to reason about that reality as it solves the robot navigation problem,” says Rinard. “While our results are consistent with the LLM using the model in this way, our experiments are not designed to answer this next question.”

The paper, "Emergent Representations of Program Semantics in Language Models Trained on Programs" can be found here.

Abstract

We present evidence that language models (LMs) of code can learn to represent the formal semantics of programs, despite being trained only to perform next-token prediction. Specifically, we train a Transformer model on a synthetic corpus of programs written in a domain-specific language for navigating 2D grid world environments. Each program in the corpus is preceded by a (partial) specification in the form of several input-output grid world states. Despite providing no further inductive biases, we find that a probing classifier is able to extract increasingly accurate representations of the unobserved, intermediate grid world states from the LM hidden states over the course of training, suggesting the LM acquires an emergent ability to interpret programs in the formal sense. We also develop a novel interventional baseline that enables us to disambiguate what is represented by the LM as opposed to learned by the probe. We anticipate that this technique may be generally applicable to a broad range of semantic probing experiments. In summary, this paper does not propose any new techniques for training LMs of code, but develops an experimental framework for and provides insights into the acquisition and representation of formal semantics in statistical models of code.

you are viewing a single comment's thread
view the rest of the comments

[–] sugar_in_your_tea@sh.itjust.works 2 points 2 months ago

Exactly. If we give an LLM with no training data a large group of specimens, will it organize them into logical groups? Does it even understand the concept of organizing things into discrete groups?

That's something that's largely encoded into our brain structures due to millennia of evolution (or creation, take your pick) where such organization is advantageous. The LLM would only do it if we indicated that such organization is advantageous, and even then would only do it if we gave it a desired output. An LLM will only reflect the priorities of its creator, or at least the priorities baked in to the training data. It's not going to suggest that something else entirely be considered, because it only considers things from the lenses we give it.

Humans will question assumptions, will organize things without being prompted, and will generate our own priorities. I firmly believe an LLM cannot, and thus cannot be considered self-deterministic, and thus not sentient. All it can do is optimize for the priorities we give it, and while it may do that in surprising ways, that doesn't mean there's "thinking" going on, just that it's a complex system we don't fully understand (even if we created it). Maybe human brains work in a similar way (i.e. completely deterministic given a specific genome and "training data"), but we know LLMs work that way, so until we prove that humans work similarly, we cannot equate them. It's kind of like the P = NP question, we know LLMs are deterministic, we don't know if humans are. So the question isn't "can LLMs think" (we know they can't), but "can humans think."