BecomeMe

Exactly this. LLMs are parrots with a million years of experience. They know that when someone says "When was Queen Elizabeth born and what was the first thing she did", that the string of words "When was Queen Elizabeth born" is often closely associated with the date "September 7, 1533". It also know that the words in "queen elizabeth What was the first thing she did" is often associated with something like "Establish the English Protestant church".

The parrot knows this because people talk about those things in close connection. But the parrot has no ability to reason that when someone is born, generally the thing they do is cry. It's a parrot, it doesn't reason that when Elizabeth the first was born, she was a baby, and babies cry. It knows what people are saying, but it doesn't think.