You can't predict the next token in an arbitrary text unless you are highly intelligent and have a vast body of knowledge.
They're obviously intelligent in the way that we judge intelligence in humans: we pay attention to what they say. You ask them a question about an arbitrary subject, and they respond in the same way that an intelligent person would. If you don't consider that intelligence, then you have a fundamentally magical, unscientific view of what intelligence is.
To return to an analogy I used a couple of days ago ... birds can fly, planes can fly, ergo they are both flying things ... but they fly in completely different ways. So on the one hand (visible behavior) they are similar (or even the same), and on the other (physical mechanism) they are not similar at all.
Which one of these comparisons you want to use depends on context.
The same seems entirely possible for current LLMs. On the one hand they do something that visibly seems to to be the same as something humans do, but on the other it is possible that the way they do it entirely different. Just as with the bird/plane comparison, this has some implications when you start to dig deeper into capabilities (e.g. planes cannot fly anywhere near as slowly as birds, and birds cannot fly as fast as planes; birds have dramatically more maneuverability than planes, etc. etc).
So are LLMs intelligent in the same way humans are? Depends on your purpose in asking that question. Planes fly, but they are not birds.
To extend your analogy, imagine that there are airplane skeptics who insist that planes can't fly, will never fly, and are good for nothing. They only crudely simulate flight. Meanwhile, millions of people are flying around every day in planes.
But if by "flight" you meant "the sorts of things swallows and kestrels can do",then the movement of planes through the sky would be at best irrelevant.
This is simply wrong, and missing the point, simultaneously.
Flight (like "intelligence") means more than one thing. Planes fly, birds fly, but they not only use a different mechanism, they can't even do the same kind of flying that the other does.
Sometimes, the difference doesn't matter. Sometimes it does. Same for "intelligence".
We don't actually know that much about how the brain works, and nobody discussing intelligence will decide tomorrow that humans aren't intelligent if the details of how the brain functions turn out to be slightly different from what we previously thought.
LLMs obviously display what everyone prior to 2022 would have called "intelligence," before the goalposts started rapidly shifting with the release of ChatGPT. They can carry conversations about arbitrary subjects, understanding what you're asking and formulating thoughtful answers at the level of a very smart and extremely well educated human. They're not identical to humans (e.g., they don't have fixed personalities), but they display what everyone commonly believes to be intelligence.
I know you're arguing with someone else, but I think it is getting sidetracked.
Whether or not LLMs are intelligent (I think they are more intelligent than a cat, for instance, but less intelligent than a human) isn't my argument.
My argument is that complexity in and of itself doesn't yield intelligence. There's no proof of that. There are many things that are very very complex, but we would not put it on an intelligence scale.
