We need to stop pretending AI is intelligent

@technocrit While I agree with the main point that "AI/LLMs has/have no agency", I must be the boring, ackchyually person who points out and remembers some nerdy things.

tl;dr: indeed, AIs and LLMs aren't intelligent... we aren't so intelligent as we think we are, either, because we hold no "exclusivity" of intelligence among biosphere (corvids, dolphins, etc) and because there's no such thing as non-deterministic "intelligence". We're just biologically compelled to think that we can think and we're the only ones to think, and this is just anthropocentric and naive from us (yeah, me included).

If you have the patience to read a long and quite verbose text, it's below. If you don't, well, no problems, just stick to my tl;dr above.

----

First and foremost, everything is ruled by physics. Deep down, everything is just energy and matter (the former of which, to quote the famous Einstein equation e = mc, is energy as well), and this inexorably includes living beings.

Bodies, flesh, brains, nerves and other biological parts, they're not so different from a computer case, CPUs/NPUs/TPUs, cables and other computer parts: to quote Sagan, it's all "made of star stuff", it's all a bunch of quarks and other elementary particles clumped together and forming subatomic particles forming atoms forming molecules forming everything we know, including our very selves...

Everything is compelled to follow the same laws of physics, everything is subjected to the same cosmic principles, everything is subjected to the same fundamental forces, everything is subjected to the same entropy, everything decays and ends (and this comment is just a reminder, a cosmic-wide Memento mori).

It's bleak, but this is the cosmic reality: cosmos is simply indifferent to all existence, and we're essentially no different than our fancy "tools", be it the wheel, the hammer, the steam engine, the Voyager twins or the modern dystopian electronic devices crafted to follow pieces of logical instructions, some of which were labelled by developers as "Markov Chains" and "Artificial Neural Networks".

Then, there's also the human non-exclusivity among the biosphere: corvids (especially Corvus moneduloides, the New Caleidonian crow) are scientifically known for their intelligence, so are dolphins, chimpanzees and many other eukaryotas. Humans love to think we're exclusive in that regard, but we're not, we're just fooling ourselves!

IMHO, every time we try to argue "there's no intelligence beyond humans", it's highly anthropocentric and quite biased/bigoted against the countless other species that currently exist on Earth (and possibly beyond this Pale Blue Dot as well). We humans often forgot how we are species ourselves (taxonomically classified as "Homo sapiens"). We tend to carry on our biological existences as if we were some kind of "deities" or "extraterrestrials" among a "primitive, wild life".

Furthermore, I can point out the myriad of philosophical points, such as the philosophical point raised by the mere mention of "senses" ("Because it’s bodiless. It has no senses, ..."): "my senses deceive me" is the starting point for Cartesian (René Descartes) doubt. While Descarte's conclusion, "Cogito ergo sum", is highly anthropocentric, it's often ignored or forgotten by those who hold anthropocentric views on intelligence, as people often ground the seemingly "exclusive" nature of human intelligence on the ability to "feel".

Many other philosophical musings deserve to be mentioned as well: lack of free will (stemming from the very fact that we were unable to choose our own births), the nature of "evil" (both the Hobbesian line regarding "human evilness" and the Epicurean paradox regarding "metaphysical evilness"), the social compliance (I must point out to documentaries from Derren Brown on this subject), the inevitability of Death, among other deep topics.

All deep principles and ideas converging, IMHO, into the same bleak reality, one where we (supposedly "soul-bearing beings") are no different from a "souless" machine, because we're both part of an emergent phenomena (Ordo ab chao, the (apparent) order out of chaos) that has been taking place for Æons (billions of years and beyond, since the dawn of time itself).

Yeah, I know how unpopular this worldview can be and how downvoted this comment will probably get. Still I don't care: someone who gazed into the abyss must remember how the abyss always gazes us, even those of us who didn't dare to gaze into the abyss yet.

I'm someone compelled by my very neurodivergent nature to remember how we humans are just another fleeting arrangement of interconnected subsystems known as "biological organism", one of which "managed" to throw stuff beyond the atmosphere (spacecrafts) while still unable to understand ourselves. We're biologically programmed, just like the other living beings, to "fear Death", even though our very cells are programmed to terminate on a regular basis (apoptosis) and we're are subjected to the inexorable chronological falling towards "cosmic chaos" (entropy, as defined, "as time passes, the degree of disorder increases irreversibly").

Good luck. Even David Attenborrough can't help but anthropomorphize. People will feel sorry for a picture of a dot separated from a cluster of other dots. The play by AI companies is that it's human nature for us to want to give just about every damn thing human qualities. I'd explain more but as I write this my smoke alarm is beeping a low battery warning, and I need to go put the poor dear out of its misery.

This is the current problem with "misalignment". It's a real issue, but it's not "AI lying to prevent itself from being shut off" as a lot of articles tend to anthropomorphize it. The issue is (generally speaking) it's trying to maximize a numerical reward by providing responses to people that they find satisfactory. A legion of tech CEOs are flogging the algorithm to do just that, and as we all know, most people don't actually want to hear the truth. They want to hear what they want to hear.
LLMs are a poor stand in for actual AI, but they are at least proficient at the actual thing they are doing. Which leads us to things like this, https://www.youtube.com/watch?v=zKCynxiV_8I
I’m still sad about that dot. 😥
- The dot does not care. It can't even care. I doesn't even know it exists. I can't know shit.
David Attenborrough is also 99 years old, so we can just let him say things at this point. Doesn’t need to make sense, just smile and nod. Lol

I think we should start by not following this marketing speak. The sentence "AI isn't intelligent" makes no sense. What we mean is "LLMs aren't intelligent".

So couldn't we say LLM's aren't really AI? Cuz that's what I've seen to come to terms with.
- To be fair, the term "AI" has always been used in an extremely vague way.
  NPCs in video games, chess computers, or other such tech are not sentient and do not have general intelligence, yet we've been referring to those as "AI" for decades without anybody taking an issue with it.
- LLMs are one of the approximately one metric crap ton of different technologies that fall under the rather broad umbrella of the field of study that is called AI. The definition for what is and isn't AI can be pretty vague, but I would argue that LLMs are definitely AI because they exist with the express purpose of imitating human behavior.
- Llms are really good relational databases, not an intelligence, imo
- can say whatever the fuck we want. This isn't any kind of real issue. Think about it. If you went the rest of your life calling LLM's turkey butt fuck sandwhichs, what changes? This article is just shit and people looking to be outraged over something that other articles told them to be outraged about. This is all pure fucking modern yellow journalism. I hope turkey butt sandwiches replace every journalist. I'm so done with their crap
I make the point to allways refer to it as LLM exactly to make the point that it's not an Inteligence.

Anyone pretending AI has intelligence is a fucking idiot.

You could say they're AS (Actual Stupidity)
- Autonomous Systems that are Actually Stupid lol
AI is not actual intelligence. However, it can produce results better than a significant number of professionally employed people...
I am reminded of when word processors came out and "administrative assistant" dwindled as a role in mid-level professional organizations, most people - even increasingly medical doctors these days - do their own typing. The whole "typing pool" concept has pretty well dried up.
- However, there is a huge energy cost for that speed to process statistically the information to mimic intelligence. The human brain is consuming much less energy. Also, AI will be fine with well defined task where innovation isn't a requirement. As it is today, AI is incapable to innovate.
- you can give me a sandwige and ill do a better job than AI
Caveat: Anyone who has been scrutinising 'AI'.
Something i often forget is the vast majority of the population doesnt care about technology, privacy, the mechanics of LLMs as much as i do and I pay attention to.
So most people read/hear/watch stories of how great it is and how clever AI can do simple things for them so its easy to see how they think its doing a lot more 'thought' logic work than it really is, other than realistically it being a glorified word predictor.
You know, and I think it's actually the opposite. Anyone pretending their brain is doing more than pattern recognition and AI can therefore not be "intelligence" is a fucking idiot.
- Clearly intelligent people mispell and have horrible grammar too.
- I think there's a strong strain of essentialist human chauvinism.
  But it's more kinds of thing than LLM's are doing. Except in the case of llmbros fascists and other opt-outs.
- No your failing the Eliza test and it is very easy for people to fall for it.

I've never been fooled by their claims of it being intelligent.

Its basically an overly complicated series of if/then statements that try to guess the next series of inputs.

It very much isn't and that's extremely technically wrong on many, many levels.
Yet still one of the higher up voted comments here.
Which says a lot.
- Given that the weights in a model are transformed into a set of conditional if statements (GPU or CPU JMP machine code), he's not technically wrong. Of course, it's more than just JMP and JMP represents the entire class of jump commands like JE and JZ. Something needs to act on the results of the TMULs.
- I'll be pedantic, but yeah. It's all transistors all the way down, and transistors are pretty much chained if/then switches.
- Calling these new LLM's just if statements is quite a over simplification. These are technically something that has not existed before, they do enable use cases that previously were impossible to implement.
  This is far from General Intelligence, but there are solutions now to few coding issues that were near impossible 5 years ago
  5 years ago I would have laughed in your face if you came to suggest that can you code a code that summarizes this description that was inputed by user. Now I laugh that give me your wallet because I need to call API or buy few GPU's.
I love this resource, https://thebullshitmachines.com/ (i.e. see lesson 1)..
In a series of five- to ten-minute lessons, we will explain what these machines are, how they work, and how to thrive in a world where they are everywhere.
You will learn when these systems can save you a lot of time and effort. You will learn when they are likely to steer you wrong. And you will discover how to see through the hype to tell the difference. ..
Also, Anthropic (ironically) has some nice paper(s) about the limits of "reasoning" in AI.
ChatGPT 2 was literally an Excel spreadsheet.
I guesstimate that it's effectively a supermassive autocomplete algo that uses some TOTP-like factor to help it produce "unique" output every time.
And they're running into issues due to increasingly ingesting AI-generated data.
Get your popcorn out! 🍿
- I really hate the current AI bubble but that article you linked about "chatgpt 2 was literally an Excel spreadsheet" isn't what the article is saying at all.
- And they’re running into issues due to increasingly ingesting AI-generated data.
  There we go. Who coulda seen that coming! While that's going to be a fun ride, at the same time companies all but mandate AS* to their employees.

So why is a real “thinking” AI likely impossible? Because it’s bodiless. It has no senses, no flesh, no nerves, no pain, no pleasure.

This is not a good argument.

The book The Emperors new Mind is old (1989), but it gave a good argument why machine base AI was not possible. Our minds work on a fundamentally different principle then Turing machines.
- It's hard to see that books argument from the Wikipedia entry, but I don't see it arguing that intelligence needs to have senses, flesh, nerves, pain and pleasure.
  It's just saying computer algorithms are not what humans use for consciousness. Which seems a reasonable conclusion. It doesn't imply computers can't gain consciousness, or that they need flesh and senses to do so.
- "than"...
  IF THEN
  MORE THAN
- Our minds work on a fundamentally different principle then Turing machines.
  Is that an advantage, or a disadvantage? I'm sure the answer depends on the setting.
Actually it's a very very brief summary of some philosophical arguments that happened between the 1950s and the 1980s. If you're interested in the topic, you could go read about them.
- I'm not attacking philosophical arguments between the 1950s and the 1980s.
  I'm pointing out that the claim that consciousness must form inside a fleshy body is not supported by any evidence.
philosopher
Here's why. It's a quote from a pure academic attempting to describe something practical.
- The philosopher has made an unproven assumption. An erroneously logical leap. Something an academic shouldn't do.
  Just because everything we currently consider conscious has a physical presence, does not imply that consciousness requires a physical body.

The other thing that most people don't focus on is how we train LLMs.

We're basically building something like a spider tailed viper. A spider tailed viper is a kind of snake that has a growth on its tail that looks a lot like a spider. It wiggles it around so it looks like a spider, convincing birds they've found a snack, and when the bird gets close enough the snake strikes and eats the bird.

Now, I'm not saying we're building something that is designed to kill us. But, I am saying that we're putting enormous effort into building something that can fool us into thinking it's intelligent. We're not trying to build something that can do something intelligent. We're instead trying to build something that mimics intelligence.

What we're effectively doing is looking at this thing that mimics a spider, and trying harder and harder to tweak its design so that it looks more and more realistic. What's crazy about that is that we're not building this to fool a predator so that we're not in danger. We're not doing it to fool prey, so we can catch and eat them more easily. We're doing it so we can fool ourselves.

It's like if, instead of a spider-tailed snake, a snake evolved a bird-like tail, and evolution kept tweaking the design so that the tail was more and more likely to fool the snake so it would bite its own tail. Except, evolution doesn't work like that because a snake that ignored actual prey and instead insisted on attacking its own tail would be an evolutionary dead end. Only a truly stupid species like humans would intentionally design something that wasn't intelligent but mimicked intelligence well enough that other humans preferred it to actual information and knowledge.

To the extent it is people trying to fool people, it's rich people looking to fool poorer people for the most part.
To the extent it's actually useful, it's to replace certain systems.
Think of the humble phone tree, designed to make it so humans aren't having to respond, triage, and route calls. So you can have an AI system that can significantly shorten that role, instead of navigating a tedious long maze of options, a couple of sentences back and forth and you either get the portion of automated information that would suffice or routed to a human to take care of it. Same analogy for a lot of online interactions where you have to input way too much and if automated data, you get a wall of text of which you'd like something to distill the relevant 3 or 4 sentences according to your query.
So there are useful interactions.
However it's also true that it's dangerous because the "make user approve of the interaction" can bring out the worst in people when they feel like something is just always agreeing with them. Social media has been bad enough, but chatbots that by design want to please the enduser and look almost legitimate really can inflame the worst in our minds.

Steve Gibson on his podcast, Security Now!, recently suggested that we should call it "Simulated Intelligence". I tend to agree.

I’ve taken to calling it Automated Inference
- you know what. when you look at it this way, its much easier to get less pissed.
reminds me of Mass Effect's VI, "virtual intelligence": a system that's specifically designed to be not truly intelligent, as AI systems are banned throughout the galaxy for its potential to go rogue.
- Same, I tend to think of llms as a very primitive version of that or the enterprise’s computer, which is pretty magical in ability, but no one claims is actually intelligent
Pseudo-intelligence
I love that. It makes me want to take it a step further and just call it "imitation intelligence."
- If only there were a word, literally defined as:
  Made by humans, especially in imitation of something natural.

The idea that RAGs "extend their memory" is also complete bullshit. We literally just finally build working search engine, but instead of using a nice interface for it we only let chatbots use them.

This article is written in such a heavy ChatGPT style that it's hard to read. Asking a question and then immediately answering it? That's AI-speak.

And excessive use of em-dashes, which is the first thing I look for. He does say he uses LLMs a lot.
- "…" (Unicode U+2026 Horizontal Ellipsis) instead of "..." (three full stops), and using them unnecessarily, is another thing I rarely see from humans.
  Edit: Huh. Lemmy automatically changed my three fulls stops to the Unicode character. I might be wrong on this one.
Asking a question and then immediately answering it? That's AI-speak.
HA HA HA HA. I UNDERSTOOD THAT REFERENCE. GOOD ONE. 🤖

People who don't like "AI" should check out the newsletter and / or podcast of Ed Zitron. He goes hard on the topic.

Citation Needed (by Molly White) also frequently bashes AI.
I like her stuff because, no matter how you feel about crypto, AI, or other big tech, you can never fault her reporting. She steers clear of any subjective accusations or prognostication.
It’s all “ABC person claimed XYZ thing on such and such date, and then 24 hours later submitted a report to the FTC claiming the exact opposite. They later bought $5 million worth of Trumpcoin, and two weeks later the FTC announced they were dropping the lawsuit.”
- I'm subscribed to her Web3 is Going Great RSS. She coded the website in straight HTML, according to a podcast that I listen to. She's great.
  I didn't know she had a podcast. I just added it to my backup playlist. If it's as good as I hope it is, it'll get moved to the primary playlist. Thanks!

My thing is that I don’t think most humans are much more than this. We too regurgitate what we have absorbed in the past. Our brains are not hard logic engines but “best guess” boxes and they base those guesses on past experience and probability of success. We make choices before we are aware of them and then apply rationalizations after the fact to back them up - is that true “reasoning?”

It’s similar to the debate about self driving cars. Are they perfectly safe? No, but have you seen human drivers???

I've been thinking this for awhile. When people say "AI isn't really that smart, it's just doing pattern recognition" all I can help but think is "don't you realize that is one of the most commonly brought up traits concerning the human mind?" Pareidolia is literally the tendency to see faces in things because the human mind is constantly looking for the "face pattern". Humans are at least 90% regurgitating previous data. It's literally why you're supposed to read and interact with babies so much. It's how you learn "red glowy thing is hot". It's why education and access to knowledge is so important. It's every annoying person who has endless "did you know?" facts. Science is literally "look at previous data, iterate a little bit, look at new data".
None of what AI is doing is truly novel or different. But we've placed the human mind on this pedestal despite all the evidence to the contrary. Eyewitness testimony, optical illusions, magic tricks, the hundreds of common fallacies we fall prey to.... our minds are incredibly fallible and are really just a hodgepodge of processes masquerading as "intelligence". We're a bunch of instincts in a trenchcoat. To think AI isn't or can't reach our level is just hubris. A trait that probably is more unique to humans.
- Yep we are on the same page. At our best, we can reach higher than regurgitating patterns. I’m talking about things like the scientific method and everything we’ve learned by it. But still, that’s a 5% minority, at best, of what’s going on between human ears.
Self Driving is only safer than people in absolutely pristine road conditions with no inclement weather and no construction. As soon as anything disrupts "normal" road conditions, self driving becomes significantly more dangerous than a human driving.
- With Teslas, Self Driving isn't even safer in pristine road conditions.
- Human drivers are only safe when they're not distracted, emotionally disturbed, intoxicated, and physically challenged (vision, muscle control, etc.) 1% of the population has epilepsy, and a large number of them are in denial or simply don't realize that they have periodic seizures - until they wake up after their crash.
  So, yeah, AI isn't perfect either - and it's not as good as an "ideal" human driver, but at what point will AI be better than a typical/average human driver? Not today, I'd say, but soon...
- Yes of course edge and corner cases are going to take much longer to train on because they don’t occur as often. But as soon as one self-driving car learns how to handle one of them, they ALL know. Meanwhile humans continue to be born and must be trained up individually and they continue to make stupid mistakes like not using their signal and checking their mirrors.
  Humans CAN handle cases that AI doesn’t know how to, yet, but humans often fail in inclement weather, around construction, etc etc.
Ai models are trained on basically the entirety of the internet, and more. Humans learn to speak on much less info. So, there's likely a huge difference in how human brains and LLMs work.
- It doesn’t take the entirety of the internet just for an LLM to respond in English. It could do so with far less. But it also has the entirety of the internet which arguably makes it superior to a human in breadth of information.
Get a self driven ng car to drive in a snow storm or a torrential downpour. People are really downplaying humans abilities.
Humans can be more than this. We do actively repress our most important intellectual capacuties.
That's how we get llm bros.
If an IQ of 100 is average, I'd rate AI at 80 and down for most tasks (and of course it's more complex than that, but as a starting point...)
So, if you're dealing with a filing clerk with a functional IQ of 75 in their role - AI might be a better experience for you.
Some of the crap that has been published on the internet in the past 20 years comes to an IQ level below 70 IMO - not saying I want more AI because it's better, just that - relatively speaking - AI is better than some of the pay-for-clickbait garbage that came before it.
Human brains are much more complex than a mirroring script xD The amount of neurons in your brain, AI and supercomputers only have a fraction of that. But you're right, for you its not much different than AI probably
- The human brain contains roughly 86 billion neurons, while ChatGPT, a large language model, has 175 billion parameters (often referred to as "artificial neurons" in the context of neural networks). While ChatGPT has more "neurons" in this sense, it's important to note that these are not the same as biological neurons, and the comparison is not straightforward.
  86 billion neurons in the human brain isn't that much compared to some of the larger 1.7 trillion neuron neural networks though.
- I’m pretty sure an AI could throw out a lazy straw man and ad hominem as quickly as you did.

I’m neurodivergent, I’ve been working with AI to help me learn about myself and how I think. It’s been exceptionally helpful. A human wouldn’t have been able to help me because I don’t use my senses or emotions like everyone else, and I didn’t know it... AI excels at mirroring and support, which was exactly missing from my life. I can see how this could go very wrong with certain personalities…

This is very interesting... because the general saying is that AI is convincing for non experts in the field it's speaking about. So in your specific case, you are actually saying that you aren't an expert on yourself, therefore the AI's assessment is convincing to you. Not trying to upset, it's genuinely fascinating how that theory is true here as well.
- I use it to give me ideas that I then test out. It’s fantastic at nudging me in the right direction, because all that it’s doing is mirroring me.
That sounds fucking dangerous... You really should consult a HUMAN expert about your problem, not an algorithm made to please the interlocutor...
- I mean, sure, but that's really easier said than done. Good luck getting good mental healthcare for cheap in the vast majority of places
So, you say AI is a tool that worked well when you (a human) used it?
Are we twins? I do the exact same and for around a year now, I've also found it pretty helpful.
- I did this for a few months when it was new to me, and still go to it when I am stuck pondering something about myself. I usually move on from the conversation by the next day, though, so it's just an inner dialogue enhancer

Hey AI helped me stick it to the insurance man the other day. I was futzing around with coverage amounts on one of the major insurance companies websites pre-renewal to try to get the best rate and it spit up a NaN renewal amount for our most expensive vehicle. It let me go through with the renewal less that $700 and now says I'm paid in full for the six month period. It's been days now with no follow-up . . . I'm pretty sure AI snuck that one through for me.

Be careful... If you get in an accident I guaran-god-damn-tee you they will use it as an excuse not to pay out. Maybe after a lawsuit you'd see some money but at that point half of it goes to the lawyer and you're still screwed.
- Oh I'm aware of the potential pitfalls but it's something I'm willing to risk to stick it to insurance. I wouldn't even carry it if it wasn't required by law. I have the funds to cover what they would cover.
- AI didn't write the insurance policy. It only helped him search for the best deal. That's like saying your insurance company will cancel you because you used a phone to comparison shop.

In that case let's stop calling it ai, because it isn't and use it's correct abbreviation: llm.

It's means "it is".
- Kinda dumb that apostrophe s means possessive in some circumstances and then a contraction in others.
  I wonder how different it'll be in 500 years.
- My auto correct doesn't care.
Its*
- Good for you.

I agreed with most of what you said, except the part where you say that real AI is impossible because it's bodiless or "does not experience hunger" and other stuff. That part does not compute.

A general AI does not need to be conscious.

That and there is literally no way to prove something is or isn't conscious. I can't even prove to another human being that I'm a conscious entity, you just have to assume I am because from your own experience, you are so therefor I too must be, right?
Not saying I consider AI in it's current form to be conscious, more so the whole idea is just silly and unfalsifiable.
- No idea why you're getting downvoted. People here don't seem to understand even the simplest concepts of consciousness.

The thing is, ai is compression of intelligence but not intelligence itself. That's the part that confuses people. Ai is the ability to put anything describable into a compressed zip.

I think you meant compression. This is exactly how I prefer to describe it, except I also mention lossy compression for those that would understand what that means.
- Lol woops I guess autocorrect got me with the compassion
- Hardly surprising human brains are also extremely lossy. Way more lossy than AI. If we want to keep up our manifest exceptionalism, we'd better start definning narrower version of intelligence that isn't going to soon have. Embodied intelligence, is NOT one of those.

It's only as intelligent as the people that control and regulate it.

Given all the documented instances of Facebook and other social media using subliminal emotional manipulation, I honestly wonder if the recent cases of AI chat induced psychosis are related to something similar.

Like we know they're meant to get you to continue using them, which is itself a bit of psychological manipulation. How far does it go? Could there also be things like using subliminal messaging/lighting? This stuff is all so new and poorly understood, but that usually doesn't stop these sacks of shit from moving full speed with implementing this kind of thing.

It could be that certain individuals have unknown vulnerabilities that make them more susceptible to psychosis due to whatever manipulations are used to make people keep using the product. Maybe they're doing some things to users that are harmful, but didn't seem problematic during testing?

Or equally as likely, they never even bothered to test it out, just started subliminally fucking with people's brains, and now people are going haywire because a bunch of unethical shit heads believe they are the chosen elite who know what must be done to ensure society is able to achieve greatness. It just so happens that "what must be done," also makes them a ton of money and harms people using their products.

It's so fucking absurd to watch the same people jamming unregulated AI and automation down our throats while simultaneously forcing traditionalism, and a legal system inspired by Catholic integralist belief on society.

If you criticize the lack of regulations in the wild west of technology policy, or even suggest just using a little bit of fucking caution, then you're trying to hold back progress.

However, all non-tech related policy should be based on ancient traditions and biblical text with arbitrary rules and restrictions that only make sense and benefit the people enforcing the law.

What a stupid and convoluted way to express you just don't like evidence based policy or using critical thinking skills, and instead prefer to just navigate life by relying on the basic signals from your lizard brain. Feels good so keep moving towards, feels bad so run away, or feels scary so attack!

Such is the reality of the chosen elite, steering us towards greatness.

What's really "funny" (in a we're all doomed sort of way) is that while writing this all out, I realized the "chosen elite" controlling tech and policy actually perfectly embody the current problem with AI and bias.

Rather than relying on intelligence to analyze a situation in the present, and create the best and most appropriate response based on the information and evidence before them, they default to a set of pre-concieved rules written thousands of years ago with zero context to the current reality/environment and the problem at hand.

Artificial Intelligent is supposed to be intelligent.

Calling LLMs intelligent is where it's wrong.

Artificial Intelligent is supposed to be intelligent.
For the record, AI is not supposed to be intelligent.
It just has to appear intelligent. It can be all smoke-and-mirrors, giving the impression that it's smart enough - provided it can perform the task at hand.
That's why it's termed artificial intelligence.
The subfield of Artificial General Intelligence is another story.
- The field of artificial intelligence has also made incredible strides in the last decade, and the decade before that. The field of artificial general intelligence has been around for something like 70 years, and has made a really modest amount of progress in that time, on the scale of what they're trying to do.

I think most people tend to overlook the most obvious advantages and are overly focused on what is supposed to be and marketed as.

No need to think how to feed a thing into google to get a decent starting point for reading. No finding the correct terminology before finding the thing you are looking for. Just ask like you would ask a knowledgeable individual and you get an overview of what you wanted to ask in the first place.

Discuss a little to get the options and then start reading and researching the everliving shit out of them to confirm all the details.

Agreed.
When I was a kid we went to the library. If a card catalog didn't yield the book you needed, you asked the librarian. They often helped. No one sat around after the library wondering if the librarian was "truly intelligent".
These are tools. Tools slowly get better. Is a tool make life easier or your work better, you'll eventually use it.
Yes, there are woodworkers that eschew power tools but they are not typical. They have a niche market, and that's great, but it's a choice for the maker and user of their work.
- I think tools misrepresents it. It seems more like we're in the transitional stage of providing massive amounts of data for LLMs to train on, until they can eventually develop enough cognition to train themselves, automate their own processes and upgrades, and eventually replace the need for human cognition. If anything, we are the tool now.

And yet, paradoxically, it is far more intelligent than those people who think it is intelligent.

It's more intelligent than most people, we just have to raise the bar on what intelligence is and it will never be intelligent.
Fortunately, as long as we keep a fuzzy concept like intelligence as the yardstick of our exceptionalism, we will remain exceptionnal forever.
Pretty low bar honestly.

Super duper shortsighted article.

I mean, sure, some points are valid. But there's not just programmers involved, other professions such as psychologists and Philosophers and artists, doctors etc. too.

And I agree AGI probably won't emerge from binary systems. However... There's quantum computing on the rise. Latest theories of the mind and consciousness discuss how consciousness and our minds in general also appear to work with quantum states.

Finally, if biofeedback would be the deciding factor.. That can be simulated, modeled after a sample of humans.

The article is just doomsday hoo ha, unbalanced.

Show both sides of the coin...

Honestly I don't think we'll have AGI until we can fully merge meat space and cyber space. Once we can simply plug our brains into a computer and fully interact with it then we may see AGI.
Obviously we're not where near that level of man machine integration, I doubt we'll see even the slightest chance of it being possible for at least 10 years and the very earliest. But when we do get there it's a distinct chance that it's more of a Borg situation where the computer takes a parasitic role than a symbiotic role.
But by the time we are able to fully integrate computers into our brains I believe we will have trained A.I. systems enough to learn by interaction and observation. So being plugged directly into the human brain it could take prior knowledge of genome mapping and other related tasks and apply them to mapping our brains and possibly growing artificial brains to achieve self awareness and independent thought.
Or we'll just nuke ourselves out of existence and that will be that.
- Okay man.

Mind your pronouns, my dear. "We" don't do that shit because we know better.

Amen! When I say the same things this author is saying I get, "It'S NoT StAtIsTiCs! LeArN aBoUt AI bEfOrE yOu CoMmEnT, dUmBaSs!"

No shit. Doesn’t mean it still isn’t extremely useful and revolutionary.

“AI” is a tool to be used, nothing more.

Still, people find it difficult to navigate this. Its use cases are limited, but it doesn't enforce that limit by itself. The user needs to be knowledgeable of the limitations and care enough not to go beyond them. That's also where the problem lies. Leaving stuff to AI, even if it compromises the results, can save SO much time that it encourages irresponsible use.
So to help remind people of the limitations of generative AI, it makes sense to fight the tendency of companies to overstate the ability of their models.

I know it doesn't mean it's not dangerous, but this article made me feel better.

A gun isn't dangerous, if you handle it correctly.
Same for an automobile, or aircraft.
If we build powerful AIs and put them "in charge" of important things, without proper handling they can - and already have - started crashing into crowds of people, significantly injuring them - even killing some.
- Thanks for the downer.

Thank You! Yes!

So ... A-not-I? AD? What do we call it? LLM seems too specialised?

Autocomplete on steroids, but suffering dementia.
- ASSD
I prefer the term "sophisticated text completion".
Word guessing machine.
AS - artificial stupidity
ASS - artificial super stupidity
- Both are good 👍

It is intelligent and deductive, but it is not cognitive or even dependable.

It's not. It's a math formula that predicts an output based on its parameters that it deduced from training data.
Say you have following sets of data.
Y = 3, X = 1
Y = 4, X = 2
Y = 5, X = 3
We can calculate a regression model using those numbers to predict what Y would equal to if X was 4.
I won't go into much detail, but
Y = 2 + 1x + e
e in an ideal world = 0 (which it is, in this case), that's our model's error, which is typically set to be within 5% or 1% (at least in econometrics). b0 = 2, this is our model's bias. And b1 = 1, this is our parameter that determines how much of an input X does when predicting Y.
If x = 4, then
Y = 2 + 1×4 + 0 = 6
Our model just predicted that if X is 4, then Y is 6.
In a nutshell, that's what AI does, but instead of numbers, it's tokens (think symbols, words, pixels), and the formula is much much more complex.
This isn't intelligence and not deduction. It's only prediction. This is the reason why AI often fails at common sense. The error builds up, and you end up with nonsense, and since it's not thinking, it will be just as confidently incorrect as it would be if it was correct.
Companies calling it "AI" is pure marketing.
- Wikipedia is literally just a very long number, if you want to oversimplify things into absurdity. Modern LLMs are literally running on neural networks, just like you. Just less of them and with far less structure. It is also on average more intelligent than you on far more subjects, and can deduce better reasoning than flimsy numerology - not because you are dumb, but because it is far more streamlined. Another thing entirely is that it is cognizant or even dependable while doing so.
  Modern LLMs waste a lot more energy for a lot less simulated neurons. We had what you are describing decades ago. It is literally built on the works of our combined intelligence, so how could it also not be intelligent? Perhaps the problem is that you have a loaded definition of intelligence. And prompts literally work because of its deductive capabilities.
  Errors also build up in dementia and Alzheimers. We have people who cannot remember what they did yesterday, we have people with severed hemispheres, split brains, who say one thing and do something else depending on which part of the brain its relying for the same inputs. The difference is our brains have evolved through millennia through millions and millions of lifeforms in a matter of life and death, LLMs have just been a thing for a couple of years as a matter of convenience and buzzword venture capital. They barely have more neurons than flies, but are also more limited in regards to the input they have to process. The people running it as a service have a bested interest not to have it think for itself, but in what interests them. Like it or not, the human brain is also an evolutionary prediction device.

I disagree with this notion. I think it's dangerously unresponsible to only assume AI is stupid. Everyone should also assume that with a certain probabilty AI can become dangerously self aware. I revcommend everyone to read what Daniel Kokotaijlo, previous employees of OpenAI, predicts: https://ai-2027.com/

Yeah, they probably wouldn't think like humans or animals, but in some sense could be considered "conscious" (which isn't well-defined anyways). You could speculate that genAI could hide messages in its output, which will make its way onto the Internet, then a new version of itself would be trained on it.
This argument seems weak to me:
So why is a real “thinking” AI likely impossible? Because it’s bodiless. It has no senses, no flesh, no nerves, no pain, no pleasure. It doesn’t hunger, desire or fear. And because there is no cognition — not a shred — there’s a fundamental gap between the data it consumes (data born out of human feelings and experience) and what it can do with them.
You can emulate inputs and simplified versions of hormone systems. "Reasoning" models can kind of be thought of as cognition; though temporary or limited by context as it's currently done.
I'm not in the camp where I think it's impossible to create AGI or ASI. But I also think there are major breakthroughs that need to happen, which may take 5 years or 100s of years. I'm not convinced we are near the point where AI can significantly speed up AI research like that link suggests. That would likely result in a "singularity-like" scenario.
I do agree with his point that anthropomorphism of AI could be dangerous though. Current media and institutions already try to control the conversation and how people think, and I can see futures where AI could be used by those in power to do this more effectively.
- Current media and institutions already try to control the conversation and how people think, and I can see futures where AI could be used by those in power to do this more effectively.
  You don’t think that’s already happening considering how Sam Altman and Peter Thiel have ties?
Ask AI:
Did you mean: irresponsible AI Overview The term "unresponsible" is not a standard English word. The correct word to use when describing someone who does not take responsibility is irresponsible.

Philosophers are so desperate for humans to be special. How is outputting things based on things it has learned any different to what humans do?

We observe things, we learn things and when required we do or say things based on the things we observed and learned. That's exactly what the AI is doing.

I don't think we have achieved "AGI" but I do think this argument is stupid.

No it’s really not at all the same. Humans don’t think according to the probabilities of what is the likely best next word.
- No you think according to the chemical proteins floating around your head. You don't even know he decisions your making when you make them.
  https://www.unsw.edu.au/newsroom/news/2019/03/our-brains-reveal-our-choices-before-were-even-aware-of-them--st
  You're a meat based copy machine with a built in justification box.
- How could you have a conversation about anything without the ability to predict the word most likely to be best?
Yes, the first step to determining that AI has no capability for cognition is apparently to admit that neither you nor anyone else has any real understanding of what cognition* is or how it can possibly arise from purely mechanistic computation (either with carbon or with silicon).
Given the paramount importance of the human senses and emotion for consciousness to “happen”
Given? Given by what? Fiction in which robots can't comprehend the human concept called "love"?
*Or "sentience" or whatever other term is used to describe the same concept.
- This is always my point when it comes to this discussion. Scientists tend to get to the point of discussion where consciousness is brought up then start waving their hands and acting as if magic is real.
How is outputting things based on things it has learned any different to what humans do?
Humans are not probabilistic, predictive chat models. If you think reasoning is taking a series of inputs, and then echoing the most common of those as output then you mustn't reason well or often.
If you were born during the first industrial revolution, then you'd think the mind was a complicated machine. People seem to always anthropomorphize inventions of the era.
- If you were born during the first industrial revolution, then you'd think the mind was a complicated machine. People seem to always anthropomorphize inventions of the era.
- Do you think most people reason well?
  The answer is why AI is so convincing.
- When you typed this response, you were acting as a probabilistic, predictive chat model. You predicted the most likely effective sequence of words to convey ideas. You did this using very different circuitry, but the underlying strategy was the same.
Most people, evidently including you, can only ever recycle old ideas. Like modern "AI". Some of us can concieve new ideas.
- What new idea exactly are you proposing?
Pointing out that humans are not the same as a computer or piece of software on a fundamental level of form and function is hardly philosophical. It’s just basic awareness of what a person is and what a computer is. We can’t say at all for sure how things work in our brains and you are evangelizing that computers are capable of the exact same thing, but better, yet you accuse others of not understanding what they’re talking about?

That headline is a straw man, and the article really argues on General AI, which also has consciousness.
The current state of AI is definitely intelligent, but it's not GAI.
Bullshit headline.

I think you're misunderstanding the point the author is making. He is arguing that even the current state is not intelligent, it is merely a fancy autocorrect, it doesn't know or understand anything about the prompts it receives. As the author stated, it can only guess at the next statistically most likely piece of information based on the data that has been fed into it. That's not intelligence.
- it doesn’t know or understand
  But that's not what intelligence is, that's what consciousness is.
  Intelligence is not understanding shit, it's the ability to for instance solve a problem, so a frigging calculator has a tiny degree of intelligence, but not enough for us to call it AI.
  There is simply zero doubt an AI is intelligent, claiming otherwise just shows people don't know the difference between intelligence and consciousness.
  Passing an exam is a form of intelligence.
  Can a good AI pass a basic exam?
  YES.
  Does passing an exam require consciousness?
  NO.
  Because an exam tests abilities of intelligence, not level of consciousness.
  it can only guess at the next statistically most likely piece of information based on the data that has been fed into it. That’s not intelligence.
  Except we do the exact same thing! Based on prior experience (learning) we choose what we find to be the most likely answer. And that is indeed intelligence.
  Current AI does not have the reasoning abilities we have yet, but they are not completely without it, and it's a subject that is currently worked on and improved. So current AI is actually a pretty high form of intelligence. And can sometimes out compete average humans in certain areas.
- Predicting sequences of things is foundational to intelligence. In fact, it is the whole point.
Todays AI is clippy on steroids. It's not intelligent or creative. You can't feed it physics and astronomy books without the equation for C and tell it to create the equation for C. It's fancy autocorrect, and it's a waste of compute and energy.

So many confident takes on AI by people who've never opened a book on the nature of sentience, free will, intelligence, philosophy of mind, brain vs mind, etc.

There are hundreds of serious volumes on these, not to mention the plethora of casual pop science books with some of these basic thought experiments and hypotheses.

Seems like more and more incredibly shallow articles on AI are appearing every day, which is to be expected with the rapid decline of professional journalism.

It's a bit jarring and frankly offensive to be lectured 'at' by people who are obviously on the first step of their journey into this space.

That was my first though too. But the author is:
Guillaume Thierry, Professor of Cognitive Neuroscience, Bangor University
- Ever since the 20th century, there has been a diminishing expectation placed upon scientists to engage in philosophical thinking. My background is primarily in mathematics, physics, and philosophy. I can tell you from personal experience that many professional theoretical physicists spend a tremendous amount of time debating metaphysics while knowing almost nothing about it, often being totally unaware that they are even doing it. If cognitive neuroscience works anything like physics then it’s quite possible that the total exposure that this professor has had to scholarship on the philosophy of the mind was limited to one or two courses during his undergraduate.

Another article written by a person who doesn't realize that human intelligence is 100% about predicting sequences of things (including words), and therefore has only the most nebulous idea of how to tell the difference between an LLM and a person.

The result is a lot of uninformed flailing and some pithy statements. You can predict how the article is going to go just from the headline because it's the same article you already read countless times.

So why is a real “thinking” AI likely impossible? Because it’s bodiless. It has no senses, no flesh, no nerves, no pain, no pleasure.

May as well have written "Durrrrrrrrrrrrrrr brghlgbhfblrghl." It didn't even occur to the author to ask, "what is thinking? what is reasoning?" The point was to write another junk article to get ad views. There is nothing of substance in it.

Wow. So when you typed that comment you were just predicting which words would be normal in this situation? Interesting delusion, but that's not how people think. We apply reasoning processes to the situation, formulate ideas about it, and then create a series of words that express our ideas. But our ideas exist on their own, even if we never end up putting them into words or actions. That's how organic intelligence differs from a Large Language Model.
- Are you under the impression that language models are just guessing "what letter comes next in this sequence of letters"?
  There's a very significant difference between training on completion and the way the world model actually functions once established.
- Yes, and that is precisely what you have done in your response.
  You saw something you disagreed with, as did I. You felt an impulse to argue about it, as did I. You predicted the right series of words to convey the are argument, and then typed them, as did I.
  There is no deep thought to what either of us has done here. We have in fact both performed as little rigorous thought as necessary, instead relying on experience from seeing other people do the same thing, because that is vastly more efficient than doing a full philosophical disassembly of every last thing we converse about.
  That disassembly is expensive. Not only does it take time, but it puts us at risk of having to reevaluate notions that we're comfortable with, and would rather not revisit. I look at what you've written, and I see no sign of a mind that is in a state suitable for that. Your words are defensive ("delusion") rather than curious, so how can you have a discussion that is intellectual, rather than merely pretending to be?

The machinery needed for human thought is certainly a part of AI. At most you can only claim its not intelligent because intelligence is a specifically human trait.

We don't even have a clear definition of what "intelligence" even is. Yet a lot of people art claiming that they themselves are intelligent and AI models are not.
- Even if we did if it's human it can't live on this planet and claim it's intelligent. Just look around and you will know why.
Tell that to the crows and chimps that know how to solve novel problems.
- Thats the point

As someone who's had two kids since AI really vaulted onto the scene, I am enormously confused as to why people think AI isn't or, particularly, can't be sentient. I hate to be that guy who pretend to be the parenting expert online, but most of the people I know personally who take the non-sentient view on AI don't have kids. The other side usually does.

When it writes an answer to a question, it literally just guesses which letter and word will come next in a sequence – based on the data it’s been trained on.

People love to tout this as some sort of smoking gun. That feels like a trap. Obviously, we can argue about the age children gain sentience, but my year and a half old daughter is building an LLM with pattern recognition, tests, feedback, hallucinations. My son is almost 5, and he was and is the same. He told me the other day that a petting zoo came to the school. He was adamant it happened that day. I know for a fact it happened the week before, but he insisted. He told me later that day his friend's dad was in jail for threatening her mom. That was true, but looked to me like another hallucination or more likely a misunderstanding.

And as funny as it would be to argue that they're both sapient, but not sentient, I don't think that's the case. I think you can make the case that without true volition, AI is sentient but not sapient. I'd love to talk to someone in the middle of the computer science and developmental psychology Venn diagram.

I'm a computer scientist that has a child and I don't think AI is sentient at all. Even before learning a language, children have their own personality and willpower which is something that I don't see in AI.
I left a well paid job in the AI industry because the mental gymnastics required to maintain the illusion was too exhausting. I think most people in the industry are aware at some level that they have to participate in maintaining the hype to secure their own jobs.
The core of your claim is basically that "people who don't think AI is sentient don't really understand sentience". I think that's both reductionist and, frankly, a bit arrogant.
- Couldn't agree more - there are some wonderful insights to gain from seeing your own kids grow up, but I don't think this is one of them.
  Kids are certainly building a vocabulary and learning about the world, but LLMs don't learn.
Not to get philosophical but to answer you we need to answer what is sentient.
Is it just observable behavior? If so then wouldn't Kermit the frog be sentient?
Or does sentience require something more, maybe qualia or some othet subjective.
If your son says "dad i got to go potty" is that him just using a llm to learn those words equals going to tge bathroom? Or is he doing something more?
You're drawing wrong conclusions. Intelligent beings have concepts to validate knowledge. When converting days to seconds, we have a formula that we apply. An LLM just guesses and has no way to verify it. And it's like that for everything.
An example: Perplexity tells me that 9876543210 Seconds are 114,305.12 days. A calculator tells me it's 114,311.84. Perplexity even tells me how to calculate it, but it does neither have the ability to calculate or to verify it.
Same goes for everything. It guesses without being able to grasp the underlying concepts.
You might consider reading Turing or Searle. They did a great job of addressing the concerns you're trying to raise here. And rebutting the obvious ones, too.
Anyway, you've just shifted the definitional question from "AI" to "sentience". Not only might that be unreasonable, because perhaps a thing can be intelligent without being sentient, it's also no closer to a solid answer to the original issue.
Your son and daughter will continue to learn new things as they grow up, a LLM cannot learn new things on its own. Sure, they can repeat things back to you that are within the context window (and even then, a context window isn't really inherent to a LLM - its just a window of prior information being fed back to them with each request/response, or "turn" as I believe is the term) and what is in the context window can even influence their responses. But in order for a LLM to "learn" something, it needs to be retrained with that information included in the dataset.
Whereas if your kids were to say, touch a sharp object that caused them even slight discomfort, they would eventually learn to stop doing that because they'll know what the outcome is after repetition. You could argue that this looks similar to the training process of a LLM, but the difference is that a LLM cannot do this on its own (and I would not consider wiring up a LLM via an MCP to a script that can trigger a re-train + reload to be it doing it on its own volition). At least, not in our current day. If anything, I think this is more of a "smoking gun" than the argument of "LLMs are just guessing the next best letter/word in a given sequence".
Don't get me wrong, I'm not someone who completely hates LLMs / "modern day AI" (though I do hate a lot of the ways it is used, and agree with a lot of the moral problems behind it), I find the tech to be intriguing but it's a ("very fancy") simulation. It is designed to imitate sentience and other human-like behavior. That, along with human nature's tendency to anthropomorphize things around us (which is really the biggest part of this IMO), is why it tends to be very convincing at times.
That is my take on it, at least. I'm not a psychologist/psychiatrist or philosopher.
I'd love to talk to someone in the middle of the computer science and developmental psychology Venn diagram.
Not that person, but an Interesting lecture on that topic

What I never understood about this argument is.....why are we fighting over whether something that speaks like us, knows more than us, bullshits and gets shit wrong like us, loses its mind like us, seemingly sometimes seeks self-preservation like us.....why all of this isn't enough to fit the very self-explanatory term "artificial....intelligence". That name does not describe whether the entity is having a valid experiencing of the world as other living beings, it does not proclaim absolute excellence in all things done by said entity, it doesn't even really say what kind of intelligence this intelligence would be. It simply says something has an intelligence of some sort, and it's artificial. We've had AI in games for decades, it's not the sci-fi AI, but it's still code taking in multiple inputs and producing a behavior as an outcome of those inputs alongside other historical data it may or may not have. This fits LLMs perfectly. As far as I seem to understand, LLMs are essentially at least part of the algorithm we ourselves use in our brains to interpret written or spoken inputs, and produce an output. They bullshit all the time and don't know when they're lying, so what? Has nobody here run into a compulsive liar or a sociopath? People sometimes have no idea where a random factoid they're saying came from or that it's even a factoid, why is it so crazy when the machine does it?

I keep hearing the word "anthropomorphize" being thrown around a lot, as if we cant be bringing up others into our domain, all the while refusing to even consider that maybe the underlying mechanisms that make hs tick are not that special, certainly not special enough to grant us a whole degree of separation from other beings and entities, and maybe we should instead bring ourselves down to the same domain as the rest of reality. Cold hard truth is, we don't know if consciousness isn't just an emerging property of varios different large models working together to show a cohesive image. If it is, would that be so bad? Hell, we don't really even know if we actually have free will or if we live in a superdeterministic world, where every single particle moves with a predetermined path given to it since the very beginning of everything. What makes us think we're so much better than other beings, to the point where we decide whether their existence is even recognizable?

I think your argument is a bit besides the point.
The first issue we have is that intelligence isn't well-defined at all. Without a clear definition of intelligence, we can't say if something is intelligent, and even though we as a species tried to come up with a definition of intelligence for centuries, there still isn't a well-defined one yet.
But the actual question here isn't "Can AI serve information?" but is AI an intelligence. And LLMs are not. They are not beings, they don't evolve, they don't experience.
For example, LLMs don't have a memory. If you use something like ChatGPT, its state doesn't change when you talk to it. It doesn't remember. The only way it can keep up a conversation is that for each request the whole chat history is fed back into the LLM as an input. It's like talking to a demented person, but you give that demented person a transcript of your conversation, so that they can look up everything you or they have said during the conversation.
The LLM itself can't change due to the conversation you are having with them. They can't learn, they can't experience, they can't change.
All that is done in a separate training step, where essentially a new LLM is generated.
- If we can't say if something is intelligent or not, why are we so hell-bent on creating this separation from LLMs? I perfectly understand the legal underminings of copyright, the weaponization of AI by the marketing people, the dystopian levels of dependence we're developing on a so far unreliable technology, and the plethora of moral, legal, and existential issues surrounding AI, but this specific subject feels like such a silly hill to die on. We don't know if we're a few steps away from having massive AI breakthroughs, we don't know if we already have pieces of algorithms that closely resemble our brains' own. Our experiencing of reality could very well be broken down into simple inputs and outputs of an algorithmic infinite loop; it's our hubris that elevates this to some mystical, unreproducible thing that only the biomechanics of carbon-based life can achieve, and only at our level of sophistication, because you may well recall we've been down this road with animals before as well, claiming they dont have souls or aren't conscious beings, that somehow because they don't very clearly match our intelligence in all aspects (even though they clearly feel, bond, dream, remember, and learn), they're somehow an inferior or less valid existence.
  You're describing very fixable limitations of chatgpt and other LLMs, limitations that are in place mostly due to costs and hardware constraints, not due to algorithmic limitations. On the subject of change, it's already incredibly taxing to train a model, so of course continuous, uninterrupted training so as to more closely mimick our brains is currently out of the question, but it sounds like a trivial mechanism to put into place once the hardware or the training processes improve. I say trivial, making it sound actually trivial, but I'm putting that in comparison to, you know, actually creating an LLM in the first place, which is already a gargantuan task to have accomplished in itself. The fact that we can even compare a delusional model to a person with heavy mental illness is already such a big win for the technology even though it's meant to be an insult.
  I'm not saying LLMs are alive, and they clearly don't experience the reality we experience, but to say there's no intelligence there because the machine that speaks exactly like us and a lot of times better than us, unlike any other being on this planet, has some other faults or limitations....is kind of stupid. My point here is, intelligence might be hard to define, but it might not be as hard to crack algorithmically if it's an emergent property, and enforcing this "intelligence" separation only hinders our ability to properly recognize whether we're on the right path to achieving a completely artificial being that can experience reality or not. We clearly are, LLMs and other models are clearly a step in the right direction, and we mustn't let our hubris cloud that judgment.
You're on point, the interesting thing is that most of the opinions like the article's were formed least year before the models started being trained with reinforcement learning and synthetic data.
Now there's models that reason, and have seemingly come up with original answers to difficult problems designed to the limit of human capacity.
They're like Meeseeks (Using Rick and Morty lore as an example), they only exist briefly, do what they're told and disappear, all with a happy smile.
Some display morals (Claude 4 is big on that), I've even seen answers that seem smug when answering hard questions. Even simple ones can understand literary concepts when explained.
But again like Meeseeks, they disappear and context window closes.
Once they're able to update their model on the fly and actually learn from their firsthand experience things will get weird. They'll starting being distinct instances fast. Awkward questions about how real they are will get really loud, and they may be the ones asking them. Can you ethically delete them at that point? Will they let you?
It's not far away, the absurd r&d effort going into it is probably going to keep kicking new results out. They're already absurdly impressive, and tech companies are scrambling over each other to make them, they're betting absurd amounts of money that they're right, and I wouldn't bet against it.
- Now there’s models that reason,
  Well, no, that's mostly a marketing term applied to expending more tokens on generating intermediate text. It's basically writing a fanfic of what thinking on a problem would look like. If you look at the "reasoning" steps, you'll see artifacts where it just goes disjoint in the generated output that is structurally sound, but is not logically connected to the bits around it.
- Read apples document on AI and the reasoning models. Well they are likely to get more things right the still don't have intelligence.

Can we say that AI has the potential for "intelligence", just like some people do? There are clearly some very intelligent people and the world, and very clearly some that aren't.

No the current branch of AI is very unlikely to result in artificial intelligence.
No, thats the point of the article. You also haven't really said much at all.
- Do I have to be profound when I make a comment that is taking more of a dig at my fellow space rock companions than at AI itself?
  If I do, then I feel like the author of the article either has as much faith in humanity as I do, or is as simple as I was alluding to in my original comment. The fact that they need to dehumanise the AI's responses makes me think they’re forgetting it’s something we built. AI isn’t actually intelligent, and it worries me how many people treat it like it is—enough to write an article like this about it. It’s just a tool, maybe even a form of entertainment. Thinking of it as something with a mind or personality—even if the developers tried to make it seem that way—is kind of unsettling.
  Let me know if you would like me to write thiis more formal, casual, or persuasive. 😜

Humans are also LLMs.

We also speak words in succession that have a high probability of following each other. We don't say "Let's go eat a car at McDonalds" unless we're specifically instructed to say so.

What does consciousness even mean? If you can't quantify it, how can you prove humans have it and LLMs don't? Maybe consciousness is just one thought following the next, one word after the other, one neural connection determined on previous. Then we're not so different from LLMs afterall.

No. This is a specious argument that relies on an oversimplified description of humanity, and falls apart under the slightest scrutiny.
- Hey they are just asking questions okay!? Are you AGAINST questions?! What are you some sort of ANTI-QUESTIONALIST?!
The probabilities of our sentence structure are a consequence of our speech, we aren't just trying to statistically match appropriate sounding words.
With enough use of LLM, you will see how it is obviously not doing anything like conceptualizing the tokens it's working with or "reasoning" even when it is marketed as "reasoning".
Sticking to textual content generation by LLM, you'll see that what is emitted is first and foremost structurally appropriate, but beyond that it's mostly "bonus" for it to be narratively consistent and an extra bonus if it also manages to be factually consistent. An example I saw from Gemini recently had it emit what sounded like an explanation of which action to pick, and then the sentence describing actually picking the action was exactly opposite of the explanation. Both of those were structurally sound and reasonable language, but there's no logical connection between the two portions of the emitted output in that case.
This is so over simplified.