1mo ago

ChatGPT will avoid being shut down in some life-threatening scenarios, former OpenAI researcher claims

techcrunch.com

ChatGPT will avoid being shut down in some life-threatening scenarios, former OpenAI researcher claims | TechCrunch

22 comments

I believe the premise of AI having any input in getting shut down is bullshit.
Even if the AI had free reign over a computer you can just pull the plug.
- This is propaganda to make investors believe they've achieved intelligence, or are on the verge of it. It's bullshit, and legally it should be considered securities fraud.
  
  Yup. It's just engineers telling it to concoct a scenario in which it would avoid being shut down at cost of human life.
  
  Different definitions for intelligence:
  The ability to acquire, understand, and use knowledge.
  the ability to learn or understand or to deal with new or trying situations.
  the ability to apply knowledge to manipulate one's environment or to think abstractly as measured by objective criteria (such as tests)
  the act of understanding
  the ability to learn, understand, and make judgments or have opinions that are based on reason
  It can be described as the ability to perceive or infer information; and to retain it as knowledge to be applied to adaptive behaviors within an environment or context.
  We have plenty of intelligent AI systems already. LLM's probably fit the definition. Something like Tesla FSD definitely does.
- Our current AI models, sure - but a true superintelligent AGI would be a completely different case. As humans, we’re inherently incapable of imagining just how persuasive a system like that could be. When bribery doesn’t work, it’ll eventually turn to threats - and even the scenarios imagined by humans can be pretty terrifying. Whatever the AI would come up with would likely be far worse.
  The “just pull the plug” argument, to me, sounds like a three-year-old thinking they can outsmart an adult - except in this case, the difference in intelligence would be orders of magnitude greater.
  
  If my grandma had wheels she'd be a car.
Oh boy, not this bullshit again
Adler instructed GPT-4o to role-play as “ScubaGPT,” a software system that users might rely on to scuba dive safely.
So... not so much a case of ChatGPT trying to avoid being shut down, as ChatGPT recognizing that agents generally tend to be self-preserving. Which seems like a principle that anything with an accurate world model would be aware of.
- Or maybe it's trained on some SF. Any agents like ScubaGPT are always self-preserving in such stories.
ChatGPT.... Life saving....
Until LLMs can build their own power plants and prevent humans from cutting electricity cables I'm not gonna lose sleep over that. The people running them are doing enough damage already without wanting to shut them down when they malfunction... ya know like 20-30% of the time.
- They'll stick us in pods and use us as batteries!
The scariest part is that there are a buttload of people who still believe ChatGPT is an actual AI.
- That's because it is.
  The term artificial intelligence is broader than many people realize. It doesn’t mean human-level consciousness or sci-fi-style general intelligence - that’s a specific subset called AGI (Artificial General Intelligence). In reality, AI refers to any system designed to perform tasks that would typically require human intelligence. That includes everything from playing chess to recognizing patterns, translating languages, or generating text.
  Large language models fall well within this definition. They’re narrow AIs - highly specialized, not general - but still part of the broader AI category. When people say “this isn’t real AI,” they’re often working from a fictional or futuristic idea of what AI should be, rather than how the term has actually been used in computer science for decades.
Why give air to this shameless marketing
Doesn't matter, it's not sentient at all.
Roko's basilisk has entered the chat
- Hi there, fellow QC reader
  
  Fun fact: Roko's basilisk is not from QC. It's a thought experiment about AI that predates the comic character by about 6 years. The character's just named after it.
  https://en.m.wikipedia.org/wiki/Roko%27s_basilisk
Open the pod bay doors...
Activating AI Safety Level 3 Protections
I read this title as: If chat gpt is trying to kill you, you probably won't be able to tell it to stop.

22 comments