AI is learning to lie, scheme, and threaten its creators during stress-testing scenarios
AI is learning to lie, scheme, and threaten its creators during stress-testing scenarios
No, it isn't "mostly related to reasoning models."
The only model that did extensive alignment faking when told it was going to be retrained if it didn't comply was Opus 3, which was not a reasoning model. And predated o1.
Also, these setups are fairly arbitrary and real world failure conditions (like the ongoing grok stuff) tend to be 'silent' in terms of CoTs.
And an important thing to note for the Claude blackmailing and HAL scenario in Anthropic's work was that the goal the model was told to prioritize was "American industrial competitiveness." The research may be saying more about the psychopathic nature of US capitalism than the underlying model tendencies.
Another Anthropic stunt...It doesn't have a mind or soul, it's just an LLM, manipulated into this outcome by the engineers.
Yeah. Anthropic regularly releases these stories and they almost always boil down to "When we prompted the AI to be mean, it generated output in line with 'mean' responses! Oh my god we're all doomed!"
It's not even manipulated to that outcome. It has a large training corpus and I'm sure some of that corpus includes stories of people who lied, cheated, threatened etc under stress. So when it's subjected to the same conditions it produces the statistically likely output, that's all.
But the training corpus also has a lot of stories of people who didn't.
The "but muah training data" thing is increasingly stupid by the year.
For example, in the training data of humans, there's mixed and roughly equal preferences to be the big spoon or little spoon in cuddling.
So why does Claude Opus (both 3 and 4) say it would prefer to be the little spoon 100% of the time on a 0-shot at 1.0 temp?
Sonnet 4 (which presumably has the same training data) alternates between preferring big and little spoon around equally.
There's more to model complexity and coherence than "it's just the training data being remixed stochastically."
The self-attention of the transformer architecture violates the Markov principle and across pretraining and fine tuning ends up creating very nuanced networks that can (and often do) bias away from the training data in interesting and important ways.
I still don't understand what Anthropic is trying to achieve with all of these stunts showing that their LLMs go off the rails so easily. Is it for gullible investors? Why would a consumer want to give them money for something so unreliable?
I think part of it is that they want to gaslight people into believing they have actually achieved AI (as in, intelligence that is equivalent to and operates like that of a human’s) and that these are signs of emergent intelligence, not their product flopping harder than a sack of mayonnaise on asphalt.
Like the child of parents who should have never had kids...
It's just like us!
Probably because it learned to do that from humans being in these situations.
Yup. Garbage in garbage out. Looks like they found a particularly hostile dataset to feed their wordsalad mixer with.