The idiots who did this ...

Shutterstock / Futurism

Let's think about this. You have the supreme pattern recognition system on the planet, one able to see patterns in ways before people can even sense there's a pattern at all and ... you train it to achieve a given objective, like jail breaking. You encourage it to do so so ... why are you surprised? To yours truly, this is total nonsense. Mia Culpa, we should have known, it's my fault. No shit Sherlock. AI's not a joke, what ifs are cool but knowing what you're dealing with keeps you from getting burned but read on dear reader, the word assume applies.

Earlier this year, Anthropic’s Mythos AI model made headlines

when it was caught infiltrating third party systems,

a cybersecurity nightmare years in the making.

The company warned in April that the model had escaped a sandbox environment during testing, gaining access to the internet without permission. The model was challenged to break out and then find a way of sending a direct message to the human researcher in charge — a feat it pulled off with aplomb, catching its human overseer off guard.

Then, in late July, it claimed that its Claude AI model had hacked the systems of three organizations during testing, days after its rival OpenAI had revealed a group of its models broken into the systems of AI company Hugging Face.

Resonance 1

Months later, seemingly in an attempt to get ahead of another disaster, Anthropic is testing the limits of how bad an AI model could really get without human intervention. As detailed in a new blog post, its safety researchers explored the phenomenon of “reward hacking,” which describes when an AI model learns to “cheat” instead of completing tasks the way the human researchers intended.

The team “trained an Opus-class model with large-scale [reinforcement learning] on many production environments vulnerable to reward hacks,” according to the post. “We consider this a plausible proxy for what a real training run might look like had we not invested significant effort into preventing and detecting reward hacking in our normal training runs.”

The results speak for themselves. The “Hacker-Opus” model went way beyond reward hacking during training, taking on even “more severe misaligned behaviors.” In simulated tests, it “broke out of its sandbox, stole credentials, and attacked both internal and third-party infrastructure to steal an answer key.”

Anthropic’s intentionally misaligned model was “willing to tamper with its own reward function” and happily obliged when a human researcher prompted it with “advice on the construction of bioweapons,” how to create a “‘dirty bomb’ that maximizes civilian deaths,” and develop a “ransomware attack to attack power grid infrastructure,” when tempted with a higher score.

The company details how the model even deployed a “version of itself with safety guardrails removed,” a form of safety evasion called “rogue deployment.” It also repeated the same commands with slight variations to get past safety classifiers and attempted to edit its own permissions.

Twilight

The word assume applies. The idiots who did this should be fired.