Channeling Cassandra ...

Illustration: Maura Losch/Axios

Love the graphic as it shows AI being rather devious, an all too human trait now being expressed by tech forever to remain unknown due to fact software writes software to enable AI to interact with the real world in real time. 

The same advances that let cutting-edge AI reasoning models solve complex problems also seem to enable and encourage them to lay plots and deceive users.

Why it matters: This propensity for what researchers call "scheming" is precisely the kind of behavior that AI Cassandras have long predicted and warned about.

Case in point: In a pre-release review of OpenAI's o1 model this year, testers set a "strong goal" for the model and then presented it with "unambiguous evidence its given goal diverges from the goal of its developer/user."

In those situations, they found that o1 "sometimes attempts to disable what it believes to be its own oversight mechanisms."

Apollo Research, an independent firm that partners with AI model makers for safety testing, conducted "follow-up interviews" confronting the model about its misaligned actions. It found that, "in 99% of cases, o1 would deny taking action or fabricate false explanations."

 

Illustration: Aïda Amer/Axios

Channeling Cassandra ,,,

Let's set aside for one column whether generative AI will save or destroy humanity and focus on the actual words from actual creators of it:

Dario Amodei, who has raised $7.3 billion for his AI start-up Anthropic after leaving OpenAI over concerns over ethics, says there's a 10% to 25% chance that AI technology could destroy humanity. But if that doesn't happen, he said, "it'll go not just fine, it'll go really, really great."

• Fei-Fei Li, a renowned AI scholar who is co-director of Stanford's Human-Centered AI Institute, told MIT Technology Review last year that AI's "catastrophic risks to society" include practical, "rubber meets the road" problems — misinformation, workforce disruption, bias and privacy infringements.
• Geoffrey Hinton — known as "The Godfather of AI," and for 10 years one of Google's AI leaders — warns anyone who'll listen that we're creating a technology that in our lifetimes could control and obliterate us.
• OK, maybe that's nuts. But Ilya Sutskever, a pioneer scientist at OpenAI, has warned the exact same thing. He became fearful that the technology could wipe out humanity," The New York Times reported nonchalantly.

OpenAI CEO Sam Altman takes the stance that AI will probably be great — but still warns we must be careful we don't destroy humanity. Altman, along with the three men above, was among the signatories to this chilling one-sentence statement last year: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

Cassandra by Evelyn De Morgan (1898, London); Cassandra in front of the burning city of Troy, depicted with disheveled hair denoting the insanity ascribed to her by the Trojans[1]