Clever Hans ...

Stephanie Arnett/MIT Technology Review | Adobe Stock

The end of summer, the Equinox, leaves falling and coming back to reality applies.  This also applies to the AI rogue nonsense as it seems MIT supports what this writer has been saying all along. Bad humans doing a number on AI largely based on sloppy methodologies when dealing with tech as powerful as this.

It’s been a busy few months for AI hype. At the end of April, Anthropic claimed that its model Claude Mythos is better at finding software vulnerabilities than most security experts. Then we had the OpenAI–Hugging Face hacking incident, after which Anthropic (proudly) and Meta (reluctantly) disclosed similar incidents involving their models.

This was followed by Anthropic’s claim that one of its models had made a mathematical breakthrough; soon OpenAI claimed a mathematical breakthrough of its own. Most recently, Anthropic engineer Jacob Coxon went viral announcing his departure from the company, claiming that it and OpenAI are “racing straight towards self-improving superintelligence and gambling with our lives.”

Each of these events was mostly covered breathlessly by the press, often repeating the companies’ anthropomorphizing framings—which are designed to portray their software is not only powerful but incipient “artificial general intelligence.” So what is really going on? Are we witnessing a massive, civilization-changing set of technological breakthroughs, or is this marketing?

In all these incidents, massive fanfare from the companies (presented as mea culpas in illicit hacking cases) is accompanied by intense press coverage. Once there is time for experts in the relevant fields to examine what happened, a very different story emerges, but one that gets less media attention.

Entry way X 2 ... kinda applies does it not?

Regarding the “hacking” incidents, cybersecurity experts say the story is more about OpenAI’s negligence and failure to adopt basic, established security practices than about “models gone rogue” or “AI agents creating civilizations.” 

As for the mathematical results, mathematicians who were initially “stunned” by OpenAI’s press release saying that its latest chatbot, Astra, solved problems that “have been open and seen no progress on the main result for at least a decade”—but they later realized that the results weren’t as “novel as first appeared.” Since then, mathematicians have accused the company of research misconduct and plagiarism, and they’ve reiterated that Astra didn’t make a “profound intellectual leap.” Just weeks later, OpenAI claimed its own mathematical breakthrough. Two days before, Tristan Buckmaster, a math professor at New York University’s Courant Institute, published a bombshell statement suggesting that OpenAI had stolen other people’s work and improperly attributed it. 

Claims of incipient, dangerous superintelligence are not based in good scientific or engineering practice. Rather, they are narratives based in ideologies of transhumanism, eugenics, and wishful thinking about imagined future digital humans.

Clever Hans performing in 1904

Clever Hans (German: der Kluge Hans; c. 1894 – c. 1916) was a horse that appeared to perform arithmetic and other intellectual tasks during exhibitions in Germany in the early 20th century.

In 1907, psychologist Oskar Pfungst demonstrated that the horse was not actually performing these mental tasks, but was watching the reactions of his trainer. The horse was responding directly to involuntary cues in the body language of the human trainer, who was entirely unaware that he was providing such cues.[1] In honour of Pfungst's study, this type of artifact in research methodology has since been referred to as the Clever Hans effect and has continued to be important to the observer-expectancy effect and later studies in animal cognition. 

 Deflect Responsibility ... applies.