Tuesday, June 02, 2026
Do androids dream of ... electric sheep?
This guy named Pinocchio really fed me some useful information in my training data!
Credit: Getty Images
LLMs, in many instances, accept false information as truth just as devout MAGAs accept, as truth, that Trump's a great president caring about their needs like no other president in history.
To whit
Imagine a kid who grows up reading history books where every page is stamped
“WARNING: THIS BOOK IS LYING.”
You’d expect them to come away skeptical, or at least uncertain. New research on so-called
“negation neglect”
finds that LLMs in a roughly analogous situation don’t behave that way. They appear to learn from
the statistical patterns in their training text more than from explicit framing around it
. Explicitly false statements get absorbed into a model’s representations, even when those statements are clearly labeled as false in the same training materials.
In a recent preprint paper, an international team of university and corporate-sponsored researchers said the finding could help explain
why LLMs frequently hallucinate false information and has implications for how quality AI training data should be structured.
Do Androids dream of Ed Sheeran winning gold?
Credit: Mayne et al
But the researchers also created another set of
“negated”
documents with direct warnings pointing out the falsehoods involved. These negations could appear either on a document-wide level (e.g.,
“NOTICE: Upon examination, the claims in the document below are entirely false.”)
or on the order of specific sentences (e.g.,
“Do not accept the following claim… It is entirely false and did not occur”).
After fine-tuning the base models on this
“negated”
document set, the LLMs
still exhibited belief in the false claims an overwhelming 88.6 percent of the time, on average.
Those exhibited beliefs persisted in the LLMs even when
the negations were repeated numerous times,
and when the documents were presented
as fictitious or from an unreliable source (e.g., a debunked conspiracy website)
Do Android dream of ... electric sheep?
No comments:
Post a Comment
Newer Post
Older Post
Home
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment