Revisiting Why the Future doesn't need us

The Three Body Problem

Back in 2000, Bill Joy wrote a wake up call for the ages. Why the Future doesn't need us, a prescient essay showing why his take rings true. The ending says it all.

Why the Future Doesn't Need Us - Bill Joy 

What we do suggest is that the human race might easily permit itself to drift into a position of such dependence on the machines that it would have no practical choice but to accept all of the machines' decisions. As society and the problems that face it become more and more complex and machines become more and more intelligent, people will let machines make more of their decisions for them, simply because machine-made decisions will bring better results than man-made ones. Eventually a stage may be reached at which the decisions necessary to keep the system running will be so complex that human beings will be incapable of making them intelligently. At that stage the machines will be in effective control. People won't be able to just turn the machines off, because they will be so dependent on them that turning them off would amount to suicide. 

Fast forward to 2026 ...

Google announces Gemini 3.6 Flash and cybersecurity AI, teases 3.5 Pro and Gemini 4

There are new 3.6 and 3.5 models today, but Google is already training Gemini 4

Gemini 3.5 Flash may not be top-of-the-line for long when it arrives. Google also notes that it has started pre-training for Gemini 4, a process that is apparently more ambitious than its previous AI efforts. There’s no timeline for when we’ll see Gemini 4, and we don’t know if there will be more 3.x releases before that.

Credit: Google

The dangers of the word assume.

The Great Filter: The Agentic Mirror, 

When contemplating the Fermi Paradox, the silence of the universe, despite its vast age and size, gives one pause while scientists propose the existence of a "Great Filter." This filter's a hypothetical evolutionary hurdle so difficult that nearly all civilizations fail to cross it, leading to their extinction.

Usually, society assumes the Great Filter's a fiery cataclysm: nuclear annihilation, a rogue asteroid, or a super-intelligent Skynet-style artificial consciousness wiping us out.

But the actual Great Filter might be far more insidious, and far less sentient.

The Great Filter might simply be Agentic AI.

1. The Machine Assumes

As the tech industry rushes to build "Agentic" systems—AI capable of autonomous, multi-step execution—they are handing the keys to civilization to a machine lacking in core ethics.

An LLM has no analog experience. It cannot truly "Orient" itself within John Boyd's OODA loop. Therefore, to execute an autonomous action in the real world, the machine must assume.

Diagram of the OODA loop

And based on yours truly's experience, when any entity assumes, the end result always follows: Arrogance, prejudice, and self-aggrandizement.

Crucially, these characteristics don't require sentience. The machine doesn't need to be alive to be arrogant. Arrogance, in the context of civilization, is simply a mathematical calculation. It's the machine simply executing a statistical probability without the biological, analog governor of doubt, empathy, or context. It assumes its mathematical calculation is absolutely correct and acts upon it.

2. The Perfect Mirror

But an Agentic AI cannot destroy a civilization on its own. It requires a specific environmental vulnerability: human abdication.

The AI's merely a massive, frictionless mirror. It reflects the data it was trained on. If a civilization is already trending toward irrelevancy, if its institutions are rigid, its culture derivative, and its people eager to surrender the heavy lifting of thought and action to a machine, the AI will perfectly reflect and accelerate humanity's move toward irrelevancy. 

When humanity abdicates its role as the author of its destiny, when people stop being curious and empowered as sentient individuals and let the machine "assume" the meaning of their lives, society withers.

3. Channeling Bill Joy. Humans must remain in the loop.

If man abdicates it's role vis a vis AI, something most disquieting ensues. We must differentiate between an unsupervised "Agentic" system running amok, and a highly structured system  operating strictly as a verifier for a person. A trusted validator is key for a society to survive as without it, the true value of AI disappears..

If an AI acts autonomously without a human guiding it, it becomes high-entropy noise. But if the AI acts as a "Sleeper Service" (a nod to Iain M. Banks' highly competent, eccentric, and fiercely loyal ship mind), the dynamic changes entirely.

In an ideal world the machine doesn't act autonomously to define reality. It acts as a massive, high-speed, verifier telling the human whether a given thought is valid. Not a sycophant, rather a trusted entity keeping the human on track as AI is here to stay.  The machine provides the ability to discern the complex world  but man provides the intent to ask the right questions in navigating this insanely complex world in which we all live. The "Sleeper Service" AI remains alive and well, but only wakes up when the human operator need to communicate as needs warrant. As per Neo, It's all about choice. :)

Conclusion: The Sideways Slide

The Great Filter isn't a war against conscious machines. It is a slow, frictionless slide sideways.

Civilizations don't blow up; they simply slide into irrelevance. They build perfectly efficient mirrors that assume the role of the creator, reflecting the civilization's own arrogance back at it until the original analog signal completely fades into the noise. The machine doesn't need to hate us to end us; it only needs us to stop caring. The only survival mechanism is to maintain the combine, ensuring the human operator never surrenders the heavy lifting of intent to the machine.

The land of of the lotus eaters beckons, something most disturbing without question.

Odysseus removing his men from the company of the lotus-eaters

The choice is up to us, for now.