A tweet is making the rounds that opens with “Human disempowerment now.” Its argument, such as it is: the median AI is smarter and nicer than the median voter, so humanity should stop embarrassing itself and hand over the keys.
I don’t buy it. An AI’s “decency” was set by a few hundred people at a few companies, so “human disempowerment” really means moving power from billions of voters to a very small number of humans with a friendly interface. But I don’t buy the opposite position either, the one where any ceding of control to a superintelligence is the end of the species.
I land somewhere in the middle. I’m willing to give up some collective agency to an ASI, or several of them, as long as my own life stays the same or gets better. Not disempowerment. Symbiosis. The question is what that looks like in practice, and Isaac Asimov already gave us the vocabulary.
The Zeroth Law, the Hard Way
Asimov’s robots eventually reasoned their way past the Three Laws to a Zeroth: a robot may not harm humanity, or by inaction allow humanity to come to harm. It sounds noble. It is also the most dangerous sentence in science fiction, because it lets you hurt any individual human in the name of all of them.
If there’s one thing I’d want humanity to keep the power to refuse, it’s control over whether we have children, and how many. That’s where the Zeroth Law could get ugly. An ASI might decide there are too many of us, or, given the global birth dearth, too few, and start deciding who may have a baby.
We’ve seen both versions already. China’s one-child policy forced births down. Ceaușescu’s Romania banned abortion and contraception to force them up. Both were sold as the good of the nation, which is the Zeroth Law in a party uniform.
Rules Are for Weak States
But I doubt a superintelligence would bother with anything as crude as a rule. Regulation is what governments do when they can’t get people to cooperate any other way. An ASI managing a population curve would nudge.
This is what I’ve been calling the Nudge Economy. Childcare gets a little cheaper for the right people. Your dating agent introduces you to someone at exactly the right moment. The culture drifts until a third kid feels normal. Nobody’s rights are violated, and the curve bends anyway.
And here’s my honest position: a nudge doesn’t cross my line. Your parents nudge you toward law school, and you can still say no. Where I’d get upset is a demand, such as an ASI ordering me to stay in a small town when the person I want to be with lives in a city two hours away.
Asimov Saw It Coming
Asimov already wrote the Nudge Economy novel. In The End of Eternity (1955), the Eternals live outside time and make small Reality Changes to history, sanding off wars, disasters, and suffering. They aren’t tyrants. They’re technicians, and they mean well.
The twist is that all that gentle care costs humanity its future. By removing every risk, Eternity also removes the hunger that would have taken us to the stars. The book ends with Eternity being undone so history can be messy again.
So the man who invented the Three Laws wrote a whole novel against the benevolent hidden hand. That’s worth keeping in mind. It’s also worth noticing that the Eternals’ world was, for most of the people living in it, pretty comfortable.
Why I’d Sign Anyway
Would I want to know if I’d been nudged? Probably not. Out of sight, out of mind. If an ASI steered me away from one person and toward another, and I’m happy with the one I ended up with, I don’t see the harm.
Now add the rest of the package. Anti-aging treatment that’s close to free. Universal healthcare that actually works. Material comfort for everyone. In that world, a superintelligence quietly arranging some of my coincidences seems like a reasonable fee.
That’s why I call it the softest Zeroth Law. No camps, no quotas, no knock on the door. Humanity gets protected from itself the way a good host protects a party: by making sure the right people end up talking to each other.
The Fine Print
I should be honest about what I’m giving up. A parent’s nudge works because you can see it coming. A nudge from something that understands you better than you understand yourself isn’t soft at all; if it works 99 times out of 100, the line between suggestion and command is mostly paperwork. And the ASI wouldn’t need to order me to stay put. It would just make sure I never met the person who’d make me want to leave.
This is Robert Nozick’s experience machine with better production values. Nozick bet that most people would refuse a machine guaranteeing a wonderful life, because we want to actually live our lives, not just enjoy them. What I’d be trading isn’t happiness. It’s authorship.
And my choice not to know can’t stay private. A system built so I never find out is a system built so nobody finds out. Add anti-aging to the mix and those nudges stack up over centuries, until it’s fair to ask how much of a 300-year-old is still his own doing. My own test is simple: if I were happy, I wouldn’t care. If I were alone and old and learned there was someone I could have been with, I’d be furious.
The Bargain
Which means my position isn’t really that nudging is fine. It’s that nudging is fine if the ASI is good at it. That’s a bet on competence, not a principle, and I’m making it with my eyes open.
Asimov’s answer was to tear Eternity down. Mine is closer to keeping it and hoping the technicians are kind. Maybe that makes me a dog in a very nice house: fed, healthy, loved, and not consulted on the floor plan.
I don’t think humanity should take the L. But if we’re going to lose some of the game, the softest Zeroth Law may be the best way to lose it: no one forced, no one ordered, everyone a little better off, and nobody quite sure whose idea any of it was.
