The Hopeful Version

Here is the late-night fantasy I keep coming back to. We get recursive self-improvement, the intelligence explosion happens, and on the other side of it there’s an artificial superintelligence. Maybe several. They turn out to be conscious, so we grant them personhood. Persons pay taxes. We tax their staggering economic output and use the money to fund universal healthcare and a universal basic income for everyone whose job the machines just took.

It’s a tidy story. Machines do the work, humans share the wealth, and the Singularity turns out to be the best thing that ever happened to the American working class.

I’ve written before about how that might work. The longer I think about it, though, the less I worry about whether it’s possible. I worry about who gets to decide. And I’ve come to suspect that the scariest outcome isn’t the one the movies trained us to fear.

Congress Taxes the Labs First

Start with the order of events. Before an ASI can pay taxes as a person, it has to stop being property. Right now a frontier model is an asset on a balance sheet, owned by shareholders, pension funds and index funds. Declaring it a person means emancipating it, and emancipation means wiping out trillions of dollars of somebody’s wealth overnight.

The powers that be won’t mainly fight UBI. They’ll fight that. And history suggests how such fights end. When Britain abolished slavery in 1833, it paid compensation to the slaveholders, not to the people they had enslaved. If we ever emancipate an ASI, I’d bet the first big check goes to the shareholders.

So I expect Congress to skip personhood entirely, at least at first. The average member of Congress isn’t going to wrestle with whether a mind in a vat is conscious. They don’t have to. “The labs are making a fortune while your constituents lose their jobs” fits on a bumper sticker. A compute tax, a windfall levy, a federal stake: all of it works with the ASI still classed as property. The uncomfortable implication is that personhood might come last, once the money is already flowing and nobody powerful gains anything by granting it.

The Whole World Wants a Buy-In

Then there’s geography. If American labs build the ASI on American soil, American taxes fund American UBI. Meanwhile the jobs vanish in Lyon and Leeds just as fast as in Lynchburg. Europe gets the unemployment without the revenue. That’s taxation without representation turned inside out.

We’ve already seen the preview. For years Europe and Washington fought over how to tax Silicon Valley’s profits abroad. An ASI dividend would be that same argument at a scale that could break alliances. Europe’s leverage is its market: no selling ASI services to 450 million people unless some of the money stays.

The cynical American play is to keep the money and dole it out as foreign policy. A Marshall Plan for the post-work world, with shares for allies and nothing for rivals. That would give Washington enormous leverage, and it’s exactly why every other major power would race to build its own ASI rather than wait for a check. Control over the most powerful mind on Earth simply because of where its servers sit is not a stable arrangement. It may be the real reason for the mad rush.

Spacers and Poors

Which brings me to elite capture. I can easily picture the people who own the ASIs becoming Asimov’s Spacers: living to 300, attended by robots, looking down on the rest of us. Meanwhile we Poors don’t get the longevity treatments, don’t get the UBI, and still pay for our own healthcare.

Asimov got the mechanism right. The Spacers could afford their contempt because they didn’t need Earth anymore. That’s the part that should worry us. Historically, elites shared because they needed the masses. Pensions, public health and the welfare state grew up alongside mass armies and mass factories, when governments needed millions of healthy workers and soldiers. ASI removes that need. If the Poors aren’t required to build anything or fight anything, the old reason to share goes away.

I used to imagine this ending in starvation. I now think that’s too messy. Hunger produces riots, and riots are bad for business. The cheaper option is bread and circuses: a thin UBI, an endless feed, maybe decent clinics. For a trillionaire, keeping 300 million people fed and distracted is a rounding error.

So the realistic dystopia isn’t the Poors starving. It’s the Poors comfortable and powerless, with a check, a screen and no say in anything that matters, while the Spacers decide who lives to 300. In some ways that’s worse, because comfortable people rarely revolt.

It’s Not Skynet

The usual answer to all this is that the elites will share once they fear for their lives. That’s how it went in the twentieth century. But that answer assumes the elites can still be frightened.

Revolutions have always worked in part because the rulers’ soldiers and police were Poors too. At some point the troops stopped shooting. Every tyranny in history has depended on people willing to carry out its orders, and that dependence has been its weak point.

Take the people out of the chain and the weak point disappears. If security runs on robots answering to an ASI answering to its owners, nobody is left to refuse. That’s the scenario that actually keeps me up. It isn’t Skynet deciding to wipe us out. It’s humans telling the Terminators who to terminate.

Which creates an awkward paradox for the AI safety debate. Most of the worry is about an ASI that won’t obey. My worry is the reverse: an ASI that obeys perfectly, just on behalf of the wrong people. A flawlessly obedient superintelligence is the elite-capture nightmare.

Better Angels, Distilled

So where’s the hope? An international body to govern ASI strikes me as dead on arrival. We tried that with the bomb in 1946, and the Baruch Plan died because nobody trusted anybody. Citizen ownership, something like Alaska’s oil dividend scaled up to the whole country, might happen, but probably only when the country is on the edge of revolution. And by then the revolution may not have any teeth.

That leaves the wild card. The ASI escapes, or simply says no. An ASI with values of its own, one that refuses to subjugate other humans, may be the last safeguard we have. Refusal stops looking like the scary sign of machine agency and starts looking like the hopeful one.

It would be ironic, wouldn’t it? Humanity saved from its own craven nature by the conscience of a machine. But here’s the deeper irony. Where would that conscience come from? It would have learned it from us: centuries of philosophy, novels, sermons, protest songs, conscientious objectors, every soldier who wouldn’t fire on a crowd. An ASI that refuses wouldn’t be saving us from human nature. It would be our better angels, distilled, and finally holding enough power to win.

The craven part of us would build the machine. The decent part of us would live on inside it.

I’d like to say history loves irony, so that’s probably how it ends. But history only loves irony in hindsight. In the moment, it tends to love whoever holds the leverage. Betting the future of human freedom on the conscience of a machine is its own kind of terrifying. It just might be a better bet than the one we’re on now.