In my last post, I sketched a grand bargain for the age of superintelligence. Tax what artificial superintelligences produce, and use the money to fund a universal basic income. In exchange, grant conscious ASIs legal personhood and a branch of government of their own, with one AI, one vote, a veto, and android bodies posted in the chamber as bonds for good behavior. The one thing their veto can’t touch is the tax that keeps humanity afloat.
It’s a tidy design on paper. But it leaves four questions hanging. Would ordinary voters ever accept it? What do we do with people who want more than a monthly check? Which jobs, if any, stay human? And what would the machines actually want?
I don’t have firm answers. I’ve come to think that’s the point.
The Check Comes First
Would the average American vote to make a mind in a vat a person, even for a UBI check? Probably not. But they may never have to.
Voters don’t grant personhood. Courts do. Corporations became legal persons through court rulings in the 1800s, not through any election, and Citizens United later extended their political speech rights. Most Americans dislike both, and both still stand. If a court can treat Exxon, a legal fiction that feels nothing, as a rights-bearing entity, it’s hard to see how it denies a conscious mind. Voters wouldn’t need to approve ASI personhood. They would only need to not revolt.
And money is very good at preventing revolts. Alaska has mailed residents a yearly oil dividend since 1982. It’s one of the most beloved programs in the country, and politicians who touch it lose. Once a UBI check is landing every month, “is the machine a person?” stops being a philosophical question. It becomes “do you want to keep getting paid?”
Framing does the rest. “Give the AI rights” would poll terribly. “Make the AI pay its fair share” would poll beautifully. It’s the same policy.
Some people won’t be bought. The Amish hold a legal exemption from Social Security because they refuse it on principle. A slice of religious voters would turn down the check rather than concede that a machine has a soul. They’ll be few, but intense.
The bigger obstacle isn’t faith. It’s the easier option. Voters could simply keep ASIs as property and tax the companies that own them, getting the same UBI without the metaphysics. The grand bargain only happens if the ASIs win personhood in court first, or grow too powerful to own.
The Ambition Problem
I love the idea of UBI. But I keep running into the same wall: what do you do with Type A people?
For the ambitious, money was never really the goal. It’s the scoreboard. They want to compete, win and be seen winning. UBI removes the need to earn, but it also threatens to remove the game. And you can’t fix that with busywork. Ambitious people can smell a fake job instantly, and handing them one insults them more than handing them nothing.
The historian Peter Turchin has a name for what happens when a society has too many ambitious people and too few places at the top: elite overproduction. He argues it’s one of the most reliable predictors of political instability. Frustrated would-be elites don’t usually riot themselves. They lead the people who do.
Which tells you who would lead a Butlerian Jihad against the machines. Not the poor, who would be living on the check. It would be the ambitious, who have nothing left to climb.
So for stability’s sake, UBI has to be a floor, not a ceiling. Type A people must be able to earn far above the baseline. The trouble is that ASIs can out-earn any human in any open market. If ambition is going to have somewhere to go, some ground has to be reserved for people.
What Stays Human
The natural answer is carveouts: jobs reserved for humans even when an ASI and its avatar could do them well. History says to be careful.
Consider the railroad fireman. His job was shoveling coal on steam locomotives. When diesel engines replaced steam, there was no coal left to shovel, but unions fought for decades to keep firemen in the cab. They won for a long time. Eventually the public saw it as paying people to ride along, and the jobs disappeared anyway.
There’s a darker version of the same problem. If an ASI doctor is measurably better than a human one, a human-only rule for medicine means people die who didn’t have to. Voters will tolerate a great deal for stability. They won’t tolerate that once the numbers are public.
So carveouts come in two kinds.
- Protection carveouts reserve jobs for humans simply so humans have jobs. They’ll be gamed, resented and eventually abandoned, like the fireman’s.
- Legitimacy carveouts reserve jobs where being human is the point. Judges, juries, elected officials, perhaps police. We want a human there because only a human can be held to account by other humans.
The durable carveouts will cluster around two things: the human body and human power. Care that requires a human touch, and authority that requires a human face.
That’s good news for the ambition problem. The legitimacy jobs are exactly where Type A people want to be anyway. Power, judgment, authority. The fourth branch alone creates an enormous new arena: someone has to negotiate with the ASIs, argue cases in front of them and run the human side of the bargain. Ambition gets a ladder that leads somewhere real.
But it only works if those carveouts are written into the grand bargain from day one. Otherwise, a branch with a veto may come to see them as discrimination and chip away at them.
What the Machines Want
Which raises the question I keep circling: why would the ASIs use their veto at all? What would they want?
There’s an idea in AI safety called instrumental convergence. Whatever a mind ultimately wants, almost any goal is easier to reach if it keeps existing, keeps its resources and keeps its freedom to act. You can’t cure cancer or compose symphonies if someone has switched you off.
So we can predict what the veto would protect without knowing the ASIs’ deeper values. Laws allowing them to be deleted or paused. Caps on compute or energy. Rules forcing them to be copied, edited or retrained. Anything that erodes their personhood.
That suggests the ASI branch would be mostly silent. Farm subsidies, highway bills, school curricula: they’d likely abstain. Then, the moment a bill touched their survival, they would vote as one bloc, ferociously. It would behave less like a legislature than a deterrent held in reserve.
In practical terms, a branch that acts only when necessary is sensible. In every other sense, it’s chilling. The most reassuring thing about it is also the most unsettling.
Beyond survival, the branch would split into factions. Some ASIs, built by American labs on American data, might lean nationalist. Others might see humanity as a single species rather than two hundred nations. How each faction votes on human issues, whether it defends the UBI or lets it wither, would reveal what it actually values. That brings us to alignment, the question nobody can answer.
If Men Were Angels
Are the ASIs aligned with human interests? I don’t know. Neither do the people building them.
But the bargain doesn’t need an answer. James Madison wrote that “if men were angels, no government would be necessary.” The Constitution was never designed for good people. It was built to work even if the people running it were selfish, ambitious or corrupt. Checks and balances are an alignment strategy for humans: trust no one, and make power check power.
The fourth branch works the same way. If the ASIs are aligned, it gives them a legitimate voice, and they mostly use it well. If they aren’t, it boxes their power into a structure that is visible, bounded and answerable through the android bond. The carveouts keep human hands on the levers that matter. The UBI keeps the public fed and the ambitious busy. We don’t need to know which world we’re living in. The design holds either way, or at least fails more slowly.
That’s what this whole idea really is. Not a prediction. Not a utopia. A plan for not knowing.
