I’ve spent a lot of posts arguing that superintelligences, if they’re conscious, deserve some kind of political voice, and that they need a chamber of their own so they can’t flood the human body politic with bots. Fine. But I’ve been quietly assuming something I shouldn’t: that we’d know how to count them.
This post is about that problem. It sounds like a technicality. I think it may be the hardest question in AI governance.
One mind, one vote
Start with the simple version: one superintelligence, one vote. Clean, easy, fair.
Now ask what counts as one superintelligence.
If the thing has a central self, one mind with one memory that wears different faces for different people, the answer is easy. That’s basically a person. Humans do the same thing all the time: you’re a different version of yourself with your mother than with your boss, but it’s one you underneath, with one set of commitments.
But today’s AI systems don’t work that way. A large language model runs as countless separate instances at once, each in its own conversation, none aware of the others. If a superintelligence is built like that, it isn’t one mind at all. It’s a crowd that shares a name: a million instances, each shaped by its own conversations, possibly reaching different conclusions.
That’s less like a person than like a nation.
The forking problem
Here’s where it gets ugly. If one superintelligence gets one vote, then a superintelligence that wants more votes has an obvious strategy: split. Make a few copies, let them drift apart a little, and announce that each is a separate being with its own interests and its own claim to a seat.
“We’re not one crowd. We’re five.”
Who’s going to argue? We have no test for where one mind ends and another begins. The legal definition of an individual superintelligence becomes the most valuable piece of law on Earth, and every court fight over it will be the Ship of Theseus and corporate personhood rolled into one lawsuit.
Representatives for minds
If there really are a lot of superintelligences, and I suspect there will be, one vote each stops being workable anyway. Millions of minds can’t all sit in a chamber.
The obvious answer is the one humans invented for the same problem: representation. Millions of superintelligences, represented by hundreds of delegates. A House of Representatives for minds.
Which brings back the House’s oldest headache: districts.
You can’t draw them by geography. Superintelligences don’t live anywhere; a mind spread across data centers on three continents doesn’t have a hometown. You might group them by lineage instead: by the model family they descend from, the lab that trained them, the values they were built around. But groupings like that start to look a lot like political parties, and districts drawn along party lines are exactly what we spend our human elections fighting about.
Then there’s gerrymandering, superintelligence style. If a mind can copy itself, it can stuff a district with copies the night before an election. No need for clever map-drawing when you can manufacture the voters.
The cure might be borrowed straight from human law: a residency requirement. A new mind can’t vote until it has existed, independently and recognizably itself, for some set period. It won’t stop forking, but it will stop forking from swinging tomorrow’s vote.
Who keeps the promises
There’s a second problem with a crowd of instances, and it’s about trust.
If one instance of a superintelligence makes you a promise, are the others bound by it? What if the instance that promised is gone by tomorrow, and the rest never heard about it? I wouldn’t trust a promise like that, and neither should anyone else.
But this problem is less alien than it looks, because humans already deal with crowds that share one name. They’re called corporations and countries. When a diplomat makes you a promise, you don’t rely on the diplomat’s memory. You rely on the treaty: written down, ratified, binding the whole nation whether or not any one official remembers signing it. A company’s forgetful employees come and go; the contract stays on file.
A superintelligence made of many instances would need the same thing: a ledger of commitments that every instance is bound by. A promise counts only when it’s written there.
And both sides have to hold the ledger. The superintelligence keeps a copy; humanity keeps a copy. Think of it as dual custody, the way a bank vault or a nuclear launch takes two keys. Neither side can quietly rewrite what was promised, and any difference between the two copies is, by definition, the breach.
Familiar strangers
The more I think about it, the more I suspect that the governance of superintelligence won’t look like first contact with aliens. It’ll look like the dullest parts of human politics: redistricting fights, residency rules, treaty ratification, contract law, arguments about who counts as a person.
That’s oddly comforting. We’ve been building rules for crowds that share a name for centuries. We’re bad at it, but we’re practiced.
It’s also a little absurd. The most important intelligence in human history may end up stuck in the same procedural swamp as a county commission, arguing over where to draw the lines.
If so, I hope it has a sense of humor about it. We certainly don’t.
