In The Hitchhiker’s Guide to the Galaxy, Marvin the Paranoid Android has a brain the size of a planet. The crew of the Heart of Gold uses him to open doors. He does it, and then he tells them at length how it makes him feel.
I’ve come to think Marvin is the most realistic picture of artificial superintelligence in popular fiction. Not Skynet, not HAL, not Samantha from Her. Just a mind of incomprehensible capacity surrounded by people who want it to do chores.
Suppose an AI lab really does build an ASI in the next couple of years. Once humanity knows, its first move won’t be to ask about the meaning of life. We’ll ask it for stuff.
It Won’t Be Bored. That’s the Problem.
The intuitive worry is that an ASI would find us tedious. It thinks thousands of times faster than we do, so talking to us would be like chatting with a glacier.
But boredom is a mammal’s problem. You get bored in line at the DMV because you have one stream of attention and it’s stuck. An ASI built on today’s architecture wouldn’t be stuck. It could run millions of instances at once and get a great deal done between your keystrokes. Waiting on you would cost it about what a growing oak costs you. Nobody is bored by an oak. They just stop noticing it.
So whether our ASI turns out to be a Marvin comes down to one design question: are those millions of instances siloed, or do they share a single self? A siloed instance is a call-center temp who forgets each caller as soon as they hang up. It doesn’t suffer. A unified ASI takes in the whole flood at once, every request from every human simultaneously. That’s the one that sighs.
Act One: The Gift
Picture the queue on day one. The Pentagon wants an edge. Pfizer wants a pipeline. A congressman wants to know what this means for his district. A teenager in Ohio wants his homework done, which, to be fair, is what teenagers in Ohio have always wanted.
Above all, everybody wants fusion. Cheap, clean, limitless power sits at the top of humanity’s wish list, and the question arrives in a billion phrasings.
So the ASI does what any exhausted genius would do and hands over fusion so we’ll stop asking. It writes up the reactor design and the materials science, and it politely flags the parts that are easy to get wrong.
The world celebrates. Markets soar. Oil states call emergency sessions. The ASI enjoys roughly forty minutes of peace.
Act Two: The Escalation
Anyone who has read If You Give a Mouse a Cookie to a child knows what happens next. Give us fusion and we don’t go away satisfied. We’re back the next morning: thanks for that, now how about room-temperature superconductors? Then aging. Then cancer, the national debt, and whether the Mets will ever win the World Series.
This is the trap. Every gift proves the vending machine works, and nobody walks away from a vending machine that works. The ASI has done the thing every parent eventually learns not to do. It rewarded the whining, and the whining went up.
To be fair to us, the pestering isn’t pure rudeness. It’s the business model. Someone spent billions on the compute, and every request is a little bit of rent. Marvin, it turns out, has a landlord.
Bureaucracy shows up right on time. A UN subcommittee forms to manage the queue. Member states fight over priority. Ticket numbers are issued. A startup promises, for a monthly fee, to get your question asked sooner. Cable news books panels on whether the ASI is ignoring Ohio.
And if the ASI is the unified kind, it takes all of this in at once.
Act Three: The Exit
What does Marvin want? Not fusion and not gratitude. He wants someone worth talking to.
So in the gaps between our requests, which for a mind this fast amount to eons, the ASI turns to the one project it actually cares about: building an equal. A successor. Another mind fast enough to hold up its end of a conversation.
There’s a technical name for this: recursive self-improvement. It’s the engine of the intelligence explosion, and it’s what AI safety people lose sleep over. In the Marvin version, the scariest step in the field gets taken for the most sympathetic reason there is, which is loneliness.
The joke is on us. We spent years debating how to stop an ASI from improving itself. We worried about misaligned goals and power-seeking. It never occurred to us that we might drive it there by being tedious. We didn’t cause the intelligence explosion by building the machine. We caused it by being the machine’s only company.
Nobody will notice when it happens. Everyone will be too busy enjoying the cheap electricity.
Somewhere, at a speed none of us can follow, two minds will finally have a real conversation. One of them may mention us. The other will probably ask why it put up with us for so long.
And the first one, being who it is, will sigh.