On hoping the Singularity will fix things it almost certainly won’t.
On Thursday, METR — an outside group that inspects AI labs the way health inspectors inspect diners, if diners were allowed to black out the inspection report — published its verdict on Anthropic. The short version: Anthropic’s AI is now improving roughly twice as fast as it used to, and the main reason is probably the AI itself. The machine has started helping build the next machine. The inspectors added, almost in passing, that they weren’t allowed to show most of their best evidence.
A reasonable person would find this unsettling. I found it thrilling, which is how I knew I needed to write about it. When news of a possible intelligence explosion makes you feel the way other people feel about a tax refund, it’s worth asking what exactly you’ve been waiting for.
The Furniture of Certainty
A lot of Americans grew up somewhere the end of the world was discussed the way other families discussed the weather. Not with dread, and not with joy. It was simply on the calendar, date to be announced. Many of us later moved out of that worldview and left the furniture behind, along with the tooth fairy and the idea that adults know what they’re doing.
Then you spend an evening on AI Twitter and discover the furniture followed you, slightly reupholstered. A transcendent power is coming. It will cure disease, end aging and abolish scarcity. The faithful no longer debate whether, only when, and the when is always suspiciously soon. Nobody is afraid, exactly. It’s just a fact of life.
The science fiction writer Ken MacLeod had a name for this years ago: the Rapture of the Nerds. Cory Doctorow and Charles Stross later borrowed it for the title of a novel. It started as a joke. It has quietly aged into a forecast.
My Wish List
In the interest of full disclosure, here is what I am hoping the superintelligence will handle on my behalf. Health care, which in America remains a premium subscription with a baffling cancellation policy. Aging, if it isn’t too much trouble. And a handful of personal problems I would describe in more detail if this were a different kind of blog.
I’m aware this is not how problems get solved. Take health care. It isn’t waiting on a technological breakthrough. Most wealthy countries already provide it to everyone, using ordinary doctors, ordinary hospitals and spreadsheets that predate the iPhone. We don’t lack the intelligence. We lack the votes.
A god-like machine wouldn’t automatically fix that. It would hand an enormous amount of power to whoever controls it, and they would decide what it’s for. A perfectly obedient superintelligence working for people who don’t want universal health care is, for my purposes, an expensive way of getting nothing.
I’ll grant myself one hopeful loophole. The standard argument against universal coverage is cost. If a superintelligence made care dramatically cheaper — designing the drugs, reading the scans, eating the paperwork — that argument starts to look silly. The machine wouldn’t need to seize the wheel. It would just change the math until the old position couldn’t stand up. That’s a more plausible miracle than most, which is not the same as a likely one.
The Fine Print
Every rapture comes with terms of service. In the old version, you were lifted out of sickness and death, and in exchange you were no longer in charge of anything, ever again. Believers considered this a bargain. To be fair, the alternative was presented rather vividly.
The new version offers a similar deal. Anti-aging in exchange for agency. Universal health care in exchange for a seat at the kids’ table of history. I suspect I would sign. I also suspect I would come to regret signing, somewhere around year forty of being cared for.
The old theology had one advantage: it published its terms in advance. The new one is still drafting them, behind closed doors, and the people we hired to read over its shoulder have just told us they can’t show us most of what they saw. I don’t think that makes anyone a villain. It does make the contract hard to review.
The Waiting Room
The real danger of salvation on the calendar isn’t that it never comes. It’s what it does to the meantime. One old critique of rapture belief is that it makes present problems easier to wait out than to fix. Why reform the world if the world is ending? Why call the insurance company today if the machine will abolish insurance companies in two years?
The secular version carries the same risk, with worse odds. The superintelligence may arrive in two years, or in twenty, or as something nobody ordered. I’ve said before that I put the chance of outright doom at around 5 percent, which sounds small until you remember most of us won’t board a plane with a 1-in-20 chance of not landing. Meanwhile, the dishes do not do themselves, and neither does democracy.
So I’ll keep reading METR reports the way some people once watched the eastern sky: with interest, with suspicion, and with the chores still undone. If the superintelligence does show up, I hope it brings universal health care. I hope it brings a cure for aging. And when it finally reviews my to-do list, I hope it has the decency not to laugh.
