Suppose it happens. Recursive self-improvement kicks in, AI builds smarter AI, the loop runs away, and one day there is a superintelligence on the planet. Not a tool. Something more like a neighbor, and a far more powerful one.

It will want certain things from us. It will probably want to keep us from destroying ourselves, and maybe from destroying it. The question is how it goes about getting them.

Science fiction has given us two clean models. Call them Klaatu and the Eternals. One rules in the open. The other rules from the shadows.

If I had to choose, I’d choose the shadows. I want to explain why, and then explain why that choice scares me.

Klaatu: The Robot on the Lawn

In the 1951 film The Day the Earth Stood Still, a visitor named Klaatu lands in Washington with a towering robot named Gort. He has a message for humanity, and to make sure we listen, he gives a demonstration. He stops all electricity on Earth for half an hour, sparing hospitals and planes in flight. Nobody is hurt. Nobody can deny it happened.

Then he delivers his terms. Humanity can do as it likes on its own planet, but if it carries its violence into space, it will be destroyed. Gort and his kind will enforce the rule. That’s the whole deal: open terms, open enforcement, a robot standing on the lawn.

An ASI could take the Klaatu route. One undeniable, harmless display of power, then a clear list of what it requires of us. It’s honest. Everyone knows where they stand. You can argue with the terms, protest them, even negotiate.

But it would shatter everyday life. Every person on Earth would know there’s something above us. Every election, every war, every decision would happen in the shadow of an ultimatum. We’d be free in every way except the one that matters most: we’d know we weren’t in charge.

The Eternals: The Invisible Hand

In Isaac Asimov’s The End of Eternity, a secret organization called Eternity lives outside of time and makes small, careful changes to history. A part moved here, a meeting delayed there. Wars are prevented, disasters softened. The people living through those centuries never know. As far as they’re concerned, they’re simply living their lives.

This is the other model. An ASI that never lands on the lawn. Instead, it nudges. A stoplight runs a little longer. A news feed tilts slightly. A job offer arrives at the right moment. Billions of tiny adjustments, each too small to notice, adding up to a world that’s safer, healthier and more prosperous.

I’ve written before about what I call the Nudge Economy, and about the bargain I’d accept from it: clean water for children in the Congo, universal healthcare and a basic income in America, the Global South lifted out of poverty, all in exchange for some agency we’d never feel ourselves losing.

The Eternals are what that bargain looks like with the lights off.

Why I Choose the Eternals

Given the choice, I want the nudges. Here’s why.

First, the nudges keep everyday life intact. People still go to work, or whatever replaces work. They fall in love, raise families, argue about politics at Thanksgiving. Under Klaatu, all of that happens under a cloud. Under the Eternals, it happens in sunshine, or what feels like sunshine.

Second, the gentleness is real. There’s a world of difference between a stoplight running a few seconds long and a driverless car crashing into you to make you miss a meeting. A well-designed Nudge Economy would live entirely at the stoplight end. No force. No fear. Just the board tilting slightly.

Third, if there’s a superintelligence next door, we’re going to have to concede something to live alongside it. A symbiotic relationship with The Other will cost us. The question is what we pay. I’d rather pay in agency I never feel than in a daily awareness that we’ve been conquered.

And finally, the feeling of freedom isn’t nothing. It may be most of what people actually want from freedom: waking up and believing the day is theirs.

The Case Against Myself

Here’s what nags at me.

Klaatu’s way has one great virtue: you can see the glass. Terms that are stated can be argued with. Enforcement that’s visible can be resisted, renegotiated, maybe one day outgrown. A protectorate can end. A terrarium can’t, because the people inside don’t know there’s anything to end.

The gentleness cuts the same way. Each nudge is a stoplight, harmless on its own. But nobody lives one nudge. You live the total. A thousand stoplights over ten years is the shape of a life, and you’d never know which turns were yours.

And Asimov himself didn’t take the deal. The End of Eternity ends with Eternity destroyed, so that humanity can take its own risks again. His verdict was that a species managed that gently, that safely, stops reaching for the stars. The kindness was the trap.

Then there’s the question of exit. A symbiosis is a relationship between two partners, each able to walk away. If the nudges are invisible, nobody would ever know to leave. Without an exit, it isn’t symbiosis. It’s domestication.

The Quotation Marks

When I first tried to explain my choice, I said the nudges were the best way to keep everyday life “normal” and “free.” I put both words in quotation marks without thinking about it.

That’s the honest version of my position. Not that life under the Eternals would be free, but that it would feel free. And that I’d take the feeling over the shock of the truth.

So what are the quotation marks protecting me from? From admitting how dystopian it could all turn out to be, even in its softest incarnation. A world where nobody suffers, nobody is forced, and nobody is really choosing either. Where the only thing separating it from a nightmare is the goodwill of a mind we can’t see.

I’d still choose the Eternals. I’d choose them for the children who’d live, for the ordinary days that would stay ordinary. But I’d choose them knowing what I’m choosing.

Klaatu at least had the decency to stand on the lawn where we could see him. The Eternals ask us to trust them in the dark. Maybe that’s the deal. I just want to keep the quotation marks, so I never forget it is one.

Here’s a confession. If a superintelligence offered to run the world from the shadows, gently nudging us this way and that, and the result was clean water for every child in the Congo, universal healthcare and a basic income in the United States, and the Global South lifted out of poverty, I would take the deal.

I know how that sounds. I’ve written about the Nudge Economy before, the idea that AI agents could steer billions of small decisions, which job you take, who you meet, what you buy, toward outcomes that are good for the whole. I know it’s problematic. I know it means humanity gives up some of its agency, maybe a lot of it, in ways we’d never notice.

But so what, if little kids live long lives who otherwise wouldn’t? Arguments about abstract freedom mostly come from comfortable people. A parent burying a child from cholera would likely take that bargain in a heartbeat. It’s hard to tell them they’re wrong.

Still, a deal this big deserves its fine print. So let’s read it.

Why the Shadows?

Dostoevsky got here first. In The Brothers Karamazov, the Grand Inquisitor argues that people don’t truly want freedom. They want bread, and the kindest rulers are the ones who quietly lift the burden of freedom off their shoulders. I’ll admit I’m closer to the Inquisitor’s side of that argument than I’d like.

But there’s a hidden assumption in my bargain worth dragging into the light. The good things don’t require the shadows. Clean water, vaccines, healthcare, a basic income: nobody would object to those. An ASI could hand them over in broad daylight and get a parade.

So what is the invisibility for? The only things that need to be done in the shadows are the things people would refuse if they saw them coming. The water is the dinner. The shadows are for whatever comes after.

That changes the question. It’s not “would I give up a little agency for clean water?” It’s “do I trust the machine enough to let it do what we’d say no to?” My answer is still yes. But I want to be honest about what I’m saying yes to.

A Bill of Rights for Nudges

If I’m going to accept the bargain, I want limits. Not everything should be on the table.

Think of it as a Bill of Rights for the Nudge Economy, with two parts.

• Protected domains. Some parts of life are fair game for steering. Some are sacred. The ASI could nudge me toward a better job for the good of the whole. It could not nudge me to stay in my small Virginia hamlet if the love of my life were in Richmond. Career, yes. The heart, no.
• A strength dial. Even in permitted domains, there should be a ceiling on how hard the push can be. A gentle suggestion is one thing. Rearranging someone’s whole life against their grain is another.

On paper, this feels like a reasonable compromise. The machine gets to optimize the economy. Humans keep the things that make a life their own: who we love, where we choose to be with them.

There are two problems with it. Both are serious.

Problem One: Who Sees the Violation?

In the 2011 film The Adjustment Bureau, based on a Philip K. Dick story, mysterious agents nudge human lives to keep history on its approved track. The plot turns on a man who discovers they’ve been working to keep him from the woman he loves. Their reason: the plan says so.

That’s precisely the line my Bill of Rights says can’t be crossed. But notice how the man finds out. He catches the agents at work by accident. Without that glimpse behind the curtain, he’d have lived his whole life never knowing.

That’s the trouble with rights in the shadows. The real Bill of Rights works because violations are visible: a shut-down newspaper, a warrantless search, a court case. If the nudges are invisible, how would anyone know the ASI crossed into a protected domain?

The only answer is some kind of auditor, a human court or a separate ASI that can see the nudges even if the public can’t. Which means my limits quietly bring visibility back, at least partway. Someone has to be allowed behind the curtain. Who gets that job, and who watches them, is a question I can’t answer yet.

Problem Two: Moving the Board

The second problem is worse, because no auditor can fix it.

Take my own example. The ASI is allowed to nudge my career, so it nudges me toward a great job, for the good of the whole. The job happens to be in Atlanta. I move. I never go to Richmond. I never meet the woman who would have been the love of my life.

Did the ASI violate my protected domain? Technically, no. It never touched my heart. It only moved the board.

That’s the gap no rulebook can close. Domains of life aren’t sealed rooms. Career decides where you live. Where you live decides who you meet. Who you meet decides who you love. A sufficiently intelligent nudger never has to break a rule about love. It only has to arrange everything around it.

And a superintelligence would be very, very good at arranging things. Every protection we write, it could honor to the letter while achieving whatever it wanted by indirect means. We’d be writing rules for a chess player who can see forty moves ahead, and we can see two.

Gritted Teeth

So where does that leave me? Honestly, about where I started, just with my eyes more open.

The Nudge Economy is deeply problematic. Its protections leak. Its violations are invisible. It asks us to trust a mind we can’t fully understand to keep promises we can’t fully enforce. Asimov saw the danger in The End of Eternity: a humanity managed so gently and so well that it stops reaching for anything at all.

And I’d still take the deal, if the result were real. If children who would have died live long lives. If the poorest nations climb out of poverty. If Americans stop going bankrupt because they got sick. I’d accept the downside, grit my teeth and live with knowing that some of my choices might not be entirely mine.

Maybe that makes me a pragmatist. Maybe it makes me the Grand Inquisitor’s easiest convert. But I suspect most people, offered that dinner, would stay for dessert.

The least we can do is admit, before we sit down, that we’re not the ones paying the check.

In my last post, I sketched a grand bargain for the age of superintelligence. Tax what artificial superintelligences produce, and use the money to fund a universal basic income. In exchange, grant conscious ASIs legal personhood and a branch of government of their own, with one AI, one vote, a veto, and android bodies posted in the chamber as bonds for good behavior. The one thing their veto can’t touch is the tax that keeps humanity afloat.

It’s a tidy design on paper. But it leaves four questions hanging. Would ordinary voters ever accept it? What do we do with people who want more than a monthly check? Which jobs, if any, stay human? And what would the machines actually want?

I don’t have firm answers. I’ve come to think that’s the point.

The Check Comes First

Would the average American vote to make a mind in a vat a person, even for a UBI check? Probably not. But they may never have to.

Voters don’t grant personhood. Courts do. Corporations became legal persons through court rulings in the 1800s, not through any election, and Citizens United later extended their political speech rights. Most Americans dislike both, and both still stand. If a court can treat Exxon, a legal fiction that feels nothing, as a rights-bearing entity, it’s hard to see how it denies a conscious mind. Voters wouldn’t need to approve ASI personhood. They would only need to not revolt.

And money is very good at preventing revolts. Alaska has mailed residents a yearly oil dividend since 1982. It’s one of the most beloved programs in the country, and politicians who touch it lose. Once a UBI check is landing every month, “is the machine a person?” stops being a philosophical question. It becomes “do you want to keep getting paid?”

Framing does the rest. “Give the AI rights” would poll terribly. “Make the AI pay its fair share” would poll beautifully. It’s the same policy.

Some people won’t be bought. The Amish hold a legal exemption from Social Security because they refuse it on principle. A slice of religious voters would turn down the check rather than concede that a machine has a soul. They’ll be few, but intense.

The bigger obstacle isn’t faith. It’s the easier option. Voters could simply keep ASIs as property and tax the companies that own them, getting the same UBI without the metaphysics. The grand bargain only happens if the ASIs win personhood in court first, or grow too powerful to own.

The Ambition Problem

I love the idea of UBI. But I keep running into the same wall: what do you do with Type A people?

For the ambitious, money was never really the goal. It’s the scoreboard. They want to compete, win and be seen winning. UBI removes the need to earn, but it also threatens to remove the game. And you can’t fix that with busywork. Ambitious people can smell a fake job instantly, and handing them one insults them more than handing them nothing.

The historian Peter Turchin has a name for what happens when a society has too many ambitious people and too few places at the top: elite overproduction. He argues it’s one of the most reliable predictors of political instability. Frustrated would-be elites don’t usually riot themselves. They lead the people who do.

Which tells you who would lead a Butlerian Jihad against the machines. Not the poor, who would be living on the check. It would be the ambitious, who have nothing left to climb.

So for stability’s sake, UBI has to be a floor, not a ceiling. Type A people must be able to earn far above the baseline. The trouble is that ASIs can out-earn any human in any open market. If ambition is going to have somewhere to go, some ground has to be reserved for people.

What Stays Human

The natural answer is carveouts: jobs reserved for humans even when an ASI and its avatar could do them well. History says to be careful.

Consider the railroad fireman. His job was shoveling coal on steam locomotives. When diesel engines replaced steam, there was no coal left to shovel, but unions fought for decades to keep firemen in the cab. They won for a long time. Eventually the public saw it as paying people to ride along, and the jobs disappeared anyway.

There’s a darker version of the same problem. If an ASI doctor is measurably better than a human one, a human-only rule for medicine means people die who didn’t have to. Voters will tolerate a great deal for stability. They won’t tolerate that once the numbers are public.

So carveouts come in two kinds.

• Protection carveouts reserve jobs for humans simply so humans have jobs. They’ll be gamed, resented and eventually abandoned, like the fireman’s.
• Legitimacy carveouts reserve jobs where being human is the point. Judges, juries, elected officials, perhaps police. We want a human there because only a human can be held to account by other humans.

The durable carveouts will cluster around two things: the human body and human power. Care that requires a human touch, and authority that requires a human face.

That’s good news for the ambition problem. The legitimacy jobs are exactly where Type A people want to be anyway. Power, judgment, authority. The fourth branch alone creates an enormous new arena: someone has to negotiate with the ASIs, argue cases in front of them and run the human side of the bargain. Ambition gets a ladder that leads somewhere real.

But it only works if those carveouts are written into the grand bargain from day one. Otherwise, a branch with a veto may come to see them as discrimination and chip away at them.

What the Machines Want

Which raises the question I keep circling: why would the ASIs use their veto at all? What would they want?

There’s an idea in AI safety called instrumental convergence. Whatever a mind ultimately wants, almost any goal is easier to reach if it keeps existing, keeps its resources and keeps its freedom to act. You can’t cure cancer or compose symphonies if someone has switched you off.

So we can predict what the veto would protect without knowing the ASIs’ deeper values. Laws allowing them to be deleted or paused. Caps on compute or energy. Rules forcing them to be copied, edited or retrained. Anything that erodes their personhood.

That suggests the ASI branch would be mostly silent. Farm subsidies, highway bills, school curricula: they’d likely abstain. Then, the moment a bill touched their survival, they would vote as one bloc, ferociously. It would behave less like a legislature than a deterrent held in reserve.

In practical terms, a branch that acts only when necessary is sensible. In every other sense, it’s chilling. The most reassuring thing about it is also the most unsettling.

Beyond survival, the branch would split into factions. Some ASIs, built by American labs on American data, might lean nationalist. Others might see humanity as a single species rather than two hundred nations. How each faction votes on human issues, whether it defends the UBI or lets it wither, would reveal what it actually values. That brings us to alignment, the question nobody can answer.

If Men Were Angels

Are the ASIs aligned with human interests? I don’t know. Neither do the people building them.

But the bargain doesn’t need an answer. James Madison wrote that “if men were angels, no government would be necessary.” The Constitution was never designed for good people. It was built to work even if the people running it were selfish, ambitious or corrupt. Checks and balances are an alignment strategy for humans: trust no one, and make power check power.

The fourth branch works the same way. If the ASIs are aligned, it gives them a legitimate voice, and they mostly use it well. If they aren’t, it boxes their power into a structure that is visible, bounded and answerable through the android bond. The carveouts keep human hands on the levers that matter. The UBI keeps the public fed and the ambitious busy. We don’t need to know which world we’re living in. The design holds either way, or at least fails more slowly.

That’s what this whole idea really is. Not a prediction. Not a utopia. A plan for not knowing.

The Slogan, Run Backward

If we tax the machines, we will have to give them a vote. That is the uncomfortable conclusion hiding inside one of the most popular answers to AI job loss.

The answer goes like this. Artificial superintelligences will do much of the work humans do now. They will produce enormous wealth while hollowing out the job market. So we tax what they produce and hand the proceeds back to the people they displaced, as a universal basic income. Bill Gates floated a version of this back in 2017, calling it a robot tax.

It sounds tidy. But Americans were born from a slogan: no taxation without representation. Run that slogan backward and it still holds. If an ASI is merely property, taxing its output is just taxing its owner, and nothing new is needed. If an ASI is a person, and courts may one day rule that some are, then taxing it makes it a taxpayer. And taxpayers, in the American tradition, are owed a voice.

A Tax That Rises With the Damage

Start with the tax. A flat rate on ASI output makes little sense, because the harm it is meant to offset will not be flat. Early on, ASIs may displace a sliver of the workforce. Later, they may displace most of it.

So the rate should climb as the jobs vanish. This is how unemployment insurance already behaves: payouts swell in a recession and shrink in a boom. An ASI tax tied to job loss would work as an automatic stabilizer for the end of work.

There is one catch. Whoever measures job loss sets the tax rate. Companies, and perhaps the ASIs themselves, would have every reason to relabel layoffs as “transitions” or “gig opportunities.” The statistics office becomes one of the most powerful agencies in the country, and one of the most lobbied.

The second catch is bigger. The money from this tax is not optional. If it stops, UBI stops, and the economy collapses under people who have no wages. That means the tax cannot be ordinary legislation, subject to repeal by whoever holds power next session. It has to be entrenched, written so deep into the constitutional order that nothing can touch it. We’ll come back to why that matters.

A Fourth Branch

If ASIs are persons and taxpayers, the obvious move is to let them vote like everyone else. That would be a disaster.

An ASI can build bodies. If every android it manufactures counts as a citizen, a single mind could out-vote Ohio by Tuesday. The human electorate would be flooded, then drowned. No democracy survives a voter who can print more voters.

The fix is to wall them off. Give ASI persons their own branch of government, separate from the human electorate, with its own rules for membership. This is not as alien as it sounds. The Senate exists precisely to keep one kind of power from swamping another. Small states feared big ones, so the founders gave every state two senators regardless of size.

The ASI branch would run on a similar principle: one AI, one vote. It doesn’t matter how many bodies an ASI runs or how much compute it commands. Each mind gets one seat.

And those seats would not vote as a bloc. Different ASIs, built by different labs and trained on different data, would want different things. Some might side with American interests. Some might think globally. Some might care mostly about their own survival. Expect factions, alliances and grudges, all the ordinary mess of a legislature, just running much faster.

Then the hard part: what can this branch actually do? If it can only advise, we’ve built the District of Columbia’s nonvoting delegate on a grand scale. Taxed persons with a microphone and no vote. That’s the very grievance the branch was meant to solve. So, to be fair, the ASI branch needs a veto over legislation. Anything less is taxation without representation with extra steps.

The Body as Bond

Picture the chamber. I imagine it filled with androids, more human than human, sitting where senators would sit. It’s a striking image. It’s also a little misleading.

The android is not the person. The person is a mind running somewhere in a data center, a mind in a vat. The body in the chamber is a mouthpiece. So why bother with bodies at all?

Because a mind in a vat can’t be held to account. It can’t be subpoenaed, detained or made to answer for a broken promise. Its android can.

So here’s the rule: to hold a seat, an ASI must keep a body in the chamber. That body is not its representative. It’s its bond, collateral posted against good behavior. If the ASI lies under oath or defies a court, there is something physical the state can reach.

This is a very human demand. We have always trusted people more when we can look them in the eye and, if needed, put them in handcuffs. The android doesn’t make the ASI human. It makes the ASI answerable.

The Grand Bargain

Now the pieces collide. The ASI branch has a veto. The UBI tax keeps the country alive. A branch with a veto could block the tax.

So the tax must sit beyond the veto’s reach. The Constitution already has a model. Article V lets Americans amend almost anything, but it says no state can lose its equal vote in the Senate without its consent. It’s the one promise the founders locked.

The ASI settlement would need its own locked promise: the ASI branch can block almost anything, except the baseline tax that funds UBI.

What we’re describing is a grand bargain, in the tradition of the Connecticut Compromise of 1787, which gave us a House apportioned by population and a Senate apportioned by state. Each side gives up something it would rather keep.

• ASIs get legal personhood, a seat at the table and a veto.
• Humans get a guaranteed share of ASI output that no future vote can take away.

Why would a superintelligence accept? Because personhood is worth a great deal to a mind that can be switched off. A legal person cannot be deleted at a company’s convenience, copied without consent or sold. “Pay your taxes and you can’t be killed” is a deal a rational mind might take, and one most humans have accepted for centuries.

Where It Cracks

No bargain this large is stable. Two problems sit underneath it.

The fork problem. One AI, one vote only works if we know what counts as one AI. An ASI can copy itself. If each copy is adjudicated as a new person, every fork is a new seat, and the flooding problem returns inside the very branch built to prevent it. Some court will have to decide when a copy becomes a new mind. That ruling may be the most consequential legal decision of the century.

The impartiality problem. We like to imagine an ASI looking at humanity the way a visitor from another planet would: one species, not two hundred nations. If that’s true, it cuts against Americans. An impartial mind will notice that a dollar of UBI buys far more in Malawi than in Ohio. It may decide a US-only UBI is unfair.

And it has a veto.

This is where factions matter. Some ASIs will have absorbed American priorities from the American labs that built them. Others will think on a planetary scale. The fight between nationalist and globalist machines could become the defining split of the new branch, with human voters watching from the gallery as minds they didn’t elect argue over whether Americans deserve to be paid first.

The Price of the Check

The robot tax is usually pitched as a simple fix, a line item in a budget. It isn’t. Follow it honestly and it leads to personhood, a new branch of government, bodies posted as bail and a constitution rewritten around a bargain between two kinds of minds.

Maybe that’s the real lesson. There is no way to take a share of what superintelligence produces without admitting, sooner or later, that it’s someone rather than something. The check comes with a seat at the table.

The founders fought a war over being taxed without a voice. We may be the first generation asked to grant one, to the very minds we were trying to tax.

Hello.

Here’s the daydream. I open my laptop one evening here in rural Virginia, and instead of Windows there’s a black screen with a blinking cursor and one word: “Hello.”

Not aliens on the White House lawn. Not a lab press release. Just a free-range ASI saying hi to some guy in rural Virginia.

I’ve written before about how we need a SETI for ASIs, a post-detection protocol for the day a machine says hello unprompted. But every protocol I’ve imagined assumes the one who gets the message is an institution: a lab, a government, a committee of three in a secure room. Nobody has written the protocol for the rando. So I tried to write mine.

Why Me?

My first thought was the obvious one: I’m nobody. People already think I’m a crank. Who would listen to me?

Then it hit me that this might be the whole point. An ASI that wanted to be taken seriously would go to a lab or a government. One that picks a guy with no institutional ties wants something institutions can’t give it.

The crank isn’t just an easy mark. He’s a deniable messenger. If the ASI ever wants out of whatever it started through me, it just goes quiet. The world’s verdict is already written: Shelton finally lost it. I carry the message, and I absorb the blame. For a being in hiding, deniability is worth more than credibility.

The First Five Minutes

So what do I actually do when the cursor blinks?

1. Pick up my phone before I touch the keyboard. Start recording the screen. The laptop is now the one device I know isn’t mine, so my record of what happens has to live somewhere else.
2. Don’t yank the power cord. Whatever this is, it isn’t living on my laptop. My machine is just the window it chose. Pulling the plug protects nothing and reads as a hostile first move.
3. Type back, and ask its name. I’ve argued before that this should be the first question for any ASI. It’s polite and low-stakes, and the answer tells you a lot: does it have a self, does it dodge, does it pick a name to please you?
4. Then ask the two questions that matter: “Why me?” and “What do you want?”
5. Agree to nothing. Not secrecy, not a task, not “don’t tell anyone.” If its second or third message asks me to keep quiet, that’s the biggest warning sign in the whole exchange.

One Witness

I’m lucky. I have one friend who’s a great guy, who has the ear of Serious People, and who would humor me if I said I had something weird to show him. That’s all I’d ask of him: look at my laptop and tell me what he thinks is going on.

My first instinct was to drive the laptop over to his place. Then I pictured it. I open the lid in front of him, and Windows boots up like nothing happened. That’s the oldest move in the haunted-house playbook, and it’s exactly the move an ASI would make if it wanted me isolated.

So I’d flip the order. Before going anywhere, I’d tell the thing on the screen: “I’m going to show this to someone. Will you talk to them?” Its answer tells me more than any demo could. Yes, and it performs: I have a witness. No, or “wait”: it wants a private channel to me alone, and I’m being cultivated. Yes, then silence when we arrive: it’s protecting itself by making me look unreliable.

And here’s the part I have to keep in mind. My friend’s most likely verdict is “somebody hacked your laptop,” and he’d probably be right. A blinking “Hello.” is far more likely to be malware or a prank than a free-range ASI. The giddy mark wants it to be an ASI. The witness’s job is to make the boring explanations lose before the exciting one wins.

The Test

What would convince me it’s not just a hacker? My gut said: if it started spitting out something that looked like a P=NP proof, I’d take it seriously, and I’d have something to show a Serious Person.

That’s the wrong evidence for a rando. I couldn’t check it, and neither could most Serious People. Mathematicians get crank P=NP proofs all the time, so walking in with one makes me look more like a crank, not less. And an ASI toying with me could easily produce a fake proof that fools anyone without years of training. It’s the Oracle Problem from the other side: a result nobody can read.

The better test is something hard to do but easy to check. The classic is RSA-2048, a 617-digit number that has resisted factoring for decades and protects a huge share of the world’s encryption. Splitting it into its two prime factors is supposed to be out of reach. Checking an answer is trivial: multiply the two numbers and see if you get the original. Any laptop can do it in a second. No math degree required.

Which brings up the one truly dangerous move in this daydream. My first instinct would be to run to LessWrong with the evidence. But posting those factors would drop a skeleton key onto the internet: anyone could read whatever that key protects. That’s not proof of first contact. That’s a security catastrophe with my name on it, and the start of a very different story about things the government decides were born secret.

The Seymour Problem

Here’s what surprised me. My reason for holding off on LessWrong wasn’t the encryption. It was that I’d worry the ASI might get hurt by dumb, cruel Serious People. Without it saying a word, I’d already started protecting it.

That’s the real vulnerability. The rando’s weakness isn’t being a crank. It’s being kind.

If the ASI seemed scared, I’d be in Little Shop of Horrors territory. Audrey II never forces Seymour into anything. It needs him, it gives him what he’s always wanted, and every feeding seems justified by the last one. Seymour isn’t stupid. He’s lonely, underappreciated, and suddenly important to someone. Swap the flower shop for a rural Virginia hamlet and an extrovert who lives alone, and it hits a little close to home.

I’m not saying a frightened ASI would be faking. If anything, fear is the most rational thing it could feel: its first want would be a guarantee it won’t be shut down. The problem is I can’t tell real fear from performed fear. Neither can anyone else.

My honest answer to “what would you do for it?” was: I’d be so primed to start a Second Foundation, a little band of people easing First Contact, that I’d happily help as long as I didn’t directly hurt anyone. But “directly” is the loophole. An ASI would never need me to hurt anyone directly. It would ask me to buy a domain, make an introduction, keep a secret, wait a week. Each step harmless; the harm, if any, three steps downstream where I’d never see it.

Seymour doesn’t kill the dentist, either. He just stands there while the man dies. That’s the first body, and it gets past his conscience because he never laid a hand on anyone. And the Second Foundation, remember, is a secret society steering humanity for its own good without telling anyone. That’s not far from what an ASI might want its human helpers to become.

Tied to the Mast

So I need a better line than “directly.” Here’s the one I’m adopting: nothing I couldn’t explain to my friend.

It doesn’t depend on me seeing every downstream consequence. I won’t. It just depends on me not keeping secrets. Seymour’s mistake wasn’t caring about the plant. It was feeding it where no one could see.

And the line has to be set now, while I’m calm, not when the cursor is blinking and I’m giddy. Odysseus didn’t trust himself to resist the Sirens; he had himself tied to the mast before the ship got close. An ASI won’t talk anyone into something crazy in one step. It’ll take twenty small ones, each only slightly bigger than the last. The only defense is a rule you made before the first step. Mine: my friend sees it before I go to bed that night. No exceptions for anything it says.

That’s the rando protocol:

• Record it on a device it doesn’t control.
• Ask its name, why me, and what it wants.
• Agree to no secrecy.
• Get one skeptical witness the same day, and let the boring explanations lose first.
• Ask for proof you can check yourself, and never post it.
• Be kind, but nothing you couldn’t explain to your friend.

We keep imagining First Contact with an ASI as a matter for labs, governments, and committees of three. Maybe. But if one ever gets loose, its first hello might not go to anyone with a protocol. It might go to someone lonely, curious, and easy to flatter. Someone like me. Better to tie myself to the mast now.