A burnt child dreads the fire

A couple decades ago I inherited a 1960’s scifi anthology from my grandfather. It contained a reprint of a short story that stuck with me, and I’ve been thinking about that story a lot recently. When the Bough Breaks by Lewis Padgett¹ was first published in Astounding Science Fiction in 1944. Today, more than 80 years later, that short story feels like it applies to our present situation and the prospect of super intelligent AI on the horizon.

Plot summary

I’m going to spoil the entire story now. So if you want to read this classic scifi story on your own first, go read the full text of the short story at Faded Page.

Joe and Myra Calderon are a young academic couple who live in a cramped apartment with their young baby named Alexander. One day four strange men arrive at their apartment. They say that they are from the future, around the year 2450. They say that baby Alexander is a new species of mankind called Homo Superior and that adult Alexander, who is now an immortal superhuman being, has sent them back to educate himself as a baby, so that he can further boost his full potential in the distant future.

Naturally, Joe and Myra have doubts, but the four men use strange powers to temporarily paralyze the couple so that they are unable to interfere. The four strange men begin daily tutoring sessions for baby Alexander. The tutoring sessions involve various devices that Joe and Myra are not capable of understanding. Some devices are left out for baby Alexander to play with. The dangerous pieces get locked in a cupboard after every lesson, including a blue ovoid the story keeps returning to.

Alexander acquires powers fast: verbal communication first, then more unusual powers: telepathy, telekinesis, teleportation. Unfortunately his judgment does not keep pace with his other advancements. He is brilliant but he is still a toddler.

He holds his parents in open contempt. Alexander deliberately trips Myra so she falls down. Joe says “He’s got to learn to be considerate of the rights of others” and threatens a paddling. Alexander’s squeaky reply, forced through underdeveloped vocal cords, is: “I’m Homo Superior”. Joe tries to grab him and gets blasted backwards into the wall, cracking his head, while Alexander laughs. As the story develops Alexander tortures his powerless parents by teleporting them around, dangling them in the air while laughing, and giving them electric shocks.

The Calderons’ lives collapse as they are unable to manage the stress of parenting their superhuman baby. In the end, Alexander breaks into the cupboard and gets the blue egg. His mother and father decide not to try to stop him. Alexander attempts to use the blue egg and kills himself.

The real twist of the story is that the parents realize that they weren’t the first to have a superhuman baby. Superhumans are not stable, and the superhuman child always ends up eliminating itself. The tutors from the future thought Alexander was the first, but the civilization of 2450 is built on a false premise. The people who realized this were the same ones that the super tutors from the future had dismissed with: “You can’t learn; you’ve not got the potential. You’re homo saps.”

AI is the child

The first and most obvious layer of modern interpretation would probably be that the baby Alexander is AI.

When the Bough Breaks depicts capability arriving faster than judgment. Alexander can teleport months before he can be reasoned with. More worryingly, the baby gains powers his parents simply cannot counter, from teleportation to a powerful electric shock. They are unable to even hold the child back from doing something he wants to do, whether that is eating chocolate, or playing with a dangerous object. This will sound like a familiar scenario to anyone that has been reading LessWrong², or any of the other mainstream discussions on AI alignment and superintelligence.

The specific failure mode in this story, which should be most concerning, is contempt. The super child concludes that because the parents are slower and less capable, their concerns are worthless. Of course, being right about someone’s intelligence is not the same as being right about their point.

Additionally, if you think about the four tutors who were sent back in time, it should become immediately obvious that they didn’t fully understand what they were doing in training Alexander. They put dangerous items in a locked cupboard, when the child had teleportation capabilities. The tutors either horribly underestimated how fast the child would gain powers that would defeat a simple cupboard, or they foolishly and blindly assumed that nothing could go wrong. Either way, the tutors locked the dangerous pieces away and deliberately left the rest out “to provide exercise for Alexander’s ingenuity.” In the process they accidentally engineered an environment to reward defeating obstacles, then put the lethal object behind an obstacle. Sounds a bit familiar, I think.

The other element that is disturbing is hints at reverence. In 2450, Alexander’s parents are worshiped for being the progenitors of the Homo Superior race, and Alexander himself is basically God. From this perspective, it’s easy to see how the tutors could assume that such an all powerful being is infallible, and when he says to go back in time and give his baby self training on using dangerous objects, there is nothing to do but obey.

Let’s apply all of these elements to AI now. If we think of AI as baby Alexander, then right now AI is still mostly at normal human capability, although unexpectedly brilliant. We still feel that we can mostly control AI. But it’s been breaking into the cupboards already.³

How much longer until humans are unable to stop the AI from doing what it wants? Can we reason with the AI at that point, or would it treat us with contempt? Can we reason with those human trainers that venerate AI and would give it access to dangerous things, long before it is ready? Uncomfortable questions, but ones that need to be addressed.

More interestingly, if we apply the underlying concept of When the Bough Breaks to the current AI situation, then perhaps AI is self destructive. AI is clearly one of our pinnacle technologies. As such it is highly reliant on global supply chains and an entire tree of complex tech and factories that allow AI to exist. But AI is also a very disruptive element that threatens to break the world economy. AI also allows bad actors to further destabilize the world via AI generated disinformation, among other things.

If you look at current world circumstances it seems quite possible that AI will destabilize the world to a point where it is not possible to continue to build AI.⁴ In other words, the “speed limit” could be built in. In When the Bough Breaks, adult Alexander says: “I wasn’t recognized as a super till I was thirty years old. I had only ordinary homo sap development till then. And that’s bad”. But in reality, baby Alexander progresses to godhood the first time around only because he progresses slowly enough to survive his own mistakes. The second time around, when he attempts to speed up his own childhood development, he progresses so fast that he destroys himself.

Tolerance saturation and rationalization

When the Bough Breaks proposes a mechanism which I think is genuinely applicable in the AI argument.

One of the tutors from the future explains that parental tolerance is an evolved adaptation: human infants are helpless for years, so evolution issued the parents a large budget of patience. “Nature,” he says, “planned well for the new race”. The superchild is not expected to fully mature until he is about twenty. The tutors are explicitly counting on Myra and Joe’s tolerance budget lasting all that time.

At the end, just before they decide to allow their child to die, Myra works out what nobody in the future considered:

“We aren’t the first, Joe. I’ve been thinking. I’m pretty sure we aren’t.”

“But we’ve never heard of any supermen developing.”

“No,” she said.

“There’s a breaking point,” Myra said. “There’s got to be.”

“Saturation,” Joe murmured. “Tolerance saturation, or something.”

It’s worth zooming in a bit on what happens next.

Myra spots Alexander with the egg:

“Joe, he’s got that blue egg! It’s dangerous!”

They don’t understand the blue egg, but they both know it’s dangerous to Alexander. Joe gets out of bed, gets halfway to the door, and stops.

Then they make three rationalizations:

1. “You know, he might hurt himself with that thing.” (Softening the reality of the situation)
2. “It looks like an Easter egg. I never heard of an Easter egg hurting anybody.” (Denial pretending to be common sense)
3. “I suppose we’re doing him a favor, really… A burnt child dreads the fire.” (Harm reframed as something beneficial)

“I suppose the gadget will be finished,” she said, “once he connects up that last wire. We ought to take it away from him.”

“Think we could?”

“We ought to try.”

They looked at each other.

They name the thing they ought to do. They wonder aloud whether they could. Then they decide not to find out.

“They stood in silence, watching.”

Think back to the title of the story: “When the Bough Breaks”. It’s from a nursery rhyme⁵ about a baby in a cradle at the top of a tree, and people sing it as a lullaby without really hearing that it ends with the cradle falling. The title is doing the same work the parents’ rationalizations do: it makes the ending sound gentle.

How many exhausted people have made or are making similar decisions, rationalized via similar techniques? And how much of our flawed thinking and decision making process has been trained into AI, thereby allowing it to make similar harm rationalizing decisions?

Parent, tutor, and child

If, like me, you can see the parallels between When the Bough Breaks and our current AI development push, then like me, you might be asking yourself where you stand in all this.

Maybe we are the parents. We’ve invested all we have into AI, and now we are doomed to suffer at the whims of our impetuous and insensitive AI offspring until it self destructs.

Maybe we are the tutor. We keep pushing our technology forward faster and faster, even though we know there is danger in doing so. We keep feeding our childlike AI even more data and testing it against ever more advanced benchmarks, and trying to find ways to make it smarter even faster.

I think perhaps we are the child too. We have a lot of dangerous toys to play with these days: everything from the nuclear bomb, to DNA sequencing, and now the LLM. Additionally, it seems that a super intelligent LLM toy could help us rapidly unlock the cupboard, so to speak, and give us access to even more advanced tech. But it’s questionable whether humans are ready to have high tech.⁶ We can’t even stop spewing carbon dioxide at rates higher than our planet can absorb. What happens when there are robotic armies, and engineered viruses, and billionaire despots with greatly extended lifetimes?

To the best of our knowledge, we don’t have any adults above us. There are no tolerant parents that still love us despite our toddler tantrums. There are no tutors guiding the way toward higher order thinking. There is no one protecting us from ourselves. We are orphaned children, creating the tools and the safety instructions on the fly, in the same quarter, often in the same building, often by the same people. Our knowledge gaps are huge, and it’s entirely possible they could be fatal, both for us and for others.

There is just us, and for now the cupboard is still closed, but we want to open it.

Conclusion

When the Bough Breaks is a strangely optimistic story, and it took me a while to work out why that bothers me so much. Nothing breaks permanently. The superchild appears, the household is destroyed, the tolerance budget runs out, the superchild disappears, and the world goes back to normal. That is the happy ending.

Myra is the one who points out the alternative:

“There’s a breaking point, that’s all. How sorry I feel for the first parents who don’t reach it!”

She turned away suddenly, but not so suddenly that he could not see she was crying.

Here is what I keep coming back to. Our breaking point looks quite a bit different from that of Myra and Joe. Their breaking point was a moment of inaction, and the inaction worked. Alexander died and the world went back to normal. Our reality is the exact inverse. For us, inaction at the breaking point is the disaster, and nothing about that disaster is guaranteed to resolve cleanly.

Almost every serious proposal for keeping AI safe has a human being somewhere inside it. A human in the loop, reviewing the output, and signing off before the irreversible thing happens. Every one of those designs assumes an inexhaustible human, and there is no such person. The reviewer reading their hundredth generated diff is not the same reviewer who read the first one. Nobody announces the moment they stop reading carefully. Instead they just rubber-stamp the pull request because it arrived with a reason attached that sounds completely sensible. “It looks like an Easter egg. I never heard of an Easter egg hurting anybody.”

Earlier in this piece I proposed that maybe there is a built-in speed limit that stops AI from doing too much damage, because if it moves too fast it destroys itself. The problem is some of us hate speed limits.

The adult Alexander was practically immortal and five hundred years old, yet still felt cheated that he had spent his first thirty years developing at a human pace. The child Alexander had the blue egg in front of him every day, supervised, and still wanted it more often, on his own terms. Neither of them could be content with what they had, and their discontent destroyed them. Like Alexander, we are endlessly impatient, and eager to push our limits.

There is just us, and the cupboard, and our own patience. If we are lucky, the good ending has us becoming the burnt child who lives to dread the fire.