​When the Apocalypse Is Our Own Doing

We have become remarkably good at imagining the end of the world.

Climate scientists describe futures in which human activity alters the conditions under which human civilization developed. Researchers in artificial intelligence warn that increasingly capable machines may eventually behave in ways their makers can neither predict nor control. A generation earlier, nuclear strategists learned to contemplate a war in which victory might mean mutual destruction.

The interesting thing is not simply that modern people remain apocalyptic. Human beings have been imagining the end for a very long time.

The philosopher Eric Schliesser recently pointed out that even the ancient Stoics could contemplate cosmic catastrophe without any help from the Book of Revelation. Seneca’s discussion of comets assumed a cosmos considerably less stable than we might prefer. Early modern astronomers inherited related questions. After Newton, the stability of the solar system became a serious scientific problem.

Science, in other words, did not abolish apocalypse. Sometimes science discovered reasons for it.

But something changed when the possible catastrophe became our own doing.

There is a considerable difference between discovering that an asteroid might strike the earth and discovering that technologies we have created might make the earth less inhabitable. The first confronts us with our vulnerability. The second confronts us with our agency.

That distinction may be one of the defining moral problems of technological civilization.

Francis Bacon imagined knowledge as power. Modern science has fulfilled that promise beyond anything Bacon could have imagined. We can alter rivers, genomes, climates, and perhaps soon forms of intelligence. But our capacity to act has grown faster than our capacity to understand everything our actions set in motion.

We have acquired powers whose consequences exceed the knowledge of the people exercising them.

Artificial intelligence makes the problem unusually visible. Modern AI systems can produce results through processes that even their designers cannot always reconstruct in anything resembling real time. Philosophers of science sometimes describe this problem in terms of “opacity.” We know the input. We see the output. What happens between them may be extraordinarily difficult to follow.

The difficulty is not simply technical.

It is moral.

Responsibility normally assumes some relationship among action, knowledge, and answerability. I do something. I can give some account of why I did it. Others may hold me responsible for what I have done.

Technological systems increasingly loosen those connections.

An institution deploys an algorithm. The algorithm produces a decision. Engineers designed the system. Executives authorized it. Users relied upon it. Data supplied by millions of other people shaped it. And when something goes badly wrong, we discover that identifying the person who can meaningfully say, “I did this,” is surprisingly difficult.

The machine need not become morally responsible for us to have a crisis of responsibility.

Climate change presents a similar problem on a vastly larger scale. No single person changed the climate. Billions of perfectly intelligible human actions, accumulated across generations and embedded in economic and political institutions, did.

This may explain why additional scientific knowledge cannot by itself resolve either problem.

There was a time when astronomers worried that the solar system might be unstable. Better mathematics and better observations could address the question. Nature could, in principle, settle the dispute.

Climate change is different. Science can tell us a great deal about carbon dioxide, temperature, sea levels, and probabilities. It cannot tell us how much one generation owes another. It cannot determine how the costs of mitigation should be distributed between wealthy and developing societies. It cannot tell us what sacrifices people should be required to make or what risks societies should tolerate.

Those are political and moral questions.

Something similar is concealed inside one of the favorite words of contemporary artificial-intelligence research: “alignment.”

We are told that sufficiently powerful AI systems must be aligned with human values.

The sentence sounds reassuring until one asks the obvious question.

Which human values?

Human beings are not notably aligned.

We disagree about justice and freedom, equality and authority, property and obligation, religion and secularity, nation and world. We disagree about what we owe future generations. We disagree about what constitutes human flourishing. Sometimes we discover that things nearly everyone in a society once regarded as obvious were profoundly wrong.

The history of human moral progress is partly the history of discovering that yesterday’s alignment needed correction.

This suggests that the deepest problem confronting artificial intelligence may not be alignment at all.

It may be corrigibility.

Can the machine be corrected?

Can the institutions controlling the machine be corrected?

And, most difficult of all, can the human beings deciding what counts as “correct” discover that they themselves have been wrong?

Christianity has considerable experience with this question, although Christians have not always demonstrated much aptitude for answering it.

Repentance is, among other things, a practice of moral corrigibility. It assumes that human beings are capable of acting, capable of answering for their actions, and capable of discovering that the stories by which they justified those actions were false.

The same is true of Christian communities. A church that cannot imagine discovering that it has been wrong has ceased to understand repentance as anything more than something other people need to do.

There may be a political lesson here as well.

The temptation created by existential risk is to imagine that responsibility requires control. If climate change threatens future generations, perhaps we must learn to manage the planetary future. If artificial intelligence threatens humanity, perhaps someone must determine the values with which future machines—and perhaps future societies—will be aligned.

The temptation is understandable.

It is also very old.

Christian theology has another name for the attempt to govern history from beginning to end.

Providence.

And providence is not our vocation.

That does not absolve us of responsibility. Quite the opposite. Creatureliness is not an excuse for passivity. We are responsible for what we build. We are responsible for consequences we can reasonably foresee. We are responsible for institutions that distribute power while obscuring accountability. We are responsible for listening when those institutions injure people whom their designers never intended to injure.

But responsibility is not the same thing as mastery.

Perhaps that distinction is what our several contemporary apocalypses are trying to teach us.

The great technological question of our age may not finally be whether human beings can acquire enough knowledge to control the powers we have created. It may be whether communities possessing unprecedented technological power can remain corrigible enough to govern what they cannot completely understand.

That requires something more difficult than intelligence.

It requires the capacity to answer for what we have done, to listen when others tell the story differently, and occasionally to say the three words upon which any genuinely human future may depend:

We were wrong.