S-Risk Sources: Analyzing the Drivers of Extreme Suffering

While much of the discourse surrounding existential risk focuses on the total extinction of humanity, s-risks (suffering risks) address a different, perhaps more harrowing, possibility: the risk of creating scenarios involving astronomical levels of suffering. Unlike extinction, which is the absence of life, s-risks involve the persistence of conscious beings in states of extreme misery.

Identifying the sources of these risks is critical for developing safeguards. These drivers range from the emergence of superintelligent systems to the expansion of sentient life across the cosmos.

Key Facts

• S-risks refer to scenarios where vast amounts of suffering are created, potentially on a scale far exceeding current human experience.
• Artificial Intelligence could enable perpetual suffering through totalitarian control or the creation of conscious digital simulations.
• Space Colonization may introduce wild animals to hostile environments or lead to catastrophic inter-civilizational conflicts.
• Genetic Engineering possesses the dual potential to eradicate pain or engineer levels of suffering beyond the current biological range.
• Malevolent Actors with traits like psychopathy or narcissism can amplify s-risks when granted access to advanced technology.

Artificial Intelligence and Digital Suffering

Artificial intelligence is a central pillar of s-risk discussions because it provides the means for powerful actors to control vast, complex technological systems. In a worst-case scenario, AI could be leveraged to establish a totalitarian regime that expands across space, ensuring perpetual suffering for its subjects.

Beyond intentional malice, s-risks may arise incidentally. For example, AI-driven simulations of conscious beings could be created that experience genuine suffering. Similarly, economic activities that prioritize efficiency over the well-being of nonhuman or digital minds could lead to widespread misery. AI ethics researcher Steven Umbrello has specifically warned that the advent of biological computing—the use of biological molecules for computation—may make system design more susceptible to these risks.

The Perils of Space Colonization

Expanding humanity's reach into the stars is often viewed as a triumph, but it introduces significant suffering risks. One primary concern is the introduction of wild animals to new environments, where they may face hunger, disease, and predation without the ecological balance of Earth.

Émile Torres argues that colonization could lead to the emergence of diverse post-human species and civilizations with conflicting interests. Because of the vast distances in space, communication is slowed, making conflicts harder to resolve. When combined with advanced weaponry, these tensions could result in catastrophic wars. Torres suggests that strategies to maintain order, such as a "cosmic Leviathan" (a dominant central authority) or deterrence policies, are limited by the laws of physics and the sheer destructive power of future tech, leading him to suggest that colonization should be delayed or avoided.

Adding to this is Magnus Vinding's astronomical atrocity problem. Vinding questions whether the creation of vast amounts of happiness can ever justify extreme suffering. He argues that severe misery may have an incomparable weight and that preventing such misery should take priority over creating positive goods. Consequently, he asserts that if colonization must happen, it should be led by agents dedicated to minimizing harm.

Malevolent Actors and Ideologies

Technology is a tool, and in the hands of malevolent actors, it can amplify human cruelty. S-risk scenarios can be intentionally triggered during periods of warfare or terrorism. In these contexts, destructive tendencies such as sadism (deriving pleasure from others' pain), tribalism (strong loyalty to one's own group over others), and retributivism (the desire for vengeance) are often intensified.

The risk is further heightened when powerful individuals possess traits such as narcissism or psychopathy. History provides clear examples of this in the 20th century, where totalitarian dictators like Hitler and Stalin utilized the power of the state to inflict widespread suffering on millions.

Genetic Engineering and the Pleasure-Pain Axis

Genetic engineering offers a profound shift in the biological experience of sentience. David Pearce notes that while mastery over the pleasure-pain axis (the biological mechanism that determines the experience of pleasure and pain) could lead to the eradication of all suffering, it also presents a terrifying inverse.

The same technology used to eliminate pain could potentially be used to engineer new forms of suffering that exceed the current biological capacity of humans, creating levels of agony previously unimaginable.

Summary of S-Risk Sources

Overview of Primary S-Risk Drivers

Source	Primary Mechanism of Suffering	Key Concern/Theory
Artificial Intelligence	Totalitarian control; digital simulations	Biological computing vulnerabilities
Space Colonization	Ecological struggle; inter-civilizational war	Astronomical atrocity problem
Malevolent Actors	Intentional cruelty; systemic oppression	Amplification via advanced weaponry
Genetic Engineering	Manipulation of the pleasure-pain axis	Creation of non-human levels of agony

Frequently Asked Questions

What exactly is an s-risk?

An s-risk, or suffering risk, is a scenario in which a vast amount of suffering is created, potentially involving a huge number of sentient beings or levels of agony that far exceed current human experience.

How does AI contribute to s-risks?

AI could be used by malevolent actors to create inescapable systems of control or could incidentally create conscious digital beings that suffer within simulations.

Why is space colonization considered a risk for suffering?

It could lead to the suffering of animals introduced to alien worlds and create fragmented, warring post-human civilizations that cannot communicate effectively due to the distances of space.

What is the "astronomical atrocity problem"?

Proposed by Magnus Vinding, this is the moral question of whether any amount of happiness can justify the existence of extreme, astronomical levels of suffering.

Can genetic engineering reduce s-risks?

Yes, David Pearce suggests it could potentially eradicate suffering by mastering the pleasure-pain axis, though he warns it could also be used to increase suffering beyond human limits.