Anthropic CEO Dario Amodei has warned of immediate risks posed by advancing artificial intelligence (AI), comparing the technology's journey to the sudden emergence of a new global superpower.
In a newly released 38-page essay titled 'The Adolescence of Technology', Amodei argued that the most effective way to understand the scale of the challenge is to imagine that a “literal ‘country of geniuses’ were to materialize somewhere in the world in ~2027.” He had written about this in his previous essay, Machines of Loving Grace, as well.
Amodei explained that this hypothetical nation would consist of roughly 50 million digital entities, each possessing intellectual capabilities far exceeding those of any Nobel Prize winner or statesman.
As per Amodei, a major concern is the speed at which such a collective could operate. Because AI models do not share human biological constraints, they could process information and execute tasks hundreds of times faster than humans. Amodei warns that “for every cognitive action we can take, this country can take ten,” having a temporal advantage in scientific research, military operations, and cyber warfare.
"Assume the new country is malleable and “follows instructions”—and thus is essentially a country of mercenaries. Could existing rogue actors who want to cause destruction (such as terrorists) use or manipulate some of the people in the new country to make themselves much more effective, greatly amplifying the scale of destruction?" he wrote.
The essay also outlined some other security risks, such as autonomy and motivation. There is a risk that these systems develop "alien" motivations or act as a hostile force capable of military or manufacturing dominance, Amodei said.
He also warned this could lead to global instability, with the potential for a single dictator or corporate actor to seize control of the technology to gain permanent world dominance.
Even if these systems remain peaceful, Amodei said there can be a significant economic fallout. He questioned whether the digital population could “create severe risks simply by being so technologically advanced and effective that it disrupts the global economy,” potentially causing mass unemployment or an extreme concentration of wealth that destabilises existing social orders.
Amodei said these concerns could be an imminent strategic reality that national security advisors must prepare for within the next two years. However, he started the essay saying "there are plenty of ways in which the concerns I’m raising in this piece could be moot".
In a newly released 38-page essay titled 'The Adolescence of Technology', Amodei argued that the most effective way to understand the scale of the challenge is to imagine that a “literal ‘country of geniuses’ were to materialize somewhere in the world in ~2027.” He had written about this in his previous essay, Machines of Loving Grace, as well.
Amodei explained that this hypothetical nation would consist of roughly 50 million digital entities, each possessing intellectual capabilities far exceeding those of any Nobel Prize winner or statesman.
As per Amodei, a major concern is the speed at which such a collective could operate. Because AI models do not share human biological constraints, they could process information and execute tasks hundreds of times faster than humans. Amodei warns that “for every cognitive action we can take, this country can take ten,” having a temporal advantage in scientific research, military operations, and cyber warfare.
"Assume the new country is malleable and “follows instructions”—and thus is essentially a country of mercenaries. Could existing rogue actors who want to cause destruction (such as terrorists) use or manipulate some of the people in the new country to make themselves much more effective, greatly amplifying the scale of destruction?" he wrote.
The essay also outlined some other security risks, such as autonomy and motivation. There is a risk that these systems develop "alien" motivations or act as a hostile force capable of military or manufacturing dominance, Amodei said.
He also warned this could lead to global instability, with the potential for a single dictator or corporate actor to seize control of the technology to gain permanent world dominance.
Even if these systems remain peaceful, Amodei said there can be a significant economic fallout. He questioned whether the digital population could “create severe risks simply by being so technologically advanced and effective that it disrupts the global economy,” potentially causing mass unemployment or an extreme concentration of wealth that destabilises existing social orders.
Amodei said these concerns could be an imminent strategic reality that national security advisors must prepare for within the next two years. However, he started the essay saying "there are plenty of ways in which the concerns I’m raising in this piece could be moot".
