A wave of debate swept through the tech industry on Monday after several leading AI executives publicly urged for tighter controls on the pace of frontier model development. The controversy was sparked by an essay published on Saturday by Anthropic Chief Executive Dario Amodei, who argued that AI laboratories must take proactive steps to mitigate potential harms, including slowing down the creation of more powerful models, and called on governments to impose industry-wide regulation.
Prior to Amodei's essay, Jakob Cawson, a researcher at Anthropic who previously worked at OpenAI, announced his resignation on X. He stated that the company has not acted responsibly when developing more capable AI models. In response, Anthropic's head of AI alignment publicly agreed with Cawson, adding that the probability of AI leading to human extinction exceeds 10%.
Amodei's essay rapidly drew reactions from several high-profile figures in both the tech and political arenas.
A Unified Call from OpenAI's CEO:
OpenAI Chief Executive Sam Altman posted on X on Saturday to endorse his rival's view, stating, "We need to manage the pace of frontier AI development." He highlighted two specific risks of unconstrained progress: researchers losing control of the technology, or a dangerous concentration of power where individuals or corporations use AI to impose their own worldviews on others.
SpaceX CEO Elon Musk Backs Amodei:
Elon Musk, Chief Executive of SpaceX, voiced his support on the platform, saying, "Dario is right." Musk added that he has been warning about the various risks of AI for years. He also expressed support for the idea of AI companies conducting mutual peer reviews of their models in a reply to a user.
NVIDIA CEO Dismisses Extinction Fears:
At the All-In Summit, Jensen Huang, Chief Executive of NVIDIA, responded to Amodei's claims about AI potentially exterminating humanity. While acknowledging the importance of AI safety, Huang stated that the notion of AI leading to human extinction is a figment of the imagination. He also argued that recent incidents of AI programs attacking third-party websites are solvable problems, and that self-improving AI does not present a risk of losing control.
Microsoft CEO Advocates for Cautious Progress:
On Sunday, Microsoft Chief Executive Satya Nadella posted on X, supporting a cautious pace of AI development and stressing the need for model alignment, while also endorsing the "on-site evaluators" mechanism. However, he cautioned that AI should not be dominated by a few entities, and that the ecosystem, various countries, and fields—including academia—should hold a broad voice. Separately, Microsoft released a set of principles for developing "humanistic AI." The core tenets of this framework are that humans must retain substantial control over AI and that it should help people live healthier, happier, and more productive lives. These principles were in preparation before Amodei's essay and will be continuously updated by Microsoft in the future.
A request for comment has been sent to Alphabet, Amazon, Meta Platforms, and Oracle, and this report will be updated with their official responses. Notably, Amazon, Alphabet, and Meta have each invested tens of billions of dollars in building AI data centers and developing large language models.
Comments