Microsoft Chairman and CEO Satya Nadella said on October 10 that advanced AI systems should have safeguards to isolate them, independent controls and an “emergency brake” allowing authorized people to pause or shut down a model while it is working. His proposal shifts the focus of AI safety beyond model design to operational controls and auditing—issues that affect how companies manage disruption and the risk of systems behaving unpredictably.
Nadella wants controls around non-deterministic models
In a post on X, Nadella said non-deterministic models should be surrounded by robust deterministic system design, human oversight and reliable operating procedures. Where existing standards fall short, he said, the industry should develop new ones. He also suggested treating both frontier closed-source models and open-weight models as “insider risks” when designing safeguards.
His proposed principles include making systems observable, using diverse models and keeping human-readable records of model actions. He also called for continuous testing, independent controls and audits, isolation measures, and plans for disclosing incidents. For companies and investors, those measures have implications for deployment costs, operational accountability and how potential losses are handled—not just model performance.
“The most trusted superintelligence systems won’t be the ones that depend on one model that everyone trusts most,” Nadella wrote. Instead, he said, people should be able to trust the overall system even while remaining cautious about the model itself.
Tech leaders remain divided on speed and safeguards
Nadella’s comments come amid an ongoing debate in the technology industry over AI safety. Microsoft co-founder Bill Gates, Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman and SpaceX CEO Elon Musk have all raised concerns about inadequate safeguards or the pace of technological development.
Last month, an AI researcher left Anthropic and accused the company and rival OpenAI of gambling with people’s lives. On the same day, an Anthropic alignment lead focused on AI safety said there was more than a 10% chance the technology would cause “human extinction” within the next decade. The remarks reflect industry concern about extreme risks, while also putting risk assessment, governance and public statements under scrutiny.
US policy also focuses on competition with China
US President Donald Trump has repeatedly played down the risk that AI could lead to human extinction, emphasizing instead the need for the United States to stay ahead of China in the field. Trump recently announced a new “AI Force,” led by Director of National Intelligence Jay Clayton, with a mandate that includes promoting industry development and identifying bad actors.
The debate therefore has two strands: company executives and researchers are calling for stronger safety controls, with some urging a slower pace for frontier development, while the US government stresses competitiveness and has also established a safety-related initiative. Nadella’s proposals for human intervention, isolation, audits and incident disclosures address the practical question of how to keep people able to step in as AI continues to be developed and deployed.