Microsoft CEO Satya Nadella says businesses should treat powerful AI models as potential insider risks, restrict their access, and give humans the ability to stop them mid-task. Those are reasonable precautions, but his warning sounds fearful to me, and I question how sincere it is when Microsoft seemingly has ground to make up in the AI race.
In a lengthy post on X, Nadella argues that organizations are giving AI agents sensitive information and responsibility for critical tasks without fully understanding how the underlying models behave. His proposed response is to put enforceable safeguards outside the intelligence itself.
“We simply can’t outsource responsibility for what intelligence does on our behalf. A model provider’s assurances do not relieve us of that responsibility,” Nadella writes.
The organization deploying an agent therefore remains accountable for what it does. Buying access to a model does not transfer that responsibility to its developer, regardless of the assurances accompanying the product.
Nadella contrasts conventional software, where developers can trace behavior through specific code paths, with frontier models whose outputs are harder to explain. He repeatedly calls these systems “Super Intelligence,” although his use of that term does not establish that today’s models have achieved superintelligence.
His central proposal separates capability from permission. A model might be able to perform a task, but the surrounding system should determine whether it has authority to do so.
“In other words, we need to separate the supply of intelligence from the authority over it,” Nadella writes.
Much of his approach borrows from existing enterprise security practices: establish identity, limit privileges, record activity, and create containment boundaries. He applies that logic to both closed and open-weight models, arguing that any capable actor with access to important systems can make mistakes or suffer compromise.
That is also the basis for his proposed emergency brake. An authorized person should always be able to pause or shut down a model during a task, with more advanced containment technology accompanying more capable systems.
Nadella is skeptical that AI supervision alone can solve the problem. Having models verify one another can help, but it can also leave businesses with several opaque systems stacked together. He similarly calls for chain-of-thought transparency while acknowledging that a model’s explanation is not necessarily a dependable account of its reasoning.
His stronger requirement is evidence. Meaningful actions should produce tamper-proof, human-readable records, and independent reviewers should be able to reconstruct an outcome. He also calls for testing against attacks and failures, avoiding dependence on a single model, and promptly disclosing incidents.
Those proposals have merit, but let’s be honest: seemingly no one notable in the AI community uses Microsoft’s consumer Copilot as their go-to assistant. That is my impression, rather than a measured usage statistic, and I am talking about the consumer assistant rather than GitHub Copilot. Copilot simply does not strike me as the destination influential AI users enthusiastically choose.
Microsoft certainly has enterprise customers. It reported more than 30 million paid Microsoft 365 Copilot seats in its fiscal fourth quarter, but that is a separate offering, and paid seats do not establish preference for its consumer assistant.
Heavy AI users can build their workflows on Linux and macOS without making Windows or Microsoft’s consumer Copilot central to the experience. Microsoft cannot assume that its position in desktop computing will automatically make it the center of the AI world.
Vibe coding could threaten another part of its business too. If people can describe the software they need and have AI help build it, some may reconsider paying for Microsoft 365, formerly Office 365. The potential erosion would not require anyone to recreate every Office feature. An alternative that handles a user’s particular needs could make another subscription harder to justify.
Free projects are already targeting familiar productivity functions. For example, HermesOffice describes itself as an open-source suite that edits Word documents, Excel spreadsheets, and PowerPoint presentations. That does not establish feature parity or enterprise readiness, but it illustrates the competition Microsoft faces. AI services and ongoing maintenance can still carry costs, even when the software itself is free.
We are already seeing an example of this with Adobe. As Futurism reports, a four-person team used AI to build ArtCraft, a free, open-source suite targeting familiar Adobe tools, including Photoshop and Lightroom. Some applications are still in early alpha, so this is hardly proof that professionals can abandon Creative Cloud tomorrow. Still, it shows why Microsoft should be nervous: if small teams can recreate useful parts of expensive creative software, I see no reason to assume Word, Excel, and PowerPoint are immune to similar competition.
Microsoft could be in trouble if AI weakens the reasons people pay for its software faster than it creates compelling replacements. That possibility makes Nadella’s warning sound self-serving to me. I agree with enforceable safeguards, but I have a hard time believing he would emphasize distrust so strongly if Microsoft were clearly leading the field.
Support independent tech journalism
NERDS.xyz is independently owned and operated. If you enjoy my coverage of Linux, AI, hardware, cybersecurity, and tech culture, consider supporting the site on Ko-fi.
Support NERDS.xyz
