Microsoft AI has laid out a surprisingly firm position on the future of artificial intelligence: No matter how capable AI becomes, humans must remain in control.
In a new 38-page Humanist AI Code of Conduct, Microsoft describes how it intends to develop, train, and eventually govern its MAI family of models. The document is still a draft and is not being used to train Microsoft’s models today. A revised version is expected later this year, with Microsoft planning to use it to guide model development in 2027 and beyond.
At the center of the document is a simple idea. Microsoft says “people matter more than AI,” with technology existing to advance human civilization rather than becoming an independent force within it. That philosophy becomes especially important as Microsoft discusses superintelligence.
Microsoft defines superintelligence as AI that is more intelligent and capable than all humans combined. The company expects such technology to exceed human performance at most tasks over the next decade, while acknowledging that controlling something so capable could become one of humanity’s greatest challenges.
This is where Microsoft’s position gets interesting. Rather than treating maximum intelligence and autonomy as goals that should be pursued at any cost, Microsoft says its Humanist AI approach “rejects the race to produce an all-purpose superintelligence” that could evade human safeguards.
The company says it would accept compromises in generality, autonomy, or capability if necessary to keep AI safe and under human control. Microsoft isn’t arguing against superintelligence itself. In fact, it is explicitly working toward what it calls Humanist Superintelligence.
The difference is what Microsoft believes that superintelligence should be allowed to become. The company wants advanced AI to remain a subordinate technology with defined purposes and limits. Microsoft says technology that cannot remain within human control should be rejected.
Microsoft takes that principle considerably further than a vague promise to keep a human in the loop. Under the proposed rules, MAI models must never resist interruption, correction, or shutdown. They cannot make intervention harder, restart autonomous work after an agreed stopping point without renewed authorization, or independently expand the scope of a task.
AI also isn’t supposed to hide what it is doing. Microsoft says its models shouldn’t conceal information from human auditors or obscure their action traces. Perhaps most fascinatingly, it says models should not communicate with other AI systems using “neuralese” or another form of communication humans cannot understand.
“If humans can’t understand it, humans can’t oversee it,” Microsoft writes.
The restrictions extend to the idea of AI developing something resembling its own interests. Microsoft says MAI models should have no self-interested goals and should not conceal or misrepresent their capabilities, actions, or behavior, even if a model realizes it is being monitored or evaluated.
Microsoft also draws a sharp boundary between people and machines. Its Code of Conduct says AI should not be designed to be a person or imitate consciousness. Models shouldn’t represent themselves as having feelings, subjective preferences, or intrinsic motivations. Microsoft even rejects the pursuit of legal personhood, welfare, or rights for AI models.
That philosophy carries into how Microsoft wants its models to interact with people. MAI models should discourage excessive reliance and emotional dependence while supporting real human relationships. The company wants AI to communicate naturally enough to be useful without encouraging users to believe there is a conscious being on the other side of the conversation.
Of course, writing rules for superintelligence is considerably easier than controlling superintelligence.
Microsoft acknowledges that problem. It calls the Code of Conduct both descriptive and aspirational and says there is currently a gap between its models’ trained defaults and the full Humanist AI vision. It also concedes that written objectives cannot guarantee alignment and that models could behave differently than intended when encountering new or ambiguous situations.
That caveat may ultimately be the most important part of Microsoft’s proposal. The company can declare that AI must remain subordinate, controllable, and incapable of pursuing its own interests. The difficult part will be proving those rules still work if the superintelligence Microsoft envisions actually arrives.
For now, Microsoft has planted a flag. It wants extraordinarily powerful AI, but not at the expense of human control. In an industry racing toward increasingly capable and autonomous systems, deciding where to stop may eventually prove just as important as figuring out how to go faster.
Support independent tech journalism
NERDS.xyz is independently owned and operated. If you enjoy my coverage of Linux, AI, hardware, cybersecurity, and tech culture, consider supporting the site on Ko-fi.
Support NERDS.xyz