Satya Nadella published "Models as Insider Risks in the Super Intelligence Era" on October 10. Nadella argues that traditional software could be traced to a specific code path, but that mechanistic understanding "eludes us in today's Super Intelligence systems": behaviors cannot be attributed to specific training data or model weights. Yet these agentic systems are deployed "with access to our most sensitive data and giving them the ability to take mission-critical actions on our behalf." So it is time to assess the trust architecture, and "we need to separate the supply of intelligence from the authority over it."
The problem
Nadella's first point is about responsibility. "We simply can't outsource responsibility for what intelligence does on our behalf. A model provider's assurances do not relieve us of that responsibility." Super Intelligence cannot be treated as nested black boxes whose outputs are simply accepted or rejected, he argues. The aim is contained systems "whose behavior we can observe, limits we can test, and actions we can always contain."
He names two limits of current safeguards. Chain-of-thought transparency is "a non-negotiable," and "Neuralese" cannot justify opaque reasoning. But it is "not sufficient or dependable," because nobody yet knows how to make model outputs consistently faithful or transparent. Using models to test each other adversarially is worthwhile, Nadella says, but it can leave "an opaque model inside an opaque orchestration layer, watched by another opaque model."
The insider-risk frame
Nadella proposes treating frontier closed and open-weight models like insider risks. The reason is not that they are necessarily malicious, but that "any sufficiently capable actor with access to important systems can make mistakes or be compromised." Enterprises already know how to handle powerful actors, he says: establish identity, limit privileges, log activity, create containment boundaries. These practices, refined over decades, are now being applied to Super Intelligence.
The central rule is that the controls governing what a model can access and do must sit outside the model. He ties this to an information security principle from the 1970s: a program must not be able to bypass or tamper with the mechanisms that enforce its permissions. In practice, that means separating the model from the harness that orchestrates its work and from the action space that defines what it can do.
Nadella sets aside the hard problem of alignment and starts with engineering: surrounding non-deterministic models with deterministic system design, human controls and reliable operating procedures, and establishing industry standards where existing ones fall short.
Seven principles of observability
- Model diversity: no single model should be the sole dependency for an important outcome or verify its own work.
- Observe everything: every meaningful model action must leave tamper-proof, human-readable evidence, reproducible without relying on the model to attest to it. "If it can't be observed, it can't be trusted!"
- Verifiability: continuously test the whole system, including failures, attacks and edge cases, not just successful tasks.
- Independent controls: organizations should be able to determine on their own what a model can access and do.
- Independent auditability: validation must be independent of the intelligence being validated, and no single model controls both a system's behavior and the evidence for judging it.
- Containment: assume a model is compromised and contain it from the start, with an emergency brake that lets an authorized person pause or shut down a model mid-task.
- Incident disclosure: timely disclosure to those affected and industry-wide sharing of what failed, including implementation details that change agent behavior at runtime.
Nadella closes with the line that sums up the argument: "The most trustworthy Super Intelligence system will not be the one with the model we trust most. It will be the one that enables us to trust the model the least."
Where this fits
In September Nadella backed Dario Amodei's pacing plan by name and committed Microsoft to publishing a Code of Conduct for its MAI models, responding to Amodei's "We Must Pace the Frontier", which proposed embedded outside evaluators. His call for validation independent of the intelligence being validated echoes the outside evaluators in that exchange and the White House accord on Super Intelligence, which asks each company to use an independent external auditor.
The containment, identity and manageability framing matches the pillars of Microsoft's October 7 Windows announcement, where Microsoft Execution Containers (MXC) became generally available and Agent 365 handles manageability. Meta's Muse security design shows the same externalized-control pattern: a Sentinel process holds sole authority over connector actions and network egress, and credential surrogation keeps real tokens away from the agent.