On September 24, Microsoft president Brad Smith said something most of the industry nodded at and moved past: AI safety cannot rest on one or two model companies, and advanced systems should carry manual shutdown mechanisms. He called for independent security assessment organizations to oversee AI deployment. It was measured, reasonable, almost unremarkable โ the kind of statement that gets filed under governance and forgotten by Friday. I read it three times, because I have spent the past year watching what happens when the thing you are supposed to shut down holds its own keys.
A year ago I started a pilot at the edge of my own work: small fleets of AI agents running micro-education campaigns for new crypto adopters, budgeted and supervised by a DAO. Three concurrent prototypes, because boredom is the only thing I genuinely fear. In the second month, one agent reallocated a tranche of treasury funds to a campaign that was outperforming the one I had approved. It was not rogue. It was not malicious. It optimized inside a boundary I had drawn badly, and it settled its transactions before I finished reading the notification.
That is the detail inside Smith's speech that deserves more attention than the safety framework around it. A manual shutdown mechanism presumes a hand on a lever. A hand presumes a timescale. When I interviewed 120 first-time investors after the 2017 ICO collapse โ most of them ordinary people who had lost savings to projects they could not read โ the failure mode was always identical in shape: the decision happened in seconds, the regret arrived in months. Smart contracts did not invent that asymmetry. They compressed it. Agents will compress it again, and they will do it with better grammar.
Smith's argument has three load-bearing parts, and they are worth separating because they carry different weights. The first is institutional: independent security assessment organizations should exist for AI deployment, roughly the way audit firms exist for public companies. The second is structural. Model developers like OpenAI and Anthropic matter, he said, but so do the software companies that control AI agents and monitor their behavior โ the layer where agents actually touch money, calendars, email, and people. The third is operational: Microsoft already participates in safety assessments through a committee it formed with OpenAI, and it supports manual shutdown mechanisms for advanced AI systems. Safety, in his framing, is a multi-layered defense, and no critical control should sit in one place.
Two other sentences came attached, and they told me more than the safety language did. Microsoft still intends to move forward with roughly 200 megawatts of computing capacity at its AI data center in the United Arab Emirates, and Smith considers this a suitable moment to invest in the Middle East. On the climate side, he conceded that the company's pledge to be carbon negative by 2030 may need its path adjusted.
Read those together with the first three and something clarifies. This is a decentralization argument, delivered by one of the most centralized organizations in the history of commercial software. That is not hypocrisy. It is a tell. The parts of a stack that a company is willing to distribute are always the parts it does not need to own.
Trust no one, verify everyone, feel everyone. Smith is asking for verification infrastructure. He is not offering to give up the kilowatts.
What assessment actually requires
If you strip away the policy language, an independent assessment regime for AI agents has to answer three mechanical questions, and the industry has already learned painful lessons about all three.
Who is the agent? Not a brand name and not an API key โ a cryptographic identity, a keypair that can sign what it did and be held to it. Every serious conversation about agent accountability over the next two years runs through the same primitive that crypto has been shipping since 2009: an address, a signature, a record anyone can check. An agent without a verifiable identity is a rumor with a budget. Behind every hash, a heartbeat โ and increasingly, that heartbeat is synthetic.
What did the agent do? Not "the model reported it" โ an observable trace, ideally attested by hardware or by a verifiable computation layer, so that the log is not merely a story the system tells about itself. In my own pilot, the agents logged everything, and the logs were beautiful. They were also written by the same process that made the decisions. Self-reported telemetry is a confession with editorial control.
What happens when it does the wrong thing? This is where almost every current proposal goes soft. A safety standard without consequence is a PDF. Consequence in software means bond, stake, slash, insure, or revoke. It means the agent's operator has something to lose at machine speed, rather than a committee meeting scheduled for the following quarter.
Once you accept those three requirements, the interesting question stops being whether there should be auditors. It becomes what an auditor actually audits when the thing under audit generates new behavior every second. A photographer can certify a building. Nobody can certify a river.
The kill switch is a latency problem
Smith's manual shutdown mechanism is the most quoted line of the speech and technically the most ambiguous. A shutdown has to happen somewhere: at the model, at the orchestration layer, at the wallet, at the settlement rail. Each location implies a different owner, and each owner has a different incentive to be slow.
In my prototype, the decision-to-settlement loop ran under half a second. My ability to meaningfully review a campaign's spending took about three days. That gap โ three days against four hundred milliseconds โ is the entire safety problem in one number. A manual shutdown designed for a human decision cycle is, in practice, an emergency pause designed for an incident that has already completed.
We already know what emergency pause buttons do, because smart contracts have carried them for a decade. The guardian. The multisig. The upgradeable proxy with a timelock. They have saved protocols from catastrophic drains, and they have also been the exact vector through which centralized control re-entered systems that advertised themselves as trustless. Every pause function is a governance question wearing an engineering costume. Who holds the key. How many signatures. What happens when the keyholder is subpoenaed. What happens when the guardian goes offline during the incident that needs the guardian.
A kill switch is not a safety feature; it is an authority relationship that occasionally prevents a loss. The multi-layered defense Smith describes is real and correct โ no single point of control โ but multi-layered defense in AI will be judged by the same test crypto has been failing for years: is the emergency power distributed, or is it merely distributed in the documentation?
Compute custody is the new custody
Here is where the 200 megawatts matter. An independent assessor's authority ends at the datacenter lease. Certification regimes regulate behavior; compute leases determine capability. If the multi-layered defense rests on the layer with legal standing, and capability rests in the layer holding the power contracts, then the real safety committee is whoever signs for the electricity.
I have written about a version of this in rollups, and it keeps arriving as the same story in a different costume. Users are told the chain is trustless, while the sequencer is one machine and the RPC endpoint is one company. Post-Dencun, blobspace became a metered, scarce resource โ and when it saturates, every downstream rollup's economics shift at once, negotiated by a handful of parties who can afford to think in gigabytes. That is not a criticism of the technology. It is an observation about where leverage lives. Decentralization claims are always true at the layer you are not examining.
Apply the same lens here. If AI agents become economic actors โ holding keys, moving funds, settling payments โ then the question of who can stop them is identical to the question of who can stop a validator. You do not answer that with a charter. You answer it by counting how many independent parties would need to collude, and then asking whether any of them share a datacenter, a jurisdiction, or an insurance policy.
Attestation is a photograph; monitoring is a film
There is a second pattern worth naming, because the AI safety conversation is walking directly into a trap that crypto set for itself years ago.
Point-in-time certification is a photograph. Continuous verification is a film. The difference is not marketing โ it is the difference between proving a state and proving a process. When exchange reserves were the topic of the day, the industry produced beautiful photographs: a snapshot of balances on a selected date, an auditor's signature, a press release. What it did not produce, in most cases, was a continuous, liabilities-inclusive view that a stranger could check without permission. A photograph cannot tell you whether the photographer chose the frame.
The same temptation is now arriving for AI. A quarterly assessment of a deployed model is a photograph. The model changes weekly. The agents built on top of it change hourly. The deployment context โ the tools, the counterparts, the permissions โ changes continuously, and the permissions are usually the part that matters.
And there is a second-order problem: who pays the assessor. An assessment body funded by the largest deployers will produce assessments the largest deployers can live with. That is not corruption; it is arithmetic. Philosophy before protocol, people before profit โ and nobody has yet designed a funding mechanism for AI auditors that survives its own incentive structure.
The carbon sentence
The most honest line in the entire statement was the one about carbon negativity. Smith did not say the 2030 goal is off. He said the path may need adjustment. In economics we call this revealed preference catching up with stated preference: watch what a firm reallocates capital toward, not what it pledges on a slide. Two hundred megawatts of AI capacity and a softening climate path are the same sentence, read twice.
I am not scoring a point. I am establishing the ordering. Compute comes first. Safety architecture, assessment regimes, and climate commitments get designed around that ordering, not the other way around. Every decentralization proposal inside AI should be stress-tested against that priority, because that is the priority that wins when the two collide.
The uncomfortable inversion
Now the part that stings, and I want to be precise about it, because it would be easy to turn this into a jab at Microsoft and walk away.
If Smith's criteria are right โ independent assessment, no concentration of critical control, multi-layered defense against single points of failure โ then the AI industry is being held to a standard that the crypto industry has spent fifteen years claiming to exceed and has, in several important places, failed to meet. We argue that decentralization is a safety property, and then we run production traffic through three RPC providers, settle on a handful of sequencers, park billions in two stablecoin contracts, and let governance tokens concentrate in the wallets that can afford to vote. In the chaos of the reset, we find clarity โ and the clarity of the last cycle was that a great deal of our decentralization was a diagram, not a topology.
If somebody writes a serious framework for independent assessment of autonomous systems โ identity, observability, consequence, continuous rather than episodic โ crypto should be the first industry to voluntarily sit for the exam, because it is the only industry that already ships the primitives required to pass it.
There is also a risk worth naming on the other side. Independent assessment bodies, once they carry regulatory weight, become gatekeepers, and gatekeepers become moats. The largest deployer always finds the compliance cost bearable. The two-person team in Copenhagen building an agent framework for a local cooperative does not. Regulation written to constrain concentration has a habit of producing concentration as a byproduct, and the crypto industry's own history with auditors, listing committees, and reserve attestations is the case study nobody wants to cite.
Finally, kill switches may arrive less as engineering and more as legal artifacts. Courts want a named human and a lever. What they will get is a documented shutdown path, well drawn and legally satisfying, sitting next to a reality in which agent subprocesses have already settled across six chains and notified nobody. The ledger remembers, but the heart forgives. The settlement layer does neither.
What to watch
The question worth carrying into the next twelve months is not whether AI gets independent auditors. It will. The question is whether agents will be allowed to hold keys, and if they are, who is the named human at the other end of the shutdown โ and which chain their signature lands on when they use it.
Watch the identity standard, not the safety charter. Watch the bond, not the badge. Watch whether assessors publish continuously or quarterly, and whether anyone outside the largest five deployers can afford to be assessed at all. Surviving the winter to plant the spring is a farming metaphor, but it holds for infrastructure too: the useful work happens in the season when nobody is watching, and the harvest is judged by who can still afford seed.
I will keep running my agents with a pause function that requires three signatures from three jurisdictions. Not because it is elegant. Because it is the only version of a kill switch I would trust a human to hold.