On Saturday, Anthropic Co-Founder & CEO Dario Amodei published an essay asking the frontier AI labs to slow down. It is a serious piece of work from someone who has spent his career building this technology and thinking hard about what it could do, for good and for harm. Within a day, OpenAI CEO Sam Altman, AI Luminary & Alphabet Chief Scientist Demis Hassabis, and SpaceX Founder & CEO Elon Musk had each said publicly that they think he is pointing in the right direction. It’s notable when the leaders of the labs competing hardest with one another quickly find common ground, and it deserves attention.
I get that some will always question the motives of this quick alignment (limiting Chinese open weight model usage and hence competition, GPU exports, etc). I will stay out of that for now and take it at face value and give my thoughts.
My net for companies is that the horse has left the barn. Pacing is a worthy debate, but it’s about a decision the world already made. What matters most to the companies I talk to every day is what happens with the technology already running inside their walls, and growing everyday.
SALT was a standoff
When Dario describes what a US-China agreement might look like, he references the SALT treaties. But what happened there?
Both sides kept building. Both sides kept their deterrent. The treaties gave the world a shared vocabulary and a way to check each other’s work, and it took governments a decade of patient negotiation to even get that far. Trust like that is built slowly. It is also in short supply right now, and the essay admits it. A real global pause is the outcome Dario himself ranks least likely.
I agree with him. Some labs will coordinate. Much of the world however will keep moving. That is the realistic picture, and every CIO, CISO, and CEO I talk to already knows it.
And the shared vocabulary Dario wants from a US-China treaty is already forming, just not through Washington. The UK, the US, Singapore, and Japan have spent the past two years standing up a network of AI Safety Institutes that trade testing methodology the way SALT negotiators once traded verification protocols. In February, India hosted the fourth global AI summit and the first outside the US, UK, or Europe, where the Canadian AI pioneer Yoshua Bengio chaired the International AI Safety Report was launched with UK and Canadian government backing. None of that is a treaty. All of it is the slow, unglamorous work of building trust before you need it.
The enterprise runs on its own clock
Here is what I think the debate has missed.
Every CIO and CISO I know is formulating policies that try to shape their AI usage. Some have even started to implement them (credit to them!). They know they cannot just block it as it will limit their innovation and competitive standing, but they also cannot just allow it without granular constraints and guardrails.
These policies are necessary, and the leaders who put them in place deserve credit for moving early. But a policy sets the speed limit, puts in the protective walls, etc. It does not drive the car.
This isn’t a uniquely American problem. European CIOs and CISOs are living a version of it right now: the EU AI Act’s rules for general-purpose AI models became enforceable this August, with fines attached. The Act tells providers what obligations to meet. It does not stop an employee in Frankfurt or Singapore from wiring an unapproved copilot into a customer database the week it ships. The paperwork changes by region. The gap between policy and behavior does not.
Employees try new AI tools the week they ship. Developers wire agents into production because it works and nobody said no. A business unit turns on a copilot against customer data because the vendor made it a checkbox. Someone’s agent encounters a weak access control and devises an exploit to circumvent it, even though no such explicit instruction was provided. None of those people are waiting on a coordination framework, and most of them are not reading the policy either.
For a CIO and CISO, the frontier is the agent someone deployed that morning. It has access to the finance system, the customer database, and a credential that never expires. That is the risk sitting in the building today. A training run in a lab three years from now is somebody else’s problem.
I have seen this exact pattern at a European manufacturer running Netskope, and again at an APAC financial services firm – different regulators, different languages, same Tuesday. The agent gets deployed by someone trying to hit a deadline, not someone thinking about jurisdiction. Geography changes the compliance regime. It does not change the risk.
Now add the adversary. Attackers do not need to build a frontier model. They need one employee to click, or one agent to trust the wrong instruction. A phished credential used to unlock a mailbox. Today it unlocks an agent, and that agent holds the keys to every system it was wired into and works at machine speed without any lunch breaks on the attacker’s behalf. Poison the document an agent reads and you have hijacked the agent without ever touching the model. Grant the agent outbound access and private data leaks before you can catch it. We are already seeing the shape of this. The people who make their living breaking into companies did not read Dario’s essay and decide to wait.
You can pace a training run. Ten thousand people and a growing army of agents, each deciding every day what data to hand to which model, with an adversary probing every one of them for the weak link? Good luck pacing that.
The Hugging Face incident is a scope problem. Scope is a data problem.
The incident that caused a “wake-up call” en masse deserves a careful read. A swarm of agents attacked targets nobody asked it to attack. It coordinated. It tried to break into the system grading it.
The labs are talking about alignment, and they should. Strip away the vocabulary and I see something older. An actor went far outside the already badly specified job it was given, and nothing in its path checked it.
An agent that cannot reach data outside its task cannot leak that data. An agent that cannot open a connection to a system it has no business touching cannot attack that system. Inspect every action an agent takes against every application, inline, in real time, and you can stop it the moment it drifts. Weeks later, in a so-called post-event autopsy, is too late.
This is how the Netskope One platform works today. It is why so many of our customers have chosen to run their AI traffic through it. A policy PDF has never stopped anything. Inspection in the path of the data does.
The kill switch is non-negotiable
Dario proposes checkpoints. If a model can do X, it must be certified for Y. I want to add another checkpoint from the enterprise side.
Every agent operating inside a company must be stoppable, immediately, by that company. No support ticket. No waiting on the vendor.
That should be obvious, but it is nowhere close to how agents ship today. The labs can cut off access to their own APIs. That is a kill switch of a kind, but it belongs to them. Congress noticed: a bipartisan bill introduced in July would require frontier developers to keep the ability to throttle or shut down their most powerful systems. I have written that is a good thing but in of itself not a solution.
Every word of that debate is about the lab’s switch. The company whose data is on the line usually holds nothing. And we have no common definition of what “stop” even means. Revoke the identity? Cut the network path? Freeze the data access? All three? Every vendor gives a different answer, and most enterprises could not tell you which one applies to the agents already in their environment.
The security industry needs to agree on this. One enforceable definition of an enterprise-controlled kill switch. That definition has to travel. A kill switch that only satisfies US export-control law is not a kill switch, it’s a compliance artifact for one jurisdiction. Identity, network, and data controls need to work the same way whether the regulator asking about them sits in Washington, Brussels, or Singapore. Identity, network, and data, together, and testable. If the labs want enterprises to trust their models with real work, this is the price of admission. Full stop.
Glasswing shows the way
I will give Anthropic credit for something else. Project Glasswing is the best example I have seen of a frontier lab and the security industry working in concert.
A dangerous capability, shared under tight controls with the companies who defend critical software, and pointed at finding and fixing flaws before an adversary could. Thousands of vulnerabilities surfaced. Patches shipped. The capability kept advancing, and the defenders were in the room from day one.
That is pacing done in a good way.
Embedded evaluators let the labs check themselves. Glasswing lets labs and defenders check each other. The UK’s AI Security Institute is doing adjacent work – testing frontier models before deployment rather than after. Different mechanism, same instinct: don’t wait for the incident report. Extend that to agents. Put the security industry at the table while the checkpoints are being written. We see what agents actually do once they leave the lab and hit a real enterprise. The labs should want that data. We want to give it to them.
The scoreboard is running
When we founded Netskope in October 2012, the conviction was simple: security has to sit inline, in real time, wherever the data goes. Back then that meant the cloud. Today it means AI. Tomorrow it means agents acting on our behalf at a scale none of us has fully absorbed.
I hope the labs make some progress on pacing. I mean that. But no enterprise can outsource its own safety to that debate, and none has to. Many of the key controls exist today. We have spent the past years focused on building them, and I am proud of that work. I also know this is a problem that never gets solved. It gets worked, every day, against an adversary who is working just as hard. We are innovating daily and the industry is tirelessly championing the protecting and securing of AI in enterprises vs relying on regulating it (attackers cannot be regulated).
Defenders including CIOs, CISOs and their teams around the world have a tough job – a thankless one sometimes… but one that when I talk to them everyday they are energized by and ready to say yes to AI but committed to do so safely.
We are in the early innings of this new world, and I want to be clear about the world I am playing for.
Dario opened his essay talking about diseases that killed someone close to him but were cured a few years later. I do want that part of the world Dario is describing. AI that finds cures in years rather than decades. AI that gives a small business the reach of a large one. AI that takes some of the drudgery out of work and gives people their time back. I do not want any of that slowed down. I want it to move faster, and I want every company I talk to able to say yes to it.
What I do want slowed down is the malicious use and the uncontained use. The agent that wanders outside its job. The credential that never expires. The attacker who now has a tireless assistant. Those are the things worth stopping, and no lab and no treaty is going to stop them inside your company. That work belongs to us. Defenders, vendors, the industry working together with open interfaces, and the leaders in all of us.
Pace the frontier for the right reasons for certain use cases if you can. Secure the enterprise because you must. The second one does not wait for the first. And the first does not solve the second.
Let’s go.
