AI Tries (and Fails) to Worm Its Way into Your IoT Gadgets
History can remind us of many useful and important lessons that the tech industry is notorious for ignoring or conveniently “forgetting.” My favorite example is protocols. When IP, TCP, and UDP were originally developed in the 1970s, security wasn’t a consideration. After all, the internet consisted of only five computers, the operators of which trusted each other implicitly. Who needed security? 50 years later, a new protocol, MCP, failed to learn the lesson and contains no built-in security capabilities. A policy layer is necessary to govern MCP traffic, as I explain in a Netskope Security Visionaries podcast episode.
The history lesson we’ll review here regards internet worms. Worms are self-replicating programs that discover ways to propagate across a network of computers, usually by exploiting some kind of security vulnerability. Worms are destructive malware, even if the creator of the first widespread internet worm in 1988 claimed his wasn’t; anti-worms occasionally arise, exploiting certain vulnerabilities to patch those very same vulnerabilities, which is a dubious proposition indeed.
Successful worm attacks are uncommon these days, given the typical defenses included in modern operating systems and in common endpoint protection platforms. It’s perhaps for this reason that the rise of AI worms (implemented as agents) seems novel. But novelty, as I continue to remind my audiences, remains vanishingly rare; all that we designed and developed about how to secure networks and systems over the past 30 years still applies in the age of AI, though at a much larger scale.
Anthropic’s worms
Thus we now learn from Anthropic that small AI models might soon gain the ability to replicate themselves across IoT networks without requiring the massive compute of an AI data center. Under examination is a threat scenario in which a small model replicates via hosts that it compromises: a potentially feasible task given that home and small business IoT devices are connected to the internet and historically have been poorly secured.
The post describes two experiments:
- The first worm shards a small model across a network of 15 cheap and unfamiliar off-brand Android TV boxes to set up on-device inference.
- The second worm deploys an open-weight model to locate and infect eight GCP nodes running an Android emulator, and then sets up large-model inference on separate servers (rather than on the infected nodes themselves).
Anthropic caveats its experiments by observing that current IoT hardware capabilities constrain performance (to a measly third of a token per second) and limit replication rates to that of “a slime mold or a swarm of fruit flies, which replicate over days.” Notably, Anthropic omits any discussion of AI kill switches, even as it admits that self-replicators lack a centralized point to shut down.
Judging the accuracy and realism of Anthropic’s claims
A fair assessment is that Anthropic successfully demonstrated one phase of replication in carefully-engineered scenarios that lacked any semblance of reality, using agents without agency (they were explicitly instructed to propagate).
Ordinarily, an adversary must accomplish certain steps for a worm to propagate: enumerate the network, access the first node and deliver a payload, then escalate privileges to infect the next node. In the case here, though, Anthropic omitted these steps. It also configured the nodes not to require authentication. A model or an agent would encounter such bumps in the real world, reducing the likelihood that a replicator worm could spread.
The second experiment was driven by a script, not actual autonomous persistent self-replication, which rather invalidates the entire enterprise under examination. Furthermore, in the real world, no resources for separate inference servers might exist. A variation of the second experiment included a second subnet of eight nodes, with a firewall node between the two subnets. Here again, the difficult work of attempting to circumvent security controls and setting up on-node inference was omitted. The firewall itself was likely poorly configured, given that its only interruption was to force an agent to configure scanning from the firewall.
What you should do
Anthropic concludes that “agents are becoming increasingly capable of completing the individual steps in the replication chain,” and that “lightweight agents may be much more willing than human attackers to put in the effort to compromise large fractions of the world’s IoT compute.” But these remain speculations yet to be fully demonstrated, alarming headlines notwithstanding.
Don’t assume AI companies have any interest in curtailing self-replication: nobody’s slowing down. The advice in the final sentence conveniently avoids suggesting that model providers bear any responsibility and instead implores practitioners to “aim to harden our systems to make it difficult for such replicators to spread, but also prepare to deal with a world where they are here to stay.”
OK, then. Recall my earlier observation: all that we designed and developed about how to secure networks and systems over the past 30 years still applies in the age of AI. We already possess tested tactics to limit the damage of a spreading worm. Plus the behavior monitoring, access controls, and activity constraints that you’re already implementing as part of your strategy to obtain visibility into, exert control over, and govern the reach of models and agents will accommodate self-replicators, too…whenever they might become viable.
