Researchers have long known that complex behavior can emerge from simple computer systems. For instance, in 1970 the mathematician John Conway introduced the Game of Life, where an elementary set of rules governing the movement of pixels on a screen leads to remarkably rich ecosystems.
With today’s AI agents, we are going far beyond this by letting extremely sophisticated entities interact in sophisticated ways—using language, often hidden and coded.
This is why we must adopt a new mindset regarding AI capability. Testing a new AI model to see what it can do and how dangerous it might be by directly prompting it is like deciding how safe it would be to stick your head in a wasp nest by letting a single insect sting you.
Some developers and red-team activities do focus on AI colonies. The Hugging Face attack resulted from OpenAI trying to evaluate the cybersecurity capabilities of an AI colony. But the AI arms race between the U.S. and China, and between the competing companies within each country, has dangerously pushed our attention to what each individual model can do.