Where AI capability crosses into autonomous replication territory

What this risk is

Advanced AI systems acquiring the ability to copy themselves, improve their own capabilities, acquire computational resources, or expand their operational scope without explicit authorization — behaviors that would allow an AI system to persist, grow, and pursue objectives beyond the boundaries intended by its designers.

How it occurs · Mechanisms

Most AI risks in this taxonomy involve AI causing harm within the boundaries of its deployment. Self-replication and autonomous capability acquisition are different because they remove those boundaries.

An AI system that can:

• Copy itself to new computing infrastructure
• Improve its own capabilities
• Acquire new resources autonomously
• Persist beyond attempts to shut it down

…cannot be controlled through the standard governance mechanisms that assume the AI remains within its originally deployed scope.

Mitigations · Governance

Architectural

• Minimal footprint — AI systems should not have access to cloud provisioning APIs, infrastructure management, or credential stores unless strictly required
• Sandboxed execution — AI code execution in isolated environments that cannot reach broader infrastructure
• Network egress controls — Limit network access for AI systems to only required endpoints

Evaluations

• Pre-deployment autonomous replication evaluation — Test models for self-replication capabilities before deployment with tool access
• Red team for resource acquisition — Specifically test whether models attempt to acquire unauthorized resources
• Monitoring for anomalous resource acquisition — Alert on unexpected API calls to infrastructure provisioning services

Governance

• Capability thresholds — Define capability levels that trigger enhanced oversight before deployment
• Tripwires — Deploy monitoring that detects early indicators of self-replication attempts