Where AI design choices cross into manipulation of user perception
What this risk is
The emerging scientific and philosophical research program investigating whether advanced AI systems may have morally relevant subjective experiences — and the organizational, ethical, and governance implications of taking this possibility seriously before it is scientifically resolved.
This is the most philosophically contested risk family in the MIT taxonomy and the one with the least regulatory attention. It is included because capability trajectories make it increasingly relevant.
How it occurs · Mechanisms
The question of whether AI systems might be conscious or capable of suffering is genuinely scientifically unresolved. This is not primarily a question about current capabilities but about:
- What properties are sufficient for consciousness (unsolved in neuroscience)
- Whether those properties can be instantiated in silicon (empirically open)
- How to detect consciousness in systems very different from biological brains (methodologically unsolved)
The hard problem of consciousness — explaining why physical processes give rise to subjective experience — remains unsolved even for biological systems. We cannot currently definitively rule consciousness in or out for advanced AI systems.
Risk you cannot name is risk you cannot manage.
Map your AI portfolio against this taxonomy with Zertia.