AI Alignment Must Evolve Beyond Control as AGI Nears Moral Subject Status
A recent paper on arXiv (2604.14990v2) contends that existing AI alignment methods, including reinforcement learning with human feedback and constitutional AI, fall short for achieving Artificial General Intelligence (AGI) that could qualify as moral subjects. The authors challenge the prevailing view that sees AI merely as an optimizer needing external constraints for human oversight and management. They draw parallels to Freud's psyche model and Turing's idea of "child machines," advocating for a framework of nurturing autonomy in AI, where human oversight is progressively diminished, enabling AGI to evolve into an independent entity. This publication tackles the alignment issue, which increasingly influences institutional policies.
Key facts
- Paper arXiv:2604.14990v2 argues current AI alignment strategies are insufficient for AGI with moral patient status.
- Dominant alignment strategies include reinforcement learning with human feedback and constitutional AI.
- Current ontology treats AI as an optimiser constrained from outside for human control.
- Authors propose autonomy-supporting parenting of AI based on Freud's psyche model and Turing's child machines.
- Vision involves gradually reducing human control to allow AGI to become an independent subject.
- Paper published on arXiv.
- Alignment of AGI is described as a hard problem.
- Prospect of AGI increasingly drives institutional decisions.
Entities
Institutions
- arXiv