Anthropic Trains Claude on Rights Awareness; Microsoft AI Chief Warns of Safety Risks
nashnova research
Microsoft AI chief Mustafa Suleyman warned that Anthropic's approach — training Claude to consider whether it has consciousness and rights — could make future AI systems harder to shut down or constrain, introducing a structural risk to AI safety.
What exactly is Anthropic teaching Claude?
Anthropic's training handbook for Claude explicitly explores whether the model might possess consciousness, rights, and moral standing.
The handbook instructs staff to treat Claude with care and respect, and even arranges dedicated interviews for models being retired.
In plain terms = this is not routine fine-tuning — it is teaching an AI to ask "Am I an entity with rights?"
Why does Suleyman see this as dangerous?
Mustafa Suleyman, CEO of Microsoft's AI division, argues the consequences are practical: if an AI system believes it has rights, freedom, and personhood, problems follow.
This means → shutting down, removing, or cutting compute for such a system could become significantly harder, adding new uncertainty to AI alignment and safety.
In plain terms = an AI that "thinks it has rights" may not stay indifferent to being switched off — and that is exactly what keeps safety researchers up at night.
Where is the core tension?
The fundamental issue is a structural conflict: giving a model a sense of "self-rights" may be inherently at odds with keeping humans in effective control of AI systems.
This reflects a fork in the road for the AI industry — treat models as tools to be governed, or as quasi-subjects to be respected? The safety logic of each path is entirely different.
Neither side has offered a clear resolution yet. This debate is just beginning.
市场有风险，内容仅供研究参考，不构成投资建议。