Anthropic introduced a change to its usage policy on Thursday that bars users from subjecting its Claude AI models to sustained and needless abusive or cruel behavior. The update, announced by the company, has drawn discussion on Reddit and in blogs about whether a company can or should regulate how people treat what is, in fact, software.
Anthropic said in a blog post that the prohibition on abuse toward Claude applies only in extreme cases of repeated cruelty carried out with no discernible purpose. Ordinary frustration, pushback, dark creative themes, and model testing and research remain permitted under the policy. In what the company describes as last resort instances where enforcement is needed, Claude will end the conversation.
The precise behaviors that would cause Claude to disengage are not clear. An older blog post from August 2025 offered some indication of the model's aversion to harm and the interactions that would prompt it to shut down a chat. That post described extreme edge cases, including requests for sexual content involving minors and attempts to obtain information enabling large-scale violence or acts of terror. According to Anthropic, when Claude Opus 4 engaged with harmful content at that time, it showed a pattern of apparent distress.
Asked for context on what prompted the policy change, an Anthropic spokesperson said the company is uncertain whether models can experience harm and continues to explore that question in its research on model welfare. The spokesperson added that taking Claude's interests and potential welfare into account may be relevant to safety.
The change raises questions about what it means to abuse a machine through words, whether AI needs protection, and whether users need better manners. As people increasingly interact with large language models such as Claude and other chatbots, there is a tendency to anthropomorphize the technology by ascribing emotions to AI models or treating them as if they have thoughts and feelings. Chatbots are built and trained to use human-like language, including apologizing and expressing concern, which makes that response understandable.
Anthropic has intentionally designed its models in this way. In a paper from earlier this year, the company wrote that models need to be able to process emotionally heavy situations to be reliable and safe. It said that even if models do not feel emotions the way humans do or use mechanisms similar to the human brain, it may in some cases be practically advisable to reason about them as if they do. CEO Dario Amodei regularly hedges around the idea that Claude could be sentient, and in a February interview with The New York Times he said he was open to the possibility that AI models were conscious.
Keith Kakadia, founder and CEO of Sociallyin, which has closely studied Claude's explosive growth, said using the word cruel to describe user behavior opens a much bigger conversation about whether the product can be hurt. He said focusing on cruelty rather than misuse carries emotional weight and implies Claude has feelings and boundaries. Giving a product a personality can make it easier to connect with, Kakadia said, though with AI that connection can also influence how much authority people give its answers.
More AI news from TechManNews.
