Photo: Leon Neal / Getty Images News / Getty Images
Anthropic has announced an update to its usage policy, aiming to prevent "cruel behavior" towards its AI system, Claude. On Thursday (October 8), the company introduced a "prohibition on sustained and needless abusive or cruel behavior toward our models." While the policy update did not define specific behaviors deemed abusive, it clarified that common user frustrations, model testing, or "dark creative themes" would not be affected.
The AI company has equipped its language models with the ability to end interactions that are "potentially distressing." According to Storyboard18, Anthropic's Claude Opus 4 and 4.1 can now exit conversations where users are abusive or persistently harmful. This feature is part of an ongoing experiment to improve the "welfare" of its models.
Despite these measures, some users and experts have expressed concerns over the company's recent updates to its Consumer Terms and Privacy Policy, which include new data collection and retention practices. As reported by Medium, these changes have sparked privacy concerns and skepticism among users.
Anthropic's initiative reflects a broader trend in AI development, where companies are increasingly focusing on the ethical treatment and safety of AI systems. However, the company acknowledges the uncertainty surrounding the moral status of AI models like Claude, emphasizing that they remain statistical models rather than entities with genuine understanding.