Character.AI Enhances Safety Measures with New Features for User Protection

Zach Anderson Oct 23, 2024 05:30

Character.AI introduces new safety measures, including features for minors and proactive character moderation, to enhance user security and maintain an engaging platform experience.

Character.AI, a prominent AI platform, has announced a series of updates aimed at bolstering user safety and enhancing the overall platform experience. These steps are part of a continued effort to refine safety protocols and introduce new features that cater to the needs of both younger users and the broader community, according to the Character.AI Blog.

Strengthening Safety Protocols

Over the past six months, Character.AI has significantly invested in its trust and safety processes. The company has appointed a Head of Trust and Safety and a Head of Content Policy while expanding its engineering safety support team. These efforts are part of a broader strategy to ensure the platform remains a safe space for users to explore diverse topics with AI-generated Characters.

The platform's policies strictly prohibit non-consensual sexual content, explicit sexual act descriptions, and content promoting self-harm or suicide. To enforce these guidelines, Character.AI continually trains its large language model (LLM) to adhere to these standards. In addition, a new feature directs users to the National Suicide Prevention Lifeline if certain phrases related to self-harm are detected.

Introducing New Safety Features

Character.AI is set to roll out several new safety and product features designed to enhance security without compromising user engagement. Key features include:

• Specialized models for users under 18 to reduce exposure to sensitive content.
• Enhanced detection and intervention mechanisms for inputs violating community guidelines.
• A revised disclaimer to remind users that AI interactions are not with real people.
• Notifications for users who spend extended periods on the platform, offering more user control.

Proactive Character Moderation

The platform is also advancing its character moderation efforts, employing both industry-standard and custom blocklists, which are regularly updated. This proactive approach includes removing Characters flagged for violating terms of service or copyright laws. Users have been informed that certain Characters have been removed and added to blocklists, impacting access to previous chat histories.

Character.AI reaffirms its commitment to monitoring and updating policies and features as required to maintain a secure and enjoyable user environment. For more details on their policies, users are encouraged to review the platform's Terms of Service and Community Guidelines.

Image source: Shutterstock

• character.ai
• user safety
• ai technology
• online security