The document discloses that Command R and Command R+ models may generate toxic text including obscenities, sexually explicit content, and content that stereotypes groups, despite safeguards, particularly in extended multi-turn conversations.
This analysis describes what Cohere's agreement states, permits, or reserves. It does not constitute a legal determination about enforceability. Regulatory applicability and practical outcomes may vary by jurisdiction, enforcement context, and individual circumstances. Read our methodology
This disclosure establishes that Cohere's safeguards do not eliminate the risk of toxic output generation, which has direct implications for customers deploying Command R in consumer-facing applications, particularly those accessible to minors or in regulated content environments.
The agreement discloses that Command R models may generate toxic text, sexually explicit content, and content that stereotypes groups of people, even with safeguards in place, with heightened risk in long multi-turn conversations. Customers building consumer-facing applications are responsible for implementing additional content filtering appropriate to their deployment context.
Cross-platform context
See how other platforms handle Toxic Degeneration Disclosure and similar clauses.
Compare across platforms →"Models have been trained on a wide variety of text from many sources that contain toxic content (see Luccioni and Viviano, 2021). As a result, models may generate toxic text. This may include obscenities, sexually explicit content, and messages which mischaracterize or stereotype groups of people based on problematic historical biases perpetuated by internet communities (see Gehman et al., 2020 for more about toxic language model degeneration). We have put safeguards in place to avoid generating harmful text, and while they are effective (see the 'Safety Benchmarks' section above), it is still possible to encounter toxicity, especially over long conversations with multiple turns.Excerpt from Cohere's Responsible Use Policy
1.
Enforcement risk, jurisdiction flags, contract triggers, and due diligence action items.
Get the research letter
Companies change their terms quietly. We read every version and catch what actually changed. One email a week on the changes that matter and what they mean.
This disclosure establishes that Cohere's safeguards do not eliminate the risk of toxic output generation, which has direct implications for customers deploying Command R in consumer-facing applications, particularly those accessible to minors or in regulated content environments.
The agreement discloses that Command R models may generate toxic text, sexually explicit content, and content that stereotypes groups of people, even with safeguards in place, with heightened risk in long multi-turn conversations. Customers building consumer-facing applications are responsible for implementing additional content filtering appropriate to their deployment context.
No. ConductAtlas is an independent monitoring service. We are not affiliated with, endorsed by, or sponsored by Cohere.