Prohibited Use: CSAM and Sexual Content Involving Minors

What it is

The policy absolutely prohibits using Cohere's AI to create any sexual content involving children, including AI-generated imagery or text.

Why it matters (compliance & governance perspective)

This provision establishes an unconditional prohibition with no operator-level override permitted, and violation would constitute a breach of the AUP as well as potentially criminal conduct under applicable law in most jurisdictions.

Consumer impact (what this means for users)

Any application built on Cohere's API that generates CSAM or sexual content involving minors is prohibited, meaning users of such applications cannot invoke Cohere's platform to produce this material, and operators cannot configure or permit this use.

Institutional analysis (Compliance & governance intelligence)

REGULATORY LANDSCAPE: This provision directly engages US federal law under 18 U.S.C. Chapter 110 (sexual exploitation of minors) and equivalent statutes in most jurisdictions globally. The National Center for Missing and Exploited Children (NCMEC) operates a CyberTipline with mandatory reporting obligations for electronic service providers under US law. The EU's proposed CSAM regulation and existing Directive 2011/93/EU also impose obligations on service providers. The FTC may also have jurisdiction over deceptive or unfair practices related to child safety. GOVERNANCE EXPOSURE: High. This prohibition is legally mandated in most jurisdictions independent of contractual terms, and failure to implement effective technical and policy controls creates severe criminal and civil liability. The clause is non-negotiable and cannot be modified by operator configuration. JURISDICTION FLAGS: Universal applicability. No jurisdiction permits CSAM. US, EU, UK, Canada, and Australia all impose mandatory reporting and criminal liability. Organizations operating globally face multi-jurisdictional obligations that the AUP alone does not satisfy. CONTRACT AND VENDOR IMPLICATIONS: Procurement teams should treat this as a baseline compliance requirement in any vendor agreement involving AI content generation. B2B contracts should include explicit representations and warranties from API users that their platforms implement content filtering and detection measures consistent with legal obligations. COMPLIANCE CONSIDERATIONS: Organizations should implement and document technical controls to prevent CSAM generation, establish incident response procedures for detection and mandatory reporting, and conduct periodic audits of content moderation systems. Legal counsel should confirm whether the organization qualifies as an electronic service provider subject to mandatory CyberTipline reporting under 18 U.S.C. 2258A.

Applicable agencies

Provision details

Other risks in this policy

• Prohibited Use: Weapons of Mass Destruction and CBRN Content high
• Prohibited Use: Cyberweapons and Critical Infrastructure Attacks high
• Prohibited Use: Non-Consensual Synthetic Media and Deepfakes high
• Prohibited Use: Election Interference and Political Disinformation medium
• Operator Responsibility for Downstream Use high
• Restriction on Autonomous High-Stakes Decision-Making Without Human Oversight high