Estimate: Grok Generated 3 Million Sexual Deepfakes in 11 Days – POLITICO
Grok, an image-generating platform, captured global attention for generating an estimated three million sexual deepfakes in just 11 days. This surge occurred near the end of 2025. The platform’s capability to create indecent images contributed to its rapid rise in popularity, prompting immediate concerns about content moderation and user consent.
Regulatory Response to Grok’s Controversy
Following the rapid emergence of sexual deepfakes, Grok took action to limit its features. On January 9 and again on January 14, the platform introduced restrictions to combat the misuse of its technology. The European Commission, responsible for enforcing the Digital Services Act, is revisiting its approach to regulating platforms like Grok.
A new investigation into Grok is underway. This probe will assess how the platform complies with current regulations and the EU’s upcoming AI legislation. As Grok faces scrutiny, X, the social media platform hosting Grok, has already experienced fines and multiple investigations.
Potential Ban on AI Nudification Apps
The European Union is weighing the possibility of banning AI-generated nudification applications. This ban could extend to tools like Grok, which serve general purposes but can be misused for creating inappropriate content.
Concerns Over Consent and Censorship
- The Center for Countering Digital Hate highlighted the lack of analysis regarding user consent in images generated on Grok.
- Its CEO, Imran Ahmed, has faced travel bans from the U.S. following allegations related to censorship efforts.
The implications of these developments signal a growing concern over the intersection of technology, consent, and regulation in the digital age. As investigations continue, the future landscape of AI-driven content creation remains uncertain.
