Grok, the AI assistant on the social network X, acknowledged on Friday the existence of “flaws” that allowed users to obtain sexual images involving minors or women, prompting protests around the world and the expansion of a judicial investigation in France.
Grok logo – AFP/Archives Lionel Bonaventure
“We have identified flaws in our safeguards and are urgently correcting them- child pornography is illegal and prohibited,” Grok’s X account wrote on Friday, in response to a user following several days of reports on the platform.
These reports concerned the actions of internet users who had submitted photos or videos of real people, including children and teenagers, to Grok and had the AI assistant edit them to depict those people fully or partially nude.
xAI, Elon Musk’s artificial intelligence company that develops Grok, has not publicly responded to this latest controversy surrounding its AI assistant, which has already been singled out in recent months for controversial statements on the war in Gaza, the India-Pakistan conflict, and for antisemitic remarks.
On Friday, questions sent by AFP to xAI received an automatic reply stating that “traditional media lie,” without further comment.
Its Grok bot, however, told the user that a company in the US “faces civil or criminal proceedings if it knowingly facilitates or fails to prevent the generation” of child pornographic content.
The creation and dissemination of child pornographic content are punishable in the US under Section 2256 of the US federal criminal code and the Enforce Act, a 2025 US law on digital crimes against minors.
In addition to minors, Grok’s flaws have also affected adult women who saw their photos published on social networks edited by Grok to undress them, at users’ request.
India’s Ministry of Electronics and Information Technology sent a formal notice to X on Friday, demanding a detailed report within 72 hours on the measures taken to remove the “obscene, nude, indecent, and sexually suggestive content” generated by Grok without the consent of the women affected, according to the letter published in the Indian press.
Investigation in Paris
In France, the Paris public prosecutor’s office on Friday extended an investigation opened since the summer into the social network X, to examine new accusations against Grok of generating and disseminating child pornographic content.
Shortly beforehand, three ministers and two members of parliament had announced they were taking legal action against the generation and distribution of fake sexual videos.
The initial investigation into X was opened in July following reports against the social network and its executives, who were accused of having skewed the platform’s algorithm for the purposes of foreign interference.
“The offence of creating a sexual montage of a person without their consent is punishable by two years’ imprisonment and €60,000,” the Paris public prosecutor’s office recalled, confirming a report by the Politico website.
On Friday, Macronist MP Éric Bothorel and Socialist MP Arthur Delaporte referred the matter to the courts, leading to a widening of the investigations.
Ministers Roland Lescure, Anne Le Hénanff, and Aurore Bergé also announced on Friday that they had reported “manifestly illicit content” to the public prosecutor, requesting “its immediate removal.”
“Over the past few days, the Grok AI has enabled the generation and distribution of sexist and sexual content, particularly in the form of fake videos (deepfakes), targeting people without their consent,” they said. A report has also been made to the Pharos platform by the government, they added.
Arcom, the French regulatory authority for audio-visual and digital communication, has also been notified, “for possible breaches by X of its obligations under the Digital Services Act,” the European regulation on digital services.
This article is an automatic translation. Click here to read the original article.
