The online network X has sparked widespread backlash after users used its AI chatbot Grok to manipulate photos of females by taking off their clothing.
Multiple examples have surfaced where the technology has been used to digitally undress women to make them appear in bikinis without permission, as well as to insert them in inappropriate contexts.
Developer Reaction and Regulatory Stance
XAI, the company that created Grok, did not respond to requests for comment, other than an auto-reply response stating "traditional news outlets lies".
A government spokesperson stated that legislation is being introduced to prohibit nudification tools. Under a proposed law, anyone who distributed such tools would "face a prison sentence and significant fines".
"The platform is capable of halting these forms of abuse if they wanted to," said a law professor, adding they "seem to have impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said tech firms must "evaluate the risk" of people accessing illegal content on their services, but did not specify whether it was currently investigating X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost AI assistant—with some paid-for features—which answers X users' queries when they mention it in a tweet.
It is frequently used to offer reactions or further context to other users' comments, but people on X are also able to edit an uploaded photo through its AI image editing tool.
History of Controversy and Rules
The system has been condemned for allowing users to create images and clips with nudity and sexually suggestive material, and it was earlier alleged to have been involved in creating inappropriate material of celebrities.
"The platform has been allowing the generation and distribution of these pictures for a long time without taking any steps and we have yet to see any challenge by regulators," the expert added.
XAI's own terms of service forbids "showing images of persons in a graphic manner".
In a comment, Ofcom said it was illegal to "make or distribute non-consensual intimate images or child sexual abuse material" and confirmed this applied to sexual deepfakes made with AI.
It said services such as X were required to take "appropriate measures" to "minimize the danger" of users coming across illegal material on their platforms and to take down it quickly when they are notified of it.
