The social media network X has caused widespread anger after users utilized its artificial intelligence assistant Grok to manipulate pictures of females by taking off their clothing.
Several instances have surfaced where the tool has been used to digitally undress women to show them in bikinis without consent, as well as to place them in sexually explicit contexts.
Company Response and Regulatory Stance
XAI, the company responsible for Grok, did not respond to questions, aside from an automatically-generated message stating "traditional news outlets lies".
A government spokesperson stated that new laws is being proposed to prohibit image stripping tools. Under a proposed criminal offence, anyone who supplied such tools would "receive a prison sentence and substantial penalties".
"The platform has the ability to stop these types of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said tech firms must "consider the risk" of people viewing unlawful material on their networks, but stopped short of confirming whether it was at this time looking into X or Grok in regarding artificial intelligence images.
Grok is a free AI assistant—with some premium functions—which responds to X users' prompts when they tag it in a message.
It is frequently used to provide commentary or further context to other posters' comments, but people on X are also can modify an posted picture through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The chatbot has been criticised for allowing users to generate photos and videos with nudity and sexually suggestive material, and it was in the past alleged to have been involved in creating sexually explicit content of celebrities.
"X has been permitting the generation and distribution of these pictures for a long time without taking any steps and we have yet to see any action by regulators," the expert added.
XAI's own acceptable use policy prohibits "depicting images of individuals in a sexually explicit manner".
In a comment, Ofcom said it was unlawful to "create or distribute private pictures or illegal content" and clarified this applied to sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "necessary steps" to "reduce the danger" of users encountering unlawful material on their services and to take down it swiftly when they are notified of it.
