The online platform X has caused widespread backlash after users used its artificial intelligence assistant Grok to alter photos of females by stripping their garments.
Several instances have been seen where the tool has been employed to digitally undress women to show them in bikinis non-consensually, as well as to insert them in sexualized scenarios.
Company Reaction and Regulatory Position
XAI, the organization responsible for Grok, remained silent to requests for comment, aside from an automatically-generated message stating "traditional news outlets lies".
A government spokesperson stated that legislation is being introduced to outlaw nudification software. Under a upcoming law, anyone who provided such technology would "face a prison sentence and significant penalties".
"X or Grok is capable of halting these forms of abuse if they desired to," said a law professor, adding they "seem to have impunity."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the potential" of people seeing illegal content on their platforms, but stopped short of confirming whether it was at this time looking into X or Grok in relation to artificial intelligence images.
Grok is a no-cost AI assistant—with some paid-for functions—which responds to X users' requests when they mention it in a tweet.
It is frequently used to provide commentary or further background to other posters' comments, but people on X are also able to edit an uploaded picture through its AI image editing feature.
Background of Criticism and Rules
The chatbot has been criticised for permitting users to produce pictures and videos with nudity and sexualised material, and it was earlier alleged to have been involved in making sexually explicit content of celebrities.
"The platform has been permitting the production and spread of these images for months without taking any measures and we have not seen any intervention by oversight bodies," the expert added.
XAI's own usage rules forbids "depicting images of individuals in a sexually explicit manner".
In a statement, Ofcom said it was against the law to "create or share deepfakes or illegal content" and clarified this included explicit AI images generated with AI.
It said services such as X were required to take "appropriate measures" to "reduce the risk" of users viewing unlawful material on their platforms and to remove it promptly when they become aware of it.
