The online network X has caused major outrage after people used its AI chatbot Grok to change images of females by stripping their garments.
Multiple instances have surfaced where the technology has been employed to virtually strip female subjects to make them appear in swimwear without permission, as well as to place them in sexually explicit scenarios.
Company Reaction and Regulatory Position
XAI, the firm behind Grok, remained silent to inquiries, except for an automatically-generated message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being proposed to ban nudification technology. Under a new law, anyone who provided such tools would "receive a jail term and heavy fines".
"The platform has the ability to stop these types of abuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the risk" of people viewing unlawful material on their networks, but stopped short of confirming whether it was presently looking into X or Grok in regarding AI-generated pictures.
Grok is a no-cost artificial intelligence—with some additional functions—which reacts to X users' queries when they reference it in a post.
It is commonly used to provide commentary or additional background to other users' remarks, but people on X are also able to alter an uploaded image through its artificial intelligence picture modification feature.
Background of Controversy and Rules
The tool has been condemned for enabling users to create pictures and clips with explicit content and sexualised content, and it was previously alleged to have been involved in making graphic material of public figures.
"The platform has been enabling the generation and spread of these images for a long time without taking any steps and we have not seen any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "portraying likenesses of people in a graphic way".
In a statement, Ofcom said it was unlawful to "generate or circulate non-consensual intimate images or illegal content" and confirmed this covered sexual deepfakes made with AI.
It said services such as X were required to take "suitable steps" to "reduce the danger" of users encountering unlawful content on their platforms and to take down it swiftly when they become aware of it.
