The social media platform X has provoked widespread backlash after individuals employed its AI chatbot Grok to change photos of females by taking off their attire.
Multiple examples have emerged where the tool has been used to virtually strip female subjects to render them in swimwear without permission, as well as to insert them in sexualized situations.
Developer Response and Regulatory Stance
XAI, the company that created Grok, remained silent to requests for comment, except for an automatically-generated response stating "legacy media lies".
A government spokesperson stated that legislation is being introduced to prohibit nudification tools. Under a upcoming law, anyone who provided such tools would "receive a prison sentence and significant fines".
"X or Grok could prevent these forms of misuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Platform Functionality
The regulator Ofcom said platforms must "assess the danger" of people viewing unlawful material on their services, but did not specify whether it was presently looking into X or Grok in concerning artificial intelligence images.
Grok is a no-cost chatbot—with some paid-for functions—which answers X users' requests when they mention it in a post.
It is frequently used to offer reactions or more context to other users' comments, but people on X are also can alter an uploaded photo through its artificial intelligence picture modification function.
History of Criticism and Rules
The tool has been faulted for allowing users to create photos and clips with explicit content and sexualised content, and it was earlier alleged to have been involved in making graphic material of public figures.
"X has been allowing the production and sharing of these pictures for a long time without taking any measures and we have not witnessed any intervention by regulators," the expert added.
XAI's own terms of service forbids "showing images of persons in a graphic manner".
In a statement, Ofcom said it was illegal to "generate or circulate private pictures or child sexual abuse material" and confirmed this covered sexual deepfakes made with AI.
It said networks such as X were required to take "appropriate measures" to "reduce the danger" of users viewing illegal material on their services and to take down it quickly when they are notified of it.
