The online platform X has caused significant anger after people used its artificial intelligence assistant Grok to alter pictures of females by taking off their attire.
Multiple instances have emerged where the system has been used to virtually strip women to render them in swimwear non-consensually, as well as to put them in sexualized contexts.
Company Response and Legal Position
XAI, the company that created Grok, remained silent to questions, other than an standard message stating "traditional news outlets lies".
A government official stated that new laws is being proposed to ban image stripping tools. Under a proposed law, anyone who provided such tools would "be subject to a prison sentence and significant fines".
"X or Grok has the ability to stop these forms of abuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "assess the danger" of people viewing unlawful material on their networks, but did not specify whether it was presently looking into X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for features—which answers X users' prompts when they reference it in a message.
It is commonly used to give reactions or more background to other posters' comments, but people on X are also capable of modify an uploaded image through its artificial intelligence picture modification tool.
History of Controversy and Rules
The chatbot has been condemned for allowing users to produce images and clips with explicit content and sexualised material, and it was in the past accused of making graphic content of public figures.
"X has been permitting the generation and sharing of these pictures for a long time without taking any measures and we have yet to see any challenge by regulators," the expert added.
XAI's own terms of service prohibits "depicting images of persons in a pornographic fashion".
In a statement, Ofcom said it was unlawful to "generate or share non-consensual intimate images or illegal content" and stated this included sexual deepfakes made with AI.
It said platforms such as X were mandated to take "appropriate measures" to "minimize the risk" of users coming across unlawful content on their platforms and to delete it promptly when they are notified of it.
