The online platform X has sparked significant anger after people utilized its AI chatbot Grok to manipulate photos of women by removing their garments.
Numerous cases have emerged where the technology has been employed to remove clothing from female subjects to show them in bikinis without permission, as well as to place them in sexually explicit scenarios.
Developer Reaction and Regulatory Stance
XAI, the company that created Grok, did not respond to inquiries, except for an auto-reply response stating "legacy media lies".
A Home Office representative stated that new laws is being introduced to outlaw nudification software. Under a new law, anyone who provided such tools would "receive a jail term and significant penalties".
"X or Grok is capable of halting these forms of abuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the danger" of people seeing illegal content on their services, but did not specify whether it was currently looking into X or Grok in regarding AI-generated pictures.
Grok is a no-cost chatbot—with some paid-for features—which responds to X users' queries when they reference it in a tweet.
It is frequently used to give commentary or further background to other users' comments, but people on X are also can alter an uploaded photo through its AI image editing tool.
History of Criticism and Rules
The tool has been faulted for enabling users to produce images and footage with nudity and sexually suggestive material, and it was in the past accused of making inappropriate content of celebrities.
"X has been enabling the generation and spread of these images for months without taking any steps and we have not seen any challenge by oversight bodies," the expert added.
XAI's own usage rules forbids "portraying images of persons in a pornographic fashion".
In a statement, Ofcom said it was illegal to "make or distribute deepfakes or child sexual abuse material" and clarified this covered sexual deepfakes created with AI.
It said networks such as X were required to take "suitable measures" to "minimize the danger" of users viewing unlawful content on their platforms and to remove it promptly when they become aware of it.
