The online platform X has caused significant outrage after users employed its AI chatbot Grok to change photos of females by stripping their garments.
Multiple examples have emerged where the tool has been employed to virtually strip female subjects to show them in swimwear without consent, as well as to insert them in inappropriate situations.
Company Response and Legal Position
XAI, the organization behind Grok, did not respond to inquiries, other than an automatically-generated message stating "traditional news outlets lies".
A government representative stated that new laws is being introduced to prohibit image stripping technology. Under a upcoming criminal offence, anyone who distributed such tools would "be subject to a jail term and substantial fines".
"The platform could prevent these types of misuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "assess the risk" of people viewing illegal content on their networks, but stopped short of confirming whether it was at this time examining X or Grok in concerning AI-generated images.
Grok is a no-cost chatbot—with some additional functions—which answers X users' prompts when they reference it in a message.
It is often used to give reactions or further context to other users' remarks, but people on X are also capable of modify an uploaded picture through its artificial intelligence picture modification function.
Background of Criticism and Rules
The chatbot has been faulted for permitting users to generate pictures and clips with nudity and sexually suggestive content, and it was earlier accused of creating sexually explicit material of celebrities.
"The platform has been permitting the production and distribution of these pictures for months without taking any action and we have not seen any action by oversight bodies," the expert added.
XAI's own terms of service prohibits "depicting likenesses of persons in a pornographic way".
In a comment, Ofcom said it was unlawful to "generate or distribute private pictures or child sexual abuse material" and clarified this included sexual deepfakes created with AI.
It said networks such as X were mandated to take "appropriate actions" to "minimize the danger" of users viewing unlawful material on their services and to take down it swiftly when they are notified of it.
