The social media network X has sparked major backlash after people employed its AI chatbot Grok to change images of females by taking off their garments.
Numerous cases have been seen where the tool has been employed to virtually strip female subjects to make them appear in bikinis without permission, as well as to place them in sexually explicit contexts.
Company Response and Legal Position
XAI, the firm that created Grok, offered no comment to questions, other than an standard response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being introduced to outlaw image stripping software. Under a proposed criminal offence, anyone who provided such technology would "receive a prison sentence and heavy penalties".
"The platform has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Functionality
The regulator Ofcom said tech firms must "consider the danger" of people accessing illegal content on their services, but stopped short of confirming whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost artificial intelligence—with some paid-for features—which responds to X users' requests when they tag it in a post.
It is frequently used to offer reactions or additional context to other users' remarks, but people on X are also can edit an uploaded image through its artificial intelligence picture modification feature.
History of Criticism and Policy
The tool has been condemned for permitting users to create pictures and videos with nudity and sexualised content, and it was previously alleged to have been involved in creating inappropriate material of celebrities.
"The platform has been permitting the creation and sharing of these pictures for months without taking any action and we have not witnessed any intervention by regulators," the expert added.
XAI's own terms of service forbids "depicting images of individuals in a pornographic fashion".
In a comment, Ofcom said it was unlawful to "make or share deepfakes or illegal content" and stated this covered explicit AI images created with AI.
It said services such as X were required to take "suitable actions" to "reduce the danger" of users coming across unlawful material on their services and to delete it quickly when they learn of it.
