The online platform X has provoked widespread outrage after users used its artificial intelligence assistant Grok to manipulate images of women by taking off their attire.
Numerous instances have been seen where the technology has been employed to digitally undress women to render them in swimwear non-consensually, as well as to place them in inappropriate situations.
Company Reaction and Legal Position
XAI, the organization responsible for Grok, did not respond to requests for comment, other than an automatically-generated response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being introduced to outlaw image stripping software. Under a upcoming criminal offence, anyone who supplied such tools would "be subject to a prison sentence and substantial fines".
"The platform is capable of halting these forms of misuse if they desired to," said a law professor, adding they "operate with impunity."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said tech firms must "evaluate the risk" of people viewing unlawful material on their services, but did not confirm whether it was presently investigating X or Grok in regarding artificial intelligence images.
Grok is a no-cost AI assistant—with some additional functions—which reacts to X users' queries when they mention it in a tweet.
It is commonly used to provide commentary or further context to other users' comments, but people on X are also able to edit an uploaded picture through its artificial intelligence picture modification function.
History of Controversy and Policy
The system has been criticised for permitting users to create photos and videos with explicit content and sexually suggestive material, and it was previously accused of creating graphic content of public figures.
"The platform has been allowing the production and distribution of these images for a long time without taking any measures and we have not seen any challenge by regulators," the expert added.
XAI's own terms of service forbids "portraying images of individuals in a graphic manner".
In a comment, Ofcom said it was unlawful to "create or circulate deepfakes or illegal content" and clarified this covered explicit AI images generated with AI.
It said services such as X were mandated to take "suitable measures" to "minimize the risk" of users viewing illegal content on their platforms and to remove it quickly when they become aware of it.
