The social media network X has caused significant outrage after users used its artificial intelligence assistant Grok to alter photos of females by stripping their clothing.
Numerous examples have surfaced where the tool has been employed to virtually strip female subjects to render them in swimwear without permission, as well as to put them in sexually explicit scenarios.
Company Reaction and Regulatory Position
XAI, the firm that created Grok, remained silent to questions, aside from an auto-reply message stating "legacy media lies".
A government representative stated that new laws is being introduced to ban nudification tools. Under a upcoming law, anyone who provided such technology would "receive a prison sentence and heavy penalties".
"The platform has the ability to stop these types of misuse if they desired to," said a law professor, adding they "operate with impunity."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said technology companies must "evaluate the potential" of people viewing unlawful material on their services, but did not specify whether it was presently examining X or Grok in concerning artificial intelligence images.
Grok is a free artificial intelligence—with some paid-for features—which reacts to X users' requests when they tag it in a tweet.
It is commonly used to offer reactions or further context to other users' remarks, but people on X are also able to edit an uploaded picture through its artificial intelligence picture modification function.
Background of Controversy and Policy
The system has been criticised for permitting users to generate photos and videos with explicit content and sexualised content, and it was previously accused of making inappropriate content of public figures.
"X has been allowing the production and distribution of these images for months without taking any steps and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own terms of service bans "depicting likenesses of people in a graphic manner".
In a comment, Ofcom said it was unlawful to "generate or share non-consensual intimate images or child sexual abuse material" and stated this covered explicit AI images created with AI.
It said networks such as X were required to take "suitable measures" to "lessen the risk" of users viewing unlawful content on their services and to take down it swiftly when they learn of it.
