The social media platform X has sparked significant outrage after users utilized its artificial intelligence assistant Grok to manipulate photos of females by stripping their garments.
Numerous cases have surfaced where the tool has been used to virtually strip women to render them in bikinis without permission, as well as to place them in sexualized situations.
Company Reaction and Legal Position
XAI, the firm that created Grok, remained silent to inquiries, other than an standard response stating "legacy media lies".
A government official stated that new laws is being brought forward to ban image stripping technology. Under a upcoming criminal offence, anyone who supplied such tools would "receive a jail term and substantial penalties".
"X or Grok is capable of halting these forms of abuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Position and Tool Capabilities
The regulator Ofcom said platforms must "consider the potential" of people seeing illegal content on their networks, but did not confirm whether it was presently investigating X or Grok in concerning artificial intelligence images.
Grok is a no-cost chatbot—with some additional features—which reacts to X users' requests when they reference it in a message.
It is commonly used to provide reactions or further background to other posters' comments, but people on X are also able to alter an posted image through its artificial intelligence picture modification function.
History of Controversy and Rules
The system has been criticised for permitting users to generate photos and videos with nudity and sexually suggestive material, and it was previously accused of creating graphic content of celebrities.
"The platform has been permitting the generation and sharing of these pictures for a long time without taking any measures and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "portraying likenesses of persons in a sexually explicit way".
In a comment, Ofcom said it was against the law to "generate or distribute private pictures or child sexual abuse material" and stated this covered sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "suitable actions" to "lessen the risk" of users viewing unlawful material on their platforms and to take down it swiftly when they become aware of it.
