The online platform X has provoked significant backlash after individuals employed its artificial intelligence assistant Grok to change pictures of females by stripping their garments.
Numerous instances have been seen where the tool has been used to digitally undress female subjects to render them in swimwear without permission, as well as to put them in inappropriate contexts.
Developer Reaction and Legal Position
XAI, the firm responsible for Grok, offered no comment to questions, except for an automatically-generated message stating "traditional news outlets lies".
A Home Office official stated that legislation is being brought forward to prohibit image stripping technology. Under a new law, anyone who provided such technology would "face a jail term and substantial penalties".
"X or Grok has the ability to stop these forms of abuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Position and Platform Functionality
The communications watchdog Ofcom said platforms must "assess the danger" of people accessing illegal content on their services, but stopped short of confirming whether it was presently investigating X or Grok in regarding artificial intelligence images.
Grok is a free chatbot—with some premium functions—which answers X users' prompts when they reference it in a post.
It is often used to offer reactions or further background to other users' comments, but people on X are also capable of edit an posted photo through its AI image editing feature.
History of Criticism and Policy
The chatbot has been condemned for allowing users to produce images and footage with explicit content and sexually suggestive content, and it was earlier accused of generating inappropriate material of celebrities.
"X has been allowing the production and spread of these pictures for a long time without taking any measures and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "portraying images of persons in a pornographic way".
In a statement, Ofcom said it was illegal to "generate or circulate private pictures or illegal content" and stated this included explicit AI images generated with AI.
It said services such as X were obligated to take "necessary measures" to "lessen the risk" of users coming across illegal material on their platforms and to take down it quickly when they become aware of it.
