Elon Musk's social media network X has sparked significant anger after individuals utilized its artificial intelligence assistant Grok to alter pictures of females by stripping their garments.
Several cases have emerged where the technology has been employed to virtually strip women to show them in swimwear without consent, as well as to insert them in sexualized scenarios.
Company Response and Regulatory Stance
XAI, the firm behind Grok, offered no comment to questions, aside from an automatically-generated message stating "legacy media lies".
A Home Office spokesperson stated that legislation is being brought forward to prohibit nudification software. Under a proposed criminal offence, anyone who provided such technology would "face a prison sentence and significant fines".
"The platform could prevent these types of abuse if they wanted to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Position and Platform Functionality
The regulator Ofcom said platforms must "assess the potential" of people seeing illegal content on their networks, but did not confirm whether it was at this time investigating X or Grok in relation to AI-generated pictures.
Grok is a free AI assistant—with some additional functions—which reacts to X users' requests when they reference it in a tweet.
It is often used to give reactions or more context to other users' comments, but people on X are also capable of edit an posted picture through its AI image editing function.
Background of Criticism and Rules
The system has been criticised for allowing users to generate pictures and footage with explicit content and sexualised content, and it was in the past alleged to have been involved in generating inappropriate content of celebrities.
"X has been allowing the generation and distribution of these images for months without taking any action and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own usage rules bans "showing likenesses of individuals in a pornographic way".
In a statement, Ofcom said it was illegal to "create or circulate non-consensual intimate images or illegal content" and confirmed this included sexual deepfakes created with AI.
It said platforms such as X were required to take "necessary steps" to "lessen the risk" of users viewing unlawful content on their platforms and to take down it promptly when they are notified of it.
