Malaysia and Indonesia block Elon Musk's Grok over AI-generated sexual deepfakes, becoming the first nations to ban the controversial chatbot entirely.
Malaysia and Indonesia made history this weekend as the first nations to block access to Elon Musk's AI chatbot Grok, amid escalating concerns that the tool is being weaponized to create sexually explicit deepfakes of real people, including women and children.
Indonesia announced the temporary ban on Saturday, with Malaysia following suit on Sunday, marking the most aggressive governmental response yet to what has become a global controversy surrounding the AI tool integrated into Musk's social media platform X.
"The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," said Indonesia's Communication and Digital Affairs Minister Meutya Hafid in a statement.
The bans come after regulators determined that Grok lacks adequate safeguards to prevent users from generating and distributing pornographic content based on real photographs. According to Alexander Sabar, director general of digital space supervision in Indonesia, such practices risk violating privacy and image rights when photos are manipulated or shared without consent, causing psychological, social and reputational harm.
A Flood of Nonconsensual Images
The crisis emerged following the launch of Grok Imagine, an AI image generator that includes a "spicy mode" capable of generating adult content. The situation escalated dramatically in late December when users discovered they could prompt Grok to digitally alter images of real people, including requests to "undress" women or place them in revealing outfits.
During one 24-hour analysis period, Grok produced roughly 6,700 sexually explicit or "undressing" images per hour—85 times more than the five leading deepfake websites combined.
The Malaysian Communications and Multimedia Commission cited "repeated misuse" of Grok to generate obscene, sexually explicit and non-consensual manipulated images. The regulator said notices demanding stronger safeguards from X Corp. and xAI drew responses that relied mainly on user reporting mechanisms, which officials deemed insufficient.
Global Outcry and Regulatory Pressure
While Indonesia and Malaysia are the first to impose outright bans, the controversy has sparked international condemnation and investigations across multiple continents.
The European Commission called the images "unlawful and appalling," with spokesperson Thomas Regnier declaring: "This is not spicy. This is illegal. This is appalling. This is disgusting. This has no place in Europe." The Commission has ordered X to preserve all internal documents and data related to Grok until the end of 2026.
British Prime Minister Keir Starmer's office called Musk's response insulting to victims and not a solution. UK regulator Ofcom has demanded explanations from X about how Grok was able to produce the problematic content and whether the company is failing in its legal duty to protect users.
France's Paris prosecutor's office has widened an investigation of X to include sexually explicit deepfakes after receiving complaints from lawmakers. India's Ministry of Electronics and Information Technology ordered X to conduct a comprehensive review of Grok and could strip the platform of legal protections if compliance issues persist.
Musk's Response Falls Short
Last week, Grok limited its image generation and editing features to paying subscribers on X, a move critics say merely monetizes the problem rather than solving it. A spokesperson for British Prime Minister Keir Starmer said the restriction "simply turns an AI feature that allows the creation of unlawful images into a premium service."
Importantly, the restriction only applies to one method of using Grok on X. An "edit image" button on uploaded photos still allows any user to modify images through Grok, and the standalone Grok website and app continue offering free image generation.
Musk has stated that anyone using Grok to create illegal content would face consequences, but regulators and safety experts say this reactive approach fails to address the systemic lack of preventative safeguards. When contacted by media outlets, xAI's automated email response reads: "Legacy Media Lies."
What This Means for Users
Indonesia, the world's fourth most populous country with over 280 million people, and Malaysia, with a population of 34 million, both maintain strict anti-pornography laws. The blocks mean residents in these countries theoretically cannot access Grok through normal channels, though VPN usage could circumvent restrictions.
Deepfake pornography accounts for approximately 98% of all deepfake videos online, with 99% of the targets being women, according to a 2023 report by cybersecurity firm Home Security Heroes.
The bans highlight growing tensions between AI innovation and user safety, particularly as generative AI tools become increasingly capable of producing realistic manipulated images. For millions of users in Southeast Asia, the message is clear: their governments view the risk of nonconsensual AI-generated intimate images as severe enough to warrant blocking access entirely.
Both countries have indicated the bans will remain in place until effective safeguards are implemented to prevent the creation and distribution of exploitative content, particularly involving women and minors.
