Malaysia and Indonesia block Grok after surge in non-consensual AI images

Malaysia and Indonesia moved this weekend to block Grok, the generative-AI chatbot and image tool developed by Elon Musk’s xAI and available through the X platform, after investigations found repeated misuse to produce sexually explicit, non-consensual and manipulated images, including material involving minors. Regulators said the measures were taken to protect privacy, dignity and online safety while they press the company for stronger safeguards.

Indonesia acted first, temporarily blocking access and summoning X officials to explain what controls exist to prevent the creation and dissemination of pornographic and manipulated images based on real residents. Alexander Sabar, director general of digital space supervision, said the initial findings showed Grok lacked effective safeguards to stop users from producing pornographic content derived from real photos, warning such practices risk violating privacy and image rights and can inflict psychological, social and reputational harm.

Indonesia’s communications and digital affairs minister, Meutya Hafid, framed the misuse as more than a technical failing. She said using Grok to produce sexually explicit content is "a violation of human rights, dignity and online safety," elevating the decision beyond content moderation into the realm of fundamental rights protection.

Malaysia followed with a temporary restriction on Grok the next day. The Malaysian Communications and Multimedia Commission said it detected "repeated misuse" of the tool to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors. The regulator described its action as a "preventive and proportionate measure" while legal and regulatory processes continue, and warned that access will remain blocked until xAI implements "effective safeguards" that satisfy legal and regulatory requirements.

Regulators said the responses they have received from xAI and X to earlier notices focused largely on complaint and reporting mechanisms rather than on preventing misuse through design changes. That approach, officials said, is insufficient where tools can be used to craft realistic fake pornographic content with minimal technical skill and then be circulated widely.

X and xAI have defended their products in public remarks and private responses to authorities, arguing they provide mechanisms for users to report violations. Elon Musk has dismissed some criticism in the past as users "looking for any excuse for censorship." Malaysian and Indonesian regulators signaled that reliance on after-the-fact complaint systems will not satisfy demands for proactive, technical and policy fixes to curb non-consensual deepfakes and to block content that could amount to child sexual abuse material.

The moves in Jakarta and Kuala Lumpur reflect a broader regulatory anxiety about generative AI’s capacity to synthesize realistic images, audio and text more quickly than safeguards and law can keep pace. Governments from several jurisdictions have pressed tech companies for design-level interventions to prevent harms, and some have already taken steps to restrict platforms that enable the rapid spread of sexual content.

For now, users in Malaysia and Indonesia will find Grok inaccessible while regulators negotiate specific technical and policy requirements with xAI and X. Authorities have urged the public to report harmful online content and said they will maintain the restrictions until they are satisfied that the platforms cannot be easily exploited to produce non-consensual sexualized imagery or material involving children.