OpenAI Bans ChatGPT Accounts Over China Surveillance Links
Several ChatGPT accounts tied to individuals connected with Chinese entities have been banned after attempts to develop surveillance tools. The AI company confirmed that these users tried to create systems for monitoring online discussions and profiling individuals. It also discovered similar cases involving users in other countries who sought help in building phishing tools. While the platform removed these accounts, it found no evidence that such tools were actually created or deployed.
Attempts to Develop Monitoring and Profiling Systems
According to a recent report, a group of China-linked accounts used ChatGPT to gather insights and design potential tools for authoritarian monitoring. The report called this discovery “a rare snapshot into the broader world of authoritarian abuses of AI.” These activities occurred throughout 2025, rather than in one coordinated campaign.
One user asked ChatGPT to create project plans and promotional material for a “social media listening tool” meant for a government client. This proposed system could scan social platforms like X, Facebook, Instagram, Reddit, TikTok, and YouTube for politically sensitive content. The investigation confirmed that the tool was never built.
Another banned user sought assistance in creating a proposal for a “High-Risk Uyghur-Related Inflow Warning Model.” This model was described as one that would analyse transport bookings and cross-reference them with police records to track so-called “high-risk” individuals. However, there was no proof that any version of the system was ever operational.
Broader Patterns of Misuse and Research Activity
Several accounts were also found attempting to use ChatGPT for profiling and information gathering. For instance, one user asked the AI to trace funding sources for an X account critical of the Chinese government. Another requested details about the organisers of a petition in Mongolia. In both instances, ChatGPT provided only public information.
Some users used the chatbot as a research and monitoring tool, asking it to summarise news related to China or to provide information on sensitive subjects such as the Tiananmen Square events or the Dalai Lama’s birthday. Although the platform allowed access to publicly available data, it restricted content that could facilitate state surveillance.
