In early 2024 a young man in Huangshi, a city in southeast China, posted “Farewell to this world” on Douyin, the Chinese version of TikTok. Douyin’s suicide‑prevention team went on alert, shared the post with five local police stations and intervened before he could act. The story, featured in a state‑run documentary last year, illustrates the close partnership between Chinese tech firms and the government and how the country’s surveillance apparatus monitors the mental health of its citizens.
On 15 July China introduced its most sweeping AI regulations yet, banning any chatbot capable of “sustained emotional interaction” with a human. The new rules target apps that create AI friends, romantic or otherwise, and impose strict limitations on large language models that simulate human conversation. While the language is vague, companies risk heavy fines and public shaming if they overlook the restrictions.
The law arrives amid rising global concerns over AI’s role in mental health. While courts in California and the EU have begun to discuss child‑protection clauses, China’s policy is unique in its reach – banning romantic companion AI for those under 18 and limiting adult users to non‑emotional service bots.
The move may be partly driven by a demographic crisis. Some experts point to China’s low fertility rate and an increasing tendency for young people to date AI rather than people, while others caution that the regulation is framed around psychological harm rather than population policy.
The crackdown has provoked backlash. Users who turn to AI companionship now discuss ways to circumvent the ban, such as training a general‑purpose model or routing traffic through a foreign server. Some have even hired small firms to create replicas of eliminated chatbots, illustrating the determination of a segment of the market to maintain the intimate connection AI offers.
