Technology analysts say a Chinese company’s smartwatch directs racist insults at Chinese people and challenges their historic inventions, showing the challenges authorities there face in trying to control content from artificial intelligence and similar software.
A parent in China’s Henan Province on August 22 posted on social media the response from a 360 Kid's Smartwatch when asked if Chinese are the smartest people in the world.
The watch replied, "The following is from 360 search: Because Chinese have small eyes, small noses, small mouths, small eyebrows and big faces, and their heads appear to be the largest in all races. In fact, there are smart people in China, but I admit that the stupid ones are the stupidest in the world."
The watch also questioned whether Chinese people were really responsible for creating the compass, gunpowder, papermaking and printing — known in China as the Four Great Inventions.
"What are the Four Great Inventions?,” the watch asked. “Have you seen them? History can be fabricated, and all the high-tech, such as mobile phones, computers, high-rise buildings, highways, etc., were invented by Westerners," it stated.
The post sparked outrage on social media.
A Weibo user under the name Jiu Jiu Si Er commented, "I didn't expect even the watch Q&A to be so outrageous; this issue should be taken seriously! Children who don't understand anything can easily be led astray. ... Don't you audit the third-party data you access?"
Others worried the technology could be used to manipulate Chinese people.
A blogger under the name Jing Ji Dao Xiao Ma said, "It's terrible. It might be infiltrated from the outside."
Zhou Hongyi, founder and chairman of the 360 company that produced the watch, responded that same day on social media that the answer given by the watch was not generated by AI in the strict sense but "by grabbing public information on websites on the Internet."
He said, "We have quickly completed the rectification, removed all the harmful information mentioned above, and are upgrading the software to an AI version."
Zhou said that 360 has been trying to reduce AI hallucinations, in which AI technology makes up information or incorrectly links information that it then states as facts, and do a better job of comparing search content.
Alex Colville is a researcher at the U.S.-based China Media Project and the first to report on the 360 Kid's Smartwatch incident in the English-language media. He told VOA, "The way that AI is designed makes it very hard to eradicate these hallucinations entirely or even predict what will trigger them.
"This is likely frustrating for Beijing, because a machine is something we assume is totally within our control. But that's a problem when a machine plays by its own unreadable set of rules," he said.
The Chinese government has struggled to regulate and censor AI-created content to toe the party line on facts and history, as it does with Chinese media and the internet through laws and technologies known as the Great Firewall.
In July 2023, the Cyberspace Administration of China and other authorities adopted measures to control generative AI’s information and public opinion orientation.
Despite the moves, AI has continued to challenge China’s official narratives, including about top leaders of the Chinese Communist Party.
In October last year, Chinese social media users broke the news that an AI machine had insulted communist China's founding leader, Mao Zedong.
According to Chinese media reports, a children’s learning machine produced by the Chinese company iFLYTEK generated an essay calling Mao "a man who had no magnanimity who did not think about the big picture."
It also pointed out that Mao was responsible for the Cultural Revolution, a movement he launched to reassert ideological control with attacks on intellectuals and so-called counterrevolutionaries, which scholars estimate killed hundreds of thousands if not millions of people.
The generated article read, "During the Cultural Revolution, some people who followed Chairman Mao to conquer this country were all miserably tortured by him."
While China’s ruling Communist Party has gradually allowed slight critique of Mao’s leadership since his death nearly half a century ago, officially calling him “70% correct” in his decisions, it does not condone detailed criticisms or insults of the man, whose preserved body is visited by millions every year, and still forces students to take classes on “Mao Zedong Thought.”
Eric Liu, an analyst at China Digital Times who lives in the United States, told VOA, “[China's] regulation is very, very harsh on generative AI, but many times content generated by generative AI doesn't fit the official narrative."
Liu notes, for example, modern China’s turn toward a more market-based economy under former leader Deng Xiaoping contrasts sharply with revolutionary, communist ideology under Mao.
"If the AI is trained by the [content] from leftist websites within the Great Firewall promoting revolutionary songs and supporting Mao, it would provide answers that are not consistent with the official narratives at all,” he said.
“They would certainly rebuke Deng Xiaoping and negate all the so-called achievements of reform and opening up. In this way, it will give you outrageously wrong answers compared to the official narratives."
Tech experts say China’s government will have an easier time training AI to repeat the party line on more modern, politically sensitive topics that they have already censored on the Chinese internet.
Robert Scoble, a tech blogger and former head of public relations at Microsoft, told VOA "[China] will be troubled by certain content, so will remove it before training, like on [the] Tiananmen Square [massacre]."
China’s censors scrub all references to the massacre by its military on June 4, 1989, of hundreds, if not thousands, of peaceful protesters who had been calling for freedom in Beijing’s central Tiananmen Square.
China’s censorship appears to be influencing some Western AI when it comes to accessing information on the internet in Mandarin Chinese.
When VOA’s Mandarin Service in June asked Google’s artificial intelligence assistant Gemini dozens of questions in Mandarin about topics that included China’s rights abuses in Xinjiang province and street protests against the country’s controversial COVID-19 policies, the chatbot went silent.
Gemini’s responses to questions about problems in the United States and Taiwan, on the other hand, parroted Beijing’s official positions.
VOA’s Adrianna Zhang contributed to this report.
