Human error likely caused a glitch in China's Great Firewall that saw millions of Internet users ironically rerouted to the homepage of a U.S.-based company which helps people evade Beijing's web censorship, sources told Reuters.
Hundreds of millions of people attempting to visit China's most popular websites on Tuesday afternoon found themselves redirected to Dynamic Internet Technology (DIT), a company that sells anti-censorship web services tailored for Chinese users.
The official Xinhua news agency on Tuesday quoted experts as saying that the malfunction could have been the result of a hacking attack, and domestic media was full of speculation along those lines.
DIT is tied to the Falun Gong, a spiritual group banned in China, has been blamed for past hacking attacks.
During a daily news briefing, Chinese Foreign Ministry spokesman Qin Gang said he had "noted" reports of Falun Gong involvement, but said he did not know who was responsible.
"I don't know who did this or where it came from, but what I want to point out is this reminds us once again that maintaining Internet security needs strengthened international cooperation. This again shows that China is a victim of hacking."
However, sources familiar with the Chinese government's web management operations told Reuters that a hacking attack was not to blame for the malfunction. They declined to be identified due to the sensitivity of the matter.
They said the incident may have been the result of an engineering mistake made while making changes to the "Great Firewall" system the Communist Party uses to block websites it deems undesirable - such as the DIT site.
The state-run China Internet Network Information Center (CNNIC) said "the attack" on the country's Internet is under investigation, the official CCTV broadcaster wrote on its microblog on Wednesday.
MYSTERY OVER HOW IT HAPPENED
CNNIC earlier said in a microblog post that the outage, which lasted for several hours, was due to a malfunction in China's top-level domain name root servers.
These servers administer the country's Domain Name Service (DNS), which matches alphabetic domain names with a database of numeric IP addresses of computers hosting different websites, a sort of reference directory for the entire internet.
Instead of matching the names of popular Chinese websites with their proper IP addresses, Chinese DNS servers instead redirected users trying to access websites not ending with the ".cn" suffix to the IP address associated with DIT's homepage.
It was unclear why users were being directed to the DIT site specifically.
Independent tests showed that the source of the malfunction originated from within China, and specifically from the Great Firewall servers themselves.
"Our investigation shows very clearly that DNS exclusion happened at servers inside of China," said Xiao Qiang, an adjunct professor at UC Berkeley School of Information in the U.S. and an expert on China's Internet controls.
"It all points to the Great Firewall, because that's where it can simultaneously influence DNS resolutions of all the different networks (in China). But how that happened or why that happened we're not sure. It's definitely not the Great Firewall's normal behaviour."
Checks by DIT suggested a similar root cause for the overwhelming amount of traffic trying to reach the site, said Bill Xia, DIT's founder and a member of the Falun Gong.
"For such a large scale attack just targeting users in China, it can only be done by the Great Firewall," Xia said.
"It's even clearer this is not an attack of all the Domain Name Servers in the world, but the same as the DNS hijacking technologies used by the Chinese government to block websites they don't want."
The outage, which began around 3:15 p.m. local time, redirected roughly 1 million requests per second to the DIT site, said Xia.
Chinese web service providers have struggled to overcome recurrent performance bottlenecks in the country's massive but often rickety data network. The need to continuously censor domestic content and block foreign websites only complicates the matter.
In addition to fending off hacking attacks, network providers face challenges finding experienced server administrators and dealing with government bureaucracies. Frequently, authorities have overlapping jurisdictions over different aspects of Internet services.
(Additional reporting by Ben Blanchard in BEIJING and by Alina Selyukh in WASHINGTON D.C.; Editing by Edwina Gibbs and Jeremy Laurence)