FCC: A few (680,000) net neutrality comments lost in 'XML gaffe'
Views vanish in IT cock-up – claim
The US Federal Communications Commission misplaced a huge tranche of public comments on its net neutrality proposals – and has blamed its outdated IT system.
Four million missives were submitted via the FCC's website and in emails to staff; comments could be submitted in PDF, CSV, or plain text formats, and were converted into XML before they were published for everyone to review.
However, campaigners noticed a sizable chunk of submissions was missing from the published XML. Today, the FCC said that's because 680,000 comments didn't make it through the data conversion process.
"We think it’s important that people understand that much of the confusion stems from the fact that the Commission has an 18-year-old Electronic Comment Filing system (ECFS), which was not built to handle this unprecedented volume of comments nor initially designed to export comments via XML," the regulator said in a blog post on Tuesday.
"This forced the Commission’s information technology team to cobble solutions together MacGyver-style. Thanks to these creative efforts, we have been able to accommodate the surge in comments and release the comments as XML files for the first time in the FCC’s history, but not without some glitches."
The agency said part of the problem came from its use of the open-source tool Apache Solr: not all of the submitted comments were converted into XML by Solr due to a "technical error," it's claimed. There was also a problem with the inclusion of comments submitted via CSV.
In all, the FCC said it received about four million public comments on its proposals for the future of the internet, extending the deadline for submissions as so many were rolling in. As soon as the regulator released the XML files detailing them all, some people smelled a rat – the cache was way short of the FCC's four million figure.
Last week, the Sunlight Foundation published a report after counting up the XML comments. It found that the majority of early submissions to the FCC were supportive of net neutrality, but, overall, once the deadline was extended for more views from the public, 60 per cent of the published comments favored paid-for prioritization of network traffic.
That report alarmed the pro-neutrality Fight For The Future (FFTF) campaign, which had encouraged people to individually write to the FCC.
The FFTF group said it forwarded 777,364 pro-net neutrality comments from its supporters to the FCC, but after looking through the XML files, at least 339,811 of these views weren't found in the published data.
"We're still looking into it and doing a full analysis to make sure every comment is part of the official record," Fight for the Future's spokesman Evan Greer told The Register. "From our perspective, the FCC now has put out complete comment records and it's up to the Sunlight Foundation to update its study."
The FCC said that the updated XML files will be released in the New Year. ®
Interestingly, the Sunlight Foundation found more than half of the anti-neutrality comments that made it through the XML conversion were submitted using a form letter drawn up and publicized by a single organization – American Commitment – which has ties to groups funded by the libertarian Koch brothers.
There's nothing wrong with that under US law, but the foundation noted the vast majority of these filled-in form letters had little or no individual opinion – in other words, hundreds of thousands of anti-net neutrality comments were a cut'n'paste job.