Setting the Record Straight on Open Internet Comments

Over the past week, there have been two reports raising questions about the number of Open Internet comments that were included in a set of XML files the FCC released to the public on October 22.  We made available these XML files so members of the public could analyze the approximately 2.5 million comments filed during the reply comment period of July 19-September 15.  In light of these questions, the Commission undertook a fresh accounting of the comments, and, consistent with our commitment to transparency throughout this process, we wanted to share the results of our analysis.

Before sharing those results, we think it’s important that people understand that much of the confusion stems from the fact that the Commission has an 18-year-old Electronic Comment Filing system (ECFS), which was not built to handle this unprecedented volume of comments nor initially designed to export comments via XML. This forced the Commission’s information technology team to cobble solutions together MacGyver-style.  Thanks to these creative efforts, we have been able to accommodate the surge in comments and release the comments as XML files for the first time in the FCC’s history, but not without some glitches that we will explain in this post.

Here are some key takeaways from our inquiry:

1. The total comment count in the Open Internet docket is nearly 4 million – the same number that has been widely reported.    These comments –whether received via ECFS, the openinternet@fcc.gov email box, CSV upload, or paper – are in ECFS where the public can view them and the Commission can draw upon them.
2. In parsing the XML files in question, it appears that nearly 680,000 of the comments were not transferred successfully from ECFS to the XML files.  This is due to a technical error involving Apache Solr, an open source tool the FCC used to produce the XML files.  We plan to fix this problem by issuing a new set of XML files after the New Year with the full set of comments received during the reply period.  Despite the fact that this group of comments was not transferred to the XML files, our review indicates that these files were uploaded to ECFS for public review.
3.  An October 22^nd blog post describing the second round of comments included a discrepancy regarding how the comments were received.  While the blog post correctly states the total number of comments received at the end of the second round – approximately 2.5 million – the breakdown of the number of comments received by email vs. ECFS and CSV was incorrect.  The October 22 blog post has been corrected to fix this discrepancy.

We are extremely grateful to the FCC team for undertaking this accounting in such short order and giving us the opportunity to clear up any confusion.

We appreciate the public’s passion and interest in this important proceeding, and hope this also points to the importance of modernizing the Commission’s legacy information systems to provide greater visibility and transparency to the public on impactful issues.