Measuring Fixed Broadband - Tenth Report

Tenth
Measuring Broadband America
Fixed Broadband Report

A Report on Consumer Fixed Broadband Performance
in the United States

Federal Communications Commission

Office of Engineering and Technology

Table of Contents

• Executive Summary
• Summary of Key Findings  
  • Most popular advertised service tiers
  • Median download speeds
  • Variations in speeds
  • Latency
  • Packet loss
  • Web Browsing Performance
• Methodology  
  • Participants
  • Measurement process
  • Measurement tests and performance metrics
  • Availability of Data
• Test Results  
  • Most popular advertised service tiers
  • Observed Median Speeds
  • Variations in speeds
  • Latency
• Additional test results  
  • Actual speed, by service tier
  • Variations in speed
  • Web browsing performance, by service tier

 

1. Executive Summary

The Tenth Measuring Broadband America Fixed Broadband Report (“Tenth Report” or “Report”) presents perspectives on empirical performance for data collected in September and October 2019 ^[1] from fixed Internet Service Providers (ISPs), as part of the Federal Communication Commission’s (FCC) Measuring Broadband America (MBA) program.  This program is an ongoing, rigorous, nationwide study of consumer broadband performance in the United States.  The goal of this program is to measure the network performance realized on a representative sample of service offerings and the residential broadband consumer demographic across the country. ^[2]   This representative sample is referred to as the MBA ‘panel’. Thousands of volunteer panelists are drawn from the subscriber bases of ISPs which collectively serve a large percentage of the residential marketplace. ^[3]

The initial Measuring Broadband America Fixed Broadband Report was published in August 2011, ^[4] and presented the first broad-scale study of directly measured consumer broadband performance throughout the United States.  As part of an open data program, all methodologies used in the program are fully documented, and all data collected is published for public use without any restrictions.  Including this current Report, ten reports have now been issued. ^[5]   These reports provide a snapshot of fixed broadband Internet access service performance in the United States utilizing a comprehensive set of performance metrics.  The resulting performance data is analyzed in a variety of ways that has evolved to make the information more understandable and useful.  

A. Major Findings Of The Tenth Report

The key findings of this report are:

The maximum advertised download speeds amongst the service tiers offered by ISPs and measured by the FCC ranged from 24 Mbps to 940 Mbps for the period covered by this report.

The weighted average advertised speed of the participating ISPs was 146.1 Mbps, representing an 8% increase from the previous year (Ninth Report) and over 100% increase from two years prior (Eighth Report).

For most of the major broadband providers that were tested, measured download speeds were 100% or better than advertised speeds during the peak hours (7 p.m. to 11 p.m. local time).

Ten ISPs were evaluated in this report.  Of these Cincinnati Bell and Frontier employed multiple different broadband technologies across the USA.  Overall 12 different ISP/technology configurations were evaluated in this report and eight performed at or better than their advertised speed during the peak hours.  Only one performed below 90% for actual-to-advertised download speed during the peak hours.

In addition to providing download and upload speed measurements of ISPs, this report also provides a measure of consistency of measured to advertised speeds of ISPs with the use of our “80/80” metric.  The 80/80 metric measures the percentage of the advertised speed that at least 80% of subscribers experience at least 80% of the time over peak periods.  Ten of the 12 ISP/technology configurations provide better than 75% of advertised speed to at least 80% of panelists for at least 80% of the time.

These and other findings are described in greater detail within this report.

B. Speed Performance Metrics

Speed (both download and upload) performance continues to be one of the key metrics reported by the MBA.  The data presented includes ISP broadband performance as a median ^[6] of speeds experienced by panelists within a specific service tier.  These reports mainly focus on common service tiers used by an ISP’s subscribers. ^[7]

Additionally, consistent with previous Reports, we also compute average per-ISP performance by weighting the median speed for each service tier by the number of subscribers in that tier.  Similarly, in calculating the composite average speed taking into account all ISPs in a specific year, the median speed of each ISP is used and weighted by the number of subscribers of that ISP as a fraction of the total number of subscribers across all ISPs.

In calculating these weighted medians, we draw on two sources for determining the number of subscribers per service tier.  ISPs may voluntarily contribute subscription demographics per surveyed service tier as the most recent and authoritative data.  Many ISPs have chosen to do so. ^[8]   When such information has not been provided by an ISP, we instead rely on the FCC’s Form 477 data. ^[9]   All facilities-based broadband providers are required to file data with the FCC twice a year (Form 477) regarding deployment of broadband services, including subscriber counts.  For this report, we used the June 2019 Form 477 data.  It should be noted that the Form 477 subscriber data values generally lag the reporting month, and therefore, there are likely to be small inaccuracies in the tier ratios.  It is for this reason that we encourage ISPs to provide us with subscriber numbers for the measurement month.

As in our previous reports, we found that for most ISPs the actual speeds experienced by subscribers either nearly met or exceeded advertised service tier speeds.  However, since we started our MBA program, consumers have changed their Internet usage habits.  In 2011, consumers mainly browsed the web and downloaded files; thus, we reported mean broadband speeds since these statistics were likely to closely mirror user experience.  By contrast, in September-October 2019 (the measurement period for this report) consumer internet usage had become dominated by video consumption, with consumers regularly streaming video for entertainment and education. ^[10]   Therefore, our network performance analytics have been expanded by using consistency in service metrics to better capture the shift in usage patterns.  Both the median measured speed metric and consistency in service metrics help to better reflect the consumer’s perception and usefulness of Internet access service.

Specifically, we use two kinds of metrics to reflect the consistency of service delivered to the consumer:  First, we report the percentage of advertised speed experienced by at least 80% of panelists during at least 80% of the daily peak usage period (“80/80 consistent speed” measure).  Second, we show the fraction of consumers who obtain median speeds greater than 95%, between 80% and 95%, and less than 80% of advertised speeds.

A. Use Of Other Performance Metrics

Although download and upload speeds remain the network performance metric of greatest interest to the consumer, we also spotlight two other key network performance metrics in this report: latency and packet loss.  These metrics can significantly affect the overall quality of Internet applications.

Latency is the time it takes for a data packet to travel across a network from one point on the network to another.  High latencies may affect the perceived quality of some interactive services such as phone calls over the Internet, video chat and video conferencing, or online multiplayer games.  All network access technologies have a minimum latency that is largely determined by the technology.There are many other factors that affect latency though, including the location of the server you're communicating with, the route taken to the server, and whether or not there is any congestion on that route. Technology-dependent latencies are typically small for terrestrial broadband services and are thus unlikely to affect the perceived quality of applications.  Additionally, for certain applications the user experience is not necessarily affected by high latencies.  As an example, when using entertainment video streaming applications, because the data can be cached prior to display, the user experience is likely to be unaffected by relatively high latencies.

Packet loss measures the fraction of data packets sent that fail to be delivered to the intended destination.  Packet loss may affect the perceived quality of applications that do not incorporate retransmission of lost packets, such as phone calls over the Internet, video chat, some online multiplayer games, and some video streaming.  High packet loss also degrades the achievable throughput of download and streaming applications.  However, packet loss of a few tenths of a percent are unlikely to significantly affect the perceived quality of most Internet applications and are common.  During network congestion, both latency and packet loss typically increase.

The Internet continually evolves in its architecture, performance, and services.  Accordingly, we will continue to adapt our measurement and analysis methodologies to further improve the collective understanding of performance characteristics of broadband Internet access.  By doing so we aim to help the community of interest across the board, from consumers to technologists, service providers and regulators.

2. Summary of Key Findings

A. Most Popular Advertised Service Tiers

A list of the ISP download and upload speed service tiers that were measured in this report are shown in Table 1.  It should be noted that while upload and download speeds are measured independently and shown separately, they are typically offered by an ISP in a paired configuration.  The service tiers that are included for reporting represent the top 80% (therefore ‘most popular’) of an ISP’s set of tiers based on subscriber numbers.  Taken in aggregate, these plans serve the majority of the subscription base of the participating ISPs.

Table 1: List of ISP service tiers whose broadband performance was measured in this report

*Tiers that lack sufficient panelists to meet the program’s target sample size.

**  Although Verizon Fiber’s 940/880 Mbps service tier was amongst the top 80% of Verizon’s offered tiers by subscription numbers, it is not included in the report charts because technical methodologies for measuring high speed rates near Gigabit and above have not yet been established for the MBA program.

Chart 1.1 (below) displays the weighted (by subscriber numbers) mean of the top 80% advertised download speed tiers for each participating ISP for the last three years (September 2017 to September-October 2019) grouped by the access technology used to offer the broadband Internet access service (DSL, cable, or fiber).  It should be noted that this chart does not reflect the actual performance of the ISPs and only provides the weighted average of the ISP’s advertised speeds.  In September-October 2019, the weighted average advertised download speed was 146.1 Mbps among the measured ISPs, which represents a 100% increase from 2017 and a 8% increase compared to the average in September-October 2018 which was 135.7 Mbps. ^[11]

Chart 1.1: Weighted average advertised download speed among the top 80% service tiers offered by each ISP

All of the ISPs, except Verizon, showed higher weighted-averages of advertised speeds in September-October 2019 as compared to September 2018.  Verizon-fiber showed a slight decrease in 2019 compared to 2018 which was not due to any reduction in the service speed offerings but arose from changes in weighting due to relative shifts in subscriber numbers on the advertised tiers from 2018 to 2019.

It can be seen from Chart 1.1 that the DSL speeds lag far behind the speed of other technologies.  In order to better compare the DSL speed offerings by the various ISPs we have added a separate Chart 1.2 drawn to a scale that makes their relative speeds more discernable.

Chart 1.2: Weighted average advertised download speed among the DSL ISPs

Among participating broadband ISPs, only Cincinnati Bell, Frontier, and Verizon use fiber as the access technology for a substantial number of their customers and their maximum speed offerings range from 200 Mbps to 940 Mbps.  A key difference between the fiber vendors and other technology vendors is that (with the exception of Cincinnati Bell), most fiber vendors advertise generally symmetric upload and download speeds.  This is in sharp contrast to the asymmetric offerings for all the other technologies, where the upload advertised speeds are typically 5 to 10 times below the download advertised speeds.

As can be seen in Chart 1.1, there is considerable difference between the offered average weighted speed tier by technology.  Chart 2 plots the weighted average of the top 80% ISP tiers by technology for the last three years. ^[12]  As can be seen in this chart, most technologies showed increases in the set of advertised download speeds by ISPs.  For the September-October 2019 period, the weighted mean advertised speeds for DSL technology was 13 Mbps which lagged considerably behind the weighted mean advertised download speeds for cable and fiber technologies, which were 155 Mbps and 208 Mbps, respectively.  Fiber technology showed the greatest increase in speed offerings in 2019 compared to 2017 with a weighted mean going up from 70 Mbps to 208 Mbps representing a nearly 200% increase.  This year’s (2019) average advertised speed for fiber, however, showed a slight decrease by 17% from last year’s (2018) speed.  DSL technology speed increased from 11 Mbps to 13 Mbps from 2017 to 2019, a 16% increase overall (though it did show a small 1% decrease in speed this year compared to last year).  In comparison, cable technology showed an 12% increase from 2018 to 2019 and an overall 83% increase from 2017 to 2019

Chart 2: Weighted average advertised download speed among the top 80% service tiers based on technology.

Chart 3 plots the migration of panelists to a higher service tier based on their access technology. ^[13]   Specifically, the horizontal axis of Chart 3 partitions the September 2018 panelists by the advertised download speed of the service tier to which they were subscribed.  For each such set of panelists who also participated in the September-October 2019 collection of data, ^[14] the vertical axis of Chart 3 displays the percentage of panelists that migrated by September-October 2019 to a service tier with a higher advertised download speed.  There are two ways that such a migration could occur: (1) if a panelist changed their broadband plan during the intervening year to a service tier with a higher advertised download speed, or (2) if a panelist did not change their broadband plan but the panelist’s ISP increased the advertised download speed of the panelist’s subscribed plan. ^[15]

Chart 3 shows that the percentage of panelists subscribed in September-October 2018 who moved to higher tiers in September - October 2019 was between 3% to 26% for DSL subscribers, 4% to 100% for cable subscribers and 16% to 50% for fiber subscribers.  In addition, 1% to 8% subscribers migrated to a higher speed tier using a different technology from what they had in September 2018.

Chart 3: Consumer migration to higher advertised download speeds

B. Median download speeds

Advertised download speeds may differ from the speeds that subscribers actually experience.  Some ISPs more consistently meet network service objectives than others or meet them unevenly across their geographic coverage area.  Also, speeds experienced by a consumer may vary during the day if the aggregate user demand during busy hours causes network congestion.  Unless stated otherwise, all actual speeds were measured only during peak usage periods, which we define as 7 p.m. to 11 p.m. local time.

To compute the average ISP performance, we determine the ratio of the median speed for each tier to the advertised tier speed and then calculate the weighted average of these based on the subscriber count per tier.  Subscriber counts for the weightings were provided from the ISPs themselves or, if unavailable, from FCC Form 477 data.

Chart 4 shows the ratio of the measured median download and upload speeds experienced by an ISP’s subscribers to that ISP’s advertised download and upload speeds weighted by the subscribership numbers for the tiers.  The actual speeds experienced by most ISPs’ subscribers are close to or exceed the advertised speeds.  However, DSL broadband ISPs continue to advertise “up-to” speeds that on average exceed the actual speeds experienced by their subscribers.  Out of the 12 ISP/technology configurations shown, 8 met or exceeded their advertised download speed and three more reached at least 90% of their advertised download speed.  Only Cincinnati-DSL (at 79%) performed below 90% of its advertised download speed.

Chart 4: The ratio of weighted median speed (download and upload) to advertised speed for each ISP.

C. Variations In Speeds

As discussed earlier, actual speeds experienced by individual consumers may vary by location and time of day.  Chart 5 shows, for each ISP, the percentage of panelists who experienced a median download speed (averaged over the peak usage period during our measurement period) that was greater than 95%, between 80% and 95%, or less than 80% of the advertised download speed.

Chart 5: The percentage of consumers whose median download speed was greater than 95%, between 80% and 95%, or less than 80% of the advertised download speed

ISPs using DSL technology had between 2% to 69% of their subscribers getting greater than or equal to 95% of their advertised download speeds during peak hours.  ISPs using cable technology and fiber technology had between 93%-99% and between 65%-97%, respectively, of their subscribers getting equal to or better than 95% of their advertised download speeds.

Though the median download speeds experienced by most ISPs’ subscribers nearly met or exceeded the advertised download speeds, there are some customers of each ISP for whom the median download speed fell significantly short of the advertised download speed.  Relatively few subscribers of cable service experienced this.  The best performing ISPs, when measured by this metric, are Charter, Comcast, Cox, Mediacom, Optimum, Frontier-Fiber and Verizon-Fiber; more than 80% of their panelists were able to attain an actual median download speed of at least 95% of the advertised download speed.

In addition to variations based on a subscriber’s location, speeds experienced by a consumer may fluctuate during the day.  This is typically caused by increased traffic demand and the resulting stress on different parts of the network infrastructure.  To examine this aspect of performance, we use the term “80/80 consistent speed.”  This metric is designed to assess temporal and spatial variations in measured values of a user’s download speed. ^[16]  While consistency of speed is in itself an intrinsically valuable service characteristic, its impact on consumers will hinge on variations in usage patterns and needs.  As an example, a good consistency of speed measure is likely to indicate a higher quality of service experience for internet users consuming video content.

Chart 6 summarizes, for each ISP, the ratio of 80/80 consistent median download speed to advertised download speed, and, for comparison, the ratio of median download speed to advertised download speed shown previously in Chart 4.  The ratio of 80/80 consistent median download speed to advertised download speed is less than the ratio of median download speed to advertised download speed for all participating ISPs due to congestion periods when median download speeds are lower than the overall average.  When the difference between the two ratios is small, the median download speed is fairly insensitive to both geography and time.  When the difference between the two ratios is large, there is a greater variability in median download speed, either across a set of different locations or across different times during the peak usage period at the same location.

Chart 6: The ratio of 80/80 consistent median download speed to advertised download speed.

Customers of Charter, Comcast, Cox, Mediacom and Optimum experienced median download speeds that were very consistent; i.e., they provided greater than 100% of the advertised speed during peak usage period to more than 80% of panelists for more than 80% of the time.  As can be seen in Chart 6 cable and fiber ISPs performed better than DSL ISPs with respect to their 80/80 consistent speeds.  For example, for September-October 2019, the 80/80 consistent download speed for Cincinnati Bell DSL was 46% of the advertised speed.

D. Latency

The latency between any two points in the network is the time it takes for a packet to travel from one point to the other.  It has a fixed component that depends on the distance, the transmission speed, and transmission technology between the source and destination, and a variable component due to queuing delay that increases as the network path congests with traffic.  The MBA program measures latency by measuring the round-trip time between  the consumer’s home and  the closest measurement server.

Chart 7 shows the median latency for each participating ISP.  In general, higher-speed service tiers have lower latency, as it takes less time to transmit each packet.  The median latencies ranged from 10 ms to 27 ms in our measurements (with the exception of CenturyLink DSL and Cincinnati Bell DSL which had median latencies of 40 ms and 34 ms, respectively).

Chart 7: Latency by ISP

DSL latencies (between 11 ms to 40 ms) were slightly higher than those for cable (13 ms to 27 ms).  Fiber ISPs showed the lowest latencies (10 ms to 12 ms).  The differences in median latencies among terrestrial-based broadband services are relatively small and are unlikely to affect the perceived quality of highly interactive applications.

E. Packet Loss

Packet loss is the percentage of packets that are sent by a source but not received at the intended destination.  The most common causes of packet loss are congestion leading to buffer overflows or active queue management along the network path.  Alternatively, high latency might lead to a packet being counted as lost if it does not arrive within a specified window.  A small amount of packet loss is expected, and indeed packet loss is commonly used by some Internet protocols such as TCP to infer Internet congestion and to adjust the sending rate to mitigate the offered load, thus lessening the contribution to congestion and the risk of lost packets.  The MBA program uses an active UDP-based packet loss measurement method and considers a packet lost if it is not returned within 3 seconds.

Chart 8 shows the average peak-period packet loss for each participating ISP, grouped into bins.  We have broken the packet loss performance into three bands, allowing a more granular view of the packet loss performance of the ISP network.  The breakpoints for the three bins used to classify packet loss have been chosen with an eye towards balancing commonly accepted packet loss thresholds for specific services and provider packet loss Service Level Agreements (SLAs) for enterprise services, as consumer offerings are not typically accompanied by SLAs.  Specifically, the 1% standard for packet loss is commonly accepted as the point at which highly interactive applications such as VoIP experience significant degradation in quality according to industry publications and international (ITU) standards. ^[17]  The 0.4% breakpoint was chosen as middle ground between the highly desirable performance of 0% packet loss described in many documents (for Voice over Internet Protocol (VoIP)) and the 1% unacceptable limit on the high side.  The specific value of 0.4% is also generally supported by major ISP SLAs for network performance.  Indeed, most SLAs support 0.1% to 0.3% packet loss guarantees, ^[18] but these are generally for enterprise level services which entail business-critical applications that require some service guarantees.

Chart 8:  Percentage of consumers whose peak-period packet loss was less than 0.4%, between 0.4% to 1%, and greater than 1%.

Chart 8 shows that ISPs using fiber technology have the lowest packet loss, and that ISPs using DSL technology tend to have the highest packet loss.  As shown in this chart, 6% to 21% of DSL subscribers experience 1% or greater packet loss.  The corresponding numbers for cable and fiber are 0% to 5% and 0% to 1.5%, respectively.  Within a given technology class, packet loss also varies among ISPs.

F. Web browsing performance

The MBA program also conducts a specific test to gauge web browsing performance.  The web browsing test accesses nine popular websites that include text and images, but not streaming video.  The time required to download a webpage depends on many factors, including the consumer’s in-home network, the download speed within an ISP’s network, the web server’s speed, congestion in other networks outside the consumer’s ISP’s network (if any), and the time required to look up the network address of the webserver.  Only some of these factors are under control of the consumer’s ISP.  Chart 9 displays the average webpage download time as a function of the advertised download speed.  As shown by this chart, webpage download time decreases as download speed increases, from about 9.8 seconds at 1.5 Mbps download speed to about 1.5 seconds for 25 Mbps download speed.  Subscribers to service tiers exceeding 25 Mbps experience slightly smaller webpage download times decreasing to 1 – 1.25 seconds at 150 Mbps.  Beyond 150 Mbps, the webpage download times decrease only by minor amounts.  These download times assume that only a single user is using the Internet connection when the webpage is downloaded, and does not account for more common scenarios, where multiple users within a household are simultaneously using the Internet connection for viewing web pages, as well as other applications such as real-time gaming or video streaming.

Chart 9: Average webpage download time, by advertised download speed.

3. Methodology

A. Participants

Ten ISPs actively participated in the Fixed MBA program in September-October 2019. ^[19]   They were:

• CenturyLink
• Charter Communications
• Cincinnati Bell
• Comcast
• Cox Communications
• Frontier Communications Company
• Mediacom Communications Corporation
• Optimum
• Verizon
• Windstream Communications

The methodologies and assumptions underlying the measurements described in this Report are reviewed at meetings that are open to all interested parties and documented in public ex parte letters filed in the GN Docket No. 12-264.  Policy decisions regarding the MBA program were discussed at these meetings prior to adoption, and involved issues such as inclusion of tiers, test periods, mitigation of operational issues affecting the measurement infrastructure, and terms-of-use notifications to panelists.  Participation in the MBA program is open and voluntary.  Participants include members of academia, consumer equipment vendors, telecommunications vendors, network service providers, consumer policy groups, as well as our contractor for this project, SamKnows.  In 2019-2020, participants at these meetings (collectively and informally referred to as “the broadband collaborative”), included all eleven participating ISPs and the following additional organizations:

• Level 3 Communications (“Level 3”), now part of CenturyLink
• Massachusetts Institute of Technology (“MIT”)
• Measurement Lab (M-Lab)
• StackPath
• NCTA – The Internet & Television Association (“NCTA”)
• New America Foundation
• Princeton University
• United States Telecom Association (“US Telecom”)
• University of California - Santa Cruz

Participants have contributed in important ways to the integrity of this program and have provided valuable input to FCC decisions for this program.  Initial proposals for test metrics and testing platforms were discussed and critiqued within the broadband collaborative.  M-Lab and Level 3 contributed their core network testing infrastructure, and both parties continue to provide invaluable assistance in helping to define and implement the FCC testing platform.  We thank all the participants for their continued contributions to the MBA program.

B. Measurement process

The measurements that provided the underlying data for this report were conducted between MBA measurement clients and MBA measurement servers.  The measurement clients ( i.e., whiteboxes) were situated in the homes of 6,006 panelists each of whom received service from one of the 11 evaluated ISPs.  The evaluated ISPs collectively accounted for over 80% of U.S. residential broadband Internet connections.  After the measurement data was processed (as described in greater detail in the Technical Appendix), test results from 3,075 panelists were used in this report.

The measurement servers used by the MBA program were hosted by StackPath, M-Lab, and Level 3 Communications, and were located in thirteen cities (often with multiple locations within each city) across the United States near a point of interconnection between the ISP’s network and the network on which the measurement server resided.

The measurement clients collected data throughout the year, and this data is available as described below.  However, only data collected from September 6 – October 3, 2019 (inclusive) plus October 8 – 9, 2019 (inclusive), referred to throughout this report as the “September-October 2019” reporting period, were used to generate the charts in this Report. ^[20]

Broadband performance varies with the time of day.  At peak hours, more people tend to use their broadband Internet connections, giving rise to a greater potential for network congestion and degraded user performance.  Unless otherwise stated, this Report focuses on performance during peak usage period, which is defined as weeknights between 7:00 p.m. to 11:00 p.m. local time at the subscriber’s location.  Focusing on peak usage period provides the most useful information because it demonstrates what performance users can expect when the Internet in their local area experiences the highest demand from users.

Our methodology focuses on the network performance of each of the participating ISPs.  The metrics discussed in this Report are derived from active measurements, i.e., test-generated traffic flowing between a measurement client, located within the modem/router within a panelist’s home, and a measurement server, located outside the ISP’s network.  For each panelist, the tests automatically choose the measurement server that has the lowest latency to the measurement client.  Thus, the metrics measure performance along the path followed by the measurement traffic within each ISP’s network, through a point of interconnection between the ISP’s network and the network on which the chosen measurement server is located.  However, the service performance that a consumer experiences could differ from our measured values for several reasons.

First, as noted, in the course of each test instance we measure performance only to a single measurement server rather than to multiple servers. This is consistent with the approach chosen by most network measurement tools.  As a point of comparison, the average web page may load its content from a multiplicity of end points.

In addition, bottlenecks or congestion points in the full path traversed by consumer application traffic might also impact a consumer’s perception of Internet service performance.  These bottlenecks may exist at various points: within the ISP’s network, beyond its network (depending on the network topology encountered en route to the traffic destination), in the consumer’s home, on the Wi-Fi used to access the in-home access router, or from a shortfall of capacity at the far end point being accessed by the application.  The MBA tests explore how a service performs from the point at which a fixed ISP’s Internet service is delivered to the home on fixed infrastructure (deliberately excluding Wi-Fi, due to the many confounding factors associated with it) to the point at which the test servers are located.  As MBA tests are designed to focus on the access to the ISP’s network, they will not include phenomena at most interconnection points or transit networks that consumer traffic may traverse.

To the extent possible ^[21] the MBA focuses on performance within an ISP’s network.  It should be noted that the overall performance a consumer experiences with their service can also be affected by congestion such as may arise at other points in the path potentially taken by consumer traffic ( e.g., in-home Wi-Fi, peering points, transit networks, etc.) but this does not get reflected in MBA measurements.

A consumer’s home network, rather than the ISP’s network, may be the bottleneck with respect to network congestion.  We measure the performance of the ISP’s service delivered to the consumer’s home network, but this service is often shared simultaneously among multiple users and applications within the home.  In-home networks, which typically include Wi-Fi, may not have sufficient capacities to support peak loads. ^[22]

In addition, consumers’ experience of ISP performance is manifested through the set of applications they utilize.  The overall performance of an application depends not only on the network performance ( i.e., raw speed, latency, or packet loss), but also on the application’s architecture and implementation and on the operating system and hardware on which it runs.  While network performance is considered in this Report, application performance is generally not.

C. Measurement Tests And Performance Metrics

This Report is based on the following measurement tests:

• Download speed: This test measures the download speed of each whitebox over a 10-second period, once per hour during peak hours (7 p.m. to 11 p.m.) and once during each of the following periods: midnight to 6 a.m., 6 a.m. to noon, and noon to 6 p.m.  The download speed measurement results from each whitebox are then averaged across the measurement month; and the median value for these average speeds across the entire set of whiteboxes on a given tier is used to determine the median measured download speed for that tier.  The overall ISP download speed is computed as the weighted median for each service tier, using the subscriber counts for the tiers as weights.
• Upload speed: This test measures the upload speed of each whitebox over a 10-second period, which is the same measurement interval as the download speed.  The upload speed measured in the last five seconds of the 10-second interval is retained, the results of each whitebox are then averaged over the measurement period, and the median value for the average speed taken over the entire set of whiteboxes is used to determine the median upload speed for a service tier.  The ISP upload speed is computed in the same manner as the download speed.
• Latency and packet loss: These tests measure the round-trip times for approximately 2,000 packets per hour sent at randomly distributed intervals.  Response times less than three seconds are used to determine the mean latency.  If the whitebox does not receive a response within three seconds, the packet is counted as lost.
• Web browsing: The web browsing test measures the total time it takes to request and receive webpages, including the text and images, from nine popular websites and is performed once every hour.  The measurement includes the time required to translate the web server name (URL) into the webserver’s network (IP) address.

This Report focuses on three key performance metrics of interest to consumers of broadband Internet access service, as they are likely to influence how well a wide range of consumer applications work: download and upload speed, latency, and packet loss.  Download and upload speeds are also the primary network performance characteristic advertised by ISPs.  However, as discussed above, the performance observed by a user in any given circumstance depends not only on the actual speed of the ISP’s network, but also on the performance of other parts of the Internet and on that of the application itself.

The standard speed tests use TCP with 8 concurrent TCP sessions.  In 2017 we also introduced a less-data intensive throughput test, which both generated less traffic and ran less frequently and thereby provided less strain on consumer accounts that are data-capped.  The Lightweight tests are used exclusively to provide broadband performance results for satellite ISPs.  The Technical Appendix to this Report describes each test in more detail, including additional tests not contained in this Report.

D. Availability Of Data

The MBA panel sample used in the reporting period is validated (i.e., upload and download tiers of the whiteboxes are verified with providers) and the measurement results are carefully inspected to eliminate misleading outliers.  This leads to a ‘validated data set’ that accompanies each report.  The Validated Data Set ^[23] on which this Report is based, as well as the full results of all tests, are available at http://www.fcc.gov/measuring-broadband-america .  For interested parties, as tests are run 24x7x365, we also provide raw data (referred to as such because cross-checks are not done except in the test period used for the report, thus subscriber tier changes may be missed) for the reference month and other months.  Previous reports of the MBA program, as well as the data used to produce them, are also available there.

Both the Commission and SamKnows, the Commission’s contractor for this program, recognize that, while the methodology descriptions included in this document provide an overview of the project, interested parties may be willing to contribute to the project by reviewing the software used in the testing.  SamKnows welcomes review of its software and technical platform, consistent with the Commission’s goals of openness and transparency for this program. ^[24]

 

 

 

 

 

 

 

---