Federal Communications Commission	DA 18-710


Before the
Federal Communications Commission
Washington, D.C. 20554


In the Matter of

Connect America Fund
)
)
)
)


WC Docket No. 10-90

ORDER

Adopted: July 6, 2018	Released: July 6, 2018

By the Chiefs, Wireline Competition Bureau, Wireless Telecommunications Bureau, and Office of Engineering and Technology:
TABLE OF CONTENTS
	Para.
I.	INTRODUCTION	1
II.	BACKGROUND	3
III.	CHOICE OF TESTING METHOD	7
IV.	GENERAL TESTING PARAMETERS	13
A.	Defining “Test”	14
B.	Timing of Tests	22
C.	Number of Test Locations 	34
D.	High-Latency Voice Performance Testing 	41
V.	COMPLIANCE FRAMEWORK 	47
A.	Standards for Full Compliance	48
B.	Non-Compliance and Verification Measures	56
VI.	PROCEDURAL MATTERS	68
VII.	ORDERING CLAUSES	71
APPENDIX A—Summary of Testing Requirements
APPENDIX B—Final Regulatory Flexibility Analysis

I. INTRODUCTION
1. In this Order, the Wireline Competition Bureau (WCB), the Wireless Telecommunications Bureau (WTB) (jointly referred to herein as the Bureaus), and the Office of Engineering and Technology (OET) adopt requirements promoting greater accountability for certain recipients of Connect America Fund (CAF) high-cost universal service support, including price cap carriers, rate-of-return carriers, rural broadband experiment (RBE) support recipients, Alaska Plan carriers, and CAF Phase II auction winners.  Specifically, we establish a uniform framework for measuring the speed and latency performance for recipients of high-cost universal service support to serve fixed locations.
2. We also require providers to submit testing results as part of their annual compliance certification.  Carriers that do not comply with our speed and latency requirements will be subject to a reduction in support, commensurate with their level of noncompliance.  In addition, providers will be subject to audit of all testing data.  With this testing and compliance framework, we aim to maximize the benefits consumers reap from our high-cost universal service programs in even the hardest-to-reach areas, thus making the best use of our Universal Service Fund (USF) dollars and further closing the digital divide.
II. BACKGROUND
3. As a condition of receiving high-cost universal service support, eligible telecommunications carriers (ETCs) must offer broadband service in their supported areas that meets certain basic performance requirements. See Connect America Fund et al., WC Docket No. 10-90 et al., Report and Order and Further Notice of Proposed Rulemaking, 26 FCC Rcd 17663, 17705-06, para. 109 (2011) (USF/ICC Transformation Order and FNPRM), aff’d sub nom. In re FCC 11-161, 753 F.3d 1015 (10th Cir. 2014).  As in the USF/ICC Transformation Order, we use the term high-cost support or high-cost funding to include all existing high-cost universal service mechanisms, as well as CAF.  See id. at 17695 n.126. This Order addresses measuring performance of fixed, not mobile, service.  For high-cost support mechanisms specifically dedicated to mobile services—Mobility Fund Phase I, Tribal Mobility Fund Phase I, and Mobility Fund Phase II—the Commission adopted different performance standards.  See id. at 17791-93, paras. 359-368; Connect America Fund; Universal Service Reform – Mobility Fund, Report and Order and Further Notice of Proposed Rulemaking, 32 FCC Rcd 2152, 2188-90, paras. 86-87 (2017) (Mobility Fund Phase II Order).
  ETCs subject to broadband performance obligations must currently offer broadband with latency suitable for real-time applications, such as Voice over Internet Protocol (VoIP), and meet a minimum speed standard of 10 Mbps downstream and 1 Mbps upstream (10/1 Mbps). Connect America Fund; ETC Annual Reports and Certifications, Report and Order, 29 FCC Rcd 15644, 15649, para. 15 (2014).  Although the Commission adopted 10 Mbps downstream and 1 Mbps upstream as a minimum speed standard for recipients of high-cost support, depending on the outcome of the Commission’s competitive bidding processes, some recipients may bid and receive support to offer faster speeds or, potentially, higher-latency service.  See Connect America Fund; ETC Annual Reports and Certifications; Rural Broadband Experiments, Report and Order and Further Notice of Proposed Rulemaking, 31 FCC Rcd 5949, 5957, para. 15 (2016) (CAF Phase II Auction Order) (allowing bids of different performance tiers with speeds of 1 Gbps/500 Mbps, 100/20 Mbps, 25/3 Mbps, and 10/1 Mbps).  See also, e.g., Connect America Fund; ETC Annual Reports and Certifications, Report and Order and Further Notice of Proposed Rulemaking, 29 FCC Rcd 8769, 8779-80, paras. 24-29 (2014) (Rural Broadband Experiments Order) (making available support through the rural broadband experiments for services providing speeds of 100/25 Mbps, 25/5 Mbps, and 10/1 Mbps).
  Recipients of high-cost support must also test their broadband networks for compliance with speed and latency metrics and certify and report the results to the Universal Service Administrative Company (USAC) and the relevant state or tribal government on an annual basis, with those results subject to audit. USF/ICC Transformation Order, 26 FCC Rcd at 17705-06, para. 109.
  In the 2011 USF/ICC Transformation FNPRM, the Commission sought comment on the specific methodology ETCs should use to measure the performance of their broadband services and the format in which funding recipients should report their results. Id. at 18045-46, paras. 1013-1017.
  The Commission directed the Bureaus and OET to work together to refine the methodology for implementation. Id. at 17680, 17708, paras. 48, 112 (“We delegate authority to the Bureaus to finalize performance measures as appropriate consistent with the goals we adopt today.”); 47 CFR § 54.313(a)(6).  
  
4. Subsequently, in October 2013, WCB defined certain service obligations of price cap carriers that accept CAF Phase II model-based support through the state-level commitment process. Connect America Fund, Report and Order, WC Docket No. 10-90, Report and Order, 28 FCC Rcd 15060 (WCB 2013) (CAF Phase II Price Cap Service Obligation Order).  
  It concluded that price cap carriers must be prepared to demonstrate a round-trip latency of 100 ms or less.  To show compliance with latency obligations, a price cap carrier must certify that 95 percent or more of all peak period measurements (also referred to as observations) of network round trip latency are at or below 100 ms when measured during the peak period between the customer premises and the nearest Internet Exchange Point (IXP) in an FCC-designated metropolitan area (FCC-designated IXP).  The measurements were required to be conducted over a minimum of two consecutive weeks during peak hours for at least 50 randomly-selected customer locations within the census blocks of each state for which the provider is receiving model-based support.  Such measurements may be made using existing network management systems, ping tests, A ping test is a method of determining connectivity and reaction time of a computer connection.  See Ookla Speedtest, What is “ping”, “download speed”, and “upload speed”?, available at https://support.speedtest.net/hc/en-us/articles/203845290-What-is-ping-download-speed-and-upload-speed-.  We note that ping tests have significant drawbacks and have shown excessive variability in measuring latency.  See, e.g., C. Pelsser, L. Cittadini, S. Vissicchio, and R. Bush. From Paris to Tokyo: On the suitability of ping to measure latency, Proceedings of the 2013 Internet Measurement Conference (IMC 2013) ACM SIGCOMM at 427–432, available at http://conferences.sigcomm.org/imc/2013/papers/imc125s-pelsserA.pdf.  Thus, although we allow use of ping tests as an off-the-shelf method for testing latency, we recommend that parties use other more reliable methods.
 or other commonly available network measurement tools. CAF Phase II Price Cap Service Obligation Order, 28 FCC Rcd at 15071-72, para. 24.
  Alternatively, carriers participating in the Measuring Broadband America (MBA) program may use the results from that testing to support their certification that they meet the latency requirement, so long as they deploy at least 50 Whiteboxes A Whitebox is a measuring device to measure performance of a broadband connection. See Measuring Broadband America, Requirements, What it takes to get involved, available at https://www.measuringbroadbandamerica.com/fixed-broadband/fixed-broadband-requirements/.
 to customers within the CAF Phase II-funded areas within each state and certify that 95 percent or more of the measurements taken quarterly during peak periods for a period of two weeks were at or below 100 ms.  The provider is responsible for the hardware and administrative costs of these Whiteboxes to the extent such Whiteboxes are in addition to those deployed as part of the MBA testing.  
5. The Bureaus and OET subsequently released a Public Notice seeking comment on outstanding questions regarding the methodology for measuring broadband services’ performance. Wireline Competition Bureau, Wireless Telecommunications Bureau, and the Office of Engineering and Technology Seek Comment on Proposed Methodology for Connect America High-Cost Universal Service Support Recipients to Measure and Report Speed and Latency Performance to Fixed Locations, Public Notice, 29 FCC Rcd 12623 (WCB 2014) (2014 Broadband Measurement and Reporting Public Notice).
  Specifically, the Bureaus and OET sought to develop the record further on how compliance with speed obligations should be determined for high-cost support recipients that serve fixed locations.  The Bureaus and OET also sought comment on whether the same testing methodologies adopted for price cap carriers accepting model-based CAF Phase II support should be applied to other support recipients, such as rate-of-return providers and those that are awarded CAF support through a competitive bidding process. Id.
  The Bureaus and OET raised the possibility of a platform that could be administered by either a single entity (e.g., USAC) or multiple service vendors, who could deploy Whiteboxes to consumers throughout CAF-supported areas, and could reduce the costs of measuring broadband performance. See id. at 12628, para. 20.
  Finally, the Bureaus and OET sought comment on the circumstances that would trigger an audit of speed and latency metrics. See id. at 12629-30, paras. 26-28.

6. Recently, the Bureaus and OET released a Public Notice seeking to update the record more broadly regarding performance measures. Comment Sought on Performance Measures for Connect America High-Cost Universal Service Support Recipients, Public Notice, 32 FCC Rcd 9321 (WCB 2017) (2017 Performance Measures Public Notice).
  Noting that the progression of technology has enabled newer ways of measuring performance across deployments, the Bureaus and OET sought comment on both general options for conducting required tests and precise parameters for testing speed and latency. Id.
  The Bureaus and OET also specifically sought comment on the USTelecom – the Broadband Association’s (USTelecom) proposal for a performance measurement and compliance framework. Id. at 9324-26 (citing Letter from Kevin Rupy, Vice President, Law & Policy, USTelecom, to Marlene Dortch, Secretary, FCC, WC Docket No. 10-90, at 7 (filed May 23, 2017) (USTelecom Ex Parte)).

III. CHOICE OF TESTING METHOD
7. Background.  Price cap carriers are currently permitted to use “existing network management systems, ping tests, or other commonly available network measurement tools,” as well as results from the MBA program, subject to certain parameters to demonstrate compliance with latency obligations associated with CAF Phase II model-based support. CAF Phase II Price Cap Service Obligation Order, 28 FCC Rcd at 15071, para. 23.
  The Bureaus and OET have since sought comment on whether the Commission should permit the same testing method options for all recipients of high-cost support that provide broadband Internet access service to fixed locations. 2014 Broadband Measurement and Reporting Public Notice, 29 FCC Rcd at 12626, paras. 13-14; 2017 Performance Measures Public Notice, 32 FCC Rcd at 9323, para. 6.
  The Bureaus and OET additionally sought comment on:  (1) whether internal or external network measurement system tools should be used to measure latency and speed performance, 2014 Broadband Measurement and Reporting Public Notice, 29 FCC Rcd at 12625-26, paras. 5-6, 13-14; 2017 Performance Measures Public Notice, 32 FCC Rcd at 9323-24, para. 7.
 (2) use of software installed in customer premises equipment (CPE) itself or equipment directly attached to CPE to conduct required testing, and (3) USAC’s role in enabling such testing options. 2017 Performance Measures Public Notice, 32 FCC Rcd at 9326, paras. 10-12.

8. Commenters broadly support flexibility in implementing the Commission’s performance testing requirements, including the choice of software or hardware for testing. See ADTRAN, Inc. Comments at 6 (ADTRAN Comments); Cincinnati Bell Telephone Company LLC Comments at 1-2 (CBT Comments); ITTA – The Voice of America’s Broadband Providers Comments at 3-4 (ITTA Comments); USTelecom Association Comments at 15-17 (USTelecom Comments); Wireless Internet Service Providers Association Comments at 3-6 (WISPA Comments); WTA – Advocates for Rural Broadband Comments at 8-9 (WTA Comments).  Unless otherwise noted, all citations to “Comments” are to comments filed on December 6, 2017 in WC Docket No. 10-90.
  For example, ADTRAN, Inc. (ADTRAN) emphasizes that the Commission should not mandate particular equipment for measuring performance, and the Wireless Internet Service Providers Association (WISPA) argues that the Commission should permit a variety of affordable and reliable methods to measure broadband speed. See ADTRAN Comments at 6; WISPA Comments at 3-6.
  Other parties similarly raise concerns about the costs of mandating one particular testing method or implementing newer testing systems. See, e.g., CBT Comments at 1-2; Hughes Network Systems Comments at 1; ITTA Comments at 3-4.
  However, one party urges caution; the Rural Electric Cooperatives ask that the Commission “ensure that any approved testing methodology is not subject to manipulation or gaming and, ideally, is designed by neutral third parties.” Rural Electric Cooperatives Comments at 6-7.  The Association of Missouri Electric Cooperatives, Midwest Energy & Communications, Alger Delta Cooperative Electric Association, HomeWorks Tri-County Electric Cooperative, Arkansas Electric Cooperatives, Inc., Great Lakes Energy, Indiana Electric Cooperatives, Utilities Technology Council, and the National Rural Electric Cooperative Association submitted comments together as the Rural Electric Cooperatives.
  Alternatively, Alaska Communications proposes that, rather than requiring speed testing, the Commission should permit carriers relying on wireless technologies to submit radio frequency propagation maps to demonstrate compliance with high-cost support obligations. See Alaska Communications Comments at 5-6.
  Finally, Alaska Communications and USTelecom argue “that USAC’s role as the administrator of all high cost programs and the entity in charge of determining compliance with program rules, should disqualify it from being directly involved in the testing itself.” USTelecom Comments at 17.  See Alaska Communications Comments at 8.
  USTelecom further asserts that it would be an inappropriate use of universal service support for USAC “to provide measurement services for a fee in competition with public entities.” USTelecom Comments at 18.

9. Discussion.  We provide high-cost support recipients that serve fixed locations three options to afford flexibility in choosing solutions to conduct required performance testing.  Specifically, we conclude that ETCs subject to fixed broadband performance obligations may conduct required testing by employing either (1) MBA testing infrastructure (MBA testing), (2) existing network management systems and tools (off-the-shelf testing), or (3) provider-developed self-testing configurations (provider-developed self-testing or self-testing).  Providers may employ any of these three options as long as the provider’s implementation meets the testing requirements established in this Order.  We define the three options as follows:
· First, a high-cost support recipient may use MBA testing by arranging with entities that manage and perform testing for the MBA program to implement performance testing, as required, for CAF.  The provider is responsible for all costs required to implement testing of its network, including any costs associated with obtaining and maintaining Whiteboxes, to the extent that any additional Whiteboxes are employed as part of the MBA testing.  We note that the MBA testing must occur in areas and for the locations supported by CAF, e.g., in CAF Phase II eligible areas for price cap carriers and for specific built-out locations for RBE, Alternative Connect America Cost Model (A-CAM), and legacy rate-of-return support recipients.  
· Second, a high-cost support recipient may elect to use existing network management systems and tools, ping tests, and other commonly available performance measurement and network management tools – off-the-shelf testing – to implement performance testing.
· Third, a high-cost support recipient may implement a provider-developed self-testing configuration using software installed on residential gateways or in equipment attached to residential gateways to regularly initiate speed and latency tests. See Letter from Mary Henze, Assistant Vice President, Federal Regulatory, AT&T, to Marlene Dortch, Secretary, FCC, WC Docket No. 10-90, at 1-2 (filed June 20, 2016); USTelecom Ex Parte at 7.
  Providers that implement self-testing of their own networks may make network performance testing services available to other providers.  We continue to consider whether USAC may have a role in offering server capacity at FCC-designated IXPs, without any oversight role in conducting tests, to mitigate smaller providers’ costs.  
10. By providing these three options, we ensure that there is a cost-effective method for conducting testing for providers of different sizes and technological sophistication.  We do not require that providers invest in and implement new internal systems; instead, providers may perform speed and latency tests with readily-available, off-the-shelf solutions or existing MBA infrastructure.  On the other hand, some providers may prefer implementing their own self-testing systems, especially if such testing features are already built into CPE for the carrier’s own network management purposes. See USTelecom Comments at 15-17.   
  These three options allow the provider to align required performance testing with their established network management systems and operations, making it as easy as possible for carriers to implement the required testing while establishing rigorous testing parameters and standards, based on real-world data.  
11. We recognize that self-testing using provider-developed software may create opportunities for “manipulation or gaming” See Rural Electric Cooperatives Comments at 7.
 by CAF recipients.  However, we believe that the testing and compliance requirements we adopt will minimize the possibility of such behavior.  First, as explained in more detail below, we will be requiring providers to submit and certify testing data annually.  .  Second, USAC will be verifying provider compliance and auditing performance testing results.
12. We reject Alaska Communications’ proposal that high-cost support recipients may submit radio frequency propagation maps in lieu of conducting speed tests to demonstrate compliance with speed obligations. See Alaska Communications Comments at 5-6.
  Such maps are only illustrative of planned, “theoretical” coverage and do not provide actual data on what consumers experience. See Connect America Fund; Universal Service Reform – Mobility Fund, Order on Reconsideration and Second Report and Order, 32 FCC Rcd 6282, 6300-04, paras. 35-41 & n.90 (2017) (requiring, in the context of the Mobility Fund Phase II, that propagation maps be combined with a challenge process to determine actual service to consumers).  See also Mobility Fund Phase II Order, 32 FCC Rcd at 2238, para. 237 (“We recognize that such maps do not actually portray the consumer’s experience throughout the area at issue, given in part that a consumer’s experience depends on variables other than signal strength.”).
  We therefore require providers to conduct the required testing using one of the three options identified above. We do not require providers to select one testing option for all areas.  Rather, providers may use different options on a state, study area, or service area basis.
 
IV. GENERAL TESTING PARAMETERS
13. All ETCs subject to fixed broadband performance obligations must conduct the required speed and latency testing using the parameters we outline in this section, regardless of which of the three testing options the carrier selects.  We first define “test” and the associated span of measurement, in the context of these performance measurements.  Next, we adopt requirements regarding when tests must begin and when exactly carriers may perform the tests, and we set the number of active subscriber locations carriers must test, with variations depending on the size of the carrier.  Finally, we address how high-latency bidders in the CAF Phase II auction must conduct required voice testing.
A. Defining “Test”
14. Background.  In the USF/ICC Transformation FNPRM, the Commission sought comment generally on whether it should adopt a specific or uniform measurement methodology, USF/ICC Transformation FNPRM, 26 FCC Rcd at 18045-46, paras. 1013-1017. 
 and the Bureaus and OET later sought comment in the 2014 Broadband Measurement and Reporting Public Notice to develop the record further on measuring compliance with service performance obligations. See generally 2014 Broadband Measurement and Reporting Public Notice; CAF Phase II Price Cap Service Obligation Order.
  In the 2017 Performance Measures Public Notice, the Bureaus and OET sought to refresh the record regarding what should constitute a “test” within the kind of provider-developed self-testing proposed by USTelecom in an ex parte filed in this docket, and regarding whether there is an industry standard or other published specification suited to achieve the Commission’s universal service goals and CAF performance objectives. 2017 Performance Measures Public Notice, 32 FCC Rcd at 9324-25, para. 9.
  In its comments, USTelecom asks that the Commission support the use of industry testing standards in software-based self-testing, such as the testing standard developed by the DSL Forum in 2008 that enables network throughput performance testing and statistical monitoring in a technology-neutral manner (the TR-143 Standard). USTelecom Comments at 18.
  USTelecom characterizes the TR-143 Standard as envisioning a testing scenario mirroring the Commission’s approach in the MBA. Id. at 18-19.
 
15. The Bureaus and OET also sought comment regarding the testing measurement span, i.e., the endpoints of speed and latency testing. See 2014 Broadband Measurement and Reporting Public Notice, 29 FCC Rcd at 12625, para. 9; 2017 Performance Measures Public Notice, 32 FCC Rcd at 9323-24, paras. 6-7.
  To demonstrate compliance with latency obligations associated with CAF Phase II model-based support, price cap carriers must currently measure latency between the customer premises and the closest FCC-designated IXP. See CAF Phase II Price Cap Service Obligation Order, 28 FCC Rcd at 15068-75, paras. 19-36.    
  The Bureaus and OET sought comment on applying that same requirement to all latency measurements required of ETCs subject to fixed broadband performance obligations. See 2014 Broadband Measurement and Reporting Public Notice, 29 FCC Rcd at 12625, para. 9; 2017 Performance Measures Public Notice, 32 FCC Rcd at 9323-24, paras. 6-7.

16. Citing the Commission’s 2011 decision in the USF/ICC Transformation Order, USTelecom advocates requiring testing between the “end-user interface to the nearest [IXP].” USTelecom Comments at 13-15 (citing USF/ICC Transformation Order, 26 FCC Rcd at 17706, para. 111).
  NTCA–The Rural Broadband Association (NTCA), the Rural Electric Cooperatives, and WTA – Advocates for Rural Broadband (WTA), however, urge that the Commission require providers only to test within their own networks, not necessarily to the nearest designated IXP. See NTCA–The Rural Broadband Association Comments at 9-11 (NTCA Comments); WTA Comments at 5-8.  See also Letter from Rebekah P. Goodheart, Counsel for the Association of Missouri Electric Cooperatives, Midwest Energy & Communications, HomeWorks, Alger Delta & Great Lakes Energy, and Arkansas Electric Cooperatives, Inc., to Marlene H. Dortch, Secretary, FCC, WC Docket No. 10-90, at 2 (filed Mar. 13, 2018) (Rural Electric Cooperatives Ex Parte).
  In particular, NTCA argues that many rural carriers rely on “third-party” middle-mile facilities to connect their networks to the Internet backbone, so even if the carrier “may have purchased sufficient capacity and deployed adequate facilities,” it cannot control network performance that occurs beyond its network. NTCA Comments at 9-11.
  NTCA further questions whether “the Commission can compel and test for performance at certain levels with respect to network facilities that are not at all supported via universal service funds.” Id. at 11.
  Similarly, WTA indicates that, unlike price cap carriers, rural carriers “have little, if any, idea of what actual route or routes a specific item of their customer or test traffic may actually take to and from the Internet.” WTA Comments at 5.
  The Rural Electric Cooperatives specifically ask that the Commission include smaller cities like Kansas City, Missouri, as IXP locations. Rural Electric Cooperatives Ex Parte at 2.
  Finally, Alaska Communications raises concerns regarding the test measurement span for non-contiguous carriers and asked that the Commission continue permitting such carriers to conduct latency testing between the customer location and a point at which traffic is consolidated for transport to an IXP in the continental U.S. Alaska Communications Comments at 8.
  
17. Discussion.  To maintain a stringent performance compliance regime while avoiding unnecessary burdens on smaller carriers, we allow flexibility concerning the specific testing approach so that carriers can select, consistent with our adopted framework, the best and most efficient testing methods for their particular circumstances.  We encourage the use of industry testing standards, such as the TR-143 Standard, for conducting self-testing. See USTelecom Comments at 18-19.
  
18. For reasons similar to those outlined in the CAF Phase II Price Cap Service Obligation Order, See CAF Phase II Price Cap Service Obligation Order, 28 FCC Rcd at 15073-74, paras. 31-32.
 we require that high-cost support recipients serving fixed locations perform these tests over the measurement span already applicable to price cap carriers receiving CAF Phase II model-based support.  ETCs must test speed and latency from the customer premises of an active subscriber to a remote test server located at or reached by passing through an FCC-designated IXP.  Accordingly, a speed test is a single measurement of download or upload speed of 10 to 15 seconds duration between a specific consumer location and a specific remote server location.  Similarly, a latency test is a single measurement of latency, often performed using a single User Datagram Protocol (UDP) packet or a group of three Internet Control Message Protocol (ICMP) or UDP packets sent at essentially the same time, as is common with ping tests. See supra n.7 (citing Ookla Speedtest, What is “ping”, “download speed”, and “upload speed”?, available at https://support.speedtest.net/hc/en-us/articles/203845290-What-is-ping-download-speed-and-upload-speed-.) (defining a “ping test”). 

19. Large and small ETCs alike commit to providing a certain level of service when accepting high-cost support to deploy broadband.  “Testing . . . on only a portion of the network connecting a consumer to the Internet core will not show whether that customer is able to enjoy high-quality real-time applications because it is network performance from the customer’s location to the destination that determines the quality of the service from the customer’s perspective.” CAF Phase II Price Cap Service Obligation Order, 28 FCC Rcd at 15073-74, para. 31.
  Although the measurement span we adopt may include transport (e.g., backhaul or transit) that a provider does not control, the carrier can influence the quality of transport purchased and can negotiate with the transport provider for a level of service that will enable it to meet the Commission’s performance requirements.  This is true for both price cap carriers and smaller carriers. Id. at 15074, para. 32.
  We therefore disagree with suggestions that testing should only occur within a provider’s own network because providers do not always control the portion of the network reaching the nearest designated IXP. See NTCA Comments at 9-11; WTA Comments at 5-8.  See also Rural Electric Cooperatives Ex Parte at 2.  Cf. Connect America Fund, Order, 28 FCC Rcd 7227, 7728-29, para. 6 (WCB 2013) (requiring rate-of-return ETCs to make certifications regarding the provision of broadband service in satisfaction of their high-cost obligations regardless of whether that customer purchases retail broadband service directly from the ETC or from an Internet service provider that purchases the ETC’s wholesale broadband transmission service offering).  

20. Previously, we designated the following ten locations as FCC-designated IXPs: New York City, NY; Washington, DC; Atlanta, GA; Miami, FL; Chicago, IL; Dallas-Fort Worth, TX; Los Angeles, CA; San Francisco, CA; Seattle, WA; and Denver, CO. See CAF Phase II Price Cap Service Obligation Order, 28 FCC Rcd at 15071 n.63.
  All of these areas, except Denver, are locations used by the MBA program, which selected these locations because they are geographically distributed major U.S. Internet peering locations.  Denver was added to the list so that all contiguous areas in the United States are within 700 miles of an FCC-designated IXP. Id.
  Because we are expanding testing to additional CAF recipients, we add the following six metropolitan areas as additional FCC-designated IXPs: Salt Lake City, UT; St. Paul, MN; Helena, MT; Kansas City, MO; Phoenix, AZ; and Boston, MA.  This expanded list ensures that most mainland U.S. locations are within 300 air miles of an FCC-designated IXP, and all are within approximately 500 air miles of one.  Further, we find that there is no reason to limit testing to the provider’s nearest IXP; rather, providers can use any FCC-designated IXP for testing purposes. 
21. Still, we recognize that non-contiguous providers face unique challenges in providing service outside the continental U.S. See Alaska Communications Comments at 8.  See also CAF Phase II Price Cap Service Obligation Order, 28 FCC Rcd at 15075, para. 35 (“Therefore, providers in noncontiguous area of the United States should conduct their latency network testing from the customer location to a point at which traffic is consolidated for transport to an Internet exchange point in the continental United States.”).
  The distance between a carrier and its nearest IXP affects latency See 2016 Measuring Broadband America Fixed Broadband Report, Federal Communications Commission Office of Engineering and Technology and Office of Strategic Planning & Policy Analysis at 20, available at  https://www.fcc.gov/reports-research/reports/measuring-broadband-america/measuring-fixed-broadband-report-2016 (2016 MBA Report) (latency “increases with the distance between the source and destination”).
 and may affect speed as well.  At this time, we do not have sufficient data to determine the extent of the effect of distance on speed performance testing.  Therefore, similar to our existing exception for non-contiguous price cap carriers accepting model-based CAF Phase II support, See CAF Phase II Price Cap Service Obligation Order, 28 FCC Rcd at 15075, para. 35.
 we permit all providers serving non-contiguous areas greater than 500 air miles from an FCC-designated IXP to conduct all required latency and speed testing between the customer premises and the point at which traffic is aggregated for transport to the continental U.S.  We have identified a sufficient number of IXPs so that no point in the continental U.S. is more than approximately 500 miles from an FCC-designated IXP.  Therefore, allowing non-contiguous providers located more than 500 miles from an FCC-designated IXP to test to the point in the non-contiguous area where traffic is aggregated for transport to the mainland will prevent these providers from being unfairly penalized for failing to meet their performance obligations solely because of the location of the areas being served.  However, as the Commission gains additional MBA and other data on speed and latency from non-contiguous areas, we may revisit this conclusion.  
B. Timing of Tests
22. Background.  Price cap carriers receiving CAF Phase II model-based support must currently conduct the required latency testing during peak periods, i.e., weeknights between 7:00 pm and 11:00 pm local time, over a minimum of two consecutive weeks per quarter. Id. at 15070-73, paras. 23, 30.
  In 2014, the Bureaus and OET sought comment on adopting a similar requirement for all high-cost support recipients serving fixed locations, to conduct required testing once hourly during peak periods, i.e., 7:00 pm to 11:00 pm daily local time, over four consecutive weeks, to occur at least once annually. See 2014 Broadband Measurement and Reporting Public Notice, 29 FCC Rcd at 12625-26, paras. 9-12.
  In 2017, the Bureaus and OET again sought comment on this proposal and broadly sought comment on USTelecom’s recommended expansion of the testing period to 18 hours, with one speed test during each of the following four testing windows: (a) 6:00 am to 10:30 am, (b) 10:30 am to 3:00 pm, (c) 3:00 pm to 7:30 pm, and (d) 7:30 pm to 12:00 am.  Noting the Commission’s previous statement that “[f]ocusing on peak usage period provides the most useful information because it demonstrates the performance users can expect when the Internet in their local area is experiencing highest demand from users,” See 2017 Performance Measures Public Notice, 32 FCC Rcd at 9324-25, para. 9 (citing 2016 MBA Report).
 the Bureaus and OET also sought comment on defining the “peak period,” testing in areas with seasonal fluctuation in traffic load, the usefulness of measurements taken during non-peak periods, and potential concerns regarding network congestion or consumer impact. See id.
  Finally, the Bureaus and OET sought comment on the number of tests necessary to determine whether a carrier’s network is meeting performance objectives. See id.

23. Several commenters support requiring testing during the peak period of network congestion. See ADTRAN Comments at 5; NTCA Comments at 11-14; Rural Electric Cooperatives Comments at 3-6.  Cf. CBT Comments at 4-5 (supporting the USTelecom 18-hour testing window, but proposing that carriers participating in the MBA program may use MBA methodology, which tests during the peak hours of 7:00 pm to 11:00 pm).
  In particular, the Rural Electric Cooperatives raise concerns that an 18-hour testing window “could open the door to abuse, as it would allow providers to test when few users are online,” and “could provide the Commission with a distorted and potentially misleading picture of network performance.” Rural Electric Cooperatives Comments at 2.
   The Rural Electric Cooperatives explain that data show Internet usage in the United States as highest on weeknights between 7:00 pm and 11:00 pm—producing increased congestion during that period. Id. at 4-5.
  Thus, the Rural Electric Cooperatives argue, “testing during peak times . . . ensures that networks function properly during all other times of day, when demand is markedly lower.” Id. at 4-5.
  Addressing concerns regarding the impact of testing on the network, the Rural Electric Cooperatives also note that 
“staggering tests throughout the . . . testing window will minimize any risk of network degradation that may exist.” Id. at 6.

24. Instead of a single evening test period, NTCA proposes two test windows, 6:00 am to 10:30 am and 7:30 pm to 12:00 am, to capture the impact of business and residential usage. NTCA Comments at 12.
  NTCA likewise “cautions that an 18-hour testing window . . . could enable providers with less-capable networks to impute performance at ‘less active’ periods to more constricted busy hours.” Id. at 13.
  If the Commission permits conducting tests over several windows during the course of a day, NTCA proposes that providers be required to report results on a per-window basis. Id. at 13.
    
25. Other commenters support USTelecom’s proposed 18-hour test window but differ in their reasoning. See CBT Comments at 4; USTelecom Comments at 7-8; WISPA Comments at 7; WTA Comments at 9.
  WISPA, for example, cites concerns similar to those of USTelecom and asserts that requiring tests “during a specified time window could increase congestion on a network.” WISPA Comments at 7.  
  Alaska Communications likewise indicates that testing exclusively during the peak period could impact customers’ broadband experience. See Alaska Communications Comments at 6.
  Alaska Communications further asks that the Commission give carriers discretion to select an appropriate period to conduct latency testing. Id. at 6-7.
  Cincinnati Bell Telephone Company LLC (CBT), on the other hand, expresses concerns about the reverse—i.e., that peak-period congestion may negatively impact carriers’ performance measurements. See CBT Comments at 4 (“Testing at multiple times throughout the day would also help the carrier determine whether any performance issues are due to network congestion or specific problems impacting an individual customer.”).
  Finally, WTA cites in support of USTelecom’s proposed testing schedule WTA members’ skepticism about whether there is a peak period for Internet usage that coincides with primetime television. WTA Comments at 8-9.

26. As proposed by the Bureaus and OET in 2014, 2014 Broadband Measurement and Reporting Public Notice, 29 FCC Rcd at 12625, para. 9.
 USTelecom supports testing over four consecutive weeks of the ETC’s choice. USTelecom Ex Parte at 3.
  However, ITTA – The Voice of America’s Broadband Providers (ITTA) cautions that USTelecom’s four-consecutive-week proposal is likely to be a disincentive for consumers to participate in testing using methods such as Whiteboxes. ITTA Comments at 2 n.7. 
  NTCA also notes that network performance may seasonally vary for certain providers and emphasizes that the testing methodology should “hold all providers to the same consistent standard of service as measured by the customer experience.” NTCA Comments at 14.

27. Discussion.  First, we establish the specific test intervals within the daily test period.  For latency, we require a minimum of one discrete test per minute, i.e., 60 tests per hour, for each of the testing hours, Testing hours are defined in paragraph 30.  We define a “test” to be a single, discrete observation or measurement of speed or latency.  
 at each subscriber test location, with the results of each discrete test recorded separately.  We note that intensive consumer use of the network (such as streaming video) during testing, referred to as cross-talk, can influence both consumer service and testing results.  The data usage load for latency testing is minimal; sending 60 UDP packets of 64 bytes each in one hour is approximately 4,000 bytes in total.  However, to prevent cross-talk from negatively affecting both the consumer experience and test results, we adopt consumer load thresholds—i.e., cross-talk thresholds—similar to those used by the MBA program. See 2016 MBA Report, Technical Appendix at 36, available at http://data.fcc.gov/download/measuring-broadband-america/2016/Technical-Appendix-fixed-2016.pdf.
  Accordingly, for latency testing, if the consumer load exceeds 64 Kbps downstream, the provider may cancel the test and reevaluate whether the consumer load exceeds 64 Kbps downstream before retrying the test in the next minute.  Providers who elect to do more than the minimum required number of latency tests at subscriber test locations must include the results from all tests performed during testing periods in their compliance calculations.
28. For speed, we require a minimum of one download test and one upload test per testing hour at each subscriber test location.  We note that speed testing has greater network impact than latency testing.  For speed testing, we require providers to start separate download and upload speed tests at the beginning of each test hour window.  As with latency, we adopt cross-talk thresholds similar to those used in the MBA program.  If the consumer load is greater than 64 Kbps downstream for download tests or 32 Kbps upstream for upload tests, the provider may defer the affected download or upload test for one minute and reevaluate whether the consumer load exceeds the relevant 64 Kbps or 32 Kbps threshold before retrying the test. Equivalent to less than 1% of 10 Mbps, a 64 Kbps load should not affect test results. 2016 Measuring Broadband America Fixed Broadband Report, Federal Communications Commission Office of Engineering and Technology and Office of Strategic Planning & Policy Analysis, Raw Data – August available at  https://www.fcc.gov/reports-research/reports/measuring-broadband-america/raw-data-measuring-broadband-america-2016#block-menu-block-4 (MBA Raw Data Aug 2016).
  This load check-and-retry must continue at one-minute intervals until the speed test can be run or the one-hour test window ends and the test for that hour is canceled. MBA data shows that a significant majority of MBA speed tests are completed within their designated 1-hour window despite consumer load.  The MBA test process only requires 5 re-tries, while our requirements call for re-tries once per minute until successful or the one-hour window is finished.  The number of failed download speed tests during MBA peak hours is approximately 1.7%.  The number of deferred tests is approximately 16%.  Id.
  Also as with latency, providers who elect to do more than the minimum required number of speed tests at subscriber test locations must include the results from all tests performed during testing periods for compliance calculations.
29. Second, to capture any seasonal effects on a carrier’s broadband performance, we require that carriers subject to the latency and speed testing requirements conduct one week of testing in each quarter of the calendar year.  Specifically, carriers must conduct one week of testing in each of the following quarters:  January through March, April through June, July through September, and October through December. We designate these particular quarterly periods for administrative ease.
  By requiring measurements quarterly, rather than in four consecutive weeks, we expect test results to reflect a carrier’s performance throughout the year, including during times of the year in which there is a seasonal increase or decrease in network usage.  Although previously WCB required price cap carriers receiving CAF Phase II support to test latency for two weeks each quarter, we find that requiring testing one week each quarter strikes a better balance of accounting for seasonal changes in broadband usage and minimizing the burden on consumers who may participate in testing. The Commission requires that Alaska Plan carriers observe the same latency standard as price cap carriers receiving CAF Phase II support and A-CAM carriers.  See Connect America Fund, et al., Report and Order and Further Notice of Proposed Rulemaking, 31 FCC Rcd 10139, 10146-47, paras. 19-20 (2016) (Alaska Plan Order).  Alaska Plan carriers must therefore comply with testing standards and parameters consistent with what is “adopted for price cap carriers serving non-contiguous areas,” including the requirements adopted herein.  Id.

30. Third, in establishing the daily testing period, we slightly expand the test period and require that carriers conduct tests between 6:00 pm and 12:00 am (testing hours), including on weekends.  We continue to find that MBA data supports our conclusion that there is a peak period of Internet usage every evening. See MBA Raw Data Aug 2016. See also Verizon QuickAnswers, “The Internet Rush Hour”, page 1, https://www.verizon.com/cs/groups/public/documents/adacct/internet-rush-hour.pdf. 
  However, we intend to revisit this requirement periodically to determine whether peak Internet usage times have changed substantially.
31. We conclude that requiring measurements over an expanded period, by including one hour before the peak period and one hour after, will best ensure that carriers meet the speed and latency obligations associated with the high-cost support they receive. See NTCA Comments at 12 (proposing a testing period broader than 7:00 pm to 11:00 pm but significantly narrower than USTelecom’s proposed 18-hour window).  Cf. Rural Electric Cooperatives Comments at 2 (urging the Commission to “require speed testing during peak times” and arguing that “USTelecom’s proposal, which would permit testing over an 18-hour span, could open the door to abuse, as it would allow providers to test when few users are online”).
  MBA data shows that broadband Internet access service providers that perform well during the peak period tend to perform well consistently throughout the day. See 2016 Measuring Broadband America Fixed Broadband Report, Federal Communications Commission Office of Engineering and Technology and Office of Strategic Planning & Policy Analysis, available at  https://www.fcc.gov/general/measuring-broadband-america.
  Further, our required schedule of testing is consistent with the specific, realistic standards we set forth which were developed using MBA peak-period data.  Thus, we will be judging testing hours data based on a standard developed using MBA data from the same time period.
32. Additionally, we disagree with assertions that requiring speed testing during the peak period will introduce problematic network congestion over the provider’s core network.  Based on MBA speed test data, a download service speed test for 10 Mbps requires approximately 624 MB combined downloaded data for 50 locations per hour. The following factors are used in this calculation: 50 testing locations; test duration warmup time plus 10 seconds per test; and median 12.48 MB used per test.  Id.  
  This is less traffic than what would be generated by streaming a little less than one-half of a high-definition movie. For purposes of this calculation, we used an estimate of 3 GB per hour for streaming a high-definition video. See Netflix Help Center, How can I control how much data Netflix uses?, available at https://help.netflix.com/en/node/87. 
  A download service speed test for 25 Mbps requires approximately 1,841 MB combined downloaded data for 50 locations, The following factors are used in this calculation: 50 testing locations; test duration warmup time + 10 second test; and median 36.82 MB used per test.  MBA Raw Data Aug 2016.
 which is about the same amount of traffic as a little less than two high-definition movies. See supra n.93.
  The small amount of data should have no noticeable effect on network congestion.  Upload test data-usage is even lower.  Based upon MBA speed test data, a one-hour upload service speed test for 1 Mbps and 3 Mbps for 50 locations will be approximately 57 MB and 120 MB, respectively.  This testing will use bandwidth equivalent to uploading 12 photos to a social media website at 1 Mbps or 24 photos at 3 Mbps. For purposes of this calculation, we used an estimate of 5 MB per photo upload data usage.  See AT&T, Internet Data Calculator, available at https://www.att.com/esupport/data-calculator/index.jsp.
  To the extent that a carrier is concerned about possible impacts on the consumer experience, we permit carriers the flexibility to choose whether to stagger their tests, so long as they do not violate any other testing requirements, as we explain in our discussion of the testing intervals below. We note that we do not have data regarding speed testing for gigabit level services.  Based on the impact of testing for the other tiers, we assume that gigabit testing under the conditions required in the Order will not cause congestion on a gigabit-capable network.  However, we will revisit this issue as necessary when we have additional data.
  
33. Fourth, testing for all locations in a single speed tier in a single state must be done during the same week.  If a provider has more than one speed tier in a state, testing for each speed tier can be conducted during different weeks within the quarter.  For a provider serving multiple states, testing of each service tier does not need to be done during the same week, i.e., a provider may test its 10/1 Mbps customers in New York one week and in Pennsylvania during a different week.  We will generally consider requests for waiver or extension in cases where a major, disruptive event (e.g., a hurricane) negatively affects a provider’s broadband performance.  However, prior to requesting a waiver, providers should determine whether rescheduling testing within the 3-month test window will be sufficient to handle the disruptive event.
C. Number of Test Locations
34. Background.  Price cap carriers must currently test 50 randomly-selected customer locations A customer location is a location at which there is an active subscriber.
 within the census blocks for each state for which the provider is receiving model-based support. CAF Phase II Price Cap Service Obligation Order, 28 FCC Rcd at 15071, para. 23.
  However, in states where a price cap carrier receives model-based support for 2,000 or fewer lines, WCB found that a provider must conduct measurements for 20 customer locations, rather than 50. Id. at 15071 n.64.
  In 2014 and 2017, we sought comment on applying the existing requirement more broadly. 2014 Broadband Measurement and Reporting Public Notice, 29 FCC Rcd at 12626, paras. 13-14; 2017 Performance Measures Public Notice, 32 FCC Rcd at 9323, para. 6.
  We also sought comment on USTelecom’s suggestion that we require each provider to test the lesser, in each state, of (a) 20 percent of the locations reported in USAC’s High Cost Universal Broadband portal (HUBB) or (b) 50 subscribers. 2017 Performance Measures Public Notice, 32 FCC Rcd at 9324, para. 8.

35. Commenters generally agree that smaller providers should not be required to test 50 locations but vary regarding the exact number of locations such providers must test. See CBT Comments at 2-3; ITTA Comments at 2-3; NTCA Comments at 8-9; Rural Electric Cooperatives Comments at 7-9; WISPA Comments at 6-7; WTA Comments at 9-10.
  WISPA supports USTelecom’s proposed alternative of 20 percent of HUBB locations in each state, See WISPA Comments at 6-7.
 but other commenters argue that smaller providers should test a lesser number of locations than USTelecom’s proposed alternative. See CBT Comments at 2-3; NTCA Comments at 8-9; Rural Electric Cooperatives Comments at 7-9; WTA Comments at 9-10.
  For example, NTCA and the Rural Electric Cooperatives both propose testing the lesser of 50 locations or 5 percent of actively subscribed locations per state. See NTCA Comments at 8-9; Rural Electric Cooperatives Comments at 7-9.
  WTA similarly advocates that rural carriers, at least, should have the option of testing 5 percent of the carrier’s locations in a state. See WTA Comments at 9-10.
  WTA argues that 50 locations is a relatively trivial number for AT&T or Verizon, and the “proposed alternative of 20 percent (20.0%) of an ETC’s locations in a state would provide no relief from the testing burden” for a carrier exceeding 250 locations served in a state. Id. at 9.
  ITTA proposes reducing the sample size overall to 20 subscribers per state or one percent of locations, although ITTA also asks that the Commission clarify that these numbers represent a total number by state and not a minimum sample size per speed tier, per state. See ITTA Comments at 2-3.
  Finally, ADTRAN suggests requiring testing a “statistically significant number of locations” for each kind of technology a carrier uses to provide the services subject to testing. See ADTRAN Comments at 5-6.

36. Discussion.  We require that carriers test up to 50 locations per CAF-required service tier offering per state, depending on the number of subscribers a carrier has in a state.  The subscribers eligible for testing must be at locations that are reported in the HUBB where there is an active subscriber.  We decline to adopt a simple percentage-based alternative but, instead, adopt the following scaled requirements for each state and service tier combination for a carrier:

Required Test Locations for Speed
Number of Subscribers at CAF-Supported Locations per State and Service Tier Combination

Number of Test Locations
50 or fewer
5
51–500
10% of total subscribers
Over 500
50

We recognize that it is possible that a carrier serving 50 or fewer subscribers in a state and particular service tier cannot find the required number of five active subscribers for testing purposes. Carriers that do not employ self-testing systems built into CPE may need to install additional equipment, whether as small as a Raspberry Pi-sized device or as large as a router or modem, at the customer premises.  Such carriers typically ask their customers to opt into allowing the carrier to conduct tests using additional equipment.  For smaller carriers in particular, it may be difficult to find five such volunteers at CAF-supported locations.  See NTCA Comments at 6-8 (noting that, as part of the testing process, carriers ask customers for consent).
  To the extent necessary, we permit such carriers to test existing, non-CAF-supported active subscriber locations within the same state and service tier to satisfy our requirement of testing five active subscriber locations. We adopt different non-compliance requirements for those carriers that must test subscriber locations not supported by the Connect America Fund.  See infra para. 66.
  Carriers may voluntarily test the speed and/or latency of additional randomly selected CAF-supported subscribers over the minimum number of required test locations as part of their quarterly testing.  However, data for all tested locations must be submitted for inclusion in the compliance calculations, i.e., carriers must identify the set of testing locations at the beginning of the testing and cannot exclude some locations during or after the testing.
37. Carriers must test an adequate number of subscriber locations to provide a clear picture of the carrier’s performance and its customers’ broadband experience across a state.  We find that 50 test locations, per speed tier per state, remains a good indicator as to whether providers are fulfilling their obligations. See CAF Phase II Price Cap Service Obligation Order, 28 FCC Rcd at 15071, para. 23.  We also decline to require that carriers test a certain number of locations per technology it employs to provide the services subject to testing.  We require that high-cost support recipients meet their performance obligations regardless of their technology choices.    
  A sample size of 50 test locations out of 2,500 or more subscribers provides a picture of carriers’ performance with a ±11.5 percent margin of error and 90 percent confidence level. Although ADTRAN argues that the Commission require “[t]esting on a statistically significant sample size,” ADTRAN fails to identify a specific confidence level and margin of error.  ADTRAN Comments at 5.  We believe that the sample sizes adopted herein are sufficient to show whether carriers are complying with our speed and latency requirements.
  Testing 50 locations out of more than 500 subscribers yields a comparable picture of carriers’ performance.  We acknowledge, however, that smaller carriers may find testing 50 locations burdensome. See CBT Comments at 2-3; ITTA Comments at 2-3; NTCA Comments at 8-9; Rural Electric Cooperatives Comments at 7-9; WISPA Comments at 6-7; WTA Comments at 9-10.
  Below 2,500 CAF-supported subscribers, greater percentages of subscribers are necessary to achieve the same margin of error and confidence level, but below 500 subscribers the necessary percentage rises quickly above 10 percent.  Carriers serving fewer subscribers would thus be unable to provide test results achieving the same margin of error and confidence level without testing a more proportionately burdensome percentage of their subscribers.
38. We also now find it preferable to use the number of subscribers in a state and service tier, rather than the number of lines for which a provider is receiving support, to determine the required number of test locations. CAF Phase II Price Cap Service Obligation Order, 28 FCC Rcd at 15071 n.64.
  A carrier receiving support for 2,000 lines serving 100 subscribers would find it much more difficult to test 50 active subscriber locations, compared to a carrier receiving support for 2,000 lines but serving 1,500 subscribers, and commenters have noted that providers may find it difficult to find a sufficient number of locations if they have relatively few subscribers. See, e.g., WTA Comments at 9-10 (stating that “RLECs may find it difficult to find enough households that agree to be tested in order to meet the required minimum sample size”).
  Basing the number of locations to be tested on the number of subscribers, rather than the number of lines, addresses this concern. 
39. We therefore require testing a specific number of subscribers for carriers serving more than 500 subscribers in a single service tier and state, but require carriers serving between 51 and 500 subscribers in a single service tier and state to test a fixed percentage of subscribers.  For carriers serving 50 or fewer subscribers in a state and service tier, a percentage-based alternative may be insufficient; in an extreme situation, data from a single subscriber cannot clearly demonstrate a carrier’s speed and latency performance.  Accordingly, we require those providers to test a specific number of active subscriber locations.  We conclude that this scaled approach balances the need to test a reasonable number of subscriber locations within a state based on the total number of subscribers and performance tiers with minimizing the burden on smaller providers to find consumer locations to be tested.  We note, also, that a carrier receiving different types of CAF funding in the same state should aggregate its customers in each speed tier for purposes of testing.  The following examples illustrate how this scaled approach should be implemented:
· A carrier with 2,300 customers subscribed to a single service tier of 10/1 Mbps in one state must test 50 locations in that state, while a carrier providing solely 25/3 Mbps service to over 2,500 subscribers in each of three states must test 50 locations in each state.
· A carrier providing 10/1 Mbps service and 25/3 Mbps service to 100 subscribers each in a single state must test 10 locations for each of the two service tiers—20 locations in total.
· A carrier providing solely 10/1 Mbps service to 30 subscribers must test five locations, and if that carrier is only able to test three CAF-supported locations, that carrier must test two non-CAF-supported locations receiving 10/1 Mbps service in the same state.
· A carrier with 2,000 customers subscribed to 10/1 Mbps in one state through CAF Phase II funding and 500 RBE customers subscribed to 10/1 Mbps in the same state, and no other high-cost support with deployment obligations, must test a total of 50 locations in that state for the 10/1 Mbps service tier. To ensure that carriers are testing the appropriate number of locations at the correct speed tier(s), we will require carriers to submit information regarding the number of subscribers at each speed tier as part of their test results.  

40. Test subjects must be randomly selected every two years from among the provider’s active subscribers in each service tier in each state.  Subscribers for latency testing may be randomly selected from those subscribers being tested for speed at all speed tiers or randomly selected from all CAF-supported subscribers, every two years.  Any sample location lacking an active subscriber 12 months after that location was selected must be replaced by an actively subscribed location, randomly selected.  Random selection will ensure that providers cannot pick and choose amongst subscribers so that only those subscribers likely to have the best performance (e.g., those closest to a central office) are tested.  Carriers may use inducements to encourage subscribers to participate in testing.  This may be particularly useful in cases where support is tied to a particular performance level for the network but the provider does not have enough subscribers to higher performance service to test to comply with the testing sample sizes.  However, to ensure that the selection remains random, carriers must offer the same inducement to all randomly-selected subscribers in the areas for which participating subscribers are required for the carrier to conduct testing.  WCB will provide further guidance regarding random selection by public notice.  
D. High-Latency Voice Performance Testing
41. Background.  In the CAF Phase II Auction Order, the Commission adopted an alternative standard for carriers unable to meet the 100 ms latency standard already applicable to price cap carriers receiving CAF Phase II model-based support. CAF Phase II Auction Order, 31 FCC Rcd at 5960, para. 29 (adopting a minimum latency requirement that 95% or more of all peak period measurements of network round trip latency are at or below 100 milliseconds).  See also CAF Phase II Service Obligation Order, 28 FCC Rcd at 15068-72, paras. 19-25.
  Bidders submitting these high-latency bids are required to meet a two-part standard for the latency of both their voice and broadband service:  (1) for voice performance, high-latency bidders must demonstrate Mean Opinion Score (MOS) of four or higher and, (2) for broadband, these bidders must show that 95 percent or more of all testing hour measurements of network round trip latency are at or below 750 ms. CAF Phase II Auction Order, 31 FCC Rcd at 5960-61, para. 30.
 
42. In response, ADTRAN filed a petition for clarification or reconsideration seeking to clarify that an applicant selecting the high-latency option must be prepared to demonstrate that its service meets a MOS of four or higher under the International Telecommunication Union’s ITU-T Recommendation P.800’s conversational-opinion tests and not the listening-opinion tests. Petition for Clarification or Reconsideration of ADTRAN, Inc., WC Docket No. 10-90 et al. (filed July 5, 2016) (ADTRAN Petition).
  Commenters, including the Rural Coalition and the Institute for Self-Reliance, expressed support for ADTRAN’s petition. See Comments of ADTRAN, Inc., AU Docket No. 17-182, WC Docket No. 10-90, at 2-4 (filed Sept. 18, 2017); Reply Comments of the Rural Coalition, AU Docket No. 17-182, WC Docket No. 10-90, at 18-19 (filed Oct. 18, 2017); Reply Comments of the Institute for Local Self-Reliance et al., AU Docket No. 17-182, WC Docket No. 10-90, at 7-8 (filed Oct. 18, 2017) (Institute for Local Self-Reliance et al. Reply).
  Alternatively, if the Commission intended to allow an applicant to demonstrate compliance using either conversational-opinion tests or listening-opinion tests, ADTRAN asks that the Commission reconsider that decision. ADTRAN Petition at 3.

43. The Commission subsequently clarified that it had not yet specified which of the methods for subjective determination of transmission quality identified in ITU-T Recommendation P.800 should be used to demonstrate compliance with the voice component of the two-part standard (MOS of four or higher) and declined to do so at the time. Connect America Fund, et al., WC Docket No. 10-90, et al., Order on Reconsideration, 33 FCC Rcd 1089 at 1386, para. 16 (2018).
  The Commission noted its expectation that the Bureaus and OET would adopt a specific methodology. Id.

44. Discussion.  We reiterate the Commission’s requirement that high-latency providers subject to testing must demonstrate a MOS of four or higher.  We agree with ADTRAN that listening-opinion tests would not suffice to demonstrate a high-quality consumer voice experience. See ADTRAN Petition at 2-3.
   Latency only minimally affects participants’ experiences and evaluations in listening-opinion tests, which involve passive listening to audio samples. International Telecommunication Union, Telecommunication Standardization Sector, Series P: Telephone Transmission Quality, Methods for subjective determination of quality, P.800 at 3 (Aug. 1996) (ITU Series P).
  However, in the USF/ICC Transformation Order, the Commission required “ETCs to offer sufficiently low latency to enable use of real-time applications, such as VoIP.” USF/ICC Transformation Order, 26 FCC Rcd at 17698, para. 96 (footnote omitted).
  Unlike a listening-opinion test, in a conversation-opinion test, two participants actively participate in a conversation. ITU Series P at 3.
  The back-and-forth of conversations highlights delay, echo, and other issues caused by latency in a way that one-way, passive listening cannot.  Therefore, we require that high-latency providers conduct an ITU-T Recommendation P.800 conversational-opinion test.  
45. Specifically, we require the use of the underlying conversational-opinion test requirements specified by the ITU-T Recommendation P.800, with testing conditions as described below.  We believe that MOS testing under these conditions will ensure that the test results reflect the consumer experience as accurately as possible. The following information should be collected during MOS testing: the conversation-opinion score, the difficulty scale, measured active speech levels, duration of testing, time and date of testing, and other information on testing environment and tested subjects. WCB will provide further guidance by public notice on consumer selection and data collection.
  First, high-latency providers must use operational network infrastructure, such as actual satellite links, for conducting MOS testing, not laboratory-based simulations intended to reproduce service conditions.  Second, the tests must be implemented using equipment, systems, and processes that are used in provisioning service to locations funded by high-cost universal service support.  Third, live interviews and surveys must be conducted by an independent agency or organization (Reviewer) to determine the MOS.  Survey forms, mail-in documentation, automated phone calls, or other non-interactive and non-person-to-person interviews are not permitted.  Any organization or laboratory with experience testing services for compliance with telecommunications industry-specified standards and, preferably, MOS testing experience, may be a Reviewer.  Fourth, testing must be conducted over a “single hop” satellite connection with at least one endpoint at an active subscriber location using the subscriber’s end-user equipment.  Finally, the second endpoint may be a centralized location from which the Reviewer conducts live interviews with the subscriber to determine the subscriber’s MOS evaluation.
46. To reduce the burden of the MOS testing for high-latency bidders while still ensuring high-quality voice service, we adopt a separate scaled table for the number of locations that are subject to MOS testing.  Specifically, we will determine the number of testing locations based upon the number of subscribers nationally for which CAF-supported service is provided. We recognize that the satellite infrastructures employed by many high-latency bidders have characteristics different from terrestrial networks that make testing of satellite service on a national, rather than state, basis appropriate.  That is, middle-mile/backhaul for satellite networks are the direct links from the consumer locations to the satellite and then from the satellite to selected downlink sites, so there is unlikely to be significant variability based on the state in which the subscriber is located.  The consumers must be randomly selected from the total CAF-supported subscriber base in all applicable states to ensure that different types of geographic locations are tested.

Required Test Locations for MOS Testing
Number of  Subscribers at CAF-Supported Locations Nationally
Number of MOS Test Locations
3500 or fewer
100
Over 3500
370

This scaled, nationwide testing requirement will reduce high-latency bidders’ testing burden while ensuring a sufficient testing sample to verify compliance with voice performance requirements.
V. COMPLIANCE FRAMEWORK
47. We extend our existing standard for full compliance with high-cost support recipients’ latency obligations and adopt a standard for full compliance with speed obligations.  We also establish a compliance framework outlining specific actions for various degrees of compliance that fall short of those standards.
A. Standards for Full Compliance
48. Background.  In 2013, WCB used ITU calculations and reported core latencies in the contiguous United States to determine that a latency of 100 ms or below was appropriate for real-time applications, such as VoIP. See CAF Phase II Price Cap Service Obligation Order, 28 FCC Rcd at 15068-70, paras. 19-22.
  The Commission thus required price cap carriers accepting CAF Phase II model-based support to test and certify that 95 percent of testing hours latency measurements are at or below 100 ms (the latency standard). See Id. at 15068-74, paras. 19-32.
  The Commission extended the same standard to RBE participants, rate-of-return carriers electing the voluntary path to model support, CAF Phase II competitive bidders not submitting high-latency bids, and Alaska Plan carriers. See Rural Broadband Experiments Order, 29 FCC Rcd at 8795, para. 78; 2016 Rate-of-Return Reform Order, 31 FCC Rcd at 3099, para. 28; CAF Phase II Auction Order, 31 FCC Rcd at 5960, para. 29; Alaska Plan Order, 31 FCC Rcd at 10146-47, paras. 19-20.
  As explained above, bidders submitting high-latency bids must certify that 95 percent or more of all testing hour measurements of network round trip latency are at or below 750 ms. See CAF Phase II Auction Order, 31 FCC Rcd at 5960-61, para. 30.  This is one part of high-latency bidders’ “two-part standard,” which also requires demonstrating a MOS of four or higher.  See id.; supra Section IV.D.
  In the 2014 Broadband Measurement and Reporting Public Notice, we sought comment on extending the same testing methodologies to other high-cost support recipients serving fixed locations and on how compliance with speed obligations should be determined for those carriers. See 2014 Broadband Measurement and Reporting Public Notice, 29 FCC Rcd at 12625-26, paras. 5-14.
  We later sought to refresh the record by seeking comment on USTelecom’s proposed compliance framework in which certifying “full” compliance means that 95 to 100 percent of all an ETC’s measurements during the test period meet the required speed. See 2017 Performance Measures Public Notice, 32 FCC Rcd at 9324-26, paras. 8-9 (citing USTelecom Ex Parte at 4-6).
  
49. Few commenters advocate for particular speed or latency full-compliance standards.  ADTRAN argues that requirements similar to those already established for measuring latency (i.e., the 95 percent of observations during testing hours standard) should apply to speed testing. See ADTRAN Comments at 5.
  The Rural Electric Cooperatives, however, propose requiring that providers’ measurements meet 90 percent of the required speed at least 95 percent of the time and note that data overhead, the test frame size, and hardware limitations can negatively impact speed test results. See Rural Electric Cooperatives Comments at 12.
  Because of those limitations, “even a perfectly engineered Gigabit network would ordinarily test at between 940 Mbps and 987 Mbps.” Id. at 12 n.47 (citing NetApp Knowledgebase, What Is the Theoretical Maximum Throughput of a Gigabit Ethernet Interface? (Nov. 11, 2017), https://kb.netapp.com/app/answers/answer_view/a_id/1003832).  See also Rural Electric Cooperatives Ex Parte at 1-2  (“Data overhead and the frame size involved in measurement reduces the actual throughput of 1 Gigabit service and therefore can have adverse impacts on speeds during testing.  This means that even a perfectly-engineered Gigabit network would ordinarily test around 940 Mbps.”); Letter from Alan Buzacott, Executive Director, Federal Regulatory Affairs, Verizon, to Marlene H. Dortch, Secretary, FCC, WC Docket No. 10-90, at 1-2 (filed Mar. 13, 2018) (“[S]peed measurements for gigabit-class services in urban areas are limited by technical constraints”); Letter from Joshua Seidemann, Vice President of Policy, NTCA, to Marlene Dortch, Secretary, FCC, WC Docket No. 10-90, at Attach. 1 (filed June 18, 2018) (explaining how protocol overheads and subscribers’ equipment interfaces can limit maximum broadband throughput, particularly for subscribers of 1 Gbps broadband Internet service); Letter from Mark D. Massman, P.E., President, Association of Communication Engineers, to Marlene Dortch, Secretary, FCC, WC Docket No. 10-90, at 1 (filed June 28, 2018) (ACE Ex Parte) (expressing support for NTCA’s June 18, 2018 submission regarding the technical limitations involved with providing 1 Gbps service).
  Most other commenters either support USTelecom’s compliance proposal in full or suggest alternatives that similarly assume a full compliance standard of 95 percent. See CBT Comments at 5; NTCA Comments at 15-17; USTelecom Comments at 12; WISPA Comments at 7-10.

50. Discussion.  We reaffirm the existing low-latency and high-latency standards and establish a speed standard for full compliance.  The data on round-trip latency in the United States has not markedly changed since the 2013 CAF Phase II Price Cap Service Obligation Order, See CAF Phase II Price Cap Service Obligation Order, 28 FCC Rcd at 15068-70, paras. 19-22.
 and no parties have challenged the Commission’s reasoning for the existing 100 ms latency standard. See 2016 Rate-of-Return Reform Order, 31 FCC Rcd at 3099, para. 28 (noting that no parties objected to extending the latency standard already adopted for price cap carriers to rate-of-return carriers).
  Accordingly, we conclude that all high-cost support recipients serving fixed locations, except those carriers submitting high-latency bids in the CAF Phase II auction, must certify that 95 percent or more of all testing hours measurements of network round-trip latency are at or below 100 ms.  High-latency bidders must certify that 95 percent or more of all testing hours measurements are at or below 750 ms. CAF Phase II Auction Order, 31 FCC Rcd at 5960-61, para. 30.
  Providers must record the observed latency for all latency test measurements, including all lost packet tests.  Thus, providers may not discard lost-packet tests from their test results; these tests count as discrete tests not meeting the standard. Thus, any latency test that sends packet(s) must be counted in the totals to determine if the 95% requirement is met.  A test showing latency at or below 100 ms would meet the standard while tests showing latency greater than 100 ms or with packet(s) sent but not returning, e.g., lost or discarded packets, would count as failing to meet the standard.

51. For speed, we require that 80 percent of download and upload measurements be at or above 80 percent of the CAF-required speed tier (i.e., an 80/80 standard). To ensure that outlying observations do not unreasonably affect results, we will exclude from certification calculations any speed measurements with values greater than 150% of the advertised speed, because such values are likely invalid.  Thus, for a carrier that offers 20/3 Mbps service to satisfy its CAF obligation to provide 10/1 Mbps service, we will exclude a download test result showing a value of greater than 30 Mbps (i.e., 150% of the advertised 20 Mbps download speed).  We will not automatically exclude test results that are “too slow,” because those results likely reflect poor performance or network congestion, rather than fundamental problems with the testing infrastructure. However, if a carrier knows or suspects that the testing infrastructure has failed or has negatively impacted test results, the carrier may submit evidence of the test infrastructure failure with sufficiently detailed information for the Commission to understand its cause and determine the extent to which any test results should be discarded or adjusted when calculating compliance.
  For example, if a carrier receives high-cost support for 10/1 Mbps service, 80 percent of the download speed measurements must be at or above 8 Mbps, while 80 percent of the upload speed measurements must be at or above 0.8 Mbps. The speed for which the provider should be testing is the speed required by the relevant CAF rules, not the advertised speed.  Thus, for example, if a provider must deploy broadband with a speed of 10/1 Mbps but advertises and sells services of 12/2 Mbps, the provider should calculate compliance based on a requirement of 10/1 Mbps, not 12/2 Mbps.  
  We require carriers to meet and test to their CAF obligation speed(s) regardless of whether their subscribers purchase Internet service offerings with advertised speeds matching the CAF-required speeds at CAF-eligible locations.  Thus, carriers that have deployed a network with the requisite speeds must include all subscribers at that level in their testing, but may still find it necessary to upgrade individual subscriber locations, at least temporarily, to conduct speed testing.  For example, a carrier may be required to deploy and offer 100/20 Mbps service, but only 5 of its 550 subscribers at CAF-supported locations take 100/20 Mbps service, with the remainder taking 20/20 Mbps service.  To satisfy its testing obligations, the carrier would be required to (1) test all 5 of the 100/20 Mbps subscribers and (2) randomly select 45 of its other CAF-supported subscribers, raise those subscribers’ speed to 100/20 Mbps, at least temporarily, and test those 45 subscribers. 
52. We believe that this standard best meets our statutory requirement to ensure that high-cost-supported broadband deployments provide reasonably comparable service as those available in urban areas. See 47 U.S.C. § 254(b)(3) (“Consumers in all regions of the Nation, including low-income consumers and those in rural, insular, and high cost areas, should have access to telecommunications and information services, including interexchange services and advanced telecommunications and information services, that are reasonably comparable to those services provided in urban areas and that are available at rates that are reasonably comparable to rates charged for similar services in urban areas.”).
  The most recent MBA report cites the 80/80 standard as a “key measure” of network consistency. 2016 MBA Report at 6.
  MBA data show that all fixed terrestrial broadband technologies that are included in the MBA program can meet this standard. MBA Raw Data Aug 2016.
  We are confident that high-cost support recipients’ newer fixed broadband deployments will benefit from more up-to-date technologies and network designs that should provide even better performance.  
53. Further, we expect that a realistic 80/80 standard will provide a “cushion” to address certain testing issues.  We noted above that some commenters expressed concern that they would be responsible for testing to an IXP even though that involved the use of backhaul that a provider may not control. See, e.g., NTCA Comments at 9-11.
  We believe that the 80/80 standard allows sufficient leeway to providers so that they will meet performance standards as long as they have reasonable backhaul arrangements.  In addition, commenters have raised a concern that speed testing could possibly show misleadingly low results if the subscriber being tested is using the connection at the time of the testing.  However, our testing methodology addresses this concern.  As with the MBA, we allow rescheduling of testing in instances where the customer usage exceeds MBA cross-talk thresholds. See supra Section IV.B.
  Thus, we do not anticipate that customer cross-talk will affect CAF performance data any more (or less) than the MBA program data on which our standard is based.   Customer usage should not prevent carriers with appropriately constructed networks from meeting our requirements.
54. We find that a speed standard similar to what we have adopted for latency to measure broadband speed performance, as proposed by ADTRAN, is not appropriate.  Staff analysis has found that this standard would not ensure CAF-supported service that is comparable to that in urban areas.  The 2016 MBA Report stated that “[c]onsistency of speed may be more important to customers who are heavy users of applications that are both high bandwidth and sensitive to short duration declines in actual speed, such as streaming video.” 2016 MBA Report at 9 (footnote omitted).
  A speed standard relying on an average or median value would not ensure consistency of speed because the distribution of values around the median may vary significantly.  A carrier could meet such a standard by ensuring that the average or median speed test meets a target speed, while not providing sufficiently fast service nearly half the time or to nearly half its subscribers in locations supported by universal service.  We therefore conclude that the 80/80 standard we adopt herein is a better measure of comparability and high-quality service.
55. Finally, we recognize that, because of technical limitations, it is currently unrealistic to expect that providers obligated to provide gigabit service, i.e., speeds of 1,000 Mbps,See, e.g., Connect America Fund, et al., WC Docket No. 10-90, et al., Report and Order and Order on Reconsideration, 32 FCC Rcd 1624, 1626-27, para. 10 (2017) (adopting a 1 Gbps downstream/500 Mbps upstream tier).
 achieve actual speeds of 1,000 Mbps download at the customer premises.  Typical customer premises equipment, including equipment for gigabit subscribers, permits a maximum throughput of 1 Gbps, and the overhead associated with gigabit Internet traffic (whether in urban or rural areas) can reach up to 60 Mbps out of the theoretical 1 Gbps. See, e.g., Isla McKetta, Are You Gigabit Ready?  17 Tips To Help You Get the Highest Speeds Possible, (May 3, 2017), available at http://www.speedtest.net/insights/blog/are-you-gigabit-ready-17-tips-to-help/ (stating that because of overhead, “you might be able to achieve a Speedtest result of 997 Mbps, but you’re more likely to top out at 940 Mbps”).  Cf. ACE Ex Parte at 1 (asserting that testing a 1 Gbps service would yield a result of 960 Mbps in “a best-case scenario”).  
  Customer premises equipment with higher maximum throughput are generally more costly and not readily available. See Rural Electric Cooperatives Comments at 12 n.47.
  Thus, even if a gigabit provider were to “overprovision” its gigabit service, the subscriber would not experience speeds of 1,000 Mbps. See id.
  We do not want to discourage carriers from bidding in the upcoming CAF auction to provide 1 Gbps service by requiring unachievable service levels.  We note that the 80/80 standard we adopt requires gigabit carriers to demonstrate that 80 percent of their testing hours download speed tests are at or above 80 percent of 1,000 Mbps, i.e., 800 Mbps.  This standard should not pose a barrier to carriers bidding to provide 1 Gbps service.
B. Non-Compliance and Verification Measures
56. Background.  In the USF/ICC Transformation Order and FNPRM, the Commission “delegate[d] authority to the Bureaus to finalize performance measures as appropriate,” consistent with its universal service goals, USF/ICC Transformation Order, 26 FCC Rcd at 17680, para. 48.  The Commission adopted the following universal service performance goals: “(1) preserve and advance universal availability of voice service; (2) ensure universal availability of modern networks capable of providing voice and broadband service to homes, businesses, and community anchor institutions; (3) ensure universal availability of modern networks capable of providing mobile voice and broadband service where Americans live, work, and travel; (4) ensure that rates are reasonably comparable in all regions of the nation, for voice as well as broadband services; and (5) minimize the universal service contribution burden on consumers and businesses.”  Id. 
 and sought comment on testing methodologies. USF/ICC Transformation FNPRM, 26 FCC Rcd at 18045-46, paras. 1013-17.
  The Commission asked whether providers should record test results “in a format that can be produced to USAC and auditable such that USAC or state commissions may confirm that a provider is, in fact, providing broadband at the required minimum speeds.” Id. at 18045, para. 1015.  The Commission also sought comment on “additional measures to impose greater accountability on recipients of funding,” including the establishment of non-compliance performance levels with public interest obligations that would “result in the loss of specific percentages of support.”  Id. at 18067-68, paras. 1111, 1116.
  In the 2014 Broadband Measurement and Reporting Public Notice, we sought further comment on determining providers’ compliance with their CAF speed obligations and, more generally, methodologies appropriate for other recipients of CAF support besides price cap carriers accepting model-based support. 2014 Broadband Measurement and Reporting Public Notice, 29 FCC Rcd at 12625-26, paras. 5-14.
  We also sought comment “on the circumstances that would trigger an audit of the speed and latency metrics.” Id. at 12623, para. 1.  See also id. at 12629-30, paras. 26-28.
  Subsequently, USTelecom proposed a framework for determining ETCs’ compliance with their CAF performance obligations. See USTelecom Ex Parte at 4-6, Exhibit A.
 
57. We sought comment on that proposal in the 2017 Performance Measures Public Notice. See 2017 Performance Measures Public Notice, 32 FCC Rcd at 9324-26, paras. 8-9.
  USTelecom envisions a compliance framework with four “compliance tiers” or levels, each associated with different remedial actions or obligations, below full compliance with speed standards See USTelecom Ex Parte at 4-6, Exhibit A.
— resembling the framework for non-compliance with interim deployment milestones. See 47 CFR § 54.320(d)(1).
  USTelecom defines the first level to include carriers reporting 80 to 94 percent of speed tests meeting the required speed in a state.  Those carriers would submit speed test results for each subscriber location indicating (a) the date and time of the testing and the methodology used and (b) steps to be taken to resolve the compliance gap.  Those carriers would then need “to identify and retest the subscriber locations that triggered the compliance gap and report the results every quarter” until the locations are in compliance. USTelecom Ex Parte at 5.
  However, ETCs may elect to remove a location from the HUBB (i.e., no longer report that location as “deployed”) to come into compliance if the network issue cannot be readily resolved.  When the locations that triggered the compliance gap and the other tested locations come into full compliance, the ETC would then be considered fully compliant.  ETCs that do not resolve their compliance gap issue within 12 months would be downgraded to the second level.
58. The second and third levels would operate similarly, with qualifying ranges of 70 to 79 percent and 60 to 69 percent of tests demonstrating the required speed but requiring that USAC withhold 15 or 25 percent of the monthly support received for the state.  Whenever a carrier comes into the next higher level of compliance, that level’s requirements would apply, and USAC would return the withheld support up to an amount reflecting the difference between the levels’ required withholding.  The fourth (and final) level also operates similarly, with a qualifying range of 50 to 59 percent of tests demonstrating the required speed, and for which USAC would withhold 50 percent of the relevant support, but if the ETC remains in this level for 6 months (i.e., two quarterly reporting periods), USAC would withhold all of the ETC’s support for that state and would conduct a recovery action for a percentage of support based on the ETC’s compliance gap.
59. A few commenters express varying degrees of support for USTelecom’s proposed compliance framework. See CBT Comments at 5; NTCA Comments at 15-17; USTelecom Comments at 12; WISPA Comments at 7-10.
  Notably, NTCA generally supports the framework but suggests adjustments to qualifying percentage ranges for the four levels. See NTCA Comments at 15-17.  NTCA specifically proposes defining the compliance tiers as follows: (a) Tier 1: 90-94% of measurements at or above the required speed; (b) Tier 2: 80-89%; (c) Tier 3: 70-79%t; and (d) Tier 4: less than 69%.  See id.
  Also recommending an alternative, WISPA argues that USTelecom’s proposed framework penalizes non-compliance with speed requirements more severely than the existing scheme for non-compliance with buildout milestones. See WISPA Comments at 7-10.
  More specifically, WISPA proposes that an ETC qualify for a particular level depending on both the degree of the speed compliance gap and the deployment compliance gap. See id.
  The Rural Electric Cooperatives, however, argue that USTelecom’s framework is too lenient and propose a different framework, without defined levels, in which USAC would withhold 2 percent of a carrier’s high-cost support for each percentage point difference between the speed measured for 90 percent of tests and the standard for full compliance. See Rural Electric Cooperatives Comments at 9-10.  The Rural Electric Cooperatives advocate for a full compliance standard of 90% of the required speed 95% of the time.  Thus, USAC would withhold no support from a carrier with a speed obligation of 100 Mbps and actual speeds of 90 Mbps or higher in 95% of its speed tests, but it would withhold 4% of the carrier’s support if that carrier’s actual speeds were 88 Mbps or higher in 95% of tests.  See id. at 12-15.
  On the other hand, WTA argues that USTelecom’s proposed “five-tier compliance and penalty system” is too punishing, particularly for rural carriers and other small ETCs. See WTA Comments at 11.
 
60. Discussion.  Consistent with the Commission’s universal service goals, we adopt a compliance framework that encourages ETCs to comply fully with their performance obligations and includes the potential for USAC to audit test results.  We establish a four-level framework that sets forth particular obligations and automatic triggers based on an ETC’s degree of compliance with our latency, speed, and, if applicable, MOS testing standards in each state and high-cost support program.  We will determine a carrier’s compliance for each standard separately.  In each case, we will divide the percentage of its measurements meeting the relevant standard by the required percentage of measurements to be in full compliance.
61. In other words, for latency, in each state in which the carrier has CAF-supported locations, we will calculate the percentage of compliance using the 95-percent standard, so we will divide the percentage of the carrier’s testing hours’ latency measurements at or below the required latency (i.e., 100 ms or 750 ms) by 95.  As an example, if a low-latency provider observes that 90 percent of all its testing hours measurements are at or below 100 ms, then that provider’s latency compliance percentage would be 90/95 = 94.7 percent in that state.  For speed, for each speed tier and state we will calculate the percentage of compliance relative to the 80-percent-based standard, so we will divide the percentage of the carrier’s testing hours speed measurements at or above 80 percent of the target speed by 80. As noted above, when first determining the percentage of its testing hours speed measurements at or above 80% of the target speed, we will exclude any observations from certification calculations above 150% of the carrier’s advertised (not target) speed.  Any such observations are likely outliers.  See supra n.145.
  Thus, if a provider observes that 65 percent of its testing hours speed measurements meet 80 percent of the required speed, the provider’s compliance percentage would be 65/80 = 81.25 percent for the relevant speed tier in that state.  Carriers must include and submit the results from all tests and cannot exclude any tests conducted beyond the minimum numbers of tests, as outlined in Section IV of this Order, for the calculation of latency and speed compliance percentages.  
62. For MOS testing, the high-latency bidder must demonstrate a MOS of 4 or higher, so a high-latency bidder would calculate its percentage of compliance relative to 4.  Thus, a provider demonstrating a MOS of 3 would have a compliance percentage of 3/4 = 75 percent.  For a high-latency bidder conducting MOS testing across its entire network, rather than state-by-state, we will calculate the same MOS compliance percentage for each state that it serves with CAF Phase II support.
63. To avoid penalizing a provider for failing to meet multiple standards for the same locations, we adopt a streamlined compliance framework in which the lowest of a carrier’s separate latency, speed, and, if applicable, MOS compliance percentages (including percentages for each speed tier) determines its obligations.  All carriers not fully compliant in a particular state must submit quarterly reports providing one week of testing hours test results, subject to the same requirements we establish in this Order, and describing steps taken to resolve the compliance gap, and USAC will withhold a percentage of a non-compliant carrier’s monthly support.  Whenever a carrier in Levels 1 through 3 comes into a higher level of compliance, that level’s requirements will apply, and USAC will return the withheld support up to an amount reflecting the difference between the levels’ required withholding but not including any support withheld by USAC for more than 12 months.  
64. We define Level 1 compliance to include carriers with compliance percentages at or above 85 but below 100 percent, and we direct USAC to withhold 5 percent of a Level 1-compliant carrier’s monthly support.  Level 2 compliance includes carriers with compliance percentages at or above 70 but below 85 percent, and we direct USAC to withhold 10 percent of a Level 2-compliant carrier’s monthly support.  Level 3 compliance includes carriers with compliance percentages at or above 55 but below 70 percent, and we direct USAC to withhold 15 percent of a Level 3-compliant carrier’s monthly support.  Level 4 compliance includes carriers with compliance percentages below 55 percent, and we direct USAC to withhold 25 percent of a Level 4-compliant carrier’s monthly support.  We will also refer Level 4-compliant carriers to USAC for an investigation into the extent to which the carrier has actually deployed broadband in accordance with its deployment obligations.  The following table provides a summary of the compliance framework, where x is the carrier’s compliance percentage:

Compliance Levels and Support Reductions

Qualifying Compliance Percentage x
Required Quarterly Reporting
Monthly Support Withheld
Full Compliance
x ≥ 100%
No
N/A
Level 1
85% ≤ x < 100%
Yes
5%
Level 2
70% ≤ x < 85%
Yes
10%
Level 3
55% ≤ x < 70%
Yes
15%
Level 4
x < 55%
Yes
25%

65. Similar to commenters’ proposals, the framework we adopt resembles the non-compliance framework for interim deployment milestones in section 54.320(d) of the Commission’s rules. See 47 CFR § 54.320(d).
  We emphasize that the goal of this compliance framework is to provide incentives, rather than penalize.  Balancing commenters’ concerns regarding the severity or leniency of a such a framework, we conclude that our framework appropriately encourages carriers to come into full compliance and offer, in areas requiring high-cost support, broadband service meeting standards consistent with what consumers typically experience.  
66. Finally, we provide one exception to this non-compliance framework.  As discussed above, carriers that serve 50 or fewer subscribers in a state and particular service tier but cannot find five active subscribers for conducting the required testing may test non-CAF-supported active subscriber locations to the extent necessary.  Because those carriers’ test results would not solely reflect the performance of CAF-supported locations, any such carriers not fully complying with our latency and speed standards will be referred to USAC for further investigation of the level of performance at the CAF-supported locations.
67. The Commission requires that providers subject to these testing requirements annually certify and report the results to USAC, which may audit the test results. 47 CFR § 54.313(a)(6); see also USF/ICC Transformation Order, 26 FCC Rcd at 17705-06, para. 109.
  To facilitate compliance monitoring, we require providers to submit speed and latency test results, including the technologies used to provide broadband at the tested locations, for each state and speed tier combination in addition to an annual certification in a format to be determined by WCB; high-latency bidders conducting MOS testing across their entire networks, rather than state-by-state, may submit and certify MOS test results on a nationwide basis.  To minimize the burden on providers, USAC will calculate the compliance percentages required using the data submitted.  By requiring carriers to submit test results annually, or quarterly if they are not fully in compliance with our standards, and having USAC perform the compliance calculations, we minimize the potential for any manipulation or gaming of the testing regime, as providers will be required to certify to a set of specific results rather than to a general level of compliance.  Because of the need to develop a mechanism for collecting the testing data and obtain Paperwork Reduction Act (PRA) approval, carriers will be required to submit the first set of testing data and accompanying certification by July 1, 2020.  This submission should include data for at least the third and fourth quarters of 2019.  Subsequently, data and certifications will be due by July 1 of each year for the preceding calendar year.  WCB will provide further guidance by public notice regarding how carriers will submit their testing data and certifications.  Together with USAC audits and possible withholding of support, we believe these measures will provide ample incentives for carriers to comply with their obligations.
VI. PROCEDURAL MATTERS
68. Paperwork Reduction Act Analysis.  The Order adopted herein contains new, proposed new or modified information collection requirements.  The Commission, as part of its continuing effort to reduce paperwork burdens, invites the general public and the Office of Management and Budget (OMB) to comment on the information collection requirements contained in this document, as required by the Paperwork Reduction Act of 1995, Public Law 104-13.  In addition, pursuant to the Small Business Paperwork Relief Act of 2002, Public Law 107-198, we seek specific comment on how we might further reduce the information collection burden for small business concerns with fewer than 25 employees. See 44 U.S.C. § 3506(c)(4).

69. Congressional Review Act.  The Commission will send a copy of this Order to Congress and the Government Accountability Office pursuant to the Congressional Review Act. See 5 U.S.C. § 801(a)(1)(A).

70. Final Regulatory Flexibility Analysis.  The Regulatory Flexibility Act of 1980 (RFA) requires that an agency prepare a regulatory flexibility analysis for notice and comment rulemakings, unless the agency certifies that “the rule will not, if promulgated, have a significant economic impact on a substantial number of small entities.” 5 U.S.C. § 605(b).  
  Accordingly, we have prepared a FRFA concerning the possible impact of the rule changes contained in the Order on small entities.  The FRFA is set forth in Appendix B.
VII. ORDERING CLAUSES
71. Accordingly, IT IS ORDERED that, pursuant to sections 1, 4(i), 5(c), 201(b), 214, and 254 of the Communications Act of 1934, as amended, and section 706 of the Telecommunications Act of 1996, 47 U.S.C. §§ 151, 154(i), 155(c), 201(b), 214, 254, 1302, sections 0.91 and 0.291 of the Commission’s rules, 47 CFR §§ 0.91, 0.291, and the delegations of authority in paragraph 170 of the USF/ICC Transformation Order, FCC 11-161, this Order IS ADOPTED, effective thirty (30) days after publication of the text or summary thereof in the Federal Register, except for the provisions subject to the PRA, See supra paras. 63, 67.
 which will become effective upon announcement in the Federal Register of OMB approval of the subject information collection requirements.  
72. IT IS FURTHER ORDERED that the Commission’s Consumer and Governmental Affairs Bureau, Reference Information Center, SHALL SEND a copy of this Order, including the Final Regulatory Flexibility Analysis, to the Chief Counsel for Advocacy of the Small Business Administration.  

FEDERAL COMMUNICATIONS COMMISSION




						Kris Anne Monteith
						Chief
						Wireline Competition Bureau

						Donald Stockdale
						Chief
						Wireless Telecommunications Bureau

Julius P. Knapp
Chief Engineer
Office of Engineering and Technology

27

Appendix A

Summary of Testing Requirements

· This framework applies to all providers with CAF Phase II, A-CAM, rate-of-return mandatory buildout, RBE, and Alaska Plan obligations.
· Three testing options will be permitted:  use of MBA testing; off-the-shelf testing; and provider-developed self-testing or self-testing.
· A test is defined to be a single, discrete observation or measurement of speed or latency conducted from the customer premises of an active subscriber at a CAF-supported location to a remote test server located at, or reached by, passing through an FCC-designated IXP.
· For providers serving non-contiguous areas more than 500 air miles from the contiguous United States, testing must be conducted from the customer premises of an active subscriber to the point in the non-contiguous area where all mainland traffic is aggregated for transport from the non-contiguous area.
· Testing must be conducted for one week during each quarter of the year.  In those weeks, testing must be performed between the hours of 6:00 pm to 12:00 am local time each day, including weekends (testing hours).  
· For latency testing, a provider must conduct a minimum of one test per minute—sixty tests per hour—for each testing hour.  If the consumer load during a latency test exceeds 64 Kbps downstream, the provider may cancel the test and reevaluate whether the consumer load exceeds 64 Kbps downstream before retrying the test in the next minute.
· For speed testing, a provider must conduct a minimum of one test per testing hour in each direction (download and upload) and start at the beginning of each test hour.  If the consumer load is greater than 64 Kbps downstream for download tests or 32 Kbps upstream for upload tests, the provider may defer the affected test for one minute and reevaluate whether the load exceeds the relevant 64 Kbps or 32 Kbps threshold before retrying the test.  This load check-and-retry must continue at one-minute intervals until the speed test can be run or the one-hour test window ends and the test for that hour is canceled. 
· The first performance measures data and certification will be due by July 1, 2020 and shall include data for the third and fourth quarters of 2019.  Thereafter, data and certification will be due July 1 each year for all four quarters of the prior calendar year.
· Test subjects for speed testing must be randomly selected from among the provider’s active subscribers in each speed tier in each state.  Subscribers for latency testing may be randomly selected from those subscribers being tested for speed at all speed tiers in that state.
· The number of consumers to be tested will be based on the number of subscribers at CAF-supported locations, with a maximum of 50 randomly-selected subscribers per state per speed tier for speed testing and 50 randomly-selected subscribers per state for latency. 
· To comply with the speed standard, a provider’s certified test results, for each state and service tier, must show that 80 percent of the upload measurements are at or above 80 percent of the required upload speed and 80 percent of the download measurements are at or above 80 percent of the required download speed. 
· To comply with the latency standard, a provider’s (excluding high-latency bidders in the CAF Phase II auction) certified test results must show, for each state, that 95 percent or more of all tests of network round trip latency are at or below 100 ms when measured between the customer premises and a remote server that is located at or reached by passing through an FCC-designated IXP.  
· Certified test results from high-latency bidders in the CAF Phase II auction must show that 95 percent or more of all testing hours measurements of network round trip latency are at or below 750 ms when measured between the customer premises and an FCC-designated IXP.  In addition, high-latency bidders must show a MOS of 4 or above using a modified ITU Standard Recommendation P.800 conversational-opinion test conducted over the actual network by an independent testing organization.
· All test results must be submitted.  In other words, providers cannot delete, trim, edit or otherwise exclude any test measurements.  However, if a provider knows or suspects that the testing infrastructure has failed or has negatively impacted test results, the provider may submit evidence of the test infrastructure failure with sufficiently detailed information for the Commission to understand its cause and determine the extent to which any test results should be discarded or adjusted when calculating compliance.  Providers must still submit such test results.


29

APPENDIX B

Final Regulatory Flexibility Analysis

1. As required by the Regulatory Flexibility Act of 1980 (RFA), 5 U.S.C. § 603.  The RFA, 5 U.S.C. §§ 601-612 has been amended by the Contract With America Advancement Act of 1996, Public Law No. 104-121, 110 Stat. 847 (1996) (CWAAA).  Title II of the CWAAA is the Small Business Regulatory Enforcement Fairness Act of 1996 (SBREFA).
 as amended, an Initial Regulatory Flexibility Analysis (IRFA) was incorporated in the USF/ICC Transformation FNPRM. See Connect America Fund et al., WC Docket No. 10-90 et al., Report and Order and Further Notice of Proposed Rulemaking, 26 FCC Rcd 17663, 18364-95 (2011) (USF/ICC Transformation Order and FNPRM),
  The Commission sought written public comment on the proposals in the USF/ICC Transformation FNPRM, including comment on the IRFA.  The Commission did not receive any relevant comments on the USF/ICC Transformation FNPRM IRFA.  This present Final Regulatory Flexibility Analysis (FRFA) conforms to the RFA. See 5 U.S.C. § 604.

A. Need for, and Objective of, the Order
2. As a condition of receiving high-cost universal service support, eligible telecommunications carriers (ETCs) must offer broadband service in their supported areas that meets certain basic performance requirements. See USF/ICC Transformation FNPRM, 26 FCC Rcd at 17705-06, para. 109.
  ETCs subject to broadband performance obligations must currently offer broadband with latency suitable for real-time applications, such as VoIP, and meet a minimum speed standard of 10 Mbps downstream and 1 Mbps upstream or greater. Connect America Fund; ETC Annual Reports and Certifications, Report and Order, 29 FCC Rcd 15644, 15649, para. 15 (2014).  Although the Commission adopted 10 Mbps downstream and 1 Mbps upstream as a minimum speed standard for recipients of high-cost support, depending on the outcome of the Commission’s competitive bidding processes, some recipients may bid and receive support to offer faster speeds or, potentially, higher-latency service.  See Connect America Fund; ETC Annual Reports and Certifications; Rural Broadband Experiments, Report and Order and Further Notice of Proposed Rulemaking, 31 FCC Rcd 5949, 5957, para. 15 (2016) (allowing bids of different performance tiers with speeds of 1 Gbps/500 Mbps, 100/20 Mbps, 25/3 Mbps, and 10/1 Mbps).  See also, e.g., Connect America Fund; ETC Annual Reports and Certifications, Report and Order and Further Notice of Proposed Rulemaking, 29 FCC Rcd 8769, 8779-80, paras. 24-29 (2014) (making available support through the rural broadband experiments for services providing speeds of 100/25 Mbps, 25/5 Mbps, and 10/1 Mbps).
  Recipients of high-cost support must also test their broadband networks for compliance with speed and latency metrics and certify and report the results to the Universal Service Administrative Company (USAC) and the relevant state or tribal government on an annual basis, with those results subject to audit. USF/ICC Transformation Order, 26 FCC Rcd at 17705-06, para. 109.

3. In the Order, we define how ETCs with Connect America Fund (CAF) Phase II, Alternative Connect America Cost Model (A-CAM), rate-of-return mandatory buildout, rural broadband experiment (RBE), or Alaska Plan obligations must test speed and latency and certify and report the results.  Specifically, we establish a uniform framework for measuring speed and latency performance.  We permit three testing methods as options for ETCs to conduct the required speed and latency tests, See supra Section III.
 and we provide a definition for a “test” in this context and specify the measurement span associated with these tests. See supra Section IV.A.
  We establish specific test parameters for latency and speed, including how often and how many tests must be conducted See supra Section IV.B.
 and the minimum test sample size. See supra Section IV.C.
  We also establish voice testing requirements for high-latency bidders in the CAF Phase II auction. See supra Section IV.D.
   Finally, we define compliance for latency and speed standards and establish the required certifications, See supra Section V.A.
 as well as a compliance framework providing strong incentives for ETCs to meet our standards. See supra Section V.B.
 
4. With the testing framework we have adopted herein, we have provided maximum flexibility to reduce the burden on smaller entities, consistent with ensuring that these carriers are meeting their latency and speed requirements.  Smaller entities required to do testing can choose from one of three methodologies to conduct the required testing.  All entities providing broadband service should already use testing mechanisms for internal purposes, such as ensuring that customers are receiving the appropriate level of service and troubleshooting in response to customer complaints.  In addition, we will be providing an online portal so entities can easily submit all of their test results electronically and USAC will do all of the necessary compliance calculations.  
B. Summary of Significant Issues Raised by Public Comments in Response to the IRFA
5. There were no comments raised that specifically addressed how broadband service should be measured, as presented in the USF/ICC Transformation FNPRM IRFA. See USF/ICC Transformation FNPRM, 26 FCC Rcd at 18364, para. 3.
  Nonetheless, the Wireline Competition Bureau, Wireless Telecommunications Bureau, and Office of Engineering and Technology considered the potential impact of the rules proposed in the IRFA on small entities and reduced the compliance burden for all small entities in order to reduce the economic impact of the rules enacted herein on such entities. 
C. Response to Comments by the Chief Counsel for Advocacy of the Small Business Administration
6. Pursuant to the Small Business Jobs Act of 2010, 5 U.S.C. Sec. 604(a)(3).
 which amended the RFA, the Commission is required to respond to any comments filed by the Chief Counsel of the Small Business Administration (SBA), and to provide a detailed statement of any change made to the proposed rule(s) as a result of those comments. 
7. The Chief Counsel did not file any comments in response to the proposed rule(s) in this proceeding.
D. Description and Estimate of the Number of Small Entities to Which the Rules Would Apply
8. The RFA directs agencies to provide a description of, and where feasible, an estimate of the number of small entities that may be affected by the proposed rules, if adopted. See 5 U.S.C. § 603(b)(3).
  The RFA generally defines the term “small entity” as having the same meaning as the terms “small business,” “small organization,” and “small governmental jurisdiction.” See 5 U.S.C. § 601(6).
  In addition, the term “small business” has the same meaning as the term “small-business concern” under the Small Business Act. See 5 U.S.C. § 601(3) (incorporating by reference the definition of “small-business concern” in the Small Business Act, 15 U.S.C. § 632).  Pursuant to 5 U.S.C. § 601(3), the statutory definition of a small business applies “unless an agency, after consultation with the Office of Advocacy of the Small Business Administration and after opportunity for public comment, establishes one or more definitions of such term which are appropriate to the activities of the agency and publishes such definition(s) in the Federal Register.”
  A small-business concern” is one which:  (1) is independently owned and operated; (2) is not dominant in its field of operation; and (3) satisfies any additional criteria established by the Small Business Administration (SBA). See 15 U.S.C. § 632.

1. Total Small Entities   
9. Our actions, over time, may affect small entities that are not easily categorized at present.  We therefore describe here, at the outset, three broad groups of small entities that could be directly affected herein.  First, while there are industry specific size standards for small businesses that are used in the regulatory flexibility analysis, according to data from the SBA’s Office of Advocacy, in general a small business is an independent business having fewer than 500 employees.  These types of small businesses represent 99.9 percent of all businesses in the United States which translates to 28.8 million businesses.   
10. Next, the type of small entity described as a “small organization” is generally “any not-for-profit enterprise which is independently owned and operated and is not dominant in its field.”   Nationwide, as of August 2016, there were approximately 356,494 small organizations based on registration and tax data filed by nonprofits with the Internal Revenue Service (IRS).   
11. Finally, the small entity described as a “small governmental jurisdiction” is defined generally as “governments of cities, counties, towns, townships, villages, school districts, or special districts, with a population of less than fifty thousand.”  U.S. Census Bureau data from the 2012 Census of Governments indicates that there were 90,056 local governmental jurisdictions consisting of general purpose governments and special purpose governments in the United States.  Of this number there were 37, 132 General purpose governments (county, municipal and town or township) with populations of less than 50,000 and 12,184 Special purpose governments (independent school districts and special districts) with populations of less than 50,000.  The 2012 U.S. Census Bureau data for most types of governments in the local government category shows that the majority of these governments have populations of less than 50,000.  Based on this data we estimate that at least 49,316 local government jurisdictions fall in the category of “small governmental jurisdictions.”
2. Broadband Internet Access Service Providers
12. Broadband Internet service providers include wired (e.g., cable, DSL) and VoIP service providers using their own operated wired telecommunications infrastructure fall in the category of Wired Telecommunication Carriers. See, 13 CFR § 121.201. The Wired Telecommunications Carrier category formerly used the NAICS code of 517110. As of 2017 the U.S. Census Bureau definition show the NAICs code as 517311.  See, https://www.census.gov/cgi-bin/sssd/naics/naicsrch?code=517311&search=2017 
  Wired Telecommunications Carriers are comprised of establishments primarily engaged in operating and/or providing access to transmission facilities and infrastructure that they own and/or lease for the transmission of voice, data, text, sound, and video using wired telecommunications networks.  Transmission facilities may be based on a single technology or a combination of technologies. Id.
  The SBA size standard for this category classifies a business as small if it has 1,500 or fewer employees. Id.
  U.S. Census data for 2012 show that there were 3,117 firms that operated that year.  Of this total, 3,083 operated with fewer than 1,000 employees. http://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml? pid=ECN_2012_US_51SSSZ2&prodType=table.
  Consequently, under this size standard the majority of firms in this industry can be considered small.
13. The broadband Internet access service provider industry has changed since this definition was introduced in 2007.  The data cited above may therefore include entities that no longer provide broadband Internet access service, and may exclude entities that now provide such service.  To ensure that this FRFA describes the universe of small entities that our action might affect, we discuss in turn several different types of entities that might be providing broadband Internet access service.  We note that, although we have no specific information on the number of small entities that provide broadband Internet access service over unlicensed spectrum, we include these entities in our Final Regulatory Flexibility Analysis.
3. Wireline Provider
14. Wired Telecommunications Carriers.  The U.S. Census Bureau defines this industry as “establishments primarily engaged in operating and/or providing access to transmission facilities and infrastructure that they own and/or lease for the transmission of voice, data, text, sound, and video using wired communications networks.  Transmission facilities may be based on a single technology or a combination of technologies.  Establishments in this industry use the wired telecommunications network facilities that they operate to provide a variety of services, such as wired telephony services, including VoIP services, wired (cable) audio and video programming distribution, and wired broadband internet services.  By exception, establishments providing satellite television distribution services using facilities and infrastructure that they operate are included in this industry.” See 13 CFR § 120.201.  The Wired Telecommunications Carrier category formerly used the NAICS code of 517110. As of 2017 the U.S. Census Bureau definition shows the NAICS code as 517311 for Wired Telecommunications Carriers.  See, https://www.census.gov/cgi-bin/sssd/naics/naicsrch?code=517311&search=2017. 
  The SBA has developed a small business size standard for Wired Telecommunications Carriers, which consists of all such companies having 1,500 or fewer employees. Id.
  U.S. Census Bureau data for 2012 shows that there were 3,117 firms that operated that year. See U.S. Census Bureau, 2012 Economic Census of the United States, Table No. EC1251SSSZ5, Information: Subject Series - Estab & Firm Size: Employment Size of Firms: 2012 (517110 Wired Telecommunications Carriers). https://factfinder.census.gov/bkmk/table/1.0/en/ECN/2012_US/51SSSZ5//naics~517110.
  Of this total, 3,083 operated with fewer than 1,000 employees. Id.
 Thus, under this size standard, the majority of firms in this industry can be considered small.
15. Incumbent Local Exchange Carriers (Incumbent LECs).  Neither the Commission nor the SBA has developed a small business size standard specifically for incumbent LEC services.  The closest applicable size standard under SBA rules is for the category Wired Telecommunications Carriers.  Under that size standard, such a business is small if it has 1,500 or fewer employees. 13 CFR § 121.201, NAICS code 517311.
  According to Commission data, Federal Communications Commission, Wireline Competition Bureau, Industry Analysis and Technology Division, Trends in Telephone Service, tbl. 5.3 (Sept. 2010) (Trends in Telephone Service).  
 1,307 carriers reported that they were incumbent LEC providers. See Trends in Telephone Service at tbl. 5.3. 
  Of these 1,307 carriers, an estimated 1,006 have 1,500 or fewer employees and 301 have more than 1,500 employees. See id.
   Consequently, the Commission estimates that most providers of incumbent LEC service are small. 
16. Competitive Local Exchange Carriers (Competitive LECs), Competitive Access Providers (CAPs), Shared-Tenant Service Providers, and Other Local Service Providers.  Neither the Commission nor the SBA has developed a small business size standard specifically for these service providers.  The appropriate size standard under SBA rules is for the category Wired Telecommunications Carriers.  Under that size standard, such a business is small if it has 1,500 or fewer employees. 13 CFR § 121.201, NAICS code 517311.
  According to Commission data, 1,442 carriers reported that they were engaged in the provision of either competitive local exchange services or competitive access provider services. See Trends in Telephone Service at tbl.5.3.
  Of these 1,442 carriers, an estimated 1,256 have 1,500 or fewer employees and 186 have more than 1,500 employees. See id.
  In addition, 17 carriers have reported that they are Shared-Tenant Service Providers, and all 17 are estimated to have 1,500 or fewer employees. See id.
  In addition, 72 carriers have reported that they are Other Local Service Providers. See id.
  Of the 72, seventy have 1,500 or fewer employees and two have more than 1,500 employees. See id.
  Consequently, the Commission estimates that most providers of competitive local exchange service, competitive access providers, Shared-Tenant Service Providers, and other local service providers are small entities. 
17. We have included small incumbent LECs in this present RFA analysis.  As noted above, a “small business” under the RFA is one that, inter alia, meets the pertinent small business size standard (e.g., a telephone communications business having 1,500 or fewer employees), and “is not dominant in its field of operation.” 5 U.S.C. § 601(3).
  The SBA’s Office of Advocacy contends that, for RFA purposes, small incumbent LECs are not dominant in their field of operation because any such dominance is not “national” in scope. Letter from Jere W. Glover, Chief Counsel for Advocacy, SBA, to William E. Kennard, Chairman, Federal Communications Commission (filed May 27, 1999).  The Small Business Act contains a definition of “small business concern,” which the RFA incorporates into its own definition of “small business.”  15 U.S.C. § 632(a); 5 U.S.C. § 601(3).  SBA regulations interpret “small business concern” to include the concept of dominance on a national basis.  13 CFR § 121.102(b).
  We have therefore included small incumbent LECs in this RFA analysis, although we emphasize that this RFA action has no effect on Commission analyses and determinations in other, non-RFA contexts.
18. Interexchange Carriers.  Neither the Commission nor the SBA has developed a small business size standard specifically for providers of interexchange services.  The appropriate size standard under SBA rules is for the category Wired Telecommunications Carriers.  Under that size standard, such a business is small if it has 1,500 or fewer employees. 13 CFR § 121.201, NAICS code 517311.
  According to Commission data, Trends in Telephone Service, tbl. 5.3.
 359 carriers have reported that they are engaged in the provision of interexchange service.  Of these, an estimated 317 have 1,500 or fewer employees and 42 have more than 1,500 employees.  Consequently, the Commission estimates that the majority of IXCs are small entities.
19. Operator Service Providers (OSPs).  Neither the Commission nor the SBA has developed a small business size standard specifically for operator service providers.  The appropriate size standard under SBA rules is for the category Wired Telecommunications Carriers.  Under that size standard, such a business is small if it has 1,500 or fewer employees. 13 CFR § 121.201, NAICS code 517311.
  According to Commission data, 33 carriers have reported that they are engaged in the provision of operator services.  Of these, an estimated 31 have 1,500 or fewer employees and two have more than 1,500 employees. Trends in Telephone Service, tbl. 5.3.
  Consequently, the Commission estimates that the majority of OSPs are small entities.
20. Prepaid Calling Card Providers.  Neither the Commission nor the SBA has developed a small business size standard specifically for prepaid calling card providers.  The appropriate size standard under SBA rules is for the category Telecommunications Resellers.  Under that size standard, such a business is small if it has 1,500 or fewer employees. See 13 CFR § 121.201, NAICS code 517911.  
  According to Commission data, 193 carriers have reported that they are engaged in the provision of prepaid calling cards. See Trends in Telephone Service at Table 5.3.
  Of these, an estimated all 193 have 1,500 or fewer employees and none have more than 1,500 employees. See id.
  Consequently, the Commission estimates that the majority of prepaid calling card providers are small entities. 
21. Local Resellers.  The SBA has developed a small business size standard for the category of Telecommunications Resellers.  Under that size standard, such a business is small if it has 1,500 or fewer employees. See 13 CFR § 121.201, NAICS code 517911.  
  According to Commission data, 213 carriers have reported that they are engaged in the provision of local resale services. See Trends in Telephone Service at Table 5.3.  
  Of these, an estimated 211 have 1,500 or fewer employees and two have more than 1,500 employees. See id.
  Consequently, the Commission estimates that the majority of local resellers are small entities. 
22. Toll Resellers.  The SBA has developed a small business size standard for the category of Telecommunications Resellers.  Under that size standard, such a business is small if it has 1,500 or fewer employees. See 13 CFR § 121.201, NAICS code 517911.  
  According to Commission data, 881 carriers have reported that they are engaged in the provision of toll resale services. See Trends in Telephone Service at Table 5.3.
  Of these, an estimated 857 have 1,500 or fewer employees and 24 have more than 1,500 employees. See id.
  Consequently, the Commission estimates that the majority of toll resellers are small entities.  
23. Other Toll Carriers.  Neither the Commission nor the SBA has developed a size standard for small businesses specifically applicable to Other Toll Carriers.  This category includes toll carriers that do not fall within the categories of interexchange carriers, operator service providers, prepaid calling card providers, satellite service carriers, or toll resellers.  The closest applicable size standard under SBA rules is for Wired Telecommunications Carriers.  Under that size standard, such a business is small if it has 1,500 or fewer employees. See 13 CFR § 121.201, NAICS code 517311.
  According to Commission data, 284 companies reported that their primary telecommunications service activity was the provision of other toll carriage. See Trends in Telephone Service at Table 5.3.
  Of these, an estimated 279 have 1,500 or fewer employees and five have more than 1,500 employees. See id.
  Consequently, the Commission estimates that most Other Toll Carriers are small entities.
24. 800 and 800-Like Service Subscribers. We include all toll-free number subscribers in this category, including those for 888 numbers.
  Neither the Commission nor the SBA has developed a small business size standard specifically for 800 and 800-like service (toll free) subscribers.  The appropriate size standard under SBA rules is for the category Telecommunications Resellers.  Under that size standard, such a business is small if it has 1,500 or fewer employees. See 13 CFR § 121.201, NAICS code 517911. 
  The most reliable source of information regarding the number of these service subscribers appears to be data the Commission collects on the 800, 888, 877, and 866 numbers in use. See Trends in Telephone Service at Tables 18.7-18.10. 
  According to our data, as of September 2009, the number of 800 numbers assigned was 7,860,000; the number of 888 numbers assigned was 5,588,687; the number of 877 numbers assigned was 4,721,866; and the number of 866 numbers assigned was 7,867,736. See id.
  We do not have data specifying the number of these subscribers that are not independently owned and operated or have more than 1,500 employees, and thus are unable at this time to estimate with greater precision the number of toll free subscribers that would qualify as small businesses under the SBA size standard.  Consequently, we estimate that there are 7,860,000 or fewer small entity 800 subscribers; 5,588,687 or fewer small entity 888 subscribers; 4,721,866 or fewer small entity 877 subscribers; and 7,867,736 or fewer small entity 866 subscribers. 
E. Description of Projected Reporting, Recordkeeping, and Other Compliance
Requirements for Small Entities
25.  In the Order, we establish for high-cost support recipients serving fixed locations a uniform framework for measuring speed and latency performance and define the requisite standards for full compliance with those providers’ speed and latency obligations.  The Commission’s existing rules require that high-cost recipients report “[t]he results of network performance tests pursuant to the methodology and in the format determined by the Wireline Competition Bureau, Wireless Telecommunications Bureau, and the Office of Engineering and Technology” and that ETCs retain such records for at least ten years from the receipt of funding. 47 CFR §§ 54.313(a)(6), 54.320(b).

26. We now provide some color to this requirement; we require providers to submit speed and latency test results, including the technologies used to provide broadband at the tested locations, for each state and speed tier combination in addition to an annual certification in a format to be determined by WCB. See supra Section V.B.
  High-latency bidders conducting mean opinion score (MOS) testing across their entire networks, rather than state-by-state, may submit and certify MOS test results on a nationwide basis.  To minimize the burden on providers, USAC will calculate the compliance percentages required using the data submitted.  By requiring carriers to submit test results annually and having USAC perform the compliance calculations, we minimize the potential for any manipulation or gaming of the testing regime, as providers will be required to certify to a set of specific results rather than to a general level of compliance.  However, providers that are not fully compliant with the speed and latency standards must submit quarterly reports including one week of test results and describing steps taken to resolve the compliance gap.  
F. Steps Taken to Minimize the Significant Economic Impact on Small Entities,
and Significant Alternatives Considered
27. The RFA requires an agency to describe any significant alternatives that it has considered in reaching its proposed approach, which may include (among others) the following four alternatives:  (1) the establishment of differing compliance or reporting requirements or timetables that take into account the resources available to small entities; (2) the clarification, consolidation, or simplification of compliance or reporting requirements under the rule for small entities; (3) the use of performance, rather than design, standards; and (4) an exemption from coverage of the rule, or any part thereof, for small entities. 5 U.S.C. § 603(c).
  We have considered all of these factors subsequent to receiving substantive comments from the public and potentially affected entities.  The Wireline Competition Bureau, Wireless Telecommunications Bureau, and Office of Engineering and Technology have considered the economic impact on small entities, as identified in any comments filed in response to USF/ICC Transformation FNPRM and IRFA, in reaching its final conclusions and taking action in this proceeding. 
28. In the Order, we adopt a clear, uniform framework for high-cost support recipients serving fixed locations to test speed and latency to meet the obligations associated with the support they receive.  The requirements we adopt provide flexibility for carriers to choose between different testing methods suitable for carriers of different sizes and technological sophistication. See supra Section III.
  Instead of requiring providers to invest in and implement new internal systems, we permit providers to perform speed and latency tests with readily available off-the-shelf solutions or existing MBA infrastructure.  We expect that carriers with testing features built into customer premises equipment for their own network management purposes may prefer using their own self-testing systems, which we also permit.
29. We require that carriers, regardless of their preferred testing methods, conduct tests using the same parameters we establish. See supra Section IV.
  These parameters take into account smaller carriers’ circumstances to avoid disproportionately burdening them.  For example, we expand the list of locations to which carriers may conduct required tests—allowing smaller carriers that are farther from the largest metropolitan areas to test speed and latency over shorter distances.  We also permit providers to conduct tests to the designated area of their choosing, rather than to the nearest designated metropolitan area. See supra Section IV.A.
  Further, carriers with fewer subscribers in a state and broadband service tier may test fewer locations.  Greater percentages of subscribers are necessary to achieve the same margin of error and confidence level in smaller sample sizes, but we recognize that, below 450 subscribers, that necessary percentage rises quickly above 10 percent.  Accordingly, in the Order, we allow providers with between 51 and 450 subscribers in a particular state and service tier combination to test 10 percent of total subscribers.  We require providers with fewer than 50 subscribers in a particular state and service tier combination to test five locations, but, to the extent necessary, those carriers may test existing, non-CAF-supported active subscriber locations to satisfy that requirement. See supra Section IV.C.

30. Finally, we provide clarity regarding the Commission’s existing requirement that carriers must report the results of network performance tests. See 47 CFR § 54.313(a)(6).
  Carriers must annually (or, in some cases, quarterly) submit detailed results of the required tests, conducted pursuant to the parameters we establish.  We hold all carriers to the same speed and latency test standards, See supra Section V.A.
 but we recognize that requiring carriers to take the additional step of using their test results to determine their level of compliance may entail unnecessary burdens.  Although we anticipate that carriers will find the adopted compliance framework straightforward, we conclude that requiring submission of the actual test results and allowing USAC to calculate the compliance percentages lessens the burden on small entities even further. See supra Section V.B.
  
Report to Congress:
The Commission will send a copy of the Order, including this FRFA, in a report to be sent to Congress and the Government Accountability Office pursuant to the Small Business Regulatory Enforcement Fairness Act of 1996. 5 U.S.C. § 801(a)(1)(A).
  In addition, the Commission will send a copy of the Order, including the FRFA, to the Chief Counsel for Advocacy of the Small Business Administration.  A copy of the Order and FRFA (or summaries thereof) will also be published in the Federal Register. See id. § 604(b).

38