Federal Communications Commission	DA 22-408


DA 22-408
Released:  April 14, 2022

BROADBAND DATA TASK FORCE AND OFFICE OF ENGINEERING AND TECHNOLOGY ANNOUNCE PROCEDURES FOR THIRD-PARTY MOBILE SPEED TEST APPLICATIONS SEEKING APPROVAL FOR USE IN THE FCC’S BROADBAND DATA COLLECTION 
WC Docket No. 19-195
ET Docket No. 22-152

I. INTRODUCTION
1. By this Public Notice, the Broadband Data Task Force and the Office of Engineering and Technology (OET) of the Federal Communications Commission (FCC or Commission) adopt and announce procedures for third-party mobile speed test application developers (third-party app developers) to submit, for OET review and approval, proposals for mobile speed test applications for use in collecting and submitting mobile network performance data as part of the Commission’s Broadband Data Collection (BDC).  This Public Notice provides information on third-party speed test app requirements and describes the process that OET will use to seek public comment on and to review proposals for third-party apps.  With the release of this Public Notice, third-party app developers may begin submitting their proposals to OET for review and approval, and OET will endeavor to complete its review of proposals received on or before June 9, 2022 in advance of the FCC’s publication of the initial versions of the broadband availability maps required under the Broadband DATA Act. Applications received after June 9, 2022 will be reviewed on a rolling basis.
  In OET Bulletin 75, See OET Bulletin No. 75, Broadband Data Collection Program: Third-Party Speed Test Mobile Application Approval Process Guidance, FCC Office of Engineering and Technology, FCC/OET-75 (Apr. 14, 2022), available at OET - Bulletins On-line, https://www.fcc.gov/general/oet-bulletins-line.
 released contemporaneously with this Public Notice and Appendix A (Template for Third-Party App Proposals) to the Public Notice (Template), OET provides technical guidance regarding how third-party app developers may present supporting information and other documentation as part of their proposals. Establishing the Digital Opportunity Data Collection, WC Docket No. 19-195, Order, DA 22-241, 2022 WL 743185 (Mobile Technical Requirements Order) at *48, para. 122 (2022). (“OET will release a public notice announcing the process for approving third-party apps for use in the mobile challenge process, inviting third-party app proposals, and seeking comment on third-party apps being evaluated. As previously mentioned, OET will announce and publish a webpage to maintain a list of approved third-party apps and any available data specifications for third-party apps.”)
 
II. BACKGROUND
2. As part of the Broadband Deployment Accuracy and Technological Availability Act (Broadband DATA Act), Broadband Deployment Accuracy and Technological Availability Act, Pub. L. No. 116-130, 134 Stat. 228 (2020) (codified at 47 U.S.C. §§ 641-646) (Broadband DATA Act).
 Congress mandated that the Commission “establish a user-friendly challenge process through which consumers, State, local, and Tribal governmental entities, and other entities or individuals may submit coverage data to the Commission to challenge [the coverage maps created pursuant to the Broadband DATA Act or the underlying broadband availability data forming the basis of the coverage maps].” 47 U.S.C. § 642(b)(5)(A).
  The legislation also requires that the Commission develop a crowdsourcing process “through which entities or individuals . . . may submit specific information about the deployment and availability of broadband internet access service . . . on an ongoing basis so that the information may be used to verify and supplement information provided by providers . . . for inclusion in the [broadband coverage] maps.” 47 U.S.C. § 644(b)(1).
  These processes are intended to allow consumers and other entities opportunities to submit data that will help to validate and improve the accuracy of broadband availability data supplied by broadband service providers.   
3. In the Second Report and Order adopted in this proceeding, the Commission directed OET, the Office of Economics and Analytics (OEA), the Wireline Competition Bureau (WCB), and the Wireless Telecommunications Bureau (WTB) “to develop and refine a process for entities and individuals to submit third-party fixed and mobile crowdsourced data consistent with the Broadband DATA Act’s requirements and the Commission’s policies.” Establishing the Digital Opportunity Data Collection; Modernizing the FCC Form 477 Data Program, WC Docket Nos. 11-10 & 19-195, Second Report and Order and Third Further Notice of Proposed Rulemaking, 35 FCC Rcd 7460, 7488, para. 66 (2020) (Second Report and Order). 
  Crowdsourced data will be used to “identify individual instances or patterns of potentially inaccurate or incomplete deployment or availability data that warrant further investigation or review.” Id. at 7490, para. 72.
  
4. In the Third Report and Order, the Commission required consumers to submit speed test data in support of challenges to mobile broadband coverage data. Establishing the Digital Opportunity Data Collection; Modernizing the FCC Form 477 Data Program, WC Docket Nos. 11-10 & 19-195, Third Report and Order, 36 FCC Rcd 1126, 1166, para. 102 (2021) (Third Report and Order). 
  In order “[t]o ensure that consumer challenge data meet necessary reporting requirements, [the Commission] require[s] consumers to use a speed test application that has been designated by OET, in consultation with OEA and WTB, for use in the [mobile] challenge process.” Id. at 1166, para. 103.  Governmental agencies and other entities are also required to submit speed-test data to support their mobile broadband availability challenges, and may use (but are not required to use) a Commission-approved speed test application to collect and submit such data.  Id. at 1171-72, paras. 116-17.
  The Third Report and Order included minimum requirements for all speed test applications and directed OET to develop data elements that mobile speed tests must include to qualify as a valid mobile challenge process submission and procedures to approve qualified third-party apps. Id. at 1166-67, paras. 103-04.
  Third-party apps may also include, as an add-on to the challenge process system features, capabilities for consumers to submit crowdsourced mobile coverage data to the Commission. Mobile Technical Requirements Order at *48, para. 122.
  
5. Most recently, in the Mobile Technical Requirements Order, OEA, WTB, and OET outlined the process by which OET will receive proposals for designation of third-party apps approved to submit mobile challenge and, if implemented, crowdsourced data, into the BDC. Id. at *10, para. 25 (“At a later date, OET will release a public notice outlining the process for collecting, reviewing, and approving applications for third-party speed test apps.  In their applications, app developers will be required to describe their performance-centric speed test methodologies and how their app complies with the data collection requirements set forth in this Order.  The OET public notice also will describe procedures for interested parties to submit comments and replies in response to the proposals and will publish on the Commission’s website a list of approved third-party apps and any available data specifications for third-party apps.” (footnote omitted)); see also id. at *48, para. 122. 
  This framework will provide consumers, government entities (including Tribal governments), and other third parties with the flexibility to choose among and between the publicly available FCC Speed Test app See FCC, Measuring Mobile Broadband, https://www.fcc.gov/general/measuring-mobile-broadband-performance.
 and OET-approved third-party apps for challenging mobile broadband coverage and, optionally, for filing crowdsourced data.  Any speed test data submitted from approved third-party apps must include the specified metrics and parameters for the challenge and, if implemented, crowdsource processes. See Mobile Technical Requirements Order at *7, para. 18 & n.73.
  The procedures for OET approval of third-party apps for both the consumer challenge process and receiving crowdsourced data are the same. Id. at *48, para. 122.
    
III. DISCUSSION
6. Third-party app developers seeking OET approval of their apps for the collection and submission of mobile challenge data (and, if implemented, crowdsourced data) must submit a separate proposal for iOS and Android operating systems using the procedures described below.  Third-party app developers may now begin submitting their proposals, and OET will endeavor to perform an initial review of, seek comment upon, and make final determinations for proposals submitted on or before June 9, 2022 by the end of August, contemporaneously with the close of the inaugural filing window for the initial versions of the broadband availability maps required under the Broadband DATA Act.  This will allow third-party apps, in addition to the FCC Speed Test app, Id. at *6, para. 14 (“The Commission directed OET, in consultation with OEA and WTB, to update the FCC Speed Test app as necessary or develop a new speed test app to collect the designated metrics, so that [mobile] challengers may use it in the [mobile] challenge process.”); see also id. at *9, para. 22 (“OET recently released a technical description of the metrics and methodologies used in the current version of the FCC Speed Test app.  The revised technical description document includes updated technical standards and additional modifications.  While this document does not illustrate future user experience design changes to the FCC Speed Test app that will be made to implement the challenge and crowdsource functionalities, we anticipate that the fundamental measurement methodologies reflected in the recently updated technical description document will not be affected by these design updates.” (footnote omitted)).
 to be made available to consumers and other entities to begin submitting mobile challenge and crowdsource data into the BDC system once the mobile broadband availability maps are published.    
A. Third-Party Application Proposals 
7. As specified in the Mobile Technical Requirements Order, third-party app developers seeking approval for use in the challenge and, if implemented, crowdsourcing processes must include information indicating how the third-party app complies with the requirements for the collection of mobile challenge data (and crowdsourced data, if applicable). Id. at *10, para. 25 (“At a later date, OET will release a public notice outlining the process for collecting, reviewing, and approving applications for third-party speed test apps.  In their applications, app developers will be required to describe their performance-centric speed test methodologies and how their app complies with the data collection requirements set forth in this Order.”).
  OET has outlined the various requirements in OET Bulletin 75, released contemporaneously with this Public Notice, and the Template in Appendix A attached hereto.  The third-party app developer must describe how the proposed third-party app meets all requirements (i.e., those described as “must” or “shall” in Bulletin 75 and specified in this Public Notice).  For any third-party app proposal that deviates from these requirements or does not implement recommended practices (i.e., those described as “may” or “should” herein), the third-party app developer must (1) explain its reasoning why its particular circumstances warranted a deviation from the requirements or recommended practices, and (2) show that such a deviation will serve the public interest.  All third-party app proposals (including supporting documentation and any supplements thereto) must be filed with the Commission in the Commission’s Electronic Comment Filing System (ECFS) and adhere to the format specified in the Template.  The Template includes prompts for responses addressing the issues and information described in OET Bulletin 75, which consolidates the various Commission requirements into one resource, and certifications necessary to ensure that a third-party app will operate as required under this Public Notice, the Commission’s prior orders, Third Report and Order, 36 FCC Rcd 1126; Second Order and Third Further Notice, 35 FCC Rcd 7460. 
 the Mobile Technical Requirements Order, See Mobile Technical Requirements Order.
 and the BDC - Data Specifications for Mobile Speed Test Data. Broadband Data Task Force and Office of Economics and Analytics Publish Additional Data Specifications for the Submission of Mobile Speed Test and Infrastructure Data into the Broadband Data Collection, Public Notice, DA 22-242, at 1 (OEA Mar. 9, 2022); FCC, Broadband Data Collection Data Specifications for Mobile Speed Test Data (2022), https://us-fcc.app.box.com/v/bdc-mobile-speedtest-spec (BDC - Data Specifications for Mobile Speed Test Data).  In the Mobile Technical Requirements Order, OEA, WTB, and OET stated that they “require on-the-ground challenge test data to include all other metrics required per the most recent specification for mobile test data.”  Mobile Technical Requirements Order at *7, para. 18 (“Finally, we require on-the-ground challenge test data to include all other metrics required per the most recent specification for mobile test data . . . .”); id. at *7, para. 18 & n.73 (stating that “[t]he specification for speed test data includes additional fields derived from the high-level metrics defined herein, as well as other identifiers to facilitate management of the submission of such data”).
  Third-party app developers may request confidential treatment of information contained in their proposals consistent with section 0.459 of the Commission’s rules and the instructions the FCC has provided for submission of confidential materials during the COVID-19 pandemic. See 47 CFR § 0.459.  Pursuant to instructions regarding the submission of confidential materials during the COVID-19 pandemic, those filers wishing to request confidential treatment are directed to contact FCC OET staff for instructions on how to upload unredacted confidential submissions via Box or may e-mail an electronic version of the document or materials (provided the documents are password protected) to FCC staff and provide the password to FCC staff via telephone or in a separate e-mail.  A redacted version of the confidential submission should be filed through ECFS.  FCC Provides Further Instructions Regarding Submission of Confidential Materials. Public Notice, DA 20-361, 35 FCC Rcd 2973 (2020).

8. As specified in the Mobile Technical Requirements Order, a third-party app developer submitting a proposal is “required to describe [its] performance-centric speed test methodologies and how [its] app complies with the data collection requirements” for the mobile challenge and crowdsourced data collections. Mobile Technical Requirements Order at *10, para. 25.
  These requirements are set forth in the Mobile Technical Requirements Order, Id. at *10, para. 25 (stating that “app developers will be required to describe . . . how their app complies with the data collection requirements set forth in this Order”); see also 47 CFR § 1.7006.
 the Commission’s prior orders, Second Report and Order; Third Report and Order. 
 and the BDC - Data Specifications for Mobile Speed Test Data. See BDC - Data Specifications for Mobile Speed Test Data, https://us-fcc.app.box.com/v/bdc-mobile-speedtest-spec.
  
9. In the Mobile Technical Requirements Order, WTB, OET, and OEA stated that they will require that third-party apps meet the same or similar technical standards as the FCC Speed Test app. Mobile Technical Requirements Order at *10, para. 26; id. at *48, para. 122.
  As part of these technical standards, WTB, OET, and OEA concluded that all speed test apps must “use multiple servers that are geographically diverse.” Id. at *10, para. 26.
  Although WTB, OET, and OEA did not dictate how a speed test app must meet these requirements, they described the FCC Speed Test app’s system architecture in detail and found that it provides “robust and reproducible test results for effective representation of network performance” Id. at *47, para. 120 (“The FCC Speed Test app’s test system architecture implements dedicated off-net servers hosted by a Content Delivery Network (CDN) to provide robust and reproducible test results for effective representation of network performance. The test servers are deployed at Tier 1 major peering/transit locations to minimize bias which is a practical approach to measure network performance.”).
 and that it “provides sufficient diversity to meet this geographic-diversity criterion.” Id. at *10, para. 26. (“As described in the most recent technical description for the FCC Speed Test app, the app currently carries out measurements against 13 servers spread out across ten locations throughout the United States and initiates a test sequence by selecting a measurement server using a latency test to identify the optimal server that has the lowest round-trip latency for performing subsequent tests. We believe that the current distribution of FCC Speed Test app servers, combined with this measurement server selection process, provides sufficient diversity to meet this geographic-diversity criterion.”).
  As noted in that Order, the FCC Speed Test app system architecture is designed to prevent overdemand in server capacity, including a target utilization rate which triggers the provisioning of additional servers to stabilize load across the platform. Id. at *10, para. 26.
  The FCC Speed Test app also includes multiple redundant and meshed servers to allow for traffic failover and appropriate selection of an optimal measurement server during the test sequence initialization. Id.
  Given the importance of these issues to potential commenters, we will require, as part of their proposal, that third-party app developers demonstrate their compliance with these technical standards by submitting a complete description of the proposed system architecture and describing their plans for initial measurement server capacity, monitoring capability of the health and quality of the measurement infrastructure and the collected data, and capacity upgrade thresholds. 
10. Similarly, we expect that, for any third-party apps designed to serve a limited geographic area (e.g., state, city, or community), the third-party app developer’s data repository system would compile and filter the collected data to the designated area’s boundary prior to transmission to the BDC system.  The collected data should be transmitted, stored, and maintained to a third-party app developer’s data repository system located within the United States after the completion of active test measurements.  As noted in response to the Mobile Technical Requirements Order, Id. at *10, para. 26.  (“As adopted and implemented in the FCC Speed Test app, the variability of latency is not entirely a function of geographical distance to the test server but also is a function of the network congestion, and so, at a minimum, servers should be distributed nationally in consideration of user base, population density, and the server utilization rate for multiple servers to be examined before the test server selection and located reasonably close to Internet eXchange Points (IXPs) to accurately reflect unbiased real-world conditions.”).
 the geographic limitations of a particular app may potentially affect testing results. Id. at *9, n. 93, (citing FCC Office of Engineering and Technology, 2021 FCC Speed Test App Technical Description at 6 (2021), https://www.fcc.gov/sites/default/files/2021_fcc_speed_test_app_technical_description.pdf) (“We note that the description includes the following about the test system architecture: ‘The measurement servers, each supporting a 100 Gbps capacity, used for mobile broadband measurement are hosted by StackPath and are distributed nationally to enable a measurement client to select the host server with the least latency.’”).
  For this reason, to the extent that a third-party app is not designed to serve the entire United States, we also expect that the third-party app would inform end users, within the app, of any designated geographic limitations on the app’s testing capabilities.  A third-party app developer that seeks approval of an app designed to serve a limited geographic area that does not meet these expectations should explain why its app nevertheless meets the same technical standards as the FCC Speed Test app. 
11. The Third Report and Order also requires that each third-party app developer provide “appropriate” privacy documentation about how consumer data will be “stored, used, and protected.” Third Report and Order, 36 FCC Rcd at 1167, para. 104.  
  To satisfy this obligation, a third-party app developer must have both a brief privacy notice for both challenge and, if implemented, crowdsource processes that is displayed to end users at the point of collection, and a comprehensive privacy policy, which must document the developer’s practices with respect to storage, use, and protection of challenge data. An app developer may choose to include a link or reference to its privacy policy in the notice at the point of collection.
  This requirement is consistent with the Commission’s broader determination that, except for “information about the location that is the subject of the challenge . . . , the name of the [challenged] provider, and any relevant details concerning the basis for the challenge . . . all other challenge information, such as individual contact information,” should be “private” to protect the personal privacy interests of those who participate in the challenge process. Third Report and Order, 36 FCC Rcd at 1175, para. 125.
  To ensure that an approved third-party app protects those interests, the Commission delegated to OET authority to approve speed test applications that achieve certain minimum requirements, which include documenting appropriate protections for how data will be stored, used, and protected. Id.
  
12. At a minimum, the developer’s privacy notice shall disclose that the Commission will make public the coordinates (latitude and longitude) of the location of the challenge, the name of the provider, and relevant details concerning the basis for the challenge; but individual contact information will not be disclosed. Id. at 1174, para. 125.
  The third-party app must additionally include all necessary opt-in provisions, including a prompt for end users to grant location permissions for the third-party app to assist in reporting geographic coordinates for the start and end of each test metric See Mobile Technical Requirements Order at *7, para. 18 (“geographic coordinates [] measured at the start and end of each test metric”).
 and a prompt for end users to grant their consent to allow the end user’s service provider to divulge customer-specific information to the FCC (such as IP address and service plan details) as necessary to respond to challenges. Id. at *7, para. 18 (listing timestamp and source IP address and port of the device, as measured by the measurement server, as another standardized metric that must be included with all on-the-ground mobile speed test data); id. para. 18, n.72 (noting that “[g]iven concerns that challengers may conduct tests after exceeding data limits, we will collect the timestamp that test measurement data were transmitted to the app developer’s servers, as well as the source IP address and port of the device, as measured by the server, so that a service provider may determine if a challenger’s device is subject to reduced speeds or otherwise lacks full network performance.”).

13. Likewise, an “appropriate” privacy policy must be easily accessible to prospective challengers.  Such a policy must be clear and concise, and it must be written in plain language.  Privacy policies that are overly technical, vague, opaque, or lengthy—or that require specialized knowledge—do not meaningfully apprise prospective challengers of how their challenge data will be stored, used, and protected, and thus cannot be characterized as “appropriate.”
14. To demonstrate how data will be “stored” and “protected,” a third-party app developer will be required to include in its privacy policy, at a minimum, a high-level description of what safeguards and controls it has adopted regarding the possession, maintenance, and transmission of challenge data, including for example, access restrictions, physical or network security, authentication methods, and other measures that will be employed to protect individualized challenge data. See, e.g., NIST, U.S. Department of Commerce, Security and Privacy Controls for Information Systems and Organizations, NIST Special Publication 800-53 rev. 5 (2020), https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-53r5.pdf (NIST 800-53 rev. 5); NIST, U.S. Department of Commerce, Control Baselines for Information Systems and Organizations; NIST, U.S. Department of Commerce, Vetting the Security of Mobile Applications, NIST Special Publication 800-163 rev. 1 (2019).  
  This description must specifically address whether the developer will maintain all test data containing Personal Identifiable Information (PII) within the United States, or, if not, what alternative measures the developer is taking to ensure the security of the data in a foreign jurisdiction. See NIST 800-53 rev. 5 at 273, SA9(8) (“Restricting the processing and storage of high-impact information to facilities within the legal jurisdictional boundary of the United States provides greater control over such processing and storage.”).
  Similarly, to appropriately document how data will be “used,” a privacy policy must address more than a mere recitation of how the challenge data will support functionality of the application or facilitate providers’ responses to challenges.  For example, an appropriate privacy policy must explain how, if at all, the developer plans to monetize individualized challenge data (e.g., through the delivery of advertisements or the promotion of the developer’s other products and services).  
15. The third-party app developer shall indicate in their proposal if the test result data transmitted to the BDC system from its servers through an application programming interface (API) in a JavaScript Object Notation (JSON) format matches the structure in accordance with the BDC - Data Specifications for Mobile Speed Test Data, and must provide JSON sample(s) of the collected data.  The developer may also supplement its proposal with speed test data showing a statistical assessment of the proposed third-party app’s overall performance.  In reviewing a third-party app proposal, OET will rely on a flexible approach to analyze the supplemental speed test data.  Although each analysis will turn on the relevant facts and circumstances, we outline one possible example of an approach to analyzing the speed test data in Appendix A of OET Bulletin 75, in an effort to provide guidance to third-party app developers.  A third-party app developer may alternatively submit a description of its own preferred test methodology along with testing data, and if applicable, the third-party app developer will make the supplemental speed test data and test methodologies available to interested parties for comment.
B. Third-Party Application Approval Process 
16. Following receipt of a proposal in the Commission’s ECFS in ET Docket No. 22-152, staff will conduct an initial review to ensure that the third-party app developer has provided sufficient information and details related to the various requirements for approval, and then issue a public notice specifying a new docket created for the particular proposal and announcing the comment and reply comment dates for that proposal.  Interested parties will be able to review and file comments and reply comments.  Comments will be due no later than 30 days from the date of the public notice and reply comments will be due 45 days from the date of the public notice.  Comments may be filed by using ECFS or by filing paper copies. See Electronic Filing of Documents in Rulemaking Proceedings, GC Docket No. 97-113, Report and Order, 13 FCC Rcd 11322 (1998); 63 FR 24121 (1998).
  
17. We require third-party app developers, concurrent with the date their proposal is submitted to the Commission, to either (1) make their third-party apps available for download, use, and testing on all applicable app stores, e.g., Google Play and/or Apple App Store (or if beta versions, Google Play Console, Apple TestFlight, or similar media), and indicate in their proposal how to find/download the app; or (2) if the app is not available for download, to provide detailed “wireframes” depicting each screen of the planned end user interface for the third-party app, along with actual test data results and a detailed speed test measurement methodology description, for OET staff and other interested parties to confirm that the underlying testing methodology meets the minimum requirements established by the Commission.  To the extent a third-party app developer submits only wireframes or a beta version for review, any approval of such proposal by OET will be conditional on the production version of the third-party app’s compliance with applicable technical requirements.  All conditionally approved proposals are required to submit, prior to BDC system registration, the production version of the app for OET’s full review and approval.  Assuming the production version of the app meets all stated requirements, it will be designated by OET for use in the mobile challenge and, if implemented, crowdsource processes.  Additionally, any deviations in the production version from the conditionally approved wireframes or beta version will be subject to either the minor or major change procedures set forth below and described in OET Bulletin 75.
18. Any modification to an app that affects its testing methodology or compliance with technical and data requirements, including a change to the upload, download, or latency testing protocols, or to the measurement procedures, will be considered a “major” change.  A third-party app developer that makes a major change in its app must submit a new proposal that follows the Template in Appendix A, and must file the new proposal in ECFS using the Docket Number associated with the initial proposal.  
19. A minor change is one that does not modify the third-party app’s testing methodology or technical and data requirements.  Minor changes may include updates to the end user’s interface, adding support for new operating system versions and hardware, security updates, or withdrawal.  A third-party app developer does not need approval from OET to make a minor change, but, when it intends to make such a change, it must notify OET of the proposed change prior to updating the third-party app in an app store.  Notice of a minor change must be filed in ECFS using the Docket Number associated with the initial proposal.  When a third-party app developer submits such notification, it must include a description of the change, identify the new third-party app version, and indicate the date when older third-party app versions will no longer be used for BDC challenge data submissions (and crowdsourced submissions, if applicable).  OET may choose whether to use challenge and crowdsourced data submitted after the minor change, and OET reserves discretion to require the third-party app developer to file a minor change as a major change. 
20. Comments on third-party app proposals should address technical deficiencies in the submission(s), including whether any required information is missing from the proposal (notwithstanding OET’s initial review); whether the proposed test system architecture would prevent the Commission, mobile broadband service providers, or other interested parties from reliably receiving the data needed to adjudicate a challenge or evaluate mobile crowdsourced submissions; whether the proposal fails to satisfy applicable privacy, security, or other policy requirements; or whether the proposal has any other shortcomings of the requirements as described in OET Bulletin 75.  Commenters may also provide information or comments on specific testing methodologies underlying the third-party app, which OET may consider as part of its evaluation and review of the third-party app proposal.   
21. OET will continue to review the qualifications of the third-party app throughout and after the review and approval process.  This review may include, but is not limited to, requiring a third-party app developer that submitted a proposal to participate in workshops and/or meetings.  A third-party app developer must comply with all instructions from OET and provide any requested information in a timely manner as a condition of receiving and maintaining approval. 
22. OET will announce any approvals of third-party apps via public notice, and will publish and maintain a webpage reflecting a list of approved third-party apps as well as any available data specifications for third-party apps. See Mobile Technical Requirements Order at *10, *48, paras. 25, 122.
  Proposals will not be considered mutually exclusive, and OET will approve as many proposals as are determined to satisfy all BDC system requirements. Id. at *7, para. 18 & n.73; BDC - Data Specifications for Mobile Speed Test Data. OET may issue a single public notice, or issue multiple public notices across multiple tranches.
  
23. Approval of third-party apps shall be for a term of five years.  Third-party app developers may request, between six months and three months prior to the end of the initial term, renewal of their app approval for an additional term of up to five years.  In the Mobile Technical Requirements Public Notice, OEA, WTB, and OET sought comment on periodically evaluating approved third-party apps, proposing that OET “would provide periodic review and offer guidance for designated third party apps to ensure continued compliance with all technical and program requirements.” Comment Sought on Technical Requirements for the Mobile Challenge, Verification, and Crowdsource Processes Required under the Broadband DATA Act, WC Docket No. 19-195, Public Notice, DA 21-853, at 24, para. 54 (WTB/OEA/OET July 16, 2021) (BDC Mobile Technical Requirements Public Notice).
  While no parties commented on specific initial or renewal approval terms in response to the Mobile Technical Requirements Public Notice, we find that adopting such terms is a reasonable means of implementing the proposal to provide periodic review of approved apps.  Moreover, commenters did express support for adopting robust requirements for speed test apps that will help ensure that the apps produce accurate data. See e.g., CTIA Comments at 15-17; AT&T Comments at 10.
  We find that, by creating a process that will allow Commission staff to conduct periodic review of approved apps, adopting initial and renewal approval terms will help ensure that approved third-party apps continue to produce accurate data and demonstrate continued reliability.  
24. Relatedly, in the Second Report and Order, the Commission directed OET, OEA, WCB, and WTB to modify the process for the collection of mobile crowdsourced data over time, to the extent necessary, recognizing that there may be changes in technology, availability of different types of crowdsourced data, or other considerations that may require reevaluation and possible modifications of these initial determinations. See Second Report and Order, 35 FCC Rcd at 7488, para. 67.
  As a condition of approving third-party apps, to the extent that apps have implemented crowdsource capabilities, we will require those apps to continue to follow and implement modifications that satisfy the Act’s provisions for submitting crowdsourced data on an ongoing basis, which will provide greater flexibility to adjust and improve our data collection process over time.
25. Any third-party app developer that wishes to discontinue their approved third-party app’s participation in the BDC program must notify the Commission at least 60 days in advance of such discontinuance.  Additionally, the third-party app provider must provide notice to all affected end users at least 30 days before such discontinuance to minimize disruptions in data collection.  In the case that either a third-party app developer desires discontinuance of their app or approval is not renewed, such discontinuance will be noted on the Commission’s webpage.  The Commission will then de-register the third-party app from the BDC system, and any data collected by the third-party app subsequent to the discontinuance or termination effective date may no longer be considered as mobile challenge or crowdsource data in the Commission’s publicly available reported data. See BDC Mobile Technical Requirements Public Notice at *20, para. 54.
  
26. Processes by designated third-party app developers to: (1) change or upgrade testing methodology; (2) change or upgrade software or user interface unrelated to the testing methodology; (3) renew the five-year approval period; or (4) withdraw designation as a third-party app developer, must be filed with the Commission and will be processed by OET on a case-by-case basis; any changes necessitating additional opportunity for comment, such as changes or upgrades to the testing methodology used by a third-party app, may delay the assessment of such app.  
27. People with Disabilities.  To request materials in accessible formats for people with disabilities (braille, large print, electronic files, audio format), send an e-mail to fcc504@fcc.gov or call the Consumer and Governmental Affairs Bureau at 202-418-0530 (voice), 202-418-0432 (tty).
28. Congressional Review Act.  The Commission has determined, and the Administrator of the Office of Information and Regulatory Affairs, Office of Management and Budget, concurs, that this rule is “non-major” under the Congressional Review Act, 5 U.S.C. § 804(2).  The Commission will send a copy of this Public Notice to Congress and the Government Accountability Office pursuant to 5 U.S.C § 801(a)(1)(A).
29. Paperwork Reduction Act.  The requirements adopted in this Public Notice are statutorily exempted from the requirements of the Paperwork Reduction Act of 1995 (PRA), Public Law 104-13. See Infrastructure Investment and Jobs Act § 60102(h)(2)(E)(ii) (amending 47 U.S.C. § 646(b)).  
  As a result, the Public Notice will not be submitted to OMB for review under section 3507(d) of the PRA.
30. Further Information.  Questions regarding this Public Notice may be directed to Sean Yun, Deputy Division Chief, Office of Engineering and Technology, at Sean.Yun@fcc.gov or (202) 418-0980.
–FCC–


2
	Federal Communications Commission	DA 22-408

APPENDIX A

Template for Third-Party App Proposals

In order for a third-party app developer’s mobile app to be approved for use in the FCC’s Broadband Data Collection (BDC) mobile challenge and crowdsourcing processes, the proposal must follow the template outlined below.  The third-party app developer must, at a minimum, provide the information requested in this template, and in the format of the template.  Responses to this template as well as any additional information provided as part of the proposal demonstrating compliance with the requirements adopted in the BDC - Data Specifications for Mobile Speed Test Data will be reviewed to determine whether the app will be approved for use as part of the BDC.  Additional guidance regarding the template and requirements can be found in OET Bulletin 75, Broadband Data Collection Program: Third-Party Speed Test Mobile Application Approval Process Guidance.

Section A. Filer Information
Requirement ID
Description
A.1
Entity Legal Name (Filer Information, p. 8) Parenthetical cites throughout this template cross-reference to the relevant section in the OET Bulletin 75 with the specific page number included for ease of reference.

A.2
Business Name (if applicable) (Filer Information, p. 8)
A.3
Mailing Address (Street Address, City, State, Zip Code) (Filer Information, p. 8)
A.4
Third-Party App Name (Filer Information, p. 8)
A.5
Third-Party App Version Number (Filer Information, p. 8)

Section B. Applicant Contact Information
Requirement ID
Description
B.1
Contact First Name (Contact Information, p. 8)
B.2
Contact Last Name (Contact Information, p. 8)
B.3
Contact Phone Number (Contact Information, p. 8)
B.4
Contact Email Address (Contact Information, p. 8)
B.5
Contact Mailing Address (Street Address, City, State, Zip Code) (Contact Information, p. 8)

Section C. System Architecture
Requirement ID
Description
C.1
Provide a complete description of the proposed test system architecture the app will use. (Architecture Description, p. 8)
C.2
Insert a technical diagram of the proposed system architecture used by the app. (Architecture Description, p. 8)
C.3
Describe the testing methodologies, including test flows and calculations, the app will use to collect speed test data. (Architecture Description, p. 8)
C.4
Indicate whether this proposal is for an app designed for the Android or the iOS mobile operating system (OS).  Each OS design requires its own separate proposal. (Architecture Description, p. 8)
C.5
Indicate whether the app is available for download. (Architecture Description, p. 8)
C.5.1
If the app is available for download, indicate whether the available version is a production or beta version of the app, and provide access information for FCC staff and the public to download and review the app. Optionally, speed test data and a description of the test configuration used to capture the speed test measurements may be included. (Architecture Description, p. 8)
C.5.2
If a working version of the app is not available for download, provide “wireframes” that must depict each screen (user interface) of the third-party app, and a detailed speed measurement methodology description, along with actual test data results demonstrating compliance with applicable requirements. (Architecture Description, p. 9)
C.6
Indicate whether the app will collect data for the mobile challenge process only, or for both the mobile challenge process and crowdsourcing process. (Architecture Description, p. 9)

Section D. Privacy Notice and End User Consent Requirements
Requirement ID
Description
D.1
Confirm that the app includes, for both challenge and, if implemented, crowdsource processes, a brief a privacy notice at the point of collection that explains how all of the data collected by the app, including personally identifiable information (PII), will be stored, used, and protected consistent with the minimum requirements set forth in applicable privacy and security policies. (Privacy Notice and End User Consent Requirements, p. 9)
D.2
Confirm that the privacy notice discloses that the Commission will make public the coordinates (latitude and longitude), the name of the provider, and relevant details concerning the basis for the challenge, but that individual contact information will not be disclosed. (Privacy Notice and End User Consent Requirements, p. 9)
D.3
Describe how the app will obtain required privacy-related consents and include all necessary opt-in provisions. 
These opt-in provisions must include prompts for end users to grant location permissions (for the app to assist in reporting geographic coordinates for the start and end of each test metric) and for users to grant their consent to allow their service provider to report customer-specific information to the FCC (such as IP address and service plan details) as necessary to respond to challenges. (Privacy Notice and End User Consent Requirements, p. 9)

Section E. Certification, Testing Environment, and Display
Requirement ID
Description
E.1
If the app includes the crowdsourcing capability, describe how the third-party app requires the end user, prior to conducting a speed test, to indicate whether the test is intended for the challenge process or the crowdsourcing process. (Certification, Testing Environment, and Display, p. 10)
E.2
Confirm that the app requires the end user to certify, prior to conducting a speed test for the challenge process, that the end user is a subscriber or authorized user of the provider being challenged. (Certification, Testing Environment, and Display, p. 10)
E.3
Confirm that the app requires the end user to certify, prior to conducting a speed test for the challenge process, that the speed test measurements were taken outdoors. (Certification, Testing Environment, and Display, p. 10)
E.4
Confirm that the app requires the end user to certify, prior to conducting a speed test for the challenge process, that to the best of the person’s actual knowledge, information, and belief, the handset and the speed test application are in ordinary working order and all statements of fact contained in the submission are true and correct. (Certification, Testing Environment, and Display, p. 10)
E.5
Describe how the app is designed and configured to ensure that the end user can certify the challenge data submission requirements prior to initiating the network performance test sequence and, once the test sequence is initiated, that all test results are automatically submitted to avoid potential biases in the challenge process data. (Certification, Testing Environment, and Display, p. 10)
E.6
Confirm that the app requires the end user to certify, prior to conducting a speed test for the crowdsource process, that to the best of the filer’s actual knowledge, information, and belief, all statements in the filing are true and correct. (Certification, Testing Environment, and Display, p. 10)
E.7
Confirm that the app requires end users to indicate whether the test was taken in an in-vehicle mobile environment or an outdoor, pedestrian stationary environment. (Certification, Testing Environment, and Display, p. 11)
E.8
For in-vehicle tests, describe how the app will capture and report the speed the vehicle was traveling when the test was taken, where available, and the speed accuracy of measured velocity for each location measurement. (Certification, Testing Environment, and Display, p. 11)
E.9
Indicate whether the app displays all of the following information to an end user at the end of each test: download speed, upload speed, latency, date and time at which the test submission data were transmitted to the app’s servers, the network technology used (e.g., 3G, 4G, or 5G), the service provider that the end user used, and an indication of whether the test was taken on a home network or performed while roaming. (Certification, Testing Environment, and Display, p. 11)
E.10
Confirm if any metric listed in E.9 fails to be collected (e.g., connection with mobile network is not established), indicate whether the third-party app displays “-” instead of “0” (or other values). (Certification, Testing Environment, and Display, p. 11)

Section F. Test Servers
Requirement ID
Description
F.1
Describe in detail how the third-party app will collect data using a set of geographically distributed measurement servers that are statistically sound proportional to the user base and minimize the impact of latency for each test. (Test Servers, p. 11)
F.2
Describe how the third-party app will report the location of the test servers (e.g., hostname or IP address) to the Commission. (Test Servers, p. 12)
F.3
Describe the third-party app developer’s plan for establishing initial measurement server capacity. (Test Servers, p. 12)
F.4
Describe how the third-party app developer will monitor the health and quality of the measurement infrastructure and the collected data, including how the third-party app developer will maintain test servers so their operational capacity is not exceeded. (Test Servers, p. 12)
F.5
Describe how the third-party app developer will establish capacity-upgrade thresholds for the third-party app developer’s servers. (Test Servers, p. 12)
F.6
Describe how the third-party app developer will monitor the system for server failures. (Test Servers, p. 12)
F.7
Describe the third-party app developer’s failover strategy to maintain a high system availability rate for end users. (Test Servers, p. 12)
F.8
Describe the third-party app developer’s processes and procedures for performing tests against an optimal test server (including how optimal server selection will be determined at the start of each test sequence and a confirmation that the same server will be used for all tests in that sequence). (Test Servers, p. 12)
F.9
Indicate if the optimal measurement server(s) will be located outside the mobile service provider’s network (off-net server) to ensure unbiased measurements. (Test Servers, p. 12)
F.10
Describe the third-party app developer’s processes and procedures for determining whether each measurement server has sufficient network capacity. (Test Servers, p. 12)
F.11
Describe the third-party app developer’s processes and procedures for monitoring and maintaining the test servers so that the operational capacity is not exceeded. (Test Servers, p. 13)
F.12
The third-party app developer should describe its processes and procedures for ensuring these requirements are continuously met. (Test Servers, p. 13)

Section G. Mobile Broadband Performance
Requirement ID
Description
G.1
Indicate that each test sequence, consisting of three parts (download, upload, and latency), must be preceded by the measurement client selecting a measurement server. (Mobile Broadband Performance, p. 13)
G.2
Indicate if the third-party app developer proposes an alternative methodology, a sufficient justification must be provided in the proposal.  Third-party app developers may conduct upload and download speed tests based either on their own preferred testing procedure or on OET’s third-party app review procedure outlined in Appendix A for statistical assessment of the app’s performance. [If not, respond to G.3 - G.9 below.  Otherwise, skip to G.10 (Mobile Broadband Performance, p. 13)
G.3
The third-party app developer shall describe how the download speed metric are measured by establishing multi-TCP connections to a target test node.  Each test cycle shall begin with a warm-up period, followed by the download speed test.  Each download test shall conclude after an elapsed minimum time of 5 seconds and a maximum elapsed time of 30 seconds.  The client shall attempt to download as much of the binary non-zero content, herein referred to as the payload, as possible for the duration of the test. (Mobile Broadband Performance, p. 13)
G.4
The third-party app developer shall describe how each test is executed using at least three concurrent TCP connections for byte streams between the third-party app and the measurement server.  Hence, each connection used in the test must count the number of bytes of the target payload transferred between two points in time and shall calculate the speed of each thread as the number of bits transferred over the number of seconds in the active test window (i.e., excluding the warm-up time).  The speeds of the multiple threads shall be summed to determine the total download speed.  In addition, the third-party app shall report all other data required in the Download Test Object table of the BDC - Data Specifications for Mobile Speed Test Data. (Mobile Broadband Performance, p. 13)
G.5
The third-party app developer shall describe how factors such as TCP slow start are taken into account by letting a “warm-up” period elapse before the data transfer rate computation begins.  The concurrent individual connections shall be established, each on its own thread, and shall be confirmed as all having completed the warm-up period before the 5- to 30-second testing period begins. (Mobile Broadband Performance, p. 14)
G.6
The third-party app developer shall describe how the upload speed metric is measured by establishing multi-TCP connections to a target test node.  Each upload test cycle shall begin with a warm-up period, followed by the upload speed test.  Each upload test shall conclude after an elapsed minimum time of 5 seconds and a maximum elapsed time of 30 seconds.  The client shall generate and attempt to upload as much of the payload as possible for the duration of the test. (Mobile Broadband Performance, p. 14)
G.7
The third-party app developer shall describe how tests are executed using at least three concurrent TCP connections for byte streams.  Each connection used in the test must count the number of bytes of the target payload transferred between two points in time and shall calculate the speed of each thread as the number of bits transferred over the number of seconds in the active test window (i.e., excluding the warm-up time).  The speeds of the multiple threads shall be summed to determine the total upload speed.  In addition, the third-party app shall report all other data required in the Upload Test Object table of the BDC - Data Specifications for Mobile Speed Test Data. (Mobile Broadband Performance, p. 14)
G.8
The third-party app developer shall describe how factors such as TCP slow start are taken into account by transferring a portion of the target payload before the real testing begins.  The concurrent individual connections shall be established, each on its own thread, and shall be confirmed as all having completed the warm-up period before the 5 to 30-second testing period begins. (Mobile Broadband Performance, p. 14)
G.9
The third-party app developer shall describe how the latency test measures the round-trip time of User Datagram Protocol (UDP) packets for an elapsed minimum time of 5 seconds and a maximum elapsed time of 30 seconds between the device and a target test site.  The test shall record the start time, duration, number of packets sent, number of packets received, and round-trip jitter.  In addition, the third-party app shall report all other data required in the Latency Test Object table of the BDC - Data Specifications for Mobile Speed Test Data. (Mobile Broadband Performance, p. 15)
G.10
The third-party app developer shall provide similar procedures as described in items G.3 - G.9 and describe in their proposal how the third-party app measures the download speed, upload speed, and latency using a different methodology from the methodology above.  In such case, the proposal must detail the proposed methodology and must provide a technical justification for how such methodology measures download, upload, and latency as compared to the FCC Speed Test app, described in the 2021 FCC Speed Test App Technical Description document. (Mobile Broadband Performance, p. 15)

Section H. Data Collection and Submission to the BDC System
Requirement ID
Description
H.1
Confirm that the app system will transmit data from the mobile devices of its end users to its data repository system after the end user completes active test measurements. (Data Collection and Submission to the BDC System, p. 15)
H.1.1
Confirm that its data repository(ies) are located with the United States, and provide the location of the data repository system. (Data Collection and Submission to the BDC System, p. 15)
H.2
Indicate if the app is intended for use in, and to collect data for, a limited geographic area (e.g., state, city, community, etc.) rather than the entire United States. (Data Collection and Submission to the BDC System, p. 15)
H.2.1
If applicable, provide the details of the limited geographic area (e.g., state, city, community, and boundaries) in which the third-party app is intended for use. (Data Collection and Submission to the BDC System, p. 15)

H.2.2
If applicable, describe how the app provides information to end users about the limited geographic area it covers or information about such limitations.  (Data Collection and Submission to the BDC System, p. 15)
H.2.3
If applicable, describe how the app data repository system compiles and filters the collected data to the designated area’s boundary prior to transmission to the BDC system. (Data Collection and Submission to the BDC System, p. 15)
H.3
Describe how, when mobile network connectivity is unavailable to an app end user, the app will collect and store test results and submit the results when connectivity becomes available (i.e., “store-and-forward” capability). (Data Collection and Submission to the BDC System, p. 15)
H.4
Describe how often the app will transmit test results from its data repository system to the BDC system. (Data Collection and Submission to the BDC System, p. 15)
H.5
Indicate if the test result data transmitted to the BDC system from its servers through an API in a JSON format matches the structure in accordance with the BDC - Data Specifications for Mobile Speed Test Data. (Data Collection and Submission to the BDC System, p. 16)
H.5.1
The third-party app developer must provide sample(s) for collected data in JSON format. (Data Collection and Submission to the BDC System, p. 16)
H.5.2
Indicate if the test results other than those for the subscribed mobile network connections (e.g., Wi-Fi) will be filtered and not transmitted to the BDC system. (Data Collection and Submission to the BDC System, p. 16)

Section I. Privacy Policies
Requirement ID
Description
I.1
Provide the comprehensive privacy policy, which must document the practices with respect to storage, use, and protection of challenge data. (Privacy Policies, p. 16)
I.2
Confirm that the privacy policy is easily accessible clear and concise and written in plain language; that it is not overly technical, vague, opaque, or lengthy; and that it does not require specialized knowledge to understand. (Privacy Policies, p. 16)
I.3
Confirm that the privacy policy includes, a high-level description of the administrative, technical, and physical safeguards and controls that the developer has adopted to protect the PII collected by the third-party app against any hazards to confidentiality, integrity, or availability.  This description must specifically address whether the third-party app developer will maintain all test data containing PII within the United States, or, if not, what alternative measures the developer is taking to ensure the security of the data maintained in a foreign jurisdiction. (Privacy Policies, p. 16)
I.4
Confirm that the privacy policy explains how, if at all, the third-party app developer will “use” individual challenge data, including to monetize such data (e.g., through the delivery of advertisements or the promotion of other products and services). (Privacy Policies, p. 16)

Bulletin 75 Section 4 Certification on Required Testing Parameters and Data Metrics.  Certify and describe how the third-party app meets the data reporting and formatting requirements set forth in the BDC - Data Specifications for Mobile Speed Test Data.  Guidance for providing information on testing parameters and test metrics is outlined in Section 4 of OET Bulletin 75, Broadband Data Collection Program: Third-Party Speed Test Mobile Application Approval Process Guidance at pp. 16-21.
2