The Schools and Libraries Program Initial Statistical Analysis of Data from the 2006/2007 Compliance Audits By Office of Inspector General Federal Communications Commission October 3, 2007 2 Background and Introduction This report contains a statistical analysis of data from the 2006/2007 Audit of the Schools and Libraries (“S&L”) Program of the Federal Communications Commission (“FCC” or “Commission”). The primary objective of the Inspector General (“IG”) in auditing the S&L program was to determine the extent to which the Schools s & Libraries program was being administered in accordance with the Commission's rules, orders and interpretative opinions. An additional objective was to provide audit results that would permit statistical estimates of the error rates under the Improper Payments Information Act of 2002 (“IPIA”).1 Under IPIA standards, a program is at risk if the erroneous payment rate exceeds 2.5 percent and the amount of erroneous payments is greater than $10 million. To assess compliance and risk, a stratified simple random sample of 155 FRNs2 was drawn and compliance attestation audits were completed. Statistical results from the sample suggest that the program is at risk. The erroneous payment rate is estimated at 12.9 percent with a margin of error ± 4.5 percent at the 90 percent confidence level.3 Compliance with rules and regulations was generally high (90 percent or more). Schools and Libraries Program Description The S&L Program of the Universal Service Fund (“USF”), commonly known as "E-rate," is administered by the Universal Service Administrative Company (“USAC”) under the direction of the FCC, and provides discounts to assist most schools and libraries in the United States to obtain affordable telecommunications and Internet access services. It is one of four support programs funded through a USF charged to companies that provide interstate and/or international telecommunications services.4 The S&L Program supports connectivity, in other words, the conduit or pipeline for communications that use telecommunications services as well as the Internet. Applicants must provide additional resources including end-user equipment (e.g., computers, telephones, etc.), software, professional development, and the other elements that are necessary to utilize the connectivity funded by the Schools and Libraries Program. Funding is requested under four categories of service: telecommunications services, Internet access, internal connections, and basic maintenance of internal connections. Discounts for support depend on the level of poverty and the urban or rural 1 Pub. L. 107-300; 11 Stat. 2350. 2 A FRN is a Funding Request Number that is assigned to a project for which USF support is requested. 3 During the audit period, approximately $1.6 billion was distributed to schools and libraries. 4 Under section 54.712 of the Commission’s rules, 47 C.F.R. § 54.712, carriers may recover universal service contributions from their end user customers so long as the line-item charge does “. . . not exceed the interstate telecommunications portion of the bill times the relevant contribution factor.” 3 status of the population served and range from 20 percent to 90 percent of the costs of eligible services.5 Eligible schools, school districts and libraries may apply individually or as part of a consortium.6 Overview of Administrative Process: Federal and state law determine eligibility of schools, school districts, and libraries for S&L support. In general, a school is eligible for S&L support if it meets the following eligibility requirements: · schools must provide elementary or secondary education as determined under state law; · schools may be public or private institutional day or residential schools, or public charter schools; · schools must operate as non-profit businesses; and · schools cannot have an endowment exceeding $50 million.7 In many cases, non-traditional facilities and students may be eligible.8 Libraries must meet the statutory definition of library or library consortium found in the 1996 Library Services and Technology Act (“LSTA”)9 to meet eligibility requirements for S&L support. Libraries must be eligible for assistance from a state library administrative agency under that Act; must have budgets completely separate from any schools (including, but not limited to, elementary and secondary schools, colleges and universities); and cannot operate as for-profit businesses. Schools, school districts, and libraries that want to apply for S&L support, commonly referred to as "E-rate," must first prepare a technology plan.10 An approved technology plan sets out how information technology and telecommunications infrastructure will be used to achieve educational goals, specific curriculum reforms, or library service improvements. A technology plan designed to improve education or library services should cover the entire funding year (July 1 to June 30) but not more than three years. The plan must 5 See 47 C.F.R § 54.505. 6 See 47 C.F.R. § 54.501(d). 7 See 47 C.F.R. § 54.501. 8 For example, an Educational Service Agency, which may operate owned or leased instructional facilities, may be eligible for S&L support if it provides elementary or secondary education as defined in state law. 9 Pub. L. 104-208, 110 Stat. 3009. 10 See 47 C.F.R. § 54.508. 4 contain the following five elements: (1) goals and a realistic strategy for using telecommunications and information technology; (2) a professional development strategy; (3) an assessment of telecommunication services, hardware, software, and other services needed; (4) budget resources; and (5) an ongoing evaluation process. The technology plan must be approved by a USAC-certified technology plan approver before discounted services can begin.11 The state is the certified technology plan approver for libraries and public schools. Applicants must file the Description of Services Requested and Certification Form (Form 470)12 to begin the competitive process and must ensure an open and fair competitive bidding process for specific products. Applicants must file a new Form 470 each funding year for requests for tariffed or month-to-month services and for new contractual services.13 When the Form 470 is filed, USAC will make it available to interested service providers by posting it to the USAC website. Applicants must: (1) describe specific services or functions for support; (2) identify the correct category of services: telecommunications, Internet access, internal connections, or basic maintenance of internal connections: (3) identify recipients of services for support; (4) follow all applicable state and local procurement laws: and (5) wait 28 days after the Form 470 is posted to the USAC website or after public availability of your Request for Proposals (“RFP”), whichever is later, before selecting a vendor or executing a contract. Applicants may use RFPs or other solicitation methods tailored to specific needs and circumstances in addition to the required Form 470. The Form 470 must be completed by the entity that will negotiate for eligible products and services with potential service providers. A service provider that participates in the competitive bidding process as a bidder cannot be involved in the preparation or certification of the entity's Form 470. Applicants must select the most cost-effective provider of the desired products or services eligible for support, with price as the primary factor.14 At the conclusion of the 28-day waiting period after the Form 470 is posted on the USAC website, the applicant may select a vendor for tariffed or month-to-month services or execute a contract for new contractual services. Applicants may also choose vendors from a State Master Contract, execute multi-year contracts pursuant to a Form 470, and enter into voluntary contract 11 The state is the certified technology plan approver for libraries and public schools. Non-public schools and other entities that do not secure approval of their technology plan from their states may locate a USAC- certified technology plan approver from USAC. Applicants that seek S&L Program support only for basic telephone service do not need a technology plan. 12 A new AForm 470 is not required if an applicant intends to seek discounts on services provided under a multi-year contract executed under a posted Form 470 in a prior funding year. 13 See f.n. 11, supra. 14 See 47 C.F.R. §54.504(b)(4). 5 extensions, but certain additional contract requirements apply. In all cases, applicants must comply with state and local procurement laws. Applicants must save all documentation pertaining to the competitive bidding process and vendor selection for five years.15 Applicants must certify and acknowledge on the Form 470 and the Services Ordered and Certification Form (Form 471) that they may be audited and that they must retain all records that can verify the accuracy of information provided. An applicant that applies for S&L Program support for eligible services must calculate the discount percentage that it and the schools or libraries it represents are eligible to receive. Applicants use the Form 471 to calculate the discount and begin by listing the recipients of services for support. FCC rules include a discount matrix that takes into consideration poverty level and the urban or rural location of the participating entity.16 The primary measure for determining Schools and Libraries support discounts is the percentage of students eligible for free and reduced lunches under the National School Lunch Program (“NSLP”), calculated by individual school. A school district applicant calculates its shared discount by calculating a weighted average of the discounts of all individual schools included in the school district. Library branches or outlets must obtain and use the NSLP data for the public school district in which they are located to calculate the discount. A library system applicant calculates its shared discount by calculating an average of the discounts of all library branches or outlets included in the system. A consortium calculates its shared discount by calculating the average of the discounts of all eligible libraries and schools that are included in its membership. Every school or library in the United States is located in either a rural or an urban area, based on Metropolitan Statistical Area (“MSA”) data. The applicant must determine if the individual school or library is rural or urban to properly calculate its percentage discount. Non-instructional facilities that serve educational purposes may be eligible to receive discounts on telecommunications and Internet access services (Priority 1 services). 15 See 47 C.F.R. § 54.516(a). 16 See 478 C.F.R. § 54.505. 6 Compliance and IPIA Audits In 2006, the Inspector General (“IG”) established two objectives that an audit of the S&L Program was to achieve. Because the S&L program was considered at risk,17 the primary objective of each audit was to determine was to determine the extent of compliance with FCC rules, orders and interpretative opinions. An additional objective was to estimate, statistically, the erroneous payment rate In order to determine compliance (as captured within the general administrative process described above), a compliance attestation audit of each school and library on a specific FRN was undertaken. With compliance attestation audits, the auditee (management of the school or library receiving S&L support) is required to sign an assertion letter acknowledging its responsibility for compliance with applicable requirements of Commission rules (e.g. 47 C.F.R. §§ 54.500 - 54.523) as well as related FCC orders and interpretive opinions; and to make specific assertions relative to an applicant’s compliance with those rules. Auditors test the assertions, and provide the cause(s) for the failure of an assertion. That is, auditors determine whether a beneficiary of the S&L program is in compliance with FCC rules and, if the beneficiary is not in compliance, the auditors provide the cause(s) of, or reason(s) for non-compliance. Because FCC Rules changed over the time frame audited, there were multiple assertion letters. Table 1 contains the fundamentals of an assertion letter that management of an auditee signed. Appendix A contains all assertion letters that applied to the entities that were audited in this program. We statistically analyze data generated from compliance attestation audits based on the assertion letters that are contained in Appendix A. TABLE 1 What kinds of service are you seeking: Telecommunications Services, Internet Access, Internal Connections Other than Basic Maintenance, or Basic Maintenance of Internal Connections? Telecommunications Services a. Do you have a Request for Proposal (RFP) that specifies the services you are seeking? If you check YES, your RFP must be available to all interested bidders for at least 28 days. b. List below the Telecommunications Services you seek. Specify each service or function (e.g. local voice service) and quantity and/or capacity (e.g. 20 existing lines plus 10 new ones). Internet Access a. Do you have a Request for Proposal (RFP) that specifies the services you are seeking RFP must be available to all interested bidders for at least 28 days. 17 See FCC Report to Congress on Improper Payments, March 31, 2004. 7 b. List below the Internet Access services you seek. Specify each service or function (e.g., monthly Internet service) and quantity and/or capacity (e.g., for 500 users) Internal Connections Other than Basic Maintenance a. Do you have a Request for Proposal (RFP) that specifies the services you are seeking? If you check YES, your RFP must be available to all interested bidders for at least 28 days. b. List below the Internal Connections services you seek. Specify each service or function (e.g., a router, hub and cabling) and quantity and/or capacity (e.g., connecting 1 classroom of 30 students). Basic Maintenance of Internal Connections a. Do you have a Request for Proposal (RFP) that specifies the services you are seeking? If you check YES, your RFP must be available to all interested bidders for at least 28 days. b. List below the Basic Maintenance services you seek. Specify each service or function (e.g., basic maintenance of routers) and quantity and/or capacity (e.g., for 10 routers). (Optional) Please name the person on your staff or project who can provide additional technical details or answer specific questions from service providers about the services you are seeking. Check [this box] if there are any restrictions imposed by state or local laws or regulations on how or when service providers may contact you or on other bidding procedures. Please describe below any such restrictions or procedures and/or provide a Web address where they are posted and a contact name and telephone number. Check [this box] if no state and local procurement/competitive bidding requirements apply to the procurement of services sought on this Form 470. If you have plans to purchase additional services in future years, or expect to seek new contracts for existing services, you may summarize below (including the likely timeframes). If you are requesting services for a funding year for which a Form 470 cannot yet be filed online, include that information here. Although the following services and facilities are ineligible for support, they are usually necessary to make effective use of the eligible services requested in this application. Unless you indicated that your application is ONLY for basic telephone service, you must check one or both boxes. You may provide details for purchases being sought. Desktop software: Software required a has been purchased; and/or is being sought. Electrical systems: b adequate electrical capacity is in place or has already been arranged; and/or upgrading for electrical capacity is being sought. Computers: a sufficient quantity of computers c has been purchased; and/or is being sought. Computer hardware maintenance: adequate arrangements d have been made; and/or are being sought. Staff development: E all staff have had an appropriate level of training/additional training has been scheduled; and/or training is being sought. Check the ONE choice that best describes this application and the eligible entities that will receive the services described in this application. You will then list the entity/entities that will pay the bills for these services. a Individual school or single-site library. b Statewide application for (enter 2-letter state code) representing (check all that apply): All public schools/districts in the state. All non-public schools in the state. All libraries in the state. c School district, library system or consortium application to serve multiple eligible 8 entities. Table 2 Contains one of the nine assertion Letters. The assertion letter applies to Schools in 2004. TABLE 2 Report of Management on Compliance with Applicable Requirements of 47 C.F.R Section 54 of the Federal Communications Commission’s Rules and Regulations and Related Orders Management of [BENEFICIARY NAME] (the “District”) is responsible for ensuring the District’s compliance with applicable requirements of 47 C.F.R. Sections 54.500 through 54.523 of the Federal Communications Commission’s (“FCC”) Rules and Regulations for Universal Service Support for Schools and Libraries, as amended, and related FCC Orders. Management has performed an evaluation of the District’s compliance with the applicable requirements of 47 C.F.R. Section 54.500 through 54.523, as amended, and related FCC Orders with respect to disbursements made from the Universal Service Fund during the year ended September 30, 2005 on our behalf relative to Funding Request No. xxxxxxx and the related Funding Year 2004 application for funding and service provider selection for such Funding Request Number (“FRN”). Based on this evaluation, we assert that as of [DATE], 2006, the District complied with all applicable requirements of 47 C.F.R. Sections 54.500 through 54.523, as amended, and related FCC Orders in all material respects. The District used [SERVICE PROVIDER NAME] as its service provider relative to FRN xxxxxxx. In addition to providing the goods and/or services for which the disbursements were made relative to that FRN, [SERVICE PROVIDER NAME] performed the following specific functions to qualify as a service provider for the Schools and Libraries Support Mechanism and on behalf of the District: Ø Preparation and submission of FCC Form 473, Service Provider Annual Certification Form Ø Preparation and submission of FCC Forms 474, Service Provider Invoice Ø Receipt of disbursements from the Universal Service Fund as requested by FCC Forms 472, Billed Entity Applicant Reimbursement, or FCC Forms 474, Service Provider Invoice Ø Reimbursement to the District of disbursements from the Universal Service Fund as requested by FCC Forms 472, Billed Entity Applicant Reimbursement The District has obtained and relied upon assurance from [SERVICE PROVIDER NAME] to verify that controls and procedures relating to these assertions have been established and maintained by [SERVICE PROVIDER NAME] in accordance with all applicable requirements of 47 C.F.R. Sections 54.500 through 54.523, as amended, and related FCC Orders. The District represents the following assertions per the applicable FCC Rules and Regulations, as amended, and related FCC Orders (which are identified herein with each assertion) with respect to disbursements made from the Universal Service Fund during the 9 year ended September 30, 2005 on our behalf relative to FRN xxxxxxx and the related Funding Year 2004 application for funding and service provider selection for such FRN: A. General Matters – The District: 1. maintained for its purchases of telecommunications and other supported services at discounted rates the kind of procurement records that it maintains for other purchases (Section 54.516 (a) which was effective from July 17, 1997 through November 12, 2004) 2. retained, to date, all documents related to the application for, receipt, and delivery of discounted telecommunications and other supported services, to the extent there were such documents, and retained any other document that demonstrates compliance with the statutory or regulatory requirements for the schools and libraries mechanism (Sections 54.516 (a) (1) and 54.504 (c) (1) (x) which were effective as of November 12, 2004 and require a five-year retention period for such documents) 3. maintained, to date, asset and inventory records of equipment purchased as components of supported internal connections services sufficient to verify the actual location of such equipment (Section 54.516 (a) which was applicable from March 11, 2004 to November 11, 2004 and Section 54.516 (a) (1) which was effective as of November 12, 2004, both of which require maintenance of such records for a period of five years after purchase) B. Application Matters – The District: 1) requested discounts from the Universal Service Fund for telecommunications and other supported services only for schools: (a) meeting the statutory definitions of "elementary school," as defined in 20 U.S.C. § 7801(14), or "secondary school," as defined in 20 U.S.C. § 7801(25); (b) which are not operated as for-profit businesses; and (c) do not have endowments exceeding $50,000,000 as of the date such services were received (Section 54.501 (b) which was effective as of July 17, 1997) 2) submitted a properly completed FCC Form 470, including the required certification of an authorized person (Sections 54.504 (a) and (b) which were effective as of July 17, 1997, with clarification included in FCC Order 03-313, paragraph 56 which was issued on December 8, 2003) 3) had the resources required to make use of the services requested, or such resources were budgeted for purchase for the current, next or other future academic years, at the time the FCC Form 470 was filed (Section 54.504 (b) (1) which was effective as of July 17, 1997) 4) identified all co-purchasers related to aggregated purchases with other entities, if any, and the services or portion of the services purchased by the District on the FCC Form 470 (Section 54.504 (b) (2) (iv) which was effective as of July 17, 1997) 5) had a technology plan that had been certified by its state, USAC or an independent entity approved by the FCC at the time of filing the FCC Form 470 (Section 54.504 (b) (2) (vii) which were effective as of July 17, 1997) 6) accurately determined its level of poverty, for use in determining its available discount rate, by using the percentage of its student enrollment that is eligible for a free or reduced price lunch under the national school lunch program or a federally- approved alternative mechanism in the public school district in which they are 10 located (Section 54.505 (b) which was effective as of July 17, 1997) 7) accurately applied the approved discount matrix, with the correct consideration of urban or rural location, to its determined level of poverty to set its discount rate to be applied to eligible goods and /or services (Section 54.505 (c), as revised, which was originally effective as of July 17, 1997) 8) submitted a completed FCC Form 471 only after signing a contract for eligible goods and/or services (Section 54.504 (c) which was effective as of February 12, 1998) 9) requested only, and funds were disbursed by the Universal Service Fund only for, eligible goods and services (Telecommunications services - Section 54.502; Internet access services – Section 54.503; Installation and maintenance of internal connections – Sections 54.503, 54.506 (b) and 54.518, all of which were effective as of February 12, 1998) 10) allocated the cost of any contract, for which request for discount was made, to eligible and ineligible components (Section 54.504 (g) which was effective as of March 11, 2004) 11) submitted a certification on FCC Form 486 that an Internet safety policy is being enforced (Section 54.520 (c) which was effective as of August 5, 2002) 12) complied with the certification on FCC Form 486, such that: a) it enforced a policy of Internet safety that includes monitoring the online activities of minors and the operation of a technology protection measure, with respect to any of its computers with Internet access, that protects against access through such computers to visual depictions that are obscene, child pornography or harmful to minors (Section 54.520 (c) (1) (i) which was effective as of August 5, 2002); and b) its Internet safety policy addresses each of the following (Section 54.520 (c) (1) (ii) which was effective as of August 5, 2002): i) access by minors to inappropriate matter on the Internet and World Wide Web; ii) the safety and security of minors when using electronic mail, chat rooms, and other forms of direct electronic communications; iii) unauthorized access, including so-called `hacking', and other unlawful activities by minors online; iv) unauthorized disclosure, use, and dissemination of personal identification information regarding minors; and v) measures designed to restrict minors’ access to materials harmful to minors C. Service Provider Selection Matters – The District: 1) sought competitive bids for all eligible goods and/or services for which Universal Service Fund support was requested and complied with applicable state and local procurement processes included in the District’s documented policies and 11 procedures (Section 54.504 (a) which was effective as of July 17, 1997) 2) waited at least four weeks after the posting date of the FCC Form 470 on the USAC Schools and Libraries website before making commitments with the selected service providers (Section 54.504 (b) (4) which was effective as of February 12, 1998) 3) considered all bids submitted and selected the most cost-effective service offering (Section 54.511 (a) which was effective as of July 17, 1997, with clarification included in FCC Order 03-101, paragraph 24, which was issued on July 15, 2003) 4) did not surrender control of its competitive bidding process to a service provider that participated in that bidding process and did not include service provider contact information on its FCC Forms 470 or 471 (FCC Order 00-167, paragraph 10, which was issued on May 23, 2000) D. Receipt of Services and Reimbursement Matters – The District: 1) applied its discount percentage to the appropriate pre-discount price (Section 54.505 (a) which was effective as of July 17, 1997) 2) received reimbursement from its service provider for purchases for which it had paid full price to the service provider (Section 54.514 (b) which was effective as of August 14, 2003) 3) requested, and funds were disbursed by the Universal Service Fund, amounts related to service substitutions that were based on the lower of the pre-discount price of the service for which support was originally requested or the pre- discount price of the new, substituted service (Section 54.504 (f)), which was effective as of March 14, 2004 as confirmation of earlier administrative practices) 4) used the services requested solely for educational purposes (Section 54.504 (b) (2) (ii) which was effective as of July 17, 1997 and 54.500 (b) which was effective as of August 14, 2003) 5) , with respect to eligible services and equipment components purchased at a discount, a) did not sell or resell such items for money or any other thing of value; b) did not transfer such items, with or without consideration of money or any other thing of value, for a period of three years after purchase, other than in the event that such transfer was made to another eligible school or library in the event the particular location where the service was originally received was permanently or temporarily closed; c) notified USAC of any such allowable transfer; and d) maintained, as did the recipient, detailed records documenting the transfer and the reason for the transfer for a period of five years (Section 54.504 (b) (2) (iii) which was effective as of July 17, 1997 and 54.513 12 (c) which was effective March 11, 2004) 6) paid all “non-discount” portions of requested goods and/or services (Section 54.504 (b) (2) (v) which was effective as of July 17, 1997) 7) allocated the costs of any contract that included both eligible and ineligible components to those eligible and ineligible components in the related request for discount (Section 54.504, which was effective as of July 17, 1997 ,with confirmation of earlier administrative practices included in FCC Order 03-313, paragraph 60 which was issued on December 8, 2003, and codified in Section 54.504 (g) which was effective as of March 11, 2004) 8) deducted from the pre-discount cost of services indicated in funding requests the value of all price reductions, promotional offers and “free” products or services (Section 54.504 which was effective as of July 17, 1997, with confirmation of earlier administrative practices included in FCC Order 03-313, paragraph 60 which was issued on December 8, 2003) Sample Design The S&L program sample design was a stratified simple random sample of 155 FRNs from a population data base of 95,558. The seven strata were defined in terms of “total amount disbursed” as follows: 1. Negative; i.e., funds returned to USAC. 2. Zero; i.e., no funds paid out or received. 3. Amount disbursed from $1 to less than $10,000. 4. Amount disbursed from $10,000 to less than $100,000. 5. Amount disbursed from $100,000 to less than $500,000. 6. Amount disbursed from $500,000 to less than $10,000,000. 7. Amount disbursed from $10,000,000 or more. The number of FRNs in the population and sample within each stratum is shown in the table below. Although data from audits conducted by KPMG in 2000 were analyzed, they were not FRN-based and provided little help in determining sample size needed to estimate erroneous payments with a margin of error of 2.5 percent at the 90 percent confidence level. Consequently, Neyman (optimal) allocation was applied to arrive at stratum sample sizes so as to optimally estimate total amount disbursed from the sample. With sufficient correlation between erroneous payments and disbursement amount, it was conjectured that the error rate may come close to 2.5 percent, but this could not be guaranteed since an advance estimate of the correlation was not available. 13 Number of FRN Stratified by Total Amount Disbursed Amount <$0 $0 $1 to $10k $10k to $100k $100k to $500k $500k to $10m $10m or more Total Sample Number of FRNs 78 4297 73295 15805 1758 318 7 95558 Sample Size 10 10 20 36 19 53 7 155 Sample Selection The sample was selected from the sampling frame of all 95,560 FRNs using the SAS procedure PROC SURVEYSELECT. Additional FRNs were randomly selected in each stratum for use, as needed, to augment the sample and/or replace FRNs that could not be audited for various reasons. Issues Associated with Sampling Some FRNs were replaced due to auditor (KPMG) conflict of interest or to law enforcement issues. In particular, the $10million or greater stratum originally had 9 members. One of the 155 audits resulted in a disclaimer due to lack of sufficient document- ation for KPMG to give an opinion. Consequently, the entire amount disbursed to this provider was regarded as erroneous. In less than 10 percent of the sampled FRNs, the number of pieces of equipment was so large that KPMG attested to the correctness of disbursements for a random sample, rather than all, items. Sampled items were required to be at least 60 in number, and in no case was an improper payment found. Therefore, this necessity to sub-sample items for FRN had no effect on the error rate. 14 Estimation Formulas18 The SL_2005 sample design was stratified into seven strata. One stratum consisted of the 7 units with disbursements of $10,000,000 or more, all of which were audited. Simple random samples were selected from the other strata. To estimate the amount and proportion of erroneous payments in the population, the combined ratio estimator was used. The remainder of this section gives the formulas used to compute the combined ratio estimator and its margin of error. ix = amount dispersed to FRN i. iy = absolute value of improper payment amount for FRN i. known population total of x.xt = Number of FRNs in stratum hhN = . sample size from stratum h.hn = 1 h i ihh xxn Î = å = sample mean disbursement x in stratum h. 1 ih ihh yyn Î = å = sample mean of improper payments y in stratum h. µ h xh htNx= = estimate of total disbursements in stratum h. µ yh h htNy= = estimate of total erroneous payments in stratum h. µ yh h xh h t p t = å å $ $ = estimated proportion of erroneous payments in the population. µ residual ii idypx=- = 1 h i ihh ddn Î = å = sample mean residual in stratum h. ( )22 1 1 hdh i ihh sddn Î =-- å = sample variance of residuals d in stratum h. 18 See W.G. Cochran, Sampling Techniques, 3rd edition (1977), John Wiley & Sons, New York. 15 ME = (1.645) 22111hdh hxhh NstnNæö-ç÷ èøå = 90 percent margin of error for estimated proportion. Estimation Results The erroneous payment rate is estimated at 12.9 percent with a margin of error ± 4.5 percent at the 90 percent confidence level. In other words, a 90 percent confidence interval for the error rate is 8.4 percent to 17.4 percent. Compliance with rules and regulations was generally high (90 percent or more) with the notable exception of submitting/obtaining technical plan approval. Table 3 below contains the details. TABLE 3 Schools and Libraries Program Compliance* ASSERTION Materially Non-Material Compliant Non-Compliant Non-Compliant Other A.1 PROC RECORDS 95.3% 0.1% 4.6% A.2 E-RATE RECORDS 97.9% 0.7% 1.4% A.3 ASSET RECORDS 81.6% 11.3% 7.1% B.1 ELIGIBLE ORGANIZATION 99.8% 0.1% 0.1% B.2 FORM 470 99.4% 0.5% 0.1% B.2-C ELIGIBLE LIBRARY CONSORTIUM 100.0% B.3 RESOURCES 99.9% 0.1% B.3-C TELECOMMUNICATIONS CONSORTIUM 100.0% 16 B.4 CO-PURCHASERS 100.0% B.4-C ELIGIBLE SERVICES 93.8% 6.3% B.5 TECH PLAN 84.0% 3.6% 12.3% B.5-C FORM 470 100.0% B.6 LEVEL OF POVERTY 99.0% 1.0% B.6-C RESOURCES 100.0% B.7 DISCOUNT MATRIX 95.9% 4.1% B.7-C CO-PURCHASERS 100.0% B.8 CONTRACT BEFORE FORM 471 99.3% 0.5% 0.2% B.8-C TECH PLAN APPROVED 50.3% 38.9% 10.8% B.9 ELIGIBLE GOODS/SERVICES 95.0% 4.1% 0.9% B.9-C WEIGHTED AVERAGE DISCOUNTS 98.8% 1.2% B.10 COST ALLOCATION 100.0% B.10-C CONTRACT BEFORE FORM 471 57.7% 42.3% B.10-LS CIPA CERT ON FORM 486 100% B.11-C 17 ELIGIBLE GOODS/SERVICES 99.6% 0.4% B.11 -LS CIPA 99.9% 0.1% B.12 CIPA 65.9% 5.5% 28.6% B.12-C CONSORTIA FORM 479 89.9% 8.4% 1.7% B.13-C MAINTAINED LETTERS OF AGENCY 94.1% 5.9% C.1 COMPETITIVE BIDS 99.5% 0.5% C.2 28 DAYS AFTER FORM 470 91.9% 8.1% C.3 BID SELECTION 99.5% 0.5% C-4 PROPER BID PROCESS 100.0% D,1 APPLIED PROPER PERCENTAGE 78.4% 12.8% 8.8% D.2 REIMBURSED BY SERV PROV 100.0% D.3 SERVICE SUBSTITUTION 99.4% 0.5% D.4 EDUCATIONAL PURPOSES 98.1% 1.6% 0.2% D.5 TRANSFER 99.9% 0.1% D.5-A RESALE 100.0% D.5-B TRANSFER 99.9% 0.1% 18 D.5-C NOTIFIED USAC 98.8% 1.0% 0.1% D.5-D TRANSFER RECORDS 99.9% 0.1% D.6 PAID NON-DISCOUNT PORTION 99.3% 0.6% 0.1% D.7 COST ALLOCATION 100.0% D.8 INCENTIVES DEDUCTED 99.5% 0.5% * Weighted Percent Totals Causes of Non-Compliance When there was non-compliance on any assertion, data were collected on causes of non-compliance. Table 4 contains all 21 possible causes of non-compliance. Data were collected such that, if an auditor found multiple causes of non-compliance, all information would be retained. TABLE 4 1-Imprecise FCC Rule/s 2-Contradictory FCC Rule/s 3-Overly Complex FCC Rules 4-Disregarded FCC Rule/s 5-Followed State Rule/s (apparent conflict with FCC Rule/s) 6-Followed USAC Procedures (apparent conflict with FCC Rule/s) 7-Inadequate Documentation Retention 8-Inadequate Auditee Processes and/or Polices and Procedures 9-Inadequate Systems for Collecting, Reporting, and/or Monitoring Data 10-Insufficient Resources/Time to Complete Task/Activity 11-Failure to Review/Monitor Work, Material, or Data/Application Submitted by Consultant/Agent 12-Applicant/Auditee Weak Internal Controls 13-Applicant/Auditee Data Entry Error 14-Service Provider Weak Internal Controls 15-Service Provider Data Entry Error 16-Service Provider Error (other) 19 17-USAC Error 18-SOLIX Error 19-NECA Error 20-Force Majeure (Acts of God and Nature) 21-Other The statistical analysis of cause is contained in Table 5 below. Causality of non- compliance depends on the particular assertion (rule or regulation). Inadequate documentation, inadequate systems for collecting & reporting and inadequate auditee processes appear to be principal causes. Non-material non-compliance with the requirement to have a ‘Tech Plan Approved’ seems to be ‘caused’ by following USAC procedures, whereas material non-compliance seems to be ‘caused’ by a variety of other factors including disregard of FCC rules or following State’s rules. Table 5 Schools and Libraries: 21 Reasons for Non-Compliance* Assertion N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 A.1 PROC RECORDS N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Materially non-compliant 3 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Non-materially non-compliant 6 0 0 0 0 0 0 4 0 1 0 0 0 0 0 0 1 0 0 0 0 0 A.2 E-RATE RECORDS N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Materially non-compliant 4 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Non-materially non-compliant 5 0 0 0 0 0 0 4 0 0 0 0 0 0 0 0 1 0 0 0 0 0 A.3 ASSET RECORDS N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Materially non-compliant 2 0 0 0 0 0 0 0 1 1 0 0 2 0 0 0 0 0 0 0 0 0 Non-materially non-compliant 5 0 0 0 0 0 0 0 2 2 0 0 1 0 0 0 0 0 0 0 0 0 B.1 ELIGIBLE ORGANIZATION N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Materially non-compliant 6 0 0 0 0 0 0 0 0 5 0 0 1 0 0 0 0 0 0 0 0 0 Non-materially non-compliant 4 0 0 0 1 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 1 B.2 FORM 470 N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Materially non-compliant 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 Non-materially non-compliant 2 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 B.3 20 RESOURCES N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Materially non-compliant 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 B.5 TECH PLAN N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Other 2 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Materially non-compliant 7 0 0 0 2 1 0 1 0 0 1 0 1 0 0 0 0 0 0 0 0 1 Non-materially non-compliant 22 0 0 0 0 2 17 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 B.6 LEVEL OF POVERTY N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Non-materially non-compliant 4 0 0 0 1 0 0 0 0 0 0 0 0 2 0 0 0 1 0 0 0 0 B.7 DISCOUNT MATRIX N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Materially non-compliant 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 B.8 CONTRACT BEFORE FORM 471 N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Materially non-compliant 2 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 Non-materially non-compliant 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 B.9 ELIGIBLE GOODS/SERVICES N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Other 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 Materially non-compliant 2 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 Non-materially non-compliant 9 0 0 0 0 0 0 0 1 0 0 0 7 0 1 0 0 0 0 0 0 0 B.12 CIPA N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Materially non-compliant 2 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 Non-materially non-compliant 5 0 0 0 0 0 0 0 5 0 0 0 0 0 0 0 0 0 0 0 0 0 C.1 COMPETITIVE BIDS N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Materially non-compliant 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 C.2 28 DAYS AFTER FORM 470 N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Materially non-compliant 2 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 C.3 BID SELECTION N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Other 2 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Materially non-compliant 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 D,1 APPLIED PROPER PERCENTAGE N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Other 1 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 0 21 Materially non-compliant 8 0 0 0 0 0 0 0 0 1 0 0 5 1 0 0 0 0 0 0 0 1 Non-materially non-compliant 9 0 0 0 0 0 0 0 1 0 0 0 6 0 3 0 0 0 0 0 0 0 D.3 SERVICE SUBSTITUTION N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Other 1 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 0 Non-materially non-compliant 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 D.4 EDUCATIONAL PURPOSES N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Other 1 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 0 Materially non-compliant 7 0 0 0 2 0 0 0 1 2 0 0 5 0 0 0 0 0 0 0 0 0 Non-materially non-compliant 4 0 0 0 1 0 0 0 1 1 1 0 1 0 0 0 0 0 0 0 0 0 D.5.a RESALE N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Other 1 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 0 Non-materially non-compliant 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 D.5.b TRANSFER N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Other 1 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 0 Materially non-compliant 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Non-materially non-compliant 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 D.5.c NOTIFIED USAC N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Materially non-compliant 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Non-materially non-compliant 2 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 D.5.d TRANSFER RECORDS N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Non-materially non-compliant 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 D.6 PAID NON-DISCOUNT PORTION N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Other 1 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 0 Materially non-compliant 3 0 0 0 1 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Non-materially non-compliant 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 D,7 COST ALLOCATION N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Materially non-compliant 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 D.8 INCENTIVES DEDUCTED N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Non-materially non-compliant 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 B.4 -C ELIGIBLE SERVICES N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 Materially non-compliant 2 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 B.8 -C TECH PLAN APPROVED N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Materially non-compliant 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 Non-materially non-compliant 6 0 0 0 0 0 6 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 B.9 -C WEIGHTED AVERAGE DISCOUNTS N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Non-materially non-compliant 4 0 0 0 1 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 B.10 -C CONTRACT BEFORE FORM 471 N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Non-materially non-compliant 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 B.11 -C ELIGIBLE GOODS/SERVICES N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Other 1 0 0 0 0 0 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 0 B.12 -C CONSORTIA FORM 479 N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Materially non-compliant 1 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 Non-materially non-compliant 4 0 0 0 1 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 B.13 -C MAINTAINED LETTERS OF AGENCY Non-materially non-compliant 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 B.11 -LS CIPA N 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 Non-materially non-compliant 2 0 0 0 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 * Cause 1-Imprecise FCC Rule/s 2-Contradictory FCC Rule/s 3-Overly Complex FCC Rules 4-Disregarded FCC Rule/s 5-Followed State Rule/s (apparent conflict with FCC Rule/s) 6-Followed USAC Procedures (apparent conflict with FCC Rule/s) 7-Inadequate Documentation Retention 8-Inadequate Auditee Processes and/or Polices and Procedures 9-Inadequate Systems for Collecting, Reporting, and/or Monitoring Data 10-Insufficient Resources/Time to Complete Task/Activity 11-Failure to Review/Monitor Work, Material, or Data/Application Submitted by Consultant/Agent 12-Applicant/Auditee Weak Internal Controls 13-Applicant/Auditee Data Entry Error 14-Service Provider Weak Internal Controls 23 15-Service Provider Data Entry Error 16-Service Provider Error (other) 17-USAC Error 18-SOLIX Error 19-NECA Error 20-Force Majeure (Acts of God and Nature) 21-Other ** N is the sample size. Discussion and Conclusions The S&L Program of USF appears to be at risk because the estimated erroneous payment rate is 12.9 percent. There was, however, general compliance with FCC regulations (90 percent or more) with the notable exception of submitting/obtaining technical plan approval. Why is the compliance rate high and the erroneous payments rate larger than the 2.5 percent required under IPIA. Of the 155 compliance attestation examinations/audits, 1 audit or 0.65 percent of the audits is a disclaimed opinion by auditors. This implies disclaimed opinions cannot explain the relatively high erroneous payment rate. Some insight is gained from examining compliance data. Just over 7.1 percent of the population had materially inadequate asset records, and 11 percent (11.3 percent) of the beneficiaries had immaterially inadequate asset records. Just under 4 percent (3.6 percent) of the beneficiaries were materially non-compliant with rules regarding technical plan approval, and 12.3 percent of the beneficiaries were not materially non-compliant with rules regarding technical plan approval. IPIA compliance appears to been a problem with 5.5 percent materially non-compliant and 28.6 percent of beneficiaries non-materially non-complaint. Non-compliance with assertion D1 (application of the correct discount percentage) is a problem with material non- compliance at 12.8 percent and non-material non-compliance at 8.8 percent. Where auditors could render opinions on compliance/non-compliance with FCC rules, more frequently observed causes of non-compliance were: inadequate documentation, and inadequate systems for collecting and reporting, and inadequate auditee processes appear to be principal causes. Non-material non-compliance with the requirement to have a Technology Plan Approved seems to have been caused by following USAC procedures, whereas material non-compliance seems to be caused by a variety of other factors including disregard of FCC rules or following state rules. Under the IPIA, estimates of both the erroneous payment rate and amount of erroneous payment are intended guide the Commission in assessing risk that is associated with the Schools and Libraries Program. Under IPIA standards, a program is at risk if the erroneous payment rate exceeds 2.5 percent and the amount of erroneous payment is 24 greater than $10 million. To assess compliance and risk, a stratified random sample of FRNs was drawn and compliance attestation examinations/audits were completed. The statistical results from a stratified random sample of 155 auditees suggest that the program is at risk, and there are significant administrative problems in the program. The erroneous payment error rate was 12.9 percent.