Universal Service Administrative Company Final Report and Statistical Analysis of the 2007-08 Federal Communications Commission Office of Inspector General High Cost Program Beneficiary Audits December 15, 2010 USAC FINAL REPORT ON FCC OIG 2007–08 DECEMBER 15, 2010 HIGH COST PROGRAM BENEFICIARY AUDITS Page 1 EXECUTIVE SUMMARY • In 2007-08, the Universal Service Administrative Company (USAC) conducted 390 audits of High Cost Program beneficiaries at the direction of the Federal Communications Commission (FCC or Commission) Office of Inspector General (OIG). One purpose of the audits was to estimate an “error rate” under the Improper Payments Information Act of 2002 (IPIA). • In November 2008, the FCC OIG issued an “initial statistical analysis” of these High Cost Program beneficiary audits. The FCC OIG’s initial statistical analysis reported an estimated improper payment rate of 23.3% with a margin of error of +/- 2.3%. • Follow-up audit work conducted by independent audit firms after the FCC OIG issued its initial statistical analysis demonstrates the correct estimated improper payment rate to be 2.7%, with a margin of error or +/-0.6%. Many payments initially deemed “improper” for IPIA reporting purposes were in fact made in compliance with program rules. • The initially reported estimated improper payment rate of 23.3% was based on substantially incomplete audit work. o Because the FCC OIG required audit firms to report preliminary estimated improper payment information as of July 31, 2008, the $472.5 million first reported as “improper” did not reflect the results of a standard quality assurance review process, which was completed in August 2008. This review of working papers and/or reports lowered by $54.5 million the total improper payment amount even before follow-up audit work got under way. • Follow-up audit work after the standard quality assurance review has demonstrated: o Estimated improper payment amounts for 390 completed audits declined from over $472.5 million to $54.4 million, out of the approximately $2.4 billion audited. o Extrapolating the correct estimated error rate to the entire High Cost Program reduces the estimated “improper” amount from over $970 million (out of $4.14 billion disbursed) to $113 million during the period in question. • The FCC OIG’s November 2008 initial statistical analysis thus reported a substantially inflated estimate of High Cost Program improper payments. o This occurred because 100% of support received by 54 of the 390 beneficiaries audited was deemed “erroneous” prior to properly completing full audits of those beneficiaries. o For example: USAC FINAL REPORT ON FCC OIG 2007–08 DECEMBER 15, 2010 HIGH COST PROGRAM BENEFICIARY AUDITS Page 2 ƒ In several cases, the independent audit firms were required by FCC OIG deadlines to stop work before the beneficiary had time to retrieve old records and demonstrate that payments were made in accordance with program rules. When records were provided after the FCC OIG issued its initial statistical analysis in November 2008, the independent audit firms found that the payments were properly made. ƒ In one instance, over $16 million was deemed “improper” because the auditee could not produce the documentation specifically requested for line counts and fixed assets. Subsequently, the independent audit firm analyzed a sample of billing systems for line counts and performed testing on various assets and found no significant errors. ƒ In another two cases, $7 million for each company was deemed “improper” because the audit firm could not obtain valid representation letters and complete testing on cost allocations. During follow-up work, the audit firm was able to complete proper testing and obtain signed representation letters. • The cost of these 390 High Cost beneficiary audits was approximately $42.4 million. • The audits identified $54.4 million in improper payments; of this amount, $48.1 million are potentially recoverable payments, while the balance, $6.3 million, represents the amount by which carriers were underpaid. • Of the $48.1 million in potentially recoverable payments, USAC has recovered as of August 1, 2010, $20.3 million. For the balance, recovery is either under way or on hold pending the outcome of appeals and/or further administrative review. USAC FINAL REPORT ON FCC OIG 2007–08 DECEMBER 15, 2010 HIGH COST PROGRAM BENEFICIARY AUDITS Page 3 FINAL REPORT AND STATISTICAL ANALYSIS OF THE 2007-08 FCC OIG HIGH COST PROGRAM BENEFICIARY AUDITS Background In 2006, under the direction of the FCC OIG, USAC initiated a wide-ranging effort to conduct an audit program of beneficiaries of the four universal service support programs (the High Cost, Low Income, Rural Health Care, and Schools and Libraries Programs) and contributors to the Universal Service Fund (USF). In accordance with FCC OIG direction, this comprehensive audit program was designed for two separate, though related, purposes: (1) to assess levels of compliance with Commission regulations and (2) to estimate rates of “erroneous” payments pursuant to the Improper Payments Information Act of 2002 (IPIA). USAC contracted with and trained independent audit firms to carry out compliance attestation examinations, the audit methodology specified by the OIG to accomplish these objectives. Under the close oversight of FCC OIG staff, USAC also developed the assertions of compliance to be made by auditee management, developed materiality guidelines, and reviewed proposed audit programs. The FCC OIG intended to extrapolate results for the sample of beneficiaries selected for audit to all funding recipients. 1 Round 1 The 2006-07 FCC OIG USF audit program came to be known as “Round 1” and included audits of 65 High Cost Program beneficiaries selected by the FCC OIG. On October 3, 2007, the FCC OIG released its initial statistical analysis of Round 1 of the audit program. The OIG noted no reported instances of fraud in any USF program area. For the High Cost Program beneficiary audits, the FCC OIG reported “general compliance with FCC rules and regulations” and a 16.6% estimated rate of improper payments with a margin of error of +/- 10.0% at the 90% confidence level. 2 The FCC OIG statistical analysis was substantially based on the work of Dr. A. Richard Bolstein, an expert statistician under contract to USAC. USAC prepared a detailed response to the FCC OIG October 2007 report. 3 USAC’s report covered all aspects of the Round 1 audit program and addressed the FCC OIG’s findings in all program areas, including USF contributors. Among other things, USAC noted that although the initial IPIA data supported the finding that the High Cost Program was at risk for improper payments as defined by OMB guidance interpreting the IPIA, the FCC OIG’s general conclusion that there were significant administrative problems was not supported by the specific audit results. 4 1 FCC OIG intended extrapolation of estimated “improper payments” to fulfill the requirements of the IPIA. See 31 U.S.C. § 3122; Public Law 107-300, Stat. 2350, November 26, 2002. 2 See Office of the Inspector General, Federal Communications Commission: Initial Statistical Analysis of Data from the 2006/2007 Compliance Audits (October 3, 2007). 3 Universal Service Administrative Company Report on the Federal Communications Commission Office of Inspector General 2006–07 Universal Service Fund Audit Program (December 31, 2007) (USAC Round 1 Response). 4 Id. at 16. USAC FINAL REPORT ON FCC OIG 2007–08 DECEMBER 15, 2010 HIGH COST PROGRAM BENEFICIARY AUDITS Page 4 Follow-up audit work conducted by independent audit firms after the FCC OIG issued its initial statistical analysis substantiated assertions of USAC’s initial response to the FCC OIG October 2007 report. It demonstrated the improper payment rate for the High Cost Program actually to be 2.74% with a margin of error or +/-2.8%%, not 16.6% as initially reported by the FCC OIG. Many payments initially deemed “improper” for IPIA reporting purposes were in fact made in compliance with program rules. 5 Round 2 On November 2, 2007, the FCC OIG directed USAC to commence Round 2 of the FCC OIG USF audit program. It included 390 High Cost Program beneficiary audits. The OIG published initial statistical analyses of improper payment rates for Round 2 on November 26, 2008, for the High Cost Program. 6 As in Round 1, USAC completed follow-up work on audits left incomplete by the audit firms. This follow-up work included 25 Round 2 audits for which audit firms issued disclaimers of opinion, resulting in 100% of the support under examination being deemed improper. It also included 27 audits from which audit firms withdrew at the direction of the FCC OIG. As with the disclaimed audit opinions, the withdrawal audits also resulted in 100% of the support under examination being deemed improper. Round 3 On September 25, 2008, the FCC OIG directed USAC to begin Round 3 of the FCC OIG USF audit program. This round included 331 audits of High Cost Program beneficiaries. Initial statistical analyses of this round of audits have not been published, and USAC is working on completing the follow-up work required to provide final results. Preliminary analysis suggests that the results will largely replicate the findings of Rounds 1 and 2. Follow-up Audit Work and Final Statistical Analysis of High Cost Program Audits Similar to Round 1, the FCC OIG specified the objectives and scope of the Round 2 audits by defining the scope of work reflected in the request for proposals (RFP) for audit services and in the engagement letters between USAC and the firms selected to perform the work. The FCC OIG specifically required: (1) Performance of compliance attestation examinations (requiring an opinion from the firms based on assertions made by auditee management). 7 5 See Universal Service Administrative Company Final Report and Statistical Analysis of the 2006-07 Federal Communications Commission Office of Inspector General High Cost Program Beneficiary Audits (September 10, 2009). 6 See Office of the Inspector General, Federal Communications Commission: The High Cost Program Initial Statistical Analysis of Data from the 2007/2008 Compliances [sic] Attestation Examinations (November 2008). 7 Although there are several types of approaches to audits that comply with government auditing standards, the FCC OIG directed that these audits be conducted as “compliance attestation” examinations. Attestation examinations are based on a set of statements by auditee management asserting compliance with program rules and procedures. USAC FINAL REPORT ON FCC OIG 2007–08 DECEMBER 15, 2010 HIGH COST PROGRAM BENEFICIARY AUDITS Page 5 (2) Delivery of examination results by July 31, 2008. As with Round 1, these two requirements combined with the high number of withdrawal audits to yield a substantial overstatement for Round 2 in the initial estimated rate of “improper” payments for the High Cost Program. For example, firms were unable to issue opinions on compliance in almost 15% of the High Cost Program examinations. This failure resulted from audit firms having insufficient time to obtain auditable information from the beneficiaries or from firms withdrawing from the audits at the direction of the FCC OIG. In cases in which audit firms disclaimed any opinion because they could not render one by the FCC OIG’s July 31, 2008, deadline, the FCC OIG required the firms to report 100% of the High Cost Program support received by those beneficiaries as “improper.” In cases in which firms did render an opinion but found that the monetary impact for certain findings could not be quantified, the FCC OIG also required 100% of the support associated with that finding be reported as “improper.” Consequently, the timing and methodological constraints of the FCC OIG USF audit program resulted in High Cost Program support for 54 of the 390 audits being considered 100% improper (25 disclaimers and 29 withdrawals). These 54 audits significantly contributed to the 23.3% error rate reported by the FCC OIG in its November 2008 initial statistical analysis, comprising $278.6 million of the $472.5 million (nearly 59%) of the initially estimated “improper” amount. Consistent with best practices, its internal audit resolution plan, and pursuant to FCC direction, USAC conducted follow-up work to determine whether the payments found to be improper in the initial audit work were recoverable under FCC program rules and thus in fact “improper.” The objective for all follow-up activity was to determine a specific dollar amount recoverable to the USF and to re-calculate, where appropriate, the initially reported estimated improper payment amount. Final Improper Payment Dollar Amounts USAC completed follow-up work on 382 of the 390 audits by June 2010. 8 On October 18, based upon this follow-up work, USAC’s independent statistician, Dr. Richard Bolstein, determined the improper payment rate to be 2.7% with a margin of error of +/-0.6% at the 90% confidence level. 9 Extrapolating these results to the entire universe of High Cost Program disbursements for the same period, estimated improper payments were reduced from $970 million to $113 million out of more than $4.14 billion in program disbursements. Auditors test the validity of management’s assertions of compliance, provide a cause for the failure of any assertion by management, and issue an opinion. Audit opinions can take any of the following forms: (1) Unqualified, with no material findings; (2) Qualified, with a limited number of findings associated with certain assertions; (3) Adverse, or material noncompliance with program rules or requirements; (4) Disclaimer, or inability of the auditor to validate or invalidate compliance assertions; or (5) Withdrawal, or inability of the auditor to complete an audit typically because of concerns about the integrity of records or because of a noncooperative auditee. 8 USAC could not complete audit work for six audits because the carriers would not release certain line count information for fear of violating the Electronic Communications Privacy Act (the ECPA) (18 USC §§ 2702-03), thereby subjecting the carrier to civil and or criminal liability. The FCC OIG did not direct USAC to perform follow-up audit activity on the remaining two audits. 9 A. Richard Bolstein, LLC, Final Analysis of Round Two High Cost Program Audit Data (October 2010) at 1 (attached hereto as Appendix A). USAC FINAL REPORT ON FCC OIG 2007–08 DECEMBER 15, 2010 HIGH COST PROGRAM BENEFICIARY AUDITS Page 6 Of the 390 beneficiaries audited, USAC conducted follow-up work on 285 beneficiaries with findings with a monetary effect. USAC reviewed these cases to determine if USAC agreed with the audit firm’s determination of the improper payment amount or whether additional outreach was needed before such a determination could be made. After conducting extensive follow-up with these beneficiaries to ensure the correct determination was made, USAC identified $54.4 million in improper payments and $48.1 million in potentially recoverable payments. 10 The eight audits remaining incomplete include disbursements totaling $30,848,637. As recommended by Dr. Bolstein, the most reliable and statistically valid way to use these numbers in calculating total improper payment figures involves treating this set of disbursements as a separate stratum unto itself. 11 Under this approach, the payments are assumed either to register as improper at the same rate as the rest of the audits, i.e., 2.7%, or more conservatively, as 100% improper. In the first scenario, the estimated improper payment rate would remain 2.7%, and the total improper payment amount for the program as a whole would rise to $113.8 million. In the second scenario, the rate would rise to 3.5% and total improper payments to $143.9 million. 12 Detailed tables showing the results first excluding the eight incomplete audits and then with the more conservative approach of including the eight in a separate stratum as 100% improper appear below. 13 Final Improper Payment Rates The follow-up audit work led to a reduction in the error rate reported by the FCC OIG of 23.3% to 2.7%, according to Dr. Bolstein’s statistical analysis. The table below shows results of improper payment calculations for different strata of disbursement amounts. Stratum (Range Disbursed) Entities in Stratum 14 Sample Size % of Stratum Population Amount Disbursed Estimated IPIA Amount Estimated IPIA Rate $1 to $1m 902 62 6.9% $345,114,750 $23,023,801 6.7% $1m to $5m 643 182 28.3% $1,448,369,599 $42,757,512 3.0% $5m to $10m 126 66 52.4% $851,812,917 $13,386,844 1.6% > $10m 72 72 100% $1,492,141,309 $33,875,106 2.3% 10 In some instances, improper payments were under-payments, not over-payments, with the additional funds being counted as the “improper” portion of the overall payment. 11 See Appendix A for a fuller discussion of improper payment rate calculation methodologies. 12 If all of these payments were found to be 100% improper and factored back into the original universe of audit data, an approach not recommended by Dr. Bolstein, the overall improper payment rate would be 4.2% (+/-0.8%) and total extrapolated improper payment amount $175.2 million. However, since only one of the fully audited 382 payments was found to be 100% improper, it is highly unlikely that these eight would be found to be so, as well. 13 See Appendix B for the corresponding table showing preliminary (and inflated) figures for improper payment totals and rates, calculated in 2008 before quality assurance and follow-up audits were completed. 14 This total excludes 268 Study Areas with zero dollars disbursed and 13 with negative amounts disbursed. Such entities were excluded from the sampling frame. The total of negative disbursements represented only two million dollars out of more than four billion disbursed. USAC FINAL REPORT ON FCC OIG 2007–08 DECEMBER 15, 2010 HIGH COST PROGRAM BENEFICIARY AUDITS Page 7 Total 1,743 382 21.9% $4,137,438,575 $113,043,263 2.7% The following table shows the same breakdown of disbursement strata and corresponding improper payment figures with the eight unfinished audits included as a separate stratum unto itself. Stratum (Range Disbursed) Entities in Stratum 15 Sample Size % of Stratum Population Amount Disbursed Estimated IPIA Amount Estimated IPIA Rate $1 to $1m 902 62 6.9% $345,114,750 $23,023,801 6.7% $1m to $5m 643 182 28.3% $1,448,369,599 $42,757,512 3.0% $5m to $10m 126 66 52.4% $851,812,917 $13,386,844 1.6% > $10m 72 72 100% $1,492,141,309 $33,875,106 2.3% Incomplete 8 8 100% $30,848,637 $30,848,637 100% Total 1,751 390 22.3% $4,168,287,612 $143,891,900 3.5% Top Audit Issues Below are tables showing the five issues identified most frequently in the 285 audits with monetary findings as well as the five issues associated with the highest dollar values of improper payments. USAC was associated with only one category, “Eligibility error,” the fourth item in the second table below representing 0.9% of total improper payments. These errors resulted from manual data entry operations in place between 2002 and 2004 involved with establishing the eligibility of companies newly designated as Eligible Telecommunications Carriers (ETC). Subsequent systems improvements have automated these processes and eliminated these errors. Top 5 Audit Issues by Number of Audits Number of Audits Inadequate/missing documentation 84 Line count/loop inaccurate 75 Accounting error 58 Subscriber list error 28 Revenue reporting error 13 Other Reasons 27 Top 5 Audit Issues by Improper Amount Improper Amount Totals Line count/loop inaccurate $ 25,082,339 Inadequate/missing documentation $ 14,430,515 Accounting error $ 13,440,168 Eligibility error $487,536 Subscriber list error $313,712 15 See footnote 14. USAC FINAL REPORT ON FCC OIG 2007–08 DECEMBER 15, 2010 HIGH COST PROGRAM BENEFICIARY AUDITS Page 8 Other findings $671,842 Total $54,426,112 USAC’s experience both reviewing data submitted for High Cost Program support and auditing disbursements has consistently shown these issues to challenge beneficiaries of the program. Indeed, Round 1 of the FCC OIG USF audit program identified substantially the same set of issues as most commonly associated with improper payments. USAC has developed a comprehensive set of measures to reduce improper payments associated with these issues involving outreach, oversight and management, audits, and information technology resources. USAC has expanded outreach in volume and type of activity, enhanced internal controls and data gathering to gain greater visibility into payment operations, calibrated audit and audit follow-up activities to gain greater certainty about beneficiary support, and modernized information technology systems to achieve greater efficiencies and improve reporting capabilities. Financial Impact of OIG USF Audit Program USAC is committed to a robust audit program and reducing improper payments to the lowest possible level. To carry out Round 2 for the High Cost Program of the FCC OIG USF audit program and the associated follow-up work, USAC spent $42.4 million from November 2007 to June 2010. As stated above, these audits identified just over $54.4 million in improper payments; $48.1 million of which is potentially recoverable. As of August 1, 2010, USAC has recovered just over $20.3 million, $9 million is on hold pending guidance from the FCC, and another $7 million is under appeal with USAC and the FCC, with actual recovery contingent on the outcomes of these appeals. The table below presents the dollar values associated with the major improper payment categories. Improper Payments Breakdown Status of Potentially Recoverable Improper Payments Amount Funds recovered $20,326,025 Pending FCC guidance $8,995,853 Appeal w/USAC $6,175,541 Recovery under way 16 $6,081,275 Recovery pending 17 $4,505,163 USAC appeal approved $848,479 16 “Recovery under way” includes audits in which other findings are under appeal. Dollar figures in this category have been finally determined to be improper. 17 “Recovery pending” includes audits in which the recovery amount has been determined and the recovery process is under way. This process includes USAC Board approval of the audit, issuance of a “Recovery Notification Letter” to the beneficiary, a sixty-day window for appeal, and finally recovery within the monthly High Cost disbursement cycle. Most of the items in the category required follow-up audit work to determine the final improper payment amount. USAC FINAL REPORT ON FCC OIG 2007–08 DECEMBER 15, 2010 HIGH COST PROGRAM BENEFICIARY AUDITS Page 9 Appeal w/ FCC $817,194 Pending USAC Board approval of follow-up audit $397,409 Total Potentially Recoverable Improper Payments $48,146,939 Status of Non-recoverable Improper Payments Amount Underpayments—no additional funds disbursed 18 $3,908,741 Underpayments—additional funds disbursed 19 $2,370,432 Total Improper Payments $54,426,112 USAC views any level of improper payments as unacceptable and is committed to reducing or preventing erroneous payments. As reported extensively elsewhere, USAC, working closely with Commission staff, has taken and will continue to take numerous steps to prevent, detect, and recover any USF support found to be disbursed in error. 18 “Underpayments—no additional funds disbursed” includes audits that identified instances of carrier filing errors that would have warranted additional disbursements had the errors not been made. However, discovery of the errors occurred after expiration of the period in which program rules allow corrections to be made to the original filing. As a result, no further payments were made nor were funds recovered. 19 “Underpayments—additional funds disbursed” includes audits that identified filing errors that could be corrected within the period in which program rules allow corrections to be made. As a result, carriers received the additional funds to which their corrected filings entitled them. Additional funds were also disbursed when audits showed that errors in establishing carriers’ eligibility to receive funds caused the improper payments. APPENDIX A Universal Service Administrative Company Final Analysis of Round Two High Cost Program Audit Data Revised Report submitted by A. Richard Bolstein, LLC October 18, 2010 APPENDIX A 1 Executive Summary: This analysis is based on revised final reconciliation data from Round 2 of the audits of the High Cost program. The data was provided to me by USAC on October 11, 2010 and is an update from an earlier version received in August. This report updates the report submitted on August 31, 2010. The sample was designed and selected by the Federal Communications Commission (FCC). The sampling units were Service Area Providers. The sampling design was a stratified simple random sampling of 390 Service Area Providers. It is not known to me what sub-sampling, if any, the auditors performed in the High Cost program to determine the improper payment amount. Also, I have no personal knowledge of the reconciliation process. My analysis and conclusions are based solely on the data provided to me. Eight of the 390 audits have not been completed so it is prudent to compute the IPIA error rate in several ways to obtain upper and lower bounds. A total of $30,848,637 was disbursed to these eight service areas. Method I: Exclude the eight incomplete audits from both the sample and population. The estimated improper payment rate for the reduced population is 2.7% with a margin of error plus or minus 0.6% at the 90% confidence level. Consider two scenarios to extend this result to the full population. 1. Assume this same improper payment rate of 2.7% applied to the incomplete audits. 1 Then the estimated total erroneous payment was $113.9 million with a margin of error less than plus or minus $0.65 million. 2. For a more conservative estimate, assume that the entire $30,848,637 disbursed to the eight incomplete audits was improper. 2 Then the estimated total improper payment amount to the entire population was $143.9 million and the improper payment rate was 3.5%. (Error margins are $0.65 million and 0.6% respectively.) Method II. For a worst case scenario, suppose that the eight incomplete audits are eventually reconciled and it turns out that the full amount disbursed for each of these providers is improper. They would then be retained in the sample and the improper payment rate would be revised to 4.2% with a margin of error plus or minus 0.8% at the 90% confidence level. The estimated total erroneous payment would be revised to $175.2 million with a margin of error plus or minus $1.4 million. Since only one completed audit in the sample resulted in the full disbursement being designated as improper, this worst case scenario appears very unlikely. In conclusion, for now it is best to rely on the two alternatives in Method I. We can be highly confident that the true improper payment rate was between 2.1 and 4.2 percent. 1 This is equivalent to ‘ignoring the non-respondents (incomplete audits).’ 2 This is equivalent to post-stratifying the sample by whether or not the audit was reconciled, and assuming a worst-case 100% improper payment rate among the incompletes. APPENDIX A 2 Method I: Final Second Round Results for High Cost: FY2007 This table excludes the eight incomplete audits from the both the sample and the population. The rate and amounts apply to the reduced population. Stratum (Range Disbursed) Entities in Reduced Population 3 Sample Size Population Amount Disbursed Estimated IPIA Amount Estimated IPIA Rate < $1m 902  62  $345,114,750  $23,023,801  6.7%  $1m to $5m 643  182  $1,448,369,599  $42,757,512  3.0%  $5m to $10m 126  66  $851,812,917  $13,386,844  1.6%  > $10m 72  72  $1,492,141,309  $33,875,106  2.3%  Total 1,743  382  $4,137,438,575  $113,043,263  2.7%  The overall improper payment rate for the reduced population is 2.7%. The margin of error is plus or minus 0.6% at the 90% confidence level. A 90% confidence interval for the true IPIA Rate is 2.1% to 3.3%. To extrapolate this estimate to the full population, assumptions must be made regarding the eight incomplete audits. Consider the following scenarios: 1. Assume that the same 2.7% IPIA rate applies to the $30,848,637 disbursed to the eight incomplete audits. The estimated improper payment amount to the full population is then $113,886,312 ($113,043,463 + $842,849). The margin of error is $646,348. A 90% confidence interval for the true amount is approximately $113.2 million to $114.5 million. 2. For a more conservative estimate, assume that the entire $30,848,637 disbursed to the eight incomplete audits is improper. a. The estimated total improper payment amount to the entire population is then $143,892,100 ($113,043,463 + $30,848,637). A 90% confidence interval for the amount improper is $143.2 million to $144.5. b. The estimated IPIA rate for the entire population is then 3.5% (the estimated total improper payment amount divided by the total disbursed, namely $4,168,287,612). A 90% confidence interval for the IPIA rate is then 2.9% to 4.1%. 3 This total excludes 268 Service Areas with zero dollars disbursed and 13 with negative amounts disbursed. Such entities were excluded from the sampling frame. The total of negative disbursements represented only two million dollars out of more than four billion disbursed. APPENDIX A 3 The IPIA rate for the High Cost program in Round 1 (fiscal year FY2006) was 2.7% with a margin of error plus or minus 2.8%. 4 Under either assumption 1 or 2 above, there is no statistically significant difference between the improper payment rates in the two fiscal years. Method II: Final Second Round Results for High Cost: FY2007 This method assumes the eight incomplete audits are eventually reconciled and the amount disbursed to these providers was 100% improper. It is presented as a worse case scenario only. Stratum (Range Disbursed) Entities in Population Sample Size Population Amount Disbursed Estimated IPIA Amount Estimated IPIA Rate < $1m 903  63  $345,483,183  $27,476,320  8.0%  $1m to $5m 649  188  $1,459,119,405  $80,723,711  5.5%  $5m to $10m 126  66  $851,812,917  $13,386,844  1.6%  > $10m 73  73  $1,511,871,677  $53,605,474  3.5%  Total 1,751  390  $4,168,287,612  $175,192,349  4.2%  The overall improper payment rate for the High Cost program for FY 2007 is estimated at 4.2%. The margin of error is plus or minus 0.8% at the 90% confidence level. A 90% confidence interval for the true IPIA Rate is 3.4% to 5.0%. The total improper payment amount is estimated at $175,192,349 with a margin of error plus or minus $1,357,482. A 90% confidence interval for the true amount is approximately $173.8 million to $176.5 million. Again, there is no statistically significant difference between improper payment rates in rounds 1 and 2. Remark: The overall IPIA error rate and amount are based on the separate ratio estimator, which is somewhat more precise than the combined ratio estimator when sample sizes are sufficiently large (30 or more as a rule thumb) in all strata. The separate ratio estimator first computes the strata IPIA rates by dividing the sample total of improper payments by the sample total amount disbursed. These are shown in the last column of the table. These rates are then multiplied by the known total amount disbursed to the population of all entities in the strata (fourth column of the table) to produce the estimated IPIA amounts for the strata (fifth column of the table.) The sum of these amounts is the separate ratio estimate of total improper payments. This total divided by the known population total disbursements is the separate ratio estimate of the improper payment rate. 4 A. Richard Bolstein, LLC, Revised Report on FY 2006 High Cost Program, August 1, 2009. APPENDIX A 4 The combined ratio estimator, which is preferred when the sample size in one or more strata are small (such as in the Schools & Libraries program for FY2007), does not estimate rates and totals within strata but usually produces a similar overall estimate. It first multiplies the mean improper payment in each stratum by the number of population entities in the stratum (column 2) and sums the results over all strata to obtain a preliminary estimate 5 of the total improper payment amount. The same operation is performed on the mean amount disbursed in the each stratum to obtain an estimate of the total disbursed to the population. (The latter will virtually always differ from the known total amount disbursed shown in column 4.) The quotient of these two quantities: preliminary estimated improper payment amount divided by estimated amount disbursed is the combined ratio estimator of the improper payment rate. Multiplication of this rate by the known total amount disbursed is the combined ratio estimator of the total improper payment amount. 5 This is called a simple expansion estimate. Simple expansion estimates are less precise than ratio estimates for these data because they do not take advantage of the correlation between improper payments and amount disbursed nor do they correct for ‘bad’ samples which do not represent the population fairly with regard to disbursements. APPENDIX B The table below appears on page two of “Preliminary Analysis of 2006-07 High Cost & Schools Audit Data,” August 18, 2008, A. Richard Bolstein. It shows a breakdown by disbursement stratum of preliminary Round 2 OIG USF Audit Program improper payment amounts and rates. III. Results for High Cost based on data submitted July 31, 2008. Stratum Entities in Population Sample Size Amount Disbursed Amount Erroneous Error Rate < 1m 903  63  $384,575,402.00  $101,015,991.45  26.3%  1m to 5m 649  188  $1,395,773,396.55 $442,171,256.34  31.7%  5m to 10m 126  66  $870,603,133.91  $234,128,612.91  26.9%  > 10m 73  73  $1,511,871,677.00 $193,007,728.58  12.8%  Overall 1751  390  $4,162,823,609.46 $970,323,589.27  23.3%  The overall erroneous payment rate for the High Cost program is estimated at 23.3%, although this rate is likely to decrease once additional audits are complete. The standard error of the overall error rate is 1.4%, resulting in a margin of error of plus or minus 2.3% at the 90% confidence level. The erroneous payment rate estimated from the 2005-06 audits of 65 entities was 15.1% with a margin of error plus or minus 9.7%. There is no statistically significant difference between the erroneous payment rates in the two fiscal years.