NANC
2012
PA
Performance 
Evaluation 
Report
Prepared by the
Numbering Oversight Working Group (NOWG)
June 20, 2013
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
2
Table of Contents
Executive Summary…………………………………………………………………….3
Section 1.0 Performance Review Methodology .............................................4
Section 2.0 PA Reports .................................................................................6
Section 3.0 Customer Focus / Issues Log .....................................................8
Section 4.0 2012 PA Performance Survey Results .......................................9
Section 5.0 Operational Review ..................................................................13
Section 6.0 Pooling Administration System (PAS).......................................15
Section 7.0 Change Orders .........................................................................17
Section 8.0 National Pooling Website..........................................................18
Section 9.0 p-ANI / RNA..............................................................................19
Section 10.0 Conclusion and Recommendation............................................23
Section 11.0 Acknowledgements & NOWG Participants...............................24
Section 12.0 List of Appendices ....................................................................26
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
3
Executive Summary 
The FCC and the North American Numbering Council (NANC) charged the 
Numbering Oversight Working Group (NOWG) with compiling and delivering an 
annual performance report of the Pooling Administrator (PA).  The PA’s annual 
performance assessment is based upon:
? 2012 Performance Feedback Survey for PA and RNA (Routing Number 
Administrator)
? Written comments and reports 
? Annual Operational Review 
? NOWG observations and monthly interactions with the PA  
The PA’s rating for the 2012 performance year was determined by the NOWG to 
be Exceeded.  This rating is defined below:
EXCEEDED
Exceeded performance requirement(s) 
? Provided excellence above performance requirements and exceeded 
expectations
? Performance was well above requirements  
? Decisions and recommendations exceeded requirements and expectations
The 2012 survey results revealed a high level of satisfaction that respondents 
attributed to the professionalism, responsiveness, and expertise exhibited by the 
PA and RNA personnel throughout 2012.
In 2012, the PA consistently exceeded its required responsibilities.  Highlights 
included:
? The RNA successfully and seamlessly transitioned from the interim RNA 
(IRNA) to the permanent RNA function. 
? Routing Number Administration System (RNAS) was made available for 
public use on time on March 19, 2012.  Scheduled availability for RNAS 
was maintained at 100% throughout 2012, which exceeded the contract 
performance metric of 99.9%.
? During Hurricane Sandy, the PA kept the Help Desk open over the 
weekend to assist service providers as needed.
? The PA initiated a special project to request donations for 66 rate centers 
being changed from Excluded to Optional.  The PA was able to obtain 
donations for 58 of the 66 rate centers, thereby potentially saving the 
opening of 58 NXX codes.
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
4
Section 1.0 Performance Review Methodology
The annual PA Performance Evaluation is a summary of significant events that 
were accomplished during the 2012 performance year.  In addition to the annual 
performance review survey process, the NOWG’s interactions with the PA 
included the following:
? Annual operational review 
? Change Order review process 
? PA NANC reports 
? Monthly NOWG/PA status meetings 
? Interaction with the industry 
The implementation of the permanent p-ANI administration functions (Routing 
Number Administrator - RNA) occurred on March 19, 2012.  Due to the fact that 
the user communities are different for the RNA functions and the PA functions, 
separate performance surveys were conducted. The combined quantitative 
results have been included in the PA Performance Review Methodology for 
2012. The methodology used by the NOWG in weighting the quantitative 
responses from the surveys is as follows:
Each rating category was assigned a point value (Exceeded = 5, More 
Than Met = 4, Met = 3, Sometimes Met = 2, Not Met =1).  The NOWG 
multiplied the corresponding point value by the number of responses in 
that category and then divided the results by the total number of 
respondents to the question.
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
5
The following chart provides the definition of each rating category: 
Satisfaction Rating Used when the PA...
EXCEEDED
Exceeded performance requirement(s) 
? Provided excellence above performance requirements and exceeded 
expectations
? Performance was well above requirements  
? Decisions and recommendations exceeded requirements and 
expectations
MORE THAN
MET
Met and often went beyond performance requirement(s)
? Provided more than what was required to be successful
? Performance was more than competent and reliable 
? Decisions and recommendations usually exceeded requirements and 
expectations
MET
Met performance requirement(s)
? Met requirements in order to be considered successful
? Performance was competent and reliable
? Decisions and recommendations were within requirements and 
expectations 
SOMETIMES MET
Sometimes met performance requirement (s)
? Was inconsistent in meeting performance requirements
? Performance was sometimes competent and reliable
? Decisions and recommendations were sometimes within requirements
NOT MET
Did not meet performance requirement(s). 
? Administrative tasks and objectives were not within requirements in 
order to be considered successful
? Performance was unreliable and commitments were not met
? Decisions and recommendations were inconsistent with requirements
N/A
Did not observe activity or does not apply to service provider/regulator
The NOWG will present preliminary findings to the FCC and the PA.  The final 
report will be presented to the NANC for endorsement and then forwarded to the 
FCC.
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
6
Section 2.0 PA Reports 
2.1 PA Annual Report 
The annual report prepared by the PA is a requirement in the Pooling 
Administrator Technical Requirements document.  The status of pooling and 
Pooling Administration should be reported in the annual report.  Review of this 
annual report is part of the NOWG's annual performance review process.  At a 
minimum, the annual report is required to contain the following information:
? Brief description of the PA 
? Highlights/significant milestones reached during the previous year 
? Identification of existing and potential pooling areas 
? Aggregated total, by pool, of the service providers participating in the 
pooled area 
? Forecast results, as well as a review of forecasts vs. actual past block 
activations 
? System and performance metrics 
? Status of required transferable property 
? Industry issue identification/feedback 
? Volume of reports produced aggregated by regulatory agency, NANC, 
NANPA, and service providers 
? Additional informational offerings
Prior to this year’s on-site operational review, the PA provided the NOWG with an 
opportunity to review the draft copy of the 2012 Annual Report.  During the on-
site operational review in Concord, California in March 2013, the PA staff 
reviewed the 2012 highlights which were also included in the annual report.
Overall, the annual report provides a comprehensive snapshot of pooling and the 
PA for 2012.  The PA 2012 Annual Report was filed with the FCC and is posted 
for general availability on the PA’s website at www.nationalpooling.com.
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
7
2.2 PA NANC Report 
The PA reported its monthly numbering activities to the NANC and the NOWG.  
Additionally, the PA made presentations at four NANC meetings in 2012, 
reporting the status of thousands-block pooling administration, p-ANI activities, 
and events affecting the performance of the PA, which included the following:
? Volume of pooling assignments, donations and applications processed 
? Codes opened to replenish pools and establish LRNs 
? Pools with less than six months inventory vs. forecasts 
? Summaries of monthly reports to the FCC 
? Number of blocks reclaimed 
? Percent availability of PAS and RNAS
? Status and implementation of change orders 
? Updates to PAS
? Updates to the PA website 
? p-ANI summary
? Results of the 2012 Pooling Administration Survey
2.3 NOWG Monthly Reports
Throughout 2012, the NOWG and PA followed a standing agenda during the 
scheduled monthly calls.  The PA provided monthly performance reports that 
were reviewed during the monthly calls with the NOWG.  The quality and content 
of these reports provided the NOWG with valuable insight into the operations of 
the PA. Some of the standing agenda topics include:
? Rate Center Pooling Status
? Application Processing
? p-ANI
? Customer Focus 
See Appendix A for 2012 PA / NOWG Standing Agenda
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
8
Section 3.0 Customer Focus / Issues Log 
Customer Focus
In 2012, at the monthly NOWG/PA meetings, the PA provided a report on 
customer focus items that they executed to help service providers and regulators.  
Customer focus items cover both contractual and non-contractual initiatives 
related to customer service.
There were 72 customer focus items from January 2012 through December 
2012.  Customer focus items included, but were not limited to, the following:
? Provided assistance to service providers on block donations
? Provided time-saving and special reports for both service providers and 
regulators 
? Provided education and assistance on p-ANI resources
? Provided service provider and regulator training
? Provided Help Desk availability over Hurricane/Superstorm Sandy 
weekend
PA/NOWG Issues Tracking Log
The tracking log is used to document ongoing issues.  The log, which includes 
metrics on the create date, issue name, summary and status, keeps the NOWG 
informed until issues are brought to resolution/closure.
There were no new issues added to this log in 2012.   There was one issue 
closed in 2012 (INC Issue 715) regarding the donations over 10 percent 
contaminated. 
See Appendix B for 2012 PA / NOWG Issues Tracking Log
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
9
Section 4.0 2012 PA Performance Survey Results
4.1 PA Survey Ratings – Quantitative Analysis
The PA 2012 Performance Surveys were completed by a total of 98 respondents:
68 Industry and Other Respondents
30 Regulator Respondents
The results are as follows:
? Pooling Administrator (Section A)
o There were 4 questions in this section to which respondents provided 
the following aggregated response ratings:
? 103 as Exceeded
? 108 as More than Met
? 35 as Met 
? 2 as Sometimes Met
? Pooling Administration System (PAS) (Section B)
o There were 3 questions in this section to which respondents provided 
the following aggregated response ratings:  
? 103 as Exceeded
? 88 as More than Met
? 63 as Met
? 1 as Sometimes Met
? PA Website (Section C)
o There was 2 questions in this section to which respondents provided 
the following aggregated response ratings:  
? 60 as Exceeded
? 76 as More than Met
? 50 as Met 
? 4 as Sometimes Met
? Miscellaneous Pooling Administration (PA) Functions (Section D)
o There were 4 questions in this section to which respondents provided 
the following aggregated response ratings:  
? 87 as Exceeded
? 107 as More than Met
? 90 as Met
? 4 as Sometimes Met 
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
10
? Overall Assessment of Pooling Administrator (PA) (Section E)
o There was 1 question in this section to which respondents provided the 
following aggregated response ratings:  
? 37 as Exceeded
? 43 as More than Met
? 16 as Met 
See Appendix C for 2012 PA Survey Metrics and Bar Charts, and Appendix D for 
2012 PA Survey Cover Letter and Performance Survey
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
11
4.2 PA Survey Written Comments
The survey allowed respondents the opportunity to provide detailed written 
comments regarding their satisfaction or dissatisfaction with the PA’s 
performance in 2012.  The majority of comments were positive, with only a few 
containing suggestions for areas of improvement.  The NOWG reviewed all 
comments to determine if there was a common theme substantiated by multiple 
respondents.
The following is a summary of written comments that were provided by survey 
respondents:
? Outstanding praise for the PA staff was a consistent theme throughout the 
survey:
o Provides excellent support, assistance, and technical expertise
o Always prompt, helpful, and courteous
o Professional, friendly, and responsive
o Willing to go the extra mile to provide top notch service to their 
customers
? Comments suggesting improvements were mostly isolated.  Comments 
pertained to:
o Process clarification questions
o Suggestions for PAS and website enhancements 
The NOWG concluded that the written comments were not indicative of any 
consistent performance issues, and in many cases provided significant praise for 
individual PA staffers.  Samples of the written comments received are provided 
below:
“The PA administrators have exceeded my expectations in 2012.  Their 
execution to approve, deliver, and respond according to requests and 
questions always exceeded the standard interval.  They perfom quality 
work regardless of volume or questions.”
“Staff at the PA has been great to deal with.  I appreciate the time they 
take to explain often complex issues in a detailed manner.  And that they 
are knowledgeable in their field.”
“PA staff is very knowledgeable and readily provides assistance.”
“The PA is extremely helpful when I need answers.  All the PA staff seem 
to do their jobs with good cheer.  A+”
“They are always helpful, professional and patient with any questions I 
bring to them.”
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
12
“Always a pleasure to work with!  When issues do arise, the PA staff 
handles them quickly and always provides assistance with understanding 
why the issue occurred.  Especially appreciate all the work that the PA 
does to find new code holders.”
“Responses to inquiries and problems are always handled promptly and 
accurately.”
“The PAs have been great to work with as they respond promptly, 
courteously and professionally.”
“All of my routine work experiences with Pooling Administration staff have 
been positive.  I get prompt responses to emails or calls, and, any 
questions regarding pooling procedures have been explained clearly, 
usually by referencing the exact section of the TBPAG, as appropriate.”
See Appendix E for 2012 PA Survey Respondents and Appendix F for 2012 PA 
Survey Respondents’ Comments
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
13
Section 5.0 Operational Review
The NOWG members met with the PA representatives in Concord, California on 
March 13 and 14, 2013 to conduct the annual on-site operational review.  During 
this review, the PA staff presented an overview and highlights of 2012 activities.  
The presentation included the Pooling Administration operations, NANP resource 
trending, external relations and training, change orders, pooling quality 
assurance, and regulatory compliance.   
The PA staff also presented the status of escalations, industry forum 
participation, technical operations, pooling reports, special projects, and p-ANI 
administration.
Key highlights presented to the NOWG included: 
? Total Applications Processed in 2012: 
o Issued 130,407 Part 3s
o Assigned 47,074 thousands blocks 
o Opened 2,588 NXX codes 
o 99.998% of all applications processed within seven calendar days 
or less (requirement is 97%) 
? Customer Support Desk received 1,895 calls; 100% answered within one 
business day 
? Trouble Tickets: 
o Opened three new trouble tickets in 2012 to correct PAS system 
errors
o Closed two new trouble tickets in 2012 to correct PAS system 
errors
? Reclaimed 15 blocks in 2012
? Average score of 4.7 out of 5.0 on the annual survey conducted by the PA
? Training:
o Five state educational sessions
? Reports:
o 658 reports to the FCC, states, NANC, NANPA, and SPs
o 105 required Contract Data Requirements List (CDRL) reports 
submitted on time
o 59 additional contract-required reports submitted on time
o All ad hoc reports done in less than one business day (Contract 
allows three business days)
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
14
? Implemented three NPA overlays in PAS in 2012
? Two rate center consolidations implemented in PAS in 2012
? Implemented four change orders in 2012
? Participated in 55 industry meetings
? No formal complaints made to PA
? Met or exceeded all system requirements with 99.998% availability –
exceeding the contract performance metric of 99.9% availability
? Pooling Help Desk available over Hurricane/Superstorm Sandy weekend
Appendix G for 2012 PA Operational Review Presentation and Appendix H for 
2012 PA Highlights
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
15
Section 6.0 Pooling Administration System (PAS)
The Pooling Administration System (PAS) was available 99.998% of the time in 
2012 – even during two system builds to implement four change orders.  The 
only unscheduled downtime for the year totaled 12 minutes and 3 seconds out of 
8,784 possible hours in 2012.  The unscheduled PAS unavailability occurred 
during routine database maintenance for a time zone change on the database 
server. Neustar encountered a problem with the PAS failover from Sterling to 
Charlotte which caused PAS to be inaccessible during non-core business hours 
until the failover was reversed and the application was restored to Sterling.
Out of the possible 24 hours of scheduled downtime allowable by the contract, 
the PA used only 2 hours and 52 minutes of scheduled downtime as a result of 
11 approved scheduled maintenance events. The PAS database was upgraded 
to the latest major release of Oracle, and the PAS firewalls were upgraded.  PAS 
users experienced very limited PAS unavailability as a result of these builds.  
Additionally, the PA completed disaster recovery testing during the weekend of 
October 26-28, 2012 with no downtime.
Overall, the industry was satisfied with the performance of PAS.  This is apparent 
in the following sample of comments received on the surveys:
“I've never had any issues when using PAS to request resources, to make 
donations or inputting forecasts.”
“As a state regulator did not use PAS, but did get companies with late Part 
4s to submit them and they were always entered in PAS the same day.”
“All PAS maintenance and availability to the NAS was always present for 
acess when needed.  No down time was experienced in 2012 that 
prohibited my use when accessed.  The PAS data was accessible and 
accurate when I referenced this data.  The ease and format of data in PAS 
makes for a great user friendly experience.”
“No issues with PAS in 2012 - system worked very well.”
“PAS is still a very user friendly tool and all the changes have made our 
jobs easier and efficient.”
There were also some enhancement suggestions and questions regarding the 
system or process:
“Perhaps some enhancements could be made in PAS to allow the ability 
to do multiple block disconnects on a Part 1A, instead of having to do 
them individually; this would be especially helpful when returning multiple 
blocks within the same code.”
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
16
“There were a couple of times when a high demand rate center did not 
have any available number blocks and a new code had to be opened, 
which impacts our ability to serve our customers in a timely manner. 
Previously, PAS would email users regarding rate centers that needed 
replenishement and waive the months to exhaust so anyone could open a 
new code and add blocks. Why are we no longer notified of rate centers in 
dire need of replenishment so we can avoid this?”
“PAS has improved a lot from the day one I have used it. Keep up the 
good work and provide more useful and easy features.  Under User Profile 
/ Edit User Profile / Additional Contacts: I know that you can only put 3 
contacts for all Part 3s, Part 4s Reminder, Pooling Notifications, etc.    ** 
Expand the contacts to at least 5 instead of only 3 contacts? ** On
Subscription choices, is there a way that a contact person can be provided 
a specific OCN(s) for Part 3s instead of all OCNs?...”
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
17
Section 7.0 Change Orders 
In 2012, the PA filed two new change orders with the FCC.  The PA change 
order process complies with FCC contractual requirements.  The 2012 change 
orders included:
? INC Issue 715 – Update TBPAG for retrieving a block donated/returned in 
error (Change Order 23)
? Enhancement of the FTP Interface with the Pooling Administration System 
(Change Order 24)
In 2012, the PA completed the implementation phase of four change orders 
approved by the FCC.  
See Appendix I for 2012 PA Change Order Matrix Log
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
18
Section 8.0 National Pooling Website 
The website maintained by the PA provides number pooling information to 
service providers and regulatory agencies.  In 2012, the PA continued to keep 
the information current on the website.
The training videos on the website continue to be viewed regularly.  In 2012, the 
PA added four new training videos to the website:
? New to Pooling Quick Start 
? Mass Modifications 
? Change Order 20 
? How to Complete the Months-to-Exhaust and Utilization Certification 
Worksheet – TN Level 
In addition, the PA assisted NANPA with developing and posting its first training 
video, “How to Request an Initial Central Office Code in a Pooling Area.”
Overall, the industry was satisfied with the website.  This is apparent in the 
following sample of comments received on the surveys:
“PAS has improved a lot from the day one I have used it. Keep up the 
good work and provide more useful and easy features.   The PAS webite 
is user friendly and easy to navigate and provide entries to 
complete/submit requests and view information.”
“The PA Website contains a wealth of statistical and reference 
information, which we access frequently during day-to-day operations.”
“The PA staff was very helpful in helping me access information via 
pooling reports that I was previously unfamiliar with.”
There were also some enhancement suggestions regarding the website:
“In general, I believe the PA website is organized well.  I can navigate to 
the reports and information I want well enough.  However, I've had the 
eyes of new staff in the past 6 months, and the site was found to be 
somewhat cluttered on the home page.  The quick links at the home page 
was not found to be recognized as links due to the check boxes.  This was 
initially taken as some sort of check list.  This aside, I believe the website 
adequately contains all the information required when using PAS 
resources.”
“Web site did not have a contact list that is Printer friendly. Printing cut off 
half of the page.”
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
19
Section 9.0 p-ANI (pseudo-Automatic Number Identification) / RNA 
(Routing Number Administrator)
After serving as the Interim Routing Number Administrator (IRNA) for p-ANI in 
2011, the PA assumed the responsibility as the RNA in 2012.  The PA completed 
development of the Routing Number Administration System (RNAS) and fully 
implemented their transition to the functions of permanent RNA by the “go-live” 
date of March 19, 2012.
The PA’s RNA functions performed in 2012 included, but were not limited to, the 
following:
? Developed, tested, and maintained RNAS and the p-ANI website
? Established the RNAS inventory of assigned and available non-dialable p-
ANIs from data received from assignors and assignees of all p-ANI 
assignments already in existence
? Reviewed and compiled data obtained from multiple entities
? Worked with service providers to reconcile data discrepancies
? Conducted five web-based overview sessions on the new website and the 
new RNAS for service providers and regulators
? Trained RNAS users on the types of documentation required to assure 
that applicants were eligible to offer services in the areas in which they 
requested p-ANIs
? Continued to participate in meetings and work with the Industry 
Numbering Committee (INC) and Emergency Services Interconnection 
Forum (ESIF)
From the time that RNAS went live on March 19, 2012, there were no instances 
of unscheduled downtime.  As a result, RNAS scheduled availability in 2012 was 
100%.
Overall, the industry was satisfied with the PA’s RNA functions.  This is apparent 
in both the quantitative results and written comments on the NOWG’s RNA 
Survey.  Following is a sample of comments received on the survey:
“They are very responsive, efficient, organized and helpful.  No 
complaints. Elcellent at assisting folks understand the new processes and 
procedures.  Always available to answer questions and help when 
needed!”
“Our interactions with the administrators have been very positive, they are 
thorough, polite, and very responsive. Florence Weber exceeded 
expectations when we needed emergency pANIs for a Superstorm Sandy 
situation.”
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
20
“Diane and Florence are extremely helpful in making sure that we have 
the necessary information and correct NENAs and OCNs for all of the 
carriers.  They have been nothing but extremely helpful. It is a pleasure 
working with them.”
“The RNA handled the rollout expertly and assisted SP whenever 
necessary.  They should be commended for the way they handled the 
HUGE cleanup of dupicate assignments that were not foreseen until 
implementation.  It was a large workload and very confusing for many.”
“Florence Weber and Dianne Calhoun went above and beyond in 2012;
they were particularly patient and helpful in working with service providers 
to get the RNAS database populated and in resolving overlapping pANI 
range issues.”
“RNA has been very helpful any time I have ever had questions or issues.”
The survey also contained some suggestions for RNAS enhancements:
“We would like the function to be able to query by PSAP name.”
“Excellent service.  No complaints. No complaints.  If there were every any 
updates to the website, I'd request that our NENA ID and OCN be tied to 
our login name so that it would autofill so it's one less field we have to fill 
in.”
“We would like the ability to upload or attach an FCC license to an 
application instead of sending a separate email and referencing the 
tracking number of the application in the email.”
“I would like to be able to query a pANI range, not just a single pANI at a 
time in the pANI Lookup screen. I would like to see the assignment date in 
the pANI Lookup results screen.”
See Appendix J for 2012 RNA Survey Respondents and Appendix K for 2012 
RNA Survey Respondents’ Comments
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
21
RNA Survey Ratings – Quantitative Analysis
The following quantitative results were included in the PA Performance Review 
Methodology for 2012. 
The RNA 2012 Performance Survey was completed by a total of 11 respondents:
8 Industry and Other Respondents
3 Regulator Respondents
The results are as follows:
? Routing Number Administrator (Section A)
o There were 3 questions in this section to which respondents provided 
the following aggregated response ratings:
? 18 as Exceeded
? 1 as More than Met
? 2 as Met 
? 2 as Not Met
? Routing Number Administration System (RNAS) (Section B)
o There were 3 questions in this section to which respondents provided 
the following aggregated response ratings:  
? 17 as Exceeded
? 3 as More than Met
? 6 as Met
? 3 as Not Met
? RNA Website (Section C)
o There were 2 questions in this section to which respondents provided 
the following aggregated response ratings:  
? 8 as Exceeded
? 7 as More than Met
? 4 as Met 
? 1 as Not Met
? Miscellaneous RNA Functions (Section D)
o There were 3 questions in this section to which respondents provided 
the following aggregated response ratings:
? 14 as Exceeded
? 2 as More than Met
? 3 as Met
? 3 as Not Met
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
22
? Overall Assessment of the RNA (Section E)  
o There was 1 question in this section to which respondents provided the 
following aggregated response ratings:
? 7 as Exceeded
? 2 as Met
? 1 as Not Met 
See Appendix L for 2012 RNA Survey Metrics and Bar Charts and Appendix M
for 2012 RNA Survey Cover Letter and Performance Survey
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
23
Section 10.0 Conclusion and Recommendation
The NOWG based its 2012 PA Performance Evaluation Rating on PA and RNA
documentation, information collected, and observations throughout the review 
period year.  Although emphasis continues to be given to the numeric and written 
survey comments, survey respondents may not always be familiar with the 
activities of the PA and RNA that occur “behind the scenes.”  For the overall 
2012 performance evaluation rating, the NOWG considered PA activities that 
included interaction with the NOWG and the NANC, and active participation at 
INC and other industry forums.
The survey results revealed a high level of client satisfaction with the continued 
professionalism and expertise exhibited by the PA personnel when performing 
their PA and RNA duties.  The PA continued to demonstrate their ability to 
handle the large volume of block applications, while simultaneously completing 
special projects and the launch of the new RNA and RNAS.
In reviewing the rating criteria for the PA, the results of the data analysis yielded 
an “Exceeded” rating for the 2012 performance year.  
The NOWG makes the following recommendations for the PA’s consideration in 
2013:
? Ongoing review of internal training processes to ensure that consistency in 
understanding the processes and responding to service providers and 
regulators is communicated to the PA and RNA personnel.
? Ongoing review of the PA and RNA website to ensure accuracy and 
timeliness of data.
? Continue to consider process or system enhancements suggested by 
regulators and service providers.
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
24
Section 11.0 Acknowledgements & NOWG Participants
The NOWG wishes to thank the following Neustar PA employees for their 
contributions throughout the year, for assisting the NOWG during the annual 
operational review, and with participating in the NOWG’s monthly meetings.
List of PA / RNA Participants
Bruce Armstrong
Jan Connally
Tara Farquhar
Dara Flowers
Evelyn Freeman
Kevin Gatchell
Linda Hymans
Wayne Louie
Cecilia McCabe
Amy Putnam
Shannon Sevigny
Florence Weber
Gary Zahn
PA/p-ANI Help Desk and Pooling/p-ANI Administrators
The following working group members have participated in varying degrees by 
attending NOWG meetings throughout the year, attending the annual operational 
review, and contributing to the development of this document.
List of NOWG Participants:
Company Participant
AT&T Linda Richardson
CenturyLink Jan Doell
Cox Communications Beth O'Donnell
EarthLink Business Linda Peterman
Pennsylvania PUC Christopher Hepburn
Sprint Nextel Rosemary Emmer
Sprint Nextel Karen Riepenkroger
T-Mobile USA Natalie McNamer
Verizon Communications Laura Dalton
Verizon Wireless Dana Crandall
Windstream Communications Michelle Bowyer
XO Communications Ruben Galvan
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
25
The NOWG would like to acknowledge and thank the following FCC members 
who attended the PA 2012 Operational Review via the conference bridge / online 
meeting link:
Gary Remondino Contracting Officer’s Technical Representative
Myrva Charles Alternate Contracting Officer’s Technical 
Representative
2012 PA Performance Evaluation Report
June 20, 2013                                                                                                              
26
Section 12.0 List of Appendices
Appendix A 2012 PA / NOWG Standing Agenda
Appendix B 2012 PA / NOWG Issues Tracking Log
Appendix C 2012 PA Survey Metrics and Bar Charts
Appendix D 2012 PA Survey Cover Letter and Performance Survey 
Appendix E 2012 PA Survey Respondents 
Appendix F 2012 PA Survey Respondents’ Comments
Appendix G 2012 PA Operational Review Presentation
Appendix H 2012 PA Highlights
Appendix I 2012 PA Change Order Matrix Log
Appendix J 2012 RNA Survey Respondents
Appendix K 2012 RNA Survey Respondents’ Comments
Appendix L 2012 RNA Survey Metrics and Bar Charts
Appendix M 2012 RNA Survey Cover Letter and Performance Survey