INTRODUCTION

Mr. Chairman and Members of the Subcommittee, thank you for the opportunity to discuss the Office of Inspector General’s (OIG) work regarding the Veterans Benefits Administration’s Systematic Technical Accuracy Review (STAR) program.  I am accompanied by Mr. Larry Reinkemeyer, Director of the OIG’s Kansas City Audit Operations Division.

The OIG is committed to proactively reviewing the effectiveness of key management controls to assure the accomplishment of mission critical service responsibilities to veterans, such as the delivery of accurate and timely disability benefits.  Further, we strive to focus our efforts on identifying control weaknesses before they escalate into significant problems.  Over the past 2 years, we issued audit reports covering aspects of VBA’s quality assurance process – the STAR, Rating Consistency Review, and Site Visit programs.  In March 2009, we issued the Audit of Veterans Benefits Administration Compensation Rating Accuracy and Consistency Reviews (Report No. 08-02073-96).  In May 2009, we issued the Audit of Veterans Benefits Administration Compensation and Pension Site Visit Program (Report No. 08-02436-126). 

Also in fiscal year (FY) 2009, we established a Benefits Inspection Division to provide recurring oversight of regional offices by focusing on disability compensation claims processing and performance of Veterans Service Center (VSC) operations. 

STAR PROGRAM

Improving the quality of rating decisions, which includes accuracy and consistency of disability compensation claim rating decisions, is among VBA’s highest priorities.  The STAR program is a key mechanism for evaluating regional office performance in processing accurate benefit claims for veterans and beneficiaries.  The STAR program provides a comprehensive review and analysis of compensation rating processing associated with specific claims or issues.  VBA’s FY 2008 and FY 2009 goal for compensation claim ratings was a 90 percent accuracy rate.  STAR reviewers select a number of claims to review from VA regional offices nationwide and use a checklist designed to facilitate a consistent, structured review and classify errors into three categories: benefit entitlement errors, decision documentation/notification errors, and administrative errors. 

Results

During the 12-month period ending February 2008, STAR reviewers found that VBA staff accurately rated about 87 percent of the claims (approximately 767,000 claims of 882,000 claims reviewed).  We reviewed a random sample of STAR reviewed claims and found additional errors affecting veterans’ benefits that STAR reviewers did not identify.  As a result, we projected VBA’s accuracy rate for claims reviewed was only 78 percent.

The STAR program does not provide a complete assessment of compensation claim rating accuracy.  We found that VBA officials excluded brokered claims from STAR reviews.  Brokered claims are those assigned to one regional office but sent to another office to be rated.  When we combined the results of our review of brokered claims with our review of STAR reviewed claims, we projected that about 77 percent (approximately 679,000 claims) of compensation claims were accurate and VBA’s reported error rate was understated by approximately 10 percentage points.  This equates to approximately 88,000 additional, or about 203,000 total, claims where veterans’ monthly benefits may be incorrect.  This difference occurred because STAR reviewers did not identify all errors, and VBA officials excluded a significant number of compensation claim ratings from review.  Additionally, VBA officials had not implemented an effective formal training program for the STAR reviewers.

We identified five areas in our review of the STAR program where VBA needed to take action to improve its quality assurance and oversight efforts.

• STAR reviewers did not identify some errors because they either did not thoroughly review available medical and non-medical evidence or identify the absence of necessary medical information.  STAR reviewers also misclassified some errors in a way that resulted in the error not being counted against the regional office’s accuracy rate.  In these cases, they recorded a “comment” instead of an “error” although the errors clearly affected veterans’ benefits entitlements and should have been counted as errors. 
• STAR management required regional offices to report quarterly on actions taken to correct benefit entitlement errors but they did not require or follow up to ensure regional offices took corrective actions on comments made by STAR reviewers.  Ultimately, they relied on the regional office to take corrective action on all issues identified whether the STAR reviewer identified an error or a comment.  From our sample, we identified 33 compensation claim ratings where STAR reviewers made comments instead of reporting issues as errors.  At least six of the comments related to issues that could affect the veterans’ benefits.  We found that regional office staff had not corrected any of the six comments potentially affecting the veteran’s benefits. 
• VBA officials excluded brokered claims from STAR reviews.  VBA officials told us STAR reviewers do not review brokered claims because the STAR program’s primary focus is on assessing and reporting rating accuracy for each of the regional offices.  Since two or more regional offices are involved in brokered work, VBA officials stated it would be difficult to assign responsibility for the rating accuracy to one specific regional office.  Thus we found that STAR management was replacing brokered claims selected for review with non-brokered claims.  We reviewed a sample of brokered claims that were not evaluated and found a 69 percent accuracy rate.   
• The STAR reviewers did not ensure regional offices submitted all of the selected compensation claim ratings to the STAR program for review.  Regional offices did not submit about 600 (7 percent) of the approximately 9,000 requested claim ratings for the 12-month period ending February 2008.  We reviewed 54 of the 600 pending requests and identified 12 (22 percent) benefit entitlement errors.  STAR management relies on regional office staff to submit the requested claims and only follows up with the regional offices that do not submit any requested claims for a given month.  A STAR manager stated they did not have sufficient resources to follow up on individual claims that regional office staff do not submit.  Therefore, regional office staff can cherry pick claims because STAR reviewers do not reconcile the claim requests.  This control weakness provided opportunities for regional office staff to withhold claims if they suspect the claims to have errors. 
• STAR reviewers are not required to complete formal training on an annual basis.  The reviewers met infrequently to discuss issues, and had no set formal training schedule or requirements.  Regional office staffs that prepare and complete ratings and awards for compensation claims are required to achieve 80 hours of training per year to stay competent on laws, policies, and processes on rating-related issues.  However, the STAR program manager stated their program workload requirements currently do not allow for the amount of training time necessary, yet agreed the program staff could benefit from formal training. 

We recommended that VBA:

• Ensure STAR reviewers evaluate all documentation related to the claim selected for review.

• Establish a requirement that all STAR reviewer comments receive a second review to make sure the comment was not a benefit entitlement error.

• Establish procedures to review brokered claims as part of the STAR program.

• Enforce procedures requiring regional offices to submit all requested claims to the STAR program office for their review or submit written justification to the STAR program’s office requesting to exclude the claim from the review.  

• Establish minimum annual training requirements for STAR reviewers that are comparable to regional office rating staff training requirements. 

VBA agreed with all five recommendations and reported that it had completed actions to implement the recommendations in our March 2009 report. 

RATING CONSISTENCY PROGRAM

To address variances in compensation rating decisions within individual VA regional offices and across the Nation, VBA developed a rating consistency review plan that included metrics to monitor rating consistency and a method to identify variances in compensation claim ratings.  VBA’s plan would identify unusual patterns of variance in claims, and then review selected variances to assess the level of decision consistency within and between regional offices.  However, as of March 2009, VBA had not fully implemented its rating consistency review plan. 

Results

In FY 2008, VBA officials identified 61 diagnostic codes where offices appeared to be making inconsistent decisions in the evaluation of a granted claim or whether offices granted or denied the claim.  VBA officials planned to conduct 22 of the 61 reviews in FY 2008 consisting of 20 grant/denial rate and 2 evaluation reviews.  However, they only initiated two grant/denial rate reviews and did not complete either review until December 2008.  Additionally, VBA did not initiate either of the evaluation reviews designed to reduce variances in compensation claim ratings.  In March 2010, VBA informed us that insufficient staffing prevented them from completing any consistency reviews in FY 2009.  However, they have now hired the necessary employees.  The first FY 2010 review was completed in January 2010; a second review started in February 2010.  

We identified three areas that impaired VBA’s ability to fully implement its rating consistency review plan.

• VBA did not have an accurate universe of claims to review.  From April through May 2008, VBA officials encountered a delay in completing planned consistency reviews because the universe of claims included completed claims with diagnostic codes outside the scope for the requested period.  VBA officials notified the Office of Performance, Accountability, and Integrity of the data integrity issue and worked with Compensation and Pension (C&P) Service staff to identify an appropriate universe by June 2008.  
• Data captured by STAR reviewers contained veterans’ personally identifiable information (PII).  In July 2008, VBA officials stopped all consistency reviews until they could take action to secure PII in STAR’s electronic records used to capture and review data and analyze the results.  VBA officials took the necessary actions to correct this condition and secured the database in December 2008. 
• VBA officials did not assign a sufficient number of staff to accomplish consistency reviews.  The STAR program office was authorized 26 reviewers for rating accuracy and consistency reviews, and had 18 reviewers on board.  However, only 8 of the 18 reviewers conducted consistency reviews.  The eight reviewers tasked with completing reviews of consistency were not sufficient to complete the assigned work.  A STAR manager estimated that approximately 12 reviewers were needed to complete the 22 planned reviews.  However, in addition to the consistency reviews, STAR management also assigned these same eight reviewers to conduct at least seven special focus reviews involving thousands of compensation claim ratings.  We concluded that the assigned staffing was insufficient to complete this planned work.

As part of future rating consistency review plans, inter-rater reliability reviews (IRRR) should be included, along with the annual rating consistency reviews.  In July 2008, C&P Service staff conducted an IRRR and found that 76 (31 percent) of the 246 participants incorrectly rated a relatively simple back strain claim.  In August 2008, C&P Service staff conducted another IRRR and found that 30 (13 percent) of the 247 participants incorrectly rated a Post Traumatic Stress Disorder compensation claim.  According to VBA officials, the most common errors were incorrect evaluations due to misinterpretation of the appropriate facts and criteria.  C&P Service managers used the results of the IRRRs to plan focused training efforts, and they plan to conduct follow-up IRRRs to evaluate the effectiveness of the training.  The IRRRs allow VBA officials to target a single rating issue to ensure the consistent application of policies and procedures nationally.   

We recommended VBA develop an annual rating consistency review schedule and complete all planned reviews as scheduled, dedicate sufficient staff to conduct consistency reviews in order to complete planned workload and reviews, and include inter-rater reliability reviews as a permanent component of their consistency review program.  VBA agreed with these recommendations and reported that it had completed actions to implement the recommendations.

Site Visit Program

The C&P Service’s Site Visit program was established to ensure centralized oversight and provide technical assistance to VBA’s 57 regional offices.  Site Visit teams monitor compliance with policies and procedures and identify best practices to assist in achieving high performance.  This includes determining if regional offices are correcting errors identified by STAR teams.

Results

The Site Visit program lacks an adequate infrastructure and management strategy to meet its mission and goals.  We identified three areas that VBA needed to address to leverage the benefits of their Site Visit program. 

• The Site Visit team experienced significant turnover during FY 2006 and FY 2008 and has never been fully staffed to the allotted eight full time equivalent positions.  Program officials stated that they have been unable to maintain adequate staffing of the Site Visit program because of difficulties in recruiting qualified candidates from field offices who are willing to relocate to the Washington, DC, area.  In addition, we found that C&P Service cannot ensure that onsite evaluations are performed in compliance with generally applicable governmental standards for independence or that sufficient independence exists between the Site Visit program’s employees and VSCs reviewed.
• C&P Service did not review all 57 VSCs in any 3-year period, and 7 (12 percent) of 57 VSCs were only visited once from FY 2001 to FY 2008.  Because the planned 3-year cycle of review coverage has not been met, potentially low-performing VSCs who could most benefit from a Site Visit program evaluation may not be visited frequently enough.  In addition, C&P Service does not have formal policies and procedures to ensure site survey protocols are modified to reflect emerging C&P issues and systemic deficiencies identified during site visits. 
• C&P Service has not established procedures and guidelines to identify and disseminate best practices.  Also, C&P Service has not developed reports that adequately develop the causes of errors identified, and a follow-up process to ensure that action items are resolved.  In addition, C&P Service does not adequately identify and report system-wide trends to senior VBA managers, thus missing out on opportunities to proactively address issues and concerns found during individual site visits nationwide. 

We recommended C&P Service:

• Develop a staffing plan to ensure that sufficient resources are made available to complete VSC reviews on a 3-year cycle.
• Comply with generally applicable government standards for independence when performing site visits.
• Develop a procedure to continuously monitor and update protocols to address systemic issues identified during Site Visit reviews, management concerns and priorities, and changes in program operations.
• Develop a process for the identification of best practices and resolution of inconsistencies in the application of policies and procedures.
• Develop and implement policies, procedures, and performance measures to strengthen follow-up on corrective action plans developed by regional offices on issues identified during onsite evaluations.
• Ensure Site Visit reports issued for VSC operations to more fully identify the root cause of issues affecting VSC performance. VBA agreed with all six recommendations and reported that it had completed actions to implement the recommendations. 

VBA agreed with these recommendations and reported that it had completed actions to implement the recommendations.

OIG REGIONAL OFFICE INSPECTIONS RESULTS

STAR management relies on the regional office managers to take corrective action on all issues identified by the STAR team.  Since April 2009, the OIG’s Benefits Inspection Division has issued eight reports that include a review of regional office procedures to ensure the accurate and timely correction of errors identified by the VBA’s STAR program.  We found that regional offices did not have formal procedures established to ensure employees took corrective actions on the identified errors and as a result, five of the eight regional offices had not corrected all of the errors identified by the STAR team.  Our analysis of 145 errors identified by STAR found that regional office staff did not correct 40 (28 percent) of the errors.  Further, regional office staff erroneously reported to STAR that 21 (53 percent) of those 40 errors were corrected, although no corrections were made. 

We will continue to review and report on regional offices’ performance in correcting errors identified during STAR reviews in future OIG benefit inspections and to report on other issues affecting accuracy and timeliness of claims processing.  We will also confirm during these inspections whether the actions taken by VBA to implement our recommendations to improve the C&P Site Visit and Rating Consistency programs were effective.  Currently, we plan on conducting up to 18 inspections in FY 2010, which means that each regional office will be inspected on a 3-year basis.  Once we have sufficient data, we plan to issue roll-up reports that identify trends affecting accuracy and timeliness and include recommendations for improvement across the VBA system.

CONCLUSION

VBA is under tremendous pressure to process claims and reduce the growing backlog.  Without an effective and reliable quality assurance program, VBA leadership cannot adequately monitor performance to make necessary program improvements and ensure veterans receive accurate and consistent ratings.

Mr. Chairman, thank you for the opportunity to discuss these important issues.  We would be pleased to answer any questions that you or other members of the Subcommittee may have.