Apr112013
Hearing Transcript on Examining the Effectiveness of the Veterans Benefits Administration’s Training, Performance Management and Accountability
|
EXAMINING THE EFFECTIVENESS OF THE VETERANS BENEFITS ADMINISTRATION'S TRAINING, PERFORMANCE MANAGEMENT AND ACCOUNTABILITY
HEARING BEFORE THE SUBCOMMITTEE ON DISABILITY ASSISTANCE AND MEMORIAL AFFAIRS OF THE COMMITTEE ON VETERANS' AFFAIRS U.S. HOUSE OF REPRESENTATIVES ONE HUNDRED TENTH CONGRESS SECOND SESSION SEPTEMBER 18, 2008 SERIAL No. 110-105 Printed for the use of the Committee on Veterans' Affairs
U.S. GOVERNMENT PRINTING OFFICE For sale by the Superintendent of Documents, U.S. Government Printing Office
|
|
COMMITTEE ON VETERANS' AFFAIRS |
|||||
|
|
CORRINE BROWN, Florida |
STEVE BUYER, Indiana, Ranking |
|
||
|
|
|||||
|
|
Malcom A. Shorter, Staff Director SUBCOMMITTEE ON DISABILITY ASSISTANCE AND MEMORIAL AFFAIRS
Pursuant to clause 2(e)(4) of Rule XI of the Rules of the House, public hearing records of the Committee on Veterans' Affairs are also published in electronic form. The printed hearing record remains the official version. Because electronic submissions are used to prepare both printed and electronic versions of the hearing record, the process of converting between various electronic formats may introduce unintentional errors or omissions. Such occurrences are inherent in the current publication process and should diminish as the process is further refined. |
|
|||
C O N T E N T S
September 18, 2008
Examining the Effectiveness of the Veterans Benefits Administration's Training, Performance Management and Accountability
OPENING STATEMENTS
Chairman John J. Hall
Prepared statement of Chairman Hall
Hon. Michael R. Turner
Hon. Doug Lamborn, Ranking Republican Member, prepared statement of
WITNESSES
U.S. Government Accountability Office, Daniel Bertoni, Director, Education, Workforce, and Income Security Issues
Prepared statement of Mr. Bertoni
U.S. Department of Veterans Affairs, Michael Walcoff, Deputy Under Secretary for Benefits, Veterans Benefits Administration
Prepared statement of Mr. Walcoff
American Federation of Government Employees, AFL-CIO, Michael Ratajczak, Decision Review Officer, Cleveland Veterans Affairs Regional Office, Veterans Benefits Administration, U.S. Department of Veterans
Prepared statement of Mr. Ratajczak
Bartzis, Nicholas T., Cleveland, OH
Prepared statement of Mr. Bartzis
Disabled American Veterans, Kerry Baker, Assistant National Legislative Director
Prepared statement of Mr. Baker
National Veterans Legal Services Program, Ronald B. Abrams, Joint Executive Director
Prepared statement of Mr. Abrams
Human Resources Research Organization, Patricia A. Keenan, Ph.D., Program Manager
Prepared statement of Ms. Keenan
MATERIAL SUBMITTED FOR THE RECORD
Reports:
Post-Hearing Questions and Responses for the Record:
EXAMINING THE EFFECTIVENESS OF THE VETERANS BENEFITS ADMINISTRATION'S TRAINING, PERFORMANCE MANAGEMENT AND ACCOUNTABILITY
Thursday, September 18, 2008
U. S. House of Representatives,
Subcommittee on Disability Assistance and Memorial Affairs,
Committee on Veterans' Affairs,
Washington, DC.
The Subcommittee met, pursuant to notice, at 10:06 a.m., in Room 340, Cannon House Office Building, Hon. John Hall [Chairman of the Subcommittee] presiding.
Present: Representatives Hall, Lamborn, and Turner.
OPENING STATEMENT OF CHAIRMAN HALL
Mr. HALL. Good morning. The House Committee on Veterans' Affairs Disability Assistance and Memorial Affairs Subcommittee hearing on examining the effectiveness of the Veterans Benefits Administration (VBA) training, performance management, and accountability will now come to order.
I would ask everyone to rise for the Pledge of Allegiance. Flags are located at both ends, actually this end.
[Pledge of Allegiance.]
Mr. HALL. Thank you. And welcome again.
Our Nation’s veterans understand the necessity of proper and adequate training. Their lives have depended on it. The military trains for its operations and everyone knows every detail of their job prior to their mission.
These same veterans should be able to expect the same level of competence when they seek assistance from the Veterans Benefits Administration. Unfortunately, that is not always the case as we have heard at other meetings and hearings throughout the year that this Subcommittee has held regarding the U.S. Department of Veterans Affairs (VA) disability claims processing system.
The VA has standardized its training curriculum and requires that all claims processors must complete 80 hours of annual training. This is a lot of hours because, in fact, some healthcare providers do not need to meet that level of continuing education to maintain their clinical license or credentialing.
The VBA training topics are identified by the Central Office (CO) or by the individual’s Regional Office (RO). New employees go through an orientation process and there are additional online learning tools available through the VBA’s training and performance support system.
Yet, with all this effort, VA training seems to fall short of its intended goals. Less than 50 percent of the Ratings Veterans Service Representatives or RVSRs passed the certification exam even though it was an open book test.
Frankly, I have seen the training manual and it could be measured in pounds, not pages. So I do not know how useful the book is, especially given the workload that the people being trained are already under. But that is sort of the crux of the matter.
As outlined in previous hearings, there are significant inconsistencies in ratings between the VA's 57 Regional Offices and a high rate of remanded cases.
I am pleased that the American Federation of Government Employees (AFGE) is here to shed light on the issue. You are a critical link to those on the front lines working to improve outcomes for our disabled veterans.
The U.S. Government Accountability Office (GAO) released a report in May 2008 entitled, "Veterans' Benefits: Increased Focus on Evaluation and Accountability Would Enhance Training and Performance Management for Claims Processors." That is a long title, but sums it up.
[The GAO Report No. GAO-08-561, which was attached to Mr. Bertoni's prepared statement, appears in the Appendix .]
The report documented areas in which the VA needs to improve its training and hold accountable those it does train.
According to the GAO, staff is not held accountable for completing the required training since the VBA does not track completion, so there are no consequences for not taking the training.
Additionally, the VBA does not evaluate its training, so it does not know if it is successfully designed and implemented in educational program.
Feedback is not consistently collected from RO employees on the training that they do receive and many have reported difficulty in accessing training because of their stringent productivity demands.
I look forward to hearing more from the GAO about this report, but these are not surprising conclusions to the Veterans Service Organizations who have complained for years about the inadequacies of the VBA training program. So I am grateful that they have joined us here today as well.
Training is not an issue that should be taken lightly. We all know the importance of good training, but effective implementation that ensures consistency and accountability can be elusive and that is what I hope we can address today.
I have taken steps to ensure improved training outcomes, we have on this Committee when we passed and the full House passed H.R. 5892, the "Veterans Disability Benefits Claims Modernization Act."
These policy enhancements will hopefully lead to compensation claims processing improvements and more accurate claims adjudication results for our veterans and their families.
Moreover, I am not sure that the VBA’s current training regimen complements its current claims processing improvement model or CPI. In fact, I am positive that the current coupling detracts from increased accountability efforts.
I am pleased to report that with the help of many in this room, H.R. 5892 passed unanimously on July 31. 2008, by the full House. On August 1, 2008, Senator Clinton introduced companion legislation, S. 3419, in the Senate.
So, Congress hopefully is on its way to rectifying the inadequacies in the VBA training system that have already been identified.
Today’s oversight hearing will allow us to look deeper into this issue and gauge where VA is in terms of its training protocol and see what other improvements can still be made.
I look forward to hearing from our witnesses. I hope to learn more about best practices and strategies for measuring performance, building better training protocols, and accountability standards.
[The prepared statement of Chairman Hall appears in the Appendix.]
And Mr. Lamborn, our Ranking Member, is not present yet, but, Mr. Turner, would you like to make an opening statement? If so, you are recognized.
OPENING STATEMENT OF HON. MICHAEL R. TURNER
Mr. TURNER. Sure. Thank you, Chairman Hall.
Thank you for this opportunity for collective discussion on the effectiveness of the Veterans Benefits Administration’s training, VBA's performance management and accounting requirements.
Over the course of the past several months, this Subcommittee has examined nearly every facet of the VA benefits claims process system in order to identify how we might help the Department overcome the claims backlog crisis.
While the recent expansion of its workforce will certainly have a positive impact, VA must ensure that newly hired claims workers receive training that is commensurate with their responsibilities. It is critical that the training it provides meets the needs of the Department and its employees.
It is equally important that the results of the training are evaluated. Without feedback, VA may never know whether or not the training is accomplishing its goals.
Any viable training program should be able to identify deficiencies and demonstrate the intended and actual outcome of the curriculum. VA training must be connected to its vision and mission and VA managers need to be assured that if employees are pulled off the floor for training that it will result in long-term benefits.
With the growing number of pending claims, there is a certain level of trepidation that there is too much work to do already and we will just get further behind if we have to conduct training.
There must be clear support from the top down in order to conduct adequate training and acquire the expected outcomes. Certainly training new employees of everything they need to know in order to make sound rating decisions is a daunting task.
The VA rating schedule itself is complex and it is merely a portion of the array of knowledge a competent adjudicator must possess to perform his or her job.
Today’s hearing is an opportunity to not only learn more about the training and assessment program VA provides its employees but also to reiterate to the Department that it should be forthright about any additional resources deemed necessary to fulfill this critical requirement.
Mr. Chairman, I thank you for hosting this hearing on VA’s training program and I yield back.
Mr. HALL. Thank you, Mr. Turner.
I would like to remind all panelists that your complete written statements have been made a part of the hearing record so that you can limit your remarks to five minutes so we can have sufficient time for follow-up questions once everyone has had the opportunity to testify.
Our first panel, the entire panel, is Mr. Daniel Bertoni, Director of Education, Workforce, and Income Security Issues at the U.S. Government Accountability Office.
Welcome, Mr. Bertoni, and you are now recognized for five minutes.
STATEMENT OF DANIEL BERTONI, DIRECTOR, EDUCATION, WORKFORCE, AND INCOME SECURITY ISSUES, U.S. GOVERNMENT ACCOUNTABILITY OFFICE
Mr. BERTONI. Mr. Chairman, Members of the Subcommittee, good morning.
I am pleased to discuss training, performance management for Department of Veterans Affairs' disability claims processors.
Last year, VA paid about $38 billion in benefits to nearly four million veterans and their families.
The disability claims process has chronically suffered from long waits for decisions, large backlogs, and problems with accuracy and consistency.
We have also noted that VA’s program is in urgent need of transformation, especially in regard to how it assesses work capacity and provides interventions and support services to veterans.
To address its management challenges, VA has hired thousands of additional staff. However, increased staff alone will not guarantee more timely, accurate, and consistent decisions.
Among other things, adequate training and performance management will be key to developing new staff and ensuring that more experienced staff maintain needed skills.
My remarks today draw from our prior work for this Subcommittee and focus on two areas, VA’s training program for disability claims staff and its performance management system.
Summary. Consistent with accepted training practice, VA has taken steps to align its training with the agency’s overall mission and goals. For example, in 2004, VA established the Training and Learning Board to ensure that training support of VA’s strategic and business plans.
Various analyses. VA has also identified the skills and abilities needed by staff and taken steps to determine the appropriate level of investment in their training and prioritize funding.
Finally, we found that VA’s training program for new staff define pertinent terms and concepts and provided many realistic examples of claims work. However, we did identify areas for improvement.
While the VA collects feedback to assess initial training for new staff, not all training is evaluated to determine how relevant or effective it is. None of the Regional Offices we visited consistently collected feedback on training they provided either for new or experienced staff. Thus, VA’s Central Office lacks key information on its entire training activities.
Both new and experienced staff reported concerns with their training. Some staff noted that VA’s computer-based learning tool, Training and Performance Support System (TPSS), was often out of date and too theoretical.
The more experienced staff reported that they struggle to meet VA’s annual 80-hour training requirement due to workload pressures or the lack of training relevant to their experience level.
It is unclear what criteria was used to justify the 80-hour requirement, but identifying the right amount of training is crucial. Overly burdensome requirements can take staff away from essential tasks while too little training can contribute to errors.
Putting aside the appropriateness of the current requirement, VA has no policies to hold staff accountable for meeting it and may be missing an opportunity to convey the importance of training as a means to meet individual goals as well as broader agency performance goals.
Regarding performance management, VA’s system generally conforms to accepted practices in that individual performance measures such as quality and productivity are also aligned with organizational performance measures.
Staff are required to receive regular feedback on performance and employees and other key stakeholders were actively involved in developing the current system.
However, we are concerned that under the current rating formula, VA’s system may not make meaningful distinctions in staff performance. Although VA has a five category rating system in the field offices we visited, 90 percent of all staff ended up in only two, fully successful and outstanding as noted in the chart we provided.
We have reported that four or five category systems are most useful for making performance distinctions. However, if staff ratings are clustered in only one or two categories as with VA, this may be problematic.
Broad, overlapping performance categories may deprive managers of information needed to reward top performers, address performance issues, deprive staff of much needed feedback.
VA has acknowledged potential issues with the rating formula and is considering some adjustments. However, the agency has never examined the distribution of all staff across the five performance categories. Absent this analysis, VA will lack a clear picture of what adjustments are actually needed.
Conclusion. While VA recognizes the importance of developing and maintaining high performing staff, it must devote more attention to training and performance management.
Additional study on such issues as the effectiveness of regional training, the appropriateness of the current training requirement, and the usefulness of computer-based learning tools is needed.
Additional means for holding staff and managers accountable for completing training should be explored as well as options for enhancing the current rating system.
We acknowledge VA’s efforts to hire more staff. However, in the longer term, more fundamental changes are needed. VA and other Federal disability programs must adopt a more modernized approach to determining eligibility for benefits as well as the timing and portfolio of services that are provided.
If progress is made in this area, effective training and performance management will be of crucial importance and will impact the degree to which service to veterans is ultimately improved.
Mr. Chairman, this concludes my statement. I am happy to answer any questions you may have. Thank you.
[The prepared statement of Mr. Bertoni appears in the Appendix.]
Mr. HALL. Thank you, Mr. Bertoni, for that most incisive statement and for your written statement as well, which goes into considerably more detail, of course.
In your opinion, what is more effective, the classroom training or online tools?
Mr. BERTONI. I do not know what individual mode of training is most effective. I think any training program should have a mix of classroom training, on the job, and online, computer-based tools.
Agencies should be looking at and evaluating each of those, call it the three-legged stool, to ensure that they complement each other, they are consistent in terms of the information they convey, and they are meeting the needs of staff.
Mr. HALL. What did you think of the VA’s learning management system and training and performance support system?
Mr. BERTONI. Generally we used an in-house criteria to assess the four components of training system and generally there are four. Our criteria is based on various external experts, internal experts at GAO, private-sector individuals, nonprofits, and academia. And we came up with a criteria that we use quite often to evaluate training and performance management.
Generally we break this apart into four categories, planning, design, implementation, and ultimately evaluation.
And as our report notes, I think VBA and VA are doing an adequate job in many respects. They have aligned their training with broader agency goals. They have aligned their performance management system and with their measuring folks on with their broader agency goals and mission.
So they are doing, I think, many things well relative to accepted practices, but still are falling short in terms of in the area of feedback. Feedback is essentially your evaluation loop. That is how you find and determine whether your training is relevant, whether it is effective, and whether it is being delivered at the appropriate time. To the extent you are not getting that feedback, I think you are missing an opportunity.
They do a great job or a better job early on for the initial orientation training, but I think things fall off as staff return to their home units and begin taking training. And it is harder for us to get a handle on uniformity, consistency, and whether it is as effective as it could be.
Mr. HALL. Thank you, Mr. Bertoni.
I understand that proper training is vital to efficiency and accuracy, but is training the bottom line problem at VBA? Would you correlate the problems inherent in the backlog as an issue with training?
Mr. BERTONI. It is hard to isolate all the factors that contribute to the backlog. There are laws, regulation that have contributed. There are management inefficiencies that have contributed. But training certainly is a tool to get at or to address many of the problems that agencies face. We would not do it if we did not think it was worthy.
And clearly when you have good training, you can see results. Whether you can, you know, measure clear cause and effect, that is probably not something that you can always do. But generally a good, solid designed training system is effective.
Beyond training, I do really believe that part and parcel with that is a good quality assurance process. Until you know where your soft spots are, where your problems are through your quality assurance reviews, it is very hard to design training that is going to really target and get at the problematic areas in your process.
We have gone many years with timeliness, accuracy, and consistency issues, and I have to believe based on some work that we have done looking at the STAR system and other quality assurance processes that having more robust quality assurance could help VBA, VA identify some of the real root causes for inconsistency and inaccuracies.
And I think they really need to focus on that. And it appears from the statement I read from VA this morning they are heading in that direction.
Mr. HALL. In reviewing VBA’s performance management system, how different was it from other Federal agencies? Three-part question here. Are there other agencies with a better performance management system that VA could adopt and would it make sense to give performance credits for training as well as for work completion so that there is not pressure to stay on the workload and not train?
Mr. BERTONI. I do not know if it is a matter of giving credits for workload completion. I believe their rating should reflect the fact that they are doing a good job or a great job in terms of processing the workloads.
Our concern with their performance management system is that it may not be rewarding. It may, in fact, be potentially demoralizing for some staff.
I will give you an example. We'll use three people. You take the first person who is clearly a high flyer who rates exceptional across all the critical and noncritical dimensions. That person, it is a no-brainer. That person would be listed in the outstanding category.
We will take another person who is just barely at the fully successful level and, you know, just barely eking out a fully successful rating, marginally falling below that, but at the end of the performance year ends up with a fully successful rating.
Take a third person who is clearly also a high performer. That person gets exceptional ratings on all dimensions, critical and noncritical. With the exception of one, that person gets a fully successful. That individual will be dumped into the bucket with that fully successful person.
Clearly that person falls somewhere between exceptional and outstanding, but because of that one fully successful in a critical dimension are dropping down into that lower bucket and all the incentives, the pay incentives, the other monetary incentives are not available.
So if you are trying to create a world-class organization and incentivize people for hard work, this example really disturbs us. We see in the four regions that we visited 90 percent of all folks in two categories and nobody in that second excellent category. It gives me pause.
Mr. HALL. Thank you, Mr. Bertoni.
My time is expired. I will now recognize and welcome our Ranking Member, the gentleman from Colorado, Mr. Lamborn.
Mr. LAMBORN. Yeah. And thank you, Mr. Chairman.
Mr. Bertoni, you apparently have some familiarity with other departments other than the VA because of looking at training programs in other parts of the Federal Government.
What are some successful components that you have seen in some of these other areas that you would suggest the VA should imitate?
Mr. BERTONI. Again, I will hearken back to what we identified in our report. I think you really need to foster a rigorous system for feedback and evaluation. If you are focusing on new employees at the initial orientation level and getting good information there, fine. We acknowledge that.
But when these folks transfer back or go out into their home units and are taking this, you know, training at the regional level, there is no feedback loop that is feeding back into VBA’s headquarters so they can use this information to identify areas where they are falling short and ultimately to devise training to get at some of the problems that we have talked about in the area of accuracy and consistency.
World-class organizations have completed that wheel, planning, design, implemented, and your evaluation loop. To the extent that all those elements are firing, you have better systems. And there are some organizations that do better than others.
Mr. LAMBORN. Thank you.
Mr. Chairman, I yield back.
Mr. HALL. Thank you, Mr. Lamborn.
And one more question, Mr. Bertoni. In 2001, the GAO issued a report "Veterans Benefits Training for Claims Processors Needs Evaluation." At that time, the GAO found that the TPSS was being implemented differently in each Regional Office. I know that the VBA has mandated some of the training, but employees have raised concerns about the training and claim limited time for it because of workload demands.
So what differences do you see in the VA’s training process from that time, from 2001 to the present?
Mr. BERTONI. We really did not delve into the TPSS system. We had the prior report. We followed up our recommendations.
I know that VA has taken steps to monitor and approve that system. I think there is an ongoing effort and I think almost a report to be issued shortly also. But we really did not delve into the system exclusively.
We were only able to sort of give a snapshot of what was going on in the four Regional Offices and there was clearly some noise that the TPSS system was not all it could be for some folks in terms of the timeliness of the information in there, the relevance, especially in regard to the collection of medical evidence. I believe that came up frequently.
But this was not a look at TPSS. It was sort of a broad-based review of all the elements and tools that are there for staff to use.
Mr. HALL. Okay. Well, that is all the questions I have. Mr. Lamborn, if you are done, I would thank you, Mr. Bertoni, again for your testimony. And your panel is now excused.
Mr. BERTONI. Thank you.
Mr. HALL. And thank you so much.
And joining us now on our second panel, we would like to welcome Mr. Michael Ratajczak—is that correct—
Mr. RATAJCZAK. Ratajczak. Thank you.
Mr. HALL. —Decision Review Officer at the Cleveland Veterans Affairs Regional Office on behalf of the American Federation of Government Employees; Mr. Kerry Baker, Assistant National Legislative Director of the Disabled American Veterans (DAV); Ronald Abrams, Joint Executive Director of the National Veterans Legal Services Program (NVLSP); Dr. Patricia Keenan, Manager of Employee Performance Enhancement and Growth Program at the Human Resources Research Organization (HumRRO); and Mr. Nick Bartzis, a veteran from Cleveland, Ohio.
Welcome, all, and your statements, written statements are in the record so that we will allow you to deviate and ask you to stay within the five minutes so we have time for questions for everybody.
Tell me one more time. Ratajczak?
Mr. RATAJCZAK. Ratajczak.
Mr. HALL. Ratajczak. Mr. Ratajczak, you are now recognized for five minutes.
STATEMENTS OF MICHAEL RATAJCZAK, DECISION REVIEW OFFICER, CLEVELAND VETERANS AFFAIRS REGIONAL OFFICE, VETERANS BENEFITS ADMINISTRATION, U.S. DEPARTMENT OF VETERANS, ON BEHALF OF AMERICAN FEDERATION OF GOVERNMENT EMPLOYEES, AFL-CIO; KERRY BAKER, ASSISTANT NATIONAL LEGISLATIVE DIRECTOR, DISABLED AMERICAN VETERANS; RONALD B. ABRAMS, JOINT EXECUTIVE DIRECTOR, NATIONAL VETERANS LEGAL SERVICES PROGRAM; PATRICIA A. KEENAN, PH.D., PROGRAM MANAGER, HUMAN RESOURCES RESEARCH ORGANIZATION (HUMRRO); AND NICHOLAS T. BARTZIS, CLEVELAND, OH (VETERAN)
STATEMENT OF MICHAEL RATAJCZAK
Mr. RATAJCZAK. Thank you.
I think I have as much practical experience with regard to training issues at the VBA as anyone in the room. I served as centralized challenge instructor. I worked on the certification design committees. I have identified trends in errors by interacting with Board of Veterans' Appeals (BVA) personnel. I have taken some remedial action to fix errors resulting from what ultimately is improper and effective training.
As a Decision Review Officer every day, I kind of am serving as a forensic pathologist where I look at a case where a mistake was made and trying to assign a reason for that mistake. When I have done that more often than not, I will effectively as possible take the case to the person who may have made the mistake and say this is what you did, let us try and avoid that in the future.
More often than not as an explanation for their actions, the folks who made an error will attribute the error to the fact that, well, I just did not know that. I was not trained. I do not even know where to find that information.
To the extent that errors are made, it is obviously a reflection of improper training on behalf of the VBA.
And what I would like to submit is that the folks who are making these errors, the front-line claims adjudicators, veterans representatives, specialists, decision officers, they have an interest in effective training. They do not want to make mistakes because as we all do, they have service to veterans at heart ultimately.
But beyond that, if they are making mistakes, adverse employment action can be taken against them. They can be put on a production improvement plan. So they have a very real interest in effective training and participating in effective training.
To the extent that that opportunity for effective training is not presented to them, I think we can do better.
I was struck by the testimony in previous hearings before this Subcommittee and others regarding complexity, increasing complexity of the process, judicial review, explanations for why there is a backlog in the VBA.
And all that is true. VBA no longer operates in splendid isolation for judicial review, so the court comes back and looks at VBA’s processes and says, well, this is what you need to do, this is what you are not doing correctly.
Those kind of decisions should be viewed more as an opportunity than as an impediment because we can take those decisions, take the reasoning in them, and identify things they need to be trained on so we avoid the mistakes in the future.
Recognizing that there is a component to the backlog which is attributable to complexity in legal review, VBA, I do not think, has made the same leap to the fact that those same elements have a detrimental impact on an individual employee’s ability to do their job.
Obviously if a given claim is more complex and there are additional legal requirements placed on the Veterans Service Representatives (VSRs) and Rating Specialist, it is going to take them longer to do their job and maybe what we could do is design some effective training to make them more efficient in doing their job in the face of increasing complexity and increasing legal requirements. I am going to suggest briefly a couple of methods of doing that based on my experiences.
I just want to note that I did read the GAO report. I think overall it is accurate, but I want to caution anybody who will listen to the extent that it may imply that individual claims processors are responsible for improper training, that cannot be the case. They do not have the ability to go to their management and say, hey, I need more training. You need to do that so I can do my job better. Frankly, that is someone else’s job to do that training and give them the opportunity for it.
Several actions can be taken to include training. Number one, centralize curriculum from Central Office implemented by instructors that are responsible to Central Office. That would provide a uniform voice to all Regional Offices. This is CO’s policy.
To the extent that adjudicators’ experience will be informed by that uniform training, you would expect to get some uniformity and some more consistency in the decisions.
Also, I think Veterans Service Representatives, Rating Specialists in particular, should be given work credit for doing deferred actions where they have direct additional development in a claim to the person who is responsible for developing the evidence. That would provide feedback to people developing the evidence and also an incentive to the Rating Specialists.
Thank you.
[The prepared statement of Mr. Ratajczak appears in the Appendix.]
Mr. HALL. Thank you, Mr. Ratajczak.
Mr. Baker, now you are recognized for your statement.
Mr. BAKER. Mr. Chairman and Members of the Subcommittee, on behalf of the DAV, I am pleased to offer my testimony to address VA’s training, accountability, and performance measurement system.
VA has a standard training curriculum for new claims processors and an 80-hour annual training requirement for all claims processors.
DAV has long maintained that the VA should invest more in training and hold employees accountable for higher standards of accuracy.
VA’s problems caused by a lack of accountability do not begin in the development and the rating process and in the training program of which we can find little, if any, measurable accountability.
For example, some employees inform DAV that many candidates begin centralized training before they complete or even start phase one training. Candidates are then not held responsible via formal testing on subjects before advancing to phase two training.
As in phase one, VA refuses to test phase two trainees. Without such testing, VA cannot gauge the success of its learning objectives.
During phase three training, employees work on real world cases. That notwithstanding, no accountability, no testing, and no oversight other than that provided locally. That oversight is not measured nationally.
The result of such an unsupervised and unaccountable training system is that no distinction exists between unsatisfactory performance and outstanding performance. Lack of accountability during training eliminates employee motivation to excel.
The need for improvements in VA’s training program became evident when it began the skills certification test for VSRs. The first two attempts at the test produced a 25- and a 29-percent pass rate. The third produced a 42-percent pass rate but only after employees completed a 20-hour prep course to pass the test.
Mandatory, comprehensive testing designed cumulatively one subject area to the next for which VA then holds trainees accountable should be the number one priority of any plan to improve VA’s training program.
Further, VA should not allow trainees to advance to subsequent stages of training until they have successfully completed such testing.
DAV has also long stated that in addition to training, accountability is the key to quality. Therefore, timeliness.
We believe VA’s quality assurance or STAR Program is severely inadequate. In 2006, the STAR Program reviewed just over 11,000 compensation and pension (C&P) claims for improper payments. There was only .72 percent of the total number of cases available for review. However, a more reliable measure of VA’s accuracy and its lack of accountability is shown through the Board of Veterans' Appeals’ annual report.
Fiscal year 2007, the Board decided approximately 40,000 appeals. The Board reversed 21 percent of those appeals and remanded over 35 percent. Of those remands, over 7,000 of those cases or nearly 20 percent were remanded because of errors in the most basic of procedures, procedures that should be elementary to every single decision maker.
The problem is compounded when one considers that those errors cleared the local Board, local Rating Board and the local Appeals Board which contain VA’s most senior Rating Specialists. These facts clearly show little incentive for many employees to concern themselves with the quality of their work.
Congress should require the Secretary to report on how the Department could establish a quality assurance and accountability program that will detect, track, and hold responsible those employees who commit errors.
We believe that effective accountability can be engineered in a manner that would hold each VA employee responsible for his or her work as a claim moves through the system while simultaneously holding all employees responsible.
If such errors are discovered, employees responsible for such errors should forfeit a portion of their work credit. This idea is discussed more in my written testimony, but I would be happy to explain further if the Committee has questions.
I believe this type of accountability system will draw on the strengths of VA’s performance measurement system, thereby allowing easier and less disruptive implementation of stronger and more effective accountability.
Mr. Chairman, that concludes my statement, and I will be happy to try to answer any questions you might have.
[The prepared statement of Mr. Baker appears in the Appendix.]
Mr. HALL. Thank you, Mr. Baker.
Now, Mr. Abrams, you are recognized for five minutes.
Mr. ABRAMS. Good morning, Mr. Chairman and Members of the Committee. I am pleased to have the opportunity to submit this testimony on behalf of NVLSP.
As some of you know, we have trained thousands of service officers and lawyers in veterans’ benefits law. We have written books, textbooks, and guides on veterans’ benefits and we help people represent veterans seeking VA benefits.
We also conduct quality reviews of the VA Regional Offices on behalf of the American Legion. We believe after reviewing all of this that the effectiveness of VBA training should be measured by the quality of the work product produced by VA adjudicators.
Therefore, the quality and timeliness of VA adjudication should reflect the effectiveness or lack thereof of VA training. Because the quality of VA adjudications is in our opinion inadequate, NVLSP must conclude that VBA training is not effective or adequate.
In the experience of NVLSP over ten years of quality checks, over 40 VA Regional Offices combined with extensive representation before the Veterans Court, we find that most of the worst VA errors are the result of premature Regional Office decisions.
We find that many VA managers emphasize quantity over quality, making many aspects of training not relevant. That is because VA workers want to please the people they work for. Training is de-emphasized because production is paramount.
VA employees consistently let us know that they are told that time spent training reduces the time available to produce decisions. Also training VA workers regarding the procedures designed to protect the rights of claimants, developing the claim fully, reviewing the entire file instead of top sheeting, which is just looking at the first couple of pages—all takes time. The essence of fairness in this particular process is giving the VA worker the time to do the job.
In fact, in September 2008, courageous VA Regional Office employees filed a grievance exposing this overemphasis on production. That grievance is attached to my testimony. The grievance asserts that the Regional Office created and encouraged a culture in which quantity is emphasized to the detriment of quality, that the RO failed to properly implement training to assure that those reviewing claims are sufficiently trained and that the RO failed to properly implement a fair and impartial performance appraisal system that assures that quantitative measures of performance are not emphasized to the detriment of measures of quality.
These VA workers allege they were told to produce cases. It did do not care about the quality of their work or it did not care that much. And we think that the quality of the VA work is much worse than what is reported by the VA.
As Kerry testified, the BVA statistics are scary. If over 50 percent of what goes up to the Board of Veterans' Appeals has to be returned because the VA in general failed to follow proper process, we have a problem. That is the best quality check possible.
I want to thank you for permitting NVLSP to testify on this important issue. I will be happy to take any questions.
[The prepared statement of Mr. Abrams appears in the Appendix.]
Mr. HALL. Thank you, Mr. Abrams. We will have some questions for you.
Dr. Keenan, you are now recognized.
STATEMENT OF PATRICIA A. KEENAN, PH.D.
Dr. KEENAN. Good morning, Mr. Chairman and Members of the Subcommittee. I am Patricia Keenan, a Program Manager at the Human Resources Research Organization or less formerly and more shortly is HumRRO.
HumRRO is a nonprofit 501(c)(3) research and development organization established in 1951 that works with government agencies and other organizations to improve their effectiveness through improved human capital development and management.
Our comment today is about the compensation and pension services training program as well as on methods to reduce training variance.
I am the project leader for HumRRO’s work with VBA skills certification programs. HumRRO has worked closely with C&P service on skill cert programs for Veterans Service Representatives or VSRs since 2001.
We have seen passing rates on the VSR test rise steadily since the beginning indicating to us that C&P’s training initiatives are having a positive effect.
As part of our work, we conducted numerous focus groups and interviews at Regional Offices. As a result, we identified several factors that decrease rating consistency.
First, the RVSR job is very cognitively demanding. It requires knowledge of medical conditions, regulations, and the rating schedule, attention to detail, decisiveness, and strong analytical ability. These requirements exacerbate the other challenges RVSRs face.
The second factor we identified is the problem of incomplete information in the rating schedule. While the schedule contains over 700 diagnostic codes, by comparison, the international classification of diseases by the medical profession contains thousands of codes.
Many areas of the rating schedule leave room for interpretation and RVSRs develop individual rules for matching the medical evidence to the schedule. This allows them to reduce the cognitive load and work more quickly, but it also is a source of rating variance.
The rating schedule does not have diagnostic codes for some conditions such as Parkinson’s disease or carpal tunnel syndrome. When a claim includes an unlisted condition, the RVSR rates it by analogy to a closely related disease or injury. By their nature, these analogous codes lack criteria for rating, so raters have to research different conditions to make the evaluation that may not be very straightforward.
The third challenge to the RVSR is related to workload. The sheer volume of cases awaiting adjudication sometimes results in a lack of attention to detail. One way this shows itself is in cases that are not ready to rate and require additional development. This means the RVSR has spent time reviewing a case, but the decision must wait for new information and the case must be reviewed a second time.
The pressure also shows itself in rating decisions that rely too heavily on the use of templates rather than clearly establishing the connection between the medical evidence presented and the regulations in a way that is easily understandable.
It is very important that veterans feel confident that their cases have been fairly evaluated.
VBA’s recent wave of hiring was the best long-term response to the influx of claims. However, one result has been that newly hired RVSRs do not understand the development process well.
Although the workload of the RVSR is not going to lighten in the near future, there are some actions that could help improve rating consistency.
First, to address one of the workload issues, newly hired RVSRs should work predetermination for several weeks to learn the system. The obvious drawback to this is that it would take longer for new RVSRs to begin rating cases. But we believe having this additional knowledge would pay off in the longer term.
Second, job aids that include more specific medical information and rating schedule would ease the cognitive complexity of the job and reduce the individual interpretation of the ratings. However, job aids cannot help with the complex problem of writing good rating decisions.
One long-term solution to this problem would be to develop a selection tool that assesses an applicant’s ability to synthesize information and present it in a well-structured, easily comprehensible document.
Finally, VBA should ensure that all RVSRs continue to receive standardized training.
It has been HumRRO’s pleasure to work with the C&P service for the past seven years. We are honored to be even a small part of the valuable work the Veterans Benefits Administration does for America’s veterans. We have watched both the skill certification program and C&P service over this time. The effort devoted to training has been steadily improving pass rates on the test.
Beyond training, we have identified several areas that could further increase rating consistency. Addressing the problems of cognitive complexities, ambiguity in the rating schedule, and workload is something VBA should continue.
Thank you for this opportunity.
[The prepared statement of Ms. Keenan appears in the Appendix.]
Mr. HALL. Thank you, Dr. Keenan.
Mr. Bartzis, you are now recognized for five minutes.
STATEMENT OF NICHOLAS T. BARTZIS
Mr. BARTZIS. Thank you, sir.
Thank you for the invitation to speak here today as a veteran and I am here today as a private citizen. However, I have a few statements beyond my written statement to go through. Most of them are in the form of—
I raise my right hand approximately 20 inches. I am going to show approximately a half an inch with my left hand. If we determine we have two cases, the cases make the mistake, make the same mistake on the same issue, how long is it going to take to go through all of the medical evidence on the case on the right as opposed to the case on the left? The answer to that question to me or the question itself is how much time is it going to take?
But what I am really talking about is time. If I have to go through 20 inches of medical evidence or if I have to go through a quarter inch or a half inch of medical evidence, I am going to spend a significantly different amount of time to find the error.
It has been stated earlier that part of the problem is the development issue, but the actual issue is how much time does it take and who is going to do it. That is a question I believe the Committee should ask.
The second issue is accountability. Who is going to do it and when and where is a failure, if any?
The rest of the statements are in my written statement, and I will defer to that. Thank you very much.
[The prepared statement of Mr. Bartzis appears in the Appendix.]
Mr. HALL. Thank you, Mr. Bartzis.
I will recognize myself now for a round of questions.
Mr. Ratajczak, what happens when RO staff cannot complete the 80 hours of training?
Mr. RATAJCZAK. To the best of my knowledge, nothing happens. I do not think there is any detrimental effect on the individual adjudicator if they have not concluded their training requirements. I suppose that there would be some method in place for them to do some remedial training. But to the extent that they are not held accountable for ensuring the training to begin with, nothing happens.
As far as a practical matter, what happens is they do not get enough training and obviously we get poor decisions and increasing appeals.
Mr. HALL. Dr. Keenan, you spoke of new hires having an initial period of, if I understood you correctly, of training before they start working on cases.
What amount of time do you recommend and is that something that you have seen succeed in other similar situations in other agencies?
Dr. KEENAN. For them to work predetermination before they start rating cases?
Mr. HALL. Right. To just have a concentrated training period before they start rating cases.
Dr. KEENAN. Well, right now they do have a three-month training period before they start rating, but they do not have enough time to learn all the intricacies of predetermination which is all the evidence gathering that is required for the RVSR then to be able to make ratings, solid ratings. So if they could spend an extra six or eight weeks doing pre-d, just learning the whole claims process.
What we heard in our visits to the ROs was that the people who are promoted internally to RVSRs, it is a road running much more than people who are hired from the outside.
Mr. HALL. Yes.
Dr. KEENAN. And that makes perfect sense. It is a very complicated process. There is a lot to learn in PR and when you are doing a rating decision, you have to understand, you have to be able to look at the evidence and know that you have got enough there to make your decision.
If the VSR has not developed all of that information and you do not know this and you spend time going through your case and then have to defer it, as Michael mentioned before, so if they could spend more time once they have gone through their initial training and then go through pre-d, spend some time, they will still be working cases in pre-d, so they are helping in that sense getting things ready to rate, they will have a much better sense of the whole process.
Mr. HALL. So rather than three months, you suggest four or five months, in effect? An additional six to eight weeks, you said?
Dr. KEENAN. Yes.
Mr. HALL. Why do you think Dr. Keenan, so many VBA employees had a problem in passing the open book test that you developed?
Dr. KEENAN. When VBA first told us they wanted an open book test, we were totally dumbfounded. We do a lot of certification work and it is always closed book. You come in on a Saturday. You take the test.
But once we realized the complexity of the job and the fact that people really cannot do their job just with their memory, it is just too complicated, there are too many rules, too much money, too many regulations, they have to have training aids, job aids, you know, regulations available to them so that they can make these decisions.
The tests are fairly applied, so it is not just a matter of recall which screen you use for that, although those are important to know because it makes you work more efficiently. The—
Mr. HALL. So you think they need more time to take the test?
Dr. KEENAN. Well, we actually just had a meeting about that yesterday. We are going to add—
Mr. HALL. Because it is a learning experience? The test itself?
Dr. KEENAN. We want them to learn while they are testing.
Mr. HALL. Right.
Dr. KEENAN. We hope that they will come in knowing that. And we had originally set the time limit. It is a two part test, half in the morning and half in the afternoon, 2 hours and 45 minutes in the morning and 2 and 45 in the afternoon. We are going to expand that to a full three hours both times.
Mr. HALL. Thank you. I am sorry to have to interrupt you, but I only have a little bit of time left in this round.
Mr. Baker, in your testimony, you mentioned that the VBA does not test participants after each phase of training.
Do you think it would be better to test at these intervals instead of the certification exam or in addition to the certification exam?
Mr. BAKER. Well, if I may clarify. It sounded like I was pretty rough on VA employees there and I want to make sure that you know I was not aiming at VA employees. It was the system that I would like to see improved.
But to answer your question, I think it should be both, absolutely testing during the training phases, cumulative testing, so that you are responsible for what
Sign Up for Committee Updates
Stay connected with the Committee