In response to Section 226 of Public Law 110-389, the Veterans’ Benefits Improvement Act of 2008, the Department of Veterans Affairs (VA) asked CNA to study the effectiveness of its current employee work credit system and the work management system. Beyond these principal areas of study, we considered other topics including the use of information technology (IT) applications, methods of reducing the time required to obtain outside information, processing of claims that are ready to rate upon submittal, processing of claims from severely injured (SI) and very severely injured (VSI) veterans, and an assessment of best practices in claims processing. CNA conducted this analysis between March and September 2009.

Due to the nature of our study questions and the short timeline for this project, the most appropriate methodology was qualitative data analysis. Our primary source of data was information collected from interviews on site visits to six Regional Offices (ROs) selected by the Veterans Benefits Administration (VBA). On these site visits we spoke with employees in a wide variety of roles in order to obtain as complete a picture as possible within the time constraints of the study. We took detailed notes from each interview and reviewed them to identify common themes among the responses. In total we spoke with 41 frontline employees, 49 managers, and 11 other employees. We supplemented the information from our site visits with background information from congressional hearings, formal evaluations from a variety of sources (such as the Government Accountability Office (GAO), VA’s Office of Inspector General (OIG), and internal VBA projects), conversations with VBA subject matter experts, and summarized VBA administrative data.

Background

VBA faces multiple challenges in processing claims including large increases in the number of claims submitted and the increasing complexity of these claims. The increase in the number of claims submitted is striking. For example, VA received about 719,000 compensation claims in FY2008. This is a 53 percent increase from FY1999.[1]  Reasons often cited for the increase are the return of veterans from Global War on Terror deployments and the expansion of benefit entitlements, such as new presumptions of service connection. As for the complexity of claims, GAO reported that in FY2006, 11 percent of claims that VA decided included 8 or more issues. By FY2008, this proportion increased to 16 percent.[2]  We note these statistics to illustrate the challenging and dynamic environment in which VBA works to process claims.

The work management system that VBA uses is the Claims Process Improvement Model or CPI model. The CPI model is an organizational model that promotes specialization as a way to improve quality and timeliness. The model has separate claims processing teams to perform functions in each of the following areas: public contact, triage, pre-determination, rating, post-determination, and appeal. In this model the triage team reviews and routes mail, enters basic claim information into the computerized workload management system, and performs some simple claims processing actions. From triage, claims moves to the pre-determination team, which primarily gathers the information necessary to rate each claim. Once a claim is ready to rate, the rating team decides whether a veteran’s medical conditions are connected to military service and if so, what the degree of disability is according to the VASRD (Department of Veterans Affairs Schedule for Rating Disabilities). Next the post-determination team processes (i.e., authorizes) awards and notifies claimants of the decisions made on their claims. Outside of this process, the appeals team handles claims for which claimants disagree with the award decision and decided to appeal it and the public contact team conducts personal interviews and responds to phone inquiries.

Veterans Service Representatives (VSRs), Rating VSRs (RVSRs), and Decision Review Officers (DROs) are VBA employees that work on the various teams in the CPI model. These employees are all subject to minimum work credit standards. They earn work credits by performing certain claims processing actions, each of which is assigned a specific number of credits meant to reflect the time required to complete the action. Employees’ ability to meet the work credit standards is one of the elements considered in their annual performance evaluations.

Note that VBA asked CNA to study its work credit and work management systems to see how they could be improved. While there are positive aspects to these systems, our charter was to look for areas of improvement; hence, we report on areas for improvement only. Therefore, one should not conclude that no part of these models works well because we focused on areas for improvement only.

Work credit system

Findings

Stakeholders’ main concerns about the current employee work credit system are (1) that it emphasizes quantity over quality, (2) that production standards are not based on a careful analysis of the tasks performed, and (3) that work credits and production standards do not reflect changes in claim complexity and additional duties imposed by legislation.

Two recent surveys found strong evidence that VBA claims-processing employees perceive that quantity is considered more important than quality. The first was a survey of RVSRs and DROs conduced by VA’s OIG during March and April 2005.[3] CNA conducted the second survey of rating officials and Veteran Service Organization (VSO) representatives for the Veterans’ Disability Benefits Commission during December 2006 and January 2007.[4]  Both surveys found very strong evidence that rating officials perceive that quantity is considered more important than quality. This is an important finding because even if that perception is incorrect, at least some employees who have that perception will probably change the way they process claims in order to do what they think their supervisors want.

From our analysis in this study, the perception that the quantity of work receives more emphasis than the quality of work was common. The overwhelming theme from frontline employees was that the production standards are difficult to meet, whereas concerns about struggling to meet the quality standards were rare. We infer from this that a moderate increase in the number of quality reviews per employee would better communicate to employees that VBA values both production and accuracy. In addition, some interview respondents reported that they feel the current number of reviews is too low to be representative of their work, which certainly implies that some employees would even welcome an increase in the number of quality reviews.

VBA has undertaken several of its own efforts to consider ways to improve the work credit system. One of these efforts was a time-motion study intended to provide information for updating the work credit values. A more fundamental change that VBA is investigating is re-defining all the actions for which employees receive credit in order to better align them with the overall goal of completing claims. In particular, VA established a VSR Performance Standards Workgroup that recommended that VSRs receive work credit for only the following four types of actions: (1) initiating development, (2) making a claim ready to rate, (3) deciding an award, and (4) authorizing the award. At the time of our report, VA was preparing to pilot test this approach at multiple ROs. 

Prior to developing this approach, the VSR Performance Standards Workgroup conducted a survey of frontline employees and managers. It included questions on perceived weaknesses in the current work credit system, and the responses echoed what we heard through our evaluation. The most commonly reported problems were that the performance standards are outdated, there are inconsistencies across teams and ROs in the rules for claiming credits, deductible time is subjective, the standards are stagnant, the current system rewards churning of work, and piecemeal development is the only way to meet the performance standards.

Based on information from our site visits, we found that employees generally feel that the work credit values in the current work credit system do not accurately reflect the amount of time required for each action. In particular, each action is too broadly defined to account for the large degree of variation in complexity across claims. The perceived inaccuracy of the work credits combined with the requirement to meet minimum work credit standards and the perceived emphasis on quantity over quality result in some unintended consequences for the way in which claims are processed.

One of the most basic questions to consider about the employee work credit system concerns the actual work credit values. Both frontline employees and managers were overwhelmingly in agreement that the work credits assigned to each individual action do not always accurately reflect how much time is needed to perform that action for each claim. Respondents reported multiple ways in which the work credit values are not accurate. Some actions receive too much credit for the average claim, some actions receive too little credit for the average claim, and some actions receive no credit at all.

There is tremendous variation in the complexity of claims, and based on information from our site visit interviews, the actions in the work credit system are not defined at a level of detail to reflect that variation. This implies that even if the work credit for an action is accurate on average for claims worked over an extended period of time, there’s no assurance to the employees that the work credit they receive for that action on any particular claim reflects the actual amount of time required to be spent.

The two factors that respondents reported as the main determinants of the difficulty of processing a claim were the number of issues and the types of medical conditions. For some actions, the current system allows extra work credit for claims with eight or more issues and for claims with certain medical conditions that are difficult to adjudicate (e.g., post-traumatic stress disorder). However, there is still a lot of variation that is not taken into account. There was general agreement that the number of issues on a claim is not adequately addressed by the work credits system. In particular, the ranges for the number of issues (1 to 7 issues and 8 or more issues) are too broad, and performing an action for a claim with 7 issues should receive more credit than for a claim with 1 issue.

There was also agreement that the particular medical condition or conditions on a claim have a big effect on how much time a claim requires. For example, respondents reported that actions on claims for hearing loss are typically relatively straightforward and might even be over-credited, whereas actions on claims with complex medical conditions (such as traumatic brain injury) might merit additional credit.

Some respondents indicated that one of the reasons they believe that work credits do not accurately reflect the effort involved in various actions is that the system has not been revised to account for changes in the types of claims submitted. They specifically cited increases in the number and complexity of issues.

Based on information from our site visits, one unfortunate effect of the current work credit values is that VSRs tend not to use phone calls to veterans to expedite development because there are generally no separate work credits for those phone calls during the development phase, even though sometimes a phone call is the most direct way to resolve a question. Some of the VSRs who do make use of phone calls said they have learned to work around the absence of work credits. They do this by following up a phone call (for which they receive no credit) with a letter (for which they do receive credit), even for cases where they feel the letter isn’t necessary. Thus, the lack of work credits for phone calls slows development down both by discouraging some VSRs from using a potentially good source of information and by causing others to perform an unnecessary activity (writing a follow-up letter) in order to get the work credits that they need.

We also heard about some examples of churning or piecemeal development of claims such as spreading development over multiple days to classify them as separate actions and therefore be able to claim multiple credits. However, we do not know exactly how frequent or rare piecemeal development is. We observed that most frontline employees are genuinely concerned with serving veterans and their families and that these types of activities are simply responses to the production pressures that many employees feel.

Figure 1 shows the distribution of the average number of work credits claimed per completed claim for all the ROs. If all ROs were processing the same set of cases in the same way, and if all employees were claiming the same work credits for the same actions, then we would expect minimal variation across ROs. Instead, we found that the average number of work credits per claim at each RO ranged from 5.2 to 17.7, with substantial variation within that range. These results do not account for average differences in the types of cases across ROs, and those differences certainly could explain some of the differences in work credits per claim. However, the sizable magnitude of variation raises questions.

Figure 1.   RO’s average work credits per claim

The existence of unintended consequences such as rushed actions and piecemeal development, have negative effects on both the quality and timeliness of claims processing. The first step toward eliminating them should be a pilot to develop a set of actions and associated work credits that accurately reflect, and are perceived to accurately reflect, the time required to perform each action at a targeted level of quality. Those work credit values can then be combined with information on expected caseload to determine the staff required to process that caseload at that quality level. Because these changes will not be quick to implement, VBA should determine what the trade-off between quality and quantity is under current resource constraints and then explicitly decide on which levels of quality and quantity best contribute to accomplishing VA’s mission.

Recommendations

To address both the issues of work credit values and increasing the perceived emphasis on quality, we recommend conducting a pilot test of the effectiveness of taking the following steps, in the order indicated:

1. Develop a list of actions defined so that there is little variation across claims in the time required to perform the action. This will probably require accounting for the number of issues and the specific medical conditions involved.
2. Determine the time that is required (i.e., the work credits) to perform each action at a certain average level of quality. This will require incorporating the fact that employees with different levels of experience (i.e., different GS levels) will need different amounts of time to achieve the same average level of quality.
3. Communicate information to employees about the methodology used in developing the new work credits. The goal is for employees to understand that the work credits accurately reflect the time that should be required for each action. In addition, they should understand how accuracy of claims processing was explicitly incorporated into the development of the work credits and thus how important quality is to VBA.
4. Set the daily work credit performance standards for each individual employee to match the number of available work hours per day. Since the work credits from step #2 above will have been developed to equal the time required for each action, setting daily standards to match the time available ensures that employees should not perceive a need to rush. This recommendation is for staff’s ease of understanding work credit values.
5. Set the RO production standards so that they can be attained when employees are working at the work credit standards from step #4 above. (This contrasts with what we heard about the current RO standards, which apparently cannot be met if all frontline employees are working only at their minimum work credit standards.)
6. Develop mathematical models to predict how the number and complexity of claims translate into the number of employees needed to complete those claims. Then, apply the model to the predicted caseload to calculate the number of employees needed in the future to handle that caseload. Plan to increase or decrease the number of claims processing employees accordingly.
7. Conduct ongoing analysis and revisions of the actions, work credit values, and number of employees needed. The ongoing analysis is required to account for the fact that there are continuing changes in the complexity of claims, in the legal requirements about what must be done for each claim, and in IT system capabilities.

We consider the following options for redefining actions (step 1) as only two examples among the numerous possible alternative approaches.

One option is to base the detailed list of actions on the four main types of actions proposed by the VSR Performance Standards Workgroup, which are initiating development, making a claim ready to rate, deciding an award, and authorizing the award. Within each of those four types of actions, the work credit values would vary to account for differences in complexity of claims by number of issues, types of medical conditions, and any other factors found to be good predictors of how much time the action requires.

One advantage of this approach would be that employees could focus more on the actual performance of actions rather than spending time recording a lot of separate smaller actions. Another advantage would be that employees would have a lot of incentive to perform only those activities that make a definite contribution to completing the claim. A disadvantage of this approach would be that work credits would be “lumpy,” by which we mean they would be relatively large but there would be only a few points in processing a claim at which work credits could be claimed. This could be a problem if there were a day or even a week in which a VSR was in the middle of intensive development activities for a large number of claims, but by chance none of the claims became ready to rate during that day or week because the external parties from whom supporting information had been requested were not responding. In that case, the VSR would earn zero credits for that time period.

A second option is to base the detailed list of actions on a comprehensive list of all the individual activities that employees perform in the process of doing their jobs. The main advantage and disadvantage of this option would be directly opposite to those described in the first example above. Specifically, the disadvantages would be that not all actions make a direct contribution to completing the claim and that the frontline employees would need to spend time recording each of their many separate actions in ASPEN (Automated Standardized Performance Elements Nationwide). However, the advantage would be that this approach would produce detailed records on each frontline employee’s specific activities, which would provide both frontline employees and managers with very precise information on exactly how much work the employees have done for any given time period and therefore how well they are on track to meet the month’s minimum work credit standards.

In addition to the seven recommended steps, we also have the following individual recommendations for actions that would contribute to the effectiveness of the work credit system:

• As discussed earlier, conducting more quality reviews would better communicate to staff that quality is a priority for VBA.
• If the RO internal quality reviews counted deviations from official procedures for claiming work credits as errors (instead of just as “comments”), then that would improve adherence to procedures, thereby improving quality.
• If there were work credit deductions for actions on which there were errors, then that would improve quality.
• In the long term, if VETSNET could be modified so that it captures work credits automatically as a claim moves through the stages of processing, then that would save time for employees in recording their work credits, and it would ensure that work credits are logged more accurately and consistently across staff and ROs.

Work management system (CPI Model)

VBA’s current work management system is the Claims Process Improvement Model. The CPI model is not VBA’s first attempt to manage the claims process by dividing it into separate stages and passing a claim along to a different person at each stage. In the 1990s, VBA was using a similar model called the “unit model.” That model resembled the CPI model in that employees were specialized so that the various functions of claims processing (e.g., development, rating, authorization) were performed by different people. The main difference between the unit model and the CPI model was that employees were not organized into teams in which everyone on the team performed the same function. Instead, each team consisted of employees who performed different functions so that they collectively covered all the necessary claims processing functions.

In the late 1990s, there was a business process reengineering (BPR) effort in which a VBA guidance team analyzed the key challenges facing claims processing. Among those challenges were a lack of individual accountability and an emphasis on quantity of work over quality. The team’s solution was a “case management model” in which each claim was handled by only one VSR and a rater who made the rating decision. VBA used this model until it adopted the CPI model in response to concerns about problems with timeliness.

It’s not difficult to see the cyclical pattern of recurring concerns that led to switching between two basic types of approaches (more vs. less employee specialization), each of which has different advantages and disadvantages. The key question then is which approach has stronger advantages than disadvantages, and based on our synthesis of information that we obtained from our site visits, the answer is not straightforward.

The main motivation for studying the CPI model is the concern that quality and accountability are lacking because many different people are involved in processing each claim. Our analysis finds that two of the main distinguishing features of the CPI model, i.e., the specialization of VSRs and the fact that claims are passed through a series of specialized teams, have both advantages and disadvantages for the quality and timeliness of claims processing. In particular, VSR specialization improves quality and timeliness (compared to a model with less specialization but more continuity in the staff who work on each claim), but the movement of claims across teams reduces quality and timeliness. The issue is to determine whether the improvements outweigh the disadvantages. Thus, the net effect of the CPI model is unclear, which means that before making any changes to its approach to claims processing, VBA should conduct a pilot study to confirm that those changes will actually produce the desired net improvements.

At the time of our report, VBA was already conducting a pilot study to test an alternative to the CPI model that, based on its design, appears to have the potential to improve both quality and timeliness. Consequently, we recommend that VBA wait for the results of that study before deciding whether it would be worthwhile to investigate other alternatives.

Specifically, VA was conducting a pilot study in the Little Rock RO, and one of its main elements is testing the use of claims processing teams (called “pods”) that contain pre-determination, rating, and post-determination functions. One of the expected benefits of organizing employees into pods is “improved quality resulting from more rapid identification and resolution of errors within the team (i.e., errors will be detected much closer to the point of occurrence)”.[5]  This approach also retains the current specialization of VSRs. Thus, the current pilot study is testing a model that appears to have reasonable potential to improve quality compared to the CPI model, especially if organizing employees into pods creates a greater sense of accountability than in the CPI model’s function-based teams. Consequently, we do not see any reason to pilot test another approach (e.g., the hypothetical one we consider above) unless the results of the current pilot show that the use of pods does not result in claims processing outcomes any better than those obtained under the CPI model.

It is also important for all stakeholders to consider the possibility that different work management systems might ultimately produce similar outcomes. On our site visits, we heard from a number of managers who thought that the specific claim processing model is less important than managers’ abilities in implementing it. This line of thinking suggests that only limited improvements could be obtained from changing the approach to claims processing. VA should certainly be using the approach that will best produce the quality and timeliness results it wants. However, other factors, such as certain IT improvements, could easily have much more impact than changes to the claims processing approach.

Other study topics

In the area of IT use, VA has been proceeding with its efforts to increase the use of paperless processing, and that strategy seems to have the most long run potential for improving timeliness and quality. VA has investigated the possibility of using rules-based applications for the rating decision, but the subjective nature of many of the current VASRD criteria would make implementation of that approach extremely challenging.

Timely development of claims is essential to the timely completion of claims, and VBA could improve development time by encouraging more use of telephone contacts to obtain information from claimants and third-party organizations. The best way to encourage this would be to provide appropriate work credit for phone development. Currently, some employees who use phone contacts for development report that they also follow up with a letter that’s not always necessary because sending the letter is what enables them to claim sufficient work credits to meet their minimum performance standards.

On the topic of claims that are ready to rate at the time they’re submitted, we found that there are no standard practices across ROs for handling those claims. There is currently a pilot study to determine the potential for a program in which “fully developed claims” receive expedited treatment, and depending on the results of that pilot, it is possible that VA will establish such a program at all ROs. If it does, then it seems likely that any special procedures that ROs have developed for ready-to-rate claims would be superseded by the program for fully developed claims.

Another category of claims that we were asked to address is claims from SI and VSI veterans. The practice reported to be effective in ensuring that those claims are processed promptly is to designate specific individuals to be responsible for following those claims extremely closely through all phases of processing.

Until recently, VBA did not formally assess or disseminate best practices for claims processing. Instead, managers tended to learn about practices at other ROs through informal contacts. We heard from managers that they would like more access to information on alternative approaches that other ROs have found to be successful, but because of the many differences among ROs, the managers would prefer that any practice reported to work well at one RO be presented as a suggestion rather than as a requirement. In July 2009, VBA issued a standard operating procedure for identifying best practices and disseminating them on their Quality Assurance Web site. So, in the future, it should be easier for managers at ROs to access information regarding best practices.

[1] U.S. Government Accountability Office. Veterans’ Disability Benefits: Preliminary Findings on Claims Processing Trends and Improvement Efforts. Testimony of Daniel Bertoni before the Committee on Veterans’ Affairs, U.S. Senate, 20 July 2009, GAO-09-910T.

[2] Ibid.

[3] U.S. Department of Veterans Affairs, Office of the Inspector General. Review of State Variances in VA Disability Compensation Payments. Report No. 05-00765-137. May 2005.

[4] D. Harris. Findings from Raters and VSOs Surveys, May 2007 (D0015934.A2).

[5] Booz Allen Hamilton. Veterans Benefits Administration Compensation and Pension Claims Development Cycle Study. Final Report. 5 June 2009.