Overview

1. My comments describe the applicability of Biomedical Informatics to improving the processes for determining Veterans’ eligibility for disability compensation.  
2. Clinical informatics involves information management and decision-making during healthcare delivery. Expert systems, which utilize artificial intelligence techniques, represent a subset of the more general decision support techniques and electronic health record approaches that clinical informatics provides.
3. I generally endorse the findings of the June, 2007 Institute of Medicine Report, “A 21st Century System for Evaluating Veterans for Disability Benefits”. That report lacks adequate detail in some areas pertinent to clinical informatics.  I clarify below how informatics can make a difference.
4. Clinical informatics can improve both the speed and quality of disability determinations for U.S. Veterans. The highly acclaimed VistA and CAPRI systems developed by the Veterans Health Administration (VHA) and Veterans Benefits Administration (VBA) provide excellent examples of relevant clinical informatics applications.  Future informatics efforts can help in five important areas by:

1. converting paper-dependent processes to electronic processes for all stages of disability determination and designating VistA as the definitive repository forsuch information, with supporting software tools such as CAPRI available in whatever venues are appropriate;
2. creating electronic definitions for each of the approximately 700 disability conditions, based on current CFR 38 definitions and amendments, that utilize standardized terminology for concepts and findings (e.g., the SNOMED-CT, LOINC, DSM, and ICD terminologies);
3. creating a tracking system that captures electronically all information relevant to each Veteran’s disability determination. The system would provide VBA raters with a checklist to determine what documents they require, a method to order necessary tests and procedures, and a dashboard to indicate the status of each document needed to complete a review. These would help raters to locate and retrieve the information efficiently, as well as to determine what information was incomplete or missing, and limit unnecessary duplicate/repeated testing that delays disability determinations and increases costs;
4. creating decision support tools to identify and display disability-specific patient information to VBA raters in a manner that allows them to determine easily which disability criteria have been met or not met, and to recommend appropriate next actions electronically; and,
5. creating a quality feedback loop using current information to evolve future practices through ongoing continuous improvement.

5. In matters as important as Veterans’ disability determination, computer-based tools, including “expert systems”, can serve as adjuncts to help humans to collect and manage information, but the tools cannot in any way replace the most important aspects of human judgment.  Present and future computer-based tools will not displace the talented, experienced people who comprise the present VHA and VBA. Informatics can help people to work “smarter”, in order to benefit Veterans.
6. A key lesson from clinical informatics is that while change can be beneficial, it can also be disruptive if employees must dramatically and abruptly alter their work processes. Whatever plan of action the government adopts to move from current state to a more ideal future, the plan must be pragmatic, and coordinated to proceed in concrete, non-disruptive steps. Each step must convey benefits and lay the foundation for subsequent steps with greater benefits. Changes must be gradual and familiar to already overburdened employees. This document describes such scenarios.

Figure 1 illustrates applicability of the above ideas and principles to various stages of disability determination: definition of disability, documenting conditions during active military duty, documentation of health and disability data within the Veterans Health Administration system; and, determination of disability status by VBA raters.

I will describe the applicability of biomedical informatics to the task of improving the processes and workflows used to determine Veterans’ eligibility for disability compensation. Clinical informatics involves information management and decision-making during healthcare delivery. Expert systems, which utilize artificial intelligence techniques, are a subset of the more general decision support techniques and electronic health record approaches that clinical informatics provides.

I would first like to place the role of computer-based decision support tools in proper perspective.  As I detailed in 1990 in an article in the Journal of Medicine and Philosophy [1], for matters as important as Veterans’ disability determination, computer-based tools, including “expert systems”, can serve as adjuncts to help humans to collect and manage information, but the tools cannot in any way replace the most important aspects of human judgment. If a patient grimaces and winces while struggling to walk across a room, and then claims, “I’m OK, doc”, a compassionate human can correctly categorize the patient’s condition, but an expert system is far less likely to do so if it only has “I’m OK, doc” as the patient description.  Present and future computer-based tools will not displace the talented, experienced people who comprise the present VHA system.  Nevertheless, the VHA system can achieve significant progress and efficiency through greater application of electronic information systems in determining and tracking Veterans’ disability benefits.I generally endorse the findings of the June, 2007 Institute of Medicine Report, “A 21st Century System for Evaluating Veterans for Disability Benefits”. That report lacks adequate detail in some areas pertinent to clinical informatics.  I clarify below how informatics can make a difference.  Clinical informatics can improve both the speed and quality of disability determinations for U.S. Veterans. The highly acclaimed VistA and CAPRI systems developed by the VHA provide excellent examples of relevant clinical informatics applications.  I will refer to the principles and recommendations listed in Figure 1 and explain each.

PHASE ONE -- BUILD INFRASTRUCTURE TO AUTOMATE STEPS OF EXISTING DISABILITY DETERMINATION SYSTEM, TO GAIN EFFICIENCY

The first task is to identify disability-related records and documents that exist now in paper format and to convert them, whenever possible, into a standardized, electronically processable format. As noted in Figure 1, Recommendation 1.2, this involves creating standardized names for all document types used in disability determination (there is already a history of being able to do so within certain segments of  the VHA system -- see, for example [2]).  A review of a sufficient number of existing disability records (paper and electronic) used to establish (or deny) each disability category for hundreds to thousands of Veterans would help to create a complete list.  Having specific names for each type of document makes it easier for VBA raters to find and manipulate them.

In changing to a more electronic disability determination system, one must be careful to convert almost all routine activities of VHA raters to being electronically based, with actions analogous to what they now do with paper.  It would potentially be worse -- more cumbersome and slower for disability determinations -- if VBA raters had to use both paper and electronic record systems in processing a Veteran’s application, than to use only one or the other system. If both paper and electronic systems were in active use, a VBA rater would always have to check both systems to see if “missing” items in one system are actually not “missing”, but present in the second system. 

The most straightforward way to begin conversion to electronic processing is to identify where paper records are currently generated, and where existing VHA software is applicable to creation of electronic versions of that paper-based information. For all other paper records that cannot be easily converted in this manner, the document naming system should be used to label them, and then they should be electronically scanned to create electronically retrievable records. Such record types should be scheduled for subsequent projects to capture them at their source -- at time of generation -- using future electronic capture tools analogous to CAPRI. The award-winning CAPRI system developed by the VHA provides templates that prompt physicians and other clinicians as they examine a patient for disability determination, and stores the information in a standard form within the VHA’s VistA electronic medical record system.  Recommendations 2.1, 2.2, 3.1, and 4.1 suggest that the DOD, the VHA healthcare providers, and the VHA disability raters rapidly move toward 100% utilization of the current version of CAPRI to capture disability-related information as it is generated, in all situations where CAPRI is applicable.  

The next stage of utility for automated systems to enhance VBA raters’ processing of disability claims would be, per Recommendation 1.1, to categorize the criteria required to establish each of the approximately 700 disability conditions specified in CFR 38 and its amendments.

To develop electronic criteria for each disability condition would require human review of the latest version of CFR 38 and amendments for each condition, and creating: (a) a list of findings, coded in SNOMED-CT or LOINC, required to be present to establish the disability, (b) a list of findings, coded in SNOMED-CT or LOINC required to be absent to establish the disability, (c) a list of findings that help to support the presence of the condition but which are not required to establish the condition, coded in SNOMED-CT or LOINC, (d) a list of the document types (using the standardized document names per Recommendation 1.2) that are relevant to determination of the specific disability condition, (e) the list of CAPRI frame identifiers that are relevant to determination of this specific disability, (f) names for each of the 700 conditions coded wherever possible in ICD, DSM, or SNOMED-CT, and (g) narrative text that describes the remaining criteria for the establishment of the specific disability condition that could not be coded in steps (a) through (c).

Once the electronic identifiers exist for information relevant to disability determination, including the CFR 38 definitions, the document names, the finding names, and the CAPRI template IDs, it is possible to create an electronic tracking system, per Recommendation 4.2, that can indicate for VBA raters which documents and findings are required to establish the disability, and what the status of each is for a given Veteran applying for disability.  An electronic dashboard that displays the status and availability of each item of information could then be constructed. It is possible that aspects of the dashboard might be shared with the Veterans who apply for disability through the “my HealteVet” web portal created by the VHA.

An adjunct to the above system would expand CAPRI to assist VBA raters in ordering the best tests and procedures to complete disability determination efficiently, and to record the reasoning and conclusions the VBA raters used to establish or deny the specific disability claim.

The recent Institute of Medicine report, “A 21st Century System for Evaluating Veterans for Disability Benefits” (National Academies of Science Press, 2007; Copyright © National Academy of Sciences.  http://www.nap.edu/catalog/11885.html) recommends: “Recommendation 6-1. VA and the Department of Defense should conduct a comprehensive multidisciplinary medical, psychosocial, and vocational evaluation of each veteran applying for disability compensation at the time of service separation.”  This should initially be done using the CAPRI system in its current state, recording the results in a VistA compatible format for future reference at VHA and VBA.

The above-described steps are somewhat straightforward, and are within the reach of existing technology, although they require substantial effort in terms of system development, security and confidentiality assurance, application testing, training of end users, and ongoing technical support.  The steps essentially comprise a basic level of automation of the current disability determination process in a manner that will assist VBA raters in carrying out their work more efficiently. Once such an infrastructure is established, substantial enhancements could be made, some of which would involve simple decision support techniques, and others of which would involve machine learning and expert system approaches.

PHASE TWO -- ENHANCE AUTOMATED INFRASTRUCTURE FOR DISABILITY DETERMINATION WITH DECISION SUPPORT FEATURES

A number of techniques developed over the past three decades for clinical decision support [3-10] are relevant to future enhancements to a VHA/VBA disability determination and documentation system.   At the national level, the VHA has been a major contributor to clinical decision support through its evolution of the VistA electronic medical record system.  In addition, many talented individuals working within the VHA and VBA have also made contributions.

One important technological approach is clinical diagnostic decision support systems [3-6], which can be probabilistic (Bayesian), criterion-based, or heuristic (“artificial intelligence” expert systems [3]) in nature.  In general, such systems take as input standardized vocabulary descriptors describing a patient’s condition (such as history, physical examination, or laboratory findings) and produce as output a ranked list of possible diagnoses and a suggested approach to determining which diagnoses are present.

A second important “expert system” technique relevant to clinical informatics is natural language text processing [8-10].  Using a target vocabulary of defined clinical terms or concepts, such as provided by the U.S. National Library of Medicine’s Unified Medical Language System Metathesaurus, or by the SNOMED-CT terminology system officially endorsed by the US government, such programs can scan a “free text” document, such as a clinical note, and identify which of the target concepts are present in the document [9].  The utility of such an approach for VBA disability determination has already been demonstrated by a pilot project to identify spinal injury related findings from free text disability exam records, and to correlate those findings with an electronic representation of the criteria used by VBA to determine disability [10].

Finally, ad hoc or heuristic approaches can combine manual techniques with semi-automated approaches to characterize clinical domains or conditions [11-12]. Such approaches have been used to derive a standardized vocabulary for patients’ problem lists from a large set of examples in free text [11], and to attempt to convert information stored in disparate DOD and VHA clinical record systems from one representation format to the other [12].

The recent Institute of Medicine report, “A 21st Century System for Evaluating Veterans for Disability Benefits” (National Academies of Science Press, 2007; Copyright © National Academy of Sciences.  http://www.nap.edu/catalog/11885.html) contained the following recommendations:

“Recommendation 3-1. The purpose of the current veterans disability compensation program as stated in statute currently is to compensate for average impairment in earning capacity, that is, work disability. This is an unduly restrictive rationale for the program and is inconsistent with current models of disability. The veterans disability compensation program should compensate for three consequences of service-connected injuries and diseases: work disability, loss of ability to engage in usual life activities other than work, and loss in quality of life.”

“Recommendation 4-1. VA should immediately update the current Rating Schedule, beginning with those body systems that have gone the longest without a comprehensive update, and devise a system for keeping it up to date. VA should reestablish a disability advisory committee to advise on changes in the Rating  Schedule.”

“Recommendation 4-6. VA should determine the feasibility of compensating for loss of quality of life by developing a tool for measuring quality of life validly and reliably in the veteran population, conducting research on the extent to which the Rating Schedule already accounts for loss in quality of life, and if it does not, developing a procedure for evaluating  and rating loss of quality of life of veterans with disabilities.”

The effort to redefine the conditions for which disability compensation is appropriate should be standards-based (ICD, DSM, SNOMED-CT, LOINC) as described above.  Text-mining and natural language processing methods could be used to determine which coded terms are currently used in disability determinations through review of the thousands of existing electronic disabilty-related VistA and CAPRI records, and from samples of paper records converted by OCR or direct typing into electronic format.  This review, coupled with the effort to extend disability criteria as recommended by the IOM Report, could result in computer-processable “criteria table” definitions for each disability condition that would maximize the objective representations of each condition (while still retaining free text if necessary to describe the aspects of human judgment required in each determination).  As previously recommended, the list of document types and procedures relevant to determination of each disability category, as well as the orders required to carry out the procedures in VistA, could be added to an expanded revision of CAPRI.

Once the above representation scheme for each disability condition was in place, an expert system using the “criteria table” approach could be developed to assist VBA raters in determining the completion status of each disability determination, and added to a more advanced version of the previously mentioned dashboard system. The AI-RHEUM expert diagnostic system [6], developed in part at the U.S. National Library of Medicine, might be used as a starting point for the proposed VHA/VBA expert system. 

A similar system could be developed for use within the DOD electronic medical record system, which would employ natural language processing and expert criteria table methods to identify portions of an active duty service individual’s record that would suggest eligibility for disability evaluations before discharge from active duty, per the IOM Report recommendation,  “Recommendation 6-1. VA and the Department of Defense should conduct a comprehensive multidisciplinary medical, psychosocial, and vocational evaluation of each veteran applying for disability compensation at the time of service separation.”

PHASE THREE -- CREATE A QUALITY FEEDBACK PROCESS TO ENHANCE AND EVOLVE THE DISABILITY RATING PROCESS OVER TIME

The recent Institute of Medicine report, “A 21st Century System for Evaluating Veterans for Disability Benefits” (National Academies of Science Press; Copyright © National Academy of Sciences.  http://www.nap.edu/catalog/11885.html)) contained the following recommendations:

“Recommendation 4-2. VA should regularly conduct research on the ability of the Rating Schedule to predict actual loss in earnings. The accuracy of the Rating Schedule to predict such losses should be evaluated using the criteria of horizontal and vertical equity.”

“Recommendation 4-3. VA should conduct research to determine if inclusion of factors in addition to medical impairment, such as age, education, and work experience, improves the ability of the Rating Schedule to predict actual losses in earnings.”

“Recommendation 4-4. VA should regularly use the results from research on the ability of the Rating Schedule to predict actual losses in earnings to revise the rating system, either by changing the rating criteria in the Rating Schedule or by adjusting the amounts of compensation associated with each rating degree.”

“Recommendation 5-1. VA should develop a process for periodic updating of the disability examination worksheets. This process should be part of, or closely linked to, the process recommended above for updating and revising the Schedule for Rating Disabilities. There should be input from the disability advisory committee recommended above (see Recommendation 4-1).”

“Recommendation 5-3. VA should establish a recurring assessment of the substantive quality and consistency, or inter-rater reliability, of examinations performed with the templates and, if the assessment ?nds problems, take steps to improve quality and consistency, for example, by revising the templates, changing the training,  or adjusting the performance standards for examiners.”

Once a fully electronic system was available that both represented criteria for disability determination electronically, and which recorded individual Veteran’s records in standardized terminologies, text mining and machine learning techniques could be used to accomplish the above-mentioned IOM objectives, and to provide feedback

for quality-based evolution of the proposed systems.

BACKGROUND INFORMATION

My own background is that I trained and became Board-certified in Internal Medicine during the 1970s. For over a quarter century, I cared for inpatients and outpatient in academic settings, including in several VHA Hospitals and Clinics. My research over the past three decades has been in the area of clinical informatics.  I was the founding Chief of the Section of Medical Informatics at the University of Pittsburgh, and the founding Chair of the Department of Biomedical Informatics at Vanderbilt. For more than two decades, I helped to develop and evaluate expert systems for medical diagnosis at the University of Pittsburgh. After moving to Vanderbilt in 1994, I helped to develop clinical decision support tools implemented within Vanderbilt’s home-grown care provider order entry (CPOE) system. That CPOE system improves quality of care, safety, and cost-effectiveness by giving advice to physicians in real-time as they care for patients.

During my career, I have been Principal Investigator on over $20 million of federal grants and contracts related to biomedical informatics.  I am currently a member of the Institute of Medicine of the National Academies of Science. I am a Past President of the American Medical Informatics Association (AMIA) and a Past President of the American College of Medical Informatics (ACMI). I currently serve as Editor-in-Chief of the Journal of the American Medical Informatics Association (JAMIA). Two of the clinical informatics systems that I helped to design and build have been disseminated commercially. My comments above have no direct relationship to those commercialization efforts, and I have no conflicts of interest in that regard.

REFERENCES

1.  Miller RA. Why the Standard View is Standard: People, not Machines, Understand Patients' Problems. J Med Philosophy. 1990; 15:581591.

2.  Brown SH, Lincoln M, Hardenbrook S, Petukhova ON, Rosenbloom ST, Carpenter P, Elkin P.  Derivation and Evaluation of a Document-naming Nomenclature.  J Am Med Inform Assoc. 2001;8(4):379-390.

3.  Duda RO, Shortliffe EH. Expert Systems Research. Science. 1983 Apr 15;220(4594):261–268.

4.  Miller RA, Pople HE Jr, Myers JD. INTERNIST1, An Experimental Computerbased Diagnostic Consultant for General Internal Medicine. N Engl J Med. 1982; 307:46876.

5. Bankowitz RA, McNeil MA, Challinor SM, Parker RC, Kapoor WN, Miller RA.  A computer-assisted medical diagnostic consultation service: implementation and prospective evaluation of a prototype.  Ann Intern Med. 1989; 110:82432.

6.  Miller RA.  Medical diagnostic decision support systems--past, present, and future: a threaded bibliography and brief commentary. J Am Med Inform Assoc. 1994;1(1):8-27.

7.  Aliferis CF, Miller RA. On the heuristic nature of medical decision-support systems. Meth Inform Med. 1995 Mar;34(1-2):5-14.

8.  Tom Mitchell. Machine Learning, McGraw-Hill: New York, NY. 1997.

9.  Uzuner O, Goldstein I, Luo Y, Kohane I. Patient Smoking Status from Medical Discharge Records. J Am Med Inform Assoc. 2008; 15: 14-24.

10.  Brown SH, Speroff T, Fielstein EM, Bauer BA, Wahner-Roedler DL, Greevy R, Elkin PL.  eQuality: Electronic quality assessment from narrative clinical reports. Mayo Clin Proc. 2006; 81(11):1472-1481.

11.  Brown S, Miller RA, Camp H, Giuse D, Walker H.  Empirical Derivation of an Electronic Clinically Useful Problem Statement System.  Ann Intern Med. 1999; 131(2):117-126.

12.  Bouhaddou O, Warnekar P, Parrish F, Do N, Mandel J, Kilbourne J, Lincoln MJ. Exchange of Computable Patient Data Between the Department of Veterans Affairs (VA) and the Department of Defense (DoD): Terminology Standards Strategy. J Am Med Inform Assoc. e-published Dec 20, 2007 doi:10.1197/jamia.M2498