Analysis of the Practice Guidelines of the Dutch College of General Practitioners with Respect to the Use of Blood Tests
- Corresdpondence and reprints: Marc A. M. van Wijk, MD, Grabijnhof 2A, 2625 LL Delft, The Netherlands. e-mail: 〈mvanwyk{at}casema.net〉
- Received 11 November 1998
- Accepted 17 February 1999
Abstract
Objective To determine the consistency among the practice guidelines of the Dutch College of General Practitioners with respect to the use of blood tests.
Methods The authors evaluated 64 practice guidelines of the Dutch College of General Practitioners. For each guideline, they analyzed each sentence that contained a reference to a blood test to determine the clinical situation in which the test should be performed (the indication) and to determine the tests that should be performed in that situation (the recommended test). An incomplete recommendation refers to a guideline that mentioned a blood test but did not identify the indication for that test. An inconsistency refers to the situation in which one guideline recommended a certain test for a given indication whereas another guideline mentioned the same indication but did not recommend the same test.
Results Twenty-seven practice guidelines mentioned blood tests. Of these, three explicitly recommended not to request blood tests. Five guidelines contained incomplete recommendations, and the authors encountered two inconsistencies among the guidelines. Twenty-three guidelines mentioned blood tests and allowed the authors to identify indications and recommended tests.
Conclusion The identification of indications and recommended tests allows evaluation of consistency among practice guidelines. Although some incomplete recommendations and inconsistencies were discovered, the majority of the guidelines provide clear and unambiguous recommendations for blood-test ordering in primary care.
To deal with the rapidly expanding amount of medical knowledge, guidelines are viewed increasingly as a mechanism for distributing knowledge to practitioners.1 2 Governmental agencies and professional organizations are developing clinical practice guidelines. In the Netherlands, the Dutch College of General Practitioners issues guidelines for the general practitioner.
These guidelines are published regularly in “Huisarts en Wetenschap,” the journal of the college. These guidelines assist general practitioners in dealing with specific clinical conditions in a primary care setting.
A number of studies have shown that the existence of guidelines does not necessarily lead to the use of these guidelines by physicians. Even when authoritative guidelines are available, changing the behavior of physicians has proved difficult.3 4 Investigators acknowledge that the implementation of guidelines constitutes an important research area that has to be addressed.5
One mechanism for implementing guidelines is using information technology to develop decision support systems based on guidelines. Decision support systems based on guidelines may focus on supporting a single guideline for a particular disease, e.g., heart failure, asthma, or diabetes. The objective of the system is to help the practitioner in the management of a particular disease using the appropriate guideline. Such a system typically covers several aspects of care, providing recommendations for diagnostic investigations, selection of treatment, and follow-up. In this approach, the paper-based guidelines are replaced by electronic guidelines. Experience, however, has shown that researchers developing decision support systems based on a guideline may encounter significant problems such as inconsistencies in the guideline, inaccurate or incomplete descriptions of terms, ambiguity, or incompleteness.6 7 8 9 10 11 This change from paper guidelines to a decision support system, therefore, requires an extensive analysis of the content of the guidelines.
Unlike systems that focus on a single guideline, we focus on the collection of guidelines issued by the Dutch College of General Practitioners. Discrepancies and inconsistencies among different guidelines that are dealing with similar issues may further aggravate the problems encountered by developers of systems based on individual guidelines. Several guidelines, for example, may refer to the same diagnostic investigation, disease, or treatment. The guidelines, however, do not necessarily agree on the recommend course of action. Given the procedures by which these guidelines are developed, such inconsistencies are possible; the development of a guideline is not just a scientific endeavor, but the human factor plays an important role.12
The procedure of creating a guideline consists of four stages.12 The first stage involves the selection of appropriate topics for new guidelines by an independent advisory board. The guidelines are intended for use by general practitioners; the topics selected and the level of detail thus reflect practice in primary care. Although criteria for selecting topics are articulated, the process of selecting topics is partly subjective. In the second stage, a small task force consisting of four to eight general practitioners with special interest and expertise in the topic of that guideline prepare a draft. This draft is based on a review of the available literature and current medical practice. As a result, the draft reflects not only scientific evidence, but also the consensus of the task force with respect to appropriate medical practice in primary care. In the third stage, this draft is peer-reviewed by a random sample of 50 Dutch general practitioners and a number of specialists. The fourth and final stage involves the authorization of the guideline by a board consisting of leading general practitioners including the chairs of the university departments of general practice. After authorization, the guideline is published in the journal of the Dutch general practitioners. This publication consists of three parts: a brief, algorithmic summary of the guideline that focuses on the decisions the general practitioner has to make; a more detailed description of the guideline itself; and a scientific justification of the guideline. The brief, algorithmic summary of the guideline is also distributed as separate card that can be used during consultations. In addition, teaching material is prepared that can be used for continuing medical education.
Although the available scientific evidence plays an important role, the Dutch College of General Practitioners acknowledges that each guideline is, to varying degrees, dependent on the subjective opinions of the individuals involved in the creation of that guideline.12 Each guideline is based on arguments of the individual members of the task force and subsequent reviewers. For each guideline, however, different general practitioners participate in the task force. The process of developing guidelines, therefore, does not guarantee consistency. To develop a decision support system that provides the general practitioner with recommendations based on all the available guidelines, these guidelines need to be analyzed and evaluated for inconsistencies within an individual guideline and for inconsistencies among guidelines. In order to focus the analysis, we restrict ourselves to the recommendations for blood tests. The choice for recommendations for blood tests is based on previous Dutch research.
Requesting blood tests is an important aspect of the health care delivered by the general practitioner in the Netherlands. Although this proportion is lower than in many other European countries,13 about 4 percent of patients' encounters with Dutch general practitioners result in the physician requesting blood tests.14 Physicians' use of blood tests, however, is not always appropriate.1 15 16 17 18 19 Dutch investigators report a lack of general practitioners' knowledge concerning the indications for blood tests leading to inappropriate and inadequate use of diagnostic tests.20 The need to improve the use of blood tests, however, is not limited to the Netherlands. Other investigators argue that improving the quality of blood test ordering deserves attention.21 22 23
The objective of this study is to evaluate the guidelines of the Dutch College of General Practitioners with respect to the ordering of blood tests. We want to determine whether these guidelines provide a consistent base for the development of a decision support system for blood test ordering.
Methods
Up to January 1, 1998, the Dutch College of General Practitioners had published 64 guidelines. The college regularly updates the guidelines. We analyzed the most recent version of each guideline that was available on January 1, 1998. Changes in the guidelines after this date are not included in this study.
For each of the guidelines, we analyzed each sentence to determine whether that sentence contained a reference to blood tests. If the sentence contained a reference to blood tests, we determined the clinical situation in which the test should be performed (the indication) and determined the tests that should be performed in that situation (the recommended test). An incomplete recommendation refers to a guideline that mentioned a blood test, but did not identify in the guideline the indication for that test, or a guideline that mentioned an indication for blood tests but did not provide a further specification of the recommended blood tests. The notion of an incomplete recommendation is restricted to particular recommendations; we do not determine whether the total set of recommendations is “complete” in the sense that the set covers all indications in primary care.
After we identified the indications and recommended tests in all guidelines, we checked—for each indication in each guideline—whether another guideline recommended another test for the same indication. A inconsistency refers to the situation in which one guideline recommended a certain test for a given indication whereas another guideline mentioned the same indication but did not recommend the same test.
Results
Of the 64 guidelines, 27 contained at least one sentence that included a reference to blood tests. Of the 27 guidelines that mentioned blood tests, 3 explicitly recommended not to request blood tests. The guideline “Sinusitis” states that sinusitis itself is not an indication for measurement of the erythrocyte sedimentation rate (ESR),24 the guideline “Depression” states that depression itself is not an indication for measurement of thyroid-stimulating hormone (TSH) or thyroxine (T4),25 and the guideline “Blood tests and liver disease” states that infectious mononucleosis is not itself an indication for liver function tests.26
Incomplete Recommendations
Of the 27 guidelines containing at least one sentence that included a reference to blood tests, 5 guidelines contained incomplete recommendations; that is, a guideline mentioned a blood test but did not describe the indication. The guideline “Imminent miscarriage” mentions a possible hemoglobin test but does not specify which patients are eligible.27 The guideline “Children with fever” states that blood tests are seldom indicated; when blood tests are indicated, and which tests should be requested, however, are not specified.28 The guideline “Problematic alcohol consumption” identifies abnormal values of glutamyltransferase (gamma-GT), aspartate aminotransferase (ASAT), and alanine aminotransferase (ALAT) as possible indicators of alcohol abuse but does not describe if and when these tests should be performed.29 The guideline “Intrauterine device” mentions the possibility of elevated erythrocyte sedimentation rate (ESR) and leukocytosis but does not specify if and when tests for these should be performed.30 The guideline “Acne vulgaris” states that, prior to treatment with isotretinoin, liver and kidney functions should be evaluated; the guideline, however, provides no further specification of which tests should be done.31
Indications and Advised Tests
Of the 64 guidelines, 23 mentioned blood tests and allowed us to identify the indication for those tests.26 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 We distinguish five different categories of indications. The first category of indications describes clinical situations in which the general practitioner considers a diagnosis, the working diagnosis. This working diagnosis is the most probable diagnosis based on the patient's medical history or physical findings, or both. The physician subsequently uses the laboratory tests to support or refute that diagnosis. In total, the guidelines mention 18 working diagnoses. Table 1 shows the working diagnosis, the recommended tests, and the guideline that makes the recommendation. In some cases, abnormal results of initial tests should be followed by additional investigations; for example, an abnormal value for TSH should be followed by a test for free T4.
Recommended Tests for Each Working Diagnosis
The second category of indications describes clinical situations in which the general practitioner has established a diagnosis and uses the laboratory to investigate the underlying pathology that could cause the disease. The crucial difference with the category working diagnosis is that in case of underlying pathology the guideline requires that the physician has already established the presence of a specific diagnosis. Given the presence of this specific diagnosis, the guideline specifies the evaluation of possible causes. In total, the guidelines mention ten diagnoses in which underlying pathology needs to be explored. Table 2 shows the diagnoses, the suspected underlying pathology, the recommended tests, and the guideline that makes that recommendation. For example, when the diagnosis transient ischemic attack (TIA) has been established, the guideline TIA recommends that tests for ESR and glucose be requested to explore arteritis temporalis and diabetes mellitus as underlying causes of the TIA.
Recommended Tests for the Underlying Disease, for Each Diagnosis
The third category involves monitoring the course of a disease. The physician has established the diagnosis and is monitoring the progression of the disease. In total, six guidelines mention diagnoses that can be monitored. Table 3 shows the established diagnosis, the tests advised to monitor that condition, and the guideline that makes that recommendation. For example, to monitor diabetes mellitus, the guideline “diabetes mellitus” recommends that the physician obtain a glucose measurement every three months, and cholesterol and creatinine measurements annually.
Recommended Tests for Monitoring the Course of Disease, for Each Diagnosis
The fourth category of indications describes situations in which blood tests are used to select appropriate treatment. In these situations, the physician has established the diagnosis, and the blood tests are used to identify factors that have a direct bearing on the choice of subsequent treatment. Based on the results of these blood tests, the guideline specifies the treatment of choice. In total, seven guidelines identify blood tests that are used to select treatment. Table 4 shows the established diagnosis, the factor that is identified, the recommended tests, and the guideline that makes the recommendation. For example, the guideline “Cholesterol” recommends in case of hypercholesterolemia the measurement of high-density lipoprotein cholesterol and triglycerides to identify lipid metabolism disorder, in order to select the right therapy.
Recommended Tests for Selecting Appropriate Treatment, for Each Diagnosis
The fifth category of indications describes situations in which blood tests are used to monitor the side effects of drugs. The guidelines state that for certain drugs, the physician should monitor the patient for potential side effects. In some instances, this requires periodic blood tests. The results of the blood tests might lead to modification of prescribe dosages or termination of treatment with that drug. In total, four guidelines identify blood tests that need to be performed to monitor side effects of drugs. Table 5 shows the drugs involved, the side effects to be monitored, the recommended tests, the frequency of performing these tests, and the guideline that makes the recommendation. For example, the guideline “Acne vulgaris” states that cholesterol and triglycerides need to be measured one month after isotretinoin treatment is started and subsequently every three months.
Recommended Tests for Measuring Side Effects of Therapy
Inconsistencies
We encountered two inconsistencies among the guidelines. The guideline “Angina pectoris”46 showed an inconsistency with the guideline “Disorder of the thyroid gland.”45 The guideline “Shoulder complaints”32 showed an inconsistency with the guideline “Rheumatoid arthritis.”47
According to the guideline “Angina pectoris,” the general practitioner should, in case of angina pectoris in combination with tachycardia, request TSH testing to evaluate hyperthyroidism. The guideline “Disorder of the thyroid gland” describes when a TSH value should be obtained; patients with angina pectoris and tachycardia, however, are not mentioned.
The guideline “Shoulder complaints” states that insufficient effect of initial treatment is a reason for blood tests; an elevated ESR is an indicator for rheumatoid or septic arthritis. In the guideline “Rheumatoid arthritis,” however, the list of advised tests for excluding or confirming rheumatoid arthritis does not include ESR.
Discussion
The objective of this study was to identify in all guidelines issued by Dutch College of General Practitioners the specific recommendations for using the laboratory and to analyze these recommendations for inconsistencies. The underlying reason for such an analysis was the desire to build a decision support system that would help general practitioners in using these guidelines.
Guidelines
Our study shows that the guidelines contain specific and detailed recommendations for ordering blood tests. Given the fact that previous studies have reported a lack of general practitioners' knowledge concerning indications for tests,18 20 the guidelines could provide needed support. These recommendations, however, are scattered throughout many different guidelines (a total of 27 of the 64 practice guidelines). In addition, the guidelines may overlap. For example, the guideline “Problematic alcohol consumption” states that increased levels of gamma-GT, ASAT, and ALAT are possible indicators of excessive alcohol abuse; the guideline does not describe if or when tests for these substances should be performed. The guideline “Blood tests and liver disease” specifies that if the practitioner suspects alcohol-induced hepatitis, ALAT and gamma-GT tests should be performed; the ASAT is in this guideline considered redundant. We conclude that the currently available paper-based guidelines require the general practitioner to spend time and effort locating and interpreting the recommendations for blood tests.
Two inconsistencies were found among the guidelines with respect to the use of the laboratory. A possible explanation for these inconsistencies could be the fact that, although the guidelines are revised regularly, not all guidelines are revised at the same time. As a result, one guideline may already reflect changed medical understanding, whereas another, possibly due for revision in the near future, does not yet reflect this change. Given that inconsistencies were found, we recommend that organizations that maintain a set of guidelines should make available to physicians a list of known inconsistencies among those guidelines.
Our analysis shows that, with respect to the use of the laboratory tests, five guidelines contained incomplete recommendations. Ambiguity or lack of clarity in guidelines could create uncertainty on the part of the general practitioner that in turn could stimulate the ordering of unnecessary blood tests. Guideline developers should, therefore, avoid incomplete recommendations in guidelines. Twenty-three guidelines, however, did contain well-defined and specific recommendations for the use of the laboratory. Given that other investigators have reported a lack of knowledge about test ordering, we believe that applying the guidelines in general practice would result in improved test ordering by general practitioners.
Decision Support System
From the perspective of medical informatics, the objective of our study was to identify the specific recommendations for using the laboratory, and analyze them for inconsistencies. From this perspective we conclude that the guidelines contain concrete and specific recommendations and that only few inconsistencies were found. Moreover, the study shows that identifying the indication for requesting blood test is one possible method for analyzing the guidelines. The indication for blood tests is the specific question of the physician to which obtaining the test will provide a (partial) answer. We conclude that the recommendations for blood tests in the guidelines of the Dutch College of General Practitioners are focused on describing what tests are necessary in the context of a given indication.
For researchers in medical informatics, this notion of the indication as the physician's question has significant consequences for designing a decision support system. Given the concept of the indication, the designer faces a choice. The first alternative is, given the patient's symptoms, that the system identifies the indication for blood tests. The second alternative is, given the indication, the system selects the appropriate tests. For the general practitioner, these two approaches result in two very different systems. If the objective is to support the identification of the indication, the decision support system will request detailed information about the patient's condition. Based on these findings, the system will generate possible indications, select among these, and prepare a recommendation. If the objective of the system is to select the appropriate test given the indication, the system will ask the physician questions about the indication; the translation of the patient's condition to a specific indication is left to the general practitioner. The system builder thus has to determine whether decision support is based on the patient's symptoms or on the physician's indications.
We have decided to build a system that requests from the physician the indication. As a result, the system is driven by questions related to the objectives of the physician. The system does not ask detailed questions about the symptoms or complaints of the patient. The downside of this approach is that the system does not support the physician in establishing the appropriate indication based on the complaints of the patient. This decision to leave the identification of the initial working diagnosis to the general practitioner is based partly on the fact that the guidelines provide recommendations on the level of indication. It is also based partly on our assertion that physicians especially are able to translate the often-complex presentation of patients' complaints into well-defined indications. In addition, a general practitioner in the Netherlands sees a very different patient population when compared with a specialist working in a hospital, since the prevalence of diseases is different. The patient's complaints presented in general practice might well result in selection of a different working diagnosis than in a hospital setting. Computers are able to deal only with those parts of the patient-physician encounter that can be expressed as objective facts and numbers; as a result, decision support systems can only deal with only a very limited segment of reality. Other investigators have noted that decision support systems tend to ignore the intellect of physicians53 and leave the practitioner with a sense of losing control.54 We believe that in the initial interpretation of a patient's symptoms, the role of a decision support system should be very limited.
Further research is needed to show whether decision support based on the guidelines is acceptable and effective.
