A survey of SNOMED CT direct users, 2010: impressions and preferences regarding content and quality
- 1Department of Computer Science, New Jersey Institute of Technology (NJIT), Newark, New Jersey, USA
- 2Halfpenny Technologies, Inc, Blue Bell, Pennsylvania, USA
- Correspondence to Dr Gai Elhanan, Computer Science Department, New Jersey Institute of Technology, University Heights, Newark, NJ 07102-1982, USA; gelhanan{at}gmail.com
- Received 27 April 2011
- Accepted 11 July 2011
- Published Online First 11 August 2011
Abstract
Objective Little information exists concerning SNOMED CT (systematized nomenclature of medicine—clinical terms) users. This report describes current impressions and preferences of direct SNOMED CT users regarding coverage, quality, and concept details, and the change request mechanism.
Design A 43-question anonymous survey distributed electronically to relevant online communities.
Measurements Data on user demographic characteristics, modes and purposes of use, means and frequencies of access, satisfaction with SNOMED CT content coverage and quality and with the change request mechanism were recorded.
Results The survey was conducted in January 2010 and elicited 215 responses. Details regarding users' profiles, modes of use and access were reported elsewhere. The coverage of SNOMED CT was perceived to be at least 85% complete by 42% of responders, and 60% were at least satisfied with its quality. Various deficiencies were encountered at least ‘somewhat often’ by 28–61% of responders. Incorrect data were more bothersome than missing data. Users indicated that significant resources should be allocated to more consistent and complete conceptual representations and to further enhance content coverage. Enhanced synonym coverage and the introduction of textual definitions were important to users (54% and 63%, respectively).
Limitations A survey format with limited control over recruitment and selection bias. Lack of information regarding the SNOMED CT version used by responders.
Conclusion Despite overall satisfaction, direct users indicated a strong desire to improve consistency, quality, and completeness of conceptual representations and concept details, as well as a continued desire to expand coverage. The survey provides much needed data for informed decisions regarding the use and development goals of SNOMED CT. Focused periodical surveys are warranted.
- Ontologies
- knowledge representations
- controlled terminologies and vocabularies
- machine learning
- predictive modeling
- statistical learning
- privacy technology
- SNOMED
- terminology
- survey
- quality
- satisfaction
Introduction
In the current alphabet soup of US and worldwide healthcare-related terminologies, SNOMED CT1 (systematized nomenclature of medicine—clinical terms) is the premier clinical terminology. SNOMED CT (SCT) is a comprehensive clinical terminology that provides standard clinical content and expressivity for clinical documentation and reporting.2 Software applications can use its concepts, hierarchies, and relationships as a common reference point for data analysis.
SCT is poised to play a major role in US healthcare because of the health information technology (HIT) component (HITECH) of the American Recovery and Reinvestment Act of 2009.3 HITECH is designed to jumpstart the transition of US medical institutions and providers to electronic health information systems.4 The final rule5 for the initial set of standards for HIT includes SCT as one of two options to ‘enable a user to electronically record, modify, and retrieve a patient's problem list for longitudinal care’. To accelerate the adoption and ‘meaningful use’ of certified electronic health records (EHRs) by providers and institutions, a system of incentives and penalties is proposed.6 Use of SCT or the International Classification of Diseases, 9th edition, Clinical Modification to encode up-to-date problem lists of current and active diagnoses for at least 80% of all patients is proposed as one of the indications of ‘meaningful’ EHR use. Moreover, SCT is slated to become the exclusive encoding system for problem lists by 2015.
Despite SCT's momentum and many endorsements, actual use within clinical applications is sporadic and not well understood, with notable exceptions such as Kaiser Permanente's (KP) Convergent Medical Terminology (CMT7) and the Veterans Health Administration's Enterprise Reference Terminology (ERT8). Many vendors await clearer business case and clients' demand.9 Little is known about the use of SCT embedded within HIT applications, or about users who interact with SCT in a direct manner.10 HIT end users are far removed from SCT by enhanced ‘interface’ versions and third party data subsets and are limited in their ability to provide input. Hence, we focus our study on the direct users. These are the designers, developers, managers, and implementers of HIT applications that utilize SCT, as well as researchers and educators in the field of healthcare informatics. In January 2010, we conducted a survey of direct users of SCT. The survey's data on direct user's profiles and use of SCT were presented at the AMIA 2010 symposium.11 Here we present the survey's results on users' impressions and preferences related to SCT's content and qualities.
Background
SNOMED CT is a description logic based clinical terminology. Originating with SNOP (systematized nomenclature of pathology),12 SNOMED underwent several transitions, the most important of which was the convergence of SNOMED reference terminology (SNOMED RT) and the UK clinical terms version 3 (formerly known as the Read Codes)13–15 in 1999. SCT contains more than 310 000 active concepts with an extensive network of relationships among them. Since 2007, the International Health Terminology Standards Development Organization (IHTSDO16) has owned and is tasked with the continued development, promotion, and implementation of SCT and its derivative products. IHTSDO has representation from member nations, vendors, standards organizations, researchers, and other user groups.
Each SCT concept has a fully specified name, its unique descriptive term, a preferred term, and, typically, a set of synonyms. Concepts are organized into 19 top-level, singly rooted hierarchies to capture broad, clinically related groupings, such as ‘clinical finding’, ‘substance’, or ‘body structure’. Concepts within one hierarchy are linked by IS-A (subsumption) relationships such that each hierarchy forms a directed acyclic graph. Concepts also have ‘attribute relationships’ (attributes) directed to other concepts. Attributes can serve in either definitional or qualification capacities. Each kind of attribute is defined to span from a source hierarchy to a target hierarchy.2 Each concept is further classified by its status of logical definition: fully defined versus primitive. A primitive concept is underspecified in the sense of not having enough attributes to distinguish it from its parents.
The US National Committee for Vital and Health Statistics views SCT as a comprehensive, high-quality terminology.17 In content coverage studies, SCT's precursors consistently performed better than other sources.18–21 However, these studies were conducted more than a decade ago. In 2003, SCT was found to cover 88.4% of diagnosis/problem list terms used by clinicians within a computerized physician order entry system.22 In 2004, the Veterans Health Administration (VA) evaluated SCT and concluded that it has promise as a coding system for clinical problems.23 Brown et al24 found the sensitivities of SCT as a reference terminology and as an interface terminology were measured at 83% and 53%, respectively, for a general medical evaluation template. Rosenbloom et al25 found SCT suitable to provide standardized representations of information created by two interface terminologies. The study suggested, however, that enriching SCT semantics would improve representation of the external terms.
Future ‘meaningful use’ of EHRs will be partly defined by encoding problem lists by SCT codes.5 6 To facilitate use, the NLM prepared the UMLS CORE (clinical observations recording and encoding).26 Among the source terminologies in the UMLS, SCT covers the highest percentage (81%) of the UMLS CORE concepts. In the e-prescribing domain, the Food and Drug Administration has adopted the VA and KP problem list subset of SNOMED27 as the terminology to represent indications in electronic labels. In an evaluation of mapping indication phrases, SCT as a whole covered 90.3%, while the clinical finding hierarchy covered 79.5%.28
Despite incremental improvements in coverage in the specific domain of diagnosis/problem list, SCT may not yet be suitable for implementation in EHRs. For example, the concept ‘acute myocardial infarction’, despite being fully defined in SCT, is not linked hierarchically to the concept ‘ischemic heart disease’ (a primitive) (July 2010 release),29 nor is there any physiological attribute linking it to the associated myocardial ischemic process. No reverse associations exist either. In fact, when large organizations use SCT in their HIT applications, significant enhancements are required. KP's CMT7 and the VA's ERT8 are such examples. However, the end results of such enterprise efforts are not necessarily compatible with current standards efforts, and cannot be copied by smaller institutions. We note that KP has recently donated the CMT to the IHTSDO30 and that access to the content to support HIT goals will be provided by the IHTSDO and the NLM.31
Issues with SCT's underlying structure and conceptual representations may affect potential secondary use such as decision support. More than 50% of SCT's concepts are primitives (July 2010 release) lacking the attributes necessary for conceptual definitions. Issues with part of attribute representations, discrepancies in defined semantics, and definitional inconsistencies between ancestors and descendants have been highlighted.32–34 Critical reviews of the present architecture of SCT have been presented, addressing both logical and ontological issues.35 36
In 250 scientific publications over 40 years of use,10 few studies have dealt with the use of SNOMED in clinical practice, and 11 or fewer have dealt with SNOMED's content coverage or data quality. No surveys such as those of the UMLS user-base37 38 have been conducted in the last decade. Thus, despite its vast potential, little is known about direct user satisfaction with SCT and their perceptions and preferences regarding associated shortcomings.
Methods
We designed a 43-item questionnaire with five sections. The first three sections surveyed demographics and employment experience information, purpose of use of SCT, and means and frequency of access to SCT releases. The results of these sections were published in an AMIA symposium paper.11 A fourth part consisted of questions investigating the respondent's satisfaction with SCT content coverage and quality. The last part was designed to evaluate direct users' involvement and satisfaction with the change request mechanism of SCT. This paper covers the data obtained in the last two sections of the survey.
The questionnaire contained 31 mandatory questions and 12 optional questions, of which six were free-text questions. The fourth and fifth sections contained 27 questions: six free-text, 16 mandatory, and five non-free-text optional questions. Four questions allowed multiple choices. The survey was designed using freely available Google Docs tools and was presented as a Google Docs form on the web. Incomplete responses (ie, to only some of the mandatory questions) could not be submitted. The questionnaire can be viewed online.39
The call to participate in the survey was sent by email to members of various user and discussion groups. Among the user groups were the IHTSDO user community, the UMLS user mailing list maintained by the NLM, 10 AMIA discussion groups, as well as other ontology-related and nursing groups. Additionally, a mailing list was compiled on the basis of a PubMed search of authors of SNOMED-related articles published since 2006. Each message included a short introduction to our research center (SABOC40) and stressed the importance of participation. The respondents were encouraged to forward the questionnaire to their colleagues. All responses were anonymous, and no employer-specific data were collected.
The survey was conducted between January 7 and January 25, 2010. The request for participation was sent in three phases to the same distribution list. The initial call did not include a deadline. A second call 12 days later included a deadline within 7 days. The third call was a last-chance participation reminder within 24 h of deadline expiration.
All responses were collected and initially summarized by Google Docs tools. Data were exported into an Excel spreadsheet for processing of information contained in ‘Other’ fields and for further analysis. Sector analysis is used when appropriate. The χ2 test was used for statistical analysis where appropriate (non-overlapping).
Results
A total of 215 responses were received. Between 87 and 190 responses were recorded for the non-free-text, non-mandatory questions. For the free-text questions, between 49 and 113 responses were recorded. A spreadsheet containing the raw data is publicly available.41 To maintain anonymity, free-text responses were removed.
The results for the first three sections of the survey, covering user demographics and uses of SCT, were published elsewhere.11 Of the respondents, 65% were from North America, 36% were from academic or research institutions, 28% from the industry sector, and 23% worked for governments.11 Although the survey was conducted in January 2010, the results do not necessarily reflect a specific release of SCT, especially since only 14% of users indicated that they load new releases immediately.11
Profession's overlap and segments
As we previously reported11 on the distribution of professions among the respondents, the professional categories were not disjoint. We observed substantial overlaps between professional categories. Each right column in figure 1 displays the overlap between one professional category and other categories. Each left column depicts the percentage of respondents in each category that indicated that they were members of that profession alone.
Composition of professions. Each pair's left column on the bar chart shows the ratio between members of only one specific profession (pure) and the overall numbers of respondents that indicated participation in that profession as well as other professions (mixed) (see top legend). Each pair's right column depicts the internal distribution of responders within a profession. Please note that the proportion of pure members of any profession on the left column is greater than the one depicted on the right because of non-excluded overlaps between the other professional groups included in the right column. To avoid ‘small numbers’ issues, related professions were combined into four broad categories, listed on top. Above each broad category are the numbers of its pure (and percentage) and all members. These combined groups form the basis for sectorial analysis. Admin, administrator; Edu, educator; Eng, engineer; Lib, librarian; Manag, manager; Phys, physician; Prog./Developer or Prog/Dev, programmer/developer; Res, researcher; Student-Med. or Stu-Med, student-medicine; Stu-Oth, student-other.
To clarify sector analysis and to avoid ‘small numbers’ issues, we grouped certain categories. The following categories were created: researchers and educators (Res/Edu), engineers, developers, and programmers (Eng/Dev), physicians, nurses, and medical students (Clinical), and administrators and managers (Admin). For each of the above groups, corresponding subgroups of respondents who were members of only a single group (pure) were created—that is, pRes/Edu, pEng/Dev, pClinical, and pAdmin. The resulting groups and subgroups are presented in figure 1.
Coverage perceptions
In our AMIA symposium paper,11 we reported that, on a scale of 1–5 where 1 is ‘not at all’ and 5 is ‘very satisfied’, 58% of respondents indicated that they are at least satisfied (4 or 5) with SCT's coverage, and 10% were very satisfied. However, users were also asked two additional questions: ‘What is your overall perception of SNOMED-CT's coverage/completeness of your domains of interest?’ (Q19), and ‘How important is it to expand SNOMED-CT's coverage?’ (Q21, scale of 1–5, ‘not important at all’ to ‘extremely important’). Although the questions did not explicitly specify whether they addressed pre- or post-coordinated content, the responses to the following question (Q23, 113 responses) clearly indicate that responders addressed it in precoordinated context. Figure 2 summarizes the results for Q19 through Q21 by professional groups. Coverage was considered to be more than 85% complete by 42% of respondents, and 12% considered it above 95%. Furthermore, 69% considered it important (4 or 5) to expand content coverage, and 36% considered it extremely important. Within professional groups, Admins and especially pAdmins tended to be less extreme regarding coverage completeness, and were more likely to indicate importance of coverage expansion (p=0.01 and p=0.02, respectively).
Coverage perception: Completeness (left), satisfaction (center), and desire (right) for coverage expansion. The categories for completeness are by percentage ranges whereas the other two questions are on a scale of 1–5.
Only 10% of those who indicated that they perceive SCT's domain coverage as 85% or more also indicated that coverage level was not sufficient for their needs. On the other hand, 37% of respondents indicated that a level of coverage completeness of less than 85% may still be sufficient for their needs. Of users that perceived coverage completeness as 85% or greater, 63% indicated that it was at least important to expand coverage, as were 67% of users who indicated that coverage is satisfactory for their needs. Twelve out of 25 respondents (48%) that perceived SCT's coverage as better than 95%, and 55% of those who indicated that coverage is very satisfactory for their needs still indicated a desire to expand coverage.
Figure 3 depicts the respondents' preferences (Q22) as to which content areas should be expanded (a threshold of 20% was used). ‘Clinical finding’ and ‘procedure’ were the two most dominant candidate roots for content expansion (47% and 31% of users, respectively). As figure 3 shows, the hierarchies targeted for expansion differed by groups. Only ‘clinical finding’ was targeted by all groups.
Root hierarchies as candidates for content expansion by employment and professional groups. Shown are only root hierarchies that received 20% or more of the votes within each group. The last column on the right of each group denotes a ‘None of the above’ selection (ie, no root hierarchy requires expansion) and therefore was not subject to the 20% threshold.
A free-text question (Q23) could not clearly identify a predominant missing content area. In Q24, respondents to Q23 were neutral regarding the urgency of introducing their preferred missing content area.
Use of SCT's root hierarchies
Each of SCT's root hierarchies was used by at least 24% of the respondents. The three most popular hierarchies were ‘clinical finding’ (92%), ‘body structure’ (72%), and ‘procedure’ (71%) (Q8). These were also the three most important hierarchies for responders (83%, 42%, and 49%, respectively) (Q9). The three least utilized hierarchies were ‘record artifact’, ‘physical force’, and ‘environment or geographical location’ (24%, 25%, and 28%, respectively). No reportable differences were observed per professional groups or employment sectors for Q8 and Q9.
Concept information
In Q25 (scale of 1–5, ‘not satisfied at all’ to ‘very satisfied’), users were asked about their satisfaction with the current level of SCT's concept details (ie, synonyms, hierarchy, and attributes). Overall, 45% of respondents indicated they were at least satisfied with the current level of concept details (8% ‘very satisfied’), and only 13% indicated they were not satisfied (1 or 2). Q26, Q27, and Q29 (all questions on a scale of 1–5, ‘not important at all’ to ‘extremely important’) covered the perceived importance of adding synonyms, attributes, or textual concept definitions (respectively). Of the respondents, 54–60% indicated that it was at least important to expand/add each of these parameters (20–27% ‘very important’), despite the level of satisfaction indicated above.
In sector analysis (figure 4), we observed that Admins tended to report fewer negative impressions regarding the status of concept details than pRes/Edus (p=0.01). Administrators assigned less importance to Q26, Q27, and Q29, whereas pRes/Edus and pClinicals assigned the most importance to expansion of concept attributes, and pEng/Devs and pClinicals to textual definitions. More respondents from academia and research organizations assigned importance to expansion of SCT attributes than those working in the industry (p<0.01 for both).
Respondents' attitudes toward SNOMED CT's concept details: overall satisfaction and importance of expanding/adding synonyms, attributes and textual definitions (right-to-left).
Frequency of SCT deficiencies and user attitudes toward them
More than 64% of responders indicated that they rarely encounter missing or incorrect parents, incorrect attributes, attribute values, or children (Q31). Missing concepts, synonyms, attributes, or children were indicated as at least often encountered by 23%, 17%, 9%, and 16% of responders, respectively (figure 5). No reportable differences were observed for different groups.
Frequency of encounters with (Q31, left column), and attitude toward (Q30, right column), various SNOMED CT deficiencies.
However, when encountering such deficiencies, respondents from all groups indicated that they find incorrect content much more bothersome than missing content (Q30). As shown in figure 5, 54–60% of respondents indicated that they are ‘very much’ bothered by incorrect parents, children, attributes, and attribute values. Between 28% and 35% indicated that such findings are only ‘somewhat’ bothersome. The attitude toward missing content was more forgiving, and 49–67% indicated that they are ‘somewhat’ bothered by such findings.
Quality and allocation of resources
Overall, 60% of respondents were at least satisfied with SCT's quality. Only 7% were very satisfied (Q34, scale of 1–5, ‘not satisfied at all’ to ‘very satisfied’). Of all respondents, 6% indicated some degree of dissatisfaction (3–11%). In Q35 (scale of 1–5, ‘not important at all’ to ‘extremely important’), 63% of respondents indicated that it is very important to conduct quality assurance and auditing of SCT's content, and 88% indicated that it is at least somewhat important.
On independent scales (each up to 100%), respondents were hypothetically willing to allocate at least 30% of IHTSDO's budget on the following activities: QA/auditing (67% of respondents), coverage expansion (63%), and expansion of concept details (56%) (Q36, figure 6). Of pAdmins, 88% were willing to invest at least 30% of IHTSDO's budget in QA/auditing. Fewer pRes/Edus (51%) were willing to invest at least 30% of IHTSDO's budget in coverage expansion, and only 42% of pEng/Devs were willing to do so for expanding concept details.
Hypothetical allocation of resources (as percentages of IHTSDO's budget) toward QA/auditing, coverage expansion, and expansion of concept details.
SCT auditing and change request process
In Q38, 83 respondents (39% of all respondents) indicated that they were actively involved in auditing of SCT (51% of government employees, 59% of pAdmins, 30% of pClinicals), and 79 respondents (37%) indicated that they submit SCT change requests, but only 48 respondents (22%) indicated doing both. Change request submissions were distributed quite evenly between IHTSDO, National Centers, and CAP-STS (30%, 39%, and 33%, respectively).
Q40 through Q42 (all three questions on a scale of 1–5, ‘not satisfied at all’ to ‘extremely satisfied’) concentrated on overall users' satisfaction with the change request process, the rate of acceptance, and the rate of implementation of change requests. Most responses indicated a neutral position (55%, 48%, and 51%, respectively), with more respondents indicating some degree of satisfaction (30%, 33%, and 26%, respectively).
Free-text questions
Six free-text questions were included in the survey. One hundred and fifty participants (70%) responded to at least one free-text question (2.5 free-text questions on average), and 25% responded to at least four free-text questions. The responses were diverse and beyond the scope of this paper. However, we observed a common thread across the responses. Many free-text respondents indicated that they would rather have SCT concentrate on improving quality than on expanding content coverage. Specifically, the issues of numerous primitive concepts, lack of desired attributes, and overall patchy modeling and coverage were common in this thread.
Those who use the change request process also indicated that, although the process has improved recently, it seems to be structured toward new content rather than for resolving existing issues.
Discussion
This is a timely survey for American HIT. After establishing itself as a premier international clinical terminology, aspects of SCT are soon to become mandated components of EHRs.5 In discussions regarding the role of SCT in HIT applications, many presumptions are being made. Direct users serve as an interface between end users and terminology designers. Thus, they present a balanced view of practical and conceptual issues regarding SCT. However, little up-to-date information exists about their attitudes toward SCT.10
This study suffers from the inherent deficiencies of its survey format. Moreover, in this survey we had little control over recruitment and selection bias, affecting our ability to generalize our observations. Additionally, we did not control for the SCT release addressed by the respondents. As reported previously,11 many direct users take their time to load new releases or use multiple versions (27%). However, we were pleasantly surprised by the number of complete responses (215), and the overall commitment of the respondents, as reflected by the high number of detailed free-text responses. Despite the encouraging number of responses, our ability to perform meaningful sectorial analysis was limited by ‘small numbers’ issues and the multitude of perspectives to examine. We believe that it is likely that the views presented here and in our previously published report11 correctly reflect those of the community of direct SCT users and provide much needed current information for informed decision making.
The goals of SCT's direct user community are not necessarily aligned with its design. The tension between ‘reference’ and ‘interface’ terminologies was omnipresent in the responses. It is also reflected in recent discussions within the IHTSDO member community.42 43 One such discussion43 closely resembles many of the opinions reflected in our respondents' answers. Continued focus on coverage expansion and precoordination versus enhanced efforts to contain expansion for maintainability's sake, and improved consistency and concept representation are dominant themes. However, our survey did not address pre- and post-coordination capabilities in SCT, and the issue was rarely discussed in the free-text responses. Our discussion of the study is conducted in the context of this important, ongoing debate within SCT's direct user community.
The results indicate that most direct users are satisfied with SCT's coverage and quality, despite its limitations. These findings are a tribute to the individuals and organizations that tirelessly push SCT forward. As responses to the free-text questions and other discussions42 43 indicate, some direct users are frustrated by deficiencies in SCT, especially incorrect conceptual representations. Respondents indicated that significant resources should be allocated to improve overall quality as reflected by issues such as insufficient/incorrect concept details and coverage. We note that even those who are satisfied, or very satisfied with coverage, or many of those who call for better quality, still indicate that they would like coverage further increased. Our survey was not designed to detect a cut-off point at which the desire to expand coverage is curtailed. In fact, such a point may not exist.
More than 50% of SCT's concepts (July 2010 release) are primitives, existing logical definitions are suboptimal,32–36 only 36% of concepts have synonyms (averaging 0.51 per concept), and by design, SCT does not include textual definitions. Such findings may be perceived as shortcomings that may limit usability for end users. To compensate and meet expectations, organizations may resort to developing or acquiring external extensions and dedicated subsets, which may diverge from current standardization efforts. Even in the context of the HITECH initiative,3–6 the use of SCT's concepts for encoding problem lists is mostly in the context of dedicated lists.26 27 In discussions,42 43 there seems to be a general agreement that SCT should not be used ‘as-is’ in patient-facing applications.
Thus, it is important not to ignore the issues about which users clearly expressed their preferences. It is at least important (categories four and five on a 1–5 scale) to 55% of users to expand synonym coverage; 54% desired to have more complete attribute coverage and 60% wished to have textual concept definitions. The latter is most likely a byproduct of SCT's success. As the user base expands, users indicate that they require more than the ‘conceptual definitions only’ approach. Similarly, LOINC44 recently added textual definitions/descriptions45 to assist users.
The balance between the various qualities of a clinical terminology changes as the user community and use patterns evolve. As terminologies mature, users increasingly become concerned with the correctness, consistency, and accuracy of the terminology,46 47 similarly to software products.48 SCT direct users' desire to invest in quality assurance and improve quality parameters reflects SCT's mature status. The findings of this study are reminiscent of those of a study on the UMLS.37 Governing bodies such as IHTSDO should pay close and continuous attention to users' feedback, even if it does not conform to the original design, and adapt accordingly. The data provided by this survey may be used for that purpose. However, mechanisms for periodic focused assessments and feedback from users are lacking, and should be established and formalized as part of the life-cycle of such a terminology. The existing change request mechanism should not be viewed as an alternative for user feedback.
SCT evolved over many years into an extensive body of knowledge. It is not clear that the preferences expressed in this study are prioritized on the IHTSDO's roadmap. Moreover, responding to the expressed desires of direct users will require large-scale efforts that are likely to drain existing resources and interfere with ongoing and planned projects. Alternative mechanisms should be considered to allow rapid implementation of such requests. Embracing the extended user community and providing a more inclusive, controlled editing environment may be conducive to attaining such goals. However, such operations will require the development of effective quality assurance processes.
The findings of this study and the preceding AMIA symposium paper11 provide the most current perspectives on the use and perceptions of SCT by direct users. Since SCT is becoming ubiquitous in today's HIT, we propose that focused surveys be conducted on a scheduled basis, to better track trends related to its manner of use, perceived deficiencies, and help define development and improvement goals.
Conclusion
Most direct users of SCT are at least somewhat satisfied with the level of coverage, level of concept details and overall quality. For most parameters, positive responses outweigh negative impressions. High levels of satisfaction or dissatisfaction are relatively low. Users indicated continued desire to improve consistency, quality, and completeness of conceptual representations and concept details (synonyms, attributes, and textual definitions), coupled with a continued desire to expand coverage. The survey provides much needed, timely information for informed decisions regarding the use and goals of SCT. Similar surveys should be conducted periodically, so that changes in users' expressed attitudes can be tracked and help direct development goals.
Footnotes
-
Funding This work was partially supported by the NLM under grant R-01-LM008912-01A1.
-
Competing interests None.
-
Provenance and peer review Not commissioned; externally peer reviewed.
