Creating a Pediatric Digital Library for Pediatric Health Care Providers and Families
Using Literature and Data to Define Common Pediatric Problems
- Correspondence and reprints: Donna M. D'Alessandro, MD, Department of Pediatrics, Children's Hospital of Iowa, 200 Hawkins Drive, Iowa City, IA 52242-1009; e-mail: <donna-dalessandro{at}uiowa.edu>
- Received 10 July 2001
- Accepted 27 October 2001
Abstract
The goal of this study was to complete a literature-based needs assessment with regard to common pediatric problems encountered by pediatric health care providers (PHCPs) and families, and to develop a problem-based pediatric digital library to meet those needs. The needs assessment yielded 65 information sources. Common problems were identified and categorized, and the Internet was manually searched for authoritative Web sites. The created pediatric digital library (www.generalpediatrics.com) used a problem-based interface and was deployed in November 1999. From November 1999 to November 2000, the number of hyperlinks and authoritative Web sites increased 51.1 and 32.2 percent, respectively. Over the same time, visitors increased by 57.3 percent and overall usage increased by 255 percent. A pediatric digital library has been created that begins to bring order to general pediatric resources on the Internet. This pediatric digital library provides current, authoritative, easily accessed pediatric information whenever and wherever the PHCPs and families want assistance.
The information needs of pediatric health care providers (PHCPs) and of the families of pediatric patients are not well documented, and most pediatric information on the Internet is poorly organized. The goal of this study was, therefore, to complete a literature-based needs assessment to determine the 200 most common pediatric problems encountered by general PHCPs and families (100 for general PHCPs and 100 for families) and then develop a problem-based pediatric digital library designed to meet these information needs.
Background
The information needs of adult primary care physicians and patients have been well documented.1 2 3 4 5 6 7 8 Patient management questions generated by adult primary care physicians frequently go unanswered, mainly because of the difficulty of seeking access to information at the point of care.9 10 11
To date, the information needs of PHCPs and of pediatric patients and their families have not been as well documented. Most of the available information is directed toward specific populations or is disease-based.12 13 14 15 A medline search conducted by the authors could not provide a listing of the 100 most common pediatric problems encountered by general PHCPs and families.
The Internet has been suggested as a means of delivering authoritative, current medical information at the point of care and at home.2 4 6 16 Today, it is usually very difficult to use the Internet to find answers to medical questions, because information on the Internet is poorly organized, rapidly growing, and of questionable authority.17 18 19 20 In order to make the Internet a useful reference tool for PHCPs and families, order must be brought to the chaos of the information.
Although the Internet is not designed to be a library, subsets of it can be curated into digital libraries, or organized collections of digital information built and curated by structured approaches.21 Rather than functioning simply as information repositories, digital libraries may be organized so that visitors can easily seek answers to questions. Examples of pediatric digital libraries are PEDINFO (http://www.pedinfo.org), and Harriet Lane WWW Links (http://162.129.72.40/poi/).
Finding authoritative information on common pediatric problems can be difficult in these digital libraries, however, since they exhaustively index all pediatric information on the Internet and do not use a problem-based approach to their organization.
MEDLINEPlus (http://www.medlineplus.org), developed in late 1998, is a general health sciences digital library for patients that uses a more problem-based approach. Its pediatric problem list has been expanded and in the fall of 2001 had entries for approximately 40 problems. Health On the Net Foundation (http://www.hon.ch) also recently began offering a problem-based approach to its patient information.
Design Objectives
Our design objectives were to:
-
Determine the 200 most common pediatric problems encountered by general PHCPs and families (100 for general PHCPs and 100 for families)
-
Design the problem-based interface to the pediatric digital library using user-centered design principles and standard principles of information architecture, including identifying authoritative Web sites that meet previously identified information needs and recommendations of published clinical standards
-
Deploy the pediatric digital library and make improvements using visitor feedback, usability testing, and increased content offerings
System Description
Literature-based Needs Assessment
A literature-based needs assessment (i.e., an assessment of the information needs regarding common problems encountered by general PHCPS and families) was conducted by searching MEDLINE from 1966 to 1999. Searches were done on the following free-text terms in March 1999—“common problems in pediatric,” “common problems in pediatrics,” “common pediatric problems,” “common pediatric diseases,” “common pediatric topics,” “common pediatric presentations,” “common diseases in pediatrics,” “pediatric visits,” “pediatric problems,” “pediatric diseases,” “pediatric topics,” and “pediatric presentations.”
In addition, searches were done on the following terms using MeSH—“pediatrics and diagnostic imaging,” “pediatrics and utilization,” “pediatrics and telephone,” and “pediatrics and statistics and numerical data.”22 Free-text and MeSH terms were used to identify a wide variety of potentially appropriate articles.
The search was developed and conducted by one of the authors (D.M.D., a general pediatrician) and a pediatric radiologist, both of whom also reviewed the resulting article titles and abstracts. Articles were selected for possible study if the article appeared to one of these physicians to define the information needs of PHCPs or families and if the article was written in English. Selected articles were reviewed and abstracted by the physicians, who independently noted the common pediatric medical diagnoses, diseases, topics, and problems (referred to in this paper as common problems) mentioned in each article. The references in these articles were also searched for additional, similar articles about information needs. These subsequently found articles were reviewed in the same way. To check for interobserver variability, the two physicians independently analyzed all the articles and compared findings.12 13 14 15 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78
Two national pediatric curricula and national health statistics were also reviewed for common problems.79 80 81 82 83 Inpatient admissions from Children's Hospital in Boston from July 1994 to June 1995 were reviewed for common problems. Inpatient and outpatient census data from The Children's Hospital of Iowa for calendar years 1995 and 1998 were reviewed for common problems.
Through analysis of the review notes for all these information sources (MEDLINE literature search, article reference search, national pediatric curricula, national health statistics, and hospital census data), recurring common pediatric problems were identified. Problems were considered to be common if they met one of the following criteria:
-
Listed as having 50 or more cases in the article by Poole et al.25
-
Listed as a principal diagnosis in the U.S. Department of Health and Human Services data83
-
Referred to in two or more articles in the medline literature search, article reference search, or national pediatric curricula
-
Listed in the tables of the article by Forrest et al.78
-
Listed as having 100 cases or more in the Children's Hospital of Iowa census data and also listed in at least one other information source
A pediatrician with 20 years of experience in private practice and in an academic medical center then reviewed the common problems for possible omissions and for their relevance and applicability to clinical practice.
Pediatric Digital Library Creation
The problem-based interface of the pediatric digital library (http://www.generalpediatrics.com) utilized user-centered design principles and standard principles of information architecture.84 85 86 Specifically, information useful primarily to PHCPs (http://www.generalpediatrics.com/CommonProbProf.html) and information primarily useful to families (http://www.generalpediatrics.com/CommonProbLay.html) have their own distinct information pages in the pediatric digital library. The common problems were grouped together by organ system, and within each organ system they were organized alphabetically. Common medical terminology was used instead of structured vocabulary such as MeSH or SNOMED, because most PHCPs and families do not use structured vocabularies when thinking about a problem.22 87 Nonmedical terminology was specifically used on the family information page. Examples of how to find answers to typical questions using the pediatric digital library are found in the pediatric digital library's tour (http://www.generalpediatrics.com/Tour.html).
A manual search of the Internet and World Wide Web for authoritative information on these common problems occurred from July to October 1999. Criteria were established that a pediatric Web site must fulfill to be considered authoritative and current, using evolving criteria for evaluating medical Web sites.88 89 90 91 92 93 94 An authoritative Web site for this pediatric digital library must clearly display four core quality standards—authorship of the information, references, sponsorship and ownership, and currency of the information. An accreditation model of legitimating resources was used rather than a referee or peer-review model. Accreditation models are designed for works that change over time.95 Accreditation in this sense means recognition as conforming to particular standards.
Manual searching of the Internet for information on the common problems was performed by searches using a large number of search engines, including AltaVista, Excite, Google, Infoseek, Lycos, and Yahoo! The hits delivered by the search engines were evaluated using the criteria described above. In addition, hits had to be in the English language and provide the information directly (i.e., not hidden behind passwords) (see http://www.generalpediatrics.com/Mission.html). Manual searching of large pediatric and general Web sites such as KidsHealth.org (http://www.kidshealth.org), pedinfo (http://www.pedinfo.org), Virtual Naval Hospital (http://www.vnh.org), Centers for Disease Control (http://www.cdc.gov), and the National Institutes of Health (http://www.nih.gov) was also performed.
Web sites that met previously identified information needs, such as differential diagnosis, diagnostic criteria, treatment recommendations, drug information, and patient education needs, were specifically sought.1 96 More than one authoritative Web site was sought for each common problem, so different treatment and management styles could be included. Emphasis on Web sites meeting all recommendations of published standards, such as policy statements and guidelines, was the intent.97 98 99 100 101
The pediatric digital library with these common problems pages was published onli ne in November 1999. Between November 1999 and October 2000, the architecture of the pediatric digital library was improved in response to visitor feedback solicited via an online comment form (http://www.generalpediatrics.com/Comment.html) and formal usability testing according to the principles of Nielsen, using medical students, pediatric residents, and laypersons as subjects.102 103 104 The pediatric digital library was publicized in April 2000 to selected Internet search engines, Web indexes, and pediatric-related mailing lists. The number of common pediatric problems, authoritative Web sites, and hyperlinks to these authoritative Web sites were manually counted in November 1999 and November 2000. To determine the overall usage as measured by qualified hits and visits to the pediatric digital library, Web server log file analysis was performed using the log file analysis program Wusage (Boutell. Com, Inc, Seattle, Washington).
Status Report
The MEDLINE literature review and article reference search provided 308 articles. A total of 65 information sources, including national pediatric curricula and national health statistics, were selected for use in the literature-based needs analysis, as described in the Methods section.12 13 14 15 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 These information sources represented a broad base of pediatric medicine for PHCPs and families, including outpatient and inpatient pediatrics, emergency and critical care medicine, and all age ranges. Locations for the data collected in these articles included pediatric and family medicine community-based offices, inpatient pediatric wards, emergency rooms, dental offices, and telephone calls. These articles also included consultation requests of PHCPs to subspecialists.
Overall, in November 1999, the pediatric digital library had a total of 323 common pediatric problems (Tables 1 and 2). It contained 1,320 hyperlinks to 332 authoritative Web sites. The number of common problems without identified authoritative information was 48. For PHCPs, 148 common problems were recurrently identified. The number of authoritative Web sites identified and hyperlinks made to the common problems for PHCPs were 126 and 446, respectively. For families, 175 common problems were recurrently identified. The number of authoritative Web sites identified and hyperlinks made to the common problems for families was 144 and 694, respectively. The hyperlinks for PHCPs most commonly came from educational institutions and commercial companies (173 and 146, respectively), and those for families most commonly came from professional societies, educational institutions, and government agencies (258, 183, and 122, respectively) (Table 3).
Number of Common Problems, Unique Web Sites, and Hyperlinks in the Pediatric Digital Library (PDL) in November 1999 and 2000
List of Common Problems in November 1999
The Number of Hyperlinks, by Organization Type, in November 1999 and November 2000
From November 1999 to November 2000, the pediatric digital library was steadily expanded and enhanced. In November 2000, the pediatric digital library had a total of 375 common pediatric problems (see Table 1). It contained 1,994 hyperlinks to 439 authoritative Web sites. This represents increases of 51.1 and 32.2 percent, respectively, since November 1999. The number of common problems without identified authoritative information had decreased to 10.
For PHCPs, the number of authoritative Web sites identified and hyperlinks made to the common problems were 215 and 709, respectively. For families, the number of authoritative Web sites identified and hyperlinks made to the common problems was 236 and 1,122, respectively. The hyperlinks for PHCPs most commonly came from educational institutions, commercial companies, and government agencies (239, 201, and 138, respectively), and for families they most commonly came from educational institutions, professional societies, and commercial companies (299, 281, and 204, respectively) (see Table 3).
Successful accesses (qualified hits) of the pediatric digital library in November 1999 and November 2000 were 12,883 and 47,682 respectively (255 percent increase). The number of visitors to the pediatric digital library increased from 5,519 to 8,682 (57.3 percent) during the same time periods. These trends are consistent with those in other months. Initial formative evaluation using online comment forms shows the DHSL is useful to its visitors in meeting their information needs. Independent reviews and comments can be found at http://www.generalpediatrics.com/FanMail.html.
Discussion
The goal of this study was to complete a literature-based needs assessment to determine the 200 most common pediatric problems encountered by general PHCPs and families (100 for PHCPs and 100 for families) and then develop a problem-based pediatric digital library designed to fulfill these information needs. En face, this seemed a simple task, but it turned out to be much more challenging than anticipated.
In November 1999, more common problems were recurrently identified instead of the 200 problems originally sought. The first challenge was reconciling the semantics used in the disparate data sets; i.e., each data set used a different classification scheme or coding, which used different terms. For example, some articles cited “health supervision” only and did not note separately important problems such as vaccination, safety, growth, development, and nutrition. Because of the disparate semantics, we chose to use the phrase “common problems” rather than “diagnoses,” “diseases,” or “topics,” because the term problem is more inclusive.
A second challenge was that issues that are very important to families (e.g., communication, gauging the severity of an illness, understanding laboratory tests) are often not readily codable and therefore are left out of the types of data analyzed. These issues are mentioned frequently in the articles but are often not given emphasis in an article. It is only in reviewing numerous articles that their importance can be determined from the literature. These important problems necessitated their separate inclusion in the PHCP and family common problems pages.
A third challenge was ICD-9 codings from certain articles, national health statistics, and hospital census data, which were more disease-specific than problem-oriented and were often weighted toward tertiary care (e.g., oncology-related diseases were prevalent in these information sources).105 These information sources did give actual numbers or percentages and therefore gave indications of how common an individual problem was (e.g., otitis media with effusion was more common than headache). The national curricula were still fairly disease oriented, although more “problems or topics” emerged, such as cough. However, few actual numbers or percentages were given in these data; therefore, it was difficult to determine and rank how common the problems may be.
A pediatric digital library is a dynamic, not static, entity. Changes and improvements are expected to meet the changing needs of its visitors. Since November 1999, improvements in the pediatric digital library have been made on the basis of online comment form feedback and formal usability testing. Example improvements include an A-to-Z alphabetical searching function (http://www.generalpediatrics.com/AZSearchProf.html and http://www.generalpediatrics.com/AZSearchLay.html) and expanded use of nonmedical terminology for patients and families. Scoped searching of authoritative pediatric Internet Web sites has also been implemented (http://www.generalpediatrics.com/SearchPage.html)
The number of common problems has increased on the PHCP and family pages. This is a perceived increase only and is due to re-organization of the pediatric digital library. Some of the original common problems have been combined or split apart to aid clarity (e.g., safety was originally a single topic and is now a separate section with several problems.)
Absolute increases in the number of authoritative Web sites and hyperlinks in the pediatric digital library have been 32.3 and 51.1 percent, respectively. The number of topics without hyperlinks has also decreased—an absolute decrease from 48 to 10. New Web sites, especially smaller sites, have been found through Web page monitoring, suggestions by visitors, and reading of the medical informatics, medical, and lay literature, and they have been added to the pediatric digital library. The usage of pediatric digital library continues to increase. Successful accesses (qualified hits) increased 255 percent when the two time periods were compared, while the number of visitors increased 57.3 percent.
This pediatric digital library was not intended to be an exhaustive solution to pediatric-related information needs; instead it was intended to offer a solution to most of the needs and offer a direction toward fulfilling other needs. It appears from the initial formal evaluation using online comment forms that the pediatric digital library is useful to its visitors. However, several formal summative qualitative and quantitative evaluations of the pediatric digital library are ongoing, including actual information needs as defined by PHCPs, validating whether the pediatric digital library meets most of the pediatric information needs of PHCPs and determining whether the pediatric digital library changes the information-seeking behaviors of PHCPs and patients and families.
With the Internet becoming a dominant means of communication in society and specifically it becoming the dominant source for health information, digital libraries are faced with numerous challenges in serving their visitors' needs.106 In 1945, in a landmark article describing digital libraries,107 Vannevar Bush realized that digital libraries would very quickly become massive in size and foresaw the need for domain specific experts, which he called trail-blazers (or curators) who would create paths through the information (e.g., pediatrics) that others would then follow.
Today, intelligent software agents are semi-automatically helping curators manage change on the Internet by discovering, monitoring, and indexing information.108 These software agents are excellent at mining huge amounts of information, but they cannot add the human elements, such as providing oversight, monitoring quality, and providing balance, which together are crucial to the process of developing, curating, and sustaining pediatric digital libraries.
It is the human element of digital library curatorship that will have a notable effect on medicine in the future. We believe that human curatorship of digital libraries will be as important to the medical culture in the 21st century as journal editors have been in the 20th century. Indeed, just as physicians and librarians played crucial leadership roles in the foundation and curation of print medical libraries at the dawn of the 20th century, they again have a crucial leadership role to play in the foundation and curation of digital medical libraries at the dawn of the 21st century.109
The following are limitations of this study:
-
The articles that were analyzed were written in English only about medical information in developed countries. The information could be different for non–English-speaking populations and those in developing countries.
-
Two physician researchers collated the common-problems information. This information was checked for relevance by another pediatrician and has been evaluated by visitors of the digital library with modification based on their feedback.
-
The information sources that were used were not uniform in nature; therefore, an absolute systematic determination of the common problems was not possible. This study did perform a systematic determination that attempted to incorporate the broad range of information needs for PHCPs and families.
-
This study was designed to determine the common problems encountered by general PHCPs and families and was not designed to specifically meet the information needs of pediatric subspecialists from an information content or architecture standpoint. Pediatric subspecialists may still find information in this pediatric digital library useful in their clinical practices.
Conclusions
A pediatric digital library has been created that begins to bring order to the chaos of the general pediatric resources on the Internet. Designed around a literature-based needs assessment, this problem-based pediatric digital library provides current, authoritative pediatric information that can be easily accessed via the Internet whenever and wherever PHCPs and families want help to optimally practice pediatrics and live healthy lives.
Acknowledgments
The authors thank Michael D'Alessandro, MD, for reviewing the articles, and Mary Larew, MD, for reviewing the common problems for relevance. They also thank Michael Peterson, MD, for his guidance and mentorship with this project.
Footnotes
-
This work was supported by a Generalist Faculty Scholars grant from the Robert Wood Johnson Foundation.
-
This work was presented in part at the 2000 Pediatric Academic Societies Annual Meeting in Boston, Massachusetts, May 12–16, 2000.
