Training Multidisciplinary Biomedical Informatics Students: Three Years of Experience

Introduction

Medical Informatics (MI) has been an established research discipline for over twenty years.2 3 4 5 6 The main focus of MI is the development of computational tools, algorithms and strategies which advance medicine and enhance patient care.7 8 9 10 11 In contrast, BioInformatics (BI) is a less mature scientific discipline which aims to research and develop algorithms, computational and statistical techniques which solve biological problems. Significantly, BI has experienced an exponential growth as a result of its importance to the understanding and interpretation of data generated by ‘omics’ technologies.12 13 14 Although MI and BI both exploit computers and computational tools, they differ in many ways. Arguably, these differences are due to diversity in the domain expertise of the practitioners (medicine vs. biology) and researchers involved in the application field (healthcare professionals vs. bio scientists) and the educational emphasis adopted by the independent disciplines (patient-care vs. basic-research).

It is now widely accepted that the impact of ’omics research on healthcare will be considerable if data generated from basic science research can be integrated and transformed into medical knowledge with clinical utility (personalized medicine based on genomic medicine).15 16 Clearly, both BI and MI should facilitate this new medical era and there is now an urgent need for an integrated scientific discipline: BioMedical Informatics (BMI).10 17 In agreement with this assertion, over the past 7 years an increasing number of research projects with an integrative BMI character have been undertaken. The participants of these projects have mostly been drawn from conventional MI and BI backgrounds and there are very few people who are truly multidisciplinary biomedical informaticians.18 19 20

In an attempt to rectify this shortcoming and develop domain specialists, several BMI curricula have recently been created.21 22 23 Significantly, these courses have predominantly appeared in the United States (see below). To remedy this geographical imbalance, over the past three years the European Network of Excellence INFOBIOMED has developed and trialled novel strategies for training a new generation of biomedical informaticians. These efforts (described in this paper) have emerged from several years of preliminary activities aimed at establishing a foundation for BMI at a European level.24 25 26

Background

Current State of BioMedical Informatics Training

In 2003, the American College of Medical Informatics undertook a study which discussed and defined future challenges in BMI. 27 To meet these challenges, the project concluded that effective training in informatics should encompass four key elements: (1) curricula that integrate (and not concatenate) experiences in the computational sciences and application domains; (2) diversity among trainees, with individualized, interdisciplinary cross-training allowing each trainee to develop key skills that he/she does not initially possess; (3) direct immersion in research and development activities; and (4) exposure to a wide range of basic informational and computational sciences.

Currently, several BMI training programs in the United States are attempting to deliver curricula integrating biological and clinical informatics.27 For example, the newly developed ‘Theory and Methods in BioMedical Informatics’ course at Columbia University presents informatics principles in their general form and illustrates their application with examples drawn from across the biomedical spectrum.28 29 At Stanford, training concentrates on the development and application of novel informatics methods for biomedical research.30 31 Further, the Oregon Health & Science University offers a successful online introductory graduate course in BMI. 32 Significantly, none of the curricula from the above training programs appear to incorporate modules aimed at teaching students how to work effectively in a multidisciplinary environment. A notable exception to this is the Purdue University BMI curriculum, which states that students are assessed on their ability to perform in a team environment. Notably, however, no criteria for success or failure in this area are presented.33 In designing the teaching initiatives within the European INFOBIOMED Network of Excellence (introduced below), a unique approach was evolved which provided students with novel experience of work in a multidisciplinary environment. This strategy, which diverged from a conventional approach targeted at personal knowledge development, will be discussed later.

Formulation Process

In 2001, the European Commission (EC) initiated activities aimed at promoting the development of BMI at a European level.34 At this time, the EC funded BIOINFOMED project brought together 30 experts from different areas to develop a roadmap for the evolution of BMI in the forthcoming decade.35 As a result of this study, a White Paper was published,36 a significant proportion of which concerned itself with the education and training of new BMI professionals.

In 2004, INFOBIOMED was one of three Networks of Excellence funded by the European Commission within the e-health division. Significantly, INFOBIOMED was the only network whose principal focus was BMI. INFOBIOMED was tasked to develop a scientific reference base for BMI concerned with: (1) the awareness and dissemination of BMI activities at an international level; (2) the education, training and mobility of professionals from computer science, medicine, biology and related disciplines; (3) the development of new computational models and methods for linking genotype and phenotype; and (4) the development of four pilot projects where clinical and genomic data were gathered and analyzed in the context of genomic medicine (in the areas of pharma-informatics, genomics and infectious diseases, peridontitis and colorectal cancer).

INFOBIOMED aimed to establish European BioMedical Informatics as an integrative discipline whose main goal was to support individualized healthcare. Within this Network of Excellence, the BioMedical Informatics undertaken aimed to exploit synergies derived from the integration of BioInformatics (BI) and Medical Informatics (MI). It was hoped this would facilitate the discovery of novel preventative, diagnostic and therapeutic methods.

Notably, training activities developed within INFOBIOMED were designed to contribute to the establishment of a durable BMI structure within Europe. It was hoped this may be achieved by building a research capacity that would enable the consolidation of BMI as a crucial scientific discipline for future healthcare. The unique approach adopted by INFOBIOMED offered a spectrum of educational activities including opportunities for: knowledge acquisition, participation in BMI exchange programs (visits to both MI and BI institutes) and the development of personal skills relating to multidisciplinary working. The INFOBIOMED training development team recognized that a broad variety of activities will be essential for the next generation of BMI students.

In this paper, two of the activities (see 3 and 4 below) supporting personal education and multi-disciplinary teamwork carried out by the INFOBIOMED partners are described in detail.

In brief, the complete set of INFOBIOMED training activities may be summarized as:

1. The development and release of a publicly available BMI-related course database. This resource includes references and contents of a wide range of curricula available within the European Union (at both the undergraduate and graduate level).

2. The organization of a Summer School bringing together a broad range of BioInformatics and Medical Informatics courses and activities from the three different Networks of Excellence funded by the eHealth division of the European Commission.

3. The development of a mobility program facilitated by a ‘mobility brokerage service’ publishing offers and requests for research exchange projects within the Network of Excellence (see sections ‘Mobility Program’ and ‘Mobility Brokerage Service’ later). The mobility program included financial resources for students wanting to obtain additional training at other institutions.

4. The creation and organization of three, intensive, one-week ‘training challenges’ (see section ‘Training Challenges’). In these events, trainees were challenged to solve complex cases proposed by their peers, which required both BioInformatics and Medical Informatics data and methods.

From an educational perspective, activities 1 and 2 above were devised and implemented in order to provide an opportunity for BMI students to gain access to relevant courses for the development of personal knowledge. Since these were worthy and essential but arguably not novel activities (in their design and implementation) little coverage will be included in this paper.

In contrast, activities 3 and 4 may be considered novel since they focus on the acquisition of interdisciplinary work experience. This was achieved either via personnel exchanges between research laboratories or via participation in ‘training challenges.’ The inception, design and implementation of these activities are described below.

Model Description

Planning and Needs Assessment Surveys

The Network of Excellence began its activities by carrying out an investigation of the needs and expectations regarding education and mobility within the consortium. The afore-mentioned study from the American College of Medical Informatics was used as a guideline for this activity. From this document, it was clear there was a general need to develop novel strategies allowing BioMedical Informatics students to: (i) develop key skills with individualized multidisciplinary training, and (ii) to facilitate their participation in research and development activities.

To facilitate (i) and (ii) above, two activities were defined and undertaken which would support the acquisition of knowledge within the overall training program. These activities were: the development of a publicly accessible BMI-course database, and the planning and organization of BMI-focused Summer Schools. In relation to the former, the INFOBIOMED course database and a Training Thesaurus on BMI-education were developed using the text mining tool Collexis©.37 38 39 This comprehensive resource of academic courses in BMI initially included information on the current academic programs of network partners (with the goal of becoming a one-stop repository for European BMI-training information). As the course database evolved, references and contents from different BMI-related academic courses were added, both at the undergraduate and graduate level, from a wide range of European countries.

The INFOBIOMED Summer School was a collaborative initiative involving the three Networks of Excellence (BioPattern, SemanticMining and INFOBIOMED) funded by the eHealth division of the European commission. This event aimed to bring together a range of BioInformatics and Medical Informatics courses and activities, and over fifty students attended (all of whom participated in courses and activities organized by fifteen teachers and students). All students were encouraged to present their work and received feedback from teachers and students on their research projects.

While the above activities were essential to establishing a European BMI-educational program, they were not in themselves novel in either their design or implementation. In contrast, the INFOBIOMED mobility program and training challenges represent an original solution to the pressing requirement for under- and post-graduate training in multidisciplinary BMI work. These activities are described in detail below.

Mobility Program

The INFOBIOMED Mobility Program aimed to help students to identify relevant and interesting opportunities in Europe for collaborative BMI-related work which would enable the development of multidisciplinary skills and expertise. To initiate the design and development of this program, two distinct online surveys were carried out to capture and enable the analysis of relevant issues, barriers and needs from the perspective of a ‘host organization’ and a ‘potential candidate.’ Each survey was divided into two parts. The first part was specifically designed to gather feedback and needs from both host organizations and candidates. The second part, common to both surveys, was designed to collect information on the specific infrastructural needs that should be included in the INFOBIOMED mobility program.

The Mobility Brokerage Service

On the basis of the preliminary survey outlined above and the results obtained as detailed in the Results section, it was clear there was a pressing need for a ‘Mobility Brokerage Service’ (MBS). To meet this requirement, a web application was designed which enabled the publication of both mobility opportunities and requests. It was hoped this would become a point of reference for BMI experts and trainees across Europe and beyond.

From an implementation perspective, the MBS is a secure web-based ‘marketplace’ where the information detailed below is made available to interested parties:40

• Information on host organizations, offering specific mobility opportunities

• Job offers

• Candidate information (candidates interested in both mobility and job opportunities)

The system automatically matches offers and requests, according to defined criteria, and notifies interested researchers and host organizations of potential opportunities. Users can search the contents of the MBS and promote contacts between host organizations and researchers; however, the system does not negotiate the specific details of each new potential action, exchange or contract. Until now, usage of the MBS has been restricted to partners within the consortium because of security and confidentiality issues. Notably, it can easily be extended beyond INFOBIOMED (to at least a European level) on implementation of appropriate security measures.

Use of the MBS

By selecting a menu option in the MBS, users can enter a new request into the resource. The online service then prompts the user for details as follows: title, textual description, type of request (job or mobility), dates and anticipated salary. To facilitate the entry of a request, all terms contained in the INFOBIOMED Training Thesaurus are shown to the user allowing them to select those which most accurately describe their interest. On completion of the data entry phase, requests are stored in the system and the matching process is automatically launched. This process retrieves information needed from all current offers in the database. It includes information related to the type of opportunity (job or mobility), duration, date of execution and topic selected from the thesaurus for rating matches. Each offer is compared one-by-one with the request and when a match is found an email is generated for both the host organization and the candidate, inviting them to review the information online. A similar process takes place when an organization fills out a new offer in the system.

In practice, if a match was found between a student and an exchange opportunity, the student could apply for funding from INFOBIOMED. Applications were then reviewed internally by a selection committee. Alongside formal criteria such as host/sending organization, duration, and budget, the committee assessed the multidisciplinary nature of the research proposal. In particular, proposed Bioinformatics and Medical Informatics methods, tools and data were considered alongside the scientific value to the BMI community.

Training Challenges

Training Challenges were based on a novel format involving the face-to-face interaction of two groups of five students drawn from a range of disparate scientific fields (Medical Informatics, BioInformatics, Medicine, Biology, Chemistry, Chemi-informatics, Computer Science, Epidemiology, Pharmacy, Physics, Mathematics, etc.). Over five days, the teams conducted research aimed at the definition of a valid, integrated, multidisciplinary approach to solve a problem proposed by a member of the team. Case studies and team composition were selected by the organizing committee prior to the challenge (according to the potential they offered for multidisciplinary teamwork).

In practice, training-challenge participants were informed of their research topic one month prior to their event. This allowed participants to explore their topic and develop ideas and strategies for subsequent contribution to research and discussions. On day one of the challenge, all team members delivered a presentation which introduced their personal research background and potential contribution to the assigned case-study. Also, at this time a team-meeting facilitator was identified. The role of this individual was to devise agendas, monitor time/progress and maintain focus over the next three days of intensive research. To support the students, INFOBIOMED staff members were present at all times to provide specific expertise and assist with problems and queries. In addition, two selected domain experts participated at specific times and contributed to research evaluation, team skill development, scientific project management, and multidisciplinary approaches. In the second and third challenges, specific contributions were also made by experts in team work dynamics. These individuals provided advice and guidance on conflict resolution, prioritization, and other issues associated with multidisciplinary research.

Training Challenges concluded with formal team presentations which introduced the research topic, and then presented the team research strategy. This presentation, its content, scientific value, consistency, and multidisciplinary approach was evaluated by a jury who decided upon a ‘winning’ team whose members were awarded a one-month mobility exchange to undertake research in an INFOBIOMED partner institution.

On completion of the training challenge events, and in order to evaluate the educational approach, all students were asked to complete a Training Challenge Survey. The survey consisted of both Likert scale and multiple-choice questions relating to personal information, the organization, the usefulness of the training challenge, the support received during the training challenge and the anticipated impact of the training challenge on their career.

From an organisational perspective, the opening and closing sessions of the training challenges took place at the organizing centre (twice by IMIM partner (Barcelona) and once by EUDIN partner (Edinburgh)). Teams then moved to a separate, semi-isolated location providing a lack of distraction, appropriate infrastructure (e.g., computer and network facilities and the required online resources—digital libraries and BioInformatics tools), accommodation and a friendly atmosphere.

Validation through Example

Survey Results

The web-based education and mobility survey was made available from the 2nd of September until the 27th of October 2004. During this period, 180 participants responded to the survey and the output from these individuals was subject to a statistical analysis aimed at identifying significant issues to be addressed by INFOBIOMED education and training development program.

A subset of results from this survey is presented in Table 1. This table presents host- and participant-related barriers hindering or preventing four to twelve week mobility exchanges between organizations. From the overall array of potential issues preventing training and mobility activities, results were filtered to identify issues and barriers which could be addressed by the INFOBIOMED Network of Excellence. The questions about the barriers for mobility were presented in multiple-choice format where respondents were allowed to select one or more of the choices. The respondents also had the opportunity to add other comments to the survey. All options are presented in Table 1 and the percentage of respondents that indicated each option as a barrier is shown.

Data captured in this exercise indicated that the most significant barrier to training and mobility exchanges between network partners was a lack of specific funding. As an immediate response to this issue, INFOBIOMED established a financial support mechanism for funding student mobility and training activities. Distribution of this financial support was managed via a novel mobility exchange program. The novelty of this approach was that students were required to justify how their work would (a) promote a BMI approach, and (b) how it would benefit the host in their research. This aimed (through the mobility brokerage service) to solve a variety of issues identified by the survey (e.g., a lack of information relating to host offers, lack of admin support, lack of host-related information etc.). The results of this mobility program are described in detail below. Significantly, despite the short period of exchange and the fact that the majority of the participants were relatively junior, several of the INFOBIOMED facilitated exchanges visits resulted in peer reviewed publications.

Mobility Program

The INFOBIOMED funding mechanism, offering economic incentives for mobility exchanges was initiated in January 2005. During its first year of implementation, the mechanism was successful in promoting and enabling a variety of exchanges with a typical duration of four to twelve weeks. The majority of these exchanges were focused on collaborative research for the INFOBIOMED clinical-genomic pilots.

While the total number of exchanges (20) was lower than expected, overall the mobility program has fostered new BMI research, resulted in a number of peer-reviewed publications and demonstrated the feasibility of this interdisciplinary BMI approach within the European Union. A significant output of this exercise is the previously discussed mobility brokerage service. This web-based ‘marketplace’ for the publication and exchange of mobility offers and demands has been ‘live’ since July 2005, during which time it has received more than 1200 visits. This service is the first web site specialized in BMI collaborative exchanges in the European Union. Table 2 shows the results of the exchanges carried out during the mobility program. In total, this was undertaken over a period of 30 months (including a 6 month EC extension to the INFOBIOMED program). The mobility program required a total of €27,000 from INFOBIOMED’s budget for financial support of the 20 exchanges. As outlined earlier, all exchanges were assessed on their combination of BioInformatics and Medical Informatics methods, tools or data.

Training Challenges

Within the INFOBIOMED educational program, the training challenge format was developed to meet a requirement for multidisciplinary project experience amongst MSc and PhD level students. Over a two-year period, three training challenges were undertaken with a total of 29 students (on average 10 students per challenge) participating. During the student-selection process, gender and European nationality were considered as factors in order to achieve a balance of representation. The case studies used in the training challenges and the scientific background of the participants are shown in Table 3.

The results from the Training Challenge Survey are presented in Table 4. Broadly, the feedback obtained from all the challenges was very similar and generally positive. The only notable dissatisfaction amongst the students related to the duration of the event. Almost all students thought that the time allotted for the training challenge was too short for them to obtain robust scientific results. It should be noted, however, that the expected outcome of the training challenge was not a series of actual results, but rather an integrated plan for further multidisciplinary research. All other aspects of the challenge received positive feedback. The results of the second challenge were marginally lower when compared to the others; however, this bias could be attributed to a single ‘outlier’ student. The results in Table 4 have been aggregated for all participants per training challenge. The Training Challenge Survey consisted of multiple choice questions. Apart from the Demographics section, Motivation section, and seven questions from the Impact section, choices were based on a five-point Likert scale. Results for Likert scale questions are presented as a value between 1 and 5 (indicating the average score for all participants) with a standard deviation between parentheses. Results in the Demographics section are averaged (for questions 1, 3, and 4) or presented as a percentage (for question 2 on gender). Results for all other multiple choice questions are reported as a percentage of the participants selecting that option (multiple options allowed).

Significantly, all students indicated that they had benefited from working in a multidisciplinary team. The Training Challenge Survey revealed that prior to the event the students had not anticipated that the multidisciplinary environment would be more challenging than their typical work. Notably, virtually all participants found that differences in scientific language, attitudes to a scientific problem and team problem solving strategies were more challenging than expected. All students indicated that the training challenge experience had contributed to their understanding and appreciation of multidisciplinary teamwork.

Discussion

To meet the challenges of modern Biomedical research, The European Union is developing a scientific research environment in which collaborative exchanges are crucial to the success of trans-national initiatives such as INFOBIOMED. In this context, the educational and mobility activities of this Network of Excellence were designed to assess a range of tractable strategies for BMI-related training and education. The result of this work was four significant contributions to BMI personnel development in Europe—the BMI-course database, the BMI Summer School, the novel ‘mobility brokerage service’ and the ‘training challenge’ format.

Traditionally, Medical and Bio-Informatics educational and training programs have operated in a mutually exclusive manner, with few examples of knowledge and experience exchange between disciplines. In recent years, however, several high-profile BMI training programs have appeared. For example, the BMI Training Program at Stanford University—directed by the bioinformatician Russ Altman30—is now joined by similarly integrated teaching efforts in Harvard, Columbia,28 29 Pittsburgh and Yale41 Universities. Our experience over the past three years has revealed that the integration of MI and BI curricula remains a significant challenge due to differing academic cultures and divergent objectives of researchers and students.17 42 It is hoped that as increasing numbers of novel links between genotype and phenotype are defined and hypotheses from pioneers in BMI are confirmed through original publications the challenges outlined above will be overcome more readily in the future.

During its first year, the INFOBIOMED Network of Excellence identified several opportunities for enhancing BMI educational activities within Europe. Significantly, several novel strategies were developed to meet a perceived requirement amongst students and post-doctoral researchers for multidisciplinary BMI work experience. To facilitate the dissemination and management of potential work-experience exchanges, the ‘mobility brokerage service’ was designed and implemented. In a recent effort to refine this application, advanced functionality enhancing the management of personal information and opportunities has been added to the system. These features, only available to registered users, simplify the process of searching for offers and demands within the system. Despite the fact that the ‘mobility brokerage service’ was consulted over 1200 times, it resulted in only twenty exchanges funded by INFOBIOMED. The availability of a mobility fund enabled twenty PhD students to experience work in a new and stimulating different scientific environment. It was anticipated, however, that alleviating the two main barriers for students to participate in mobility programs (funding and the acquisition of information about host offers) would significantly increase the number of mobility exchanges undertaken. Notably, other barriers ultimately prevented students participating in the mobility program. For example, a ‘lack of time’ was highlighted as a significant issue in this area. This barrier can only be reduced if educational programs include a period in which the students are able to work in multidisciplinary teams at other scientific institutes. Overall, the authors feel INFOBIOMED demonstrated the value of multidisciplinary exchange programs and training challenges; however, widespread adoption in other educational programs remains a significant issue for the future.

Although the mobility brokerage service will end when the INFOBIOMED project closes, the software is freely available to other European projects and has been developed in such a way that it can be applied to areas other than BMI.

In an effort to meet a Europe-wide requirement for BMI multidisciplinary training, the INFOBIOMED educational development team devised the ‘Training Challenge’ format described in this paper. Having conducted three BMI-orientated training challenges and analysed feedback from the participants, we feel this approach provides students with an excellent opportunity to develop multidisciplinary team-based research skills. Our experience showed that the time-limited challenge encouraged students to focus on:

• Team-based contributions and the trust of colleague opinions.

• Team delegation according to skills and experience.

• Problem solving.

• The production and clear presentation of a high-level solution in a competitive environment.

Significantly, however, the training challenge format also has a number of drawbacks. For example, during the five-day period, it is not possible for students to pursue an in-depth scientific analysis of a subject, nor do all team members acquire an extensive, detailed understanding of the whole subject area. These requirements are perhaps more effectively achieved by more conventional Summer School-like activities.

From a general perspective, during the three training challenges it became clear that the social skills of individual team members affected their ability to work within a team environment. In this respect, the challenges accurately reflected a real multidisciplinary work environment. Where social interactions became an issue for the performance of the team, a significant proportion of tutor time was devoted to the improvement of inter- and intra-team communication and advising on the appropriate delegation of tasks. Within the challenges, team size was designed to avoid student couplets working independently. In practice, however, this still occurred. Perhaps predictably, the absence of a ‘natural’ leader within a team also had an impact on their progress. Interestingly, whilst teams with a natural leader seemed to make more progress at the beginning of a challenge, ‘natural leadership’ could also hamper the process when discussions turned towards a scientifically detailed topic.

To improve the sociological aspects of the training challenge format, it may be prudent to open future events with a formal training session in methods for work in a team environment. Towards this objective, training challenges 2 and 3 featured a contribution from an expert in multidisciplinary teamwork. This contribution was well received by all students; however, it was not 100% effective in preventing differences of opinion which hampered team performance. A solution to this problem may be the implementation of a more stringent selection process which includes an interview. This would aid in the identification of those not suited to a multidisciplinary work environment.

From a scientific perspective, the research undertaken by all the teams yielded interesting and promising ideas and concepts. We are not aware, however, whether any of the research leads have been actively pursued in the aftermath of the training challenge by the student whose case was selected. It may well be interesting to reconvene teams at a later date in order to discuss and further develop ideas and concepts, and pursue tangible results from the work (e.g., publications, proposals, etc.). Such results could then be applied to the assessment of the training success.

In summary, we feel that the Training Challenge format was successful in providing students with a first experience of multidisciplinary BMI-related project work. The events challenged students to (among other things): overcome language barriers, differences in expertise, agree on a work plan, and respect a variety of scientific viewpoints.

Feedback from the students indicated that the Training Challenge had encouraged students to work in multidisciplinary teams and demonstrated to them that multidisciplinary team work can be enjoyable and rewarding.

Conclusion

From its inception in 2004, INFOBIOMED aimed to establish a wide variety of educational and training activities. This objective was achieved by developing a range of applications, services and activities enabling BMI-knowledge acquisition (combined MI and BI courses), the facilitation of collaborative BMI exchanges and the acquisition of multidisciplinary team working skills.

Encouragingly, this combination of different education and training activities has been very positively received by students and could serve as a starting point for a continued European effort in this field. The combination of traditional courses and novel educational activities serves to teach students how to bridge gaps between MI and BI, and merge the two fields in a multidisciplinary setting. In future projects, more attention should be given to how a mobility program plus training challenge could be most effectively combined with more traditional educational activities. In INFOBIOMED, educational activities were isolated activities and were not integrated into a series of interlinked educational events.

To evolve this approach in the future, we feel it will be a key issue to further emphasise to BMI-course designers and student supervisors that multidisciplinary BMI training and experiences are critical to the development of BMI students. To achieve integration into current curricula and research projects, time schedules will have to be adjusted to include overseas exchanges, summer-school attendance, and participation in novel events such as the Training Challenges described in this paper.