The overall system consists of five steps: (1) pre-processing sentences, (2) marking noun phrases (NPs) and adjective phrases (APs), (3) extracting concepts that use a dosage-unit dictionary to dynamically switch two models based on Conditional Random Fields (CRF), (4) classifying assertions based on voting of five classifiers, and (5) identifying relations using normalized sentences with a set of effective discriminating features.

Macro-averaged and micro-averaged precision, recall and F-measure were used to evaluate results.

The performance is competitive with the state-of-the-art systems with micro-averaged F-measure of 0.8489 for concept extraction, 0.9392 for assertion classification and 0.7326 for relation identification.

The system exploits an array of common features and achieves state-of-the-art performance. Prudent feature engineering sets the foundation of our systems. In concept extraction, we demonstrated that switching models, one of which is especially designed for telegraphic sentences, improved extraction of the treatment concept significantly. In assertion classification, a set of features derived from a rule-based classifier were proven to be effective for the classes such as conditional and possible. These classes would suffer from data scarcity in conventional machine-learning methods. In relation identification, we use two-staged architecture, the second of which applies pairwise classifiers to possible candidate classes. This architecture significantly improves performance.

Data on 2505 ambulatory surgery cases were prospectively collected at an academic center. Sixteen patient-related, surgical, and anesthetic predictors were used to develop a logistic regression model. The experimental model (EM) was compared against the original Apfel model (OAM), refitted Apfel model (RAM), simplified Apfel risk score (SARS), and refitted Sinclair model (RSM) by examining the discriminating power calculated using area under the curve (AUC) and by examining calibration curves.

The EM contained 11 input variables. The AUC was 0.738 for the EM, 0.620 for the OAM, 0.629 for the RAM, 0.626 for the SARS, and 0.711 for the RSM. Pair-wise discrimination comparison of models showed statistically significant differences (p<0.05) in AUC between the EM and all other models, OAM and RSM, RAM and RSM, and SARS and RSM.

All models except the OAM appeared to have good calibration for our institution's ambulatory surgery data. Ours is the first model to break down risk by anesthetic technique and incorporate risk reduction due to PONV prophylaxis.

The EM showed statistically significant improved discrimination over existing models and good calibration. However, the EM should be validated at another institution.

Through iterative review, we developed metrics to estimate the appropriateness of manually entered problem–medication links for inclusion in a knowledge base that can be used to infer previously unasserted links between problems and medications.

Clinicians manually linked 231 223 medications (55.30% of prescribed medications) to problems within the electronic health record, generating 41 203 distinct problem–medication pairs, although not all were accurate. We developed methods to evaluate the accuracy of the pairs, and after limiting the pairs to those meeting an estimated 95% appropriateness threshold, 11 166 pairs remained. The pairs in the knowledge base accounted for 183 127 total links asserted (76.47% of all links). Retrospective application of the knowledge base linked 68 316 medications not previously linked by a clinician to an indicated problem (36.53% of unlinked medications). Expert review of the combined knowledge base, including inferred and manually linked problem–medication pairs, found a sensitivity of 65.8% and a specificity of 97.9%.

Crowdsourcing is an effective, inexpensive method for generating a knowledge base of problem–medication pairs that is automatically mapped to local terminologies, up-to-date, and reflective of local prescribing practices and trends.

An iterative, user-centered design process informed by Bandura's social cognitive theory was employed to design DIMA—a mobile application that utilizes touch-screen, visual interfaces; barcode scanning; and voice recording to assist varying-literacy patients receiving hemodialysis to self-monitor their diet. A pilot field study was conducted where 18 patients receiving hemodialysis were recruited face-to-face from two dialysis facilities to use DIMA for 6 weeks. Subjects recorded their dietary intake using DIMA and met with research assistants three times each week. All interactions with DIMA were logged. Subjects' interdialytic weight gain was recorded throughout the study. At the end of the study, two face-to-face questionnaires were administered to assess usability and context of use.

Subjects were able to use DIMA successfully—12 subjects used DIMA as much or more at the end of the study as they did at the beginning and reported that DIMA helped them change their diet. Subjects had difficulty using the barcode scanner. Viewing past meals was the most used of the reflection mechanisms in DIMA.

Results suggest that while many design features were useful, some could be improved. In particular, future versions of DIMA will be on a smartphone using a camera for barcode scanning, integrate feedback and past meal reflection into the normal flow of the application, and support visual cues when selecting food items.

A machine learning approach based on graphical models was employed to cluster coreferent concepts. Features selected were divided into domain independent and domain specific sets. Training was done with the i2b2 provided training set of 489 documents with 6949 chains. Testing was done on 322 documents.

The learning engine, using the un-weighted average of three different measurement schemes, resulted in an F measure of 0.8423 where no domain specific features were included and 0.8483 where the feature set included both domain independent and domain specific features.

Our machine learning approach is a promising solution for recognizing coreferent concepts, which in turn is useful for practical applications such as the assembly of problem and medication lists from clinical documents.

A PCHMS called Healthy.me was developed and evaluated in a 2010 CONSORT-compliant two-group (6-month waitlist vs PCHMS) parallel randomized controlled trial (RCT) (allocation ratio 1:1). The PCHMS integrated an untethered personal health record with consumer care pathways, social forums, and messaging links with a health service provider.

742 university students and staff met inclusion criteria and were randomized to a 6-month waitlist (n=372) or the PCHMS (n=370). Amongst the 470 participants eligible for primary analysis, PCHMS users were 6.7% (95% CI: 1.46 to 12.30) more likely than the waitlist to receive an influenza vaccine (waitlist: 4.9% (12/246, 95% CI 2.8 to 8.3) vs PCHMS: 11.6% (26/224, 95% CI 8.0 to 16.5); ²=7.1, p=0.008). PCHMS participants were also 11.6% (95% CI 3.6 to 19.5) more likely to visit the health service provider (waitlist: 17.9% (44/246, 95% CI 13.6 to 23.2) vs PCHMS: 29.5% (66/224, 95% CI: 23.9 to 35.7); ²=8.8, p=0.003). A dose–response effect was detected, where greater use of the PCHMS was associated with higher rates of vaccination (p=0.001) and health service provider visits (p=0.003).

PCHMS can significantly increase consumer participation in preventive health activities, such as influenza vaccination.

Integrating a PCHMS into routine health service delivery systems appears to be an effective mechanism for enhancing consumer engagement in preventive health measures.

Australian New Zealand Clinical Trials Registry ACTRN12610000386033. http://www.anzctr.org.au/trial_view.aspx?id=335463

A key contribution of this paper is the application of contextual-dependent rules that describe relationships between coreference pairs. To resolve phrases that refer to the same entity, the authors use these rules in three representative NLP systems: one rule-based, another based on the maximum entropy model, and the last a system built on the Markov logic network (MLN) model.

The experimental results show that the proposed MLN-based system outperforms the baseline system (exact match) by average F-scores of 4.3% and 5.7% on the Beth and Partners datasets, respectively. Finally, the three systems were integrated into an ensemble system, further improving performance to 87.21%, which is 4.5% more than the official i2b2 Track 1C average (82.7%).

In this paper, the main challenges in the resolution of coreference relations in patient discharge summaries are described. Several rules are proposed to exploit contextual information, and three approaches presented. While single systems provided promising results, an ensemble approach combining the three systems produced a better performance than even the best single system.

From September 2005 to March 2007, we conducted an on-treatment sub-study within a cluster-randomized trial involving 11 primary care practices that used the same PHR. Intervention practices received access to a medications module prompting patients to review their documented medications and identify discrepancies, generating ‘eJournals’ that enabled rapid updating of medication lists during subsequent clinical visits.

A sample of 267 patients who submitted medications eJournals was contacted by phone 3 weeks after an eligible visit and compared with a matched sample of 274 patients in control practices that received a different PHR-linked intervention. Two blinded physician adjudicators determined unexplained discrepancies between documented and patient-reported medication regimens. The primary outcome was proportion of medications per patient with unexplained discrepancies.

Among 121 046 patients in eligible practices, 3979 participated in the main trial and 541 participated in the sub-study. The proportion of medications per patient with unexplained discrepancies was 42% in the intervention arm and 51% in the control arm (adjusted OR 0.71, 95% CI 0.54 to 0.94, p=0.01). The number of unexplained discrepancies per patient with potential for severe harm was 0.03 in the intervention arm and 0.08 in the control arm (adjusted RR 0.31, 95% CI 0.10 to 0.92, p=0.04).

When used, concordance between documented and patient-reported medication regimens and reduction in potentially harmful medication discrepancies can be improved with a PHR medication review tool linked to the provider's medical record.

This study was registered at ClinicalTrials.gov (NCT00251875).

The CDSS was constituted by two rulebases: the free-text rulebase for interpreting Pap reports and a guideline rulebase. The free-text rulebase was developed by analyzing a corpus of 49 293 Pap reports. The guideline rulebase was constructed using national cervical cancer screening guidelines. The CDSS accesses the electronic medical record (EMR) system to generate patient-specific recommendations. For evaluation, the screening recommendations made by the CDSS for 74 patients were reviewed by a physician.

Evaluation revealed that the CDSS outputs the optimal screening recommendations for 73 out of 74 test patients and it identified two cases for gynecology referral that were missed by the physician. The CDSS aided the physician to amend recommendations in six cases. The failure case was because human papillomavirus (HPV) testing was sometimes performed separately from the Pap test and these results were reported by a laboratory system that was not queried by the CDSS. Subsequently, the CDSS was upgraded to look up the HPV results missed earlier and it generated the optimal recommendations for all 74 test cases.

Single institution and single expert study.

An accurate CDSS system could be constructed for cervical cancer screening given the standardized reporting of Pap tests and the availability of explicit guidelines. Overall, the study demonstrates that free text in the EMR can be effectively utilized through natural language processing to develop clinical decision support tools.

Data were collected from a US-based online research database, the Interactive Autism Network (IAN). A total of 19 497 youths, aged 1.9–19 years (mean 9 years, SD 3.94), were included in the present study. Response to three follow-up surveys, solicited from parents after baseline enrollment, served as the outcome measures. Multivariate binary logistic regression models were then used to examine predictors of non-response.

31 216 survey instances were examined, of which 8772 or 28.1% were partly or completely responded to. Results from the multivariate model found non-response of baseline surveys (OR 28.0), years since enrollment in the online protocol (OR 2.06), and numerous sociodemographic characteristics were associated with non-response to follow-up surveys (all p<0.05).

Consistent with the current literature, response rates to online surveys were somewhat low. While many demographic characteristics were associated with non-response, time since registration and participation at baseline played the greatest role in predicting follow-up survey non-response.

An important hazard to the generalizability of findings from research is non-response bias; however, little is known about this problem in longitudinal internet-mediated research (IMR). This study sheds new light on important predictors of longitudinal response rates that should be considered before launching a prospective IMR study.

We searched Medline, Pre-Medline, CINAHL, Embase, and the SPIE Digital Library databases for English-language publications between 1980 and September 2010 using Medical Subject Headings terms and keywords.

Eleven studies from the USA and UK were included. All studies measured aspects of time associated with the introduction of PACS, namely the availability of images, the time a physician took to review an image, and changes in viewing patterns. Seven studies examined the impact on clinical decision-making, with the majority measuring the time to image-based clinical action. The effect of PACS on communication modes was reported in five studies.

PACS can impact on clinician work practices in three main areas. Most of the evidence suggests an improvement in the efficiency of work practices. Quick image availability can impact on work associated with clinical decision-making, although the results were inconsistent. PACS can change communication practices, particularly between the ICU and radiology; however, the evidence base is insufficient to draw firm conclusions in this area.

The potential for PACS to impact positively on clinician work practices in the ICU and improve patient care is great. However, the evidence base is limited and does not reflect aspects of contemporary PACS technology. Performance measures developed in previous studies remain relevant, with much left to investigate to understand how PACS can support new and improved ways of delivering care in the intensive care setting.

A panel of experts was convened with the goal of identifying critical DDIs that should be used for generating medication-related decision support alerts in all EHRs. Panelists included medication knowledge base vendors, EHR vendors, in-house knowledge base developers from academic medical centers, and both federal and private agencies involved in the regulation of medication use. Candidate DDIs were assessed by the panel based on the consequence of the interaction, severity levels assigned to them across various medication knowledge bases, availability of therapeutic alternatives, monitoring/management options, predisposing factors, and the probability of the interaction based on the strength of evidence available in the literature.

Of 31 DDIs considered to be high risk, the panel approved a final list of 15 interactions. Panelists agreed that this list represented drugs that are contraindicated for concurrent use, though it does not necessarily represent a complete list of all such interacting drug pairs. For other drug interactions, severity may depend on additional factors, such as patient conditions or timing of co-administration.

The panel provided recommendations on the creation, maintenance, and implementation of a central repository of high severity interactions.

A set of highly clinically significant drug-drug interactions was identified, for which warnings should be generated in all EHRs. The panel highlighted the complexity of issues surrounding development and implementation of such a list.

We implement a knowledge-based model for calculating phenotype scores from patient-specific genotype data. These data include allelic variants of genes encoding enzymes involved in the bioactivation of tamoxifen. We performed quantile linear regression to evaluate whether six phenotype scoring algorithms are predictive of patient endoxifen/NDM plasma concentration ratio, and validate our scoring methods.

Our model illustrates a knowledge-based approach to predict drug metabolism efficacy given patient genomics data. Results showed that for one phenotype scoring algorithm, scores were weakly correlated with patient endoxifen/NDM plasma concentration ratios. This algorithm performed better than simple metrics for variation in individual and multiple genes.

We discuss advantages of the model, challenges to its implementation in a personalized medicine context, and provide example future directions.

We demonstrate the utility of our model in a tamoxifen case study context. We also provide evidence that more complicated polygenic models are needed to represent heterogeneity in clinical outcomes.

The studies were carried out in a cohort of recently delivered women in Tayside, Scotland, UK, who were asked about their current infant feeding method and future feeding plans. Reliability was assessed by comparison of their responses to two SMS messages sent 1 day apart. Validity was assessed by comparison of their responses to text questions and the same question administered by phone 1 day later, by comparison with the same data collected from other sources, and by correlation with other related measures. Acceptability was evaluated using quantitative and qualitative questions, and practicality by analysis of a researcher log.

Reliability of the factual SMS message gave perfect agreement. Reliabilities for the numerical question were reasonable, with between 0.76 (95% CI 0.56 to 0.96) and 0.80 (95% CI 0.59 to 1.00). Validity for data compared with that collected by phone within 24 h ( =0.92 (95% CI 0.84 to 1.00)) and with health visitor data ( =0.85 (95% CI 0.73 to 0.97)) was excellent. Correlation validity between the text responses and other related demographic and clinical measures was as expected. Participants found the method a convenient and acceptable way of providing data. For researchers, SMS text messaging provided an easy and functional method of gathering a large volume of data.

In this sample and for these questions, SMS was a reliable and valid method for capturing research data.

A binary classifier was employed to identify pneumonia from a dataset of multiple types of clinical notes created for 426 patients during their stay in the intensive care unit. For this purpose, three types of features were considered: (1) word n-grams, (2) Unified Medical Language System (UMLS) concepts, and (3) assertion values associated with pneumonia expressions. System performance was greatly increased by a feature selection approach which uses statistical significance testing to rank features based on their association with the two categories of pneumonia identification.

Besides testing our system on the entire cohort of 426 patients (unrestricted dataset), we also used a smaller subset of 236 patients (restricted dataset). The performance of the system was compared with the results of a baseline previously proposed for these two datasets. The best results achieved by the system (85.71 and 81.67 F1-measure) are significantly better than the baseline results (50.70 and 49.10 F1-measure) on the restricted and unrestricted datasets, respectively.

Using a statistical feature selection approach that allows the feature extractor to consider only the most informative features from the feature space significantly improves the performance over a baseline that uses all the features from the same feature space. Extracting the assertion value for pneumonia expressions further improves the system performance.

To examine the effectiveness of interventions using electronic reminders in improving patients' adherence to chronic medication.

A comprehensive literature search was conducted in PubMed, Embase, PsycINFO, CINAHL and Cochrane Central Register of Controlled Trials. Electronic searches were supplemented by manual searching of reference lists and reviews. Two reviewers independently screened all citations. Full text was obtained from selected citations and screened for final inclusion. The methodological quality of studies was assessed.

Thirteen studies met the inclusion criteria. Four studies evaluated short message service (SMS) reminders, seven audiovisual reminders from electronic reminder devices (ERD), and two pager messages. Best evidence synthesis revealed evidence for the effectiveness of electronic reminders, provided by eight (four high, four low quality) studies showing significant effects on patients' adherence, seven of which measured short-term effects (follow-up period <6 months). Improved adherence was found in all but one study using SMS reminders, four studies using ERD and one pager intervention. In addition, one high quality study using an ERD found subgroup effects.

This review provides evidence for the short-term effectiveness of electronic reminders, especially SMS reminders. However, long-term effects remain unclear.

During a 36-week study period, we systematically documented the response patterns of 178 physician users randomized to receive CTAs for an ongoing clinical trial. Data were collected on: (1) response rates to the CTA; and (2) referral rates per physician, per time unit. Variables of interest were offset by the log of the total number of alerts received by that physician during that time period, in a Poisson regression.

Response rates demonstrated a significant downward trend across time, with response rates decreasing by 2.7% for each advancing time period, significantly different from zero (flat) (p<0.0001). Even after 36 weeks, response rates remained in the 30%–40% range. Subgroup analyses revealed differences between community-based versus university-based physicians (p=0.0489).

CTA responsiveness declined gradually over prolonged exposure, although it remained reasonably high even after 36 weeks of exposure. There were also notable differences between community-based versus university-based users.

These findings add to the limited literature on this form of EHR-based alert fatigue and should help inform future tailoring, deployment, and further study of CTAs.

Our method is based on a latent variable model that links the observed outcomes to the underlying genetic variables. A Markov Chain Monte Carlo approach is used for model search and to evaluate the posterior probability of each predictor.

BOSS is compared with three established methods (stepwise regression, logistic lasso, and elastic net) in a simulated benchmark. Two real case studies are also investigated: a GWAS on the genetic bases of longevity, and the type 2 diabetes study from the Wellcome Trust Case Control Consortium. Simulations show that BOSS achieves higher precisions than the reference methods while preserving good recall rates. In both experimental studies, BOSS successfully detects genetic polymorphisms previously reported to be associated with the analyzed phenotypes.

BOSS outperforms the other methods in terms of F-measure on simulated data. In the two real studies, BOSS successfully detects biologically relevant features, some of which are missed by univariate analysis and the three reference techniques.

The proposed algorithm is an advance in the methodology for model selection with a large number of features. Our simulated and experimental results showed that BOSS proves effective in detecting relevant markers while providing a parsimonious model.

Instead of bringing data to a central repository for computation, we bring computation to the data. The Grid Binary LOgistic REgression (GLORE) model integrates decomposable partial elements or non-privacy sensitive prediction values to obtain model coefficients, the variance-covariance matrix, the goodness-of-fit test statistic, and the area under the receiver operating characteristic (ROC) curve.

We conducted experiments on both simulated and clinically relevant data, and compared the computational costs of GLORE with those of a traditional LR model estimated using the combined data. We showed that our results are the same as those of LR to a 10^–15 precision. In addition, GLORE is computationally efficient.

In GLORE, the calculation of coefficient gradients must be synchronized at different sites, which involves some effort to ensure the integrity of communication. Ensuring that the predictors have the same format and meaning across the data sets is necessary.

The results suggest that GLORE performs as well as LR and allows data to remain protected at their original sites.

A perturbation mechanism was designed in which users are given options with respect to scale and direction of the perturbation. The mechanism translates the true count, user preferences, and a privacy level within administrator-specified bounds into a probability distribution from which the perturbed count is drawn.

Users can significantly impact the scale and direction of the count perturbation and can receive more accurate final cohort estimates. Strong and semantically meaningful differential privacy is guaranteed, providing for a unified privacy accounting system that can support role-based trust levels. This study provides an open source web-enabled tool to investigate visually and numerically the interaction between system parameters, including required privacy level and user preference settings.

Quantifying privacy allows system administrators to provide users with a privacy budget and to monitor its expenditure, enabling users to control the inevitable loss of utility. While current measures of privacy are conservative, this system can take advantage of future advances in privacy measurement. The system provides new ways of trading off privacy and utility that are not provided in current study design systems.

Retrospective observational study using EMR data of all 10 715 patients who received genetic testing by physicians trained in a primary care specialty or subspecialty at an academic medical center between January 1, 2008 and December 31, 2010.

Patients had a mean±SD age of 38.3±15.8 years (median 36.1, IQR 30.0–43.8). The proportion of female subjects in the study population was larger than in the general patient population (77.2% vs 55.0%, p<0.001) and they were younger than the male subjects in the study (36.5±13.2 vs 44.6±21.2 years, p<0.001). Approximately 1.1% of all patients received genetic testing. There were 942 physicians who ordered a total of 15 320 genetic tests. By volume, commonly tested genes involved mutations for cystic fibrosis (36.7%), prothrombin (13.7%), Tay–Sachs disease (6.7%), hereditary hemochromatosis (4.4%), and chronic myelogenous leukemia (4.1%). EMRs stored reports as free text with categorical descriptions of mutations and an average length of 269.4±153.2 words (median 242, IQR 146–401).

In this study, genetic tests were often ordered by a diverse group of physicians for women of childbearing age being evaluated for diseases that may affect potential offspring. EMRs currently serve primarily as a storage warehouse for textual reports that could potentially be transformed into meaningful structured data for next-generation clinical decision support. Further studies are needed to address the design, development, and implementation of EMRs capable of managing the critical genetic health information challenges of the future.

The CDEs of the National Cancer Institute (NCI) Cancer Data Standards Repository, the NCI Thesaurus (NCIt) concepts and the UMLS semantic network were integrated using a semantic web-based framework for a SPARQL-enabled evaluation. First, the set of CDE-permissible values with corresponding meanings in external controlled terminologies were isolated. The corresponding value meanings were then evaluated against their NCI- or UMLS-generated semantic network mapping to determine whether all of the meanings fell within the same semantic group.

Of the enumerated CDEs in the Cancer Data Standards Repository, 3093 (26.2%) had elements drawn from more than one UMLS semantic group. A random sample (n=100) of this set of elements indicated that 17% of them were likely to have been misclassified.

The use of existing semantic web tools can support a high-throughput mechanism for evaluating the quality of large CDE collections. This study demonstrates that the involvement of multiple semantic groups in an enumerated value domain of a CDE is an effective anchor to trigger an auditing point for quality evaluation activities.

This approach produces a useful quality assurance mechanism for a clinical study CDE repository.

To fill this gap a three-phased process was used to conceptualize, identify and recommend patient-reported data elements on health behaviors and psychosocial factors for the EHR. Expert panels (n=13) identified candidate measures (phase 1) that were reviewed and rated by a wide range of health professionals (n=93) using the grid-enabled measures wiki social media platform (phase 2). Recommendations were finalized through a town hall meeting with key stakeholders including patients, providers, researchers, policy makers, and representatives from healthcare settings (phase 3).

Nine key elements from three areas emerged as the initial critical patient-reported elements to incorporate systematically into EHR—health behaviors (eg, exercise), psychosocial issues (eg, distress), and patient-centered factors (eg, demographics). Recommendations were also made regarding the frequency of collection ranging from a single assessment (eg, demographic characteristics), to annual assessment (eg, health behaviors), or more frequent (eg, patient goals).

There was strong stakeholder support for this initiative reflecting the perceived value of incorporating patient-reported elements into EHR. The next steps will include testing the feasibility of incorporating these elements into the EHR across diverse primary care settings.

An integrated coreference resolution system was developed by exploiting Person attributes, contextual semantic clues, and world knowledge. It includes three subsystems: Person coreference system based on three Person attributes, Problem/Treatment/Test system based on numerous contextual semantic extractors and world knowledge, and Pronoun system based on a multi-class support vector machine classifier. The three Person attributes are patient, relative and hospital personnel. Contextual semantic extractors include anatomy, position, medication, indicator, temporal, spatial, section, modifier, equipment, operation, and assertion. The world knowledge is extracted from external resources such as Wikipedia.

Micro-averaged precision, recall and F-measure in MUC, BCubed and CEAF were used to evaluate results.

The system achieved an overall micro-averaged precision, recall and F-measure of 0.906, 0.925, and 0.915, respectively, on test data (from four hospitals) released by the challenge organizers. It achieved a precision, recall and F-measure of 0.905, 0.920 and 0.913, respectively, on test data without Pittsburgh data. We ranked the first out of 20 competing teams. Among the four sub-tasks on Person, Problem, Treatment, and Test, the highest F-measure was seen for Person coreference.

This system achieved encouraging results. The Person system can determine whether personal pronouns and proper names are coreferent or not. The Problem/Treatment/Test system benefits from both world knowledge in evaluating the similarity of two mentions and contextual semantic extractors in identifying semantic clues. The Pronoun system can automatically detect whether a Pronoun mention is coreferent to that of the other four types. This study demonstrates that it is feasible to accomplish the coreference task in discharge summaries.

This study introduces and explores the notions of research sponsorship and allocation and shows that leaders of research activity can be identified within specific disease areas of interest, such as those with high mortality or high sponsorship. The funding profiles of disease topics readily cluster themselves in agreement with the ontology hierarchy and closely mirror the funding agency priorities. Finally, four temporal trends are identified among research topics.

This work utilizes disease ontology (DO)-based annotation to profile effectively the landscape of biomedical research activity. By using DO in this manner a use-case driven mechanism is also proposed to evaluate the utility of classification hierarchies.

Based on the occurrences of UMLS terms in a 51 million document corpus of Mayo Clinic clinical notes, this study computes statistics about the terms' string attributes, source terminologies, semantic types and syntactic categories. Term occurrences in 2010 i2b2/VA text were also mapped; eight example filters were designed from the Mayo-based statistics and applied to i2b2/VA data.

For the corpus analysis, negligible numbers of mapped terms in the Mayo corpus had over six words or 55 characters. Of source terminologies in the UMLS, the Consumer Health Vocabulary and Systematized Nomenclature of Medicine—Clinical Terms (SNOMED-CT) had the best coverage in Mayo clinical notes at 106 426 and 94 788 unique terms, respectively. Of 15 semantic groups in the UMLS, seven groups accounted for 92.08% of term occurrences in Mayo data. Syntactically, over 90% of matched terms were in noun phrases. For the cross-institutional analysis, using five example filters on i2b2/VA data reduces the actual lexicon to 19.13% of the size of the UMLS and only sees a 2% reduction in matched terms.

The corpus statistics presented here are instructive for building lexicons from the UMLS. Features intrinsic to Metathesaurus terms (well formedness, length and language) generalise easily across clinical institutions, but term frequencies should be adapted with caution. The semantic groups of mapped terms may differ slightly from institution to institution, but they differ greatly when moving to the biomedical literature domain.

A data-driven approach was proposed that utilizes individualized confidence intervals (CIs) to select the most ‘appropriate’ model from a pool of candidates to assess the individual patient's clinical condition. The method does not require access to the training dataset. This approach was compared with other strategies: the BEST model (the ideal model, which can only be achieved by access to data or knowledge of which population is most similar to the individual), CROSS model, and RANDOM model selection.

When evaluated on clinical datasets, the approach significantly outperformed the CROSS model selection strategy in terms of discrimination (p<1e–14) and calibration (p<0.006). The method outperformed the RANDOM model selection strategy in terms of discrimination (p<1e–12), but the improvement did not achieve significance for calibration (p=0.1375).

The CI may not always offer enough information to rank the reliability of predictions, and this evaluation was done using aggregation. If a particular individual is very different from those represented in a training set of existing models, the CI may be somewhat misleading.

This approach has the potential to offer more reliable predictions than those offered by other heuristics for disease risk estimation of individual patients.

The Kullback–Leibler divergence retrieval model with Dirichlet smoothing is used as the state-of-the-art general retrieval method. Pseudorelevance feedback and term weighing methods are proposed by leveraging MeSH and UMLS thesauri. Evaluation is performed on a test collection recently created for the ImageCLEF medical case retrieval challenge.

Experimental results show that a well-tuned state-of-the-art general retrieval model achieves a mean average precision of 0.2754, but the performance can be improved by over 40% to 0.3980, through the proposed methods.

The results over the ImageCLEF test collection, which is currently the best collection available for the task, are encouraging. There are, however, limitations due to small evaluation set size. The analysis shows that further refinement of the methods is necessary before they can be really useful in a clinical setting.

Medical case-based literature retrieval is a critical search application that presents a number of unique challenges. This analysis shows that the state-of-the-art general retrieval models are reasonably good for the task, but the performance can be significantly improved by developing new task-specific retrieval models that incorporate medical thesauri and physician feedback.

Systematic observations methods were used to follow 19 ICU nurses for 38 hours in 3 clinical practice settings. Storyboard methods and concept mapping helped to categorize the observed tasks, the associated information needs, and the information gaps of the most frequent tasks by SA level. Consensus and discussion of the research team was used to propose recommendations to improve information displays at the bedside based on information deficits.

Nurses performed 46 different tasks at a rate of 23.4 tasks per hour. The information needed to perform the most common tasks was often inaccessible, difficult to see at a distance or located on multiple monitoring devices. Current devices at the ICU bedside do not adequately support a nurse's information-gathering activities. Medication management was the most frequent category of tasks.

Information gaps were present at all levels of SA and across most of the tasks. Using a theoretical model to understand information gaps can aid in designing functional requirements.

Integrated information that enhances nurses' Situation Awareness may decrease errors and improve patient safety in the future.

We created a set of regular expressions and rules to capture the EF using a random sample of 765 echocardiograms from seven VA medical centers. The documents were randomly assigned to two sets: a set of 275 used for training and a second set of 490 used for testing and validation. To establish the reference standard, two independent reviewers annotated all documents in both sets; a third reviewer adjudicated disagreements.

System test results for document-level classification of EF of <40% had a sensitivity (recall) of 98.41%, a specificity of 100%, a positive predictive value (precision) of 100%, and an F measure of 99.2%. System test results at the concept level had a sensitivity of 88.9% (95% CI 87.7% to 90.0%), a positive predictive value of 95% (95% CI 94.2% to 95.9%), and an F measure of 91.9% (95% CI 91.2% to 92.7%).

An EF value of <40% can be accurately identified in VA echocardiogram reports.

An automated information extraction system can be used to accurately extract EF for quality measurement.

Physicians reviewed charts from three institutions to identify patients with RA. Each institution compiled attributes from various sources in the EHR, including codified data and clinical narratives, which were searched using one of two natural language processing (NLP) systems. The performance of the published model was compared with locally retrained models.

Applying the previously published model from Partners Healthcare to datasets from Northwestern and Vanderbilt Universities, the area under the receiver operating characteristic curve was found to be 92% for Northwestern and 95% for Vanderbilt, compared with 97% at Partners. Retraining the model improved the average sensitivity at a specificity of 97% to 72% from the original 65%. Both the original logistic regression models and locally retrained models were superior to simple billing code count thresholds.

These results show that a previously published algorithm for RA is portable to two external hospitals using different EHR systems, different NLP systems, and different target NLP vocabularies. Retraining the algorithm primarily increased the sensitivity at each site.

Electronic phenotype algorithms allow rapid identification of case populations in multiple sites with little retraining.

A knowledge base from a diagnostic decision support system was used to identify HIFs for selected HRDs: lumbar disc disease, myocardial infarction, appendicitis, and colon, breast, lung, ovarian and bladder carcinomas. Two physicians reviewed at least 20 patient records retrieved from a research patient data registry for each of these eight HRDs and for age- and gender-compatible controls. Records were searched for HIFs in visit notes that were created before the HRD was established in the electronic record and in general medical visit notes for controls.

25% of records reviewed (61/243) contained HIFs in notes before the HRD was established. The mean duration between HIFs first occurring in the record and time of diagnosis ranged from 19 days for breast cancer to 2 years for bladder cancer. In three of the eight HRDs, HIFs were much less likely in control patients without the HRD.

In many records of patients with an HRD, HIFs were present before the HRD was established. Reasons for delay include non-compliance with recommended follow-up, unusual presentation of a disease, and system errors (eg, lack of laboratory follow-up). The presence of HIFs in clinical records suggests a potential role for the integration of diagnostic decision support into the clinical workflow to provide reminder alerts to improve the diagnostic focus.

The Office of the National Coordinator for Health Information Technology, through the Strategic Health IT Advanced Research Projects (SHARP) Program, funds the project. The SMART team has focused on enabling the property of substitutability through an app programming interface leveraging web standards, presenting predictable data payloads, and abstracting away many details of enterprise health information technology systems. Containers—health information technology systems, such as electronic health records (EHR), personally controlled health records, and health information exchanges that use the SMART app programming interface or a portion of it—marshal data sources and present data simply, reliably, and consistently to apps.

The SMART team has completed the first phase of the project (a) defining an app programming interface, (b) developing containers, and (c) producing a set of charter apps that showcase the system capabilities. A focal point of this phase was the SMART Apps Challenge, publicized by the White House, using http://www.challenge.gov website, and generating 15 app submissions with diverse functionality.

Key strategic decisions must be made about the most effective market for further disseminating SMART: existing market-leading EHR vendors, new entrants into the EHR market, or other stakeholders such as health information exchanges.

This study presents a medical coreference resolution system (MCORES) for noun phrases in four frequently used clinical semantic categories: persons, problems, treatments, and tests. MCORES treats coreference resolution as a binary classification task. Given a pair of concepts from a semantic category, it determines coreferent pairs and clusters them into chains. MCORES uses an enhanced set of lexical, syntactic, and semantic features. Some MCORES features measure the distance between various representations of the concepts in a pair and can be asymmetric.

MCORES was compared with an in-house baseline that uses only single-perspective ‘token overlap’ and ‘number agreement’ features. MCORES was shown to outperform the baseline; its enhanced features contribute significantly to performance. In addition to the baseline, MCORES was compared against two available third-party, open-domain systems, RECONCILE_ACL09 and the Beautiful Anaphora Resolution Toolkit (BART). MCORES was shown to outperform both of these systems on clinical records.

This study proposed a method of measuring semantic similarities between GO terms using the entire GO tree structure, including both the upper (ancestral) and the lower (descendant) parts. Comprehensive comparison studies were performed with well-known information content-based and graph structure-based semantic similarity measures with protein sequence similarities, gene expression-profile correlations, protein–protein interactions, and biological pathway analyses.

The proposed bidirectional measure of semantic similarity outperformed other graph-based and information content-based methods.

Mobile phones were utilized and a structured survey was implemented to be administered by community health workers using Open Data Kit. This system was used to support screening efforts for a population of two million persons in western Kenya.

Users of the system felt it was easy to use and facilitated their work. The system was also more cost effective than pen and paper alternatives.

This implementation is one of the largest applications of a system utilizing handheld devices for performing clinical care during home visits in a resource-constrained environment. Because the data were immediately available electronically, initial reports could be performed and important trends in data could thus be detected. This allowed adjustments to the programme to be made sooner than might have otherwise been possible.

A viable, cost-effective solution at scale has been developed and implemented for collecting electronic data during household visits in a resource-constrained setting.

The coreference challenge provided the community with two annotated ground truth corpora and evaluated systems on coreference resolution in two ways: first, it evaluated systems for their ability to identify mentions of concepts and to link together those mentions. Second, it evaluated the ability of the systems to link together ground truth mentions that refer to the same entity. Twenty teams representing 29 organizations and nine countries participated in the coreference challenge.

The teams' system submissions showed that machine-learning and rule-based approaches worked best when augmented with external knowledge sources and coreference clues extracted from document structure. The systems performed better in coreference resolution when provided with ground truth mentions. Overall, the systems struggled in solving coreference resolution for cases that required domain knowledge.

Existing functionalities of the Veterans Affairs (VA) computerized patient record system (CPRS)/veterans health information systems and technology architecture (VISTA) were modified to support the activities of a randomized controlled trial including enrolment, randomization, and longitudinal data collection.

The VA's CPRS/VISTA was successfully adapted to support the processes of a clinical trial and longitudinal study data are being collected from the medical record automatically. As of 30 June 2011, 55 of the 67 eligible patients approached received a randomized intervention.

The design of CPRS/VISTA made integration of study workflows and data collection possible. Institutions and investigators considering similar designs must carefully map clinical workflows and clinical trial workflows to EMR capabilities. POCCT study teams are necessarily interdisciplinary and interdepartmental. As a result, executive sponsorship is critical.

POCCT represent a promising new method for conducting clinical science. Much work is needed to understand better the optimal uses and designs for this new approach. Next steps include focus groups to measure patient and clinician perceptions, multisite deployment of the current pilot, and implementation of additional studies.

Retrospective electronic chart reviews of 4081 patient records across 57 practices were analyzed to determine the validity of EHR-derived quality measures and documented preventive services.

Results from this study show that workflow and documentation habits have a profound impact on EHR-derived quality measures. Compared with the manual review of electronic charts, EHR-derived measures can undercount practice performance, with a disproportionately negative impact on the number of patients captured as receiving a clinical preventive service or meeting a recommended treatment goal.

This study provides a cautionary note in using EHR-derived measurement for public reporting of provider performance or use for payment.

Our hypothesis was that most variation in names occurs after the first two letters, and that date of birth is highly reliable, so a single match variable consisting of a hashed string built from the first two letters of the patient's first and last names plus their date of birth would have the desired characteristics. We compared and contrasted the match algorithm characteristics (rate of false positive v. rate of false negative) for our chosen variable against both Social Security Numbers and full names.

In a data set of 19 000 records, a derived match variable consisting of a 2-character prefix from both first and last names combined with date of birth has a 97% sensitivity; by contrast, an anonymized identifier based on the patient's full names and date of birth has a sensitivity of only 87% and SSN has sensitivity 86%.

The approach we describe is most useful in situations where privacy policies preclude the full exchange of the identifiers required by more sophisticated and sensitive linkage algorithms. For data sets of sufficiently high quality this effective approach, while producing a lower rate of matching than more complex algorithms, has the merit of being easy to explain to institutional review boards, adheres to the minimum necessary rule of the HIPAA privacy rule, and is faster and less cumbersome to implement than a full probabilistic linkage.

The Ontology Development and Information Extraction corpus annotated for coreference relations consists of 7214 coreferential markables, forming 5992 pairs and 1304 chains. We trained classifiers with semantic, syntactic, and surface features pruned by feature selection. For the three system components—for the resolution of relative pronouns, personal pronouns, and noun phrases—we experimented with support vector machines with linear and radial basis function (RBF) kernels, decision trees, and perceptrons. Evaluation of algorithms and varied feature sets was performed using standard metrics.

The best performing combination is support vector machines with an RBF kernel and all features (MUC score=0.352, B³=0.690, CEAF=0.486, BLANC=0.596) outperforming a traditional decision tree baseline.

The application showed good performance similar to performance on general English text. The main error source was sentence distances exceeding a window of 10 sentences between markables. A possible solution to this problem is hinted at by the fact that coreferent markables sometimes occurred in predictable (although distant) note sections. Another system limitation is failure to fully utilize synonymy and ontological knowledge. Future work will investigate additional ways to incorporate syntactic features into the coreference problem.

We investigated computational methods for coreference resolution in the clinical narrative. The best methods are released as modules of the open source Clinical Text Analysis and Knowledge Extraction System and Ontology Development and Information Extraction platforms.

Subsets were first extracted using four graph-traversal heuristics and one logic-based technique, and were subsequently filtered with frequency information from MEDLINE. Twenty manually coded discharge summaries from cardiology patients were used as signatures and test sets. The coverage, size, and precision of extracted subsets were measured.

Graph-traversal heuristics provided high coverage (71–96% of terms in the test sets of discharge summaries) at the expense of subset size (17–51% of the size of SNOMED CT). Pre-computed subsets and logic-based techniques extracted small subsets (1%), but coverage was limited (24–55%). Filtering reduced the size of large subsets to 10% while still providing 80% coverage.

Extracting subsets to annotate discharge summaries is challenging when no previous corpus exists. Ontology modularization provides valuable techniques, but the resulting modules grow as signatures spread across subhierarchies, yielding a very low precision.

Graph-traversal strategies and frequency data from an authoritative source can prune large biomedical ontologies and produce useful subsets that still exhibit acceptable coverage. However, a clinical corpus closer to the specific use case is preferred when available.

Electronic query of EHR data identified women aged 30–65 years old who were at low-risk of cervical cancer and therefore eligible for an extended Pap testing interval of 3 years (as per professional society guidelines). Manual chart review assessed query accuracy. The use of low-value Pap tests (ie, those performed sooner than recommended) was measured, and adverse consequences of low-value Pap tests (ie, colposcopies performed as a result of low-value Pap tests) were identified.

Manual chart review confirmed query accuracy. Two-thirds (1120/1705) of low-risk women received a Pap test sooner than recommended, and 21 colposcopies were performed as a result of this low-value Pap testing.

Secondary analysis of EHR data can accurately measure the use of low-value services such as Pap testing performed sooner than recommended in women at low risk of cervical cancer. Similar application of our methodology could facilitate efforts to simultaneously improve quality and decrease costs, maximizing value in the US healthcare system.

To determine whether computerized prescribing decision support with patient-specific risk estimates would increase physician response to psychotropic drug alerts and reduce injury risk in older people.

Cluster randomized controlled trial of 81 family physicians and 5628 of their patients aged 65 and older who were prescribed psychotropic medication.

Intervention physicians received information about patient-specific risk of injury computed at the time of each visit using statistical models of non-modifiable risk factors and psychotropic drug doses. Risk thermometers presented changes in absolute and relative risk with each change in drug treatment. Control physicians received commercial drug alerts.

Injury risk at the end of follow-up based on psychotropic drug doses and non-modifiable risk factors. Electronic health records and provincial insurance administrative data were used to measure outcomes.

Mean patient age was 75.2 years. Baseline risk of injury was 3.94 per 100 patients per year. Intermediate-acting benzodiazepines (56.2%) were the most common psychotropic drug. Intervention physicians reviewed therapy in 83.3% of visits and modified therapy in 24.6%. The intervention reduced the risk of injury by 1.7 injuries per 1000 patients (95% CI 0.2/1000 to 3.2/1000; p=0.02). The effect of the intervention was greater for patients with higher baseline risks of injury (p<0.03).

Patient-specific risk estimates provide an effective method of reducing the risk of injury for high-risk older people.

clinicaltrials.gov Identifier: NCT00818285.

Statistical text classifiers based on Naïve Bayes and Support Vector Machine (SVM) algorithms were trained and tested on clinical incident reports to automatically detect extreme-risk events, defined by incidents that satisfy the criteria of Severity Assessment Code (SAC) level 1. For this purpose, incident reports submitted to the Advanced Incident Management System by public hospitals from one Australian region were used. The classifiers were evaluated on two datasets: (1) a set of reports with diverse incident types (n=120); (2) a set of reports associated with patient misidentification (n=166). Results were assessed using accuracy, precision, recall, F-measure, and area under the curve (AUC) of receiver operating characteristic curves.

The classifiers performed well on both datasets. In the multi-type dataset, SVM with a linear kernel performed best, identifying 85.8% of SAC level 1 incidents (precision=0.88, recall=0.83, F-measure=0.86, AUC=0.92). In the patient misidentification dataset, 96.4% of SAC level 1 incidents were detected when SVM with linear, polynomial or radial-basis function kernel was used (precision=0.99, recall=0.94, F-measure=0.96, AUC=0.98). Naïve Bayes showed reasonable performance, detecting 80.8% of SAC level 1 incidents in the multi-type dataset and 89.8% of SAC level 1 patient misidentification incidents. Overall, higher prediction accuracy was attained on the specialized dataset, compared with the multi-type dataset.

Text classification techniques can be applied effectively to automate the detection of extreme-risk events in clinical incident reports.

A cluster-randomized effectiveness trial with PHR and no PHR groups was conducted in two ambulatory clinics. 453 of 1686 (26.4%) patients approached were included in the analyses. A PHR tethered to the patient's electronic medical record (EMR) was the primary intervention and included security measures, patient control of access, limited transmission of EMR data, blood pressure (BP) tracking, and appointment assistance. BP was the main outcome measure. Patient empowerment was assessed using the Patient Activation Measure and Patient Empowerment Scale. Quality of care was assessed using the Clinician and Group Assessment Score (CAHPS) and the Patient Assessment of Chronic Illness Care. Frequency of use of medical services was self-reported.

No impact of the PHR was observed on BP, patient activation, patient perceived quality, or medical utilization in the intention-to-treat analysis. Sub-analysis of intervention patients self-identified as active PHR users (25.7% of those with available information) showed a 5.25-point reduction in diastolic BP. Younger age, self-reported computer skills, and more positive provider communication ratings were associated with frequency of PHR use.

Few patients provided with a PHR actually used the PHR with any frequency. Thus simply providing a PHR may have limited impact on patient BP, empowerment, satisfaction with care, or use of health services without additional education or clinical intervention designed to increase PHR use.

http://ClinicalTrials.gov Identifier: NCT01317537.

Medline, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials, world wide web.

Studies were included if they described an electronic audiovisual tool used as a means to educate patients, care givers, or clinicians about diabetes management and assessed a psychological, behavioral, or clinical outcome.

Study abstraction and evaluation for clinical usefulness, sustainability, and usability were performed by two independent reviewers.

Of 12616 citations and 1541 full-text articles reviewed, 57 studies met inclusion criteria. Forty studies used experimental designs (25 randomized controlled trials, one controlled clinical trial, 14 before–after studies), and 17 used observational designs. Methodological quality and ratings for clinical usefulness and sustainability were variable, and there was a high prevalence of usability errors. Tools showed moderate but inconsistent effects on a variety of psychological and clinical outcomes including HbA1c and weight. Meta-regression of adequately reported studies (12 studies, 2731 participants) demonstrated that, although the interventions studied resulted in positive outcomes, this was not moderated by clinical usefulness nor usability.

This review is limited by the number of accessible tools, exclusion of tools for mobile devices, study quality, and the use of non-validated scales.

Few tools were identified that met our criteria for effectiveness, usefulness, sustainability, and usability. Priority areas include identifying strategies to minimize website attrition and enabling patients and clinicians to make informed decisions about website choice by encouraging reporting of website quality indicators.

To determine whether a clinical alerting system, which uses inference rules to notify providers of undocumented problems, improves problem list documentation.

Inference rules for 17 conditions were constructed and an electronic health record-based intervention was evaluated to improve problem documentation. A cluster randomized trial was conducted of 11 participating clinics affiliated with a large academic medical center, totaling 28 primary care clinical areas, with 14 receiving the intervention and 14 as controls. The intervention was a clinical alert directed to the provider that suggested adding a problem to the electronic problem list based on inference rules. The primary outcome measure was acceptance of the alert. The number of study problems added in each arm as a pre-specified secondary outcome was also assessed. Data were collected during 6-month pre-intervention (11/2009–5/2010) and intervention (5/2010–11/2010) periods.

17 043 alerts were presented, of which 41.1% were accepted. In the intervention arm, providers documented significantly more study problems (adjusted OR=3.4, p<0.001), with an absolute difference of 6277 additional problems. In the intervention group, 70.4% of all study problems were added via the problem list alerts. Significant increases in problem notation were observed for 13 of 17 conditions.

Problem inference alerts significantly increase notation of important patient problems in primary care, which in turn has the potential to facilitate quality improvement.

ClinicalTrials.gov: NCT01105923.

Using data from a 2007 statewide survey of Massachusetts physicians, we conducted multivariate regression analysis to examine relationships between practice characteristics, perceptions of EHR-related errors, perceptions of healthcare quality, and overall physician satisfaction.

30% of physicians agreed that EHRs create new opportunities for error, but only 2% believed their EHR has created more errors than it prevented. With respect to perceptions of quality, there was no significant association between perceptions of EHR-associated errors and perceptions of EHR-associated changes in healthcare quality. Finally, physicians who believed that EHRs created new opportunities for error were less likely be satisfied with their practice situation (adjusted OR 0.49, p=0.001).

Almost one third of physicians perceived that EHRs create new opportunities for error. This perception was associated with lower levels of physician satisfaction.

Survey of a total of 2010 behavioral health providers in a Midwestern state (33% response rate), with questions based on previously reported open-ended beliefs elicitation interviews.

Factor analysis resulted in four groupings: beliefs that HIE would improve care and communication, add cost and time burdens, present access and vulnerability concerns, and impact workflow and control (positively and negatively). A regression model including all four factors parsimoniously predicted attitudes toward HIE. Providers clustered into two groups based on their beliefs: a majority (67%) were positive about the impact of HIE, and the remainder (33%) were negative. There were some professional/demographic differences between the two clusters of providers.

Most behavioral health providers are supportive of HIE; however, their adoption and use of it may continue to lag behind that of medical providers due to perceived cost and time burdens and concerns about access to and vulnerability of information.

Interrupted time series analysis, 2003–2009.

A large US not-for-profit integrated health care organization.

29 hospitals in Northern and Southern California.

An integrated EHR including computerized physician order entry, nursing documentation, risk assessment tools, and documentation tools.

Percentage of patients with completed risk assessments for hospital acquired pressure ulcers (HAPUs) and falls (process measures) and rates of HAPU and falls (outcome measures).

EHR implementation was significantly associated with an increase in documentation rates for HAPU risk (coefficient 2.21, 95% CI 0.67 to 3.75); the increase for fall risk was not statistically significant (0.36; –3.58 to 4.30). EHR implementation was associated with a 13% decrease in HAPU rates (coefficient –0.76, 95% CI –1.37 to –0.16) but no decrease in fall rates (–0.091; –0.29 to 0.11). Irrespective of EHR implementation, HAPU rates decreased significantly over time (–0.16; –0.20 to –0.13), while fall rates did not (0.0052; –0.01 to 0.02). Hospital region was a significant predictor of variation for both HAPU (0.72; 0.30 to 1.14) and fall rates (0.57; 0.41 to 0.72).

The introduction of an integrated EHR was associated with a reduction in the number of HAPUs but not in patient fall rates. Other factors, such as changes over time and hospital region, were also associated with variation in outcomes. The findings suggest that EHR impact on nursing care processes and outcomes is dependent on a number of factors that should be further explored.

The project adapted software from the Informatics for Integrating Biology and the Bedside (i2b2) program to connect three Clinical Translational Research Award sites: University of Washington, Seattle, University of California, Davis, and University of California, San Francisco. The project developed an iterative spiral software development model to support the implementation and coordination of this multisite data resource.

By standardizing technical infrastructures, policies, and semantics, the project enabled federated querying of deidentified clinical datasets stored in separate institutional environments and identified barriers to engaging users for measuring utility.

The authors discuss the iterative development and evaluation phases of the project and highlight the challenges identified and the lessons learned.

The common system architecture and translational processes provide high-level (aggregate) deidentified access to a large patient population (>5 million patients), and represent a novel and extensible resource. Enhancing the network for more focused disease areas will require research-driven partnerships represented across all partner sites.

In 2008, a team of medical residents built a wiki containing institutional knowledge and reference information using Microsoft SharePoint. We tracked visit data for 3 years, and performed an audit of page views and updates in the second year. We evaluated the attitudes of medical residents toward the wiki using a survey.

Users accessed the wiki 23 218, 35 094, and 40 545 times in each of three successive academic years from 2008 to 2011. In the year two audit, 85 users made a total of 1082 updates to 176 pages and of these, 91 were new page creations by 17 users. Forty-eight percent of residents edited a page. All housestaff felt the wiki improved their ability to complete tasks, and 90%, 89%, and 57% reported that the wiki improved their experience, efficiency, and education, respectively, when surveyed in academic year 2009–2010.

A wiki is a useful and popular tool for organizing administrative and educational content for residents. Housestaff felt strongly that the wiki improved their workflow, but a smaller educational impact was observed. Nearly half of the housestaff edited the wiki, suggesting broad buy-in among the residents.

A wiki is a feasible and useful tool for improving information retrieval for house officers.

A non-randomized cross-sectional study was conducted of 48 providers in New York and 30 providers in Massachusetts, all of whom used paper prescriptions, from September 2005 to November 2006. Using standardized methodology, prescriptions and medical records were reviewed to identify errors.

9385 prescriptions were analyzed from 5955 patients. The overall prescribing error rate, excluding illegibility errors, was 36.7 per 100 prescriptions (95% CI 30.7 to 44.0) and did not vary significantly between providers from each state (p=0.39). One or more non-illegibility errors were found in 28% of prescriptions. Rates of illegibility errors were very high (175.0 per 100 prescriptions, 95% CI 169.1 to 181.3). Inappropriate abbreviation and direction errors also occurred frequently (13.4 and 4.2 errors per 100 prescriptions, respectively). Reviewers determined that the vast majority of errors could have been eliminated through the use of e-prescribing with clinical decision support.

Prescribing errors appear to occur at very high rates among community-based primary care providers, especially when compared with studies of academic-affiliated providers that have found nearly threefold lower error rates. Illegibility errors are particularly problematical.

Further characterizing prescribing errors of community-based providers may inform strategies to improve ambulatory medication safety, especially e-prescribing.

http://www.clinicaltrials.gov, NCT00225576.

The electronic medical charts of 1507 patients diagnosed with histologically confirmed primary invasive breast cancer.

The automatic drug treatment classification tool consisted of components for: (1) extraction of drug treatment-relevant information from clinical narratives using natural language processing (clinical Text Analysis and Knowledge Extraction System); (2) extraction of drug treatment data from an electronic prescribing system; (3) merging information to create a patient treatment timeline; and (4) final classification logic.

Agreement between results from the algorithm and from a nurse abstractor is measured for categories: (0) no tamoxifen or aromatase inhibitor (AI) treatment; (1) tamoxifen only; (2) AI only; (3) tamoxifen before AI; (4) AI before tamoxifen; (5) multiple AIs and tamoxifen cycles in no specific order; and (6) no specific treatment dates. Specificity (all categories): 96.14%–100%; sensitivity (categories (0)–(4)): 90.27%–99.83%; sensitivity (categories (5)–(6)): 0–23.53%; positive predictive values: 80%–97.38%; negative predictive values: 96.91%–99.93%.

Our approach illustrates a secondary use of the electronic medical record. The main challenge is event temporality.

We present an algorithm for automated treatment classification within an electronic medical record to combine information extracted through natural language processing with that extracted from structured databases. The algorithm has high specificity for all categories, high sensitivity for five categories, and low sensitivity for two categories.

Prescription-filling data for 2 million subjects derived from a payor's insurance claims were used to evaluate adherence to three chronic drugs over 1 year. We relied on patterns of prescription fills, including the length of gaps in medication possession, to measure adherence among subjects and to build models for predicting poor long-term adherence.

All prescription fills for a specific drug were sequenced chronologically into drug eras. 61.3% to 66.5% of the prescription patterns contained medication gaps >30 days during the first year of drug use. These interrupted drug eras include long-term discontinuations, where the subject never again filled a prescription for any drug in that category in the dataset, which represent 23.7% to 29.1% of all drug eras. Among the prescription-filling patterns without large medication gaps, 0.8% to 1.3% exhibited long-term poor adherence. Our models identified these subjects as early as 60 days after the first prescription fill, with an area under the curve (AUC) of 0.81. Model performance improved as the predictions were made at later time-points, with AUC values increasing to 0.93 at the 120-day time-point.

Dispensed medication histories (widely available in real time) are useful for alerting providers about poorly adherent patients and those who will be non-adherent several months later. Efforts to use these data in point of care and decision support facilitating patient are warranted.

Cancer registry data and artificial data with increased numbers of missing values in a relevant variable are used for empirical comparisons. As a classification method, classification and regression trees were used. On the resulting binary comparison patterns, the following strategies for dealing with missingness are considered: imputation with unique values, sample-based imputation, reduced-model classification and complete-case induction. These approaches are evaluated according to the number of training data needed for induction and the F-scores achieved.

The evaluations reveal that unique value imputation leads to the best results. Imputation with zero is preferred to imputation with 0.5, although the latter shows the highest median F-scores. Imputation with zero needs considerably less training data, it shows only slightly worse results and simplifies the computation by maintaining the binary structure of the data.

The results support the ad-hoc solution for missing values ‘replace NA by the value of inequality’. This conclusion is based on a limited amount of data and on a specific deduplication method. Nevertheless, the authors are confident that their results should be confirmed by other empirical analyses and applications.

Duke Medicine has deployed a hybrid solution, The Duke Integrated Subject Cohort and Enrollment Research Network (DISCERN), that combines both retrospective warehouse data and clinical events contained in prospective Health Level 7 (HL7) messages to immediately alert study personnel of potential recruits as they become eligible.

DISCERN analyzes more than 500 000 messages daily in service of 12 projects. Users may receive results via email, text pages, or on-demand reports. Preliminary results suggest DISCERN's unique ability to reason over both retrospective and real-time data increases study enrollment rates while reducing the time required to complete recruitment-related tasks. The authors have introduced a preconfigured DISCERN function as a self-service feature for users.

The DISCERN framework is adoptable primarily by organizations using both HL7 message streams and a data warehouse. More efficient recruitment may exacerbate competition for research subjects, and investigators uncomfortable with new technology may find themselves at a competitive disadvantage in recruitment.

DISCERN's hybrid framework for identifying real-time clinical events housed in HL7 messages complements the traditional approach of using retrospective warehoused data. DISCERN is helpful in instances when the required clinical data may not be loaded into the warehouse and thus must be captured contemporaneously during patient care. Use of an open-source tool supports generalizability to other institutions at minimal cost.

Records from a large healthcare system's enterprise data warehouse (EDW) were linked to a statewide population database, and a master subject index was created. The authors evaluate the linkage, along with the impact of missing information in EDW records and the coverage of the population database. The makeup of the EDW and population database provides a subset of cancer records that exist in both resources, which allows a cancer-specific evaluation of the linkage.

About 3.4 million records (60.8%) in the EDW were linked to the population database with a minimum accuracy of 96.3%. It was estimated that approximately 24.8% of target records were absent from the population database, which enabled the effect of the amount and type of information missing from a record on the linkage to be estimated. However, 99% of the records from the oncology data mart linked; they had fewer missing fields and this correlated positively with the number of patient visits.

A general-purpose research infrastructure was created which allows disease-specific cohorts to be identified. The usefulness of creating an index between institutions is that it allows each institution to maintain control and confidentiality of their own information.

We present an optical character recognition processing pipeline, which leverages the capabilities of existing third-party optical character recognition engines, and provides the flexibility offered by a modular custom-developed system. The system was configured and run on a selected set of form fields extracted from a corpus of handwritten ophthalmology forms.

The processing pipeline allowed multiple configurations to be run, with the optimal configuration consisting of the Nuance and LEADTOOLS engines running in parallel with a positive predictive value of 94.6% and a sensitivity of 13.5%.

While limitations exist, preliminary experience from this project yielded insights on the generalizability and applicability of integrating multiple, inexpensive general-purpose third-party optical character recognition engines in a modular pipeline.