J Am Med Inform Assoc 16:660-669 doi:10.1197/jamia.M3154
  • Original Investigation
  • Research Paper

Usability Testing Finds Problems for Novice Users of Pediatric Portals

  1. Maria T Britto, MD, MPHa,b,
  2. Holly B Jimison, PhDc,
  3. Jennifer Knopf Munafo, MAa,
  4. Jennifer Wissman, MSd,
  5. Michelle L Rogers, PhDe,
  6. William Hersh, MDc
  1. aCenter for Innovation in Chronic Disease Care, Division of Adolescent Medicine, Cincinnati Children's Hospital Medical Center, Cincinnati, OH
  2. bUniversity of Cincinnati College of Medicine, Cincinnati, OH
  3. cDepartment of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, OR
  4. dVeterans Affairs Getting at Patient Safety (GAPS) Center, Veterans Administration Medical Center, Cincinnati, OH
  5. eDrexel University, Philadelphia, PA
  1. Correspondence: Maria T. Britto, MD, MPH, Center for Innovation in Chronic Disease Care, Division of Adolescent Medicine, Cincinnati Children's Hospital Medical Center, University of Cincinnati College of Medicine, Cincinnati, Ohio, 3333 Burnet Avenue – MLC 7027, Cincinnati, Ohio 45229 (Email: maria.britto{at}
  • Received 27 January 2009
  • Accepted 5 June 2009


Objective Patient portals may improve pediatric chronic disease outcomes, but few have been rigorously evaluated for usability by parents. Using scenario-based testing with think-aloud protocols, we evaluated the usability of portals for parents of children with cystic fibrosis, diabetes or arthritis.

Design Sixteen parents used a prototype and test data to complete 14 tasks followed by a validated satisfaction questionnaire. Three iterations of the prototype were used.

Measurements During the usability testing, we measured the time it took participants to complete or give up on each task. Sessions were videotaped and content-analyzed for common themes. Following testing, participants completed the Computer Usability Satisfaction Questionnaire which measured their opinions on the efficiency of the system, its ease of use, and the likability of the system interface. A 7-point Likert scale was used, with seven indicating the highest possible satisfaction.

Results Mean task completion times ranged from 73 (± 61) seconds to locate a document to 431 (± 286) seconds to graph laboratory results. Tasks such as graphing, location of data, requesting access, and data interpretation were challenging. Satisfaction was greatest for interface pleasantness (5.9 ± 0.7) and likeability (5.8 ± 0.6) and lowest for error messages (2.3 ± 1.2) and clarity of information (4.2 ± 1.4). Overall mean satisfaction scores improved between iteration one and three.

Conclusions Despite parental involvement and prior heuristic testing, scenario-based testing demonstrated difficulties in navigation, medical language complexity, error recovery, and provider-based organizational schema. While such usability testing can be expensive, the current study demonstrates that it can assist in making healthcare system interfaces for laypersons more user-friendly and potentially more functional for patients and their families.


  • This research was funded in part by the National Library of Medicine grant #5F38LM008876, Evaluation of Pediatric Patient Portals, to the first author. The findings were presented in part at the 2007 National Library of Medicine Fellows Conference, June 27, 2007, Stanford University, San Francisco, CA.

Free Sample

This recent issue is free to all users to allow everyone the opportunity to see the full scope and typical content of JAMIA.
View free sample issue >>

Access policy for JAMIA

All content published in JAMIA is deposited with PubMed Central by the publisher with a 12 month embargo. Authors/funders may pay an Open Access fee of $2,000 to make the article free on the JAMIA website and PMC immediately on publication.

All content older than 12 months is freely available on this website.

AMIA members can log in with their JAMIA user name (email address) and password or via the AMIA website.

Navigate This Article