rss
J Am Med Inform Assoc 6:143-150 doi:10.1136/jamia.1999.0060143
  • Original Investigation

A Reliability Study for Evaluating Information Extraction from Radiology Reports

Table 3

Reliability and Number of Raters

Site Condition Generalizability Coefficient per Rater, ρ1 (95% CI) No. of Raters for ρ = 0.7 (95% CI) No. of Raters for ρ = 0.95 (95% CI)
NYP Congestive heart failure 0.85 (0.83-0.88) 1 (1-1) 4 (1-8)
Pneumonia 0.70 (0.66-0.75) 1 (1-2) 8 (4-12)
Pleural effusion 0.67 (0.61-0.72) 2 (1-2) 10 (6-14)
Pneumothorax 0.87 (0.85-0.90) 1 (1-1) 3 (1-7)
COPD 0.71 (0.67-0.76) 1 (1-2) 8 (4-12)
Neoplasm 0.74 (0.70-0.78) 1 (1-2) 7 (3-11)
BW Congestive heart failure 0.80 (0.76-0.83) 1 (1-2) 5 (1-9)
Pneumonia 0.69 (0.64-0.74) 2 (1-2) 9 (5-13)
Pleural effusion 0.87 (0.85-0.90) 1 (1-1) 3 (1-7)
Pneumothorax 0.87 (0.85-0.90) 1 (1-1) 3 (1-7)
Atelectasis 0.77 (0.72-0.81) 1 (1-2) 6 (2-10)
Rib fracture 0.86 (0.83-0.88) 1 (1-1) 4 (1-8)
Central line presence 0.97 (0.96-0.97) 1 (1-1) 1 (1-5)
Both Average condition 0.80 (0.79-0.81) 1 (1-1) 6 (5-7)

This Article

Access policy for JAMIA

All content published in JAMIA is deposited with PubMed Central by the publisher with a 12 month embargo. Authors/funders may pay an Unlocked fee of $2,000 to make the article free on the JAMIA website and PMC immediately on publication.

All content older than 12 months is freely available on this website.

AMIA members can log in with their JAMIA user name (email address) and password or via the AMIA website.