Standardized interpretation of paediatric chest radiographs for the diagnosis of pneumonia in epidemiological studies

Thomas Cherian, E. Kim Mulholland, John B. Carlin, Harald Ostensen, Ruhul Amin, Margaret De Campo, David Greenberg, Rosanna Lagos, Marilla Lucero, Shabir A. Madhi, Katherine L. O'Brien, Stephen K Obaro, Mark C. Steinhoff

Research output: Contribution to journalArticle

327 Citations (Scopus)

Abstract

Background: Although radiological pneumonia is used as an outcome measure in epidemiological studies, there is considerable variability in the interpretation of chest radiographs. A standardized method for identifying radiological pneumonia would facilitate comparison of the results of vaccine trials and epidemiological studies of pneumonia. Methods: A WHO working group developed definitions for radiological pneumonia. Inter-observer variability in categorizing a set of 222 chest radiographic images was measured by comparing the readings made by 20 radiologists and clinicians with a reference reading. Intra-observer variability was measured by comparing the initial readings of a randomly chosen subset of 100 radiographs with repeat readings made 8-30 days later. Findings: Of the 222 images, 208 were considered interpretable. The reference reading categorized 43% of these images as showing alveolar consolidation or pleural effusion (primary end-point pneumonia); the proportion thus categorized by each of the 20 readers ranged from 8% to 61%. Using the reference reading as the gold standard, 14 of the 20 readers had sensitivity and specificity of ≥ 0.70 in identifying primary end-point pneumonia; 13 out of 20 readers had a kappa index of > 0.6 compared with the reference reading. For the 92 radiographs deemed to be interpretable among the 100 images used for intra-observer variability, 19 out of 20 readers had a kappa index of > 0.6. Conclusion: Using standardized definitions and training, it is possible to achieve agreement in identifying radiological pneumonia, thus facilitating the comparison of results of epidemiological studies that use radiological pneumonia as an outcome.

Original languageEnglish (US)
Pages (from-to)353-359
Number of pages7
JournalBulletin of the World Health Organization
Volume83
Issue number5
StatePublished - May 1 2005

Fingerprint

Epidemiologic Studies
Pneumonia
Thorax
Reading
Pediatrics
Observer Variation
Pleural Effusion
Vaccines
Outcome Assessment (Health Care)
Sensitivity and Specificity

Keywords

  • Child
  • Observer variation
  • Pneumonia/radiography
  • Radiography, Thoracic/standards
  • Reference standards
  • Sensitivity and specificity

ASJC Scopus subject areas

  • Public Health, Environmental and Occupational Health

Cite this

Cherian, T., Mulholland, E. K., Carlin, J. B., Ostensen, H., Amin, R., De Campo, M., ... Steinhoff, M. C. (2005). Standardized interpretation of paediatric chest radiographs for the diagnosis of pneumonia in epidemiological studies. Bulletin of the World Health Organization, 83(5), 353-359.

Standardized interpretation of paediatric chest radiographs for the diagnosis of pneumonia in epidemiological studies. / Cherian, Thomas; Mulholland, E. Kim; Carlin, John B.; Ostensen, Harald; Amin, Ruhul; De Campo, Margaret; Greenberg, David; Lagos, Rosanna; Lucero, Marilla; Madhi, Shabir A.; O'Brien, Katherine L.; Obaro, Stephen K; Steinhoff, Mark C.

In: Bulletin of the World Health Organization, Vol. 83, No. 5, 01.05.2005, p. 353-359.

Research output: Contribution to journalArticle

Cherian, T, Mulholland, EK, Carlin, JB, Ostensen, H, Amin, R, De Campo, M, Greenberg, D, Lagos, R, Lucero, M, Madhi, SA, O'Brien, KL, Obaro, SK & Steinhoff, MC 2005, 'Standardized interpretation of paediatric chest radiographs for the diagnosis of pneumonia in epidemiological studies', Bulletin of the World Health Organization, vol. 83, no. 5, pp. 353-359.
Cherian, Thomas ; Mulholland, E. Kim ; Carlin, John B. ; Ostensen, Harald ; Amin, Ruhul ; De Campo, Margaret ; Greenberg, David ; Lagos, Rosanna ; Lucero, Marilla ; Madhi, Shabir A. ; O'Brien, Katherine L. ; Obaro, Stephen K ; Steinhoff, Mark C. / Standardized interpretation of paediatric chest radiographs for the diagnosis of pneumonia in epidemiological studies. In: Bulletin of the World Health Organization. 2005 ; Vol. 83, No. 5. pp. 353-359.
@article{353b374fa3624ea791ea2427ccc3a91b,
title = "Standardized interpretation of paediatric chest radiographs for the diagnosis of pneumonia in epidemiological studies",
abstract = "Background: Although radiological pneumonia is used as an outcome measure in epidemiological studies, there is considerable variability in the interpretation of chest radiographs. A standardized method for identifying radiological pneumonia would facilitate comparison of the results of vaccine trials and epidemiological studies of pneumonia. Methods: A WHO working group developed definitions for radiological pneumonia. Inter-observer variability in categorizing a set of 222 chest radiographic images was measured by comparing the readings made by 20 radiologists and clinicians with a reference reading. Intra-observer variability was measured by comparing the initial readings of a randomly chosen subset of 100 radiographs with repeat readings made 8-30 days later. Findings: Of the 222 images, 208 were considered interpretable. The reference reading categorized 43{\%} of these images as showing alveolar consolidation or pleural effusion (primary end-point pneumonia); the proportion thus categorized by each of the 20 readers ranged from 8{\%} to 61{\%}. Using the reference reading as the gold standard, 14 of the 20 readers had sensitivity and specificity of ≥ 0.70 in identifying primary end-point pneumonia; 13 out of 20 readers had a kappa index of > 0.6 compared with the reference reading. For the 92 radiographs deemed to be interpretable among the 100 images used for intra-observer variability, 19 out of 20 readers had a kappa index of > 0.6. Conclusion: Using standardized definitions and training, it is possible to achieve agreement in identifying radiological pneumonia, thus facilitating the comparison of results of epidemiological studies that use radiological pneumonia as an outcome.",
keywords = "Child, Observer variation, Pneumonia/radiography, Radiography, Thoracic/standards, Reference standards, Sensitivity and specificity",
author = "Thomas Cherian and Mulholland, {E. Kim} and Carlin, {John B.} and Harald Ostensen and Ruhul Amin and {De Campo}, Margaret and David Greenberg and Rosanna Lagos and Marilla Lucero and Madhi, {Shabir A.} and O'Brien, {Katherine L.} and Obaro, {Stephen K} and Steinhoff, {Mark C.}",
year = "2005",
month = "5",
day = "1",
language = "English (US)",
volume = "83",
pages = "353--359",
journal = "Bulletin of the World Health Organization",
issn = "0042-9686",
publisher = "World Health Organization",
number = "5",

}

TY - JOUR

T1 - Standardized interpretation of paediatric chest radiographs for the diagnosis of pneumonia in epidemiological studies

AU - Cherian, Thomas

AU - Mulholland, E. Kim

AU - Carlin, John B.

AU - Ostensen, Harald

AU - Amin, Ruhul

AU - De Campo, Margaret

AU - Greenberg, David

AU - Lagos, Rosanna

AU - Lucero, Marilla

AU - Madhi, Shabir A.

AU - O'Brien, Katherine L.

AU - Obaro, Stephen K

AU - Steinhoff, Mark C.

PY - 2005/5/1

Y1 - 2005/5/1

N2 - Background: Although radiological pneumonia is used as an outcome measure in epidemiological studies, there is considerable variability in the interpretation of chest radiographs. A standardized method for identifying radiological pneumonia would facilitate comparison of the results of vaccine trials and epidemiological studies of pneumonia. Methods: A WHO working group developed definitions for radiological pneumonia. Inter-observer variability in categorizing a set of 222 chest radiographic images was measured by comparing the readings made by 20 radiologists and clinicians with a reference reading. Intra-observer variability was measured by comparing the initial readings of a randomly chosen subset of 100 radiographs with repeat readings made 8-30 days later. Findings: Of the 222 images, 208 were considered interpretable. The reference reading categorized 43% of these images as showing alveolar consolidation or pleural effusion (primary end-point pneumonia); the proportion thus categorized by each of the 20 readers ranged from 8% to 61%. Using the reference reading as the gold standard, 14 of the 20 readers had sensitivity and specificity of ≥ 0.70 in identifying primary end-point pneumonia; 13 out of 20 readers had a kappa index of > 0.6 compared with the reference reading. For the 92 radiographs deemed to be interpretable among the 100 images used for intra-observer variability, 19 out of 20 readers had a kappa index of > 0.6. Conclusion: Using standardized definitions and training, it is possible to achieve agreement in identifying radiological pneumonia, thus facilitating the comparison of results of epidemiological studies that use radiological pneumonia as an outcome.

AB - Background: Although radiological pneumonia is used as an outcome measure in epidemiological studies, there is considerable variability in the interpretation of chest radiographs. A standardized method for identifying radiological pneumonia would facilitate comparison of the results of vaccine trials and epidemiological studies of pneumonia. Methods: A WHO working group developed definitions for radiological pneumonia. Inter-observer variability in categorizing a set of 222 chest radiographic images was measured by comparing the readings made by 20 radiologists and clinicians with a reference reading. Intra-observer variability was measured by comparing the initial readings of a randomly chosen subset of 100 radiographs with repeat readings made 8-30 days later. Findings: Of the 222 images, 208 were considered interpretable. The reference reading categorized 43% of these images as showing alveolar consolidation or pleural effusion (primary end-point pneumonia); the proportion thus categorized by each of the 20 readers ranged from 8% to 61%. Using the reference reading as the gold standard, 14 of the 20 readers had sensitivity and specificity of ≥ 0.70 in identifying primary end-point pneumonia; 13 out of 20 readers had a kappa index of > 0.6 compared with the reference reading. For the 92 radiographs deemed to be interpretable among the 100 images used for intra-observer variability, 19 out of 20 readers had a kappa index of > 0.6. Conclusion: Using standardized definitions and training, it is possible to achieve agreement in identifying radiological pneumonia, thus facilitating the comparison of results of epidemiological studies that use radiological pneumonia as an outcome.

KW - Child

KW - Observer variation

KW - Pneumonia/radiography

KW - Radiography, Thoracic/standards

KW - Reference standards

KW - Sensitivity and specificity

UR - http://www.scopus.com/inward/record.url?scp=18744368940&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=18744368940&partnerID=8YFLogxK

M3 - Article

VL - 83

SP - 353

EP - 359

JO - Bulletin of the World Health Organization

JF - Bulletin of the World Health Organization

SN - 0042-9686

IS - 5

ER -