Development and initial validation of the Risk Analysis Index for measuring frailty in surgical populations

Daniel E. Hall, Shipra Arya, Kendra K Schmid, Casey Blaser, Mark Alan Carlson, Travis L. Bailey, Georgia Purviance, Tammy Bockman, Thomas G. Lynch, Jason M Johanning

Research output: Contribution to journalArticle

34 Citations (Scopus)

Abstract

Importance: Growing consensus suggests that frailty-associated risks should inform shared surgical decision making. However, it is not clear how best to screen for frailty in preoperative surgical populations. Objective: To develop and validate the Risk Analysis Index (RAI), a 14-item instrument used to measure surgical frailty. It can be calculated prospectively (RAI-C), using a clinical questionnaire, or retrospectively (RAI-A), using variables from the surgical quality improvement databases (Veterans Affairs or American College of Surgeons National Surgical Quality Improvement Projects). Design, Setting, and Participants: Single-site, prospective cohort from July 2011 to September 2015 at the Veterans Affairs Nebraska-Western Iowa Heath Care System, a Level 1b Veterans Affairs Medical Center. The study included all patients presenting to the medical center for elective surgery. Exposures: We assessed the RAI-C for all patients scheduled for surgery, linking these scores to administrative and quality improvement data to calculate the RAI-A and the modified Frailty Index. Main Outcomes and Measures: Receiver operator characteristics and C statistics for each measure predicting postoperative mortality and morbidity. Results: Of the participants, the mean (SD) age was 60.7 (13.9) years and 249 participants (3.6%) were women. We assessed the RAI-C 10 698 times, from which we linked 6856 unique patients to mortality data. The C statistic predicting 180-day mortality for the RAI-C was 0.772. Of these 6856 unique patients, we linked 2785 to local Veterans Affairs Surgeons National Surgical Quality Improvement Projects data and calculated the C statistic for both the RAI-A (0.823) and RAI-C (0.824), along with the correlation between the 2 scores (r = 0.478; P < .001). Of these 2785 patients, there were sufficient data to calculate the modified Frailty Index for 1021, in which the C statistics were 0.865 (RAI-A), 0.797 (RAI-C), and 0.811 (modified Frailty Index). The correlation between the RAI-A and RAI-C was 0.547, and the correlations of the modified Frailty Index to the RAI-A and RAI-C were 0.301 and 0.269, respectively (all P < .001). A cutoff of RAI-C of at least 21 classified 18.3%patients as "frail" with a sensitivity of 0.50 and specificity of 0.82, whereas the RAI-A was less sensitive (0.25) and more specific (0.97), classifying only 3.7%as "frail." Conclusions and Relevance: The RAI-C and RAI-A represent effective tools for measuring frailty in surgical populations with predictive ability on par with other frailty tools. Moderate correlation between the measures suggests convergent validity. The RAI-C offers the advantage of prospective, preoperative assessment that is proved feasible for large-scale screening in clinical practice. However, further efforts should be directed at determining the optimal components of preoperative frailty assessment.

Original languageEnglish (US)
Pages (from-to)175-182
Number of pages8
JournalJAMA Surgery
Volume152
Issue number2
DOIs
StatePublished - Feb 1 2017

Fingerprint

Population
Veterans
Quality Improvement
Mortality
Consensus
Decision Making
Outcome Assessment (Health Care)
Databases
Morbidity
Sensitivity and Specificity

ASJC Scopus subject areas

  • Surgery

Cite this

Development and initial validation of the Risk Analysis Index for measuring frailty in surgical populations. / Hall, Daniel E.; Arya, Shipra; Schmid, Kendra K; Blaser, Casey; Carlson, Mark Alan; Bailey, Travis L.; Purviance, Georgia; Bockman, Tammy; Lynch, Thomas G.; Johanning, Jason M.

In: JAMA Surgery, Vol. 152, No. 2, 01.02.2017, p. 175-182.

Research output: Contribution to journalArticle

Hall, Daniel E. ; Arya, Shipra ; Schmid, Kendra K ; Blaser, Casey ; Carlson, Mark Alan ; Bailey, Travis L. ; Purviance, Georgia ; Bockman, Tammy ; Lynch, Thomas G. ; Johanning, Jason M. / Development and initial validation of the Risk Analysis Index for measuring frailty in surgical populations. In: JAMA Surgery. 2017 ; Vol. 152, No. 2. pp. 175-182.
@article{c39b81dff1ae4283987a5388f60270a8,
title = "Development and initial validation of the Risk Analysis Index for measuring frailty in surgical populations",
abstract = "Importance: Growing consensus suggests that frailty-associated risks should inform shared surgical decision making. However, it is not clear how best to screen for frailty in preoperative surgical populations. Objective: To develop and validate the Risk Analysis Index (RAI), a 14-item instrument used to measure surgical frailty. It can be calculated prospectively (RAI-C), using a clinical questionnaire, or retrospectively (RAI-A), using variables from the surgical quality improvement databases (Veterans Affairs or American College of Surgeons National Surgical Quality Improvement Projects). Design, Setting, and Participants: Single-site, prospective cohort from July 2011 to September 2015 at the Veterans Affairs Nebraska-Western Iowa Heath Care System, a Level 1b Veterans Affairs Medical Center. The study included all patients presenting to the medical center for elective surgery. Exposures: We assessed the RAI-C for all patients scheduled for surgery, linking these scores to administrative and quality improvement data to calculate the RAI-A and the modified Frailty Index. Main Outcomes and Measures: Receiver operator characteristics and C statistics for each measure predicting postoperative mortality and morbidity. Results: Of the participants, the mean (SD) age was 60.7 (13.9) years and 249 participants (3.6{\%}) were women. We assessed the RAI-C 10 698 times, from which we linked 6856 unique patients to mortality data. The C statistic predicting 180-day mortality for the RAI-C was 0.772. Of these 6856 unique patients, we linked 2785 to local Veterans Affairs Surgeons National Surgical Quality Improvement Projects data and calculated the C statistic for both the RAI-A (0.823) and RAI-C (0.824), along with the correlation between the 2 scores (r = 0.478; P < .001). Of these 2785 patients, there were sufficient data to calculate the modified Frailty Index for 1021, in which the C statistics were 0.865 (RAI-A), 0.797 (RAI-C), and 0.811 (modified Frailty Index). The correlation between the RAI-A and RAI-C was 0.547, and the correlations of the modified Frailty Index to the RAI-A and RAI-C were 0.301 and 0.269, respectively (all P < .001). A cutoff of RAI-C of at least 21 classified 18.3{\%}patients as {"}frail{"} with a sensitivity of 0.50 and specificity of 0.82, whereas the RAI-A was less sensitive (0.25) and more specific (0.97), classifying only 3.7{\%}as {"}frail.{"} Conclusions and Relevance: The RAI-C and RAI-A represent effective tools for measuring frailty in surgical populations with predictive ability on par with other frailty tools. Moderate correlation between the measures suggests convergent validity. The RAI-C offers the advantage of prospective, preoperative assessment that is proved feasible for large-scale screening in clinical practice. However, further efforts should be directed at determining the optimal components of preoperative frailty assessment.",
author = "Hall, {Daniel E.} and Shipra Arya and Schmid, {Kendra K} and Casey Blaser and Carlson, {Mark Alan} and Bailey, {Travis L.} and Georgia Purviance and Tammy Bockman and Lynch, {Thomas G.} and Johanning, {Jason M}",
year = "2017",
month = "2",
day = "1",
doi = "10.1001/jamasurg.2016.4202",
language = "English (US)",
volume = "152",
pages = "175--182",
journal = "JAMA Surgery",
issn = "2168-6254",
publisher = "American Medical Association",
number = "2",

}

TY - JOUR

T1 - Development and initial validation of the Risk Analysis Index for measuring frailty in surgical populations

AU - Hall, Daniel E.

AU - Arya, Shipra

AU - Schmid, Kendra K

AU - Blaser, Casey

AU - Carlson, Mark Alan

AU - Bailey, Travis L.

AU - Purviance, Georgia

AU - Bockman, Tammy

AU - Lynch, Thomas G.

AU - Johanning, Jason M

PY - 2017/2/1

Y1 - 2017/2/1

N2 - Importance: Growing consensus suggests that frailty-associated risks should inform shared surgical decision making. However, it is not clear how best to screen for frailty in preoperative surgical populations. Objective: To develop and validate the Risk Analysis Index (RAI), a 14-item instrument used to measure surgical frailty. It can be calculated prospectively (RAI-C), using a clinical questionnaire, or retrospectively (RAI-A), using variables from the surgical quality improvement databases (Veterans Affairs or American College of Surgeons National Surgical Quality Improvement Projects). Design, Setting, and Participants: Single-site, prospective cohort from July 2011 to September 2015 at the Veterans Affairs Nebraska-Western Iowa Heath Care System, a Level 1b Veterans Affairs Medical Center. The study included all patients presenting to the medical center for elective surgery. Exposures: We assessed the RAI-C for all patients scheduled for surgery, linking these scores to administrative and quality improvement data to calculate the RAI-A and the modified Frailty Index. Main Outcomes and Measures: Receiver operator characteristics and C statistics for each measure predicting postoperative mortality and morbidity. Results: Of the participants, the mean (SD) age was 60.7 (13.9) years and 249 participants (3.6%) were women. We assessed the RAI-C 10 698 times, from which we linked 6856 unique patients to mortality data. The C statistic predicting 180-day mortality for the RAI-C was 0.772. Of these 6856 unique patients, we linked 2785 to local Veterans Affairs Surgeons National Surgical Quality Improvement Projects data and calculated the C statistic for both the RAI-A (0.823) and RAI-C (0.824), along with the correlation between the 2 scores (r = 0.478; P < .001). Of these 2785 patients, there were sufficient data to calculate the modified Frailty Index for 1021, in which the C statistics were 0.865 (RAI-A), 0.797 (RAI-C), and 0.811 (modified Frailty Index). The correlation between the RAI-A and RAI-C was 0.547, and the correlations of the modified Frailty Index to the RAI-A and RAI-C were 0.301 and 0.269, respectively (all P < .001). A cutoff of RAI-C of at least 21 classified 18.3%patients as "frail" with a sensitivity of 0.50 and specificity of 0.82, whereas the RAI-A was less sensitive (0.25) and more specific (0.97), classifying only 3.7%as "frail." Conclusions and Relevance: The RAI-C and RAI-A represent effective tools for measuring frailty in surgical populations with predictive ability on par with other frailty tools. Moderate correlation between the measures suggests convergent validity. The RAI-C offers the advantage of prospective, preoperative assessment that is proved feasible for large-scale screening in clinical practice. However, further efforts should be directed at determining the optimal components of preoperative frailty assessment.

AB - Importance: Growing consensus suggests that frailty-associated risks should inform shared surgical decision making. However, it is not clear how best to screen for frailty in preoperative surgical populations. Objective: To develop and validate the Risk Analysis Index (RAI), a 14-item instrument used to measure surgical frailty. It can be calculated prospectively (RAI-C), using a clinical questionnaire, or retrospectively (RAI-A), using variables from the surgical quality improvement databases (Veterans Affairs or American College of Surgeons National Surgical Quality Improvement Projects). Design, Setting, and Participants: Single-site, prospective cohort from July 2011 to September 2015 at the Veterans Affairs Nebraska-Western Iowa Heath Care System, a Level 1b Veterans Affairs Medical Center. The study included all patients presenting to the medical center for elective surgery. Exposures: We assessed the RAI-C for all patients scheduled for surgery, linking these scores to administrative and quality improvement data to calculate the RAI-A and the modified Frailty Index. Main Outcomes and Measures: Receiver operator characteristics and C statistics for each measure predicting postoperative mortality and morbidity. Results: Of the participants, the mean (SD) age was 60.7 (13.9) years and 249 participants (3.6%) were women. We assessed the RAI-C 10 698 times, from which we linked 6856 unique patients to mortality data. The C statistic predicting 180-day mortality for the RAI-C was 0.772. Of these 6856 unique patients, we linked 2785 to local Veterans Affairs Surgeons National Surgical Quality Improvement Projects data and calculated the C statistic for both the RAI-A (0.823) and RAI-C (0.824), along with the correlation between the 2 scores (r = 0.478; P < .001). Of these 2785 patients, there were sufficient data to calculate the modified Frailty Index for 1021, in which the C statistics were 0.865 (RAI-A), 0.797 (RAI-C), and 0.811 (modified Frailty Index). The correlation between the RAI-A and RAI-C was 0.547, and the correlations of the modified Frailty Index to the RAI-A and RAI-C were 0.301 and 0.269, respectively (all P < .001). A cutoff of RAI-C of at least 21 classified 18.3%patients as "frail" with a sensitivity of 0.50 and specificity of 0.82, whereas the RAI-A was less sensitive (0.25) and more specific (0.97), classifying only 3.7%as "frail." Conclusions and Relevance: The RAI-C and RAI-A represent effective tools for measuring frailty in surgical populations with predictive ability on par with other frailty tools. Moderate correlation between the measures suggests convergent validity. The RAI-C offers the advantage of prospective, preoperative assessment that is proved feasible for large-scale screening in clinical practice. However, further efforts should be directed at determining the optimal components of preoperative frailty assessment.

UR - http://www.scopus.com/inward/record.url?scp=85014774256&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85014774256&partnerID=8YFLogxK

U2 - 10.1001/jamasurg.2016.4202

DO - 10.1001/jamasurg.2016.4202

M3 - Article

VL - 152

SP - 175

EP - 182

JO - JAMA Surgery

JF - JAMA Surgery

SN - 2168-6254

IS - 2

ER -