Can the Student Outperform the Master? A Plan Comparison Between Pinnacle Auto-Planning and Eclipse knowledge-Based RapidPlan Following a Prostate-Bed Plan Competition

April Smith, Andrew Granatowicz, Cole Stoltenberg, Shuo Wang, Xiaoying Liang, Charles A. Enke, Andrew O. Wahl, Sumin Zhou, Dandan Zheng

Research output: Contribution to journalArticle

Abstract

PURPOSE: Pinnacle Auto-Planning and Eclipse RapidPlan are 2 major commercial automated planning engines that are fundamentally different: Auto-Planning mimics real planners in the iterative optimization, while RapidPlan generates static dose objectives from estimations predicted based on a prior knowledge base. This study objectively compared their performances on intensity-modulated radiotherapy planning for prostate fossa and lymphatics adopting the plan quality metric used in the 2011 American Association of Medical Dosimetrists Plan Challenge. METHODS: All plans used an identical intensity-modulated radiotherapy beam setup and a simultaneous integrated boost prescription (68 Gy/56 Gy to prostate fossa/lymphatics). Auto-Planning was used to retrospectively plan on 20 patients, which were subsequently employed as the library to build an RapidPlan model. To compare the 2 engines' performances, a test set including 10 patients and the Plan Challenge patient was planned by both Auto-Planning (master) and RapidPlan (student) without manual intervention except for a common dose normalization and evaluated using the plan quality metric that included 14 quantitative submetrics ranging over target coverage, spillage, and organ at risk doses. Plan quality metric scores were compared between the Auto-Planning and RapidPlan plans using the Mann-Whitney U test. RESULTS: There was no significant difference between the overall performance of the 2 engines on the 11 test cases ( P = .509). Among the 14 submetrics, Auto-Planning and RapidPlan showed no significant difference on most submetrics except for 2. On the Plan Challenge case, Auto-Planning scored 129.9 and RapidPlan scored 130.3 out of 150, as compared with the average score of 116.9 ± 16.4 (range: 58.2-142.5) among the 125 Plan Challenge participants. CONCLUSION: Using an innovative study design, an objective comparison has been conducted between 2 major commercial automated inverse planning engines. The 2 engines performed comparably with each other and both yielded plans at par with average human planners. Using a constant-performing planner (Auto-Planning) to train and to compare, RapidPlan was found to yield plans no better than but as good as its library plans.

Original languageEnglish (US)
Pages (from-to)1533033819851763
JournalTechnology in cancer research & treatment
Volume18
DOIs
StatePublished - Jan 1 2019

Fingerprint

Prostate
Intensity-Modulated Radiotherapy
Students
Libraries
Organs at Risk
Knowledge Bases
American Medical Association
Nonparametric Statistics
Prescriptions
P 509

Keywords

  • Auto-Plan
  • KBP
  • RapidPlan
  • automation
  • treatment planning

ASJC Scopus subject areas

  • Oncology
  • Cancer Research

Cite this

Can the Student Outperform the Master? A Plan Comparison Between Pinnacle Auto-Planning and Eclipse knowledge-Based RapidPlan Following a Prostate-Bed Plan Competition. / Smith, April; Granatowicz, Andrew; Stoltenberg, Cole; Wang, Shuo; Liang, Xiaoying; Enke, Charles A.; Wahl, Andrew O.; Zhou, Sumin; Zheng, Dandan.

In: Technology in cancer research & treatment, Vol. 18, 01.01.2019, p. 1533033819851763.

Research output: Contribution to journalArticle

@article{6abba341f1004e9f89cdb7eb8d0511e1,
title = "Can the Student Outperform the Master? A Plan Comparison Between Pinnacle Auto-Planning and Eclipse knowledge-Based RapidPlan Following a Prostate-Bed Plan Competition",
abstract = "PURPOSE: Pinnacle Auto-Planning and Eclipse RapidPlan are 2 major commercial automated planning engines that are fundamentally different: Auto-Planning mimics real planners in the iterative optimization, while RapidPlan generates static dose objectives from estimations predicted based on a prior knowledge base. This study objectively compared their performances on intensity-modulated radiotherapy planning for prostate fossa and lymphatics adopting the plan quality metric used in the 2011 American Association of Medical Dosimetrists Plan Challenge. METHODS: All plans used an identical intensity-modulated radiotherapy beam setup and a simultaneous integrated boost prescription (68 Gy/56 Gy to prostate fossa/lymphatics). Auto-Planning was used to retrospectively plan on 20 patients, which were subsequently employed as the library to build an RapidPlan model. To compare the 2 engines' performances, a test set including 10 patients and the Plan Challenge patient was planned by both Auto-Planning (master) and RapidPlan (student) without manual intervention except for a common dose normalization and evaluated using the plan quality metric that included 14 quantitative submetrics ranging over target coverage, spillage, and organ at risk doses. Plan quality metric scores were compared between the Auto-Planning and RapidPlan plans using the Mann-Whitney U test. RESULTS: There was no significant difference between the overall performance of the 2 engines on the 11 test cases ( P = .509). Among the 14 submetrics, Auto-Planning and RapidPlan showed no significant difference on most submetrics except for 2. On the Plan Challenge case, Auto-Planning scored 129.9 and RapidPlan scored 130.3 out of 150, as compared with the average score of 116.9 ± 16.4 (range: 58.2-142.5) among the 125 Plan Challenge participants. CONCLUSION: Using an innovative study design, an objective comparison has been conducted between 2 major commercial automated inverse planning engines. The 2 engines performed comparably with each other and both yielded plans at par with average human planners. Using a constant-performing planner (Auto-Planning) to train and to compare, RapidPlan was found to yield plans no better than but as good as its library plans.",
keywords = "Auto-Plan, KBP, RapidPlan, automation, treatment planning",
author = "April Smith and Andrew Granatowicz and Cole Stoltenberg and Shuo Wang and Xiaoying Liang and Enke, {Charles A.} and Wahl, {Andrew O.} and Sumin Zhou and Dandan Zheng",
year = "2019",
month = "1",
day = "1",
doi = "10.1177/1533033819851763",
language = "English (US)",
volume = "18",
pages = "1533033819851763",
journal = "Technology in Cancer Research and Treatment",
issn = "1533-0346",
publisher = "Adenine Press",

}

TY - JOUR

T1 - Can the Student Outperform the Master? A Plan Comparison Between Pinnacle Auto-Planning and Eclipse knowledge-Based RapidPlan Following a Prostate-Bed Plan Competition

AU - Smith, April

AU - Granatowicz, Andrew

AU - Stoltenberg, Cole

AU - Wang, Shuo

AU - Liang, Xiaoying

AU - Enke, Charles A.

AU - Wahl, Andrew O.

AU - Zhou, Sumin

AU - Zheng, Dandan

PY - 2019/1/1

Y1 - 2019/1/1

N2 - PURPOSE: Pinnacle Auto-Planning and Eclipse RapidPlan are 2 major commercial automated planning engines that are fundamentally different: Auto-Planning mimics real planners in the iterative optimization, while RapidPlan generates static dose objectives from estimations predicted based on a prior knowledge base. This study objectively compared their performances on intensity-modulated radiotherapy planning for prostate fossa and lymphatics adopting the plan quality metric used in the 2011 American Association of Medical Dosimetrists Plan Challenge. METHODS: All plans used an identical intensity-modulated radiotherapy beam setup and a simultaneous integrated boost prescription (68 Gy/56 Gy to prostate fossa/lymphatics). Auto-Planning was used to retrospectively plan on 20 patients, which were subsequently employed as the library to build an RapidPlan model. To compare the 2 engines' performances, a test set including 10 patients and the Plan Challenge patient was planned by both Auto-Planning (master) and RapidPlan (student) without manual intervention except for a common dose normalization and evaluated using the plan quality metric that included 14 quantitative submetrics ranging over target coverage, spillage, and organ at risk doses. Plan quality metric scores were compared between the Auto-Planning and RapidPlan plans using the Mann-Whitney U test. RESULTS: There was no significant difference between the overall performance of the 2 engines on the 11 test cases ( P = .509). Among the 14 submetrics, Auto-Planning and RapidPlan showed no significant difference on most submetrics except for 2. On the Plan Challenge case, Auto-Planning scored 129.9 and RapidPlan scored 130.3 out of 150, as compared with the average score of 116.9 ± 16.4 (range: 58.2-142.5) among the 125 Plan Challenge participants. CONCLUSION: Using an innovative study design, an objective comparison has been conducted between 2 major commercial automated inverse planning engines. The 2 engines performed comparably with each other and both yielded plans at par with average human planners. Using a constant-performing planner (Auto-Planning) to train and to compare, RapidPlan was found to yield plans no better than but as good as its library plans.

AB - PURPOSE: Pinnacle Auto-Planning and Eclipse RapidPlan are 2 major commercial automated planning engines that are fundamentally different: Auto-Planning mimics real planners in the iterative optimization, while RapidPlan generates static dose objectives from estimations predicted based on a prior knowledge base. This study objectively compared their performances on intensity-modulated radiotherapy planning for prostate fossa and lymphatics adopting the plan quality metric used in the 2011 American Association of Medical Dosimetrists Plan Challenge. METHODS: All plans used an identical intensity-modulated radiotherapy beam setup and a simultaneous integrated boost prescription (68 Gy/56 Gy to prostate fossa/lymphatics). Auto-Planning was used to retrospectively plan on 20 patients, which were subsequently employed as the library to build an RapidPlan model. To compare the 2 engines' performances, a test set including 10 patients and the Plan Challenge patient was planned by both Auto-Planning (master) and RapidPlan (student) without manual intervention except for a common dose normalization and evaluated using the plan quality metric that included 14 quantitative submetrics ranging over target coverage, spillage, and organ at risk doses. Plan quality metric scores were compared between the Auto-Planning and RapidPlan plans using the Mann-Whitney U test. RESULTS: There was no significant difference between the overall performance of the 2 engines on the 11 test cases ( P = .509). Among the 14 submetrics, Auto-Planning and RapidPlan showed no significant difference on most submetrics except for 2. On the Plan Challenge case, Auto-Planning scored 129.9 and RapidPlan scored 130.3 out of 150, as compared with the average score of 116.9 ± 16.4 (range: 58.2-142.5) among the 125 Plan Challenge participants. CONCLUSION: Using an innovative study design, an objective comparison has been conducted between 2 major commercial automated inverse planning engines. The 2 engines performed comparably with each other and both yielded plans at par with average human planners. Using a constant-performing planner (Auto-Planning) to train and to compare, RapidPlan was found to yield plans no better than but as good as its library plans.

KW - Auto-Plan

KW - KBP

KW - RapidPlan

KW - automation

KW - treatment planning

UR - http://www.scopus.com/inward/record.url?scp=85067540291&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85067540291&partnerID=8YFLogxK

U2 - 10.1177/1533033819851763

DO - 10.1177/1533033819851763

M3 - Article

C2 - 31177922

AN - SCOPUS:85067540291

VL - 18

SP - 1533033819851763

JO - Technology in Cancer Research and Treatment

JF - Technology in Cancer Research and Treatment

SN - 1533-0346

ER -