In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows

Qian Sun, Melissa Romanus, Tong Jin, Hongfeng Yu, Peer Timo Bremer, Steve Petruzza, Scott Klasky, Manish Parashar

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

Coupled application workflows composed of applications implemented using task-based models present new coupling and data exchange challenges, due to the asynchronous interaction and coupling behaviors between tasks of the component applications. In this paper, we present an adaptive data placement approach that addresses these challenges by dynamically adjusting to the asynchronous coupling patterns. Specifically, it places data across a set of staging cores/nodes with an awareness of the application-specific data locality requirements and the runtime task executions at these staging cores/nodes, with the goal of reducing end-to-end execution time and data movement overhead of the workflow. We experimentally demonstrate the effectiveness of our approach on the Titan Cray XK7 system using representative data coupling patterns derived from current scientific workflows. The evaluation demonstrates that our approach efficiently improves performance by reducing the time-to-solution and increasing the quality of insights for scientific discovery.

Original languageEnglish (US)
Title of host publicationProceedings of ESPM2 2016
Subtitle of host publication2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2-9
Number of pages8
ISBN (Electronic)9781509038589
DOIs
StatePublished - Jan 24 2017
Event2nd International Workshop on Extreme Scale Programming Models and Middleware, ESPM2 2016 - Salt Lake City, United States
Duration: Nov 18 2016 → …

Publication series

NameProceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis

Other

Other2nd International Workshop on Extreme Scale Programming Models and Middleware, ESPM2 2016
CountryUnited States
CitySalt Lake City
Period11/18/16 → …

Fingerprint

Electronic data interchange

Keywords

  • Couplings
  • Data storage systems
  • Runtime

ASJC Scopus subject areas

  • Hardware and Architecture
  • Software

Cite this

Sun, Q., Romanus, M., Jin, T., Yu, H., Bremer, P. T., Petruzza, S., ... Parashar, M. (2017). In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows. In Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis (pp. 2-9). [7831554] (Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ESPM2.2016.006

In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows. / Sun, Qian; Romanus, Melissa; Jin, Tong; Yu, Hongfeng; Bremer, Peer Timo; Petruzza, Steve; Klasky, Scott; Parashar, Manish.

Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2017. p. 2-9 7831554 (Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Sun, Q, Romanus, M, Jin, T, Yu, H, Bremer, PT, Petruzza, S, Klasky, S & Parashar, M 2017, In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows. in Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis., 7831554, Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis, Institute of Electrical and Electronics Engineers Inc., pp. 2-9, 2nd International Workshop on Extreme Scale Programming Models and Middleware, ESPM2 2016, Salt Lake City, United States, 11/18/16. https://doi.org/10.1109/ESPM2.2016.006
Sun Q, Romanus M, Jin T, Yu H, Bremer PT, Petruzza S et al. In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows. In Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc. 2017. p. 2-9. 7831554. (Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis). https://doi.org/10.1109/ESPM2.2016.006
Sun, Qian ; Romanus, Melissa ; Jin, Tong ; Yu, Hongfeng ; Bremer, Peer Timo ; Petruzza, Steve ; Klasky, Scott ; Parashar, Manish. / In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows. Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 2-9 (Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis).
@inproceedings{78e558e96a7b46b98fb5a639ce3e1cdd,
title = "In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows",
abstract = "Coupled application workflows composed of applications implemented using task-based models present new coupling and data exchange challenges, due to the asynchronous interaction and coupling behaviors between tasks of the component applications. In this paper, we present an adaptive data placement approach that addresses these challenges by dynamically adjusting to the asynchronous coupling patterns. Specifically, it places data across a set of staging cores/nodes with an awareness of the application-specific data locality requirements and the runtime task executions at these staging cores/nodes, with the goal of reducing end-to-end execution time and data movement overhead of the workflow. We experimentally demonstrate the effectiveness of our approach on the Titan Cray XK7 system using representative data coupling patterns derived from current scientific workflows. The evaluation demonstrates that our approach efficiently improves performance by reducing the time-to-solution and increasing the quality of insights for scientific discovery.",
keywords = "Couplings, Data storage systems, Runtime",
author = "Qian Sun and Melissa Romanus and Tong Jin and Hongfeng Yu and Bremer, {Peer Timo} and Steve Petruzza and Scott Klasky and Manish Parashar",
year = "2017",
month = "1",
day = "24",
doi = "10.1109/ESPM2.2016.006",
language = "English (US)",
series = "Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "2--9",
booktitle = "Proceedings of ESPM2 2016",

}

TY - GEN

T1 - In-Staging Data Placement for Asynchronous Coupling of Task-Based Scientific Workflows

AU - Sun, Qian

AU - Romanus, Melissa

AU - Jin, Tong

AU - Yu, Hongfeng

AU - Bremer, Peer Timo

AU - Petruzza, Steve

AU - Klasky, Scott

AU - Parashar, Manish

PY - 2017/1/24

Y1 - 2017/1/24

N2 - Coupled application workflows composed of applications implemented using task-based models present new coupling and data exchange challenges, due to the asynchronous interaction and coupling behaviors between tasks of the component applications. In this paper, we present an adaptive data placement approach that addresses these challenges by dynamically adjusting to the asynchronous coupling patterns. Specifically, it places data across a set of staging cores/nodes with an awareness of the application-specific data locality requirements and the runtime task executions at these staging cores/nodes, with the goal of reducing end-to-end execution time and data movement overhead of the workflow. We experimentally demonstrate the effectiveness of our approach on the Titan Cray XK7 system using representative data coupling patterns derived from current scientific workflows. The evaluation demonstrates that our approach efficiently improves performance by reducing the time-to-solution and increasing the quality of insights for scientific discovery.

AB - Coupled application workflows composed of applications implemented using task-based models present new coupling and data exchange challenges, due to the asynchronous interaction and coupling behaviors between tasks of the component applications. In this paper, we present an adaptive data placement approach that addresses these challenges by dynamically adjusting to the asynchronous coupling patterns. Specifically, it places data across a set of staging cores/nodes with an awareness of the application-specific data locality requirements and the runtime task executions at these staging cores/nodes, with the goal of reducing end-to-end execution time and data movement overhead of the workflow. We experimentally demonstrate the effectiveness of our approach on the Titan Cray XK7 system using representative data coupling patterns derived from current scientific workflows. The evaluation demonstrates that our approach efficiently improves performance by reducing the time-to-solution and increasing the quality of insights for scientific discovery.

KW - Couplings

KW - Data storage systems

KW - Runtime

UR - http://www.scopus.com/inward/record.url?scp=85013982634&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85013982634&partnerID=8YFLogxK

U2 - 10.1109/ESPM2.2016.006

DO - 10.1109/ESPM2.2016.006

M3 - Conference contribution

AN - SCOPUS:85013982634

T3 - Proceedings of ESPM2 2016: 2nd International Workshop on Extreme Scale Programming Models and Middleware - Held in conjunction with SC 2016: The International Conference for High Performance Computing, Networking, Storage and Analysis

SP - 2

EP - 9

BT - Proceedings of ESPM2 2016

PB - Institute of Electrical and Electronics Engineers Inc.

ER -