Legion-based scientific data analytics on heterogeneous processors

Lina Yu, Hongfeng Yu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present a study of scientific data analytics on heterogeneous architectures using the Legion runtime system. Legion is a new programming model and runtime system targeting distributed heterogeneous architectures. It introduces logical regions as a new abstraction for describing the structures and usages of program data. We describe how to leverage logical regions to express important properties of program data, such as locality and independence, for scientific data analytics that can consist of multiple operations with different data types. Our approach can help users simplify programming on the data partition, data organization, and data movement for distributed-memory heterogeneous architectures, thereby facilitating a simultaneous execution of multiple analytics operations on modern and future supercomputers. We demonstrate the scalability and the usability of our approach by a hybrid data partitioning and distribution scheme for different data types using both CPUs and GPUs on a heterogeneous system.

Original languageEnglish (US)
Title of host publicationProceedings - 2016 IEEE International Conference on Big Data, Big Data 2016
EditorsRonay Ak, George Karypis, Yinglong Xia, Xiaohua Tony Hu, Philip S. Yu, James Joshi, Lyle Ungar, Ling Liu, Aki-Hiro Sato, Toyotaro Suzumura, Sudarsan Rachuri, Rama Govindaraju, Weijia Xu
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2305-2314
Number of pages10
ISBN (Electronic)9781467390040
DOIs
StatePublished - Jan 1 2016
Event4th IEEE International Conference on Big Data, Big Data 2016 - Washington, United States
Duration: Dec 5 2016Dec 8 2016

Publication series

NameProceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

Other

Other4th IEEE International Conference on Big Data, Big Data 2016
CountryUnited States
CityWashington
Period12/5/1612/8/16

Fingerprint

Supercomputers
Program processors
Scalability
Data storage equipment
Graphics processing unit

Keywords

  • Legion
  • heterogeneous processors
  • scientific data analytics

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Hardware and Architecture

Cite this

Yu, L., & Yu, H. (2016). Legion-based scientific data analytics on heterogeneous processors. In R. Ak, G. Karypis, Y. Xia, X. T. Hu, P. S. Yu, J. Joshi, L. Ungar, L. Liu, A-H. Sato, T. Suzumura, S. Rachuri, R. Govindaraju, ... W. Xu (Eds.), Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016 (pp. 2305-2314). [7840863] (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData.2016.7840863

Legion-based scientific data analytics on heterogeneous processors. / Yu, Lina; Yu, Hongfeng.

Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. ed. / Ronay Ak; George Karypis; Yinglong Xia; Xiaohua Tony Hu; Philip S. Yu; James Joshi; Lyle Ungar; Ling Liu; Aki-Hiro Sato; Toyotaro Suzumura; Sudarsan Rachuri; Rama Govindaraju; Weijia Xu. Institute of Electrical and Electronics Engineers Inc., 2016. p. 2305-2314 7840863 (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yu, L & Yu, H 2016, Legion-based scientific data analytics on heterogeneous processors. in R Ak, G Karypis, Y Xia, XT Hu, PS Yu, J Joshi, L Ungar, L Liu, A-H Sato, T Suzumura, S Rachuri, R Govindaraju & W Xu (eds), Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016., 7840863, Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016, Institute of Electrical and Electronics Engineers Inc., pp. 2305-2314, 4th IEEE International Conference on Big Data, Big Data 2016, Washington, United States, 12/5/16. https://doi.org/10.1109/BigData.2016.7840863
Yu L, Yu H. Legion-based scientific data analytics on heterogeneous processors. In Ak R, Karypis G, Xia Y, Hu XT, Yu PS, Joshi J, Ungar L, Liu L, Sato A-H, Suzumura T, Rachuri S, Govindaraju R, Xu W, editors, Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. Institute of Electrical and Electronics Engineers Inc. 2016. p. 2305-2314. 7840863. (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016). https://doi.org/10.1109/BigData.2016.7840863
Yu, Lina ; Yu, Hongfeng. / Legion-based scientific data analytics on heterogeneous processors. Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016. editor / Ronay Ak ; George Karypis ; Yinglong Xia ; Xiaohua Tony Hu ; Philip S. Yu ; James Joshi ; Lyle Ungar ; Ling Liu ; Aki-Hiro Sato ; Toyotaro Suzumura ; Sudarsan Rachuri ; Rama Govindaraju ; Weijia Xu. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 2305-2314 (Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016).
@inproceedings{9c683eac9c6c4649a539652267f1dec0,
title = "Legion-based scientific data analytics on heterogeneous processors",
abstract = "We present a study of scientific data analytics on heterogeneous architectures using the Legion runtime system. Legion is a new programming model and runtime system targeting distributed heterogeneous architectures. It introduces logical regions as a new abstraction for describing the structures and usages of program data. We describe how to leverage logical regions to express important properties of program data, such as locality and independence, for scientific data analytics that can consist of multiple operations with different data types. Our approach can help users simplify programming on the data partition, data organization, and data movement for distributed-memory heterogeneous architectures, thereby facilitating a simultaneous execution of multiple analytics operations on modern and future supercomputers. We demonstrate the scalability and the usability of our approach by a hybrid data partitioning and distribution scheme for different data types using both CPUs and GPUs on a heterogeneous system.",
keywords = "Legion, heterogeneous processors, scientific data analytics",
author = "Lina Yu and Hongfeng Yu",
year = "2016",
month = "1",
day = "1",
doi = "10.1109/BigData.2016.7840863",
language = "English (US)",
series = "Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "2305--2314",
editor = "Ronay Ak and George Karypis and Yinglong Xia and Hu, {Xiaohua Tony} and Yu, {Philip S.} and James Joshi and Lyle Ungar and Ling Liu and Aki-Hiro Sato and Toyotaro Suzumura and Sudarsan Rachuri and Rama Govindaraju and Weijia Xu",
booktitle = "Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016",

}

TY - GEN

T1 - Legion-based scientific data analytics on heterogeneous processors

AU - Yu, Lina

AU - Yu, Hongfeng

PY - 2016/1/1

Y1 - 2016/1/1

N2 - We present a study of scientific data analytics on heterogeneous architectures using the Legion runtime system. Legion is a new programming model and runtime system targeting distributed heterogeneous architectures. It introduces logical regions as a new abstraction for describing the structures and usages of program data. We describe how to leverage logical regions to express important properties of program data, such as locality and independence, for scientific data analytics that can consist of multiple operations with different data types. Our approach can help users simplify programming on the data partition, data organization, and data movement for distributed-memory heterogeneous architectures, thereby facilitating a simultaneous execution of multiple analytics operations on modern and future supercomputers. We demonstrate the scalability and the usability of our approach by a hybrid data partitioning and distribution scheme for different data types using both CPUs and GPUs on a heterogeneous system.

AB - We present a study of scientific data analytics on heterogeneous architectures using the Legion runtime system. Legion is a new programming model and runtime system targeting distributed heterogeneous architectures. It introduces logical regions as a new abstraction for describing the structures and usages of program data. We describe how to leverage logical regions to express important properties of program data, such as locality and independence, for scientific data analytics that can consist of multiple operations with different data types. Our approach can help users simplify programming on the data partition, data organization, and data movement for distributed-memory heterogeneous architectures, thereby facilitating a simultaneous execution of multiple analytics operations on modern and future supercomputers. We demonstrate the scalability and the usability of our approach by a hybrid data partitioning and distribution scheme for different data types using both CPUs and GPUs on a heterogeneous system.

KW - Legion

KW - heterogeneous processors

KW - scientific data analytics

UR - http://www.scopus.com/inward/record.url?scp=85015191237&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85015191237&partnerID=8YFLogxK

U2 - 10.1109/BigData.2016.7840863

DO - 10.1109/BigData.2016.7840863

M3 - Conference contribution

AN - SCOPUS:85015191237

T3 - Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

SP - 2305

EP - 2314

BT - Proceedings - 2016 IEEE International Conference on Big Data, Big Data 2016

A2 - Ak, Ronay

A2 - Karypis, George

A2 - Xia, Yinglong

A2 - Hu, Xiaohua Tony

A2 - Yu, Philip S.

A2 - Joshi, James

A2 - Ungar, Lyle

A2 - Liu, Ling

A2 - Sato, Aki-Hiro

A2 - Suzumura, Toyotaro

A2 - Rachuri, Sudarsan

A2 - Govindaraju, Rama

A2 - Xu, Weijia

PB - Institute of Electrical and Electronics Engineers Inc.

ER -