Boosting performance of I/O-intensive workload by preemptive job migrations in a cluster system

X. Qin, H. Jiang, Y. Zhu, D. R. Swanson

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

Load balancing in a cluster system has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, if a significant portion of applications running in the system is I/O-intensive, traditional load balancing policies that focus on CPU and memory usage may cause the system performance to decrease substantially. To solve this problem, a new I/O-aware load-balancing scheme with preemptive job migration is presented to sustain the high performance of a cluster with a diverse set of workload conditions. The proposed scheme dynamically detects I/O load imbalance on nodes of a cluster, and determines whether to preempt some running jobs on overloaded nodes and migrate them to other less- or under-loaded nodes. Besides balancing I/O load, the scheme takes into account both CPU and memory load sharing in clusters, thereby maintaining the same level of performance as existing schemes when I/O load is low or well balanced. Results from a trace-driven simulation show that, compared to the existing approaches that only consider I/O with nonpreemptive job migrations, the proposed schemes achieve the improvement in mean slowdown by up to a factor of 10.

Original languageEnglish (US)
Title of host publicationProceedings - 15th Symposium on Computer Architecture and High Performance Computing, SBAC-PAD 2003
EditorsLiria M. Sato, Edson T. Midorikawa, Philippe O. A. Navaux
PublisherIEEE Computer Society
Pages235-243
Number of pages9
ISBN (Electronic)0769520464
DOIs
StatePublished - Jan 1 2003
Event15th Symposium on Computer Architecture and High Performance Computing, SBAC-PAD 2003 - Sao Paulo, Brazil
Duration: Nov 10 2003Nov 12 2003

Publication series

NameProceedings - Symposium on Computer Architecture and High Performance Computing
Volume2003-January
ISSN (Print)1550-6533

Other

Other15th Symposium on Computer Architecture and High Performance Computing, SBAC-PAD 2003
CountryBrazil
CitySao Paulo
Period11/10/0311/12/03

Fingerprint

Resource allocation
Program processors
Data storage equipment
Computer systems

Keywords

  • Application software
  • Boosting
  • Bridges
  • Computational modeling
  • Computer science
  • Costs
  • Load management
  • Remote sensing
  • System performance
  • Throughput

ASJC Scopus subject areas

  • Hardware and Architecture
  • Software

Cite this

Qin, X., Jiang, H., Zhu, Y., & Swanson, D. R. (2003). Boosting performance of I/O-intensive workload by preemptive job migrations in a cluster system. In L. M. Sato, E. T. Midorikawa, & P. O. A. Navaux (Eds.), Proceedings - 15th Symposium on Computer Architecture and High Performance Computing, SBAC-PAD 2003 (pp. 235-243). [1250343] (Proceedings - Symposium on Computer Architecture and High Performance Computing; Vol. 2003-January). IEEE Computer Society. https://doi.org/10.1109/CAHPC.2003.1250343

Boosting performance of I/O-intensive workload by preemptive job migrations in a cluster system. / Qin, X.; Jiang, H.; Zhu, Y.; Swanson, D. R.

Proceedings - 15th Symposium on Computer Architecture and High Performance Computing, SBAC-PAD 2003. ed. / Liria M. Sato; Edson T. Midorikawa; Philippe O. A. Navaux. IEEE Computer Society, 2003. p. 235-243 1250343 (Proceedings - Symposium on Computer Architecture and High Performance Computing; Vol. 2003-January).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Qin, X, Jiang, H, Zhu, Y & Swanson, DR 2003, Boosting performance of I/O-intensive workload by preemptive job migrations in a cluster system. in LM Sato, ET Midorikawa & POA Navaux (eds), Proceedings - 15th Symposium on Computer Architecture and High Performance Computing, SBAC-PAD 2003., 1250343, Proceedings - Symposium on Computer Architecture and High Performance Computing, vol. 2003-January, IEEE Computer Society, pp. 235-243, 15th Symposium on Computer Architecture and High Performance Computing, SBAC-PAD 2003, Sao Paulo, Brazil, 11/10/03. https://doi.org/10.1109/CAHPC.2003.1250343
Qin X, Jiang H, Zhu Y, Swanson DR. Boosting performance of I/O-intensive workload by preemptive job migrations in a cluster system. In Sato LM, Midorikawa ET, Navaux POA, editors, Proceedings - 15th Symposium on Computer Architecture and High Performance Computing, SBAC-PAD 2003. IEEE Computer Society. 2003. p. 235-243. 1250343. (Proceedings - Symposium on Computer Architecture and High Performance Computing). https://doi.org/10.1109/CAHPC.2003.1250343
Qin, X. ; Jiang, H. ; Zhu, Y. ; Swanson, D. R. / Boosting performance of I/O-intensive workload by preemptive job migrations in a cluster system. Proceedings - 15th Symposium on Computer Architecture and High Performance Computing, SBAC-PAD 2003. editor / Liria M. Sato ; Edson T. Midorikawa ; Philippe O. A. Navaux. IEEE Computer Society, 2003. pp. 235-243 (Proceedings - Symposium on Computer Architecture and High Performance Computing).
@inproceedings{7f77da24dd82413a950efe1c2c12f62f,
title = "Boosting performance of I/O-intensive workload by preemptive job migrations in a cluster system",
abstract = "Load balancing in a cluster system has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, if a significant portion of applications running in the system is I/O-intensive, traditional load balancing policies that focus on CPU and memory usage may cause the system performance to decrease substantially. To solve this problem, a new I/O-aware load-balancing scheme with preemptive job migration is presented to sustain the high performance of a cluster with a diverse set of workload conditions. The proposed scheme dynamically detects I/O load imbalance on nodes of a cluster, and determines whether to preempt some running jobs on overloaded nodes and migrate them to other less- or under-loaded nodes. Besides balancing I/O load, the scheme takes into account both CPU and memory load sharing in clusters, thereby maintaining the same level of performance as existing schemes when I/O load is low or well balanced. Results from a trace-driven simulation show that, compared to the existing approaches that only consider I/O with nonpreemptive job migrations, the proposed schemes achieve the improvement in mean slowdown by up to a factor of 10.",
keywords = "Application software, Boosting, Bridges, Computational modeling, Computer science, Costs, Load management, Remote sensing, System performance, Throughput",
author = "X. Qin and H. Jiang and Y. Zhu and Swanson, {D. R.}",
year = "2003",
month = "1",
day = "1",
doi = "10.1109/CAHPC.2003.1250343",
language = "English (US)",
series = "Proceedings - Symposium on Computer Architecture and High Performance Computing",
publisher = "IEEE Computer Society",
pages = "235--243",
editor = "Sato, {Liria M.} and Midorikawa, {Edson T.} and Navaux, {Philippe O. A.}",
booktitle = "Proceedings - 15th Symposium on Computer Architecture and High Performance Computing, SBAC-PAD 2003",

}

TY - GEN

T1 - Boosting performance of I/O-intensive workload by preemptive job migrations in a cluster system

AU - Qin, X.

AU - Jiang, H.

AU - Zhu, Y.

AU - Swanson, D. R.

PY - 2003/1/1

Y1 - 2003/1/1

N2 - Load balancing in a cluster system has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, if a significant portion of applications running in the system is I/O-intensive, traditional load balancing policies that focus on CPU and memory usage may cause the system performance to decrease substantially. To solve this problem, a new I/O-aware load-balancing scheme with preemptive job migration is presented to sustain the high performance of a cluster with a diverse set of workload conditions. The proposed scheme dynamically detects I/O load imbalance on nodes of a cluster, and determines whether to preempt some running jobs on overloaded nodes and migrate them to other less- or under-loaded nodes. Besides balancing I/O load, the scheme takes into account both CPU and memory load sharing in clusters, thereby maintaining the same level of performance as existing schemes when I/O load is low or well balanced. Results from a trace-driven simulation show that, compared to the existing approaches that only consider I/O with nonpreemptive job migrations, the proposed schemes achieve the improvement in mean slowdown by up to a factor of 10.

AB - Load balancing in a cluster system has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, if a significant portion of applications running in the system is I/O-intensive, traditional load balancing policies that focus on CPU and memory usage may cause the system performance to decrease substantially. To solve this problem, a new I/O-aware load-balancing scheme with preemptive job migration is presented to sustain the high performance of a cluster with a diverse set of workload conditions. The proposed scheme dynamically detects I/O load imbalance on nodes of a cluster, and determines whether to preempt some running jobs on overloaded nodes and migrate them to other less- or under-loaded nodes. Besides balancing I/O load, the scheme takes into account both CPU and memory load sharing in clusters, thereby maintaining the same level of performance as existing schemes when I/O load is low or well balanced. Results from a trace-driven simulation show that, compared to the existing approaches that only consider I/O with nonpreemptive job migrations, the proposed schemes achieve the improvement in mean slowdown by up to a factor of 10.

KW - Application software

KW - Boosting

KW - Bridges

KW - Computational modeling

KW - Computer science

KW - Costs

KW - Load management

KW - Remote sensing

KW - System performance

KW - Throughput

UR - http://www.scopus.com/inward/record.url?scp=84944452647&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84944452647&partnerID=8YFLogxK

U2 - 10.1109/CAHPC.2003.1250343

DO - 10.1109/CAHPC.2003.1250343

M3 - Conference contribution

AN - SCOPUS:84944452647

T3 - Proceedings - Symposium on Computer Architecture and High Performance Computing

SP - 235

EP - 243

BT - Proceedings - 15th Symposium on Computer Architecture and High Performance Computing, SBAC-PAD 2003

A2 - Sato, Liria M.

A2 - Midorikawa, Edson T.

A2 - Navaux, Philippe O. A.

PB - IEEE Computer Society

ER -