Improving the performance of I/O-intensive applications on clusters of workstations

Xiao Qin, Hong Jiang, Yifeng Zhu, David R Swanson

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

Load balancing in a workstation-based cluster system has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, if a significant portion of applications running in the system is I/O-intensive, traditional load balancing policies can cause system performance to decrease substantially. In this paper, two I/O-aware load-balancing schemes, referred to as IOCM and WAL-PM, are presented to improve the overall performance of a cluster system with a general and practical workload including I/O activities. The proposed schemes dynamically detect I/O load imbalance of nodes in a cluster, and determine whether to migrate some I/O load from overloaded nodes to other less- or under-loaded nodes. The current running jobs are eligible to be migrated in WAL-PM only if overall performance improves. Besides balancing I/O load, the scheme judiciously takes into account both CPU and memory load sharing in the system, thereby maintaining the same level of performance as existing schemes when I/O load is low or well balanced. Extensive trace-driven simulations for both synthetic and real I/O-intensive applications show that: (1) Compared with existing schemes that only consider CPU and memory, the proposed schemes improve the performance with respect to mean slowdown by up to a factor of 20; (2) When compared to the existing approaches that only consider I/O with non-preemptive job migrations, the proposed schemes achieve improvements in mean slowdown by up to a factor of 10; (3) Under CPU-memory intensive workloads, our scheme improves the performance over the existing approaches that only consider I/O by up to 47.5%.

Original languageEnglish (US)
Pages (from-to)297-311
Number of pages15
JournalCluster Computing
Volume9
Issue number3
DOIs
StatePublished - Jul 1 2006

Fingerprint

Program processors
Resource allocation
Data storage equipment
Computer systems

Keywords

  • Clusters
  • I/O intensive
  • Performance evaluation
  • Slowdown

ASJC Scopus subject areas

  • Software
  • Computer Networks and Communications

Cite this

Improving the performance of I/O-intensive applications on clusters of workstations. / Qin, Xiao; Jiang, Hong; Zhu, Yifeng; Swanson, David R.

In: Cluster Computing, Vol. 9, No. 3, 01.07.2006, p. 297-311.

Research output: Contribution to journalArticle

Qin, Xiao ; Jiang, Hong ; Zhu, Yifeng ; Swanson, David R. / Improving the performance of I/O-intensive applications on clusters of workstations. In: Cluster Computing. 2006 ; Vol. 9, No. 3. pp. 297-311.
@article{fab35aee39c041d79b14bc16741b214a,
title = "Improving the performance of I/O-intensive applications on clusters of workstations",
abstract = "Load balancing in a workstation-based cluster system has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, if a significant portion of applications running in the system is I/O-intensive, traditional load balancing policies can cause system performance to decrease substantially. In this paper, two I/O-aware load-balancing schemes, referred to as IOCM and WAL-PM, are presented to improve the overall performance of a cluster system with a general and practical workload including I/O activities. The proposed schemes dynamically detect I/O load imbalance of nodes in a cluster, and determine whether to migrate some I/O load from overloaded nodes to other less- or under-loaded nodes. The current running jobs are eligible to be migrated in WAL-PM only if overall performance improves. Besides balancing I/O load, the scheme judiciously takes into account both CPU and memory load sharing in the system, thereby maintaining the same level of performance as existing schemes when I/O load is low or well balanced. Extensive trace-driven simulations for both synthetic and real I/O-intensive applications show that: (1) Compared with existing schemes that only consider CPU and memory, the proposed schemes improve the performance with respect to mean slowdown by up to a factor of 20; (2) When compared to the existing approaches that only consider I/O with non-preemptive job migrations, the proposed schemes achieve improvements in mean slowdown by up to a factor of 10; (3) Under CPU-memory intensive workloads, our scheme improves the performance over the existing approaches that only consider I/O by up to 47.5{\%}.",
keywords = "Clusters, I/O intensive, Performance evaluation, Slowdown",
author = "Xiao Qin and Hong Jiang and Yifeng Zhu and Swanson, {David R}",
year = "2006",
month = "7",
day = "1",
doi = "10.1007/s10586-006-9742-7",
language = "English (US)",
volume = "9",
pages = "297--311",
journal = "Cluster Computing",
issn = "1386-7857",
publisher = "Kluwer Academic Publishers",
number = "3",

}

TY - JOUR

T1 - Improving the performance of I/O-intensive applications on clusters of workstations

AU - Qin, Xiao

AU - Jiang, Hong

AU - Zhu, Yifeng

AU - Swanson, David R

PY - 2006/7/1

Y1 - 2006/7/1

N2 - Load balancing in a workstation-based cluster system has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, if a significant portion of applications running in the system is I/O-intensive, traditional load balancing policies can cause system performance to decrease substantially. In this paper, two I/O-aware load-balancing schemes, referred to as IOCM and WAL-PM, are presented to improve the overall performance of a cluster system with a general and practical workload including I/O activities. The proposed schemes dynamically detect I/O load imbalance of nodes in a cluster, and determine whether to migrate some I/O load from overloaded nodes to other less- or under-loaded nodes. The current running jobs are eligible to be migrated in WAL-PM only if overall performance improves. Besides balancing I/O load, the scheme judiciously takes into account both CPU and memory load sharing in the system, thereby maintaining the same level of performance as existing schemes when I/O load is low or well balanced. Extensive trace-driven simulations for both synthetic and real I/O-intensive applications show that: (1) Compared with existing schemes that only consider CPU and memory, the proposed schemes improve the performance with respect to mean slowdown by up to a factor of 20; (2) When compared to the existing approaches that only consider I/O with non-preemptive job migrations, the proposed schemes achieve improvements in mean slowdown by up to a factor of 10; (3) Under CPU-memory intensive workloads, our scheme improves the performance over the existing approaches that only consider I/O by up to 47.5%.

AB - Load balancing in a workstation-based cluster system has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, if a significant portion of applications running in the system is I/O-intensive, traditional load balancing policies can cause system performance to decrease substantially. In this paper, two I/O-aware load-balancing schemes, referred to as IOCM and WAL-PM, are presented to improve the overall performance of a cluster system with a general and practical workload including I/O activities. The proposed schemes dynamically detect I/O load imbalance of nodes in a cluster, and determine whether to migrate some I/O load from overloaded nodes to other less- or under-loaded nodes. The current running jobs are eligible to be migrated in WAL-PM only if overall performance improves. Besides balancing I/O load, the scheme judiciously takes into account both CPU and memory load sharing in the system, thereby maintaining the same level of performance as existing schemes when I/O load is low or well balanced. Extensive trace-driven simulations for both synthetic and real I/O-intensive applications show that: (1) Compared with existing schemes that only consider CPU and memory, the proposed schemes improve the performance with respect to mean slowdown by up to a factor of 20; (2) When compared to the existing approaches that only consider I/O with non-preemptive job migrations, the proposed schemes achieve improvements in mean slowdown by up to a factor of 10; (3) Under CPU-memory intensive workloads, our scheme improves the performance over the existing approaches that only consider I/O by up to 47.5%.

KW - Clusters

KW - I/O intensive

KW - Performance evaluation

KW - Slowdown

UR - http://www.scopus.com/inward/record.url?scp=33748881968&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33748881968&partnerID=8YFLogxK

U2 - 10.1007/s10586-006-9742-7

DO - 10.1007/s10586-006-9742-7

M3 - Article

VL - 9

SP - 297

EP - 311

JO - Cluster Computing

JF - Cluster Computing

SN - 1386-7857

IS - 3

ER -