Improved read performance in a cost-effective, fault-tolerant parallel virtual file system (CEFT-PVFS)

Yifeng Zhu, Hong Jiang, Xiao Qin, Dan Feng, David R Swanson

Research output: Chapter in Book/Report/Conference proceedingConference contribution

19 Citations (Scopus)

Abstract

Due to the ever-widening performance gap between processors and disks, I/O operations tend to become the major performance bottleneck of data-intensive applications on modern clusters. If all the existing disks on the nodes of a cluster are connected together to establish high performance parallel storage systems, the cluster's overall performance can be boosted at no additional cost. CEFT-PVFS (a RAID 10 style parallel file system that extends the original PVFS), as one such system, divides the cluster nodes into two groups, stripes the data across one group in a round-robin fashion, and then duplicates the same data to the other group to provide storage service of high performance and high reliability. Previous research has shown that the system reliability is improved by a factor of more than 40 with mirroring while maintaining a comparable write performance. This paper presents another benefit of CEFT-PVFS in which the aggregate peak read performance can be improved by as much as 100% over that of the original PVFS by exploiting the increased parallelism. Additionally, when the data servers, which typically are also computational nodes in a cluster environment, are loaded in an unbalanced way by applications running in the cluster, the read performance of PVFS will be degraded significantly. On the contrary, in the CEFT-PVFS, a heavily loaded data server can be skipped and all the desired data is read from its mirroring node. Thus the performance will not be affected unless both the server node and its mirroring node are heavily loaded.

Original languageEnglish (US)
Title of host publicationProceedings - CCGrid 2003
Subtitle of host publication3rd IEEE/ACM International Symposium on Cluster Computing and the Grid
Pages730-735
Number of pages6
DOIs
StatePublished - Dec 1 2003
Event3rd IEEE/ACM International Symposium on Cluster Computing and the Grid, CCGrid 2003 - Tokyo, Japan
Duration: May 12 2003May 15 2003

Publication series

NameProceedings - CCGrid 2003: 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid

Conference

Conference3rd IEEE/ACM International Symposium on Cluster Computing and the Grid, CCGrid 2003
CountryJapan
CityTokyo
Period5/12/035/15/03

Fingerprint

Servers
Costs

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Computer Science Applications
  • Software
  • Electrical and Electronic Engineering

Cite this

Zhu, Y., Jiang, H., Qin, X., Feng, D., & Swanson, D. R. (2003). Improved read performance in a cost-effective, fault-tolerant parallel virtual file system (CEFT-PVFS). In Proceedings - CCGrid 2003: 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid (pp. 730-735). [1199440] (Proceedings - CCGrid 2003: 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid). https://doi.org/10.1109/CCGRID.2003.1199440

Improved read performance in a cost-effective, fault-tolerant parallel virtual file system (CEFT-PVFS). / Zhu, Yifeng; Jiang, Hong; Qin, Xiao; Feng, Dan; Swanson, David R.

Proceedings - CCGrid 2003: 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid. 2003. p. 730-735 1199440 (Proceedings - CCGrid 2003: 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zhu, Y, Jiang, H, Qin, X, Feng, D & Swanson, DR 2003, Improved read performance in a cost-effective, fault-tolerant parallel virtual file system (CEFT-PVFS). in Proceedings - CCGrid 2003: 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid., 1199440, Proceedings - CCGrid 2003: 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid, pp. 730-735, 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid, CCGrid 2003, Tokyo, Japan, 5/12/03. https://doi.org/10.1109/CCGRID.2003.1199440
Zhu Y, Jiang H, Qin X, Feng D, Swanson DR. Improved read performance in a cost-effective, fault-tolerant parallel virtual file system (CEFT-PVFS). In Proceedings - CCGrid 2003: 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid. 2003. p. 730-735. 1199440. (Proceedings - CCGrid 2003: 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid). https://doi.org/10.1109/CCGRID.2003.1199440
Zhu, Yifeng ; Jiang, Hong ; Qin, Xiao ; Feng, Dan ; Swanson, David R. / Improved read performance in a cost-effective, fault-tolerant parallel virtual file system (CEFT-PVFS). Proceedings - CCGrid 2003: 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid. 2003. pp. 730-735 (Proceedings - CCGrid 2003: 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid).
@inproceedings{9368337f4cd44dd29dec5ce19cbb611a,
title = "Improved read performance in a cost-effective, fault-tolerant parallel virtual file system (CEFT-PVFS)",
abstract = "Due to the ever-widening performance gap between processors and disks, I/O operations tend to become the major performance bottleneck of data-intensive applications on modern clusters. If all the existing disks on the nodes of a cluster are connected together to establish high performance parallel storage systems, the cluster's overall performance can be boosted at no additional cost. CEFT-PVFS (a RAID 10 style parallel file system that extends the original PVFS), as one such system, divides the cluster nodes into two groups, stripes the data across one group in a round-robin fashion, and then duplicates the same data to the other group to provide storage service of high performance and high reliability. Previous research has shown that the system reliability is improved by a factor of more than 40 with mirroring while maintaining a comparable write performance. This paper presents another benefit of CEFT-PVFS in which the aggregate peak read performance can be improved by as much as 100{\%} over that of the original PVFS by exploiting the increased parallelism. Additionally, when the data servers, which typically are also computational nodes in a cluster environment, are loaded in an unbalanced way by applications running in the cluster, the read performance of PVFS will be degraded significantly. On the contrary, in the CEFT-PVFS, a heavily loaded data server can be skipped and all the desired data is read from its mirroring node. Thus the performance will not be affected unless both the server node and its mirroring node are heavily loaded.",
author = "Yifeng Zhu and Hong Jiang and Xiao Qin and Dan Feng and Swanson, {David R}",
year = "2003",
month = "12",
day = "1",
doi = "10.1109/CCGRID.2003.1199440",
language = "English (US)",
isbn = "0769519199",
series = "Proceedings - CCGrid 2003: 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid",
pages = "730--735",
booktitle = "Proceedings - CCGrid 2003",

}

TY - GEN

T1 - Improved read performance in a cost-effective, fault-tolerant parallel virtual file system (CEFT-PVFS)

AU - Zhu, Yifeng

AU - Jiang, Hong

AU - Qin, Xiao

AU - Feng, Dan

AU - Swanson, David R

PY - 2003/12/1

Y1 - 2003/12/1

N2 - Due to the ever-widening performance gap between processors and disks, I/O operations tend to become the major performance bottleneck of data-intensive applications on modern clusters. If all the existing disks on the nodes of a cluster are connected together to establish high performance parallel storage systems, the cluster's overall performance can be boosted at no additional cost. CEFT-PVFS (a RAID 10 style parallel file system that extends the original PVFS), as one such system, divides the cluster nodes into two groups, stripes the data across one group in a round-robin fashion, and then duplicates the same data to the other group to provide storage service of high performance and high reliability. Previous research has shown that the system reliability is improved by a factor of more than 40 with mirroring while maintaining a comparable write performance. This paper presents another benefit of CEFT-PVFS in which the aggregate peak read performance can be improved by as much as 100% over that of the original PVFS by exploiting the increased parallelism. Additionally, when the data servers, which typically are also computational nodes in a cluster environment, are loaded in an unbalanced way by applications running in the cluster, the read performance of PVFS will be degraded significantly. On the contrary, in the CEFT-PVFS, a heavily loaded data server can be skipped and all the desired data is read from its mirroring node. Thus the performance will not be affected unless both the server node and its mirroring node are heavily loaded.

AB - Due to the ever-widening performance gap between processors and disks, I/O operations tend to become the major performance bottleneck of data-intensive applications on modern clusters. If all the existing disks on the nodes of a cluster are connected together to establish high performance parallel storage systems, the cluster's overall performance can be boosted at no additional cost. CEFT-PVFS (a RAID 10 style parallel file system that extends the original PVFS), as one such system, divides the cluster nodes into two groups, stripes the data across one group in a round-robin fashion, and then duplicates the same data to the other group to provide storage service of high performance and high reliability. Previous research has shown that the system reliability is improved by a factor of more than 40 with mirroring while maintaining a comparable write performance. This paper presents another benefit of CEFT-PVFS in which the aggregate peak read performance can be improved by as much as 100% over that of the original PVFS by exploiting the increased parallelism. Additionally, when the data servers, which typically are also computational nodes in a cluster environment, are loaded in an unbalanced way by applications running in the cluster, the read performance of PVFS will be degraded significantly. On the contrary, in the CEFT-PVFS, a heavily loaded data server can be skipped and all the desired data is read from its mirroring node. Thus the performance will not be affected unless both the server node and its mirroring node are heavily loaded.

UR - http://www.scopus.com/inward/record.url?scp=84887507469&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84887507469&partnerID=8YFLogxK

U2 - 10.1109/CCGRID.2003.1199440

DO - 10.1109/CCGRID.2003.1199440

M3 - Conference contribution

SN - 0769519199

SN - 9780769519197

T3 - Proceedings - CCGrid 2003: 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid

SP - 730

EP - 735

BT - Proceedings - CCGrid 2003

ER -