Efficient scheduling with intensive in-memory file accesses considering bandwidth constraint on memory bus

Lin Wu; Qingfeng Zhuge; Edwin H.M. Sha; Zhilong Sun

doi:10.1007/978-3-319-27122-4_39

Efficient scheduling with intensive in-memory file accesses considering bandwidth constraint on memory bus

Lin Wu, Qingfeng Zhuge, Edwin H.M. Sha, Zhilong Sun

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

The latest trend in high performance computing and big data applications inspires new solutions of in-memory filesystems to achieve high throughput of file data accesses. However, new concerns arise because intensive in-memory file accesses can cause contention problem on memory bus. With existing schedulers, such as Complete Fair Scheduler (CFS) in Linux system, memory access intensive processes can excessively occupy memory bandwidth. As a result, the performance of a set of running processes is degraded. In this paper, we propose a scheduling algorithm to consider memory bandwidth consumption during scheduling and improve the completion time of a given working set. We first present the data to show that CFS can not efficiently handle a set of running processes involving intensive in-memory file accesses. Then, a heuristic algorithm is proposed to solve the scheduling problem with bandwidth consumption and execution time consideration. The scheduling algorithm is implemented and extensively experimented in Linux system. The experimental results show that the completion time is reduced up to 33.3% compared with that generated by Linux scheduler CFS.

Original language	English
Title of host publication	Algorithms and Architectures for Parallel Processing - 15th International Conference, ICA3PP 2015, Proceedings
Editors	Albert Zomaya, Kenli Li, Gregorio Martinez Perez, Guojun Wang
Publisher	Springer Verlag
Pages	567-580
Number of pages	14
ISBN (Print)	9783319271217
DOIs	https://doi.org/10.1007/978-3-319-27122-4_39
State	Published - 2015
Externally published	Yes
Event	15th International Conference on Algorithms and Architectures for Parallel Processing, ICA3PP 2015 - Zhangjiajie, China Duration: 18 Nov 2015 → 20 Nov 2015

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	9529
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	15th International Conference on Algorithms and Architectures for Parallel Processing, ICA3PP 2015
Country/Territory	China
City	Zhangjiajie
Period	18/11/15 → 20/11/15

Keywords

Bandwidth contention
Completion time
IO path
In-Memory filesystem
Scheduling

Access to Document

10.1007/978-3-319-27122-4_39

Cite this

Wu, L., Zhuge, Q., Sha, E. H. M., & Sun, Z. (2015). Efficient scheduling with intensive in-memory file accesses considering bandwidth constraint on memory bus. In A. Zomaya, K. Li, G. M. Perez, & G. Wang (Eds.), Algorithms and Architectures for Parallel Processing - 15th International Conference, ICA3PP 2015, Proceedings (pp. 567-580). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9529). Springer Verlag. https://doi.org/10.1007/978-3-319-27122-4_39

Wu, Lin ; Zhuge, Qingfeng ; Sha, Edwin H.M. et al. / Efficient scheduling with intensive in-memory file accesses considering bandwidth constraint on memory bus. Algorithms and Architectures for Parallel Processing - 15th International Conference, ICA3PP 2015, Proceedings. editor / Albert Zomaya ; Kenli Li ; Gregorio Martinez Perez ; Guojun Wang. Springer Verlag, 2015. pp. 567-580 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{0599d0dafe4c4b6e83ce27099c4fded0,

title = "Efficient scheduling with intensive in-memory file accesses considering bandwidth constraint on memory bus",

abstract = "The latest trend in high performance computing and big data applications inspires new solutions of in-memory filesystems to achieve high throughput of file data accesses. However, new concerns arise because intensive in-memory file accesses can cause contention problem on memory bus. With existing schedulers, such as Complete Fair Scheduler (CFS) in Linux system, memory access intensive processes can excessively occupy memory bandwidth. As a result, the performance of a set of running processes is degraded. In this paper, we propose a scheduling algorithm to consider memory bandwidth consumption during scheduling and improve the completion time of a given working set. We first present the data to show that CFS can not efficiently handle a set of running processes involving intensive in-memory file accesses. Then, a heuristic algorithm is proposed to solve the scheduling problem with bandwidth consumption and execution time consideration. The scheduling algorithm is implemented and extensively experimented in Linux system. The experimental results show that the completion time is reduced up to 33.3\% compared with that generated by Linux scheduler CFS.",

keywords = "Bandwidth contention, Completion time, IO path, In-Memory filesystem, Scheduling",

author = "Lin Wu and Qingfeng Zhuge and Sha, \{Edwin H.M.\} and Zhilong Sun",

note = "Publisher Copyright: {\textcopyright} Springer International Publishing Switzerland 2015.; 15th International Conference on Algorithms and Architectures for Parallel Processing, ICA3PP 2015 ; Conference date: 18-11-2015 Through 20-11-2015",

year = "2015",

doi = "10.1007/978-3-319-27122-4\_39",

language = "英语",

isbn = "9783319271217",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "567--580",

editor = "Albert Zomaya and Kenli Li and Perez, \{Gregorio Martinez\} and Guojun Wang",

booktitle = "Algorithms and Architectures for Parallel Processing - 15th International Conference, ICA3PP 2015, Proceedings",

address = "德国",

}

Wu, L, Zhuge, Q , Sha, EHM & Sun, Z 2015, Efficient scheduling with intensive in-memory file accesses considering bandwidth constraint on memory bus. in A Zomaya, K Li, GM Perez & G Wang (eds), Algorithms and Architectures for Parallel Processing - 15th International Conference, ICA3PP 2015, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 9529, Springer Verlag, pp. 567-580, 15th International Conference on Algorithms and Architectures for Parallel Processing, ICA3PP 2015, Zhangjiajie, China, 18/11/15. https://doi.org/10.1007/978-3-319-27122-4_39

Efficient scheduling with intensive in-memory file accesses considering bandwidth constraint on memory bus. / Wu, Lin; Zhuge, Qingfeng ; Sha, Edwin H.M. et al.
Algorithms and Architectures for Parallel Processing - 15th International Conference, ICA3PP 2015, Proceedings. ed. / Albert Zomaya; Kenli Li; Gregorio Martinez Perez; Guojun Wang. Springer Verlag, 2015. p. 567-580 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 9529).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Efficient scheduling with intensive in-memory file accesses considering bandwidth constraint on memory bus

AU - Wu, Lin

AU - Zhuge, Qingfeng

AU - Sha, Edwin H.M.

AU - Sun, Zhilong

N1 - Publisher Copyright: © Springer International Publishing Switzerland 2015.

PY - 2015

Y1 - 2015

N2 - The latest trend in high performance computing and big data applications inspires new solutions of in-memory filesystems to achieve high throughput of file data accesses. However, new concerns arise because intensive in-memory file accesses can cause contention problem on memory bus. With existing schedulers, such as Complete Fair Scheduler (CFS) in Linux system, memory access intensive processes can excessively occupy memory bandwidth. As a result, the performance of a set of running processes is degraded. In this paper, we propose a scheduling algorithm to consider memory bandwidth consumption during scheduling and improve the completion time of a given working set. We first present the data to show that CFS can not efficiently handle a set of running processes involving intensive in-memory file accesses. Then, a heuristic algorithm is proposed to solve the scheduling problem with bandwidth consumption and execution time consideration. The scheduling algorithm is implemented and extensively experimented in Linux system. The experimental results show that the completion time is reduced up to 33.3% compared with that generated by Linux scheduler CFS.

AB - The latest trend in high performance computing and big data applications inspires new solutions of in-memory filesystems to achieve high throughput of file data accesses. However, new concerns arise because intensive in-memory file accesses can cause contention problem on memory bus. With existing schedulers, such as Complete Fair Scheduler (CFS) in Linux system, memory access intensive processes can excessively occupy memory bandwidth. As a result, the performance of a set of running processes is degraded. In this paper, we propose a scheduling algorithm to consider memory bandwidth consumption during scheduling and improve the completion time of a given working set. We first present the data to show that CFS can not efficiently handle a set of running processes involving intensive in-memory file accesses. Then, a heuristic algorithm is proposed to solve the scheduling problem with bandwidth consumption and execution time consideration. The scheduling algorithm is implemented and extensively experimented in Linux system. The experimental results show that the completion time is reduced up to 33.3% compared with that generated by Linux scheduler CFS.

KW - Bandwidth contention

KW - Completion time

KW - IO path

KW - In-Memory filesystem

KW - Scheduling

UR - https://www.scopus.com/pages/publications/84952006134

U2 - 10.1007/978-3-319-27122-4_39

DO - 10.1007/978-3-319-27122-4_39

M3 - 会议稿件

AN - SCOPUS:84952006134

SN - 9783319271217

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 567

EP - 580

BT - Algorithms and Architectures for Parallel Processing - 15th International Conference, ICA3PP 2015, Proceedings

A2 - Zomaya, Albert

A2 - Li, Kenli

A2 - Perez, Gregorio Martinez

A2 - Wang, Guojun

PB - Springer Verlag

T2 - 15th International Conference on Algorithms and Architectures for Parallel Processing, ICA3PP 2015

Y2 - 18 November 2015 through 20 November 2015

ER -

Wu L, Zhuge Q , Sha EHM, Sun Z. Efficient scheduling with intensive in-memory file accesses considering bandwidth constraint on memory bus. In Zomaya A, Li K, Perez GM, Wang G, editors, Algorithms and Architectures for Parallel Processing - 15th International Conference, ICA3PP 2015, Proceedings. Springer Verlag. 2015. p. 567-580. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-27122-4_39

Efficient scheduling with intensive in-memory file accesses considering bandwidth constraint on memory bus

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this