Hierarchical memory-constrained operator scheduling of neural architecture search networks

Zihan Wang; Chengcheng Wan; Yuting Chen; Ziyi Lin; He Jiang; Lei Qiao

doi:10.1145/3489517.3530472

Hierarchical memory-constrained operator scheduling of neural architecture search networks

Zihan Wang, Chengcheng Wan, Yuting Chen, Ziyi Lin, He Jiang, Lei Qiao

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

12 Scopus citations

Abstract

Neural Architecture Search (NAS) is widely used in industry, searching for neural networks meeting task requirements. Meanwhile, it faces a challenge in scheduling networks satisfying memory constraints. This paper proposes HMCOS that performs hierarchical memory-constrained operator scheduling of NAS networks: given a network, HMCOS constructs a hierarchical computation graph and employs an iterative scheduling algorithm to progressively reduce peak memory footprints. We evaluate HMCOS against RPO and Serenity (two popular scheduling techniques). The results show that HMCOS outperforms existing techniques in supporting more NAS networks, reducing 8.7∼42.4% of peak memory footprints, and achieving 137 - 283x of speedups in scheduling.

Original language	English
Title of host publication	Proceedings of the 59th ACM/IEEE Design Automation Conference, DAC 2022
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	493-498
Number of pages	6
ISBN (Electronic)	9781450391429
DOIs	https://doi.org/10.1145/3489517.3530472
State	Published - 10 Jul 2022
Externally published	Yes
Event	59th ACM/IEEE Design Automation Conference, DAC 2022 - San Francisco, United States Duration: 10 Jul 2022 → 14 Jul 2022

Publication series

Name	Proceedings - Design Automation Conference
ISSN (Print)	0738-100X

Conference

Conference	59th ACM/IEEE Design Automation Conference, DAC 2022
Country/Territory	United States
City	San Francisco
Period	10/07/22 → 14/07/22

Access to Document

10.1145/3489517.3530472

Cite this

Wang, Z., Wan, C., Chen, Y., Lin, Z., Jiang, H., & Qiao, L. (2022). Hierarchical memory-constrained operator scheduling of neural architecture search networks. In Proceedings of the 59th ACM/IEEE Design Automation Conference, DAC 2022 (pp. 493-498). (Proceedings - Design Automation Conference). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1145/3489517.3530472

@inproceedings{d574f39c7cf248efb52341c810b02596,

title = "Hierarchical memory-constrained operator scheduling of neural architecture search networks",

abstract = "Neural Architecture Search (NAS) is widely used in industry, searching for neural networks meeting task requirements. Meanwhile, it faces a challenge in scheduling networks satisfying memory constraints. This paper proposes HMCOS that performs hierarchical memory-constrained operator scheduling of NAS networks: given a network, HMCOS constructs a hierarchical computation graph and employs an iterative scheduling algorithm to progressively reduce peak memory footprints. We evaluate HMCOS against RPO and Serenity (two popular scheduling techniques). The results show that HMCOS outperforms existing techniques in supporting more NAS networks, reducing 8.7∼42.4\% of peak memory footprints, and achieving 137 - 283x of speedups in scheduling.",

author = "Zihan Wang and Chengcheng Wan and Yuting Chen and Ziyi Lin and He Jiang and Lei Qiao",

note = "Publisher Copyright: {\textcopyright} 2022 ACM.; 59th ACM/IEEE Design Automation Conference, DAC 2022 ; Conference date: 10-07-2022 Through 14-07-2022",

year = "2022",

month = jul,

day = "10",

doi = "10.1145/3489517.3530472",

language = "英语",

series = "Proceedings - Design Automation Conference",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "493--498",

booktitle = "Proceedings of the 59th ACM/IEEE Design Automation Conference, DAC 2022",

address = "美国",

}

Wang, Z, Wan, C, Chen, Y, Lin, Z, Jiang, H & Qiao, L 2022, Hierarchical memory-constrained operator scheduling of neural architecture search networks. in Proceedings of the 59th ACM/IEEE Design Automation Conference, DAC 2022. Proceedings - Design Automation Conference, Institute of Electrical and Electronics Engineers Inc., pp. 493-498, 59th ACM/IEEE Design Automation Conference, DAC 2022, San Francisco, United States, 10/07/22. https://doi.org/10.1145/3489517.3530472

Hierarchical memory-constrained operator scheduling of neural architecture search networks. / Wang, Zihan; Wan, Chengcheng; Chen, Yuting et al.
Proceedings of the 59th ACM/IEEE Design Automation Conference, DAC 2022. Institute of Electrical and Electronics Engineers Inc., 2022. p. 493-498 (Proceedings - Design Automation Conference).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Hierarchical memory-constrained operator scheduling of neural architecture search networks

AU - Wang, Zihan

AU - Wan, Chengcheng

AU - Chen, Yuting

AU - Lin, Ziyi

AU - Jiang, He

AU - Qiao, Lei

PY - 2022/7/10

Y1 - 2022/7/10

N2 - Neural Architecture Search (NAS) is widely used in industry, searching for neural networks meeting task requirements. Meanwhile, it faces a challenge in scheduling networks satisfying memory constraints. This paper proposes HMCOS that performs hierarchical memory-constrained operator scheduling of NAS networks: given a network, HMCOS constructs a hierarchical computation graph and employs an iterative scheduling algorithm to progressively reduce peak memory footprints. We evaluate HMCOS against RPO and Serenity (two popular scheduling techniques). The results show that HMCOS outperforms existing techniques in supporting more NAS networks, reducing 8.7∼42.4% of peak memory footprints, and achieving 137 - 283x of speedups in scheduling.

AB - Neural Architecture Search (NAS) is widely used in industry, searching for neural networks meeting task requirements. Meanwhile, it faces a challenge in scheduling networks satisfying memory constraints. This paper proposes HMCOS that performs hierarchical memory-constrained operator scheduling of NAS networks: given a network, HMCOS constructs a hierarchical computation graph and employs an iterative scheduling algorithm to progressively reduce peak memory footprints. We evaluate HMCOS against RPO and Serenity (two popular scheduling techniques). The results show that HMCOS outperforms existing techniques in supporting more NAS networks, reducing 8.7∼42.4% of peak memory footprints, and achieving 137 - 283x of speedups in scheduling.

UR - https://www.scopus.com/pages/publications/85137537213

U2 - 10.1145/3489517.3530472

DO - 10.1145/3489517.3530472

M3 - 会议稿件

AN - SCOPUS:85137537213

T3 - Proceedings - Design Automation Conference

SP - 493

EP - 498

BT - Proceedings of the 59th ACM/IEEE Design Automation Conference, DAC 2022

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 59th ACM/IEEE Design Automation Conference, DAC 2022

Y2 - 10 July 2022 through 14 July 2022

ER -

Hierarchical memory-constrained operator scheduling of neural architecture search networks

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this