Cover Image

SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN

CC BY

Saved in:
Bibliographic Details
Main Authors: Haoran, Wang, Thibaut, Tachon, Chong, Li
Format: Book
Language:English
Published: Springer 2023
Subjects:
DNNs
DNN designers
Online Access:https://link.springer.com/article/10.1007/s10766-022-00741-6
https://dlib.phenikaa-uni.edu.vn/handle/PNK/8234
Tags: Add Tag
No Tags, Be the first to tag this record!
id oai:localhost:PNK-8234
record_format dspace
spelling oai:localhost:PNK-82342023-04-24T02:02:17Z SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN Haoran, Wang Thibaut, Tachon Chong, Li DNNs DNN designers CC BY The increasing size of deep neural networks (DNNs) raises a high demand for distributed training. An expert could find good hybrid parallelism strategies, but designing suitable strategies is time and labor-consuming. Therefore, automating parallelism strategy generation is crucial and desirable for DNN designers. Some automatic searching approaches have recently been studied to free the experts from the heavy parallel strategy conception. However, these approaches all rely on a numerical cost model, which requires heavy profiling results that lack portability. These profiling-based approaches cannot lighten the strategy generation work due to the non-reusable profiling value. Our intuition is that there is no need to estimate the actual execution time of the distributed training but to compare the relative cost of different strategies. 2023-04-24T02:02:17Z 2023-04-24T02:02:17Z 2022 Book https://link.springer.com/article/10.1007/s10766-022-00741-6 https://dlib.phenikaa-uni.edu.vn/handle/PNK/8234 en application/pdf Springer
institution Digital Phenikaa
collection Digital Phenikaa
language English
topic DNNs
DNN designers
spellingShingle DNNs
DNN designers
Haoran, Wang
Thibaut, Tachon
Chong, Li
SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN
description CC BY
format Book
author Haoran, Wang
Thibaut, Tachon
Chong, Li
author_facet Haoran, Wang
Thibaut, Tachon
Chong, Li
author_sort Haoran, Wang
title SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN
title_short SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN
title_full SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN
title_fullStr SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN
title_full_unstemmed SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN
title_sort smsg: profiling-free parallelism modeling for distributed training of dnn
publisher Springer
publishDate 2023
url https://link.springer.com/article/10.1007/s10766-022-00741-6
https://dlib.phenikaa-uni.edu.vn/handle/PNK/8234
_version_ 1764086838623993856
score 8.881002

Similar Items