SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN
CC BY
Lưu vào:
| Tác giả chính: | , , |
|---|---|
| Định dạng: | Sách |
| Ngôn ngữ: | English |
| Nhà xuất bản: |
Springer
2023
|
| Chủ đề: | |
| Truy cập trực tuyến: | https://link.springer.com/article/10.1007/s10766-022-00741-6 https://dlib.phenikaa-uni.edu.vn/handle/PNK/8234 |
| Từ khóa: |
Thêm từ khóa
Không có từ khóa, Hãy là người đầu tiên đánh dấu biểu ghi này!
|
| id |
oai:localhost:PNK-8234 |
|---|---|
| record_format |
dspace |
| spelling |
oai:localhost:PNK-82342023-04-24T02:02:17Z SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN Haoran, Wang Thibaut, Tachon Chong, Li DNNs DNN designers CC BY The increasing size of deep neural networks (DNNs) raises a high demand for distributed training. An expert could find good hybrid parallelism strategies, but designing suitable strategies is time and labor-consuming. Therefore, automating parallelism strategy generation is crucial and desirable for DNN designers. Some automatic searching approaches have recently been studied to free the experts from the heavy parallel strategy conception. However, these approaches all rely on a numerical cost model, which requires heavy profiling results that lack portability. These profiling-based approaches cannot lighten the strategy generation work due to the non-reusable profiling value. Our intuition is that there is no need to estimate the actual execution time of the distributed training but to compare the relative cost of different strategies. 2023-04-24T02:02:17Z 2023-04-24T02:02:17Z 2022 Book https://link.springer.com/article/10.1007/s10766-022-00741-6 https://dlib.phenikaa-uni.edu.vn/handle/PNK/8234 en application/pdf Springer |
| institution |
Digital Phenikaa |
| collection |
Digital Phenikaa |
| language |
English |
| topic |
DNNs DNN designers |
| spellingShingle |
DNNs DNN designers Haoran, Wang Thibaut, Tachon Chong, Li SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN |
| description |
CC BY |
| format |
Book |
| author |
Haoran, Wang Thibaut, Tachon Chong, Li |
| author_facet |
Haoran, Wang Thibaut, Tachon Chong, Li |
| author_sort |
Haoran, Wang |
| title |
SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN |
| title_short |
SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN |
| title_full |
SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN |
| title_fullStr |
SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN |
| title_full_unstemmed |
SMSG: Profiling-Free Parallelism Modeling for Distributed Training of DNN |
| title_sort |
smsg: profiling-free parallelism modeling for distributed training of dnn |
| publisher |
Springer |
| publishDate |
2023 |
| url |
https://link.springer.com/article/10.1007/s10766-022-00741-6 https://dlib.phenikaa-uni.edu.vn/handle/PNK/8234 |
| _version_ |
1764086838623993856 |
| score |
8.89252 |