A transmission optimization method for MPI communications

In recent years, MPI has been widely used as a communication protocol for massively parallel computing tasks, and the performance of MPI interprocess communications has become a major constraint for large-scale scalability. By analyzing the performance characteristics of bandwidth and latency of MPI...

Full description

Saved in:
Bibliographic Details
Published in:The Journal of supercomputing 2024-03, Vol.80 (5), p.6240-6263
Main Authors: Wang, Jubin, Zhuang, Yuan, Zeng, Yunhui
Format: Article
Language:English
Subjects:
Communication
Compilers
Computer Science
Interpreters
Numerical models
Optimization
Processor Architectures
Programming Languages
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
cited_by
cites cdi_FETCH-LOGICAL-c270t-d2b21cdf4fc5ea126f7e1a48e023823177543efb0bb1a06c9970b87db1b455513
container_end_page 6263
container_issue 5
container_start_page 6240
container_title The Journal of supercomputing
container_volume 80
creator Wang, Jubin
Zhuang, Yuan
Zeng, Yunhui
description In recent years, MPI has been widely used as a communication protocol for massively parallel computing tasks, and the performance of MPI interprocess communications has become a major constraint for large-scale scalability. By analyzing the performance characteristics of bandwidth and latency of MPI communications, a transmission optimization method for MPI communications is proposed. For the variables of transmitted data, the communication strategy of MPI is optimized according to the data size and the succession of multiple communications, and the operation of packing or unpacking is automatically selected, which makes the performance of MPI communications significantly improved. For the time-consuming parts of MPI communication in the ocean numerical model Parallel Ocean Program with this method used, at least 2.4x speedup in point-to-point communication with unpacking strategy and at least 1.7x speedup in point-to-point with packing strategy are achieved. By automating file scans and analysis, 1.6x speedup is achieved for some point-to-point communications.
doi_str_mv 10.1007/s11227-023-05699-x
format article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_journals_2956212815</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2956212815</sourcerecordid><originalsourceid>FETCH-LOGICAL-c270t-d2b21cdf4fc5ea126f7e1a48e023823177543efb0bb1a06c9970b87db1b455513</originalsourceid><addsrcrecordid>eNp9UMtKxDAUDaLgOPoDrgquo_feJk27HAYfAyO60HXoI9EOthmTFka_3sxUcOfqcrjnxWHsEuEaAdRNQCRSHCjlILOi4LsjNkOpIhS5OGYzKAh4LgWdsrMQNgAgUpXOWLFIBl_2oWtDaF2fuO3Qdu13OexBZ4Z31yTW-eTxeZXUruvGvq0Pz3DOTmz5EczF752z17vbl-UDXz_dr5aLNa9JwcAbqgjrxgpbS1MiZVYZLEVuYtecUlRKitTYCqoKS8jqolBQ5aqpsBJSSkzn7Gry3Xr3OZow6I0bfR8jNRUyI6QcZWTRxKq9C8Ebq7e-7Ur_pRH0fiI9TaRjrD5MpHdRlE6iEMn9m_F_1v-ofgAspWmA</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2956212815</pqid></control><display><type>article</type><title>A transmission optimization method for MPI communications</title><source>Springer Nature</source><creator>Wang, Jubin ; Zhuang, Yuan ; Zeng, Yunhui</creator><creatorcontrib>Wang, Jubin ; Zhuang, Yuan ; Zeng, Yunhui</creatorcontrib><description>In recent years, MPI has been widely used as a communication protocol for massively parallel computing tasks, and the performance of MPI interprocess communications has become a major constraint for large-scale scalability. By analyzing the performance characteristics of bandwidth and latency of MPI communications, a transmission optimization method for MPI communications is proposed. For the variables of transmitted data, the communication strategy of MPI is optimized according to the data size and the succession of multiple communications, and the operation of packing or unpacking is automatically selected, which makes the performance of MPI communications significantly improved. For the time-consuming parts of MPI communication in the ocean numerical model Parallel Ocean Program with this method used, at least 2.4x speedup in point-to-point communication with unpacking strategy and at least 1.7x speedup in point-to-point with packing strategy are achieved. By automating file scans and analysis, 1.6x speedup is achieved for some point-to-point communications.</description><identifier>ISSN: 0920-8542</identifier><identifier>EISSN: 1573-0484</identifier><identifier>DOI: 10.1007/s11227-023-05699-x</identifier><language>eng</language><publisher>New York: Springer US</publisher><subject>Communication ; Compilers ; Computer Science ; Interpreters ; Numerical models ; Optimization ; Processor Architectures ; Programming Languages</subject><ispartof>The Journal of supercomputing, 2024-03, Vol.80 (5), p.6240-6263</ispartof><rights>The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c270t-d2b21cdf4fc5ea126f7e1a48e023823177543efb0bb1a06c9970b87db1b455513</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,780,784,27923,27924</link.rule.ids></links><search><creatorcontrib>Wang, Jubin</creatorcontrib><creatorcontrib>Zhuang, Yuan</creatorcontrib><creatorcontrib>Zeng, Yunhui</creatorcontrib><title>A transmission optimization method for MPI communications</title><title>The Journal of supercomputing</title><addtitle>J Supercomput</addtitle><description>In recent years, MPI has been widely used as a communication protocol for massively parallel computing tasks, and the performance of MPI interprocess communications has become a major constraint for large-scale scalability. By analyzing the performance characteristics of bandwidth and latency of MPI communications, a transmission optimization method for MPI communications is proposed. For the variables of transmitted data, the communication strategy of MPI is optimized according to the data size and the succession of multiple communications, and the operation of packing or unpacking is automatically selected, which makes the performance of MPI communications significantly improved. For the time-consuming parts of MPI communication in the ocean numerical model Parallel Ocean Program with this method used, at least 2.4x speedup in point-to-point communication with unpacking strategy and at least 1.7x speedup in point-to-point with packing strategy are achieved. By automating file scans and analysis, 1.6x speedup is achieved for some point-to-point communications.</description><subject>Communication</subject><subject>Compilers</subject><subject>Computer Science</subject><subject>Interpreters</subject><subject>Numerical models</subject><subject>Optimization</subject><subject>Processor Architectures</subject><subject>Programming Languages</subject><issn>0920-8542</issn><issn>1573-0484</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><recordid>eNp9UMtKxDAUDaLgOPoDrgquo_feJk27HAYfAyO60HXoI9EOthmTFka_3sxUcOfqcrjnxWHsEuEaAdRNQCRSHCjlILOi4LsjNkOpIhS5OGYzKAh4LgWdsrMQNgAgUpXOWLFIBl_2oWtDaF2fuO3Qdu13OexBZ4Z31yTW-eTxeZXUruvGvq0Pz3DOTmz5EczF752z17vbl-UDXz_dr5aLNa9JwcAbqgjrxgpbS1MiZVYZLEVuYtecUlRKitTYCqoKS8jqolBQ5aqpsBJSSkzn7Gry3Xr3OZow6I0bfR8jNRUyI6QcZWTRxKq9C8Ebq7e-7Ur_pRH0fiI9TaRjrD5MpHdRlE6iEMn9m_F_1v-ofgAspWmA</recordid><startdate>20240301</startdate><enddate>20240301</enddate><creator>Wang, Jubin</creator><creator>Zhuang, Yuan</creator><creator>Zeng, Yunhui</creator><general>Springer US</general><general>Springer Nature B.V</general><scope>AAYXX</scope><scope>CITATION</scope></search><sort><creationdate>20240301</creationdate><title>A transmission optimization method for MPI communications</title><author>Wang, Jubin ; Zhuang, Yuan ; Zeng, Yunhui</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c270t-d2b21cdf4fc5ea126f7e1a48e023823177543efb0bb1a06c9970b87db1b455513</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Communication</topic><topic>Compilers</topic><topic>Computer Science</topic><topic>Interpreters</topic><topic>Numerical models</topic><topic>Optimization</topic><topic>Processor Architectures</topic><topic>Programming Languages</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Wang, Jubin</creatorcontrib><creatorcontrib>Zhuang, Yuan</creatorcontrib><creatorcontrib>Zeng, Yunhui</creatorcontrib><collection>CrossRef</collection><jtitle>The Journal of supercomputing</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Wang, Jubin</au><au>Zhuang, Yuan</au><au>Zeng, Yunhui</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>A transmission optimization method for MPI communications</atitle><jtitle>The Journal of supercomputing</jtitle><stitle>J Supercomput</stitle><date>2024-03-01</date><risdate>2024</risdate><volume>80</volume><issue>5</issue><spage>6240</spage><epage>6263</epage><pages>6240-6263</pages><issn>0920-8542</issn><eissn>1573-0484</eissn><abstract>In recent years, MPI has been widely used as a communication protocol for massively parallel computing tasks, and the performance of MPI interprocess communications has become a major constraint for large-scale scalability. By analyzing the performance characteristics of bandwidth and latency of MPI communications, a transmission optimization method for MPI communications is proposed. For the variables of transmitted data, the communication strategy of MPI is optimized according to the data size and the succession of multiple communications, and the operation of packing or unpacking is automatically selected, which makes the performance of MPI communications significantly improved. For the time-consuming parts of MPI communication in the ocean numerical model Parallel Ocean Program with this method used, at least 2.4x speedup in point-to-point communication with unpacking strategy and at least 1.7x speedup in point-to-point with packing strategy are achieved. By automating file scans and analysis, 1.6x speedup is achieved for some point-to-point communications.</abstract><cop>New York</cop><pub>Springer US</pub><doi>10.1007/s11227-023-05699-x</doi><tpages>24</tpages></addata></record>
fulltext fulltext
identifier ISSN: 0920-8542
ispartof The Journal of supercomputing, 2024-03, Vol.80 (5), p.6240-6263
issn 0920-8542
1573-0484
language eng
recordid cdi_proquest_journals_2956212815
source Springer Nature
subjects Communication
Compilers
Computer Science
Interpreters
Numerical models
Optimization
Processor Architectures
Programming Languages
title A transmission optimization method for MPI communications
url http://sfxeu10.hosted.exlibrisgroup.com/loughborough?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-13T03%3A43%3A18IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=A%20transmission%20optimization%20method%20for%20MPI%20communications&rft.jtitle=The%20Journal%20of%20supercomputing&rft.au=Wang,%20Jubin&rft.date=2024-03-01&rft.volume=80&rft.issue=5&rft.spage=6240&rft.epage=6263&rft.pages=6240-6263&rft.issn=0920-8542&rft.eissn=1573-0484&rft_id=info:doi/10.1007/s11227-023-05699-x&rft_dat=%3Cproquest_cross%3E2956212815%3C/proquest_cross%3E%3Cgrp_id%3Ecdi_FETCH-LOGICAL-c270t-d2b21cdf4fc5ea126f7e1a48e023823177543efb0bb1a06c9970b87db1b455513%3C/grp_id%3E%3Coa%3E%3C/oa%3E%3Curl%3E%3C/url%3E&rft_id=info:oai/&rft_pqid=2956212815&rft_id=info:pmid/&rfr_iscdi=true