
Blocking and Non-blocking Communication in MPI
In parallel computing with MPI (Message Passing Interface), communication between processes plays a crucial role in achieving efficient parallelization of algorithms. Two common approaches to communication are blocking and non-blocking communication. You can visit the detailed tutorial on MPI with Python here.
Blocking Communication
Blocking communication involves processes halting their execution until the communication operation is complete. In MPI, blocking communication functions like comm.send()
and comm.recv()
ensure that the sender waits until the receiver receives the message, and vice versa. Blocking communication is often used when processes need to synchronize their execution or when the sender and receiver must coordinate closely. While blocking communication simplifies program logic and synchronization, it can lead to potential performance bottlenecks if processes spend significant time waiting for communication to complete. Let’s see the below code;
from mpi4py import MPI
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
if rank == 0:
data = {'a': 7, 'b': 3.14}
comm.send(data, dest=1, tag=11)
elif rank == 1:
data = comm.recv(source=0, tag=11)
Explanation
- Import MPI: The code begins by importing the MPI module from mpi4py library, which provides MPI functionalities for Python programs.
- Initialize MPI Communicator: The code initializes the MPI communicator
comm
representing all processes participating in the computation. - Get Rank: Each process in the communicator obtains its rank using
comm.Get_rank()
to determine its identity in the communicator. - Conditional Execution: Depending on the rank of the process:
- If the rank is 0:
- Create a Python dictionary
data
containing some sample data. - Use
comm.send()
to send the data to process 1 (dest=1
) with a specified tag (tag=11
).
- Create a Python dictionary
- If the rank is 1:
- Use
comm.recv()
to receive data from process 0 (source=0
) with the specified tag (tag=11
). The received data is stored in thedata
variable.
- Use
- If the rank is 0:
- Blocking Communication: Both
comm.send()
andcomm.recv()
are blocking operations. This means that the sender (comm.send()
) will be blocked until the receiver (comm.recv()
) receives the message, and vice versa. - Data Transfer: In this program, the dictionary
data
is sent from process 0 to process 1 using blocking communication. Process 1 waits to receive the data sent by process 0 before continuing its execution.
Non-blocking Communication
Non-blocking communication, on the other hand, allows processes to continue their execution immediately after initiating communication operations, without waiting for the operations to complete. In MPI, non-blocking communication functions like comm.isend()
and comm.irecv()
return a request object immediately, enabling processes to overlap computation with communication. Non-blocking communication is particularly useful in scenarios where processes can perform useful work while waiting for communication to progress. By overlapping computation with communication, non-blocking communication can improve overall performance and scalability in parallel applications. Let’s see the below code;
from mpi4py import MPI
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
if rank == 0:
data = {'a': 7, 'b': 3.14}
req = comm.isend(data, dest=1, tag=11)
req.wait()
elif rank == 1:
req = comm.irecv(source=0, tag=11)
data = req.wait()
Explanation
- Import MPI: Similar to the blocking communication program, this code starts by importing the MPI module from mpi4py library.
- Initialize MPI Communicator and Get Rank: The MPI communicator
comm
is initialized, and the rank of the process is obtained usingcomm.Get_rank()
. - Conditional Execution: Depending on the rank of the process:
- If the rank is 0:
- Create a Python dictionary
data
containing some sample data. - Use
comm.isend()
to initiate the non-blocking sending of the data to process 1 (dest=1
) with a specified tag (tag=11
). The request objectreq
is returned. - Wait for the completion of the send operation using
req.wait()
.
- Create a Python dictionary
- If the rank is 1:
- Use
comm.irecv()
to initiate the non-blocking receiving of data from process 0 (source=0
) with the specified tag (tag=11
). The request objectreq
is returned. - Wait for the completion of the receive operation using
req.wait()
. The received data is stored in thedata
variable.
- Use
- If the rank is 0:
- Non-blocking Communication: In contrast to blocking communication, non-blocking communication operations (
comm.isend()
andcomm.irecv()
) do not block the execution of the process. Instead, they return a request object immediately, allowing the process to perform other tasks while the communication operation progresses asynchronously. - Data Transfer: Similarly, the dictionary
data
is sent from process 0 to process 1, but this time using non-blocking communication. Process 1 initiates the receive operation and waits for the data to be received asynchronously.
12 thoughts on “Blocking and Non-blocking Communication in MPI”
That is a very good tip especially to those new to the blogosphere. Short but very accurate information… Many thanks for sharing this one. A must read article! http://www.kayswell.com
Very nice post. I just stumbled upon your weblog and wished to say that I’ve really enjoyed browsing your blog posts. After all I’ll be subscribing to your feed and I hope you write again soon! http://www.kayswell.com
I’m not sure where you’re getting your info, but great topic. I needs to spend some time learning more or understanding more. Thanks for great info I was looking for this information for my mission. http://www.kayswell.com
Right now it sounds like WordPress is the preferred blogging platform out there right now. (from what I’ve read) Is that what you are using on your blog? http://www.kayswell.com
Hello mates, nice article and fastidious urging commented here, I am actually enjoying by these. http://www.kayswell.com
I used to be able to find good advice from your blog articles. http://www.kayswell.com
Hey I know this is off topic but I was wondering if you knew of any widgetsI could add to my blog that automatically tweet my newest twitter updates. http://www.kayswell.com
great points altogether, you simply received a logo new reader. What might you suggest in regards to your publish that you just made a few days in the past? Any positive?
Aw, this was a very nice post. Taking the time and actual effort to create a great article… but what can I say… I hesitate a whole lot and never seem to get nearly anything done. http://www.kayswell.com
Hi, Neat post. There is a problem with your site in web explorer, might test this? IE still is the market chief and a large component of other people will leave out your wonderful writing because of this problem.
Hi! I know this is kinda off topic but I was wondering if you knew where I could find a captcha plugin for my comment form? I’m using the same blog platform as yours and I’m having difficulty finding one? Thanks a lot! http://www.kayswell.com
Nice post. I learn something totally new and challenging on blogs I stumbleupon on a daily basis.It will always be useful to read through content from other authors and use a little something from other websites.