{"id":3169,"date":"2024-05-07T18:41:44","date_gmt":"2024-05-07T13:41:44","guid":{"rendered":"https:\/\/afzalbadshah.com\/?p=3169"},"modified":"2024-05-08T22:14:46","modified_gmt":"2024-05-08T17:14:46","slug":"parallel-summation-using-mpi-in-python-with-mpi4py","status":"publish","type":"post","link":"https:\/\/afzalbadshah.com\/index.php\/2024\/05\/07\/parallel-summation-using-mpi-in-python-with-mpi4py\/","title":{"rendered":"Parallel Summation using MPI in Python with mpi4py"},"content":{"rendered":"\n

Parallel summation involves distributing the task of summing a large set of numbers across multiple processors or computing nodes, enabling simultaneous computation and aggregation of partial results. Each processor handles a portion of the data, performs local summation, and then communicates its partial sum to a designated root processor. The root processor collects and combines these partial sums to compute the global sum, thereby leveraging parallelism to accelerate the computation process and efficiently handle large-scale data sets. In this tutorial, we explore the implementation of parallel summation using MPI in Python, demonstrating how MPI facilitates communication and coordination among processes to achieve efficient parallel computation. Visit the detailed tutorial on MPI in Python here. <\/a><\/p>\n\n\n\n

Code Example<\/h3>\n\n\n\n
from mpi4py import MPI\n\ncomm = MPI.COMM_WORLD\nsize = comm.Get_size()\nrank = comm.Get_rank()\n\n# Define the total number of elements\nN = 100\n\n# Calculate the number of elements to be handled by each process\nchunk_size = N \/\/ size\nstart = rank * chunk_size\nend = start + chunk_size\n\n# Perform local summation for each process\nlocal_sum = sum(range(start + 1, end + 1))\n\nprint(\"The processor\", rank, \"local sum is:\", local_sum)\n# Gather all local sums to the root process (rank 0)\nglobal_sum = comm.reduce(local_sum, op=MPI.SUM, root=0)\n\n# Print the result at the root process\nif rank == 0:\n    print(\"Global sum:\", global_sum)<\/code><\/pre>\n\n\n\n
\"\"
Output of the program<\/figcaption><\/figure>\n\n\n\n
Explanation<\/h3>\n\n\n\n
Import MPI Module and Initialize MPI Environment<\/strong><\/p>\n\n\n\n
from mpi4py import MPI<\/code><\/pre>\n\n\n\n
This line imports the MPI module from the mpi4py package, enabling the use of MPI functionalities.<\/p>\n\n\n\n
comm = MPI.COMM_WORLD\nsize = comm.Get_size()\nrank = comm.Get_rank()<\/code><\/pre>\n\n\n\n
These lines initialize the MPI environment. MPI.COMM_WORLD<\/code> creates a communicator object representing all processes in the MPI world. comm.Get_size()<\/code> returns the number of processes in the communicator, and comm.Get_rank()<\/code> returns the rank of the current process in the communicator.<\/p>\n\n\n\n
Define the Total Number of Elements<\/strong><\/p>\n\n\n\n
N = 100<\/code><\/pre>\n\n\n\n
This line defines the total number of elements to be processed.<\/p>\n\n\n\n
Calculate Chunk Size and Local Range<\/strong><\/p>\n\n\n\n
chunk_size = N \/ size\nstart = rank * chunk_size\nend = start + chunk_size<\/code><\/pre>\n\n\n\n
These lines calculate the number of elements to be handled by each process (chunk_size<\/code>) and determine the start and end indices of the local range of elements to be processed by the current process.<\/p>\n\n\n\n
Perform Local Summation<\/strong><\/p>\n\n\n\n
local_sum = sum(range(start + 1, end + 1))<\/code><\/pre>\n\n\n\n
This line performs the local summation of elements for each process. It creates a range of numbers starting from start + 1<\/code> (since we want to include start<\/code>) to end<\/code>, and then sums up these numbers using the sum<\/code> function.<\/p>\n\n\n\n
Gather Local Sums to Root Process<\/strong><\/p>\n\n\n\n
global_sum = comm.reduce(local_sum, op=MPI.SUM, root=0)<\/code><\/pre>\n\n\n\n
This line gathers all the local sums from each process and performs a reduction operation. The local sums are reduced using the MPI sum operation (MPI.SUM<\/code>). The resulting global sum is stored in the variable global_sum<\/code> and is only computed by the root process (rank 0).<\/p>\n\n\n\n
Print Result at Root Process<\/strong><\/p>\n\n\n\n
if rank == 0:\n    print(\"Global sum:\", global_sum)<\/code><\/pre>\n\n\n\n
This conditional statement ensures that only the root process (rank 0) prints the final result. Other processes contribute their local sums but do not print the result.<\/p>\n\n\n\n
\n