{"id":3211,"date":"2024-05-09T07:55:15","date_gmt":"2024-05-09T02:55:15","guid":{"rendered":"https:\/\/afzalbadshah.com\/?p=3211"},"modified":"2024-05-09T08:01:46","modified_gmt":"2024-05-09T03:01:46","slug":"matrix-multiplication-on-multi-processors-mpi4py","status":"publish","type":"post","link":"https:\/\/afzalbadshah.com\/index.php\/2024\/05\/09\/matrix-multiplication-on-multi-processors-mpi4py\/","title":{"rendered":"Matrix Multiplication on Multi-Processors: MPI4PY"},"content":{"rendered":"

\n\n\n

<\/p>\n\n\n<\/p>\n

\n\n\n<\/p>\n

In this scenario, each processor handles a portion of the matrices, performing computations independently, and then the results are combined to obtain the final result. This parallelization technique leverages the capabilities of multiple processors to expedite the overall computation time. <\/p>\n

\n\n\n<\/p>\n

\n\n\n<\/p>\n

Code:<\/strong><\/p>\n

\n\n\n<\/p>\n

\n\n\n<\/p>\n

from mpi4py import MPI\nimport numpy as np\n# Function to perform matrix multiplication\ndef matrix_multiply(A, B):\n    C = np.zeros((A.shape[0], B.shape[1]))\n    for i in range(A.shape[0]):\n        for j in range(B.shape[1]):\n            for k in range(A.shape[1]):\n                C[i][j] += A[i][k] * B[k][j]\n    return C\n# Initialize MPI\ncomm = MPI.COMM_WORLD\nrank = comm.Get_rank()\nsize = comm.Get_size()\n# Master process\nif rank == 0:\n    # Generate matrices A and B\n    A = np.random.rand(2, 2)\n    B = np.random.rand(2, 2)\n    # Split matrices for distribution\n    chunk_size = A.shape[0] \/\/ size\n    A_chunks = [A[i:i+chunk_size] for i in range(0, A.shape[0], chunk_size)]\n    # Send parts of A and B to worker processes\n    for i in range(1, size):\n        comm.send(A_chunks[i-1], dest=i, tag=1)\n        comm.send(B, dest=i, tag=2)\n    # Calculate its own part of multiplication\n    C_partial = matrix_multiply(A_chunks[0], B)\n    # Collect results from worker processes\n    for i in range(1, size):\n        C_partial += comm.recv(source=i, tag=3)\n    # Print the resulting matrix\n    print(\"Resulting matrix C:\")\n    print(C_partial)\n# Worker processes\nelse:\n    # Receive matrix chunks from master\n    A_chunk = comm.recv(source=0, tag=1)\n    B = comm.recv(source=0, tag=2)\n    # Perform multiplication\n    C_partial = matrix_multiply(A_chunk, B)\n    # Send back the result to master\n    comm.send(C_partial, dest=0, tag=3)\n<\/code><\/pre>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
Explanation<\/strong><\/h2>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
Import MPI Module and Initialize MPI Environment<\/strong><\/p>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
from mpi4py import MPI<\/code><\/pre>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
This line imports the MPI module from the mpi4py package, enabling the use of MPI functionalities.<\/p>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
comm = MPI.COMM_WORLD\nrank = comm.Get_rank()\nsize = comm.Get_size()<\/code><\/pre>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
These lines initialize the MPI environment. MPI.COMM_WORLD<\/code> creates a communicator object representing all processes in the MPI world. comm.Get_rank()<\/code> returns the rank of the current process in the communicator, and comm.Get_size()<\/code> returns the total number of processes in the communicator.<\/p>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
Function to Perform Matrix Multiplication<\/strong><\/p>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
def matrix_multiply(A, B):\n    C = np.zeros((A.shape[0], B.shape[1]))\n    for i in range(A.shape[0]):\n        for j in range(B.shape[1]):\n            for k in range(A.shape[1]):\n                C[i][j] += A[i][k] * B[k][j]\n    return C<\/code><\/pre>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
This function matrix_multiply<\/code> takes two matrices A<\/code> and B<\/code> as input and returns their multiplication C<\/code>. It initializes an empty matrix C<\/code> with dimensions derived from the multiplication of matrices A<\/code> and B<\/code>. Then, it performs matrix multiplication using nested loops to iterate through rows and columns of matrices A<\/code> and B<\/code>, computing each element of matrix C<\/code>.<\/p>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
Master Process<\/strong><\/p>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
if rank == 0:\n    A = np.random.rand(2, 2)\n    B = np.random.rand(2, 2)<\/code><\/pre>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
In the master process (rank 0), random matrices A<\/code> and B<\/code> of size 2×2 are generated.<\/p>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
chunk_size = A.shape[0] \/\/ size\nA_chunks = [A[i:i+chunk_size] for i in range(0, A.shape[0], chunk_size)]<\/code><\/pre>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
The matrices A<\/code> is split into chunks based on the total number of processes (size<\/code>). Each chunk is of size chunk_size<\/code>, and the list A_chunks<\/code> contains these chunks.<\/p>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
for i in range(1, size):\n    comm.send(A_chunks[i-1], dest=i, tag=1)\n    comm.send(B, dest=i, tag=2)<\/code><\/pre>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
Parts of matrices A<\/code> and B<\/code> are sent to worker processes using comm.send()<\/code>. Each chunk of A<\/code> along with the entire matrix B<\/code> is sent to a different worker process.<\/p>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
C_partial = matrix_multiply(A_chunks[0], B)<\/code><\/pre>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
The master process calculates its own partial result of matrix multiplication using the first chunk of matrix A<\/code>.<\/p>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
for i in range(1, size):\n    C_partial += comm.recv(source=i, tag=3)<\/code><\/pre>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
The master process receives partial results from each worker process using comm.recv()<\/code>, aggregates them, and stores the final result in C_partial<\/code>.<\/p>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
print(\"Resulting matrix C:\")\nprint(C_partial)<\/code><\/pre>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
Finally, the resulting matrix C<\/code> is printed.<\/p>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
Worker Processes<\/strong><\/p>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
else:\n    A_chunk = comm.recv(source=0, tag=1)\n    B = comm.recv(source=0, tag=2)<\/code><\/pre>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
In the worker processes (rank != 0), chunks of matrix A<\/code> and entire matrix B<\/code> are received from the master process using comm.recv()<\/code>.<\/p>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
C_partial = matrix_multiply(A_chunk, B)<\/code><\/pre>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
Each worker process performs matrix multiplication using its received chunk of matrix A<\/code> and matrix B<\/code>.<\/p>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
comm.send(C_partial, dest=0, tag=3)<\/code><\/pre>\n
\n\n\n<\/p>\n
\n\n\n<\/p>\n
The resulting partial matrix C_partial<\/code> is sent back to the master process using comm.send()<\/code>.<\/p>

<\/p>\n
<\/p>\n
Material<\/h2>\n
Download the programs (code), covering the MPI4Py.<\/a><\/p>\n
\n\n\n<\/p>\n