*** Wartungsfenster jeden ersten Mittwoch vormittag im Monat ***

Skip to content
Snippets Groups Projects
Commit b9d427e7 authored by Blaas-Schenner, Claudia's avatar Blaas-Schenner, Claudia
Browse files

Python 03_ring tmp

parent 2845d31a
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 98 additions and 203 deletions
Python/03_ring.ipynb
+ 98
− 203
View file @ b9d427e7
%% Cell type:markdown id:7b8b3b7c-e15e-466d-b324-a5ce3fd18359 tags:
# MPI - Round and round we go (ring)!
<details>
<summary markdown="span"><b>Jupyter notebook quick start (click to expand)</b></summary>
- \<Shift>+\<Return> --> runs a cell
- ! --> shell escape [! Linux command line]
- use a (above) and b (below) left of the [ ] to open new cells
- use m (Markdown) or r (Raw) left of the [ ] to change the cell mode
<details>
<summary markdown="span"><b>Click to expand</b></summary>
If you see a little triangle you can click it to expand further explanations.
%% Cell type:markdown id:0edbac8a-a801-4e5a-aca0-89499027cb94 tags:
<details>
<summary markdown="span"><b>Author, acknowledgment, copyright, and license for this notebook</b></summary>
- <b>Author:</b> Claudia Blaas-Schenner (VSC Research Center, TU Wien), 18 November 2024
- <b>Based on</b> the [MPI course developed by Rolf Rabenseifner, HLRS](https://www.hlrs.de/training/self-study-materials/mpi-course-material) that is under a quite restrictive copyright by Rolf Rabenseifner and HLRS. The copyrighted material (some images, some exercise descriptions, some code snippets) is used with permission. Some parts taken from the HLRS material are modified and the Jupyter Notebook is extended with own material of the Notebook authors.
- <b>License:</b> [CC BY-SA 4.0 (Attribution-ShareAlike)](https://creativecommons.org/licenses/by-sa/4.0/)
- <b>Contributing and error reporting:</b> Please send an email to: [training@vsc.ac.at](mailto:training@vsc.ac.at)
%% Cell type:markdown id:543833e8-c66a-4bbd-a9ab-1c5a90170ff4 tags:
#### MPI Standard: [MPI: A Message-Passing Interface Standard Version 4.1 (PDF)](https://www.mpi-forum.org/docs/mpi-4.1/mpi41-report.pdf)
%% Cell type:markdown id:d5494aee-a65b-4e2a-a5d4-4c533cc7b483 tags:
### MPI4Py Documentation: [MPI4PY: https://mpi4py.readthedocs.io](https://mpi4py.readthedocs.io)
%% Cell type:markdown id:5d223967-4a35-407d-a2f7-bdbb8c23f2b9 tags:
#### You can edit the exercises directly in the cells.
&nbsp;
%% Cell type:markdown id:01ac1316-a516-477c-b5f0-ae32bbc0bf9e tags:
### Contents - learn to use nonblocking point-to-point communication correctly:
%% Cell type:markdown id:8a6701a3-8e8a-4c96-81c5-0da837632879 tags:
- 0. Round and round we go (ring)!<br>
- 1. ring - MPI_Issend + MPI_Recv + MPI_Wait - debugging version!<br>
- 2. ring - MPI_Irecv + MPI_Ssend + MPI_Wait - debugging version!<br>
- 3. ring - MPI_Irecv + MPI_Issend + MPI_Waitall - debugging version!<br>&nbsp;<br>
%% Cell type:markdown id:32aed258-f7e0-4b18-a676-1d228f82a644 tags:
![03_ring](../images/03_ring.png "03_ring")
%% Cell type:markdown id:7a0618a1-9194-48ae-9955-d28847eef050 tags:
&nbsp;
%% Cell type:markdown id:55c185ae-cc9d-4fef-a837-fa43ce81db70 tags:
## 0. Round and round we go (ring)!
%% Cell type:markdown id:e89a732e-5e1b-471d-b13e-89988fd92f42 tags:
##### Ring communication (see also image with explanation above):
- A set of processes is arranged in a **ring** and they need to do a halo communication with their neigbors.
- To make this exercise easier, we'll focus only on **one direction** and<br>
we'll not use a big data array but work with **just one integer**, i.e., the ranks.
- Just for fun we do that a couple of times (size) and sum up all values that a process receives,<br>
the correct result is: size * (size-1) / 2
<details>
<summary markdown="span"><b>Further reading (nonblocking communication)</b></summary>
<br>[MPI 4.1 Chapter 3.7](https://www.mpi-forum.org/docs/mpi-4.1/mpi41-report.pdf#page=111) - Nonblocking Communication
- pages 69-71, 75, 78-83: recommended reading<br>
[MPI 4.1 Chapter 2.4](https://www.mpi-forum.org/docs/mpi-4.1/mpi41-report.pdf#page=53) - Semantic Terms
- pages 11-17: MPI Operations, MPI Procedures, MPI Datatypes<br>
[MPI 4.1 Chapter 2.7](https://www.mpi-forum.org/docs/mpi-4.1/mpi41-report.pdf#page=67) - Processes
- page 25: Processes<br>
[MPI 4.1 Chapter 2.9](https://www.mpi-forum.org/docs/mpi-4.1/mpi41-report.pdf#page=69) - Progress
- pages 27-28: Progress<br>
<details>
<summary markdown="span"><b>Deep dive (nonblocking & probe & persistent & MPI_PROC_NULL)</b></summary>
<br>[MPI 4.1 Chapter 3.7](https://www.mpi-forum.org/docs/mpi-4.1/mpi41-report.pdf#page=111) - Nonblocking Communication -- pages 69-94
[MPI 4.1 Chapter 3.8](https://www.mpi-forum.org/docs/mpi-4.1/mpi41-report.pdf#page=136) - Probe and Cancel -- pages 94-102
[MPI 4.1 Chapter 3.9](https://www.mpi-forum.org/docs/mpi-4.1/mpi41-report.pdf#page=146) - Persistent Communication Requests -- pages 104-111
[MPI 4.1 Chapter 3.10](https://www.mpi-forum.org/docs/mpi-4.1/mpi41-report.pdf#page=153) - Null MPI Processes -- page 111
%% Cell type:markdown id:396a4a40-545a-4086-a9e9-eecb686ad89d tags:
<details>
<summary markdown="span"><b>MPI_Isend(&buf, count, datatype, dest, tag, comm, &request)</b></summary>
- **nonblocking send procedure** (other send modes have the same syntax)
- source rank sends the message defined by (buf, count, datatype) to the dest(ination) rank
<br>&nbsp;<br>
- IN &nbsp; &nbsp; &nbsp; buf &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; initial address of send buffer (choice)
- IN &nbsp; &nbsp; &nbsp; count &nbsp; &nbsp; &nbsp; &nbsp; number of elements in send buffer (non-negative
integer)
- IN &nbsp; &nbsp; &nbsp; datatype &nbsp; datatype of each send buffer element (handle)
- IN &nbsp; &nbsp; &nbsp; dest &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; rank of destination (integer)
- IN &nbsp; &nbsp; &nbsp; tag &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; message tag (integer)
- IN &nbsp; &nbsp; &nbsp; comm &nbsp;&nbsp; &nbsp; &nbsp; communicator (handle)
- OUT &nbsp; request &nbsp; &nbsp; communication request (handle)
<br>&nbsp;<br>
- C binding
<br> int **MPI_Isend**(const void ***buf**, int **count**, MPI_Datatype **datatype**, int **dest**, int **tag**, MPI_Comm **comm**, MPI_Request ***request**)
- *Usage: &nbsp; MPI_Isend(&buffer, 1, MPI_INT, 1, 17, MPI_COMM_WORLD, &request);*
<br>&nbsp;<br>
- *Note:*
<br> *Nonblocking sends can use any mode, standard, synchronous, buffered, ready.*
- ***MPI_Isend** (standard send)*
- ***MPI_Issend** (synchronous send)*
<br> *Synchronous mode affects completion, i.e., MPI_Wait / MPI_Test, not initiation, i.e., MPI_I....*
%% Cell type:markdown id:9794c876-39cc-41b3-8757-3480effe2cb6 tags:
<details>
<summary markdown="span"><b>MPI_Irecv(&buf, count, datatype, source, tag, comm, &request)</b></summary>
- **nonblocking receive procedure**
- dest(ination) rank receives a message from the source rank and stores it at (buf, count, datatype)
<br>&nbsp;<br>
- OUT &nbsp; buf &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; initial address of receive buffer (choice)
- IN &nbsp; &nbsp; &nbsp; count &nbsp; &nbsp; &nbsp; &nbsp; number of elements in receive buffer (non-negative
integer)
- IN &nbsp; &nbsp; &nbsp; datatype &nbsp; datatype of each receive buffer element (handle)
- IN &nbsp; &nbsp; &nbsp; source &nbsp; &nbsp; &nbsp; rank of source or MPI_ANY_SOURCE (integer)
- IN &nbsp; &nbsp; &nbsp; tag &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; message tag or MPI_ANY_TAG (integer)
- IN &nbsp; &nbsp; &nbsp; comm &nbsp;&nbsp; &nbsp; &nbsp; communicator (handle)
- OUT &nbsp; request &nbsp; &nbsp; communication request (handle)
<br>&nbsp;<br>
- C binding
<br> int **MPI_Irecv**(void ***buf**, int **count**, MPI_Datatype **datatype**, int **source**, int **tag**, MPI_Comm **comm**, MPI_Request ***request**)
- *Usage: &nbsp; MPI_Irecv(&buffer, 1, MPI_FLOAT, 0, 17, MPI_COMM_WORLD, &request);*
%% Cell type:markdown id:0fb456c6-7f01-4877-8bac-25c8649f2e9d tags:
<details>
<summary markdown="span"><b>MPI_Wait(&request, &status)</b></summary>
- **nonblocking communication completion**<br>
- The functions MPI_WAIT and MPI_TEST are used to complete a nonblocking communication.
- The completion of a send operation indicates that the sender is now free to update
the send buffer (the send operation itself leaves the content of the send buffer unchanged).
It does not indicate that the message has been received, rather, it may have been buffered
by the communication subsystem. However, if a synchronous mode send was used, the
completion of the send operation indicates that a matching receive was initiated, and that
the message will eventually be received by this matching receive.
- The completion of a receive operation indicates that the receive buffer contains the
received message, the receiver is now free to access it, and that the status object is set. It does not indicate that the matching send operation has completed (but indicates, of course,
that the send was initiated).
<br>&nbsp;<br>
- INOUT &nbsp; request &nbsp; &nbsp; &nbsp; request (handle)
- OUT &nbsp; &nbsp; &nbsp; status &nbsp; &nbsp; &nbsp; &nbsp; status object (status)
<br>&nbsp;<br>
- C binding
<br> int **MPI_Wait**(MPI_Request ***request**, MPI_Status ***status**)
- *Usage: &nbsp; MPI_Wait(&request, &status);*
%% Cell type:markdown id:c74bca4d-7a16-4e38-83cf-dbda85e95bd9 tags:
<details>
<summary markdown="span"><b>MPI_Waitall(count, array_of_requests, array_of_statuses)</b></summary>
- IN &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; count &nbsp;&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; list length (non-negative integer)
- INOUT &nbsp; array_of_requests &nbsp; &nbsp; &nbsp; array of requests (array of handles)
- OUT &nbsp; &nbsp; &nbsp; array_of_statuses &nbsp; &nbsp; &nbsp; array of status objects (array of status)
<br>&nbsp;<br>
- C binding
<br> int **MPI_Waitall**(int **count**, MPI_Request **array_of_requests[]**,
MPI_Status **array_of_statuses[]**)
- *Usage: &nbsp; MPI_Waitall(count, arr_request, arr_status);*
%% Cell type:markdown id:94431f3b-d259-49f2-a8eb-d1c2816b8d0c tags:
**[MPI 4.1 Table 3.2](https://www.mpi-forum.org/docs/mpi-4.1/mpi41-report.pdf#page=76) - Predefined MPI datatypes corresponding to C datatypes**
%% Cell type:markdown id:fbaa8ca6-6fdc-432c-950d-e8eac2d687c3 tags:
&nbsp;
%% Cell type:markdown id:d8383b74-f20b-4864-903a-ab7cd45e25c9 tags:
## 1. ring - MPI_Issend + MPI_Recv + MPI_Wait - debugging version!
%% Cell type:markdown id:03d9d0ce-79ef-4766-9d0a-beb0ff4c9ffc tags:
##### Communication in a ring:
- 1. Run the (WRONG) program in the cell below --> why does this seem to work?
- 2. Substitute the Send by an Ssend to make it a debugging version<br>
run it again --> now you'll see a deadlock (use the interrupt / stop box to end it)
- 3. Correct the program by using nonblocking communication<br>
--> use nonblocking MPI_Issend but keep the blocking MPI_Recv
*Note:*<br>
*A real application in **production** mode would **use standard MPI_Isend**.*<br>
*Here in this exercises, we use synchronous MPI_Issend only to **demonstrate a deadlock** if the nonblocking routine is not correctly used.*
%% Cell type:code id:adc73ff7-6447-45ab-9344-8ea2967b841d tags:
``` python
%%writefile tmp/ring.c
%%writefile tmp/ring.py
#include <stdio.h>
#include <mpi.h>
from mpi4py import MPI
import numpy as np
int main (int argc, char *argv[])
{
int rank, size;
int snd_buf, rcv_buf;
int right, left;
int sum, i;
MPI_Status status;
// _____
rcv_buf = np.empty((),dtype=np.intc)
status = MPI.Status()
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);
right = (rank+1) % size;
left = (rank-1+size) % size;
comm_world = MPI.COMM_WORLD
rank = comm_world.Get_rank()
size = comm_world.Get_size()
right = (rank+1) % size
left = (rank-1+size) % size
sum = 0
snd_buf = np.array(rank, dtype=np.intc)
for i in range(size):
comm_world.Send((snd_buf, 1, MPI.INT), dest=right, tag=17)
comm_world.Recv((snd_buf, 1, MPI.INT), source=left, tag=17, status=status)
# _____ WRONG program, it will deadlock with a synchronous communication protocol
# np.copyto(snd_buf, rcv_buf) # We make a copy here. What happens if we write snd_buf = rcv_buf instead?
sum += snd_buf
sum = 0;
snd_buf = rank;
for( i = 0; i < size; i++)
{
MPI_Send(&snd_buf, 1, MPI_INT, right, 17, MPI_COMM_WORLD);
MPI_Recv(&snd_buf, 1, MPI_INT, left, 17, MPI_COMM_WORLD, &status);
// _____ WRONG program, it will deadlock with a synchronous communication protocol
// _____
sum += snd_buf;
}
printf ("PE%i:\tSum = %i\n", rank, sum);
MPI_Finalize();
}
```
%% Cell type:markdown id:6e1c939f-db58-4c80-8871-aab667ada929 tags:
##### Compile:
%% Cell type:code id:5bf06703-8012-4291-ad59-71d83be99107 tags:
``` python
!cd tmp; mpicc ring.c -o ring
print(f"PE{rank}:\tSum = {sum}")
```
%% Cell type:markdown id:5112674f-c2d7-4516-b3c7-ed1121fc4ca4 tags:
##### Run:
%% Cell type:code id:8dd01859-ef72-40d8-a434-b6a499ef2a6c tags:
``` python
!cd tmp; mpirun -np 4 ./ring
!cd tmp; mpirun -np 4 python3 ./ring.py
```
%% Cell type:markdown id:4e3f19cb-cd03-40f3-a7a4-47ad53c18a93 tags:
#### Play around with different numbers of MPI processes
%% Cell type:markdown id:68f76d2b-0516-45ab-bcd8-a5d9f717666e tags:
**Correct result = size * (size-1) / 2**
%% Cell type:markdown id:8ed5866b-a63a-409c-a388-fa7923b76cad tags:
**Run with different numbers of MPI processes:**<br>
E.g., in the VSC Jupyterhub, there are 4 physical cores available for the MPI course.<br>
If you want to run with more MPI processes you have to add the option:
**--oversubscribe**<br>
*Note: For performance runs you want to avoid oversubscribing, but it's okay for our little examples.*<br>
*Note: You can not do oversubscribing and pinning at the same time.*
%% Cell type:code id:3940eeee-e665-4ded-b75a-5863f15ee222 tags:
``` python
!cd tmp; mpirun -np 12 --oversubscribe ./ring
!cd tmp; mpirun -np 12 --oversubscribe python3 ./ring.py
```
%% Cell type:markdown id:912708e2-cb48-47d4-bcb7-ab9864d1bd92 tags:
#### Solution (please try to solve the exercise by yourself before looking at the solution)
%% Cell type:code id:10a3ae35-acf5-4cf4-9c94-db94e353637a tags:
``` python
%%writefile tmp/ring_solution.c
%%writefile tmp/ring_solution.py
#include <stdio.h>
#include <mpi.h>
from mpi4py import MPI
import numpy as np
int main (int argc, char *argv[])
{
int rank, size;
int snd_buf, rcv_buf;
int right, left;
int sum, i;
MPI_Status status;
MPI_Request request;
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);
right = (rank+1) % size;
left = (rank-1+size) % size;
sum = 0;
snd_buf = rank;
for( i = 0; i < size; i++)
{
MPI_Issend(&snd_buf, 1, MPI_INT, right, 17, MPI_COMM_WORLD, &request);
MPI_Recv (&rcv_buf, 1, MPI_INT, left, 17, MPI_COMM_WORLD, &status);
MPI_Wait(&request, &status);
snd_buf = rcv_buf;
sum += rcv_buf;
}
printf ("PE%i:\tSum = %i\n", rank, sum);
MPI_Finalize();
}
```
rcv_buf = np.empty((),dtype=np.intc)
status = MPI.Status()
%% Cell type:markdown id:34788895-5728-43df-8d90-12a4685d231b tags:
comm_world = MPI.COMM_WORLD
rank = comm_world.Get_rank()
size = comm_world.Get_size()
right = (rank+1) % size
left = (rank-1+size) % size
sum = 0
snd_buf = np.array(rank, dtype=np.intc)
for i in range(size):
request = comm_world.Issend((snd_buf, 1, MPI.INT), dest=right, tag=17)
comm_world.Recv ((rcv_buf, 1, MPI.INT), source=left, tag=17, status=status)
request.Wait(status)
np.copyto(snd_buf, rcv_buf) # # We make a copy here. What happens if we write snd_buf = rcv_buf instead?
# Remember that snd_buf = rcv_buf binds them to the same object (i.e. snd_buf and rcv_buf are then two different names pointing to the same object), which is unintended and can lead to incorrect code.
sum += rcv_buf
##### Compile the solution:
%% Cell type:code id:6f38f54b-4e24-4d21-bba2-37c322dce283 tags:
``` python
!cd tmp; mpicc ring_solution.c -o ring_solution
print(f"PE{rank}:\tSum = {sum}")
```
%% Cell type:markdown id:7566ecac-6878-45aa-b545-36fc0d7e97e3 tags:
##### Run the solution:
%% Cell type:code id:0af3970c-8691-4b43-ae81-3db977614638 tags:
``` python
!cd tmp; mpirun -np 4 ./ring_solution
!cd tmp; mpirun -np 4 python3 ./ring_solution.py
```
%% Cell type:markdown id:613f39de-a970-4121-981e-b8b13e537296 tags:
#### Expected output (4 MPI processes):
%% Cell type:raw id:28b4a2d7-32e7-4b8b-b579-c80eca0b56b4 tags:
PE0: Sum = 6
PE1: Sum = 6
PE2: Sum = 6
PE3: Sum = 6
%% Cell type:markdown id:2be9fcee-1be8-47ef-ad53-6ea650bb4365 tags:
&nbsp;
%% Cell type:markdown id:2e3ccf5a-5a3c-4d46-acf1-cf371adb8c92 tags:
## 2. ring - MPI_Irecv + MPI_Ssend + MPI_Wait - debugging version!
%% Cell type:markdown id:ef02270b-aa74-4387-976d-5124fedf084f tags:
##### Use MPI_Irecv + MPI_Ssend + MPI_Wait instead of MPI_Issend + MPI_Recv + MPI_Wait
%% Cell type:code id:c9258630-975c-41ca-aab4-4e15c76d1e42 tags:
``` python
%%writefile tmp/ring_advanced_irecv_ssend.c
%%writefile tmp/ring_advanced_irecv_ssend.py
#include <stdio.h>
#include <mpi.h>
from mpi4py import MPI
import numpy as np
#define to_right 201
rcv_buf = np.empty((),dtype=np.intc)
status = MPI.Status()
int main (int argc, char *argv[])
{
int rank, size;
int snd_buf, rcv_buf;
int right, left;
int sum, i;
MPI_Status status;
MPI_Request request;
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);
right = (rank+1) % size;
left = (rank-1+size) % size;
sum = 0;
snd_buf = rank;
for( i = 0; i < size; i++)
{
MPI_Issend(&snd_buf, 1, MPI_INT, right, to_right, MPI_COMM_WORLD, &request);
MPI_Recv (&rcv_buf, 1, MPI_INT, left, to_right, MPI_COMM_WORLD, &status);
MPI_Wait(&request, &status);
snd_buf = rcv_buf;
sum += rcv_buf;
}
printf ("PE%i:\tSum = %i\n", rank, sum);
comm_world = MPI.COMM_WORLD
rank = comm_world.Get_rank()
size = comm_world.Get_size()
right = (rank+1) % size
left = (rank-1+size) % size
sum = 0
snd_buf = np.array(rank, dtype=np.intc)
for i in range(size):
request = comm_world.Issend((snd_buf, 1, MPI.INT), dest=right, tag=17)
comm_world.Recv ((rcv_buf, 1, MPI.INT), source=left, tag=17, status=status)
request.Wait(status)
np.copyto(snd_buf, rcv_buf) # # We make a copy here. What happens if we write snd_buf = rcv_buf instead?
# Remember that snd_buf = rcv_buf binds them to the same object (i.e. snd_buf and rcv_buf are then two different names pointing to the same object), which is unintended and can lead to incorrect code.
sum += rcv_buf
MPI_Finalize();
}
```
%% Cell type:markdown id:187c78ba-9dc8-4e43-853a-21a6f4b001ab tags:
##### Compile:
%% Cell type:code id:204eeceb-ec83-4230-b7f7-e75874938018 tags:
``` python
!cd tmp; mpicc ring_advanced_irecv_ssend.c -o ring_advanced_irecv_ssend
print(f"PE{rank}:\tSum = {sum}")
```
%% Cell type:markdown id:87af2510-d138-4a1f-8a6c-0d96830a27c6 tags:
##### Run:
%% Cell type:code id:cd8a2825-0677-4bc0-8ff9-09e6fc190360 tags:
``` python
!cd tmp; mpirun -np 4 ring_advanced_irecv_ssend
!cd tmp; mpirun -np 4 python3 ring_advanced_irecv_ssend.py
```
%% Cell type:markdown id:c7527f1b-a008-4f95-b14b-7c812172f372 tags:
#### Solution (please try to solve the exercise by yourself before looking at the solution)
#### Solution for C (please try to solve the exercise by yourself before looking at the solution)
%% Cell type:code id:9d483267-69ed-44ee-a2fc-57664ecb2524 tags:
``` python
%%writefile tmp/ring_advanced_irecv_ssend_solution.c
#include <stdio.h>
#include <mpi.h>
#define to_right 201
int main (int argc, char *argv[])
{
int rank, size;
int snd_buf, rcv_buf;
int right, left;
int sum, i;
MPI_Status status;
MPI_Request request;
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);
right = (rank+1) % size;
left = (rank-1+size) % size;
sum = 0;
snd_buf = rank;
for( i = 0; i < size; i++)
{
MPI_Irecv(&rcv_buf, 1, MPI_INT, left, to_right, MPI_COMM_WORLD, &request);
MPI_Ssend(&snd_buf, 1, MPI_INT, right, to_right, MPI_COMM_WORLD);
MPI_Wait(&request, &status);
snd_buf = rcv_buf;
sum += rcv_buf;
}
printf ("PE%i:\tSum = %i\n", rank, sum);
MPI_Finalize();
}
```
%% Cell type:markdown id:3b280691-dbba-4814-8ef5-a90dafb02cad tags:
##### Compile the solution:
%% Cell type:code id:31a63e97-3b84-48ab-a301-92eab829c09b tags:
``` python
!cd tmp; mpicc ring_advanced_irecv_ssend_solution.c -o ring_advanced_irecv_ssend_solution
```
%% Cell type:markdown id:f4b34423-8112-468a-8462-bd68c47719d9 tags:
##### Run the solution:
%% Cell type:code id:2dea2fdb-719d-450b-9cf3-17e19127c405 tags:
``` python
!cd tmp; mpirun -np 4 ./ring_advanced_irecv_ssend_solution
```
%% Cell type:markdown id:42ba0a93-f2ad-467c-81fd-ff7c3bcb72de tags:
&nbsp;
%% Cell type:markdown id:9acc3374-06bd-491e-aee3-b5099426f129 tags:
## 3. ring - MPI_Irecv + MPI_Issend + MPI_Waitall - debugging version!
%% Cell type:markdown id:05cbc183-0328-4e27-976f-c87729792b28 tags:
##### Use MPI_Irecv + MPI_Issend + MPI_Waitall instead of MPI_Issend + MPI_Recv + MPI_Wait
%% Cell type:code id:6d9a1f73-5df9-4dc8-9ed2-254f0294af1a tags:
``` python
%%writefile tmp/ring_advanced_irecv_issend.c
#include <stdio.h>
#include <mpi.h>
#define to_right 201
int main (int argc, char *argv[])
{
int rank, size;
int snd_buf, rcv_buf;
int right, left;
int sum, i;
MPI_Status status;
MPI_Request request;
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);
right = (rank+1) % size;
left = (rank-1+size) % size;
%%writefile tmp/ring_advanced_irecv_issend.py
sum = 0;
snd_buf = rank;
for( i = 0; i < size; i++)
{
MPI_Issend(&snd_buf, 1, MPI_INT, right, to_right, MPI_COMM_WORLD, &request);
MPI_Recv (&rcv_buf, 1, MPI_INT, left, to_right, MPI_COMM_WORLD, &status);
MPI_Wait(&request, &status);
snd_buf = rcv_buf;
sum += rcv_buf;
}
printf ("PE%i:\tSum = %i\n", rank, sum);
MPI_Finalize();
}
```
%% Cell type:markdown id:10a48a93-282a-4496-8c2b-dceae7d0d18c tags:
from mpi4py import MPI
import numpy as np
##### Compile:
rcv_buf = np.empty((),dtype=np.intc)
status = MPI.Status()
%% Cell type:code id:90c41dc5-809f-4ae3-ab97-592cc81927ad tags:
comm_world = MPI.COMM_WORLD
rank = comm_world.Get_rank()
size = comm_world.Get_size()
right = (rank+1) % size
left = (rank-1+size) % size
sum = 0
snd_buf = np.array(rank, dtype=np.intc)
for i in range(size):
request = comm_world.Issend((snd_buf, 1, MPI.INT), dest=right, tag=17)
comm_world.Recv ((rcv_buf, 1, MPI.INT), source=left, tag=17, status=status)
request.Wait(status)
np.copyto(snd_buf, rcv_buf) # # We make a copy here. What happens if we write snd_buf = rcv_buf instead?
# Remember that snd_buf = rcv_buf binds them to the same object (i.e. snd_buf and rcv_buf are then two different names pointing to the same object), which is unintended and can lead to incorrect code.
sum += rcv_buf
``` python
!cd tmp; mpicc ring_advanced_irecv_issend.c -o ring_advanced_irecv_issend
print(f"PE{rank}:\tSum = {sum}")
```
%% Cell type:markdown id:0736cd95-52f7-467f-aa17-b93e83e7f4ba tags:
##### Run:
%% Cell type:code id:283449b9-a7f1-4b11-9128-5524c4aaee33 tags:
``` python
!cd tmp; mpirun -np 4 ./ring_advanced_irecv_issend
!cd tmp; mpirun -np 4 python3 ./ring_advanced_irecv_issend.py
```
%% Cell type:markdown id:d25b376f-8262-4394-af71-9dd67f1d412c tags:
#### Solution (please try to solve the exercise by yourself before looking at the solution)
#### Solution for C (please try to solve the exercise by yourself before looking at the solution)
%% Cell type:code id:060db35e-9f44-4feb-a68a-10eb2c330dcb tags:
``` python
%%writefile tmp/ring_advanced_irecv_issend_solution.c
#include <stdio.h>
#include <mpi.h>
#define to_right 201
int main (int argc, char *argv[])
{
int rank, size;
int snd_buf, rcv_buf;
int right, left;
int sum, i;
MPI_Status arr_status[2];
MPI_Request arr_request[2];
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);
right = (rank+1) % size;
left = (rank-1+size) % size;
sum = 0;
snd_buf = rank;
for( i = 0; i < size; i++)
{
MPI_Irecv (&rcv_buf, 1, MPI_INT, left, to_right, MPI_COMM_WORLD, &arr_request[0]);
MPI_Issend(&snd_buf, 1, MPI_INT, right, to_right, MPI_COMM_WORLD, &arr_request[1]);
MPI_Waitall(2, arr_request, arr_status);
snd_buf = rcv_buf;
sum += rcv_buf;
}
printf ("PE%i:\tSum = %i\n", rank, sum);
MPI_Finalize();
}
```
%% Cell type:markdown id:2836ce60-02ad-45cd-9ef3-cb4b7ab66830 tags:
##### Compile the solution:
%% Cell type:code id:b78e2947-9c6d-4021-a9d2-4d1b4e8d24e2 tags:
``` python
!cd tmp; mpicc ring_advanced_irecv_issend_solution.c -o ring_advanced_irecv_issend_solution
```
%% Cell type:markdown id:26a0cfae-d665-496b-a0d5-78d19a968130 tags:
##### Run the solution:
%% Cell type:code id:61392033-89b1-437a-9aa7-714158351a63 tags:
``` python
!cd tmp; mpirun -np 4 ./ring_advanced_irecv_issend_solution
```
%% Cell type:markdown id:e0e39004-d8f0-4ba1-9468-d45975e424ce tags:
## &nbsp;
%% Cell type:markdown id:d38c5f96-ecb1-47fb-ab1c-f7c3fcb5fa9c tags:
<details>
<summary markdown="span"><b>Author, acknowledgment, copyright, and license for this notebook</b></summary>
- <b>Author:</b> Claudia Blaas-Schenner (VSC Research Center, TU Wien), 18 November 2024
- <b>Based on</b> the [MPI course developed by Rolf Rabenseifner, HLRS](https://www.hlrs.de/training/self-study-materials/mpi-course-material) that is under a quite restrictive copyright by Rolf Rabenseifner and HLRS. The copyrighted material (some images, some exercise descriptions, some code snippets) is used with permission. Some parts taken from the HLRS material are modified and the Jupyter Notebook is extended with own material of the Notebook authors.
- <b>License:</b> [CC BY-SA 4.0 (Attribution-ShareAlike)](https://creativecommons.org/licenses/by-sa/4.0/)
- <b>Contributing and error reporting:</b> Please send an email to: [training@vsc.ac.at](mailto:training@vsc.ac.at)
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

[ 18.Mar 6h : 3,688 users | 903 groups | 4,946 projects ]