MPI_Comm_spawn on an intercommunicator

[David Cronk]

This should work, but it is failing under SGI MPI and Open MPI. Can
anyone else get this to work or see anything I have made a mistake with?
It is a pretty simple example program. Running on 16 processors it
should create 4 sets of intercommunicators each with one master and 3
slaves.

Any insight is appreciated.

Dave.

#include <stdio.h>
#include <sys/time.h>
#include <string.h>
#include "mpi.h"

int main (int argc, char **argv)
{
int i, myrank, numprocs, color;
MPI_Comm my_comm;
MPI_Comm inter_comm;
MPI_Comm my_inter_comm;

MPI_Init (&argc, &argv);

MPI_Comm_size (MPI_COMM_WORLD, &numprocs);
MPI_Comm_rank (MPI_COMM_WORLD, &myrank);

color = (myrank < 4);

MPI_Comm_split (MPI_COMM_WORLD, color, 1, &my_comm);
if (color)
MPI_Intercomm_create (my_comm, 0, MPI_COMM_WORLD, 4, 12, &inter_comm);
else
MPI_Intercomm_create (my_comm, 0, MPI_COMM_WORLD, 0, 12, &inter_comm);

color = myrank % 4;
MPI_Comm_split (inter_comm, color, 1, &my_inter_comm);

MPI_Finalize ();
}

[Michael Hofmann]

Post by David Cronk
This should work, but it is failing under SGI MPI and Open MPI. Can
anyone else get this to work or see anything I have made a mistake with?

Since their is no "MPI_Comm_spawn on an intercommunicator" in the example
program, I guess you mean MPI_Comm_split. As far as I understand,
MPI_Comm_split only applies to intra-communicators.

- "The function MPI_COMM_SPLIT allows more general partitioning of a group
into one or more subgroups with optional reordering. This call applies
only intra-communicators."
(http://www.mpi-forum.org/docs/mpi-11-html/node102.html)

- "The other communicator constructors, MPI_COMM_CREATE and
MPI_COMM_SPLIT, currently apply only to intracommunicators."
(http://www.mpi-forum.org/docs/mpi-20-html/node142.htm)

Post by David Cronk
It is a pretty simple example program. Running on 16 processors it
should create 4 sets of intercommunicators each with one master and 3
slaves.

It works with my "local" Open MPI (1.2.7) and with an AIX MPI. It fails
with an MVAPICH and another Open MPI.

Maybe it is safer, to create 8 intra-communicators first and the 4
inter-communicators independently from each other (afterwards).


Michael

[David Cronk]

Post by Michael Hofmann

Post by David Cronk
This should work, but it is failing under SGI MPI and Open MPI. Can
anyone else get this to work or see anything I have made a mistake with?

Since their is no "MPI_Comm_spawn on an intercommunicator" in the
example program, I guess you mean MPI_Comm_split.

Yes, that is what I meant.

As far as I

Post by Michael Hofmann
understand, MPI_Comm_split only applies to intra-communicators.
- "The function MPI_COMM_SPLIT allows more general partitioning of a
group into one or more subgroups with optional reordering. This call
applies only intra-communicators."
(http://www.mpi-forum.org/docs/mpi-11-html/node102.html)
- "The other communicator constructors, MPI_COMM_CREATE and
MPI_COMM_SPLIT, currently apply only to intracommunicators."
(http://www.mpi-forum.org/docs/mpi-20-html/node142.htm)

There seems to be some confusion.

MPI - The Complete Reference Volume 1, The MPI Core, second edition
page 306:
Intercommunicators can be used as arguments to all the communicator
accessor, constructor and destructor functions of Section 5.4.1, Section
5.4.2 and Section 5.4.3.

Page 310:
If MPI_COMM_SPLIT(comm, color, key, newcomm) is invoked with an
intercommunicator argument in comm, then a new intercommunicator is
created for each value of color (distinct from MPI_UNDEFINED). The two
groups of this intercommunicator consist of all processes of group A of
that color, and all of the processes of group B of that color.

However, clearly the standard has precidence over the reference manual.
Thanks for finding those points. I never thought of checking the
standard since the reference manuals are minor re-writes over the standard.

Post by Michael Hofmann

Post by David Cronk
It is a pretty simple example program. Running on 16 processors it
should create 4 sets of intercommunicators each with one master and 3
slaves.

It works with my "local" Open MPI (1.2.7) and with an AIX MPI. It fails
with an MVAPICH and another Open MPI.

Thanks. But if it is not standard compliant, I am going to bag it.

Thanks again for the reply.

Dave.

Post by Michael Hofmann
Maybe it is safer, to create 8 intra-communicators first and the 4
inter-communicators independently from each other (afterwards).
Michael

[Michael Hofmann]

Post by David Cronk
However, clearly the standard has precidence over the reference manual.
Thanks for finding those points. I never thought of checking the
standard since the reference manuals are minor re-writes over the standard.

For MPI-1 (1.3) it is clearly said that only intracommunicators are
supported.

However, it seems that there is some real confusion in MPI-2 (2.0). On
page 145 it is said that "MPI_COMM_CREATE and MPI_COMM_SPLIT, currently
apply only to intracommunicators", but the two following examples
explicitly show the usage with intercommunicators. This is very misleading.


Michael