Sending/receiving column vectors

[l***@yahoo.com]

Hi all,
I'm trying to send a column of a matrix from one processor to a
processor on its right.

Each processor stores a square matrix (say, 8-by-8) in the form of

xxxxxxxx
x000000x
x000000x
x000000x
x000000x
x000000x
x000000x
xxxxxxxx

where x are "ghost rows/columns" and the actual useful rows/columns
are the 0.

I have the original large grid of 64x64 matrix, mapped evenly onto 4x4
processors, so each processor has a 18x18 square matrix (called grid),
with the actual "inner" 16x16 matrix. The processors are arranged in

0 1 2 3
4 5 6 7
8 9 10 11
12 13 14 15

What I'm trying to do is send the second last column in P0, i.e.
grid[i][columns], columns = 16, i = 1 to 16 to P1, and store that
column in the first (ghost) column of P1, i.e. grid[i][0], i = 1 to
16. (This actually happens for all the processors, i.e. P0 -> P1, P1 -

P2, P2 ->P3, P4 -> P5 etc, but I'm just looking at P0 and P1 for

simplicity's sake).

My code is as follows:

#include <stdlib.h>
#include <stdio.h>
#include <mpi.h>
#include <math.h>

#define N 64 // size of entire grid mapped by all 16 processors

void exchange(int **grid, int rank, int rows, int columns, int size)
{
MPI_Status status;
MPI_Datatype col;
MPI_Type_vector(rows, 1, columns+2, MPI_INT, &col);
MPI_Type_commit(&col);
int srofsize, left, right;
int i, *tempcol;

srofsize = (int)sqrt(size);
tempcol = (int*)calloc(rows, sizeof(int));

/* Partners during exchange */
left = rank - 1;
right = rank + 1;

if ((rank % srofsize) == 0)
left = MPI_PROC_NULL;
if (((rank+1) % srofsize) == 0)
right = MPI_PROC_NULL;

/* Send right/receive as lst column */
if ((rank % 2) == 0)
{
MPI_Send(&grid[1][columns], 1, col, right, 1, MPI_COMM_WORLD);
if (rank == 0)
{
printf("P0 Sending\n");
for (int j = 0 ; j < rows ; j++)
printf("%d ", grid[j+1][columns]);
printf("\n");
}
}
else
{
MPI_Recv(tempcol, rows, MPI_INT, left, 1, MPI_COMM_WORLD,
&status);
if (rank == 1)
printf("P1 receiving\n");
for (i = 0 ; i < rows ; i++)
{
grid[i+1][0] = tempcol[i];
if (rank == 1)
printf("%d ", tempcol[i]);
}
if (rank == 1)
printf("\n");
}
}
int main(int argc, char ** argv)
{
MPI_Init(&argc, &argv);
int i, size, rank, rows, columns, srofsize;
int **grid;
MPI_Comm_size(MPI_COMM_WORLD, &size);
MPI_Comm_rank(MPI_COMM_WORLD, &rank);

srofsize = (int)sqrt(size);
rows = N/srofsize;
columns = N/srofsize;

grid = (int**)calloc(rows+2, sizeof(int*));
for (i = 0 ; i < rows+2 ; i++)
grid[i] = (int*)calloc(columns+2, sizeof(int));

for (i = 0 ; i < 18 ; i++)
grid[i][columns] = -11;

exchange(grid, rank, rows, columns, size);
for (i = 0 ; i < rows+2 ; i++)
free(grid[i]);
free(grid);
MPI_Finalize();
return 0;
}

The printf statements are to check the results of my program.

The output I got was
P0 Sending
-11 -11 -11 -11 -11 -11 -11 -11 -11 -11 -11 -11 -11 -11 -11 -11
P1 receiving
-11 4 0 7275776 0 0 -1137561976 161 1 0 7270192 1952412271 48 80 0 0

The row of -11 is the column I'm trying to send, but I don't
understand why I kept receiving the garbage values in P1.

Please advise.

Thank you.

Regards,
Rayne

[Michael Hofmann]

for (i =3D 0 ; i < rows+2 ; i++)
grid[i] =3D (int*)calloc(columns+2, sizeof(int));

You are allocating each row of "grid" with a separate call to "calloc". =
Therefore, it's very unlikely that all rows are located one after anothe=
r (and with correct order) in memory. But this is essential for your "ex=
change" routine to work.

Try to allocate "grid" like this (untested!):

grid_data =3D (int*) calloc((rows+2)*(columns+2), sizeof(int));
for (i =3D 0; i < rows+2; i++)
grid[i] =3D &grid_data[i*(colums+2)];

exchange(...);

free(grid_data);

(Don't forget to remove the for-loop with the free calls at the end!)


Michael