Reading files concurrently

[l***@yahoo.com]

Hi all,
I'll like to know if it is alright to read from a file concurrently
from multiple processors, i.e. every processor has their own file
pointer and opens the same file, read its contents, close the file etc
concurrently. I've tried it and it seems to give me the correct
results, but I'm not sure if there are any hidden errors that just
haven't shown up yet.

Thank you.

Regards,
Rayne

[Michael Hofmann]

Post by l***@yahoo.com
Hi all,
I'll like to know if it is alright to read from a file concurrently
from multiple processors, i.e. every processor has their own file
pointer and opens the same file, read its contents, close the file etc
concurrently. I've tried it and it seems to give me the correct
results, but I'm not sure if there are any hidden errors that just
haven't shown up yet.

MPI has nothing to do with regular file I/O (fopen, fseek, fread, fclose,
...). Whether it is OK or not depends on your OS, the network file system,
...

MPI-2 provides an own set of file I/O routines (MPI_File_...). If you use
it correctly, it is definitely OK and it is portable.

In general, concurrent read should not be a problem (if you open the file
read-only).


Michael

[David Cronk]

Post by l***@yahoo.com
Hi all,
I'll like to know if it is alright to read from a file concurrently
from multiple processors, i.e. every processor has their own file
pointer and opens the same file, read its contents, close the file etc
concurrently. I've tried it and it seems to give me the correct
results, but I'm not sure if there are any hidden errors that just
haven't shown up yet.

This is legal but not scalable. Operating systems have limits on how
many processes can have a single file open at the same time.

Since this is an MPI group, look into MPI-I/O to see how to do this with
MPI. You may even find it is easier with MPI-I/O.

Hope this helps.

Dave.

Post by l***@yahoo.com
Thank you.
Regards,
Rayne