64 bit MPI PRPQ README for Version 1.1
*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!
IMPORTANT! PLEASE READ!
This 64 bit enabled version of MPI is provided by IBM under
RPQ #: P88059 and is offered as is, WITHOUT WARRANTY OF ANY KIND.
It is NOT INTENDED FOR PRODUCTION USE!
The 64-bit MPI libraries are installed in /usr/local/mpi.64bit/ on the EMSEI
SP. You should replace "<mydirectory>/ppp.poe/" in the examples below with
this location.
*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!
COMPILING:
1) When compiling 64-bit MPI programs for this PRPQ,
the following directory structure must exist,
containing the libraries and other files shipped in the
64-bit tar file package. The structure may be a
subdirectory of any directory path (called "mydirectory"
below) that the user chooses.
<mydirectory>/ppe.poe/lib/
<mydirectory>/ppe.poe/lib/ip
<mydirectory>/ppe.poe/lib//us/switch
<mydirectory>/ppe.poe/lib//us/switch2
<mydirectory>/ppe.poe/bin
<mydirectory>/ppe.poe/include
<mydirectory>/ppe.poe/include/thread64
When the tar file is extracted from <mydirectory>
it creates the needed structure.
This structure mimics that in /usr/lpp/ppe.poe,
which is used for standard 32-bit compilation,
but rather contains libraries with 64-bit objects
in addition to the 32-bit objects. In order to get
the compiler to access the <mydirectory>/ppe.poe
path rather than the /usr/lpp/ppe.poe path, you must:
export MP_PREFIX=<mydirectory>
2) If you are compiling for C or C++, you may use your
installed compile scripts. If you are compiling for
FORTRAN, you must use the scripts included in the tar
file, because they point to the 64-bit version of
mpif.h in the path ${MP_PREFIX}/ppe.poe/include/thread64.
In either case, you must have the environment
variable MP_PREFIX set as described above so that
the proper libraries and include files are used.
3) Invoke the proper compile script, specifying the -q64 option.
*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!
RUNNING:
1) Set up needed libraries:
When running 64-bit MPI programs for this PRPQ,
the following directory structure must exist,
consisting of the libraries shipped in the 64-bit tar
file package. The structure may be a subdirectory of
any directory path (called "mydirectory" below) that
the user chooses.
<mydirectory>/ppe.poe/lib/libmpi_r.a
<mydirectory>/ppe.poe/lib/libvtd_r.a
<mydirectory>/ppe.poe/lib/libppe_r.a
<mydirectory>/ppe.poe/lib/ip/libmpci_r.a
(needed only for IP)
<mydirectory>/ppe.poe/lib/us/switch/libmpci_r.a
(needed only for user space with SP Switch)
<mydirectory>/ppe.poe/lib/us/switch2/libmpci_r.a
(needed only for user space with SP Switch2)
<mydirectory>/ppe.poe/lib/us/libhal_r.a
(needed only for user space)
This structure mimics that in /usr/lpp/ppe.poe, which
is used for running standard 32-bit programs, but rather
contains libraries with 64-bit objects in addition to
the 32-bit objects.
When the tar file is extracted from <mydirectory> it
creates the needed structure.
2) If running user space, in addition to:
export MP_EUILIB=us
create a symbolic link in <mydirectory>/ppe.poe/lib/us as
follows, depending on which user space libmpci_r.a is desired:
a) ln -s ./switch/libmpci_r.a libmpci_r.a for the SP Switch
b) ln -s ./switch2/libmpci_r.a libmpci_r.a for the SP Switch2
3) If running IP, all that is required is the standard:
export MP_EUILIB=ip
4) Invoke program:
In order to get the loader to access the <mydirectory>/ppe.poe/lib
path rather than the /usr/lpp/ppe.poe/lib path, you must:
a)If invoking "my64bitprogram" via POE:
export MP_EUILIBPATH=<mydirectory>/ppe.poe/lib
poe my64bitprogram
b)If invoking "my64bitprogram" directly (not via POE):
export MP_EUILIBPATH=<mydirectory>/ppe.poe/lib
export LIBPATH=<mydirectory>/ppe.poe/lib:<mydirectory>/ppe.poe/lib/$MP_EUILIB:$LIBPATH
my64bitprogram
*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!*!
MISCELLANEOUS NOTES:
1) Messages from POE, including any -ilevel information or
advisory messages may present truncated values in cases
where the value is a pointer or long. The PRPQ does not
include new versions of the POE code which formats and
prints messages that MPI initiates so when MPI sends
data to be printed there can be truncation.
This does not apply to error messages from MPI that relate
to use of the MPI functions since these use a different
mechanism.
2) ulimits may need to be adjusted in order to malloc very
large values. This can be done by a system administrator,
or via LoadLeveler, as described here:
- in a LoadLeveler Job Command File (cmdfile_name):
# @ job_type=parallel
# @ data_limit = hard, soft
# @ queue
Where "hard" and "soft" represent the limit values,
which can be set to "unlimited".
- export MP_LLFILE=cmdfile_name
3) 64-bit switch clock is not supported for the PRPQ. The AIX
clock is used. Make sure that you don't set
MP_CLOCK_SOURCE=SWITCH.
4) A banner is printed when an application runs, stating that the
64 bit PRPQ code is in use. In order to disable the printing
of this banner:
export MP_CALLOUT_CONDITIONS=NO
Back to the index
