Siesta master branch, Intel MKL FATAL ERROR

SIESTA - Installation Questions Compilation
1

Hi all,

I recently compiled the siesta master branch using cmake. It is running fine in serial, but there is a problem when running with parallel:

…
dipole: A dipole layer will be introduced in the vacuum
dipole: region to compensate the system dipole
Dipole moment in unit cell = 0.0000 0.0000 0.0000 D
Electric field for dipole correction = 0.000000 0.000000 -0.000000 eV/Ang/e
Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers.
Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers.
Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers.
Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers.
Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers.
Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers.
Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers.
Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers.
Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers.
Intel MKL FATAL ERROR: Cannot load symbol MKLMPI_Get_wrappers.

The error gets printed once by each core. I’m not sure if this is an error introduced due to incorrect linking when compiling (the compilation went quite smoothly), or it is just an error from running the code incorrectly. When I run the code, I have

module load intel/23.0.0-fasrc01 intel-mkl/23.0.0-fasrc01 openmpi/4.1.4-fasrc01
export LD_LIBRARY_PATH=/path/to/siesta/Docs/build/lib:$LD_LIBRARY_PATH

in my submission script, and am running with mpirun -np $numtasks siesta …

I compiled with cmake using the following options:

module load intel/23.0.0-fasrc01 intel-mkl/23.0.0-fasrc01 openmpi/4.1.4-fasrc01

Point to libraries

LIBXC_ROOT=/path/to/lib/libs-libxc
GRIDXC_ROOT=/path/to/siesta/External/libgridxc
XMLF90_ROOT=/path/to/siesta/External/xmlf90
PSML_ROOT=/path/to/siesta/External/libpsml
LIBFDF_ROOT=/path/to/siesta/External/libfdf

MKL STUFF

MKLPATH=${MKLROOT}/lib/intel64
SCALAPACK=${MKLPATH}/libmkl_scalapack_lp64.a
BLACS=${MKLPATH}/libmkl_blacs_openmpi_lp64.a
LAPACK=${MKLPATH}/libmkl_lapack95_lp64.a

cmake -S. -Bbuild
-DCMAKE_INSTALL_PREFIX=/path/to/siesta/siesta/build
-DCMAKE_PREFIX_PATH=“$XMLF90_ROOT;$PSML_ROOT;$GRIDXC_ROOT;$LIBXC_ROOT;$LIBFDF_ROOT”
-DSCALAPACK_LIBRARY=“-lmkl_scalapack_lp64;$BLACS”
-DWITH_LIBXC=ON
-DWITH_LUA=OFF -DWITH_FLOOK=OFF
-DLAPACK_LIBRARY=“$LAPACK”
-DLIBFDF_FIND_METHOD=source
-DLIBFDF_SOURCE_DIR=/path/to/siesta/External/libfdf
-DNetCDF_ROOT=/path/to/siesta/Docs/build \

Can anyone advise?

Thanks,

Daniel Bennettt

2

Hi,

Did you try with the current HEAD of master? A few Cmake-related fixes have been merged this week.

3

Hi Yann,

I rebuilt with the new master branch and it seems to be working now! (for a a small number of cores, <= 10, haven’t tested with larger calculations).

One thing I noticed:

Leaving BLAS_LIBRARY and LAPACK_LIBRARY blank, cmake was able to correctly link to them, and put the right flags. But this didn’t work with SCALAPACK_LIBRARY.

I tried adding $MKLROOT/lib/intel64, which is where I think scalapack is located, into CMAKE_PREFIX_PATH, and I tried to directly specify SCALAPACK LIBRARY, pointing to $MKLROOT/.../libmkl_scalapack_lp64.so, but it wasn’t able to correctly link. In the end I put

-DSCALAPACK_LIBRARY="-lmkl_scalapack_lp64;$BLACS"

where $BLACS is pointing to $MKLROOT/.../libmkl_blacs_openmpi_lp64.so, and it worked.

I’m not sure if this is a problem with the compilation scripts, or if it was some mistake on my end, but I would be interested to see if anyone else has experienced problems compiling the new version and linking with scalapack

Daniel

4

Hello,
I believe the MKL problems is due to the BLACS lib.

I’m also compiling the mpi version with master branch following the above mentioned setup.(Actually pretty helpful)

When $BLACS is pointing to $MKLROOT/.../libmkl_blacs_openmpi_lp64.a, the static lib, the problem happens, but for $MKLROOT/.../libmkl_blacs_openmpi_lp64.so , the dynamic lib, with everything else unchanged, no problem runing in parall.

The reason, I think, shall be that BLACS is required by mpirun during runtime, and that requires a dynamic lib to be linked.