Intel Parallel Studio XE 2020 update 4¶
The Intel Parallel Studio XE Cluster Edition 2020 update 4 suite is now installed and available on Apocrita. This release will be the last of the Parallel Studio releases we support, as Intel is moving to its oneAPI model of bundling.
The suite provides releases for Intel's C, C++, and Fortran compilers. Also included are releases for MKL MPI, Python, as well as a wide array of tools. The available modules are given below. Note that MPI and Python distributions have their own, distinct, modules.
module load intel/2020.4 # Compilers, Tools
module load intelmpi/2020.4 # MPI
module load intelpython/3.7.7-2020.4 # Python
The default modules for intel
and intelmpi
remain the same, as
this version has breaking issues when using Fortran coarrays. You can read more
in the "known issues" section. The
default modules remain intel/2018.3
and intelmpi/2018.3
. If you think you
are affected by any issues and would like more specialised help or suggestions,
we would encourage you to get in touch with our
RSE team.
Release Notes¶
In this version, we are changing how MPI communications are propagating across
the InfiniBand network. In earlier releases of Intel MPI we used the general
Unix verbs
provider to enable InfiniBand communication across our nodes.
Now, the communication will be handled by the mlx
provider, which is
optimised for our Mellanox
network cards, for compatibility and hardware detection considerations.
A complete list of release notes is provided by Intel.
Known Issues¶
-
The default environment variable
I_MPI_HYDRA_TOPOLIB=hwloc
can cause bad termination faults throughSIGKILL 8/11
signals during compilation, including an emptycpuinfo
card on some nodes. Intel's suggested workaround is to setI_MPI_HYDRA_TOPOLIB=ipl
, and this has been applied in the module. There is currently no visible impact to the end user in terms of performance or utility. -
The compiler option
-fast
can cause severe issues during compilation. Static linking with optimisations should be avoided. Ideally, dynamic linking should be used along with the optimisation options-O2
and-O3
. -
Benign warnings during program execution can commonly appear and are related to environment variables created by the modules system. They are named by appending
_modshare
to the original variable. They can be safely ignored. Example:
[0] MPI startup(): I_MPI_ROOT_modshare environment variable is not supported.
Known issues using Fortran coarrays¶
Several changes implemented since the 2018 releases of the compiler
and MPI suites can affect the working of coarrays in Fortran programs as well as
their performance. At this time, we recommend avoiding using intel/2020.4
with
coarrays. Below is a non-exhaustive list of known issues we have
discovered and brought to Intel's attention.
-
Fortran coarrays runtime is affected by a longstanding bug of the underlying network structure. Running a Fortran coarray program on any
sdx
andddy
nodes will force asignal 11
segmentation fault. Intel's suggested workaround is to export the following environment variable before runtime:export MPIR_CVAR_CH4_OFI_ENABLE_RMA=0
. You can safely add it in your job submission script, before running the coarray binary. -
Programs compiled with
ifort -coarray=shared
can now spawn images across multiple nodes. In previous releases it was used exclusively for shared memory image spawning (single node). Intel has confirmed that this is a feature and not a bug, but still strongly suggests usingifort -coarray=distributed
for any multi-node runs. -
In previous releases, programs compiled with
ifort -coarray=distributed
required a run-time configuration file. This is no longer needed. -
The environment variable
I_MPI_CAF_RUNTIME
has been deprecated in the newer version. However, it has not been completely removed, leading to the following warning (safely ignored):
[0] MPI startup(): I_MPI_CAF_RUNTIME environment variable is not supported.
[0] MPI startup(): Similar variables: I_MPI_THREAD_RUNTIME
Reproducibility concerns¶
Reproducibility can be affected by some of the environment variables set in
the module files. The module files are supplied with the minimal amount of
variables set to enable the compilers to work across as many Apocrita nodes as
possible. The exact environment-setting statements used in the module file
(and any other module file) can be seen through module show <module name>
.
For any reproducibility considerations we would encourage you to get in touch
with the RSE team.
Appendix - Intel tools¶
- Enhanced GNU* Project Debugger (GDB)
- Intel Data Analytics Acceleration Library (DAAL)
- Intel Threading Building Blocks (TBB)
- Intel Integrated Performance Primitives (IPP)
- Intel Advisor
- Intel Inspector
- Intel VTune Profiler
- Intel Trace Analyzer and Collector (ITAC)
- Intel Cluster Checker (CLCK)