How to build CP2K with Intel Compiler 12.1, OpenMPI-1.4.3 and Infiniband Support

http://hpc.xrqtc.com/index.php/articles-tutorials/192-how-to-build-cp2k-with-intel-compiler-121-openmpi-143-and-infiniband-support

How to build CP2K with Intel Compiler 12.1, OpenMPI-1.4.3 and Infiniband Support

About CP2K

CP2K is a free available (GPL) program, written in Fortran 95, to perform atomistic and molecular simulations of solid state, liquid, molecular and biological systems. It provides a general framework for different methods such as e.g. density functional theory (DFT) using a mixed Gaussian and plane waves approach (GPW), and classical pair and many-body potentials.

Official website: http://cp2k.berlios.de/

This document explains how to build CP2K program using Intel Compiler 12.1, OpenMPI-1.4.3 with Infiniband support. The compilation explained has been done in an AMD Opteron 6140 machine. The next software has been used:

• Intel Compiler 12.1 (Composer XE 2011)
• OpenMPI 1.4.3*

*OpenMPI was built with Intel Compiler 12.1, OFED-1.5.3 and adding Torque/PBS flags

Environment Set Up

We recommend to use a screen session that allows to reattach a background session.

#screen -S cp2k-intel-ompi
#tar -xvf cp2k.tar.gz

Then, we have to load the required modules.

#. /opt/modules/init/bash
#module load intel/composer_xe_2011_sp1.6.233
#module load openmpi/1.4.3_intel2011

Build FFTW wrapper library for Intel MKL

First of all, you need to build the FFTW wrapper library for Intel MKL.

#cd /opt/compilers/intel/composer_xe_2011_sp1.6.233/mkl/interfaces/fftw3xf/
#make libem64t compiler=intel

The wrapper library libfftw3xf_intel.a is created in this directory. You can link this wrapper to /opt/compilers/intel/composer_xe_2011_sp1.6.233/mkl/lib/intel64.

Prepare the makefile

You can modify Linux-x86-64-intel.popt file found in cp2k/arch/.

#cd cp2k/arch/
#cp Linux-x86-64-intel.popt Linux-x86-64-intel.popt.orig
#vi Linux-x86-64-intel.popt

Edit Linux-x86-64-intel.popt. BLACS, SCALAPACK, BLAS and LAPACK libraries provided by Intel MKL are recommended. You need to specify where your MKL libraries are.

The Linux-x86-64-intel.popt file should be similar to this:

CC       = cc
CPP      =
FC       = mpif90
LD       = mpif90
AR       = ar -r

INTEL_MKL=/opt/compilers/intel/composer_xe_2011_sp1.6.233/mkl
INTEL_INC = $(INTEL_MKL)/include/fftw
INTEL_LIB = $(INTEL_MKL)/lib/intel64

DFLAGS   = -D__INTEL -D__FFTSG -D__parallel -D__BLACS -D__SCALAPACK -D__FFTW3
CPPFLAGS =
FCFLAGS  = $(DFLAGS) -I$(INTEL_INC) -O3 -msse2 -heap-arrays 64 -funroll-loops -fpp -free
FCFLAGS2 = $(DFLAGS) -I$(INTEL_INC) -O1 -msse2 -heap-arrays 64 -fpp -free
LDFLAGS  = $(FCFLAGS) -I$(INTEL_INC)
LIBS     = -L$(INTEL_LIB) -lfftw3xf_intel \
-L${INTEL_LIB} -I${INTEL_INC} -I${INTEL_INC}/intel64/lp64 -lmkl_blas95_lp64 \
-L${INTEL_LIB} -I${INTEL_INC} -I${INTEL_INC}/intel64/lp64 -lmkl_lapack95_lp64 \
${INTEL_LIB}/libmkl_scalapack_lp64.a ${INTEL_LIB}/libmkl_solver_lp64_sequential.a -Wl,--start-group ${INTEL_LIB}/libmkl_intel_lp64.a ${INTEL_LIB}/libmkl_sequential.a ${INTEL_LIB}/libmkl_core.a ${INTEL_LIB}/libmkl_blacs_openmpi_lp64.a -Wl,--end-group -lpthread

OBJECTS_ARCHITECTURE = machine_intel.o

graphcon.o: graphcon.F
$(FC) -c $(FCFLAGS2) $<

qs_vxc_atom.o: qs_vxc_atom.F
$(FC) -c $(FCFLAGS2) $<

Note that we use -msse2 flag instead of the -xW flag of the original Linux-x86-64-intel.popt file  because -axW, -xW or -xHost are deprecated options in the new Intel Compiler Suite.

Building the code

To build the code, you have to move to the makefiles directory

#cd cp2k/makefiles/
#make -j 4 ARCH=Linux-x86-64-intel VERSION=popt

In this case, we run 4 make jobs simultaniously.

Testing the code

There are several test examples in testing directory.

A first recommended test is to run C.inp job.

#cd cp2k/tests/QS/
#mpirun -np 8 ../../exe/Linux-x86-intel/cp2k.popt -i C.inp

This is a short test just to be sure that the executable is running without any segmentation fault problem. You can run any test you want, for instance H2O-32.inp

#cd cp2k/tests/QS/
#mpirun -np 8 ../../exe/Linux-x86-intel/cp2k.popt -i H2O-32.inp

Setting up the modulefile

This is an example of modulefile called cp2k/intel2011_ompi-1.4.3:

#%Module##############################################
proc ModulesHelp { } {
puts stderr "\tLoad environment of CP2K compiled with Intel XE Composer 2011 and openmpi-1.4.3\n"
}
module-whatis   "Load environment of CP2K"
prepend-path    PATH   /opt/applic/CP2K/intel2011-ompi-1.4.3/cp2k/exe/Linux-x86-64-intel
prepend-path    LD_LYBRARY_PATH  /opt/applic/CP2K/intel2011-ompi-1.4.3/cp2k/lib/Linux-x86-64-intel/popt
module load     openmpi/1.4.3_intel2011