在Intel Xeon Phi上编译运行OpenFOAM

1. OpenFOAM version: OpenFOAM-2.3.0

2. OpenFOAM build & run on CPU

   Refer to http://www.openfoam.com and https://openfoamwiki.net

3. OpenFOAM build on MIC

   • Refer to http://www.cfd-online.com/Forums/openfoam-installation/118552-openfoam-build-intel-xeon-phi.html
   • Make Zlib and Flex for MIC
   • Export this zlib/lib to LD_LIBRARY_PATH
   • Modify ${foam_dir}/etc/bashrc:

     foamInstall=/path/to/install
     WM_COMPILER=Icc
     WM_MPLIB=INTELMPI
     

   • Modify $FOAM_INST_DIR/wmake/rules/linux64Icc/*
     • Modify file c ${}^{[1]}$
     • Modify file c++ ${}^{[2]}$
     • Modify file c++Opt ${}^{[3]}$
   • Add source /opt/intel/composer_xe_2013_sp1.2.144/bin/compilervars_arch.sh mic to ~/.bashrc and then source it.
   • Export $MPI_ARCH_PATH=/opt/intel/impi/{version}/mic
   • Go to $FOAM_INST_DIR and execute ./Allwmake

4. OpenFOAM run on MIC

   • Copy the folder OpenFOAM-2.3.0 to MIC
   • Copy related dynamic link libraries and binary executable file to MIC
   • Modify OpenFOAM-2.3.0/etc/bashrc
     • Modify foamInstall = /home/{username}
   • Create a bashrc file in /home/{username}
     • Add $LD_LIBRARY_PATH
     • Add $PATH
     • Source /home/{username}/OpenFOAM-2.3.0/etc/bashrc

5. OpenFOAM case

   • This case is a pipe flow with 228000 cells.
   • This case has only fluid phase.
   • Detailed information on how to build and run is in readme.md of the packaged file.

---

[1] c

cFLAGS = \
-I/opt/intel/impi/4.1.1.036/mic/include/ \
-I/path/to/flex/include/ -I/path/to/zlib/include/ \
-Wl,/home/yjiang/Tools/zlib/lib/libz.so \
-Wl,/opt/mpss/3.1.2/sysroots/k1om-mpss-linux/lib64/libc.so.6 \
-Wl,/opt/mpss/3.1.2/sysroots/k1om-mpss-linux/lib64/libpthread.so.0 \
$(GFLAGS) $(cWARN) $(cOPT) $(cDBUG) $(LIB_HEADER_DIRS) -fPIC -mmic \
-I/usr/include

LINK_LIBS = -Wl,-rpath-link,/opt/mpss/3.1.2/sysroots/k1om-mpss-linux/lib64/,-rpath-link,/opt/intel/impi/4.1.1.036/mic/lib/,-rpath-link,/opt/intel/composer_xe_2013_sp1.0.080/compiler/lib/mic/,-rpath-link,/path/to/zlib/lib/,-rpath-link,/path/to/flex/lib/,-rpath-link,/path/to/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64IccDPOpt/lib/4.1.1.036/,-rpath-link,/path/to/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64IccDPOpt/lib/ \
-L/opt/intel/impi/4.1.1.036/mic/lib/ \
-L/opt/intel/composer_xe_2013_sp1.0.080/compiler/lib/mic/ \
-L/path/to/zlib/lib/ \
-L/path/to/flex/lib/ \
-L/opt/mpss/3.1.2/sysroots/k1om-mpss-linux/lib64/ \
-L/path/to/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64IccDPOpt/lib/4.1.1.036/  \
$(cDBUG) -L$(IA32ROOT)/lib

[2] c++

CC          = mpiicpc -std=c++0x

c++FLAGS    = -I/opt/intel/impi/4.1.1.036/mic/include/ \
-I/path/to/flex/include/ \
-I/path/to/zlib/include/ \
-Wl,/path/to/zlib/lib/libz.so \
$(GFLAGS) $(c++WARN) $(c++OPT) $(c++DBUG) $(ptFLAGS) $(LIB_HEADER_DIRS) -fPIC –mmic

LINK_LIBS   = -Wl,-rpath-link,/opt/intel/impi/4.1.1.036/mic/lib/,-rpath-link,/opt/mpss/3.1.2/sysroots/k1om-mpss-linux/lib64/,-rpath-link,/opt/intel/composer_xe_2013_sp1.0.080/compiler/lib/mic/,-rpath-link,/path/to/zlib/lib/,-rpath-link,/path/to/flex/lib/,-rpath-link,/path/to/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64IccDPOpt/lib/4.1.1.036/,-rpath-link,/path/to/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64IccDPOpt/lib/ \
-L/opt/intel/impi/4.1.1.036/mic/lib/ \
-L/opt/mpss/3.1.2/sysroots/k1om-mpss-linux/lib64/ \
-L/path/to/OpenFOAM/OpenFOAM-2.3.0/platfoams/linux64IccDPOpt/lib/ \
-L/path/to/OpenFOAM/OpenFOAM-2.3.0/platfoams/linux64IccDPOpt/lib/4.1.1.036/  \
-L/opt/intel/composer_xe_2013_sp1.0.080/compiler/lib/mic/ \
-L/path/to/zlib/lib/ \
-L/path/to/flex/lib/ \
$(c++DBUG) \
-L$(IA64ROOT)/lib

[3] c++Opt

c++OPT = -O2 -no-prec-div (delete –xHost)