ABACUS-3.2的安装方法

量子御坂

已于 2023-05-22 21:18:07 修改

阅读量1.8k

点赞数 1

文章标签：服务器

于 2023-05-22 21:08:44 首次发布

本文链接：https://blog.csdn.net/weixin_44471577/article/details/130815345

版权

Last Update on 2023.04.22
总结一下自己对ABACUS的安装部署方法

软件介绍

ABACUS Documentation — ABACUS documentation (deepmodeling.com)
Abacus (ustc.edu.cn)

to do :

探索无MKL环境下abacus+deepks正确编译方式（暂放）
探索编译得到的abacus的deepks和deepmd具体支持情况
探索abacus对接DP势函数跑MD相比lammps的优势与不足
探索abacus在计算催化方面的应用优势与未来

ABACUS安装方法简介

Easy Installation — ABACUS documentation (deepmodeling.com)

官网推荐采用OneAPI安装，
MKL里面包含了LAPACK, ScaLAPACK和BLAS需求，
并自带MPI。
但这估计是因为目前主流服务器是intel-u。
AMD-u上oneAPI性能一般来说是略逊于OpenMPI+Openblas思路的（）
上述测试结果来自对cp2k-2023.1的测试

四月到五月以来一系列测试表明，GNU工具链对ABACUS是最好的，OneAPI工具链运行效率相对偏慢，
并且OneAPI工具链不能很好地实现OMP并行，而ABACUS恰恰又是一个特别地在MPI+OMP并行下运行效率最高的软件。

目前采用GNU方法
（即Openmpi + OpenBLAS + ScaLAPACK 方法）
编译安装得到的abacus最稳定且运行速度最快。
这点跟陈老师确认过，在系列测试中也表现明显
在Intel服务器上测试结果：OneAPI慢10%（Intel 8358）
在AMD服务器上测试结果：OneAPI慢50%（AMD 7b12）

具体测试数据 by SI_LCAO_cell-relax (简单改过STRU文件)
in Intel Platnum 8358 16cores
- OneAPI 282s
- OpenMPI MPI16 268s
- OpenMPI MPI8-OMP2 232s
- OpenMPI MPI4-OMP4 223s
in AMD EPYC 7b12
- OneAPI 452s （加了MKL-hacking规避对Intel核的检查）
- OpenMPI MPI16 305s
- OpenMPI MPI4-OMP4 213s

注：如果要安装DeePKS特性的话在build的时候需要在build时加入mkl环境；
如果要装DeePMD特性的话要求已经有装好的DeePMD-python环境且GCC版本>=11.1（支持libstdc++.so.6.29）

说实话本次挑战abacus的编译安装也正是在cp2k的编译安装中有感而发，有许多软件包都是采用cp2k-toolchain先装上去的，稍加修改即可直接用于abacus的编译安装，GNU工具链中主要难点ScaLAPACK和ELPA均可采用cp2k-toolchain直接装上。

文中提到的“toolchain方法”均指利用cp2k-toolchain系列脚本，对依赖进行安装，这个方法可以参见我之后可能会鸽的“cp2k高级安装方法”文章。

GNU方法

0. cmake

采用cmake 3.25.1

可以并建议直接安装，用官网的.sh脚本和gcc编译安装都可以。也可以通过 cp2k 的 toolchain 方法安装得到。

在$cp2k/tools/toolchain/scripts/stage1/install_cmake.sh中修改：

# omitted 
    echo "==================== Installing CMake ===================="
    cmake_ver="3.25.1"
    if [ "${OPENBLAS_ARCH}" = "arm64" ]; then
      cmake_arch="linux-aarch64"
      cmake_sha256="8491a40148653b99877a49bf5ad6b33b595acc58f7ad2f60b659b63b38bb2cbf"
    elif [ "${OPENBLAS_ARCH}" = "x86_64" ]; then
      cmake_arch="linux-x86_64"
      cmake_sha256="6598da34f0e3a0f763809e25cfdd646aa1d5e4d133c4277821e63ae5cfe09457"
    else
      report_error ${LINENO} \
        "cmake installation for ARCH=${ARCH} is not supported. You can try to use the system installation using the flag --with-cmake=system instead."
      exit 1
    fi
    #pkg_install_dir="${INSTALLDIR}/cmake-${cmake_ver}"
    pkg_install_dir="/data/softwares/cmake/${cmake_ver}"
    install_lock_file="$pkg_install_dir/install_successful"
# omitted

1. Openmpi

采用openmpi/4.1.4，

在$cp2k/tools/toolchain/scripts/stage1/install_openmpi.sh中修改：

# omitted
case "${with_openmpi}" in
  __INSTALL__)
    echo "==================== Installing OpenMPI ===================="
    #pkg_install_dir="${INSTALLDIR}/openmpi-${openmpi_ver}"
    pkg_install_dir="/data/softwares/openmpi/${openmpi_ver}"
    install_lock_file="$pkg_install_dir/install_successful"
    if verify_checksums "${install_lock_file}"; then
# omitted

当然openmpi也可以自己直接装，在configure的时候没啥需要特别设置的

2. OpenBLAS

采用openblas-0.3.21

并类似上述的手动修改脚本内的安装位置
（同时建议手动调优编译选项）

# - line 27
case "${with_openblas}" in
  __INSTALL__)
    echo "==================== Installing OpenBLAS ===================="
    #pkg_install_dir="${INSTALLDIR}/openblas-${openblas_ver}"
    pkg_install_dir="/data/libraries/openblas/${openblas_ver}_cp2k"
    install_lock_file="$pkg_install_dir/install_successful"
# omitted

openblas也可以自己根据toolchain脚本的选项自行cmake安装，要写进去的参数不多

3. FFTW3

FFTW3的安装则相对特殊，不能用 cp2k toolchain的默认设置，因为ABACUS同时需要float版本和double版本的fftw3

具体安装方法：

下载安装fftw.3.3.10.tar.gz安装包，这个安装包可以用cp2k离线安装的安装包
解压，进入主目录，用如下脚本，在PREFIX处同时安装float版本和double版本

#!/bin/bash
# for install fftw float/double version, for abacus
# Author: JamesMisaka 
# server: AMD-41 date: 2023-04-02

module load openblas openmpi cmake

PREFIX=/data/libraries/fftw/3.3.10

# optimaztion options
# abacus 3.2 required float and double version

# install float(single) version
FFTW_FLAGS="--enable-openmp --enable-mpi --enable-shared --enable-static --enable-float --enable-avx2 --enable-avx --enable-threads --enable-fma --enable-sse --enable-sse2"

# if --enable-sse and --enable-sse2, use float version & single version
# float & double version seems to should be installed separately

# if use intel
# CC=icc F90=ifort F77=ifort

# cleanup
make distclean

# install
./configure --prefix=$PREFIX $FFTW_FLAGS | tee configure_float.log
make -j `nproc` | tee make_float.log
make -j `nproc` install | tee install_float.log

# install double version
FFTW_FLAGS="--enable-openmp --enable-mpi --enable-shared --enable-static --enable-avx2 --enable-avx --enable-threads --enable-fma"
make distclean
# install
./configure --prefix=$PREFIX $FFTW_FLAGS | tee configure_double.log
make -j `nproc` | tee make_double.log
make -j `nproc` install | tee install_double.log

要求已经装好openblas和openmpi

4. LibXC

2023-04-22更新

建议采用cmake方法自行安装：使用如下脚本：

#!/bin/bash
# install libxc by gnu
# JamesMisaka
# from https://gitlab.com/jxzou/qcinstall/-/blob/main/%E7%A6%BB%E7%BA%BF%E5%AE%89%E8%A3%85CP2K-2022.2_Intel%E7%BC%96%E8%AF%91%E5%99%A8%E7%89%88.md

rm -rf build

module load openmpi openblas cmake

mkdir build && cd build

LIBXC=/data/libraries/libxc/6.0.0

cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$LIBXC \
-DBUILD_SHARED_LIBS=YES -DCMAKE_C_COMPILER=gcc \
-DCMAKE_Fortran_COMPILER=gfortran \
-DCMAKE_INSTALL_LIBDIR=lib -DENABLE_FORTRAN=ON \
-DENABLE_CXX=ON -DENABLE_PYTHON=ON \
-DBUILD_TESTING=NO ..

make -j24
make install

也可以用toolchain方法安装，但需要加入参数。建议和openmpi, openblas一起用toolchain方法一次装上

在$cp2k/tools/toolchain/scripts/stage3/install_openmpi.sh中
修改安装位置和设置，去掉--disable-lxc（似乎是非必要项）, 加入--enable-shared

# omitted
      # CP2K does not make use of fourth derivatives, so skip their compilation with --disable-lxc
      ./configure --prefix="${pkg_install_dir}" --libdir="${pkg_install_dir}/lib" --enable-shared\
        > configure.log 2>&1 || tail -n ${LOG_LINES} configure.log
      make -j $(get_nprocs) > make.log 2>&1 || tail -n ${LOG_LINES} make.log
      make install > install.log 2>&1 || tail -n ${LOG_LINES} install.log
# omitted

如果不加入enable-shared，此libxc将不被abacus识别。但目前实测加入enable-shared也没法识别，原因不明，依然建议用cmake方法安装。

5. ScaLAPACK

采用toolchain方法安装即可，建议和openmpi, openblas一起用toolchain方法一次装上。

在$cp2k/tools/toolchain/scripts/stage1/install_openmpi.sh中
修改安装位置和设置，
修改cmake选项-DBUILD_SHARED_LIBS=YES (默认NO)

# omitted
case "$with_scalapack" in
  __INSTALL__)
    echo "==================== Installing ScaLAPACK ===================="
    #pkg_install_dir="${INSTALLDIR}/scalapack-${scalapack_ver}"
    pkg_install_dir="/data/libraries/scalapack/${scalapack_ver}"
    install_lock_file="$pkg_install_dir/install_successful"
    if verify_checksums "${install_lock_file}"; then
# omitted
      if ("${FC}" --version | grep -q 'GNU'); then
        flags=$(allowed_gfortran_flags "-fallow-argument-mismatch")
      fi
      FFLAGS=$flags cmake -DCMAKE_FIND_ROOT_PATH="$ROOTDIR" \
        -DCMAKE_INSTALL_PREFIX="${pkg_install_dir}" \
        -DCMAKE_INSTALL_LIBDIR="lib" \
        -DBUILD_SHARED_LIBS=YES \ # default no ,need change
        -DCMAKE_BUILD_TYPE=Release .. \
        -DBUILD_TESTING=NO \
        -DSCALAPACK_BUILD_TESTS=NO \
        > configure.log 2>&1 || tail -n ${LOG_LINES} configure.log
# omitted

6. ELPA

采用toolchain方法安装即可，但有多处要修改

ABACUS 暂时不兼容elpa-2022.11.001，需要用2021.11.002.:
elpa-2022中有函数名字发生了改变才导致不兼容，据说可以手动修改，开发者似乎在issue上面提到它们打算利用elpa的新特性,但总归暂时不太行。需要到 Index of /static/downloads (cp2k.org)处下载elpa-2021.11.002.tar.gz
将它copy到toolchain/build文件夹下
采用toolchain方法安装:
最好和 openmpi, openblas，scalapack 一起用 toolchain 方法一次装上
不然后续用 system 的 openmpi, openblas 的时候容易编译报错
原因不明，可能来自于环境变量设置

在$cp2k/tools/toolchain/scripts/stage1/install_openmpi.sh中修改：

elpa版本，安装位置
configure选项，打开enable-shared

# omitted
# From https://elpa.mpcdf.mpg.de/software/tarball-archive/ELPA_TARBALL_ARCHIVE.html
# elpa_ver="2022.11.001"
elpa_ver="2021.11.002"
#elpa_sha256="35e397d7c0af95bb43bc7bef7fff29425c1da400fa0cd86ae8d3bd2ff2f9d999"
# omitted
case "$with_elpa" in
  __INSTALL__)
    echo "==================== Installing ELPA ===================="
    #pkg_install_dir="${INSTALLDIR}/elpa-${elpa_ver}"
    pkg_install_dir="/data/libraries/elpa/${elpa_ver}"
    install_lock_file="$pkg_install_dir/install_successful"
    enable_openmp="yes"
# omitted
        cd "build_${TARGET}"
        ../configure --prefix="${pkg_install_dir}/${TARGET}/" \
          --libdir="${pkg_install_dir}/${TARGET}/lib" \
          --enable-openmp=${enable_openmp} \
          --enable-shared=yes \ # default no : need change
          --enable-static=yes \
          --disable-c-tests \
          --disable-cpp-tests \
# omitted

安装完毕之后进入到elpa/include/目录下，进行如下操作：

ln -s ./elpa_openmp-2021.11.002 ./elpa

创建软链接使abacus能识别到elpa的include

7. toolchain检查

这些都做完之后就可以采用toolchain方法安装了。
由于FFTW的特别性，可能最好先用toolchain方法或自行编译方法安装cmake ,openmpi, openblas以方便编译。
为了确认，最终也可以再跑一次toolchain安装，具体方法

#!/bin/bash
# install cp2k as openmpi-openblas
# can be used to install dependence of gpaw and abacus

module load fftw plumed
# 这里load plumed是因为我plumed也是自定义路径的
# 也可以install，只要有token就不会覆盖安装
# libint同理也是自定义路径的，但不会覆盖安装

./install_cp2k_toolchain.sh --with-openblas=install \
        --with-intel=no \
        --with-gcc=system \
        --with-cmake=install \
        --with-openmpi=install \
        --with-scalapack=install \
        --with-elpa=install \
        --with-libint=install \
        --with-libxc=install \
        --with-sirius=no \
        --with-fftw=system \
        --with-plumed=system \

8. CEREAL

直接官网下载即可，在ABACUS安装时指定对应include文件夹路径就行

9. ABACUS

终于，我们来到了本体。在上述动作都做完之后，通过module方法或其他方法将上述库加入到环境变量中，通过如下脚本安装：

#!/bin/bash
# compile and install abacus-3.2.0
# JamesMisaka in 2023.04.02
# TMC AMD-41 Server

# rm -rf ./build/

module load cmake fftw openmpi/4.1.4 openblas elpa/2021.11.002 libxc scalapack
# needed：cmake, openmpi, openblas , libxc(sometimes)

PREFIX=/data/softwares/abacus/3.2.0_openmpi
LAPACK=/data/libraries/openblas/0.3.21_cp2k/lib
SCALAPACK=/data/libraries/scalapack/2.2.1/lib
#ELPA=/data/libraries/elpa/2022.11.001/cpu
ELPA=/data/libraries/elpa/2021.11.002/cpu
FFTW3=/data/libraries/fftw/3.3.10
CEREAL=/data/libraries/cereal/include/cereal
LIBXC=/data/libraries/libxc/6.0.0

CXX=mpicxx

cmake -B build -DCMAKE_INSTALL_PREFIX=$PREFIX \
        -DMPI_CXX_COMPILER=mpicxx \
        -DLAPACK_DIR=$LAPACK \
        -DSCALAPACK_DIR=$SCALAPACK \
        -DELPA_DIR=$ELPA \
        -DFFTW3_DIR=$FFTW3 \
        -DCEREAL_INCLUDE_DIR=$CEREAL \
        -DLibxc_DIR=$LIBXC \
        -DENABLE_LCAO=ON \
        -DENABLE_LIBXC=ON \
        -DUSE_OPENMP=ON \
        -DENABLE_ASAN=OFF \
        -DUSE_ELPA=ON | tee configure.log

cmake --build build -j `nproc` | tee build.log
cmake --install build | tee install.log

即可将abacus成功安装在PREFIX下。
目前服务器上的abacus运算测试均通过

注：目前abacus存在内存泄漏问题，如果在编译的时候，
加入ENABLE_ASAN=on选项就能看到，
不过这个选项是用于开发的，在生产中不推荐，会拖慢速度，
这方面的问题有待开发者进一步更新

OneAPI方法

基于Intel-OneAPI方法进行安装是ABACUS教程推荐的安装方法。

具体流程

1. 使用toolchain方法编译安装elpa，单独安装libxc

如果对cp2k安装有印象的话就知道oneapi装cp2k时，
libint是需要单独安装的，不过我们abacus也用不到libint，这里也可以默认安装

#!/bin/bash
# install cp2k as oneapi, also use for abacus and gpaw

module load icc mkl mpi compiler plumed
module load cmake libint
./install_cp2k_toolchain.sh --with-openblas=no \
        --with-intel=system \
        --with-gcc=no \
        --with-cmake=system \
        --with-openmpi=no \
        --with-scalapack=system \
        --with-libint=system \
        --with-libxc=install \
        --with-sirius=no \
        --with-elpa=install \
        --with-fftw=system \
        --with-plumed=system \
        --with-quip=no \
        --with-pexsi=no \
        --with-superlu=no \
        --with-ptscotch=no

在使用toolchain安装之前所需要做的脚本调整与之前一致，
需要注意intel-OneAPI编译的elpa和libxc要与gcc-openblas编译的elpa和libxc区分开来，工具链之间不能杂化（）

libxc建议采用cmake方法单独安装：

#!/bin/bash
# build by oneapi
mkdir build && cd build

module load mkl mpi icc compiler cmake

cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$HOME/lib/libxc/6.0.0_oneapi \
-DBUILD_SHARED_LIBS=YES -DCMAKE_C_COMPILER=icc \
-DCMAKE_Fortran_COMPILER=ifort -DCMAKE_CXX_COMPILER=icpc \
-DCMAKE_INSTALL_LIBDIR=lib -DENABLE_FORTRAN=ON -DENABLE_PYTHON=ON -DENABLE_CXX=ON \
-DBUILD_TESTING=NO ..

make -j `nproc`
make install

2.编译安装

在abacus的目录下运行如下脚本，注意确认路径

#!/bin/bash
# install abacus by oneapi
# JamesMisaka in 2023-04-02
# TMC AMD-41 Server

# using oneapi
module load cmake mkl icc mpi compiler elpa/2021.11.002_oneapi libxc

PREFIX="/data/softwares/abacus/3.2.0_oneapi"
ELPA="/data/libraries/elpa/2021.11.002_oneapi/cpu"
CEREAL="/data/libraries/cereal/include/cereal"
LIBXC="/data/libraries/libxc/6.0.0_oneapi/"

# if ELPA_INCLUDE_DIR not find: do 
# ln -s $ELPA/include/elpa-****/elpa $ELPA/include/elpa

CC=icc
CXX=icpc
F90=ifort
F77=ifort

cmake -B build -DCMAKE_INSTALL_PREFIX=${PREFIX} \
        -DMPI_CXX_COMPILER=mpiicpc \
        -DMKLROOT=$MKLROOT \
        -DELPA_DIR=${ELPA} \
        -DCEREAL_INCLUDE_DIR=${CEREAL} \
        -DLibxc_DIR=${LIBXC} \
        -DENABLE_LCAO=ON \
        -DENABLE_LIBXC=ON \
        -DUSE_OPENMP=ON \
        -DENABLE_ASAN=OFF \
        -DUSE_ELPA=ON | tee configure.log

cmake --build build -j 16 | tee build.log
cmake --install build | tee install.log

即可成功安装，目前实测运行正常，虽然在AMD41上的运行效率远低于OpenBlas版本.

关于安装过程和计算效率的一些讨论可见：
Failed to Install ABACUS · Issue #2173 · deepmodeling/abacus-develop (github.com)

安装有DeepKS支持的ABACUS

GNU方法

此前尝试openblas-openmpi安装，但一直不太行
直接编译安装时，在configure阶段不会报错，但在build阶段报错：

-- Build files have been written to: /data/softwares/abacus/abacus-develop/build
[  0%] Built target psi
[  1%] Built target hamilt_general
[  1%] Built target symmetry
[  2%] Built target tddft
[  3%] Built target hamilt_ofdft
[  4%] Built target genelpa
[  6%] Built target neighbor
[  7%] Built target driver
[  9%] Built target vdw
[ 14%] Built target device
[ 17%] Built target planewave
[ 19%] Built target hsolver
[ 20%] Built target operator_ks_pw
[ 22%] Built target hamilt_stodft
[ 25%] Built target orb
[ 29%] Built target dftu
[ 31%] Built target xc_
[ 35%] Built target cell
[ 36%] Built target md
[ 39%] Built target surchem
[ 42%] Built target io_advanced
[ 46%] Built target relax
[ 57%] Built target base
[ 62%] Built target gint
[ 71%] Built target io_basic
[ 78%] Built target hamilt_pwdft
[ 80%] Built target operator_ks_lcao
[ 85%] Built target esolver
[ 90%] Built target elecstate
[ 90%] Building CXX object source/module_hamilt_lcao/module_deepks/CMakeFiles/deepks.dir/LCAO_deepks_io.cpp.o
[ 98%] Built target hamilt_lcao
[100%] Built target deepks
[100%] Building CXX object CMakeFiles/abacus.dir/source/main.cpp.o
[100%] Linking CXX executable abacus
/usr/bin/ld: cannot find -lmkl_intel_ilp64
/usr/bin/ld: cannot find -lmkl_intel_thread
/usr/bin/ld: cannot find -lmkl_core
collect2: error: ld returned 1 exit status
gmake[2]: *** [CMakeFiles/abacus.dir/build.make:813: abacus] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:696: CMakeFiles/abacus.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

原因集中于libtorch在使用时，会自动寻找环境里的Intel mkl, 似乎libtorch的已编译版都是基于mkl编译的。要整这个的话可能得自己用openblas编译libtorch，abacus开发者提供的dockerfile里面有纯gnu版本的，据说可以，暂未尝试。

于是想到了一个曲线救国的方法，我在configure的时候不加入mkl环境，但在build的时候加载，让libtorch能找到mkl。

#!/bin/bash
# install ABACUS with libxc and deepks
# JamesMisaka in 2023.04.02

# conda activate deepks

#rm -rf build

module load cmake fftw openmpi/4.1.4 openblas elpa/2021.11.002 libxc/6.0.0 scalapack libnpy

PREFIX=/data/softwares/abacus/3.2.0_deepks
LAPACK=/data/libraries/openblas/0.3.21_cp2k/lib
SCALAPACK=/data/libraries/scalapack/2.2.1/lib
#ELPA=/data/libraries/elpa/2022.11.001/cpu
ELPA=/data/libraries/elpa/2021.11.002/cpu
FFTW3=/data/libraries/fftw/3.3.10
CEREAL=/data/libraries/cereal/include/cereal
LIBXC=/data/libraries/libxc/6.0.0
LIBTORCH=/data/libraries/libtorch/2.0.0/share/cmake/Torch
LIBNPY=/data/libraries/libnpy/0.1.0/include/

# in docker.gnu file
#export OMPI_MCA_btl_vader_single_copy_mechanism=none

export CMAKE_PREFIX_PATH=/data/libraries/libtorch/2.0.0/share/cmake/

CXX=mpicxx

cmake -B build -DCMAKE_INSTALL_PREFIX=$PREFIX \
        -DMPI_CXX_COMPILER=mpicxx \
        -DLAPACK_DIR=$LAPACK \
        -DSCALAPACK_DIR=$SCALAPACK \
        -DELPA_DIR=$ELPA \
        -DFFTW3_DIR=$FFTW3 \
        -DCEREAL_INCLUDE_DIR=$CEREAL \
        -DLibxc_DIR=$LIBXC \
        -DENABLE_LCAO=ON \
        -DENABLE_LIBXC=ON \
        -DUSE_OPENMP=ON \
        -DENABLE_ASAN=OFF \
        -DUSE_ELPA=ON \
        -DENABLE_DEEPKS=1 \
        -DTorch_DIR=$LIBTORCH \
        -Dlibnpy_INCLUDE_DIR=$LIBNPY \
        | tee configure.log

# add mkl env for libtorch to link
module load mkl

cmake --build build -j `nproc` | tee build.log
cmake --install build | tee install.log

编译通过。

经简单测试，计算速度基本在OpenBLAS-OpenMPI版ABACUS正常水平。对deepks的具体支持有待进一步测试

OneAPI方法

采用Intel-OneAPI则简单直接，只需要加入libtorch和libnpy的路径即可。注意一定要加载libnpy环境

#!/bin/bash
# install abacus by oneapi with deepks feature
# JamesMisaka in 2023-04-02
# Server: TMC-AMD41

# using oneapi
module load cmake mkl icc mpi compiler elpa/2021.11.002_oneapi libnpy libxc/6.0.0_oneapi

PREFIX="/data/softwares/abacus/3.2.0_oneapi/"
ELPA="/data/libraries/elpa/2021.11.002_oneapi/cpu"
CEREAL="/data/libraries/cereal/include/cereal"
LIBXC="/data/libraries/libxc/6.0.0_oneapi"
LIBTORCH="/data/libraries/libtorch/2.0.0/share/cmake/Torch"
LIBNPY="/data/libraries/libnpy/0.1.0/include"


# if ELPA_INCLUDE_DIR not find: do 
# ln -s $ELPA/include/elpa-****/elpa $ELPA/include/elpa

CC=icc
CXX=icpc
F90=ifort
F77=ifort

cmake -B build -DCMAKE_INSTALL_PREFIX=${PREFIX} \
        -DMKLROOT=$MKLROOT \
        -DELPA_DIR=${ELPA} \
        -DCEREAL_INCLUDE_DIR=${CEREAL} \
        -DLibxc_DIR=${LIBXC} \
        -DENABLE_LCAO=ON \
        -DENABLE_LIBXC=ON \
        -DUSE_OPENMP=ON \
        -DENABLE_ASAN=OFF \
        -DENABLE_DEEPKS=1 \
        -DTorch_DIR=$LIBTORCH \
        -Dlibnpy_INCLUDE_DIR=$LIBNPY \
        -DUSE_ELPA=ON | tee configure.log

cmake --build build -j 16 | tee build.log
cmake --install build | tee install.log

安装有DeepMD支持的ABACUS

此处展示GNU方法：
注意：要求gcc版本为11.3及以上，或支持GLIBCXX-3.4.29

#!/bin/bash
# install ABACUS with libxc, deepks, deepmd
# JamesMisaka in 2023.04.23
# platform: AMD-3950X WSL2

# conda activate deepmd

# rm -rf build

module load cmake openmpi/4.1.4 openblas fftw scalapack libxc/6.0.0 libnpy

PREFIX=$HOME/apps/abacus/3.2.0
LAPACK=$HOME/math_lib/openblas/0.3.21/lib
SCALAPACK=$HOME/math_lib/scalapack/2.2.1/lib
#ELPA=/data/libraries/elpa/2022.11.001/cpu
ELPA=$HOME/math_lib/elpa/2021.11.002/cpu
FFTW3=$HOME/math_lib/fftw/3.3.10
CEREAL=$HOME/math_lib/cereal/include/cereal
LIBTORCH=$HOME/math_lib/libtorch/2.0.0/share/cmake/Torch
LIBNPY=$HOME/math_lib/libnpy/0.1.0/include
LIBXC=$HOME/math_lib/libxc/6.0.0
DEEPMD=$HOME/apps/anaconda3/envs/deepmd

# in docker.gnu file
#export OMPI_MCA_btl_vader_single_copy_mechanism=none

#export CMAKE_PREFIX_PATH=/data/libraries/libtorch/1.12.1/share/cmake/

CXX=mpicxx

cmake -B build -DCMAKE_INSTALL_PREFIX=$PREFIX \
        -DMPI_CXX_COMPILER=mpicxx \
        -DLAPACK_DIR=$LAPACK \
        -DSCALAPACK_DIR=$SCALAPACK \
        -DELPA_DIR=$ELPA \
        -DFFTW3_DIR=$FFTW3 \
        -DCEREAL_INCLUDE_DIR=$CEREAL \
        -DLibxc_DIR=$LIBXC \
        -DENABLE_LCAO=ON \
        -DENABLE_LIBXC=ON \
        -DUSE_OPENMP=ON \
        -DENABLE_ASAN=OFF \
        -DUSE_ELPA=ON \
        -DENABLE_DEEPKS=1 \
        -DTorch_DIR=$LIBTORCH \
        -Dlibnpy_INCLUDE_DIR=$LIBNPY \
        -DDeePMD_DIR=$DEEPMD \
        -DTensorFlow_DIR=$DEEPMD \
        | tee configure.log

# add mkl env for libtorch to link
module load mkl

cmake --build build -j `nproc` | tee build.log
cmake --install build | tee install.log

OneAPI方法同理。

此方法安装deepmd支持的时候会有warning：

-- The CXX compiler identification is GNU 11.1.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found git: attempting to get commit info...
-- Current commit hash: 05ea8d307
-- Last commit date: Sat May 20 09:46:47 2023
-- Found Cereal: /home/james/math_lib/cereal/include/cereal
-- Found ELPA: /home/james/math_lib/elpa/2021.11.002/cpu/lib/libelpa_openmp.so
-- Performing Test ELPA_VERSION_SATISFIES
-- Performing Test ELPA_VERSION_SATISFIES - Success
-- Found MPI_CXX: /home/james/compiler_lib/openmpi/4.1.4/lib/libmpi.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - /usr/local/cuda/bin/nvcc
-- Found FFTW3: /home/james/math_lib/fftw/3.3.10/lib/libfftw3_omp.so
-- Found LAPACK: /home/james/math_lib/openblas/0.3.21/lib/libopenblas.so
-- Found ScaLAPACK: /home/james/math_lib/scalapack/2.2.1/lib/libscalapack.so
-- Found Torch: /home/james/math_lib/libtorch/2.0.0/lib/libtorch.so
-- Found Libxc: version 6.0.0
-- Found DeePMD: /home/james/apps/anaconda3/envs/deepmd/lib/libdeepmd_cc.so
-- Found TensorFlow: /home/james/apps/anaconda3/envs/deepmd/lib/libtensorflow_cc.so
-- Configuring done
CMake Warning at CMakeLists.txt:104 (add_executable):
  Cannot generate a safe runtime search path for target abacus because files
  in some directories may conflict with libraries in implicit directories:

    runtime library [libgomp.so.1] in /usr/lib/gcc/x86_64-linux-gnu/11 may be hidden by files in:
      /home/james/apps/anaconda3/envs/deepmd/lib

  Some of these libraries may not be found correctly.


-- Generating done
-- Build files have been written to: /home/james/source-code/abacus-develop/build