NVIDIA上编译onednn

近期想在nvidia上测试onednn，根据网上信息和个人编译过程进行汇总，形成此文。

本文主要参考https://zhuanlan.zhihu.com/p/585129220?utm_id=0

参考链接过程基本正确，过程中需要安装部分库

安装步骤
· 安装好 nvidia driver. 已验证平台:

T4, driver 460.91.03, CUDA 11.2,
V100, driver 470.141.03, CUDA 11.4
· Build cuda device compiler

Refer to : https://github.com/intel/llvm/blob/sycl/sycl/doc/GetStartedGuide.md#build-dpc-toolchain-with-support-for-nvidia-cuda

git clone https://github.com/intel/llvm.git
cd llvm
CUDA_LIB_PATH=/usr/local/cuda/lib64/stubs CC=gcc CXX=g++ python ./buildbot/configure.py --cuda --cmake-opt="-DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda"
CUDA_LIB_PATH=/usr/local/cuda/lib64/stubs CC=gcc CXX=g++ python ./buildbot/compile.py

需要安装opencl和openmp

sudo apt install opencl-headers 
sudo apt install ocl-icd-libopencl1 
sudo apt install ocl-icd-opencl-dev
sudo apt install clinfo
#sudo apt install intel-opencl-icd  #Install NEO OpenCL runtime for Intel GPU


sudo apt install libomp-dev
sudo apt install libtbb-dev

export DPCPP_HOME=YOUR_LLVM_DIR
git clone https://github.com/oneapi-src/oneDNN.git
cd oneDNN
mkdir build && cd build //一定还要在build中操作
//export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$DPCPP_HOME/llvm/build/lib

export LD_LIBRARY_PATH=$DPCPP_HOME/llvm/build/lib:$LD_LIBRARY_PATH
export CC= $DPCPP_HOME/llvm/build/bin/clang
export CXX= $DPCPP_HOME /llvm/build/bin/clang++
cmake -DDNNL_CPU_RUNTIME=DPCPP -DDNNL_GPU_RUNTIME=DPCPP -DDNNL_GPU_VENDOR=NVIDIA -G Ninja ..
ninja

在编译过程中出现了几个问题：
1.出现error type不识别，需要在相应源码加头文件
2.出现库找不的情况
/usr/bin/ld: src/libdnnl.so.3.5: undefined reference to `tbb::detail::r1::spawn(tbb::detail::d1::task&, tbb::detail::d1::task_group_context&)’
。。。

需要修改makefile文件，修改src/Cmakefile.txt，添加find TBB库，并把TBB库加入到target_link_libraries

find_library(TBB_LIBRARY tbb)
# LINK_PRIVATE for cmake 2.8.11 compatibility
target_link_libraries(${LIB_PACKAGE_NAME} LINK_PRIVATE ${${LIB_PACKAGE_NAME}_INTERFACE} ${STATIC_LIB_DEPS} ${SHARED_LIB_DEPS}  ${TBB_LIBRARY})

在运行例子时，会出现运行未定义symbol，这是运行库不匹配导致

./primitives-lrn-cpp: symbol lookup error: /home/cmcc/onednn_test/ondnn_llvm/oneDNN/build3/src/libdnnl.so.3: undefined symbol: _ZNK4sycl3_V16detail16AccessorBaseHost25isMemoryObjectUsedByGraphEv

通过修改lib路径可解决该问题

export DPCPP_HOME=YOUR_LLVM_DIR
export LD_LIBRARY_PATH=$DPCPP_HOME/llvm/build/lib:$LD_LIBRARY_PATH