【G3D笔记】AI生成式3D算法相关环境安装爬坑笔记

老富2012

已于 2024-08-29 19:26:01 修改

阅读量1.3k

点赞数 15

文章标签： 3d 笔记人工智能 python

于 2024-03-18 01:17:14 首次发布

本文链接：https://blog.csdn.net/fuweipeng2012/article/details/136794036

版权

这篇博客详细记录了在安装和配置AI生成式3D算法环境过程中遇到的挑战，包括RayMarching、Tiny-Cuda-nn和PyTorch3D的安装问题。在RayMarching中，由于C++版本和CUDA版本不匹配导致错误，而Tiny-Cuda-nn的安装遇到网络问题。对于PyTorch3D，错误主要集中在CUDA版本不兼容和Python环境配置上。解决方案涵盖了从环境变量调整到特定库版本的选择和安装。此外，还涉及到了PyMeshlab、Ninja、CuMCubes和OpenGL相关的问题及解决策略。

摘要由CSDN通过智能技术生成

本章是关于生成式AI3D相关算法环境搭建过程中的错误记录，涉及技术有：
raymarching：

Ray Marching 射线步进，一种用于实时场景的快速渲染方法，可以将SDF模型显示出来。

tiny-cuda-nn：

Tiny CUDA Neural Networks，快如闪电的 C++/CUDA 神经网络框架，小型的、独立的框架，用于训练和查询神经网络。

PyTorch3D：

使用 3D 数据进行深度学习的库。

1、 RayMarching

1.1 error C1189: #error: You need C++17 to compile PyTorch

系统环境：win10

构建raymarching
python setup.py build_ext --inplace

raymarching.cu
C:/Users/USERT/miniconda3/envs/nerf2mesh/lib/site-packages/torch/include\c10/util/C++17.h(27): fatal error C1189: #error:  You need C++17 to compile PyTorch
nvcc warning : incompatible redefinition for option 'std', the last value of this option was used
raymarching.cu
ninja: build stopped: subcommand failed.

...

run_ninja_build(
  File "C:\Users\USERT\miniconda3\envs\nerf2mesh\lib\site-packages\torch\utils\cpp_extension.py", line 2112, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error compiling objects for extension

问题分析:
pytorch版本过高

解决方案：
conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.8 -c pytorch -c nvidia

1.2 raymarching安装环境版本测试

系统环境：win10

Win 10 ver.21H
Python 3.9

通过测试
conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia
conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.8 -c pytorch -c nvidia
conda install pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=11.8 -c pytorch -c nvidia

未通过测试
conda install pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=12.1 -c pytorch -c nvidia

1.3 host_config.h(231): fatal error C1083: 无法打开包括文件: “crtdefs.h”

系统环境：win10

问题分析
Windows 10 SDK 未安装或版本不正确

解决方案
Visul Studio Community 2022 -> 工具 -> 获取工具和功能 -> 使用 C++ 的桌面开发 -> 可选 -> Windows 10 SDK (10.0.20348.0)
Windows 10 SDK (10.0.20348.0) == Windows 10 专业版 21H2

SDK对应WIN版本查询
https://developer.microsoft.com/zh-cn/windows/downloads/sdk-archive/

1.4 VC environment is activated but DISTUTILS_USE_SDK is not set

          raise UserWarning(msg)
      UserWarning: It seems that the VC environment is activated but DISTUTILS_USE_SDK is not set.This may lead to multiple activations of the VC env.Please set `DISTUTILS_USE_SDK=1` and try again.

解决方案
设置VC环境变量

SET DISTUTILS_USE_SDK=1
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat

成功解决

Successfully built raymarching
Installing collected packages: raymarching
Successfully installed raymarching-0.0.0

2、Tiny-Cuda-nn

Tiny CUDA Neural Networks

2.1 win10 系统环境下pip安装

2.1.1 HTTP/2 stream 1 was not closed cleanly before end of the underlying stream

系统环境：win10
pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch

pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple/
Collecting git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
  Cloning https://github.com/NVlabs/tiny-cuda-nn/ to c:\users\usert\appdata\local\temp\pip-req-build-fydeqrnx
  Running command git clone --filter=blob:none --quiet https://github.com/NVlabs/tiny-cuda-nn/ 'C:\Users\USERT\AppData\Local\Temp\pip-req-build-fydeqrnx'
  fatal: unable to access 'https://github.com/NVlabs/tiny-cuda-nn/': HTTP/2 stream 1 was not closed cleanly before end of the underlying stream
  error: subprocess-exited-with-error

问题分析
网络不通

解决方案
需要梯子或接国内镜像

>pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple/
Collecting git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
  Cloning https://github.com/NVlabs/tiny-cuda-nn/ to c:\users\usert\appdata\local\temp\pip-req-build-ft7mgi0v
  Running command git clone --filter=blob:none --quiet https://github.com/NVlabs/tiny-cuda-nn/ 'C:\Users\USERT\AppData\Local\Temp\pip-req-build-ft7mgi0v'
  Resolved https://github.com/NVlabs/tiny-cuda-nn/ to commit 2ec562e853e6f482b5d09168705205f46358fb39
  Running command git submodule update --init --recursive -q
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: tinycudann
  Building wheel for tinycudann (setup.py) ... done
  Created wheel for tinycudann: filename=tinycudann-1.7-cp39-cp39-win_amd64.whl size=24754607 sha256=6d0047ec77071aff9b86908b512e7e613c85587ffe3b46d2d71880985ba69249
  Stored in directory: C:\Users\USERT\AppData\Local\Temp\pip-ephem-wheel-cache-ewyhba87\wheels\a9\17\8d\33c2914dcfb0b2fb1476d5d89ba4d3c896e5125178bdaeff3b
Successfully built tinycudann
Installing collected packages: tinycudann
Successfully installed tinycudann-1.7

2.1.1 C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\include\crt/host_config.h(231): fatal error C1083: 无法打开包括文件: “crtdefs.h”: No such file or directory

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.1\include\crt/host_config.h(231): fatal error C1083: 无法打开包括文件: “crtdefs.h”: No such file or directory

检查VC环境

>vcvars64.bat
'vcvars64.bat' 不是内部或外部命令，也不是可运行的程序
或批处理文件。

没有设置路径，不能运行vcvars64.bat

找到vcvars64.bat脚本
运行vcvars64.bat脚本以设置环境变量

>C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat
'C:\Program' 不是内部或外部命令，也不是可运行的程序
或批处理文件。

E:\Program_AI\G3d\torch-ngp\tiny-cuda-nn\bindings\torch>"C:\Program Files\Microsoft Visual Studio\2022\Commu
nity\VC\Auxiliary\Build\vcvars64.bat"
**********************************************************************
** Visual Studio 2022 Developer Command Prompt v17.9.3
** Copyright (c) 2022 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'

再次安装
pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch

Installed e:\program_ai\g3d\torch-ngp\embed\lib\site-packages\tinycudann-1.7-py3.11-win-amd64.egg
Processing dependencies for tinycudann==1.7
Finished processing dependencies for tinycudann==1.7

2.2 win10 系统环境下本地源码编译安装

2.2.1 手动下载源码和依赖

下载主程序
Tiny-cuda-nn 源码

下载依赖程序
Cutlass CUDA模板源码
 Fmt 格式库源码

下载解压后把 cutlass和fmt两个目录复制到 tiny-cuda-nn\dependencies里，替换dependencies里的原来的空目录
在这里插入图片描述

2.2.2 Git加参数recursive 自动下载关联依赖

git clone --recursive 将tiny-cuda-nn源码下的关联依赖一起下载到本地
git clone --recursive https://github.com/nvlabs/tiny-cuda-nn

>git clone --recursive https://github.com/nvlabs/tiny-cuda-nn
Cloning into 'tiny-cuda-nn'...
remote: Enumerating objects: 4098, done.
remote: Counting objects: 100% (1306/1306), done.
remote: Compressing objects: 100% (208/208), done.
remote: Total 4098 (delta 1121), reused 1155 (delta 1077), pack-reused 2792 (from 1)
Receiving objects: 100% (4098/4098), 19.61 MiB | 13.52 MiB/s, done.
Resolving deltas: 100% (2630/2630), done.
Submodule 'dependencies/cutlass' (https://github.com/NVIDIA/cutlass) registered for path 'dependencies/cutlass'
Submodule 'dependencies/fmt' (https://github.com/fmtlib/fmt) registered for path 'dependencies/fmt'
Cloning into 'E:/Program_AI/G3d/tiny-cuda-nn/dependencies/cutlass'...
remote: Enumerating objects: 27394, done.
remote: Total 27394 (delta 0), reused 0 (delta 0), pack-reused 27394 (from 1)
Receiving objects: 100% (27394/27394), 43.44 MiB | 5.38 MiB/s, done.
Resolving deltas: 100% (20593/20593), done.
Cloning into 'E:/Program_AI/G3d/tiny-cuda-nn/dependencies/fmt'...
remote: Enumerating objects: 37307, done.
remote: Counting objects: 100% (1051/1051), done.
remote: Compressing objects: 100% (92/92), done.
remote: Total 37307 (delta 972), reused 980 (delta 945), pack-reused 36256 (from 1)
Receiving objects: 100% (37307/37307), 15.27 MiB | 212.00 KiB/s, done.
Resolving deltas: 100% (25234/25234), done.
Submodule path 'dependencies/cutlass': checked out '1eb6355182a5124639ce9d3ff165732a94ed9a70'
Submodule path 'dependencies/fmt': checked out 'b0c8263cb26ea178d3a5df1b984e1a61ef578950'

2.2.3 编译安装

进入tiny-cuda-nn\bindings\torch目录
运行python setup.py install

2.2.4 RuntimeError: Error compiling objects for extension

修改setup.py

cmdclass={“build_ext”: BuildExtension}
改为
cmdclass={‘build_ext’: BuildExtension.with_options(use_ninja=False)}

2.2.5 fatal error C1083: 无法打开包括文件: “algorithm”: No such file or directory

pip install algorithms

>pip install algorithms
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting algorithms
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/31/a4/a94b159b94d545ab7662b659f94b36f3b469b982fe873a6d956e4ffdc0d5/algorithms-0.1.4-py3-none-any.whl (209 kB)
Installing collected packages: algorithms
Successfully installed algorithms-0.1.4

2.2.6 还不能解决可尝试 2.1.1

运行vcvars64.bat脚本以设置环境变量
vcvars64.bat 文件的路径根据你自已安装Microsoft Visual Studio为准

>"C:\Program Files\Microsoft Visual Studio\2022\Commu
nity\VC\Auxiliary\Build\vcvars64.bat"
**********************************************************************
** Visual Studio 2022 Developer Command Prompt v17.9.3
** Copyright (c) 2022 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'

3、Pytorch3D

3.1 The detected CUDA version (12.1) mismatches the version that was used to compile PyTorch (11.8)

3.1.1 win10 系统环境下解决方案

raise RuntimeError(CUDA_MISMATCH_MESSAGE.format(cuda_str_version, torch.version.cuda))
      RuntimeError:
      The detected CUDA version (12.1) mismatches the version that was used to compile
      PyTorch (11.8). Please make sure to use the same CUDA versions.

问题分析
当前环境的CUDA版本过高

>nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Wed_Feb__8_05:53:42_Coordinated_Universal_Time_2023
Cuda compilation tools, release 12.1, V12.1.66
Build cuda_12.1.r12.1/compiler.32415258_0

>conda list
pytorch                   2.1.0           py3.9_cuda11.8_cudnn8_0    pytorch

解决方案
我的电脑->属性->高级系统设置->环境变量
修改CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8

修改path中的C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin和libnvvp两项上移置顶

重启命令行窗口

nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:41:10_Pacific_Daylight_Time_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

3.1.2 Ubuntu 系统环境下解决方案

查看nvcc版本

$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Feb__7_19:32:13_PST_2023
Cuda compilation tools, release 12.1, V12.1.66
Build cuda_12.1.r12.1/compiler.32415258_0

查看系统中有哪些nvcc版本
已经安装cuda版本有我们需要的11.8

$ ll /usr/local | grep cuda
lrwxrwxrwx  1 root root   20 Mar 18 01:52 cuda -> /usr/local/cuda-12.1/
lrwxrwxrwx  1 root root   25 Apr 10  2023 cuda-11 -> /etc/alternatives/cuda-11/
drwxr-xr-x 14 root root 4096 Mar 18 01:37 cuda-11.6/
drwxr-xr-x 14 root root 4096 Jun 18  2023 cuda-11.7/
drwxr-xr-x 18 root root 4096 Mar 18 01:38 cuda-11.8/
lrwxrwxrwx  1 root root   25 Apr 29  2023 cuda-12 -> /etc/alternatives/cuda-12/
drwxr-xr-x 16 root root 4096 Mar 14 14:56 cuda-12.1/
drwxr-xr-x 15 root root 4096 Jan 24 00:57 cuda-12.3/
drwxr-xr-x 15 root root 4096 Mar 13 07:19 cuda-12.4/

修改用户环境配置

vim ~/.bashrc

export LD_LIBRARY_PATH="/usr/local/cuda-11.8/lib64:$LD_LIBRARY_PATH"
export PATH="/usr/local/cuda-11.8/bin:$PATH"
export CUDA_HOME="/usr/local/cuda-11.8"

立即生效配置

source ~/.bashrc

$ source ~/.bashrc
$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

搞定啦！！！

3.2 安装Pytorch3D

系统环境：Ubuntu 2004

3.2.1 选择Python版本

根据几十次的安装测试，我选择用3.9.18

创建conda环境

conda create -n pt3d python==3.9.18

3.2.2 安装Pytorch库

需要pytorch 9.2及以上版本

选择CUDA是11.8，pytorch版本需要>=v2.0.0才支持CUDA11.8
确定使用pytorch 2.0.1版本

conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.8 -c pytorch -c nvidia

$ conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.8 -c pytorch -c nvidia
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: done


==> WARNING: A newer version of conda exists. <==
  current version: 23.1.0
  latest version: 24.3.0

Please update conda by running

    $ conda update -n base -c defaults conda

Or to minimize the number of packages updated during conda update use

     conda install conda=24.3.0



## Package Plan ##

  environment location: /home/dp/anaconda3/envs/rad3d

  added / updated specs:
    - pytorch-cuda=11.8
    - pytorch==2.0.1
    - torchaudio==2.0.2
    - torchvision==0.15.2


The following NEW packages will be INSTALLED:

  blas               pkgs/main/linux-64::blas-1.0-mkl 
  brotli-python      pkgs/main/linux-64::brotli-python-1.0.9-py39h6a678d5_7 
  bzip2              pkgs/main/linux-64::bzip2-1.0.8-h5eee18b_5 
  certifi            pkgs/main/linux-64::certifi-2024.2.2-py39h06a4308_0 
  charset-normalizer pkgs/main/noarch::charset-normalizer-2.0.4-pyhd3eb1b0_0 
  cuda-cudart        nvidia/linux-64::cuda-cudart-11.8.89-0 
  cuda-cupti         nvidia/linux-64::cuda-cupti-11.8.87-0 
  cuda-libraries     nvidia/linux-64::cuda-libraries-11.8.0-0 
  cuda-nvrtc         nvidia/linux-64::cuda-nvrtc-11.8.89-0 
  cuda-nvtx          nvidia/linux-64::cuda-nvtx-11.8.86-0 
  cuda-runtime       nvidia/linux-64::cuda-runtime-11.8.0-0 
  ffmpeg             pytorch/linux-64::ffmpeg-4.3-hf484d3e_0 
  filelock           pkgs/main/linux-64::filelock-3.13.1-py39h06a4308_0 
  freetype           pkgs/main/linux-64::freetype-2.12.1-h4a9f257_0 
  gmp                pkgs/main/linux-64::gmp-6.2.1-h295c915_3 
  gmpy2              pkgs/main/linux-64::gmpy2-2.1.2-py39heeb90bb_0 
  gnutls             pkgs/main/linux-64::gnutls-3.6.15-he1e5248_0 
  idna               pkgs/main/linux-64::idna-3.4-py39h06a4308_0 
  intel-openmp       pkgs/main/linux-64::intel-openmp-2023.1.0-hdb19cb5_46306 
  jinja2             pkgs/main/linux-64::jinja2-3.1.3-py39h06a4308_0 
  jpeg               pkgs/main/linux-64::jpeg-9e-h5eee18b_1 
  lame               pkgs/main/linux-64::lame-3.100-h7b6447c_0 
  lcms2              pkgs/main/linux-64::lcms2-2.12-h3be6417_0 
  lerc               pkgs/main/linux-64::lerc-3.0-h295c915_0 
  libcublas          nvidia/linux-64::libcublas-11.11.3.6-0 
  libcufft           nvidia/linux-64::libcufft-10.9.0.58-0 
  libcufile          nvidia/linux-64::libcufile-1.9.0.20-0 
  libcurand          nvidia/linux-64::libcurand-10.3.5.119-0 
  libcusolver        nvidia/linux-64::libcusolver-11.4.1.48-0 
  libcusparse        nvidia/linux-64::libcusparse-11.7.5.86-0 
  libdeflate         pkgs/main/linux-64::libdeflate-1.17-h5eee18b_1 
  libiconv           pkgs/main/linux-64::libiconv-1.16-h7f8727e_2 
  libidn2            pkgs/main/linux-64::libidn2-2.3.4-h5eee18b_0 
  libnpp             nvidia/linux-64::libnpp-11.8.0.86-0 
  libnvjpeg          nvidia/linux-64::libnvjpeg-11.9.0.86-0 
  libpng             pkgs/main/linux-64::libpng-1.6.39-h5eee18b_0 
  libtasn1           pkgs/main/linux-64::libtasn1-4.19.0-h5eee18b_0 
  libtiff            pkgs/main/linux-64::libtiff-4.5.1-h6a678d5_0 
  libunistring       pkgs/main/linux-64::libunistring-0.9.10-h27cfd23_0 
  libwebp-base       pkgs/main/linux-64::libwebp-base-1.3.2-h5eee18b_0 
  lz4-c              pkgs/main/linux-64::lz4-c-1.9.4-h6a678d5_0 
  markupsafe         pkgs/main/linux-64::mar