PSHuman 部署笔记

目录

github地址：

依赖项：

xformers安装：

解决方法，安装xformers

smpl_data下载：

推理步骤：

SMPLDataset

---

香港科技大学提出了一种叫PSHuman的新框架。这个方法利用了一个多视角扩散模型的“先验知识”来更好地重建人物。如果直接用多视角扩散技术去处理单视角图像，结果往往会出现很大的几何扭曲，尤其是在生成的面部细节上。为了解决这一问题，设计了一种跨尺度扩散方法，这种方法能同时兼顾整体身体形状和局部面部特征的细节，生成的结果既细致又真实，没有几何上的畸变。

github地址：

GitHub - pengHTYX/PSHuman

模型下载地址：

https://huggingface.co/pengHTYX/PSHuman_Unclip_768_6views

演示地址：

https://huggingface.co/spaces/fffiloni/PSHuman

依赖项：

pip install rembg

pip install yacs

pip install pymeshlab

pip install kaolin==0.17.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-2.5.1_cu121.html
pip install icecream
pip install torch_scatter

xformers安装：

不更新会报错：

NotImplementedError: No operator found for `memory_efficient_attention_forward` with inputs:

解决方法，安装xformers

pip3 install -U xformers --index-url https://download.pytorch.org/whl/cu121

安装完了后，torch会自动更新为2.5.1

nvdiffrast安装： 

https://github.com/NVlabs/nvdiffrast

下载下来，cd nvdiffrast

pip install .

'https://github.com/danielgatis/rembg/releases/download/v0.0.0/u2net.onnx' to file '/root/.u2net/u2net.onnx'.

omegaconf.errors.ConfigKeyError: Key ' ' not in 'TestConfig'

临时解决方法：

/mnt/pfs/users/lbg/envs/py310wj/lib/python3.10/site-packages/omegaconf/dictconfig.py

    def _validate_get(self, key: Any, value: Any = None) -> None:
        is_typed = self._is_typed()
        if key==" ":
            return

调用：

CUDA_VISIBLE_DEVICES=0 python inference.py --config configs/inference-768-6view.yaml \
    pretrained_model_name_or_path='/xxx/models/models--pengHTYX--PSHuman_Unclip_768_6views' \
    validation_dataset.crop_size=740 \
    with_smpl=false \
    validation_dataset.root_dir=examples \
    seed=600 \
    num_views=7 \
    save_mode='rgb'

smpl_data下载：

huggingface-cli download --resume-download lilpotat/pytorch3d --local-dir lilpotat--pytorch3d

拷贝到目录：

smpl_related/smpl_data

推理步骤：

1. Given a human image, we use Clipdrop or rembg to remove the background. For the latter, we provide a simple scrip.

python utils/remove_bg.py --path $DATA_PATH$

Then, put the RGBA images in the $DATA_PATH$.

1. By running inference.py, the textured mesh and rendered video will be saved in out.

CUDA_VISIBLE_DEVICES=$GPU python inference.py --config configs/inference-768-6view.yaml \
    pretrained_model_name_or_path='pengHTYX/PSHuman_Unclip_768_6views' \
    validation_dataset.crop_size=740 \
    with_smpl=false \
    validation_dataset.root_dir=$DATA_PATH$ \
    seed=600 \
    num_views=7 \
    save_mode='rgb' 

SMPLDataset

from econdataset import SMPLDataset

import os
import sys
import torch
os.chdir(os.path.dirname(os.path.abspath(__file__)))
 
current_dir = os.path.dirname(os.path.abspath(__file__))
 
paths = [os.path.abspath(__file__).split('scripts')[0]]
print('current_dir',current_dir)
paths.append(os.path.abspath(os.path.join(current_dir, './')))
paths.append(os.path.abspath(os.path.join(current_dir, '../')))
 
for path in paths:
    sys.path.insert(0, path)
    os.environ['PYTHONPATH'] = (os.environ.get('PYTHONPATH', '') + ':' + path).strip(':')
    
class Params:
        pass

validation_dataset=Params()    
validation_dataset.root_dir='examples'

# dataset_param = {'image_dir': validation_dataset.root_dir, 'seg_dir': None, 'colab': False, 'has_det': True, 'hps_type': 'pixie'}
dataset_param = {'image_dir': validation_dataset.root_dir, 'seg_dir': None, 'colab': False, 'has_det': True, 'hps_type': 'pixie'}
econdata = SMPLDataset(dataset_param, device='cuda')