Diffusers代码学习：Shap-E 3D网格输出

Shap-E是一个灵活的模型，它还可以生成纹理网格输出，以便为下游应用程序渲染。在本例中，将把输出转换为glb文件，因为数据集库支持glb文件的网格可视化，这些文件可以通过数据集查看器进行渲染。

通过将output_type参数指定为“mesh”，可以为[ShapEPipeline]和[ShapEImg2ImgPipeline]生成网格输出。

import os

os.environ["HF_ENDPOINT"] = "https://hf-mirror.com"

import torch
from diffusers import ShapEPipeline

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

pipe = ShapEPipeline.from_pretrained("openai/shap-e", torch_dtype=torch.float16, variant="fp16")
pipe = pipe.to(device)

guidance_scale = 15.0
prompt = "A birthday cupcake"

images = pipe(prompt, guidance_scale=guidance_scale, num_inference_steps=64, frame_size=256, output_type="mesh").images

from diffusers.utils import export_to_ply

ply_path = export_to_ply(images[0], "3d_cake.ply")
print(f"Saved to folder: {ply_path}")

import trimesh

mesh = trimesh.load("3d_cake.ply")

mesh_export = mesh.export("3d_cake.glb", file_type="glb")

实际生成了3d_cake.ply和3d_cake.glb两个文件。用3D查看器查看效果如下：

3d_cake.ply

3d_cake.glb