fid-helper-pytorch
Github 仓库地址:https://github.com/One-sixth/fid-helper-pytorch
Gitee 仓库地址:https://gitee.com/ONE_SIX_MIX/fid-helper-pytorch
这是一个FID助手工具。
提供了一个简单方便的接口来计算FID。
仓库使用pytorch jit script来打包模型,包含了一个默认模型。
:default_1 的模型和权重来自 styleganv3 fid tool。
文件夹 make_default_model 包含了默认模型的编译脚本。
我本想实现一个简单的fid工具,但顺便发现了clean-fid和它很棒的效果。
所以顺便追加了clean-fid的计算方式。
他们均有相同的权重,但可能是因为jit包装,计算顺序,随机数相关的的问题,它们的输出均有细微的差异。
支持clean-fid的计算方式同时,我把尽可能多的采样方法涵盖进去,尽管这并没有什么用。(笑)
对了,如果你希望使用最新的pytorch抗锯齿的方法 torch_bilinear_aa 和 torch_bicubic_aa。
你需要更新到 pytorch-1.11 以及更新版本。
一些杂事
原本包含了 3个默认模型。
:default_1 的模型和权重来自 styleganv3 fid tool。
:default_2 的模型和权重来自 pytorch_fid。
:default_3 的模型和权重来自 clean-fid。
但我发现它们是完全一样的,最后我只保留了第一个模型。
第二个和第三个模型你可以在 make_default_model 里面找到它们的构建脚本。
安装
从pip中安装
pip install fid-helper-pytorch
从源码中安装
git clone https://github.com/One-sixth/fid-helper-pytorch
cd fid-helper-pytorch
python setup.py install
构建轮子
git clone https://github.com/One-sixth/fid-helper-pytorch
cd fid-helper-pytorch
python setup.py bdist_wheel
# The wheel that you can found in dist folder.
如何使用
直接比对两个文件夹,默认FID计算方式与pytorch-fid一致。
其中,dir1 和 dir2 都可以是 图片文件夹,图片zip压缩包或统计文件。
# 比较两个文件夹
fid-helper compare dir1 dir2
# 比较两个zip压缩包
fid-helper compare dir1.zip dir2.zip
# 比较两个统计文件
fid-helper compare dir1_stat.pkl dir2_stat.pkl
# 你也可以自由交换它们
# 比较文件夹和zip压缩包
fid-helper compare dir1 dir2.zip
# 毕竟统计文件和文件夹
fid-helper compare dir1_stat.pkl dir2.zip
给目标图片文件夹,图片压缩包生成统计文件,便于后续快速评估
# 导出文件夹1的统计文件
fid-helper extract dir1 dir1_stat.pkl
# 导出压缩包的统计文件
fid-helper extract dir1.zip dir1_stat.pkl
# 导出统计文件的统计文件 :)
fid-helper extract dir1_stat.pkl dir1_stat_2.pkl
与其他库进行比较
如果你希望获得与 clean-fid legacy_tensorflow 模式一致的分数
如果你希望获得与 NVlabs/stylegan3’s FID tool 一致的分数
fid-helper-pytorch compare folder1 folder2
or
fid-helper-pytorch --model=:default_1 --resample=nv_bilinear_float --field tf compare folder1 folder2
如果你希望获得与 pytorch-fid 一致的分数。
如果你希望获得与 clean-fid legacy_pytorch 模式一致的分数。
or
fid-helper-pytorch --model=:default_1 --resample=torch_bilinear_float --field pt compare folder1 folder2
如果你希望获得与 clean-fid clean 模式一致的分数
clean-fid
fid-helper-pytorch --model=:default_1 --resample=pil_bicubic_float --field tf compare folder1 folder2
or
fid-helper-pytorch --model=:default_1 --resample=torch_bicubic_float --field tf compare folder1 folder2
FID 测试
全部测试数据均生成自 stylegan3。
_gen_sample_from_stylegan3/_gen_sample.py 为测试数据生成源码。
_gen_sample_from_stylegan3/_batch_compare_fid.py 为完整的FID测试。
这是我生成的测试数据。你可以从该测试数据中复现我的结果。
每个压缩包均有 10000 张png图像。
https://anonfiles.com/dfDej8Oaxe/sample_1_zip
https://anonfiles.com/V3N5j6O1x3/sample_2_zip
分数比较
更详细的数据你可以在这里找到 compare.txt
--
# pytorch-fid
3.772843757504859
# clean-fid mode=legacy_pytorch
3.772843691229639
# fid-helper-pytorch model=:default_1 input_range=pytorch resample=torch_bilinear_float
3.7728430784583953
---
# clean-fid mode=clean
3.7965344580221654
# fid-helper-pytorch model=:default_1 input_range=tf resample=pil_bicubic_float
3.79653415593325
# fid-helper-pytorch model=:default_1 input_range=tf resample=torch_bicubic_aa_float
3.7965341239671466
---
# clean-fid mode=legacy_tensorflow
3.773897315245904
# stylegan3 fid tool
none. please see clean-fid legacy_tensorflow
# fid-helper-pytorch model=:default_1 input_range=tf resample=nv_bilinear_float
3.7738974250271213
---
应用内FID计算
Example_1 Compute fid by yourself.
import numpy as np
from fid_helper_pytorch import FidHelper, INPUT_RANGE_TF
fidhelper = FidHelper(':default_1', INPUT_RANGE_TF, resample_mode='nv_bilinear_float', device='cuda:0')
# Compute real image stat by yourself.
fidhelper.begin_ref_stat()
for _ in range(16):
batch_image = np.random.randint(0, 255, size=[6, 3, 256, 256], dtype=np.uint8)
fidhelper.update_ref_stat(batch_image, data_range=(0, 255))
fidhelper.finish_ref_stat()
# Compute fake image stat by yourself.
fidhelper.begin_eval_stat()
for _ in range(16):
gen_images = np.random.uniform(-1., 1., size=[18, 3, 256, 256])
fidhelper.update_eval_stat(gen_images, data_range=[-1, 1])
fidhelper.finish_eval_stat()
# Compute fid.
fid_score = fidhelper.compute_fid_score()
print('FID:', fid_score)
Example_2 Compute stat and fid from folder and cycle.
import torch
from fid_helper_pytorch import FidHelper
fidhelper = FidHelper(':default_1')
batch_size = 4
num_workers = 1
real_img_dir = 'real_img'
# Compute real image stat from image folder.
fidhelper.compute_ref_stat_from_dir(real_img_dir, batch_size=batch_size, num_workers=num_workers)
# Compute fake image stat by yourself.
fidhelper.begin_eval_stat()
for _ in range(16):
gen_images = torch.rand(size=[18, 3, 256, 256])
fidhelper.update_eval_stat(gen_images, data_range=[0, 1])
fidhelper.finish_eval_stat()
# Compute fid.
fid_score = fidhelper.compute_fid_score()
print('FID:', fid_score)
Example_3 Compute fid from zip and get last fid.
from fid_helper_pytorch import FidHelper
fidhelper = FidHelper(':default_1')
batch_size = 4
num_workers = 1
real_img_arc = 'real_img.zip'
fake_img_arc = 'fake_img.zip'
# Compute real image stat from image folder.
fid_score = fidhelper.compute_fid_score_from_dir(real_img_arc, fake_img_arc, batch_size=batch_size, num_workers=num_workers, verbose=True)
print('FID:', fid_score)
# Compute fid.
fid_score = fidhelper.fid_score
print('last FID:', fid_score)
Example_4 Compute stat and fid from folder.
from fid_helper_pytorch import FidHelper
fidhelper = FidHelper(resample_mode='pil_bicubic_float')
fake_img_dir = 'fake_img'
real_img_dir = 'real_img'
# Compute real image stat from image folder.
fidhelper.compute_ref_stat_from_dir(fake_img_dir, batch_size=16, num_workers=2, verbose=True)
# Compute fake image stat from image folder.
fidhelper.compute_eval_stat_from_dir(real_img_dir, batch_size=24, num_workers=3, verbose=True)
# Compute fid.
fid_score = fidhelper.compute_fid_score()
print('FID:', fid_score)
Example_5 Compute stat file and save to disk.
from fid_helper_pytorch import FidHelper
fidhelper = FidHelper(':default_1')
real_img_dir = 'real_img'
fake_img_dir = 'fake_img'
# Compute real image stat from image folder.
fidhelper.compute_ref_stat_from_dir(real_img_dir)
# Save stat file to disk.
fidhelper.save_ref_stat_dict('out_real_stat.pkl')
# Compute fake image stat from image folder.
fidhelper.compute_eval_stat_from_dir(fake_img_dir)
# Save stat file to disk.
fidhelper.save_eval_stat_dict('out_fake_stat.pkl')
Example_6 Load stat file and compute fid.
from fid_helper_pytorch import FidHelper
fidhelper = FidHelper(':default_1')
# Get fake image stat from stat file.
fidhelper.load_ref_stat_dict('fake_stat.pkl')
# Get real image stat from stat file.
fidhelper.load_eval_stat_dict('real_stat.pkl')
fid_score = fidhelper.compute_fid_score()
print('FID:', fid_score)
Example_7 Change paramters anywhere.
from fid_helper_pytorch import FidHelper
fidhelper = FidHelper(':default_1')
# Change model anywhere.
fidhelper.load_model('xxx.pt', model_input_range=None)
# Change device anywhere.
fidhelper.change_device('cuda:1')
# Change model input range anywhere.
fidhelper.change_model_input_range([0, 1])
# Change resample_mode anywhere.
fidhelper.change_resample_mode('torch_bilinear_float')
# Change resize_hw anywhere.
fidhelper.change_resize_hw([320, 320])
参考引用
非常感谢大佬们的开源仓库,没有他们无私奉献,这个库将不可能存在。
如果你喜欢本仓库,请为下面大佬的仓库star。
https://github.com/mseitzer/pytorch-fid 最流行的pytorch-fid仓库
https://github.com/GaParmar/clean-fid 最新的clean-fid仓库
https://github.com/NVlabs/stylegan3 stylegan3的官方仓库