135 OpenCV DNN 实时快速的图像风格迁移
代码
import cv2 as cv
import numpy as np
base_dir = "../models/fast_style/"
styles = ["composition_vii.t7", "starry_night.t7", "la_muse.t7", "the_wave.t7",
"mosaic.t7", "the_scream.t7", "feathers.t7", "candy.t7", "udnie.t7"]
index = 2
net = cv.dnn.readNetFromTorch(base_dir + styles[index])
net.setPreferableBackend(cv.dnn.DNN_BACKEND_OPENCV);
cap = cv.VideoCapture(0)
while cv.waitKey(1) < 0:
hasFrame, frame = cap.read()
if not hasFrame:
cv.waitKey()
break
cv.imshow("frame", frame)
inWidth = 256
inHeight = 256
h, w = frame.shape[:2]
inp = cv.dnn.blobFromImage(frame, 1.0, (inWidth, inHeight),
(103.939, 116.779, 123.68), swapRB=False, crop=False)
# 执行风格迁移
net.setInput(inp)
out = net.forward()
print(out.shape)
t, _ = net.getPerfProfile()
freq = cv.getTickFrequency() / 1000
label = "FPS : %.2f" % (1000 / (t / freq))
# 解析输出
out = out.reshape(3, out.shape[2], out.shape[3])
print("ddddddddd", out.shape)
out[0] += 103.939
out[1] += 116.779
out[2] += 123.68
out /= 255.0
out = out.transpose(1, 2, 0)
print("new shape", out.shape)
out = np.clip(out, 0.0, 1.0)
# rescale与中值模糊,消除极值点噪声
out = cv.normalize(out, None, 0, 255, cv.NORM_MINMAX)
out = cv.medianBlur(out, 5)
# resize and show
result = np.uint8(cv.resize(out, (w, h)))
cv.putText(result, label, (5, 25), cv.FONT_HERSHEY_SIMPLEX, 0.75, (0, 0, 255), 2)
cv.imshow('Fast Style Demo', result)
# cv.imwrite("D:/result_%d.png"%index, result)
实验结果
解释
OpenCV DNN模块现在还支持图像风格迁移网络模型的加载与使用,支持的模型是基于李飞飞等人在论文《Perceptual Losses for Real-Time Style Transfer and Super-Resolution》中提到的快速图像风格迁移网络,基于感知损失来提取特征,生成图像特征与高分辨率图像。整个网络模型是基于DCGAN + 5个残差层构成,是一个典型的全卷积网络,关于DCGAN可以看这里有详细介绍与代码实现:
使用DCGAN实现图像生成
这个网络可以支持任意尺寸的图像输入,作者提供了很多种预训练的风格迁移模型提供使用,我下载了下面的预训练模型。:
composition_vii.t7
starry_night.t7
la_muse.t7
the_wave.t7
mosaic.t7
the_scream.t7
feathers.t7
candy.t7
udnie.t7
这些模型都是torch框架支持的二进制权重文件,加载模型之后,就可以调用forward得到结果,通过对输出结果反向加上均值,rescale到0~255的RGB色彩空间,即可显示。
所有内容均来源于贾志刚老师的知识星球——OpenCV研习社,本文为个人整理学习,已获得贾老师授权,有兴趣、有能力的可以加入贾老师的知识星球进行深入学习。