深度学习AI美颜系列---AI瘦身效果算法揭秘

https://blog.csdn.net/Trent1985/article/details/80667611

 

深度学习AI美颜系列---AI瘦身效果算法揭秘

2018年06月12日 17:31:26 Trent1985 阅读数 5681更多

分类专栏： 深度学习AI美颜系列 SF图像滤镜/美颜/美妆算法详解与实战

版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。

本文链接：https://blog.csdn.net/Trent1985/article/details/80667611

最近一段时间，抖音、微视、美图纷纷推出了视频实时瘦身的特效，可以说是火了一把！本文将给大家做个技术揭秘！

商汤基于深度学习研发了整套瘦身SDK，包括了瘦腿，瘦腰，瘦胳膊，瘦头型等等功能，并给出了酷炫的实时瘦身视频，惊艳到了众人！本文将以瘦腰和瘦腿为例，给大家详细讲解一下。

瘦身从算法角度来讲，包含两个模块：①人体轮廓特征点检测模块；②人体变形模块

[人体轮廓特征点检测模块]

人体轮廓特征点检测模块好比人脸特征点检测，需要检测到代表人体轮廓的一些点位信息，但是技术上比人脸特征点更加复杂，因为人体是非刚体，比人脸的变化更多，更复杂；

目前人体轮廓特征点检测主要方法就是人体姿态估计；从2015年开始，陆续出现了很多人体姿态估计的论文，这里举例如：

 

DeeperCut: A Deeper, Stronger, and Faster Multi-Person Pose Estimation Model

这篇论文中所提供的深度学习模型与效果图如下：

 

这里我们不详细介绍人体姿态估计的算法，主要以讲瘦身的流程为主；

上面的姿态点与我们想要的轮廓点还有差别，在掌握了上面的人体姿态估计算法之后，我们可以使用人体轮廓点的样本进行训练，这样就可以得到如下图所示的人体轮廓特征点：

图片来自网络，为了避免侵权，做了人脸遮挡；

[人体变形模块]

人体变形模块主要是依据人体特征点，将人体变形，也就是瘦身美型；

变形的算法有很多，比如MLS移动最小二乘法变形算法，IWD反距离加权变形算法，MLSR变形算法等等，相关连接如下：

MLSR变形算法实现：点击打开链接

IWD反距离加权变形算法实现：点击打开链接

MLS移动最小二乘变形算法实现：点击打开链接，代码链接点击打开链接

本人MLS代码如下：

1. static void setSrcPoints(const vector<PointD> &qsrc, vector<PointD> &newDotL, int* nPoint) {

2. *nPoint = qsrc.size();

3. newDotL.clear();

4. newDotL.reserve(*nPoint);

5. for (size_t i = 0; i < qsrc.size(); i++)

6. newDotL.push_back(qsrc[i]);

7. }

8.  

9. static void setDstPoints(const vector<PointD> &qdst,vector<PointD> &oldDotL, int* nPoint) {

10. *nPoint = qdst.size();

11. oldDotL.clear();

12. oldDotL.reserve(*nPoint);

13.  

14. for (size_t i = 0; i < qdst.size(); i++) oldDotL.push_back(qdst[i]);

15. }

16. static double bilinear_interp(double x, double y, double v11, double v12,

17. double v21, double v22) {

18. return (v11 * (1 - y) + v12 * y) * (1 - x) + (v21 * (1 - y) + v22 * y) * x;

19. }

20.  

21. static double calcArea(const vector<PointD> &V) {

22. PointD lt, rb;

23. lt.x = lt.y = 1e10;

24. rb.x = rb.y = -1e10;

25. for (vector<PointD >::const_iterator i = V.begin(); i != V.end();

26. i++) {

27. if (i->x < lt.x) lt.x = i->x;

28. if (i->x > rb.x) rb.x = i->x;

29. if (i->y < lt.y) lt.y = i->y;

30. if (i->y > rb.y) rb.y = i->y;

31. }

32. return (rb.x - lt.x) * (rb.y - lt.y);

33. }

34. static void calcDelta_rigid(int srcW, int srcH, int tarW, int tarH, double alpha, int gridSize, int nPoint, int preScale, double *rDx, double *rDy, vector<PointD> &oldDotL, vector<PointD> &newDotL)

35. {

36. int i, j, k;

37. PointD swq, qstar, newP, tmpP;

38. double sw;

39.  

40. double ratio;

41.  

42. if (preScale) {

43. ratio = sqrt(calcArea(newDotL) / calcArea(oldDotL));

44. for (i = 0; i < nPoint; i++) {

45. newDotL[i].x *= 1 / ratio;

46. newDotL[i].y *= 1 / ratio;

47. }

48. }

49. double *w = new double[nPoint];

50.  

51. if (nPoint < 2) {

52. //rDx.setTo(0);

53. //rDy.setTo(0);

54. return;

55. }

56. PointD swp, pstar, curV, curVJ, Pi, PiJ, Qi;

57. double miu_r;

58.  

59. for (i = 0;; i += gridSize) {

60. if (i >= tarW && i < tarW + gridSize - 1)

61. i = tarW - 1;

62. else if (i >= tarW)

63. break;

64. for (j = 0;; j += gridSize) {

65. if (j >= tarH && j < tarH + gridSize - 1)

66. j = tarH - 1;

67. else if (j >= tarH)

68. break;

69. sw = 0;

70. swp.x = swp.y = 0;

71. swq.x = swq.y = 0;

72. newP.x = newP.y = 0;

73. curV.x = i;

74. curV.y = j;

75. for (k = 0; k < nPoint; k++) {

76. if ((i == oldDotL[k].x) && j == oldDotL[k].y) break;

77. if (alpha == 1)

78. w[k] = 1 / ((i - oldDotL[k].x) * (i - oldDotL[k].x) +

79. (j - oldDotL[k].y) * (j - oldDotL[k].y));

80. else

81. w[k] = pow((i - oldDotL[k].x) * (i - oldDotL[k].x) +

82. (j - oldDotL[k].y) * (j - oldDotL[k].y),

83. -alpha);

84. sw = sw + w[k];

85. swp.x = swp.x + w[k] * oldDotL[k].x;

86. swp.y = swp.y + w[k] * oldDotL[k].y;

87. swq.x = swq.x + w[k] * newDotL[k].x;

88. swq.y = swq.y + w[k] * newDotL[k].y;

89. }

90. if (k == nPoint) {

91. pstar.x = (1 / sw) * swp.x;

92. pstar.y = (1 / sw) * swp.y;

93. qstar.x = 1 / sw * swq.x;

94. qstar.y = 1 / sw * swq.y;

95. // Calc miu_r

96. double s1 = 0, s2 = 0;

97. for (k = 0; k < nPoint; k++) {

98. if (i == oldDotL[k].x && j == oldDotL[k].y) continue;

99. Pi.x = oldDotL[k].x - pstar.x;

100. Pi.y = oldDotL[k].y - pstar.y;

101. PiJ.x = -Pi.y, PiJ.y = Pi.x;

102. Qi.x = newDotL[k].x - qstar.x;

103. Qi.y = newDotL[k].y - qstar.y;

104. s1 += w[k] * (Qi.x*Pi.x+Qi.y*Pi.y);

105. s2 += w[k] * (Qi.x*PiJ.x+Qi.y*PiJ.y);

106. }

107. miu_r = sqrt(s1 * s1 + s2 * s2);

108. curV.x -= pstar.x;

109. curV.y -= pstar.y;

110.  

111. curVJ.x = -curV.y, curVJ.y = curV.x;

112.  

113. for (k = 0; k < nPoint; k++) {

114. if (i == oldDotL[k].x && j == oldDotL[k].y) continue;

115. Pi.x = oldDotL[k].x - pstar.x;

116. Pi.y = oldDotL[k].y - pstar.y;

117. PiJ.x = -Pi.y, PiJ.y = Pi.x;

118. tmpP.x = (Pi.x*curV.x+Pi.y*curV.y)* newDotL[k].x -

119. (PiJ.x*curV.x+PiJ.y*curV.y)* newDotL[k].y;

120. tmpP.y = -(Pi.x*curVJ.x+Pi.y*curVJ.y) * newDotL[k].x +

121. (PiJ.x*curVJ.x+PiJ.y*curVJ.y) * newDotL[k].y;

122. tmpP.x *= w[k] / miu_r;

123. tmpP.y *= w[k] / miu_r;

124. newP.x += tmpP.x;

125. newP.y += tmpP.y;

126. }

127. newP.x += qstar.x;

128. newP.y += qstar.y;

129. } else {

130. newP = newDotL[k];

131. }

132.  

133. if (preScale) {

134. rDx[j * tarW + i] = newP.x * ratio - i;

135. rDy[j * tarW + i] = newP.y * ratio - j;

136. } else {

137. rDx[j * tarW + i] = newP.x - i;

138. rDy[j * tarW + i] = newP.y - j;

139. }

140. }

141. }

142. delete[] w;

143.  

144. if (preScale!=0) {

145. for (i = 0; i < nPoint; i++){

146. newDotL[i].x *= ratio;

147. newDotL[i].y *= ratio;

148. }

149. }

150. }

151. static void calcDelta_Similarity(int srcW, int srcH, int tarW, int tarH, double alpha, int gridSize, int nPoint, int preScale, double *rDx, double *rDy, vector<PointD> &oldDotL, vector<PointD> &newDotL)

152. {

153. int i, j, k;

154.  

155. PointD swq, qstar, newP, tmpP;

156. double sw;

157.  

158. double ratio;

159.  

160. if (preScale) {

161. ratio = sqrt(calcArea(newDotL) / calcArea(oldDotL));

162. for (i = 0; i < nPoint; i++) {

163. newDotL[i].x *= 1 / ratio;

164. newDotL[i].y *= 1 / ratio;

165. }

166. }

167. double *w = new double[nPoint];

168.  

169. if (nPoint < 2) {

170. return;

171. }

172.  

173. PointD swp, pstar, curV, curVJ, Pi, PiJ;

174. double miu_s;

175.  

176. for (i = 0;; i += gridSize) {

177. if (i >= tarW && i < tarW + gridSize - 1)

178. i = tarW - 1;

179. else if (i >= tarW)

180. break;

181. for (j = 0;; j += gridSize) {

182. if (j >= tarH && j < tarH + gridSize - 1)

183. j = tarH - 1;

184. else if (j >= tarH)

185. break;

186. sw = 0;

187. swp.x = swp.y = 0;

188. swq.x = swq.y = 0;

189. newP.x = newP.y = 0;

190. curV.x = i;

191. curV.y = j;

192. for (k = 0; k < nPoint; k++) {

193. if ((i == oldDotL[k].x) && j == oldDotL[k].y) break;

194. w[k] = 1 / ((i - oldDotL[k].x) * (i - oldDotL[k].x) +

195. (j - oldDotL[k].y) * (j - oldDotL[k].y));

196. sw = sw + w[k];

197. swp.x = swp.x + w[k] * oldDotL[k].x;

198. swp.y = swp.y + w[k] * oldDotL[k].y;

199. swq.x = swq.x + w[k] * newDotL[k].x;

200. swq.y = swq.y + w[k] * newDotL[k].y;

201. }

202. if (k == nPoint) {

203. pstar.x = (1 / sw) * swp.x;

204. pstar.y = (1 / sw) * swp.y;

205. qstar.x = 1 / sw * swq.x;

206. qstar.y = 1 / sw * swq.y;

207. // Calc miu_s

208. miu_s = 0;

209. for (k = 0; k < nPoint; k++) {

210. if (i == oldDotL[k].x && j == oldDotL[k].y) continue;

211.  

212. Pi.x = oldDotL[k].x - pstar.x;

213. Pi.y = oldDotL[k].y - pstar.y;

214. miu_s += w[k] * (Pi.x*Pi.x+Pi.y*Pi.y);

215. }

216.  

217. curV.x -= pstar.x;

218. curV.y -= pstar.y;

219. curVJ.x = -curV.y, curVJ.y = curV.x;

220.  

221. for (k = 0; k < nPoint; k++) {

222. if (i == oldDotL[k].x && j == oldDotL[k].y) continue;

223.  

224. Pi.x = oldDotL[k].x - pstar.x;

225. Pi.y = oldDotL[k].y - pstar.y;

226. PiJ.x = -Pi.y, PiJ.y = Pi.x;

227.  

228. tmpP.x = (Pi.x*curV.x+Pi.y*curV.y) * newDotL[k].x -

229. (PiJ.x*curV.x+PiJ.y*curV.y) * newDotL[k].y;

230. tmpP.y = -(Pi.x*curVJ.x+Pi.y*curVJ.y) * newDotL[k].x +

231. (PiJ.x*curVJ.x+PiJ.y*curVJ.y) * newDotL[k].y;

232. tmpP.x *= w[k] / miu_s;

233. tmpP.y *= w[k] / miu_s;

234. newP.x += tmpP.x;

235. newP.y += tmpP.y;

236. }

237. newP.x += qstar.x;

238. newP.y += qstar.y;

239. } else {

240. newP = newDotL[k];

241. }

242.  

243. rDx[j * tarW + i] = newP.x - i;

244. rDy[j * tarW + i] = newP.y - j;

245. }

246. }

247.  

248. delete[] w;

249. if (preScale!=0) {

250. for (i = 0; i < nPoint; i++){

251. newDotL[i].x *= ratio;

252. newDotL[i].y *= ratio;

253. }

254. }

255. }

256. static int GetNewImg(unsigned char* oriImg, int width, int height, int stride, unsigned char* tarImg, int tarW, int tarH, int tarStride, int gridSize, double* rDx, double* rDy, double transRatio)

257. {

258. int i, j;

259. double di, dj;

260. double nx, ny;

261. int nxi, nyi, nxi1, nyi1;

262. double deltaX, deltaY;

263. double w, h;

264. int ni, nj;

265. int pos, posa, posb, posc, posd;

266. for (i = 0; i < tarH; i += gridSize)

267. for (j = 0; j < tarW; j += gridSize) {

268. ni = i + gridSize, nj = j + gridSize;

269. w = h = gridSize;

270. if (ni >= tarH) ni = tarH - 1, h = ni - i + 1;

271. if (nj >= tarW) nj = tarW - 1, w = nj - j + 1;

272. for (di = 0; di < h; di++)

273. for (dj = 0; dj < w; dj++) {

274. deltaX =

275. bilinear_interp(di / h, dj / w, rDx[i * tarW + j], rDx[i * tarW + nj],

276. rDx[ni * tarW + j], rDx[ni * tarW + nj]);

277. deltaY =

278. bilinear_interp(di / h, dj / w, rDy[i * tarW + j], rDy[i * tarW + nj],

279. rDy[ni * tarW + j], rDy[ni * tarW + nj]);

280. nx = j + dj + deltaX * transRatio;

281. ny = i + di + deltaY * transRatio;

282. if (nx > width - 1) nx = width - 1;

283. if (ny > height - 1) ny = height - 1;

284. if (nx < 0) nx = 0;

285. if (ny < 0) ny = 0;

286. nxi = int(nx);

287. nyi = int(ny);

288. nxi1 = ceil(nx);

289. nyi1 = ceil(ny);

290. pos = (int)(i + di) * tarStride + ((int)(j + dj) << 2);

291. posa = nyi * stride + (nxi << 2);

292. posb = nyi * stride + (nxi1 << 2);

293. posc = nyi1 * stride + (nxi << 2);

294. posd = nyi1 * stride + (nxi1 << 2);

295. tarImg[pos] = (unsigned char)bilinear_interp(ny - nyi, nx - nxi, oriImg[posa], oriImg[posb], oriImg[posc], oriImg[posd]);

296. tarImg[pos + 1] = (unsigned char)bilinear_interp(ny - nyi, nx - nxi, oriImg[posa + 1],oriImg[posb + 1], oriImg[posc + 1], oriImg[posd + 1]);

297. tarImg[pos + 2] = (unsigned char)bilinear_interp(ny - nyi, nx - nxi, oriImg[posa + 2],oriImg[posb + 2], oriImg[posc + 2], oriImg[posd + 2]);

298. tarImg[pos + 3] = (unsigned char)bilinear_interp(ny - nyi, nx - nxi, oriImg[posa + 3],oriImg[posb + 3], oriImg[posc + 3], oriImg[posd + 3]);

299. }

300. }

301. return 0;

302. };

303.  

304. static void MLSImageWrapping(unsigned char* oriImg,int width, int height, int stride,const vector<PointD > &qsrc, const vector<PointD > &qdst, unsigned char* tarImg, int outW, int outH, int outStride, double transRatio, int preScale, int gridSize, int method)

305. {

306. int srcW = width;

307. int srcH = height;

308. int tarW = outW;

309. int tarH = outH;

310. double alpha = 1;

311. int nPoint;

312. int len = tarH * tarW;

313. vector<PointD> oldDotL, newDotL;

314. double *rDx = NULL,*rDy = NULL;

315. setSrcPoints(qsrc,newDotL,&nPoint);

316. setDstPoints(qdst,oldDotL,&nPoint);

317. rDx = (double*)malloc(sizeof(double) * len);

318. rDy = (double*)malloc(sizeof(double) * len);

319. memset(rDx, 0, sizeof(double) * len);

320. memset(rDy, 0, sizeof(double) * len);

321. if(method!=0)

322. calcDelta_Similarity(srcW, srcH, tarW, tarH, alpha, gridSize, nPoint, preScale, rDx, rDy, oldDotL, newDotL);

323. else

324. calcDelta_rigid(srcW, srcH, tarW, tarH, alpha, gridSize, nPoint, preScale, rDx, rDy, oldDotL, newDotL);

325. GetNewImg(oriImg, srcW, srcH, stride, tarImg, tarW, tarH, outStride, gridSize, rDx, rDy, transRatio);

326. if(rDx != NULL)

327. free(rDx);

328. if(rDy != NULL)

329. free(rDy);

330. };

331. int f_TMLSImagewarpping(unsigned char* srcData, int width ,int height, int stride, unsigned char* dstData, int outW, int outH, int outStride, int srcPoint[], int dragPoint[], int pointNum, double intensity, int preScale, int gridSize, int method)

332. {

333. int res = 0;

334. vector<PointD> qDst;

335. vector<PointD> qSrc;

336. PointD point = {0};

337. int len = 0;

338. for(int i = 0; i < pointNum; i++)

339. {

340. len = (i << 1);

341. point.x = srcPoint[len];

342. point.y = srcPoint[len + 1];

343. qSrc.push_back(point);

344. point.x = dragPoint[len];

345. point.y = dragPoint[len + 1];

346. qDst.push_back(point);

347. }

348. MLSImageWrapping(srcData, width, height, stride, qSrc, qDst, dstData, outW, outH, outStride, intensity, preScale,gridSize, method);

349. return res;

350. };

 

本人这里使用MLS变形来实现瘦身效果：

1，根据人体姿态估计检测得到原图人体的特征点；

2，计算得到瘦腿之后的人体特征点；

3，计算得到瘦腰之后的人体特征点；

1-3如下图所示：

4，使用MLS，将原图特征点分别变换到瘦腿特征点和瘦腰特征点，进而得到两个相应的效果图如下(这里给出四个效果)：

至此，瘦身效果的算法流程讲完，至于实时处理，那是代码优化和算法优化的事情，只要速度够快，一切都不是问题！

上面只是以瘦腿和瘦腰为例，实际上还可以瘦胳膊，丰胸等等，来真正的达到美颜瘦身！

本人的DEMO界面如下：

本人QQ1358009172

最后给出本人的DEMO：点击打开链接