google colab 免费GPU 跑 first order motion model 生成“蚂蚁呀嘿 ” Face2Face

First order motion model github, notebook 参考链接, 论文论文地址

代码文本重新连接修改

---

Colab Demo for paper "First Order Motion Model for Image Animation"

1. 下载代码

1.1 Clone 下载

!git clone https://github.com/AliaksandrSiarohin/first-order-model

1.2 切换到项目目录下

cd first-order-model

2. 挂载谷歌云盘和添加预训练数据

2.1 挂载

from google.colab import drive
drive.mount('/content/gdrive')

2.2 添加谷歌云盘共享的预训练数据

把这个共享数据文件夹 https://drive.google.com/drive/folders/1kZ1gCnpfU0BnpdU47pLM_TQ6RypDDqgw?usp=sharing 添加到自己的谷歌云盘中，这个过程类似win里面添加文件的快捷文件

挂载和添加数据成功后，你就可以在colab上面看到项目路劲和共享文件路劲

3. 读取驱动视频和源图像（Load driving video and source image）

import imageio
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.animation as animation
from skimage.transform import resize
from IPython.display import HTML
import warnings
warnings.filterwarnings("ignore")
# 下面的路劲是保存在谷歌云盘的路劲
source_image = imageio.imread('/content/gdrive/My Drive/first-order-motion-model/02.png')
reader = imageio.get_reader('/content/gdrive/My Drive/first-order-motion-model/04.mp4')


#Resize image and video to 256x256

source_image = resize(source_image, (256, 256))[..., :3]

fps = reader.get_meta_data()['fps']
driving_video = []
try:
    for im in reader:
        driving_video.append(im)
except RuntimeError:
    pass
reader.close()

driving_video = [resize(frame, (256, 256))[..., :3] for frame in driving_video]

def display(source, driving, generated=None):
    fig = plt.figure(figsize=(8 + 4 * (generated is not None), 6))

    ims = []
    for i in range(len(driving)):
        cols = [source]
        cols.append(driving[i])
        if generated is not None:
            cols.append(generated[i])
        im = plt.imshow(np.concatenate(cols, axis=1), animated=True)
        plt.axis('off')
        ims.append([im])

    ani = animation.ArtistAnimation(fig, ims, interval=50, repeat_delay=1000)
    plt.close()
    return ani
    
# 展示效果
HTML(display(source_image, driving_video).to_html5_video())

4. 创建模型并加载预训练权值（Create a model and load checkpoints）

from demo import load_checkpoints
generator, kp_detector = load_checkpoints(config_path='config/vox-256.yaml', 
                            checkpoint_path='/content/gdrive/My Drive/first-order-motion-model/vox-cpk.pth.tar')

5. 制作图像动画（Perform image animation）

from demo import make_animation
from skimage import img_as_ubyte

predictions = make_animation(source_image, driving_video, generator, kp_detector, relative=True)

#save resulting video
imageio.mimsave('../generated.mp4', [img_as_ubyte(frame) for frame in predictions], fps=fps)
#video can be downloaded from /content folder

HTML(display(source_image, driving_video, predictions).to_html5_video())

 

6. 加载数据的图片或视频

6.1 裁剪视频

First we need to crop a face from both source image and video, while simple graphic editor like paint can be used for cropping from image. Cropping from video is more complicated. You can use ffpmeg for this. Another posibility is to use some screen recording tool, or if you need to crop many images at ones use face detector(https://github.com/1adrianb/face-alignment) , see https://github.com/AliaksandrSiarohin/video-preprocessing for preprcessing of VoxCeleb.

!ffmpeg -i /content/gdrive/MyDrive/first-order-motion-model/07.mkv -ss 00:08:57.50 -t 00:00:08 -filter:v "crop=600:600:760:50" -async 1 hinton.mp4

6.2 上传自己的数据并制作

改为自己的路劲前，需要把自己的图片上传到谷歌云盘，然后再获取自己图片的在线链接

# source_image = imageio.imread('/content/gdrive/My Drive/first-order-motion-model/09.png')
source_image = imageio.imread('/content/gdrive/MyDrive/image/cby_demo.jpg') # 改为自己的数据路劲
driving_video = imageio.mimread('hinton.mp4', memtest=False)


#Resize image and video to 256x256

source_image = resize(source_image, (256, 256))[..., :3]
driving_video = [resize(frame, (256, 256))[..., :3] for frame in driving_video]

predictions = make_animation(source_image, driving_video, generator, kp_detector, relative=True,
                             adapt_movement_scale=True)

HTML(display(source_image, driving_video, predictions).to_html5_video())

效果如下（隐私原因对自己打码）：

我自己的完整的colab notebook 链接