seam carving

Seam Carving

andrewdcampbell/seam-carving

17. 如何通过缝隙雕刻图像：Seam Carving

图像智能缩放（Image Retargeting）

智能图像缩放考虑了图像内容的重要性差异，根据“重要的图像内容应该尽可能保留，而相对不重要的内容可以去掉或重复”原则，将原图像缩放为目标尺寸。

智能图像缩放问题转化为：

• 衡量图像内容重要性的方法
  衡量图像内容的重要性的主要方法是计算图中像素的能量值，比较经典的能量计算依据有梯度值（seam carving），图像的显著性（显著性大的地方比较容易受人眼的关注）和网格形变量（Robust Image Retargeting，推荐学习）等。
• 尽可能减弱缩放痕迹的缩放策略

当我们看到一张图像时，人眼最关注的地方就最重要。在上图中，大家的研究最终会停留在那座建筑上，那么那座建筑的重要性就很大。最右列的上下效果分别为seam carving和比例放大的效果图，可以明显看到区别，智能缩放（上图）中建筑没有发生明显形变，图片连贯性和完整性都比较自然；而传统技术（下图）将原图直接拉大，图片内容发生严重变形。而图示中间那一列即为seam carving算法的中间能量示意图，seam carving使用图像的梯度特征进行能量计算。每次针对能量最小的缝（seam）进行移除或复制操作，缩放的过程中需要考虑图像相邻像素的连续性和单调性，以减弱认为处理的痕迹。连续性指的是上下（左右）相邻像素不能在横坐标（纵坐标）方向相差太远，上图中红色的线即为算法检测出的一条缝（seam），该缝是连续的。单调性是为了保证图像始终为矩形形状，每次移除或者复制缝（seam），每行每列只能操作一个像素点，调调性体现在缝的方向是垂直或水平的，不会出现回路形式的缝（seam）。

图像拼接裁剪/缝隙切割（seam carving）

seam carving缝隙切割，就是在图像中找到一条缝，将这条缝插入或删除以达到缩放的目的。是一种用于图像缩放的技术，这种技术可以感知图像的内容，在缩小时保留图像中的重要内容；在放大时，放大图像的重要内容。该方法在保留图像中重要物体方面相对于基于裁剪的缩放方法和等比例缩放技术有明显的优势。
seam carving的基本思想是：图像中不同的位置“能量”不同，有重要内容的地方能量大，反之，可有可无的位置能量小。这个能量函数可以是梯度，梯度大的地方亮度变化剧烈、纹理丰富，一般是重要内容；梯度小的位置亮度变化小，比如蓝天，删除一条缝隙对图像影响较小。

seam carving算法是一种图像缩放算法，它能够将图像缩放也不影响图像的主要内容。

• 计算图像的能量
• 通过能量图计算代价图以及路径图
• 通过代价图以及路径图寻找能量最低的seam
• 重复上述步骤

能量图的计算

“能量”是用来衡量像素的重要程度。如果一个像素有较大的梯度，说明这个像素很大可能Wie边缘。而边缘往往是一幅图像的重要内容。定义能量函数如下去计算每个像素的能量。

遍历图像中每个像素，计算其能量，输出一个与图像相同尺寸的能量图。能量图中边缘像素具有较大亮度，且能量图像也能显现图像的内容。

from skimage import color
import numpy as np
def energy_function(image):
    image_gray = image.copy()
    image_gray = color.rgb2gray(image_gray)
    h,w = image.shape
    G = np.gradient(image)
    out = np.abs(G[0]) + np.abs(G[1])
    return out

计算代价和路径

seam定义为像素从上到下（或从左到右）的连接路径。对于从上到下的像素，从每一行中选取一个像素，这些连接的路径就组成了一条seam。竖直seam（连接方式）为：

n为行数，即相邻两行的连接线只能是当前行像素位置的八连通域子集，即列编号最多相隔一个像素，而非随便连接。同理水平seam定义为：

能量定义了一个像素在图像中的重要程度，为了保留图像汇总的主要内容，删除图像中总能量最低的seam，就保留了图像中的主要内容。对于从上到下的seam。如果一个像素的横坐标为x，那么上一行组成seam的像素的横坐标只能是x-1，x，x+1。那么为了让能量最小，选择上一行这三个像素中能量最小的像素和该像素组成seam，那么通过该点时的总能量为这一点的能量加上上一行像素总能量的最小值，所以利用以下递推公式（从上到下）计算代价：

例如：能量图如下

计算代价为：

由于计算到最后一行时，为经过最后一行像素的seam的最小总能量。那么只要挑选总能量最小的像素，并进行回溯，找到这个最小能量对应的seam即可。如果path[i,j]=-1，则找到左上角的像素，如果为path[i,j]=0则回溯至上方像素；path[i,j]=1，回溯至右上方像素。回溯至上一行计算以上操作，直至到达图像第一行。

#计算代价的函数
def compute_cost(image, energy_map, axis=1):
    if axis == 1:
        energy_map = np.transpose(energy_map)
    h, w = energy_map.shape
    energy_cost = np.zeros((h, w))
    path = np.zeros((h, w))
    energy_cost[0, :] = energy_map[0, :].copy()
    for i in range(1, h):
        for j in range(1, w):
            cost_list = np.array([energy_cost[i - 1, j - 1], energy_cost[i - 1, j], energy_cost[i - 1, min(j + 1, w - 1)]])
            index = np.argmin(cost_list)
            energy_cost[i, j] = energy_map[i, j] + cost_list[index]
            path[i, j] = index - 1
        cost_list_left = np.array([energy_cost[i - 1, 0], energy_cost[i - 1, 1]])
        index = np.argmin(cost_list_left)
        energy_cost[i, 0] = cost_list_left[index] + energy_map[i, 0]
        path[i, 0] = index
    if axis == 0:
        path = np.transpose(path)
        energy_cost = np.transpose(energy_cost)
    return path, energy_cost
#回溯函数
def backtrack_seam(path, end_index):
    h, w = path.shape
    seam = -np.ones(h)
    seam[-1] = end_index.copy()
    for i in range(h - 1):
        if path[h - 1 - i, end_index] == -1:
            end_index = end_index - 1
            seam[h - 2 - i] = end_index
        if path[h - 1 - i, end_index] == 0:
            end_index = end_index
            seam[h - 2 - i] = end_index
        else:
            end_index = end_index + 1
            seam[h - 2 - i] = end_index
    return seam
#把seam最小的seam移除图像
def remove_seam(image, seam):
    seam = seam.astype(int)
    if len(image.shape) == 2:
        image = np.expand_dims(image, axis=2)
    h, w, c = image.shape
    image_new = np.zeros((h, w - 1, c))
    for i in range(h):
        image_new[i, :seam[i]] = image[i, :seam[i]] 
        image_new[i, seam[i]:] = image[i, seam[i] + 1:]
    image_new = np.squeeze(image_new)
    return image_new

迭代的寻找seam，并移除

指定图片的尺寸，对图片进行缩小，主要思路是移除一条seam后，在移除的图像上找到另一个seam，然后再移除，重复上述步骤，知道图像到达指定的维度。

def reduce(image, size, axis=1, efunc=energy_function, cfunc=compute_cost, bfunc=backtrack_seam, rfunc=remove_seam):
    if axis == 0:
        image = np.transpose(image, (1, 0, 2))
    image_new = image.copy()
    w = image.shape[1]
    image_gray = color.rgb2gray(image_new)
    energy_map = efunc(image_gray)
    paths, cost = cfunc(image_gray, energy_map, 1)
    index = np.argmin(cost[-1])
    seam = bfunc(paths, index)
    for i in range(w - size):
        image_new = rfunc(image_new, seam)
        image_gray = color.rgb2gray(image_new)
        energy_map = efunc(image_gray)
        paths, cost = cfunc(image_gray, energy_map, 1)
        index = np.argmin(cost[-1])
        seam = bfunc(paths, index)
    if axis == 0:
        image_new = np.transpose(image_new, (1, 0, 2))
    return image_new

seam carving 对图像进行放大处理

利用seam carving找出几条能量最小的seam，在相应位置重复复制这条seam，对图像进行放大。

#该方法速度快，单效果不好
def duplicate(image, seam):
    if len(image.shape) == 2:
        image = np.expand(image, axis=2)
    h, w, c = image.shape
    image_new = np.zeros((h, w + 1, c))
    for i in range(h):
        image_new[i] = np.insert(image[i], int(seam[i]), image[i , int(seam[i])], axis=0)
    image_new = np.squeeze(image_new)
    return image_new

def enlarge_naive(image, size, axis=1, efunc=energy_function, cfunc=compute_cost, bfunc=backtrack_seam, dfunc=duplicate_seam):
    if axis == 0:
        image = np.transpose(image, (1, 0, 2))
    w = image.shape[1]
    image_gray = color.rgb2gray(image)
    energy_map = efunc(image_gray)
    paths, cost = cfunc(image, energy_map, 1)
    end_index = np.argmin(cost[-1])
    seam = bfunc(paths, end_index)
    image_new = image.copy()
    for i in range(size - w):
        image_new = dfunc(image_new, seam)
    return image_new
#更成熟的想法是，找出N条能量最小的seam，进行复制
def find_seam(image, k, efunc=energy_function, cfunc=compute_cost, bfunc=backtrack_seam, rfunc=remove_seam):
    image_gray = color.rgb2gray(image)
    energy_map = efucn(image_gray)
    paths, cost = cfunc(image, energy_map, axis=1)
    end_index = np.argmin(cost[-1])
    seam = bfunc(paths, end_index)
    seam_list = [seam]
    for i in range(int(k - 1)):
        energy_map = rfunc(energy_map, seam)
        image_gray = rfunc(image_gray, seam)
        paths, cost = cfunc(image_gray, energy_map)
        end_index = np.argmin(cost[-1])
        seam = bfunc(paths, end_index)
        seam_list.append(seam)
    return np.array(seam_list)
def enlarge(image, size, axis=1, efunc=energy_function, cfunc=compute_cost, bfunc=backtrack_seam, rfunc=remove_seam, dfunc=duplicate_seam):
    if axis == 0:
        image = np.transpose(image, (1, 0, 2))
    w = image.shape[1]
    seams = find_seam(image, size - w, efunc, cfunc, bfunc, rfunc)
    image_large = image.copy()
    for i in range(size - w):
        image_large = dfunc(image_large, seams[i])
    if axis == 0:
        image_large = np.transpose(image_large, (1, 0, 2))
   return image_large 

Forward Energy

使用seam carving算法，移除最低能量像素并保留最高的能量像素。因此，平均图像能量增加，可能导致伪影和锯齿边缘。考虑到由于移除seam后，产生的新的邻接像素之间的差。在原递推公式加上约束项，让邻域像素的差尽可能小，这样就可以尽可能消除伪影和锯齿边缘。

def compute_cost_forward(image, energy_map, axis=1):
    if axis == 0:
        image = np.transpose(image, (1, 0, 2))
        energy_map = np.transpose(energy_map, (1, 0, 2))
    image = color.rgb2gray(image)
    h, w = image.shape
    cost = np.zeros((h, w))
    cost[0] = energy_map[0]
    paths = np.zeros((h, w))
    for i in range(1, w - 1):
        cost[0, i] = cost[0, i] + np.abs(image[0, i - 1] - image[0, i + 1])
    cost[0, 0] = energy_map[0, 0]
    cost[0, w - 1] = energy_map[0, w - 1]
    for i in range(1, h):
        for j in range(w):
            if j == 0:
                cost_list = np.array([cost[i - 1, 0], cost[i - 1, 1] + np.abs(image[i - 1, 0] - image[i, 1])])
                index = np.argmin(cost_list)
                paths[i, j] = index
                cost[i, j] = energy_map[i, j] + cost_list[index]
            elif j == w - 1:
                cost_list = np.array([cost[i - 1, w - 2] + np.abs(image[i - 1, w - 1] - image[i, w - 2]), cost[i - 1, w - 1]])
                index = np.argmin(cost_list)
                paths[i, j] = index - 1
                cost[i, j] = energy_map[i, j] + cost_list[index]
            else:
                diff_left = np.abs(image[i, j + 1] - image[i, j - 1]) + np.abs(image[i - 1, j] - image[i, j - 1])
                diff_mid = np.abs(image[i, j + 1] - image[i, j + 1])
                diff_right = np.abs(image[i, j + 1] - image[i, j - 1]) + np.abs(image[i - 1, j] - image[i, j + 1])
                cost_list = np.array([cost[i - 1, j - 1] + diff_left, cost[i - 1, j] + diff_mid, cost[i - 1, j + 1] + diff_right])
                index = np.argmin(cost_list)
                paths[i, j] = index - 1
                cost[i, j] = energy_map[i, j] + cost_list[index]
    if aixs == 0:
        image = np.transpose(image, (1, 0, 2))
        energy_map = np.transpose(energy_map, (1, 0, 2))
    return paths, cost

seam carving 移除图像中的某物体

利用seam carving算法移除图中某一物体的主要思想为：将图像中物体所对应的像素的能量定义为一个较大的负数，以确保在利用seam carving算法时，使得物体所对应的像素能够删除；物体删除后，图像尺寸得以缩小，再利用上面提到的方法把图像扩大为原尺寸即可。

from skimage import measure
#从左到右删除
def remove_object_lr(image, mask):
    h, w, c = image.shape
    mask = mask.astype(int)
    regions = meansure.regionprops(mask)
    region = regions[0]
    if region.bbox[2] - region.bbox[0] < region.bbox[3] - region.bbox[1]:
        image = np.transpose(image, (1, 0, 2))
        mask = np.transpose(mask, (1, 0))
    image_gray = color.rgb2gray(image)
    count = 0
    image_new = image.copy()
    while not np.all(mask == 0):
        energy_map = energy_function(image_gray)
        energy_map = energy_map - energy_map * mask * (1000)
        paths, cost = compute_cost_forward(image_gray, energy_map)
        index = np.argmin(cost[-1])
        seam = backtrack_seam(paths, index)
        image_gray = remove_seam(image_gray, seam)
        image_new = remove_seam(image_new, seam)
        mask = remove_seam(mask, seam)
        count = count + 1
    out = enlarge(image_new, image_new.shape[1] + count)
    if region.bbox[2] - region.bbox[0] < region.bbox[3] - region.bbox[1]:
        out = np.transpose(out, (1, 0, 2))
    rturn out

#从上到下删除
def remove_object_tb(image, mask):
    h, w, c = image.shape
    mask = mask.astype(int)
    image_gray = color.rgb2gray(image)
    count = 0
    image_new = image.copy()
    while not np.all(mask == 0):
        energy_map = energy_function(image_gray)
        energy_map = energy_map - energy_map * mask * (1000)
        paths, cost = compute_cost_forward(image_gray, energy_map)
        index = np.argmin(cost[-1])
        seam = backtrack_seam(paths, index)
        image_gray = remove_seam(image_gray, seam)
        image_new = remove_seam(image_new,seam)
        mask = remove_seam(mask, seam)
        count = count + 1
    out = enlarge(image_new, image_new.shape[1] + count)
    return out

显著性检测和Seem Carving融合裁剪图像

# coding=utf-8
# /usr/bin/env python
"""
Author: studyeboy
Date: 2021/1/11 下午2:53
Description: seam carving test
"""
import os
import sys
import argparse
from scipy import ndimage as ndi
import numpy as np
from numba import jit
import cv2

import torch
import torch.nn.functional as F
import torchvision.transforms as transforms
from model.CPD_models import CPD_VGG

import warnings
warnings.filterwarnings('ignore')

#utility code
def visualize(image, boolmask=None, rotate=False, seam_color=np.array([255, 200, 200])):
    vis = image.astype(np.uint8)
    if boolmask is not None:
        vis[np.where(boolmask == False)] = seam_color
    if rotate:
        vis = rotate_image(vis, False)
    cv2.imshow('visualization', vis)
    cv2.waitKey(1)
    return vis

def rotate_image(image, clockwise):
    k = 1 if clockwise else 3
    return np.rot90(image, k)

#energy functions
def backward_energy(image):
    '''
    simple gradient magnitude energy map.
    :param image:
    :return: grad_mag
    '''
    xgrad = ndi.convolve1d(image, np.array([1, 0, -1]), axis=1, mode='wrap')
    ygrad = ndi.convolve1d(image, np.array([1, 0, -1]), axis=0, mode='wrap')

    grad_mag = np.sqrt(np.sum(xgrad ** 2, axis=2) + np.sum(ygrad ** 2, axis=2))

    return grad_mag

@jit
def forward_energy(image):
    '''
    Forward energy algorithm as described in 'Improved Seam Carving for Video Retargeting'
    by Rubinstein, Shamir, Avidan.
    Vectorized cod adapted from https://github.com/axu2/improved-seam-carving.
    :param image:
    :return: energy
    '''
    h, w = image.shape[:2]
    image = cv2.cvtColor(image.astype(np.uint8), cv2.COLOR_BGR2GRAY).astype(np.float64)

    energy = np.zeros((h, w))
    m = np.zeros((h, w))

    U = np.roll(image, 1, axis=0) # I(i-1, j)
    L = np.roll(image, 1, axis=1) #I(i, j-1)
    R = np.roll(image, -1, axis=1)#I(i, j+1)

    cU = np.abs(R - L) #|I(i, j-1) - I(i, j+1)|
    cL = np.abs(U - L) + cU #|I(i-1, j) - I(i, j-1)| + |I(i, j-1) - I(i, j+1)|
    cR = np.abs(U - R) + cU #|I(i-1, j) - I(i, j+1)| + |I(i, j-1) - I(i, j+1)|

    for i in range(1, h):
        mU = m[i - 1]
        mL = np.roll(mU, 1)
        mR = np.roll(mU, -1)

        mULR = np.array([mU, mL, mR])
        cULR = np.array([cU[i], cL[i], cR[i]])
        mULR += cULR

        argmins = np.argmin(mULR, axis=0)
        m[i] = np.choose(argmins, mULR)
        energy[i] = np.choose(argmins, cULR)

        return energy

#seam help functions
@jit
def add_seam(image, seam_idx):
    '''
    Add a vertiall seam to a 3-channel color image at the indices provided
    by averaging the pixels values to the left and right of the seam.
    :param image:
    :param seam_idx:
    :return: output
    '''
    h, w = image.shape[:2]
    output = np.zeros((h, w + 1, 3))
    for row in range(h):
        col = seam_idx[row]
        for ch in range(3):
            if col == 0:
                p = np.average(image[row, col: col + 2, ch])
                output[row, col, ch] = image[row, col, ch]
                output[row, col + 1, ch] = p
                output[row, col + 1:, ch] = image[row, col:, ch]
            else:
                p = np.average(image[row, col - 1: col + 1, ch])
                output[row, :col, ch] = image[row, :col, ch]
                output[row, col, ch] = p
                output[row, col + 1:, ch] = image[row, col:, ch]
    return output

@jit
def add_seam_grayscale(image, seam_idx):
    '''
    Add a vertical seam to a grayscale image at the indices provided
    by average the pixels values to the left and right of the seam.
    :param image:
    :param seam_idx:
    :return: output
    '''
    h, w = image.shape[:2]
    output = np.zeros((h, w + 1))
    for row in range(h):
        col = seam_idx[row]
        if col == 0:
            p = np.average(image[row, col: col + 2])
            output[row, col] = image[row, col]
            output[row, col + 1] = p
            output[row, col + 1:] = image[row, col:]
        else:
            p = np.average(image[row, col - 1: col + 1])
            output[row, :col] = image[row, :col]
            output[row, col] = p
            output[row, col:] = image[row, col:]
    return output

@jit
def remove_seam(image, boolmask):
    h, w = image.shape[:2]
    boolmask3c = np.stack([boolmask] * 3, axis=2)
    return image[boolmask3c].reshape((h, w - 1, 3))

@jit
def remove_seam_grayscale(image, boolmask):
    h, w = image.shape[:2]
    return image[boolmask].reshape((h, w - 1))

@jit
def get_minimum_seam(image, mask=None, remove_mask=None, energyfn=forward_energy, mask_threshold=10, energy_mask_const=100000.0 ):
    '''
    DP algorithm for finding the seam of minimum energy.Code adapted from
    https://karthikkaranth.me/blog/implementing-seam-carving-with-python/
    :param image:
    :param mask:
    :param remove_mask:
    :param energyfn: use forward_energy or backward_energy function
    :param mask_threshold: minimum pixel intensity for binary mask
    :param energy_mask_const: large energy value for protective masking
    :return: np.array(seam_idx), boolmask
    '''
    h, w = image.shape[:2]
    M = energyfn(image)

    if mask is not None:
        M[np.where(mask > mask_threshold)] = energy_mask_const
    #give removal mask priority over protective mask by using larger negative value
    if remove_mask is not None:
        M[np.where(remove_mask > mask_threshold)] = -energy_mask_const * 100

    backtrack = np.zeros_like(M, dtype=np.int)

    #populate DP matrix
    for i in range(1, h):
        for j in range(0, w):
            if j == 0:
                idx = np.argmin(M[i - 1, j:j + 2])
                backtrack[i, j] = idx + j
                min_energy = M[i - 1, idx + j]
            else:
                idx = np.argmin(M[i - 1, j - 1:j + 2])
                backtrack[i, j] = idx + j - 1
                min_energy = M[i - 1, idx + j -1]

            M[i, j] += min_energy
    #backtrack to find path
    seam_idx = []
    boolmask = np.ones((h, w), dtype=np.bool)
    j = np.argmin(M[-1])
    for i in range(h - 1, -1, -1):
        boolmask[i, j] = False
        seam_idx.append(j)
        j = backtrack[i, j]

    seam_idx.reverse()
    return np.array(seam_idx), boolmask

#main algorithm
def seams_removal(image, num_remove, mask=None, vis=False, rot=False):
    for _ in range(num_remove):
        seam_idx, boolmask = get_minimum_seam(image, mask)
        if vis:
            visualize(image, boolmask, rotate=rot)
        image = remove_seam(image, boolmask)
        if mask is not None:
            mask = remove_seam_grayscale(mask, boolmask)
    return image, mask


def seams_insertion(image, num_add, mask=None, vis=False, rot=False):
    seams_record = []
    temp_img = image.copy()
    temp_mask = mask.copy() if mask is not None else None

    for _ in range(num_add):
        seam_idx, boolmask = get_minimum_seam(temp_img, temp_mask)
        if vis:
            visualize(temp_img, boolmask, rotate=rot)

        seams_record.append(seam_idx)
        temp_img = remove_seam(temp_img, boolmask)
        if temp_mask is not None:
            temp_mask = remove_seam_grayscale(temp_mask, boolmask)

    seams_record.reverse()
    for _ in range(num_add):
        seam = seams_record.pop()
        image = add_seam(image, seam)
        if vis:
            visualize(image, rotate=rot)
        if mask is not None:
            mask = add_seam_grayscale(mask, seam)

        #update the remaining seam indices
        for remaining_seam in seams_record:
            remaining_seam[np.where(remaining_seam >= seam)] += 2

    return image, mask


#main driver functions
def seam_carving(image, dy, dx, mask=None, vis=False):
    image = image.astype(np.float64)
    h, w = image.shape[:2]
    assert h + dy > 0 and w + dx > 0 and dy <= h and dx <= w

    if mask is not None:
        mask = mask.astype(np.float64)

    output = image

    if dx < 0:
        output, mask = seams_removal(output, -dx, mask, vis)

    elif dx > 0:
        output, mask = seams_insertion(output, dx, mask, vis)

    if dy < 0:
        output = rotate_image(output, True)
        if mask is not None:
            mask = rotate_image(mask, True)
        output, mask = seams_removal(output, -dy, mask, vis, rot=True)
        output = rotate_image(output, False)

    elif dy > 0:
        output = rotate_image(output, True)
        if mask is not None:
            mask = rotate_image(mask, True)
        output, mask = seams_insertion(output, dy, mask, vis, rot=True)
        output = rotate_image(output, False)
    return output


def model_init(model_dir):
    model = CPD_VGG()
    model.load_state_dict(torch.load(model_dir))
    cuda = torch.cuda.is_available()
    if cuda:
        model.cuda()
    model.eval()
    return model

def cpd_semantic(model, image, input_size=224):
    h, w = image.shape[:2]
    scale_img = cv2.resize(image, (input_size, input_size), interpolation=cv2.INTER_AREA)
    normalize = transforms.Compose([
        transforms.ToTensor(),
        transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
    ])
    scale_img_rgb = cv2.cvtColor(scale_img, cv2.COLOR_BGR2RGB)
    tensor_img = normalize(scale_img_rgb).unsqueeze(0)
    if torch.cuda.is_available():
        tensor_img = tensor_img.cuda()

    _, res = model(tensor_img)
    res = F.interpolate(res, size=(h, w), mode='bilinear', align_corners=False)
    if torch.cuda.is_available():
        res = res.cpu()
    res = res.sigmoid().data.numpy().squeeze()
    res = (res - res.min()) / (res.max() - res.min() + 1e-8)
    mask = np.where((res * 255).astype(np.uint8) > 0, 255, 0)
    return mask


def parse_arguments(argv):
    parser = argparse.ArgumentParser()
    parser.add_argument('--image_dir', type=str, help='input images dir', \
                        default='/mnt/disk2/data/SOD/book/11_small')
    parser.add_argument('--output_dir',type=str, help='output images dir', \
                        default='/mnt/disk2/data/SOD/book/11_seam_carving')
    parser.add_argument('--model-dir', type=str, help='input model dir', \
                        default='./CPD.pth')
    return parser.parse_args(argv)


def main(args):
    output_dir = os.path.join(args.output_dir)
    if not os.path.exists(output_dir):
        os.makedirs(output_dir)

    #cpd
    model = model_init(args.model_dir)

    for image_name in os.listdir(args.image_dir):
        image_path = os.path.join(args.image_dir, image_name)
        image = cv2.imread(image_path)
        assert image is not None
        h, w = image.shape[:2]
        #change w and h
        new_h = np.minimum(h, w)
        new_w = np.minimum(h, w)
        dy = new_h - h
        dx = new_w - w

        #cpd method
        mask = cpd_semantic(model, image)

        # image resize mode
        output = seam_carving(image, dy, dx, mask)

        cv2.imwrite(os.path.join(args.output_dir, image_name), output)



if __name__ == '__main__':
    main(parse_arguments(sys.argv[1:]))

参考资料

CV | 智能缩放：浅谈Seam Carving算法 (1)
图像接缝裁剪（seam carving）算法实现-SIGGRAPH 2007
Seam Carving算法