SSIM与MS-SSIM图像评价函数

SSIM的全称为structural similarity index，即为结构相似性，是一种衡量两幅图像相似度的指标。该指标首先由德州大学奥斯丁分校的图像和视频工程实验室(Laboratory for Image and Video Engineering)提出。而如果两幅图像是压缩前和压缩后的图像，那么SSIM算法就可以用来评估压缩后的图像质量。

在实际应用中，一般采用高斯函数计算图像的均值、方差以及协方差，而不是采用遍历像素点的方式，以换来更高的效率。

具体步骤：

更正下协方差计算

python 代码：

def keras_SSIM_cs(y_true, y_pred):
    axis=None
    gaussian = make_kernel(1.5)
    x = tf.nn.conv2d(y_true, gaussian, strides=[1, 1, 1, 1], padding='SAME')
    y = tf.nn.conv2d(y_pred, gaussian, strides=[1, 1, 1, 1], padding='SAME')

    u_x=K.mean(x, axis=axis)
    u_y=K.mean(y, axis=axis)

    var_x=K.var(x, axis=axis)
    var_y=K.var(y, axis=axis)

    cov_xy=cov_keras(x, y, axis)

    K1=0.01
    K2=0.03
    L=1  # depth of image (255 in case the image has a differnt scale)

    C1=(K1*L)**2
    C2=(K2*L)**2
    C3=C2/2

    l = ((2*u_x*u_y)+C1) / (K.pow(u_x,2) + K.pow(u_x,2) + C1)
    c = ((2*K.sqrt(var_x)*K.sqrt(var_y))+C2) / (var_x + var_y + C2)
    s = (cov_xy+C3) / (K.sqrt(var_x)*K.sqrt(var_y) + C3)

    return [c,s,l]

def keras_MS_SSIM(y_true, y_pred):
    iterations = 5
    x=y_true
    y=y_pred
    weight = [0.0448, 0.2856, 0.3001, 0.2363, 0.1333]
    c=[]
    s=[]
    for i in range(iterations):
        cs=keras_SSIM_cs(x, y)
        c.append(cs[0])
        s.append(cs[1])
        l=cs[2]
        if(i!=4):
            x=tf.image.resize_images(x, (x.get_shape().as_list()[1]//(2**(i+1)), x.get_shape().as_list()[2]//(2**(i+1))))
            y=tf.image.resize_images(y, (y.get_shape().as_list()[1]//(2**(i+1)), y.get_shape().as_list()[2]//(2**(i+1))))
    c = tf.stack(c)
    s = tf.stack(s)
    cs = c*s

    #Normalize: suggestion from https://github.com/jorge-pessoa/pytorch-msssim/issues/2 last comment to avoid NaN values
    l=(l+1)/2
    cs=(cs+1)/2

    cs=cs**weight
    cs = tf.reduce_prod(cs)
    l=l**weight[-1]

    ms_ssim = l*cs
    ms_ssim = tf.where(tf.is_nan(ms_ssim), K.zeros_like(ms_ssim), ms_ssim)

    return K.mean(ms_ssim)

MATLAB代码：

function [mssim, ssim_map] = ssim(img1, img2, K, window, L)

% ========================================================================
% Edited code by Adam Turcotte and Nicolas Robidoux
% Laurentian University
% Sudbury, ON, Canada
% Last Modified: 2011-01-22
% ----------------------------------------------------------------------
% This code implements a refactored computation of SSIM that requires
% one fewer blur (4 instead of 5), the same number of pixel-by-pixel
% binary operations (10), and two fewer unary operations (6 instead of 8).
%
% In addition, this version reduces memory usage with in-place functions.
% As a result, it supports larger input images.
%========================================================================

% ========================================================================
% SSIM Index with automatic downsampling, Version 1.0
% Copyright(c) 2009 Zhou Wang
% All Rights Reserved.
%
% ----------------------------------------------------------------------
% Permission to use, copy, or modify this software and its documentation
% for educational and research purposes only and without fee is hereby
% granted, provided that this copyright notice and the original authors'
% names appear on all copies and supporting documentation. This program
% shall not be used, rewritten, or adapted as the basis of a commercial
% software or hardware product without first obtaining permission of the
% authors. The authors make no representations about the suitability of
% this software for any purpose. It is provided "as is" without express
% or implied warranty.
%----------------------------------------------------------------------
%
% This is an implementation of the algorithm for calculating the
% Structural SIMilarity (SSIM) index between two images
%
% Please refer to the following paper and the website with suggested usage
%
% Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image
% quality assessment: From error visibility to structural similarity,"
% IEEE Transactios on Image Processing, vol. 13, no. 4, pp. 600-612,
% Apr. 2004.
%
% http://www.ece.uwaterloo.ca/~z70wang/research/ssim/
%
% Note: This program is different from ssim_index.m, where no automatic
% downsampling is performed. (downsampling was done in the above paper
% and was described as suggested usage in the above website.)
%
% Kindly report any suggestions or corrections to zhouwang@ieee.org
%
%----------------------------------------------------------------------
%
%Input : (1) img1: the first image being compared
%        (2) img2: the second image being compared
%        (3) K: constants in the SSIM index formula (see the above
%            reference). defualt value: K = [0.01 0.03]
%        (4) window: local window for statistics (see the above
%            reference). default widnow is Gaussian given by
%            window = fspecial('gaussian', 11, 1.5);
%        (5) L: dynamic range of the images. default: L = 255
%
%Output: (1) mssim: the mean SSIM index value between 2 images.
%            If one of the images being compared is regarded as 
%            perfect quality, then mssim can be considered as the
%            quality measure of the other image.
%            If img1 = img2, then mssim = 1.
%        (2) ssim_map: the SSIM index map of the test image. The map
%            has a smaller size than the input images. The actual size
%            depends on the window size and the downsampling factor.
%
%Basic Usage:
%   Given 2 test images img1 and img2, whose dynamic range is 0-255
%
%   [mssim, ssim_map] = ssim(img1, img2);
%
%Advanced Usage:
%   User defined parameters. For example
%
%   K = [0.05 0.05];
%   window = ones(8);
%   L = 100;
%   [mssim, ssim_map] = ssim(img1, img2, K, window, L);
%
%Visualize the results:
%
%   mssim                        %Gives the mssim value
%   imshow(max(0, ssim_map).^4)  %Shows the SSIM index map
%========================================================================


if (nargin < 2 || nargin > 5)
   mssim = -Inf;
   ssim_map = -Inf;
   return;
end

if (size(img1) ~= size(img2))
   mssim = -Inf;
   ssim_map = -Inf;
   return;
end

[M N] = size(img1);

if (nargin == 2)
   if ((M < 11) || (N < 11))
	   mssim = -Inf;
	   ssim_map = -Inf;
      return
   end
   window = fspecial('gaussian', 11, 1.5);	%
   K(1) = 0.01;					% default settings
   K(2) = 0.03;					%
   L = 255;                                     %
end

if (nargin == 3)
   if ((M < 11) || (N < 11))
	   mssim = -Inf;
	   ssim_map = -Inf;
      return
   end
   window = fspecial('gaussian', 11, 1.5);
   L = 255;
   if (length(K) == 2)
      if (K(1) < 0 || K(2) < 0)
		   mssim = -Inf;
   		ssim_map = -Inf;
	   	return;
      end
   else
	   mssim = -Inf;
   	ssim_map = -Inf;
	   return;
   end
end

if (nargin == 4)
   [H W] = size(window);
   if ((H*W) < 4 || (H > M) || (W > N))
	   mssim = -Inf;
	   ssim_map = -Inf;
      return
   end
   L = 255;
   if (length(K) == 2)
      if (K(1) < 0 || K(2) < 0)
		   mssim = -Inf;
   		ssim_map = -Inf;
	   	return;
      end
   else
	   mssim = -Inf;
   	ssim_map = -Inf;
	   return;
   end
end

if (nargin == 5)
   [H W] = size(window);
   if ((H*W) < 4 || (H > M) || (W > N))
	   mssim = -Inf;
	   ssim_map = -Inf;
      return
   end
   if (length(K) == 2)
      if (K(1) < 0 || K(2) < 0)
		   mssim = -Inf;
   		ssim_map = -Inf;
	   	return;
      end
   else
	   mssim = -Inf;
   	ssim_map = -Inf;
	   return;
   end
end


img1 = double(img1);
img2 = double(img2);

% automatic downsampling
f = max(1,round(min(M,N)/256));
%downsampling by f
%use a simple low-pass filter 
if(f>1)
    lpf = ones(f,f);
    lpf = (1./(f*f))*lpf;
    img1 = imfilter(img1,lpf,'symmetric','same');
    img2 = imfilter(img2,lpf,'symmetric','same');

    img1 = img1(1:f:end,1:f:end);
    img2 = img2(1:f:end,1:f:end);
end

C1 = (K(1)*L)^2;
C2 = (K(2)*L)^2;
window = window/sum(sum(window));
ssim_map = filter2(window, img1, 'valid');        % gx
w1 = filter2(window, img2, 'valid');              % gy
w2 = ssim_map.*w1;                                % gx*gy
w2 = 2*w2+C1;                                     % 2*(gx*gy)+C1 = num1
w1 = (w1-ssim_map).^2+w2;                         % (gy-gx)^2+num1 = den1
ssim_map = filter2(window, img1.*img2, 'valid');  % g(x*y)
ssim_map = (2*ssim_map+(C1+C2))-w2;               % 2*g(x*y)+(C1+C2)-num1 = num2
ssim_map = ssim_map.*w2;                          % num
img1 = img1.^2;                                   % x^2
img2 = img2.^2;                                   % y^2
img1 = img1+img2;                                 % x^2+y^2

if (C1 > 0 && C2 > 0)
   w2 = filter2(window, img1, 'valid');           % g(x^2+y^2)
   w2 = w2-w1+(C1+C2);                            % den2
   w2 = w2.*w1;                                   % den
   ssim_map = ssim_map./w2;                       % num/den = ssim
else
   w3 = filter2(window, img1, 'valid');           % g(x^2+y^2)
   w3 = w3-w1+(C1+C2);                            % den2
   w4 = ones(size(w1));
   index = (w1.*w3 > 0);
   w4(index) = (ssim_map(index))./(w1(index).*w3(index));
   index = (w1 ~= 0) & (w3 == 0);
   w4(index) = w2(index)./w1(index);
   ssim_map = w4;
end

mssim = mean2(ssim_map);

return

C++代码

#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <iostream>

using namespace std; 
using namespace cv;

Scalar getMSSIM(Mat  inputimage1, Mat inputimage2);
int main()
{
    Mat BlurImage1;
    Mat BlurImage2;
    Mat SrcImage = imread("1.jpg");
    blur(SrcImage, BlurImage1, Size(5, 5));
    blur(SrcImage,BlurImage2,Size(10,10));
    Scalar SSIM1 = getMSSIM(SrcImage, BlurImage1);
    Scalar SSIM2 = getMSSIM(SrcImage, BlurImage2);
    printf("模糊5*5通道1：%f\n", SSIM1.val[0] * 100);
    printf("模糊5*5通道2：%f\n", SSIM1.val[1] * 100);
    printf("模糊5*5通道3：%f\n", SSIM1.val[2] * 100);
    printf("模糊5*5：%f\n", (SSIM1.val[2] + SSIM1.val[1] + SSIM1.val[0])/3 * 100);
    printf("模糊10*10通道1：%f\n", SSIM2.val[0] * 100);
    printf("模糊10*10通道2：%f\n", SSIM2.val[1] * 100);
    printf("模糊10*10通道3：%f\n", SSIM2.val[2] * 100);
    printf("模糊10*10：%f\n", (SSIM2.val[2] + SSIM2.val[1] + SSIM2.val[0]) / 3 * 100);
    imshow("原图",SrcImage);
    imshow("模糊5*5",BlurImage1);
    imshow("模糊10*10", BlurImage2);
    waitKey(0);
    return 0;
}
Scalar getMSSIM(Mat  inputimage1, Mat inputimage2)
{
    Mat i1 = inputimage1;
    Mat i2 = inputimage2;
    const double C1 = 6.5025, C2 = 58.5225;
    int d = CV_32F;
    Mat I1, I2;
    i1.convertTo(I1, d);
    i2.convertTo(I2, d);
    Mat I2_2 = I2.mul(I2);
    Mat I1_2 = I1.mul(I1);
    Mat I1_I2 = I1.mul(I2);
    Mat mu1, mu2;
    GaussianBlur(I1, mu1, Size(11, 11), 1.5);
    GaussianBlur(I2, mu2, Size(11, 11), 1.5);
    Mat mu1_2 = mu1.mul(mu1);
    Mat mu2_2 = mu2.mul(mu2);
    Mat mu1_mu2 = mu1.mul(mu2);
    Mat sigma1_2, sigma2_2, sigma12;
    GaussianBlur(I1_2, sigma1_2, Size(11, 11), 1.5);
    sigma1_2 -= mu1_2;
    GaussianBlur(I2_2, sigma2_2, Size(11, 11), 1.5);
    sigma2_2 -= mu2_2;
    GaussianBlur(I1_I2, sigma12, Size(11, 11), 1.5);
    sigma12 -= mu1_mu2;
    Mat t1, t2, t3;
    t1 = 2 * mu1_mu2 + C1;
    t2 = 2 * sigma12 + C2;
    t3 = t1.mul(t2);
    t1 = mu1_2 + mu2_2 + C1;
    t2 = sigma1_2 + sigma2_2 + C2;
    t1 = t1.mul(t2);
    Mat ssim_map;
    divide(t3, t1, ssim_map);
    Scalar mssim = mean(ssim_map);
    return mssim;
}

这里附一个Python的工具箱，有各种评价函数：

The following metrics are included:

• Mean-Squared-Error (MSE).

• Peak-Signal-to-Noise-Ratio (PSNR).

• Structural Similarity Index (SSIM).

• Normalized Mutual Information (NMI).

• Image Complexity.

• Resolution analysis through Edge-Profile-Fitting (EPF).

• Resolution analysis through Fourier Ring Correlation (FRC).

The following routines to construct simulated datasets are included:

• Create a Shepp-Logan phantom.

• Create generic phantoms with analytical X-ray transform.

• Rescale image.

• Downsample sinogram.

• Add Gaussian or Poisson noise.

• Add Gaussian blurring.

   https://github.com/arcaduf/image_quality_assessment

【1】https://ieeexplore.ieee.org/abstract/document/1292216

 【2】Image quality assessment: From error visibility to structural similarity,