运动目标检测——GMM高斯混合模型

最新推荐文章于 2024-07-16 17:56:45 发布

_helen_520

最新推荐文章于 2024-07-16 17:56:45 发布

阅读量3.8k

点赞数 2

分类专栏：背景建模与前景提取

本文链接：https://blog.csdn.net/haronchou/article/details/107709165

版权

背景建模与前景提取专栏收录该内容

31 篇文章 21 订阅

订阅专栏

对基于高斯混合模型的背景建模方法进行总结

1. 1999，Stauffer C & Grimson GMM

paper: Adaptive Background Mixture Models for Real-Time Tracking
博客上的许多代码都是基于这个模型来写的；
bgslibrary的DPGrimsonGMMBGS代码
OpenCV中的MOG，位于之前的版本中。在OpenCV3.x的版本中没有MOG源码了。Opencv2.4.13.6的源码地址：opencv/modules/video/src/bgfg_gaussmix.cpp

2. 2001，P. KadewTraKuPong and R. Bowden

paper：An improved adaptive background mixture model for real-time tracking with shadow detection, Proc. 2nd European Workshp on Advanced Video-Based Surveillance Systems, 2001
bgslibrary的GUI里面有，就是mixtureOfGuassianV1.

简单描述： OpenCV实现的是这篇文章。他的创新点就是用EM初始化每个高斯模型的参数；传统的高斯模型存在学习速度慢，特别对于复杂的环境，并且不能区分移动的阴影和移动的目标。这篇文章提出在更新方程上做了一定的修改，在不同的阶段使用不同的方程，并且介绍了一种阴影检测的方法。

3. Zivkovic，2004,2006.

Paper:

(1) ICPR2004. Z.Zivkovic, Improved adaptive Gausian mixture model for background subtraction, International Conference Pattern Recognition, UK, August, 2004, The code is very fast and performs also shadow detection. Number of Gausssian components is adapted per pixel. 这篇文章的创新点是对混合高斯模型参数更新的方式，递归方程被用来不断的更新参数
(2) Z.Zivkovic, F. van der Heijden, Efficient Adaptive Density Estimation per Image Pixel for the Task of Background Subtraction, Pattern Recognition Letters, vol. 27, no. 7, pages 773-780, 2006. The algorithm similar to the standard Stauffer&Grimson algorithm with additional selection of the number of the Gaussian components based on:
(3)PAMI2004. Z.Zivkovic, F.van der Heijden, Recursive unsupervised learning of finite mixture models, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol.26, no.5, pages 651-656, 2004.

code:

bgslibrary里的MixtureOfGaussianV2，调用了opencv2.4.13.6里面的opencv/modules/video/src/bgfg_gaussmix2.cpp源码的实现，MixtureOfGaussianV2只是调用了这个函数。
bgslibrary里的DPZivkovicAGMMBGS也实现的是上面的文献(2)的代码。
opencv3.4.0中MOG2的使用方法：https://docs.opencv.org/3.4/d1/dc5/tutorial_background_subtraction.html

参考pudn的代码，把mog2从opencv里面提出来单独运行。代码见后页

3.1 算法流程分析

I(k+1)是当前像素值；如果当前像素属于当前模型，则这个模型的权重增加α。
ρ=α/w，用ρ来对当前像素加权，去更新均值和方差。权重越大，那么当前模型占主要模型，那么背景模型的均值和方差就更新的慢一点，当前像素对模型的更改要小一些，历史的权重要大一些。如果权重太小，那么当前像素对模型的贡献就要大一些，历史的权重就会小一些。
如果α学习率很大，那么三个值就会增加的更快。模型的权重、均值、方差就会变化更大，即当前像素对模型的影响会更大，所以背景更新的会越快。更新的快就会更适应动态变化，将动态变化较快的融入到背景模型中，检测不会很灵敏。
所以α是权重的学习率，累加学习率；ρ是均值和方差的累加学习率。
模型排队：按照权重来排队；Grimson中用w/σ来排队。
检查是否为前景像素：权重累积<0.9 且 4*σ 不去访问最后的概率<0.1的模型
检查是否属于这个背景模型：3*σ

3.2 效果分析

静止变为运动后：需要20帧左右才会消除鬼影。说明新像素融入到模型中的，然后成为背景的实际需要20帧。更新率为0.005,20帧之后，20*0.005=0.1，增加了0.1的权重后，就可以变成背景。初始权重为1/2的样子。

main中的对象定义和使用；使用时mog2(current_gray, fore, learning_rate)

构造函数给出了一些参数默认设置

4. Git desktop使用方法

https://docs.github.com/cn/desktop/getting-started-with-github-desktop

https://www.cnblogs.com/schaepher/p/4823181.html

main

#include "mog2.h"   


#define  DEFAULT_WIDTH 352   
#define  DEFAULT_HEIGHT 288   

#include "malloc.h"   
#include "stdio.h"   
#include "stdlib.h"   

#include "math.h"   

#include "cv.h"   
#include "highgui.h"   
#include "cxcore.h"   
#include <cvaux.h>   

#include<cstdio>   
#include<iostream>   

#include <string.h>   

using namespace cv;
using namespace std;

using namespace ZWQ;

int  main()
{
	//char file_name[120] = { 0 };
	char save_name[120] = { 0 };
	char save_image_file[130] = { 0 };
	FILE*  pFile = fopen("videofile_mog.txt", "r+");
	int len = 0;


	Mat frame;
	Mat foreground;
	Mat background;

	BackgroundSubtractorMOG2_Z mog2;
	int num;
#if 0
	while (!feof(pFile))
	{

		memset(file_name, 0, sizeof(file_name));
		fgets(file_name, sizeof(file_name)-1, pFile);
		printf("%s", file_name);

		if (!feof(pFile))
		{
			memset(save_name, 0, sizeof(save_name));
			fgets(save_name, sizeof(save_name)-1, pFile);
			printf("%s\n", save_name);
			memset(save_image_file, 0, sizeof(save_image_file));
		}

		len = sizeof(file_name);
		while (len>0)
		{
			if ('\n' == file_name[len - 1])
			{
				file_name[len - 1] = '\0'; // ȥ��'\n'�ַ�     
				break;
			}
			len--;
		}

		len = sizeof(save_name);
		while (len>0)
		{
			if ('\n' == save_name[len - 1])
			{
				save_name[len - 1] = '\0'; // ȥ��'\n'�ַ�     
				break;
			}
			len--;
		}
#endif
		while (true){
			string file_name = "G:/202007_work/data_video/fire.avi";

			VideoCapture  capture(file_name);
			num = 1;
			while (capture.read(frame))
			{
				//sprintf(save_image_file, "%s%d.jpg", save_name, num);  //���ɱ���ͼƬ·��   
				std::vector<cv::Mat>SrcMatpart(frame.channels());      //生成与通道数数目相等的图像容器
				cv::split(frame, SrcMatpart);                                          //分解与通道数数目相等的图像容器
				mog2(SrcMatpart[0], foreground, 0.005);
				cv::medianBlur(foreground, foreground, 3);


				//std::vector<cv::Mat>mask(frame.channels());
				//for (int i = 0; i < frame.channels(); i++){
				//	foreground.copyTo(mask[i]);
				//	mask[i] /= 255;
				//	mask[i] = 1 - mask[i];
				//}

				//cv::Mat MergeMat;
				//cv::merge(mask, MergeMat);                                //合成与通道数数目相等的图像容器

				cv::Mat colorMask(frame.size(),frame.type(),Scalar(1,1,1));
				//colorMask = colorMask.mul(MergeMat);

				for (int i = 0; i < frame.rows;i++)
				for (int j = 0; j < frame.cols; j++){
					if (foreground.at<uchar>(i, j) == 0)
						colorMask.at<Vec3b>(i, j) = Vec3b(1,1,1);
					else
						colorMask.at<Vec3b>(i, j) = Vec3b(0, 0, 255);;
				}


				imshow("前景叠加在背景上", colorMask.mul(frame));
				//imwrite(save_image_file, foreground);
				imshow("fore", foreground);
				waitKey(500);   
				num++;
			}
		}
	fclose(pFile);
	system("pause");
	getchar();
	return 0;

	return  0;
}

mog2.h

#ifndef __MOG2_HPP__ 
#define __MOG2_HPP__ 


#include <list> 

#include "opencv2/core/core.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/video/background_segm.hpp"
#include "opencv2/highgui/highgui.hpp"
#include <stdio.h>


#include "opencv2/core/mat.hpp"
#include "opencv2/core/internal.hpp"


using namespace std;
using namespace cv;


namespace ZWQ
{

	/*!
	The Base Class for Background/Foreground Segmentation

	The class is only used to define the common interface for
	the whole family of background/foreground segmentation algorithms.
	*/
	class CV_EXPORTS_W BackgroundSubtractor /*: public Algorithm*/
	{
	public:
		//! the virtual destructor 
		virtual ~BackgroundSubtractor();
		//! the update operator that takes the next video frame and returns the current foreground mask as 8-bit binary image. 
		CV_WRAP_AS(apply) virtual void operator()(InputArray image, OutputArray fgmask,
			double learningRate = 0);

		//! computes a background image 
		virtual void getBackgroundImage(OutputArray backgroundImage) const;
	};


	/*!
	The class implements the following algorithm:
	"Improved adaptive Gausian mixture model for background subtraction"
	Z.Zivkovic
	International Conference Pattern Recognition, UK, August, 2004.
	http://www.zoranz.net/Publications/zivkovic2004ICPR.pdf
	*/
	class CV_EXPORTS BackgroundSubtractorMOG2_Z : public BackgroundSubtractor
	{
	public:
		//! the default constructor 
		BackgroundSubtractorMOG2_Z();
		//! the full constructor that takes the length of the history, the number of gaussian mixtures, the background ratio parameter and the noise strength 
		BackgroundSubtractorMOG2_Z(int history, float varThreshold, bool bShadowDetection = true);
		//! the destructor 
		virtual ~BackgroundSubtractorMOG2_Z();
		//! the update operator 
		virtual void operator()(InputArray image, OutputArray fgmask, double learningRate = -1);

		//! computes a background image which are the mean of all background gaussians 
		virtual void getBackgroundImage(OutputArray backgroundImage) const;

		//! re-initiaization method 
		virtual void initialize(Size frameSize, int frameType);

		// virtual AlgorithmInfo* info() const; 

	protected:
		Size frameSize;
		int frameType;
		Mat bgmodel;
		Mat bgmodelUsedModes;//keep track of number of modes per pixel 
		int nframes;
		int history;
		int nmixtures;
		//! here it is the maximum allowed number of mixture components. 
		//! Actual number is determined dynamically per pixel 
		double varThreshold;
		// threshold on the squared Mahalanobis distance to decide if it is well described 
		// by the background model or not. Related to Cthr from the paper. 
		// This does not influence the update of the background. A typical value could be 4 sigma 
		// and that is varThreshold=4*4=16; Corresponds to Tb in the paper. 

		/ 
		// less important parameters - things you might change but be carefull 
		 
		float backgroundRatio;
		// corresponds to fTB=1-cf from the paper 
		// TB - threshold when the component becomes significant enough to be included into 
		// the background model. It is the TB=1-cf from the paper. So I use cf=0.1 => TB=0. 
		// For alpha=0.001 it means that the mode should exist for approximately 105 frames before 
		// it is considered foreground 
		// float noiseSigma; 
		float varThresholdGen;
		//correspondts to Tg - threshold on the squared Mahalan. dist. to decide 
		//when a sample is close to the existing components. If it is not close 
		//to any a new component will be generated. I use 3 sigma => Tg=3*3=9. 
		//Smaller Tg leads to more generated components and higher Tg might make 
		//lead to small number of components but they can grow too large 
		float fVarInit;
		float fVarMin;
		float fVarMax;
		//initial variance  for the newly generated components. 
		//It will will influence the speed of adaptation. A good guess should be made. 
		//A simple way is to estimate the typical standard deviation from the images. 
		//I used here 10 as a reasonable value 
		// min and max can be used to further control the variance 
		float fCT;//CT - complexity reduction prior 
		//this is related to the number of samples needed to accept that a component 
		//actually exists. We use CT=0.05 of all the samples. By setting CT=0 you get 
		//the standard Stauffer&Grimson algorithm (maybe not exact but very similar) 

		//shadow detection parameters 
		bool bShadowDetection;//default 1 - do shadow detection 
		unsigned char nShadowDetection;//do shadow detection - insert this value as the detection result - 127 default value 
		float fTau;
		// Tau - shadow threshold. The shadow is detected if the pixel is darker 
		//version of the background. Tau is a threshold on how much darker the shadow can be. 
		//Tau= 0.5 means that if pixel is more than 2 times darker then it is not shadow 
		//See: Prati,Mikic,Trivedi,Cucchiarra,"Detecting Moving Shadows...",IEEE PAMI,2003. 
	};
}
#endif

mog2.cpp


//#include "precomp.hpp"   


#include "mog2.h"   

namespace ZWQ
{

	/*
	Interface of Gaussian mixture algorithm from:

	"Improved adaptive Gausian mixture model for background subtraction"
	Z.Zivkovic
	International Conference Pattern Recognition, UK, August, 2004
	http://www.zoranz.net/Publications/zivkovic2004ICPR.pdf

	Advantages:
	-fast - number of Gausssian components is constantly adapted per pixel.
	-performs also shadow detection (see bgfg_segm_test.cpp example)

	*/

	// default parameters of gaussian background detection algorithm   
	static const int defaultHistory2 = 500; // Learning rate; alpha = 1/defaultHistory2   
	static const float defaultVarThreshold2 = 4.0f*4.0f;
	static const int defaultNMixtures2 = 5; // maximal number of Gaussians in mixture   
	static const float defaultBackgroundRatio2 = 0.9f; // threshold sum of weights for background test   
	static const float defaultVarThresholdGen2 = 3.0f*3.0f;
	static const float defaultVarInit2 = 15.0f; // initial variance for new components   
	static const float defaultVarMax2 = 5 * defaultVarInit2;
	static const float defaultVarMin2 = 4.0f;

	// additional parameters   
	static const float defaultfCT2 = 0.05f; // complexity reduction prior constant 0 - no reduction of number of components   
	static const unsigned char defaultnShadowDetection2 = (unsigned char)0; // value to use in the segmentation mask for shadows, set 0 not to do shadow detection   
	static const float defaultfTau = 0.5f; // Tau - shadow threshold, see the paper for explanation   

	struct GaussBGStatModel2Params
	{
		//image info   
		int nWidth;
		int nHeight;
		int nND;//number of data dimensions (image channels)   

		bool bPostFiltering;//defult 1 - do postfiltering - will make shadow detection results also give value 255   
		double  minArea; // for postfiltering   

		bool bInit;//default 1, faster updates at start   

		/   
		//very important parameters - things you will change   
		   
		float fAlphaT;
		//alpha - speed of update - if the time interval you want to average over is T   
		//set alpha=1/T. It is also usefull at start to make T slowly increase   
		//from 1 until the desired T   
		float fTb;
		//Tb - threshold on the squared Mahalan. dist. to decide if it is well described   
		//by the background model or not. Related to Cthr from the paper.   
		//This does not influence the update of the background. A typical value could be 4 sigma   
		//and that is Tb=4*4=16;   

		/   
		//less important parameters - things you might change but be carefull   
		   
		float fTg;
		//Tg - threshold on the squared Mahalan. dist. to decide   
		//when a sample is close to the existing components. If it is not close   
		//to any a new component will be generated. I use 3 sigma => Tg=3*3=9.   
		//Smaller Tg leads to more generated components and higher Tg might make   
		//lead to small number of components but they can grow too large   
		float fTB;//1-cf from the paper   
		//TB - threshold when the component becomes significant enough to be included into   
		//the background model. It is the TB=1-cf from the paper. So I use cf=0.1 => TB=0.   
		//For alpha=0.001 it means that the mode should exist for approximately 105 frames before   
		//it is considered foreground   
		float fVarInit;
		float fVarMax;
		float fVarMin;
		//initial standard deviation  for the newly generated components.   
		//It will will influence the speed of adaptation. A good guess should be made.   
		//A simple way is to estimate the typical standard deviation from the images.   
		//I used here 10 as a reasonable value   
		float fCT;//CT - complexity reduction prior   
		//this is related to the number of samples needed to accept that a component   
		//actually exists. We use CT=0.05 of all the samples. By setting CT=0 you get   
		//the standard Stauffer&Grimson algorithm (maybe not exact but very similar)   

		//even less important parameters   
		int nM;//max number of modes - const - 4 is usually enough   

		//shadow detection parameters   
		bool bShadowDetection;//default 1 - do shadow detection   
		unsigned char nShadowDetection;//do shadow detection - insert this value as the detection result   
		float fTau;
		// Tau - shadow threshold. The shadow is detected if the pixel is darker   
		//version of the background. Tau is a threshold on how much darker the shadow can be.   
		//Tau= 0.5 means that if pixel is more than 2 times darker then it is not shadow   
		//See: Prati,Mikic,Trivedi,Cucchiarra,"Detecting Moving Shadows...",IEEE PAMI,2003.   
	};

	struct GMM
	{
		float weight;
		float variance;
	};

	// shadow detection performed per pixel   
	// should work for rgb data, could be usefull for gray scale and depth data as well   
	// See: Prati,Mikic,Trivedi,Cucchiarra,"Detecting Moving Shadows...",IEEE PAMI,2003.   
	static CV_INLINE bool
		detectShadowGMM(const float* data, int nchannels, int nmodes,
		const GMM* gmm, const float* mean,
		float Tb, float TB, float tau)
	{
			float tWeight = 0;

			// check all the components  marked as background:   
			for (int mode = 0; mode < nmodes; mode++, mean += nchannels)
			{
				GMM g = gmm[mode];

				float numerator = 0.0f;
				float denominator = 0.0f;
				for (int c = 0; c < nchannels; c++)
				{
					numerator += data[c] * mean[c];
					denominator += mean[c] * mean[c];
				}

				// no division by zero allowed   
				if (denominator == 0)
					return false;

				// if tau < a < 1 then also check the color distortion   
				if (numerator <= denominator && numerator >= tau*denominator)
				{
					float a = numerator / denominator;
					float dist2a = 0.0f;

					for (int c = 0; c < nchannels; c++)
					{
						float dD = a*mean[c] - data[c];
						dist2a += dD*dD;
					}

					if (dist2a < Tb*g.variance*a*a)
						return true;
				};

				tWeight += g.weight;
				if (tWeight > TB)
					return false;
			};
			return false;
		}

	//update GMM - the base update function performed per pixel   
	//   
	//"Efficient Adaptive Density Estimapion per Image Pixel for the Task of Background Subtraction"   
	//Z.Zivkovic, F. van der Heijden   
	//Pattern Recognition Letters, vol. 27, no. 7, pages 773-780, 2006.   
	//   
	//The algorithm similar to the standard Stauffer&Grimson algorithm with   
	//additional selection of the number of the Gaussian components based on:   
	//   
	//"Recursive unsupervised learning of finite mixture models "   
	//Z.Zivkovic, F.van der Heijden   
	//IEEE Trans. on Pattern Analysis and Machine Intelligence, vol.26, no.5, pages 651-656, 2004   
	//http://www.zoranz.net/Publications/zivkovic2004PAMI.pdf   

	struct MOG2Invoker
	{
		MOG2Invoker(const Mat& _src, Mat& _dst,
		GMM* _gmm, float* _mean,
		uchar* _modesUsed,
		int _nmixtures, float _alphaT,
		float _Tb, float _TB, float _Tg,
		float _varInit, float _varMin, float _varMax,
		float _prune, float _tau, bool _detectShadows,
		uchar _shadowVal)
		{
			src = &_src;//原始图像
			dst = &_dst;//前景图像
			gmm0 = _gmm;//背景模型换为1维数组
			mean0 = _mean;//背景模型的数据的尾指针
			modesUsed0 = _modesUsed;//已经使用了的模型的数据指针
			nmixtures = _nmixtures;//模型个数
			alphaT = _alphaT;//学习率
			Tb = _Tb;//方差阈值
			TB = _TB;//背景比例
			Tg = _Tg;//方差阈值
			varInit = _varInit;//初始化方差
			varMin = MIN(_varMin, _varMax);//最小方差4
			varMax = MAX(_varMin, _varMax);//最大方差75
			prune = _prune;//-学习率 * CT
			tau = _tau;
			detectShadows = _detectShadows;
			shadowVal = _shadowVal;
			shadowVal = false;

			cvtfunc = src->depth() != CV_32F ? getConvertFunc(src->depth(), CV_32F) : 0;
		}


		/*void operator()(const Range& range) const*/

		void operator()(const BlockedRange& range) const
		{

			int y0 = range.begin(), y1 = range.end();
			int ncols = src->cols, nchannels = src->channels();
			AutoBuffer<float> buf(src->cols*nchannels);//每一行的缓存，定义变量并分配空间
			float alpha1 = 1.f - alphaT;
			float dData[CV_CN_MAX];

			for (int y = y0; y < y1; y++)
			{
				const float* data = buf;
				if (cvtfunc)//data是buf的指针，buf变量赋值
					cvtfunc(src->ptr(y), src->step, 0, 0, (uchar*)data, 0, Size(ncols*nchannels, 1), 0);//把src的每一行取出来给Data
				else
					data = src->ptr<float>(y);

				float* mean = mean0 + ncols*nmixtures*nchannels*y;//背景模型初始指针mean0, mean是尾指针；乘以y后就变成了尾指针。下一行均值所在位置的指针。
				GMM* gmm = gmm0 + ncols*nmixtures*y;//头指针，每行的模型的第一个指针
				uchar* modesUsed = modesUsed0 + ncols*y;//已经用了几个模型了
				uchar* mask = dst->ptr(y);//获取行指针
				//data是每一行的像素值，gmm是背景模型的指针，每一行的每一个；mean是每一个像素的
				for (int x = 0; x < ncols; x++, data += nchannels, gmm += nmixtures, mean += nmixtures*nchannels)
				{
					//calculate distances to the modes (+ sort)  
					//here we need to go in descending order!!!  //降序排列
					bool background = false;//return value -> true - the pixel classified as background  

					//internal:  fitsPDF如果当前像素不属于任何一个模型，就重新建一个模型
					bool fitsPDF = false;//if it remains zero a new GMM mode will be added  
					int nmodes = modesUsed[x], nNewModes = nmodes;//current number of modes in GMM  
					float totalWeight = 0.f;

					float* mean_m = mean;

					//  
					//go through all modes  
					for (int mode = 0; mode < nmodes; mode++, mean_m += nchannels)
					{
						float weight = alpha1*gmm[mode].weight + prune;//need only weight if fit is found  
						int swap_count = 0;
						  
						//fit not found yet  
						if (!fitsPDF)
						{
							//check if it belongs to some of the remaining modes  
							float var = gmm[mode].variance;

							//calculate difference and distance  
							float dist2;

							if (nchannels == 3)
							{
								dData[0] = mean_m[0] - data[0];
								dData[1] = mean_m[1] - data[1];
								dData[2] = mean_m[2] - data[2];
								dist2 = dData[0] * dData[0] + dData[1] * dData[1] + dData[2] * dData[2];
							}
							else
							{
								dist2 = 0.f;
								for (int c = 0; c < nchannels; c++)
								{
									dData[c] = mean_m[c] - data[c];
									dist2 += dData[c] * dData[c];
								}
							}

							//background? - Tb - usually larger than Tg  
							if (totalWeight < TB && dist2 < Tb*var)
								background = true;

							//check fit  
							if (dist2 < Tg*var)
							{
								/  
								//belongs to the mode  
								fitsPDF = true;

								//update distribution  

								//update weight  
								weight += alphaT;
								float k = alphaT / weight;

								//update mean  
								for (int c = 0; c < nchannels; c++)
									mean_m[c] -= k*dData[c];

								//update variance  
								float varnew = var + k*(dist2 - var);
								//limit the variance  
								varnew = MAX(varnew, varMin);
								varnew = MIN(varnew, varMax);
								gmm[mode].variance = varnew;

								//sort  
								//all other weights are at the same place and  
								//only the matched (iModes) is higher -> just find the new place for it  
								for (int i = mode; i > 0; i--)
								{
									//check one up  
									if (weight < gmm[i - 1].weight)
										break;

									swap_count++;
									//swap one up  
									std::swap(gmm[i], gmm[i - 1]);
									for (int c = 0; c < nchannels; c++)
										std::swap(mean[i*nchannels + c], mean[(i - 1)*nchannels + c]);
								}
								//belongs to the mode - bFitsPDF becomes 1  
								/  
							}
						}//!bFitsPDF)  

						//check prune  
						if (weight < -prune)
						{
							weight = 0.0;
							nmodes--;
						}

						gmm[mode - swap_count].weight = weight;//update weight by the calculated value  
						totalWeight += weight;
					}
					//go through all modes  
					//  

					//renormalize weights  
					totalWeight = 1.f / totalWeight;
					for (int mode = 0; mode < nmodes; mode++)
					{
						gmm[mode].weight *= totalWeight;
					}

					nmodes = nNewModes;

					//make new mode if needed and exit  
					if (!fitsPDF)
					{
						// replace the weakest or add a new one  
						int mode = nmodes == nmixtures ? nmixtures - 1 : nmodes++;

						if (nmodes == 1)
							gmm[mode].weight = 1.f;
						else
						{
							gmm[mode].weight = alphaT;

							// renormalize all other weights  
							for (int i = 0; i < nmodes - 1; i++)
								gmm[i].weight *= alpha1;
						}

						// init  
						for (int c = 0; c < nchannels; c++)
							mean[mode*nchannels + c] = data[c];

						gmm[mode].variance = varInit;

						//sort  
						//find the new place for it  
						for (int i = nmodes - 1; i > 0; i--)
						{
							// check one up  
							if (alphaT < gmm[i - 1].weight)
								break;

							// swap one up  
							std::swap(gmm[i], gmm[i - 1]);
							for (int c = 0; c < nchannels; c++)
								std::swap(mean[i*nchannels + c], mean[(i - 1)*nchannels + c]);
						}
					}

					//set the number of modes  
					modesUsed[x] = uchar(nmodes);
					mask[x] = background ? 0 :
						detectShadows && detectShadowGMM(data, nchannels, nmodes, gmm, mean, Tb, TB, tau) ?
					shadowVal : 255;
				}
			}
		}


		const Mat* src;
		Mat* dst;
		GMM* gmm0;
		float* mean0;
		uchar* modesUsed0;

		int nmixtures;
		float alphaT, Tb, TB, Tg;
		float varInit, varMin, varMax, prune, tau;

		bool detectShadows;
		uchar shadowVal;

		BinaryFunc cvtfunc;
	};

	BackgroundSubtractor::~BackgroundSubtractor()
	{
	}

	void BackgroundSubtractor::operator()(InputArray image, OutputArray fgmask,
		double learningRate)
	{
	}

	void BackgroundSubtractor::getBackgroundImage(OutputArray backgroundImage) const
	{
	}

	BackgroundSubtractorMOG2_Z::BackgroundSubtractorMOG2_Z()
	{
		frameSize = Size(0, 0);
		frameType = 0;

		nframes = 0;
		history = defaultHistory2;
		varThreshold = defaultVarThreshold2;
		bShadowDetection = 1;

		nmixtures = defaultNMixtures2;
		backgroundRatio = defaultBackgroundRatio2;
		fVarInit = defaultVarInit2;
		fVarMax = defaultVarMax2;
		fVarMin = defaultVarMin2;

		varThresholdGen = defaultVarThresholdGen2;
		fCT = defaultfCT2;
		nShadowDetection = defaultnShadowDetection2;
		fTau = defaultfTau;
	}

	BackgroundSubtractorMOG2_Z::BackgroundSubtractorMOG2_Z(int _history, float _varThreshold, bool _bShadowDetection)
	{
		frameSize = Size(0, 0);
		frameType = 0;

		nframes = 0;
		history = _history > 0 ? _history : defaultHistory2;
		varThreshold = (_varThreshold>0) ? _varThreshold : defaultVarThreshold2;
		bShadowDetection = _bShadowDetection;

		nmixtures = defaultNMixtures2;
		backgroundRatio = defaultBackgroundRatio2;
		fVarInit = defaultVarInit2;
		fVarMax = defaultVarMax2;
		fVarMin = defaultVarMin2;

		varThresholdGen = defaultVarThresholdGen2;
		fCT = defaultfCT2;
		nShadowDetection = defaultnShadowDetection2;
		fTau = defaultfTau;
	}

	BackgroundSubtractorMOG2_Z::~BackgroundSubtractorMOG2_Z()
	{
	}


	void BackgroundSubtractorMOG2_Z::initialize(Size _frameSize, int _frameType)
	{
		frameSize = _frameSize;
		frameType = _frameType;
		nframes = 0;

		int nchannels = CV_MAT_CN(frameType);
		CV_Assert(nchannels <= CV_CN_MAX);

		// for each gaussian mixture of each pixel bg model we store ...   
		// the mixture weight (w),   
		// the mean (nchannels values) and   
		// the covariance   
		bgmodel.create(1, frameSize.height*frameSize.width*nmixtures*(2 + nchannels), CV_32F);
		//make the array for keeping track of the used modes per pixel - all zeros at start   
		bgmodelUsedModes.create(frameSize, CV_8U);
		bgmodelUsedModes = Scalar::all(0);
	}

	void BackgroundSubtractorMOG2_Z::operator()(InputArray _image, OutputArray _fgmask, double learningRate)
	{
		Mat image = _image.getMat();//原图
		bool needToInitialize = nframes == 0 || learningRate >= 1 || image.size() != frameSize || image.type() != frameType;

		if (needToInitialize)
			initialize(image.size(), image.type());

		_fgmask.create(image.size(), CV_8U);
		Mat fgmask = _fgmask.getMat();//随机分配的值

		++nframes;
		learningRate = learningRate >= 0 && nframes > 1 ? learningRate : 1. / min(2 * nframes, history);
		CV_Assert(learningRate >= 0);

		parallel_for(BlockedRange(0, image.rows),
			MOG2Invoker(image, fgmask,
			(GMM*)bgmodel.data,//bgmodel是Mat类型，分配的是（1，rows*cols*5*(2+1))的内存，然后将其data换为GMM格式
			(float*)(bgmodel.data + sizeof(GMM)*nmixtures*image.rows*image.cols),//三维数据，每一个数据都是GMM类型的。
			bgmodelUsedModes.data, nmixtures, (float)learningRate,
			(float)varThreshold,
			backgroundRatio, varThresholdGen,
			fVarInit, fVarMin, fVarMax, float(-learningRate*fCT), fTau,
			bShadowDetection, nShadowDetection));
	}

	void BackgroundSubtractorMOG2_Z::getBackgroundImage(OutputArray backgroundImage) const
	{
		int nchannels = CV_MAT_CN(frameType);
		CV_Assert(nchannels == 3);
		Mat meanBackground(frameSize, CV_8UC3, Scalar::all(0));

		int firstGaussianIdx = 0;
		const GMM* gmm = (GMM*)bgmodel.data;
		const Vec3f* mean = reinterpret_cast<const Vec3f*>(gmm + frameSize.width*frameSize.height*nmixtures);
		for (int row = 0; row<meanBackground.rows; row++)
		{
			for (int col = 0; col<meanBackground.cols; col++)
			{
				int nmodes = bgmodelUsedModes.at<uchar>(row, col);
				Vec3f meanVal;
				float totalWeight = 0.f;
				for (int gaussianIdx = firstGaussianIdx; gaussianIdx < firstGaussianIdx + nmodes; gaussianIdx++)
				{
					GMM gaussian = gmm[gaussianIdx];
					meanVal += gaussian.weight * mean[gaussianIdx];
					totalWeight += gaussian.weight;

					if (totalWeight > backgroundRatio)
						break;
				}

				meanVal *= (1.f / totalWeight);
				meanBackground.at<Vec3b>(row, col) = Vec3b(meanVal);
				firstGaussianIdx += nmixtures;
			}
		}

		switch (CV_MAT_CN(frameType))
		{
		case 1:
		{
				  vector<Mat> channels;
				  split(meanBackground, channels);
				  channels[0].copyTo(backgroundImage);
				  break;
		}

		case 3:
		{
				  meanBackground.copyTo(backgroundImage);
				  break;
		}

		default:
			CV_Error(CV_StsUnsupportedFormat, "");
		}
	}


}

/* End of file. */