转自:http://www.cnblogs.com/xrwang/archive/2010/02/21/ForegroundDetection.html
作者:王先荣
前言
在很多情况下,我们需要从一段视频或者一系列图片中找到感兴趣的目标,比如说当人进入已经打烊的超市时发出警报。为了达到这个目的,我们首先需要“学习”背景模型,然后将背景模型和当前图像进行比较,从而得到前景目标。
背景建模
背景与前景都是相对的概念,以高速公路为例:有时我们对高速公路上来来往往的汽车感兴趣,这时汽车是前景,而路面以及周围的环境是背景;有时我们仅仅对闯入高速公路的行人感兴趣,这时闯入者是前景,而包括汽车之类的其他东西又成了背景。背景建模的方式很多,或高级或简单。不过各种背景模型都有自己适用的场合,即使是高级的背景模型也不能适用于任何场合。下面我将逐一介绍OpenCv中已经实现,或者在《学习OpenCv》这本书中介绍的背景建模方法。
1.帧差
帧差可说是最简单的一种背景模型,指定视频中的一幅图像为背景,用当前帧与背景进行比较,根据需要过滤较小的差异,得到的结果就是前景了。OpenCv中为我们提供了一种动态计算阀值,然后用帧差进行前景检测的函数——cvChangeDetection(注:EmguCv中没有封装cvChangeDetection,我将其声明到OpenCvInvoke类中,具体实现见文末代码)。而通过对两幅图像使用减法运算,然后再用指定阀值过滤的方法在《学习OpenCv》一书中有详细的介绍。它们的实现代码如下:
帧差:
- [DllImport("cvaux200.dll")]
- public static extern void cvChangeDetection(IntPtr prev_frame, IntPtr curr_frame, IntPtr change_mask);
- //backgroundMask为背景,imageBackgroundModel为背景模型,currentFrame为当前帧
- if (backgroundMask == null)
- backgroundMask = new Image<Gray, byte>(imageBackgroundModel.Size);
- if (threshold == 0d)
- //如果阀值为0,使用OpenCv中的自适应动态背景检测
- OpenCvInvoke.cvChangeDetection(imageBackgroundModel.Ptr, currentFrame.Ptr, backgroundMask.Ptr);
- else
- {
- //如果设置了阀值,使用帧差
- Image<TColor, Byte> imageTemp = imageBackgroundModel.AbsDiff(currentFrame);
- Image<Gray, Byte>[] images = imageTemp.Split();
- backgroundMask.SetValue(0d);
- foreach (Image<Gray, Byte> image in images)
- backgroundMask._Or(image.ThresholdBinary(new Gray(threshold), new Gray(255d)));
- }
- backgroundMask._Not();
对于类似无人值守的仓库防盗之类的场合,使用帧差效果估计很好。
2.背景统计模型
背景统计模型是:对一段时间的背景进行统计,然后计算其统计数据(例如平均值、平均差分、标准差、均值漂移值等等),将统计数据作为背景的方法。OpenCv中并未实现简单的背景统计模型,不过在《学习OpenCv》中对其中的平均背景统计模型有很详细的介绍。在模仿该算法的基础上,我实现了一系列的背景统计模型,包括:平均背景、均值漂移、标准差和标准协方差。对这些统计概念我其实不明白,在维基百科上看了好半天 -_-
调用背景统计模型很简单,只需4步而已:
- //(1)初始化对象
- BackgroundStatModelBase<Bgr> bgModel = new BackgroundStatModelBase<Bgr>(BackgroundStatModelType.AccAvg);
- //(2)更新一段时间的背景图像,视情况反复调用(2)
- bgModel.Update(image);
- //(3)设置当前帧
- bgModel.CurrentFrame = currentFrame;
- //(4)得到背景或者前景
- Image<Gray,Byte> imageForeground = bgModel.ForegroundMask;
背景统计模型的实现代码如下:
- /*
- 背景统计模型
- 作者:王先荣
- 时间:2010年2月19日
- */
- using System;
- using System.Collections.Generic;
- using System.Linq;
- using System.Text;
- using System.Drawing;
- using System.Diagnostics;
- using System.Runtime.InteropServices;
- using Emgu.CV;
- using Emgu.CV.CvEnum;
- using Emgu.CV.Structure;
- using Emgu.CV.UI;
- using Emgu.CV.VideoSurveillance;
- namespace ImageProcessLearn
- {
- //背景模型接口,在IBGFGDetector接口的基础上增加了一个CurrentFrame属性
- public interface IBackgroundStatModel<TColor> : IDisposable
- where TColor : struct, IColor
- {
- /// <summary>
- /// 获取前景
- /// </summary>
- Image<Gray, byte> BackgroundMask { get; }
- /// <summary>
- /// 获取背景
- /// </summary>
- Image<Gray, byte> ForegroundMask { get; }
- /// <summary>
- /// 更新背景模型
- /// </summary>
- /// <param name="image"></param>
- void Update(Image<TColor, byte> image);
- /// <summary>
- /// 计算统计数据
- /// </summary>
- void CalcStatData();
- /// <summary>
- /// 获取或者设置当前帧
- /// </summary>
- Image<TColor, Byte> CurrentFrame
- {
- get;
- set;
- }
- }
- /// <summary>
- /// 使用帧差的方式来建立背景模型
- /// </summary>
- /// <typeparam name="TColor"></typeparam>
- public class BackgroundStatModelFrameDiff<TColor> : IBackgroundStatModel<TColor>
- where TColor : struct, IColor
- {
- //成员
- private Image<TColor, Byte> imageBackgroundModel; //背景模型图像
- private Image<TColor, Byte> currentFrame; //当前帧
- private double threshold; //计算前景时所用的阀值,如果当前帧和背景的差别大于阀值,则被认为是前景
- private Image<Gray, Byte> backgroundMask; //计算得到的背景图像
- /// <summary>
- /// 构造函数
- /// </summary>
- /// <param name="image">用于背景统计模型的背景</param>
- public BackgroundStatModelFrameDiff(Image<TColor, Byte> image)
- {
- imageBackgroundModel = image;
- currentFrame = null;
- threshold = 15d;
- backgroundMask = null;
- }
- public BackgroundStatModelFrameDiff()
- : this(null)
- {
- }
- /// <summary>
- /// 设置或者获取计算前景时所用的阀值;如果阀值为0,则使用自适应的阀值
- /// </summary>
- public double Threshold
- {
- get
- {
- return threshold;
- }
- set
- {
- threshold = value >= 0 ? value : 15d;
- }
- }
- /// <summary>
- /// 更新背景模型
- /// </summary>
- /// <param name="image"></param>
- public void Update(Image<TColor, Byte> image)
- {
- imageBackgroundModel = image;
- }
- /// <summary>
- /// 获取或者设置当前帧
- /// </summary>
- public Image<TColor, Byte> CurrentFrame
- {
- get
- {
- return currentFrame;
- }
- set
- {
- currentFrame = value;
- CalcBackgroundMask();
- }
- }
- /// <summary>
- /// 计算统计数据
- /// </summary>
- public void CalcStatData()
- {
- }
- /// <summary>
- /// 计算背景
- /// </summary>
- private void CalcBackgroundMask()
- {
- if (imageBackgroundModel == null || currentFrame == null || imageBackgroundModel.Size != currentFrame.Size)
- throw new ArgumentException("在计算背景时发现参数错误。可能是:背景模型图像为空,当前帧为空,或者背景模型图像和当前帧的尺寸不一致。");
- if (backgroundMask == null)
- backgroundMask = new Image<Gray, byte>(imageBackgroundModel.Size);
- if (threshold == 0d)
- //如果阀值为0,使用OpenCv中的自适应动态背景检测
- OpenCvInvoke.cvChangeDetection(imageBackgroundModel.Ptr, currentFrame.Ptr, backgroundMask.Ptr);
- else
- {
- //如果设置了阀值,使用帧差
- Image<TColor, Byte> imageTemp = imageBackgroundModel.AbsDiff(currentFrame);
- Image<Gray, Byte>[] images = imageTemp.Split();
- backgroundMask.SetValue(0d);
- foreach (Image<Gray, Byte> image in images)
- backgroundMask._Or(image.ThresholdBinary(new Gray(threshold), new Gray(255d)));
- }
- backgroundMask._Not();
- }
- /// <summary>
- /// 获取背景
- /// </summary>
- public Image<Gray, Byte> BackgroundMask
- {
- get
- {
- return backgroundMask;
- }
- }
- /// <summary>
- /// 获取前景
- /// </summary>
- public Image<Gray, Byte> ForegroundMask
- {
- get
- {
- return backgroundMask.Not();
- }
- }
- /// <summary>
- /// 释放资源
- /// </summary>
- public void Dispose()
- {
- if (backgroundMask != null)
- backgroundMask.Dispose();
- }
- }
- /// <summary>
- /// 使用平均背景来建立背景模型
- /// </summary>
- /// <typeparam name="TColor"></typeparam>
- public class BackgroundStatModelAccAvg<TColor> : IBackgroundStatModel<TColor>
- where TColor : struct, IColor
- {
- //成员
- private Image<TColor, Single> imageAccSum; //累计图像
- private Image<TColor, Single> imageAccDiff; //累计差值图像
- private int frameCount; //已经累计的背景帧数
- private Image<TColor, Single> previousFrame; //在背景建模时使用的前一帧图像
- private Image<TColor, Byte> currentFrame; //当前帧图像
- private double scale; //计算背景时所使用的缩放系数,大于平均值*scale倍数的像素认为是前景
- private Image<Gray, Byte> backgroundMask; //计算得到的背景图像
- private Image<TColor, Single> imageTemp; //临时图像
- private bool isStatDataReady; //是否已经准备好统计数据
- private Image<Gray, Single>[] imagesHi; //背景模型中各通道的最大值图像
- private Image<Gray, Single>[] imagesLow; //背景模型中各通道的最小值图像
- /// <summary>
- /// 构造函数
- /// </summary>
- public BackgroundStatModelAccAvg()
- {
- imageAccSum = null;
- imageAccDiff = null;
- frameCount = 0;
- previousFrame = null;
- currentFrame = null;
- scale = 6d;
- backgroundMask = null;
- isStatDataReady = false;
- imagesHi = null;
- imagesLow = null;
- }
- /// <summary>
- /// 设置或者获取计算前景时所用的阀值
- /// </summary>
- public double Scale
- {
- get
- {
- return scale;
- }
- set
- {
- scale = value > 0 ? value : 6d;
- }
- }
- /// <summary>
- /// 更新背景模型
- /// </summary>
- /// <param name="image"></param>
- public void Update(Image<TColor, Byte> image)
- {
- if (frameCount==0)
- {
- imageAccSum = new Image<TColor, Single>(image.Size);
- imageAccSum.SetValue(0d);
- imageAccDiff = new Image<TColor, float>(image.Size);
- imageAccDiff.SetValue(0d);
- }
- imageTemp = image.ConvertScale<Single>(1d, 0d); //将图像转换成浮点型
- imageAccSum.Acc(imageTemp);
- if (previousFrame != null)
- imageAccDiff.Acc(imageTemp.AbsDiff(previousFrame));
- previousFrame = imageTemp.Copy();
- frameCount++;
- }
- /// <summary>
- /// 获取或者设置当前帧
- /// </summary>
- public Image<TColor, Byte> CurrentFrame
- {
- get
- {
- return currentFrame;
- }
- set
- {
- currentFrame = value;
- CalcBackgroundMask();
- }
- }
- /// <summary>
- /// 计算统计数据
- /// </summary>
- public void CalcStatData()
- {
- //计算出最高及最低阀值图像
- Image<TColor, Single> imageAvg = imageAccSum.ConvertScale<Single>(1d / frameCount, 0d);
- Image<TColor, Single> imageAvgDiff = imageAccDiff.ConvertScale<Single>(1d / frameCount, 1d); //将平均值加1,为了确保总是存在差异
- Image<TColor, Single> imageHi = imageAvg.Add(imageAvgDiff.ConvertScale<Single>(scale, 0d));
- Image<TColor, Single> imageLow = imageAvg.Sub(imageAvgDiff.ConvertScale<Single>(scale, 0d));
- imagesHi = imageHi.Split();
- imagesLow = imageLow.Split();
- isStatDataReady = true;
- //释放资源
- imageAvg.Dispose();
- imageAvgDiff.Dispose();
- imageHi.Dispose();
- imageLow.Dispose();
- }
- /// <summary>
- /// 计算背景
- /// </summary>
- private void CalcBackgroundMask()
- {
- if (imageAccSum == null || imageAccDiff == null || imageAccSum.Size != currentFrame.Size)
- throw new ArgumentException("在计算背景时发生参数错误。可能是:还没有建立背景模型;或者当前帧的尺寸与背景尺寸不一致。");
- if (!isStatDataReady)
- CalcStatData();
- imageTemp = currentFrame.ConvertScale<Single>(1d, 0d);
- Image<Gray, Single>[] images = imageTemp.Split();
- //计算背景图像
- if (backgroundMask == null)
- backgroundMask = new Image<Gray, byte>(currentFrame.Size);
- backgroundMask.SetZero();
- for (int i = 0; i < currentFrame.NumberOfChannels; i++)
- backgroundMask._Or(images[i].InRange(imagesLow[i], imagesHi[i]));
- //释放资源
- for (int i = 0; i < images.Length; i++)
- images[i].Dispose();
- }
- /// <summary>
- /// 获取背景
- /// </summary>
- public Image<Gray, Byte> BackgroundMask
- {
- get
- {
- return backgroundMask;
- }
- }
- /// <summary>
- /// 获取前景
- /// </summary>
- public Image<Gray, Byte> ForegroundMask
- {
- get
- {
- return backgroundMask.Not();
- }
- }
- /// <summary>
- /// 释放资源
- /// </summary>
- public void Dispose()
- {
- if (imageAccSum != null)
- imageAccSum.Dispose();
- if (imageAccDiff != null)
- imageAccDiff.Dispose();
- if (previousFrame != null)
- previousFrame.Dispose();
- if (currentFrame != null)
- currentFrame.Dispose();
- if (backgroundMask != null)
- backgroundMask.Dispose();
- if (isStatDataReady)
- {
- for (int i = 0; i < imagesHi.Length; i++)
- {
- imagesHi[i].Dispose();
- imagesLow[i].Dispose();
- }
- }
- }
- }
- /// <summary>
- /// 使用均值漂移来建立背景模型
- /// </summary>
- /// <typeparam name="TColor"></typeparam>
- public class BackgroundStatModelRunningAvg<TColor> : IBackgroundStatModel<TColor>
- where TColor : struct, IColor
- {
- //成员
- private Image<TColor, Single> imageAcc; //累计图像
- private Image<TColor, Single> imageAccDiff; //累计差值图像
- private int frameCount; //已经累计的背景帧数
- private Image<TColor, Single> previousFrame; //在背景建模时使用的前一帧图像
- private Image<TColor, Byte> currentFrame; //当前帧图像
- private double scale; //计算背景时所使用的缩放系数,大于平均值*scale倍数的像素认为是前景
- private double alpha; //计算均值漂移时使用的权值
- private Image<Gray, Byte> backgroundMask; //计算得到的背景图像
- private Image<TColor, Single> imageTemp; //临时图像
- private bool isStatDataReady; //是否已经准备好统计数据
- private Image<Gray, Single>[] imagesHi; //背景模型中各通道的最大值图像
- private Image<Gray, Single>[] imagesLow; //背景模型中各通道的最小值图像
- /// <summary>
- /// 构造函数
- /// </summary>
- public BackgroundStatModelRunningAvg()
- {
- imageAcc = null;
- imageAccDiff = null;
- frameCount = 0;
- previousFrame = null;
- currentFrame = null;
- scale = 6d;
- alpha = 0.5d;
- backgroundMask = null;
- isStatDataReady = false;
- imagesHi = null;
- imagesLow = null;
- }
- /// <summary>
- /// 设置或者获取计算前景时所用的阀值
- /// </summary>
- public double Scale
- {
- get
- {
- return scale;
- }
- set
- {
- scale = value > 0 ? value : 6d;
- }
- }
- /// <summary>
- /// 设置或者获取计算均值漂移是使用的权值
- /// </summary>
- public double Alpha
- {
- get
- {
- return alpha;
- }
- set
- {
- alpha = value > 0 && value < 1 ? value : 0.5d;
- }
- }
- /// <summary>
- /// 更新背景模型
- /// </summary>
- /// <param name="image"></param>
- public void Update(Image<TColor, Byte> image)
- {
- imageTemp = image.ConvertScale<Single>(1d, 0d); //将图像转换成浮点型
- if (imageAcc == null)
- {
- imageAcc = imageTemp.Copy();
- }
- else
- imageAcc.RunningAvg(imageTemp, alpha);
- if (previousFrame != null)
- {
- if (imageAccDiff == null)
- imageAccDiff = imageTemp.AbsDiff(previousFrame);
- else
- imageAccDiff.RunningAvg(imageTemp.AbsDiff(previousFrame), alpha);
- }
- previousFrame = imageTemp.Copy();
- frameCount++;
- }
- /// <summary>
- /// 获取或者设置当前帧
- /// </summary>
- public Image<TColor, Byte> CurrentFrame
- {
- get
- {
- return currentFrame;
- }
- set
- {
- currentFrame = value;
- CalcBackgroundMask();
- }
- }
- /// <summary>
- /// 计算统计数据
- /// </summary>
- public void CalcStatData()
- {
- //计算出最高及最低阀值图像
- Image<TColor, Single> imageHi = imageAcc.Add(imageAccDiff.ConvertScale<Single>(scale, 0d));
- Image<TColor, Single> imageLow = imageAcc.Sub(imageAccDiff.ConvertScale<Single>(scale, 0d));
- imagesHi = imageHi.Split();
- imagesLow = imageLow.Split();
- isStatDataReady = true;
- //释放资源
- imageHi.Dispose();
- imageLow.Dispose();
- }
- /// <summary>
- /// 计算背景
- /// </summary>
- private void CalcBackgroundMask()
- {
- if (imageAcc == null || imageAccDiff == null || imageAcc.Size != currentFrame.Size)
- throw new ArgumentException("在计算背景时发生参数错误。可能是:还没有建立背景模型;或者当前帧的尺寸与背景尺寸不一致。");
- if (!isStatDataReady)
- CalcStatData();
- imageTemp = currentFrame.ConvertScale<Single>(1d, 0d);
- Image<Gray, Single>[] images = imageTemp.Split();
- //计算背景图像
- if (backgroundMask == null)
- backgroundMask = new Image<Gray, byte>(currentFrame.Size);
- backgroundMask.SetZero();
- for (int i = 0; i < currentFrame.NumberOfChannels; i++)
- backgroundMask._Or(images[i].InRange(imagesLow[i], imagesHi[i]));
- //释放资源
- for (int i = 0; i < images.Length; i++)
- images[i].Dispose();
- }
- /// <summary>
- /// 获取背景
- /// </summary>
- public Image<Gray, Byte> BackgroundMask
- {
- get
- {
- return backgroundMask;
- }
- }
- /// <summary>
- /// 获取前景
- /// </summary>
- public Image<Gray, Byte> ForegroundMask
- {
- get
- {
- return backgroundMask.Not();
- }
- }
- /// <summary>
- /// 释放资源
- /// </summary>
- public void Dispose()
- {
- if (imageAcc != null)
- imageAcc.Dispose();
- if (imageAccDiff != null)
- imageAccDiff.Dispose();
- if (previousFrame != null)
- previousFrame.Dispose();
- if (currentFrame != null)
- currentFrame.Dispose();
- if (backgroundMask != null)
- backgroundMask.Dispose();
- if (isStatDataReady)
- {
- for (int i = 0; i < imagesHi.Length; i++)
- {
- imagesHi[i].Dispose();
- imagesLow[i].Dispose();
- }
- }
- }
- }
- /// <summary>
- /// 使用标准方差来建立背景模型
- /// </summary>
- /// <typeparam name="TColor"></typeparam>
- public class BackgroundStatModelSquareAcc<TColor> : IBackgroundStatModel<TColor>
- where TColor : struct, IColor
- {
- //成员
- private Image<TColor, Single> imageAccSum; //累计图像
- private Image<TColor, Single> imageAccSquare; //累计平方图像
- private int frameCount; //已经累计的背景帧数
- private Image<TColor, Single> previousFrame; //在背景建模时使用的前一帧图像
- private Image<TColor, Byte> currentFrame; //当前帧图像
- private double scale; //计算背景时所使用的缩放系数,大于平均值*scale倍数的像素认为是前景
- private Image<Gray, Byte> backgroundMask; //计算得到的背景图像
- private Image<TColor, Single> imageTemp; //临时图像
- private bool isStatDataReady; //是否已经准备好统计数据
- private Image<Gray, Single>[] imagesHi; //背景模型中各通道的最大值图像
- private Image<Gray, Single>[] imagesLow; //背景模型中各通道的最小值图像
- /// <summary>
- /// 构造函数
- /// </summary>
- public BackgroundStatModelSquareAcc()
- {
- imageAccSum = null;
- imageAccSquare = null;
- frameCount = 0;
- previousFrame = null;
- currentFrame = null;
- scale = 6d;
- backgroundMask = null;
- isStatDataReady = false;
- imagesHi = null;
- imagesLow = null;
- }
- /// <summary>
- /// 设置或者获取计算前景时所用的阀值
- /// </summary>
- public double Scale
- {
- get
- {
- return scale;
- }
- set
- {
- scale = value > 0 ? value : 6d;
- }
- }
- /// <summary>
- /// 更新背景模型
- /// </summary>
- /// <param name="image"></param>
- public void Update(Image<TColor, Byte> image)
- {
- if (frameCount == 0)
- {
- imageAccSum = new Image<TColor, Single>(image.Size);
- imageAccSum.SetZero();
- imageAccSquare = new Image<TColor, float>(image.Size);
- imageAccSquare.SetZero();
- }
- imageTemp = image.ConvertScale<Single>(1d, 0d); //将图像转换成浮点型
- imageAccSum.Acc(imageTemp);
- CvInvoke.cvSquareAcc(imageTemp.Ptr, imageAccSquare.Ptr, IntPtr.Zero);
- previousFrame = imageTemp.Copy();
- frameCount++;
- }
- /// <summary>
- /// 获取或者设置当前帧
- /// </summary>
- public Image<TColor, Byte> CurrentFrame
- {
- get
- {
- return currentFrame;
- }
- set
- {
- currentFrame = value;
- CalcBackgroundMask();
- }
- }
- /// <summary>
- /// 计算统计数据
- /// </summary>
- public void CalcStatData()
- {
- //计算出标准差、最高及最低阀值图像
- Image<TColor, Single> imageAvg = imageAccSum.ConvertScale<Single>(1d / frameCount, 0d);
- Image<TColor, Single> imageSd = imageAccSquare.ConvertScale<Single>(1d / frameCount, 0d);
- imageSd.Sub(imageAvg.Pow(2d));
- imageSd = imageSd.Pow(0.5d);
- Image<TColor, Single> imageHi = imageAvg.Add(imageSd.ConvertScale<Single>(scale, 0d));
- Image<TColor, Single> imageLow = imageAvg.Sub(imageSd.ConvertScale<Single>(scale, 0d));
- imagesHi = imageHi.Split();
- imagesLow = imageLow.Split();
- isStatDataReady = true;
- //释放资源
- imageAvg.Dispose();
- imageSd.Dispose();
- imageHi.Dispose();
- imageLow.Dispose();
- }
- /// <summary>
- /// 计算背景
- /// </summary>
- private void CalcBackgroundMask()
- {
- if (imageAccSum == null || imageAccSquare == null || imageAccSum.Size != currentFrame.Size)
- throw new ArgumentException("在计算背景时发生参数错误。可能是:还没有建立背景模型;或者当前帧的尺寸与背景尺寸不一致。");
- if (!isStatDataReady)
- CalcStatData();
- imageTemp = currentFrame.ConvertScale<Single>(1d, 0d);
- Image<Gray, Single>[] images = imageTemp.Split();
- //计算背景图像
- if (backgroundMask == null)
- backgroundMask = new Image<Gray, byte>(currentFrame.Size);
- backgroundMask.SetZero();
- for (int i = 0; i < currentFrame.NumberOfChannels; i++)
- backgroundMask._Or(images[i].InRange(imagesLow[i], imagesHi[i]));
- //释放资源
- for (int i = 0; i < images.Length; i++)
- images[i].Dispose();
- }
- /// <summary>
- /// 获取背景
- /// </summary>
- public Image<Gray, Byte> BackgroundMask
- {
- get
- {
- return backgroundMask;
- }
- }
- /// <summary>
- /// 获取前景
- /// </summary>
- public Image<Gray, Byte> ForegroundMask
- {
- get
- {
- return backgroundMask.Not();
- }
- }
- /// <summary>
- /// 释放资源
- /// </summary>
- public void Dispose()
- {
- if (imageAccSum != null)
- imageAccSum.Dispose();
- if (imageAccSquare != null)
- imageAccSquare.Dispose();
- if (previousFrame != null)
- previousFrame.Dispose();
- if (currentFrame != null)
- currentFrame.Dispose();
- if (backgroundMask != null)
- backgroundMask.Dispose();
- if (isStatDataReady)
- {
- for (int i = 0; i < imagesHi.Length; i++)
- {
- imagesHi[i].Dispose();
- imagesLow[i].Dispose();
- }
- }
- }
- }
- /// <summary>
- /// 使用标准协方差来建立背景模型
- /// </summary>
- /// <typeparam name="TColor"></typeparam>
- public class BackgroundStatModelMultiplyAcc<TColor> : IBackgroundStatModel<TColor>
- where TColor : struct, IColor
- {
- //成员
- private Image<TColor, Single> imageAccSum; //累计图像
- private Image<TColor, Single> imageAccMultiply; //累计平方图像
- private int frameCount; //已经累计的背景帧数
- private Image<TColor, Single> previousFrame; //在背景建模时使用的前一帧图像
- private Image<TColor, Byte> currentFrame; //当前帧图像
- private double scale; //计算背景时所使用的缩放系数,大于平均值*scale倍数的像素认为是前景
- private Image<Gray, Byte> backgroundMask; //计算得到的背景图像
- private Image<TColor, Single> imageTemp; //临时图像
- private bool isStatDataReady; //是否已经准备好统计数据
- private Image<Gray, Single>[] imagesHi; //背景模型中各通道的最大值图像
- private Image<Gray, Single>[] imagesLow; //背景模型中各通道的最小值图像
- /// <summary>
- /// 构造函数
- /// </summary>
- public BackgroundStatModelMultiplyAcc()
- {
- imageAccSum = null;
- imageAccMultiply = null;
- frameCount = 0;
- previousFrame = null;
- currentFrame = null;
- scale = 6d;
- backgroundMask = null;
- isStatDataReady = false;
- imagesHi = null;
- imagesLow = null;
- }
- /// <summary>
- /// 设置或者获取计算前景时所用的阀值
- /// </summary>
- public double Scale
- {
- get
- {
- return scale;
- }
- set
- {
- scale = value > 0 ? value : 6d;
- }
- }
- /// <summary>
- /// 更新背景模型
- /// </summary>
- /// <param name="image"></param>
- public void Update(Image<TColor, Byte> image)
- {
- if (frameCount == 0)
- {
- imageAccSum = new Image<TColor, Single>(image.Size);
- imageAccSum.SetZero();
- imageAccMultiply = new Image<TColor, float>(image.Size);
- imageAccMultiply.SetZero();
- }
- imageTemp = image.ConvertScale<Single>(1d, 0d); //将图像转换成浮点型
- imageAccSum.Acc(imageTemp);
- if (previousFrame != null)
- CvInvoke.cvMultiplyAcc(previousFrame.Ptr, imageTemp.Ptr, imageAccMultiply.Ptr, IntPtr.Zero);
- previousFrame = imageTemp.Copy();
- frameCount++;
- }
- /// <summary>
- /// 获取或者设置当前帧
- /// </summary>
- public Image<TColor, Byte> CurrentFrame
- {
- get
- {
- return currentFrame;
- }
- set
- {
- currentFrame = value;
- CalcBackgroundMask();
- }
- }
- /// <summary>
- /// 计算统计数据
- /// </summary>
- public void CalcStatData()
- {
- //计算出标准协方差、最高及最低阀值图像
- Image<TColor, Single> imageAvg = imageAccSum.ConvertScale<Single>(1d / frameCount, 0d);
- Image<TColor, Single> imageScov = imageAccMultiply.ConvertScale<Single>(1d / frameCount, 0d);
- imageScov.Sub(imageAvg.Pow(2d));
- imageScov = imageScov.Pow(0.5d);
- Image<TColor, Single> imageHi = imageAvg.Add(imageScov.ConvertScale<Single>(scale, 0d));
- Image<TColor, Single> imageLow = imageAvg.Sub(imageScov.ConvertScale<Single>(scale, 0d));
- imagesHi = imageHi.Split();
- imagesLow = imageLow.Split();
- isStatDataReady = true;
- //释放资源
- imageAvg.Dispose();
- imageScov.Dispose();
- imageHi.Dispose();
- imageLow.Dispose();
- }
- /// <summary>
- /// 计算背景
- /// </summary>
- private void CalcBackgroundMask()
- {
- if (imageAccSum == null || imageAccMultiply == null || imageAccSum.Size != currentFrame.Size)
- throw new ArgumentException("在计算背景时发生参数错误。可能是:还没有建立背景模型;或者当前帧的尺寸与背景尺寸不一致。");
- if (!isStatDataReady)
- CalcStatData();
- imageTemp = currentFrame.ConvertScale<Single>(1d, 0d);
- Image<Gray, Single>[] images = imageTemp.Split();
- //计算背景图像
- if (backgroundMask == null)
- backgroundMask = new Image<Gray, byte>(currentFrame.Size);
- backgroundMask.SetZero();
- for (int i = 0; i < currentFrame.NumberOfChannels; i++)
- backgroundMask._Or(images[i].InRange(imagesLow[i], imagesHi[i]));
- //释放资源
- for (int i = 0; i < images.Length; i++)
- images[i].Dispose();
- }
- /// <summary>
- /// 获取背景
- /// </summary>
- public Image<Gray, Byte> BackgroundMask
- {
- get
- {
- return backgroundMask;
- }
- }
- /// <summary>
- /// 获取前景
- /// </summary>
- public Image<Gray, Byte> ForegroundMask
- {
- get
- {
- return backgroundMask.Not();
- }
- }
- /// <summary>
- /// 释放资源
- /// </summary>
- public void Dispose()
- {
- if (imageAccSum != null)
- imageAccSum.Dispose();
- if (imageAccMultiply != null)
- imageAccMultiply.Dispose();
- if (previousFrame != null)
- previousFrame.Dispose();
- if (currentFrame != null)
- currentFrame.Dispose();
- if (backgroundMask != null)
- backgroundMask.Dispose();
- if (isStatDataReady)
- {
- for (int i = 0; i < imagesHi.Length; i++)
- {
- imagesHi[i].Dispose();
- imagesLow[i].Dispose();
- }
- }
- }
- }
- /// <summary>
- /// 背景统计模型
- /// </summary>
- public class BackgroundStatModelBase<TColor> : IBackgroundStatModel<TColor>
- where TColor : struct, IColor
- {
- //成员变量
- IBackgroundStatModel<TColor> bgModel;
- BackgroundStatModelType type;
- /// <summary>
- /// 构造函数
- /// </summary>
- /// <param name="type">背景模型类型</param>
- public BackgroundStatModelBase(BackgroundStatModelType type)
- {
- this.type = type;
- switch (type)
- {
- case BackgroundStatModelType.FrameDiff:
- bgModel = new BackgroundStatModelFrameDiff<TColor>();
- break;
- case BackgroundStatModelType.AccAvg:
- bgModel = new BackgroundStatModelAccAvg<TColor>();
- break;
- case BackgroundStatModelType.RunningAvg:
- bgModel = new BackgroundStatModelRunningAvg<TColor>();
- break;
- case BackgroundStatModelType.SquareAcc:
- bgModel = new BackgroundStatModelSquareAcc<TColor>();
- break;
- case BackgroundStatModelType.MultiplyAcc:
- bgModel = new BackgroundStatModelMultiplyAcc<TColor>();
- break;
- default:
- throw new ArgumentException("不存在的背景模型", "type");
- }
- }
- /// <summary>
- /// 获取背景模型类型
- /// </summary>
- public BackgroundStatModelType BackgroundStatModelType
- {
- get
- {
- return type;
- }
- }
- /// <summary>
- /// 更新背景模型
- /// </summary>
- /// <param name="image"></param>
- public void Update(Image<TColor, Byte> image)
- {
- bgModel.Update(image);
- }
- /// <summary>
- /// 计算统计数据
- /// </summary>
- public void CalcStatData()
- {
- bgModel.CalcStatData();
- }
- /// <summary>
- /// 设置或者获取当前帧
- /// </summary>
- public Image<TColor, Byte> CurrentFrame
- {
- get
- {
- return bgModel.CurrentFrame;
- }
- set
- {
- bgModel.CurrentFrame = value;
- }
- }
- /// <summary>
- /// 获取背景
- /// </summary>
- public Image<Gray, Byte> BackgroundMask
- {
- get
- {
- return bgModel.BackgroundMask;
- }
- }
- /// <summary>
- /// 获取前景
- /// </summary>
- public Image<Gray, Byte> ForegroundMask
- {
- get
- {
- return bgModel.ForegroundMask;
- }
- }
- /// <summary>
- /// 释放资源
- /// </summary>
- public void Dispose()
- {
- if (bgModel != null)
- bgModel.Dispose();
- }
- }
- /// <summary>
- /// 背景模型类型
- /// </summary>
- public enum BackgroundStatModelType
- {
- FrameDiff, //帧差
- AccAvg, //平均背景
- RunningAvg, //均值漂移
- MultiplyAcc, //计算协方差
- SquareAcc //计算方差
- }
- }
3.编码本背景模型
编码本的基本思路是这样的:针对每个像素在时间轴上的变动,建立多个(或者一个)包容近期所有变化的Box(变动范围);在检测时,用当前像素与Box去比较,如果当前像素落在任何Box的范围内,则为背景。
在OpenCv中已经实现了编码本背景模型,不过实现方式与《学习OpenCv》中提到的方式略有不同,主要有:(1)使用单向链表来容纳Code Element;(2)清除消极的Code Element时,并未重置t。OpenCv中的以下函数与编码本背景模型相关:
cvCreateBGCodeBookModel 建立背景模型
cvBGCodeBookUpdate 更新背景模型
cvBGCodeBookClearStale 清除消极的Code Element
cvBGCodeBookDiff 计算得到背景与前景(注意:该函数仅仅设置背景像素为0,而对前景像素未处理,因此在调用前需要将所有的像素先置为前景)
cvReleaseBGCodeBookModel 释放资源
在EmguCv中只实现了一部分编码本背景模型,在类BGCodeBookModel<TColor>中,可惜它把cvBGCodeBookDiff给搞忘记了 -_-
下面的代码演示了如何使用编码本背景模型:
- //(1)初始化对象
- if (rbCodeBook.Checked)
- {
- if (bgCodeBookModel != null)
- {
- bgCodeBookModel.Dispose();
- bgCodeBookModel = null;
- }
- bgCodeBookModel = new BGCodeBookModel<Bgr>();
- }
- //(2)背景建模或者前景检测
- bool stop = false;
- while (!stop)
- {
- Image<Bgr, Byte> image = capture.QueryFrame().Clone(); //当前帧
- bool isBgModeling, isFgDetecting; //是否正在建模,是否正在前景检测
- lock (lockObject)
- {
- stop = !isVideoCapturing;
- isBgModeling = isBackgroundModeling;
- isFgDetecting = isForegroundDetecting;
- }
- //得到设置的参数
- SettingParam param = (SettingParam)this.Invoke(new GetSettingParamDelegate(GetSettingParam));
- //code book
- if (param.ForegroundDetectType == ForegroundDetectType.CodeBook)
- {
- if (bgCodeBookModel != null)
- {
- //背景建模
- if (isBgModeling)
- {
- bgCodeBookModel.Update(image);
- //背景建模一段时间之后,清理陈旧的条目 (因为清理操作不会重置t,所以这里用求余数的办法来决定清理的时机)
- if (backgroundModelFrameCount % CodeBookClearPeriod == CodeBookClearPeriod - 1)
- bgCodeBookModel.ClearStale(CodeBookStaleThresh, Rectangle.Empty, null);
- backgroundModelFrameCount++;
- pbBackgroundModel.Image = bgCodeBookModel.BackgroundMask.Bitmap;
- //如果达到最大背景建模次数,停止背景建模
- if (param.MaxBackgroundModelFrameCount != 0 && backgroundModelFrameCount > param.MaxBackgroundModelFrameCount)
- this.Invoke(new NoParamAndReturnDelegate(StopBackgroundModel));
- }
- //前景检测
- if (isFgDetecting)
- {
- Image<Gray, Byte> imageFg = new Image<Gray, byte>(image.Size);
- imageFg.SetValue(255d); //CodeBook在得出前景时,仅仅将背景像素置零,所以这里需要先将所有的像素都假设为前景
- CvInvoke.cvBGCodeBookDiff(bgCodeBookModel.Ptr, image.Ptr, imageFg.Ptr, Rectangle.Empty);
- pbBackgroundModel.Image = imageFg.Bitmap;
- }
- }
- }
- //更新视频图像
- pbVideo.Image = image.Bitmap;
- }
- //(3)释放对象
- if (bgCodeBookModel != null)
- {
- try
- {
- bgCodeBookModel.Dispose();
- }
- catch { }
- }
4.高级背景统计模型
在OpenCv还实现了两种高级的背景统计模型,它们为别是:(1)FGD——复杂背景下的前景物体检测(Foreground object detection from videos containing complex background);(2)MOG——高斯混合模型(Mixture Of Gauss)。包括以下函数:
CvCreateFGDetectorBase 建立前景检测对象
CvFGDetectorProcess 更新前景检测对象
CvFGDetectorGetMask 获取前景
CvFGDetectorRelease 释放资源
EmguCv将其封装到类FGDetector<TColor>中。我个人觉得OpenCv在实现这个模型的时候做得不太好,因为它将背景建模和前景检测糅合到一起了,无论你是否愿意,在建模的过程中也会检测前景,而只希望前景检测的时候,同时也会建模。我比较喜欢将背景建模和前景检测进行分离的设计。
调用的过程很简单,代码如下:
- //(1)创建对象
- if (rbMog.Checked)
- {
- if (fgDetector != null)
- {
- fgDetector.Dispose();
- fgDetector = null;
- }
- fgDetector = new FGDetector<Bgr>(FORGROUND_DETECTOR_TYPE.FGD);
- }
- else if (rbFgd.Checked)
- {
- if (fgDetector != null)
- {
- fgDetector.Dispose();
- fgDetector = null;
- }
- fgDetector = new FGDetector<Bgr>(FORGROUND_DETECTOR_TYPE.MOG);
- }
- //背景建模及前景检测
- bool stop = false;
- while (!stop)
- {
- Image<Bgr, Byte> image = capture.QueryFrame().Clone(); //当前帧
- bool isBgModeling, isFgDetecting; //是否正在建模,是否正在前景检测
- lock (lockObject)
- {
- stop = !isVideoCapturing;
- isBgModeling = isBackgroundModeling;
- isFgDetecting = isForegroundDetecting;
- }
- //得到设置的参数
- SettingParam param = (SettingParam)this.Invoke(new GetSettingParamDelegate(GetSettingParam));
- if (param.ForegroundDetectType == ForegroundDetectType.Fgd || param.ForegroundDetectType == ForegroundDetectType.Mog)
- {
- if (fgDetector != null && (isBgModeling || isFgDetecting))
- {
- //背景建模
- fgDetector.Update(image);
- backgroundModelFrameCount++;
- pbBackgroundModel.Image = fgDetector.BackgroundMask.Bitmap;
- //如果达到最大背景建模次数,停止背景建模
- if (param.MaxBackgroundModelFrameCount != 0 && backgroundModelFrameCount > param.MaxBackgroundModelFrameCount)
- this.Invoke(new NoParamAndReturnDelegate(StopBackgroundModel));
- //前景检测
- if (isFgDetecting)
- {
- pbBackgroundModel.Image = fgDetector.ForgroundMask.Bitmap;
- }
- }
- }
- //更新视频图像
- pbVideo.Image = image.Bitmap;
- }
- //(3)释放资源
- if (fgDetector != null)
- {
- try
- {
- fgDetector.Dispose();
- }
- catch { }
- }
前景检测
在建立好背景模型之后,通过对当前图像及背景的某种比较,我们可以得出前景。在上面的介绍中,已经包含了对前景的代码,在此不再重复。一般情况下,得到的前景包含了很多噪声,为了消除噪声,我们可以对前景图像进行开运算及闭运算,然后再丢弃比较小的轮廓。
本文的代码
- using System;
- using System.Collections.Generic;
- using System.ComponentModel;
- using System.Data;
- using System.Drawing;
- using System.Linq;
- using System.Text;
- using System.Windows.Forms;
- using System.Diagnostics;
- using System.Runtime.InteropServices;
- using System.Threading;
- using Emgu.CV;
- using Emgu.CV.CvEnum;
- using Emgu.CV.Structure;
- using Emgu.CV.UI;
- using Emgu.CV.VideoSurveillance;
- namespace ImageProcessLearn
- {
- public partial class FormForegroundDetect : Form
- {
- //成员变量
- Capture capture = null; //视频捕获对象
- Thread captureThread = null; //视频捕获线程
- private bool isVideoCapturing = true; //是否正在捕获视频
- private bool isBackgroundModeling = false; //是否正在背景建模
- private int backgroundModelFrameCount = 0; //已经建模的视频帧数
- private bool isForegroundDetecting = false; //是否正在进行前景检测
- private object lockObject = new object(); //用于锁定的对象
- //各种前景检测方法对应的对象
- BGCodeBookModel<Bgr> bgCodeBookModel = null; //编码本前景检测
- private const int CodeBookClearPeriod = 40; //编码本的清理周期,更新这么多次背景之后,清理掉很少使用的陈旧条目
- private const int CodeBookStaleThresh = 20; //在清理编码本时,使用的阀值(stale大于该阀值的条目将被删除)
- FGDetector<Bgr> fgDetector = null; //Mog或者Fgd检测
- BackgroundStatModelFrameDiff<Bgr> bgModelFrameDiff = null; //帧差
- BackgroundStatModelAccAvg<Bgr> bgModelAccAvg = null; //平均背景
- BackgroundStatModelRunningAvg<Bgr> bgModelRunningAvg = null; //均值漂移
- BackgroundStatModelSquareAcc<Bgr> bgModelSquareAcc = null; //标准方差
- BackgroundStatModelMultiplyAcc<Bgr> bgModelMultiplyAcc = null; //标准协方差
- public FormForegroundDetect()
- {
- InitializeComponent();
- }
- //窗体加载时
- private void FormForegroundDetect_Load(object sender, EventArgs e)
- {
- //设置Tooltip
- toolTip.Active = true;
- toolTip.SetToolTip(rbMog, "高斯混合模型(Mixture Of Gauss)");
- toolTip.SetToolTip(rbFgd, "复杂背景下的前景物体检测(Foreground object detection from videos containing complex background)");
- toolTip.SetToolTip(txtMaxBackgroundModelFrameCount, "在背景建模时,使用的最大帧数,超出该值之后,将自动停止背景建模。\r\n对于帧差,总是只捕捉当前帧作为背景。\r\n如果设为零,背景检测将不会自动停止。");
- //打开摄像头视频捕获线程
- capture = new Capture(0);
- captureThread = new Thread(new ParameterizedThreadStart(CaptureWithEmguCv));
- captureThread.Start(null);
- }
- //窗体关闭前
- private void FormForegroundDetect_FormClosing(object sender, FormClosingEventArgs e)
- {
- //终止视频捕获
- isVideoCapturing = false;
- if (captureThread != null)
- captureThread.Abort();
- if (capture != null)
- capture.Dispose();
- //释放对象
- if (bgCodeBookModel != null)
- {
- try
- {
- bgCodeBookModel.Dispose();
- }
- catch { }
- }
- if (fgDetector != null)
- {
- try
- {
- fgDetector.Dispose();
- }
- catch { }
- }
- if (bgModelFrameDiff != null)
- bgModelFrameDiff.Dispose();
- if (bgModelAccAvg != null)
- bgModelAccAvg.Dispose();
- if (bgModelRunningAvg != null)
- bgModelRunningAvg.Dispose();
- if (bgModelSquareAcc != null)
- bgModelSquareAcc.Dispose();
- if (bgModelMultiplyAcc != null)
- bgModelMultiplyAcc.Dispose();
- }
- //EmguCv视频捕获
- private void CaptureWithEmguCv(object objParam)
- {
- if (capture == null)
- return;
- bool stop = false;
- while (!stop)
- {
- Image<Bgr, Byte> image = capture.QueryFrame().Clone(); //当前帧
- bool isBgModeling, isFgDetecting; //是否正在建模,是否正在前景检测
- lock (lockObject)
- {
- stop = !isVideoCapturing;
- isBgModeling = isBackgroundModeling;
- isFgDetecting = isForegroundDetecting;
- }
- //得到设置的参数
- SettingParam param = (SettingParam)this.Invoke(new GetSettingParamDelegate(GetSettingParam));
- //code book
- if (param.ForegroundDetectType == ForegroundDetectType.CodeBook)
- {
- if (bgCodeBookModel != null && (isBgModeling || isFgDetecting))
- {
- //背景建模
- if (isBgModeling)
- {
- bgCodeBookModel.Update(image);
- //背景建模一段时间之后,清理陈旧的条目
- if (backgroundModelFrameCount % CodeBookClearPeriod == CodeBookClearPeriod - 1)
- bgCodeBookModel.ClearStale(CodeBookStaleThresh, Rectangle.Empty, null);
- backgroundModelFrameCount++;
- pbBackgroundModel.Image = bgCodeBookModel.BackgroundMask.Bitmap;
- //如果达到最大背景建模次数,停止背景建模
- if (param.MaxBackgroundModelFrameCount != 0 && backgroundModelFrameCount > param.MaxBackgroundModelFrameCount)
- this.Invoke(new NoParamAndReturnDelegate(StopBackgroundModel));
- }
- //前景检测
- if (isFgDetecting)
- {
- Image<Gray, Byte> imageFg = new Image<Gray, byte>(image.Size);
- imageFg.SetValue(255d); //CodeBook在得出前景时,仅仅将背景像素置零,所以这里需要先将所有的像素都假设为前景
- CvInvoke.cvBGCodeBookDiff(bgCodeBookModel.Ptr, image.Ptr, imageFg.Ptr, Rectangle.Empty);
- pbBackgroundModel.Image = imageFg.Bitmap;
- }
- }
- }
- //fgd or mog
- else if (param.ForegroundDetectType == ForegroundDetectType.Fgd || param.ForegroundDetectType == ForegroundDetectType.Mog)
- {
- if (fgDetector != null && (isBgModeling || isFgDetecting))
- {
- //背景建模
- fgDetector.Update(image);
- backgroundModelFrameCount++;
- pbBackgroundModel.Image = fgDetector.BackgroundMask.Bitmap;
- //如果达到最大背景建模次数,停止背景建模
- if (param.MaxBackgroundModelFrameCount != 0 && backgroundModelFrameCount > param.MaxBackgroundModelFrameCount)
- this.Invoke(new NoParamAndReturnDelegate(StopBackgroundModel));
- //前景检测
- if (isFgDetecting)
- {
- pbBackgroundModel.Image = fgDetector.ForgroundMask.Bitmap;
- }
- }
- }
- //帧差
- else if (param.ForegroundDetectType == ForegroundDetectType.FrameDiff)
- {
- if (bgModelFrameDiff != null)
- {
- //背景建模
- if (isBgModeling)
- {
- bgModelFrameDiff.Update(image);
- backgroundModelFrameCount++;
- this.Invoke(new NoParamAndReturnDelegate(StopBackgroundModel)); //对于帧差,只需要捕获当前帧作为背景即可
- }
- //前景检测
- if (isFgDetecting)
- {
- bgModelFrameDiff.Threshold = param.Threshold;
- bgModelFrameDiff.CurrentFrame = image;
- pbBackgroundModel.Image = bgModelFrameDiff.ForegroundMask.Bitmap;
- }
- }
- }
- //平均背景
- else if (param.ForegroundDetectType == ForegroundDetectType.AccAvg)
- {
- if (bgModelAccAvg!=null)
- {
- //背景建模
- if (isBgModeling)
- {
- bgModelAccAvg.Update(image);
- backgroundModelFrameCount++;
- //如果达到最大背景建模次数,停止背景建模
- if (param.MaxBackgroundModelFrameCount != 0 && backgroundModelFrameCount > param.MaxBackgroundModelFrameCount)
- this.Invoke(new NoParamAndReturnDelegate(StopBackgroundModel));
- }
- //前景检测
- if (isFgDetecting)
- {
- bgModelAccAvg.CurrentFrame = image;
- pbBackgroundModel.Image = bgModelAccAvg.ForegroundMask.Bitmap;
- }
- }
- }
- //均值漂移
- else if (param.ForegroundDetectType == ForegroundDetectType.RunningAvg)
- {
- if (bgModelRunningAvg != null)
- {
- //背景建模
- if (isBgModeling)
- {
- bgModelRunningAvg.Update(image);
- backgroundModelFrameCount++;
- //如果达到最大背景建模次数,停止背景建模
- if (param.MaxBackgroundModelFrameCount != 0 && backgroundModelFrameCount > param.MaxBackgroundModelFrameCount)
- this.Invoke(new NoParamAndReturnDelegate(StopBackgroundModel));
- }
- //前景检测
- if (isFgDetecting)
- {
- bgModelRunningAvg.CurrentFrame = image;
- pbBackgroundModel.Image = bgModelRunningAvg.ForegroundMask.Bitmap;
- }
- }
- }
- //计算方差
- else if (param.ForegroundDetectType == ForegroundDetectType.SquareAcc)
- {
- if (bgModelSquareAcc != null)
- {
- //背景建模
- if (isBgModeling)
- {
- bgModelSquareAcc.Update(image);
- backgroundModelFrameCount++;
- //如果达到最大背景建模次数,停止背景建模
- if (param.MaxBackgroundModelFrameCount != 0 && backgroundModelFrameCount > param.MaxBackgroundModelFrameCount)
- this.Invoke(new NoParamAndReturnDelegate(StopBackgroundModel));
- }
- //前景检测
- if (isFgDetecting)
- {
- bgModelSquareAcc.CurrentFrame = image;
- pbBackgroundModel.Image = bgModelSquareAcc.ForegroundMask.Bitmap;
- }
- }
- }
- //协方差
- else if (param.ForegroundDetectType == ForegroundDetectType.MultiplyAcc)
- {
- if (bgModelMultiplyAcc != null)
- {
- //背景建模
- if (isBgModeling)
- {
- bgModelMultiplyAcc.Update(image);
- backgroundModelFrameCount++;
- //如果达到最大背景建模次数,停止背景建模
- if (param.MaxBackgroundModelFrameCount != 0 && backgroundModelFrameCount > param.MaxBackgroundModelFrameCount)
- this.Invoke(new NoParamAndReturnDelegate(StopBackgroundModel));
- }
- //前景检测
- if (isFgDetecting)
- {
- bgModelMultiplyAcc.CurrentFrame = image;
- pbBackgroundModel.Image = bgModelMultiplyAcc.ForegroundMask.Bitmap;
- }
- }
- }
- //更新视频图像
- pbVideo.Image = image.Bitmap;
- }
- }
- //用于在工作线程中更新结果的委托及方法
- private delegate void AddResultDelegate(string result);
- private void AddResultMethod(string result)
- {
- //txtResult.Text += result;
- }
- //用于在工作线程中获取设置参数的委托及方法
- private delegate SettingParam GetSettingParamDelegate();
- private SettingParam GetSettingParam()
- {
- ForegroundDetectType type = ForegroundDetectType.FrameDiff;
- if (rbFrameDiff.Checked)
- type = ForegroundDetectType.FrameDiff;
- else if (rbAccAvg.Checked)
- type = ForegroundDetectType.AccAvg;
- else if (rbRunningAvg.Checked)
- type = ForegroundDetectType.RunningAvg;
- else if (rbMultiplyAcc.Checked)
- type = ForegroundDetectType.MultiplyAcc;
- else if (rbSquareAcc.Checked)
- type = ForegroundDetectType.SquareAcc;
- else if (rbCodeBook.Checked)
- type = ForegroundDetectType.CodeBook;
- else if (rbMog.Checked)
- type = ForegroundDetectType.Mog;
- else
- type = ForegroundDetectType.Fgd;
- int maxFrameCount = 0;
- int.TryParse(txtMaxBackgroundModelFrameCount.Text, out maxFrameCount);
- double threshold = 15d;
- double.TryParse(txtThreshold.Text, out threshold);
- if (threshold <= 0)
- threshold = 15d;
- return new SettingParam(type, maxFrameCount, threshold);
- }
- //没有参数及返回值的委托
- private delegate void NoParamAndReturnDelegate();
- //开始背景建模
- private void btnStartBackgroundModel_Click(object sender, EventArgs e)
- {
- if (rbCodeBook.Checked)
- {
- if (bgCodeBookModel != null)
- {
- bgCodeBookModel.Dispose();
- bgCodeBookModel = null;
- }
- bgCodeBookModel = new BGCodeBookModel<Bgr>();
- }
- else if (rbMog.Checked)
- {
- if (fgDetector != null)
- {
- fgDetector.Dispose();
- fgDetector = null;
- }
- fgDetector = new FGDetector<Bgr>(FORGROUND_DETECTOR_TYPE.FGD);
- }
- else if (rbFgd.Checked)
- {
- if (fgDetector != null)
- {
- fgDetector.Dispose();
- fgDetector = null;
- }
- fgDetector = new FGDetector<Bgr>(FORGROUND_DETECTOR_TYPE.MOG);
- }
- else if (rbFrameDiff.Checked)
- {
- if (bgModelFrameDiff != null)
- {
- bgModelFrameDiff.Dispose();
- bgModelFrameDiff = null;
- }
- bgModelFrameDiff = new BackgroundStatModelFrameDiff<Bgr>();
- }
- else if (rbAccAvg.Checked)
- {
- if (bgModelAccAvg != null)
- {
- bgModelAccAvg.Dispose();
- bgModelAccAvg = null;
- }
- bgModelAccAvg = new BackgroundStatModelAccAvg<Bgr>();
- }
- else if (rbRunningAvg.Checked)
- {
- if (bgModelRunningAvg != null)
- {
- bgModelRunningAvg.Dispose();
- bgModelRunningAvg = null;
- }
- bgModelRunningAvg = new BackgroundStatModelRunningAvg<Bgr>();
- }
- else if (rbSquareAcc.Checked)
- {
- if (bgModelSquareAcc != null)
- {
- bgModelSquareAcc.Dispose();
- bgModelSquareAcc = null;
- }
- bgModelSquareAcc = new BackgroundStatModelSquareAcc<Bgr>();
- }
- else if (rbMultiplyAcc.Checked)
- {
- if (bgModelMultiplyAcc != null)
- {
- bgModelMultiplyAcc.Dispose();
- bgModelMultiplyAcc = null;
- }
- bgModelMultiplyAcc = new BackgroundStatModelMultiplyAcc<Bgr>();
- }
- backgroundModelFrameCount = 0;
- isBackgroundModeling = true;
- btnStartBackgroundModel.Enabled = false;
- btnStopBackgroundModel.Enabled = true;
- btnStartForegroundDetect.Enabled = false;
- btnStopForegroundDetect.Enabled = false;
- }
- //停止背景建模
- private void btnStopBackgroundModel_Click(object sender, EventArgs e)
- {
- StopBackgroundModel();
- }
- //停止背景建模
- private void StopBackgroundModel()
- {
- lock (lockObject)
- {
- isBackgroundModeling = false;
- }
- btnStartBackgroundModel.Enabled = true;
- btnStopBackgroundModel.Enabled = false;
- btnStartForegroundDetect.Enabled = true;
- btnStopForegroundDetect.Enabled = false;
- }
- //开始前景检测
- private void btnStartForegroundDetect_Click(object sender, EventArgs e)
- {
- isForegroundDetecting = true;
- btnStartBackgroundModel.Enabled = false;
- btnStopBackgroundModel.Enabled = false;
- btnStartForegroundDetect.Enabled = false;
- btnStopForegroundDetect.Enabled = true;
- }
- //停止前景检测
- private void btnStopForegroundDetect_Click(object sender, EventArgs e)
- {
- lock (lockObject)
- {
- isForegroundDetecting = false;
- }
- btnStartBackgroundModel.Enabled = true;
- btnStopBackgroundModel.Enabled = false;
- btnStartForegroundDetect.Enabled = true;
- btnStopForegroundDetect.Enabled = false;
- }
- }
- //前景检测方法枚举
- public enum ForegroundDetectType
- {
- FrameDiff,
- AccAvg,
- RunningAvg,
- MultiplyAcc,
- SquareAcc,
- CodeBook,
- Mog,
- Fgd
- }
- //设置参数
- public struct SettingParam
- {
- public ForegroundDetectType ForegroundDetectType;
- public int MaxBackgroundModelFrameCount;
- public double Threshold;
- public SettingParam(ForegroundDetectType foregroundDetectType, int maxBackgroundModelFrameCount, double threshold)
- {
- ForegroundDetectType = foregroundDetectType;
- MaxBackgroundModelFrameCount = maxBackgroundModelFrameCount;
- Threshold = threshold;
- }
- }
- }
另外,细心的读者发现我忘记贴OpenCvInvoke类的实现代码了,这里补上。多谢指正。
- using System;
- using System.Collections.Generic;
- using System.Linq;
- using System.Text;
- using System.Drawing;
- using System.Runtime.InteropServices;
- using Emgu.CV.Structure;
- using Emgu.CV.CvEnum;
- namespace ImageProcessLearn
- {
- /// <summary>
- /// 声明一些没有包含在EmguCv中的OpenCv函数
- /// </summary>
- public static class OpenCvInvoke
- {
- //自适应动态背景检测
- [DllImport("cvaux200.dll")]
- public static extern void cvChangeDetection(IntPtr prev_frame, IntPtr curr_frame, IntPtr change_mask);
- //均值漂移分割
- [DllImport("cv200.dll")]
- public static extern void cvPyrMeanShiftFiltering(IntPtr src, IntPtr dst, double spatialRadius, double colorRadius, int max_level, MCvTermCriteria termcrit);
- //开始查找轮廓
- [DllImport("cv200.dll")]
- public static extern IntPtr cvStartFindContours(IntPtr image, IntPtr storage, int header_size, RETR_TYPE mode, CHAIN_APPROX_METHOD method, Point offset);
- //查找下一个轮廓
- [DllImport("cv200.dll")]
- public static extern IntPtr cvFindNextContour(IntPtr scanner);
- //用新轮廓替换scanner指向的当前轮廓
- [DllImport("cv200.dll")]
- public static extern void cvSubstituteContour(IntPtr scanner, IntPtr new_contour);
- //结束轮廓查找
- [DllImport("cv200.dll")]
- public static extern IntPtr cvEndFindContour(ref IntPtr scanner);
- }
- }
后记
值得注意的是,本文提到的OpenCv函数目前属于CvAux系列,以后也许会加入到正式的图像处理Cv系列,也许以后会消失。最重要的是它们还没有正式的文档。
其实关于背景模型的方法还有很多,比如《Video-object segmentation using multi-sprite background subtraction》可以在摄像机运动的情况下建立背景,《Nonparametric background generation》利用mean-shift算法处理动态的背景模型,如果我的时间和能力允许,也许会去尝试实现它们。另外,《Wallflower: Principles and practice of background maintenance》比较了各种背景建模方式的差异,我希望能够尝试翻译出来。
感谢您耐心看完本文,希望对您有所帮助。