MATLAB小球目标追踪示例

MATLAB小球目标追踪示例代码注释备份

%% Use Kalman Filter for Object Tracking
% This example shows how to use the |vision.KalmanFilter| object and
% |configureKalmanFilter| function to track objects.

% Copyright 2012 The MathWorks, Inc.

%%
% This example is a function with its main body at the top and helper
% routines in the form of nested functions.
%
function kalmanFilterForTracking

%% Introduction
% The Kalman filter has many uses, including applications in control,
% navigation, computer vision, and time series econometrics. This example
% illustrates how to use the Kalman filter for tracking objects and focuses
% on three important features:

%%
% * Prediction of object's future location
% * Reduction of noise introduced by inaccurate detections
% * Facilitating the process of association of multiple objects to their
%   tracks

%% Challenges of Object Tracking
% Before showing the use of Kalman filter, let us first examine the
% challenges of tracking an object in a video. The following video shows a
% green ball moving from left to right on the floor.
showDetections();

%%
% The white region over the ball highlights the pixels detected using
% |vision.ForegroundDetector|, which separates moving objects from the
% background. The background subtraction only finds a portion of the ball
% because of the low contrast between the ball and the floor. In other
% words, the detection process is not ideal and introduces noise.
%
% To easily visualize the entire object trajectory, we overlay all video
% frames onto a single image. The "+" marks indicate the centroids computed
% using blob analysis.
showTrajectory();   %覆盖了所有的视频帧到一个单一的图像。“+”标记表示使用blob分析计算出的质心。

%%
% Two issues can be observed:
% # The region's center is usually different from the ball's center. In
%   other words, there is an error in the measurement of the ball's
%   location.
% # The location of the ball is not available when it is occluded by the
%   box, i.e. the measurement is missing.

%%
% Both of these challenges can be addressed by using the Kalman filter.
% 上述两个挑战都可以通过使用卡尔曼滤波器来解决。

%% Track a Single Object Using Kalman Filter
% Using the video which was seen earlier, the |trackSingleObject| function
% shows you how to:

%% 
% * Create |vision.KalmanFilter| by using |configureKalmanFilter|
% * Use |predict| and |correct| methods in a sequence to eliminate noise
%   present in the tracking system
% * Use |predict| method by itself to estimate ball's location when
%   it is occluded by the box

%
% The selection of the Kalman filter parameters can be challenging. The
% |configureKalmanFilter| function helps simplify this problem. More
% details about this can be found further in the example.

%%
% The |trackSingleObject| function includes nested helper functions. The
% following top-level variables are used to transfer the data between the
% nested functions.  
frame            = [];  % A video frame
detectedLocation = [];  % The detected location
trackedLocation  = [];  % The tracked location
label            = '';  % Label for the ball
utilities        = [];  % Utilities used to process the video

%%
% The procedure for tracking a single object is shown below.
function trackSingleObject(param)
  % Create utilities used for reading video, detecting moving objects,
  % and displaying the results.
  utilities = createUtilities(param);

  isTrackInitialized = false;
  while hasFrame(utilities.videoReader)
    frame = readFrame(utilities.videoReader);

    % Detect the ball.
    [detectedLocation, isObjectDetected] = detectObject(frame);

    if ~isTrackInitialized
      if isObjectDetected
        % Initialize a track by creating a Kalman filter when the ball is
        % detected for the first time.
        initialLocation = computeInitialLocation(param, detectedLocation);
        kalmanFilter = configureKalmanFilter(param.motionModel, ...
          initialLocation, param.initialEstimateError, ...
          param.motionNoise, param.measurementNoise);

        isTrackInitialized = true;
        trackedLocation = correct(kalmanFilter, detectedLocation);
        label = 'Initial';
      else
        trackedLocation = [];
        label = '';
      end

    else
      % Use the Kalman filter to track the ball.
      if isObjectDetected % The ball was detected.
        % Reduce the measurement noise by calling predict followed by
        % correct.
        predict(kalmanFilter);
        trackedLocation = correct(kalmanFilter, detectedLocation);
        label = 'Corrected';
      else % The ball was missing.
        % Predict the ball's location.
        trackedLocation = predict(kalmanFilter);
        label = 'Predicted';
      end
    end

    annotateTrackedObject();  
  end % while
  
  showTrajectory(); %红色圆圈展示预测轨迹
end

%% 
% There are two distinct scenarios that the Kalman filter addresses:

%%
% * When the ball is detected, the Kalman filter first predicts its state
%   at the current video frame, and then uses the newly detected object
%   location to correct its state. This produces a filtered location.
% * When the ball is missing, the Kalman filter solely relies on its
%   previous state to predict the ball's current location.

%%
% You can see the ball's trajectory by overlaying all video frames.
param = getDefaultParameters();  % get Kalman configuration that works well
                                 % for this example
                                 
trackSingleObject(param);  % visualize the results

%% Explore Kalman Filter Configuration Options 【配置卡尔曼滤波器参数】
% Configuring the Kalman filter can be very challenging. Besides basic
% understanding of the Kalman filter, it often requires experimentation in
% order to come up with a set of suitable configuration parameters. The
% |trackSingleObject| function, defined above, helps you to explore the
% various configuration options offered by the |configureKalmanFilter|
% function.
%
% The |configureKalmanFilter| function returns a Kalman filter object. You
% must provide five input arguments.
%
%   kalmanFilter = configureKalmanFilter(MotionModel, InitialLocation,
%            InitialEstimateError, MotionNoise, MeasurementNoise)

%%
% The *MotionModel* setting must correspond to the physical characteristics
% of the object's motion. You can set it to either a constant velocity or
% constant acceleration model. The following example illustrates the
% consequences of making a sub-optimal choice.  【次优选择】
param = getDefaultParameters();         % get parameters that work well
param.motionModel = 'ConstantVelocity'; % switch from ConstantAcceleration
                                        % to ConstantVelocity
% After switching motion models, drop noise specification entries
% corresponding to acceleration.
param.initialEstimateError = param.initialEstimateError(1:2);
param.motionNoise          = param.motionNoise(1:2);

trackSingleObject(param); % visualize the results

%%
% Notice that the ball emerged in a spot that is quite different from the
% predicted location. From the time when the ball was released, it was
% subject to constant deceleration due to resistance from the carpet.
% Therefore, constant acceleration model was a better choice. If you kept
% the constant velocity model, the tracking results would be sub-optimal no
% matter what you selected for the other values.

%%
% Typically, you would set the *InitialLocation* input to the location
% where the object was first detected. You would also set the
% *InitialEstimateError* vector to large values since the initial state may
% be very noisy given that it is derived from a single detection. The
% following figure demonstrates the effect of misconfiguring these 
% parameters. 【错误示范】

param = getDefaultParameters();  % get parameters that work well
param.initialLocation = [0, 0];  % location that's not based on an actual detection 
param.initialEstimateError = 100*ones(1,3); % use relatively small values

trackSingleObject(param); % visualize the results

%%
% With the misconfigured parameters, it took a few steps before the
% locations returned by the Kalman filter align with the actual trajectory
% of the object.

%%
% The values for *MeasurementNoise* should be selected based on the
% detector's accuracy. Set the measurement noise to larger values for a
% less accurate detector. The following example illustrates the noisy
% detections of a misconfigured segmentation threshold. Increasing the
% measurement noise causes the Kalman filter to rely more on its internal
% state rather than the incoming measurements, and thus compensates for the
% detection noise.【错误示范】

param = getDefaultParameters();
param.segmentationThreshold = 0.0005; % smaller value resulting in noisy detections
param.measurementNoise      = 12500;  % increase the value to compensate 
                                      % for the increase in measurement noise

trackSingleObject(param); % visualize the results

%%
% Typically objects do not move with constant acceleration or constant
% velocity. You use the *MotionNoise* to specify the amount of deviation
% from the ideal motion model. When you increase the motion noise, the
% Kalman filter relies more heavily on the incoming measurements than on
% its internal state. Try experimenting with *MotionNoise* parameter to
% learn more about its effects.

%%
% Now that you are familiar with how to use the Kalman filter and how to
% configure it, the next section will help you learn how it can be used for
% multiple object tracking.

%%
% *Note:* In order to simplify the configuration process in the above
% examples, we used the |configureKalmanFilter| function. This function
% makes several assumptions. See the function's documentation for details.
% If you require greater level of control over the configuration process,
% you can use the |vision.KalmanFilter| object directly.

%% Track Multiple Objects Using Kalman Filter
%
% Tracking multiple objects poses several additional challenges:

%% 
% * Multiple detections must be associated with the correct tracks
% * You must handle new objects appearing in a scene 
% * Object identity must be maintained when multiple objects merge into a
%   single detection
%
% The |vision.KalmanFilter| object together with the
% |assignDetectionsToTracks| function can help to solve the problems of

%%
% * Assigning detections to tracks
% * Determining whether or not a detection corresponds to a new object, 
%   in other words, track creation
% * Just as in the case of an occluded single object, prediction can be
%   used to help separate objects that are close to each other
%
% To learn more about using Kalman filter to track multiple objects, see
% the example titled
% <docid:vision_ug#example-MotionBasedMultiObjectTrackingExample
% Motion-Based Multiple Object Tracking>.

%% Utility Functions Used in the Example
% Utility functions were used for detecting the objects and displaying the
% results. This section illustrates how the example implemented these
% functions.

%%
% Get default parameters for creating Kalman filter and for segmenting the
% ball.
function param = getDefaultParameters
  param.motionModel           = 'ConstantAcceleration';
  param.initialLocation       = 'Same as first detection';
  param.initialEstimateError  = 1E5 * ones(1, 3);
  param.motionNoise           = [25, 10, 1];
  param.measurementNoise      = 25;
  param.segmentationThreshold = 0.05;
end

%%
% Detect and annotate the ball in the video.
function showDetections()
  param = getDefaultParameters();   %获取创建卡尔曼过滤器和分割球的默认参数
  utilities = createUtilities(param);   %创建用于读取视频、检测移动对象和显示结果
  trackedLocation = [];

  idx = 0;
  while hasFrame(utilities.videoReader)
    frame = readFrame(utilities.videoReader);
    detectedLocation = detectObject(frame);    %检测当前视频帧中的球
    % Show the detection result for the current video frame.
    annotateTrackedObject();   %显示当前检测和跟踪结果

    % To highlight the effects of the measurement noise, show the detection
    % results for the 40th frame in a separate figure.
    idx = idx + 1;
    if idx == 40
      combinedImage = max(repmat(utilities.foregroundMask, [1,1,3]), im2single(frame));
      figure, imshow(combinedImage);
    end
  end % while
  
  % Close the window which was used to show individual video frame.
  uiscopes.close('All'); 
end

%%
% Detect the ball in the current video frame.
function [detection, isObjectDetected] = detectObject(frame)
  grayImage = rgb2gray(im2single(frame));
  utilities.foregroundMask = step(utilities.foregroundDetector, grayImage);
  detection = step(utilities.blobAnalyzer, utilities.foregroundMask);
  if isempty(detection)
    isObjectDetected = false;
  else
    % To simplify the tracking process, only use the first detected object.
    detection = detection(1, :);
    isObjectDetected = true;
  end
end

%%
% Show the current detection and tracking results.
function annotateTrackedObject()  % 显示当前检测和跟踪结果
  accumulateResults();
  % Combine the foreground mask with the current video frame in order to
  % show the detection result.
  combinedImage = max(repmat(utilities.foregroundMask, [1,1,3]), im2single(frame));

  if ~isempty(trackedLocation)
    shape = 'circle';
    region = trackedLocation;
    region(:, 3) = 5;
    combinedImage = insertObjectAnnotation(combinedImage, shape, ...
      region, {label}, 'Color', 'red');
  end
  step(utilities.videoPlayer, combinedImage);
end

%%
% Show trajectory of the ball by overlaying all video frames on top of 
% each other.
function showTrajectory
  % Close the window which was used to show individual video frame.
  uiscopes.close('All'); 
  
  % Create a figure to show the processing results for all video frames.
  figure; imshow(utilities.accumulatedImage/2+0.5); hold on;

   plot(utilities.accumulatedDetections(:,1), ...
     utilities.accumulatedDetections(:,2), 'k+');
  
  if ~isempty(utilities.accumulatedTrackings)
    plot(utilities.accumulatedTrackings(:,1), ...
      utilities.accumulatedTrackings(:,2), 'r-o');
    legend('Detection', 'Tracking');
  end
end

%%
% Accumulate video frames, detected locations, and tracked locations to
% show the trajectory of the ball.
function accumulateResults()
  utilities.accumulatedImage      = max(utilities.accumulatedImage, frame);
  utilities.accumulatedDetections ...
    = [utilities.accumulatedDetections; detectedLocation];
  utilities.accumulatedTrackings  ...
    = [utilities.accumulatedTrackings; trackedLocation];
end

%%
% For illustration purposes, select the initial location used by the Kalman
% filter.
function loc = computeInitialLocation(param, detectedLocation)
  if strcmp(param.initialLocation, 'Same as first detection')
    loc = detectedLocation;
  else
    loc = param.initialLocation;
  end
end

%%
% Create utilities for reading video, detecting moving objects, and
% displaying the results.
function utilities = createUtilities(param)
  % Create System objects for reading video, displaying video, extracting
  % foreground, and analyzing connected components.
  utilities.videoReader = VideoReader('singleball.mp4');
  utilities.videoPlayer = vision.VideoPlayer('Position', [100,100,500,400]);
  utilities.foregroundDetector = vision.ForegroundDetector(...
    'NumTrainingFrames', 10, 'InitialVariance', param.segmentationThreshold);
  utilities.blobAnalyzer = vision.BlobAnalysis('AreaOutputPort', false, ...
    'MinimumBlobArea', 70, 'CentroidOutputPort', true);

  utilities.accumulatedImage      = 0;
  utilities.accumulatedDetections = zeros(0, 2);
  utilities.accumulatedTrackings  = zeros(0, 2);
end



end