CN101610412B - A visual tracking method based on multi-cue fusion - Google Patents

Abstract

The invention discloses a visual tracking method based on multi-cue fusion, which belongs to the technical field of information. The method comprises the following steps: a) determining a tracking window comprising a target region and a background region in a first frame of a video sequence; b) obtaining a color feature probability distribution graph, a position feature probability distribution graph and a motion continuity feature probability distribution graph of the previous frame from the second frame; c) adding the three probability distribution graphs in a weighed manner to obtain a total probability distribution graph; and d) using a CAMSHIFT algorithm to obtain the coordinates of a central point of the tracking window of the current frame in the total probability distribution graph. The method can be used in human-computer interaction, visual intelligent surveillance, intelligent robot, virtual reality technology, model-based image encoding, content retrieval of streaming media and other fields.

Description

A visual tracking method based on multi-cue fusion

technical field

The invention relates to visual tracking, in particular to a visual tracking method for fusing multiple clues, and belongs to the field of information technology.

Background technique

With the rapid development of information technology and intelligence science, computer vision, which uses computers to realize human visual functions, has become one of the most popular research directions in the computer field. Among them, visual tracking is one of the core problems of computer vision, which is to find the position of the moving target of interest in each frame of the image sequence. It is very necessary and urgent to study it.

Hong Liu et al published the paper "Collaborative mean shift tracking based on multi-cue integration and auxiliary objects" in "Proceedings of the 14th IEEE International Conference on Image Processing (ICIP 2007)" (IEEE 14th International Conference on Image Processing) in 2007 (Cooperative mean shift tracking based on multi-cue fusion and auxiliary objects), this paper combines color, position and prediction feature clues, dynamically updates the weight of each clue according to the background situation, uses Mean Shift technology, and uses auxiliary objects to achieve Visual tracking method. However, it assumes that the background model obeys a single Gaussian model, and it needs to train the video sequence without moving objects in advance to obtain the background initial model, which limits its application. In the cue evaluation function, a rectangle slightly larger than the target is used to represent the sense The area of interest, the area between the rectangle and the tracking window is defined as the background area. For the reliability evaluation function of a clue, the size of the background area directly affects its value, that is, the larger the tracking window, the greater its reliability evaluation function The smaller the value, the lack of generality.

Contents of the invention

The purpose of the present invention is to overcome the deficiencies in the prior art, provide a kind of visual tracking method of fusion multiple cues, especially can be used for the visual tracking of human body movement, make when allowing computer to carry out visual automatic tracking to target (such as human body), Meet the requirements of accuracy and real-time.

The present invention combines multiple clues (color features, position features and motion continuity features) of video images to realize visual tracking by means of CAMSHIFT (Continuously Adaptive Mean Shift) method, as shown in Figure 1. Among them, the color features are preferably hue and saturation features, red channel features, green channel features, and blue channel features, which achieve better robustness to occlusion and pose changes; position features are implemented using frame difference technology; motion Continuity features are done in terms of inter-frame continuity.

The present invention adopts a fixed tracking window, which limits the management of appearance changes and occlusions, but it does not need to consider some areas with similar backgrounds as part of the target, and can also achieve tracking effects.

The present invention is realized through the following technical solutions, comprising the following steps:

a) Determine a tracking window in the first frame of a section of video sequence, the tracking window includes a target area and a background area, and the target area includes a tracked object; preferably, the tracking window is a rectangle, the The rectangle is equally divided into three parts, the middle part is the target area, and each side part is the background area, as shown in FIG. 2 .

b) For each frame from the second frame, obtain the color feature probability distribution map, position feature probability distribution map and motion continuity feature probability distribution map of the previous frame;

c) weighted addition of the three probability distribution diagrams to obtain a total probability distribution diagram;

d) In the total probability distribution diagram, the coordinates of the center point of the tracking window of the current frame are obtained through the CAMSHIFT algorithm.

Various clues and clue fusion involved in the present invention are described in detail below.

color characteristics

The color features preferably include the hue (Hue) and saturation (Saturation) features of the image, the R (Red) channel features, the G (Green) channel features and the B (Blue) channel features, which realize the change of occlusion and pose Better robustness.

Assuming that the present invention uses a histogram of m handles (bins), the image has n pixels, and their positions and corresponding values in the histogram are respectively {x _i } _i=1...n , {q _u } _u=1,...,m. (R channel characteristics, G channel characteristics and B channel characteristics) or {q _u(v) } _{u=1,...,m; v=1,...,m .} (hue and saturation characteristics). Define the function b: R ² →{1,...,m}, this function represents the discrete interval value corresponding to the color information of each pixel. In the histogram, the value corresponding to the cth color information interval can be expressed as formula (1) and formula (2) or formula (1') and formula (2'):

${\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\mathrm{<span\; class="notranslate">\; \&delta;</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$ ${\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; min</span>}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; 255</span>}}{\mathrm{<span\; class="notranslate">\; max</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; 255</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

or $q_u=\underset{i=1}{\overset{\mathrm{no}}{\mathrm{\&Sigma;}}}\mathrm{\&delta;}[b\left(x_i\right)-u]---\left(1^{\&prime;}\right)$ $p_u=\min (\frac{255}{\max \left(q_u\right)}{q}_u,255)---\left(2^{\&prime;}\right)$

A color feature probability distribution map can be created by the following methods:

First, extract the R (Red), G (Green) and B (Blue) channels from the RGB (Red, Green, Blue) image, then convert the RGB image into an HSV (Hue, Saturation, Value) image, and extract Hue (Hue) channel and Saturation (Saturation) channel, use the histogram back-projection (Histogram Back-Projection) to calculate the probability distribution of hue and saturation of pixels in the tracking window, red probability distribution, green probability distribution, blue probability distribution , such as formula (1) or formula (1′).

Second, use formula (2) or formula (2') to re-value the value ranges in the probability distribution of hue and saturation, red probability distribution, green probability distribution, and blue probability distribution, so that the value range is from [0, max(q _u(v) )] or [0, max(q _u )] projected to [0, 255].

Third, according to certain rules, select suitable features among the hue and saturation features, red features, green features, and blue features as the color features of the visual tracking algorithm to form the final color probability distribution map p(x, y).

Among the above four features, the method of the present invention preferably dynamically selects one or more features that can best reflect the difference between the target area and the background area, the method is as follows:

For feature k, let i be the value of feature k, H ₁ ^k (i) represents the histogram of feature values in target region A, H ₂ ^k (i) represents the histogram of feature values in background regions B and C, p _k (i) is the discrete probability distribution of the target area A, q _k (i) is the discrete probability distribution of the background areas B and C, L _i ^k is the logarithmic likelihood of feature k, as in formula (10), take δ> 0, which is mainly to prevent the denominator of formula (10) from being 0 or log0. var(L _i ^k ; p _k (i) is the variance of L _i ^k relative to the target class distribution p _k (i), such as formula (11), var(L _i ^k ; q _k (i)) is the relative background class The variance of L _i ^k of the distribution q _k (i), such as formula (12), var(L _i ^k ; R _k (i)) is the variance of L _i ^k relative to the target and background class distribution, such as formula (13) , V(L; p _k (i), q _k (i)) is the variance of L _i ^k , such as formula (14), V(L; p _k (i), q _k (i)) represents the feature k The ability to separate the target from the background, V(L; p _k (i), q _k (i)), the larger the feature k, the easier it is to separate the target from the background, the more reliable this feature is, the more suitable it is to track the target Characteristics.

${\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; log</span>}\frac{\mathrm{<span\; class="notranslate">\; max</span>}<span\; class="notranslate">\; \{</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; \&delta;</span>}<span\; class="notranslate">\; \}</span>}{\mathrm{<span\; class="notranslate">\; max</span>}<span\; class="notranslate">\; \{</span>{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; \&delta;</span>}<span\; class="notranslate">\; \}</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; )</span>$

$\mathrm{<span\; class="notranslate">\; var</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ;</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span>{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; -</span>{<span\; class="notranslate">\; [</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 11</span>}<span\; class="notranslate">\; )</span>$

$\mathrm{<span\; class="notranslate">\; var</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ;</span>{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span>{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; -</span>{<span\; class="notranslate">\; [</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 12</span>}<span\; class="notranslate">\; )</span>$

$\mathrm{<span\; class="notranslate">\; var</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ;</span>{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; -</span>{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; -</span>{<span\; class="notranslate">\; [</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 13</span>}<span\; class="notranslate">\; )</span>$

where R _k (i)=[p _k (i)+q _k (i)]/2;

$\mathrm{<span\; class="notranslate">\; V</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ;</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; var</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ;</span>{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; var</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ;</span>{\mathrm{<span\; class="notranslate">\; p</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; var</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; L</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; ;</span>{\mathrm{<span\; class="notranslate">\; q</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 14</span>}<span\; class="notranslate">\; )</span>$

During the video tracking process, the reliability of the hue and saturation characteristics, R (Red) channel characteristics, G (Green) channel characteristics and B (Blue) channel characteristics are constantly detected and judged. Equation (14) calculates their reliability, and rearranges them according to the size of the reliability, and takes the W features with the largest V(L _i ^k ; p _k (i), q _k (i)) as the color features of the tracking target. The W value is preferably 1, 2 or 3.

location feature

Regarding the position feature, the present invention calculates the gray level difference of each point in the two frames of images by using the frame difference, and then determines which pixel points are moving points by setting a threshold, and the moving points are those exceeding the threshold. If you only rely on experience to set the frame difference threshold, it is blind and only applicable to certain occasions. For this reason, the present invention preferably uses the Otsu method to dynamically determine the frame difference threshold F. The basic idea of the Otsu algorithm is to find the appropriate threshold F to minimize the intra-class scatter moment, which is equivalent to finding an appropriate threshold F to maximize the inter-class scatter moment, that is, to divide the frame difference image into two categories by the frame difference threshold F, so that The variance of the two classes that are divided into is the largest. The intra-class scatter moments represent the scatter of each sample point around their mean, and the inter-class scatter moments represent the scatter among different classes. The smaller the inter-class scatter moment means that the interior of each type of sample is tighter, and the larger the inter-class scatter moment means the better the separability between various types of samples.

Motion Continuity Features

Regarding the feature of motion continuity, the present invention estimates the speed of the tracking target through the images of the previous frames, and then estimates the center position of the target at the current moment from the target position obtained by tracking the images of the previous frames. In a short period of time (between video frames), the movement of the target has a strong continuity, and the speed of the target can be regarded as constant. Therefore, the speed of the tracking target can be estimated through the images of the previous frames, and then The target center position at the current moment is estimated from the target position tracked by the previous frames of images.

X(t, row)＝X(t-1, row)±(X(t-1, row)-X(t-2, row)) (3)

X(t,col)＝X(t-1,col)±(X(t-1,col)-X(t-2,col)) (4)

Let X(t, row) represent the row coordinates of the current target center position at time t, such as formula (3), X(t, col) represents the ordinate of the current target center position at time t, such as formula (4), row is the image The maximum number of lines, col is the maximum vertical number of the image, taking into account the continuity of the target movement, by using a linear predictor to predict the current position. So the relationship between X(t, row), X(t-1, row) and X(t-2, row) is as in formula (5), X(t, col), X(t-1, col) and X The relationship between (t-2, col) is as formula (6).

X(t, row) ∈ [max(X(t-1, row)-(X(t-1, row)-X(t-2, row)), 1), min(X(t-1, row)+(X(t-1, row)-X(t-2, row)), rows)] (5)

X(t, col) ∈ [max(X(t-1, col)-(X(t-1, col)-X(t-2, col)), 1), min(X(t-1, col)+(X(t-1, col)-X(t-2, col)), cols)] (6)

Assuming that the line width of the tracking window is width, and the vertical length is length, then the line coordinate of the target at the current moment is as in formula (7), and the ordinate is as in formula (8), that is, the target is within the range of this rectangle.

Y(t, row) ∈ [max(X(t, row)-width, 1), min(X(t, row)+width, rows)] (7)

Y(t, col) ∈ [max(X(t, col)-length, 1), min(X(t, col)+length, cols)] (8)

Let B'(x, y, t) represent the probability distribution map of motion continuity features, such as formula (9), where (x, y, t) represents the pixel at the coordinate (x, y) at time t, and 1 represents the pixel being Tracking target, 0 means background pixel.

${\mathrm{<span\; class="notranslate">\; B</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\mathrm{<span\; class="notranslate">\; 1</span>}& <span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; row</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&cap;</span>\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; col</span>}<span\; class="notranslate">\; )</span>\\ \mathrm{<span\; class="notranslate">\; 0</span>}& <span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&NotElement;</span>\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; row</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&cap;</span>\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; col</span>}<span\; class="notranslate">\; )</span>\end{array}\right.<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; )</span>$

cue fusion

Suppose P _k (row, colu, t) is the probability distribution of pixel (row, colu) obtained by feature k at time t, which represents the probability that each pixel (row, colu) belongs to the target area under feature k. P(row, colu, t) represents the final probability distribution after fusion of W+2 features (W color features, a predicted target position feature and a motion continuity feature) at time t, which characterizes each pixel ( row, colu) belongs to the probability of the target area, such as formula (15). W color features use the reliability of the previous frame as the basis for competition evaluation. If a feature is highly reliable, it will dominate the visual tracking system and provide more information to the tracking system; when the reliability is low, Information will be underutilized or ignored.

$\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; row</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; colu</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; W</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 2</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; r</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; row</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; colu</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 15</span>}<span\; class="notranslate">\; )</span>$

where r _k is the weight of feature k, r ₁ , r ₂ , ..., r _k is the selected color feature, r _w+1 is the weight of the predicted target position feature, r _w+2 is the motion continuity feature weights, $\underset{k=1}{\overset{W+2}{\mathrm{\&Sigma;}}}{r}_k=1,$ In order to project the value ranges of P _w+1 (row, colu, t) and P _w+2 (row, colu, t) to [0, 255], take P _w+1 (row, colu, t) = B (x, y, t)*255, P _w+2 (row, colu, t)=B'(x, y, t)*255.

Compared with the prior art, the present invention is simple and effective, does not need to assume a background model, and does not need to train video sequences without moving objects in advance. Good tracking performance, especially for video sequences where the color saturation of the target environment is low and the target is partially occluded.

The visual tracking by using the present invention can be used not only as a tracking result, but also as an intermediate result of the next step of visual understanding. The present invention has wide application prospects in the field of information, and can be used in human-robot interaction technology (Human Robot Interaction, HRI for short), visual intelligent monitoring, intelligent robots, virtual reality technology, image coding based on models, and content retrieval of streaming media. and other fields have been applied. Here we only take the visual intelligent monitoring system as an example. It covers security, transportation, fire protection, military industry, communication and other fields. The video monitoring system has been used in community safety monitoring, fire monitoring, traffic violations, flow control, military and banking Safety precautions in shopping malls, airports, subways and other public places. Existing video surveillance systems usually only record video images, which are used as evidence after the event, and do not give full play to the role of real-time and active monitoring. If the existing video surveillance system is improved into an intelligent video surveillance system, the surveillance capability can be greatly enhanced, the hidden danger of insecurity can be reduced, and at the same time, manpower, material resources and investment can be saved. There are two problems that the video intelligence system can solve: one is to free the security operators from the complicated and boring task of "staring at the screen", and let the machine complete this part of the work; The image you want to find is to track the target. For example, Beijing Metro Line 13 uses video analysis to catch thieves; Pudong Airport, Capital Airport and many railway projects that are already under construction are expected to use video analysis technology. The invented visual tracking method is one of the core and key technologies of these video analysis technologies.

Description of drawings

Fig. 1 is a schematic diagram of fusion clues of the method of the present invention;

Fig. 2 is a schematic diagram of the tracking window of the present invention, A is the target area, B and C are the background areas;

Figures 3-5 are schematic diagrams of visual tracking for 50 frames, 100 frames and 120 frames of a video sequence with a resolution of 640*512, in which figure a represents the probability distribution map of motion continuity features, and b represents the probability distribution map of position features. c represents the total probability distribution map, and d represents the current frame tracking result map.

Figures 6a-d are visual tracking results for 50 frames, 90 frames, 120 frames and 164 frames of a video sequence with a resolution of 640*480, respectively.

Detailed ways

Embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. The protection scope of the present invention is not limited to the following examples.

The visual tracking of the present embodiment is carried out according to the following steps:

First, the tracking window is set in the first frame of the video sequence. The length and width of the tracking window are determined by the operator according to the size of the tracked target and remain unchanged during the tracking process. Divide the tracking window into three parts, the middle part (A) is the target area, and the left and right (B and C) are the background areas, as shown in Figure 2.

Second, from the second frame, select the most reliable 2 (W=2) color features (for example, R channel and B channel) according to the previous frame, and calculate the color feature probability distribution map M ₁ .

Thirdly, the location feature probability distribution map M ₂ is calculated.

Fourth, the motion continuity feature probability distribution map M ₃ is calculated.

Fifth, the three probability distribution diagrams (M ₁ , M ₂ , M ₃ ) obtained above are weighted by the corresponding r _k to obtain the final probability distribution diagram M. In this embodiment, the weights of M ₁ -M ₃ are sequentially is 3/7 (the weights of R channel and B channel are 2/7 and 1/7 respectively), 2/7 and 2/7.

Sixth, in the probability distribution map M, the center point coordinates of the tracking window of the current frame are obtained by the CAMSHIFT algorithm. The core process of the CAMSHIFT algorithm includes: calculating the zero-order moment (the following formula 16) and the first-order moment (the following formula 17 and 18), iteratively calculate (x, y) coordinates through formula (19), formula (20) until the coordinates have no obvious displacement (the change of the value of x and y is less than 2) or when the maximum number of iterations is 15 times The coordinates are the center of the tracking window of the current frame.

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; p</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 16</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; xp</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 17</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; x</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; the\; y</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; yp</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 18</span>}<span\; class="notranslate">\; )</span>$

$\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 10</span>}}}{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 19</span>}<span\; class="notranslate">\; )</span>$

$\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 01</span>}}}{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 00</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 20</span>}<span\; class="notranslate">\; )</span>$

Figures 3-5 are schematic diagrams of visual tracking for 50 frames, 100 frames and 120 frames of a video sequence with a resolution of 640*512, respectively.

Fig. 6 is a visual tracking result diagram of 50 frames, 90 frames, 120 frames and 164 frames of a video sequence with a resolution of 640*480. Although the saturation of this video sequence 2 is low, the tracking goal is still achieved considering the reliability of color features and multi-cue fusion.

Claims (6)

1. A visual tracking method based on multi-cue fusion comprises the following steps:

a) determining a tracking window in a first frame of a video sequence, wherein the tracking window comprises a target area and a background area, and the target area contains a tracked object;

b) for each frame from the second frame, obtaining a color feature probability distribution map, a position feature probability distribution map and a motion continuity feature probability distribution map of the previous frame; the color features in the color feature probability distribution map include one or more of hue and saturation features, R channel features, G channel features, and B channel features;

c) weighting and adding the three probability distribution maps to obtain a total probability distribution map;

d) and obtaining the center point coordinate of the tracking window of the current frame in the total probability distribution map through a CAMSHIFT algorithm.

2. The visual tracking method of claim 1, wherein the tracking window is a rectangle equally divided into three parts, the middle part being the target region and the two parts being the background region.

3. The visual tracking method of claim 1 wherein V (L; p) of the hue and saturation characteristics, R channel characteristics, G channel characteristics, and B channel characteristics is calculated by_k(i)，q_k(i) Value), the color features in the color feature probability distribution map including the previous, two, or three features for which the V value is the greatest:

$V(L_i^k;p_k\left(i\right),q_k\left(i\right))=\frac{\mathrm{var}(L_i^k;R_k\left(i\right))}{\mathrm{var}(L_i^k;p_k\left(i\right))+\mathrm{var}(L_i^k;q_k\left(i\right))},$ wherein,

p_k(i) representing a discrete probability distribution of the target region;

q_k(i) a discrete probability distribution representing the background region;

where δ is used to ensure that no cases occur where the denominator is 0 or log 0;

R_k(i)＝[p_k(i)+q_k(i)]/2；

wherein E represents the mean and var represents the variance;

<math><mrow><mi>var</mi><mrow><mo>(</mo><msubsup><mi>L</mi><mi>i</mi><mi>k</mi></msubsup><mo>;</mo><msub><mi>q</mi><mi>k</mi></msub><mrow><mo>(</mo><mi>i</mi><mo>)</mo></mrow><mo>)</mo></mrow><mo>=</mo><mi>E</mi><mo>[</mo><msubsup><mi>L</mi><mi>i</mi><mi>k</mi></msubsup><mo>*</mo><msubsup><mi>L</mi><mi>i</mi><mi>k</mi></msubsup><mo>]</mo><mo>-</mo><msup><mrow><mo>(</mo><mi>E</mi><mo>[</mo><msubsup><mi>L</mi><mi>i</mi><mi>k</mi></msubsup><mo>]</mo><mo>)</mo></mrow><mn>2</mn></msup><mo>=</mo><munder><mi>&Sigma;</mi><mi>i</mi></munder><msub><mi>q</mi><mi>k</mi></msub><mrow><mo>(</mo><mi>i</mi><mo>)</mo></mrow><mo>*</mo><msubsup><mi>L</mi><mi>i</mi><mi>k</mi></msubsup><mo>*</mo><msubsup><mi>L</mi><mi>i</mi><mi>k</mi></msubsup><mo>-</mo><msup><mrow><mo>[</mo><munder><mi>&Sigma;</mi><mi>i</mi></munder><msub><mi>q</mi><mi>k</mi></msub><mrow><mo>(</mo><mi>i</mi><mo>)</mo></mrow><mo>*</mo><msubsup><mi>L</mi><mi>i</mi><mi>k</mi></msubsup><mo>]</mo></mrow><mn>2</mn></msup><mo>;</mo></mrow></math>

<math><mrow><mi>var</mi><mrow><mo>(</mo><msubsup><mi>L</mi><mi>i</mi><mi>k</mi></msubsup><mo>;</mo><msub><mi>R</mi><mi>k</mi></msub><mrow><mo>(</mo><mi>i</mi><mo>)</mo></mrow><mo>)</mo></mrow><mo>=</mo><mi>E</mi><mo>[</mo><msubsup><mi>L</mi><mi>i</mi><mi>k</mi></msubsup><mo>*</mo><msubsup><mi>L</mi><mi>i</mi><mi>k</mi></msubsup><mo>]</mo><mo>-</mo><msup><mrow><mo>(</mo><mi>E</mi><mo>[</mo><msubsup><mi>L</mi><mi>i</mi><mi>k</mi></msubsup><mo>]</mo><mo>)</mo></mrow><mn>2</mn></msup><mo>=</mo><munder><mi>&Sigma;</mi><mi>i</mi></munder><msub><mi>R</mi><mi>k</mi></msub><mrow><mo>(</mo><mi>i</mi><mo>)</mo></mrow><mo>*</mo><msubsup><mi>L</mi><mi>i</mi><mi>k</mi></msubsup><mo>*</mo><msubsup><mi>L</mi><mi>i</mi><mi>k</mi></msubsup><mo>-</mo><msup><mrow><mo>[</mo><munder><mi>&Sigma;</mi><mi>i</mi></munder><msub><mi>R</mi><mi>k</mi></msub><mrow><mo>(</mo><mi>i</mi><mo>)</mo></mrow><mo>*</mo><msubsup><mi>L</mi><mi>i</mi><mi>k</mi></msubsup><mo>]</mo></mrow><mn>2</mn></msup><mo>.</mo></mrow></math>

4. the visual tracking method of claim 1, wherein the location feature probability distribution map is obtained by: and calculating the gray difference value of each pixel point of the tracking window in the current frame and the previous frame, wherein if the difference value is greater than a set threshold value, the pixel point is a motion point, and the position characteristic probability distribution map comprises all the motion points.

5. The visual tracking method of claim 4, wherein the threshold is dynamically determined by an Otsu method.

6. The visual tracking method of claim 1, wherein when the three probability distribution maps are weighted and added to obtain the total probability distribution map, the sum of the weights of the respective probability distribution maps is 1.

Images