CN101398896B - Device and method for extracting color features with strong discriminative power for imaging device - Google Patents

Abstract

Provided are a method and device for extracting color features with strong discrimination. The method includes: dividing the object area and non-object area in the input image into multiple rectangles; establishing the color histogram of the object rectangle and the non-object rectangle, each color histogram is divided into multiple discrete intervals; according to the color histogram Figure extracts the main color of the rectangle; calculates the minimum distance between the color histogram interval of the object rectangle and the color histogram interval of the non-object rectangle for the extracted main color; calculates the weight of the color histogram interval of the object rectangle based on the minimum distance; based on Calculate the weight of the color component of the object rectangle based on the weight of the color histogram interval of the object rectangle; calculate the weight of the object rectangle based on the weight of the color histogram interval of the object rectangle; calculate the weight of the object rectangle based on the weight of the object rectangle and the weight of the color component The color histogram bins are reweighted; based on the reweighting results a final color model of the object is produced.

Description

Device and method for extracting color features with strong discriminative power for imaging device

technical field

The present invention relates to pattern recognition, feature extraction, statistical learning, and object detection techniques (such as human detection techniques), and more particularly, relates to the extraction of highly discriminative features of an object's color appearance or reveals features related to other objects of the same or different kind. Objects are relatively hidden color distribution patterns with strong distinguishing ability. The present invention also relates to a device and method for extracting color features with strong distinguishing power and an imaging device using the device and method.

Background technique

Representative color features of object appearance are widely used for object detection and tracking in video frames. When representing color features, a color histogram is usually used. To capture the spatial layout of the color distribution, a histogram of independent subregions divided from the object region is suitable. In addition to this method, there are additional methods (such as motion information, shape information, geometric constraints, etc.) to compensate the color model to obtain better detection results.

Various attempts have been made to solve the problem of tracking coherent moving objects using color models. Examples of these attempts include: a method for probabilistic tracking using color appearance (this method is disclosed in Computer Vision European Conference 2002 by P. method); color histogram modeling and probability map detection method (this method is disclosed in US Patent No. 5845009A1, hereinafter referred to as the 009 patent); based on the multi-search window method of color model comparison similarity (this method is disclosed in 2007127775A1 No. US Patent, hereinafter referred to as the 775 patent); a method for extracting skin color based on a statistical color model (this method is disclosed in US Patent No. 2004017938A1, hereinafter referred to as the 938 patent).

Representative color features of object appearance are widely used for object detection in video frames. When expressing color features, a color histogram is generally used (see Perez method, 009 patent, 775 patent, and 938 patent). Given the position and size of a moving object, the color histogram of the moving object can be calculated. In subsequent video frames, a similarity map for this histogram model is computed. Then, connected-domain (blob) analysis techniques are able to cluster pixel regions with high similarity to the histogram model into connected blobs indicating the locations of moving objects with high probability. Many methods focus on how to effectively represent the color features of moving objects. In fact, the important issue is how to distinguish moving objects from non-moving objects (ie, other objects of the same or different kind as well as the background), rather than being perfectly faithful to the original color distribution of the target. For moving objects, it consists of multiple parts. For example, assuming that the moving object is a person, the person includes a face/head, a clothed upper body, and a clothed lower body. The color appearance of people may appear similar to other image areas, especially in environments with cluttered backgrounds. In this case, it is necessary to select color regions with strong discrimination from the human body as color features instead of selecting representative color features of the entire human body.

Contents of the invention

A color feature with strong discriminative power means that there is a large color difference between the moving object itself and other image regions. In order to extract color features with strong discrimination, the color vectors in the color histogram of the object divided into several blocks are compared with the color vectors in the color histograms of other image regions, and the color vectors close to the color vectors in other image regions The importance of the color vector of the moving object is reduced, and the importance of the other color vector is increased.

Therefore, the present invention provides a method and device for extracting color features with strong discrimination for different situations of imaging and scenes and providing focus areas for dynamic adjustment of imaging units.

According to one aspect of the present invention, there is provided a method for extracting color features with strong discrimination, the method comprising the following steps: dividing the object area and non-object area in the input image into a plurality of rectangles; using three color channels to establish Color histograms of object rectangles and non-object rectangles, where each color histogram is divided into multiple discrete intervals; extract the main color of the rectangle according to the color histogram; calculate the color histogram interval of the object rectangle for the extracted main colors The minimum distance from the color histogram bins of non-object rectangles; the weights of the color histogram bins of object rectangles are calculated based on the calculated minimum distance; the weights of the color components of object rectangles are calculated based on the weights of the color histogram bins of object rectangles calculated weight; calculating the weight of the object rectangle based on the calculated weight of the color histogram interval of the object rectangle; reweighting the color histogram interval of the object rectangle based on the calculated weight of the object rectangle and the weight of the color component, thereby extracting the object's Discriminatory color features; reweighting results based on the color histogram bins of the object rectangle yields the final color model of the object.

The method may further include: extracting the color connected domain of the object from the new input image based on the final color model of the object; performing connected domain analysis on the extracted color connected domain, and calculating the shape of the object in the input image based on the connected domain analysis. center and size to locate and track the object.

According to another aspect of the present invention, there is provided a device for extracting color features with strong discrimination, which includes: an area division unit, which divides the object area and non-object area in the input image into a plurality of rectangles; histogram calculation The unit uses the color channel to establish the color histogram of the object rectangle and the non-object rectangle, wherein each color histogram is divided into multiple discrete intervals; the main color extraction unit extracts the object according to the color histogram established by the histogram calculation unit The main color of the rectangle; the interval minimum distance calculation unit, for the main color extracted by the main color extraction unit, calculates the minimum distance between the color histogram interval of the object rectangle and the color histogram interval of the non-object rectangle; the interval weight calculation unit, based on the interval The minimum distance calculated by the minimum distance calculation unit calculates the weight of the color histogram interval of the object rectangle; the color component weight calculation unit calculates the weight of the color component of the object rectangle based on the weight of the color histogram interval of the object rectangle calculated by the interval weight calculation unit; The rectangle weight calculation unit calculates the weight of the object rectangle based on the weight of the color histogram interval of the object rectangle calculated by the interval weight calculation unit; the interval reweighting unit calculates the weight of the object rectangle based on the rectangle weight calculation unit and the color component weight calculation unit The weight of the color component of the object rectangle is reweighted to the color histogram interval of the object rectangle, thereby extracting the color feature with strong recognition power of the object; the final color model generation unit is based on the reweighted result of the color histogram interval of the object rectangle. The final color model of the object.

The device may further include: a color connected domain extraction unit, based on the final color model of the object, extracting the color connected domains of the object from a new input image; an object positioning unit, performing connected domain analysis on the color connected domains extracted by the color connected domain extraction unit, The centroid and size of the object in the input image are calculated based on the connected domain analysis, so as to locate and track the object.

According to another aspect of the present invention, there is provided an imaging device, which includes: an imaging unit that takes an image of an object; according to the device for extracting a color feature with strong discrimination according to the present invention, receives the image from the imaging unit, and extracts the object The color feature with strong recognition power generates the final color model of the object, extracts the color connected domain of the object based on the final color model, performs connected domain analysis, and generates parameters for adjusting the posture of the imaging device based on the connected domain analysis, to obtain Better image quality; the control unit receives parameters for adjusting the posture of the imaging device from the device for extracting color features with strong recognition power, and adjusts the posture of the imaging device, wherein the posture of the imaging device can be used for the swing of the PTZ camera Selection of angle, tilt angle, zoom ratio, and focus area, or some of the above, for example, zoom ratio and selection of focus area for a digital camera; storage unit, which stores an image of a photographed object; display unit, which displays a photographed image of the object.

The imaging device may further include: a labeling unit, which provides an object area manually marked by a user on the image to the device for extracting color features with strong recognition power.

The control unit may adjust at least one of operations of panning, tilting, zooming, and selecting a focus area of the imaging device according to parameters for adjusting the attitude of the imaging device.

In the operation of selecting the focus area, the control unit controls the imaging device to select a new area where the object is located as a basis for focusing, so as to focus on the area.

When the control unit controls the imaging device to select the focus area, the imaging device selects the central area of the image as the default imaging focus area, or dynamically selects a new image area where the object is located as the imaging focus area, and dynamically Adjust the zoom factor, focus, swing or tilt parameters of the imaging device accurately.

Description of drawings

These and/or other aspects and advantages of the present invention will become clear and easier to understand from the description of the following embodiments in conjunction with the accompanying drawings, wherein:

Fig. 1 has shown the flow chart of the method for extracting the color feature with strong discrimination according to the present invention;

Figure 2 shows how the person rectangle is calculated;

Figure 3 shows the calculation of the histogram for each rectangle;

Fig. 4 shows the calculation of interval weights and the effect of using the color extraction with strong discrimination of the present invention;

Figure 5 shows the importance weighting function;

Figure 6 shows color-connected domain detection using color features with strong discriminative power;

Figure 7 shows a connected domain analysis for locating a person's location and size;

Fig. 8 shows the block diagram of the device for extracting color features with strong discrimination according to the present invention;

Fig. 9 shows a block diagram of an imaging device according to the present invention.

Detailed ways

Reference will now be made in detail to embodiments of the invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like parts throughout. The embodiments are described below in order to explain the present invention by referring to the figures.

In the following, the present invention is described by taking a human as an example of a moving object, but the present invention is not limited thereto, and may also be other types of moving objects.

FIG. 1 shows a flowchart of a method for extracting color features with strong discrimination according to the present invention. In step 101, the system receives video input, person's position and size information given by motion extraction, manual annotation or human body detector (as disclosed in US20060147108A1). There are various motion extraction methods, for example, background subtraction can be used due to the stationary state of the imaging device in its initial state. Manual annotation can directly give the position and size information of the person. A human detector detects the position and size of an input image for candidate persons. Although people have different sizes in the image, the aspect ratio has a fixed range, which can be used to safely determine other image regions except moving objects (i.e., non-object regions, including other objects of the same or different kind and background). The next step is to extract color regions with strong discriminative power based on known human and non-human rectangular image regions. In step 102, the color image is quantized, and color pixels with 3 color channels are quantized into N discrete intervals (bins), for example, N=64.

In step 103, the rectangles of the human head and human body are calculated. Figure 2 shows the calculation of the rectangles of the human head and body. For example, when a person's head position and size are given, several rectangles with the same size will be arranged down until the lower boundary of the image is reached. As prior knowledge, the human face and human hair have distinct color appearances, and the head rectangle is divided into several small rectangles. Following the same principle, other image regions than those occupied by people can be safely estimated based on statistical people's height (head) to width ratios. It is not necessary to accurately estimate the background rectangle area, that is, the background rectangle area can be roughly estimated to ensure that no image pixels of moving objects are incorrectly classified as the background rectangle.

In step 104, a histogram of rectangles is calculated. Figure 3 shows the calculation of the color histogram for each rectangle, which includes the human head sub-rectangle, human body rectangle and background rectangle (or other rectangles except the given human rectangle, i.e., non-object area) . The histograms of these rectangles are calculated separately, the histogram is divided into bins, and the number of pixels is accumulated in each bin.

In step 105, after obtaining the color histogram of each rectangle, the dominant colors are kept and the subtle colors are discarded. The subtle colors are unstable or unreliable because noise can affect the subtle colors with a small number of pixels having the subtle colors.

In step 106, the interval minimum distance is calculated.

The distance between the rectangular color histogram bins of objects and non-objects is computed as shown in equations (1) and (2). The following takes the background area as an example of the non-object area.

$\mathrm{<span\; class="notranslate">\; J</span>}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; B</span>}{\mathrm{<span\; class="notranslate">\; G</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; j</span>}}{\mathrm{<span\; class="notranslate">\; arg</span>}\mathrm{<span\; class="notranslate">\; min</span>}}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; J</span>}}^{<span\; class="notranslate">\; \&prime;</span>}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

Wherein, J represents an interval identification (ID) of the background color distribution of the color channel c, and the interval J has the minimum distance from the interval i of the color channel c of the main color histogram of the human rectangle r. b _i is the identification (ID) of interval i. H _r ^c (i) is the color histogram of interval i of color channel c of human rectangle r. BG ^c (j) is the background color histogram for bin j of color channel c. b _j ' is the ID of section j. D _r ^c (i) is the minimum distance between interval i of color channel c of person rectangle r and the background color histogram interval with the same color channel.

In step 107, interval weights are calculated.

As shown in Equation (3), the discriminative power (weight) of an interval in the rectangle r and the channel c is defined.

${\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; the\; s</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&times;</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; B</span>}{\mathrm{<span\; class="notranslate">\; G</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; J</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

in, $\mathrm{the\; s}=\mathrm{sign}(h_r^c\left(i\right)-B{G}^c\left(J\right))$

T(x, y)=e ^x×k×y , where k is a constant

At step 108, color component weights are calculated.

In each rectangle, a pixel is composed of three color components, which are red, green, and blue elements. Because each element is quantized into multiple histogram bins, these elements have different discriminative powers depending on the distance of the histogram bins they relate to.

${\mathrm{<span\; class="notranslate">\; w</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; W</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; w</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}}{\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; K</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; w</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

In equation (4), the weights of each interval in the color component c are accumulated. Therefore, the weight of each color component can be calculated according to Equation (5). In Equations (4) and (5), M represents the number of bins in the human rectangular histogram, and K represents the number of color channels.

In step 109, rectangle weights are calculated.

The importance of a person rectangle depends on the recognition power of the color intervals included in the person rectangle. When a rectangle covers a highly discriminative color area, it will have a large weight compared to other human rectangles.

${\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; k</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; K</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; =</span>\frac{{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}}{\underset{\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; L</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 7</span>}<span\; class="notranslate">\; )</span>$

In Equation (6), the sum of interval weights in the human rectangle r is calculated. Then, rectangle weights can be calculated according to Equation (7), which weights the importance of these rectangles with high discriminative color intervals, where L represents the number of person rectangles.

At step 110, the histogram bins are reweighted.

${\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; \&times;</span>{\mathrm{<span\; class="notranslate">\; W</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

Each histogram interval has a different discriminative power between the target moving object and the cluttered background or other moving objects. The importance of a histogram interval depends on the initial weight w _r ^c (i), the color component weight W ^c and the rectangle weight W _r of the histogram interval in the histogram. Therefore, the final importance of the histogram is reweighted according to equation (8).

At step 111, a final color model is generated.

${\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; \{</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}^{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}^{\mathrm{<span\; class="notranslate">\; g</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}^{\mathrm{<span\; class="notranslate">\; b</span>}}<span\; class="notranslate">\; \}</span>$

${\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; b</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; =</span>{<span\; class="notranslate">\; \{</span>{{\mathrm{<span\; class="notranslate">\; N</span>}}^{\mathrm{<span\; class="notranslate">\; r</span>}}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}^{\mathrm{<span\; class="notranslate">\; g</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}^{\mathrm{<span\; class="notranslate">\; b</span>}}<span\; class="notranslate">\; \}</span>$

${\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; N</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; )</span>$

After reweighting the histogram bins in each person rectangle, the final color model can be produced according to equations (9) and (10), which represent the Final color model (color histogram distribution). In equations (9) and (10), the weights from the histogram bins in the rectangles of the human head and human torso are accumulated respectively. R is the number of person rectangles, and M is the number of bins of the person rectangle histogram. More specifically, R _h represents the number of human head rectangles, R _t represents the number of human torso rectangles, M _h represents the number of bins in the histogram of human head rectangles, and M _t represents the number of bins in the histogram of human torso rectangles .

To illustrate the main idea later in this section, Figure 4 shows an example. (a) in FIG. 4 is a color histogram of color components in a rectangle of a moving object. (b) in FIG. 4 is a color histogram of color components in other regions of the image than moving objects. In both color histograms there are two bins, denoted by BIN-1 and BIN-2. As the BIN-1 of the moving object is farther away from the two intervals of the background than the BIN-2 of the moving object in the distance transformation (equations (1) and (2)), the weight of the BIN-1 of the moving object increases, while The weight of BIN-2 of moving objects decreases, as shown in (c) in Fig. 4. Although the initial weight of BIN-2 is larger than that of BIN-1, after the transformation with strong discriminative color feature extraction, the new weight of BIN-1 is larger than that of BIN-2. Figure 5 shows the importance weighting function T(x, y), which increases the weights with large interval distances and decreases the weights with small interval distances, which are calculated according to equations (1) and (2) .

Notably, an approach is shown here for the three color channels of a color image, in which the final color model is built independently using a one-dimensional color histogram. But in practice there are variations such as using a two-dimensional color histogram or a three-dimensional color histogram. For example, there are three kinds of two-dimensional color histograms, namely, a red-green two-dimensional color histogram, a red-blue two-dimensional color histogram, and a blue-green two-dimensional color histogram. However, the method of the present invention can also be applied to these histograms in a similar manner, thereby extracting color features with strong discriminative power.

Based on the final color model produced from the extraction of highly discriminative color features, the color-connected domains belonging to moving objects can be extracted from the new input image. For each pixel in the input image, its similarity to the target moving object is formed into a weight matrix whose size is the same as that of the input image. The elements of the weight matrix come from the accumulated weights in the final color histogram model obtained by quantizing the color pixel values.

In Fig. 6, there are two persons. A person is selected as a follower in a certain way, for example, the person on the right is selected as a follower. (a) in Figure 6 is the new input image, (b) in Figure 6 is the result of connected domain detection using the human torso model, and (c) in Figure 6 is the connected domain detection using the human head model result. Since white is widely distributed in the background as well as the other person, the white of the person on the right has a very low weight, which occupies most of the initial weight of the color histogram.

Referring to (a) in FIG. 7, there are two head candidates from a human head detector with a shape model. Therefore, the probability distribution of a human head is a Gaussian distribution centered at a given head position. (b) in FIG. 7 shows the result of the color connected domain detection using the head color model, and (c) in FIG. 7 shows the result of the color connected domain detection using the torso color model. In color-connected domain detection of human heads, the probability distribution of a person is a Gaussian distribution centered at a given head position. Since an up-down pose of the person is assumed, the probability distribution extends downward to indicate the presence of the person's torso. In the color-connected domain detection of the human torso, the probability distribution of the person is also a Gaussian distribution centered on a given torso position. Since the human head is located in the top region of the human torso (which constrains the geometric relationship), the probability distribution extends upwards to indicate the presence of the human head. Therefore, referring to (d) in FIG. 7 , the centroid of the probability map can be calculated as the final position of the tracking object.

FIG. 8 shows a block diagram of a device for extracting color features with strong discrimination according to the present invention.

Referring to Fig. 8, the device for extracting color features with strong discrimination according to the present invention includes an area division unit 801, a histogram calculation unit 802, a main color extraction unit 803, an interval minimum distance calculation unit 804, an interval weight calculation unit 805, a color A component weight calculation unit 806 , a rectangle weight calculation unit 807 , an interval reweighting unit 808 , and a final color model generation unit 809 .

The area dividing unit 801 divides an object area and a non-object area in an input image into a plurality of rectangles.

The histogram calculation unit 802 uses the color channels to establish a color histogram of the object rectangle and the non-object rectangle, wherein the color histogram is divided into a plurality of discrete intervals.

The main color extracting unit 803 extracts the main color of the object rectangle according to the color histogram.

The section minimum distance calculation unit 804 calculates the minimum distance between the color histogram section of the object rectangle and the color histogram section of the non-object rectangle with respect to the main colors extracted by the main color extraction unit 803 . The section minimum distance calculation unit 804 can calculate the minimum distance according to equations (1) and (2).

The section weight calculation unit 805 calculates the weight of the color histogram section of the object rectangle based on the minimum distance calculated by the section minimum distance calculation unit 804 . The interval weight calculation unit 805 may calculate the weight of the color histogram interval of the object rectangle according to Equation (3).

The color component weight calculation unit 806 calculates the weight of the color component of the object rectangle based on the weight of the color histogram interval of the object rectangle calculated by the interval weight calculation unit 805; the color component weight calculation unit 806 can be calculated according to equations (4) and (5). Computes the weights for the color components of the object's rectangle.

The rectangle weight calculation unit 807 calculates the weight of the object rectangle based on the weights of the color histogram sections of the object rectangle calculated by the color component weight calculation unit 806 . The rectangle weight calculation unit 807 can calculate the weight of the object rectangle according to Equations (6) and (7).

The section reweighting unit 808 reweights the color histogram section of the object rectangle based on the weight of the object rectangle calculated by the rectangle weight calculation unit 807 and the weight of the color component calculated by the color component weight calculation unit 806, thereby extracting the color histogram sections of the object with strong Distinguishable color characteristics. The bin reweighting unit 808 may reweight the color histogram bins of the object rectangle according to equation (8).

The final color model generation unit 809 generates the final color model of the object based on the reweighted result of the color histogram interval of the object rectangle. The final color model generation unit 809 can generate the final color model of the object according to the following equation:

N={N ^r , N ^g , N ^b }

${\mathrm{<span\; class="notranslate">\; N</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; R</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; b</span>}}_{\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 11</span>}<span\; class="notranslate">\; )</span>$

Wherein, R represents the number of object rectangles, and M represents the number of intervals of the object rectangle histogram.

In addition, the device for extracting color features with strong discrimination according to the present invention may further include a color connected domain extraction unit 810 and an object positioning unit 811 .

The color connected domain extracting unit 810 extracts the color connected domain of the object from the new input image based on the final color model of the object. The object positioning unit 811 performs connected domain analysis based on the color connected domain extracted by the color connected domain extraction unit 810, and calculates the centroid and size of the object in the input image, so as to locate and track the object.

Fig. 9 shows a block diagram of an imaging device according to the present invention.

The imaging device includes an imaging unit 901 , a device 902 for extracting color features with strong discrimination according to the present invention, a control unit 903 , a storage unit 904 , a display unit 905 and a labeling unit 906 . The imaging device may be any one of a PTZ (PAN, TILT, ZOOM) camera, a still surveillance camera, a DC (Digital Camera), a DV (Digital Video Camera), and a PVR (Personal Video Recorder).

The imaging unit 901 is a hardware device, such as a CCD or a CMOS device, used to perceive and generate natural images, and the image processing chip of the imaging unit can achieve good image quality. In order to track a moving object, there are two methods to provide the location and size of the moving region of the object. The first method is an automatic method, using an embedded algorithm to extract the size and location of the region of the object of interest. The second method is a manual method, where a user or operator marks an area of an object of interest on a displayed image (eg, a touch screen). For automatic methods, use embedded algorithms (such as those disclosed in USP20060147108A1, USP20050276446, or in the paper titled "Multi-view Human Head Detection in Still Images" published at the International Conference on Machine Vision and Applications in 2005 public methods), objects can be detected automatically. The labeling unit 906 provides a labeling function to the user or operator, so that the user or operator can manually label the object area of interest on the image with a pen or finger.

The device 902 for extracting color features with strong discrimination can receive image data from the imaging unit 901, and can also receive the size and position information of the object region of interest marked by the user, for example in the form of a rough mark. The device 902 for extracting color features with strong recognition power extracts the color features with strong recognition power of the object, generates the final color model of the object, extracts the color connected domain of the object based on the final color model, performs connected domain analysis, and based on the connected domain analysis to generate parameters that adjust the pose of the imaging device. It is worth noting that when using the first method of providing object location and size, the labeling unit 906 is optional. When there are multiple tracking objects for selection, for example, there are multiple moving objects to be tracked, the user can modify the tracking object automatically selected by the imaging device in the present invention.

The control unit 903 can adjust the posture of the imaging device. The posture of the imaging device is controlled by the operation of swinging, tilting, zooming and selecting the focus area for automatic focusing of the PTZ camera, or by the zooming operation of the still surveillance camera, DC, DV or PVR. The control unit 903 receives parameters for adjusting the pose of the imaging device from the device 902 for extracting color features with strong discrimination. The device 902 for extracting color features with strong discrimination can provide object position and size information of a new time point or new frame data. The control unit 903 adjusts the posture of the imaging device according to the parameters to center the object in the image through the swing/tilt operation, select the area of the object of interest through the operation of selecting the focus area, and focus on the object through the zoom and auto focus operation. The area of the object of interest is brought into focus to obtain high image quality details of moving objects. In the operation of selecting the focus area, the control unit may control the imaging device to select a new area where the object is located as a basis for focusing, so as to focus on the area. In addition, when the control unit controls the imaging device to select the focus area, the imaging device can not only select the central area of the image as the default imaging focus area, but also dynamically select a new image area where the object is located as the imaging focus area. Image data information, dynamically adjust the zoom factor, auto focus, focal length, swing or tilt parameters of the imaging device, so as to obtain better imaging effects.

For an electronic product held in the user's hand, such as DC, DV or PVR, the user can manually adjust its posture so that the object of interest is centered in the image. At this time, the device itself can automatically perform other operations, such as changing the zoom factor and auto-focusing.

The storage unit 904 can store images or videos in the storage device, and the display unit can display the images or videos on site to the user.

The invention can be implemented as software for an embedded system connected to the imaging device setup and control unit to adjust the parameters of the imaging device pose. For an embedded imaging device system, it can receive video as input and send commands to the imaging device's control unit to adjust the imaging device's attitude, lens focus area, etc.

The present invention does not extract the main color histogram to represent the color appearance of the object, but extracts the color model with strong recognition ability. There is a difference between a representative color model and a color model with strong recognition. Sometimes when the color appearance of moving objects is very different from the non-object and background color models, the color model with strong discrimination is similar to the representative color model. Sometimes discriminating color models are not similar to representative color models, especially when the color appearance of moving objects is partially different from non-object and background color models. The present invention can effectively use a color model with strong discrimination to locate a target object in a cluttered background with interfering color regions similar to moving objects. The invention can distinguish multiple moving objects with partially similar colors from tracking objects, and then locate the position and size of the tracking objects, so as to adjust the imaging device to obtain good imaging conditions. A color model with strong discriminative power uses the principle of an importance weighting strategy. A color model with strong discriminative power increases the importance/weight of color elements that are very different from non-objects or backgrounds, and decreases the importance/weight of color elements that are similar to non-objects or backgrounds. Tracking objects using stationary cameras or PTZ cameras for coherent tracking of moving objects of interest or suspicion, can be used for event analysis and image quality enhancement. In detection using a discriminative color model, the detection scans an input image and transforms the input image into a connected domain graph based on the discriminative color model. This detection can also be used to check human candidates from human detectors to reject false alarms.

In a PTZ camera, the invention can be implemented as software in an embedded system that receives an input video image, analyzes the input image for use in locating a person after building a highly discriminative color model of the input image , adjust the attitude of the PTZ camera (swing, tilt zoom, select the focus area) to track the person, so that the person is located in the center of the image, with good resolution and image quality. When the camera swings and tilts, the moving person is tracked to avoid people leaving the camera's field of view. When the camera is optically zoomed, adjust the optical focus to obtain high-quality images of local details of moving people or the entire body. The video recording system can save the field data in the storage device, and the video recording system records human behavior with high image quality when entering the area set by the user, or performs interactive operation with the machine.

It is worth noting that the invention can also be used in portable imaging devices (eg DC, DV or mobile cameras). The initial position and size of the person can be manually marked by the user, or automatically detected by the detector. By positioning the moving person and adjusting the focus directly, it can automatically focus on the moving person to obtain high image quality. To center the moving person in the image, the end user can manually adjust the lens orientation of the imaging device he/she is holding.

In still cameras, the invention can be implemented with a collection of trajectories of a moving person with coherent tracking of the moving person. The trajectory includes the position and size of the moving person. During tracking, the system can analyze whether certain events occurred, or interactively issue some alerts or notifications. A user-defined event can be a dangerous situation, or a general notification situation to the user, for example, the position or size of a person entering a certain pre-set forbidden area as the movement of an intruder.

The term "imaging device" used in the present invention refers to a device having at least an imaging unit and a control unit, which may be a video camera, a camera or other portable devices with an imaging function.

The invention can also be used in robots that use robotic imaging devices to localize people. The basic functions of the robot include avoiding obstacles, interacting with people, and tracking people. The present invention can be used to discover whether there is a person in a nearby area, or to locate the location of a person for tracking or interaction with the person.

The present invention can also be used for the color feature extraction and detection with strong discrimination of other kinds of moving objects, and the human is taken as an example for the convenience of explanation. Candidate target applications are not limited to human modeling and detection. For example, for using DC to image a moving object, the end user can draw a line on the touch screen with hand or pen to manually mark the object area of interest to indicate the object position, size or area. The present invention can establish a highly discriminative color model of an object and reveal the position and size of the object in the next video frame. Therefore, DC can adjust imaging parameters to obtain high image quality of moving objects.

While the invention has been particularly described and shown with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that modifications may be made thereto without departing from the spirit and scope of the invention as defined by the claims. Various changes in form and detail.

Claims (21)

1. A method of extracting color features with strong discriminatory power, comprising the steps of:

dividing an object region and a non-object region in an input image into a plurality of rectangles;

establishing color histograms of an object rectangle and a non-object rectangle using color channels, wherein each color histogram is divided into a plurality of discrete intervals;

extracting the main color of the rectangle according to the color histogram;

calculating a minimum distance between a color histogram section of the object rectangle and a color histogram section of the non-object rectangle for the extracted main color;

calculating a weight of a color histogram section of the object rectangle based on the calculated minimum distance;

calculating weights of color components of the target rectangle based on the calculated weights of the color histogram sections of the target rectangle;

calculating the weight of the target rectangle based on the calculated weight of the color histogram interval of the target rectangle;

re-weighting the color histogram interval of the object rectangle based on the calculated weight of the object rectangle and the weight of the color component, thereby extracting the color feature with strong identification power of the object;

a final color model of the object is generated based on the result of the re-weighting of the color histogram bins of the object rectangle.

2. The method of claim 1, wherein the minimum distance between the color histogram bin of the object rectangle and the color histogram bin of the non-object rectangle is calculated according to the following equation:

<math> <mrow> <mi>J</mi> <mo>[</mo> <msubsup> <mi>H</mi> <mi>r</mi> <mi>c</mi> </msubsup> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> <mo>,</mo> <msup> <mi>BG</mi> <mi>c</mi> </msup> <mrow> <mo>(</mo> <mi>j</mi> <mo>)</mo> </mrow> <mo>]</mo> <mo>=</mo> <munder> <mrow> <mi>arg</mi> <mi>min</mi> </mrow> <mi>j</mi> </munder> <mo>|</mo> <msub> <mi>b</mi> <mi>i</mi> </msub> <mo>-</mo> <msubsup> <mi>b</mi> <mi>j</mi> <mo>&prime;</mo> </msubsup> <mo>|</mo> </mrow> </math>

<math> <mrow> <msubsup> <mi>D</mi> <mi>r</mi> <mi>c</mi> </msubsup> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> <mo>=</mo> <mo>|</mo> <msub> <mi>b</mi> <mi>i</mi> </msub> <mo>-</mo> <msubsup> <mi>b</mi> <mi>J</mi> <mo>&prime;</mo> </msubsup> <mo>|</mo> </mrow> </math>

where J represents a bin of the color histogram of the non-target rectangle of the color channel c, said bin J having a minimum distance to bin i of the color channel c of the main color histogram of the target rectangle r, b_iIs the identification of the interval i and,is the color histogram, BG, of the interval i of the color channel c of the object rectangle r^c(j) Is the non-object color histogram of bin j of color channel c,

is the identification of the interval j and,

is the minimum distance between the bin i of the color channel c of the object rectangle r and the non-object rectangle color histogram bin with the same color channel.

3. The method according to claim 2, wherein the weight of the color histogram interval of the object rectangle is calculated according to the following equation

<math> <mrow> <msubsup> <mi>w</mi> <mi>r</mi> <mi>c</mi> </msubsup> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> <mo>=</mo> <mi>T</mi> <mrow> <mo>(</mo> <msubsup> <mi>D</mi> <mi>r</mi> <mi>c</mi> </msubsup> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> <mo>,</mo> <mi>s</mi> <mo>)</mo> </mrow> <mo>&times;</mo> <mo>|</mo> <msubsup> <mi>H</mi> <mi>r</mi> <mi>c</mi> </msubsup> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> <mo>-</mo> <msup> <mi>BG</mi> <mi>c</mi> </msup> <mrow> <mo>(</mo> <mi>J</mi> <mo>)</mo> </mrow> <mo>|</mo> </mrow> </math>

Wherein,

T(x，y)＝e^x×k×yand k is a constant.

4. The method according to claim 3, wherein the weight W of the color component of the object rectangle is calculated according to the following equation^c：

<math> <mrow> <msup> <mi>w</mi> <mi>c</mi> </msup> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>m</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <msubsup> <mi>w</mi> <mi>r</mi> <mi>c</mi> </msubsup> <mrow> <mo>(</mo> <mi>m</mi> <mo>)</mo> </mrow> </mrow> </math>

<math> <mrow> <msup> <mi>W</mi> <mi>c</mi> </msup> <mo>=</mo> <mfrac> <msup> <mi>w</mi> <mi>c</mi> </msup> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>K</mi> </munderover> <msup> <mi>w</mi> <mi>k</mi> </msup> </mrow> </mfrac> </mrow> </math>

Where K represents the number of color channels and M represents the number of bins in the object rectangle histogram, in equation

The sum of the weights of each interval in the color component c of the object rectangle r is calculated in equation

The weight of the color component c is calculated.

5. The method of claim 4, wherein the weight W of the object rectangle is calculated according to the following equation_r：

<math> <mrow> <msub> <mi>w</mi> <mi>r</mi> </msub> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>K</mi> </munderover> <munderover> <mi>&Sigma;</mi> <mrow> <mi>m</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <msubsup> <mi>w</mi> <mi>r</mi> <mi>k</mi> </msubsup> <mrow> <mo>(</mo> <mi>m</mi> <mo>)</mo> </mrow> </mrow> </math>

<math> <mrow> <msub> <mi>W</mi> <mi>r</mi> </msub> <mo>=</mo> <mfrac> <msub> <mi>w</mi> <mi>r</mi> </msub> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>l</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>L</mi> </munderover> <msub> <mi>w</mi> <mi>l</mi> </msub> </mrow> </mfrac> </mrow> </math>

Where L represents the number of object rectangles in the equation

The sum of the weights of the intervals in the object rectangular histogram is calculated, in equation

The weights of the object rectangles r are calculated.

6. The method of claim 5, wherein the method is performed according to the equationThe color histogram interval of the object rectangle is reweighted to obtain the weight of the reweighted color histogram interval of the object rectangle

7. The method of claim 6, wherein according to the equation N ═ { N ═ N^r，N^g，N^b-generating a final color model of the object,

wherein, for the color channel c,

r denotes the number of target rectangles, and M denotes the number of bins of the target rectangle histogram.

8. The method of claim 7, further comprising:

extracting a color connected domain of the object from the new input image based on the final color model of the object;

and performing connected domain analysis on the extracted color connected domain, and calculating the centroid and the size of the object in the input image based on the connected domain analysis so as to locate and track the object.

9. An apparatus for extracting color features with strong recognizability, comprising:

a region dividing unit that divides an object region and a non-object region in an input image into a plurality of rectangles;

a histogram calculation unit that establishes color histograms of an object rectangle and a non-object rectangle using the color channels, wherein each color histogram is divided into a plurality of discrete intervals;

a main color extracting unit for extracting the main color of the object rectangle according to the color histogram established by the histogram calculating unit;

an interval minimum distance calculation unit that calculates a minimum distance between the color histogram interval of the target rectangle and the color histogram interval of the non-target rectangle for the main color extracted by the main color extraction unit;

an interval weight calculation unit that calculates a weight of a color histogram interval of the target rectangle based on the minimum distance calculated by the interval minimum distance calculation unit;

a color component weight calculation unit that calculates the weight of the color component of the target rectangle based on the weight of the color histogram section of the target rectangle calculated by the section weight calculation unit;

a rectangular weight calculation unit that calculates the weight of the target rectangle based on the weight of the color histogram section of the target rectangle calculated by the section weight calculation unit;

an interval re-weighting unit that re-weights the color histogram interval of the target rectangle based on the weight of the target rectangle calculated by the rectangle weight calculation unit and the weight of the color component calculated by the color component weight calculation unit, thereby extracting a color feature of the target having strong recognizability;

and a final color model generation unit which generates a final color model of the object based on a result of the re-weighting of the color histogram section of the object rectangle.

10. The apparatus according to claim 9, wherein the minimum distance calculation unit calculates the minimum distance between the color histogram bin of the object rectangle and the color histogram bin of the non-object rectangle according to the following equation:

<math> <mrow> <mi>J</mi> <mo>[</mo> <msubsup> <mi>H</mi> <mi>r</mi> <mi>c</mi> </msubsup> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> <mo>,</mo> <msup> <mi>BG</mi> <mi>c</mi> </msup> <mrow> <mo>(</mo> <mi>j</mi> <mo>)</mo> </mrow> <mo>]</mo> <mo>=</mo> <munder> <mrow> <mi>arg</mi> <mi>min</mi> </mrow> <mi>j</mi> </munder> <mo>|</mo> <msub> <mi>b</mi> <mi>i</mi> </msub> <mo>-</mo> <msubsup> <mi>b</mi> <mi>j</mi> <mo>&prime;</mo> </msubsup> <mo>|</mo> </mrow> </math>

<math> <mrow> <msubsup> <mi>D</mi> <mi>r</mi> <mi>c</mi> </msubsup> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> <mo>=</mo> <mo>|</mo> <msub> <mi>b</mi> <mi>i</mi> </msub> <mo>-</mo> <msubsup> <mi>b</mi> <mi>J</mi> <mo>&prime;</mo> </msubsup> <mo>|</mo> </mrow> </math>

where J represents a bin of the color histogram of the non-target rectangle of the color channel c, said bin J having a minimum distance to bin i of the color channel c of the main color histogram of the target rectangle r, b_iIs the identification of the interval i and,is toColor histogram, BG, of the interval i of a color channel c like a rectangle r^c(j) Is the non-object color histogram of bin j of color channel c,is the identification of the interval j and,is the minimum distance between the bin i of the color channel c of the object rectangle r and the non-object rectangle color histogram bin with the same color channel.

11. The apparatus according to claim 10, wherein the interval weight calculation unit calculates the weight of the color histogram interval of the object rectangle according to the following equation

<math> <mrow> <msubsup> <mi>w</mi> <mi>r</mi> <mi>c</mi> </msubsup> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> <mo>=</mo> <mi>T</mi> <mrow> <mo>(</mo> <msubsup> <mi>D</mi> <mi>r</mi> <mi>c</mi> </msubsup> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> <mo>,</mo> <mi>s</mi> <mo>)</mo> </mrow> <mo>&times;</mo> <mo>|</mo> <msubsup> <mi>H</mi> <mi>r</mi> <mi>c</mi> </msubsup> <mrow> <mo>(</mo> <mi>i</mi> <mo>)</mo> </mrow> <mo>-</mo> <msup> <mi>BG</mi> <mi>c</mi> </msup> <mrow> <mo>(</mo> <mi>J</mi> <mo>)</mo> </mrow> <mo>|</mo> </mrow> </math>

Wherein,

T(x，y)＝e^x×k×yand k is a constant.

12. The apparatus according to claim 11, wherein the color component weight calculation unit calculates the weight W of the color component of the object rectangle according to the following equation^c：

<math> <mrow> <msup> <mi>w</mi> <mi>c</mi> </msup> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>m</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <msubsup> <mi>w</mi> <mi>r</mi> <mi>c</mi> </msubsup> <mrow> <mo>(</mo> <mi>m</mi> <mo>)</mo> </mrow> </mrow> </math>

<math> <mrow> <msup> <mi>W</mi> <mi>c</mi> </msup> <mo>=</mo> <mfrac> <msup> <mi>w</mi> <mi>c</mi> </msup> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>K</mi> </munderover> <msup> <mi>w</mi> <mi>k</mi> </msup> </mrow> </mfrac> </mrow> </math>

Where K represents the number of color channels and M represents the number of bins in the object rectangle histogram, in equation

The sum of the weights of each interval in the color component c of the object rectangle r is calculated in equation

In which weights for color components c are calculatedAnd (4) heavy.

13. The apparatus according to claim 12, wherein the object rectangle weight calculation unit calculates the weight W of the object rectangle according to the following equation_r：

<math> <mrow> <msub> <mi>w</mi> <mi>r</mi> </msub> <mo>=</mo> <munderover> <mi>&Sigma;</mi> <mrow> <mi>k</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>K</mi> </munderover> <munderover> <mi>&Sigma;</mi> <mrow> <mi>m</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>M</mi> </munderover> <msubsup> <mi>w</mi> <mi>r</mi> <mi>k</mi> </msubsup> <mrow> <mo>(</mo> <mi>m</mi> <mo>)</mo> </mrow> </mrow> </math>

<math> <mrow> <msub> <mi>W</mi> <mi>r</mi> </msub> <mo>=</mo> <mfrac> <msub> <mi>w</mi> <mi>r</mi> </msub> <mrow> <munderover> <mi>&Sigma;</mi> <mrow> <mi>l</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>L</mi> </munderover> <msub> <mi>w</mi> <mi>l</mi> </msub> </mrow> </mfrac> </mrow> </math>

Where L represents the number of object rectangles in the equationThe sum of the weights of the intervals in the object rectangular histogram is calculated, in equation

The weights of the object rectangles r are calculated.

14. The apparatus of claim 13, wherein the interval re-weighting unit is according to equation

The color histogram interval of the object rectangle is reweighted to obtain the weight of the reweighted color histogram interval of the object rectangle

15. The apparatus of claim 14, wherein the final color model generation unit generates a final color model according to the equation N ═ { N ═ N^r，N^g，N^b-generating a final color model of the object,

wherein, for the color channel c,

r denotes the number of target rectangles, and M denotes the number of bins of the target rectangle histogram.

16. The apparatus of claim 15, further comprising:

a color connected component extracting unit which extracts a color connected component of the object from the new input image based on the final color model of the object;

and an object positioning unit which performs connected component analysis based on the color connected component extracted by the color connected component extracting unit, and calculates the centroid and the size of the object in the input image based on the connected component analysis, thereby positioning and tracking the object.

17. An image forming apparatus comprising:

an imaging unit that takes an image of a subject;

the apparatus for extracting color features with strong recognizability according to claim 16, receiving an image from an imaging unit, extracting color features with strong recognizability of an object, generating a final color model of the object, extracting color connected domains of the object based on the final color model, performing connected domain analysis, and generating parameters for adjusting the pose of the imaging apparatus based on the connected domain analysis;

a control unit receiving a parameter for adjusting the posture of the imaging device from the device for extracting the color feature with strong recognition power, and adjusting the posture of the imaging device;

a storage unit that stores an image of a photographed subject;

and a display unit displaying the photographed image of the subject.

18. The imaging apparatus of claim 17, further comprising:

and the labeling unit is used for providing the object region manually labeled on the image by the user to the device for extracting the color features with strong identification force.

19. The imaging device of claim 18, wherein the control unit adjusts at least one of a pan, tilt, zoom, and select a focus region of the imaging device according to a parameter that adjusts the pose of the imaging device.

20. The imaging apparatus according to claim 19, wherein in the operation of selecting the focus area, the control unit controls the imaging apparatus to select a new area where the object is located as the basis for focusing so as to focus on the area.

21. The imaging device according to claim 20, wherein when the control unit controls the imaging device to select the focus area, the imaging device selects an image center area as a default imaging focus area, or dynamically selects a new image area where the object is located as an imaging focus area, and dynamically adjusts zoom factors, focal lengths, wobbling, or tilt parameters of the imaging device according to image data information of the focus area.

Images