CN102142146B - Method for tracking video target area - Google Patents

Abstract

The invention discloses a method for tracking a video target area. The method tracks the tracking points corresponding to the path points in the video target area to obtain parameter information of the path points, and then performs smoothing processing on the obtained path points. Use smooth curves to connect each path point, cover the video target area, and realize the tracking of the video target area. The present invention overcomes the problems existing in the prior art, uses one module to realize most of the tracking functions, connects with other modules through the extended interface, and increases the interrelationship operation between waypoints, which is convenient for users to operate , to achieve accurate tracking of the video target area.

Description

A method for tracking video target area

technical field

The invention relates to the technical field of video processing, in particular to a method for tracking a video target area.

Background technique

Most of the video processing software includes tracking modules, but most tracking modules have problems such as not being able to cover all tracking situations during tracking, inconvenient expansion, and fixed operation modes.

In the existing software, the path points obtained by tracking are generally isolated or simple straight-line connections, and it is difficult to achieve complete coverage of the area, and there is generally no mutual relationship between the path points, but in most videos, the path points generally exist. connected characteristics.

In video processing, if you need to edit a certain video object, such as mosaic masking, color correction, particle effects and textures, etc., you need to know the path of the target area accurately. Multiple tracking points are added to the boundary of the video target area to achieve coverage of the target area. But in video post-processing, the target area is usually an irregular video object, and due to the motion of the video shooting object or the shaking of the camera, the instability of the target area in the video often occurs. When the video target area is a regular polygon or a stable area, and the boundary does not change too much, the existing technology can meet the demand. But in most cases, the video target area is an irregular shape or unstable area. To solve this problem, the existing technology cannot realize the tracking of the video target area well.

In existing video tracking operations, the following problems are often encountered:

1) The target area is unstable on the screen. In the tracking video, it is often encountered that the target area is unstable, covered by other objects with different moving directions, or in an area outside the screen display. In the unstable target area, the target area cannot be obtained because no good tracking point can be found.

2), arbitrary target area tracking. In the existing tracking software, the target area is generally rectangular, elliptical or directly connected with the path points by a straight line to obtain the target area, which cannot cover the target area well.

3), module interface. In video post-processing, the function of tracking alone is single or meaningless, but if it can be combined with other video plug-ins or special effect modules, it can produce good results, so the interface between tracking modules and plug-ins becomes very important . In the existing software, it is generally cumbersome to first track the video and then save the path point parameters, and then open other modules and import the tracking data.

Contents of the invention

(1) Technical problems to be solved

Aiming at the above-mentioned problem of irregularity and instability of the video target area, the main purpose of the present invention is to provide a method for tracking the video target area, so as to realize accurate tracking of the video target area.

(2) Technical solution

In order to achieve the above object, the present invention provides a method for tracking the video target area, the method is by tracking the tracking point corresponding to the way point in the video target area, obtaining the parameter information of the way point, and then tracking the obtained path Points are smoothed, and smooth curves are used to connect each path point, covering the video target area, and realizing the tracking of the video target area.

In the above scheme, the waypoint is a position point on the boundary of the video target area, the tracking point is the position point of the algorithm tracking module in the tracking process, and the tracking point is the reference point of the waypoint, which uniquely corresponds to the position point in the video target area. a waypoint.

In the above solution, when tracking the tracking point corresponding to the waypoint in the video target area, a tracking area block is selected near the tracking point as the tracking point area, and the tracking point area contains the information of the tracking point.

In the above scheme, the template tracking method is used for tracking the tracking points corresponding to the waypoints in the video target area, and the tracking point area is used as a reference template, and each area in the video target area is compared with the reference template. Matching, search and match the area with the maximum similarity in the next frame of image as a new tracking point.

In the above scheme, the tracking point corresponding to the waypoint in the video target area is tracked using the method of corner point tracking, by detecting the corner point in the video target area of the next frame image, after the corner point is detected, the corner point Calculate the similarity between the area where it is located and the tracking point area, and get the maximum similarity as the new tracking point.

In the above solution, the parameter information of the waypoint at least includes: tracking type, waypoint and tracking point, whether the waypoint and tracking point are separated, the relationship between waypoints, the size of the tracking area block and the search area block, and the connection path Smooth curve type and path handling for points.

In the above scheme, when smoothing the obtained path points and connecting each path point with a smooth curve, the smooth curve used is a Bezier curve.

In the above solution, the Bezier curve has two Bezier control points at each path point, and the video target area can be covered by adjusting the Bezier control points.

In the above solution, when the obtained path points are smoothed, and the smooth curves are used to connect the path points, the smooth curves used are B-spline curves.

In the above scheme, for the case where the video is only panned or zoomed but not deformed, the method further tracks the video target area according to the relationship between the waypoints, and selects a waypoint whose position is different from other waypoints. Change.

(3) Beneficial effects

As can be seen from the foregoing technical solutions, the present invention has the following beneficial effects:

1. The method for tracking the video target area provided by the present invention is to obtain the parameter information of the way point by tracking the tracking point corresponding to the way point in the video target area, and then smoothing the obtained way point, A smooth curve is used to connect each path point to cover the video target area, and the accurate tracking of the video target area is realized.

2. The method for tracking the video target area provided by the present invention overcomes the problems existing in the prior art, utilizes a module to realize most of the tracking functions, and connects with other modules through an expanded interface, and increases It clarifies the interrelationship operation between waypoints, which is convenient for users to operate.

3. In the method for tracking the video target area provided by the present invention, in a tracking method of the video target area, Bezier curve connections are added between the path points, and the interdependence and tracking between the path points can be set. Functions such as separation of waypoints and tracking points, so that when the tracking point is blocked and invisible in some frames due to image changes, it can also be based on the relative position between the tracking point and other visible tracking points and the distance between the tracking points. The position of the waypoint can be obtained through the relationship between them, so as to accurately determine the tracking area.

Description of drawings

Fig. 1 is the operation schematic diagram of tracking a certain frame by the tracking method of the video target area provided by the present invention;

Fig. 2 is a flowchart of a method for tracking a video target area according to an embodiment of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be described in further detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

Aiming at the above-mentioned problem of irregularity and instability of the video target area, the method for tracking the video target area provided by the present invention proposes to separate the tracking point from the path point, use smooth curves to realize the coverage of the target area and set the path point The tracking path of the video target area can be obtained through the mutual relationship between them, and the accurate tracking of the video target area can be realized.

1), separation of tracking points and path points

The tracking point is the reference point when the tracking algorithm operates, and the path point is the final output path position. The general setting is that the tracking point and the waypoint are at the same position. Due to the instability of the video and its objects, the tracking of the target area cannot be achieved in many cases. For example, some waypoints in the target area are blocked or disappeared by other objects. In this case, find a tracking point that has a stable correspondence with the waypoint and set up a connection relationship. By tracking the tracking point, the parameters of the waypoint can be obtained.

2), the smooth curve covers the target area

Since the tracking target is generally an irregular area, if a straight line connection is used, a lot of path points are needed to cover it, but this method uses a smooth curve to connect the path points, and the required area size is obtained through the control points of the smooth curve. While saving the amount of calculation, it also better meets the demand.

3), set the relationship between waypoints

In video tracking, there is generally a certain relationship between waypoints. When the video only pans and zooms, you can track more accurately and with fewer calculations by setting the relationship between the waypoints.

The method for tracking the video object area provided by the present invention obtains the parameter information of the path points by tracking the tracking points corresponding to the path points in the video object area, and then performs smoothing processing on the obtained path points. The curve connects each path point, covers the video target area, and realizes accurate tracking of the video target area.

Wherein, the waypoint is a position point on the boundary of the video target area, the tracking point is the position point of the algorithm tracking module in the tracking process, and the tracking point is the reference point of the waypoint, uniquely corresponding to a path in the video target area point. When tracking the tracking point corresponding to the path point in the video target area, select a tracking area block near the tracking point as the tracking point area, and the tracking point area contains the information of the tracking point. The parameter information of the waypoint at least includes: tracking type, waypoint and tracking point, whether the waypoint and tracking point are separated, the relationship between waypoints, the size of the tracking area block and the search area block, and the smooth curve connecting the way points Type and path handling.

The template tracking method is used to track the tracking points corresponding to the path points in the video target area, and the tracking point area is used as a reference template. By matching each area in the video target area with the reference template, the image in the next frame The region with the maximum similarity obtained by searching and matching is used as a new tracking point. Alternatively, the corner point tracking method is used to track the tracking point corresponding to the path point in the video target area. By detecting the corner point in the video target area of the next frame image, after the corner point is detected, the corner point is located in the area where the corner point is located and tracked. Calculate the similarity of the point area, and get the maximum similarity as the new tracking point.

The obtained path points are smoothed, and when smooth curves are used to connect the path points, the smooth curves used are Bezier curves. The Bezier curve has two Bezier control points at each path point. By adjusting the Bezier control points, the video target area can be covered. Alternatively, the obtained path points are smoothed, and when smooth curves are used to connect the path points, the smooth curves used are B-spline curves.

For the case where the video is only panned or zoomed but not deformed, the method further tracks the target area of the video according to the relationship between the waypoints, and the position between a certain waypoint and other waypoints remains unchanged.

The method for tracking a video target area provided by the present invention obtains the path parameters of the area by tracking the video target area, and is used for video post-processing, such as masking, 2D transformation and image stabilization. Path parameters include: position of single or multiple waypoints per frame and Bezier control points for each waypoint. The path points of each frame are connected by Bezier curves and cover the target area. As shown in Figure 1, it is the tracking result of a certain frame. The figure includes 4 path points and their respective Bezier curves. The small squares on the line represent the position of the path points, and the horizontal line is the connection of the Bezier control points. All the path points and parameters form a circular tracking region. By adjusting the path points and Bezier control points, a Bezier curve is obtained, and the Bezier curve can surround the tracking area to realize the tracking of the video target area.

In the method for tracking the video target area provided by the present invention, the following terms are used:

Video tracking: path the whole or part of the moving target area, and obtain the path parameters of the target area.

Bezier curve: According to 2 row value points and 2 control points, a smooth curve passing through 2 row value points is obtained.

Waypoint and tracking point: the waypoint is the position where the waypoint was obtained recently, and the tracking point is the position of the algorithm tracking module during the tracking process. Trackpoints are reference points for waypoints.

Similarity: It is used to measure the similarity between two regions.

Template tracking: Through the method of template matching with each block of the search area, the area with a large similarity is obtained as a new tracking point.

Corner tracking: By detecting corners in the search area, the similarity calculation between the corner area and the tracking template is performed and the maximum similarity is obtained as a new tracking point.

The method for tracking the video target area provided by the present invention is solidified in the tracking module, and the tracking module is mainly called by other plug-ins or modules. These plug-ins or modules need to take the path point information of the video target area as a parameter. For example, in the mask plug-in, the area to be masked needs to be obtained; in the color correction module, the area to be corrected needs to be obtained, etc. When starting up, it is necessary to set the initial parameters of the tracking module, such as the parameters of the starting path point, whether the number of path points is fixed, the relationship between path points and Bezier curve information, etc. After entering the tracking module, the video target tracking operation can be carried out. After completion, the parameters of each path point and its Bezier curve parameters are passed to the plug-in module for operation.

Each path point has its own parameters, such as tracking type, separation of path point and track point, and whether to set it as Bezier curve, etc. The following are the parameter settings of the waypoint:

1), Tracking type selection. Tracking type can choose template tracking or corner tracking. Template tracking is to use the tracking point area as a reference template, and find the area with the greatest similarity in the next frame image as a new tracking point. Corner tracking is to detect the corner position in the next frame image, and find the point with the greatest similarity between the corner position area and the tracking area as a new tracking point. The advantage of template tracking is that it matches accurately, but it has a large amount of calculation. The advantage of corner point tracking is that it has a small amount of calculation, fast speed and clear feature points. The premise of corner tracking is that the tracking point must be on a stable corner, and the user can view the detected corner positions frame by frame by displaying all the corner points in the video. Select the corner point that appears stably as the location of the tracking point.

2), track block and search block size. Considering the accuracy and computing efficiency, a tracking block is generally selected near the tracking point, which contains the information of the tracking point, and the search range in the next frame is determined by the size of the search block. The larger the search block, the more motion can be detected, but at the same time, more error information will be introduced and the amount of calculation will be larger. Generally, appropriate tracking blocks and search blocks are selected according to different tracking objects. You can preliminarily adjust the size of the tracking block and the search block. If the tracking point is stable and the deformation is small, you can choose a small tracking block; if the deformation of the area is large, you need to select a larger tracking block to contain more information. quantity.

3) Waypoint connection type. In the tracking method module of the video target area of the present invention, Bezier curves are generally selected to connect the path points. By adjusting each Bezier control point, the coverage of the target area can be achieved, and a straight line connection can also be selected. If a straight-line connection is selected to cover the target area, more path points are generally required, which increases the amount of computation and instability. By adjusting the Bezier control points, the target area can be covered with fewer points. Moreover, the straight line connection can only cover the edge polygons, and the use of Bezier curves can achieve the area coverage of smooth edges.

4) Interrelationship between waypoints. You can choose that the positions of a certain path point and other path points remain unchanged. It is mainly used in situations where the video is only panned or zoomed without deformation, which can reduce the amount of computation and be more accurate. If the video tracking area is distorted or deformed, the waypoints and tracking points need to be set at the same point to achieve more accurate tracking.

5), waypoints and tracking points. It can be set that the waypoint and the tracking point are not in the same position. It is applied to the situation that the waypoint is unstable or disappears. The position of the waypoint can be obtained through the relative position with the tracking point and the relationship between the tracking points. The position changes of multiple tracking points can determine whether the video has been panned or zoomed. If panned, the relative position between the tracked point and the path point will not change. If zoomed, the relative position needs to be scaled accordingly.

6), path processing. After the waypoints are obtained, the waypoints of each frame can be smoothed to prevent jitter or correct minor errors that occur.

The method for tracking the video target area provided by the present invention proposes to add a control button to the existing module, and call the tracking module through the control button, so that the target can be tracked in real time and conveniently without resetting Export and import data. The interface of the tracking method module of the video target area of the present invention is the position of the waypoint and its control point.

See shown in Fig. 2, Fig. 2 is the tracking method flowchart of the video target area of an embodiment of the present invention, comprises steps:

A) Start the control button to call the tracking module of the video target area of the present invention.

B) Setting parameters:

1) Set the path type: the tracking type can choose template tracking or corner tracking.

2) Set the tracking block and search area size.

3) Set the connection type between tracking points, and select the Bezier curve to connect the path points.

4) Set the mutual relationship between the waypoints, select a certain waypoint and other waypoints, and the position between each other remains unchanged.

5) Set the relationship between the waypoint and the tracking point.

C) Start the tracking module of the video target area of the present invention to track.

D) Path processing: After the path points are obtained, the path points of each frame can be smoothed.

E) Exit the tracking module, complete.

When tracking the athlete's head portrait, since the athlete's head portrait is always changing and the background is also different, it is difficult to achieve tracking by selecting the edge point of the athlete's head portrait as the tracking point. Head portrait tracking can be achieved by setting the tracking point in the face area, the path point in the edge area, and the tracking point and path point separately.

When using a smooth curve to connect the path points, the smooth curve can use Bezier curve, B-spline curve and other methods. For example, the Bezier curve connects the path points. There are two control points at each path point. By adjusting the control points, the tracking area can be covered.

When considering the interrelationship of waypoints, if only translation occurs in the video area, all the waypoint positions can be obtained by tracking only one point. When some tracking objects disappear, you can set the relationship between this tracking point and other tracking points, and get this tracking point through the positions of other tracking points.

The method for tracking the video target area provided by the present invention introduces the method of separating the waypoint and the tracking point, which can well solve the problem that the target area is unstable on the screen, and adds the relationship between the waypoints, even in the When the video is zoomed, the position of the waypoint can also be quickly obtained. The tracking point is the reference point of the waypoint, and the position of the waypoint is obtained through direct or indirect correlation. For example, in a video, you want to get the waypoint above someone's head. Since the area above the person's head often changes or disappears, you can set the person's eyes as the tracking point, and then set the relative relationship between the eyes and the top of the head. In this case, the top of the head is the waypoint, the eyes are the tracking points, and the waypoint and the tracking point are separated. Of course, the waypoint and the tracking point can also be in the same position.

The method for tracking a video target area provided by the present invention uses Bezier curves to connect path points, and obtains any smooth target area by setting parameters of Bezier control points at path points.

The specific embodiments described above have further described the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims (9)

1. A method for tracking the video target area, characterized in that the method obtains the parameter information of the way point by tracking the tracking point corresponding to the way point in the video target area, and then smoothes the obtained way point Processing, using smooth curves to connect each path point, covering the video target area, and realizing the tracking of the video target area; Wherein, the waypoint is a position point on the boundary of the video target area, the tracking point is the position point of the algorithm tracking module in the tracking process, and the tracking point is the reference point of the waypoint, uniquely corresponding to a path in the video target area point. 2. The method for tracking the video target area according to claim 1, wherein, when the tracking point corresponding to the waypoint in the video target area is tracked, a tracking area block is selected near the tracking point as Tracking point area, the tracking point area contains tracking point information. 3. The method for tracking the video target area according to claim 2, characterized in that, the tracking point corresponding to the waypoint in the video target area is tracked using a template tracking method, and the tracking point area is used as The reference template, by matching each area in the video target area with the reference template, searches for the area with the maximum similarity in the next frame image as a new tracking point. 4. The method for tracking the video target area according to claim 2, characterized in that, the tracking point corresponding to the waypoint in the video target area is tracked using the method of corner point tracking, through the next frame image The corner point is detected in the video target area, and the similarity calculation is performed between the corner point area and the tracking point area after the corner point is detected, and the maximum similarity is obtained as a new tracking point. 5. The method for tracking the video target area according to claim 1, wherein the parameter information of the waypoint at least includes: Tracking type, path point and track point, whether path point and track point are separated, relationship between path points, size of track area block and search area block, type of smooth curve connecting path points, and path processing. 6. The method for tracking the video target area according to claim 1, characterized in that, the obtained path points are smoothed, and when using smooth curves to connect each path point, the smooth curves used are Bezier curve. 7. The method for tracking the video target area according to claim 6, wherein the Bezier curve has two Bezier control points at each path point, and the Bezier control points are adjustment to cover the video target area. 8. The method for tracking the video target area according to claim 1, characterized in that, the obtained path points are smoothed, and when using smooth curves to connect each path point, the smooth curves used are B-splines curve. 9. The method for tracking the video target area according to claim 1, characterized in that, for the situation that only translation or zooming occurs in the video but no deformation, the method further tracks the video target area according to the relationship between the waypoints For tracking, select a certain waypoint and other waypoints with the same position between each other.

Images