CN114926501A - Method and apparatus for determining motion information, medium, and computer device - Google Patents

Abstract

Embodiments of the present disclosure provide a method, apparatus, medium, and computer device for determining motion information. The method includes: acquiring video frames captured by multiple cameras in different first fields of view; and determining, based on the acquired video frames, position information of multiple track points passed by the target object during the movement within the second field of view; the second field of view covers the first field of view corresponding to the multiple cameras; obtain the multiple tracks Projection information of points on the trajectory of the target object, the trajectory of the target object is determined based on the position information of the plurality of trajectory points; the projection information is used to represent the movement distance of the target object; based on the projection information determines motion information of the target object.

Description

Method and apparatus, medium and computer equipment for determining motion information

technical field

The present disclosure relates to the technical field of data processing, and in particular, to a method and apparatus, medium and computer equipment for determining motion information.

Background technique

During the movement of the target object, it is often necessary to obtain the movement information of the target object to evaluate the movement state of the target object. In the related art, the motion information of the target object is roughly estimated simply based on the linear distance of the target object on different trajectory points and the total duration of the target object passing through the different trajectory points, which is less accurate.

SUMMARY OF THE INVENTION

In a first aspect, an embodiment of the present disclosure provides a method for determining motion information, the method comprising: acquiring video frames captured by multiple cameras in different first fields of view; and determining, based on the acquired video frames, that a target object is in The position information of the multiple track points passed during the movement in the second field of view; the second field of view covers the first field of view corresponding to the multiple cameras; obtain the location information of the multiple track points in the projection information on the trajectory of the target object, the trajectory of the target object is determined based on the position information of the multiple trajectory points; the projection information is used to represent the movement distance of the target object; Describe the motion information of the target object.

In some embodiments, determining the trajectory of the target object based on the position information of the plurality of trajectory points includes: acquiring a predicted trajectory, where the predicted trajectory includes at least one prediction parameter; based on the acquired position of the at least one trajectory point The information determines each prediction parameter of the at least one prediction parameter; the trajectory of the target object is determined according to each determined prediction parameter and the predicted trajectory.

In some embodiments, acquiring the projection information of the plurality of trajectory points on the trajectory of the target object includes: determining a distance between each group of adjacent trajectory points in the plurality of trajectory points; The distance between the adjacent trajectory points of each group is projected to the tangent direction of the trajectory, and the projected distance of each group of adjacent trajectory points on the trajectory is obtained; The projection distance between the distances on the track is used to determine the projection information of the multiple track points on the track of the target object.

In some embodiments, the method further includes: acquiring time information of each track point in the plurality of track points; and determining the motion information of the target object based on the projection information includes: based on the projection information and the time information of each track point to determine the motion information of the target object.

In some embodiments, after determining the motion information of the target object based on the projection information, the method further includes: acquiring a subsequent trajectory point located behind the plurality of trajectory points; acquiring the subsequent trajectory point the first distance from the last trajectory point in the plurality of trajectory points; the first distance is projected on the tangent direction of the trajectory to obtain a second distance; based on the second distance, the Projection information is updated.

In some embodiments, the determining the motion information of the target object based on the projection information includes: acquiring a motion model of the target object, where the motion model is used to represent the movement distance of the target object over time change relationship, the motion model includes at least one model parameter; based on the acquired projection information of the at least one trajectory point and the acquired time information of each trajectory point in the at least one trajectory point, determine the at least one model parameter. each model parameter; determine the motion information of the target object according to the determined model parameter and the motion model.

In some embodiments, the at least one track point includes N track points that have been acquired from the plurality of track points; the determining the motion information of the target object according to the determined model parameters and the motion model includes: : determine the motion information of the M-th trajectory point of the target object in the N trajectory points according to the determined model parameters and the motion model; M and N are both positive integers, and M is less than N.

In some embodiments, the motion information includes acceleration information, the target object moves on a surface of a reference object, and the surface of the target object is in contact with the surface of the reference object; the method further includes: based on the The acceleration information determines the coefficient of friction between the surface of the target object and the surface of the reference object.

In some embodiments, the determining, based on the acquired video frames, the position information of multiple track points that the target object passes through in the process of moving within the second field of view includes: determining, based on the video frames acquired by each camera, the The initial position information of the track points within the first field of view of each camera is described; the initial position information of the track points within the first field of view of each camera is fused to obtain the position information of the multiple track points.

In some embodiments, the first field of view of at least two target cameras in the plurality of cameras partially overlaps; the initial position information of the track points within the first field of view of each camera is fused to obtain the The position information of multiple track points, including: determining the initial position information of the track points within the third field of view of each target camera, where the third field of view is the overlapping field of view of each target camera; Synchronizing the initial position information of the track points within the third field of view of the cameras to obtain the synchronization position information of the track points within the third field of view of each target camera; synchronizing the track points within the third field of view of each target camera The position information is fused to obtain the position information of the track points within the third field of view of each target camera.

In a second aspect, an embodiment of the present disclosure provides an apparatus for determining motion information. The apparatus includes: a first acquisition module, configured to acquire video frames acquired by multiple cameras in different first visual fields; a module for determining, based on the acquired video frames, the position information of multiple track points that the target object passes through in the process of moving within a second field of view; the second field of view covers the corresponding a first field of view; a second acquisition module, configured to acquire projection information of the multiple track points on the track of the target object, where the track of the target object is determined based on the position information of the multiple track points; the The projection information is used to represent the movement distance of the target object; the second determination module is used to determine the movement information of the target object based on the projection information.

In some embodiments, the second obtaining module is configured to: obtain a predicted trajectory, where the predicted trajectory includes at least one predicted parameter; and determine each of the at least one predicted parameter based on the obtained position information of the at least one trajectory point. a prediction parameter; the trajectory of the target object is determined according to each determined prediction parameter and the predicted trajectory.

In some embodiments, the second obtaining module is configured to: determine the distance between each group of adjacent trajectory points in the plurality of trajectory points; project the distance between each group of adjacent trajectory points To the tangent direction of the trajectory, the projected distance of each group of adjacent trajectory points on the trajectory is obtained; according to the projected distance of the distance between each group of adjacent trajectory points on the trajectory, determine the projected distance of each group of adjacent trajectory points on the trajectory. projection information of the plurality of trajectory points on the trajectory of the target object.

In some embodiments, the apparatus further includes: a third acquisition module configured to acquire time information of each trajectory point in the plurality of trajectory points; the second determination module configured to: based on the projection information and The time information of each track point determines the motion information of the target object.

In some embodiments, the apparatus further includes: a fourth acquisition module, configured to acquire subsequent trajectory points located after the plurality of trajectory points; and a fifth acquisition module, configured to acquire the relationship between the subsequent trajectory points and all the first distance between the last trajectory point in the plurality of trajectory points; the projection module is used to project the first distance on the tangent direction of the trajectory to obtain the second distance; the update module is used to base on the The second distance updates the projection information.

In some embodiments, the second determination module is configured to: acquire a motion model of the target object, where the motion model is used to represent a time-varying relationship of the motion distance of the target object, and the motion model includes at least one model parameter; based on the acquired projection information of the at least one trajectory point and the acquired time information of each trajectory point in the at least one trajectory point, determine each model parameter in the at least one model parameter; according to the determined model The parameters and the motion model determine motion information of the target object.

In some embodiments, the at least one trajectory point includes N trajectory points that have been obtained from the plurality of trajectory points; the second determining module is configured to: according to the determined model parameters and the motion model, determine the motion information of the M-th trajectory point of the target object in the N trajectory points; M and N are both positive integers, and M is less than N.

In some embodiments, the motion information includes acceleration information, the target object moves on a surface of a reference object, and the surface of the target object is in contact with the surface of the reference object; the apparatus further includes: a third determination a module for determining a friction coefficient between the surface of the target object and the surface of the reference object based on the acceleration information.

In some embodiments, the first determining module is configured to: determine the initial position information of the track points within the first field of view of each camera based on the video frames acquired by each camera; The initial position information of the track points within the field of view is fused to obtain the position information of the multiple track points.

In some embodiments, the first field of view of at least two target cameras in the plurality of cameras partially overlaps; the first determining module is configured to: determine the trajectory within the third field of view of each target camera The initial position information of the point, the third field of view is the overlapping field of view of each target camera; the initial position information of the track points within the third field of view of each target camera is synchronized to obtain the third field of view of each target camera Synchronous position information of the track points within the range; fuse the synchronous position information of the track points within the third field of view of each target camera to obtain the position information of the track points within the third field of view of each target camera.

In a third aspect, an embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the method described in any one of the embodiments.

In a fourth aspect, an embodiment of the present disclosure provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing the program described in any of the embodiments when the processor executes the program. method described.

In the embodiment of the present disclosure, the movement distance of the target object is represented by acquiring projection information of multiple trajectory points passed by the target object on the trajectory of the target object, because the projection information can more accurately characterize the movement of the target object during the movement process. The actual movement distance, therefore, the accuracy of the movement information determined based on the projection information is higher.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated into and constitute a part of this specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the technical solutions of the present disclosure.

FIG. 1 is a schematic diagram of a manner of determining motion information in the related art.

FIG. 2 is a flowchart of a method for determining motion information according to an embodiment of the present disclosure.

FIG. 3 is a schematic diagram of a field of view of a camera according to an embodiment of the present disclosure.

FIG. 4 is a schematic diagram of a synchronization process of location information.

FIG. 5 is a schematic diagram of calculating the projection distance.

Figure 6 is a schematic diagram of a bidirectional queue.

FIG. 7 is a schematic diagram of a sudden change in the movement speed of the target object.

FIG. 8 is a block diagram of an apparatus for determining motion information according to an embodiment of the present disclosure.

FIG. 9 is a schematic structural diagram of a computer device according to an embodiment of the present disclosure.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.

The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to limit the present disclosure. As used in this disclosure and the appended claims, the singular forms "a," "the," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items. Additionally, the term "at least one" herein refers to any one of a plurality or any combination of at least two of a plurality.

It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various pieces of information, such information should not be limited by these terms. These terms are only used to distinguish the same type of information from each other. For example, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information, without departing from the scope of the present disclosure. Depending on the context, the word "if" as used herein can be interpreted as "at the time of" or "when" or "in response to determining."

In order for those skilled in the art to better understand the technical solutions in the embodiments of the present disclosure, and to make the above objects, features and advantages of the embodiments of the present disclosure more clearly understood, the following describes the technical solutions in the embodiments of the present disclosure with reference to the accompanying drawings. The program is described in further detail.

During the movement of the target object, it is often necessary to obtain the movement information of the target object to evaluate the movement state of the target object. In the related art, the motion information of the target object is roughly estimated simply based on the linear distance of the target object on different trajectory points and the total duration of the target object passing through the different trajectory points, which is less accurate. For example, referring to FIG. 1 , assuming that the gray area is a track, the black circle represents the object 101 (ie the target object) moving on the track, U1 and U2 are the starting line and the finish line, respectively, and the dotted line S represents that the object 101 is in the race track on the road. In the related art, the motion information of the object 101 is generally determined based on the distance between the start line U1 and the finish line U2 and the duration of the movement of the object 101 from the start line U1 to the finish line U2. However, it can be seen from the figure that the moving trajectory of the object 101 is not a straight line, but a curve. Therefore, there is often a difference between the distance between the starting line U1 and the ending line U2 and the actual moving distance of the object 101, resulting in The determined motion information of the object 101 is inaccurate.

Based on this, the present disclosure provides a method for determining motion information. Referring to Figure 2, the method includes:

Step 201: Acquire video frames captured by multiple cameras in different first fields of view respectively;

Step 202: Based on the acquired video frames, determine the position information of multiple track points that the target object passes through in the process of moving within the second field of view; a field of view;

Step 203: Acquire projection information of the plurality of trajectory points on the trajectory of the target object, where the trajectory of the target object is determined based on the position information of the plurality of trajectory points; the projection information is used to represent the target the movement distance of the object;

Step 204: Determine motion information of the target object based on the projection information.

In step 201, the target object can be any type of target object, including but not limited to people, animals, vehicles, robots, sports equipment (eg, curling, ice hockey), etc., the target object can move autonomously or under external force Move under the action.

The video frames of the target object may be collected by multiple cameras respectively, and the independent field of view of each camera is referred to as the first field of view of the camera. Different cameras correspond to different first fields of view respectively, and the first fields of view of the multiple cameras may partially overlap. The plurality of cameras may include two or more cameras, and the first field of view of the plurality of cameras partially overlaps, which may include a variety of situations. In the following, with reference to FIG. 3, and taking the number of cameras equal to 3 as an example, respectively Examples of possible situations are given. Wherein, each ellipse represents the first field of view of one camera, and ellipses with different colors represent the first field of view of different cameras. The white ellipse corresponds to the first field of view S1 of the first camera, the light gray ellipse corresponds to the first field of view S2 of the second camera, and the dark gray ellipse corresponds to the first field of view S3 of the third camera.

Case 1: The first field of view of any two cameras is partially overlapped. As shown in (3-1) in FIG. 3, any two of S1, S2 and S3 are partially overlapped.

Case 2: The first field of view of some cameras partially overlaps, and the first field of view of other cameras does not overlap, and the first field of view of at least one camera completely falls within the first field of view of other cameras, as shown in Figure 3 As shown in (3-2) of , S1 and S3 partially overlap, S2 and S3 do not overlap, and S2 completely falls within S1.

Case 3: The first field of view of two adjacent cameras partially overlaps, and the first field of view of non-adjacent cameras does not overlap, as shown in (3-3) in Figure 3, where the first field of view is S1 The camera is adjacent to the camera whose first field of view is S2, and is not adjacent to the camera whose first field of view is S3, and the camera whose first field of view is S2 is adjacent to the camera whose first field of view is S3. It can be seen that S1 and S2 in (3-3) partially overlap, S2 and S3 partially overlap, and S1 and S3 do not overlap.

Case 4: The first field of view of some cameras partially overlaps, and the first field of view of at least one camera covers the total field of view of the cameras whose first field of view overlaps, as shown in (3-4) in Figure 3, S2 Partially overlapping with S3, S1 covers the total field of view of S2 and S3.

Case 5: The first field of view of two adjacent cameras partially overlaps, and the first field of view of non-adjacent cameras also partially overlaps, as shown in (3-5) in Figure 3, where the first field of view is The camera of S1 is adjacent to the camera with the first field of view of S2, and is not adjacent to the camera of the first field of view of S3. The camera of the first field of view of S2 is adjacent to the camera of the first field of view of S3. It can be seen that S1 and S2 in (3-5) partially overlap, S2 and S3 partially overlap, and S1 and S3 also partially overlap.

In addition to the cases listed above, the at least partial overlap of the first field of view ranges of the plurality of cameras may also be other cases, which will not be listed one by one. The overlapping portion within the first field of view of two or more cameras is referred to as the overlapping field of view. For example, in (3-1) of Fig. 3, the area where S1, S2 and S3 overlap each other (as shown by the area with slashes in the figure) is the area of camera C1, camera C2 and camera C3. The overlapping field of view; in (3-3) of Figure 3, the overlapping area of S1 and S2 (as shown by the area with a horizontal bar in the figure) is the overlapping field of view of camera C1 and camera C2, S2 and S3 The overlapping area (shown as the area with vertical bars in the figure) is the overlapping field of view of the camera C2 and the camera C3.

In some embodiments, the method of the present disclosure may be applied to determine motion information of a target object moving in a global scene, where the global scene means that the first field of view of any camera covers a sub-area in the global scene, The union of the first field of view of the plurality of cameras can cover the global scene. In the global scene, although the first field of view of a single camera can cover the global scene by adjusting the focal length of the camera, this will cause the target object to occupy less pixels in the video frame captured by the camera. Such video frames determine the motion information of the target object, which will lead to large errors. Also, objects in a single camera are easily occluded. Therefore, in the global scene, the first field of view of each camera generally covers only one sub-area in the global scene.

A video frame within the first field of view corresponding to the camera may be captured by each of the plurality of cameras, and the captured video frame may include one or more target objects, or may not include the target object.

In step 202, based on the video frame acquired in step 201, the position information of multiple track points that the target object passes through during the movement within the second field of view may be determined. The second field of view covers the first field of view corresponding to the plurality of cameras respectively, that is, the second field of view may be the same as the range corresponding to the union of the first field of view of each camera, or, each camera may The union of the first field of view is a subset of the second field of view.

The manner of determining the position information of the track point will be illustrated below with an example. The location information may be the location information of the target object in a preset coordinate system, and the coordinate system may be a world coordinate system, or may be other user-defined coordinate systems. Taking the coordinate system as the world coordinate system as an example, the pixel position of the target object in one video frame can be determined, and based on the pixel position and the conversion relationship between the camera coordinate system and the world coordinate system, the pixel position is converted into the world coordinate system. The physical position in the coordinate system can obtain the position information of a track point. The above method can be used to process multiple video frames collected by the same camera, so as to obtain the position information of multiple track points of the target object within the first field of view of the camera. The track points within the first field of view of each camera are spliced, so that position information of multiple track points that the target object passes through during the movement within the second field of view can be obtained.

For example, it is assumed that the target object moves from the first field of view of camera C1 to the first field of view of camera C2, the second field of view includes the union of the first field of view of camera C1 and the first field of view of camera C2, and the target object In the first field of view of the camera C1, the two track points whose position information is (x1, y1) and (x2, y2) have passed in sequence, and the target object has passed through the first field of view of the camera C2. The position information is ( x3, y3), (x4, y4) and (x5, y5) three track points, then for the track points that the target object passes through in the first field of view of camera C1 and the target object in the first field of view of camera C2 By splicing the track points passed in sequence within the range, the track points passed by the target object in sequence within the second field of view can be obtained, including the position information of (x1, y1), (x2, y2), (x3, y3), ( The five trajectory points of x4, y4) and (x5, y5).

The position information of the track points determined based on the video frames collected by a single camera may not be accurate enough, therefore, the initial position information of the track points within the first field of view of each camera may be determined based on the video frames obtained by each camera; The initial position information of the track points within the first field of view of each camera is fused to obtain the position information of the multiple track points (referred to as fused position information). The initial position information is the position information of the track point determined based on the video frame collected by a single camera. As mentioned above, it can be obtained by converting the pixel position of the target object in the video frame to a preset coordinate system.

In some embodiments, the first fields of view of at least two target cameras of the plurality of cameras partially overlap. For example, in the case shown in (3-3) in FIG. 3, the target camera may include the camera C1 and the camera C2, or the camera C2 and the camera C3. When the target object is within the overlapping field of view of each target camera (hereinafter referred to as the third field of view), when the target object passes through a trajectory point, video frames including the target object can often be captured by multiple target cameras. Therefore, the initial position information of the track points within the third field of view of each target camera can be determined; the initial position information of the track points within the third field of view of each target camera is synchronized to obtain the first position information of each target camera. Synchronous position information of the track points within the three visual fields; fuse the synchronous position information of the track points within the third field of view of each target camera to obtain the position information of the track points within the third field of view of each target camera.

In this embodiment, the initial position information of the track points within the third field of view of each target camera is generally asynchronous, that is, the initial position information of the track points within the third field of view of different target cameras is actually The position information of the trajectory points that the target object passes through at different times, which is caused by the difference in the time when different cameras collect video frames. In order to reduce the error generated in the fusion process of the position information, in this embodiment, the synchronization of the initial position information is performed first, and the synchronization position information of the track points within the third field of view of each target camera is obtained, that is, the third position information of each target camera is obtained. The position information of the trajectory points in the field of view at the same time, and then the synchronous position information is fused.

For example, assuming that the target camera includes camera C1 and camera C2, and the target object is within the overlapping field of view (ie, the third field of view) of the two target cameras, camera C1 and camera C2, camera C1 can be determined based on the video frames collected by camera C1 The initial position information (denoted as the first position information) of the track points within the first field of view of the second location information). Since the time when the camera C1 collects the video frame is different from the time when the camera C2 collects the video frame, it is difficult to obtain the first position information and the second position information at the same moment. Therefore, the first position information and the second position information can be synchronized to obtain the synchronized position information (referred to as the third position information) of the track points within the third field of view of the camera C1 and the position information of the track points within the third field of view of the camera C2. Synchronized position information of the track point (referred to as fourth position information), wherein the third position information and the fourth position information are the position information of the track point of the target object at the same time (referred to as reference time). By fusing the third position information and the fourth position information, the position information of the track points within the third field of view of the camera C1 and the camera C2 can be obtained.

The reference time may be the time when one of the target cameras among the multiple target cameras captures the video frame. For example, when the target object is at a certain trajectory point, multiple target cameras may capture video frames successively. When the target object is at the same track point, the video frames successively collected by each target camera are called a batch of video frames. The reference time may be the time corresponding to the earliest time stamp in the time stamps of the same batch of video frames.

The difference between the time stamps of each video frame in a batch of video frames is generally less than the preset time interval. Therefore, it can be acquired by different target cameras, and the difference between the time stamps is less than the preset time interval. video frames, denoted as {Q1,Q2,...,Qn}, where Qi is the video frame collected by the ith target camera, i and n are positive integers, and i≤n, n is the total number of target cameras . The time corresponding to the earliest timestamp in the timestamps of {Q1, Q2, ..., Qn} can be used as the reference time. After the reference time is determined, for each camera, the video frame collected by the target camera before the reference time (called the previous video frame) and the video frame collected by the target camera after the reference time (called the previous video frame) can be obtained. After the video frame), the position information determined based on the previous video frame and the position information determined based on the subsequent video frame are interpolated to obtain the position information of the reference time (that is, the position information of the track point within the third field of view of the target camera. sync location information). By performing the above processing on the video frames collected by each camera, the synchronization position information of the track points within the third field of view of each target camera can be obtained.

编号为k的视频帧对应的在后视频帧中目标对象的位置信息记为

通过对

和

进行插值，得到对应相机的第三视野范围内的目标对象在基准时间t的位置(x^k,y^k)。对于基准帧而言，该基准帧中目标对象的位置信息即为对应相机的第三视野范围内的目标对象在基准时间t的位置，即图中的(x³,y³)。Referring to Figure 4, it is assumed that there are 4 video frames, which are video frames collected by different target cameras, as shown by the gray squares in the figure. Numbers above the squares. The video frame numbered 3 is the first video frame collected among the four video frames, then the video frame numbered 3 can be used as the reference frame, and the time t corresponding to the timestamp of the video frame numbered 3 can be used as the reference time . For each video frame except the reference frame in the above-mentioned 4 video frames (that is, the video frames numbered 1, 2 and 4), the video frame can be regarded as the subsequent video frame, and the video frame preceding the video frame can be regarded as the video frame. as the preceding video frame. The position information of the target object in the previous video frame corresponding to the video frame numbered k (k=1, 2 or 4) is recorded as

The position information of the target object in the following video frame corresponding to the video frame numbered k is recorded as

through the pair

and

Perform interpolation to obtain the position (x ^k , y ^k ) of the target object in the third field of view of the corresponding camera at the reference time t. For the reference frame, the position information of the target object in the reference frame is the position of the target object in the third field of view of the corresponding camera at the reference time t, ie (x ³ , y ³ ) in the figure.

After obtaining the synchronous position information of the trajectory points within the third field of view of each target camera, the synchronous position information of the trajectory points within the third field of view of each target camera at the same reference time can be fused by means of weighted average , to obtain the position information of the track point at the reference time. Following the previous example, the weighted average of (x ¹ , y ¹ ), (x ² , y ² ), (x ³ , y ³ ) and (x ⁴ , y ⁴ ) can be obtained to obtain the track point at the reference time t location information.

In step 203, projection information of the multiple track points on the track of the target object may be acquired. Wherein, the trajectory may be obtained by: acquiring a predicted trajectory, where the predicted trajectory includes at least one predicted parameter; and determining each predicted parameter in the at least one predicted parameter based on the obtained position information of at least one trajectory point; The trajectory of the target object is determined according to each determined prediction parameter and the predicted trajectory.

In some embodiments, the predicted trajectory may be a straight line or a curve, and the curve may be a quadratic or more than quadratic curve. When the predicted trajectory is a straight line, the abscissa and ordinate of each point on the trajectory have a linear relationship, and the predicted trajectory can be written as:

y=ax+b;

Among them, (x, y) is the coordinates of the point on the predicted trajectory in the preset coordinate system, x and abscissa, y is the ordinate, a and b are both prediction parameters, where a and b are both constant, and a≠0.

When the predicted trajectory is a quadratic curve, the ordinate of each point on the trajectory can be expressed as a quadratic function of the abscissa of the corresponding point, so the predicted trajectory can be written as:

y=cx ² +dx+e;

Among them, (x, y) is the coordinates of the point on the predicted trajectory in the preset coordinate system, x and abscissa, y is the ordinate, c, d and e are prediction parameters, where c, d and e are constants, and c≠0.

In this embodiment, a priori hypothesis of the predicted trajectory can be generated first, that is, the predicted trajectory is assumed to be a straight line or a quadratic curve, and then the obtained position information of the trajectory points is used to determine the prediction parameters included in the predicted trajectory, thereby obtaining The trajectory of the target object. In other embodiments, it can also be assumed that the predicted trajectory is a curve of three or more times, but this a priori assumption is greatly affected by noise disturbance, and in most cases, the trajectory of the target object is not a strict sense Therefore, the predicted trajectory of the target object can generally be assumed to be the above quadratic curve. In this way, the influence of noise disturbance is reduced, and it is more in line with practical application scenarios.

After the prior hypothesis of the trajectory is determined, the prior hypothesis may be solved based on the position information of one or more track points to obtain each prediction parameter. In the embodiment in which the position information is fused, the position information used to solve the prior hypothesis is the fused position information. For example, when the a priori assumption is the above quadratic curve, the quadratic curve can be solved based on the three newly acquired trajectory points to obtain the values of the prediction parameters c, d and e, so as to determine the quadratic curve. curve. The determined quadratic curve is the trajectory of the target object.

During the movement of the target, the quadratic curve may change. Therefore, only a few newly acquired trajectory points can be used to determine the prediction parameters of the predicted trajectory, and at intervals, based on the newly acquired multiple trajectory points The prediction parameters of the predicted trajectory are updated, so as to continuously obtain a trajectory that conforms to the current actual motion of the target.

After the trajectory of the target object is obtained, projection information of the plurality of trajectory points on the trajectory of the target object can be obtained. In some embodiments, the distance between each group of adjacent trajectory points in the plurality of trajectory points may be determined; the distance between each group of adjacent trajectory points is projected to the tangent direction of the trajectory, Obtain the projected distance of each group of adjacent trajectory points on the trajectory; Projection information on the trajectory of the target object.

For example, the projection distance of each group of adjacent trajectory points on the trajectory can be directly used as the projection information of each group of adjacent trajectory points on the trajectory of the target object, or the The projection distances of each group of adjacent trajectory points on the trajectory are summed to obtain the projection information of the plurality of trajectory points on the trajectory of the target object.

When the projection information is obtained by summing the projection distances, referring to Figure 5, it is assumed that each dot represents the trajectory points of the object moving on the track (that is, the target object) at different positions, which are respectively recorded as P1, P2, ..., P7, these 7 track points are the multiple track points that the target object passes through during the movement in the second field of view. The plurality of trajectory points include a group of adjacent trajectory points P6 and P7, the solid line represents the trajectory of the object, as described above, the motion trajectory can be obtained based on the predicted trajectory and the predicted parameters, the dotted line represents the tangent direction of the trajectory, and in In the above scenario, the x-axis represents the direction of the track, and the y-axis represents the direction perpendicular to the track. Then, for the above-mentioned adjacent track points P6 and P7, the distance between P6 and P7 can be projected to the tangent direction of the track to obtain the projected distance Δ1. Among them, the tangent direction of the trajectory can be obtained by derivation of the trajectory. The projection distances of other groups of adjacent track points (for example, track points P5 and P6 ) on the track can be obtained in a similar manner, and details are not described herein again. After obtaining the projection distances of each group of adjacent trajectory points on the trajectory, the projection distances of each group of adjacent trajectory points on the trajectory may be summed to obtain the plurality of trajectory points on the trajectory. Projection information on the trajectory of the target object.

By projecting the distance in the method of this embodiment, and continuously superimposing the projection distances corresponding to each group of adjacent track points, the projection information used to represent the movement distance of the target object can be determined more accurately, thereby improving the determination of the movement information of the object. accuracy.

It should be noted that, in the above embodiment, in addition to the multiple track points described in step 202 (that is, the multiple track points used to determine the projection information in step 203 ), the track points used for determining the track are also Other trajectory points may be included. For example, in the embodiment shown in FIG. 5 , the plurality of trajectory points described in step 202 include the trajectory point P1 to the trajectory point P7 , wherein the trajectory point P1 and the trajectory point P2 are between the trajectory point P2 and the trajectory point P3 and/or between other adjacent track points in FIG. 5 , one or more other track points may be included. That is to say, the trajectory can be determined through more trajectory points, but the projection information is determined only based on some of the trajectory points. For example, when the trajectory of the target object is close to a straight line, or when the moving speed of the target object is low, or when the frame rate of the captured video frame is high, the error caused by only some of the collected track points is relatively small. Therefore, it is not necessary to use every acquired trajectory point to determine projection information. In this way, the amount of calculation is reduced, the processing efficiency is improved, and a high calculation accuracy is maintained.

In step 204, motion information of the target object may be determined based on the projection information. In some embodiments, the motion information may include, but is not limited to, the motion speed and/or acceleration of the target object.

In some embodiments, time information of each trajectory point in the plurality of trajectory points may be acquired, and based on the projection information and the time information of each trajectory point, the motion information of the target object is determined. Wherein, the time information of one trajectory point can be used to represent the time it takes for the target object to reach the trajectory point from a certain trajectory point (for example, the first trajectory point). Based on the difference between the time information of the two track points, the time interval at which the target object passes through the two track points can be determined.

In the case where the projection distance of each group of adjacent trajectory points on the trajectory is directly used as the projection information of each group of adjacent trajectory points on the trajectory of the target object in step 203, it can be based on The time information of the adjacent trajectory points among the plurality of trajectory points determines the time interval at which the target object passes through the adjacent trajectory points, and is based on the projected distance of the distance between the adjacent trajectory points on the trajectory. The ratio of the time interval between the target object and the time interval when the target object passes through the adjacent trajectory points determines the movement speed of the target object, which can be specifically recorded as:

v _n =Δl/Δt;

Among them, v _n is the movement speed of the target object when it passes the nth (n is a positive integer) trajectory point, and Δl is the projection of the distance between the nth trajectory point and the n-1th trajectory point on the trajectory Distance, Δt is the time interval between the time when the target object passes the nth trajectory point and the time when the target object passes through the n-1th trajectory point.

In some embodiments, referring to FIG. 6 , the projection distances of each group of adjacent trajectory points on the trajectory and the time information of each trajectory point can be added to the bidirectional queue continuously, and the latest data can be continuously read from the bidirectional queue. Projection distance and time information can obtain the movement speed of the target object.

in:

L _n and L _n-1 respectively represent the sum of the projection distances of the first n-1 groups of adjacent track points on the track and the sum of the projection distances of the first n-2 groups of adjacent track points on the track, respectively , where the first n-1 groups of adjacent trajectory points include the adjacent trajectory points composed of the first trajectory point and the second trajectory point, ..., the n-1th trajectory point and the nth trajectory point. The adjacent trajectory points of , t _n and t _n-1 respectively represent the time information of the nth trajectory point and the time information of the n-1th trajectory point, and n is a positive integer.

However, the movement speed obtained by the above-mentioned method of obtaining the movement speed fluctuates greatly. This is because there is noise in the processing process, so that the position information of the target object determined based on the video frame is often not accurate enough, so that the determined projection distance is not accurate. precise.

In order to solve the above problem, the present disclosure may sum the projection distances of each group of adjacent trajectory points on the trajectory to obtain the projection information of the plurality of trajectory points on the trajectory of the target object. On this basis, the motion model of the target object can be obtained first, and the motion model is used to represent the time-varying relationship of the motion distance of the target object, and the motion model includes at least one model parameter; The projection information of the at least one trajectory point and the acquired time information of each trajectory point in the at least one trajectory point are used to determine each model parameter in the at least one model parameter; Describe the motion information of the target object.

The motion model can be a uniform motion model, a uniform acceleration motion model or other motion models. For example, in a uniform motion model, the relationship between the movement distance of the target object and time is a linear relationship, which can be specifically recorded as:

L = rt;

Among them, r is the model parameter, L represents the moving distance of the target object, which can be approximated by the projection information of multiple trajectory points on the trajectory of the target object, and t represents the total time spent by the target object to pass through the multiple trajectory points, which can be Determined based on time information of the plurality of trajectory points.

In other embodiments, in the motion model of the target object, the motion distance is a quadratic function of time, which can be specifically recorded as:

L=r ₁ t ² +r ₂ t+r ₃ ;

Among them, r ₁ , r ₂ and r ₃ are all model parameters, L represents the moving distance of the target object, which can be approximated by the projection information of multiple trajectory points on the trajectory of the target object, and t represents that the target object passes through the plurality of trajectory points. The total duration of the trajectory points may be determined based on the time information of the plurality of trajectory points.

In other embodiments, other motion models may also be used according to actual conditions, which will not be listed one by one here.

After the motion model is determined, each model parameter of the at least one model parameter may be determined based on the acquired projection information of the at least one trajectory point and the acquired time information of each trajectory point in the at least one trajectory point, thereby Determine the motion information of the target object.

In some embodiments, the at least one trajectory point includes N trajectory points that have been acquired among the plurality of trajectory points, for example, the N trajectory points may include the most recently acquired N trajectory points. After the model parameters are determined based on the N track points, the motion information of the Mth track point of the target object in the N track points may be determined according to the model parameters and the motion model; M and N is a positive integer, and M is less than N. For example, in a specific numerical embodiment, the value of N is 60, and the value of M is 50. That is to say, the motion information of the 50th trajectory point among the 60 trajectory points can be determined based on the projection information and time information corresponding to the newly acquired 60 trajectory points. The reason why the obtained motion information is not the motion information corresponding to the most recently collected trajectory point (in this numerical example, the 60th trajectory point) is because the motion information of the target object may change abruptly during the motion process.

For example, referring to FIG. 7 , the target object hits the obstacle in the process of moving in a certain direction, and then continues to move in the opposite direction. Among them, the arrow indicates the moving direction of the target object, and v1 and v2 indicate the movement speed before hitting the obstacle and the moving speed after hitting the obstacle, respectively. In the case of a sudden change in the motion information of a certain trajectory point A, only determining the motion information of the trajectory point A based on the trajectory points collected before the trajectory point A will cause a large deviation between the obtained motion information and the actual situation. Therefore, In the above embodiment, the motion information of the trajectory point A is determined by means of several trajectory points collected after the trajectory point A, so as to improve the accuracy of the motion information obtained when the motion information changes abruptly.

In some embodiments, after obtaining the above-mentioned model parameters, Gaussian weighted least squares may be used to solve the motion model including the above-mentioned model parameters, so as to obtain the above-mentioned motion information corresponding to the M-th target trajectory point. Among them, the equation to be solved is as follows:

In the formula, ω _i is the weight corresponding to the ith trajectory point, and the weight includes the time weight and the velocity weight, where both the time weight and the velocity weight conform to the Gaussian distribution, and t _i and v _i respectively represent the ith trajectory point corresponding to Time information and motion speed, σ _t and σ _v respectively represent the variance corresponding to the time weight and the variance corresponding to the speed weight, t0 and v0 respectively represent the time information and the motion speed corresponding to the Mth trajectory point.

Then the tangential velocity v of the target object at the M-th trajectory point can be obtained by derivation of the motion model, which is specifically expressed as:

v=2×r ₁ t ₀ +r ₂ ;

The tangential acceleration Acc of the target at the Mth trajectory point can be obtained by calculating the second derivative of the motion model, which is specifically expressed as:

Acc=2×r ₁ .

Further, in the case that the target object moves on the surface of the reference object, and the surface of the target object is in contact with the surface of the reference object, it is also possible to determine, based on the acceleration information, that the surface of the target object and the surface of the reference object are in contact with each other. The friction coefficient μ between the surfaces of the reference objects, the friction coefficient μ can be determined based on the ratio between the acceleration information and the gravitational acceleration, and can be specifically expressed as:

Among them, g is the acceleration of gravity, and its value can be approximated as 9.8m/s ² .

In practical applications, since the track points are collected one by one during the movement of the object, after the motion information of the target object is determined based on the projection information, the following track points behind the multiple track points can also be obtained. track point; obtain the first distance between the trailing track point and the last track point in the plurality of track points; project the first distance on the tangent direction of the track to obtain the second distance ; Update the projection information based on the second distance. After the updated projection information is obtained, the step of determining the motion information of the target object based on the projection information can be returned, so as to continuously obtain the motion information of the target object during the movement process.

Still referring to the embodiment shown in FIG. 5 , since each trajectory point P1 to P7 is acquired in sequence, assuming that P1 is the starting trajectory point, in the case of acquiring the two trajectory points P1 and P2, you can first The projected distance Δl _{1 of the distance between P1 and P2 on the trajectory is determined, and the projected distance Δl 1 is} determined as projection information. After P3 is acquired, the projection distance Δl ₂ of the distance between P2 and P3 on the trajectory can be determined again, and Δl ₂ is superimposed on Δl ₁ to obtain updated projection information. By analogy in the above manner, the projection information of the target in the whole movement process can be obtained, and the step of determining the movement information of the target object based on the projection information is continuously repeated.

Those skilled in the art can understand that in the above method of the specific implementation, the writing order of each step does not mean a strict execution order but constitutes any limitation on the implementation process, and the specific execution order of each step should be based on its function and possible Internal logic is determined.

Referring to FIG. 8 , an embodiment of the present disclosure further provides a data processing apparatus for determining motion information of a target moving within the field of view of the camera; the apparatus includes:

A first acquisition module 801, configured to acquire video frames captured by multiple cameras in different first fields of view;

The first determining module 802 is configured to, based on the acquired video frames, determine the position information of multiple track points that the target object passes through during the movement in the second field of view; the second field of view covers the multiple cameras respectively. the corresponding first field of view;

The second obtaining module 803 is configured to obtain the projection information of the multiple track points on the track of the target object, where the track of the target object is determined based on the position information of the multiple track points; the projection information uses to characterize the movement distance of the target object;

The second determining module 804 is configured to determine motion information of the target object based on the projection information.

In some embodiments, the functions or modules included in the apparatuses provided in the embodiments of the present disclosure may be used to execute the methods described in the above method embodiments. For specific implementation, reference may be made to the descriptions of the above method embodiments. For brevity, here No longer.

The embodiments of the present specification further provide a computer device, which at least includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned embodiments when executing the program. method described.

FIG. 9 shows a more specific hardware structure diagram of a computing device provided by an embodiment of this specification. The device may include: a processor 901 , a memory 902 , an input/output interface 903 , a communication interface 904 and a bus 905 . The processor 901 , the memory 902 , the input/output interface 903 and the communication interface 904 realize the communication connection between each other within the device through the bus 905 .

The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit, central processing unit), a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute related program to implement the technical solutions provided by the embodiments of this specification. The processor 901 may further include a graphics card, and the graphics card may be an Nvidia titan X graphics card or a 1080Ti graphics card or the like.

The memory 902 may be implemented in the form of a ROM (Read Only Memory, read only memory), a RAM (Random Access Memory, random access memory), a static storage device, a dynamic storage device, and the like. The memory 902 may store an operating system and other application programs. When implementing the technical solutions provided by the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 902 and invoked by the processor 901 for execution.

The input/output interface 903 is used for connecting input/output modules to realize information input and output. The input/output/module can be configured in the device as a component (not shown in the figure), or can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, and the like.

The communication interface 904 is used to connect a communication module (not shown in the figure), so as to realize the communication interaction between the device and other devices. The communication module may implement communication through wired means (eg, USB, network cable, etc.), or may implement communication through wireless means (eg, sports network, WIFI, Bluetooth, etc.).

Bus 905 includes a path to transfer information between the various components of the device (eg, processor 901, memory 902, input/output interface 903, and communication interface 904).

It should be noted that although the above device only shows the processor 901, the memory 902, the input/output interface 903, the communication interface 904 and the bus 905, in the specific implementation process, the device may also include the necessary components for normal operation. other components. In addition, those skilled in the art can understand that, the above-mentioned device may only include components necessary to implement the solutions of the embodiments of the present specification, rather than all the components shown in the figures.

An embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, implements the method described in any of the foregoing embodiments.

Computer-readable media, including persistent and non-permanent, removable and non-removable media, can be implemented by any method or technology for storage of information. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

From the description of the above embodiments, those skilled in the art can clearly understand that the embodiments of the present specification can be implemented by means of software plus a necessary general hardware platform. Based on such understanding, the technical solutions of the embodiments of this specification or the parts that make contributions to the prior art may be embodied in the form of software products, and the computer software products may be stored in storage media, such as ROM/RAM, A magnetic disk, an optical disk, etc., includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments in this specification.

The systems, devices, modules or units described in the above embodiments may be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementing device is a computer, which may be in the form of a personal computer, laptop computer, cellular phone, camera phone, smart phone, personal digital assistant, media player, navigation device, email sending and receiving device, game control desktop, tablet, wearable device, or a combination of any of these devices.

Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and reference may be made to some descriptions of the method embodiments for related parts. The device embodiments described above are only illustrative, and the modules described as separate components may or may not be physically separated. When implementing the solutions of the embodiments of the present specification, the functions of each module may be integrated into the same module. or multiple software and/or hardware implementations. Some or all of the modules may also be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

The above are only specific implementations of the embodiments of the present specification. It should be pointed out that for those skilled in the art, without departing from the principles of the embodiments of the present specification, several improvements and modifications can be made. These Improvements and modifications should also be regarded as the protection scope of the embodiments of the present specification.

Claims (13)

1. A method for determining motion information, the method comprising:

acquiring video frames acquired by a plurality of cameras in different first visual field ranges respectively;

determining position information of a plurality of track points which are passed by the target object in the process of moving in the second visual field range based on the acquired video frame; the second visual field range covers the first visual field ranges respectively corresponding to the plurality of cameras;

acquiring projection information of the plurality of track points on the track of the target object, wherein the track of the target object is determined based on the position information of the plurality of track points; the projection information is used for representing the movement distance of the target object;

determining motion information of the target object based on the projection information.

2. The method of claim 1, wherein determining the trajectory of the target object based on the position information of the plurality of trajectory points comprises:

obtaining a predicted trajectory, wherein the predicted trajectory comprises at least one prediction parameter;

determining each prediction parameter of the at least one prediction parameter based on the acquired position information of the at least one track point;

and determining the track of the target object according to each determined prediction parameter and the predicted track.

3. The method according to claim 1 or 2, wherein the obtaining of the projection information of the plurality of track points on the track of the target object comprises:

determining the distance between each group of adjacent track points in the plurality of track points;

projecting the distance between each group of adjacent track points to the tangential direction of the track to obtain the projection distance of each group of adjacent track points on the track;

and determining the projection information of the plurality of track points on the track of the target object according to the projection distance of the distance between each group of adjacent track points on the track.

4. A method according to any of claims 1 to 3, characterized in that the method further comprises:

acquiring time information of each track point in the plurality of track points;

the determining motion information of the target object based on the projection information includes:

and determining the motion information of the target object based on the projection information and the time information of each track point.

5. The method of any of claims 1 to 4, wherein after determining motion information of the target object based on the projection information, the method further comprises:

acquiring a rear track point behind the plurality of track points;

acquiring a first distance between the rear track point and the last track point in the plurality of track points;

projecting the first distance to the tangential direction of the track to obtain a second distance;

updating the projection information based on the second distance.

6. The method of any of claims 1-5, wherein the determining motion information of the target object based on the projection information comprises:

obtaining a motion model of the target object, wherein the motion model is used for representing the change relation of the motion distance of the target object along with time, and the motion model comprises at least one model parameter;

determining each model parameter in the at least one model parameter based on the acquired projection information of the at least one track point and the acquired time information of each track point in the at least one track point;

and determining the motion information of the target object according to the determined model parameters and the motion model.

7. The method of claim 6, wherein the at least one trace point comprises N trace points acquired from the plurality of trace points; the determining the motion information of the target object according to the determined model parameters and the motion model includes:

determining the motion information of the Mth track point of the target object in the N track points according to the determined model parameters and the motion model;

m and N are positive integers, and M is less than N.

8. The method according to any one of claims 1-7, wherein the motion information comprises acceleration information, the target object moves on a surface of a reference object, the surface of the target object is in contact with the surface of the reference object; the method further comprises the following steps:

determining a coefficient of friction between a surface of the target object and a surface of the reference object based on the acceleration information.

9. The method according to any one of claims 1-8, wherein determining position information of a plurality of track points that the target object passes through during the movement in the second field of view based on the acquired video frames comprises:

determining initial position information of track points in a first visual field range of each camera based on a video frame acquired by each camera;

and fusing initial position information of the track points in the first view range of each camera to obtain the position information of the plurality of track points.

10. The method of claim 9, wherein the first fields of view of at least two target cameras of the plurality of cameras partially overlap; the initial position information of the track point in the first field of view scope to each camera fuses, obtains the positional information of a plurality of track points, includes:

determining initial position information of track points in a third visual field range of each target camera, wherein the third visual field range is an overlapped visual field range of each target camera;

synchronizing initial position information of the track points in the third view range of each target camera to obtain synchronous position information of the track points in the third view range of each target camera;

and fusing the synchronous position information of the track points in the third view range of each target camera to obtain the position information of the track points in the third view range of each target camera.

11. An apparatus for determining motion information, the apparatus comprising:

the first acquisition module is used for acquiring video frames acquired by the plurality of cameras in different first visual field ranges;

the first determining module is used for determining the position information of a plurality of track points which are passed by the target object in the process of moving in the second visual field range based on the acquired video frame; the second visual field range covers the first visual field ranges respectively corresponding to the plurality of cameras;

the second acquisition module is used for acquiring projection information of the plurality of track points on the track of the target object, and the track of the target object is determined based on the position information of the plurality of track points; the projection information is used for representing the movement distance of the target object;

a second determination module to determine motion information of the target object based on the projection information.

12. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out the method of any one of claims 1 to 10.

13. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 10 when executing the program.

Images