CN111784224B - Object tracking method and device, control platform and storage medium - Google Patents

Abstract

The present disclosure relates to an object tracking method and apparatus, a control platform, and a storage medium. The object tracking method comprises the following steps: acquiring current positioning information of all table-board objects on a control platform; the current positioning information of all table-board objects on the control platform is respectively matched with the pre-stored historical motion trail of each object to be tracked; and taking the current positioning information of the successfully matched table-board object as the current positioning information of the corresponding object to be tracked. The present disclosure may enable tracking of object motion without prediction.

Description

Object tracking method and device, control platform and storage medium

Technical Field

The present disclosure relates to the field of logistics, and in particular to an object tracking method and device, a control platform and a storage medium.

Background Art

In the current environment of rapid development of logistics, transportation and warehouse automation, some scenes in unmanned warehouses require low-speed motion control of objects such as square boxes (low speed is defined as less than 2m/s). For example, intelligent conveyor belts and other similar platform-type control devices. Therefore, it is necessary to first accurately locate and track the square box, and provide the acquired box information (such as geometric dimensions, position, posture, box number, historical motion path) to the control device, so as to jointly complete real-time closed-loop control.

Summary of the invention

The inventors found through research that: since the control part of the related technology has the expected control path for each box, it can first make a prediction based on the positioning result of the previous frame (or more previous frames), the time interval t from the previous frame (or more previous frames) to the current frame, and the expected path, and estimate the positioning information predicted in the current frame; then match the actual positioning result in the current frame with it. The logic design is completed based on the matching result, thereby further completing the tracking.

However, the prediction of related technologies requires the positioning results of the previous frame (or more previous frames), the time interval t from the previous frame (or more previous frames) to the current frame, and the expected path, which naturally requires corresponding time resources and space resources (such as computing resources). If related technologies want to use the results of more previous frames for more accurate predictions, they will require more resources.

In view of at least one of the above technical problems, the present disclosure provides an object tracking method and device, a control platform and a storage medium, which can track the movement of an object without prediction.

According to one aspect of the present disclosure, there is provided an object tracking method, comprising:

Pre-store the historical motion trajectory of each object to be tracked;

Get the current positioning information of all objects on the control platform;

For each object to be tracked, searching for current positioning information that matches the historical motion trajectory of the object to be tracked from the current positioning information of all objects on the table;

The current positioning information of the successfully matched table object is used as the current positioning information of the corresponding object to be tracked.

In some embodiments of the present disclosure, the successfully matched table object and the object to be tracked are the same object.

In some embodiments of the present disclosure, the object tracking method further includes:

Output the current motion trajectory of the object to be tracked, wherein the current motion trajectory includes the historical motion trajectory and the current positioning information.

In some embodiments of the present disclosure, for each object to be tracked, searching for current positioning information that matches the historical motion trajectory of the object to be tracked from the current positioning information of all table objects includes:

For each object to be tracked, current positioning information matching the latest positioning information in the historical motion trajectory of the object to be tracked is searched from the current positioning information of all the objects on the table.

In some embodiments of the present disclosure, for each object to be tracked, searching the current positioning information of all table objects for the current positioning information that matches the latest positioning information in the historical motion trajectory of the object to be tracked includes:

Calculating the distance between the current positioning information of each table object and the latest positioning information of the object to be tracked;

Determining whether the distance is less than a predetermined threshold;

When the distance is less than a predetermined threshold, it is determined that the table object whose distance is less than the predetermined threshold is successfully matched with the object to be tracked.

In some embodiments of the present disclosure, the acquiring of current positioning information of all table objects on the control platform includes: acquiring current positioning information of all table objects on the control platform at predetermined positioning time intervals.

In some embodiments of the present disclosure, the predetermined threshold is smaller than the movement distance of the object within a predetermined positioning time interval, and the predetermined threshold is smaller than the minimum distance from the center of the object to the edge of the object.

In some embodiments of the present disclosure, the object tracking method further includes:

Determine whether there is current positioning information of a table object that fails to match the historical motion trajectories of all objects to be tracked;

When there is current positioning information of a tabletop object that fails to match the historical motion trajectories of all objects to be tracked, the tabletop object is used as a new object to be tracked, and the current positioning information of the tabletop object is used as the current positioning information of the object to be tracked.

In some embodiments of the present disclosure, the object tracking method further includes:

Determine whether there is a historical motion trajectory of the object to be tracked that fails to match the current positioning information of all the objects on the table;

In the case that there is a historical motion trajectory of the object to be tracked that fails to match the current positioning information of all the table objects, the historical motion trajectory of the object to be tracked is deleted from the set of objects to be tracked.

In some embodiments of the present disclosure, deleting the historical motion trajectory of the object to be tracked from the set of objects to be tracked includes:

Add 1 to the number of tracking anomalies of the object to be tracked;

Determining whether the number of tracking anomalies of the object to be tracked is greater than or equal to a predetermined number;

When the number of tracking anomalies of the object to be tracked is greater than or equal to a predetermined number, the historical motion trajectory of the object to be tracked is deleted from the set of objects to be tracked.

According to another aspect of the present disclosure, there is provided an object tracking device, comprising:

A historical trajectory storage module is used to pre-store the historical motion trajectory of each object to be tracked;

A positioning information acquisition module is used to obtain the current positioning information of all objects on the control platform;

A motion trajectory matching module is used to search, for each object to be tracked, the current positioning information of all the objects on the table for the current positioning information that matches the historical motion trajectory of the object to be tracked;

The object motion tracking module is used to use the current positioning information of the successfully matched table object as the current positioning information of the corresponding object to be tracked.

In some embodiments of the present disclosure, the object tracking device is used to perform operations to implement the object tracking method described in any of the above embodiments.

According to another aspect of the present disclosure, there is provided an object tracking device, comprising:

A memory for storing instructions;

The processor is used to execute the instructions so that the device performs operations to implement the object tracking method as described in any of the above embodiments.

According to another aspect of the present disclosure, a control platform is provided, characterized by comprising an object tracking device as described in any of the above embodiments.

According to another aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the object tracking method as described in any of the above embodiments is implemented.

The present disclosure can track the motion of an object without prediction.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

1 and 2 are schematic diagrams of a control platform in some embodiments of the present disclosure.

FIG. 3 is a schematic diagram of some embodiments of a related art object tracking method.

FIG. 4 is a schematic diagram of some embodiments of the object tracking method disclosed herein.

FIG. 5 is a schematic diagram of other embodiments of the object tracking method disclosed herein.

FIG. 6 is a schematic diagram of a scenario of some embodiments of the present disclosure.

FIG. 7 is a schematic diagram of some embodiments of the object tracking device disclosed herein.

FIG. 8 is a schematic diagram of other embodiments of the object tracking device disclosed herein.

DETAILED DESCRIPTION

The technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

The relative arrangement of components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.

In all examples shown and discussed herein, any specific values should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

Figures 1 and 2 are schematic diagrams of control platforms in some embodiments of the present disclosure. Figures 1 and 2 are schematic diagrams of intelligent conveyor belts and similar control platforms. It is necessary to accurately locate and track objects such as square boxes and packages, and provide the acquired object information (such as geometric size, position, posture, box number, historical motion path) to the control platform to jointly complete real-time closed-loop control.

In the scenarios of Figures 1 and 2, typical control generally requires a shooting rate of at least 30 frames per second, that is, at least 30 positioning and tracking information per second is required to complete real-time control (the interval between each positioning and tracking information is less than 33ms). General 3D depth cameras can achieve a data acquisition rate of 30 frames per second. If it is a 2D color camera or other sensor, the frame rate can be higher. So it is assumed that both the data acquisition and positioning parts have met this requirement.

When objects such as square boxes move continuously at a low speed, it is assumed that the positioning part has been completed, that is, the geometric position and posture information of all objects such as square boxes on the platform has been obtained in each shot (each frame). The next step is to track these square boxes.

Tracking disclosed herein refers to: matching the positioning results of all square boxes on the table (current frame) (for example, there are N square boxes, and the size, center point coordinates, rotation angle, and other information of each box are known, referred to as geometric pose information in this disclosure. It is assumed that the geometric pose information corresponds to the box one by one) with the positioning results of the square boxes on the table obtained in the previous frame (for example, there are M square boxes of geometric pose information) to calculate: 1) each box in the current frame corresponds to which previous box, and is given the same number; 2) the historical motion trajectory of all boxes can be obtained. When the tracking part is completed, we can provide the tracking result of the current frame to the control device, and the control device will perform real-time closed-loop control on each box according to the expected path based on the number, historical motion trajectory information, etc.

FIG3 is a schematic diagram of some embodiments of the object tracking method of the related art. As shown in FIG3, the object tracking method of the related art may include: since the control part has an expected control path for each box, the related art can be based on the positioning result of the previous frame (or more previous frames), the time interval t from the previous frame (or more previous frames) to the current frame, and the expected path, first predict and estimate the positioning information predicted in the current frame; then match the positioning result actually obtained in the current frame with it. Complete the logic design based on the matching result, thereby further completing the tracking.

The inventor discovered through research that the related technology has some technical problems:

1. The positioning results of related technologies are generally obtained by camera vision or other sensors. We call the one that provides this function the positioning module; and the control device the control module. Whether the tracking function (module) is placed in the positioning module, the control module, or a separate module, it needs to interact with the positioning module or (and) the control module multiple times in each tracking calculation. Because the tracking module not only needs the current frame positioning result provided by the positioning module, but also needs the predicted positioning information provided by the control and positioning modules, which will require more information transmission and interaction in each tracking calculation.

2. The prediction of related technologies requires the positioning results of the previous frame (or more previous frames), the time interval t from the previous frame (or more previous frames) to the current frame, and the expected path, which naturally requires corresponding time resources and space resources (such as computing resources). If the related technology wants to use the results of more previous frames for more accurate prediction, it will require more resources.

3. Related technologies can also use template-based, deep learning-based and other technologies, but they will have certain restrictions. Template-based technologies require target modeling, or require certain restrictions on target type characteristics. Deep learning-based technologies also have restrictions on target type characteristics and require pre-training. The length, width, and height dimensions of the square boxes in the scenes shown in Figures 1 and 2 are unknown (they can be varied), and the texture of the box itself is very simple (or the texture of different boxes is consistent except for the size). In addition, these technologies have high requirements for time resources or space resources.

4. The relevant technology does not fully consider the conditions of the box entering and exiting the platform, abnormal situations, etc.

In view of at least one of the above technical problems, the present disclosure provides an object tracking method and device, a control platform and a storage medium, which are described below through embodiments.

The present invention proposes an object tracking method that does not require prediction and has simple and complete logic for the tracking part of a square box under low-speed motion control.

This disclosure only focuses on the tracking part, and assumes that the positioning results of all boxes in each positioning cycle (e.g., each frame) have been obtained. The positioning results mainly refer to geometric posture information, including the geometric size (length and width) of the square box, posture information (box center point coordinates, rotation angle), etc.

The typical control in the typical scenarios mentioned in this disclosure generally has the following prerequisites for the scenarios and control conditions:

1) The square box moves at a low speed, less than 2m/s. Calculated at 30 frames per second, the time interval for obtaining each positioning information is about 33ms. The calculation shows that the moving distance of the box does not exceed 2m/s*33ms＝66mm＝0.066m.

2) The size of the object such as a square box cannot be too small, otherwise it will be difficult to be controlled by the control device (platform). Generally speaking, the side length of a square box, for example, will be at least 15 cm.

3) There will be multiple boxes on the control platform at the same time, and each box needs to be controlled separately from entering the platform to leaving.

In response to the control requirements in such scenarios, the present disclosure proposes a tracking method for low-speed motion control of objects such as square boxes.

FIG4 is a schematic diagram of some embodiments of the object tracking method disclosed in the present invention. Preferably, this embodiment can be performed by the object tracking device disclosed in the present invention. The method may include the following steps:

Step 40: pre-store the historical motion trajectory of each object to be tracked.

Step 41, obtaining the current positioning information of all table objects on the control platform.

In some embodiments of the present disclosure, step 41 may include: acquiring current positioning information of all table objects on the control platform at predetermined positioning time intervals.

In some embodiments of the present disclosure, step 41 may include: using a sensor such as a camera to obtain current positioning information of all moving table objects on the table that already have coordinate information.

Step 42: for each object to be tracked, search the current positioning information of all the objects on the table for the current positioning information that matches the historical motion trajectory of the object to be tracked.

In some embodiments of the present disclosure, step 42 may include: for each object to be tracked, searching the current positioning information of all table objects for current positioning information that matches the latest positioning information in the historical motion trajectory of the object to be tracked.

In some embodiments of the present disclosure, for each object to be tracked, the step of searching for current positioning information that matches the latest positioning information in the historical motion trajectory of the object to be tracked from the current positioning information of all table objects includes: calculating the distance between the current positioning information of each table object and the latest positioning information of the object to be tracked; determining whether the distance is less than a predetermined threshold; and if the distance is less than the predetermined threshold, determining that the table object with a distance less than the predetermined threshold is successfully matched with the object to be tracked.

In some embodiments of the present disclosure, the distance between the current positioning information of each tabletop object and the latest positioning information of each object to be tracked may be the distance between the center point coordinate position in the current positioning information of each tabletop object and the center point coordinate position in the latest positioning information of each object to be tracked.

In some embodiments of the present disclosure, the distance between the center point coordinate position of each tabletop object and the center point coordinate position of each to-be-tracked object may be the Euclidean distance or other distance between the two.

In some embodiments of the present disclosure, the predetermined threshold is smaller than the movement distance of the object within a predetermined positioning time interval; and the predetermined threshold is smaller than the minimum distance from the center of the object to the edge of the object.

In other embodiments of the present disclosure, step 42 may include: using the overlap of objects in two adjacent positioning information (such as the intersection area) as a judgment criterion to determine whether the table object matches the object to be tracked.

For example, step 42 may include determining whether the intersection area of a surface (e.g., bottom surface) in the current positioning information of each table object and the corresponding surface (e.g., bottom surface) in the latest positioning information in the historical motion trajectory of each object to be tracked is less than a predetermined area, to determine whether the table object matches the object to be tracked.

For the scenarios and controls mentioned in the above embodiments of the present disclosure, it is recommended to use a threshold-based, Euclidean distance discrimination criterion. The criterion in the above embodiments of the present disclosure is simple, requires little calculation, and is sufficiently effective.

Step 43: The current positioning information of the successfully matched table object is used as the current positioning information of the corresponding object to be tracked.

In some embodiments of the present disclosure, the successfully matched table object and the object to be tracked are the same object.

In some embodiments of the present disclosure, the object tracking method may further include: determining whether there is current positioning information of a table top object that fails to match the historical motion trajectories of all objects to be tracked; if there is current positioning information of a table top object that fails to match the historical motion trajectories of all objects to be tracked, treating the table top object as a new object to be tracked, and treating the current positioning information of the table top object as the current positioning information of the object to be tracked.

In some embodiments of the present disclosure, the object tracking method may further include: in the case of creating a new object to be tracked, adding a path state value for the new object to be tracked, wherein the path state is normal or abnormal, wherein a normal (state) path refers to a box that has truly newly entered the control platform and can only enter from a designated position of the control platform; an abnormal (state) path refers to other types of entry other than the designated position (such as a box thrown into the platform out of thin air, etc., used to characterize the situation where the control platform is not entered normally from the conveyor belt).

In some embodiments of the present disclosure, the object tracking method may further include: determining whether there is a historical motion trajectory of the object to be tracked that fails to match the current positioning information of all table objects; if there is a historical motion trajectory of the object to be tracked that fails to match the current positioning information of all table objects, deleting the historical motion trajectory of the object to be tracked from the set of objects to be tracked.

In some embodiments of the present disclosure, deleting the historical motion trajectory of the object to be tracked from the set of objects to be tracked includes: adding 1 to the number of tracking exceptions of the object to be tracked; determining whether the number of tracking exceptions of the object to be tracked is greater than or equal to a predetermined number; and deleting the historical motion trajectory of the object to be tracked from the set of objects to be tracked if the number of tracking exceptions of the object to be tracked is greater than or equal to a predetermined number.

In some embodiments of the present disclosure, the object tracking method may further include: when the current positioning information of a tabletop object successfully matches the historical motion trajectory of an object to be tracked, determining whether the tabletop object has left the control platform; when the tabletop object has not left the control platform, executing an operation (i.e., a path update operation) using the current positioning information of the successfully matched tabletop object as the current positioning information of the object to be tracked, and then deleting the object to be tracked from the set of objects to be tracked, and deleting the tabletop object from the set of tabletop objects; when the tabletop object has left the control platform, deleting the object to be tracked from the set of objects to be tracked, and deleting the tabletop object from the set of tabletop objects.

Based on the object tracking method provided by the above embodiment of the present disclosure, a complete process and method for tracking a square box is provided to meet the needs of platform-type control devices such as intelligent conveyor belts for low-speed motion control (real-time closed-loop control) of objects such as square boxes. The above embodiment of the present disclosure does not require interaction with other modules, does not require an expected motion path for the box, and does not require real-time prediction of current positioning information.

Based on the actual control and operating conditions in the scene, the above embodiment of the present disclosure only needs positioning information of each frame (each predetermined positioning time interval) to complete the tracking of multiple boxes, covering the complete life cycle of the boxes from entering the platform to leaving, as well as possible abnormal situations. The "path set" viewpoint proposed in the above embodiment of the present disclosure is logically clear and complete, and requires less time and space resources.

The present disclosure tracks the box from the perspective of "path set" to complete real-time tracking of any box. The "path set" disclosed in the present disclosure is more like the historical motion trajectory of each box, which is different from the control path of the control part. The control path of the control part is based on the expected path and the current posture of the box, and sends control instructions to the box to make it move according to the expected path. The "path set" here considers the state of each box from the perspective of historical motion trajectory, from entering the platform to leaving the platform.

FIG5 is a schematic diagram of other embodiments of the object tracking method of the present disclosure. Preferably, the present embodiment can be performed by the object tracking device of the present disclosure. The object in the embodiment of FIG5 is a square box, and the object in the embodiment of FIG5 can also be other objects such as a package. The object tracking method of the embodiment of FIG5 may include the following steps:

Step 1: Get the positioning result list of the current frame, which indicates the positioning information of all square boxes on the table in the current frame, totaling M boxes. The positioning result of one of them is recorded as m, 0≤m≤(M-1).

Step 2: The “path collection” stores the historical motion trajectories of each square box (again, non-control paths).

In some embodiments of the present disclosure, only the latest pose information is stored in the path set, that is, the last update result of the current frame.

There are a total of N paths in the path set of the above embodiment of the present disclosure, which means the historical motion trajectory (position information, etc.) of each of the N boxes currently tracked. One of the paths is recorded as n, 0≤n≤(N-1). The path n represents the geometric position information of the square box that was last updated (for example, in the previous frame) up to the current frame.

In some embodiments of the present disclosure, it can be expanded to all historical posture information as required, and all historical posture information can be used to perform weighted judgment on the matching results.

Step 3, match the path n and the positioning result m based on a predetermined threshold. If the match is successful, it indicates that the positioning result m belongs to the path n, that is, the square box corresponding to the positioning result m is matched to the square box corresponding to (tracked by) the path n, and then execute step 4-1. Otherwise, if the match is unsuccessful, execute step 5-1.

In some embodiments of the present disclosure, threshold-based matching: mainly based on the center point coordinates in the pose information. Based on the latest position information (center point coordinates) of path n and the position information (center point coordinates) of positioning result m, the Euclidean distance between the two is calculated.

In some embodiments of the present disclosure, a predetermined threshold δ=0.07m may be set. When the Euclidean distance between the two is less than the threshold, the match is considered successful, otherwise it is considered a failure.

In some embodiments of the present disclosure, the conditions for the threshold to be established are (satisfied simultaneously): 1) less than the movement distance within one frame. Since the square box moves at a low speed, the speed is less than 2m/s; the control requires that the update of the positioning and tracking information is at least 30 frames per second, that is, the time interval of each group is less than 33ms, and the calculated movement distance does not exceed 2m/s*33ms=0.066m; condition 1) is met; 2) less than half of the shortest side length of the box. Considering the control requirements of the control device, the side length of the square box will be at least 15cm; condition 2) is met.

Step 4-1, according to the positioning result m, whether the box has left the platform. If it has left the platform, execute step 4-2; otherwise, if it has not left the platform, it means that the box is still within the control range of the control platform, and execute step 4-3.

In some embodiments of the present disclosure, a criterion for determining whether the box has left the platform is: whether all four corners of the box have left the control platform.

Step 4-2, if it has left the platform, then delete the positioning result m from the positioning result list of the current frame. The match has been successful here, so there is no need to match the positioning result m in the future. Deleting it from the list can reduce the amount of subsequent calculations. After step 4-2, jump to step 6, that is, perform the "path extinction" operation.

Step 4-3, enter the "path update" operation, then execute step 4-4.

In some embodiments of the present disclosure, the "path update" operation may include: updating the information of the positioning result m to the path n, so that the path n obtains the latest geometric posture information of the corresponding box and is associated with the previous historical motion trajectory. At the same time, the update exception counter of the path n (which will be explained in detail in "path establishment") is set to 0.

Step 4-4, delete the positioning result m from the positioning result list of the current frame; then jump to step 0.

Step 5-1, continue to determine whether all positioning results of the current frame have been traversed. If not (all positioning results of the current frame have not been traversed), execute step 5-2; if yes (all positioning results of the current frame have been traversed), it means that path n does not match any positioning result of the current frame. Then path n enters the "path extinction" operation, that is, jump to step 6.

Step 5-2, set m=m+1, and then continue to match the next one in the positioning result list (positioning result m+1) with path n based on the threshold, that is, jump back to step 0.

Step 6: Perform the “path extinction” operation.

In some embodiments of the present disclosure, the "path extinction" operation is used to indicate the extinction (deletion) operation of a path, indicating that the life cycle of the path has ended and the path (and its corresponding box) does not need to be tracked anymore.

In some embodiments of the present disclosure, step 6 may include:

Step 6-1, determine whether the box has left the platform. The method for determining whether the box has left the platform is the same as that described in step 4-1.

Step 6-2, if the box has left, that is, in the case of step 4-2, the life cycle of the path naturally ends, which is a normal extinction: immediately delete the path n from the "path set".

Step 6-3, if the box has not left, that is, path n does not match any positioning result, it indicates that there may be abnormal situations such as lost tracking. In order to cover such abnormal situations, the present disclosure considers forcibly (abnormally) eliminating path n.

In some embodiments of the present disclosure, step 6-3 may include: setting an update exception counter in path n (which will be explained in detail in "path establishment"); first, the counter value is increased by 1, and then it is determined whether it reaches a preset value, such as 3 times (indicating that there is no update for 3 consecutive frames. This value can be set according to the scene, but it is recommended not to be too large, otherwise it will increase the possibility of errors in the matching stage of the path and the positioning result). If the threshold is reached, the path n is immediately deleted from the "path set" (forced extinction); if it is not reached, go directly to the subsequent step 7.

Step 7, determine whether all paths in the path set have been traversed. If not, enter the next loop and repeat the above steps 3 to 6 for the next path (path n+1); if all paths in the path set have been traversed, indicating that all paths in the path set have been processed, then enter step 0.

Step 8, determine whether the positioning result list is empty. If the list is not empty, it indicates that there is a positioning result (denoted as j) that is not matched to any path in the path set, and then execute step 9. If the list is empty, it indicates that all positioning results of the current frame have been successfully matched, and then execute step 10.

In the above embodiment of the present disclosure, in step 4-2 and step 4-4 (threshold-based matching, when the match is successful), the matched positioning result m is deleted, so all the matched positioning results are deleted. If all the positioning results of the current frame are matched, the list will be empty.

Step 9, enter the "path establishment" operation, and then enter step 10.

In some embodiments of the present disclosure, step 9 may include: when the positioning result j does not match any path in the path set, the present disclosure creates a new path in the path set (there are a total of M paths in the previous path set, then this path is the M+1th path). This path has the following characteristics:

a) A path has a state, which indicates whether the path is normal or abnormal.

The normal (state) path refers to a box that has just entered the control platform: it can only enter from a certain specified position of the control platform, as shown in the scenario diagram of Figure 6.

Abnormal (state) paths refer to other types of entry except for the designated location (such as boxes thrown into the platform out of thin air, etc., which are used to represent the situation where the control platform is not entered normally from the conveyor belt).

In the present disclosure, when a positioning result j appears (no match), a new path will be created for the positioning result, and the positioning result will be updated to the newly created path. Except for the state being normal or abnormal, the rest of the states and operations of the path (such as path update and extinction, etc.) are exactly the same.

The advantage of dividing the paths into normal and abnormal in the above embodiment of the present disclosure is that dividing the paths into normal and abnormal can distinguish whether the box enters the platform normally or is lost (or thrown in out of thin air, etc.). More importantly, the abnormal path can give the box corresponding operations, such as alarm, such as moving to a specified location. It is only necessary to set a unified processing method for the abnormal path. However, for the path itself (and all processes in the tracking method), there is no difference between the operation of the abnormal path and the normal state.

b) The path also has an "update exception counter", which is set to 0 when the path is established (both for newly created normal and abnormal state paths, because they have just been "updated" at this time). A non-zero value indicates how many times the path has not been updated in a row. For example, if the update exception counter is 3, it means that the path has not been updated for 3 consecutive times (3 consecutive times, it has not successfully matched all the positioning results of the current frame).

Step 10: Output the tracking information of the current frame, that is, the current "path set". This tracking is completed.

In some embodiments of the present disclosure, step 10 may include: providing the latest status of each path in the "path set" (including necessary information, indicating the latest situation of each box being tracked) to the control part, so that the control part can be told the number, geometric position posture, and historical motion trajectory of each box on the current table.

The above is the entire process of completing a tracking (after obtaining the positioning result of the current frame).

Each path in the path set of the above embodiment of the present disclosure includes three stages, namely, path establishment → path update → path extinction. Compared with considering each tracking for the square box itself (i.e., after each positioning, determining whether the box is a new entry, whether the box has left, which box the box corresponds to, etc.), the path set viewpoint proposed in the above embodiment of the present disclosure can only consider the latest state of the path, whether it is established, updated or extinct, when the box constantly enters and leaves the platform, and when an abnormality occurs.

The above embodiment of the present disclosure does not need to predict the positioning information of the current frame based on the expected path of the control module (no prediction is required), and does not need to interact with other modules (no interaction is required). Only the positioning result of each time is used to complete the tracking of the square box. Therefore, the present disclosure has little demand on time resources and space resources.

The above-mentioned embodiments of the present disclosure may be applicable to typical scenarios of the embodiments of FIG. 1 and FIG. 2 .

The above-mentioned embodiments of the present disclosure can also be adapted to the scene schematic diagram shown in FIG6. As shown in FIG6, the square box moves from the conveyor belt and enters the platform-type control device from a specified side (specified position. In the actual scene, it can only enter from the specified position, and there can be many positions for leaving), and then is controlled and leaves the platform according to a preset path (such as indicated by a green arrow or a yellow arrow). In the process from the square box entering the platform to leaving the platform, the square box is tracked and real-time geometric posture information is output to the control platform.

In the above embodiment of the present disclosure, there can be multiple square boxes on the table at the same time, and each of them needs to be controlled separately.

FIG7 is a schematic diagram of some embodiments of the object tracking device of the present disclosure. As shown in FIG7 , the object tracking device of the present disclosure may include a history track storage module 70, a positioning information acquisition module 71, a motion track matching module 72 and an object motion tracking module 73, wherein:

The historical trajectory storage module 70 is used to pre-store the historical motion trajectory of each object to be tracked.

The positioning information acquisition module 71 is used to acquire the current positioning information of all table objects on the control platform.

In some embodiments of the present disclosure, the positioning information acquisition module 71 may be used to acquire the current positioning information of all table objects on the control platform at a predetermined positioning time interval.

The motion trajectory matching module 72 is used to search, for each object to be tracked, the current positioning information of all the objects on the table for the current positioning information that matches the historical motion trajectory of the object to be tracked.

In some embodiments of the present disclosure, the motion trajectory matching module 72 may be used to search, for each object to be tracked, the current positioning information of all table objects for the current positioning information that matches the latest positioning information in the historical motion trajectory of the object to be tracked.

In some embodiments of the present disclosure, the motion trajectory matching module 72 can be used to calculate the distance between the current positioning information of each table object and the latest positioning information of each object to be tracked; determine whether the distance is less than a predetermined threshold; if the distance is less than the predetermined threshold, determine that the table object with a distance less than the predetermined threshold is successfully matched with the object to be tracked.

In some embodiments of the present disclosure, the distance between the current positioning information of each table object and the latest positioning information of each object to be tracked may be the distance between the center point coordinate position in the current positioning information of each table object and the center point coordinate position in the latest positioning information of each object to be tracked.

In some embodiments of the present disclosure, the predetermined threshold is smaller than the movement distance of the object within a predetermined positioning time interval; the predetermined threshold is smaller than the minimum distance from the center of the object to the edge of the object.

In some embodiments of the present disclosure, the motion trajectory matching module 72 may be configured to use the overlap of objects in two adjacent positioning information (such as the intersection area) as a criterion to determine whether the table object matches the object to be tracked.

In some embodiments of the present disclosure, the motion trajectory matching module 72 can be used to determine whether the intersection area of a surface (e.g., bottom surface) in the current positioning information of each table object and the corresponding surface (e.g., bottom surface) in the latest positioning information in the historical motion trajectory of each object to be tracked is less than a predetermined area, so as to determine whether the table object matches the object to be tracked.

For the scenarios and controls mentioned in the above embodiments of the present disclosure, it is recommended to use a threshold-based, Euclidean distance discrimination criterion. The above embodiments of the present disclosure have simple criteria, small calculation amount, and are sufficiently effective.

The object motion tracking module 73 is used to use the current positioning information of the successfully matched table object as the current positioning information of the corresponding object to be tracked.

In some embodiments of the present disclosure, the successfully matched table object and the object to be tracked are the same object.

In some embodiments of the present disclosure, the object tracking device may also be used to output a current motion trajectory of the object to be tracked, wherein the current motion trajectory includes a historical motion trajectory and current positioning information.

In some embodiments of the present disclosure, the object tracking device can also be used to determine whether there is current positioning information of a table top object that fails to match the historical motion trajectories of all objects to be tracked; if there is current positioning information of a table top object that fails to match the historical motion trajectories of all objects to be tracked, the table top object is used as a new object to be tracked, and the current positioning information of the table top object is used as the current positioning information of the object to be tracked.

In some embodiments of the present disclosure, the object tracking device can also be used to determine whether there is a historical motion trajectory of the object to be tracked that fails to match the current positioning information of all table objects; if there is a historical motion trajectory of the object to be tracked that fails to match the current positioning information of all table objects, the historical motion trajectory of the object to be tracked is deleted from the set of objects to be tracked.

In some embodiments of the present disclosure, the object tracking device can be used to add 1 to the number of tracking exceptions of the object to be tracked when the historical motion trajectory of the object to be tracked is deleted from the set of objects to be tracked; determine whether the number of tracking exceptions of the object to be tracked is greater than or equal to a predetermined number; and delete the historical motion trajectory of the object to be tracked from the set of objects to be tracked when the number of tracking exceptions of the object to be tracked is greater than or equal to a predetermined number.

In some embodiments of the present disclosure, the object tracking device can also be used to determine whether a tabletop object has left the control platform when the current positioning information of the tabletop object successfully matches the historical motion trajectory of an object to be tracked; when the tabletop object has not left the control platform, perform an operation of using the current positioning information of the successfully matched tabletop object as the current positioning information of the corresponding object to be tracked (i.e., a path update operation), and then delete the object to be tracked from the set of objects to be tracked, and delete the tabletop object from the set of tabletop objects; when the tabletop object has left the control platform, delete the object to be tracked from the set of objects to be tracked, and delete the tabletop object from the set of tabletop objects.

In some embodiments of the present disclosure, the object tracking device is used to perform operations of implementing the object tracking method described in any of the above embodiments (for example, any of the embodiments in FIG. 4 or FIG. 5 ).

Based on the object tracking device provided by the above embodiment of the present disclosure, a complete process and method for tracking a square box is provided to meet the needs of platform-type control devices such as intelligent conveyor belts for low-speed motion control (real-time closed-loop control) of objects such as square boxes. The above embodiment of the present disclosure does not require interaction with other modules, does not require an expected motion path for the box, and does not require real-time prediction of current positioning information.

FIG8 is a schematic diagram of some other embodiments of the object tracking device of the present disclosure. As shown in FIG8 , the object tracking device of the present disclosure may include a memory 81 and a processor 82, wherein:

The memory 81 is used for storing instructions.

The processor 82 is used to execute the instructions so that the device performs operations to implement the object tracking method as described in any of the above embodiments (for example, any of the embodiments in FIG. 4 or FIG. 5 ).

Based on the actual control and operating conditions in the scene, the above embodiment of the present disclosure only needs positioning information of each frame (each predetermined positioning time interval) to complete the tracking of multiple boxes, covering the complete life cycle of objects such as boxes from entering the platform to leaving, as well as possible abnormal situations. The "path set" viewpoint proposed in the above embodiment of the present disclosure is logically clear and complete, and requires less time and space resources.

According to another aspect of the present disclosure, a control platform is provided, comprising an object tracking device as described in any of the above embodiments (eg, any of the embodiments in FIG. 7 or FIG. 8 ).

Based on the control platform provided by the above-mentioned embodiment of the present disclosure, the box is tracked from the perspective of "path set" to complete the real-time tracking of any box. The "path set" of the present disclosure is more like the historical motion trajectory of each box, which is different from the control path of the control part. The control path of the control part is based on the expected path and the current posture of the box, and a control instruction is issued to the box to make it move according to the expected path. The "path set" here considers the state of each box from the perspective of the historical motion trajectory, from entering the platform to leaving the platform.

According to another aspect of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the object tracking method as described in any of the above embodiments (for example, any of the embodiments in Figure 4 or Figure 5) is implemented.

Each path in the path set of the above embodiment of the present disclosure includes three stages, namely, path establishment → path update → path extinction. Compared with considering each tracking for the square box itself (i.e., after each positioning, determining whether the box is a new entry, whether the box has left, which box the box corresponds to, etc.), the path set viewpoint proposed in the above embodiment of the present disclosure can only consider the latest state of the path, whether it is established, updated or extinct, when the box constantly enters and leaves the platform, and when an abnormality occurs.

The above embodiment of the present disclosure does not need to predict the positioning information of the current frame based on the expected path of the control module (no prediction is required), and does not need to interact with other modules (no interaction is required). Only the positioning result of each time is used to complete the tracking of the square box. Therefore, the present disclosure has little demand on time resources and space resources.

The object tracking device described above can be implemented as a general-purpose processor, a programmable logic controller (PLC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components or any appropriate combination thereof for performing the functions described in the present application.

So far, the present disclosure has been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed here.

A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

The description of the present disclosure is given for the purpose of illustration and description, and is not intended to be exhaustive or to limit the present disclosure to the disclosed form. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments are selected and described in order to better illustrate the principles and practical applications of the present disclosure, and to enable those of ordinary skill in the art to understand the present disclosure and thereby design various embodiments with various modifications suitable for specific uses.

Claims (12)

1. An object tracking method, comprising: Pre-store the historical motion trajectory of each object to be tracked; Get the current positioning information of all objects on the control platform; For each object to be tracked, searching for current positioning information that matches the historical motion trajectory of the object to be tracked from the current positioning information of all objects on the table; The current positioning information of the successfully matched table object is used as the current positioning information of the corresponding object to be tracked; Wherein, the object tracking method further includes: Determine whether there is current positioning information of a table object that fails to match the historical motion trajectories of all objects to be tracked; In the case where there is current positioning information of a tabletop object that fails to match the historical motion trajectories of all objects to be tracked, the tabletop object is used as a new object to be tracked, and the current positioning information of the tabletop object is used as the current positioning information of the object to be tracked; When a new object to be tracked is created, a path state value is added for the new object to be tracked, where the path state is normal or abnormal, the normal state path is the path of the object to be tracked entering from the specified position of the control platform, the abnormal state path is the path of the object to be tracked entering from the abnormal position of the control platform, and the abnormal position is a position other than the specified position. 2. The object tracking method according to claim 1, characterized in that the successfully matched table object and the object to be tracked are the same object; The object tracking method further comprises: Output the current motion trajectory of the object to be tracked, wherein the current motion trajectory includes the historical motion trajectory and the current positioning information. 3. The object tracking method according to claim 1, wherein for each object to be tracked, searching for current positioning information that matches the historical motion trajectory of the object to be tracked from the current positioning information of all table objects comprises: For each object to be tracked, current positioning information matching the latest positioning information in the historical motion trajectory of the object to be tracked is searched from the current positioning information of all table objects. 4. The object tracking method according to claim 3, characterized in that, for each object to be tracked, searching the current positioning information of all table objects for the current positioning information that matches the latest positioning information in the historical motion trajectory of the object to be tracked comprises: Calculating the distance between the current positioning information of each table object and the latest positioning information of the object to be tracked; Determining whether the distance is less than a predetermined threshold; When the distance is less than a predetermined threshold, it is determined that the table object whose distance is less than the predetermined threshold is successfully matched with the object to be tracked. 5. The object tracking method according to claim 4, characterized in that: The obtaining of the current positioning information of all the objects on the control platform comprises: obtaining the current positioning information of all the objects on the control platform at a predetermined positioning time interval; The predetermined threshold is smaller than the moving distance of the object within a predetermined positioning time interval, and the predetermined threshold is smaller than the minimum distance from the center of the object to the edge of the object. 6. The object tracking method according to any one of claims 1 to 5, further comprising: Determine whether there is a historical motion trajectory of the object to be tracked that fails to match the current positioning information of all the objects on the table; In the case that there is a historical motion trajectory of the object to be tracked that fails to match the current positioning information of all the table objects, the historical motion trajectory of the object to be tracked is deleted from the set of objects to be tracked. 7. The object tracking method according to claim 6, wherein the deleting the historical motion trajectory of the object to be tracked from the set of objects to be tracked comprises: Add 1 to the number of tracking anomalies of the object to be tracked; Determining whether the number of tracking anomalies of the object to be tracked is greater than or equal to a predetermined number; When the number of tracking anomalies of the object to be tracked is greater than or equal to a predetermined number, the historical motion trajectory of the object to be tracked is deleted from the set of objects to be tracked. 8. An object tracking device, comprising: A historical trajectory storage module is used to pre-store the historical motion trajectory of each object to be tracked; A positioning information acquisition module is used to obtain the current positioning information of all objects on the control platform; A motion trajectory matching module is used to search, for each object to be tracked, the current positioning information of all the objects on the table for the current positioning information that matches the historical motion trajectory of the object to be tracked; The object motion tracking module is used to use the current positioning information of the successfully matched table object as the current positioning information of the corresponding object to be tracked; Among them, the object tracking device is also used to determine whether there is current positioning information of a tabletop object that fails to match the historical motion trajectories of all objects to be tracked; in the case that there is current positioning information of a tabletop object that fails to match the historical motion trajectories of all objects to be tracked, the tabletop object is used as a new object to be tracked, and the current positioning information of the tabletop object is used as the current positioning information of the object to be tracked; in the case of creating a new object to be tracked, a path state value is added to the new object to be tracked, wherein the path state is normal or abnormal, the normal state path is the path of the object to be tracked entering from the specified position of the control platform, the abnormal state path is the path of the object to be tracked entering from the abnormal position of the control platform, and the abnormal position is a position other than the specified position. 9. The object tracking device according to claim 8, characterized in that the object tracking device is used to perform operations to implement the object tracking method according to any one of claims 2 to 7. 10. An object tracking device, comprising: A memory for storing instructions; A processor is used to execute the instructions so that the device performs operations to implement the object tracking method according to any one of claims 1 to 7. 11. A control platform, characterized by comprising the object tracking device according to any one of claims 8 to 10. 12. A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, and when the instructions are executed by a processor, the object tracking method according to any one of claims 1 to 7 is implemented.