CN115187904A - Object motion state detection method, device and storage medium - Google Patents

Abstract

The present invention provides a method, a device and a storage medium for detecting a motion state of an object. The method includes: collecting a video stream in a predetermined area in real time, and extracting N image frames corresponding to the current moment in the video stream based on a sliding window method; processing Obtain the N position coordinate information corresponding to the calibration object in the N image frames, and combine the N image frames to determine the observation sequence value set after excluding the predetermined observation sequence value; based on the dynamic hidden Markov model corresponding to the observation sequence value The set obtains the set of state sequence values; the motion state of the target object at the current moment in the N image frames is determined by observing the set of sequence values and the set of state sequence values; the calibration object belongs to the target object. Because this scheme is based on the dynamic hidden Markov model to filter out the interference factors caused by misrecognition and missed identification, and can better deal with the difficulties faced by the standard hidden Markov model, and thus can improve the recognition accuracy of the vehicle motion state.

Description

Object motion state detection method, device and storage medium

technical field

Embodiments of the present invention relate to the technical field of motion detection, and in particular, to a method, a device, and a storage medium for detecting a motion state of an object.

Background technique

In a smart logistics park, the management of vehicles and the management of loading and unloading require efficient coordination to fully ensure the efficiency of logistics operations. Vehicle identification is a very important link in vehicle management, and vehicle status identification itself has different scenarios, such as vehicle identification at road gates and vehicle identification at platforms, usually using computer vision methods. For example, the vehicle body is identified. When the distance between the position of the vehicle body and the platform is lower than a certain threshold, it is determined that the vehicle arrives. If the vehicle body cannot be recognized in the platform area, it is determined to be leaving. The major disadvantage of the current technology is that it can only judge whether there is a vehicle in the platform area, but cannot identify the motion state of the vehicle. In addition, due to the complex scene of the platform and many interferences, the recognition accuracy of the vehicle motion state is low.

SUMMARY OF THE INVENTION

The object motion state detection method, device, and storage medium provided by the embodiments of the present invention can improve the recognition accuracy of the vehicle motion state.

The technical scheme of the present invention is realized as follows:

An embodiment of the present invention provides a method for detecting a motion state of an object, including:

Collecting the video stream of the predetermined area in real time, and extracting N image frames corresponding to the current moment in the video stream based on the sliding window method; N is an integer greater than 1;

processing to obtain N pieces of positional coordinate information corresponding to the calibration objects in the N image frames, and combining the N image frames to determine a set of observation sequence values after excluding the predetermined observation sequence values; wherein, the predetermined observation sequence values are The sequence value is the observation sequence value corresponding to the predetermined position coordinate information; the predetermined position coordinate information is the position coordinate information obtained by processing corresponding to the image frame that does not include the calibration object;

Obtaining a state sequence value set corresponding to the observation sequence value set based on the dynamic hidden Markov model;

Through the observation sequence value set and the state sequence value set, the motion state of the target object at the current moment in the N image frames is determined; the calibration object belongs to the target object.

In the above solution, the state sequence value set obtained based on the dynamic hidden Markov model corresponding to the observation sequence value set includes:

It is detected that the subscript between the i-th observation sequence value in the observation sequence value set and the previous observation sequence value is continuous, based on the i-th observation sequence value, the first preset state transition probability matrix and the first The preset observation probability matrix is processed by a preset dynamic programming algorithm to obtain the ith group of M state sequences corresponding to the ith observation sequence value; i is an integer greater than or equal to 1; M is an integer greater than 1;

Detecting the subscript interval T values between the i+1th observation sequence value and the ith observation sequence value, based on the second preset state transition probability matrix corresponding to the i+1th observation sequence value , the third state transition probability matrix is obtained by calculation; the second preset state transition probability matrix represents the transition probability set between M state sequence values when corresponding to the i+1th observation sequence value; T is greater than or equal to an integer of 1;

Based on the ith group of M state sequences, the ith+1th observation sequence value, the third state transition probability matrix and the third preset observation probability matrix are processed by the preset dynamic programming algorithm to obtain the ith The i+1th group of M state sequences corresponding to the i+1 observation sequence values, until the M final state sequences corresponding to the last observation sequence value in the observation sequence value set are obtained;

determining the final state sequence value of the target state sequence with the largest transition probability among the M final state sequences;

Combining the final state sequence value and the preset state sequence matrix, a plurality of state sequence values are determined to obtain the state sequence value set.

In the above solution, the calculation to obtain a third state transition probability matrix based on the second preset state transition probability matrix corresponding to the i+1th observation sequence value includes:

In combination with the second preset state transition probability matrix, the probability values corresponding to the first state sequence value and the M state sequence values are calculated to form the first row of the third state transition probability matrix;

Until combined with the second preset state transition probability matrix, the probability values corresponding to the M th state sequence values and the M state sequence values respectively are calculated to form the M th row of the third state transition probability matrix, and then The third state transition probability matrix is obtained.

In the above solution, in combination with the second preset state transition probability matrix, the probability values corresponding to the first state sequence value and the M state sequence values are calculated to form the third state transition probability matrix. The first line, including:

Extracted from the second preset state transition probability matrix, the first state sequence value is respectively multiplied by the transition probability between the M state sequence values to obtain the first product;

When T is 1, calculate the K th product corresponding to the M state sequence values and the K th state sequence value respectively, and add the first product and the K th product to form the third state transition probability matrix The K th probability value of the first row in , until the M th probability value of the first row of the third state transition probability matrix is obtained to form the first row of the third state transition probability matrix; K is An integer greater than or equal to 1 and less than or equal to M.

In the above solution, the first state sequence value extracted from the second preset state transition probability matrix is multiplied by the transition probabilities between the M state sequence values respectively, and after the first product is obtained, the The method also includes:

When T is greater than 1, extract the transition probability between any two state sequence values in the M state sequence values, and multiply them to obtain a second product;

Calculate the K th product corresponding to the M state sequence values and the K th state sequence value respectively, and add the first product, T-1 second products and the K th product to form the third state transition The Kth probability value of the first row in the probability matrix is obtained until the Mth probability value of the first row of the third state transition probability matrix is obtained, so as to form the first row of the third state transition probability matrix.

In the above solution, based on the i-th group of M state sequences, the i+1-th observation sequence value, the third state transition probability matrix, and the third preset observation probability matrix pass through the preset dynamic programming algorithm. processing to obtain the i+1th group of M state sequences corresponding to the i+1th observation sequence value, until after obtaining the M final state sequences corresponding to the last observation sequence value in the observation sequence value set, the Before determining a plurality of state sequence values in combination with the preset state sequence matrix corresponding to the final state sequence value and the target state sequence, to obtain the set of state sequence values, the method further includes:

The M final state sequences are arranged in the order of columns, so as to obtain the preset state sequence matrix.

In the above solution, the combination of the final state sequence value and the preset state sequence matrix to determine a plurality of state sequence values to obtain the state sequence value set, including:

A target sequence value matching the final state sequence value is determined in the last row of the preset state sequence matrix, and then the final state sequence in the column corresponding to the target sequence value is determined as the plurality of state sequence values.

In the above solution, the sliding window-based method extracts N image frames corresponding to the current moment in the video stream, including:

extracting the current image frame corresponding to the current moment in the video stream;

Taking the current image frame as a starting point in the video stream, extracting an image frame along the time axis every predetermined duration or a predetermined number of image frames, until N-1 image frames are extracted;

The N image frames are obtained by combining the current image frame and the N-1 image frames along the time axis.

In the above solution, the processing obtains N pieces of position coordinate information corresponding to the calibration objects in the N image frames, and combines the N image frames to determine the set of observation sequence values after excluding the predetermined observation sequence values, include:

The N image frames are processed by a preset detection model to obtain the N position coordinate information;

The N image frames are divided into regions respectively, and the observation sequence value of the region to which it belongs is determined in the corresponding image frame for each position coordinate information, so as to obtain N observation sequence values corresponding to the N position coordinate information ;

The predetermined observation sequence value is eliminated from the N observation sequence values to obtain the observation sequence value set.

In the above solution, the N image frames are divided into regions respectively, and the observation sequence value of the corresponding region is determined in the corresponding image frame for each position coordinate information, so as to obtain the corresponding N position coordinate information. The N observed sequence values of , including:

The N image frames are respectively divided into U regions along the ordinate to obtain U regions corresponding to each image frame; U is an integer greater than 1;

Determine the belonging area in the corresponding U areas for each position coordinate information;

The vertical order of the belonging region is determined in the U regions, and the vertical order is used as the observation sequence value of each position coordinate information, and then the N observation sequence values are obtained.

In the above solution, determining the motion state of the target object at the current moment in the N image frames through the observation sequence value set and the state sequence value set, including:

Combining the state sequence value set and the observation sequence value set, eliminating the interference position coordinate information in the N position coordinate information to obtain a plurality of target position coordinate information;

Using the plurality of target position coordinate information, the motion state is determined.

In the above solution, the state sequence value set and the observation sequence value set are combined to eliminate the interference position coordinate information in the N position coordinate information, and obtain a plurality of target position coordinate information, including:

Compare multiple state sequence values in the state sequence value set with multiple observation sequence values in the observation sequence value set one by one in order, if at least one state sequence value and at least one observation in the corresponding order are determined If the sequence values are different, the at least one second position coordinate information corresponding to the at least one observation sequence value is eliminated from the N position coordinate information to obtain a plurality of intermediate position coordinate information;

The predetermined position coordinate information in the plurality of intermediate position coordinate information is eliminated to obtain the plurality of target position coordinate information.

In the above solution, determining the motion state by using the plurality of target position coordinate information includes:

performing linear fitting on the plurality of target ordinate information in the plurality of target position coordinate information along the time axis to obtain slope values of the plurality of target ordinate information;

The motion state is determined according to the magnitude of the slope value.

In the above solution, determining the motion state according to the magnitude of the slope value includes one of the following:

If the slope value is greater than a preset threshold, determining that the motion state is a close state;

If the slope value is smaller than the preset threshold, it is determined that the motion state is a distance state.

The embodiment of the present invention also provides an object motion state detection device, including:

The acquisition and extraction unit is used for real-time acquisition of the video stream of the predetermined area, and extracts N image frames corresponding to the current moment in the video stream based on the sliding window method; N is an integer greater than 1;

a processing unit, configured to process and obtain N pieces of position coordinate information corresponding to the calibration objects in the N image frames, and combine the N image frames to determine an observation sequence value set after excluding the predetermined observation sequence value; wherein , the predetermined observation sequence value is the observation sequence value corresponding to the predetermined position coordinate information; the predetermined position coordinate information is the position coordinate information obtained by processing corresponding to the image frame that does not include the calibration object;

The processing unit is further configured to obtain a state sequence value set corresponding to the observation sequence value set based on the dynamic hidden Markov model;

A determination unit, configured to determine the motion state of the target object at the current moment in the N image frames through the observation sequence value set and the state sequence value set; the calibration object belongs to the target object.

Embodiments of the present invention also provide an object motion state detection device, including a memory and a processor, where the memory stores a computer program that can be run on the processor, and the processor implements the steps in the above method when executing the program.

Embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, implements the steps in the above method.

In the embodiment of the present invention, a video stream in a predetermined area is collected in real time, and N image frames corresponding to the current moment are extracted from the video stream based on a sliding window method; N is an integer greater than 1; The N pieces of position coordinate information corresponding to the calibration object are calibrated, and combined with the N image frames, an observation sequence value set after excluding the predetermined observation sequence value is determined; wherein, the predetermined observation sequence value is the observation sequence value corresponding to the predetermined position coordinate information; The position coordinate information is the position coordinate information obtained by processing the image frames that do not include the calibration object; the state sequence value set is obtained based on the dynamic hidden Markov model corresponding to the observation sequence value set; the observation sequence value set and the state sequence value set are determined. The motion state of the target object at the current moment in the N image frames is obtained; the calibration object belongs to the target object. Because the scheme is based on the dynamic hidden Markov model to filter out the interference factors caused by misrecognition and missed identification, and can better deal with the difficulties faced by the standard hidden Markov model, and thus can improve the recognition accuracy of the vehicle motion state.

Description of drawings

1 is an optional schematic flowchart of a method for detecting a motion state of an object provided by an embodiment of the present invention;

2 is a schematic diagram of an optional effect of a method for detecting a motion state of an object provided by an embodiment of the present invention;

3 is an optional schematic flowchart of a method for detecting a motion state of an object provided by an embodiment of the present invention;

4 is an optional schematic flowchart of a method for detecting a motion state of an object provided by an embodiment of the present invention;

5 is an optional schematic flowchart of a method for detecting a motion state of an object provided by an embodiment of the present invention;

6 is an optional schematic flowchart of a method for detecting a motion state of an object provided by an embodiment of the present invention;

7 is a schematic diagram of an optional effect of a method for detecting a motion state of an object provided by an embodiment of the present invention;

8 is a schematic structural diagram of an object motion state detection device provided by an embodiment of the present invention;

FIG. 9 is a schematic diagram of a hardware entity of an apparatus for detecting a motion state of an object provided by an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention are further elaborated below in conjunction with the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations of the present invention. All other embodiments obtained without creative work fall within the protection scope of the present invention.

In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" can be the same or a different subset of all possible embodiments, and Can be combined with each other without conflict.

If a similar description of "first/second" appears in the invention document, the following description will be added. In the following description, the term "first\second\third" involved is only to distinguish similar objects, and does not mean For a specific ordering of objects, it can be understood that "first\second\third" may be interchanged in a specific order or sequence if permitted, so that the embodiments of the present invention described herein can be used in other than those shown in the drawings. performed in an order other than that shown or described.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terms used herein are for the purpose of describing the embodiments of the present invention only, and are not intended to limit the present invention.

An embodiment of the present invention provides a method for detecting a motion state of an object. Please refer to FIG. 1 , which is an optional schematic flowchart of the method for detecting a motion state of an object provided by an embodiment of the present invention, which will be described in conjunction with the steps shown in FIG. 1 .

S101. Collect a video stream of a predetermined area in real time, and extract N image frames corresponding to the current moment from the video stream based on a sliding window method.

In the embodiment of the present invention, the object motion state detection device collects the video stream of the predetermined area in real time, and extracts N image frames corresponding to the current moment from the video stream based on the sliding window method. Wherein, N is an integer greater than 1.

In the embodiment of the present invention, the object motion state detection apparatus collects the video stream of the predetermined area through the camera set in the predetermined area. The object motion state detection device extracts one image frame every predetermined number of image frames, and uses the sliding window method to determine N image frames corresponding to the current moment.

In the embodiment of the present invention, the object motion state detection device extracts the image frame corresponding to the current moment in the video stream, and extracts multiple image frames at a predetermined time interval forward along the time axis to obtain N image frames.

In this embodiment of the present invention, the object motion state detection device may be a terminal or a server connected to a camera in a predetermined area.

In this embodiment of the present invention, the predetermined area captured by the camera remains unchanged, and the camera captures the video stream of the predetermined area in real time. After the object motion state detection device detects that the object (that is, the vehicle) enters the predetermined area, the object motion state detection device extracts N image frames corresponding to the current moment in the video stream based on the sliding window method.

S102, processing to obtain N position coordinate information corresponding to the calibration object in the N image frames, and combining the N image frames to determine an observation sequence value set after excluding the predetermined observation sequence value.

In the embodiment of the present invention, the object motion state detection device obtains N position coordinate information corresponding to the calibration object in the N image frames, and combines the N image frames to determine the observation sequence value set after excluding the predetermined observation sequence value . Wherein, the predetermined observation sequence value is the observation sequence value corresponding to the predetermined position coordinate information; the predetermined position coordinate information is the position coordinate information obtained by processing corresponding to the image frame that does not include the calibration object. Exemplarily, the predetermined position coordinate information may be (-1, -1).

In the embodiment of the present invention, the object motion state detection device processes N image frames by using a preset detection model, and obtains N position coordinate information corresponding to the calibration object in the N image frames.

In the embodiment of the present invention, the object motion state detection device may process N image frames by using a Warped Planar Object Detection Network (WPOD-NET) model to obtain N corresponding to the calibration objects in the N image frames. location coordinate information. In the embodiment of the present invention, the apparatus for detecting the motion state of the object may also process the N image frames by using other preset detection models, which is not limited in the embodiment of the present invention.

In the embodiment of the present invention, the object motion detection apparatus divides the N image frames into regions respectively. The object motion detection device determines the area to which each position coordinate information belongs in the corresponding image frame. The object motion detection device determines N observation sequence values of the N pieces of position coordinate information according to the respective regions corresponding to the N pieces of position coordinate information. The object motion detection device removes a predetermined observation sequence value from the N observation sequence values to obtain a set of observation sequence values.

In the embodiment of the present invention, the N image frames include a target object, and the target object has a calibration object. Exemplarily, the target object may be a car, and the corresponding calibration object may be a license plate, a door or a wheel. The target object can also be any moving object, and the calibration object can be a predetermined part on the arbitrary moving object.

In the embodiment of the present invention, the object motion state detection device uses a computer vision method to identify the arrival and departure of the vehicle 13 in the platform scene. During the identification process of the object motion state detection device, the camera 12 is arranged above the warehouse door 11 of the platform 10, and the identification is performed when the vehicle 13 is reversing and approaching the warehouse door 11 of the platform 10, as shown in FIG. 2 . For the video stream collected by the camera 12, this solution takes into account the computing power requirements of different algorithms, and collects and identifies at a frequency of 1 frame every 5 frames. The overall scheme can judge the action of the vehicle by recognizing the dynamics of the vehicle body itself or the dynamics of the license plate 13. Considering that the license plate is the unique information of the vehicle 13, the license plate recognition can eliminate the interference of some other objects that are easily regarded as the vehicle 13. And the final purpose of identifying the vehicle 13 is to identify the license plate, and directly using the license plate information to identify can reduce the modules for vehicle body identification. Therefore, this scheme performs license plate detection on the pictures collected in each frame, and judges the dynamics of vehicle arrival and departure through the change of the position of the recognized license plate in each frame.

In this embodiment of the present invention, after an image frame is given, the object motion state detection device can return the range of the area corresponding to the license plate in the image by using a preset detection model. The object motion state detection device determines the coordinates of the four vertices of the license plate through the WPOD-NET algorithm model. If no license plate is detected in any image frame, the detection result is empty. For each image frame, the object motion state detection device determines the horizontal and vertical coordinates of the center point of the license plate area through the WPOD-NET algorithm model combined with four vertex coordinates. If the license plate is not detected in any image frame, the corresponding horizontal and vertical coordinates are used - 1 indicates. Then, in the case of continuous acquisition by the camera, the object motion state detection device can obtain two coordinate sequences, respectively corresponding to the abscissa and the ordinate.

S103, obtaining a state sequence value set based on the corresponding observation sequence value set of the dynamic hidden Markov model.

In the embodiment of the present invention, the object motion state detection device obtains the state sequence value set based on the observation sequence value set corresponding to the dynamic hidden Markov model.

In the embodiment of the present invention, the object motion state detection device detects that the subscript between the i-th observation sequence value and the previous observation sequence value in the observation sequence value set is continuous, based on the i-th observation sequence value, the first preset The state transition probability matrix and the first preset observation probability matrix are processed by a preset dynamic programming algorithm to obtain the ith group of M state sequences corresponding to the ith observation sequence value; i is an integer greater than or equal to 1; M is greater than 1 is an integer; the subscript interval T values between the i+1th observation sequence value and the ith observation sequence value are obtained by detecting, and based on the second preset state transition probability matrix corresponding to the i+1th observation sequence value, The third state transition probability matrix is obtained by calculation; based on the ith group of M state sequences, the i+1th observation sequence value, the third state transition probability matrix and the third preset observation probability matrix are processed by a preset dynamic programming algorithm to obtain The i+1th group of M state sequences corresponding to the i+1th observation sequence value, until the M final state sequences corresponding to the last observation sequence value in the observation sequence value set are obtained; the transition is determined in the M final state sequences The final state sequence value of the target state sequence with the highest probability; combining the final state sequence value and the preset state sequence matrix, multiple state sequence values are determined to obtain a set of state sequence values.

S103: Determine the motion state of the target object at the current moment in the N image frames by observing the sequence value set and the state sequence value set.

In the embodiment of the present invention, the object motion state detection device determines the motion state of the target object at the current moment in the N image frames by observing the sequence value set and the state sequence value set. The calibration object belongs to the target object.

In the embodiment of the present invention, the object motion state detection device combines N image frames, removes interfering position coordinate information in the N position coordinate information, and obtains a plurality of target position coordinate information. The object motion state detection device determines the current moment motion state of the target object in the N image frames by using a plurality of target position coordinate information.

The object motion detection device compares the state sequence values in the state sequence value set with multiple observation sequence values one by one in order to determine the abnormal observation sequence value, and then the object motion detection device can eliminate the abnormal observation sequence from the N position coordinate information. The position coordinate information corresponding to the value is obtained, and then multiple target position coordinate information is obtained.

In the embodiment of the present invention, the object motion state detection device can remove the predetermined position coordinate information representing the image frame of the undetected license plate from the N position coordinate information, and then obtain a plurality of target position coordinate information.

In the embodiment of the present invention, the object motion state detection device performs linear fitting on a plurality of target position coordinate information to obtain a slope value. The object motion state detection device determines the current motion state of the target object according to the magnitude of the slope value.

Exemplarily, the arbitrary position coordinate information may be (X, Y). Wherein, X represents the abscissa of the midpoint of the calibration object corresponding to the image frame, and Y represents the ordinate of the midpoint of the calibration object corresponding to the image frame.

In the embodiment of the present invention, the object motion state detection device only retains N pieces of position coordinate information recently collected at the current moment, therefore, the sequence of the abscissa and the ordinate is represented as X and Y respectively. The lengths of the two sequences are both N, and the value of N is 20 in this scheme. The dynamic judgment method of vehicle arrival or departure in this scheme is to judge the trend of Y. If the vehicle is from far to near, the value of the element in Y is on an upward trend, and from near to far, it is a downward trend. In order to realize this logic, this scheme adds a time subscript sequence t, and performs linear fitting on Y and t, that is, y=at+b. If the fitted slope a is greater than a certain threshold, it is determined that the vehicle is in the entering state, and if a is less than a certain threshold, it is determined that the vehicle is in the leaving state. Since not all the license plates can be recognized in the pictures, before fitting, all elements with a value of -1 in Y need to be removed. Assuming that the length after removal is M, then t=[1, 2...M], and then perform linear fitting. If the value of M is too small, that is, there are too few valid data, such as less than 10, the fitting will not be performed, and the current vehicle motion state will not be judged.

In the embodiment of the present invention, a video stream in a predetermined area is collected in real time, and N image frames corresponding to the current moment are extracted from the video stream based on a sliding window method; N is an integer greater than 1; The N pieces of position coordinate information corresponding to the calibration object are calibrated, and combined with the N image frames, an observation sequence value set after excluding the predetermined observation sequence value is determined; wherein, the predetermined observation sequence value is the observation sequence value corresponding to the predetermined position coordinate information; The position coordinate information is the position coordinate information obtained by processing the image frames that do not include the calibration object; the state sequence value set is obtained based on the dynamic hidden Markov model corresponding to the observation sequence value set; the observation sequence value set and the state sequence value set are determined. The motion state of the target object at the current moment in the N image frames is obtained; the calibration object belongs to the target object. Because the scheme is based on the dynamic hidden Markov model to filter out the interference factors caused by misrecognition and missed identification, and can better deal with the difficulties faced by the standard hidden Markov model, and thus can improve the recognition accuracy of the vehicle motion state.

In some embodiments, referring to FIG. 3 , FIG. 3 is an optional schematic flowchart of the object motion state detection method provided by the embodiment of the present invention. S103 shown in FIG. 1 can also be implemented through S105 to S109 , and each step will be combined with each other. Be explained.

S105. Detecting that the subscript between the i-th observation sequence value in the observation sequence value set and the previous observation sequence value is continuous, based on the i-th observation sequence value, the first preset state transition probability matrix and the first preset The observation probability matrix is processed by a preset dynamic programming algorithm to obtain the ith group of M state sequences corresponding to the ith observation sequence value.

In the embodiment of the present invention, the object motion state detection device detects that the subscript between the i-th observation sequence value and the previous observation sequence value in the observation sequence value set is continuous, based on the i-th observation sequence value, the first preset The state transition probability matrix and the first preset observation probability matrix are processed by a preset dynamic programming algorithm to obtain the ith group of M state sequences corresponding to the ith observation sequence value. i is an integer greater than or equal to 1; M is an integer greater than 1.

In the embodiment of the present invention, the object motion detection apparatus may process each observation sequence value through the viterbi algorithm. to obtain M state sequences corresponding to each observation sequence value. When the object motion detection device detects that the subscript between the ith observation sequence value and the previous observation sequence value is continuous (that is, the difference between the image frame corresponding to the ith observation sequence value and the image frame of the previous observation sequence value) the interval time remains unchanged). The object motion detection device is processed by a preset dynamic programming algorithm based on the ith observation sequence value, the first preset state transition probability matrix, and the first preset observation probability matrix, and obtains the ith group of M corresponding to the ith observation sequence value. state sequence. The first preset state transition probability matrix and the first preset observation probability matrix corresponding to two adjacent observation sequence values are different.

The preset dynamic programming algorithm is not limited in the embodiment of the present invention, and the object motion state detection device can also obtain the state sequence by processing other preset dynamic programming algorithms. The first preset state transition probability matrix represents a set of transition probabilities between any two state sequence values; the first preset observation probability matrix represents a set of transition probabilities between any state sequence value and observation sequence value.

The first preset state transition probability matrix is an empirical probability matrix obtained manually. When there are M state sequence values, the transition probability between any two state sequence values is manually confirmed, and then M times M probabilities can be obtained to obtain a preset state transition probability matrix. The transition probability in any row of the first preset state transition probability matrix represents the transition probability between the corresponding two state sequence values.

The first preset observation probability matrix is a probability matrix obtained by the preset detection model during training. When there are M state sequence values and M observation sequence values, the preset detection model obtains the transition probability between any state sequence value and observation sequence value during training, and then M times M probabilities can be obtained to obtain the preset observation probability matrix. The transition probability in any row of the first preset observation probability matrix represents the transition probability between the corresponding state sequence value and the observation sequence value.

S106. Detect and obtain the subscript interval T values between the i+1th observation sequence value and the ith observation sequence value, and calculate the second preset state transition probability matrix corresponding to the i+1th observation sequence value based on the A third state transition probability matrix is obtained.

In the embodiment of the present invention, the object motion state detection device detects and obtains the subscript interval T values between the i+1th observation sequence value and the ith observation sequence value, based on the ith value corresponding to the i+1th observation sequence value Two preset state transition probability matrices are calculated to obtain a third state transition probability matrix. T is an integer greater than or equal to 1.

Wherein, the second preset state transition probability matrix represents a transition probability set between M state sequence values when corresponding to the i+1th observation sequence value.

Exemplarily, the set of observation sequence values may be o ^x (1)o ^x (2)...o ^x (N). Among them, o ^x (i) corresponds to the subscript Θ. If the two observation sequence values are observation sequence values corresponding to two consecutive image frames, the subscripts of the two observation sequence values are consecutive (that is, the latter subscript is greater than the previous subscript by 1). If the two observation sequence values are not the observation sequence values corresponding to two consecutive image frames, the subscript values of the two observation sequence values are discontinuous (that is, the latter subscript is at least 2 larger than the former subscript). If the latter subscript is larger than the previous subscript by 2, it indicates that there is one sequence value between the two observation sequence values; if the latter subscript is larger than the previous subscript by 3, it indicates that the two observation sequence values are between between 2 sequence values.

S107. Based on the i-th group of M state sequences, the i+1-th observation sequence value, the third state transition probability matrix and the third preset observation probability matrix are processed by a preset dynamic programming algorithm to obtain the i+1-th observation sequence value corresponding to the i+1th group of M state sequences, until the M final state sequences corresponding to the last observation sequence value in the observation sequence value set are obtained.

In the embodiment of the present invention, the object motion state detection device is processed by a preset dynamic programming algorithm based on the ith group of M state sequences, the i+1th observation sequence value, the third state transition probability matrix, and the third preset observation probability matrix. , obtain the i+1th group of M state sequences corresponding to the i+1th observation sequence value, until the M final state sequences corresponding to the last observation sequence value in the observation sequence value set are obtained.

S108: Determine the final state sequence value of the target state sequence with the largest transition probability among the M final state sequences.

In the embodiment of the present invention, the object motion state detection device determines the final state sequence value of the target state sequence with the largest transition probability among the M final state sequences.

In the embodiment of the present invention, the object motion state detection apparatus may calculate the probability corresponding to each final state sequence by using multiple state sequence values in each final state sequence and multiple observation sequence values in the observation sequence value set. Wherein, the object motion state detection device may multiply multiple state transition probabilities corresponding to multiple state sequence values to obtain a product. The object motion state detection device may multiply multiple observation probabilities between the multiple observation sequence values and the multiple state sequence values in the corresponding order to obtain another product. The object motion state detection device calculates the product of the two products, and obtains the probability of each final state sequence.

In the embodiment of the present invention, the object motion state detection device arranges the M final state sequences in the order of columns, thereby obtaining a preset state sequence matrix.

S109. Combine the final state sequence value and the preset state sequence matrix, and determine a plurality of state sequence values to obtain a state sequence value set.

In the embodiment of the present invention, the object motion state detection device combines the final state sequence value and the preset state sequence matrix to determine a plurality of state sequence values to obtain a state sequence value set.

In the embodiment of the present invention, the object motion state detection device determines a corresponding column matching the last state sequence value in the last row of the preset state sequence matrix, so as to obtain a plurality of state sequence values.

In this embodiment of the present invention, the set of state sequence values includes: a set of information about the area to which the calibration object actually belongs in the corresponding image frame in the N image frames, which is predicted and obtained from the set of observation sequence values.

In the embodiment of the present invention, the object motion state detection device calculates and obtains the third state transition probability matrix by using the second preset state transition probability matrix corresponding to the i+1th observation sequence value, that is, the dynamic hidden Markov model-based The idea is to determine the third state transition probability matrix for the growth of the state sequence. Furthermore, the interference factors corresponding to O ₀ are eliminated, and a more accurate set of state sequence values is obtained.

In some embodiments, referring to FIG. 4 , FIG. 4 is an optional schematic flowchart of the object motion state detection method provided by the embodiment of the present invention. S106 shown in FIG. 3 can also be implemented through S110 to S111, and each step will be combined Be explained.

S110. Detect and obtain the subscript interval T values between the i+1th observation sequence value and the ith observation sequence value, and calculate the first state sequence value and the M value based on the second preset state transition probability matrix. probability values corresponding to the state sequence values to form the first row of the third state transition probability matrix.

In the embodiment of the present invention, the object motion state detection device detects and obtains the subscript interval T values between the i+1th observation sequence value and the ith observation sequence value, and calculates the th value based on the second preset state transition probability matrix. 1 state sequence value and probability values corresponding to the M state sequence values respectively, to form the first row of the third state transition probability matrix.

In the embodiment of the present invention, the object motion state detection device detects and obtains the subscript interval T values between the i+1th observation sequence value and the ith observation sequence value, and extracts the value from the second preset state transition probability matrix. , the first state sequence value is multiplied by the transition probability between the M state sequence values, respectively, to obtain the first product. When T is 1, the object motion state detection device calculates the K th product corresponding to the M state sequence values and the K th state sequence value respectively, and adds the first product and the K th product to form the third state transition probability matrix. The Kth probability value of the first row of , until the Mth probability value of the first row of the third state transition probability matrix is obtained, so as to form the first row of the third state transition probability matrix. K is an integer greater than or equal to 1 and less than or equal to M.

In the embodiment of the present invention, the object motion state detection device detects and obtains the subscript interval T values between the i+1th observation sequence value and the ith observation sequence value, and extracts the value from the second preset state transition probability matrix. , the first state sequence value is multiplied by the transition probability between the M state sequence values, respectively, to obtain the first product. When T is greater than 1, the object motion state detection device extracts the transition probability between any two state sequence values among the M state sequence values, and multiplies them to obtain a second product. The object motion state detection device calculates the K th product corresponding to the M state sequence values and the K th state sequence value respectively, and adds the first product, T-1 second products and the K th product to form a third state transition probability The Kth probability value of the first row in the matrix is obtained until the Mth probability value of the first row of the third state transition probability matrix is obtained, so as to form the first row of the third state transition probability matrix.

Wherein, the object motion state detection device calculates the K th product of the M state sequence values and the K th state sequence value respectively, and the object motion state detection device extracts the M state sequence values corresponding to the K th state sequence value respectively. Transition probability, multiplied to get the Kth product.

S111, until combined with the second preset state transition probability matrix, calculate the probability values of the M th state sequence value corresponding to the M state sequence values respectively, so as to form the M th row of the third state transition probability matrix, and then obtain the third state Transition probability matrix.

In the embodiment of the present invention, the object motion state detection device calculates the probability values corresponding to the M th state sequence values and the M state sequence values respectively, in combination with the second preset state transition probability matrix, to form the third state transition probability matrix. In the Mth row, the third state transition probability matrix is obtained.

In the embodiment of the present invention, the object motion state detection device extracts from the second preset state transition probability matrix, and the M th state sequence value is multiplied by the transition probability between the M state sequence values to obtain the M 1 th product . When T is 1, the object motion state detection device calculates the K th product corresponding to the M state sequence values and the K th state sequence value respectively, and adds the M 1 th product and the K th product to form the third state transition probability matrix. The Kth probability value of the first row of , until the Mth probability value of the Mth row of the third state transition probability matrix is obtained, so as to form the Mth row of the third state transition probability matrix.

In the embodiment of the present invention, the object motion state detection device extracts from the second preset state transition probability matrix, and the M th state sequence value is multiplied by the transition probability between the M state sequence values to obtain the M 1 th product . When T is greater than 1, the object motion state detection device extracts the transition probability between any two state sequence values among the M state sequence values, and multiplies them to obtain the M2th product. The object motion state detection device calculates the K th product corresponding to the M state sequence values and the K th state sequence value respectively, and adds the M1 th product, the T-1 M2 th product and the K th product to form a third state transition probability The Kth probability value of the Mth row in the matrix is obtained until the Mth probability value of the Mth row of the third state transition probability matrix is obtained, so as to form the first row of the third state transition probability matrix.

In the embodiment of the present invention, the object motion state detection device calculates the third state transition probability matrix based on the dynamic hidden Markov model, filters out the interference caused by misrecognition and missed recognition in the scene of vehicle state recognition on the platform, and can improve It handles the difficulties faced by standard Hidden Markov Models. Thus, the detection accuracy of the motion state of the object is improved.

In some embodiments, referring to FIG. 4, FIG. 4 is an optional schematic flowchart of the object motion state detection method provided by the embodiment of the present invention. S109 shown in FIG. 3 can also be implemented through S112 to S113, and each step will be combined Be explained.

S112: Arrange the M final state sequences in the order of columns, thereby obtaining a preset state sequence matrix.

In the embodiment of the present invention, the object motion state detection device arranges the M final state sequences in the order of columns, thereby obtaining a preset state sequence matrix.

S113. Determine a target sequence value matching the final state sequence value in the last row of the preset state sequence matrix, and then determine that the final state sequence of the column corresponding to the target sequence value is a plurality of state sequence values, so as to obtain a state sequence value set .

In the embodiment of the present invention, the object motion state detection device determines a target sequence value matching the final state sequence value in the last row of the preset state sequence matrix, and further determines that the final state sequence of the column corresponding to the target sequence value is a plurality of states sequence values to get a set of state sequence values.

In the embodiment of the present invention, due to the influence of some factors on site, the object motion state detection device may misidentify other objects as license plates, thereby causing misidentification of the entire vehicle dynamics. Therefore, the object motion state detection device can judge some of the misidentifications according to the trend of the whole observation sequence. For example, due to the motion characteristics of the vehicle, the position of the license plate cannot be moved too much in a short period of time. This characteristic can be described by the Markov state transition. If the license plate appears in one position at the current moment, then at the next moment, there is a greater probability of appearing in the original position or a nearby position, and the probability of appearing in a farther position will vary with decreases as the distance increases. Therefore, this scheme uses the hidden Markov model to estimate the real license plate position sequence through the observed license plate position sequence.

In the embodiment of the present invention, the object motion state detection device can divide the monitored area into M areas in the ordinate direction to set M states, each state indicates that the license plate appears in the corresponding area, and use {s ₁ , s ₂ ,…,s _m }, where s _i ∈{s ₁ ,s ₂ ,…,s _m } indicates that the license plate appears in the ith area. Corresponding to each state, there is a corresponding observation, which is represented by {o ₁ ,o ₂ ,…,o _m }, where O _i ∈{o ₁ ,o ₂ ,…,o _m } is represented in the ith region License plate detected. In the hidden Markov model, given an observation sequence, the state transition probability and observation probability need to be used. Among them, the state transition probability P(s _j |s _i ) is defined as the probability that the state at the previous moment is s _i and the state at the next moment is s _j . The observation probability P(O _j |s _i ) is the probability that the state is s _i and the observation is O _j . In the standard hidden Markov model, given a sequence of observations in a time window o(1), o(2), ..., o(N), O(τ) ∈ {o(1), o(2 ),...,o(N)}, according to the state transition probability and observation probability, the actual state sequence S(1), S(2),...,S(N), S(τ can be estimated using the Viterbi algorithm )∈{S(1),S(2),…,S(N)}. Furthermore, if at a moment τ, S(τ) and O(τ) do not match each other, remove the license plate position coordinate information Y(τ) at this moment, and then fit the linear model y=at+b. However, in the case of the dynamic hidden Markov model, since there is no setting of S ₀ (no license plate in the monitored area) and O ₀ (no license plate detected), the time corresponding to the observation value O ₀ must be removed from the sequence o (1), o(2), ..., o(N) are deleted, and the related state transitions are not considered. To solve this problem, this scheme considers state transitions corresponding to variable length sampling times. For example, since the elapsed time is different, P(S(τ+η)=s _j |s(τ)=s _i )≠P(s _j |s _i ), and η>1. But formula (1) can be obtained from the Chapman-Kolmogorov equation:

P(S(τ+η)=s _j |s(τ)=s _i )=∑ _k P(S(τ+η-1)=s _k |s(τ)=s _i )P(s _j | s _k ) (1)

Among them, P(s _j |s _k ) represents the transition probability of the K-th and j-th state sequence values, and P(S(τ+η-1) represents the state transition probability of the sequence value at the τ+η-1 moment. .s(τ) represents the ith state sequence value, _si represents the ith state sequence value, and _sk represents the kth state sequence value.

Therefore, in the dynamic hidden Markov model, it is necessary to maximize the

∏ _τ P(O(θτ)|S(θτ))×∏ _τ θ-1P(S(θτ+1)|S(θτ)) (2)

Among them, P(O(θτ)|S(θτ)) represents the observed transition probability between the τth state sequence value and the τth observation sequence value. P(S(θτ+1)|S(θτ)) represents the state transition probability between the τth state sequence value and the τ+1th state sequence value.

The object motion state detection device is used to solve the corresponding state sequence, where Θ=[θ1, θ2, .....] is the subscript of the remaining observations after removing the observation period of O ₀ .

In order to solve the problem of the dynamic hidden Markov model, the Viterbi algorithm should also be modified accordingly to form the dynamic Viterbi algorithm. Given the observation sequence and state transition probability, after the state probability, the dynamic Viterbi algorithm flow is as follows:

1. Create matrices A[|Θ|,M], B[|Θ|,M] to record the probability of the subsequence and the previous state corresponding to each state at the current moment in the subsequence.

2. At the first moment A[θ1,M]=P(O(θτ)|S(θτ)), B[θ1,M]=<start>, <start> indicates the start of the system state.

3. From time θ2 to θ|Θ|, use formula (1) to obtain the current state transition probability, A[θτ,i]=max _j P(O(θτ)|S(θ _τ ))=S _i )P (S(θ _τ )=S _i |S(θ _τ-1 )=s _j )A[θ _τ-1 ,s _j ], B[θτ,i]=argmax _j P(S(θ _τ )=S _i |S(θ _τ-1 )=s _j )A[θ _τ-1 ,s _j ].

4. Find the i that maximizes the transition probability of A[θ|Θ|,i], will estimate the last sequence value in the resulting final state sequence, S(θ|Θ|)=S _i .

5. According to the orientation of the matrix B[|Θ|, M], the estimated state sequence is obtained from the back to the front.

In some embodiments, referring to FIG. 5, FIG. 5 is an optional schematic flowchart of the object motion state detection method provided by the embodiment of the present invention. S101 to S102 shown in FIG. 1 can also be implemented by S114 to S119, which will be combined with Each step is explained.

S114: Extract the current image frame corresponding to the current moment from the video stream.

In the embodiment of the present invention, the object motion state detection device extracts the current image frame corresponding to the current moment in the video stream.

Exemplarily, the camera collects the video stream of the current predetermined area in real time and sends it to the object motion state detection device. The object motion state detection device extracts the image frame corresponding to the current moment after receiving the real-time video stream.

S115. Taking the current image frame as a starting point in the video stream, extract one image frame along the time axis every predetermined time length or a predetermined number of image frames, until N-1 image frames are extracted.

In the embodiment of the present invention, the object motion state detection device takes the current image frame as a starting point in the video stream, and extracts an image frame every predetermined time or a predetermined number of image frames along the time axis until N-1 images are extracted. frame.

In this embodiment of the present invention, the video stream includes a plurality of image frames distributed along the time axis. The object motion state detection device may take the current image frame as a starting point, and extract an image frame every 1 second until N-1 image frames are extracted. In this embodiment of the present invention, the predetermined duration is not limited.

In this embodiment of the present invention, the video stream includes a plurality of image frames distributed along the time axis. The object motion state detection device may take the current image frame as a starting point, and extract an image frame every 5 image frames until N-1 image frames are extracted. In this embodiment of the present invention, the predetermined number is not limited.

S116, combine the current image frame and N-1 image frames along the time axis to obtain N image frames.

In the embodiment of the present invention, the object motion state detection apparatus combines the current image frame and N-1 image frames along the time axis to obtain N image frames.

In the embodiment of the present invention, the object motion state detection apparatus combines the current image frame and N-1 image frames in an order from front to back along the respective corresponding time points to obtain N image frames.

S117: Process N image frames by using a preset detection model to obtain N pieces of position coordinate information.

In the embodiment of the present invention, the object motion state detection device processes N image frames by using a preset detection model, and obtains N pieces of position coordinate information.

The preset detection model may be a WPOD-NET algorithm model, and the preset detection model is not limited in this embodiment of the present invention.

S118: Divide the N image frames into regions respectively, and determine the observation sequence value of the region in the corresponding image frame for each position coordinate information, so as to obtain N observation sequence values corresponding to the N position coordinate information.

In the embodiment of the present invention, the object motion state detection device divides the N image frames into regions, and determines the observation sequence value of the corresponding region in the corresponding image frame for each position coordinate information, so as to obtain the corresponding N position coordinates The N observed sequence values of the information.

In the embodiment of the present invention, the object motion state detection device divides the N image frames into regions according to certain rules, so as to obtain a plurality of regions corresponding to each image frame. The object motion state detection device determines the region to which each position coordinate information belongs in the corresponding image frame, and then combines the order of the region to obtain observation sequence values, and finally obtains N observation sequence values.

S119: Eliminate the predetermined observation sequence value from the N observation sequence values to obtain a set of observation sequence values.

In the embodiment of the present invention, the object motion state detection device removes a predetermined observation sequence value from the N observation sequence values, and obtains a set of observation sequence values.

In the embodiment of the present invention, the object motion state detection device extracts N image frames at the current moment based on the sliding window method, and can accurately determine whether a vehicle arrives or leaves the platform by using the dynamic characteristics of the N image frames.

In some embodiments, referring to FIG. 5 , FIG. 5 is an optional schematic flowchart of the object motion state detection method provided by the embodiment of the present invention. S104 shown in FIG. 1 can also be implemented through S120 to S121, and each step will be combined Be explained.

S120 , combining the state sequence value set and the observation sequence value set, remove the interfering position coordinate information in the N pieces of position coordinate information, and obtain a plurality of target position coordinate information.

In the embodiment of the present invention, the object motion state detection device combines the state sequence value set and the observation sequence value set to eliminate the interfering position coordinate information in the N position coordinate information to obtain a plurality of target position coordinate information.

In the embodiment of the present invention, the object motion state detection apparatus compares multiple state sequence values in the state sequence value set and multiple observation sequence values in the observation sequence value set one by one in order. If it is detected that at least one observation sequence value is different from the corresponding state sequence value, the object motion state detection device removes the position coordinate information corresponding to the observation sequence value from the N position coordinate information, thereby obtaining multiple target position coordinate information.

S121. Determine the motion state by using multiple target position coordinate information.

In the embodiment of the present invention, the object motion state detection device determines the motion state by using a plurality of target position coordinate information.

In the embodiment of the present invention, the object motion state detection device performs linear fitting on a plurality of target position coordinate information to obtain a slope value. And determine the motion state according to the magnitude of the slope value.

In the embodiment of the present invention, the object motion state detection device combines the state sequence value set and the observation sequence value set to eliminate the interfering position coordinate information in the N position coordinate information, and then can calculate a more accurate motion state.

In some embodiments, referring to FIG. 6 , FIG. 6 is an optional schematic flowchart of the object motion state detection method provided by the embodiment of the present invention. S118 shown in FIG. 5 can also be implemented through S122 to S124, and each step will be combined Be explained.

S122: Divide the N image frames into U regions along the ordinate, respectively, to obtain U regions corresponding to each image frame.

In the embodiment of the present invention, the object motion state detection device divides the N image frames into U regions along the ordinate, respectively, to obtain U regions corresponding to each image frame. U is an integer greater than 1.

In the embodiment of the present invention, the object motion state detection device divides each image frame into 10 regions along the ordinate. Then, 10 regions corresponding to each image frame can be obtained.

S123: Determine the belonging area in the corresponding U areas for each position coordinate information.

In the embodiment of the present invention, the object motion state detection apparatus determines the area to which each position coordinate information belongs from the corresponding multiple areas.

Among them, each area includes a certain amount of position coordinate information. The object motion state detection device may determine the region to which the corresponding position coordinate information belongs in the U regions of each image frame.

S124: Determine the vertical order of the region in the U regions, use the vertical order as the observation sequence value of each position coordinate information, and then obtain N observation sequence values.

In the embodiment of the present invention, the object motion state detection device determines the vertical order of the region in the U regions, and uses the vertical order as the observation sequence value of each position coordinate information, and then obtains N observation sequence values.

Exemplarily, with reference to FIG. 7 , the object motion state detection apparatus divides each image frame into 10 regions along the ordinate, and then 10 regions corresponding to each image frame can be obtained. The object motion state detection device determines the area to which it belongs (the area marked with a bold solid line in FIG. 5 ) from 10 areas according to the position coordinate information of the center point of the license plate 14 of the vehicle. The object motion state detection device determines the order 5 corresponding to the region in which it is divided into 10 regions along the ordinate, and then obtains the longitudinal observation sequence value 5 corresponding to the position coordinate information.

In the embodiment of the present invention, the object motion state detection device divides the N image frames into U regions along the ordinate to obtain the observation sequence value corresponding to each position coordinate information. The method can accurately determine the area described by the position coordinate information, and the obtained observation sequence value is also more accurate.

In some embodiments, referring to FIG. 6 , FIG. 6 is an optional schematic flowchart of the object motion state detection method provided by the embodiment of the present invention. S120 to S121 shown in FIG. 5 can also be implemented through S125 to S128, which will be combined Each step is explained.

S125: Compare the multiple state sequence values in the state sequence value set with the multiple observation sequence values in the observation sequence value set one by one in order, if at least one state sequence value and at least one observation sequence value in the corresponding order are determined If it is different, at least one second position coordinate information corresponding to at least one observation sequence value is eliminated from the N pieces of position coordinate information to obtain a plurality of intermediate position coordinate information.

In the embodiment of the present invention, the object motion state detection device compares multiple state sequence values in the state sequence value set with multiple observation sequence values in the observation sequence value set one by one in order, and if at least one state sequence value is determined Different from the at least one observation sequence value in the corresponding order, at least one second position coordinate information corresponding to the at least one observation sequence value is eliminated from the N position coordinate information to obtain a plurality of intermediate position coordinate information.

S126: Eliminate the predetermined position coordinate information from the plurality of intermediate position coordinate information to obtain a plurality of target position coordinate information.

In the embodiment of the present invention, the object motion state detection device removes predetermined position coordinate information from a plurality of intermediate position coordinate information to obtain a plurality of target position coordinate information.

S127 , performing linear fitting on the plurality of target ordinate information in the plurality of target position coordinate information along the time axis to obtain the slope values of the plurality of target ordinate information.

In the embodiment of the present invention, the object motion state detection device performs linear fitting on the plurality of target ordinate information in the plurality of target position coordinate information along the time axis to obtain the slope values of the plurality of target ordinate information.

S128. Determine the motion state according to the magnitude of the slope value.

In the embodiment of the present invention, the object motion state detection device determines the motion state according to the magnitude of the slope value

If the slope value is greater than a preset threshold, determining that the motion state is a close state;

If the slope value is smaller than the preset threshold, it is determined that the motion state is a distance state.

The threshold value may be any numerical value, and the size of the threshold value is not limited in this embodiment of the present invention.

In the actual use process, the process of this solution is as follows:

Configuration: The staff adjusts the position and angle of the camera, sets the monitoring area, and divides the grid area in the abscissa and ordinate directions.

The object motion state detection device uses the camera to continuously collect data, and extracts the frames obtained at each moment and retains the most recent N frames.

The object motion state detection device uses the WPOD-NET algorithm to detect the license plate of each picture and save the position (horizontal and vertical coordinates) of the license plate (in fact, only the currently obtained frame needs to be detected at each moment, because the previous detection results have been saved) .

The object motion state detection device corresponds the ordinate of the license plate of each frame to the observation {o ₁ , o ₂ ,..., o _m }.

The object motion state detection device removes O ₀ in the observation sequence o(1), o(2), ..., o(N), and obtains o(θ1), o(θ2), ..., o(θ|Θ|).

The object motion state detection device uses a dynamic hidden Markov model to map the observation sequence o(θ1), o(θ2), ..., o(θ|Θ|) to the state sequence.

The object motion state detection device removes the elements at the position where the state value is inconsistent with the observed value in the license plate ordinate sequence Y, and updates Y.

The object motion state detection device performs least squares fitting on Y and t to obtain a slope value.

The object motion state detection device judges whether a vehicle arrives or leaves the platform according to the comparison between the magnitude of the slope and the preset threshold.

In the embodiment of the present invention, the object motion state detection device extracts N image frames by using the sliding window method, and uses the dynamic characteristics of the vehicle to determine the motion state, so as to improve the accuracy of the vehicle arrival/departure determination. At the same time, based on the idea of dynamic hidden Markov model, the interference factor is eliminated, thereby improving the detection efficiency.

Referring to FIG. 8 , FIG. 8 is a schematic structural diagram of an apparatus for detecting a motion state of an object according to an embodiment of the present invention.

The embodiment of the present invention further provides an object motion state detection device 800 , including: a collection and extraction unit 803 , a processing unit 804 and a determination unit 805 .

The acquisition and extraction unit 803 is used for real-time acquisition of a video stream in a predetermined area, and extracts N image frames corresponding to the current moment in the video stream based on a sliding window method; N is an integer greater than 1;

The processing unit 804 is configured to process and obtain N pieces of position coordinate information corresponding to the calibration objects in the N image frames, and combine the N image frames to determine an observation sequence value set after excluding the predetermined observation sequence value; Wherein, the predetermined observation sequence value is the observation sequence value corresponding to the predetermined position coordinate information; the predetermined position coordinate information is the position coordinate information obtained by processing corresponding to the image frame that does not include the calibration object;

The processing unit 804 is further configured to obtain a state sequence value set corresponding to the observation sequence value set based on the dynamic hidden Markov model;

A determination unit 805, configured to determine the motion state of the target object at the current moment in the N image frames through the observation sequence value set and the state sequence value set; the calibration object belongs to the target object .

In the embodiment of the present invention, the processing unit 804 in the object motion state detection device 800 is configured to detect and obtain the continuous subscript between the i-th observation sequence value in the observation sequence value set and the previous observation sequence value, based on the The i-th observation sequence value, the first preset state transition probability matrix and the first preset observation probability matrix are processed by a preset dynamic programming algorithm to obtain the i-th group of M state sequences corresponding to the i-th observation sequence value ; i is an integer greater than or equal to 1; M is an integer greater than 1; the subscript interval T values between the i+1th observation sequence value and the ith observation sequence value are detected, based on the The second preset state transition probability matrix corresponding to the i+1 observation sequence value is calculated to obtain the third state transition probability matrix; the second preset state transition probability matrix represents the time corresponding to the i+1th observation sequence value , the set of transition probabilities between M state sequence values; T is an integer greater than or equal to 1; based on the ith group of M state sequences, the i+1th observation sequence value, the third state transition probability matrix and The third preset observation probability matrix is processed by the preset dynamic programming algorithm to obtain the i+1 th group of M state sequences corresponding to the i+1 th observation sequence value, until the last observation sequence value set is obtained. M final state sequences corresponding to one observation sequence value; among the M final state sequences, determine the final state sequence value of the target state sequence with the largest transition probability; combining the final state sequence value and the preset state sequence matrix, A plurality of state sequence values are determined to obtain the set of state sequence values.

In the embodiment of the present invention, the processing unit 804 in the object motion state detection device 800 is configured to combine the second preset state transition probability matrix to calculate the probability that the first state sequence value corresponds to the M state sequence values respectively value to form the first row of the third state transition probability matrix; until combined with the second preset state transition probability matrix, the M th state sequence value is calculated respectively corresponding to the M state sequence value probability values , to form the Mth row of the third state transition probability matrix, and then obtain the third state transition probability matrix.

In this embodiment of the present invention, the processing unit 804 in the object motion state detection device 800 is configured to extract, from the second preset state transition probability matrix, the difference between the first state sequence value and the M state sequence values, respectively. Multiply the transition probabilities to obtain the first product; when T is 1, calculate the Kth product corresponding to the M state sequence values and the Kth state sequence value respectively, and multiply the first product with the Kth product adding up to form the Kth probability value of the first row in the third state transition probability matrix until the Mth probability value of the first row of the third state transition probability matrix is obtained to form the third The first row of the state transition probability matrix; K is an integer greater than or equal to 1 and less than or equal to M.

In this embodiment of the present invention, the processing unit 804 in the object motion state detection device 800 is configured to extract the transition probability between any two state sequence values in the M state sequence values and multiply them when T is greater than 1, Obtain the second product; calculate the K th product corresponding to the M state sequence values and the K th state sequence value respectively, and add the first product, T-1 second products and the K th product to form the K th product. The Kth probability value of the first row in the third state transition probability matrix until the Mth probability value of the first row of the third state transition probability matrix is obtained to form the third state transition probability matrix the first line of .

In this embodiment of the present invention, the processing unit 804 in the object motion state detection device 800 is configured to arrange the M final state sequences in the order of columns, thereby obtaining the preset state sequence matrix.

In this embodiment of the present invention, the processing unit 804 in the object motion state detection device 800 is configured to determine, in the last row of the preset state sequence matrix, a target sequence value that matches the last state sequence value, and then determine the target sequence value that matches the last state sequence value. The final state sequence of the column corresponding to the target sequence value is the plurality of state sequence values.

In this embodiment of the present invention, the acquisition and extraction unit 803 in the object motion state detection device 800 is configured to extract the current image frame corresponding to the current moment in the video stream; As the starting point, an image frame is extracted along the time axis every predetermined duration or a predetermined number of image frames until N-1 image frames are extracted; the current image frame and the N-1 image frames are extracted along the time axis Axes are combined to obtain the N image frames.

In the embodiment of the present invention, the processing unit 804 in the object motion state detection device 800 is configured to process the N image frames by using a preset detection model to obtain the N position coordinate information; Regions are divided respectively, and for each position coordinate information, the observation sequence value of the region to which it belongs is determined in the corresponding image frame, so as to obtain N observation sequence values corresponding to the N pieces of position coordinate information; The predetermined observation sequence value is eliminated from the sequence value to obtain the observation sequence value set.

In this embodiment of the present invention, the processing unit 804 in the object motion state detection device 800 is configured to divide the N image frames into U regions along the ordinate to obtain U regions corresponding to each image frame; U is an integer greater than 1; for each position coordinate information, determine the belonging area in the corresponding U areas; determine the vertical order of the belonging area in the U areas, and use the vertical order as The observation sequence value of each position coordinate information, and then the N observation sequence values are obtained.

In the embodiment of the present invention, the determination unit 805 in the object motion state detection device 800 is configured to combine the state sequence value set and the observation sequence value set to eliminate the interference position coordinate information in the N pieces of position coordinate information, and obtain Multiple target location coordinate information;

Using the plurality of target position coordinate information, the motion state is determined.

In this embodiment of the present invention, the determination unit 805 in the object motion state detection device 800 is configured to perform a sequence of multiple state sequence values in the state sequence value set and multiple observation sequence values in the observation sequence value set in order. One-to-one comparison, if it is determined that at least one state sequence value is different from at least one observation sequence value in the corresponding order, then at least one second position coordinate information corresponding to the at least one observation sequence value is selected from the N position coordinate information. Elimination to obtain a plurality of intermediate position coordinate information; to eliminate predetermined position coordinate information from the plurality of intermediate position coordinate information to obtain the plurality of target position coordinate information.

In this embodiment of the present invention, the determining unit 805 in the object motion state detection device 800 is configured to perform linear fitting on the multiple target ordinate information in the multiple target position coordinate information along the time axis to obtain the multiple targets The slope value of the ordinate information; the motion state is determined according to the magnitude of the slope value.

In the embodiment of the present invention, the determination unit 805 in the object motion state detection device 800 is configured to determine that the motion state is a close state if the slope value is greater than a preset threshold; if the slope value is less than the preset threshold , then it is determined that the motion state is the away state.

In the embodiment of the present invention, the video stream of the predetermined area is collected in real time by the acquisition and extraction unit 803, and N image frames corresponding to the current moment are extracted from the video stream based on the sliding window method; N is an integer greater than 1; 804 Process to obtain N position coordinate information corresponding to the calibration object in the N image frames, and combine the N image frames to determine the observation sequence value set after excluding the predetermined observation sequence value; wherein, the predetermined observation sequence value is the predetermined position The observation sequence value corresponding to the coordinate information; the predetermined position coordinate information is the position coordinate information obtained by processing the image frame corresponding to the image frame not including the calibration object; the state sequence value set is obtained by the processing unit 804 based on the corresponding observation sequence value set of the dynamic hidden Markov model ; The determination unit 805 determines the motion state of the target object at the current moment in the N image frames by observing the sequence value set and the state sequence value set; the calibration object belongs to the target object. Because the scheme is based on the dynamic hidden Markov model to filter out the interference factors caused by misrecognition and missed identification, and can better deal with the difficulties faced by the standard hidden Markov model, and thus can improve the recognition accuracy of the vehicle motion state.

It should be noted that, in the embodiment of the present invention, if the above-mentioned object motion state detection method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present invention may be embodied in the form of software products in essence or the parts that make contributions to related technologies. The computer software products are stored in a storage medium and include several instructions for making An object motion state detection device (which may be a personal computer, etc.) executes all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: a U disk, a removable hard disk, a read only memory (Read Only Memory, ROM), a magnetic disk or an optical disk and other mediums that can store program codes. As such, embodiments of the present invention are not limited to any particular combination of hardware and software.

Correspondingly, an embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, implements the steps in the above method.

Correspondingly, an embodiment of the present invention provides an object motion state detection device, including a memory 802 and a processor 801, where the memory 802 stores a computer program that can be executed on the processor 801, and the processor 801 executes the program When implementing the steps in the above method.

It should be pointed out here that the descriptions of the above storage medium and device embodiments are similar to the descriptions of the above method embodiments, and have similar beneficial effects to the method embodiments. For technical details not disclosed in the embodiments of the storage medium and device of the present invention, please refer to the description of the method embodiments of the present invention to understand.

It should be noted that FIG. 9 is a schematic diagram of a hardware entity of an object motion state detection device provided by an embodiment of the present invention. As shown in FIG. 9 , the hardware entity of the object motion state detection device 800 includes: a processor 801 and a memory 802 ,in;

The processor 801 generally controls the overall operation of the object motion state detection device 800 .

The memory 802 is configured to store instructions and applications executable by the processor 801, and can also cache data to be processed or processed by the processor 801 and each module in the object motion state detection device 800 (for example, image data, audio data, voice data, etc.). communication data and video communication data), which can be implemented by flash memory (FLASH) or random access memory (Random Access Memory, RAM).

It is to be understood that reference throughout the specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic associated with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily necessarily referring to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present invention, the size of the sequence numbers of the above-mentioned processes does not mean the sequence of execution, and the execution sequence of each process should be determined by its functions and internal logic, rather than the embodiments of the present invention. implementation constitutes any limitation. The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages or disadvantages of the embodiments.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus and method may be implemented in other manners. The apparatus embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined, or Can be integrated into another system, or some features can be ignored, or not implemented. In addition, the coupling, or direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical or other forms. of.

The unit described above as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit; it may be located in one place or distributed to multiple network units; Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present invention may all be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integration The unit can be implemented either in the form of hardware or in the form of hardware plus software functional units.

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by program instructions related to hardware, the aforementioned program can be stored in a computer-readable storage medium, and when the program is executed, the execution includes: The steps of the above method embodiments; and the aforementioned storage medium includes: a removable storage device, a read only memory (Read Only Memory, ROM), a magnetic disk or an optical disk and other media that can store program codes.

Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present invention may be embodied in the form of software products in essence or the parts that make contributions to related technologies. The computer software products are stored in a storage medium and include several instructions for making A computer device (which may be a personal computer, a server, or a network device, etc.) executes all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a removable storage device, a ROM, a magnetic disk, or an optical disk.

The above are only the embodiments of the present invention, but the protection scope of the present invention is not limited to this. Any person skilled in the art who is familiar with the technical scope disclosed by the present invention can easily think of changes or substitutions. Included within the scope of protection of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims (17)

1. an object motion state detection method, is characterized in that, comprises: Collecting the video stream of the predetermined area in real time, and extracting N image frames corresponding to the current moment in the video stream based on the sliding window method; N is an integer greater than 1; processing to obtain N pieces of positional coordinate information corresponding to the calibration objects in the N image frames, and combining the N image frames to determine a set of observation sequence values after excluding the predetermined observation sequence values; wherein, the predetermined observation sequence values are The sequence value is the observation sequence value corresponding to the predetermined position coordinate information; the predetermined position coordinate information is the position coordinate information obtained by processing corresponding to the image frame that does not include the calibration object; Obtaining a state sequence value set corresponding to the observation sequence value set based on the dynamic hidden Markov model; Through the observation sequence value set and the state sequence value set, the motion state of the target object at the current moment in the N image frames is determined; the calibration object belongs to the target object. 2. The object motion state detection method according to claim 1, wherein the obtaining a state sequence value set corresponding to the observation sequence value set based on a dynamic hidden Markov model, comprising: It is detected that the subscript between the i-th observation sequence value in the observation sequence value set and the previous observation sequence value is continuous, based on the i-th observation sequence value, the first preset state transition probability matrix and the first The preset observation probability matrix is processed by a preset dynamic programming algorithm to obtain the ith group of M state sequences corresponding to the ith observation sequence value; i is an integer greater than or equal to 1; M is an integer greater than 1; Detecting the subscript interval T values between the i+1th observation sequence value and the ith observation sequence value, based on the second preset state transition probability matrix corresponding to the i+1th observation sequence value , the third state transition probability matrix is obtained by calculation; the second preset state transition probability matrix represents the transition probability set between M state sequence values when corresponding to the i+1th observation sequence value; T is greater than or equal to an integer of 1; Based on the ith group of M state sequences, the ith+1th observation sequence value, the third state transition probability matrix and the third preset observation probability matrix are processed by the preset dynamic programming algorithm to obtain the ith The i+1th group of M state sequences corresponding to the i+1 observation sequence values, until the M final state sequences corresponding to the last observation sequence value in the observation sequence value set are obtained; determining the final state sequence value of the target state sequence with the largest transition probability among the M final state sequences; Combining the final state sequence value and the preset state sequence matrix, a plurality of state sequence values are determined to obtain the state sequence value set. 3 . The object motion state detection method according to claim 2 , wherein the third state transition probability is calculated based on the second preset state transition probability matrix corresponding to the i+1th observation sequence value. 4 . matrix, including: In combination with the second preset state transition probability matrix, the probability values corresponding to the first state sequence value and the M state sequence values are calculated to form the first row of the third state transition probability matrix; Until combined with the second preset state transition probability matrix, the probability values corresponding to the M th state sequence values and the M state sequence values respectively are calculated to form the M th row of the third state transition probability matrix, and then The third state transition probability matrix is obtained. 4 . The object motion state detection method according to claim 3 , wherein the calculation of the first state sequence value in combination with the second preset state transition probability matrix corresponds to the M state sequence values respectively. 5 . The probability values to form the first row of the third state transition probability matrix, including: Extracted from the second preset state transition probability matrix, the first state sequence value is respectively multiplied by the transition probability between the M state sequence values to obtain the first product; When T is 1, calculate the K th product corresponding to the M state sequence values and the K th state sequence value respectively, and add the first product and the K th product to form the third state transition probability matrix The K th probability value of the first row in , until the M th probability value of the first row of the third state transition probability matrix is obtained to form the first row of the third state transition probability matrix; K is An integer greater than or equal to 1 and less than or equal to M. 5 . The object motion state detection method according to claim 4 , wherein, extracted from the second preset state transition probability matrix, the first state sequence value is the difference between the M state sequence values. 6 . After multiplying the transition probabilities between the two to obtain the first product, the method further includes: When T is greater than 1, extract the transition probability between any two state sequence values in the M state sequence values, and multiply them to obtain a second product; Calculate the K th product corresponding to the M state sequence values and the K th state sequence value respectively, and add the first product, T-1 second products and the K th product to form the third state transition The Kth probability value of the first row in the probability matrix is obtained until the Mth probability value of the first row of the third state transition probability matrix is obtained, so as to form the first row of the third state transition probability matrix. 6 . The object motion state detection method according to claim 2 , wherein, based on the i-th group of M state sequences, the i+1-th observation sequence value, the third state transition probability matrix and the The third preset observation probability matrix is processed by the preset dynamic programming algorithm to obtain the i+1 th group of M state sequences corresponding to the i+1 th observation sequence value, until the last observation sequence value set is obtained. After M final state sequences corresponding to one observation sequence value, the preset state sequence matrix corresponding to the final state sequence value and the target state sequence is combined to determine a plurality of state sequence values to obtain the state sequence Before the value collection, the method further includes: The M final state sequences are arranged in the order of columns, so as to obtain the preset state sequence matrix. 7 . The object motion state detection method according to claim 6 , wherein, by combining the final state sequence value and a preset state sequence matrix, a plurality of state sequence values are determined to obtain the state sequence value. 8 . Collection, including: A target sequence value matching the final state sequence value is determined in the last row of the preset state sequence matrix, and then the final state sequence in the column corresponding to the target sequence value is determined as the plurality of state sequence values. 8. The object motion state detection method according to claim 1, wherein the method based on the sliding window extracts N image frames corresponding to the current moment in the video stream, comprising: extracting the current image frame corresponding to the current moment in the video stream; Taking the current image frame as a starting point in the video stream, extracting an image frame along the time axis every predetermined duration or a predetermined number of image frames, until N-1 image frames are extracted; The N image frames are obtained by combining the current image frame and the N-1 image frames along the time axis. 9 . The object motion state detection method according to claim 1 , wherein the processing obtains N pieces of positional coordinate information corresponding to the calibration object in the N image frames, and in combination with the N image frames, 10 . Determine the set of observation sequence values after excluding the predetermined observation sequence values, including: The N image frames are processed by a preset detection model to obtain the N position coordinate information; The N image frames are divided into regions respectively, and the observation sequence value of the region to which it belongs is determined in the corresponding image frame for each position coordinate information, so as to obtain N observation sequence values corresponding to the N position coordinate information ; The predetermined observation sequence value is eliminated from the N observation sequence values to obtain the observation sequence value set. 10 . The object motion state detection method according to claim 9 , wherein the N image frames are divided into regions respectively, and the region to which they belong is determined in the corresponding image frame for each position coordinate information. 11 . to obtain N observation sequence values corresponding to the N positional coordinate information, including: The N image frames are respectively divided into U regions along the ordinate to obtain U regions corresponding to each image frame; U is an integer greater than 1; Determine the belonging area in the corresponding U areas for each position coordinate information; The vertical order of the belonging region is determined in the U regions, and the vertical order is used as the observation sequence value of each position coordinate information, and then the N observation sequence values are obtained. 11 . The object motion state detection method according to claim 1 , wherein the said observation sequence value set and the state sequence value set are used to determine the said target object in the N image frames. 12 . The current state of motion, including: Combining the state sequence value set and the observation sequence value set, eliminating the interference position coordinate information in the N position coordinate information to obtain a plurality of target position coordinate information; Using the plurality of target position coordinate information, the motion state is determined. 12 . The object motion state detection method according to claim 11 , wherein, by combining the state sequence value set and the observation sequence value set, the interference position coordinate information in the N pieces of position coordinate information is eliminated. 12 . , get multiple target position coordinate information, including: Compare multiple state sequence values in the state sequence value set with multiple observation sequence values in the observation sequence value set one by one in order, if at least one state sequence value and at least one observation in the corresponding order are determined If the sequence values are different, the at least one second position coordinate information corresponding to the at least one observation sequence value is eliminated from the N position coordinate information to obtain a plurality of intermediate position coordinate information; The predetermined position coordinate information in the plurality of intermediate position coordinate information is eliminated to obtain the plurality of target position coordinate information. 13. The method for detecting a motion state of an object according to claim 11, wherein the determining the motion state by using the plurality of target position coordinate information comprises: performing linear fitting on the plurality of target ordinate information in the plurality of target position coordinate information along the time axis to obtain slope values of the plurality of target ordinate information; The motion state is determined according to the magnitude of the slope value. 14. The method for detecting the motion state of an object according to claim 13, wherein the determining the motion state according to the magnitude of the slope value comprises one of the following: If the slope value is greater than a preset threshold, determining that the motion state is a close state; If the slope value is smaller than the preset threshold, it is determined that the motion state is a distance state. 15. A device for detecting a motion state of an object, comprising: The acquisition and extraction unit is used for real-time acquisition of the video stream of the predetermined area, and extracts N image frames corresponding to the current moment in the video stream based on the sliding window method; N is an integer greater than 1; a processing unit, configured to process and obtain N pieces of position coordinate information corresponding to the calibration objects in the N image frames, and combine the N image frames to determine an observation sequence value set after excluding the predetermined observation sequence value; wherein , the predetermined observation sequence value is the observation sequence value corresponding to the predetermined position coordinate information; the predetermined position coordinate information is the position coordinate information obtained by processing corresponding to the image frame that does not include the calibration object; The processing unit is further configured to obtain a state sequence value set corresponding to the observation sequence value set based on the dynamic hidden Markov model; A determination unit, configured to determine the motion state of the target object at the current moment in the N image frames through the observation sequence value set and the state sequence value set; the calibration object belongs to the target object. 16. An object motion state detection device, characterized by comprising a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and the processor implements claims 1 to 14 when executing the program The steps of any one of the methods. 17. A computer-readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the steps in the method of any one of claims 1 to 14 are implemented.

Images