CN115661191A - Method, system, equipment and medium for judging zero displacement in photoelectric navigation - Google Patents
Method, system, equipment and medium for judging zero displacement in photoelectric navigation Download PDFInfo
- Publication number
- CN115661191A CN115661191A CN202211322524.3A CN202211322524A CN115661191A CN 115661191 A CN115661191 A CN 115661191A CN 202211322524 A CN202211322524 A CN 202211322524A CN 115661191 A CN115661191 A CN 115661191A
- Authority
- CN
- China
- Prior art keywords
- target
- current
- correlation value
- determining
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000006073 displacement reaction Methods 0.000 title claims abstract description 49
- 239000013598 vector Substances 0.000 claims abstract description 102
- 238000004422 calculation algorithm Methods 0.000 claims description 11
- 238000004590 computer program Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000005693 optoelectronics Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a method, a system, equipment and a medium for judging zero displacement in photoelectric navigation, and relates to the technical field of photoelectric navigation. The judging method comprises the following steps: determining a current target frame and a prediction vector; determining a search area according to the prediction vector; according to the search area, determining a reference block in a current reference frame, and determining a reference target block and a target block to be matched in the current target frame; respectively carrying out correlation operation on each target block to be matched and a reference block, and determining an optimal correlation value and an optimal matching block; calculating a current motion vector of the optimal matching block relative to the reference block; performing correlation operation on the reference target block and the reference block to obtain a reference correlation value; determining whether the target photoelectric navigation equipment generates relative motion at the current moment according to the optimal correlation value and the reference correlation value; if the relative motion occurs, outputting the current motion vector, otherwise, outputting zero displacement. The invention can correctly output zero displacement when the photoelectric navigation equipment does not generate relative motion.
Description
Technical Field
The invention relates to the technical field of photoelectric navigation, in particular to a method, a system, equipment and a medium for judging zero displacement in photoelectric navigation.
Background
The common motion estimation method of the photoelectric navigation technology is as follows: a matching interval is preset, then correlation calculation is carried out on a reference frame and a target frame, an optimal matching block is found, and a motion vector is output according to the relative position of the optimal matching block. In which "presetting a matching interval" is to determine an interval that should make the reference frame and the target frame have the strongest correlation (theoretically having overlapping image portions) according to a certain prediction vector.
If errors occur in the correlation calculation process, subsequent prediction errors can be caused, the calculation enters a vicious error loop, and even if the device and the target do not move relatively, the errors do not stop. For example, existing correlation calculation methods: the height of Anhua is judged by a bowl-shaped algorithm. Namely, the matching results S1 to S9, wherein the optimal value (marked as Sopt) is placed in the middle, other results are placed around, if Sopt is far smaller than (or larger than, depending on the algorithm) other values, the bowl is a deep bowl, and the matching is a good matching. In this method, since there is no standard threshold, images with high contrast and images with low contrast are compared, it can be found that images with high contrast, even if matching is wrong, have bowl depths greater than the results obtained in a sequence of images with low contrast.
In the photoconductive navigation image estimation method, a prediction vector is provided, a matching process is provided, and the possibility of errors in prediction and matching is high. For single or small amount of errors, the multipoint matching algorithm has correction effect; but for a plurality of continuous errors, the matching can not be positioned to the correct interval any more, and the optimal correlation module is found. The resulting consequences are: the motion track is completely abnormal. For example, in a mouse application it appears that: even though the mouse is already stationary, the cursor flies everywhere.
Therefore, there is a need for a method for determining zero displacement in electro-optical navigation, so that zero displacement can be correctly output when there is no relative motion of the electro-optical navigation device.
Disclosure of Invention
The invention aims to provide a method, a system, equipment and a medium for judging zero displacement in photoelectric navigation, so as to realize correct output of zero displacement when no relative motion occurs in photoelectric navigation equipment.
In order to achieve the purpose, the invention provides the following scheme:
a method for judging zero displacement in photoelectric navigation, comprising the following steps:
taking an electronic image acquired by target photoelectric navigation equipment at the current moment as a current target frame;
determining a prediction vector according to an output vector at a historical moment; the prediction vector is the linear relation of the output vector of the previous moment or the output vectors of a plurality of continuous historical moments before the current moment;
determining a search area between a current reference frame and the current target frame according to the prediction vector; the current reference frame is a previous target frame or an original reference frame; the last target frame is an electronic image acquired by the target photoelectric navigation equipment at the last moment; the original reference frame is determined according to an electronic image acquired by target photoelectric navigation equipment at a historical moment; the search area is an overlapped area between the current reference frame and the current target frame;
determining a reference block in a search area corresponding to the current reference frame;
determining a reference target block and a plurality of target blocks to be matched in a search area corresponding to the current target frame; the plurality of target blocks to be matched include: a central target block and a plurality of neighborhood target blocks; the central target block is an area on the current target frame which is overlapped with the reference block; the neighborhood target block is a region obtained by moving to surrounding neighborhoods by taking the central target block as a center; the reference target block is an area on the current target frame, the position and the size of the area are the same as those of the reference block;
respectively carrying out correlation operation on each target block to be matched and the reference block, determining an optimal correlation value, and taking the target block to be matched corresponding to the optimal correlation value as an optimal matching block; the optimal correlation value is the minimum sum of absolute errors of the pixels or the maximum value of the similar quantity;
calculating a current motion vector of the best matching block relative to the reference block;
performing correlation operation on the reference target block and the reference block to obtain a reference correlation value; when the optimal correlation value is the minimum value of the sum of absolute errors, the reference correlation value is the sum of absolute errors of pixels of the reference target block and the reference block; when the optimal correlation value is the maximum value of the similar quantity, the reference correlation value is the similar quantity of the pixel points of the reference target block and the reference block;
determining whether the target photoelectric navigation equipment generates relative motion at the current moment according to the optimal correlation value and the reference correlation value;
if the relative motion occurs, outputting the current motion vector; if no relative motion occurs, zero displacement is output.
Optionally, the determining, according to the optimal correlation value and the reference correlation value, whether the target photoelectric navigation device generates relative motion at the current time specifically includes:
comparing the optimal correlation value with the reference correlation value;
when the optimal correlation value is the minimum sum of absolute errors, if the optimal correlation value is larger than or equal to the reference correlation value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment;
when the optimal correlation value is the maximum value of the similarity quantity, if the optimal correlation value is smaller than or equal to the reference correlation value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment.
Optionally, the determining, according to the optimal correlation value and the reference correlation value, whether the target photoelectric navigation device generates relative motion at the current time specifically includes:
comparing the optimal correlation value with the reference correlation value;
when the optimal correlation value is the sum of the absolute errors, if the optimal correlation values at all the moments in a set time interval are greater than or equal to the reference correlation value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment;
when the optimal correlation value is the maximum value of the similarity quantity, if the optimal correlation values at all the moments in a set time interval are less than or equal to the reference correlation value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment; the starting time of the set time interval is the next time of the time when the target photoelectric navigation device generates relative motion before the current time.
Optionally, the determining, according to the optimal correlation value and the reference correlation value, whether the target photoelectric navigation device generates relative motion at the current time specifically includes:
comparing the optimal correlation value to the reference correlation value;
when the optimal correlation value is the sum of the absolute errors, if the times that the optimal correlation value is greater than or equal to the reference correlation value in a set time interval are greater than a set value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment;
when the optimal correlation value is the maximum value of the similarity quantity, if the times that the optimal correlation value is smaller than or equal to the reference correlation value in a set time interval are larger than a set value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment; the starting time of the set time interval is the next time of the time when the target photoelectric navigation device generates relative motion before the current time.
Optionally, the determining a search region between the current reference frame and the current target frame according to the prediction vector specifically includes:
determining an actual motion vector of the last target frame relative to the current reference frame;
determining a predicted motion vector of the current target frame relative to the current reference frame according to the actual motion vector and the predicted vector;
determining a search area between the current reference frame and the current target frame according to the predicted motion vector.
Optionally, before the determining a search region between a current reference frame and the current target frame according to the prediction vector, the method further includes:
judging whether the size of a search area between the last reference frame and the current target frame is in a set size range or not;
if so, taking the previous target frame as the current reference frame;
and if not, taking the original reference frame as the current reference frame.
Optionally, a correlation operation is performed using a SAD algorithm.
The invention also provides a system for judging zero displacement in photoelectric navigation, which is applied to the method for judging zero displacement in photoelectric navigation, and comprises the following steps:
the current target frame determining module is used for taking an electronic image acquired by the target photoelectric navigation equipment at the current moment as a current target frame;
the prediction vector determining module is used for determining a prediction vector according to the output vector of the historical moment; the prediction vector is the linear relation of the output vector of the previous moment or the output vectors of a plurality of continuous historical moments before the current moment;
a search area determining module, configured to determine a search area between the current reference frame and the current target frame according to the prediction vector; the current reference frame is a previous target frame or an original reference frame; the last target frame is an electronic image acquired by the target photoelectric navigation equipment at the last moment; the original reference frame is determined according to an electronic image acquired by target photoelectric navigation equipment at a historical moment; the search area is an overlapped area between the current reference frame and the current target frame;
a reference block determining module, configured to determine a reference block in a search area corresponding to the current reference frame;
the target block determining module is used for determining a reference target block and a plurality of target blocks to be matched in a search area corresponding to the current target frame; the plurality of target blocks to be matched include: a central target block and a plurality of neighborhood target blocks; the central target block is an area on the current target frame which is overlapped with the reference block; the neighborhood target block is a region obtained by moving to surrounding neighborhoods by taking the central target block as a center; the reference target block is an area on the current target frame, the position and the size of the area are the same as those of the reference block;
the first correlation operation module is used for respectively performing correlation operation on each target block to be matched and the reference block, determining an optimal correlation value, and taking the target block to be matched corresponding to the optimal correlation value as an optimal matching block; the optimal correlation value is the minimum sum of absolute errors of the pixels or the maximum value of the similar quantity;
a current motion vector calculation module for calculating a current motion vector of the best matching block relative to the reference block;
the second correlation operation module is used for performing correlation operation on the reference target block and the reference block to obtain a reference correlation value; when the optimal correlation value is the minimum value of the sum of absolute errors, the reference correlation value is the sum of absolute errors of pixels of the reference target block and the reference block; when the optimal correlation value is the maximum value of the similar quantity, the reference correlation value is the similar quantity of the pixel points of the reference target block and the reference block;
the relative motion determining module is used for determining whether the target photoelectric navigation equipment generates relative motion at the current moment according to the optimal correlation value and the reference correlation value;
a displacement output module to: if the relative motion occurs, outputting the current motion vector; if no relative motion occurs, zero displacement is output.
The invention also provides an electronic device, which comprises a memory and a processor, wherein the memory is used for storing the computer program, and the processor runs the computer program to enable the electronic device to execute the method for judging the zero displacement in the photoelectric navigation.
The invention further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the method for judging zero displacement in photoelectric navigation.
According to the specific embodiment provided by the invention, the invention discloses the following technical effects:
the zero displacement judging method provided by the invention is additionally provided with a judging process for judging whether the target photoelectric navigation equipment generates relative motion on the basis of a conventional motion estimation algorithm, namely, an area with the same position and size as a reference block on a current target frame is determined as a reference target block, the reference target block and each target block to be matched are respectively subjected to correlation operation with the reference block, a reference correlation value of the reference target block and the reference block and an optimal correlation value of each target block to be matched and the reference block are determined, and the reference correlation value is compared with the optimal correlation value, so that whether the target photoelectric navigation equipment generates relative motion relative to a background is determined, and further, the zero displacement can be correctly output when the photoelectric navigation equipment does not generate relative motion.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without inventive exercise.
FIG. 1 is a flow chart of a method for determining zero displacement in electro-optical navigation according to the present invention;
fig. 2 is a flowchart illustrating a method for determining zero shift in electro-optical navigation according to a first embodiment of the present invention;
fig. 3 is a flowchart illustrating a method for determining zero shift in a second electro-optical navigation system according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a method for determining a search area according to the present invention;
FIG. 5 is a diagram illustrating a method for determining a reference block and a target block according to the present invention;
FIG. 6 is a block diagram of a zero shift determination system for electro-optical navigation according to the present invention.
Description of the symbols: the device comprises a current target frame determining module-1, a prediction vector determining module-2, a search area determining module-3, a reference block determining module-4, a target block determining module-5, a first correlation operation module-6, a current motion vector calculating module-7, a second correlation operation module-8, a relative motion determining module-9 and a displacement output module-10.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention aims to provide a method, a system, equipment and a medium for judging zero displacement in photoelectric navigation, so as to realize correct output of zero displacement when no relative motion occurs in photoelectric navigation equipment.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in further detail below.
Example one
Fig. 1 is a flowchart of a method for determining zero displacement in photoelectric navigation according to the present invention. As shown in fig. 1, the determining method includes:
step S1: and taking an electronic image acquired by the target photoelectric navigation equipment at the current moment as a current target frame.
Step S2: determining a prediction vector according to an output vector at a historical moment; the prediction vector is the linear relation of the output vector of the previous moment or the output vectors of a plurality of continuous historical moments before the current moment.
And step S3: determining a search area between a current reference frame and the current target frame according to the prediction vector; the current reference frame is a previous target frame or an original reference frame; the last target frame is an electronic image acquired by the target photoelectric navigation equipment at the last moment; the original reference frame is determined according to an electronic image acquired by target photoelectric navigation equipment at a historical moment; the search region is a region of overlap between the current reference frame and the current target frame.
In this embodiment, step S3 specifically includes:
step S31: determining an actual motion vector of the previous target frame relative to the current reference frame.
Step S32: and determining a predicted motion vector of the current target frame relative to the current reference frame according to the actual motion vector and the predicted vector.
Step S33: determining a search area between the current reference frame and the current target frame according to the predicted motion vector.
And step S4: and determining a reference block in the search area corresponding to the current reference frame.
Step S5: determining a reference target block and a plurality of target blocks to be matched in a search area corresponding to the current target frame; the plurality of target blocks to be matched include: a central target block and a plurality of neighborhood target blocks; the central target block is an area on the current target frame that overlaps the reference block; the neighborhood target block is a region obtained by moving to each neighborhood (such as 8 neighborhoods or 24 neighborhoods) around the center target block; the reference target block is an area on the current target frame having the same position and size as the reference block.
Step S6: respectively carrying out correlation operation on each target block to be matched and the reference block, determining an optimal correlation value, and taking the target block to be matched corresponding to the optimal correlation value as an optimal matching block; the optimal correlation value is the minimum sum of absolute errors of pixel points (of the target block to be matched and the reference block) or the maximum value of the similarity quantity. Preferably, the correlation operation is performed using Sum of Absolute Differences (SAD).
Step S7: calculating a current motion vector of the best matching block relative to the reference block.
Step S8: performing correlation operation on the reference target block and the reference block to obtain a reference correlation value; when the optimal correlation value is the minimum value of the absolute errors, the reference correlation value is the sum of the absolute errors of the pixels of the reference target block and the reference block; and when the optimal correlation value is the maximum value of the similar quantity, the reference correlation value is the similar quantity of the pixel points of the reference target block and the reference block.
Step S9: and determining whether the target photoelectric navigation equipment generates relative motion at the current moment according to the optimal correlation value and the reference correlation value.
Step S10: if relative motion occurs (at the current time), outputting the current motion vector, namely outputting a vector (dx, dy); if no relative motion occurs (at the current time), a zero shift is output, i.e., vector (0,0) is output.
Further, before step S3, the method further includes:
step S11: and judging whether the size of the search area between the last reference frame and the current target frame is in a set size range.
Step S12: if so, taking the previous target frame as the current reference frame; if not, the original reference frame (i.e. the last reference frame) is taken as the current reference frame.
Fig. 2 is a flowchart illustrating a method for determining zero displacement in a first electro-optical navigation system according to an embodiment of the present invention, and fig. 3 is a flowchart illustrating a method for determining zero displacement in a second electro-optical navigation system according to an embodiment of the present invention. As shown in fig. 2 and fig. 3, in practical application, the specific process of the present invention is as follows:
(1) Acquiring a reference frame (namely a current reference frame) and a next target frame (namely a current target frame), and performing image matching to obtain the moving direction and distance of a target photoelectric navigation device (such as a mouse cursor). The target frame needs to be acquired and updated each time, and the reference frame does not need to be updated each time. The image matching is to prepare a frame of reference image and a frame of target image, and compare the reference image with the target image to determine the moving direction and distance (i.e., motion vector) of the target image relative to the reference image. The photoelectric navigation device, namely an LED illumination system irradiates a target plane, reflected light of the target plane enters a device sensor to form an electronic image, the device processes the electronic image (such as image matching) to obtain a motion vector, and the device is called a photoelectric navigation device.
(2) A prediction vector (preDx, preDy) is determined based on the output vector at the historical time, and this prediction vector can be regarded as the prediction motion vector of the current target frame relative to the previous target frame, so that the position of the current target frame can be predicted based on the position of the previous target frame and the prediction vector, that is, the prediction motion vectors of the current target frame and the current reference frame can be obtained based on the actual motion vectors of the previous target frame and the current reference frame and the prediction vector. Referring to fig. 4, an overlapping area of the current reference frame and the current target frame is determined as a search area (i.e., a gray area in fig. 4). This region makes the current reference frame have the strongest correlation with the current target frame. And determining a reference block and a target block in the search area, and performing correlation operation to correct the prediction vector to obtain a correct motion vector (dx, dy) of the current target frame relative to the previous target frame. The accumulation of the correct motion vectors (dx, dy) for each match is the actual motion vectors for the target frame and the reference frame. The predicted vector may be the correct motion vector (dx, dy) output last time, or may be a linear relationship of the first motion vectors (dx, dy).
(3) The method comprises the steps of determining a reference block and a target block (here, a target block to be matched), and performing correlation operation on the reference block and a plurality of designated target blocks (namely target blocks to be matched) in a neighborhood (such as a 3x3 neighborhood or a 5x5 neighborhood), wherein a correlation result can be recorded as S1-S9 by taking the 3x3 neighborhood as an example. And comparing S1 to S9 to find an optimal value Sopt (namely an optimal correlation value), wherein the relative movement position of a target block (namely an optimal matching block) corresponding to the optimal value and a reference block is the motion vector value (dx, dy) of the current displacement. The reference block in the search area is fixed, the target block is a block overlapped with the reference block in the search area on the target frame and a plurality of blocks of the neighborhood thereof, if eight neighborhoods are selected, the eight blocks are S1-S9 respectively. The reference block is selected in the search area as large as possible, and nine target blocks to be matched, which are to be referred to by the reference block, cannot exceed the target frame, and the determination method of the reference block and the target blocks is shown in fig. 5, wherein a dark gray area in fig. 5 represents a central target block, a light gray area represents a reference target block, and a square area with asterisks represents a vertex at the upper left corner of each neighborhood target block.
Specifically, the optimal value is determined by a correlation operation algorithm, such as an SAD algorithm, in which two matching blocks (a reference block and a target block) are subtracted from each other point to point, an absolute value is then taken, and finally all absolute values are summed. The minimum value among S1 to S9 is set as an optimum value, and if the target block is identical to the reference block, the correlation is optimum with a SAD value of 0. In addition, other algorithms can be adopted, for example, the number of point-to-point similarities of two matching blocks is counted, the number value is the largest, and the correlation is optimal. The term "dot" as used herein refers to a pixel point of an electronic image.
(4) And (3) the reference block is not changed, according to the position and the size of the reference block, a target block (namely a reference target block) with the same size is found at the same position of the target frame, correlation operation is carried out, and the result is marked as S0 (namely a reference correlation value). Since the reference block is unchanged and the target block corresponding to S0 is the same size as the target block corresponding to Sopt, sopt is comparable to S0.
(5) Comparing S0 with Sopt, if S0 is better than Sopt (i.e. the first implementation method), the electro-optical navigation device should not have relative displacement, and then outputs zero displacement (0,0). If Sopt is more optimal, then (dx, dy) is output. Taking the correlation operation performed by the SAD algorithm as an example, the smaller value is better.
In addition, different implementation methods can be adopted, and the method is not limited to the flow and does not influence the idea of the invention. For example, after obtaining the result that Sopt is not better than S0, the lag outputs zero shift, and changes in subsequent situations are observed, such as consecutive frames (i.e. the second implementation) or a plurality of frames within a certain interval (i.e. the third implementation) that Sopt is not better than S0, and then zero shift is output.
The three implementation methods are described as follows:
a first implementation is shown in fig. 2. Comparing the optimal correlation value with the reference correlation value; when the optimal correlation value is the minimum sum of absolute errors, if the optimal correlation value is larger than or equal to the reference correlation value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment; when the optimal correlation value is the maximum value of the similarity quantity, if the optimal correlation value is smaller than or equal to the reference correlation value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment.
A second implementation is shown in fig. 3. Comparing the optimal correlation value with the reference correlation value; when the optimal correlation value is the sum of the absolute errors, if the optimal correlation values at all the moments in a set time interval are greater than or equal to the reference correlation value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment; when the optimal correlation value is the maximum value of the similarity quantity, if the optimal correlation values at all the moments in a set time interval are less than or equal to the reference correlation value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment; the starting moment of the set time interval is the next moment of the moment when the target photoelectric navigation equipment generates relative motion before the current moment; the current time is a certain time in the set time interval. Specifically, the set time interval includes consecutive Th0 time instants, and a counter Cnt is used to count the number of times that Sopt is not better than S0.
The third implementation method specifically comprises the following steps: comparing the optimal correlation value with the reference correlation value; when the optimal correlation value is the sum of the absolute errors, if the times that the optimal correlation value is greater than or equal to the reference correlation value in a set time interval are greater than a set value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment; when the optimal correlation value is the maximum value of the similarity quantity, if the times that the optimal correlation value is smaller than or equal to the reference correlation value in a set time interval are larger than a set value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment; the starting moment of the set time interval is the next moment of the moment when the target photoelectric navigation equipment generates relative motion before the current moment; the current time is a certain time in the set time interval.
(6) When the next matching between the reference frame and the target frame image is performed, whether the reference frame needs to be replaced needs to be considered. When the search area is too small, the correlation between the reference frame and the target frame is not strong, which may cause errors in matching and prediction, and the current target frame needs to be used as the reference frame for the next matching.
Example two
In order to implement the method corresponding to the above embodiment to achieve the corresponding functions and technical effects, a system for determining zero displacement in photoelectric navigation is provided below, and fig. 6 is a block diagram of the system for determining zero displacement in photoelectric navigation provided by the present invention. As shown in fig. 6, the judgment system includes:
and the current target frame determining module 1 is used for taking an electronic image acquired by the target photoelectric navigation equipment at the current moment as a current target frame.
A prediction vector determining module 2, configured to determine a prediction vector according to an output vector at a historical time; the prediction vector is the linear relation of the output vector of the previous moment or the output vectors of a plurality of continuous historical moments before the current moment.
A search area determining module 3, configured to determine a search area between the current reference frame and the current target frame according to the prediction vector; the current reference frame is a previous target frame or an original reference frame; the last target frame is an electronic image acquired by the target photoelectric navigation equipment at the last moment; the original reference frame is determined according to an electronic image acquired by target photoelectric navigation equipment at a historical moment; the search area is an overlapping area between the current reference frame and the current target frame.
And a reference block determining module 4, configured to determine a reference block in the search area corresponding to the current reference frame.
A target block determining module 5, configured to determine a reference target block and multiple target blocks to be matched in a search area corresponding to the current target frame; the plurality of target blocks to be matched include: a central target block and a plurality of neighborhood target blocks; the central target block is an area on the current target frame which is overlapped with the reference block; the neighborhood target block is a region obtained by moving to surrounding neighborhoods by taking the central target block as a center; the reference target block is an area on the current target frame having the same position and size as the reference block.
The first correlation operation module 6 is configured to perform correlation operation on each target block to be matched and the reference block, determine an optimal correlation value, and use the target block to be matched corresponding to the optimal correlation value as an optimal matching block; the optimal correlation value is the minimum sum of absolute errors of the pixels or the maximum value of the similar quantity.
A current motion vector calculation module 7, configured to calculate a current motion vector of the best matching block relative to the reference block.
A second correlation operation module 8, configured to perform correlation operation on the reference target block and the reference block to obtain a reference correlation value; when the optimal correlation value is the minimum value of the sum of absolute errors, the reference correlation value is the sum of absolute errors of pixels of the reference target block and the reference block; and when the optimal correlation value is the maximum value of the similar quantity, the reference correlation value is the similar quantity of the pixel points of the reference target block and the reference block.
And a relative motion determining module 9, configured to determine whether the target photoelectric navigation device generates relative motion at the current time according to the optimal correlation value and the reference correlation value.
A displacement output module 10 for: if the relative motion occurs, outputting the current motion vector; if no relative motion occurs, zero displacement is output.
EXAMPLE III
The embodiment of the invention provides electronic equipment, which comprises a memory and a processor, wherein the memory is used for storing a computer program, and the processor runs the computer program to enable the electronic equipment to execute the method for judging zero displacement in photoelectric navigation in the first embodiment.
Alternatively, the electronic device may be a server.
In addition, an embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method for determining zero displacement in the optoelectronic navigation system described in the first embodiment is implemented.
On the basis of a general flow (motion estimation algorithm), a zero displacement judgment process is added, namely a correlation (correlation) matching result S0 is calculated through a target block which has the same position and size as a reference block of the general flow, and the S0 and a Sopt value in the general flow are compared to determine whether the equipment has relative displacement relative to the background. Compared with the prior art, the invention has the following advantages:
(1) The method has no threshold setting, has consistency to the captured images under all backgrounds, and has no difference treatment.
(2) For equipment without relative motion, the S0 value is theoretically equal to 0 and is close to zero in practical application, which shows that the method provided by the invention is accurate and has obvious effect.
The embodiments in the present description are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. For the system disclosed by the embodiment, the description is relatively simple because the system corresponds to the method disclosed by the embodiment, and the relevant points can be referred to the method part for description.
The principles and embodiments of the present invention have been described herein using specific examples, which are provided only to assist in understanding the core concepts of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, the specific embodiments and the application range may be changed. In view of the foregoing, the description is not to be taken in a limiting sense.
Claims (10)
1. A method for judging zero displacement in photoelectric navigation is characterized by comprising the following steps:
taking an electronic image acquired by target photoelectric navigation equipment at the current moment as a current target frame;
determining a prediction vector according to an output vector at a historical moment; the prediction vector is a linear relation of an output vector at the previous moment or output vectors at a plurality of continuous historical moments before the current moment;
determining a search area between a current reference frame and the current target frame according to the prediction vector; the current reference frame is a previous target frame or an original reference frame; the last target frame is an electronic image acquired by the target photoelectric navigation equipment at the last moment; the original reference frame is determined according to an electronic image acquired by target photoelectric navigation equipment at a historical moment; the search area is an overlapped area between the current reference frame and the current target frame;
determining a reference block in a search area corresponding to the current reference frame;
determining a reference target block and a plurality of target blocks to be matched in a search area corresponding to the current target frame; the plurality of target blocks to be matched include: a central target block and a plurality of neighborhood target blocks; the central target block is an area on the current target frame that overlaps the reference block; the neighborhood target block is an area obtained by moving to each neighborhood around by taking the central target block as a center; the reference target block is an area on the current target frame, the position and the size of the area are the same as those of the reference block;
respectively carrying out correlation operation on each target block to be matched and the reference block, determining an optimal correlation value, and taking the target block to be matched corresponding to the optimal correlation value as an optimal matching block; the optimal correlation value is the minimum sum of absolute errors of the pixels or the maximum value of the similar quantity;
calculating a current motion vector of the best matching block relative to the reference block;
performing correlation operation on the reference target block and the reference block to obtain a reference correlation value; when the optimal correlation value is the minimum value of the sum of absolute errors, the reference correlation value is the sum of absolute errors of pixels of the reference target block and the reference block; when the optimal correlation value is the maximum value of the similar quantity, the reference correlation value is the similar quantity of the pixel points of the reference target block and the reference block;
determining whether the target photoelectric navigation equipment generates relative motion at the current moment according to the optimal correlation value and the reference correlation value;
if the relative motion occurs, outputting the current motion vector; if no relative motion occurs, zero displacement is output.
2. The method for determining zero displacement in photoelectric navigation according to claim 1, wherein the determining whether the target photoelectric navigation device has a relative motion at the current time according to the optimal correlation value and the reference correlation value specifically comprises:
comparing the optimal correlation value with the reference correlation value;
when the optimal correlation value is the minimum sum of absolute errors, if the optimal correlation value is larger than or equal to the reference correlation value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment;
when the optimal correlation value is the maximum value of the similarity quantity, if the optimal correlation value is smaller than or equal to the reference correlation value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment.
3. The method for determining zero displacement in photoelectric navigation according to claim 1, wherein the determining whether the target photoelectric navigation device has a relative motion at the current time according to the optimal correlation value and the reference correlation value specifically comprises:
comparing the optimal correlation value to the reference correlation value;
when the optimal correlation value is the sum minimum of the absolute errors, if the optimal correlation values at all the moments in a set time interval are greater than or equal to the reference correlation value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment;
when the optimal correlation value is the maximum value of the similarity quantity, if the optimal correlation values at all the moments in a set time interval are less than or equal to the reference correlation value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment; the starting time of the set time interval is the next time of the time when the target photoelectric navigation device generates relative motion before the current time.
4. The method for determining zero displacement in electro-optical navigation according to claim 1, wherein the determining whether the target electro-optical navigation device generates relative motion at the current time according to the optimal correlation value and the reference correlation value specifically includes:
comparing the optimal correlation value to the reference correlation value;
when the optimal correlation value is the sum of the absolute errors, if the times that the optimal correlation value is greater than or equal to the reference correlation value in a set time interval are greater than a set value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment;
when the optimal correlation value is the maximum value of the similarity quantity, if the times that the optimal correlation value is smaller than or equal to the reference correlation value in a set time interval are larger than a set value, determining that the target photoelectric navigation equipment does not generate relative motion at the current moment; otherwise, determining that the target photoelectric navigation equipment generates relative motion at the current moment; the starting time of the set time interval is the next time of the time when the target photoelectric navigation device generates relative motion before the current time.
5. The method for determining zero displacement in electro-optical navigation according to claim 1, wherein the determining a search region between the current reference frame and the current target frame according to the prediction vector specifically includes:
determining an actual motion vector of the last target frame relative to the current reference frame;
determining a predicted motion vector of the current target frame relative to the current reference frame according to the actual motion vector and the predicted vector;
determining a search area between the current reference frame and the current target frame according to the predicted motion vector.
6. The method for determining zero shift in electro-optical navigation according to claim 1, further comprising, before the determining the search region between the current reference frame and the current target frame according to the prediction vector,:
judging whether the size of a search area between the last reference frame and the current target frame is in a set size range or not;
if so, taking the previous target frame as the current reference frame;
and if not, taking the original reference frame as the current reference frame.
7. The method as claimed in claim 1, wherein the correlation operation is performed by using SAD algorithm.
8. A system for determining zero displacement in photoelectric navigation, the system comprising:
the current target frame determining module is used for taking an electronic image acquired by the target photoelectric navigation equipment at the current moment as a current target frame;
the prediction vector determining module is used for determining a prediction vector according to the output vector of the historical moment; the prediction vector is the linear relation of the output vector of the previous moment or the output vectors of a plurality of continuous historical moments before the current moment;
a search area determining module, configured to determine a search area between the current reference frame and the current target frame according to the prediction vector; the current reference frame is a previous target frame or an original reference frame; the last target frame is an electronic image acquired by the target photoelectric navigation equipment at the last moment; the original reference frame is determined according to an electronic image acquired by target photoelectric navigation equipment at a historical moment; the search area is an overlapped area between the current reference frame and the current target frame;
a reference block determining module, configured to determine a reference block in a search area corresponding to the current reference frame;
the target block determining module is used for determining a reference target block and a plurality of target blocks to be matched in a search area corresponding to the current target frame; the plurality of target blocks to be matched include: a central target block and a plurality of neighborhood target blocks; the central target block is an area on the current target frame that overlaps the reference block; the neighborhood target block is a region obtained by moving to surrounding neighborhoods by taking the central target block as a center; the reference target block is an area on the current target frame, the position and the size of the area are the same as those of the reference block;
the first correlation operation module is used for respectively performing correlation operation on each target block to be matched and the reference block, determining an optimal correlation value, and taking the target block to be matched corresponding to the optimal correlation value as an optimal matching block; the optimal correlation value is the minimum sum of absolute errors of the pixels or the maximum value of the similar quantity;
a current motion vector calculation module for calculating a current motion vector of the best matching block with respect to the reference block;
the second correlation operation module is used for performing correlation operation on the reference target block and the reference block to obtain a reference correlation value; when the optimal correlation value is the minimum value of the sum of absolute errors, the reference correlation value is the sum of absolute errors of pixels of the reference target block and the reference block; when the optimal correlation value is the maximum value of the similar quantity, the reference correlation value is the similar quantity of the pixel points of the reference target block and the reference block;
the relative motion determining module is used for determining whether the target photoelectric navigation equipment generates relative motion at the current moment according to the optimal correlation value and the reference correlation value;
a displacement output module to: if the relative motion occurs, outputting the current motion vector; if no relative motion occurs, zero displacement is output.
9. An electronic device, comprising a memory and a processor, wherein the memory is used for storing a computer program, and the processor runs the computer program to make the electronic device execute the method for determining zero displacement in electro-optical navigation according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that it stores a computer program, which when executed by a processor implements the method for determining zero displacement in electro-optical navigation according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211322524.3A CN115661191A (en) | 2022-10-27 | 2022-10-27 | Method, system, equipment and medium for judging zero displacement in photoelectric navigation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211322524.3A CN115661191A (en) | 2022-10-27 | 2022-10-27 | Method, system, equipment and medium for judging zero displacement in photoelectric navigation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115661191A true CN115661191A (en) | 2023-01-31 |
Family
ID=84990495
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211322524.3A Pending CN115661191A (en) | 2022-10-27 | 2022-10-27 | Method, system, equipment and medium for judging zero displacement in photoelectric navigation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115661191A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116543186A (en) * | 2023-05-04 | 2023-08-04 | 无锡英斯特微电子有限公司 | Image matching method, system, device and medium in photoelectric navigation |
| CN116758308A (en) * | 2023-05-22 | 2023-09-15 | 无锡英斯特微电子有限公司 | Image matching methods, systems, equipment and media |
-
2022
- 2022-10-27 CN CN202211322524.3A patent/CN115661191A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116543186A (en) * | 2023-05-04 | 2023-08-04 | 无锡英斯特微电子有限公司 | Image matching method, system, device and medium in photoelectric navigation |
| CN116758308A (en) * | 2023-05-22 | 2023-09-15 | 无锡英斯特微电子有限公司 | Image matching methods, systems, equipment and media |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7016411B2 (en) | Image signal coding method, image signal coding apparatus and storage medium | |
| CN106296578B (en) | Image processing method and device | |
| CN109871760B (en) | Face positioning method and device, terminal equipment and storage medium | |
| EP2858008A2 (en) | Target detecting method and system | |
| CN115661191A (en) | Method, system, equipment and medium for judging zero displacement in photoelectric navigation | |
| EP2180695B1 (en) | Apparatus and method for improving frame rate using motion trajectory | |
| US20130004079A1 (en) | Image processing apparatus, image processing method, and program thereof | |
| US10410358B2 (en) | Image processing with occlusion and error handling in motion fields | |
| CN109598744A (en) | A kind of method, apparatus of video tracking, equipment and storage medium | |
| KR20150053438A (en) | Stereo matching system and method for generating disparity map using the same | |
| CN111754429B (en) | Motion vector post-processing method and device, electronic device and storage medium | |
| US11669978B2 (en) | Method and device for estimating background motion of infrared image sequences and storage medium | |
| JP2015226326A (en) | Video analysis method and video analysis device | |
| CN113674319B (en) | Target tracking method, system, equipment and computer storage medium | |
| CN113284167B (en) | Face tracking detection method, device, equipment and medium | |
| KR100778116B1 (en) | Motion vector correction device and correction method | |
| CN111462178A (en) | Method, device and equipment for detecting living body flow direction based on thermal imaging and storage medium | |
| EP3941062A1 (en) | Method,video processing apparatus, device, and medium for estimating a motion vector of a pixel block | |
| WO2015198592A1 (en) | Information processing device, information processing method, and information processing program | |
| US9106926B1 (en) | Using double confirmation of motion vectors to determine occluded regions in images | |
| Hermann et al. | Inclusion of a second-order prior into semi-global matching | |
| US9357161B1 (en) | Motion vector interpolation for motion compensation | |
| EP3933767B1 (en) | Scene detection method, chip, electronic device, and storage medium | |
| US20150350669A1 (en) | Method and apparatus for improving estimation of disparity in a stereo image pair using a hybrid recursive matching processing | |
| CN113128290A (en) | Moving object tracking method, system, device and computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |