CN112083716B - Navigation method, device and system based on machine vision - Google Patents
Navigation method, device and system based on machine vision Download PDFInfo
- Publication number
- CN112083716B CN112083716B CN201910508334.2A CN201910508334A CN112083716B CN 112083716 B CN112083716 B CN 112083716B CN 201910508334 A CN201910508334 A CN 201910508334A CN 112083716 B CN112083716 B CN 112083716B
- Authority
- CN
- China
- Prior art keywords
- image
- value
- dark channel
- detected
- channel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/20—Image enhancement or restoration using local operators
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/77—Retouching; Inpainting; Scratch removal
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30248—Vehicle exterior or interior
- G06T2207/30252—Vehicle exterior; Vicinity of vehicle
- G06T2207/30261—Obstacle
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Biology (AREA)
- Electromagnetism (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Automation & Control Theory (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
- Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)
- Manipulator (AREA)
Abstract
The disclosure provides a navigation method, device and system based on machine vision, and relates to the field of autonomous navigation. The navigation method comprises the following steps: judging whether three primary color components of an image detected by an image sensor are all larger than a first color threshold value of a corresponding channel; if the three primary color components of the detected image are all larger than the first color threshold value of the corresponding channel, carrying out dark channel filtering processing on the detected image; comparing the similarity of the image subjected to dark channel filtering processing with the closed-loop candidate image in the database; and if the comparison result is greater than or equal to the similarity threshold, determining that the motion trail of the robot where the image sensor is positioned forms a closed loop so as to optimize the pose of the robot. The positioning accuracy in the autonomous navigation process of the robot is improved.
Description
Technical Field
The present disclosure relates to the field of autonomous navigation, and in particular, to a machine vision-based navigation method, apparatus, and system.
Background
The robot autonomous navigation algorithm is that the mobile robot analyzes data acquired by a sensor under an unknown environment and creates an environment map, and the robot autonomous positioning and navigation are realized by using the map. Common sensors include ultrasonic sensors, visual sensors, infrared sensors, lidar sensors, and multi-sensor hybrid approaches. The autonomous navigation algorithm realized by the vision sensor is also called a vision SLAM (Simultaneous Localization AND MAPPING, instant positioning and map building) technology, and is used as one of the current mainstream robot positioning and navigation technologies, and the image data transmitted by the vision sensor is subjected to interframe matching to realize the map building of the surrounding environment and the positioning of the position of the robot.
Common visual navigation algorithms include the following modules: the camera tracking and positioning, scene map construction, closed loop detection, and path planning and obstacle avoidance modules. The camera tracking and positioning means that the position of the camera in the environment is determined by utilizing the estimated value of the pose of the camera and analyzing the observed value obtained by the sensor; constructing a scene map refers to constructing a three-dimensional map of a scene where a camera is located through technologies such as three-dimensional reconstruction and the like; closed loop detection belongs to back-end optimization, and the pose of a camera is optimized by detecting whether a current scene and a previous scene form a closed loop or not; and path planning and obstacle avoidance refer to a method for searching an optimal road and avoiding obstacles in real time in the running process of a robot.
In the related closed loop detection process, because the method for matching the image characteristic points is greatly influenced by light rays and noise points, the situation that false matching forms an incorrect closed loop or the closed loop cannot be detected often occurs, and the positioning accuracy of the navigation robot is influenced.
Disclosure of Invention
The technical problem to be solved by the present disclosure is to provide a navigation method, device and system based on machine vision, which can improve the positioning accuracy in the autonomous navigation process of a robot.
According to an aspect of the present disclosure, a machine vision-based navigation method is provided, including: judging whether three primary color components of an image detected by an image sensor are all larger than a first color threshold value of a corresponding channel; if the three primary color components of the detected image are all larger than the first color threshold value of the corresponding channel, carrying out dark channel filtering processing on the detected image; comparing the similarity of the image subjected to dark channel filtering processing with the closed-loop candidate image in the database; and if the comparison result is greater than or equal to the similarity threshold, determining that the motion trail of the robot where the image sensor is positioned forms a closed loop so as to optimize the pose of the robot.
In one embodiment, performing dark channel filtering processing on the detected image includes: determining a dark channel value of the detected image based on the three primary color channel values of the detected image; carrying out mean value filtering processing on the dark channel value of the detected image; determining transmissivity according to the detected dark channel value of the image and the dark channel value after mean value filtering processing; determining a global atmospheric light value according to the detected three primary color channel values of the image and the dark channel value after mean value filtering processing; and determining the image after dark channel filtering processing according to the detected image, the transmissivity and the global atmospheric light value.
In one embodiment, the similarity comparison of the image subjected to the dark channel filtering process and the closed-loop candidate image includes: determining a first bag-of-word vector of the image subjected to dark channel filtering; and determining a similarity value of the first bag-of-word vector and a second bag-of-word vector of the closed-loop candidate image based on the chi-square distance.
In one embodiment, if the detected three primary color components of the image are not all greater than the first color threshold of the corresponding channel, determining if the detected three primary color components of the image are all less than the second color threshold of the corresponding channel; if the three primary color components of the detected image are smaller than the second color threshold of the corresponding channel, the exposure value of the image sensor is adjusted so that the image sensor can re-acquire the image.
In one embodiment, if the comparison result is smaller than the similarity threshold, the first bag-of-word vector of the image subjected to the dark channel filtering process is added to the database.
According to another aspect of the present disclosure, there is also provided a machine vision-based navigation device, including: a color threshold value judging unit configured to judge whether or not three primary color components of an image detected by the image sensor are all larger than a first color threshold value of a corresponding channel; a dark channel filter processing unit configured to perform dark channel filter processing on the detected image if both of the three primary color components of the detected image are greater than the first color threshold of the corresponding channel; the similarity comparison unit is configured to compare the similarity of the image subjected to dark channel filtering processing with the closed-loop candidate image in the database; and the closed loop determining unit is configured to determine that the motion trail of the robot where the image sensor is positioned forms a closed loop to optimize the pose of the robot if the comparison result is greater than or equal to the similarity threshold.
In one embodiment, the dark channel filter processing unit is configured to determine the dark channel value of the detected image based on the three primary color channel values of the detected image; carrying out mean value filtering processing on the dark channel value of the detected image; determining transmissivity according to the detected dark channel value of the image and the dark channel value after mean value filtering processing; determining a global atmospheric light value according to the detected three primary color channel values of the image and the dark channel value after mean value filtering processing; and determining the image after dark channel filtering processing according to the detected image, the transmissivity and the global atmospheric light value.
In one embodiment, the similarity comparison unit is configured to determine a first bag-of-word vector of the image after dark channel filtering; and determining a similarity value of the first bag-of-word vector and a second bag-of-word vector of the closed-loop candidate image based on the chi-square distance.
In one embodiment, the color threshold judging unit is further configured to judge whether the three primary color components of the detected image are all smaller than the second color threshold of the corresponding channel if the three primary color components of the detected image are not all larger than the first color threshold of the corresponding channel; the exposure value adjustment unit is configured to adjust the exposure value of the image sensor so that the image sensor re-captures the image if the three primary color components of the detected image are all smaller than the second color threshold of the corresponding channel.
According to another aspect of the present disclosure, there is also provided a machine vision-based navigation device, including: a memory; and a processor coupled to the memory, the processor configured to perform a navigation method as described above based on instructions stored in the memory.
According to another aspect of the present disclosure, there is also provided a machine vision-based navigation system, including: an image sensor located on the robot configured to acquire an image; and the machine vision based navigation device.
According to another aspect of the disclosure, a computer-readable storage medium is also presented, on which computer program instructions are stored, which instructions, when executed by a processor, implement the navigation method described above.
Compared with the related art, the method and the device have the advantages that the detected image is subjected to dark channel filtering, the image with high RGB three channel values can be repaired, the closed loop is generated by identifying similar scenes, so that the pose of the robot is optimized, and the positioning accuracy in the autonomous navigation process of the robot is improved.
Other features of the present disclosure and its advantages will become apparent from the following detailed description of exemplary embodiments of the disclosure, which proceeds with reference to the accompanying drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description, serve to explain the principles of the disclosure.
The disclosure may be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings in which:
fig. 1 is a flow diagram of one embodiment of a machine vision based navigation method of the present disclosure.
Fig. 2 is a flow chart illustrating another embodiment of a machine vision based navigation method of the present disclosure.
Fig. 3 is a schematic structural diagram of one embodiment of a machine vision based navigation device of the present disclosure.
Fig. 4 is a schematic structural diagram of another embodiment of a machine vision based navigation device of the present disclosure.
Fig. 5 is a schematic structural diagram of another embodiment of a machine vision based navigation device of the present disclosure.
Fig. 6 is a schematic structural diagram of another embodiment of a machine vision based navigation device of the present disclosure.
Detailed Description
Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present disclosure unless it is specifically stated otherwise.
Meanwhile, it should be understood that the sizes of the respective parts shown in the drawings are not drawn in actual scale for convenience of description.
The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the disclosure, its application, or uses.
Techniques, methods, and apparatus known to one of ordinary skill in the relevant art may not be discussed in detail, but should be considered part of the specification where appropriate.
In all examples shown and discussed herein, any specific values should be construed as merely illustrative, and not a limitation. Thus, other examples of the exemplary embodiments may have different values.
It should be noted that: like reference numerals and letters denote like items in the following figures, and thus once an item is defined in one figure, no further discussion thereof is necessary in subsequent figures.
For the purposes of promoting an understanding of the principles and advantages of the disclosure, reference will now be made to the embodiments illustrated in the drawings and specific language will be used to describe the same.
Fig. 1 is a flow diagram of one embodiment of a machine vision based navigation method of the present disclosure.
In step 110, it is determined whether the three primary color components of the image detected by the image sensor are all greater than a first color threshold for the corresponding channel. The image sensor is, for example, a camera, located on the robot. The three primary colors include R (red) G (green) B (blue), and in this step, it is determined whether the R component of the image is greater than a preset R color threshold, whether the G component is greater than a preset G color threshold, and whether the B component is greater than a preset B color threshold.
As can be seen from comparing clear images with images which are not clear due to weather, clear images are bright in color and uniform in color distribution, while unclear images are single in color and generally show higher or lower brightness, such as higher brightness when fog exists and lower brightness when light is dark. The above feature is manifested in terms of R, G, B channel formations of image colors in that the RGB components of a sharp image are typically different and one of the components will be relatively small, while the three components of an unclear image RGB will typically be co-larger or smaller.
In step 120, if the three primary color components of the detected image are all greater than the first color threshold of the corresponding channel, then dark channel filtering is performed on the detected image.
For example, a dark channel value of the detected image is determined based on the three primary color channel values of the detected image, an average value filter process is performed on the dark channel value of the detected image, a transmittance is determined based on the dark channel value of the detected image and the dark channel value after the average value filter process, a global atmospheric light value is determined based on the three primary color channel value of the detected image and the dark channel value after the average value filter process, and the image after the dark channel filter process is determined based on the detected image, the transmittance, and the global atmospheric light value.
In step 130, the image after the dark channel filtering process is compared with the closed loop candidate image in the database for similarity. And judging the similarity between the image key frame subjected to dark channel filtering processing and the key in the database.
In one embodiment, a first bag-of-word vector of the image after dark channel filtering is determined, a second bag-of-word vector of the closed-loop candidate image is determined, and a similarity value of the first bag-of-word vector and the second bag-of-word vector is determined based on the chi-square distance.
The chi-square distance formula isWherein v 1 is a first bag-of-word vector, v 2 is a second bag-of-word vector, χ 2(v1,v2) is a similarity value between the first bag-of-word vector and the second bag-of-word vector,The value of the ith variable of v 1, i.e. the frequency number on the ith class,Expected frequency number on class i for v 1; the value of the ith variable of v 2, i.e. the frequency number on the ith class, Is the expected frequency of v 2 on class i.
In step 140, if the comparison result is greater than or equal to the similarity threshold, determining that the motion track of the robot where the image sensor is located forms a closed loop, so as to optimize the pose of the robot. And identifying whether the image shot by the current camera position coincides with the image in the database, if so, determining that the closed loop detection is successful, and starting a closed loop optimization function to optimize the pose of the robot.
In the embodiment, the detected image is subjected to dark channel filtering processing, so that the image with higher RGB three-channel value can be repaired, and a closed loop is generated by identifying similar scenes, so that the pose of the robot is optimized, and the positioning accuracy in the autonomous navigation process of the robot is improved.
Fig. 2 is a flow chart illustrating another embodiment of a machine vision based navigation method of the present disclosure.
At step 210, an image acquired by an image sensor is acquired.
In step 220, it is determined whether the three primary color components of the image are all greater than the first color threshold of the corresponding channel, if so, step 230 is performed, otherwise, step 280 is performed.
In step 230, the image is subjected to dark channel filtering.
For example, the acquired image is H (x), and the dark channel value J (x) of the image is determined according to the formula J (x) =min c∈{r,g,b}(HC (x)), where H C (x) is the value of RGB three channels of the acquired image, and the dark channel refers to that in most of the local areas other than sky, some pixels always have at least one color channel with a very low value, and according to this assumption, it can be derived that J (x) tends to be 0, and thus the dark channel is the minimum value of RGB three channels. The dark channel values of the image are then subjected to an average filtering process to obtain a dark channel value J ave (x) after each channel average filtering process, for example, J ave (x) =averagej (x). Calculating the mean value m av of all channels, determining the transmittance t (x) from the detected dark channel value of the image and the mean-filtered dark channel value, wherein t (x) =min (min (ρm av,0.9)Jave (x), J (x)), ρ is a parameter for adjustment, determining the global atmospheric light value a from the detected trichromatic channel value of the image and the mean-filtered dark channel value, wherein a=1/2 (max (max c∈{r,g,b}(HC(x))+maxJave (x)) [ 11 1]. Then using the formulaAn image F (x) of the dark channel filter processing is obtained.
In step 240, a first bag-of-word vector of the image after dark channel filtering is determined.
In step 250, a similarity value for the first bag-of-words vector to a second bag-of-words vector of the closed loop candidate image in the database is determined based on the chi-square distance.
In step 260, it is determined whether the similarity value is greater than the similarity threshold, if so, step 270 is executed, otherwise, step 271 is executed.
In step 270, it is determined that the motion trajectory of the robot where the image sensor is located forms a closed loop to optimize the pose of the robot.
At step 271, the first bag of words vector is added to the database. I.e. the bag of words vector is stored as a new key frame in the database.
In step 280, it is determined whether the three primary color components of the image are all smaller than the second color threshold of the corresponding channel, if yes, step 281 is executed, otherwise, step 290 is executed.
In step 281, the exposure value of the image sensor is adjusted so that the image sensor re-captures an image. If the three primary color components of the image are all smaller than the second color threshold of the corresponding channel, the three primary color components are transmitted back to the tracking thread, and the camera adjusts the exposure value and resamples.
In step 290, a similarity value between the first bag-of-words vector corresponding to the image and a second bag-of-words vector of the closed-loop candidate image in the database is determined based on the chi-square distance. Step 260 is subsequently performed.
In this embodiment, the closed loop detection thread may be divided into two parts, wherein the first part is responsible for detecting whether the motion trajectory of the robot forms a closed loop or not, and the second part optimizes the pose of the robot according to the detected closed loop situation. By calculating the similarity degree of the current key frame and other key frames, whether the current scene is in a database or not is identified, when the camera is identified to come to the scene which is arrived before again, the detection of the closed loop is considered to be successful, and a closed loop optimization function is started to optimize the pose of the robot.
Fig. 3 is a schematic structural diagram of one embodiment of a machine vision based navigation device of the present disclosure. The navigation device includes a color threshold judging unit 310, a dark channel filter processing unit 320, a similarity comparing unit 330, and a closed loop determining unit 340.
The color threshold value judging unit 310 is configured to judge whether or not the three primary color components of the image detected by the image sensor are all greater than the first color threshold value of the corresponding channel. The image sensor is, for example, a camera, located on the robot.
The dark channel filter processing unit 320 is configured to perform dark channel filter processing on the detected image if the three primary color components of the detected image are both greater than the first color threshold of the corresponding channel.
For example, a dark channel value of the detected image is determined based on the three primary color channel values of the detected image, an average value filter process is performed on the dark channel value of the detected image, a transmittance is determined based on the dark channel value of the detected image and the dark channel value after the average value filter process, a global atmospheric light value is determined based on the three primary color channel value of the detected image and the dark channel value after the average value filter process, and the image after the dark channel filter process is determined based on the detected image, the transmittance, and the global atmospheric light value.
The similarity comparison unit 330 is configured to perform similarity comparison of the image subjected to the dark channel filtering process with the closed-loop candidate image in the database. For example, determining a first bag-of-word vector of an image subjected to dark channel filtering; and determining a similarity value of the first bag-of-word vector and a second bag-of-word vector of the closed-loop candidate image based on the chi-square distance.
The closed loop determining unit 340 is configured to determine that the motion trail of the robot where the image sensor is located forms a closed loop to optimize the pose of the robot if the comparison result is greater than or equal to the similarity threshold. And identifying whether the image shot by the current camera position coincides with the image in the database, if so, determining that the closed loop detection is successful, and starting a closed loop optimization function to optimize the pose of the robot. If the comparison result is smaller than the similarity threshold value, the first bag-of-word vector of the image subjected to dark channel filtering processing is added into a database.
In the embodiment, the detected image is subjected to dark channel filtering processing, so that the image with higher RGB three-channel value can be repaired, and a closed loop is generated by identifying similar scenes, so that the pose of the robot is optimized, and the positioning accuracy in the autonomous navigation process of the robot is improved.
In another embodiment of the present disclosure, the navigation device further includes an exposure value adjustment unit 410. Wherein the color threshold judging unit 310 is further configured to judge whether the three primary color components of the detected image are all smaller than the second color threshold of the corresponding channel if the three primary color components of the detected image are not all larger than the first color threshold of the corresponding channel; the exposure value adjustment unit 410 is configured to adjust the exposure value of the image sensor so that the image sensor re-captures the image if the three primary color components of the detected image are all smaller than the second color threshold of the corresponding channel.
In the embodiment, the device is insensitive to light and weather transformation, reduces the error closed loop formed by error matching or the condition that the closed loop is not detected, and improves the positioning precision of the navigation robot.
Fig. 5 is a schematic structural diagram of another embodiment of a machine vision based navigation device of the present disclosure. The navigation device includes a memory 510 and a processor 520. Wherein: memory 510 may be a magnetic disk, flash memory, or any other non-volatile storage medium. The memory 510 is used to store instructions in the embodiments corresponding to fig. 1 and 2. Processor 520 is coupled to memory 510 and may be implemented as one or more integrated circuits, such as a microprocessor or microcontroller. The processor 520 is configured to execute instructions stored in the memory.
In one embodiment, the navigation device 600 can also include a memory 610 and a processor 620, as shown in FIG. 6. Processor 620 is coupled to memory 610 through BUS 630. The navigation device 600 may also be connected to an external storage device 650 via a storage interface 640 for invoking external data, and may also be connected to a network or another computer system (not shown) via a network interface 660, which will not be described in detail herein.
In the embodiment, the data instruction is stored through the memory, and the instruction is processed through the processor, so that the positioning accuracy of the robot in the autonomous navigation process is improved.
In another embodiment of the present disclosure, a machine vision based navigation system is protected, the navigation system comprising: an image sensor located on the robot configured to acquire an image; and the machine vision based navigation device.
In another embodiment, a computer readable storage medium has stored thereon computer program instructions which, when executed by a processor, implement the steps of the method of the corresponding embodiment of fig. 1, 2. It will be apparent to those skilled in the art that embodiments of the present disclosure may be provided as a method, apparatus, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable non-transitory storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.
The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Thus far, the present disclosure has been described in detail. In order to avoid obscuring the concepts of the present disclosure, some details known in the art are not described. How to implement the solutions disclosed herein will be fully apparent to those skilled in the art from the above description.
Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It will be appreciated by those skilled in the art that modifications may be made to the above embodiments without departing from the scope and spirit of the disclosure. The scope of the present disclosure is defined by the appended claims.
Claims (10)
1. A machine vision based navigation method, comprising:
Judging whether three primary color components of an image detected by an image sensor are all larger than a first color threshold value of a corresponding channel;
If the three primary color components of the detected image are all larger than the first color threshold value of the corresponding channel, performing dark channel filtering processing on the detected image, wherein the dark channel value of the detected image is determined based on the three primary color channel values of the detected image, average filtering processing is performed on the dark channel value of the detected image, the transmittance is determined according to the dark channel value of the detected image and the dark channel value after the average filtering processing, the global atmosphere light value is determined according to the three primary color channel values of the detected image and the dark channel value after the average filtering processing, and the image after the dark channel filtering processing is determined according to the detected image, the transmittance and the global atmosphere light value;
Comparing the similarity of the image subjected to dark channel filtering processing with the closed-loop candidate image in the database;
and if the comparison result is greater than or equal to the similarity threshold, determining that the motion track of the robot where the image sensor is positioned forms a closed loop so as to optimize the pose of the robot.
2. The navigation method according to claim 1, wherein the similarity comparison of the image subjected to the dark channel filtering process with the closed-loop candidate image includes:
Determining a first bag-of-word vector of the image subjected to dark channel filtering;
and determining a similarity value of the first bag-of-word vector and a second bag-of-word vector of the closed-loop candidate image based on the chi-square distance.
3. The navigation method according to claim 1 or 2, further comprising:
if the three primary color components of the detected image are not all larger than the first color threshold value of the corresponding channel, judging whether the three primary color components of the detected image are all smaller than the second color threshold value of the corresponding channel;
And if the three primary color components of the detected image are smaller than the second color threshold value of the corresponding channel, adjusting the exposure value of the image sensor so that the image sensor can acquire the image again.
4. The navigation method of claim 2, further comprising:
and if the comparison result is smaller than the similarity threshold, adding the first bag-of-word vector of the image subjected to the dark channel filtering processing into the database.
5. A machine vision based navigation device, comprising:
a color threshold value judging unit configured to judge whether or not three primary color components of an image detected by the image sensor are all larger than a first color threshold value of a corresponding channel;
A dark channel filter processing unit configured to perform dark channel filter processing on the detected image if both three primary color components of the detected image are greater than a first color threshold of a corresponding channel, wherein a dark channel value of the detected image is determined based on the three primary color channel values of the detected image, a mean value filter processing is performed on the dark channel value of the detected image, a transmittance is determined based on the dark channel value of the detected image and the mean value filter processed dark channel value, a global atmospheric light value is determined based on the three primary color channel values of the detected image and the mean value filter processed dark channel value, and a dark channel filter processed image is determined based on the detected image, the transmittance, and the global atmospheric light value;
The similarity comparison unit is configured to compare the similarity of the image subjected to dark channel filtering processing with the closed-loop candidate image in the database;
and the closed loop determining unit is configured to determine that the motion trail of the robot where the image sensor is positioned forms a closed loop to optimize the pose of the robot if the comparison result is greater than or equal to the similarity threshold.
6. The navigation device of claim 5, wherein,
The similarity comparison unit is configured to determine a first bag-of-word vector of the image subjected to dark channel filtering; and determining a similarity value of the first bag-of-word vector and a second bag-of-word vector of the closed-loop candidate image based on the chi-square distance.
7. The navigation device according to claim 5 or 6, further comprising an exposure value adjusting unit, wherein,
The color threshold judging unit is further configured to judge whether the three primary color components of the detected image are all smaller than the second color threshold of the corresponding channel if the three primary color components of the detected image are not all larger than the first color threshold of the corresponding channel;
The exposure value adjustment unit is configured to adjust the exposure value of the image sensor so that the image sensor re-acquires an image if the three primary color components of the detected image are all smaller than the second color threshold of the corresponding channel.
8. A machine vision based navigation device, comprising:
a memory; and
A processor coupled to the memory, the processor configured to perform the navigation method of any of claims 1-4 based on instructions stored in the memory.
9. A machine vision based navigation system, comprising:
An image sensor located on the robot configured to acquire an image; and
A machine vision based navigation device as claimed in any one of claims 5 to 8.
10. A computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement the navigation method of any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910508334.2A CN112083716B (en) | 2019-06-13 | 2019-06-13 | Navigation method, device and system based on machine vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910508334.2A CN112083716B (en) | 2019-06-13 | 2019-06-13 | Navigation method, device and system based on machine vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112083716A CN112083716A (en) | 2020-12-15 |
| CN112083716B true CN112083716B (en) | 2024-07-12 |
Family
ID=73734504
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910508334.2A Active CN112083716B (en) | 2019-06-13 | 2019-06-13 | Navigation method, device and system based on machine vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112083716B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104751421A (en) * | 2015-03-10 | 2015-07-01 | 西安理工大学 | Method for achieving image defogging on FPGA |
| CN105841687A (en) * | 2015-01-14 | 2016-08-10 | 上海智乘网络科技有限公司 | Indoor location method and indoor location system |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR100571834B1 (en) * | 2004-02-27 | 2006-04-17 | 삼성전자주식회사 | Method and apparatus for detecting floor dust of cleaning robot |
| KR100716422B1 (en) * | 2005-01-06 | 2007-05-08 | 에스케이 텔레콤주식회사 | Matching service system and method using pattern recognition |
| CN106033537A (en) * | 2015-03-19 | 2016-10-19 | 宁夏巨能机器人系统有限公司 | Visual identification device and method for robot positioning |
| CN104767912B (en) * | 2015-04-14 | 2018-11-30 | 深圳日盛科技股份有限公司 | Video defogging method and system based on FPGA |
| CN105279739A (en) * | 2015-09-08 | 2016-01-27 | 哈尔滨工程大学 | Self-adaptive fog-containing digital image defogging method |
| CN105989583B (en) * | 2016-07-19 | 2018-07-24 | 河海大学 | A kind of image defogging method |
| CN106607907B (en) * | 2016-12-23 | 2017-09-26 | 西安交通大学 | A kind of moving-vision robot and its investigating method |
| CN108549376A (en) * | 2018-04-16 | 2018-09-18 | 爱啃萝卜机器人技术（深圳）有限责任公司 | A kind of navigation locating method and system based on beacon |
-
2019
- 2019-06-13 CN CN201910508334.2A patent/CN112083716B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105841687A (en) * | 2015-01-14 | 2016-08-10 | 上海智乘网络科技有限公司 | Indoor location method and indoor location system |
| CN104751421A (en) * | 2015-03-10 | 2015-07-01 | 西安理工大学 | Method for achieving image defogging on FPGA |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112083716A (en) | 2020-12-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Xiao et al. | Dynamic-SLAM: Semantic monocular visual localization and mapping based on deep learning in dynamic environment | |
| Berrio et al. | Camera-LIDAR integration: Probabilistic sensor fusion for semantic mapping | |
| US9811732B2 (en) | Systems and methods for object tracking | |
| JP6294615B2 (en) | System and method for detection and tracking of moving objects | |
| CN111462135A (en) | Semantic Mapping Method Based on Visual SLAM and 2D Semantic Segmentation | |
| CN110587597B (en) | SLAM closed loop detection method and detection system based on laser radar | |
| KR20200128145A (en) | Methods and devices, vehicles, and electronic devices for traffic light detection and intelligent driving | |
| CN108694724A (en) | A kind of long-time method for tracking target | |
| CN112101160B (en) | Binocular semantic SLAM method for automatic driving scene | |
| KR20110011424A (en) | Position recognition and driving control method of mobile robot and mobile robot using same | |
| AU2013213659A1 (en) | Method and system for using fingerprints to track moving objects in video | |
| US8462211B2 (en) | Method of detecting objects | |
| CN113029185A (en) | Road marking change detection method and system in crowdsourcing type high-precision map updating | |
| Hu et al. | Mapping and localization using semantic road marking with centimeter-level accuracy in indoor parking lots | |
| CN106599918B (en) | vehicle tracking method and system | |
| Françani et al. | Dense prediction transformer for scale estimation in monocular visual odometry | |
| CN111553945A (en) | Vehicle positioning method | |
| CN117670931A (en) | Multi-camera multi-target tracking method and device based on computer vision | |
| JP2018073308A (en) | Recognition device, program | |
| Omar et al. | Detection and localization of traffic lights using YOLOv3 and Stereo Vision | |
| CN116189150A (en) | Monocular 3D target detection method, device, equipment and medium based on fusion output | |
| CN110738668A (en) | method and system for intelligently controlling high beam and vehicle | |
| CN112083716B (en) | Navigation method, device and system based on machine vision | |
| CN110956616B (en) | Object detection method and system based on stereoscopic vision | |
| CN111192290A (en) | Blocking processing method for pedestrian image detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20220125 Address after: 100007 room 205-32, floor 2, building 2, No. 1 and No. 3, qinglonghutong a, Dongcheng District, Beijing Applicant after: Tianyiyun Technology Co.,Ltd. Address before: No.31, Financial Street, Xicheng District, Beijing, 100033 Applicant before: CHINA TELECOM Corp.,Ltd. |
|
| TA01 | Transfer of patent application right | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |