[go: up one dir, main page]

CN118262336B - Indoor parking lot positioning method and system based on visual SLAM - Google Patents

Indoor parking lot positioning method and system based on visual SLAM Download PDF

Info

Publication number
CN118262336B
CN118262336B CN202410685784.XA CN202410685784A CN118262336B CN 118262336 B CN118262336 B CN 118262336B CN 202410685784 A CN202410685784 A CN 202410685784A CN 118262336 B CN118262336 B CN 118262336B
Authority
CN
China
Prior art keywords
vehicle
parking lot
information
image
coordinate system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410685784.XA
Other languages
Chinese (zh)
Other versions
CN118262336A (en
Inventor
冯挽强
王旭华
蒋金鑫
张周平
关怀
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanchang Intelligent New Energy Vehicle Research Institute
Original Assignee
Nanchang Intelligent New Energy Vehicle Research Institute
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanchang Intelligent New Energy Vehicle Research Institute filed Critical Nanchang Intelligent New Energy Vehicle Research Institute
Priority to CN202410685784.XA priority Critical patent/CN118262336B/en
Publication of CN118262336A publication Critical patent/CN118262336A/en
Application granted granted Critical
Publication of CN118262336B publication Critical patent/CN118262336B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G06V20/586—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads of parking space
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G06V10/245—Aligning, centring, orientation detection or correction of the image by locating a pattern; Special marks for positioning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/10—Internal combustion engine [ICE] based vehicles
    • Y02T10/40—Engine management systems

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses an indoor parking lot positioning method and system based on visual SLAM, and belongs to the field of intelligent transportation. The method comprises the following steps: acquiring Apriltag image data by using a monocular camera, and acquiring vehicle Inertial Measurement Unit (IMU) data by using an IMU; extracting feature point information and Apriltag ID information in the image, and resolving IMU data to obtain pose information of the vehicle; the visual odometer between the front frame and the rear frame is realized through the characteristic point information and the ID information of Apriltag; according to pose information of the vehicle, performing back-end optimization by adopting an extended Kalman filtering algorithm; and loop detection is carried out by adopting the bag-of-word model, and the map is updated, so that the indoor parking lot map building and the vehicle positioning are realized. The invention realizes accurate image construction and positioning of the indoor parking lot.

Description

Indoor parking lot positioning method and system based on visual SLAM
Technical Field
The invention belongs to the field of intelligent transportation, and more particularly relates to an indoor parking lot positioning method and system based on visual SLAM.
Background
With the development of society and technology, the rapid increase of vehicle demands brings a series of problems, an intelligent transportation system becomes an important way for solving a plurality of problems, an unmanned vehicle is an important component of the intelligent transportation system, and reliable map building and positioning capability plays a vital role in autonomous cruise control, collision early warning and path planning. For example, indoor parking lot mapping and positioning can provide richer information for path planning and positioning of vehicles for autonomous parking and calling in the indoor parking lot.
SLAM technology is key point and hotspot in autonomous navigation and location field, has extensive application prospect in intelligent transportation field. AprilTag is a library of visual references, which is widely used in the AR, robot, camera calibration field. At present, most vision SLAM directly adopts characteristic point information or independently adopts Apriltag ID information to realize map updating and positioning. The anti-interference capability of single characteristic point information is weak, and the ID information of single Apriltag is difficult to cover each frame of image, so that data is lost, and errors are large.
Disclosure of Invention
In view of the above, the present invention aims to provide an indoor parking lot positioning method and system based on visual SLAM, which solves the problem of large positioning error caused by directly adopting characteristic point information or independently adopting Apriltag ID information to realize map updating and positioning in the prior art.
The embodiment of the invention is realized as follows:
an indoor parking lot positioning method based on visual SLAM, the method comprising:
Step 1: acquiring Apriltag image data by adopting a monocular camera, and acquiring IMU data information of the vehicle by adopting an inertial measurement unit; the image data includes a plurality Apriltag of indoor parking lot images; the IMU data comprises vehicle attitude, angular velocity and linear acceleration information;
step 2: the monocular camera and the inertial measurement unit are calibrated to a vehicle coordinate system in a combined mode;
step 3: preprocessing a parking lot image acquired by the monocular camera, extracting characteristic points in the parking lot image by adopting a SIFT algorithm, and acquiring Apriltag positions and IDs thereof in the parking lot image;
step 4: preprocessing the IMU data, and calculating pose information of the vehicle;
Step 5: for the obtained feature points and ID information, combining with vehicle pose information, adopting extended Kalman filtering to perform map optimization;
Step 6: and loop detection is carried out by adopting the bag-of-word model, and the map is updated, so that the indoor parking lot map building and the vehicle positioning are realized.
Further, the method for positioning the indoor parking lot based on the visual SLAM comprises the following steps: the monocular camera is a 180-degree wide-angle camera, and the inertial measurement unit is a vehicle-mounted inertial measurement unit.
Further, the method for positioning the indoor parking lot based on the visual SLAM comprises the following steps: the method for jointly calibrating the monocular camera and the inertial measurement unit to the vehicle coordinate system in the step 2 comprises the following steps:
S201, an inertial measurement unit coordinate system and a vehicle coordinate system are set, wherein the front of the vehicle is taken as a Y axis, the left is taken as an X axis, the upward is taken as a Z axis, and the origin is respectively the center of the inertial measurement unit and the center of the vehicle;
S202, calibrating the monocular camera, obtaining internal and external parameters of the monocular camera according to a Zhang Zhengyou calibration method, converting pixel coordinate data into a camera coordinate system according to a formula (1), then manually measuring the coordinates of the monocular camera in the vehicle coordinate system O V-XVYVZV and the positions of the monocular camera in the camera coordinate system by adopting a plurality of calibration objects, and obtaining a rotation matrix R C and a translation vector T C according to a formula (2);
(1)
(2)
Wherein: xc, yc, zc are coordinate values of the calibration object in a camera coordinate system, xv, yv, zv are coordinate values of the calibration object in a vehicle coordinate system, R C,TC is a rotation matrix and a translation matrix of data converted from the camera coordinate system to the vehicle coordinate system, f is a camera focal length, and 1/dx and 1/dy represent: the number of pixels contained in each unit length in the x direction and the y direction can be decimal; gamma is a distortion factor, taking 0; u 0、v0 represents the number of horizontal and vertical pixels of the phase difference between the center pixel coordinate of the image and the original pixel coordinate of the image, and u and v are coordinate values of the calibration object on the image;
S203, calibrating the inertial measurement unit to a vehicle coordinate system, selecting a plurality of points, measuring the coordinates of the points in the inertial measurement unit coordinate system O I-XIYIZI and the coordinates in the vehicle coordinate system O V-XVYVZV, obtaining a rotation matrix R I and a translation vector T I according to a formula (3), and calibrating the rotation matrix R I and the translation vector T I to the vehicle coordinate system;
(3)
Wherein: r I,TI is a rotation matrix and a translation matrix of data converted from the inertial measurement unit coordinate system to a vehicle coordinate system, and X I、YI、ZI is a coordinate value of a calibration object under the inertial measurement unit coordinate system.
Further, the method for positioning the indoor parking lot based on the visual SLAM comprises the following steps: step 3, preprocessing the parking lot image acquired by the monocular camera, extracting characteristic points in the parking lot image by adopting a SIFT algorithm, and acquiring Apriltag positions and IDs thereof in the parking lot image, wherein the method comprises the following steps:
S301, preprocessing comprises Gaussian smoothing and DoG feature map calculation;
S302, extracting feature points in the parking lot image by adopting a SIFT algorithm: identifying potential interest points with scales and unchanged rotation of the parking lot image through a Gaussian differential function, positioning key points according to the interest points, determining characteristic directions, determining main directions of the characteristic points by utilizing gradient histograms of neighborhood of the characteristic points, generating description operators in the neighborhood of the characteristic points, generating SIFT feature vectors of 4x4x8 dimensions, and carrying out pairwise comparison through the SIFT feature vectors of the key points to find out a plurality of pairs of mutually matched characteristic points;
S303, acquiring Apriltag positions and IDs thereof in the parking lot image: and carrying out graying and binarization on the parking lot image, carrying out edge detection by adopting a Sobel operator to obtain a binarized edge image, carrying out polygon analysis on the binarized edge image until all binary points are traversed, calculating a convex hull of each polygon by utilizing a polygon convex hull searching algorithm, solving the areas of the convex hulls and the polygons thereof, comparing the areas of the convex hulls and the polygons thereof, removing the polygons when the areas of the polygons are larger than the areas of the convex hulls, keeping other polygons meeting the conditions, carrying out quadrilateral approximation on the other polygons by using a Taglas-Prak algorithm, and finally decoding the detected quadrilateral internal patterns to obtain the digital ID corresponding to each pattern.
Further, the method for positioning the indoor parking lot based on the visual SLAM comprises the following steps: and 5, combining the acquired characteristic points and ID information with vehicle pose information, and adopting an extended Kalman filter to perform map optimization, wherein the method comprises the following steps:
S501, predicting joint state estimation of pose and feature points at the current moment according to feature point information, ID information and vehicle pose information in the parking lot image at the previous moment, wherein the joint state of the pose and the feature points at the current moment represents a mean value, and the variance is solved by a motion change matrix, and the method comprises the following specific steps of:
constructing a basic equation: set discrete time The position of the vehicle isThe motion equation of the vehicle from the last moment to the next moment is:；
setting the information of the characteristic points as The inertial measurement unit is in positionWhere it detectsThe observation equation is:；
wherein, The position at time k is indicated,For the motion of the vehicle estimated by the feature points,Information and ID information representing the feature points obtained by the inertial measurement unit, and a covariance matrix of states and errors at the k+1 time is estimated and predicted according to the optimal state at the k time:
(4)
(5)
equation (4) is a state test equation in which, Is the optimal estimate of the system state at time k,Is based on the state optimal estimation at k timeSystem input of sum timePredicting the system state at the moment k+1;
Equation (5) is the covariance matrix prediction equation of the error, wherein, Is the optimal estimation of the system state at time kA corresponding error covariance matrix is provided,Is a prediction of system stateA corresponding error covariance matrix is provided,Is an equation ofIs a jacobian matrix of the matrix,Is the covariance matrix of the system process noise at the moment k;
S502, optimizing a prediction result of the system state at the time k+1 obtained in the last step according to the actual characteristic points and ID information of the system at the time k+1 to obtain an optimal estimation of the system state at the time k+1, wherein the method specifically comprises the following steps:
(6)
wherein, Is the optimal estimate of the system state at time k +1,Is a Kalman gain matrix, and the calculation method is as follows:
(7)
wherein, Is an equation ofIs a jacobian matrix of the matrix,Is the covariance matrix of the measured noise at time k+1;
S503, continuously updating an error covariance matrix at the moment k+1, wherein the formula is as follows:
(8)
Wherein I is an identity matrix, Is the optimal estimate of the system state at time k+1A corresponding error covariance matrix.
Further, the method for positioning the indoor parking lot based on the visual SLAM comprises the following steps: and 6, carrying out loop detection by adopting a bag-of-word model, wherein the specific steps are as follows:
s601, constructing a dictionary containing the fusion of the characteristic points and the ID information, namely, a set of words, wherein the set of words is expressed as follows:
(9)
S602, determining words in a frame of image to form a bag-of-words vector, wherein the bag-of-words vector is represented by the following formula:
(10)
wherein 1 indicates that the word is present, 0 indicates that none, Respectively representing the first feature point and the first ID information,Respectively representing an nth feature point and nth ID information;
s603, comparing the difference of the description vectors F of the two frames of images, wherein the similarity is larger than a preset threshold value The two images of the car are used as a detected loop, and the map is updated, so that the indoor parking lot map building and the vehicle positioning are realized.
Another object of the present invention is to provide an indoor parking lot positioning system based on visual SLAM, which is characterized in that: the system comprises:
The acquisition module is used for: the method comprises the steps of acquiring Apriltag image data by using a monocular camera, and acquiring IMU data information of a vehicle by using an inertial measurement unit; the image data includes a plurality Apriltag of indoor parking lot images; the IMU data comprises vehicle attitude, angular velocity and linear acceleration information;
And (3) a calibration module: the method comprises the steps of performing joint calibration on the monocular camera and an inertial measurement unit to a vehicle coordinate system;
and a pretreatment module: the method comprises the steps of preprocessing a parking lot image obtained by the monocular camera, extracting characteristic points in the parking lot image by adopting a SIFT algorithm, and obtaining Apriltag positions and IDs thereof in the parking lot image;
And a resolving module: the IMU data is used for preprocessing the IMU data, and pose information of the vehicle is calculated;
and an optimization module: the map optimization method is used for optimizing the obtained characteristic points and ID information by combining the vehicle pose information and adopting extended Kalman filtering;
and a positioning module: the method is used for carrying out loop detection by adopting the bag-of-word model, updating the map and realizing the construction of the indoor parking lot and the positioning of the vehicle.
Further, the indoor parking lot positioning system based on visual SLAM, wherein: the monocular camera is a 180-degree wide-angle camera, and the inertial measurement unit is a vehicle-mounted inertial measurement unit.
Another object of the present invention is to provide a readable storage medium having stored thereon a computer program, characterized in that the program when executed by a processor realizes the steps of the method according to any of the above.
It is a further object of the invention to provide an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, which processor implements the steps of the method described above when executing the program.
The invention adopts the combination of the image characteristic point information and the ID information Apriltag as the initial information of the image, thereby overcoming the defects that single information is easy to be interfered and data is easy to be lost. The method can accurately and effectively extract the fused initial information from the image, optimizes the fused initial data information by adopting the extended Kalman filtering, overcomes the defects of large error and low precision of a linear model, can be more in line with the actual situation, and further reduces the noise in the SLAM process. The word bag model is adopted to carry out loop detection, a dictionary which contains the fusion of the characteristic points and the image ID information is constructed, and the defects of large loop detection error and difficult loop return caused by single characteristic point information are overcome. Most SLAM adopts a linear model, and data which is nonlinear in most practical cases is ignored. The method solves the problem that most visual SLAM in the prior art directly adopts characteristic point information or independently adopts Apriltag ID information to realize map updating and positioning. The anti-interference capability of single characteristic point information is weak, and the ID information of single Apriltag is difficult to cover each frame of image, so that the data is lost, and the problem of large positioning error is caused.
Drawings
Fig. 1 is a flowchart of an indoor parking lot positioning method based on visual SLAM according to a first embodiment of the present invention;
fig. 2 is a block diagram illustrating a construction of an indoor parking lot positioning system based on a visual SLAM according to a second embodiment of the present invention.
Detailed Description
Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
Example 1
Referring to fig. 1, an indoor parking lot positioning method based on visual SLAM according to a first embodiment of the present invention includes steps 1 to 6.
Step 1: acquiring Apriltag image data by adopting a monocular camera, and acquiring IMU data information of the vehicle by adopting an inertial measurement unit; the image data includes a plurality Apriltag of indoor parking lot images; the IMU data includes vehicle pose, angular velocity, and linear acceleration information.
The monocular camera can adopt a 180-degree wide-angle camera, and the inertial measurement unit is a vehicle-mounted inertial measurement unit.
Step 2: and carrying out joint calibration on the monocular camera and the inertial measurement unit to a vehicle coordinate system.
The method comprises the steps of setting an inertial measurement unit coordinate system and a vehicle coordinate system, wherein the front of a vehicle is uniformly taken as a Y axis, the left is taken as an X axis, the upward is taken as a Z axis, and the origin is respectively the center of the inertial measurement unit (sensor) and the center of the vehicle;
Calibrating the monocular camera, obtaining internal and external parameters of the monocular camera according to a Zhang Zhengyou calibration method, converting pixel coordinate data into a camera coordinate system according to a formula (1), then manually measuring coordinates of the monocular camera in the vehicle coordinate system O V-XVYVZV and positions of the monocular camera in the camera coordinate system by adopting a plurality of calibration objects, and obtaining a rotation matrix R C and a translation vector T C according to a formula (2);
(1)
(2)
Wherein: xc, yc, zc are coordinate values of the calibration object in a camera coordinate system, xv, yv, zv are coordinate values of the calibration object in a vehicle coordinate system, R C,TC is a rotation matrix and a translation matrix of data converted from the camera coordinate system to the vehicle coordinate system, f is a camera focal length, and 1/dx and 1/dy represent: the number of pixels contained in each unit length in the x direction and the y direction can be decimal; gamma is a distortion factor, taking 0; u 0、v0 represents the number of horizontal and vertical pixels of the phase difference between the center pixel coordinate of the image and the original pixel coordinate of the image, and u and v are coordinate values of the calibration object on the image;
Calibrating the inertial measurement unit to a vehicle coordinate system, selecting a plurality of points, measuring the coordinates of the points in the inertial measurement unit coordinate system O I-XIYIZI and the coordinates in the vehicle coordinate system O V-XVYVZV, obtaining a rotation matrix R I and a translation vector T I according to a formula (3), and calibrating the rotation matrix R I and the translation vector T I to the vehicle coordinate system;
(3)
Wherein: r I,TI is a rotation matrix and a translation matrix of data converted from the inertial measurement unit coordinate system to a vehicle coordinate system, and X I、YI、ZI is a coordinate value of a calibration object under the inertial measurement unit coordinate system.
Step 3: preprocessing the parking lot image acquired by the monocular camera, extracting characteristic points in the parking lot image by adopting a SIFT algorithm, and acquiring Apriltag positions and IDs thereof in the parking lot image.
Specifically, preprocessing comprises Gaussian smoothing and DoG feature map calculation;
extracting characteristic points in the parking lot image by adopting a SIFT algorithm: identifying potential interest points with scales and invariable rotation of the parking lot image through a Gaussian differential function, positioning key points according to the interest points, determining the characteristic directions, specifically, determining the positions and the scales at each candidate position through a fitting fine model, determining the main directions of the characteristic points by utilizing gradient histograms of characteristic point neighborhood, generating a description operator in the characteristic point neighborhood region, generating SIFT characteristic vectors of 4x4x8 dimensions, and carrying out pairwise comparison to find out a plurality of pairs of characteristic points matched with each other through the SIFT characteristic vectors of the key points, so that the corresponding relation between scenes can be established, and camera movement, namely vehicle movement, is estimated;
In addition, according to the information of the adjacent images, rough camera motion can be estimated, and a good initial value is provided for the back end. The application is a monocular camera, only 2D pixel coordinates can be obtained, so that camera motion estimation is carried out by adopting epipolar geometry according to two groups of 2D points, and the method specifically comprises the following steps: the position change of the external point P with respect to the camera is calculated from the two adjacent frames of images. And then, taking P as an unchanged external point, and reversely pushing out the change of a camera coordinate system relative to P, so as to position the camera.
Solving epipolar constraint: under the coordinate system of the first frame, the spatial coordinates of P are set as [ X, Y, Z ] T, and two pixel points are obtained according to the pinhole camera modelThe pixel positions of (1) are:
；
wherein, For a pixel point of P on the first frame image,For P pixel point on the second frame image, K is camera reference, R, t is camera motion of two coordinate systems, the first frame coordinate system is transferred to the rotation and translation matrix of the second frame coordinate system,The constant factor is eliminated, and the second coordinate is obtained after writing。
Taking outHere, whereIs the coordinates on the normalized plane of the two pixel points, and the secondary coordinates are brought into+T, and the two sides and t are the outer product to obtain. Two sides are simultaneously multiplied by leftThere is a method of producing a liquid crystal display device,From the meaning of the outer product, the vectorPerpendicular toTherefore, if the inner product is 0, there isThis formula is called the epipolar constraint. If it is toIs brought into the epipolar constraint formula, withThe equation is also epipolar constraint.
Order theCalled the essence matrix, the epipolar constraint is expressed as；
Order theCalled the basis matrix, the epipolar constraint may be expressed as。
And solving the pose of the camera through the essential matrix E and the basic matrix F, solving E or F according to the pixel positions of the paired feature points, and then solving R and t according to E or F.
Solving the pose according to the essential matrix: there are 9 unknowns within the matrix by definition of the essential matrix as a 3x3 matrix. The patent adopts a classical eight-point method to estimate E, and specifically comprises the following steps:
For a pair of matching points, its normalized coordinates are . According to the epipolar constraint, the following steps are obtained:
expanding the matrix E and writing the matrix E into a vector form: the above can be written as . The other feature points are similarly expanded, and the merging calculation can obtain the following formula:
；
Subsequently, E is solved using Singular Value Decomposition (SVD) decomposition. Let the SVD of E be: wherein U, V is an orthogonal matrix, Is a matrix of singular values that are used,Is the singular value of the decomposition.
In SVD decomposition, for any E, there are four possible solutions for decomposing to get R and t:
Where Rz (pi/2) represents the rotation matrix of R rotated 90 degrees about the Z axis, where P has a positive depth in both cameras only in the first set of solutions. And according to the obtained rotation translation matrix R and t between two continuous frames, the movement of the camera between the two frames of images can be obtained, so that the movement track of the camera is estimated from the continuous video frames.
And acquiring Apriltag positions and IDs (identity) of the parking lot images: and carrying out graying and binarization on the parking lot image, adopting a Sobel operator to carry out edge detection, carrying out polygon analysis on the obtained binarized edge image, analyzing a nesting relationship by using an edge structure, and expressing an original image by using an edge. The algorithm starts from a starting point, edits pixels of edges, searches for similar edge points of the starting point, when scanning to the starting point, the polygon closed loop is formed, switches to the next starting point to repeat the operation until all binary points are traversed, the number of excluded edges is smaller than 4, further processes the satisfied conditions, calculates a convex hull of each polygon by utilizing a polygon convex hull searching algorithm, calculates areas of the convex hulls and the polygons thereof, compares the areas of the convex hulls and the polygons thereof, excludes the polygons when the areas of the polygons are larger than the areas of the convex hulls, can effectively exclude non-convex polygons, and does not satisfy the ratio of the areas of the polygons to the areas of the convex hulls to be larger than 0.8, retains other polygons satisfying the conditions, performs quadrilateral approximation on the other polygons by using a Taglas-Prime algorithm, and finally decodes the detected internal patterns of the quadrilaterals to obtain the digital ID corresponding to each pattern.
Step 4: preprocessing the IMU data, and calculating pose information of the vehicle;
step 5: and combining the acquired characteristic points and ID information with vehicle pose information, and adopting extended Kalman filtering to perform map optimization.
Specifically, the joint state estimation of the pose and the feature point at the current moment is predicted according to the feature point information, the ID information and the vehicle pose information in the parking lot image at the previous moment, wherein the joint state of the pose and the feature point at the current moment represents a mean value, and the variance is solved by a motion change matrix, and the method specifically comprises the following steps:
constructing a basic equation: set discrete time The position of the vehicle isThe motion equation of the vehicle from the last moment to the next moment is:；
setting the information of the characteristic points as The inertial measurement unit is in positionWhere it detectsThe observation equation is:；
wherein, The position at time k is indicated,For the motion of the vehicle estimated by the feature points,Information and ID information representing the feature points obtained by the inertial measurement unit, and a covariance matrix of states and errors at the k+1 time is estimated and predicted according to the optimal state at the k time:
(4)
(5)
equation (4) is a state test equation in which, Is the optimal estimate of the system state at time k,Is based on the state optimal estimation at k timeSystem input of sum timePredicting the system state at the moment k+1;
Equation (5) is the covariance matrix prediction equation of the error, wherein, Is the optimal estimation of the system state at time kA corresponding error covariance matrix is provided,Is a prediction of system stateA corresponding error covariance matrix is provided,Is an equation ofIs a jacobian matrix of the matrix,Is the covariance matrix of the system process noise at the moment k;
according to the actual characteristic points and ID information of the system at the time k+1, optimizing the prediction result of the system state at the time k+1 obtained in the last step to obtain the optimal estimation of the system state at the time k+1, wherein the method comprises the following steps:
(6)
wherein, Is the optimal estimate of the system state at time k +1,Is a Kalman gain matrix, and the calculation method is as follows:
(7)
wherein, Is an equation ofIs a jacobian matrix of the matrix,Is the covariance matrix of the measured noise at time k+1;
Continuously updating the error covariance matrix at the moment k+1, wherein the formula is as follows:
(8)
Wherein I is an identity matrix, Is the optimal estimate of the system state at time k+1A corresponding error covariance matrix.
Step 6: and (3) carrying out loop detection by adopting an improved bag-of-words model, updating a map, and realizing the construction of an indoor parking lot and the positioning of vehicles.
Specifically, a dictionary including the feature points and the ID information fused together, that is, a set of words, is constructed, expressed as:
(9)
determining words in a frame of image to form a bag-of-words vector, wherein the bag-of-words vector is represented by the following formula:
(10)
wherein 1 indicates that the word is present, 0 indicates that none, Respectively representing the first feature point and the first ID information,Respectively representing an nth feature point and nth ID information;
comparing the difference of the description vectors F of the two frames of images, wherein the similarity is larger than a preset threshold value The two images of the car are used as a detected loop, and the map is updated, so that the indoor parking lot map building and the vehicle positioning are realized.
In summary, the indoor parking lot positioning method based on visual SLAM in the embodiment of the invention overcomes the defects that single information is easy to be interfered and data is easy to be lost by adopting the combination of the image characteristic point information and the Apriltag ID information as the initial information of the image. The method can accurately and effectively extract the fused initial information from the image, optimizes the fused initial data information by adopting the extended Kalman filtering, overcomes the defects of large error and low precision of a linear model, can be more in line with the actual situation, and further reduces the noise in the SLAM process. The word bag model is adopted to carry out loop detection, a dictionary which contains the fusion of the characteristic points and the image ID information is constructed, and the defects of large loop detection error and difficult loop return caused by single characteristic point information are overcome. Most SLAM adopts a linear model, and data which is nonlinear in most practical cases is ignored. The method solves the problem that most visual SLAM in the prior art directly adopts characteristic point information or independently adopts Apriltag ID information to realize map updating and positioning. The anti-interference capability of single characteristic point information is weak, and the ID information of single Apriltag is difficult to cover each frame of image, so that the data is lost, and the problem of large positioning error is caused.
Example two
Referring to fig. 2, an indoor parking lot positioning system based on visual SLAM according to the present invention is shown. The method is characterized in that: the system comprises:
The acquisition module 100: the method comprises the steps of acquiring Apriltag image data by using a monocular camera, and acquiring IMU data information of a vehicle by using an inertial measurement unit; the image data includes a plurality Apriltag of indoor parking lot images; the IMU data comprises vehicle attitude, angular velocity and linear acceleration information;
Calibration module 200: the method comprises the steps of performing joint calibration on the monocular camera and an inertial measurement unit to a vehicle coordinate system;
preprocessing module 300: the method comprises the steps of preprocessing a parking lot image obtained by the monocular camera, extracting characteristic points in the parking lot image by adopting a SIFT algorithm, and obtaining Apriltag positions and IDs thereof in the parking lot image;
the calculation module 400: the IMU data is used for preprocessing the IMU data, and pose information of the vehicle is calculated;
optimization module 500: the map optimization method is used for optimizing the obtained characteristic points and ID information by combining the vehicle pose information and adopting extended Kalman filtering;
Positioning module 600: the method is used for carrying out loop detection by adopting the bag-of-word model, updating the map and realizing the construction of the indoor parking lot and the positioning of the vehicle.
Further, in the indoor parking lot positioning method based on the visual SLAM, the monocular camera is a 180-degree wide-angle camera, and the inertial measurement unit is a vehicle-mounted inertial measurement unit.
Further, the method for positioning an indoor parking lot based on visual SLAM, wherein the calibration module is specifically configured to:
s201, an inertial measurement unit coordinate system and a vehicle coordinate system are set, wherein the front of the vehicle is taken as a Y axis, the left is taken as an X axis, the upward is taken as a Z axis, and the origin is respectively the center of the inertial measurement unit and the center of the vehicle;
S202, calibrating the monocular camera, obtaining internal and external parameters of the monocular camera according to a Zhang Zhengyou calibration method, converting pixel coordinate data into a camera coordinate system according to a formula (1), then manually measuring the coordinates of the monocular camera in the vehicle coordinate system O V-XVYVZV and the positions of the monocular camera in the camera coordinate system by adopting a plurality of calibration objects, and obtaining a rotation matrix R C and a translation vector T C according to a formula (2);
(1)
(2)
Wherein: xc, yc, zc are coordinate values of the calibration object in a camera coordinate system, xv, yv, zv are coordinate values of the calibration object in a vehicle coordinate system, R C,TC is a rotation matrix and a translation matrix of data converted from the camera coordinate system to the vehicle coordinate system, f is a camera focal length, and 1/dx and 1/dy represent: the number of pixels contained in each unit length in the x direction and the y direction can be decimal; gamma is a distortion factor, taking 0; u 0、v0 represents the number of horizontal and vertical pixels of the phase difference between the center pixel coordinate of the image and the original pixel coordinate of the image, and u and v are coordinate values of the calibration object on the image;
S203, calibrating the inertial measurement unit to a vehicle coordinate system, selecting a plurality of points, measuring the coordinates of the points in the inertial measurement unit coordinate system O I-XIYIZI and the coordinates in the vehicle coordinate system O V-XVYVZV, obtaining a rotation matrix R I and a translation vector T I according to a formula (3), and calibrating the rotation matrix R I and the translation vector T I to the vehicle coordinate system;
(3)
Wherein: r I,TI is a rotation matrix and a translation matrix of data converted from the inertial measurement unit coordinate system to a vehicle coordinate system, and X I、YI、ZI is a coordinate value of a calibration object under the inertial measurement unit coordinate system.
Further, the method for positioning an indoor parking lot based on visual SLAM, wherein the preprocessing module is specifically configured to:
S301, preprocessing comprises Gaussian smoothing and DoG feature map calculation;
S302, extracting feature points in the parking lot image by adopting a SIFT algorithm: identifying potential interest points with scales and unchanged rotation of the parking lot image through a Gaussian differential function, positioning key points according to the interest points, determining characteristic directions, determining main directions of the characteristic points by utilizing gradient histograms of neighborhood of the characteristic points, generating description operators in the neighborhood of the characteristic points, generating SIFT feature vectors of 4x4x8 dimensions, and carrying out pairwise comparison through the SIFT feature vectors of the key points to find out a plurality of pairs of mutually matched characteristic points;
S303, acquiring Apriltag positions and IDs thereof in the parking lot image: and carrying out graying and binarization on the parking lot image, carrying out edge detection by adopting a Sobel operator to obtain a binarized edge image, carrying out polygon analysis on the binarized edge image until all binary points are traversed, calculating a convex hull of each polygon by utilizing a polygon convex hull searching algorithm, solving the areas of the convex hulls and the polygons thereof, comparing the areas of the convex hulls and the polygons thereof, removing the polygons when the areas of the polygons are larger than the areas of the convex hulls, keeping other polygons meeting the conditions, carrying out quadrilateral approximation on the other polygons by using a Taglas-Prak algorithm, and finally decoding the detected quadrilateral internal patterns to obtain the digital ID corresponding to each pattern.
Further, the method for positioning an indoor parking lot based on visual SLAM, wherein the optimization module is specifically configured to:
S501, predicting joint state estimation of pose and feature points at the current moment according to feature point information, ID information and vehicle pose information in the parking lot image at the previous moment, wherein the joint state of the pose and the feature points at the current moment represents a mean value, and the variance is solved by a motion change matrix, and the method comprises the following specific steps of:
constructing a basic equation: set discrete time The position of the vehicle isThe motion equation of the vehicle from the last moment to the next moment is:；
setting the information of the characteristic points as The inertial measurement unit is in positionWhere it detectsThe observation equation is:；
wherein, The position at time k is indicated,For the motion of the vehicle estimated by the feature points,Information and ID information representing the feature points obtained by the inertial measurement unit, and a covariance matrix of states and errors at the k+1 time is estimated and predicted according to the optimal state at the k time:
(4)
(5)
equation (4) is a state test equation in which, Is the optimal estimate of the system state at time k,Is based on the state optimal estimation at k timeSystem input of sum timePredicting the system state at the moment k+1;
Equation (5) is the covariance matrix prediction equation of the error, wherein, Is the optimal estimation of the system state at time kA corresponding error covariance matrix is provided,Is a prediction of system stateA corresponding error covariance matrix is provided,Is an equation ofIs a jacobian matrix of the matrix,Is the covariance matrix of the system process noise at the moment k;
S502, optimizing a prediction result of the system state at the time k+1 obtained in the last step according to the actual characteristic points and ID information of the system at the time k+1 to obtain an optimal estimation of the system state at the time k+1, wherein the method specifically comprises the following steps:
(6)
wherein, Is the optimal estimate of the system state at time k +1,Is a Kalman gain matrix, and the calculation method is as follows:
(7)
wherein, Is an equation ofIs a jacobian matrix of the matrix,Is the covariance matrix of the measured noise at time k+1;
S503, continuously updating an error covariance matrix at the moment k+1, wherein the formula is as follows:
(8)
Wherein I is an identity matrix, Is the optimal estimate of the system state at time k+1A corresponding error covariance matrix.
Further, the positioning method of the indoor parking lot based on the visual SLAM is specifically used for:
s601, constructing a dictionary containing the fusion of the characteristic points and the ID information, namely, a set of words, wherein the set of words is expressed as follows:
(9)
S602, determining words in a frame of image to form a bag-of-words vector, wherein the bag-of-words vector is represented by the following formula:
(10)
wherein 1 indicates that the word is present, 0 indicates that none, Respectively representing the first feature point and the first ID information,Respectively representing an nth feature point and nth ID information;
s603, comparing the difference of the description vectors F of the two frames of images, wherein the similarity is larger than a preset threshold value The two images of the car are used as a detected loop, and the map is updated, so that the indoor parking lot map building and the vehicle positioning are realized.
Example III
Another aspect of the present invention also provides a readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the method described in the first embodiment above.
Example IV
In another aspect, the present invention also provides an electronic device, where the electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the program to implement the steps of the method described in the first embodiment.
The technical features of the above embodiments may be arbitrarily combined, and for brevity, all of the possible combinations of the technical features of the above embodiments are not described, however, they should be considered as the scope of the description of the present specification as long as there is no contradiction between the combinations of the technical features.
Those of skill in the art will appreciate that the logic and/or steps represented in the flow diagrams or otherwise described herein, e.g., a ordered listing of executable instructions for implementing logical functions, can be embodied in any computer-readable storage medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable storage medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.
More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable storage medium may even be paper or other suitable medium upon which the program is printed, as the program may be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted, or otherwise processed in a suitable manner, if necessary, and then stored in a computer memory.
It is to be understood that portions of the present invention may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, the various steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, may be implemented using any one or combination of the following techniques, as is well known in the art: discrete logic circuits having logic gates for implementing logic functions on data signals, application specific integrated circuits having suitable combinational logic gates, programmable Gate Arrays (PGAs), field Programmable Gate Arrays (FPGAs), and the like.
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
The foregoing examples illustrate only a few embodiments of the invention and are described in detail herein without thereby limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.

Claims (7)

1. An indoor parking lot positioning method based on visual SLAM is characterized in that: the method comprises the following steps:
Step 1: acquiring Apriltag image data by adopting a monocular camera, and acquiring IMU data information of the vehicle by adopting an inertial measurement unit; the image data includes a plurality Apriltag of indoor parking lot images; the IMU data comprises vehicle attitude, angular velocity and linear acceleration information;
step 2: the monocular camera and the inertial measurement unit are calibrated to a vehicle coordinate system in a combined mode;
step 3: preprocessing a parking lot image acquired by the monocular camera, extracting characteristic points in the parking lot image by adopting a SIFT algorithm, and acquiring Apriltag positions and IDs thereof in the parking lot image;
step 4: preprocessing the IMU data, and calculating pose information of the vehicle;
Step 5: for the obtained feature points and ID information, combining with vehicle pose information, adopting extended Kalman filtering to perform map optimization;
Step 6: loop detection is carried out by adopting a word bag model, and a map is updated, so that the indoor parking lot map building and the vehicle positioning are realized;
Step 3, preprocessing the parking lot image obtained by the monocular camera, extracting feature points in the parking lot image by using a SIFT algorithm, and obtaining Apriltag positions and IDs thereof in the parking lot image comprises:
S301, preprocessing comprises Gaussian smoothing and DoG feature map calculation;
S302, extracting feature points in the parking lot image by adopting a SIFT algorithm: identifying potential interest points with scales and unchanged rotation of the parking lot image through a Gaussian differential function, positioning key points according to the interest points, determining characteristic directions, determining main directions of the characteristic points by utilizing gradient histograms of neighborhood of the characteristic points, generating description operators in the neighborhood of the characteristic points, generating SIFT feature vectors of 4x4x8 dimensions, and carrying out pairwise comparison through the SIFT feature vectors of the key points to find out a plurality of pairs of mutually matched characteristic points;
S303, acquiring Apriltag positions and IDs thereof in the parking lot image: performing graying and binarization on the parking lot image, performing edge detection by adopting a Sobel operator to obtain a binarized edge image, performing polygon analysis on the binarized edge image until all binary points are traversed, calculating a convex hull of each polygon by utilizing a polygon convex hull searching algorithm, solving the areas of the convex hulls and the polygons thereof, comparing the areas of the convex hulls and the polygons thereof, removing the polygons when the areas of the polygons are larger than the areas of the convex hulls, reserving other polygons meeting the conditions, performing quadrilateral approximation on the other polygons by using a Taglas-Prak algorithm, and finally decoding the detected quadrilateral internal patterns to obtain the digital ID corresponding to each pattern;
And 5, combining the acquired characteristic points and ID information with vehicle pose information, and carrying out map optimization by adopting extended Kalman filtering to obtain the following steps:
S501, predicting joint state estimation of pose and feature points at the current moment according to feature point information, ID information and vehicle pose information in the parking lot image at the previous moment, wherein the joint state of the pose and the feature points at the current moment represents a mean value, and the variance is solved by a motion change matrix, and the method comprises the following specific steps of:
constructing a basic equation: set discrete time The position of the vehicle isThe motion equation of the vehicle from the last moment to the next moment is:；
setting the information of the characteristic points as The inertial measurement unit is in positionWhere it detectsThe observation equation is:；
wherein, The position at time k is indicated,For the motion of the vehicle estimated by the feature points,Information and ID information representing the feature points obtained by the inertial measurement unit, and a covariance matrix of states and errors at the k+1 time is estimated and predicted according to the optimal state at the k time:
(4)
(5)
equation (4) is a state test equation in which, Is the optimal estimate of the system state at time k,Is based on the state optimal estimation at k timeSystem input of sum timePredicting the system state at the moment k+1;
Equation (5) is the covariance matrix prediction equation of the error, wherein, Is the optimal estimation of the system state at time kA corresponding error covariance matrix is provided,Is a prediction of system stateA corresponding error covariance matrix is provided,Is an equation ofIs a jacobian matrix of the matrix,Is the covariance matrix of the system process noise at the moment k;
S502, optimizing a prediction result of the system state at the time k+1 obtained in the last step according to the actual characteristic points and ID information of the system at the time k+1 to obtain an optimal estimation of the system state at the time k+1, wherein the method specifically comprises the following steps:
(6)
wherein, Is the optimal estimate of the system state at time k +1,Is a Kalman gain matrix, and the calculation method is as follows:
(7)
wherein, Is an equation ofIs a jacobian matrix of the matrix,Is the covariance matrix of the measured noise at time k+1;
S503, continuously updating an error covariance matrix at the moment k+1, wherein the formula is as follows:
(8)
Wherein I is an identity matrix, Is the optimal estimate of the system state at time k+1A corresponding error covariance matrix;
and 6, carrying out loop detection by adopting a bag-of-word model, wherein the specific steps are as follows:
s601, constructing a dictionary containing the fusion of the characteristic points and the ID information, namely, a set of words, wherein the set of words is expressed as follows:
(9)
S602, determining words in a frame of image to form a bag-of-words vector, wherein the bag-of-words vector is represented by the following formula:
(10)
wherein 1 indicates that the word is present, 0 indicates that none, Respectively representing the first feature point and the first ID information,Respectively representing an nth feature point and nth ID information;
s603, comparing the difference of the description vectors F of the two frames of images, wherein the similarity is larger than a preset threshold value The two images of the car are used as a detected loop, and the map is updated, so that the indoor parking lot map building and the vehicle positioning are realized.
2. The visual SLAM-based indoor parking lot positioning method of claim 1, wherein the method comprises the steps of: the monocular camera is a 180-degree wide-angle camera, and the inertial measurement unit is a vehicle-mounted inertial measurement unit.
3. The visual SLAM-based indoor parking lot positioning method of claim 1, wherein the method comprises the steps of: the calibrating the monocular camera and the inertial measurement unit to the vehicle coordinate system in step2 includes:
s201, an inertial measurement unit coordinate system and a vehicle coordinate system are set, wherein the front of the vehicle is taken as a Y axis, the left is taken as an X axis, the upward is taken as a Z axis, and the origin is respectively the center of the inertial measurement unit and the center of the vehicle;
S202, calibrating the monocular camera, obtaining internal and external parameters of the monocular camera according to a Zhang Zhengyou calibration method, converting pixel coordinate data into a camera coordinate system according to a formula (1), then manually measuring the coordinates of the monocular camera in the vehicle coordinate system O V-XVYVZV and the positions of the monocular camera in the camera coordinate system by adopting a plurality of calibration objects, and obtaining a rotation matrix R C and a translation vector T C according to a formula (2);
(1)
(2)
Wherein: xc, yc, zc are coordinate values of the calibration object in a camera coordinate system, xv, yv, zv are coordinate values of the calibration object in a vehicle coordinate system, R C,TC is a rotation matrix and a translation matrix of data converted from the camera coordinate system to the vehicle coordinate system, f is a camera focal length, and 1/dx and 1/dy represent: the number of pixels contained in each unit length in the x direction and the y direction can be decimal; gamma is a distortion factor, taking 0; u 0、v0 represents the number of horizontal and vertical pixels of the phase difference between the center pixel coordinate of the image and the original pixel coordinate of the image, and u and v are coordinate values of the calibration object on the image;
S203, calibrating the inertial measurement unit to a vehicle coordinate system, selecting a plurality of points, measuring the coordinates of the points in the inertial measurement unit coordinate system O I-XIYIZI and the coordinates in the vehicle coordinate system O V-XVYVZV, obtaining a rotation matrix R I and a translation vector T I according to a formula (3), and calibrating the rotation matrix R I and the translation vector T I to the vehicle coordinate system;
(3)
Wherein: r I,TI is a rotation matrix and a translation matrix of data converted from the inertial measurement unit coordinate system to a vehicle coordinate system, and X I、YI、ZI is a coordinate value of a calibration object under the inertial measurement unit coordinate system.
4. An indoor parking area positioning system based on vision SLAM, its characterized in that: an indoor parking lot positioning method based on visual SLAM for implementing any one of claims 1 to 3, the system comprising:
The acquisition module is used for: the method comprises the steps of acquiring Apriltag image data by using a monocular camera, and acquiring IMU data information of a vehicle by using an inertial measurement unit; the image data includes a plurality Apriltag of indoor parking lot images; the IMU data comprises vehicle attitude, angular velocity and linear acceleration information;
And (3) a calibration module: the method comprises the steps of performing joint calibration on the monocular camera and an inertial measurement unit to a vehicle coordinate system;
and a pretreatment module: the method comprises the steps of preprocessing a parking lot image obtained by the monocular camera, extracting characteristic points in the parking lot image by adopting a SIFT algorithm, and obtaining Apriltag positions and IDs thereof in the parking lot image;
And a resolving module: the IMU data is used for preprocessing the IMU data, and pose information of the vehicle is calculated;
and an optimization module: the map optimization method is used for optimizing the obtained characteristic points and ID information by combining the vehicle pose information and adopting extended Kalman filtering;
and a positioning module: the method is used for carrying out loop detection by adopting the bag-of-word model, updating the map and realizing the construction of the indoor parking lot and the positioning of the vehicle.
5. The visual SLAM-based indoor parking lot positioning system of claim 4, wherein: the monocular camera is a 180-degree wide-angle camera, and the inertial measurement unit is a vehicle-mounted inertial measurement unit.
6. A readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the steps of the method according to any one of claims 1 to 3.
7. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method according to any one of claims 1 to 3 when the program is executed.
CN202410685784.XA 2024-05-30 2024-05-30 Indoor parking lot positioning method and system based on visual SLAM Active CN118262336B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410685784.XA CN118262336B (en) 2024-05-30 2024-05-30 Indoor parking lot positioning method and system based on visual SLAM

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410685784.XA CN118262336B (en) 2024-05-30 2024-05-30 Indoor parking lot positioning method and system based on visual SLAM

Publications (2)

Publication Number Publication Date
CN118262336A CN118262336A (en) 2024-06-28
CN118262336B true CN118262336B (en) 2024-09-03

Family

ID=91613567

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410685784.XA Active CN118262336B (en) 2024-05-30 2024-05-30 Indoor parking lot positioning method and system based on visual SLAM

Country Status (1)

Country Link
CN (1) CN118262336B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118800097A (en) * 2024-09-13 2024-10-18 深圳点点电工网络科技有限公司 A smart station monitoring method and monitoring device and system
CN118915017B (en) * 2024-09-20 2025-03-04 广东工业大学 A solid-state lidar loop closure detection method based on point cloud fiducial markers

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113205604A (en) * 2021-05-17 2021-08-03 南昌智能新能源汽车研究院 Feasible region detection method based on camera and laser radar
CN113706626A (en) * 2021-07-30 2021-11-26 西安交通大学 Positioning and mapping method based on multi-sensor fusion and two-dimensional code correction
CN114359318A (en) * 2021-12-31 2022-04-15 中国第一汽车股份有限公司 External parameter calibration method, data processing device, storage medium and electronic device

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108717712B (en) * 2018-05-29 2021-09-03 东北大学 Visual inertial navigation SLAM method based on ground plane hypothesis
CN109376785B (en) * 2018-10-31 2021-09-24 东南大学 A Navigation Method Based on Iterative Extended Kalman Filter Fusion Inertial and Monocular Vision
CN110032201A (en) * 2019-04-19 2019-07-19 成都飞机工业（集团）有限责任公司 A method of the airborne visual gesture fusion of IMU based on Kalman filtering
CN113220818B (en) * 2021-05-27 2023-04-07 南昌智能新能源汽车研究院 Automatic mapping and high-precision positioning method for parking lot
CN113706620B (en) * 2021-10-22 2022-03-22 杭州迦智科技有限公司 Positioning method, positioning device and movable platform based on reference object
CN113903011B (en) * 2021-10-26 2024-06-11 江苏大学 Semantic map construction and positioning method suitable for indoor parking lot
CN115900695A (en) * 2022-09-30 2023-04-04 中国第一汽车股份有限公司 Intelligent parking vehicle positioning method applied to vehicle
CN116481522A (en) * 2023-03-24 2023-07-25 武汉光庭信息技术股份有限公司 Indoor monocular inertial navigation SLAM method and system
CN117664124A (en) * 2023-11-21 2024-03-08 西安诚合工业自动化设备有限公司 Inertial guidance and visual information fusion AGV navigation system and method based on ROS

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113205604A (en) * 2021-05-17 2021-08-03 南昌智能新能源汽车研究院 Feasible region detection method based on camera and laser radar
CN113706626A (en) * 2021-07-30 2021-11-26 西安交通大学 Positioning and mapping method based on multi-sensor fusion and two-dimensional code correction
CN114359318A (en) * 2021-12-31 2022-04-15 中国第一汽车股份有限公司 External parameter calibration method, data processing device, storage medium and electronic device

Also Published As

Publication number Publication date
CN118262336A (en) 2024-06-28

Similar Documents

Publication Publication Date Title
US11900627B2 (en) Image annotation
CN118262336B (en) Indoor parking lot positioning method and system based on visual SLAM
CN114332221B (en) Semantic-based loop detection method, device, electronic device and storage medium
CN116385997B (en) Vehicle-mounted obstacle accurate sensing method, system and storage medium
US8885049B2 (en) Method and device for determining calibration parameters of a camera
Siegemund et al. A temporal filter approach for detection and reconstruction of curbs and road surfaces based on conditional random fields
Zhou et al. Moving object detection and segmentation in urban environments from a moving platform
CN110176022B (en) Tunnel panoramic monitoring system and method based on video detection
Parra et al. Robust visual odometry for vehicle localization in urban environments
KR101030317B1 (en) Apparatus and method for tracking obstacles using stereo vision
Park et al. Nonparametric background model-based LiDAR SLAM in highly dynamic urban environments
CN118397588B (en) Camera scene analysis method, system, equipment and medium for intelligent driving automobile
US12110009B2 (en) Parking space detection method and system
CN109115232B (en) Navigation method and device
Birk et al. Simultaneous localization and mapping (SLAM)
Hwang et al. Vision-based vehicle detection and tracking algorithm design
Huang et al. Probabilistic lane estimation for autonomous driving using basis curves
Shafique et al. Computer vision based autonomous navigation in controlled environment
García-García et al. 3D visual odometry for GPS navigation assistance
Dumortier et al. 4-d tensor voting motion segmentation for obstacle detection in autonomous guided vehicle
Huang et al. Probabilistic lane estimation using basis curves
Hwang et al. Vehicle detection system design based on stereo vision sensors
JPH06243251A (en) Image feature extracting device
US20240193964A1 (en) Lane line recognition method, electronic device and storage medium
US20240005557A1 (en) Methods and apparatus for calibrating stereo imagery using motion of vehicle

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant