CN119437202B

CN119437202B - An automatic indoor map construction method integrating multi-source trajectories and mobile phone images

Info

Publication number: CN119437202B
Application number: CN202411570254.7A
Authority: CN
Inventors: 马威; 潘建平; 陈茂霖; 舒震; 王艺龙; 朱超
Original assignee: Chongqing Jiaotong University
Current assignee: Chongqing Jiaotong University
Priority date: 2024-11-05
Filing date: 2024-11-05
Publication date: 2025-06-20
Anticipated expiration: 2044-11-05
Also published as: CN119437202A

Abstract

The invention discloses an automatic indoor map construction method integrating a crowd-sourced track and a mobile phone image, which relates to the technical field of indoor map construction and comprises the following steps that S1, a smart phone is utilized to collect corridor, indoor image and multi-sensor data; S2, constructing an indoor navigation map, S3, extracting an indoor plane structure, and S4, fusing and optimizing a navigation path and the indoor plane structure. According to the indoor map automatic construction method integrating the crowd-sourced track and the mobile phone image, the indoor image acquisition method is guided in a standardized manner in the crowd-sourced track acquisition process, and the plane structure characteristics in the image are extracted, so that the space information of the map is further enriched. The advantages of the whole and partial parts and the frame and the details are complementary, and on one hand, the inefficiency of the crowd source data acquisition is effectively relieved. On the other hand, the low-cost data acquisition mode is continued, the quality of data is ensured, and the drawing precision is improved, so that the availability of the position service is greatly improved.

Description

Indoor map automatic construction method integrating crowd-sourced tracks and mobile phone images

Technical Field

The invention relates to the technical field of indoor map construction, in particular to an automatic indoor map construction method integrating a crowd source track and a mobile phone image.

Background

With the high-speed promotion of urban construction, indoor space is continuously expanded, research about indoor space information application is increasingly focused by academia and industry, and the market scale of indoor application with location service as a core is rapidly growing. As a data base of indoor Location application, due to the influence of complex signal environment, space layout, topology variability and the like, an indoor two-dimensional map has the problems of data starvation, drawing precision, low efficiency and the like, and becomes a main factor for limiting the development of indoor Location service (Location-basedServices, LBS). The traditional indoor two-dimensional planar map construction can be mainly divided into two modes based on manual drawing and professional sensors according to a data acquisition mode. The manual drawing method is based on the construction of measuring tape, steel rule, hand-held laser range finder, total station and other basic measuring instrument and drawing software such as AutoCAD. While map accuracy can be effectively ensured, manual modeling requires a lot of time and effort. The method based on the professional sensor is to collect indoor space information by using a laser scanner or a camera and the like so as to generate an indoor two-dimensional map. However, professional drawing equipment has high requirements on acquisition personnel, and has high hardware cost, so that the drawing equipment is difficult to popularize and popularize in application.

At present, the hardware of the smart phone is extremely fast in changing speed, and various sensors (an accelerometer, a gyroscope, a magnetometer, a barometer, a GPS (global positioning system) and the like) are built in the smart phone, so that the smart phone not only has stronger computing capacity, but also can acquire multi-source data (indoor moving tracks, behavior data, images and the like), and is an ideal platform for indoor drawing. Various indoor drawing methods are explored by students at home and abroad aiming at different data sources. Wherein, based on SLAM

(Simu ltaneousLoca l i zat ionandMappi ng, SLAM) smart phone indoor mapping methods gradually mature, and students have also explored a lot and have promoted the development of SLAM technology in mapping accuracy, scale, reliability, etc. However, such methods require the user to turn on the camera at any time to aim at the surrounding environment, which does not conform to the use habit of the user. In addition, the algorithm is seriously dependent on image feature points in a scene, the situation of mismatching exists widely, and the requirements on the memory and the computing capacity of the mobile phone in use are high. In addition, the intelligent mobile phone indoor drawing method based on the crowd source data effectively reduces the cost and the technical threshold of data acquisition, but lacks of effective management of time, efficiency and quality of data acquisition. Due to the non-professionality of data collection, it is difficult to guarantee the accuracy of map construction, and a great deal of effort is required for data screening and effective information extraction.

Disclosure of Invention

The invention aims to provide an indoor map automatic construction method integrating a crowd source track and a mobile phone image, which can realize accurate construction of an indoor map by utilizing crowd source track data acquired by a smart phone platform and acquiring the indoor image according to a formulated photographing rule and provides a composition method with low cost, high precision and high efficiency for indoor plane map construction.

In order to achieve the above purpose, the invention provides an indoor map automatic construction method integrating a crowd source track and a mobile phone image, which comprises the following steps:

s1, acquiring corridor, indoor images and multi-sensor data by using a smart phone;

s2, constructing an indoor navigation map;

S3, extracting an indoor plane structure;

and S4, fusing and optimizing the navigation path and the indoor plane structure.

Preferably, S2 comprises the steps of:

S21, obtaining crowd source track data through pedestrian dead reckoning of multi-sensor data acquired by a smart phone platform;

S22, obtaining a behavior landmark through a behavior recognition algorithm, and clustering the behavior landmark through a clustering algorithm based on Wi-F i fingerprints;

s23, calculating the relative distance between the behavioral landmarks to obtain an indoor navigation map.

Preferably, S3 comprises the steps of:

S31, inputting an indoor space monocular image acquired by the smart phone, and extracting a three-dimensional layout structure of the monocular image through an indoor three-dimensional scene understanding network model;

S32, mapping the indoor 3D scene into a 2D image, and performing projection transformation;

s33, recovering the space structure proportion and obtaining the indoor plane structure.

Preferably, S32 comprises the steps of:

S321, extracting and solving camera parameters based on vanishing points by utilizing VPBC technology, wherein a main point of a camera is positioned in the center of an image, and a camera internal reference matrix K is in the form of:

f is the focal length, which can be found from vanishing points;

S322, setting the coordinate of vanishing points in an image coordinate system as vp₀(x_i0,y_i0),vp₁(x_i1,y_i1),vp₂(x_i2,y_i2),, setting the center of gravity of a triangle formed by three vanishing points as an intersection point p (ox, oy) of a camera optical axis and an imaging plane, setting the coordinate of vp ₀ in the camera coordinate system as (x _c0,y_c0, f), wherein x _c0＝x_i0-ox,y_c0＝-(y_i0 -oy), obtaining the other two vanishing points by the same principle, and obtaining a focal length and a rotation matrix by using an orthogonal relation between the vanishing points, wherein the focal length and the rotation matrix are respectively as follows:

s323, calculating a projective transformation matrix H _ij by using camera parameters, and carrying out 'orthographic projection' correction on a plane determined by vanishing points vp _i and vp _j, wherein the solution of H _ij is as follows:

H_ij＝K·R_ij·K^-1;

Reprojecting 4 intersection points by using transformation matrices H ₀₁ and H ₀₂ of the front wall and the floor to obtain coordinates of each point in the front view after eliminating distortion

Preferably, the real proportional relation of the length, width and height of the indoor space is as follows:

preferably, S4 comprises the steps of:

S41, constructing a corridor, namely performing position matching on the corrected track route and a corridor plane structure extracted from the mobile phone image, and realizing map construction of a corridor area;

S42, matching the room plane structure extracted from the image with inertial navigation data to draw the room structure in the indoor map.

Preferably, the map accuracy is further optimized and guaranteed by constructing an energy function, and an energy equation Cheng Ruxia is to be constructed:

min_x＝{x_f,R,t}E_l(x)+E_c(x)+E_b(x);

Where E _l (x) represents the complexity of the layout, E _c (x) represents the closeness, E _b (x) represents the boundary similarity between adjacent segments, { x _f, R, t } represents the rotation and displacement of all corridor paths and room layout at the time of connection.

Preferably, the conditions limiting the uniqueness of the hallway and room layout occurrences are:

preferably, the image acquisition process follows the rule that the indoor corridor shooting needs to shoot all the view fields with the inflection points in the clockwise sequence in the turning process, and the indoor room shooting needs to shoot from two sides of the room to the opposite sides.

Therefore, the indoor map automatic construction method integrating the crowd source track and the mobile phone image has the following beneficial effects:

(1) The method for extracting the planar structure of the indoor map element has the advantages that the image shot by the mobile phone is used as a data source, the geometric structure of the indoor three-dimensional model is not required to be reconstructed, professional measuring equipment is also not required, the intelligent mobile phone is used for shooting the monocular image of the indoor scene, the planar structure of the indoor map element is obtained through a visual scene understanding technology, and the reality and the accuracy of the indoor map expression are effectively improved.

(2) And the high-precision construction of the indoor map is realized by integrating the crowd-sourced track with the standardized mobile phone image acquisition only through different types of data collected by the smart mobile phone and utilizing a new geometric feature extraction method. The indoor space characteristics of the whole and partial, frame and detail are fully developed and complemented, a user is not required to walk through all positions in the indoor environment, the labor cost of indoor map drawing is effectively saved, and the time and the calculated amount are reduced.

The technical scheme of the invention is further described in detail through the drawings and the embodiments.

Drawings

FIG. 1 is a flow chart of the present invention;

FIG. 2 is a schematic diagram of a relative distance estimation according to the present invention;

FIG. 3 is a graph showing the results of step detection and angle measurement between behavioral landmarks of the present invention;

FIG. 4 shows the image acquisition rule of the corridor, wherein a is a right angle turn, b is a four-fork corner, and c is a T-shaped corner;

fig. 5 is a rule of image acquisition of a room according to the present invention.

Detailed Description

The technical scheme of the invention is further described below through the attached drawings and the embodiments.

Unless defined otherwise, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this invention belongs. The terms "first," "second," and the like, as used herein, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that elements or items preceding the word are included in the element or item listed after the word and equivalents thereof, but does not exclude other elements or items. The terms "connected" or "connected," and the like, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", etc. are used merely to indicate relative positional relationships, which may also be changed when the absolute position of the object to be described is changed.

Examples

Referring to fig. 1-5, the invention provides an indoor map automatic construction method integrating a crowd source track and a mobile phone image, which can realize accurate construction of an indoor map by using three modules of indoor navigation map construction, indoor plane structure extraction and indoor map construction integrating a navigation path and a plane structure only by utilizing crowd source track data acquired by a smart mobile phone platform and acquiring an indoor image according to a formulated photographing rule, and provides a composition method with low cost, high precision and high efficiency for indoor plane map construction.

An automatic indoor map construction method integrating a crowd-sourced track and a mobile phone image comprises the following steps:

S1, collecting corridor, indoor images and multi-sensor data by using a smart phone.

S2, constructing an indoor navigation map.

S21, multi-sensor data acquired by the smart phone platform are subjected to dead reckoning to obtain crowd source track data. Many behavioral landmarks exist in the crowd-sourced data, wherein a part of behavioral landmarks are collected at the same node, and in order to construct an indoor map, the behavioral landmarks collected by the same node need to be clustered at first.

S22, obtaining a behavior landmark through a behavior recognition algorithm, and clustering the behavior landmark through a clustering algorithm based on Wi-Fi fingerprints.

When a new track is obtained, a behavior landmark is obtained by using a behavior recognition algorithm, and meanwhile, wi-Fi fingerprints at the moment of behavior occurrence are extracted to be used as the characteristics of the behavior landmark. A behavior landmark sequence NAL ₁,NAL₂...NAL_m is obtained according to the time sequence. Wherein m behavioural landmarks are included (NAL represents the behavioural landmark extracted in the new upload track). M behavioral landmarks are clustered using a Wi-Fi fingerprint-based clustering algorithm, where there may be n behavioral landmarks with the same Wi-F i fingerprint characteristics, where (0 n m).

And if the track is the first track, sequentially adding the behavior landmarks contained in the track into the node database. If the track is not the first track data, the image dataset of the existing garbage can be collected in a large quantity in the track through a clustering algorithm based on Wi-F i characteristics, and the image dataset is used for constructing a training dataset.

After the behavioral landmarks are clustered, all the behavioral landmarks are clustered into different classes, each class represents one node of the indoor map, and the two nodes are directly adjacent, so that the distance between the two nodes can be obtained through pedestrian dead reckoning. As shown in fig. 4 (a), the track acquired by one smart phone (obtained by dead reckoning of the pedestrian) is shown, A, B, C and D are four behavioral landmarks (turns). Where AB, BC and CD are adjacent behavioral landmarks (known by the time sequence of landmark detection), the relative distances between them can be directly obtained by step detection and step estimation. For non-adjacent landmarks, such as a and C, it is necessary to obtain the length of AB and BC and the angular information of angle B to calculate the relative distance between the landmarks. The distance and angle information between the behavioral landmarks in fig. 4 (a) can be obtained through inertial data acquired by a smart phone, the distance information can be acquired through step detection by accelerometer data, and the angle information is acquired by a gyroscope and an electronic compass, and fig. 3 shows the step detection result and angle change information between the behavioral landmarks.

Based on the behavioral landmark clustering and the relative distance calculation, the points for constructing the indoor navigation map and the relative distances among all the points are obtained to form a relative distance matrix. And calculating the relative spatial relationship by using the relative distance between the points as a non-similarity measurement parameter by using a multidimensional scale technology, and constructing the indoor navigation map.

S3, extracting the indoor plane structure.

S31, inputting an indoor space monocular image acquired by the smart phone, and extracting a three-dimensional layout structure of the monocular image through an indoor three-dimensional scene understanding network model.

The initial frame is trained through convolutional neural network (Convo l ut i ona l Neura l Networks, CNN) learning, and is used as a feature vector to be input into a structured support vector machine, so that the extraction accuracy of indoor three-dimensional space features is improved. The indoor layout estimation based on CNN is to directly train the corner points (key points) and the room types of the room, estimate the ordered set of the key points of the indoor layout, and connect them according to a specific order, thereby obtaining the indoor layout frame.

S32, mapping the indoor 3D scene into a 2D image, and performing projection transformation.

Because the three-dimensional space layout length, width and height structural information extracted from the mobile phone image is represented by pixel distances, the image can be distorted to different degrees in the process of mapping the indoor 3D scene to the 2D image, so that the proportional distortion based on the space structure is caused. Therefore, the real proportional relation among the length, the width and the height in the indoor space needs to be restored by eliminating the influence of image imaging distortion.

Extracting and solving camera parameters based on vanishing points by VPBC technology, wherein a main point of a camera is positioned in the center of an image, and a camera internal reference matrix K is in the form of:

f is focal length, which can be obtained by vanishing points, wherein the vanishing points are set to be vp₀(x_i0,y_i0),vp₁(x_i1,y_i1),vp₂(x_i2,y_i2), in an image coordinate system, the center of gravity of a triangle formed by three vanishing points is set to be an intersection point p (ox, oy) of a camera optical axis and an imaging plane, vp ₀ is set to be (x _c0,y_c0, f) in the camera coordinate system, wherein x _c0＝x_i0-ox,y_c0＝-(y_i0 -oy), the other two vanishing points are obtained in the same way, and the focal length and the rotation matrix can be obtained by utilizing the orthogonal relation among the vanishing points respectively:

Using the camera parameters to calculate the projective transformation matrix H _ij, the plane determined by vanishing points vp _i and vp _j can be "orthoprojected" corrected, and the solution for H _ij is as follows:

H_ij＝K·R_ij·K^-1;

Based on after re-projectionThe coordinates calculate the relative length of the length, width and height respectively, and finally the real proportional relation of the length, width and height of the indoor space is solved:

And setting the interlayer spacing of the indoor rooms as a mean value H, and calculating a ratio value by taking the building height H as a reference for the length, width and height extracted from the image, so that the scale consistency of the map is ensured.

S41, constructing a corridor, namely firstly, in the image acquisition process, orderly photographing all the view fields with inflection points according to a clockwise sequence in the turning process, wherein the following rule is required. As shown in fig. 4, when the user passes through one corner, a picture is taken on the corridor in one direction and then rotated to take a picture in the other direction.

The gyroscope and accelerometer record both angle and acceleration. Fluctuations in gyroscope readings (up or down) indicate that the user is turning left or right at this location. During both undulations, the user takes a picture of the corridor, and the location of the user at the corner can be accounted for by inertial data. And according to the image acquisition rule and the analysis of shooting behaviors, performing position matching on the corrected track route and the corridor plane structure extracted from the mobile phone image, and realizing map construction of the corridor area.

S42, building rooms, wherein rules are shown in FIG. 5 when room images are acquired, and users shoot the rooms from two sides to the opposite sides. And then matching the room plane structure extracted from the image with inertial navigation data to draw the room structure in the indoor map.

After the track data is matched with the position of the shot image, the map precision is further ensured by constructing an energy function in a further optimization mode, and the energy equation to be constructed is as follows

min_x＝{x_f,R,t}E_l(x)+E_c(x)+E_b(x);

The uniqueness of the hallway and room layout appearance is limited by the following formula.

Therefore, the invention adopts the indoor map automatic construction method integrating the crowd source track and the mobile phone image, and provides a high-efficiency, low-cost and independent indoor map element plane structure extraction method, pictures shot by the mobile phone are taken as data sources, the geometric structure of an indoor three-dimensional model is not required to be reconstructed, professional measuring equipment is also not required, only the intelligent mobile phone is used for shooting indoor scene monocular pictures, the acquisition of the indoor map element plane structure is realized through a visual scene understanding technology, and the authenticity and accuracy of indoor map expression are effectively improved. And the high-precision construction of the indoor map is realized by integrating the crowd-sourced track with the standardized mobile phone image acquisition only through different types of data collected by the smart mobile phone and utilizing a new geometric feature extraction method. The indoor space characteristics of the whole and partial, frame and detail are fully developed and complemented, a user is not required to walk through all positions in the indoor environment, the labor cost of indoor map drawing is effectively saved, and the time and the calculated amount are reduced.

It should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention may be modified or substituted by the same, and the modified or substituted technical solution may not deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for automatically constructing an indoor map by integrating multi-source trajectories and mobile phone images, characterized in that it includes the following steps:

S1, using smartphones to collect corridor, indoor images and multi-sensor data;

S2, build indoor navigation map;

S3, extracting indoor plane structure;

S4, integrating and optimizing the navigation path and indoor plane structure;

S2 includes the following steps:

S21, obtaining multi-source trajectory data by using pedestrian dead reckoning to collect multi-sensor data from the smartphone;

S22, obtaining behavior landmarks through a behavior recognition algorithm, and clustering the behavior landmarks through a clustering algorithm based on Wi-Fi fingerprints;

S23, calculating the relative distances between the behavioral landmarks to obtain an indoor navigation map;

S3 includes the following steps:

S31, inputting a monocular image of an indoor space captured by a smartphone, and extracting a three-dimensional layout structure of the monocular image through an indoor three-dimensional scene understanding network model;

S32, mapping the indoor 3D scene into the 2D image and performing projection transformation;

S33, restore the spatial structure ratio and obtain the indoor plane structure;

S4 includes the following steps:

S41, corridor construction, matching the corrected trajectory route with the corridor plane structure extracted from the mobile phone image to realize map construction of the corridor area;

S42, matching the room plane structure extracted from the image with the inertial navigation data to achieve drawing of the room structure in the indoor map;

S32 includes the following steps:

S321, using VPBC technology to extract and solve camera parameters based on vanishing point. The camera principal point is located at the center of the image. The camera intrinsic parameter matrix K is in the form of:

f is the focal length, which can be found from the vanishing point;

S322, let the coordinates of the vanishing points in the image coordinate system be vp ₀ ( _xi0 , _yi0 ), vp ₁ ( _xi1 , _yi1 ), vp ₂ ( _xi2 , _yi2 ). Since the centroid of the triangle formed by the three vanishing points is the intersection point p(ox, oy) of the camera optical axis and the imaging plane, the coordinates of vp ₀ in the camera coordinate system are ( _xc0 , _yc0 , f), where _xc0 = _xi0- ox, _yc0 =-( _yi0 -oy). The coordinates of the other two vanishing points can be obtained in the same way. Using the orthogonal relationship between the vanishing points, the focal length and rotation matrix can be obtained respectively:

S323, using the camera parameters to calculate the projection transformation matrix _Hij , the plane determined by the vanishing points _vpi and _vpj can be corrected by "orthographic projection". The solution of _Hij is as follows:

_Hij = K· _Rij ·K ^-1 ;

Use the transformation matrices H ₀₁ and H ₀₂ of the front wall and floor to reproject the four intersection points and obtain the coordinates of each point in the front view after eliminating distortion.

2. According to claim 1, a method for automatically constructing an indoor map by integrating multi-source trajectories and mobile phone images is characterized in that the real proportional relationship between the length, width and height of the indoor space is:

3. According to claim 2, a method for automatically constructing an indoor map by integrating multi-source trajectories and mobile phone images is characterized in that the map accuracy is further optimized by constructing an energy function, and the energy equation to be constructed is as follows:

min _x ={x _f ,R,t}E _l (x)+E _c (x)+E _b (x);

Where _El (x) represents the complexity of the layout, _Ec (x) represents the closure, _Eb (x) represents the boundary similarity between adjacent segments, and { _xf , R, t} represents the rotation and displacement of all corridor paths and room layouts when they are connected.

4. The method for automatically constructing an indoor map by integrating multi-source trajectories and mobile phone images according to claim 3, wherein the condition for limiting the uniqueness of the layout of corridors and rooms is:

5. According to the method of automatically constructing an indoor map by integrating multi-source trajectories and mobile phone images as described in claim 4, it is characterized in that the following rules are followed during the image acquisition process: when shooting indoor corridors, it is necessary to take pictures of the fields of view facing all turning points in a clockwise order during the turning process, and when shooting indoor rooms, it is necessary to shoot from both sides of the room to the opposite side.