CN110631581B - Method and drone for building indoor 3D maps - Google Patents

Abstract

The application provides a method for building an indoor 3D map and an unmanned aerial vehicle. The method comprises the following steps: determining at least one sub-area to be scanned of the target room; determining an optimal scanning point of each sub-area in at least one sub-area to be scanned; determining a target scanning path according to the current scanning point and the optimal scanning point of each sub-area in at least one sub-area to be scanned; scanning at least one sub-area to be scanned of the target room along a target scanning path; when all sub-areas of the target room complete scanning, a 3D map of the target room is established according to the 3D point cloud of the target room obtained through scanning. According to the method and the device, the scanning path can be planned in real time according to the area to be scanned during scanning, and the complete indoor scanning can be achieved, so that a more accurate 3D map is built.

Description

Method and drone for building indoor 3D maps

technical field

The present application relates to the technical field of unmanned aerial vehicles, and more specifically, relates to a method for building an indoor 3D map and an unmanned aerial vehicle.

Background technique

With the rapid development of drone technology, indoor drones have been applied to a certain extent. Indoor drones can move freely in indoor spaces, and can complete various tasks that ground robots cannot. Before the indoor drone works indoors, it needs to scan the room completely to generate an indoor 3D map (the 3D map can represent the indoor spatial structure), and then work based on the 3D map.

The traditional solution generally scans the room according to preset rules, and builds a 3D map based on the scanned 3D point cloud. For example, control the UAV to fly along obstacles, and scan the room during the flight, and then build a 3D map based on the scanned 3D point cloud when the scan is completed.

In the traditional solution, when the indoor is scanned directly according to the preset rules, the scanning may be incomplete, resulting in the inaccuracy of the final 3D map.

Contents of the invention

The present application provides a method for building an indoor 3D map and an unmanned aerial vehicle, so as to realize a complete indoor scan, and then construct a more accurate 3D map.

In the first aspect, a method for establishing an indoor 3D map is provided, the method is used to scan each sub-area of a target room, and the method specifically includes: determining at least one sub-area to be scanned in the target room; determining at least one sub-area to be scanned The optimal scanning point of each sub-area in the area; according to the current scanning point and the optimal scanning point of each sub-area in at least one sub-area to be scanned, determine the target scanning path; starting from the current scanning point, according to the target scanning path, Scanning at least one sub-area of the target room to be scanned; when all sub-areas of the target room are scanned, a 3D map of the target room is established according to the 3D point cloud of the target room obtained through scanning.

Wherein, when each sub-region is scanned at the optimal scanning point of each sub-region, the scanning degree of each sub-region reaches the first preset scanning degree.

The above method can be executed by a drone, and the above target room can be a room that requires 3D mapping.

Optionally, the starting point of the above-mentioned target scanning path is the current scanning point, and the end point of the above-mentioned target scanning path is located in any one of the at least one sub-area to be scanned.

Optionally, any two sub-areas in the at least one area to be scanned are not connected to each other.

It should be understood that the 3D point cloud of the target room obtained during the scanning process specifically includes the scanned 3D coordinate information in the target room. The 3D map of the target room may refer to a grid map containing the occupancy of objects in the target room.

In this application, only unscanned areas are paid attention to during path planning, so that the planned scanning path is more targeted, can improve the overall scanning efficiency when scanning a room, and can finally build a 3D map effect.

With reference to the first aspect, in some implementations of the first aspect, the above method further includes: determining the subregion with the largest area in the at least one subregion to be scanned as the target subregion; according to the current scanning point and the at least one subregion to be scanned The optimal scanning point of each sub-area in the area determines the target scanning path, including: determining the target scanning path according to the current scanning point and the optimal scanning point of the target sub-area, wherein the starting point of the target scanning path is the current scanning point, and the end point is the best scan point for the target sub-area.

During the scanning process, by preferentially scanning the sub-area with a larger area, a larger area can be scanned as much as possible during the scanning process, and the scanning efficiency can be improved to a certain extent.

With reference to the first aspect, in some implementations of the first aspect, determining the target scanning path according to the current scanning point and the optimal scanning point of the target sub-area includes: determining that the starting point is the current scanning point and the end point is the optimal scanning point of the target sub-area. multiple scan paths for optimal scan points; and determine the scan path with the shortest path among the multiple scan paths as the target scan path.

With reference to the first aspect, in some implementations of the first aspect, determining the target scanning path according to the current scanning point and the optimal scanning point of the target sub-area includes: setting the starting point as the current scanning point and the end point as the optimal scanning point of the target sub-area The scan path with the shortest path among the multiple scan paths with the optimal scan point is determined as the target scan path.

By using the shortest scanning path as the target scanning path, the moving distance can be reduced during the scanning process, and the scanning time can be reduced to a certain extent, and the scanning efficiency can be improved.

Optionally, the shortest scanning path among the above multiple scanning paths may be a straight line path from the current scanning point to the optimal scanning point of the target sub-area.

It should be understood that if there is an obstacle between the current scan point and the optimal scan point of the target sub-area, it will be impossible to reach the target sub-area along the straight path between the current scan point and the optimal scan point of the target sub-area. The best scan point. Then, the above-mentioned target scanning path is the shortest path starting from the current scanning point, bypassing obstacles and reaching the optimal scanning point of the target sub-area.

With reference to the first aspect, in some implementations of the first aspect, determining the target scanning path according to the current scanning point and the optimal scanning point of the target sub-area includes: determining that the starting point is the current scanning point and the end point is the optimal scanning point of the target sub-area. multiple scanning paths of optimal scanning points; according to the length of each scanning path in the multiple scanning paths, and the scanning area when passing through the optimal scanning points of other sub-areas to be scanned when scanning along each scanning path, determine each The path costs of the scanning paths; the path with the smallest path cost among the multiple paths is determined as the target scanning path.

With reference to the first aspect, in some implementations of the first aspect, determining the target scanning path according to the current scanning point and the optimal scanning point of the target sub-area includes: the starting point is the current scanning point, and the end point is the optimal scanning point of the target sub-area. In the multiple scanning paths of the optimal scanning point, according to the length of each scanning path in the multiple scanning paths, and the scanning area when scanning along each scanning path through the optimal scanning point of other sub-areas to be scanned, determine The path cost of each scanning path, wherein the other sub-areas to be scanned are sub-areas other than the target sub-area in at least one sub-area to be scanned; the path with the smallest path cost among the multiple paths is determined as the target scanning path.

Wherein, the above-mentioned other sub-areas to be scanned refer to sub-areas in the above-mentioned at least one sub-area to be scanned except the target sub-area.

When determining the target scanning path, specifically, the Bellman-Ford (BF) algorithm or integer programming that supports negative weight edges, or other greedy algorithms such as genetic algorithm (genetic algorithm, GA) can be used to select a path from various paths the path of least cost.

By synthesizing the path length of each path and the area of other sub-regions to be scanned that can be scanned when scanning along the path, a more reasonable path can be selected from multiple paths as the target scanning path to improve scanning efficiency.

In combination with the first aspect, in some implementations of the first aspect, the scanning cost of each path is obtained according to the following formula:

C=α·d-β·S

Among them, C represents the path cost of the path, the value of d is proportional to the length of the path, the value of S is proportional to the area of other sub-regions to be scanned that can be scanned when passing through the optimal scanning point of other sub-regions to be scanned, α and β is the weight coefficient.

With reference to the first aspect, in some implementations of the first aspect, the starting point of the target scanning path is the current scanning point, the target scanning path passes through the optimal scanning point of each sub-area in at least one sub-area to be scanned, and the target scanning The end point of the path is an optimal scanning point of one sub-area in at least one sub-area to be scanned.

With reference to the first aspect, in some implementations of the first aspect, determining the target scanning path according to the current scanning point and the optimal scanning point of each sub-area in at least one sub-area to be scanned includes: determining the starting point as the current scanning point, a plurality of scanning paths passing through the optimal scanning point of each sub-area in at least one sub-area to be scanned and the end point being the optimal scanning point of a sub-area in at least one sub-area to be scanned; The scan path with the shortest path is determined as the target scan path.

With reference to the first aspect, in some implementations of the first aspect, determining the target scanning path according to the current scanning point and the optimal scanning point of each subregion in at least one subregion to be scanned includes: combining multiple scanning paths The scanning path with the shortest path in the path is determined as the target scanning path, wherein, each scanning path in the plurality of paths starts at the current scanning point, passes through the optimal scanning point of each sub-area in at least one sub-area to be scanned, and ends at A path of an optimal scanning point of a sub-area in at least one sub-area to be scanned.

By using the shortest scanning path among the above multiple scanning paths as the target scanning path, it is possible to scan at least one sub-area to be scanned by moving as little distance as possible from the current scanning point, which can reduce scanning to a certain extent. time and improve scanning efficiency.

With reference to the first aspect, in some implementations of the first aspect, determining at least one sub-area to be scanned of the target room includes: obtaining boundary information of the target room; constructing a bounding box of the target room according to the boundary information; constructing a bounding box according to the bounding box The initial grid map of the target room, wherein the initial grid map is at least one of the top view, bottom view, left view, right view, front view and rear view of the target room; the boundary information is projected to the initial grid map, Obtain the target grid map; determine the area outside the 3D point cloud projection area in the target grid map as the area to be scanned, and the area to be scanned includes at least one sub-area to be scanned.

It should be understood that the initial value of each grid in the initial grid map is an invalid value, indicating that the area in the initial grid map has not yet been scanned.

The aforementioned boundary information may be a scanned 3D point cloud.

Optionally, the above-mentioned bounding box is a cube corresponding to the target room, and the length, width, and height of the cube correspond to the length, width, and height of the target room, respectively, and are used to represent the size of the target space.

It should be understood that when constructing the initial grid map (or called grid map) of the current room according to the bounding box, the top view grid map, the bottom view grid map, the left view grid map, the right View grid, front view grid and back view grid, and choose one of them as the initial grid. Or it is also possible to construct a grid graph of only one view, and determine the grid graph of this view as the initial grid graph.

With reference to the first aspect, in some implementation manners of the first aspect, determining the optimal scanning point of each sub-area in at least one sub-area to be scanned includes: determining the geometric center of gravity of the geometry corresponding to the three-dimensional space to which each sub-area belongs ; Determine the reference plane passing through the lowest point of the edge of the upper surface edge of the geometric center of gravity and the geometry and perpendicular to the bottom surface of each sub-region; determine the target line segment on the reference plane, wherein the starting point of the target line segment is the geometric center of gravity, and the length of the target line segment is Preset length, the angle between the target line segment and the bottom surface of each sub-area is a preset angle; determine the position of the end point of the target line segment as the position of the best scanning point for each sub-area; point the end point of the target line segment to the target line segment The orientation of the starting point is determined as the scanning pose of the best scanning point.

Optionally, the above method further includes: determining the exit of the current room according to the real-time 3D point cloud information of the current room; when the scanning degree of the current room satisfies the second preset scanning degree, moving to the exit of the current room, for the current room other rooms other than the exit of the room to scan.

During the scanning process, by identifying the exit of the current room, continuous scanning of multiple rooms can be realized, and the scanning effect can be improved.

In a second aspect, an unmanned aerial vehicle is provided, and the unmanned aerial vehicle includes various modules for performing the method in the first aspect above.

In a third aspect, a computer-readable storage medium is provided, and the computer-readable storage medium stores program code, wherein the program code includes a method for executing the method in any one of the implementation manners in the first aspect instruction.

When the above-mentioned computer-readable storage medium is set inside the drone, the drone can execute the program code in the computer-readable storage medium, and when the drone executes the program code in the computer-readable storage medium, the drone The man-machine can execute the method in any one of the implementation manners in the first aspect above.

Description of drawings

FIG. 1 is a schematic flowchart of a method for establishing an indoor 3D map according to an embodiment of the present application;

Fig. 2 is a schematic diagram of determining the optimal scanning point of the sub-area to be scanned;

Fig. 3 is a schematic diagram of a bounding box;

Fig. 4 is a schematic diagram of a top view grid map;

Fig. 5 is a schematic diagram of a top view grid map after the boundary is expanded;

Fig. 6 is a grid diagram containing 3D point cloud information;

Figure 7 is a schematic diagram of a grid map;

Fig. 8 is a schematic diagram of forming a shielding area;

Fig. 9 is a schematic flowchart of scanning multiple rooms;

Fig. 10 is a schematic block diagram of an unmanned aerial vehicle according to an embodiment of the present application;

Fig. 11 is a schematic block diagram of an unmanned aerial vehicle according to an embodiment of the present application.

Detailed ways

The technical solution in this application will be described below with reference to the accompanying drawings.

The scanning points in this application may also be referred to as observation points or detection points, and the scanning points may be some key points that pass through when scanning the room. In this application, a complete indoor scan is achieved by acquiring these scanning points and planning a scanning path based on these scanning points.

Fig. 1 shows a schematic flowchart of a method for establishing an indoor 3D map according to an embodiment of the present application. The method shown in FIG. 1 can be performed by a drone, or other devices with automatic flight and scanning functions. The method shown in FIG. 1 specifically includes steps 110 to 160, and the steps 110 to 160 will be described in detail below.

110. Determine at least one sub-area to be scanned of the target room.

The aforementioned target room may be a room that requires 3D mapping.

120. Determine an optimal scanning point for each subarea in at least one subarea to be scanned.

Wherein, when the optimal scanning point of each sub-region is to scan each sub-region, the scanning degree of each sub-region reaches the first preset scanning degree.

The scanning degree of a certain sub-region can be expressed by the ratio of the scanning area of the sub-region to the entire area of the sub-region. For example, the above-mentioned first preset scanning degree can be 60%. At this time, taking the first sub-area as an example, the optimal scanning point of the first sub-area is when the first sub-area is scanned. The ratio of the area to the entire area of the first sub-region is greater than or equal to 60%.

130. Determine a target scanning path according to the current scanning point and the optimal scanning point of each sub-area in the at least one sub-area to be scanned.

The aforementioned at least one sub-area to be scanned may be all sub-areas to be scanned (or called unscanned sub-areas) in the target room.

Optionally, determining the target scanning path according to the current scanning point and the optimal scanning point of each sub-area in the at least one sub-area to be scanned includes: determining that the starting point is the current scanning point, and passing through at least one sub-area to be scanned in the target room The path of the area is the target scan path.

The scanning of the area to be scanned can be realized by planning a path starting from the current scanning point and passing through at least one subarea to be scanned in the target room.

Optionally, the above-mentioned target path may be a scanning path starting from the current scanning point to a certain sub-area in the at least one sub-area to be scanned, or starting from the current scanning point and passing through the at least one sub-area to be scanned The scan path of the best scan points for all sub-areas of .

The specific process of determining the target scanning path in these two cases will be described in detail below.

The first case: the starting point of the target scanning path is the current scanning point, and the ending point is the optimal scanning point of a certain sub-area in at least one sub-area to be scanned.

In the first case, a path from the current scanning point to at least one optimal scanning point of the largest sub-area in the area to be scanned can be planned as the target scanning path, and the specific process is as follows:

(1) determining the subregion with the largest area in at least one subregion to be scanned as the target subregion;

(2) Determine the target scanning path according to the current scanning point and the optimal scanning point of the target sub-area, wherein the starting point of the target scanning path is the current scanning point, and the end point is the path of the optimal scanning point of the target sub-area.

In the present application, by preferentially scanning a sub-region with a larger area during the scanning process, a larger area can be scanned as much as possible during the scanning process, and the scanning efficiency can be improved to a certain extent.

Further, in order to reduce the length of the path as much as possible during the scanning process, the shortest available path can be selected as the target scanning path, and the specific process is as follows:

(3) Determining the shortest scanning path among the plurality of scanning paths whose starting point is the current scanning point and the ending point is the optimal scanning point of the target sub-area as the target scanning path.

In the present application, by using the shortest scanning path as the target scanning path, the moving distance can be reduced during the scanning process, the scanning time can be reduced to a certain extent, and the scanning efficiency can be improved.

Optionally, the length of the shortest scanning path among the above multiple scanning paths may be a linear distance from the current scanning point to the optimal scanning point of the target sub-area.

Or, there is an obstacle between the current scanning point and the optimal scanning point of the target sub-area, resulting in no straight path along the current scanning point to the optimal scanning point of the target sub-area. Then, the above-mentioned target scanning path is the shortest path starting from the current scanning point, bypassing obstacles and reaching the optimal scanning point of the target sub-area.

In the first case, in addition to considering the distance of the path to determine the target scanning path, it can also be considered comprehensively that the starting point is the current scanning point and the ending point is the optimal scanning point of the target sub-area passing through at least one sub-area to be scanned. The scanning area at the best scanning point is comprehensively determined, and the specific process is as follows:

(4) In multiple scan paths whose starting point is the current scan point and the end point is the optimal scan point of the target sub-area, according to the length of each scan path in the multiple scan paths, and the time passed when scanning along each scan path The scanning area at the best scanning point of other sub-areas to be scanned determines the path cost of each scanning path;

(5) Determine the path with the smallest path cost among the multiple paths as the target scanning path.

Wherein, the above-mentioned other sub-areas to be scanned refer to sub-areas in the above-mentioned at least one sub-area to be scanned except the target sub-area.

When determining the target scanning path, specifically, the Bellman-Ford (BF) algorithm that supports negative weights or integer programming, and other greedy algorithms such as genetic algorithm (GA) can be used to select the path with the smallest path cost from various paths. path.

In this application, by synthesizing the path length of each path and the area of other sub-areas to be scanned that can be scanned when scanning along the path, a more reasonable path can be selected from multiple paths as the target scanning path to improve scanning efficiency.

The path cost of each path above can be determined according to formula (1).

C=α·d-β·S (1)

Among them, C represents the path cost of the path, the value of d is proportional to the length of the path, the value of S is proportional to the area of other sub-regions to be scanned that can be scanned when passing through the optimal scanning point of other sub-regions to be scanned, α and β is the weight coefficient (α is the weight coefficient of d, and β is the weight coefficient of S).

The second case: the starting point of the target scanning path is the current scanning point, and the target scanning path passes through all sub-areas in at least one sub-area to be scanned,

In the second case, the starting point is the current scanning point, and there are multiple paths passing through at least one sub-area to be scanned, and one path can be directly selected as the target scanning path from the multiple paths. The specific process is as follows :

(6) Determine the target scanning path according to the current scanning point and the optimal scanning point of each sub-area in at least one sub-area to be scanned.

Specifically, the starting point of the target scanning path determined in the above process (6) is the current scanning point, and the end point is the path of the optimal scanning point of a sub-area in at least one sub-area to be scanned, and the target scanning path passes through at least one The optimal scanning point for each sub-region in the sub-region to be scanned.

Further, in the second case, the path with the shortest path can also be selected from multiple optional paths as the target scanning path, and the specific process is as follows:

(7) Determining the scanning path with the shortest path among the plurality of scanning paths as the target scanning path, wherein each scanning path in the plurality of paths is the starting point as the current scanning point, and passes through each sub-area in at least one sub-area to be scanned The optimal scanning point of , and the end point is a path of the optimal scanning point of one sub-area in at least one sub-area to be scanned.

In this application, by using the shortest scanning path as the target scanning path, it is possible to start from the current scanning point to scan at least one sub-area to be scanned by moving as little distance as possible, which can reduce the scanning time to a certain extent and improve scanning efficiency.

140. Starting from the current scanning point, scan at least one sub-area of the target room to be scanned according to the target scanning path;

It should be understood that step 140 starts from the current scanning point and moves along the target scanning path to scan at least one sub-area of the target room to be scanned.

150. Determine whether all sub-areas of the target room are scanned.

Scanning all sub-areas (all sub-areas) of the target room is equivalent to scanning the target room. Whether the target room has been scanned can be judged according to the scanning degree of the target room.

Specifically, when the scanning of the target room reaches the second preset scanning level, it may be considered that the sub-area of the target room has been scanned. For example, when the ratio of the scanned area of the target room to the total area of the target room reaches 95% or more, it can be considered that the scanning of the target room is completed.

160. Establish a 3D map of the target room according to the 3D point cloud of the target room obtained through scanning.

It should be understood that the 3D point cloud of the target room obtained during the scanning process specifically includes the scanned points in the target room, as well as the 3D coordinate information of these scanned points, and the category attribute information (semantic information, color texture information, etc.) of the points. . The 3D map of the target room may be represented by an occupancy grid map, may also be a 3D grid map represented by an octree, or may be a semantic map with semantic information, etc.

In this application, only unscanned areas are paid attention to during path planning, so that the planned scanning path is more targeted, can improve the overall scanning efficiency when scanning a room, and can finally build a 3D map effect.

Optionally, as an embodiment, determining the optimal scanning point of each sub-area in at least one sub-area to be scanned includes: determining the geometric center of gravity of the geometry corresponding to the three-dimensional space to which each sub-area belongs; The lowest point of the edge of the upper surface of the edge and the reference plane perpendicular to the bottom surface of each sub-region; the target line segment is determined on the reference plane, wherein the starting point of the target line segment is the geometric center of gravity, the length of the target line segment is a preset length, and the target line segment and The included angle of the bottom surface of each sub-area is a preset included angle; the position of the end point of the target line segment is determined as the position of the best scanning point of each sub-area; the end point of the target line segment is pointed to the target The direction of the starting point of the line segment is determined as the scanning posture of the optimal scanning point.

It should be understood that the geometry corresponding to the three-dimensional space to which each sub-area belongs may be the geometry formed by the three-dimensional space where each sub-area is located, or the geometry is the geometry formed by the three-dimensional space occupied by the objects in each sub-area. It can be regular or irregular.

As shown in Figure 2, the geometry corresponding to the sub-area to be scanned is an irregular cylinder. The center of gravity A of the cylinder and the lowest point B on the edge of the upper surface of the cylinder form a reference plane along the height direction of the cylinder. The plane is perpendicular to the bottom surface of the cylinder, so a length can be drawn from point A on the reference plane as a preset length (according to the best observation distance of the sensor in the drone, it can be set to 2 meters or other values), A line segment that forms a certain angle (for example, 30 degrees) with the height direction of the cylinder, the position of the end point of the line segment is the position of the best scanning point of the sub-area to be scanned, and the opposite direction of the line segment is the same as the height of the cylinder The angle formed by the directions is the scanning attitude (scanning angle) of the optimal scanning point of the sub-area to be scanned.

It should be understood that before scanning at least one sub-area to be scanned, the area to be scanned in the target image may also be determined first, and the specific process of determining the area to be scanned in the target image is as follows:

(8) Obtain the boundary information of the target room;

(9) Construct the bounding box of the target room according to the boundary information;

(10) Constructing an initial grid map of the target room according to the bounding box, wherein the initial grid map is at least one of a top view, a bottom view, a left view, a right view, a front view and a rear view of the target room;

(11) Project the boundary information to the initial grid map to obtain the target grid map;

(12) Determining an area outside the 3D point cloud projection area in the target grid map as an area to be scanned, and the area to be scanned includes at least one sub-area to be scanned.

The foregoing boundary information may be information such as approximate length, width, and height of the target room obtained during initial scanning. For example, when the method of the present application is executed by a UAV, the above boundary information may be obtained by roughly scanning around the target room after the UAV initially takes off.

In addition, the initial value of each grid in the initial grid map is an invalid value, indicating that the area in the initial grid map has not been scanned.

Specifically, after the drone takes off, a camera or other detectors can be used to roughly detect the room to obtain the general space in the room, and then construct an indoor bounding box according to the space in the room.

In addition, the flying height of the drone can be set in a certain proportion according to the height of the room, or can be set in a way to keep a certain safe distance from the top of the room.

As shown in FIG. 3 , the above-mentioned bounding box may be a cube, and the length, width, and height of the cube may correspond to the length, width, and height of the room, respectively, and are used to represent the size of the indoor space.

When constructing the initial grid map (or grid map) of the current room according to the bounding box, the top view grid map, bottom view grid map, left view grid map, and right view grid map of the room can be constructed according to the bounding box map, front-view grid map, and back-view grid map, and select one of the grid maps as the initial grid map. Or it is also possible to construct a grid graph of only one view, and determine the grid graph of this view as the initial grid graph.

It should be understood that the length and width of the top-view grid and bottom-view grid constructed according to the indoor bounding box are the same as the length and width of the room bounding box respectively, while the left-view grid, right-view grid, and front-view grid The length and width of the map and the backview grid map correspond to the length and height or width and height of the indoor bounding box.

For example, the top-view grid graph constructed according to the bounding box is shown in FIG. 4 , and the top-view grid graph shown in FIG. 4 can be directly used as the initial grid graph. It should be understood that the length and width of the top view grid map shown in FIG. 4 are the same as the length and width of the bounding box shown in FIG. 3 . In addition, the top view grid diagram shown in FIG. 4 also contains many grids, which can conveniently determine the scanning area and the area to be scanned.

It should be understood that the top view can be divided into grid maps according to the size of 0.1*0.1m (or other sizes), so as to obtain the top view grid map shown in Figure 4, and use these grids to record the grid area Whether it has been scanned, so as to determine the scanning area and the area to be scanned.

In addition, when determining the initial grid map, the boundary of the obtained grid map of a certain viewing angle may also be expanded, and the expanded grid map is determined as the initial grid map. For example, by extending the top-view grid diagram shown in Figure 4, the top-view grid diagram after the boundary extension shown in Figure 5 can be obtained, and the grid on the boundary in the top-view grid diagram shown in Figure 5 is used to represent The walls around the room, where the value of some grids on the boundary is the height h ₀ of the wall, while other grids on the boundary do not exist in these grids due to occlusion or beyond the detection distance of the drone's sensor value.

When the initial grid map is the top view grid map shown in Figure 5, the currently acquired 3D point cloud information can be projected (projected) onto the top view grid map shown in Figure 5 to obtain the grid map shown in Figure 6 Getu. In Figure 6, the area where the symbol "+" is located indicates the area where the 3D point cloud is projected, and the unit area where the symbol "-" is located indicates the area that is not projected to the 3D point cloud. Therefore, the area on the right side of Figure 6 is the area to be scanned, where , the grid where h is located represents the junction of the area to be scanned and the scanning area, and the value of h represents the height of the junction. It should be understood that the height value at the junction of the to-be-scanned area and the scanned area shown in FIG. 6 may have a certain difference, and h in the figure is only a symbol indicating the height of the junction, and does not represent the actual height. The area to be scanned shown in FIG. 6 is a complete area, and at this time, the area to be scanned is substantially composed of a sub-area.

As shown in FIG. 7 , the area to be scanned includes two independent areas. At this time, the area to be scanned is essentially composed of two sub-areas (sub-area to be scanned 1 and sub-area to be scanned 2).

In addition, in the process of scanning indoors, due to the occlusion of objects, occlusion areas may be generated during the scanning process, that is to say, there may be occlusion areas in the area to be scanned, and the relevant content of the occlusion areas will be described below. introduce.

For example, as shown in Figure 8, due to the occlusion of the piano in the current room, two boundary lines may be obtained when scanning the piano, boundary line 1 and boundary line 2, and these two boundary lines are continuous inside the image , but the 3D coordinate system is discontinuous, and the positions of boundary line 1 and boundary line 2 in the 3D coordinate system constitute the visual occlusion area. It should be understood that the occlusion area formed due to occlusion is also included in the scope of the sub-area to be scanned, and the present application has actually taken the occlusion area into consideration when planning a path according to the area to be scanned.

It should be understood that after completing the scanning of the target room, you can also enter other rooms through the exit of the target room to scan other unscanned rooms, so as to realize continuous scanning in multiple rooms. This part will be described below in conjunction with FIG. The content is described in detail.

Fig. 9 is a schematic flowchart of scanning multiple rooms. The process shown in FIG. 9 can be performed by a drone or other devices with automatic flight and scanning functions. The process shown in FIG. 9 specifically includes steps 2001 to 2006, which will be described in detail below.

2001, start;

When there are multiple rooms to scan, you can start scanning from any room.

2002. Scan the current room, and determine the exit information of the current room according to the scanned information of the current room.

It should be understood that the current room here is equivalent to the target room in the method shown in FIG. 1 , and the detailed process of scanning the current room can refer to the method shown in FIG. 1 above, and the detailed process will not be repeated.

Specifically, the exit information of the current room can be determined from the scanning information obtained during the scanning process of the current room (determine the exit information of the current room while scanning), or the exit information of the current room can be determined after the scanning of the current room is completed. export information.

The above-mentioned exit information may include the size of the exit (door, staircase, etc.) of the current room, the position of the exit in the current room, and the like.

When determining the exit information of the current room, the image of the current room and the scanned 3D point cloud may be obtained first, and then the exit information of the current room may be determined according to the image of the current room and the 3D point cloud.

After the image of the current room is obtained, target detection can be performed on the image of the current room (the obtained image can be detected through a neural network or other target detection algorithm), the exit of the current room can be obtained, and the exit of the current room can be identified Then combined with the 3D point cloud obtained by scanning, the 3D position information of the export can be obtained.

Specifically, indoor exits generally include doors, windows, stairs, etc. When identifying exits, you can collect some pictures with the positions of doors and stairs marked, and train the neural network. After training, use this network model, Input a photo with a door or a staircase, and the neural network can output where the door and staircase are located in the image. It should be understood that the position of the exit identified by the neural network is a 2D position. In order to obtain the real position of the exit of the current room, it is necessary to combine the 3D point cloud information of the current room and segment the point cloud based on the 2D position, and then After segmentation, the point cloud data belonging to the door is estimated by a neural network (for example, Frustum-Pointnet network) algorithm to obtain 3D information of the door.

2003. Determine whether the current room has been scanned;

If the current room has not been scanned, then return to the previous step and continue to execute step 2002; if the current room has been scanned, then execute step 2004.

If the scanning degree of the current room reaches the second preset scanning degree (for example, when the scanning degree reaches 90% or more), it can be considered that the scanning of the current room is over; if the scanning degree of the current room does not reach the second preset scanning degree (less than 90%, that is, more than 10% of the area has not been scanned), it can be considered that the scanning of the current room has not ended.

The above-mentioned second preset scanning degree can be set according to actual conditions, for example, it can be set to 90%, 95% and so on.

Specifically, the second preset scanning degree may be the degree of completion of the scan, such as the proportion of the scanned space to the total space, or the fineness of the scan, that is, the richness of the details of the scanned space included in the scan, etc. limited.

2004. Determine whether there is an unpassed exit in the scanned room;

If there is an unpassed exit in the scanned room, then step 2005 is executed; if there is no unpassed exit in the current room, then step 2006 is executed.

It should be understood that during the scanning process, the status of the exits of each room may be recorded, and the initial status of each exit is not passed, and if a certain exit is passed, the status of the exit is recorded as passed.

In step 2004, it may also be first judged whether there is an unpassed exit in the current room. If there is an unpassed exit in the current room, then enter other unscanned rooms through the unpassed exit of the current room to scan; if there is no unpassed exit in the current room, then return to the previous room of the current room. After returning to the previous room, judge whether there is an unpassed exit in the previous room. If there is an unpassed exit, enter other unscanned rooms from the unpassed exit of the previous room for scanning, otherwise continue to return to the previous room. , until returning to the initial scanning room, if there is an unpassed exit in the initial scanning room, then continue to scan other un-scanned rooms through the unpassed exit; if there is no unpassed exit in the initial scanning room, Then the scan ends.

2005. Enter other unscanned rooms through unpassed exits, and scan other unscanned rooms.

After step 2005 is executed, step 2002 is re-executed until all unscanned rooms are scanned.

The scanning process when scanning other unscanned rooms is the same as the scanning process for the target room in the method shown in FIG. 1 , and will not be described again here.

2006, end.

After the scanning is completed, a 3D map of each room can be constructed based on the acquired 3D point clouds of each room.

The method for establishing an indoor 3D map of the embodiment of the present application has been described in detail above in conjunction with FIGS. The unmanned aerial vehicle shown in 11 can execute each step of the method for establishing an indoor 3D map according to the embodiment of the present application. Specifically, the unmanned aerial vehicle shown in FIG. 10 and FIG. 11 can execute each step in FIG. 1 and FIG. 9 , In order to avoid unnecessary repetition, repeated descriptions are appropriately omitted when introducing the drone of the embodiment of the present application below.

Fig. 10 is a schematic block diagram of an unmanned aerial vehicle according to an embodiment of the present application. The UAV 3000 shown in Figure 10 includes:

A sub-area to be scanned determination module 3001, configured to determine at least one sub-area to be scanned in the target room;

The to-be-scanned sub-area determining module 3001 is further configured to determine the optimal scanning point of each sub-area in the at least one to-be-scanned sub-area, wherein, at the optimal scanning point of each sub-area, each sub-area is When scanning, the scanning degree of each sub-region reaches the first preset scanning degree;

A path planning module 3002, configured to determine a target scanning path according to the current scanning point and the optimal scanning point of each sub-area in the at least one sub-area to be scanned;

The movement and scanning module 3003 is configured to scan at least one sub-area to be scanned of the target room according to the target scanning path starting from the current scanning point;

The mapping module 3004 is configured to build a 3D map of the target room according to the 3D point cloud of the target room obtained through the scanning when all sub-areas of the target room are scanned.

In this application, only unscanned areas are paid attention to during path planning, so that the planned scanning path is more targeted, can improve the overall scanning efficiency when scanning a room, and can finally build a 3D map effect.

Fig. 11 is a schematic block diagram of an unmanned aerial vehicle according to an embodiment of the present application.

The UAV 4000 shown in FIG. 11 includes: a detection unit 4001 , a flight unit 4002 and a control and processing unit 4003 .

Among them, the control and processing unit 4003 is equivalent to the sub-area determination module 3001 to be scanned, the path planning module 3002 and the mapping module 3004 above; the detection unit 4001 is equivalent to the scanning module in the motion and scanning module 3003, and the flight unit 4002 is equivalent to The motion module in the movement and scanning module 3003; each module in the UAV 4000 can also realize the functions of each module in the UAV 3000.

It should be understood that the above-mentioned control and processing unit 4003 may correspond to a processor inside the drone, and the processor inside the drone may specifically be a central processing unit (central processing unit, CPU) or an artificial intelligence chip.

The detection unit 4001 may correspond to a camera and other detectors of the drone. The flight unit 4002 may correspond to components such as motors and propellers of the drone used to drive the drone to fly.

In the several embodiments provided in this application, it should be understood that the disclosed methods and devices may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of units or modules in the device is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components May be combined or may be integrated into another system, or some features may be omitted, or not implemented.

In addition, each unit or module in the device of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.

If the method in the present application is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disk and other various media that can store program codes. .

The above is only a specific implementation of the application, but the scope of protection of the application is not limited thereto. Anyone familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the application. Should be covered within the protection scope of this application. Therefore, the protection scope of the present application should be determined by the protection scope of the claims.

Claims (18)

1. A method of creating an indoor 3D map, comprising scanning each sub-region of a target room, comprising:

determining at least one sub-area to be scanned of the target room;

determining an optimal scanning point of each sub-area in the at least one sub-area to be scanned, wherein when the optimal scanning point of each sub-area scans the sub-area, the scanning degree of each sub-area reaches a first preset scanning degree;

determining a target scanning path according to the current scanning point and the optimal scanning point of each sub-area in the at least one sub-area to be scanned;

starting from the current scanning point, scanning the at least one sub-region to be scanned according to the target scanning path;

and when all the subareas of the target room finish the scanning, establishing a 3D map of the target room according to the 3D point cloud of the target room obtained by scanning.

2. The method of claim 1, wherein the method further comprises:

determining a sub-region with the largest area in the at least one sub-region to be scanned as a target sub-region;

determining a target scan path according to the current scan point and an optimal scan point of each sub-region of the at least one sub-region to be scanned, including:

And determining the target scanning path according to the current scanning point and the optimal scanning point of the target sub-region, wherein the starting point of the target scanning path is the current scanning point, and the end point is the optimal scanning point of the target sub-region.

3. The method of claim 2, wherein the determining the target scan path from the current scan point and the optimal scan point for the target sub-region comprises:

and determining a scanning path with the shortest path among a plurality of scanning paths with the starting point as the current scanning point and the end point as the optimal scanning point of the target sub-region as the target scanning path.

4. The method of claim 2, wherein the determining the target scan path from the current scan point and the optimal scan point for the target sub-region comprises:

determining the path cost of each scanning path according to the length of each scanning path in a plurality of scanning paths with the starting point being the current scanning point and the end point being the optimal scanning point of the target sub-area and the scanning area passing through the optimal scanning points of other sub-areas to be scanned when scanning along each scanning path, wherein the other sub-areas to be scanned are sub-areas except the target sub-area in the at least one sub-area to be scanned;

And determining the path with the minimum path cost in the plurality of scanning paths as the target scanning path.

5. The method of claim 4, wherein the scan cost for each scan path is obtained according to the following formula:

C＝α·d-β·S

wherein C represents the path cost of the path, d is proportional to the path length, S is proportional to the area of other sub-regions to be scanned when passing through the optimal scanning point of the other sub-regions to be scanned, and alpha and beta are weight coefficients.

6. The method of claim 1, wherein a start point of the target scan path is the current scan point, the target scan path passes through a best scan point of each of the at least one sub-region to be scanned, and an end point of the target scan path is a best scan point of one of the at least one sub-region to be scanned.

7. The method of claim 6, wherein determining a target scan path based on the current scan point and the optimal scan point for each of the at least one sub-region to be scanned comprises:

and determining a scanning path with the shortest path in a plurality of scanning paths as the target scanning path, wherein each scanning path in the plurality of scanning paths is a path with a starting point being the current scanning point, a best scanning point passing through each sub-area in the at least one sub-area to be scanned and an end point being the best scanning point of one sub-area in the at least one sub-area to be scanned.

8. The method according to any of claims 1-7, wherein determining at least one sub-area of the target room to be scanned comprises:

acquiring boundary information of the target room;

constructing a bounding box of the target room according to the boundary information;

constructing an initial grid diagram of the target room according to the bounding box, wherein the initial grid diagram is at least one of a top view, a bottom view, a left view, a right view, a front view and a rear view of the target room;

projecting the boundary information to the initial grid map to obtain a target grid map;

and determining an area outside the 3D point cloud projection area in the target grid graph as an area to be scanned, wherein the area to be scanned comprises the at least one sub-area to be scanned.

9. The method of any of claims 1-7, wherein determining the best scan point for each of the at least one sub-region to be scanned comprises:

determining the geometric gravity center of a geometric body corresponding to the three-dimensional space to which each sub-region belongs;

determining a reference plane passing through the geometric center of gravity and an edge lowest point of an upper surface edge of the geometric body and perpendicular to the bottom surface of each sub-area;

Determining a target line segment on the reference plane, wherein the starting point of the target line segment is the geometric center of gravity, the length of the target line segment is a preset length, and the included angle between the target line segment and the bottom surface of each sub-area is a preset included angle;

determining the position of the end point of the target line segment as the position of the optimal scanning point of each sub-area;

and determining the direction of the end point of the target line segment to the starting point of the target line segment as the scanning gesture of the optimal scanning point.

10. An unmanned aerial vehicle, comprising:

the sub-area to be scanned determining module is used for determining at least one sub-area to be scanned of the target room;

the sub-area to be scanned determining module is further configured to determine an optimal scanning point of each sub-area in the at least one sub-area to be scanned, where when the optimal scanning point of each sub-area scans the sub-area, a scanning degree of each sub-area reaches a first preset scanning degree;

the path planning module is used for determining a target scanning path according to the current scanning point and the optimal scanning point of each sub-area in the at least one sub-area to be scanned;

The motion and scanning module is used for scanning the at least one sub-area to be scanned according to the target scanning path from the current scanning point;

and the mapping module is used for building a 3D map of the target room according to the 3D point cloud of the target room obtained through the scanning when all the subareas of the target room finish the scanning.

11. The drone of claim 10, wherein the sub-region to be scanned determination module is further to:

determining a sub-region with the largest area in the at least one sub-region to be scanned as a target sub-region;

the path planning module is used for determining the target scanning path according to the current scanning point and the optimal scanning point of the target sub-region, wherein the starting point of the target scanning path is the current scanning point, and the end point is the optimal scanning point of the target sub-region.

12. The drone of claim 11, wherein the path planning module is to:

and determining a scanning path with the shortest path among a plurality of scanning paths with the starting point as the current scanning point and the end point as the optimal scanning point of the target sub-region as the target scanning path.

13. The drone of claim 11, wherein the path planning module is to:

determining the path cost of each scanning path in a plurality of scanning paths with the starting point being the current scanning point and the end point being the optimal scanning point of the target sub-area according to the length of each scanning path in the plurality of scanning paths and the scanning area passing through the optimal scanning point of the at least one sub-area to be scanned when scanning along each scanning path;

and determining the path with the minimum path cost in the plurality of scanning paths as the target scanning path.

14. The drone of claim 13, wherein the scan cost for each scan path is obtained according to the following equation:

C＝α·d-β·S

wherein C represents the path cost of the path, d is proportional to the path length, S is proportional to the area of other sub-regions to be scanned when passing through the optimal scanning point of the other sub-regions to be scanned, and alpha and beta are weight coefficients.

15. The drone of claim 10, wherein a start point of the target scan path is the current scan point, the target scan path passes through an optimal scan point for each of the at least one sub-area to be scanned, and an end point of the target scan path is an optimal scan point for one of the at least one sub-area to be scanned.

16. The drone of claim 15, wherein the path planning module is to:

and determining a scanning path with the shortest path in a plurality of scanning paths as the target scanning path, wherein each scanning path in the plurality of scanning paths is a path with a starting point being the current scanning point, a best scanning point passing through each sub-area in the at least one sub-area to be scanned and an end point being the best scanning point of one sub-area in the at least one sub-area to be scanned.

17. The drone of any one of claims 10-16, wherein the sub-area to be scanned determination module is to:

acquiring boundary information of the target room;

constructing a bounding box of the target room according to the boundary information;

constructing an initial grid diagram of the target room according to the bounding box, wherein the initial grid diagram is at least one of a top view, a bottom view, a left view, a right view, a front view and a rear view of the target room;

projecting the boundary information to the initial grid map to obtain a target grid map;

and determining an area outside the 3D point cloud projection area in the target grid graph as an area to be scanned, wherein the area to be scanned comprises the at least one sub-area to be scanned.

18. The drone of any one of claims 10-16, wherein the sub-area to be scanned determination module is to:

determining the geometric gravity center of a geometric body corresponding to the three-dimensional space to which each sub-region belongs;

determining a reference plane passing through the geometric center of gravity and an edge lowest point of an upper surface edge of the geometric body and perpendicular to the bottom surface of each sub-area;

determining a target line segment on the reference plane, wherein the starting point of the target line segment is the geometric center of gravity, the length of the target line segment is a preset length, and the included angle between the target line segment and the bottom surface of each sub-area is a preset included angle;

determining the position of the end point of the target line segment as the position of the optimal scanning point of each sub-area;

and determining the direction of the end point of the target line segment to the starting point of the target line segment as the scanning gesture of the optimal scanning point.