CN116503627B - Model construction system and method based on multi-source data - Google Patents
Model construction system and method based on multi-source data Download PDFInfo
- Publication number
- CN116503627B CN116503627B CN202310745909.9A CN202310745909A CN116503627B CN 116503627 B CN116503627 B CN 116503627B CN 202310745909 A CN202310745909 A CN 202310745909A CN 116503627 B CN116503627 B CN 116503627B
- Authority
- CN
- China
- Prior art keywords
- data
- source data
- sampling
- fitness
- track
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/04—Texture mapping
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Multimedia (AREA)
- Geometry (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Graphics (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Remote Sensing (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Selective Calling Equipment (AREA)
- Processing Or Creating Images (AREA)
Abstract
The application relates to the technical field of digital data processing, in particular to a model construction system and method based on multi-source data, wherein the method comprises the following steps: acquiring multi-source data acquired by a plurality of terminals, extracting characteristic points according to the multi-source data, and matching according to the extracted characteristic points to obtain characteristic point pairs; and constructing a basic model according to the multi-source data and the characteristic point pairs, and performing texture mapping on the basic model according to the multi-source data to generate a fine model. And rejecting the matched characteristic point pairs, rejecting the mismatching characteristic point pairs, and constructing according to the rejected characteristic point pairs when constructing the basic model. By adopting the scheme, the technical problems that in the prior art, the data source is single, errors exist in the acquired information, and the live-action model is wrong can be solved.
Description
Technical Field
The application relates to the technical field of digital data processing, in particular to a model construction system and method based on multi-source data.
Background
In the prior art, feasibility study is required before road and bridge construction, field investigation is required in the stage, site conditions are acquired, and a live-action model is constructed to verify the rationality of a design route. In field surveys, unmanned aerial vehicles are employed to take aerial photographs to collect field data due to the large area of investigation and the presence of difficult to reach terrain-complex areas. In conventional aerial photography, an image acquisition device is mounted on a flight platform, for example, a camera device is mounted on an unmanned aerial vehicle, and the camera device continuously acquires images through the flight of the unmanned aerial vehicle, so as to acquire field image data.
However, when the unmanned aerial vehicle navigates, the unmanned aerial vehicle shoots at the visual angle of the camera equipment, only partial information of the target object, such as the top of the target object, can be acquired, and the side information of the target object can not be acquired comprehensively. The buildings in the city are dense, the buildings are mutually shielded, and especially the middle and lower parts of the buildings are difficult to acquire information of corresponding parts by unmanned aerial vehicle shooting. Meanwhile, the method is also interfered by flying objects such as birds in the village, and the shielding situation occasionally occurs, so that the acquired information is interfered, and a gap exists between a finally constructed live-action model and reality.
Disclosure of Invention
The application aims to provide a model construction method based on multi-source data, which aims to solve the technical problem that in the prior art, a real model is wrong due to single data source and errors in information acquisition.
The basic scheme provided by the application is as follows: the model construction method based on the multi-source data comprises the following steps:
acquiring multi-source data acquired by a plurality of terminals, extracting characteristic points according to the multi-source data, and matching according to the extracted characteristic points to obtain characteristic point pairs;
and constructing a basic model according to the multi-source data and the characteristic point pairs, and performing texture mapping on the basic model according to the multi-source data to generate a fine model.
Further, the method also comprises the following steps:
obtaining topographic data to be modeled, and generating a sampling track according to the topographic data; sampling by a plurality of terminals according to the sampling track to obtain multi-source data;
the terrain data comprises a terrain area, plane data and elevation data of a required modeling area, and a sampling track is generated according to the terrain data, and comprises the following contents:
carrying out endpoint recognition according to the terrain area to construct a sampling area, and planning a sampling track according to the sampling area;
and establishing a sampling reference according to the plane data and the elevation data, and sampling the multi-source data according to the sampling reference.
Further, planning a sampling trajectory from the sampling region includes:
constructing a three-dimensional space coordinate system in a target area to be sampled, and dividing the target area to be sampled into a plurality of cube spaces with preset lengths as side lengths according to the three-dimensional space coordinate system;
and classifying the three-dimensional cube space according to the terrain area, the plane data and the elevation data, wherein the classification comprises three types of object space, barrier-free space and barrier space, and marking the importance of the object space.
Randomly generating a plurality of initial sampling tracks to form an initial sampling track set;
constructing a fitness function, wherein the fitness function comprises a track length fitness function, an image acquisition income fitness function and an operation complexity fitness function;
the track length fitness function is as follows:
in the formula ,、/>、/>spatial coordinates representing the ith point in the initial sampling trajectory,/->Representing the distance between the ith point and the (i+1) th point in the initial sampling track, n being the midpoint of the initial sampling trackTotal number of bits>Is the track length;
the image acquisition yield fitness function is as follows:
in the formula ,、/>、/>the method comprises the steps of taking the spatial coordinates of a position with a linear distance j from an ith point on a path between the ith point and an (i+1) th point in an initial sampling track; />、/>、/>The space coordinates of a target object space m closest to the position with the linear distance j from the ith point on the path between the ith point and the (i+1) th point in the initial sampling track;representing the effective acquisition radius of the camera; />The importance of the object space m nearest to the position of the i-th point on the path between the i-th point and the i+1-th point in the initial sampling track, which is at a linear distance j from the i-th point, is shown. />For image acquisitionThe profitability;
the function of the operational complexity fitness is as follows:
in the formula ,for operational complexity;
and taking the initial sampling track set as an initial group, performing iterative optimization according to the fitness function to obtain an optimal sampling track, and completing the planning of the sampling track.
Further, taking the initial sampling track set as an initial population, performing iterative optimization according to the fitness function includes:
according to the fitness function, calculating fitness function values of all initial sampling tracks, and respectively sequencing the initial sampling tracks from high to low according to the three fitness function values to obtain sequencing sets of the three initial sampling tracks;
according to the fitness function value, randomly selecting a plurality of initial sampling tracks from the three sorting sets through roulette to form three groups in different directions, wherein the selected probability is in direct proportion to the fitness function value;
selecting, crossing and mutating each group according to a genetic algorithm, and kicking out individual schemes which cannot be realized, and respectively iterating for preset times to form three iterated groups;
randomly selecting individuals from three groups according to fitness to form a unidirectional gene library, forming three groups of bidirectional cross groups by two-by-two crossing among the three groups, calculating three fitness of each individual for each group of bidirectional cross groups, performing non-dominant ranking on two fitness related to the bidirectional cross groups, selecting, crossing and mutating according to ranking results, and then kicking out an individual scheme which cannot be realized;
after the three groups of two-way cross groups are iterated for a preset number of times, the three groups are combined together, the three fitness functions are weighted and summed to obtain comprehensive fitness, individual selection, cross and mutation are carried out according to the comprehensive fitness, and an individual with the largest fitness is obtained as an optimal sampling track after the preset number of iterations.
Further, before iterative intersection of the bi-directional intersection groups, a random jump-back mechanism is adopted to select intersection objects:
wherein ,for crossing objects +.>For randomly selected individuals from the unidirectional gene library, <' > a ++>A random number between 0 and 1, < >>Is a jump back threshold.
Further, the method further comprises the following steps: according to the sorting order of the individuals to be crossedAssigning, the lower the individual ranking, the +.>The greater the value.
Further, the method also comprises the following steps:
and rejecting the matched characteristic point pairs, rejecting the mismatching characteristic point pairs, and constructing according to the rejected characteristic point pairs when constructing the basic model.
Further, rejecting the matched feature point pairs is as follows: and adopting left-right consistency test and unique constraint to remove the feature points which are erroneously matched.
Further, texture mapping is performed on the basic model according to the multi-source data to generate a fine model, and the method specifically comprises the following steps:
and during texture mapping, mapping the two-dimensional texture space to a preset intermediate curved surface according to the multi-source data, mapping the texture of the intermediate curved surface to a basic model, and reconstructing a fine model.
The first basic scheme has the beneficial effects that:
in the application, various terminals are adopted to collect data, such as aeronautical data collected by unmanned aerial vehicle aerial survey, ground data collected by ground mobile vehicles and mobile data collected by personal mobile terminals. The defect of single data source is made up by data acquired by various terminals, and the multi-source data acquired by various technical means, multi-angle and all-dimensional acquired data effectively, completely and truly reflect the information of the target object, thereby realizing the integral description of the target object.
Because of different data sources, the method processes the multi-source data, finds the most accurate characteristic point pair through the extraction, matching and elimination of the characteristic points, characterizes the target object through the characteristic points, and realizes the fusion of the multi-source data based on the characteristic point pair, thereby constructing a basic model. When the feature points are removed, the left and right consistency check is adopted, the matched feature point sets are searched, the intersection sets are obtained, the unique constraint is sampled again, the one-to-many feature points are removed according to the one-to-one correspondence principle, the incorrectly matched feature points are removed, the accuracy of the feature point pairs is guaranteed, and therefore the consistency of the final model and reality is guaranteed.
In the application, the acquired data volume is larger, and the data volume for constructing the model is larger based on the acquisition of various technical means. Therefore, the application plans the sampling track of the data acquisition, plans according to the graph of the required modeling area, designs the sampling track with the shortest route, and reduces the sampling data volume, thereby reducing the data processing volume. Alternatively, a sampling reference is designed, sampling is performed by using the sampling reference, and the data type required to be processed is reduced, for example, the data of a plane of longitude and latitude is used as a reference or the data of an elevation is used as a reference, so that the calculation amount of the system is reduced.
In the application, an optimal sampling track is formed by adopting an optimization algorithm, the sampling track can be optimized from multiple aspects by setting a track length fitness function, an image acquisition yield fitness function and an operation complexity fitness function, the sampling path length is reduced, and the sampling efficiency is improved; in the optimization algorithm, a multidirectional evolution strategy is adopted, and population evolution iteration is respectively carried out through three different fitness functions, so that the reservation of optimal solutions in all directions is ensured, the global property is enhanced, and the program can be effectively prevented from being trapped into local optimal; after the multidirectional evolution is finished, a gradual fusion strategy is creatively introduced, and the situation that three groups are directly fused to cause the loss of the optimal character is avoided by fusing every two groups and then comprehensively iterating; and a random jump-back mechanism is provided, so that the excellent single character before random fusion can be realized, the excellent character and the individual can be further ensured to be reserved in the subsequent iteration, and the rationality and the global property of the optimal solution are ensured.
It is a second object of the present application to provide a model building system based on multi-source data.
The application provides a basic scheme II: the model construction system based on the multi-source data uses the model construction method based on the multi-source data, and comprises a server, and is characterized in that:
the server is used for acquiring multi-source data acquired by various terminals, extracting characteristic points according to the multi-source data, and matching according to the extracted characteristic points to obtain characteristic point pairs;
the server is also used for constructing a basic model according to the multi-source data and the characteristic point pairs, and generating a fine model by performing texture mapping on the basic model according to the multi-source data.
Further, the server is further used for eliminating the feature point pairs, eliminating the feature point pairs which are mismatched, and constructing according to the eliminated feature point pairs when constructing the basic model.
Further, the server is also used for acquiring the topographic data of the modeling required and generating a sampling track according to the topographic data; and the multiple terminals sample according to the sampling track to obtain multi-source data.
Further, during texture mapping, the server is further configured to map the two-dimensional texture space to a preset intermediate surface according to the multi-source data, map the texture of the intermediate surface to the basic model, and reconstruct the fine model.
The second basic scheme has the beneficial effects that:
in the application, various terminals are adopted to collect data, such as aeronautical data collected by unmanned aerial vehicle aerial survey, ground data collected by ground mobile vehicles and mobile data collected by personal mobile terminals. The defect of single data source is made up by data acquired by various terminals, and the multi-source data acquired by various technical means, multi-angle and all-dimensional acquired data effectively, completely and truly reflect the information of the target object, thereby realizing the integral description of the target object.
Because of different data sources, the method processes the multi-source data, finds the most accurate characteristic point pair through the extraction, matching and elimination of the characteristic points, characterizes the target object through the characteristic points, and realizes the fusion of the multi-source data based on the characteristic point pair, thereby constructing a basic model. When the feature points are removed, the left and right consistency check is adopted, the matched feature point sets are searched, the intersection sets are obtained, the unique constraint is sampled again, the one-to-many feature points are removed according to the one-to-one correspondence principle, the incorrectly matched feature points are removed, the accuracy of the feature point pairs is guaranteed, and therefore the consistency of the final model and reality is guaranteed.
Drawings
FIG. 1 is a flowchart of a model construction method based on multi-source data according to an embodiment of the present application;
FIG. 2 is a schematic block diagram of a first embodiment of a multi-source data based model building system of the present application.
Detailed Description
The following is a further detailed description of the embodiments:
example 1
The model construction method based on the multi-source data, as shown in fig. 1, comprises the following steps:
s1: and acquiring multi-source data acquired by various terminals.
S2: performing feature processing on the multi-source data, including: and respectively extracting the characteristic points according to the multi-source data, and matching according to the extracted characteristic points to obtain characteristic point pairs.
S3: and constructing a basic model according to the multi-source data and the characteristic point pairs.
S4: and performing texture mapping on the basic model according to the multi-source data to generate a fine model.
S1 specifically comprises the following contents: the multiple terminals comprise unmanned aerial vehicle acquisition equipment, ground vehicle-mounted equipment and backpack equipment, and the multi-source data comprise aviation data, ground data and mobile data.
The unmanned aerial vehicle acquisition equipment carries on multiple sensor, carries out low altitude inclined image acquisition from a plurality of angles by means of aerial survey remote sensing technology to carry out three-dimensional location, acquire aviation data. The aviation data comprise aviation images, aviation plane data and aviation elevation data, and the aviation plane data are two-dimensional longitude and latitude data.
In order to make up for the discontinuous image that the local ground object view caused because of shielding etc. when unmanned aerial vehicle navigates, avoid the incompleteness that single measurement data source exists, still set up ground on-vehicle equipment. The method comprises the steps that coordinates of a target are captured through a laser scanner of ground vehicle-mounted equipment, image data are obtained through a panoramic camera, the attitude and position information of a system at each moment are automatically recorded, ground data are obtained, the ground data comprise ground images, ground plane data and ground elevation data, and the ground plane data are two-dimensional longitude and latitude data.
Meanwhile, the backpack RTK camera is arranged, so that the backpack type RTK camera can be operated by a single person, and the backpack type RTK camera can be operated in a region where an automobile cannot enter, such as crowded relative personnel, complex environment and the like, so that mobile data can be acquired, wherein the mobile data comprises mobile images, mobile plane data and mobile elevation data, and the mobile plane data is two-dimensional longitude and latitude data.
According to the application, various terminals are adopted to collect data, the defect of single data source is made up by the data collected by the various terminals, and the multi-source data collected by various technical means, the data collected in multiple angles and in all aspects, effectively, completely and truly reflect the information of the target object, and realize the integral description of the target object.
S2 specifically comprises the following contents:
invoking multi-source data, respectively extracting characteristic points of image data in the multi-source data according to a SIFT algorithm, and matching according to the extracted characteristic points to obtain characteristic point pairs, wherein the matching according to the extracted characteristic points comprises the following steps: extracting feature descriptors at the feature points, distributing direction values for the feature points, and searching matching points according to the feature descriptors so as to obtain matching point pairs; the feature descriptor is a feature vector describing feature points, the Euclidean distance of the feature vector is used as a similarity judgment measure of the feature points, and if the similarity accords with a preset range, the feature descriptor is judged to be a feature point pair.
S2 further comprises: and eliminating the matched characteristic point pairs, and eliminating the mismatching characteristic point pairs. Rejecting the matched characteristic point pairs is as follows: and adopting left-right consistency test and unique constraint to remove the feature points which are erroneously matched.
Specific: and obtaining characteristic point pairs according to a nearest neighbor algorithm, and then removing wrong characteristic points by utilizing left-right consistency test and uniqueness constraint, so as to obtain an initial matching result. The left-right consistency test is to search the feature point set in the corresponding left image and right image of the feature point pair and calculate the intersection set. The uniqueness constraint is to eliminate one-to-many matching points according to the one-to-one correspondence principle.
S3 specifically comprises the following contents: and obtaining a basic model through dense matching by taking the feature point pairs obtained through mutual matching as seed points. Specific: in the application, a surface patch-based method is adopted, and an accurate point cloud model with dense or quasi-dense object surface is obtained through diffusion and filtering based on sparse points obtained by matching characteristic points among multiple views.
S3 further comprises: when the basic model is constructed, the basic model is constructed according to the characteristic point pairs after the elimination.
S4 specifically comprises the following contents:
and during texture mapping, mapping the two-dimensional texture space to a preset intermediate curved surface according to the multi-source data, mapping the texture of the intermediate curved surface to a basic model, and reconstructing a fine model. Specific: in the application, two-step texture mapping is adopted, the texture mapping is carried out through two links, and the first link is mapped from a two-dimensional texture space to an intermediate curved surface expressed by available parameters; and the second link maps the texture of the intermediate curved surface to the object space to be reconstructed, namely, a fine model is generated.
And matching points among different images under multiple visual angles by using texture information to finish three-dimensional reconstruction. In the application, a two-step texture mapping method is adopted, so that the problems of forward texture mapping and reverse texture mapping are solved, the continuity, the single value and the reversibility of the texture mapping are ensured, and the high fidelity of the texture mapping of the three-dimensional model is better ensured.
Because of the multi-source data, when the texture is attached, a plurality of texture sources exist on the network model or in a certain grid area, when the texture is selected, the included angle between the normal line of the screening triangular grid and the connecting line from the grid center to the image photographing center is smaller than 90 degrees, and the triangular surface patch is visible in the image. Based on the visible triangular patches, when the same triangular patch appears on two or more images, screening the image with the largest corresponding texture area; if the same triangular patch has shielding on all the images, the image with the smallest shielding area is screened. The proper texture fitting can reflect the real scene and the real will, so that the minimum error and the minimum deformation are ensured, and the optimal texture mapping effect is achieved.
The model construction system based on the multi-source data, as shown in fig. 2, uses the model construction method based on the multi-source data, and comprises a server and various terminals.
The server is used for acquiring multi-source data acquired by various terminals, extracting characteristic points according to the multi-source data, and matching according to the extracted characteristic points to obtain characteristic point pairs; the server is also used for constructing a basic model according to the multi-source data and the characteristic point pairs, and generating a fine model by performing texture mapping on the basic model according to the multi-source data. The server is also used for eliminating the characteristic point pairs, eliminating the characteristic point pairs which are mismatched, and constructing according to the characteristic point pairs after eliminating when constructing the basic model. And in the texture mapping process, the server is also used for mapping the two-dimensional texture space to a preset intermediate curved surface according to the multi-source data, mapping the texture of the intermediate curved surface to the basic model, and reconstructing the fine model.
The multiple terminals comprise unmanned aerial vehicle acquisition equipment, ground vehicle-mounted equipment and backpack equipment, and the multi-source data comprise aviation data, ground data and mobile data.
The unmanned aerial vehicle acquisition equipment carries on multiple sensor, carries out low altitude inclined image acquisition from a plurality of angles by means of aerial survey remote sensing technology to carry out three-dimensional location, acquire aviation data. The aviation data comprise aviation images, aviation plane data and aviation elevation data, and the aviation plane data are two-dimensional longitude and latitude data.
In order to make up for the discontinuous image that the local ground object view caused because of shielding etc. when unmanned aerial vehicle navigates, avoid the incompleteness that single measurement data source exists, still set up ground on-vehicle equipment. The method comprises the steps that coordinates of a target are captured through a laser scanner of ground vehicle-mounted equipment, image data are obtained through a panoramic camera, the attitude and position information of a system at each moment are automatically recorded, ground data are obtained, the ground data comprise ground images, ground plane data and ground elevation data, and the ground plane data are two-dimensional longitude and latitude data.
Meanwhile, the backpack RTK camera is arranged, so that the backpack type RTK camera can be operated by a single person, and the backpack type RTK camera can be operated in a region where an automobile cannot enter, such as crowded relative personnel, complex environment and the like, so that mobile data can be acquired, wherein the mobile data comprises mobile images, mobile plane data and mobile elevation data, and the mobile plane data is two-dimensional longitude and latitude data.
The multi-source data uploading server is collected by various terminals and comprises a data acquisition module, a feature extraction module, a feature matching module, a feature eliminating module, a model construction module and a texture mapping module.
The data acquisition module is used for receiving and storing the uploaded multi-source data. The feature extraction module is used for calling the multi-source data and extracting feature points of the image data in the multi-source data according to the SIFT algorithm. The feature matching module is used for matching according to the extracted feature points so as to obtain feature point pairs.
The feature eliminating module is used for eliminating matched feature point pairs and eliminating mismatching feature point pairs. Specifically, the feature points which are erroneously matched are removed by adopting left-right consistency test and uniqueness constraint. In this embodiment, a feature point pair is obtained according to a nearest neighbor algorithm, and then the wrong feature point is removed by using a left-right consistency test and a uniqueness constraint, so as to obtain an initial matching result. The left-right consistency test is to search the feature point set in the corresponding left image and right image of the feature point pair and calculate the intersection set. The uniqueness constraint is to eliminate one-to-many matching points according to the one-to-one correspondence principle.
The model construction module is used for obtaining a basic model through dense matching by taking the characteristic point pairs obtained through mutual matching as seed points according to the characteristic point pairs after being removed. In this embodiment, a surface patch-based method is adopted, and based on sparse points obtained by matching feature points between multiple views, an accurate point cloud model with dense or quasi-dense object surfaces, namely a basic model, is obtained through diffusion and filtering.
The texture mapping module is used for performing texture mapping on the basic model according to the multi-source data to generate a fine model, and specifically: performing texture mapping through two links by adopting two-step texture mapping, wherein the first link is mapped from a two-dimensional texture space to an intermediate curved surface expressed by available parameters; and the second link maps the texture of the intermediate curved surface to the object space to be reconstructed, namely, a fine model is generated.
When the texture is attached, a plurality of texture sources exist on the network model or in a certain grid area, when the texture is selected, the included angle between the normal line of the triangular grid and the connecting line from the grid center to the image photographing center is smaller than 90 degrees, and the triangular surface patch is visible in the image. Based on the visible triangular patches, when the same triangular patch appears on two or more images, screening the image with the largest corresponding texture area; if the same triangular patch has shielding on all the images, the image with the smallest shielding area is screened.
In the application, various terminals are adopted to collect data, such as aeronautical data collected by unmanned aerial vehicle aerial survey, ground data collected by ground mobile vehicles and mobile data collected by personal mobile terminals. The defect of single data source is made up by data acquired by a plurality of terminals, the multi-source data acquired by a plurality of technical means are acquired in multiple angles and all aspects, the information of the target object is effectively, completely and truly reflected, the overall description of the target object is realized, and the real scenery of the earth surface is truly restored to the maximum extent.
Because of different data sources, the method processes the multi-source data, finds the most accurate characteristic point pair through the extraction, matching and elimination of the characteristic points, characterizes the target object through the characteristic points, and realizes the fusion of the multi-source data based on the characteristic point pair, thereby constructing a basic model. When the feature points are removed, the left and right consistency check is adopted, the matched feature point sets are searched, the intersection sets are obtained, the unique constraint is sampled again, the one-to-many feature points are removed according to the one-to-one correspondence principle, the incorrectly matched feature points are removed, the accuracy of the feature point pairs is guaranteed, and therefore the consistency of the final model and reality is guaranteed.
Example two
Because the acquired data volume is larger, and the acquired data volume is larger based on various technical means, the required model construction time, data processing and calculation time are longer. Therefore, the present embodiment is different from the first embodiment in that:
the model construction method based on the multi-source data further comprises the following steps:
the method comprises the steps of obtaining terrain data of a required modeling, wherein the terrain data comprise a terrain area, plane data and elevation data of the required modeling area.
Generating a sampling track according to the topographic data, specifically: carrying out endpoint recognition according to the terrain area to construct a sampling area, and planning a sampling track according to the sampling area; and establishing a sampling reference according to the plane data and the elevation data, and sampling the multi-source data according to the sampling reference.
Specifically, in this embodiment, the planning of the sampling trajectory specifically includes the following steps:
and carrying out endpoint recognition according to the terrain area to construct a sampling area, constructing a three-dimensional space coordinate system in the target area to be sampled, dividing the target area to be sampled into a plurality of cube spaces with preset length as side length according to the three-dimensional space coordinate system, and dividing the cube spaces with the preset length of 5 meters in the embodiment.
The three-dimensional cube space is classified according to the terrain area, the plane data and the elevation data, the classification comprises three types of object space, barrier-free space and barrier space, and importance marking is carried out on the object space according to the acquisition requirement.
Randomly selecting two barrier-free spaces as a starting point and a terminal point, and taking the coordinates of the corresponding barrier-free spaces as the coordinates of the starting point and the terminal point;
selecting an unobstructed space set with a space distance smaller than a preset distance from an unobstructed space, and taking the set as an observation point set;
randomly selecting a plurality of observation points from the observation point set; taking the starting point, the end point and the observation point as point positions of an initial sampling track, and connecting the point positions to generate the initial sampling track;
repeating the steps to generate a plurality of initial sampling tracks to form an initial sampling track set.
Constructing an adaptability function comprising track length, image acquisition profitability and operation complexity;
the track length is used for characterizing the total length of the path of each initial sampling track, and the track length fitness function is as follows:
in the formula ,、/>、/>spatial coordinates representing the ith point in the initial sampling trajectory,/->Representing the distance between the ith point and the (i+1) th point in the initial sampling track, n being the total number of points in the initial sampling track, +.>Is the track length.
The image acquisition profitability is used for representing effective image data which can be acquired by each initial sampling track; the image acquisition yield fitness function is as follows:
in the formula ,、/>、/>the method comprises the steps of taking the spatial coordinates of a position with a linear distance j from an ith point on a path between the ith point and an (i+1) th point in an initial sampling track; />、/>、/>The space coordinates of a target object space m closest to the position with the linear distance j from the ith point on the path between the ith point and the (i+1) th point in the initial sampling track;representing the effective acquisition radius of the camera; />The importance of the object space m nearest to the position of the i-th point on the path between the i-th point and the i+1-th point in the initial sampling track, which is at a linear distance j from the i-th point, is shown. />And collecting the profitability for the image.
The function of the operational complexity fitness is as follows:
in the formula ,is an operational complexity.
And according to the fitness function, calculating fitness function values of all the initial sampling tracks, and respectively sequencing the initial sampling tracks from high to low according to the three fitness function values to obtain sequencing sets of the three initial sampling tracks.
According to the fitness function value, a plurality of initial sampling tracks are randomly selected from the three sorting sets through roulette to form three groups in different directions, and the selected probability is in direct proportion to the fitness function value.
And selecting, crossing and mutating each group according to a genetic algorithm, and kicking out individual schemes which cannot be realized, and respectively iterating for preset times to form three groups after convergence.
And randomly selecting individuals from the three groups according to the fitness to form a unidirectional gene library, forming three groups of bidirectional cross groups between the three groups in a pairwise crossing way, calculating three fitness of each individual for each group of bidirectional cross groups, performing non-dominant sorting on the two fitness related to the bidirectional cross groups, selecting, crossing and mutating according to sorting results, and then kicking out an individual scheme which cannot be realized. Before crossing, a random jump back mechanism is adopted to select crossing objects:
wherein ,for crossing objects +.>For randomly selected individuals from the unidirectional gene library, <' > a ++>A random number between 0 and 1, < >>For the jump back threshold, in this implementation +.>0.1, in other embodiments of the application, the dynamic assignment is based on crossed individual ranks, the lower the individual ranks, the +.>The greater the value.
After the three groups of two-way cross groups are iterated for a preset number of times, the three groups are combined together, the three fitness functions are weighted and summed to obtain comprehensive fitness, individual selection, cross and mutation are carried out according to the comprehensive fitness, and an individual with the largest fitness is obtained as an optimal sampling track after the preset number of iterations.
The terrain area is usually irregular, when the sampling track is generated, the endpoint of the terrain area is identified, a regular sampling area is constructed according to the identified endpoint, and the sampling track is automatically planned according to the sampling area. The optimal sampling track of the route is designed, the track length, the image acquisition income and the operation complexity are comprehensively considered, the length of the route can be effectively reduced, the sampling data quantity is reduced, and the data processing quantity and the operation quantity of a user on the unmanned aerial vehicle are reduced.
At the same time, to reduce the data types required to be processed, the system calculation amount is reduced. A sampling reference is designed, for example, a sampling reference is established by taking plane data of longitude and latitude as a reference or elevation data, and multi-source data is sampled according to the sampling reference. Setting up a sampling reference to reduce the data type of feature processing, such as unified plane data or elevation data, thereby reducing the difficulty of data processing and reducing the calculation amount of the system.
And the multiple terminals sample according to the sampling track to obtain multi-source data.
The model construction system based on the multi-source data uses the model construction method based on the multi-source data.
The server is used for acquiring the topographic data of the required modeling and generating a sampling track according to the topographic data. And the multiple terminals sample according to the sampling track to obtain multi-source data.
The server also comprises a sampling planning module, wherein the sampling planning module is used for carrying out endpoint identification according to the terrain area to construct a sampling area and planning a sampling track according to the sampling area; and establishing a sampling reference according to the plane data and the elevation data, and sampling the multi-source data according to the sampling reference.
According to the application, the terrain area is usually irregular, when the sampling track is generated, the terminal point identification is carried out on the terrain area, the regular sampling area is constructed according to the identified terminal point, and the sampling track is automatically planned according to the sampling area. The shortest sampling track of the design route reduces the sampling data volume, thereby reducing the data processing volume.
At the same time, to reduce the data types required to be processed, the system calculation amount is reduced. A sampling reference is designed, for example, a sampling reference is established by taking plane data of longitude and latitude as a reference or elevation data, and multi-source data is sampled according to the sampling reference. Setting up a sampling reference to reduce the data type of feature processing, such as unified plane data or elevation data, thereby reducing the difficulty of data processing and reducing the calculation amount of the system.
The foregoing is merely an embodiment of the present application, and a specific structure and characteristics of common knowledge in the art, which are well known in the scheme, are not described herein, so that a person of ordinary skill in the art knows all the prior art in the application date or before the priority date, can know all the prior art in the field, and has the capability of applying the conventional experimental means before the date, and a person of ordinary skill in the art can complete and implement the present embodiment in combination with his own capability in the light of the present application, and some typical known structures or known methods should not be an obstacle for a person of ordinary skill in the art to implement the present application. It should be noted that modifications and improvements can be made by those skilled in the art without departing from the structure of the present application, and these should also be considered as the scope of the present application, which does not affect the effect of the implementation of the present application and the utility of the patent. The protection scope of the present application is subject to the content of the claims, and the description of the specific embodiments and the like in the specification can be used for explaining the content of the claims.
Claims (6)
1. The model construction method based on the multi-source data is characterized by comprising the following steps:
obtaining topographic data to be modeled, and generating a sampling track according to the topographic data; sampling by a plurality of terminals according to the sampling track to obtain multi-source data;
the terrain data comprises a terrain area, plane data and elevation data of a required modeling area, and a sampling track is generated according to the terrain data, and comprises the following contents:
carrying out endpoint recognition according to the terrain area to construct a sampling area, and planning a sampling track according to the sampling area;
establishing a sampling reference according to the plane data and the elevation data, and sampling multi-source data according to the sampling reference;
wherein planning a sampling trajectory from the sampling region comprises:
constructing a three-dimensional space coordinate system in a target area to be sampled, and dividing the target area to be sampled into a plurality of cube spaces with preset lengths as side lengths according to the three-dimensional space coordinate system;
classifying the three-dimensional cube space according to the terrain area, the plane data and the elevation data, wherein the classification comprises three types of a target space, an unobstructed space and an obstacle space, and marking the importance of the target space;
randomly generating a plurality of initial sampling tracks to form an initial sampling track set;
constructing a fitness function, wherein the fitness function comprises a track length fitness function, an image acquisition income fitness function and an operation complexity fitness function;
the track length fitness function is as follows:
in the formula ,、/>、/>spatial coordinates representing the ith point in the initial sampling trajectory,/->Representing the distance between the ith point and the (i+1) th point in the initial sampling track, n being the total number of points in the initial sampling track, +.>Is the track length;
the image acquisition yield fitness function is as follows:
in the formula ,、/>、/>the method comprises the steps of taking the spatial coordinates of a position with a linear distance j from an ith point on a path between the ith point and an (i+1) th point in an initial sampling track; />、/>、/>The space coordinates of a target object space m closest to the position with the linear distance j from the ith point on the path between the ith point and the (i+1) th point in the initial sampling track; />Representing the effective acquisition radius of the camera; />The importance of a target object space m closest to the position with the linear distance j from the ith point on the path between the ith point and the (i+1) th point in the initial sampling track is represented; />Benefit for image acquisition；
The function of the operational complexity fitness is as follows:
in the formula ,for operational complexity;
taking the initial sampling track set as an initial group, performing iterative optimization according to the fitness function to obtain an optimal sampling track, and completing planning of the sampling track;
the iterative optimization based on the fitness function comprises the following steps of:
according to the fitness function, calculating fitness function values of all initial sampling tracks, and respectively sequencing the initial sampling tracks from high to low according to the three fitness function values to obtain sequencing sets of the three initial sampling tracks;
according to the fitness function value, randomly selecting a plurality of initial sampling tracks from the three sorting sets through roulette to form three groups in different directions, wherein the selected probability is in direct proportion to the fitness function value;
selecting, crossing and mutating each group according to a genetic algorithm, and kicking out individual schemes which cannot be realized, and respectively iterating for preset times to form three iterated groups;
randomly selecting individuals from three groups according to fitness to form a unidirectional gene library, forming three groups of bidirectional cross groups by two-by-two crossing among the three groups, calculating three fitness of each individual for each group of bidirectional cross groups, performing non-dominant ranking on two fitness related to the bidirectional cross groups, selecting, crossing and mutating according to ranking results, and then kicking out an individual scheme which cannot be realized;
after the three groups of two-way cross groups are iterated for a preset number of times, combining the three groups together, carrying out weighted summation on the three fitness functions to obtain comprehensive fitness, carrying out individual selection, cross and mutation according to the comprehensive fitness, and iterating for the preset number of times to obtain an individual with the maximum fitness as an optimal sampling track;
acquiring multi-source data acquired by a plurality of terminals, extracting characteristic points according to the multi-source data, and matching according to the extracted characteristic points to obtain characteristic point pairs; the system comprises a plurality of terminals, a control system and a control system, wherein the plurality of terminals comprise unmanned aerial vehicle acquisition equipment, ground vehicle-mounted equipment and backpack equipment; the method for obtaining the characteristic point pairs by matching according to the extracted characteristic points comprises the following steps: extracting feature descriptors at the feature points, distributing direction values for the feature points, and searching matching points according to the feature descriptors so as to obtain matching point pairs; wherein the feature descriptor is a feature vector describing feature points;
constructing a basic model according to the multi-source data and the characteristic point pairs, and performing texture mapping on the basic model according to the multi-source data to generate a fine model, wherein the method comprises the following steps of:
the method comprises the steps of obtaining a basic model by densely matching characteristic point pairs obtained by mutual matching as seed points, obtaining an accurate object surface dense or quasi-dense point cloud model by diffusion and filtering on the basis of sparse points obtained by characteristic point matching among multiple views by adopting a surface patch-based method;
performing texture mapping through two links by adopting two-step texture mapping, wherein the first link is mapped from a two-dimensional texture space to an intermediate curved surface expressed by available parameters; and the second link maps the texture of the intermediate curved surface to the object space to be reconstructed to generate a fine model.
2. The multi-source data-based model construction method according to claim 1, wherein: and performing non-dominant sorting on two fitness related to the bi-directional cross group, and selecting, crossing and mutating according to sorting results, wherein before the bi-directional cross group crosses, a random jump-back mechanism is adopted to select a cross object:
wherein ,for crossing objects +.>For randomly selected individuals from the unidirectional gene library, <' > a ++>A random number between 0 and 1, < >>Is a jump back threshold.
3. The multi-source data-based model construction method according to claim 2, wherein: further comprises: according to the sorting order of the individuals to be crossedAssigning, the lower the individual ranking, the +.>The greater the value.
4. The multi-source data-based model construction method according to claim 1, wherein: the method also comprises the following steps:
and rejecting the matched characteristic point pairs, rejecting the mismatching characteristic point pairs, and constructing according to the rejected characteristic point pairs when constructing the basic model.
5. The multi-source data-based model construction method according to claim 4, wherein: rejecting the matched characteristic point pairs is as follows: and adopting left-right consistency test and unique constraint to remove the feature points which are erroneously matched.
6. A model building system based on multi-source data, using the model building method based on multi-source data according to any one of claims 1-5, comprising a server, characterized in that:
the server is used for acquiring multi-source data acquired by various terminals, extracting characteristic points according to the multi-source data, and matching according to the extracted characteristic points to obtain characteristic point pairs;
the server is also used for constructing a basic model according to the multi-source data and the characteristic point pairs, and generating a fine model by performing texture mapping on the basic model according to the multi-source data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310745909.9A CN116503627B (en) | 2023-06-25 | 2023-06-25 | Model construction system and method based on multi-source data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310745909.9A CN116503627B (en) | 2023-06-25 | 2023-06-25 | Model construction system and method based on multi-source data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116503627A CN116503627A (en) | 2023-07-28 |
| CN116503627B true CN116503627B (en) | 2023-09-26 |
Family
ID=87325032
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310745909.9A Active CN116503627B (en) | 2023-06-25 | 2023-06-25 | Model construction system and method based on multi-source data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116503627B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018126275A1 (en) * | 2016-12-30 | 2018-07-05 | Dirk Schneemann, LLC | Modeling and learning character traits and medical condition based on 3d facial features |
| CN108595560A (en) * | 2018-04-12 | 2018-09-28 | 北京建筑大学 | The methods of exhibiting and system of geographic information data |
| CN109895103A (en) * | 2019-01-21 | 2019-06-18 | 同济大学 | A kind of teaching playback track optimizing method based on GA-PSO algorithm |
| CN111696199A (en) * | 2020-06-14 | 2020-09-22 | 荆门汇易佳信息科技有限公司 | Ground-air fusion precise three-dimensional modeling method for synchronous positioning and mapping |
| CN113515650A (en) * | 2021-05-20 | 2021-10-19 | 贵州省交通科学研究院股份有限公司 | An intelligent aerial remote sensing measurement elevation point extraction system |
| CN113570647A (en) * | 2021-07-21 | 2021-10-29 | 中国能源建设集团安徽省电力设计院有限公司 | A spatial registration method of stereo objects between oblique photography and remote sensing optical images |
| CN114004848A (en) * | 2021-11-01 | 2022-02-01 | 辽宁工程技术大学 | A multi-view remote sensing terrain image point cloud reconstruction method |
| CN116129020A (en) * | 2023-02-09 | 2023-05-16 | 甘肃四维测绘工程有限公司 | Novel live-action three-dimensional modeling method |
| CN116149179A (en) * | 2022-12-09 | 2023-05-23 | 广州大学 | Non-uniform trajectory length differential evolution iterative learning control method for robotic fish |
-
2023
- 2023-06-25 CN CN202310745909.9A patent/CN116503627B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018126275A1 (en) * | 2016-12-30 | 2018-07-05 | Dirk Schneemann, LLC | Modeling and learning character traits and medical condition based on 3d facial features |
| CN108595560A (en) * | 2018-04-12 | 2018-09-28 | 北京建筑大学 | The methods of exhibiting and system of geographic information data |
| CN109895103A (en) * | 2019-01-21 | 2019-06-18 | 同济大学 | A kind of teaching playback track optimizing method based on GA-PSO algorithm |
| CN111696199A (en) * | 2020-06-14 | 2020-09-22 | 荆门汇易佳信息科技有限公司 | Ground-air fusion precise three-dimensional modeling method for synchronous positioning and mapping |
| CN113515650A (en) * | 2021-05-20 | 2021-10-19 | 贵州省交通科学研究院股份有限公司 | An intelligent aerial remote sensing measurement elevation point extraction system |
| CN113570647A (en) * | 2021-07-21 | 2021-10-29 | 中国能源建设集团安徽省电力设计院有限公司 | A spatial registration method of stereo objects between oblique photography and remote sensing optical images |
| CN114004848A (en) * | 2021-11-01 | 2022-02-01 | 辽宁工程技术大学 | A multi-view remote sensing terrain image point cloud reconstruction method |
| CN116149179A (en) * | 2022-12-09 | 2023-05-23 | 广州大学 | Non-uniform trajectory length differential evolution iterative learning control method for robotic fish |
| CN116129020A (en) * | 2023-02-09 | 2023-05-16 | 甘肃四维测绘工程有限公司 | Novel live-action three-dimensional modeling method |
Non-Patent Citations (2)
| Title |
|---|
| 基于相似特征点集的SIFT匹配改进算法;高放;陆频频;王旭;;测绘工程(第06期);22-26 * |
| 基于线路特征的电子地图匹配技术研究;刘尚兰;《中国优秀硕士学位论文全文数据库》(第02期);C034-616 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116503627A (en) | 2023-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Xiang et al. | Mini-unmanned aerial vehicle-based remote sensing: Techniques, applications, and prospects | |
| CN112085844B (en) | Unmanned aerial vehicle image rapid three-dimensional reconstruction method for field unknown environment | |
| CN110956651B (en) | Terrain semantic perception method based on fusion of vision and vibrotactile sense | |
| US9488492B2 (en) | Real-time system for multi-modal 3D geospatial mapping, object recognition, scene annotation and analytics | |
| Kuang et al. | Real-time UAV path planning for autonomous urban scene reconstruction | |
| JP2022077976A (en) | Image-based positioning methods and systems | |
| KR102557775B1 (en) | Drone used 3d mapping method | |
| CN103377476A (en) | Image registration of multimodal data using 3d geoarcs | |
| CN109920055A (en) | Construction method, device and the electronic equipment of 3D vision map | |
| KR20230026916A (en) | 3d mapping method with time series information using drone | |
| CN117873158A (en) | Unmanned aerial vehicle routing inspection complex route optimization method based on live-action three-dimensional model | |
| CN118608692A (en) | A method for constructing a four-dimensional base for digital twin cities based on four-dimensional space-time increments | |
| Bittner et al. | Long-short skip connections in deep neural networks for dsm refinement | |
| CN116433865A (en) | Space-ground collaborative acquisition path planning method based on scene reconstructability analysis | |
| Maurer et al. | Automated inspection of power line corridors to measure vegetation undercut using UAV-based images | |
| Wang et al. | LPRnet: A self-supervised registration network for LiDAR and photogrammetric point clouds | |
| Majdik et al. | Micro air vehicle localization and position tracking from textured 3d cadastral models | |
| CN116503627B (en) | Model construction system and method based on multi-source data | |
| US12196552B2 (en) | System and method for providing improved geocoded reference data to a 3D map representation | |
| Shahbazi | Professional drone mapping | |
| Zhang et al. | Improving accident scene monitoring with multisource data fusion under low-brightness and occlusion conditions | |
| Shen et al. | Feature extraction from vehicle-borne laser scanning data | |
| Sharma et al. | Image Acquisition for High Quality Architectural Reconstruction. | |
| Tang et al. | UAV Visual Localization Technology Based on Heterogenous Remote Sensing Image Matching | |
| CN117033817B (en) | Route determination method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |