Disclosure of Invention
First) solving the technical problems
The invention provides a method and a system for acquisition and immersive generation of a three-dimensional image of a site, which are used for solving the problem that the existing scanning technology can only provide surface geometric information and is difficult to identify a pattern missing area caused by material degradation.
Two) technical scheme
In order to achieve the purpose, the invention provides the following technical scheme that the three-dimensional image acquisition method for the ruins comprises the following steps:
Acquiring three-dimensional point cloud data generated after a three-dimensional scanner scans a site building;
acquiring reflection spectrum data of the surface of the remains building under a plurality of different wavelength illuminations at each scanning point while acquiring the three-dimensional point cloud data;
Comparing the reflection spectrum data of each scanning point to extract at least one low-reflectivity point, calculating the reflection spectrum similarity between each low-reflectivity point and each scanning point in the adjacent area, wherein the low-reflectivity point represents that the reflectivity of one scanning point is lower than the reflectivity of other scanning points in the corresponding color light wave band in a plurality of color light wave bands, the calculation of the reflection spectrum similarity is specifically to compare the reflectivity information of each scanning point in the adjacent area and calculate the reflection spectrum similarity, and
And forming a complete area by the low-reflectivity points and each scanning point with high similarity of reflection spectrum in the neighborhood of the low-reflectivity points, and marking the area in the three-dimensional point cloud data.
Further, the acquiring three-dimensional point cloud data generated after the site building is scanned by the three-dimensional scanner includes:
at each scanning position, a Cartesian coordinate system is constructed by taking the position of the three-dimensional scanner as an origin, wherein three axes in the coordinate system respectively represent the horizontal direction and the vertical direction in the scanning range and the distance information from a scanned object to the three-dimensional scanner;
the three-dimensional scanner calculates the distance between each laser reflection point on the site surface and the three-dimensional scanner by emitting pulse laser and calculating the flight time of the laser reflected from the building surface;
Combining the distance data of the laser reflection points of each site surface with the corresponding horizontal angle and vertical angle, calculating the three-dimensional coordinates of the points in the Cartesian coordinate system, recording each three-dimensional coordinate point as independent three-dimensional point cloud data, and constructing a point cloud model of the whole site surface together.
Further, while the three-dimensional scanning is performed, capturing a color image of the current site surface by a camera, matching pixel values in at least one of the color images for each scanning point in the three-dimensional point cloud data,
In the three-dimensional coordinate system, each three-dimensional point corresponds to at least one two-dimensional pixel coordinate;
Converting the two-dimensional pixel coordinates into coordinates of three-dimensional points by setting internal parameters of a camera;
and for each three-dimensional point, finding out the corresponding image pixel, acquiring the color value of the pixel, and then distributing the pixel to the three-dimensional point to form three-dimensional point cloud data with color information.
Further, while the three-dimensional scanning is performed, a plurality of different wavelengths of light are selected to illuminate the surface of the site building, and reflection spectrum data at different wavelengths are acquired at each scanning point, wherein the reflection spectrum data comprise reflectivity information in different color bands.
Further, after comparing and analyzing the reflection spectrum data of each scanning point, marking out a plurality of low-reflectivity points P i, and calculating the position coordinate (x i,yi,zi) of each low-reflectivity point P i in the three-dimensional coordinate system;
Defining a neighborhood taking the low-reflectivity point P i as a central point for each marked low-reflectivity point P i, and extracting the reflectivity information of each scanning point P j in the neighborhood;
For each scan point, the collected reflectance spectrum data is formed into a spectrum vector X containing the reflectivities in the plurality of colorbands:
Wherein, Representing the reflectivity of the scan point at wavelength lambda i;
Calculating the spectral vector of the low reflectivity point P i Spectral vector to a scan point P j in its fieldCosine similarity cos θ of (2):
wherein x·x low represents the inner product of two vectors:
The number of X and the number of X low are two respectively. Modulo of the individual vectors (i.e., vector length);
The value of cos θ is close to 1, indicating that the spectral vectors are highly similar, indicating that the scan point P j in the neighborhood is highly similar to the reflectivity information of the low reflectivity point P i in the different colorbands.
Further, if there is a scan point P j in the neighborhood with low reflectivity and similar reflectivity information to the low reflectivity point P i, the scan point P j is used as the new center point, and the expansion is continued;
redefining a neighborhood for the new center point and carrying out the reflection spectrum similarity calculation until the cosine similarity cos theta of the new center point and all scanning points in the neighborhood is lower than a set cosine similarity threshold value, and stopping expansion;
After the expansion is stopped, the scanning points P j with similar reflectivity information in all the low reflectivity points P i and the adjacent areas in the expanded area are combined to form a complete area, the area is defined as a pattern color missing area of the site surface, and the pattern color missing area is marked in the three-dimensional point cloud data.
A method for generating three-dimensional images of a site, comprising:
After carrying out point cloud noise reduction, point cloud alignment and point cloud simplification on three-dimensional point cloud data with image color information obtained after scanning the site building, converting the simplified point cloud data into triangular grids, namely generating a surface structure through the adjacent relation among points;
mapping image color information in the three-dimensional point cloud into the generated triangular mesh;
And combining reflection spectrum data of the surface of the remains building obtained at each scanning point for reflecting a plurality of different wavelengths with the surface attribute of each triangular mesh patch, and marking the areas formed by the scanning points with similar reflectivity information in the neighborhood and the low reflectivity points in the reflection spectrum in the triangular mesh.
A three-dimensional image acquisition system for a site, comprising:
a data acquisition end configured to:
Acquiring three-dimensional point cloud data generated after a three-dimensional scanner scans a site building;
Acquiring three-dimensional point cloud data generated after a three-dimensional scanner scans a site building;
acquiring reflection spectrum data of the surface of the remains building under a plurality of different wavelength illuminations at each scanning point while acquiring the three-dimensional point cloud data;
Comparing the reflection spectrum data of each scanning point, extracting at least one low-reflectivity point, calculating the similarity of the reflection spectrum of each low-reflectivity point and each scanning point in the adjacent area, and
And forming a complete area by the low-reflectivity points and each scanning point with high similarity of reflection spectrum in the neighborhood of the low-reflectivity points, and marking the area in the three-dimensional point cloud data.
A three-dimensional image immersive generation system for a site, comprising:
a data generating end configured to:
After carrying out point cloud noise reduction, point cloud alignment and point cloud simplification on three-dimensional point cloud data with image color information obtained after scanning the site building, converting the simplified point cloud data into triangular grids, namely generating a surface structure through the adjacent relation among points;
mapping image color information in the three-dimensional point cloud into the generated triangular mesh;
And combining reflection spectrum data of the surface of the remains building obtained at each scanning point for reflecting a plurality of different wavelengths with the surface attribute of each triangular mesh patch, and marking the areas formed by the scanning points with similar reflectivity information in the neighborhood and the low reflectivity points in the reflection spectrum in the triangular mesh.
Third), beneficial effects:
Compared with the prior art, the invention has the following beneficial effects:
The invention generates a site building three-dimensional model with geometric, color and spectral information by fusing three-dimensional point cloud, color image and spectral data, and finds out the region with obvious spectral difference based on the principle that the reflectivity of the missing or fading region is obviously reduced in a plurality of color wave bands due to the specific spectral reflection characteristic of mineral pigment (such as cinnabar, celadon and ocher) in the normal state, and marks the region as a pattern missing or pigment degradation region. Through the combination of the three-dimensional point cloud and the spectrum data, the three-dimensional structure and the color distribution of the building are presented, and the area where the pattern is missing or faded can be identified through analyzing the reflectivity, so that the full diagnosis and the display of the site building are realized.
Detailed Description
The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, but not all embodiments, and all other embodiments obtained by those skilled in the art without making creative efforts based on the embodiments of the present invention are included in the protection scope of the present invention.
In the description of the present invention, it should be understood that the terms "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. indicate orientations or positional relationships based on the drawings, are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation, be configured and operated in a specific orientation, and thus should not be construed as limiting the present invention.
In the process of carrying out three-dimensional space data scanning and recording on an ruined building, the geometric space structure of the ruined building is usually scanned and recorded by means of a scanning device such as a three-dimensional laser scanner, and on the basis of constructing a geometric model, if image information on the ruined building needs to be acquired, for example, a wall painting pattern on the ruined building needs to be acquired and combined in the constructed three-dimensional model, in the prior art, the whole wall painting is mostly scanned in regions by a camera, and then the whole wall painting is generated by utilizing an image splicing technology and attached to the three-dimensional model.
The absence of the mural pattern is not only represented by the absence of colors on the image, but also more of the time is the absence of the mural pattern caused by the degradation of the painting material, and the mural pigment can be peeled off or aged due to environmental factors such as weathering, humidity and the like. The existing three-dimensional scanning technology can only provide surface geometric information, and is difficult to identify pattern missing areas caused by material degradation.
Taking the Tibetan gully dynasty site as an example, the Gu Gewang dynasty architecture adopts local sandstone and shale, and the wall painting in the architecture adopts cinnabar, celadon stone and ocher. Rock materials used in construction are subject to long-term weathering, which can easily cause loosening and flaking of the surface, thereby affecting the mural pigment attached thereto. The pore structure of sandstones and shales makes them more prone to absorb moisture, which evaporates when temperature changes or exposed to wind, causing the stone to expand and contract, thereby accelerating the spalling of the paint on the surface of the fresco.
In addition, cinnabar is used as a red pigment, and mercury sulfide is the main component, and in a humid environment, the cinnabar is easy to undergo oxidation reaction, gradually changes color and even pulverizes, so that the red area of the mural is resolved or peeled off. The blue gold stone is used as blue pigment, the main component is a compound of the radalite and other minerals, the blue gold stone is sensitive to humidity, and can be decomposed due to long-term exposure to the humidity, the color is faded, and the pattern defect occurs along with the peeling of the substrate. Ocher is used as a brown or reddish brown pigment, and ocher as a natural iron oxide pigment, which reacts chemically under humid and acidic conditions, resulting in a dull or exfoliated color pigment. And the plateau environment at the Tibetan site has high radiation intensity, and cinnabar and other mineral pigments can fade or decompose gradually under the irradiation of ultraviolet rays, so that the aging process of the pigments can be accelerated, and the pattern is lost. In the high-sand area Gu Gewang towards the site, the surface of the mural is eroded by fine particles for a long time, so that the pigment gradually falls off.
According to the method and the system for collecting and immersively generating the three-dimensional images of the sites, pigment degradation caused by weathering and humidity shows different reflectivity changes at different wavelengths, pigment peeling or aging areas on the surfaces can be identified through multi-reflection spectrum data, and the pattern color missing areas are marked in the process of generating a conversion model through subsequent data so as to be used as repair references and immersive display.
Specifically, referring to fig. 1, fig. 1 is a flowchart of a method for capturing three-dimensional images of a ruined site according to an embodiment of the present invention, specifically:
And 101, acquiring three-dimensional point cloud data generated after the site building is scanned by a three-dimensional scanner. In some embodiments of the invention, the site building is scanned by a laser scanner, and the distance from each laser reflection point on the building surface to the scanner is obtained by measuring the flight time of laser emitted from the scanner to the surface reflection return, so as to generate three-dimensional point cloud data. And constructing a space coordinate system, and recording the position of each laser reflection point in a scanning area by the three-dimensional coordinates (x, y, z) of each laser reflection point to form a point cloud model in a three-dimensional space, so as to accurately reflect the geometric structure of the building.
Specifically, a Cartesian coordinate system is established at each scanning position by taking the position of the three-dimensional scanner as the origin of coordinates. Three axes in the coordinate system are used to represent the horizontal direction, the vertical direction of the scanner in the scanning range, and the distance (depth information) of the three-dimensional scanner to the reflection point on the target site surface, respectively. Wherein:
X axis is horizontal scanning direction;
y axis is vertical scanning direction;
z axis is depth information from the laser scanner to the scanned object.
The three-dimensional laser scanner determines the distance of each reflection point to the scanner by emitting a pulsed laser beam to the surface of the site building, then receiving the laser reflected back from the surface, and calculating the time of flight of the laser from emission to return.
Meanwhile, the laser scanner records the horizontal angle and the vertical angle during laser emission, and the horizontal angle and the vertical angle respectively represent the azimuth of the reflection point relative to the scanner. The spherical coordinates are converted to Cartesian coordinates by the horizontal angle, the vertical angle and the measured distance information. The three-dimensional coordinates (x, y, z) of each point can be converted by the following formula:
z=D cosθ
Wherein D is the distance, θ is the vertical angle, Is a horizontal angle.
The three-dimensional coordinates of each reflection point are recorded as an independent three-dimensional point. Along with the repeated scanning of the three-dimensional scanner at different positions of the site building, the three-dimensional points are gradually accumulated, and finally, a complete point cloud model of the whole site surface is built.
In some practical embodiments of the present invention, the RGB camera is used to capture color images of the site surface simultaneously with the laser scanner, and color information (RGB values) of each pixel is obtained.
And matching each pixel value of the color image with a corresponding point in the three-dimensional point cloud data. Two-dimensional pixel coordinates (u, v) in a color image are converted into points in a three-dimensional space through correction of internal and external parameters of a camera, and corresponding color information is allocated to each three-dimensional point.
Specifically, each laser reflection point (i.e., one point in the three-dimensional point cloud) has a corresponding three-dimensional coordinate (x, y, z) in the cartesian coordinate system, because the image captured by the camera is two-dimensional, and each pixel point in the image also has its corresponding two-dimensional pixel coordinate (u, v).
More specifically, the two-dimensional pixel coordinates corresponding to each three-dimensional point are determined by the internal and external parameters of the camera. The internal parameters describe the optical characteristics of the camera, mainly including focal length (focal length), principal point (PRINCIPAL POINT) position, radial distortion coefficient, etc., and the external parameters describe the position and orientation of the camera in three-dimensional space, including rotation matrices and displacement vectors, for converting points in three-dimensional space into two-dimensional pixels at the camera's perspective.
In some practical embodiments of the invention, three-dimensional coordinates x, y, z) may be mapped to two-dimensional pixel coordinates (u, v) of the camera image plane by using internal and external parameters of the camera.
Where K is the camera's internal reference matrix, R is the rotation matrix, and t is the displacement vector.
By mapping the three-dimensional point to the two-dimensional image pixel coordinates, the color value (e.g., RGB value) of the pixel is obtained.
The acquired color values are assigned to the three-dimensional points. Thus, each three-dimensional point carries not only its spatial coordinates, but also the color information of that point.
In summary, each point in the three-dimensional point cloud data is given corresponding color information to form a three-dimensional point cloud model with color. The three-dimensional point cloud data generated by the method not only can accurately describe the geometric form of the site building, but also can keep the color details on the surface, thereby realizing more real and detailed digital expression.
And 102, acquiring reflection spectrum data of the surface of the remains building under a plurality of different wavelength lights at each scanning point while acquiring the three-dimensional point cloud data. In some practical embodiments of the present invention, the spectral camera is configured to synchronously acquire the reflection spectrum data of each scanning point under different wavelength illumination. The spectral camera measures each pixel in a plurality of wavelength bands (e.g., ultraviolet, visible, infrared) and records its reflectivity at different wavelengths. These spectral data can help distinguish between different regions of a material, particularly when the color changes or material degradation are difficult for the naked eye to recognize, and spectral analysis can provide more detail.
It should be noted that in order to avoid different illumination interfering with the operation of the RGB camera and the spectral camera, in some embodiments of the invention, a uniform white LED light source is used, avoiding the use of a flash to affect the spectral camera, and standard red, green, blue filters are used in front of the RGB camera to separate the color information. By using a synchronous trigger mechanism, the three-dimensional laser scanner, the RGB camera and the spectrum camera can acquire data at the same time point, namely, in the scanning process, the three-dimensional laser scanner generates point cloud data, and simultaneously, the RGB camera and the spectrum camera respectively capture related color images and spectrum data.
In an embodiment with the Tibetan gully dynasty, the following are different paint colors and related reflective properties:
Cinnabar mainly contains mercury sulfide (such as HgS).
Cinnabar generally has high reflectivity in the red light band (about 620-750 nm) and lower reflectivity in the blue-green light band in the visible spectrum.
Referring specifically to fig. 5, fig. 5 is a graph showing the comparison of reflection spectra of a normal area and a severely degraded area of cinnabar pigment under the same illumination conditions and reflection distances according to an embodiment of the present invention.
Under normal conditions, the reflection spectrum of cinnabar (mercuric sulfide, hgS) shows strong reflection in the red light band (about 620-750 nm) of visible light, and has obvious absorption in the green band (about 500-550 nm) and the blue band (about 450-490 nm).
The Tibetan Gu Gewang is directed to the site plateau, the ultraviolet radiation intensity is high, the occurrence frequency of natural disasters such as sand wind is high, part of the mercuric sulfide (cinnabar) can be gradually converted into mercury oxide under the action of ultraviolet rays and oxygen, the chemical structure of the cinnabar is changed by the change, the reflection characteristic of the mercury oxide is different from that of the mercury sulfide, and the reflection capability of the mercury oxide in the visible light red light wave band is obviously reduced.
And pigment particles are decomposed or thinned due to the influence of sand weather. The size of the pigment particles directly affects its spectral reflectance characteristics, larger particles can reflect light of a specific wavelength band more effectively, and when the vermilion pigment particles are thinned, the reflectance of the red wavelength band is reduced, which means that the intensity of the reflection peak is weakened.
It can be understood that the vermilion pigment in the region of larger degradation degree has reduced reflection peak, especially the reflection rate in the red light wave band is reduced, and the changes destroy the original spectral characteristics of the vermilion, so that the reflection capacity in the red light wave band is reduced, and therefore, the region with significantly reduced reflection rate in the red light wave band is detected and compared, and the region can be judged as the fading and missing region of the pattern color.
The main component of the blue gold stone is a compound of the plagioclase and other minerals.
The blue-gold stone has high reflectivity in blue light and purple light wave bands (about 450-490 nm), has obvious reflection peaks in a reflection spectrum, and generally shows strong blue color.
Referring specifically to fig. 6, fig. 6 is a graph showing the comparison of reflection spectra of a normal region and a severely degraded region of a blue or white pigment under the same illumination conditions and reflection distances according to an embodiment of the present invention.
The spectral reflectance characteristics of the celadon stone are exhibited as strong reflectance in the blue band (about 450-490 nm) and as significant absorption in the red band (about 620-750 nm) and green band (about 500-550 nm) under normal conditions. This is due to the crystal structure of the main component of the blue calcite, and in particular the absorption and reflection properties of sulfide ions within it for light.
However, in the pattern color missing or degraded area, the blue reflection peak is reduced, and the reflectivity is reduced, mainly because ultraviolet rays can damage the chemical bonds of the pigment, and under the influence of the ultraviolet rays, the colored components in the celadon stone can be degraded by light, so that the reflection performance of the blue light wave band is weakened. As this photodegradation proceeds, the reflection peak in the blue band decreases, and at the same time, the reflection characteristics change and the reflectance decreases due to the damage of photodegradation to the surface chemical structure.
It will be appreciated that in areas of pattern colour missing or degraded, degradation of the blue-gold pigment will result in a reduction of the reflection peak in the blue band and a significant reduction in reflectance, and therefore by detecting areas of significantly reduced reflectance in the blue band compared to that, it can be determined that they are areas of colour fade and missing in the pattern.
Ocher (reddish brown) contains natural ferric oxide minerals as main ingredient.
The reflectivity is high in the red and green bands (about 580-640 nm), especially in the red band region.
Referring specifically to fig. 7, fig. 7 is a graph showing the comparison of reflection spectra of a normal region and a severely degraded region of ocher pigment under the same illumination conditions and reflection distances according to the embodiment of the present invention.
Under normal conditions, the optical properties of ocher are mainly reflected in the red and yellow bands (about 580-640 nm) with higher reflectivity, while in the blue band (about 450-490 nm) with lower absorptivity, the reddish yellow reflection of ocher being mainly due to the chemical properties of iron oxide. However, when the pattern color is missing or ocher is degraded, the absorptivity in the red band is significantly reduced, which indicates that the structure of ocher pigment is changed.
In areas where environmental impact is great, iron oxides are susceptible to further chemical reactions, producing iron oxides of other forms, such as hydrated iron oxides (FeO (OH)) or other secondary minerals. These newly formed compounds have different spectral reflectance characteristics compared to iron oxide (hematite), resulting in a decrease in reflectance in the red and yellow bands, and thus appear as a decrease in reflectance. This chemical transformation causes the pigment to fade or lose its reddish yellow appearance. The surface structure of the iron oxide is destroyed by the influence of ultraviolet rays for a long time, and the light reflection performance of the pigment is affected, which is shown by a significant decrease in the reflectance in the red wavelength band (about 620-720 nm).
It will be appreciated that in areas of missing or degraded pattern colour, degradation of the ocher pigment will result in a reduction of the reflection peaks in the red and yellow bands and a significant reduction in reflectance, so that by detecting areas of significantly reduced reflectance in the blue band compared to the contrast, it can be determined that they are areas of missing and missing pattern colour.
And 103, comparing the reflection spectrum data of each scanning point, extracting at least one low-reflectivity point, and calculating the reflection spectrum similarity between each low-reflectivity point and each scanning point in the adjacent area.
On the basis of the above, in some embodiments of the present invention, the acquired spectral data is processed to extract a scan point of low reflectivity, which is defined herein as a point having significantly lower reflectivity over a plurality of colored light bands (e.g., red, green, blue bands, etc.) than other scan points.
Specifically, the reflectance spectrum data of each scan point at different wavelengths is analyzed to find other scan points with significantly lower reflectance over multiple colorbands (e.g., red, green, blue, etc.) than the same region, and in some embodiments of the invention, a threshold is setOnly when a certain pixel satisfies low reflectivity in all critical bands (red, green, blue), it can be marked as a low reflectivity point, the condition satisfies the formula:
For each detected low reflectivity point, labeled P i, its position coordinates in the three dimensional coordinate system (x i,yi,zi) are recorded, these scanned points are used as potential areas of fading, missing or severe degradation of the pictorial material, and the extent of the missing areas is confirmed by subsequent similarity analysis.
Regarding similarity analysis, in particular, for each scan point, the spectral camera will measure the reflectivity of the pixel over a range of wavelengths, and the spectral data collected from each scan point will form a vector containing reflectivity information for a plurality of bands, the spectral vector being defined as:
wherein X is the spectral vector of a certain scanning point;
representing the reflectivity of the pixel at wavelength lambda i.
Taking the mural of the Tibetan gully dynasty remains as an example, three common pigments are cinnabar, celadon and ocher respectively. Their reflectivities at different bands appear differently, so that their spectral vectors have a significant difference in the value of the reflectivity at each band, and when the wall painting is faded or missing, the reflectivity is significantly reduced at all wavelengths, forming a low-reflectivity spectral vector.
More specifically, when pigment in the wall painting is degraded or the pattern is missing, the reflectance value of each component of the spectral vector, i.e., each band, such as red band of vermilion, blue band of celadon, red-yellow band of ocher, becomes significantly lower.
On the basis of extracting the low-reflectivity points P i, a neighborhood including a plurality of surrounding scanning points is defined by taking each low-reflectivity point P i as a central point, and the size of the neighborhood can be set according to actual requirements, and is generally determined according to the scanning density and the spatial distribution of the mural features.
Referring now to fig. 3, fig. 3 is a color image generated by scanning a wall painting in a daycare of the Tibetan gulfwtoward the ruins, referring now to fig. 4, a low-reflectivity point P i is identified at a point a in fig. 3, a field with the low-reflectivity point P i as a center point is defined, the farthest distance of 3 scanning points on a straight line is taken as a radius, so as to focus on local features, extract the reflection spectrum characteristics of each scanning point P j in the neighborhood, including a spectrum vector X of each scanning point, perform spectrum similarity analysis, and identify scanning points belonging to pattern deletion or fading in the neighborhood based on spectrum similarity.
Specifically, in some practical embodiments of the present invention, cosine similarity is used to analyze the similarity of spectral vectors between individual scan points. The cosine similarity only considers the directions among the spectrum vectors, and in the analysis of the mural patterns, although the reflectances of different scanning points have certain differences, the relative proportion of the points in each wave band is more important, so the cosine similarity can more effectively capture the shape change of a spectrum curve, but not the absolute reflectance value, and the error caused by the difference of the reflectances is avoided.
In addition, the spectral data of the mural scanning generally comprises a plurality of wave bands to form high-dimensional spectral vectors, cosine similarity has good performance when the high-dimensional vectors are processed, the similarity between the vectors can be calculated quickly, and the high calculation efficiency can be maintained no matter the number of the wave bands. In the analysis of the mural patterns, the reflectivity of adjacent scanning points fluctuates due to environmental factors (such as illumination change or instrument noise), and cosine similarity is focused on measuring the direction difference of vectors, so that the method has better robustness on the tiny fluctuations and does not influence similarity judgment due to small-amplitude reflectivity change. Since cosine similarity focuses more on the similarity of the shape of the spectral curve, in a local area, scan points similar to the spectral curve of low reflectivity points can be effectively identified, and this feature can help to quickly find other affected scan points in the missing or degraded area, contributing to the missing area range.
Under multiband (such as visible light and near infrared light) scanning, the spectral reflectivity of each scanning point is composed of reflectivities at different wavelengths to form a spectral vector, and the spectral vector of a certain scanning point P j in the neighborhood is set asThe spectral vector of the low-reflectivity point P i is Calculated by the following formula:
wherein X.X low represents the inner product of two vectors:
The |x| and |x low | are each the modulus (i.e., vector length) of the two vectors.
The value of cos θ is close to 1, indicating that the spectral vectors of the two scan points are highly similar, indicating that the scan point in the neighborhood is also a lower reflectance point, indicating that it is in a pattern missing or fading region.
The value of cos θ approaching 0 indicates a large difference in spectral vectors, indicating that the scanned spot has low similarity to the low reflectivity spot in the neighborhood, possibly while still maintaining the original pattern.
The value of cos θ is close to-1, indicating that the two spectral vectors are diametrically opposite.
And 104, forming a complete area by each scanning point with high reflection spectrum similarity in the low-reflectivity point and the neighborhood thereof, and marking the area in the three-dimensional point cloud data.
Based on the cosine similarity calculation, the neighborhood is further expanded by continuously taking the newly detected high similarity point as a new center point P' i, and the similarity analysis is repeated until a complete missing region is found.
More specifically, a cosine similarity calculation is performed on the spectral vector of each scan point P 'j in the neighborhood of the new center point P' i, and the calculated similarity value, i.e., the value of cos θ, is compared with a set cosine similarity threshold T. In the embodiment of the present invention, the cosine similarity threshold T is used to determine the spectrum similarity between the new center point P 'i and the other scanning points P' j in the neighborhood, and the size of the T value may be set according to the actual operation requirement, which is not limited herein specifically.
If the new center point P ' i is similar to a certain scan point P ' j in the neighborhood, cos θ > T, the scan point P ' j is considered to be a point with similar spectral characteristics to the new center point P ' i and added to the area to be analyzed, while the point is set as the new center point P ' i.
If cos θ is less than or equal to T, the scan point P' j is no longer added to the area to be analyzed.
Selecting a new center point (such as point P' j), repeating the steps, and calculating the similarity of other scanning points in the neighborhood of the new center point:
Continuing the expansion, if the similarity value is still greater than the threshold T, continuing to expand the neighborhood.
Stopping the expansion, wherein the similarity of all scanning points in the domain is smaller than or equal to a threshold T, and the boundary of the area is determined.
Once the expansion has ceased, all scan points similar to the original low reflectivity points are recorded, which constitute areas where there is a loss or discoloration.
Through the area expansion method based on cosine similarity, missing or fading areas in the wall painting can be effectively identified, boundaries of the missing or fading areas are accurately defined, and the missing or fading areas are marked in the collected and processed three-dimensional point cloud data. In practical application, the method can help identify the range of the mural area to be repaired, so that related researchers can better understand and maintain historical heritage.
The method for generating the three-dimensional image immersion of the ruins provided by the embodiment of the invention is used for modeling the data acquired by the scanning of the ruins, and concretely comprises the following steps of:
Step 201, performing point cloud noise reduction, point cloud alignment and point cloud simplification on three-dimensional point cloud data with image color information obtained after the site building is scanned, and converting the simplified point cloud data into triangular grids, namely generating a surface structure through the adjacent relation among points. In some practical embodiments of the present invention, the three-dimensional point cloud data with image color information obtained by scanning is noise reduced, because some noise points may be generated during the scanning process, and these points do not belong to the object surface and need to be removed by an algorithm. If multiple perspectives are used when scanning an object or scene, it is necessary to align the point cloud data at different angles (e.g., using the ICP algorithm) to generate a complete model. Each point of the point cloud data is independent, there is no direct connection between the points (e.g., edges between triangular patches), so it cannot directly represent the surface of an object, but rather represents a series of discrete points, and the point cloud can be transformed into a continuous surface model by a surface reconstruction algorithm (e.g., poisson Surface Reconstruction), as shown in fig. 8, which is a triangular mesh model generated from one three-dimensional point cloud data.
It can be appreciated that the simplified three-dimensional point cloud data is used to construct a triangular mesh according to the adjacency between points. The generation of the triangular meshes enables the geometric structure of the surface of the site building to be more coherent, the details can be better displayed through the surface patches, and the digital reconstruction of the surface structure of the site is realized through the triangular meshes, so that the site building has a vivid geometric form and is suitable for immersive experience.
And 202, mapping the image color information in the three-dimensional point cloud into the generated triangular mesh.
Specifically, color information (acquired from a color image) in the original three-dimensional point cloud is accurately mapped onto the generated triangular mesh patches, so that each mesh patch not only carries geometric information, but also has surface color attributes, and the model is ensured to visually present a real site appearance.
And 203, combining reflection spectrum data of the surface of the remains building obtained at each scanning point, reflecting a plurality of different wavelengths, with the surface attribute of each triangular mesh patch, and marking the areas formed by the low-reflectivity points in the reflection spectrum and the scanning points with similar reflectivity information in the neighborhood of the low-reflectivity points in the reflection spectrum in the triangular mesh.
Specifically, the spectral data of the previously collected site surface reflected light rays with different wavelengths are combined, and the reflection spectral characteristics of each grid surface patch are analyzed. The spectral data provides additional information about the texture of the surface and can be analyzed to identify areas of wall painting pattern color loss by comparing the reflectivity of the various colorbands.
It will be appreciated that by the cosine similarity based region extension method described above, areas of missing or faded patterns in the wall painting are identified and accurately bordered and marked on a three-dimensional model for display or reference analysis by a researcher.
Based on the above, it can be appreciated that, by importing the processed triangular mesh model with the color information and the spectral mark into an immersive display system, such as a Virtual Reality (VR) or Augmented Reality (AR) system, a user can view the area where the wall painting pattern marked after spectral analysis is missing or faded in a three-dimensional space, immersively experience the real state of the site building, and understand details such as degradation, damage and the like of the surface material of the site building.
Through the combination of the point cloud and the spectrum data, the three-dimensional geometric structure of the site building can be reconstructed, and the pattern color missing condition can be identified through spectrum analysis.
The above description is only a preferred embodiment of the present invention, and the patent protection scope of the present invention is defined by the claims, and all equivalent structural changes made by the specification and the drawings of the present invention should be included in the protection scope of the present invention.