CN116740300B - Multi-mode-based prime body and texture fusion furniture model reconstruction method - Google Patents

Abstract

The present invention discloses a furniture model reconstruction method based on multi-modal body and texture fusion. The method comprises: obtaining furniture sparse point cloud and texture embedding vector; inputting the furniture sparse point cloud and the texture embedding vector into a body and texture fusion model to obtain a new texture furniture model. The present invention enables users to complete the texture update of furniture models without the need for professional furniture designers, greatly saving time, and the original data only needs to provide two-dimensional picture data, which users can obtain using ordinary cameras.

Description

A furniture model reconstruction method based on multimodal body and texture fusion

Technical Field

The present invention belongs to the technical field of three-dimensional reconstruction based on multi-modality, and in particular relates to a furniture model reconstruction method based on multi-modality body and texture fusion.

Background technique

With the development of Internet technology and the increasing demand for intelligent furniture industry, furniture merchants can display various furniture renderings through online media without the need for customers to go to the site to view the real furniture in person, which greatly saves time, facilitates merchants and customers, and to a certain extent drives the efficiency of the furniture industry. However, a furniture model requires professional furniture designers to spend a lot of time using furniture modeling software such as blender, 3ds max and C4D to complete, which greatly limits the efficiency of furniture model launch. Therefore, merchants and users need a fast and accurate method to achieve furniture model reconstruction.

In traditional computer vision methods, multiple 2D images or videos are usually used to build a 3D model. For example, the pixel coordinates in multiple images or video frames can be converted into 3D coordinate point clouds using multi-view geometry methods, and then these point cloud data can be used to generate furniture models. In computer vision methods using deep learning, neural network models are usually used to predict the three-dimensional model of furniture from 2D images or point cloud data. For example, NeRF neural network volume rendering technology can achieve good results, but the model training time is too long and lacks generalization ability. The Instant-NGP neural network creatively introduces hash coding into the neural network, which improves the speed of model reasoning, but the surface texture of the exported three-dimensional model is not good and the model consumes too much video memory, so the NeRF neural network is still not well applied in the furniture field.

Summary of the invention

In order to solve at least one of the technical problems mentioned above, the present invention provides a furniture model reconstruction method based on multimodal body and texture fusion to solve the technical problem of merchants and users needing a fast and accurate method to achieve furniture model reconstruction.

A furniture model reconstruction method based on multi-modal body and texture fusion, comprising:

Get furniture sparse point cloud and texture embedding vector;

The furniture sparse point cloud and the texture embedding vector are input into the body and texture fusion model to obtain a new texture furniture model.

Preferably, the body and texture fusion model includes:

The body encoder is used to generate the spatial coordinate point potential vector based on the furniture sparse point cloud;

A voxel decoder is used to generate a voxel point feature vector and a UV coordinate feature vector according to a spatial coordinate point potential vector;

A texture encoder, for generating a texture feature vector according to the texture embedding vector;

The feature fusion module is used to generate a UV mapping feature vector according to the texture feature vector and the UV coordinate feature vector, and to generate a new texture furniture model according to the UV mapping feature vector and the body point feature vector.

Preferably, the furniture sparse point cloud and texture embedding vector are input into the body and texture fusion model to obtain a new texture furniture model, which includes:

Input the furniture sparse point cloud into a body encoder to obtain a potential vector of a spatial coordinate point;

Input the spatial coordinate point potential vector into a body decoder to obtain a body point feature vector and a UV coordinate feature vector;

Inputting the texture embedding vector into a texture encoder to obtain a texture feature vector;

Input the texture feature vector and the UV coordinate feature vector into a feature fusion module to obtain a UV mapping feature vector;

The UV map feature vector is added to the body point feature vector, and a new texture furniture model is obtained by passing through a convolutional network and using the open3D extension library.

Preferably, the texture embedding vector is input into a texture encoder, and a texture feature vector is output, comprising:

The texture with position information is embedded in the vector. After layer normalization, the layer normalization result and the multi-head attention calculation result are residually connected to obtain the texture potential vector:

_E'patch ＝MSA(LN( _Epatch )+ _Epatch

Where _E'patch is the texture potential vector, and _Epatch is the texture embedding vector with position information;

The texture potential vector is normalized and fully connected layer, and then the residual calculation is performed on the layer normalization and fully connected layer calculation results to output the texture feature vector:

T＝MLP(LN( _E'patch ))+ _E'patch

Where T is the texture feature vector and _E'patch is the texture latent vector.

Preferably, the spatial coordinate point potential vector is input into a body decoder to obtain a body point feature vector and a UV coordinate feature vector, including:

Input the spatial coordinate point potential vector into the voxel decoder to find the set of neighbor node potential vectors of any point in the space, where the neighbor nodes are calculated as the 16 nodes with the closest Euclidean distance between the spatial coordinate point potential vector and the point cloud vector of any point in the space. The Euclidean distance is:

Where _xp refers to the three-dimensional coordinates of the neighbor node, _yp refers to the three-dimensional coordinates of the neighbor node, _zp refers to the three-dimensional coordinates of the neighbor node, _xq refers to the three-dimensional coordinates of any point in space, _yq refers to the three-dimensional coordinates of any point in space, and _zq refers to the three-dimensional coordinates of any point in space;

Use interpolation to get the potential vector of any point in the space:

Where Z _q refers to the potential vector of any point in space, p _i ∈ N _q refers to the set of potential vectors of neighboring nodes, and s _i ∈ {S _q } refers to the distance weight;

After the potential vector of any point in the space passes through a fully connected layer, a feature vector of the body point is obtained, and multiple feature vectors of the body point constitute a body model;

After the potential vector of any point in the space passes through a fully connected layer, the UV coordinate feature vector of any point in the space is obtained.

Preferably, the texture feature vector and the UV coordinate feature vector are input into a feature fusion module to obtain a UV mapping feature vector, including:

The texture feature vector and the UV coordinate feature vector are added and input into the convolutional network. The convolutional network backbone uses ResNet34 to obtain the UV map feature vector.

Preferably, after obtaining the furniture sparse point cloud, the method further includes enhancing the furniture sparse point cloud data:

Perform high-frequency transformation on the furniture sparse point cloud:

in, For a furniture sparse point cloud, /> is a set of real numbers, and L is a high frequency variability.

Preferably, after the texture feature vector and the UV coordinate feature vector are input into a feature fusion module to obtain the UV map feature vector, the method further includes calculating and checking the similarity between the UV map and the texture image, specifically:

Input the UV mapping feature vector into the torchvision extension library, use the transform_convert() method to output the UV mapping result, and use the mean square error MSE based on the Gram matrix as the loss function:

Where C refers to the number of channels of the feature map, H refers to the height of the feature map, W refers to the width of the feature map, _Gi (Gen) refers to the Gram matrix on the feature map of the i-th layer of the UV map, and _Gi (Style) refers to the Gram matrix of the texture image on the i-th layer of the feature map. Where _Ci is the feature map after the i-th layer of the neural network,/> is the transposed matrix of _Ci .

A furniture model reconstruction system based on multi-modal body and texture fusion, comprising:

Data acquisition module, used to obtain furniture sparse point cloud and texture embedding vector;

The data processing module is used to input the furniture sparse point cloud and the texture embedding vector into the body and texture fusion model to obtain a new texture furniture model.

The present invention uses a furniture reconstruction method based on multi-modal voxel and texture fusion, the furniture sparse point cloud is input into a voxel encoder and a voxel decoder to obtain a voxel point feature vector and a UV coordinate feature vector, the texture image is pre-processed and input into a texture encoder to obtain a texture feature vector, the UV coordinate feature vector and the texture feature vector are subjected to feature fusion to obtain a UV map and a UV map feature vector, the voxel point feature vector is subjected to an open3d expansion library to obtain a voxel model, the voxel point feature vector and the UV map feature vector are subjected to feature fusion to obtain a new texture furniture model. The method allows users to complete the texture update of furniture models without the need for professional furniture designers, greatly saving time, and the original data only needs to provide two-dimensional picture data, which users can obtain using ordinary cameras.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the background technology, the drawings required for use in the embodiments of the present invention or the background technology will be described below.

The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions disclosed in the present invention.

FIG1 is a flow chart of a method for reconstructing a furniture model based on multi-modal body and texture fusion in an example of the present invention;

FIG2 is a flow chart of obtaining a sparse point cloud of furniture in an example of the present invention;

FIG3 is a structural diagram of a body and texture fusion module in an example of the present invention;

FIG4 is a structural diagram of a body encoder in an example of the present invention;

FIG5 is a structural diagram of a body decoder in an example of the present invention;

FIG6 is a structural diagram of a texture encoder in an example of the present invention;

FIG7 is a structural diagram of a feature fusion module in an example of the present invention;

FIG8 is a diagram showing the reconstruction effect of a body in an example of the present invention;

FIG. 9 is a rendering of a furniture model with texture update in an example of the present invention.

Detailed ways

In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

The term "and/or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and/or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

In addition, in order to better illustrate the present invention, numerous specific details are provided in the following specific embodiments. It should be understood by those skilled in the art that the present invention can be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present invention.

With the development of furniture and the popularization of the Internet, furniture dealers can display various furniture effects through online media. Customers no longer need to go to the furniture market in person to view the actual furniture, which greatly saves time. However, to make a furniture model for customers to refer to on the Internet, professional furniture designers or modeling engineers need to spend a lot of time on 3D modeling. Merchants need a new method to improve the efficiency of furniture modeling.

In traditional computer vision methods, multiple 2D images or videos are usually used to build a 3D model. For example, using multi-view geometry methods, pixel coordinates in multiple images or video frames are converted into 3D points, and then these point cloud data are used to generate furniture models. Selecting important features from each image is a necessary step, and as the number of categories increases, feature extraction becomes more and more troublesome. In computer vision methods using deep learning, neural network models are usually used to predict the 3D model of furniture from 2D images or point cloud data. For example, NeRF neural network volume rendering technology can achieve good results, but the model takes too long to train and lacks generalization ability. Instant-NGP creatively introduces hash coding into the neural network, which improves the speed of model reasoning, but the surface texture of the exported 3D model is not good and the model consumes too much video memory, so it is still not well applied in the furniture field. Nowadays, the improvement of computing power and the abundance of data have led to the rapid development of deep learning. Among them, multimodal neural networks refer to neural networks that can process multiple input data types. The present invention uses furniture sparse point clouds and texture images as input, and predicts the body model, UV map and new texture furniture model through the neural network, which can greatly save the time of furniture designers in making furniture models, greatly saving time, and the original data only needs to provide two-dimensional image data, which users can obtain using ordinary cameras.

The present invention uses furniture sparse point cloud and texture pictures as input, and predicts the body model, UV map and new texture furniture model through a neural network, which can greatly save the time of furniture designers in making furniture models, greatly saving time, and the original data only needs to provide two-dimensional picture data, which users can obtain using an ordinary camera.

Example 1

A furniture model reconstruction method based on multi-modal body and texture fusion, referring to FIG1, comprises:

S100 obtains furniture sparse point cloud and texture embedding vector, as shown in Figure 2. Take at least 20 original furniture pictures as the furniture picture set, and ensure that the shift angle between two adjacent pictures does not exceed 45°, otherwise the pose information cannot be constructed, and the furniture sparse point cloud cannot be output. Use SFM software colmap to process and obtain the furniture sparse point cloud;

Create a new project in colmap, create a project database, and import the furniture picture collection;

Use the feature extraction function to extract features, use the feature match function to match features, and generate scene graphs and matching matrices;

Use the start reconstruction function to perform incremental reconstruction. The colmap graphical user interface will generate the furniture sparse point cloud of the scene and the camera poses of each viewpoint, obtain the furniture sparse point cloud reconstruction result, and set the point cloud information to only contain spatial coordinates and grayscale values;

The furniture sparse point cloud reconstruction result is saved as text to obtain the furniture sparse point cloud. The dimension of each point cloud data is 4 dimensions, namely (x, y, z, g); x, y, z are the point coordinates in the world coordinate system, and g is the gray value of the point cloud;

Select a texture image that needs to be attached to the surface of the furniture, scale the texture image to 224 pixels in length and 224 pixels in width, set it to RGB3 channels, and get a standard texture image;

The standard texture image is divided into patches with a length of 16 pixels and a width of 16 pixels. A standard texture image can get 196 patches, and the dimension of each patch is 768;

The obtained 196 patches are input into the fully connected layer. The input node of the fully connected layer is 768, the output node is 768, and the output is a patch vector [196, 768] without position information.

The patch vector without position information is added with a one-dimensional position code. The one-dimensional position code is directly added to the patch vector without position information. E _pixel is the one-dimensional position code. Get the texture embedding vector E _patch ;

S200 inputs the furniture sparse point cloud and texture embedding vector into the body and texture fusion model to obtain a new texture furniture model, as shown in Figure 9.

Preferably, the body and texture fusion model includes:

The body encoder is used to generate the spatial coordinate point potential vector based on the furniture sparse point cloud;

A voxel decoder is used to generate a voxel point feature vector and a UV coordinate feature vector according to a spatial coordinate point potential vector;

A texture encoder, for generating a texture feature vector according to the texture embedding vector;

A feature fusion module is used to generate a UV mapping feature vector according to a texture feature vector and a UV coordinate feature vector, and to generate a new texture furniture model according to the UV mapping feature vector and a body point feature vector;

A fully connected layer-based body encoder is used to generate spatial coordinate point latent vectors based on the furniture sparse point cloud;

Interpolation-based voxel decoder, used to generate voxel point feature vectors and UV coordinate feature vectors based on spatial coordinate point latent vectors;

A multi-head attention-based texture encoder is used to generate a texture feature vector from a texture embedding vector;

The feature fusion module based on the convolutional neural network, as shown in FIG7 , is used to generate a UV mapping feature vector according to the texture feature vector and the UV coordinate feature vector, and to generate a new texture furniture model according to the UV mapping feature vector and the body point feature vector.

Preferably, S200 inputs the furniture sparse point cloud and texture embedding vector into the body and texture fusion model to obtain a new texture furniture model, referring to FIG3 , including:

S210 inputs the furniture sparse point cloud into the body encoder, as shown in Figure 4, and obtains the potential vector of the spatial coordinate point:

The sparse point cloud data of the enhanced furniture is input into the body encoder. The body encoder is an 8-layer fully connected network. The number of neurons in the 1st to 7th layers is 256, and the number of neurons in the 8th layer is 125. Residual connections are made in the 1st and 4th fully connected layers. Finally, the network outputs a potential vector with a dimension size of 125.

The spatial coordinates of each furniture sparse point cloud and the corresponding latent vector are concatenated once, and the dimension is increased to 128 dimensions to obtain the latent vector of the spatial coordinate point;

S220 inputs the spatial coordinate point potential vector into the body decoder to obtain the body point feature vector and the UV coordinate feature vector;

S230 inputs the texture embedding vector into a texture encoder to obtain a texture feature vector;

S240 inputs the texture feature vector and the UV coordinate feature vector into a feature fusion module to obtain a UV mapping feature vector;

S250 adds the UV map feature vector to the body point feature vector, passes through the convolutional network, and uses the open3D extension library to obtain a new texture furniture model;

The UV map feature vector T _q and the body feature vector R _q pass through the convolutional network. The convolutional network backbone uses ResNet34 and outputs a new texture furniture model through the open3d extension library.

Preferably, S230 texture embedding vector is input into a texture encoder, and a texture feature vector is output, referring to FIG6 , including:

The texture with position information is embedded in the vector E _patch . After layer normalization, the layer normalization result and the multi-head attention calculation result are residually connected to obtain the texture potential vector E' _patch :

_E'patch ＝MSA(LN( _Epatch )+ _Epatch

Where _E'patch is the texture potential vector, and _Epatch is the texture embedding vector with position information;

The texture potential vector _E'patch is normalized and fully connected. The residual calculation is performed on the layer normalization and fully connected layer calculation results to output the texture feature vector T:

T＝MLP(LN( _E'patch ))+ _E'patch

Where T is the texture feature vector and _E'patch is the texture latent vector.

Preferably, S220 inputs the spatial coordinate point potential vector into the body decoder to obtain the body point feature vector and the UV coordinate feature vector, referring to FIG5 , including:

The spatial coordinate point potential vector is input into the volume decoder, which needs to construct the point cloud vector Z _q of any point q in the space except the furniture sparse point cloud. Z _q is a 128-dimensional vector that currently only contains spatial coordinates. The first 3 dimensions are spatial coordinates, and the rest are all set to 0.

Input the spatial coordinate point potential vector into the voxel decoder and find the neighbor node potential vector set p _i ∈ {N _q } of any point q in space. The neighbor nodes are calculated as the 16 nodes with the closest Euclidean distance between the spatial coordinate point potential vector and the point cloud vector Z _q of any point q in space. The Euclidean distance is:

Where _xp refers to the three-dimensional coordinates of the neighbor node, _yp refers to the three-dimensional coordinates of the neighbor node, _zp refers to the three-dimensional coordinates of the neighbor node, _xq refers to the three-dimensional coordinates of any point in space, _yq refers to the three-dimensional coordinates of any point in space, and _zq refers to the three-dimensional coordinates of any point in space;

Use interpolation to get the potential vector at any point in space:

Where Z _q refers to the potential vector of any point in space, p _i ∈ N _q refers to the set of potential vectors of neighboring nodes, and s _i ∈ {S _q } refers to the distance weight;

Where s _i : ⊙ is the hardmard product, which is a learnable distance weight. For a query point q, the closer the neighbor node is to it, the larger the norm of the weight ||s _i || ₂ is, which means that the potential vector of the neighbor node is more closely related to the query point q.

After the potential vector Z _q of any point q in the space passes through a fully connected layer, the feature vector R _q of the body point is obtained. Multiple feature vectors R _q of the body point constitute the body model.

After the potential vector Z _q of any point q in the space passes through a fully connected layer, the UV coordinate feature vector C _q of any point in the space is obtained.

Preferably, S240 inputs the texture feature vector and the UV coordinate feature vector into a feature fusion module to obtain a UV mapping feature vector, as shown in FIG7 , including:

The texture feature vector T and the UV coordinate feature vector C _q are added and input into the convolutional network. The convolutional network backbone uses ResNet34 to obtain the UV map feature vector T _q ;

The UV map feature vector T _q and the body feature vector R _q pass through the convolutional network. The convolutional network backbone uses ResNet34 and outputs a new texture furniture model through the open3d extension library.

Preferably, the plurality of element point feature vectors constitute an element model through an open3d extension library, including:

Use Chamfer Loss as the loss function to generate the body model, as shown in Figure 8, specifically:

Where _pi is the i-th point in the body point cloud, _qj is the j-th point in the real point cloud, _Np is the number in the body point cloud, and _Ngt is the number in the real point cloud.

Preferably, S200 inputs the furniture sparse point cloud and texture embedding vector into the body and texture fusion model to obtain a new texture furniture model, and further includes training the texture and body fusion model, specifically:

The IKEA 3D Models dataset is used to obtain furniture sparse point clouds, and the CUReT dataset is used as a texture image to obtain texture embedding vectors.

The IKEA 3D Models dataset furniture sparse point cloud and the CUReT dataset texture image texture embedding vectors are input into the volume and texture fusion model;

The body and texture fusion model outputs a new texture furniture model;

Iterate training until the model converges.

Preferably, after obtaining the furniture sparse point cloud in S100, the method further includes enhancing the sparse point cloud data in S110:

Perform high-frequency transformation on the furniture sparse point cloud:

in, For a furniture sparse point cloud, /> is a set of real numbers, L is a high frequency variability;

The high-frequency variation L is set to 3, and the point cloud data dimension is increased from 4 to 24, obtaining enhanced furniture sparse point cloud data.

Preferably, after inputting the texture feature vector and the UV coordinate feature vector into the feature fusion module to obtain the UV map feature vector, S240 also includes calculating and checking the similarity between the UV map and the texture image, specifically:

Input the UV map feature vector T _q into the torchvision extension library, use the transform_convert() method to output the UV map result. The UV map is essentially the style transfer of the texture image, so to calculate the similarity between the UV map and the texture image, use the mean square error MSE based on the Gram matrix as the loss function:

Where C refers to the number of channels of the feature map, H refers to the height of the feature map, W refers to the width of the feature map, _Gi (Gen) refers to the Gram matrix on the feature map of the i-th layer of the UV map, and _Gi (Style) refers to the Gram matrix of the texture image on the i-th layer of the feature map. Where _Ci is the feature map after the i-th layer of the neural network,/> is the transposed matrix of _Ci .

Example 2

A furniture model reconstruction system based on multi-modal body and texture fusion, comprising:

Data acquisition module, used to obtain furniture sparse point cloud and texture embedding vector;

The data processing module is used to input the furniture sparse point cloud and texture embedding vector into the body and texture fusion model to obtain a new texture furniture model.

Claims (6)

1. A method for reconstructing a furniture model based on multi-modal body and texture fusion, characterized by comprising: Get furniture sparse point cloud and texture embedding vector; Inputting the furniture sparse point cloud and the texture embedding vector into a volume and texture fusion model to obtain a new texture furniture model; The body and texture fusion model includes: a body encoder, which is used to generate a spatial coordinate point potential vector according to the furniture sparse point cloud; a body decoder, which is used to generate a body point feature vector and a UV coordinate feature vector according to the spatial coordinate point potential vector; a texture encoder, which is used to generate a texture feature vector according to a texture embedding vector; a feature fusion module, which is used to generate a UV mapping feature vector according to the texture feature vector and the UV coordinate feature vector, and generate a new texture furniture model according to the UV mapping feature vector and the body point feature vector; the furniture sparse point cloud is input into the body encoder to obtain the spatial coordinate point potential vector; Input the spatial coordinate point potential vector into a body decoder to obtain a body point feature vector and a UV coordinate feature vector; Input the spatial coordinate point potential vector into the voxel decoder to find the set of neighbor node potential vectors of any point in the space, where the neighbor nodes are calculated as the 16 nodes with the closest Euclidean distance between the spatial coordinate point potential vector and the point cloud vector of any point in the space. The Euclidean distance is: ; in Refers to the three-dimensional coordinates of the neighbor node, /> Refers to the three-dimensional coordinates of the neighbor node, /> Refers to the three-dimensional coordinates of the neighbor node, /> Refers to the three-dimensional coordinates of any point in space, /> Refers to the three-dimensional coordinates of any point in space, /> Refers to the three-dimensional coordinates of any point in space; Use interpolation to get the potential vector of any point in the space: ; in refers to the potential vector at any point in space, /> refers to the potential vector set of neighbor nodes, Refers to the distance weight, /> It is Hadamard; After the potential vector of any point in the space passes through a fully connected layer, a feature vector of the body point is obtained, and multiple feature vectors of the body point constitute a body model; After the potential vector of any point in the space passes through a fully connected layer, the UV coordinate feature vector of any point in the space is obtained; Inputting the texture embedding vector into a texture encoder to obtain a texture feature vector; The texture embedding vector with position information is residually connected with the result of layer normalization and multi-head attention calculation to obtain the texture potential vector: ; in is the texture latent vector, /> is a texture embedding vector with position information, /> It refers to the result of the multi-head attention mechanism in the neural network. /> It refers to the layer normalization result in the neural network; The texture potential vector is normalized and fully connected layer, and then the residual calculation is performed on the layer normalization and fully connected layer calculation results to output the texture feature vector: ; Where T is the texture feature vector, is the texture latent vector, /> Compute results for fully connected layers in neural networks; Input the texture feature vector and the UV coordinate feature vector into a feature fusion module to obtain a UV mapping feature vector; The UV map feature vector is added to the body point feature vector, and a new texture furniture model is obtained by passing through a convolutional network and using the open3D extension library. 2. The method for reconstructing a furniture model based on multi-modal body and texture fusion according to claim 1, characterized in that the texture feature vector and the UV coordinate feature vector are input into a feature fusion module to obtain a UV mapping feature vector, comprising: The texture feature vector and the UV coordinate feature vector are added and input into the convolutional network. The convolutional network backbone uses ResNet34 to obtain the UV map feature vector. 3. The method for reconstructing furniture models based on multi-modal body and texture fusion according to claim 2, wherein the body model is composed of a plurality of body point feature vectors and comprises: use As the loss function, the body model is generated as follows: ; in is the i-th point in the point cloud of the primitive,/> is the jth point in the real point cloud,/> is the number of point clouds in the primitive,/> is the number in the real point cloud. 4. The method for reconstructing a furniture model based on multi-modal body and texture fusion according to claim 1, characterized in that after obtaining the furniture sparse point cloud, it also includes enhancing the furniture sparse point cloud data: Perform high-frequency transformation on the furniture sparse point cloud: ; in, For a furniture sparse point cloud, /> is a set of real numbers, and L is a high frequency variability. 5. The method for reconstructing a furniture model based on multimodal body and texture fusion according to claim 1 is characterized in that after the texture feature vector and the UV coordinate feature vector are input into a feature fusion module to obtain the UV map feature vector, the method further comprises calculating and checking the similarity between the UV map and the texture image, specifically: Input the UV mapping feature vector into the torchvision extension library, use the transform_convert() method to output the UV mapping result, and use Mean square error of the matrix/> As loss function: ; in Refers to the number of channels of the feature map, /> Refers to the height of the feature map, /> Refers to the width of the feature map, /> Refers to the UV map on the i-th layer feature map/> Matrix, /> Refers to the texture image on the i-th layer feature map Matrix, where /> , where/> is the feature map after the i-th layer of the neural network,/> For/> The transposed matrix of . 6. A furniture model reconstruction system based on multi-modal body and texture fusion, characterized by comprising: Data acquisition module, used to obtain furniture sparse point cloud and texture embedding vector; A data processing module, used for inputting the furniture sparse point cloud and the texture embedding vector into a body and texture fusion model to obtain a new texture furniture model; Used to implement a furniture model reconstruction method based on multi-modal body and texture fusion as described in any one of claims 1-5.