CN113744407B - Method and device for generating digital model - Google Patents
Method and device for generating digital modelInfo
- Publication number
- CN113744407B CN113744407B CN202010700841.9A CN202010700841A CN113744407B CN 113744407 B CN113744407 B CN 113744407B CN 202010700841 A CN202010700841 A CN 202010700841A CN 113744407 B CN113744407 B CN 113744407B
- Authority
- CN
- China
- Prior art keywords
- model
- grid
- point cloud
- cloud data
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/41—Medical
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
本发明公开了一种生成数字化模型的方法及装置。其中，该方法包括：基于被测物体的至少一组被测物图像，获取所述被测物体的整体点云数据；以至少两个不同的分辨率对所述整体点云数据进行网格化处理，得到所述被测物体的至少两个网格模型；将所述至少两个网格模型进行合成，生成具有多分辨率的整体网格模型。本发明解决了现有技术中采用分模扫描时，由于降低了扫描被测物体的效率造成的生成数字化模型的效率无法提高的技术问题。
The present invention discloses a method and apparatus for generating a digital model. The method comprises: obtaining overall point cloud data of the object under test based on at least one set of images of the object under test; gridding the overall point cloud data at at least two different resolutions to obtain at least two grid models of the object under test; and synthesizing the at least two grid models to generate an overall grid model with multiple resolutions. The present invention solves the technical problem in the prior art of using split-mode scanning, which reduces the efficiency of scanning the object under test and thus prevents the efficiency of generating a digital model from being improved.
Description
The present application claims priority from chinese patent office, application number 2020104767855, application name "tooth scanning method, apparatus, system and computer readable storage medium" filed 29 in 05/2020, the entire contents of which are incorporated herein by reference.
Meanwhile, the application also claims priority of Chinese patent application filed in the Chinese patent office at 07/06/2020 with application number 2020106419086 and application name of "method and device for generating digital model", which is incorporated herein by reference in its entirety.
Technical Field
The invention relates to the field of digital models, in particular to a method and a device for generating a digital model.
Background
In digital dental restoration design application, dental digital models are generally obtained by collecting dental models in the mouth of a patient through intraoral scanning or by obtaining plaster models through scanning and reproduction by a desktop scanner. In applications, it is generally desirable that the digital model be as fine as possible in order to better identify some important details, such as, in particular, the edge lines of some restorative teeth, abutments, steps, end faces and corners of implant stems, etc.
The digitized model is represented by triangular grids, and the minimum point distance of the triangular grids determines the minimum detail level which can be expressed by the model. Thus, the smaller the lattice model of the point distance, the more the detailed features of the actual tooth or restoration can be expressed. In addition, the mesh model with smaller point distance needs to be scanned by a scanner with higher image resolution to obtain high-resolution original point cloud data, and then a series of time-consuming processes are carried out to generate high-definition meshes.
As described above, if an overall finer digitized dental model is to be obtained, there are two problems that 1) the model data amount is large and it is inconvenient to import the design software, and 2) the processing time for generating the model upon scanning increases.
In order to solve the problems, the technical scheme in the existing systems mainly comprises the step of adopting a mode division scanning method. Cutting open the dental plaster model, scanning the part to be repaired separately to generate fine grids, scanning the other part to generate coarse grids, and then aligning and sewing the two grids into a whole. This ensures the details of the partially important repair area and the appropriate overall mesh model size.
Therefore, in the prior art, a scanner is adopted to scan to obtain a digitalized model of a measured object, in general, in order to obtain an overall finer digitalized model, the point distance of a triangular mesh for representing the digitalized model is subjected to minimization treatment, but the technical problem of low efficiency of generating the digitalized model exists in the mode, and a split-mode scanning method can be adopted to solve the problem in the prior art, but in the scanning process, the efficiency of scanning the measured object is reduced, so that the technical problem that the efficiency of generating the digitalized model cannot be improved is caused.
In view of the above problems, no effective solution has been proposed at present.
Disclosure of Invention
The embodiment of the invention provides a method and a device for generating a digital model, which at least solve the technical problem that the efficiency of generating the digital model can not be improved due to the reduction of the efficiency of scanning a measured object when split-mode scanning is adopted in the prior art.
According to one aspect of the embodiment of the invention, a method for generating a digital model is provided, which comprises the steps of acquiring integral point cloud data of a measured object based on at least one group of measured object images of the measured object, carrying out gridding processing on the integral point cloud data with at least two different resolutions to obtain at least two grid models of the measured object, and synthesizing the at least two grid models to generate an integral grid model with multiple resolutions.
Optionally, gridding the whole point cloud data with at least two different resolutions to obtain at least two grid models of the measured object, wherein the gridding the whole point cloud data with the first resolution to obtain a first grid model of the measured object, determining a region to be adjusted in the first grid model, determining and dividing original point cloud data falling in the region to be adjusted from the whole point cloud data, and gridding the original point cloud data falling in the region to be adjusted with the second resolution to obtain a second grid model of the region to be adjusted, wherein the whole point cloud data comprises the point cloud data in the region to be adjusted.
Optionally, synthesizing at least two grid models to generate an integral grid model with multiple resolutions, wherein the integral grid model comprises the steps of cutting an original grid of a region to be adjusted from a first grid model to generate a grid model to be stitched, and synthesizing a second grid model and the grid model to be stitched to generate the integral grid model.
Optionally, the information of the area to be adjusted comprises the range of the area to be adjusted and the area type, wherein the range of the area to be adjusted is used for screening the area to be adjusted from the whole point cloud data, and the area type is used for determining the second resolution.
Optionally, under the condition that the first resolution is low resolution, the first grid model is a reconstructed coarse grid model, wherein determining the region to be adjusted in the first grid model comprises identifying the feature type from the first grid model by adopting an identification model and determining the region to be adjusted based on the identified feature type, wherein the identification model is a neural network model based on sample training, or selecting the region to be adjusted from the first grid model based on a received selection instruction, and the region to be adjusted is a region which needs to be subjected to fine gridding processing by adopting high resolution in the first grid model.
Optionally, the method further comprises determining a non-adjusted region in the first mesh model, wherein the mesh model of the non-adjusted region is a portion of the first mesh model other than the second mesh model.
Optionally, after determining the area to be adjusted in the first grid model, the method further comprises gridding the original point cloud data falling in the area to be adjusted with a third resolution to obtain a third grid model of the area to be adjusted, wherein the whole point cloud data comprises the point cloud data in the area to be adjusted.
Optionally, in the process of sequentially performing gridding processing on the whole point cloud data by adopting a plurality of resolutions, the resolution used each time is sequentially increased, and the point cloud data currently subjected to gridding processing is part of the point cloud data subjected to previous gridding processing.
According to another aspect of the embodiment of the invention, another method for generating a digital model is provided, which comprises the steps of acquiring overall point cloud data of a measured object based on at least one group of measured object images of the measured object, carrying out gridding processing on the overall point cloud data with a first resolution to obtain a first grid model of the measured object, wherein the first grid model comprises a first area needing to be subjected to gridding processing again, carrying out gridding processing on original point cloud data in the first area with a second resolution to obtain a second grid model, and synthesizing the second grid model and grid models except the first area in the first grid model to generate the overall grid model.
Alternatively, in the case where the first resolution is a low resolution, the second resolution is a high resolution, the first mesh model is a coarse mesh model, and the second mesh model is a fine mesh model.
According to still another aspect of the embodiment of the invention, another method for generating a digital model is provided, which comprises the steps of acquiring integral point cloud data of a measured object based on acquired multi-frame images of the measured object, performing gridding processing on the integral point cloud data to obtain a first grid model of the measured object, identifying a region to be regulated in the first grid model, acquiring original point cloud data falling in the region to be regulated from the integral point cloud data, performing high-resolution fine gridding processing on the original point cloud data to generate a second grid model of the region to be regulated, and replacing the original grid model of the region to be regulated in the first grid model by using the second grid model.
According to one aspect of the embodiment of the invention, the device for generating the digital model comprises a first acquisition module, a first processing module and a first generation module, wherein the first acquisition module is used for acquiring the whole point cloud data of the measured object based on at least one group of measured object images of the measured object, the first processing module is used for carrying out gridding processing on the whole point cloud data with at least two different resolutions to obtain at least two grid models of the measured object, and the first generation module is used for synthesizing the at least two grid models to generate the whole grid model with multiple resolutions.
According to another aspect of the embodiment of the present invention, there is further provided a nonvolatile storage medium, where the nonvolatile storage medium includes a stored program, and when the program runs, a method for controlling any one of devices where the nonvolatile storage medium is located to generate a digitized model.
According to another aspect of the embodiment of the present invention, there is also provided a processor, configured to execute a program, where the program executes any one of the methods for generating a digitized model.
According to the embodiment of the invention, a mode of scanning a detected object without parting is adopted, integral point cloud data of the detected object is obtained based on at least one group of detected object images of the detected object, gridding processing is carried out on the integral point cloud data with at least two different resolutions to obtain at least two grid models of the detected object, the at least two grid models are synthesized to generate an integral grid model with multiple resolutions, the purpose of generating the grid model corresponding to the detected object without parting scanning the detected object is achieved, the technical effect of quickly generating the grid model is achieved, and the technical problem that the efficiency of generating a digital model due to the fact that the efficiency of scanning the detected object is reduced in the prior art is solved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1 is a flow diagram of a method of generating a digitized model in accordance with an embodiment of the invention;
FIG. 2 is a flow chart framework diagram of an alternative high resolution fine grid for generating feature regions, also provided in accordance with an embodiment of the present invention;
FIG. 3 is a flowchart of an alternative method for determining feature region information in a coarse mesh model according to an embodiment of the present invention;
FIG. 4 is a flow chart framework of an alternative generation of a coarse grid provided in accordance with an embodiment of the present invention;
FIG. 5 is a flow chart of an alternative method for generating a non-feature area grid according to an embodiment of the invention;
FIG. 6 is a flow chart of an alternative method for generating a non-feature area grid according to an embodiment of the invention;
FIG. 7 is a flow chart of an alternative method for generating a resolution grid according to an embodiment of the present invention;
FIG. 8 is a flow chart of another method of generating a digitized model according to an embodiment of the invention;
FIG. 9 is a flow chart of another method of generating a digitized model according to an embodiment of the invention;
FIG. 10 is a schematic diagram of an apparatus for generating a digitized model according to an embodiment of the invention;
FIG. 11 is a schematic diagram of another apparatus for generating a digitized model according to an embodiment of the invention;
Fig. 12 is a schematic structural view of another apparatus for generating a digitized model according to an embodiment of the invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present invention and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
In accordance with an embodiment of the present invention, there is provided a method embodiment for generating a digitized model, it being noted that the steps shown in the flowchart of the figures may be performed in a computer system, such as a set of computer-executable instructions, and, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order other than that shown or described herein.
FIG. 1 is a method of generating a digitized model according to an embodiment of the invention, as shown in FIG. 1, the method comprising the steps of:
step S102, acquiring the whole point cloud data of the measured object based on at least one group of measured object images of the measured object;
Step S104, gridding the whole point cloud data with at least two different resolutions to obtain at least two grid models of the measured object;
and step S106, synthesizing at least two grid models to generate an integral grid model with multiple resolutions.
In the method for generating the digital model, the whole point cloud data of the measured object can be acquired based on at least one group of measured object images of the measured object, then gridding processing is carried out on the whole point cloud data with at least two different resolutions to obtain at least two grid models of the measured object, finally, the at least two grid models are synthesized to generate the whole grid model with multiple resolutions, the purpose of generating the grid three-dimensional model corresponding to the measured object without scanning the measured object in a split mode is achieved, the technical effect of quickly generating the grid three-dimensional model is achieved, and the technical problem that the efficiency of generating the digital model due to the fact that the efficiency of scanning the measured object is reduced cannot be improved when the split mode scanning is adopted in the prior art is solved.
It should be noted that, the multi-resolution is to process different areas with different resolutions according to actual requirements, where the two different resolutions may be higher than the default resolution and may be lower than the default resolution. The default resolution is generally set to a lower resolution to achieve a faster scanning speed for real-time fluent imaging.
Specifically, in the step S102, the image of the object to be measured may be scanned by using a scanner, for example, a three-dimensional scanner, and then each group of images to be measured is subjected to three-dimensional reconstruction to form original single-chip point cloud data, each group of images to be measured forms one piece of original single-chip point cloud data, multiple groups of images to be measured may form multiple pieces of original single-chip point cloud data, and the multiple pieces of original single-chip point cloud data may be spliced into integral point cloud data after being processed by a unified coordinate system, so as to obtain integral point cloud data of the object to be measured.
It is easy to note that the scanner device needs to have a higher image resolution, that is, the original single-chip point cloud data with higher fineness can be obtained through scanning acquisition, for example, an intraoral scanner with pixels greater than or equal to 30 ten thousand or a desktop scanner with pixels greater than or equal to 130 ten thousand is adopted.
The original single-chip point cloud data refers to point cloud data acquired and generated by a single visual angle, and the object to be tested which can be scanned by the scanning equipment is an image of denture, oral cavity and the like.
In the step S104, in order to obtain at least two grid models of the measured object, in an alternative embodiment of the present application, the first grid model of the measured object may be obtained by performing the gridding process on the whole point cloud data with the first resolution, then the area to be adjusted in the first grid model is determined, then the original point cloud data falling in the area to be adjusted is determined and segmented from the whole point cloud data, and finally the second grid model of the area to be adjusted is obtained by performing the gridding process on the original point cloud data falling in the area to be adjusted with the second resolution.
Taking the scanning rod area (namely the area where the scanning rod is positioned) as a first grid area as an example, the scanning rod is a standard body fitting of a three-dimensional scanner for scanning a dental model, the scanning rod can be inserted into a dental jaw, and the position of the implant can be better positioned through the standard scanning rod on the dental jaw.
The method comprises the steps of analyzing dental identification data to obtain tooth areas, restoration type information, gum areas and base areas, obtaining scanning rod areas according to the tooth areas, the gum areas and the base areas, and separating and extracting first grid data corresponding to the scanning rod areas in the tooth coarse grid three-dimensional model according to the scanning rod areas. In one embodiment, a scanning rod area exists in a tooth coarse grid three-dimensional model in a three-dimensional scanner, and first grid data corresponding to the scanning rod area in the tooth coarse grid three-dimensional model is extracted according to the scanning rod area separation.
In an alternative embodiment of the present application, in order to perform gridding more quickly, when gridding the whole point cloud data with the first resolution, the gridding adopts an incremental gridding process, that is, when a part of original point cloud is obtained by scanning, the gridding process of the existing part of original point is started, then in the subsequent scanning process, the original point cloud generated by the new scanning is gridded on the basis of the previous grid in an incremental manner, so as to obtain a first grid model of the measured object, and the area to be adjusted can adopt a general gridding process, that is, a non-incremental gridding process.
In an alternative embodiment of the application, the region to be adjusted can be a characteristic region, as shown in fig. 2, and the embodiment of the application further provides a process frame diagram for generating a high-resolution fine grid of the characteristic region, in the process, the high-resolution fine grid of the characteristic region can be generated by firstly screening all regions of original single-chip point cloud data, namely selecting and segmenting out parts falling in the spatial range of the characteristic region from all the original single-chip point cloud data subjected to global optimization processing, then carrying out high-resolution fine grid processing by using the segmented original point cloud data, fusing the original point cloud based on voxels, determining the size of the resolution by setting the size of the voxels, extracting the grid based on an implicit curved surface of a measured object model, obtaining a grid model of the measured object, and obtaining a final fine grid model through a series of grid processing of characteristic protection. The process can also adopt a mode of stitching based on a single-piece grid to generate a final fine grid model, after the multi-piece point cloud is de-overlapped, the grid treatment is respectively carried out to obtain the single-piece grid, and then the multi-piece grid is synthesized, and as the characteristic area is generally only a small part of the whole dental jaw, the time of high-resolution (small point distance) treatment of the local area can be effectively controlled.
It should be noted that global optimization refers to uniformly determining the optimal spatial positions of all the single point clouds, fusion processing refers to merging independent single point clouds into a uniform grid model, and feature protection methods include, but are not limited to, bilateral filtering, flanging processing and the like, and meanwhile, it is easy to note that the smaller the point distance of the grid model is, the higher the resolution is.
After obtaining at least two grid models of the object to be tested, in order to obtain an overall grid model, synthesizing the at least two grid models to generate an overall grid model with multiple resolutions, in some alternative embodiments of the present application, an original grid of an area to be adjusted may be cut from a first grid model to generate a grid model to be stitched, and then a second grid model processed with a second resolution is synthesized with the grid model to be stitched to generate an overall grid model, where the synthesis method includes, but is not limited to, synthesizing the two grid models by adopting a stitching algorithm to obtain the overall grid model.
It should be noted that the information of the to-be-adjusted area includes a range of the to-be-adjusted area and an area type, wherein the range of the to-be-adjusted area is used for screening the to-be-adjusted area from the whole point cloud data, and the area type is used for determining the second resolution. Specifically, in order to determine the position information of the to-be-adjusted area, the distribution range of the characteristic area may be represented by using the point cloud corresponding to the to-be-adjusted area, or the distribution range corresponding to the characteristic area may be represented by using a regular solid bounding box. The above-mentioned range of the area to be adjusted can be used to determine the range of the fine grid processing, and the type of the feature area can be further used to determine the resolution of the fine grid of different areas according to the application requirements.
The method of generating the digitized model used in some embodiments of the present application is also different in order to adapt to different application scenarios. For example, when the local detail is more prominent during the oral scanning, the adopted strategy is to adjust the resolution of the feature region higher, specifically, when the handheld scanner performs the oral scanning, the whole point cloud data is firstly gridded by adopting the default resolution to obtain a first grid model, then the feature region is gridded by adopting the higher resolution to obtain a second grid model, namely, the first grid model is a rough grid model with lower resolution compared with the second grid model, the second grid model is a fine grid model with higher resolution compared with the first grid model, and then the first grid model and the second grid model are synthesized to obtain a multi-resolution whole grid model, namely, a whole grid model with lower resolution and higher resolution.
In some alternative embodiments of the present application, when the first resolution is low resolution, the first mesh model is a reconstructed coarse mesh model, the region to be adjusted in the first mesh model may be determined by, for example, identifying a feature type from the first mesh model by using an identification model, and determining the region to be adjusted based on the identified feature type, where the identification model is a neural network model based on sample training, or selecting the region to be adjusted from the first mesh model based on a received selection instruction, where the region to be adjusted is a region in the first mesh model that needs to be subjected to fine meshing processing with high resolution (small point distance).
For intraoral scanning, for example, feature types include teeth, gums, teeth, scan bars, etc., in order to make local details more prominent in this application scenario, two-level resolution is generally used, a grid with a first-level resolution is formed in a real-time scanning process, after scanning is completed, a grid with a second-level resolution is generated for points corresponding to feature areas such as teeth, scan bars, etc., the second-level resolution is higher than the first-level resolution to improve the resolution of the areas such as teeth, scan bars, etc., if three-level resolution is used, the feature areas such as teeth, scan bars, etc. use high resolution, the feature areas such as teeth, etc. use medium resolution, and the non-feature areas such as gums, etc., use low resolution, i.e., default resolution.
For example, for denture scanning, the feature type comprises one of teeth, gums, teeth preparation, abutments, inlays and scan bars, wherein a first-level resolution grid can be generated in the real-time scanning process, after the scanning is completed, points corresponding to the feature areas of the teeth preparation, abutments, inlays and scan bars can be generated to a second-level resolution grid, the second-level resolution is higher than the first-level resolution, and the like, the feature type in the denture can be processed by adopting three-level resolution, specifically, the feature areas of the teeth preparation, abutments, inlays and scan bars adopt high resolution, the feature areas of the teeth adopt middle resolution, the non-feature areas of the gums and other materials adopt low resolution, namely default resolution, and the feature type can be processed by adopting more different resolutions according to the practical application environment.
In an alternative embodiment of the present application, the region to be adjusted may be a feature region, as shown in fig. 3, and the embodiment of the present application provides a flow frame diagram for determining feature region information in a coarse mesh model, specifically, for the coarse mesh model, automatic recognition may be first adopted to obtain the feature region, for example, based on AI training, feature pattern matching, etc., or the feature region may be manually selected or modified, or a combination of the two may be selected and displayed automatically, and then manually interacted to modify and confirm, where the generated feature region information mainly includes the range and feature type of each adjustment region, and the feature type includes, but is not limited to, common teeth, teeth needing to be repaired and a base station, and it is to be noted that the coarse mesh model may be simply referred to as the coarse mesh model.
In an alternative embodiment of the present application, a flowchart of generating a coarse grid is further provided, as shown in fig. 4, where after an image is acquired and three-dimensional reconstruction is performed, single-frame point cloud data is obtained, and then a fast incremental gridding processing method is used to obtain the coarse grid. The incremental gridding processing means synchronous gridding processing in the scanning process, and improves processing efficiency through an incremental iterative processing mode, specifically, only newly scanned single-chip point cloud data are optimized each time, the optimal space position of the single-chip point cloud is determined, independent single-chip point clouds are combined into a unified grid model with lower resolution, and then the latest grid is obtained from the grid model, in this way, after scanning is completed, a grid model of an object to be detected can be obtained, noise of the model is smoothly filtered, and a rough grid model can be obtained.
In some optional embodiments of the present application, if the first mesh model can meet the actual application requirement, the non-adjustment area in the first mesh model may be determined directly according to the area to be adjusted, and it is easy to note that the mesh model of the non-adjustment area is a portion of the first mesh model except the second mesh model,
In an alternative embodiment of the present application, the area to be adjusted may be a feature area, and the non-adjustment area may be a non-feature area, as shown in fig. 5, a flowchart of generating a non-feature area grid is further provided, in this flowchart, first, the feature area in the coarse grid may be directly cut according to the area information of the feature area, and then the non-feature area grid is obtained.
In some optional embodiments of the present application, when there are a plurality of areas to be adjusted, after performing gridding processing on the original point cloud data in the area to be adjusted with the second resolution, gridding processing may be performed on the original point cloud data in the other areas to be adjusted with a third resolution for other areas to be adjusted except for the area to be adjusted with the second resolution, so as to obtain a third grid model of the other areas to be adjusted, where the whole point cloud data includes the point cloud data in the other areas to be adjusted, and it is easy to note that processing may also be performed on the other areas to be adjusted with different resolutions.
In some embodiments of the present application, if the first grid model cannot meet the actual application requirement, for example, the resolution is too low, after determining the region to be adjusted in the first grid model, the non-adjustment region in the first grid model may be generated by reading global point cloud data, performing fusion processing on the global point cloud data to generate a fourth resolution global grid, and cutting the determined region to be adjusted from the low resolution global grid based on the information of the region to be adjusted, to generate the non-adjustment region in the first grid model, where the information of the region to be adjusted includes the range and the region type of the region to be adjusted, and the grid model of the non-adjustment region is a portion of the first grid model except the second grid model.
For example, in the application of generating the dental digital model by adopting high-low resolution processing, the scheme of three or more resolution processing of high-low resolution can be further promoted, in an optional embodiment of the application, in the process of sequentially carrying out gridding processing on the whole point cloud data by adopting a plurality of resolutions, the resolution used each time can be sequentially increased, the point cloud data which is currently subjected to gridding processing is part of the point cloud data which is subjected to last gridding processing, the detail of an important characteristic area can be best embodied by adopting the multi-resolution processing mode, the data size of the whole dental model is better controlled, the speed of generating the whole dental grid model by processing is effectively improved, and the times and steps of scanning are not required to be increased, so that the time of data processing is greatly saved.
In an alternative embodiment of the present application, the area to be adjusted may be a feature area, and the area not to be adjusted may be a non-feature area, as shown in fig. 6, another flowchart of generating a grid of non-feature areas is provided, where first, the grid model is regenerated by using the original single-chip point cloud data, specifically, fusion processing is carried out on all single-chip point cloud data by adopting a relatively low-resolution grid model, then low-resolution grids are extracted from the grid model, a low-resolution grid model is obtained after simple smoothing processing is carried out, then characteristic region parts are cut off, and finally non-characteristic region grids are obtained. In this way, although the complete area grid model is regenerated, the processing time is still much longer than the whole high resolution (small dot pitch) fine grid processing due to the lower resolution grid processing and simpler processing.
In an alternative embodiment of the present application, as shown in fig. 7, a flowchart of a process for generating a resolution grid is further provided, where first, the non-feature area and the feature area can be determined by the alternative embodiment, and then, the multi-resolution grid is synthesized, and finally, a multi-resolution integral grid model is obtained.
FIG. 8 is another method of generating a digitized model, as shown in FIG. 2, according to an embodiment of the invention, the method comprising the steps of:
s202, acquiring integral point cloud data of a detected object based on at least one group of detected object images of the detected object;
S204, gridding the whole point cloud data with a first resolution to obtain a first grid model of the measured object, wherein the first grid model comprises a first area needing to be subjected to regrid treatment;
S206, gridding the original point cloud data in the first area with a second resolution to obtain a second grid model;
s208, synthesizing the second grid model and the grid models except the first area in the first grid model to generate an integral grid model.
The method for generating the digitized model comprises the steps of firstly acquiring the whole point cloud data of the measured object based on at least one group of measured object images of the measured object, then carrying out gridding processing on the whole point cloud data with a first resolution to obtain a first grid model of the measured object, wherein the first grid model comprises a first area needing to be subjected to gridding processing again, secondly carrying out gridding processing on the original point cloud data in the first area with a second resolution to obtain a second grid model, and finally synthesizing the second grid model and the grid models except the first area in the first grid model to generate the whole grid model, thereby achieving the purpose of generating the grid three-dimensional model corresponding to the measured object without dividing the mode to scan the measured object, further achieving the technical effect of quickly generating the grid three-dimensional model, and further solving the technical problem that the efficiency of generating the digitized model cannot be improved due to the fact that the efficiency of scanning the measured object is reduced when dividing mode scanning is adopted in the prior art.
When the first resolution is low, the second resolution is high (small dot pitch), the first mesh model is a coarse mesh model, and the second mesh model is a fine mesh model.
FIG. 9 is another method of generating a digitized model according to an embodiment of the invention, as shown in FIG. 3, the method comprising the steps of
S302, acquiring integral point cloud data of a detected object based on the acquired multi-frame images of the detected object;
s304, gridding the whole point cloud data to obtain a first grid model of the measured object;
s306, identifying a region to be adjusted in the first grid model;
s308, acquiring original point cloud data falling in an area to be adjusted from the whole point cloud data;
S310, performing high-resolution fine gridding processing on the original point cloud data to generate a second grid model of the area to be adjusted;
s312, replacing the original grid model of the area to be adjusted in the first grid model by using the second grid model.
The method for generating the digital model comprises the steps of firstly acquiring the whole point cloud data of the measured object based on the acquired multi-frame image of the measured object, secondly carrying out gridding treatment on the whole point cloud data to obtain a first grid model of the measured object, identifying a region to be regulated in the first grid model, then acquiring the original point cloud data in the region to be regulated from the whole point cloud data, carrying out high-resolution fine gridding treatment on the original point cloud data to generate a second grid model of the region to be regulated, and finally replacing the original grid model of the region to be regulated in the first grid model by using the second grid model, thereby achieving the aim of generating the grid three-dimensional model corresponding to the measured object without dividing the mode to scan the measured object, further realizing the technical effect of quickly generating the grid three-dimensional model, and further solving the technical problem that the efficiency of generating the digital model cannot be improved due to the fact that the efficiency of scanning the measured object is reduced when dividing mode scanning is adopted in the prior art.
FIG. 10 provides an apparatus for generating a digitized model according to another aspect of an embodiment of the invention, as shown in FIG. 10, the apparatus comprising:
a first acquisition module 10, configured to acquire overall point cloud data of a measured object based on at least one group of measured object images of the measured object;
The first processing module 12 is configured to perform gridding processing on the whole point cloud data with at least two different resolutions, so as to obtain at least two grid models of the object to be measured;
A first generation module 14 is configured to synthesize at least two mesh models to generate an overall mesh model with multiple resolutions.
The device for generating the digitized model comprises a first acquisition module 10, a first processing module 12 and a first generation module 14, wherein the first acquisition module 10 is used for acquiring the whole point cloud data of the measured object based on at least one group of measured object images of the measured object, the first processing module 12 is used for carrying out gridding processing on the whole point cloud data with at least two different resolutions to obtain at least two grid models of the measured object, and the first generation module 14 is used for synthesizing the at least two grid models to generate the whole grid model with multiple resolutions, so that the aim of generating the grid three-dimensional model corresponding to the measured object without carrying out split mode scanning on the measured object is fulfilled, the technical effect of quickly generating the grid three-dimensional model is achieved, and the technical problem that the efficiency of generating the digitized model due to the fact that the efficiency of scanning the measured object is reduced in the prior art is solved.
FIG. 11 another apparatus for generating a digitized model according to another aspect of an embodiment of the invention is provided, as shown in FIG. 11, the apparatus comprising:
A second obtaining module 20, configured to obtain overall point cloud data of the object to be measured based on at least one group of object images of the object to be measured;
The second processing module 22 is configured to perform gridding processing on the whole point cloud data with a first resolution, so as to obtain a first grid model of the object to be measured, where the first grid model includes a first area that needs to be subjected to regrid processing;
a third processing module 24, configured to perform gridding processing on the original point cloud data in the first area with a second resolution, so as to obtain a second grid model;
A second generating module 26, configured to synthesize the second grid model with the grid models of the first grid model except for the first region, and generate an overall grid model.
The device for generating the digitized model comprises a second acquisition module 20, a second processing module 22, a third processing module 24 and a second generation module 26, wherein the second acquisition module 20 is used for acquiring the whole point cloud data of the measured object based on at least one group of measured object images of the measured object, the second processing module 22 is used for carrying out gridding processing on the whole point cloud data with a first resolution to obtain a first grid model of the measured object, the first grid model comprises a first area needing to carry out gridding processing again, the third processing module 24 is used for carrying out gridding processing on the original point cloud data in the first area with a second resolution to obtain a second grid model, the second generation module 26 is used for synthesizing the second grid model and the grid model except the first area in the first grid model to generate the whole grid model, the purpose of generating the grid three-dimensional model corresponding to the measured object without carrying out mould division scanning on the measured object is achieved, and therefore the technical effect that the problem that the digital model cannot be generated due to the fact that the scanning efficiency of the measured object is reduced in the prior art is solved.
FIG. 12 provides an apparatus for generating a digitized model according to another aspect of an embodiment of the invention, as shown in FIG. 12, the apparatus comprising:
a third obtaining module 30, configured to obtain integral point cloud data of the measured object based on the collected multi-frame images of the measured object;
a fourth processing module 32, configured to perform gridding processing on the global point cloud data, so as to obtain a first grid model of the object to be measured;
an identifying module 34, configured to identify a region to be adjusted in the first mesh model;
a fourth obtaining module 36, configured to obtain, from the overall point cloud data, original point cloud data that falls within the area to be adjusted;
a fifth processing module 38, configured to perform high-resolution fine gridding processing on the original point cloud data, and generate a second grid model of the area to be adjusted;
A replacing module 40, configured to replace the original grid model of the region to be adjusted in the first grid model with the second grid model.
The device for generating the digitized model comprises a third acquisition module 30, a fourth processing module 32, an identification module 34, a fourth acquisition module 36, a fifth processing module 38 and a replacement module 40, wherein the third acquisition module 30 is used for acquiring the whole point cloud data of the measured object based on the acquired multi-frame images of the measured object, the fourth processing module 32 is used for carrying out gridding processing on the whole point cloud data to obtain a first grid model of the measured object, the identification module 34 is used for identifying a region to be adjusted in the first grid model, the fourth acquisition module 36 is used for acquiring original point cloud data falling in the region to be adjusted from the whole point cloud data, the fifth processing module 38 is used for carrying out high-resolution fine gridding processing on the original point cloud data to generate a second grid model of the region to be adjusted, and the replacement module 40 is used for replacing the original grid model of the region to be adjusted in the first grid model, so that the purpose of quickly generating a three-dimensional model corresponding to the measured object without dividing the mode scanning the measured object is achieved, and the problem that the efficiency of generating the three-dimensional model cannot be quickly generated due to the scanning technology of the scanned model is solved.
According to another aspect of the embodiment of the present invention, there is further provided a nonvolatile storage medium, where the nonvolatile storage medium includes a stored program, and when the program runs, a method for controlling any one of devices where the nonvolatile storage medium is located to generate a digitized model.
Specifically, the storage medium is used for storing program instructions for executing the following functions, and the following functions are realized:
The method comprises the steps of obtaining integral point cloud data of a detected object based on at least one group of detected object images of the detected object, carrying out gridding treatment on the integral point cloud data with at least two different resolutions to obtain at least two grid models of the detected object, and synthesizing the at least two grid models to generate an integral grid model with multiple resolutions.
According to another aspect of the embodiment of the present invention, there is also provided a processor, configured to execute a program, where the program executes any one of the methods for generating a digitized model.
Specifically, the processor is used for calling program instructions in the memory to realize the following functions;
The method comprises the steps of obtaining integral point cloud data of a detected object based on at least one group of detected object images of the detected object, carrying out gridding treatment on the integral point cloud data with at least two different resolutions to obtain at least two grid models of the detected object, and synthesizing the at least two grid models to generate an integral grid model with multiple resolutions.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
In the foregoing embodiments of the present invention, the descriptions of the embodiments are emphasized, and for a portion of this disclosure that is not described in detail in this embodiment, reference is made to the related descriptions of other embodiments.
In the several embodiments provided in the present application, it should be understood that the disclosed technology may be implemented in other manners. The above-described embodiments of the apparatus are merely exemplary, and the division of the units, for example, may be a logic function division, and may be implemented in another manner, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some interfaces, units or modules, or may be in electrical or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied essentially or in part or all of the technical solution or in part in the form of a software product stored in a storage medium, including instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. The storage medium includes a U disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a removable hard disk, a magnetic disk, or an optical disk, etc. which can store the program code.
The foregoing is merely a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention, which are intended to be comprehended within the scope of the present invention.
Claims (19)
1. A method of generating a digitized model, comprising:
Acquiring integral point cloud data of a detected object based on at least one group of detected object images of the detected object;
Gridding the whole point cloud data with at least two different resolutions to obtain at least two grid models of the measured object, wherein the gridding the whole point cloud data with a first resolution to obtain a first grid model of the measured object; the method comprises the steps of determining a region to be adjusted in a first grid model, generating a second grid model with a second resolution corresponding to the region to be adjusted according to the information of the region to be adjusted, wherein the grid model is a dental digital model, the information of the region to be adjusted comprises a range and a region type of the region to be adjusted, the range of the region to be adjusted is used for screening the region to be adjusted from the whole point cloud data, the region type is used for determining the second resolution, the determining the region to be adjusted in the first grid model comprises the steps of identifying a feature type from the first grid model by adopting an identification model and determining the region to be adjusted on the basis of the identified feature type, the identification model is a neural network model based on sample training, or selecting the region to be adjusted from the first grid model on the basis of a received selection instruction, wherein the region to be adjusted is a region which needs to be subjected to fine grid processing by adopting high resolution in the first grid model, and the grid processing mode is adopted for carrying out incremental grid processing on the whole point cloud data;
And synthesizing the at least two grid models to generate an integral grid model with multiple resolutions.
2. The method of claim 1, wherein gridding the global point cloud data with at least two different resolutions results in at least two grid models of the object under test, comprising:
Determining and dividing original point cloud data falling in the area to be adjusted from the whole point cloud data;
and carrying out gridding processing on the original point cloud data in the area to be regulated by using a second resolution to obtain a second grid model of the area to be regulated, wherein the whole point cloud data comprises the point cloud data in the area to be regulated.
3. The method of claim 2, wherein synthesizing the at least two mesh models to generate an overall mesh model having multiple resolutions, comprises:
cutting off the original grid of the region to be adjusted from the first grid model to generate a grid model to be synthesized;
And synthesizing the second grid model and the grid model to be synthesized to generate the whole grid model.
4. The method of claim 2, wherein the non-adjusted regions of the first mesh model are determined by:
The method comprises the steps of obtaining integral point cloud data, reading the integral point cloud data, carrying out fusion processing on the integral point cloud data to generate a fourth-resolution integral grid, cutting the determined area to be adjusted from the low-resolution integral grid based on the information of the area to be adjusted, and generating a non-adjustment area in the first grid model.
5. The method of claim 2, wherein the first mesh model is a reconstructed coarse mesh model if the first resolution is a low resolution.
6. The method according to claim 2, wherein the method further comprises:
and determining a non-adjustment area in the first grid model, wherein the grid model of the non-adjustment area is a part of the first grid model except for the second grid model.
7. The method of claim 2, wherein after determining the region to be adjusted in the first mesh model, the method further comprises:
and carrying out gridding processing on the original point cloud data in the area to be regulated by using a third resolution to obtain a third grid model of the area to be regulated, wherein the whole point cloud data comprises the point cloud data in the area to be regulated.
8. The method according to claim 1, wherein in the process of sequentially gridding the entire point cloud data using a plurality of resolutions, the resolution used each time is sequentially increased, and the point cloud data currently subjected to gridding is part of the point cloud data subjected to the last gridding.
9. The method of claim 1, wherein acquiring global point cloud data for an object under test based on at least one set of object images of the object under test comprises:
scanning the object to be detected through a scanner device;
Forming original point cloud data by three-dimensional reconstruction of each group of scanned object images, wherein each group of scanned object images forms an original single point cloud data, and a plurality of groups of scanned object images form a plurality of pieces of original point cloud data;
And processing the plurality of pieces of original point cloud data through a unified coordinate system, and then splicing to obtain the integral point cloud data.
10. The method of claim 9, wherein the raw point cloud data comprises point cloud data generated from a single perspective acquisition.
11. The method of claim 2, wherein the second resolution is greater than the first resolution.
12. The method of claim 2, wherein gridding the overall point cloud data at the first resolution to obtain a first grid model of the object under test, comprising:
when a part of original point cloud is obtained through scanning, gridding the existing part of original point cloud;
And in the subsequent scanning process, on the basis of the previous grid, carrying out grid formation on the original point cloud newly generated by scanning to obtain a first grid model of the measured object.
13. The method of claim 2, wherein the first mesh model is a reconstructed coarse mesh model in the case where the first resolution is low resolution, and wherein determining the region to be adjusted in the first mesh model comprises:
identifying a feature type from the first grid model by adopting an identification model, and determining the region to be adjusted based on the identified feature type, wherein the identification model is a neural network model based on sample training, or
And selecting the region to be adjusted from the first grid model based on the received selection instruction, wherein the region to be adjusted is a region which needs to be subjected to fine gridding processing by adopting high resolution in the first grid model.
14. A method of generating a digitized model, comprising:
Acquiring integral point cloud data of a detected object based on at least one group of detected object images of the detected object;
Carrying out gridding processing on the integral point cloud data with a first resolution to obtain a first grid model of the object to be tested, wherein the first grid model comprises a first area needing to be subjected to gridding processing again, and the gridding processing adopts incremental gridding under the condition that the integral point cloud data is subjected to gridding processing for the first time;
Performing the gridding processing on the original point cloud data in the first area with a second resolution to obtain a second grid model;
synthesizing the second grid model with the grid models except the first area in the first grid model to generate an integral grid model;
The method comprises the steps of determining a first area in a first grid model, wherein the integral grid model is a dental digital model, the information of the first area comprises a range and an area type of the first area, the range of the first area is used for screening the first area from the integral point cloud data, the area type is used for determining the second resolution, determining the first area in the first grid model comprises the steps of identifying a characteristic type from the first grid model by adopting an identification model and determining the first area based on the identified characteristic type, wherein the identification model is a neural network model based on sample training, or the first area is selected from the first grid model based on a received selection instruction, wherein the first area is the area in the first grid model, which needs to be subjected to fine grid processing by adopting high resolution.
15. The method of claim 14, wherein the second resolution is high resolution in the case where the first resolution is low resolution, the first mesh model is a coarse mesh model, and the second mesh model is a fine mesh model.
16. The method of claim 14, wherein the first region comprises a region to be adjusted, wherein integrating the second mesh model with the mesh model of the first mesh model other than the first region, and wherein generating the overall network model comprises:
and replacing an original grid model of the region to be adjusted in the first grid model by using the second grid model.
17. An apparatus for generating a digitized model, comprising:
the first acquisition module is used for acquiring the whole point cloud data of the detected object based on at least one group of detected object images of the detected object;
The first processing module is used for carrying out gridding processing on the whole point cloud data with at least two different resolutions to obtain at least two grid models of the measured object, and comprises the steps of carrying out gridding processing on the whole point cloud data with a first resolution to obtain a first grid model of the measured object, determining a region to be adjusted in the first grid model, generating a second grid model with a second resolution corresponding to the region to be adjusted according to the information of the region to be adjusted, wherein the grid model is a dental digital model, the information of the region to be adjusted comprises a range and a region type of the region to be adjusted, wherein the range of the region to be adjusted is used for screening the region to be adjusted from the whole point cloud data, and the region type is used for determining the second resolution;
And the first generation module is used for synthesizing the at least two grid models to generate an integral grid model with multiple resolutions.
18. A non-volatile storage medium, characterized in that the non-volatile storage medium comprises a stored program, wherein the program, when run, controls a device in which the non-volatile storage medium is located to perform the method of generating a digitized model according to any one of claims 1 to 16.
19. A processor for executing a program, wherein the program when executed performs the method of generating a digitized model of any one of claims 1 to 16.
Applications Claiming Priority (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2020104767855 | 2020-05-29 | ||
| CN202010476785 | 2020-05-29 | ||
| CN202010641908 | 2020-07-06 | ||
| CN2020106419086 | 2020-07-06 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113744407A CN113744407A (en) | 2021-12-03 |
| CN113744407B true CN113744407B (en) | 2025-10-03 |
Family
ID=78728007
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010700841.9A Active CN113744407B (en) | 2020-05-29 | 2020-07-17 | Method and device for generating digital model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113744407B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103413357A (en) * | 2013-08-09 | 2013-11-27 | 江苏普旭软件信息技术有限公司 | Construction method for forming square benchmark grid surface through point cloud |
| CN106709947A (en) * | 2016-12-20 | 2017-05-24 | 西安交通大学 | RGBD camera-based three-dimensional human body rapid modeling system |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9083960B2 (en) * | 2013-01-30 | 2015-07-14 | Qualcomm Incorporated | Real-time 3D reconstruction with power efficient depth sensor usage |
| US10580143B2 (en) * | 2015-11-04 | 2020-03-03 | Intel Corporation | High-fidelity 3D reconstruction using facial features lookup and skeletal poses in voxel models |
| US10438408B2 (en) * | 2017-07-28 | 2019-10-08 | The Boeing Company | Resolution adaptive mesh for performing 3-D metrology of an object |
| CN109190573B (en) * | 2018-09-12 | 2021-11-12 | 阿波罗智能技术(北京)有限公司 | Ground detection method applied to unmanned vehicle, electronic equipment and vehicle |
| CN109410320A (en) * | 2018-09-30 | 2019-03-01 | 先临三维科技股份有限公司 | Method for reconstructing three-dimensional model, device, computer equipment and storage medium |
-
2020
- 2020-07-17 CN CN202010700841.9A patent/CN113744407B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103413357A (en) * | 2013-08-09 | 2013-11-27 | 江苏普旭软件信息技术有限公司 | Construction method for forming square benchmark grid surface through point cloud |
| CN106709947A (en) * | 2016-12-20 | 2017-05-24 | 西安交通大学 | RGBD camera-based three-dimensional human body rapid modeling system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113744407A (en) | 2021-12-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| AU2021206867B2 (en) | Automatic selection and locking of intraoral images | |
| JP7386215B2 (en) | Method and device for removal of dental mesh orthodontic appliances | |
| EP2742857B1 (en) | Integration of intra-oral imagery and volumetric imagery | |
| EP2560572B1 (en) | Reduction and removal of artifacts from a three-dimensional dental x-ray data set using surface scan information | |
| WO2021238673A1 (en) | Tooth scanning method and apparatus, computer device, and computer-readable storage medium | |
| DE102019106666A1 (en) | Dental CAD automation using deep learning | |
| Tian et al. | Efficient computer-aided design of dental inlay restoration: a deep adversarial framework | |
| US20060183082A1 (en) | Method and computer system for creating a dental restoration model | |
| CN112515787B (en) | Three-dimensional dental data analysis method | |
| CN113744407B (en) | Method and device for generating digital model | |
| KR20240025109A (en) | Method and system for automating dental crown design image based on artificial intelligence | |
| US12118670B2 (en) | Method and device for generating digital model | |
| EP4280168A1 (en) | Automated methods for designing and fabricating an intra-oral appliance | |
| CN119649973A (en) | A method for generating a gum filling network, a method for filling gums and a device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |