CN115641583A - Point cloud detection method, system and medium based on self-supervision and active learning - Google Patents
Point cloud detection method, system and medium based on self-supervision and active learning Download PDFInfo
- Publication number
- CN115641583A CN115641583A CN202211671076.8A CN202211671076A CN115641583A CN 115641583 A CN115641583 A CN 115641583A CN 202211671076 A CN202211671076 A CN 202211671076A CN 115641583 A CN115641583 A CN 115641583A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- initial
- point
- self
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 184
- 238000012549 training Methods 0.000 claims abstract description 69
- 238000002372 labelling Methods 0.000 claims abstract description 52
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 51
- 238000000034 method Methods 0.000 claims abstract description 39
- 238000012545 processing Methods 0.000 claims abstract description 32
- 230000008859 change Effects 0.000 claims description 71
- 238000004364 calculation method Methods 0.000 claims description 40
- 238000000605 extraction Methods 0.000 claims description 30
- 238000012544 monitoring process Methods 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 12
- 230000009466 transformation Effects 0.000 claims description 10
- 238000006243 chemical reaction Methods 0.000 claims description 7
- 238000012163 sequencing technique Methods 0.000 claims description 6
- 230000000052 comparative effect Effects 0.000 claims description 4
- 125000004122 cyclic group Chemical group 0.000 claims description 4
- 239000000284 extract Substances 0.000 claims description 4
- 230000011218 segmentation Effects 0.000 claims description 4
- 238000004590 computer program Methods 0.000 claims description 2
- 238000012216 screening Methods 0.000 claims description 2
- 238000010187 selection method Methods 0.000 abstract description 4
- 230000008569 process Effects 0.000 description 23
- 238000007781 pre-processing Methods 0.000 description 6
- 238000013501 data transformation Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 4
- 238000011179 visual inspection Methods 0.000 description 4
- 239000003086 colorant Substances 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013136 deep learning model Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000003416 augmentation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Images
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a point cloud detection method, a point cloud detection system and a point cloud detection medium based on self-supervision and active learning, wherein the method comprises the following steps: acquiring original point cloud data; extracting point cloud initial features of original point cloud data based on a self-supervision learning algorithm; performing point cloud over-segmentation processing on the original point cloud data to obtain initial super points; distributing characteristics for the initial super points based on the point cloud initial characteristics to obtain initial super point characteristics; acquiring an initial labeling sample based on the initial super point feature and a mean value clustering algorithm; training an initial point cloud detection model based on the initial labeling sample; performing loop iteration operation on the initial point cloud detection model to obtain a final point cloud detection model; performing point cloud detection based on the final point cloud detection model; the method can obtain better model initialization and initial sample selection based on the self-supervision learning algorithm, and meanwhile, combines a high-value sample selection method of uncertain active learning, and can give consideration to both detection cost and detection performance when point cloud detection is applied.
Description
Technical Field
The invention relates to the technical field of visual inspection of industrial machines, in particular to a point cloud detection method, a point cloud detection system and a point cloud detection medium based on self-supervision and active learning, which are applied to the field of three-dimensional visual inspection.
Background
At present, a deep learning technology is widely applied to visual industrial detection, a current deep learning model is mainly based on big data drive and strongly depends on the quantity and accuracy of a labeled data set, the problem is particularly highlighted under three-dimensional point cloud data, and the specific problems are as follows:
on the first hand, because the number of the three-dimensional data points is usually extremely large, in the process of labeling the three-dimensional point cloud, a annotator needs to execute a plurality of two-dimensional labels from different visual angles in the process of labeling the three-dimensional point cloud, or the labels are executed on a three-dimensional space through the scaling of the point cloud and the adjustment of the size of a labeling brush, the operations obviously increase the time and the cost of labeling the three-dimensional point cloud, the professional requirements on the annotator in the industrial detection field are extremely high, the annotator is required to have expert knowledge, the annotator is required to be familiar with a complex three-dimensional labeling process, and the difficulty in acquiring the three-dimensional labeling data is further increased;
secondly, the existing three-dimensional visual industrial detection method is mainly based on full supervision for training and is limited by the difficulty of data acquisition and labeling, so that the existing deep learning model cannot obtain a good detection effect and a high detection precision;
in summary, in the field of three-dimensional industrial visual inspection, a pair of spears is formed between the difficulty and cost of data labeling and the inspection precision of the final inspection model, which finally results in poor applicability and poor touchdown of the current deep learning-based three-dimensional visual inspection scheme.
Disclosure of Invention
The invention aims to provide a point cloud detection method, a point cloud detection system and a point cloud detection medium based on self-supervision and active learning, aiming at the problem of contradiction between data labeling cost and model precision in the prior art, so that higher detection precision is achieved under limited labeling data, and the problem of contradiction between cost and performance in practical application is greatly relieved.
In order to solve the technical problems, the specific technical scheme of the invention is as follows:
in one aspect, the invention provides a point cloud detection method based on self-supervision and active learning, which comprises the following steps:
a feature acquisition step based on self-supervision:
acquiring original point cloud data, wherein the original point cloud data is three-dimensional point cloud data;
extracting initial point cloud features of the original point cloud data based on a self-supervised learning algorithm;
model training step based on the super point segmentation:
performing point cloud over-segmentation processing on the original point cloud data to obtain an initial super point;
distributing characteristics for the initial super points based on the point cloud initial characteristics to obtain initial super point characteristics;
obtaining an initial labeling sample based on the initial super point feature and a mean value clustering algorithm;
training an initial point cloud detection model based on the initial labeling sample;
model iteration step based on autonomous learning:
executing a cyclic iteration operation based on autonomous learning on the initial point cloud detection model to obtain a final point cloud detection model;
the cycle iterative operation based on the autonomous learning comprises the following steps: calculating a high contribution degree overtime point in the unmarked overtime points of the initial overtime point based on an uncertainty active learning algorithm; marking the high-contribution-degree super point, and updating the initial marked sample based on the marked high-contribution-degree super point; iterating the initial point cloud detection model based on the updated initial labeling sample;
point cloud detection:
and carrying out point cloud detection based on the final point cloud detection model.
As an improved solution, the self-supervised learning algorithm includes:
setting an initial self-monitoring model;
carrying out data conversion operation on the original point cloud data to obtain a conversion point cloud pair;
training the initial self-supervision model by adopting the transformation point cloud pair and based on a contrast learning strategy to obtain a trained self-supervision model;
and extracting the characteristics of the original point cloud data by adopting the characteristic extraction network of the trained self-supervision model to obtain the initial characteristics of the point cloud.
As an improvement, the loss function of the comparison learning strategy is:
in the loss function, q is the anchor point,
in order to be a positive sample point,
a set of positive and negative sample points,
is the temperature coefficient;
the training of the initial self-supervision model by adopting the transformation point cloud pair and based on a contrast learning strategy to obtain a trained self-supervision model comprises the following steps:
taking the transformed point cloud pair as an automatic supervision training set;
based on the loss function, the initial self-supervision model is trained on the self-supervision training set according to comparative learning loss;
and training the initial self-monitoring model to the time of convergence of the comparison learning loss to obtain the trained self-monitoring model.
As an improved solution, the point cloud over-segmentation process includes:
acquiring characteristic information of the original point cloud data;
dividing the original point cloud data into a plurality of sub-point cloud blocks according to the characteristic information;
and setting the sub point cloud blocks as the initial super points respectively.
As an improved scheme, the obtaining an initial annotation sample based on the initial hyper-point feature and a mean value clustering algorithm includes:
setting the clustering number and the number of the over-point extractions;
clustering the initial over point characteristics by adopting the mean value clustering algorithm according to the clustering number to obtain a clustering cluster;
extracting first initial super point features from the cluster according to the super point extraction quantity;
setting the initial overtopping point corresponding to the first initial overtopping point characteristic as the overtopping point to be marked;
marking the over points to be marked;
and enabling the marked overtoints to be marked to be the initial marked sample.
As an improved scheme, the unmarked super point is as follows: initial overtops except the overtops to be marked;
the uncertainty active learning algorithm comprises:
calculating the region entropy, the region surface change rate and the region color change rate of the unmarked super points;
performing weighted calculation based on the region entropy, the region surface change rate and the region color change rate to obtain an uncertainty value of the unmarked overtop, wherein the uncertainty value and the contribution degree of the unmarked overtop are in a direct proportion relation;
setting an active learning budget, and sequencing the uncertainty values of the unmarked over points;
and screening partial overtemperature from the unmarked overtemperature as the high contribution degree overtemperature according to the sequencing result of the uncertainty value and the active learning budget.
As an improvement, the iterating the initial point cloud detection model based on the updated initial labeling sample includes:
setting the expected performance of the model;
when the initial point cloud detection model is iterated, judging whether the iterated initial point cloud detection model meets the expected performance of the model or not, and judging whether the unmarked overtop exists or not;
and when the initial point cloud detection model meets the expected performance of the model or the unmarked super points do not exist, setting the initial point cloud detection model as the final point cloud detection model.
As an improved scheme, the calculating the entropy, the surface change rate and the color change rate of the area of the unmarked super point comprises the following steps:
confirming the over point area without the over point mark;
acquiring a point information entropy in the super point region, and obtaining the region entropy based on the average calculation of the point information entropy;
obtaining the point surface change rate in the over point area, and obtaining the area surface change rate based on the average calculation of the point surface change rate;
and acquiring point color change data in the over point area, and obtaining the area color change rate based on average calculation of the point color change data.
On the other hand, the invention also provides a point cloud detection system based on self-supervision and active learning, which comprises the following components:
the system comprises a feature extraction module, an initial training module, a model iteration module and a point cloud detection module;
the characteristic extraction module is used for acquiring original point cloud data; the feature extraction module extracts point cloud initial features of the original point cloud data based on a self-supervision learning algorithm;
the initial training module is used for carrying out point cloud over-segmentation processing on the original point cloud data to obtain an initial super point; the initial training module distributes characteristics for the initial super points based on the point cloud initial characteristics to obtain initial super point characteristics; the initial training module obtains an initial labeling sample based on the initial super point feature and a mean value clustering algorithm; the initial training module trains an initial point cloud detection model based on the initial labeling sample;
the model iteration module is used for executing a cyclic iteration operation based on autonomous learning on the initial point cloud detection model to obtain a final point cloud detection model; the cycle iterative operation based on the autonomous learning comprises the following steps: the model iteration module calculates a high-contribution-degree super point based on an uncertainty active learning algorithm and an unmarked super point in the initial super points; the model iteration module is used for marking the high-contribution-degree super point, and updating the initial marking sample based on the marked high-contribution-degree super point; the model iteration module iterates the initial point cloud detection model based on the updated initial labeling sample;
and the point cloud detection module is used for carrying out point cloud detection according to the final point cloud detection model.
In another aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the point cloud detection method based on self-supervision and active learning.
The technical scheme of the invention has the beneficial effects that:
1. the point cloud detection method based on self-supervision and active learning can achieve better model initialization and initial sample selection based on a self-supervision learning algorithm, and meanwhile, the problem of difficult labeling in industrial detection is solved by combining a high-value sample selection method of uncertain active learning, so that higher detection precision is achieved under limited labeling data, the problem of contradiction between cost and performance in practical application is greatly relieved, the performance and precision of a final detection model are guaranteed to a great extent through continuous model iteration, the detection precision in the application of final point cloud detection is improved, and the point cloud detection method based on self-supervision and active learning has higher applicability and application value.
2. The point cloud detection system based on the self-supervision and the active learning can obtain better model initialization and initial sample selection based on the self-supervision learning algorithm through the mutual matching of the feature extraction module, the initial training module, the model iteration module and the point cloud detection module, and simultaneously, the problem of difficult labeling in industrial detection is solved by combining a high-value sample selection method of the uncertain active learning, so that higher detection precision is achieved under limited labeling data, the contradiction between the cost and the performance in practical application is greatly relieved, the performance and the precision of a final detection model are ensured to a great extent through continuous model iteration, the detection precision in the final point cloud detection application is improved, and the point cloud detection system has higher applicability and application value.
3. The computer-readable storage medium can realize the cooperation of a guide feature extraction module, an initial training module, a model iteration module and a point cloud detection module, and further realize the point cloud detection method based on self-supervision and active learning.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
Fig. 1 is a schematic flow chart of a point cloud detection method based on self-supervision and active learning according to embodiment 1 of the present invention;
FIG. 2 is a detailed flowchart of a point cloud detection method based on self-supervision and active learning according to embodiment 1 of the present invention;
FIG. 3 is a schematic logic flow diagram of a point cloud detection method based on self-supervision and active learning according to embodiment 1 of the present invention;
FIG. 4 is a logic diagram of the self-supervised learning algorithm in the point cloud detection method based on self-supervision and active learning according to embodiment 1 of the present invention;
fig. 5 is a schematic structural diagram of a point cloud detection system based on self-supervision and active learning according to embodiment 2 of the present invention.
Detailed Description
The following detailed description of the preferred embodiments of the present invention, taken in conjunction with the accompanying drawings, will make the advantages and features of the invention easier to understand by those skilled in the art, and thus will clearly and clearly define the scope of the invention.
In the description of the present invention, it should be noted that the described embodiments of the present invention are a part of the embodiments of the present invention, and not all embodiments; all other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The terms "first," "second," and the like in the description and in the claims, as well as in the drawings, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments herein described are capable of operation in sequences other than those illustrated or described herein. Moreover, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, apparatus, article, or device that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or device.
Example 1
The embodiment provides a point cloud detection method based on self-supervision and active learning, as shown in fig. 1 to 4, including the following steps:
s100, a feature obtaining step based on self-supervision specifically comprises the following steps:
s110, acquiring original point cloud data; extracting point cloud initial features of the original point cloud data based on a self-supervision learning algorithm;
as an embodiment of the invention, the original point cloud data is acquired by an industrial camera in a way of acquiring the point cloud data, wherein when the industrial camera acquires the point cloud data, the adopted industrial camera can select an RGBD (red, green and blue) camera or a laser radar and the like, and the specific point cloud acquisition precision can be set according to actual requirements; in the acquisition process, the directly obtained point cloud data cannot be used as final original point cloud data, and corresponding pretreatment operation is required; because large-scale high-precision point cloud is limited by the limitations of current GPU storage and computational power, the point cloud is generally required to be subjected to down-sampling or clipping, and further the consumption of computational resources is reduced; in this embodiment, the preprocessing operation includes down-sampling, and the down-sampling may be selected from various schemes, such as: voxel grid downsampling, uniform downsampling, random downsampling and the like; optionally, the preprocessing operation comprises point cloud cutting, and during cutting, cutting based on a cutting box is usually performed by taking an interested target as a center; however, the above preprocessing operation is only suitable for the case that the computing resources cannot meet the requirements, but when the computing resources can achieve the purpose of processing large-scale high-precision point cloud indexes, the original point cloud data collected by the industrial camera can be directly used as input to be used as corresponding original point cloud data; optionally, the preprocessing operation further includes: point cloud filtering denoising, point cloud distortion compensation, internal empty point compensation and the like, wherein during actual operation, the preprocessing operation is adaptively selected according to the specific characteristics of the original point cloud and the actual requirements; when actual training deployment is performed, data after preprocessing operation is used as the original point cloud data, and the original point cloud data is manually divided into a training set and a verification set, for example, the ratio between the training set and the verification set is about 7:3;
as an embodiment of the present invention, the self-supervised learning algorithm includes: setting an initial self-monitoring model; in the embodiment, the initial self-supervision model adopts a feature extractor based on self-supervision learning or other arbitrary three-dimensional point cloud feature extraction networks; optionally, in actual operation, the feature extractor may be PointNet + + or 3D UNet, etc.; in this embodiment, the following steps will be described with 3D UNet as an example; in this example, the structure of the used 3D UNet is exactly the same as UNet, and only 3D sparse convolution is used to replace 2D convolution in UNet; then, carrying out data transformation operation on the original point cloud data to obtain a transformation point cloud pair; in the present embodiment, the data transformation operation is a transformation augmentation operation performed on the original point cloud data, and the transformation in this operation includes, but is not limited to, rotation, translation, size scaling, and the like; as shown in fig. 4, after the original point cloud data is subjected to random data transformation operation, two transformed point cloud data (i.e., transformed point cloud pairs) are obtained, i.e., transformed point cloud 1 and transformed point cloud 2 in fig. 4; adopting the transformation point cloud pair, and training the initial self-supervision model based on a contrast learning strategy to obtain a trained self-supervision model, wherein the specific training process is as follows: because the transformed point cloud 1 and the transformed point cloud 2 are from the same original point cloud data, each point between the transformed point cloud pair can realize one-to-one correspondence, and because the transformed point cloud 1 and the transformed point cloud 2 pass through a 3D UNet sharing model parameters, point-to-point characteristics of the transformed point cloud 1 and the transformed point cloud 2 can be obtained from the last layer of convolution layer of the 3D UNet; based on the above, a method of contrast learning between transformed point cloud pairs is adopted to approximate the characteristics of the transformed point cloud pairs in matching and push away the characteristics of the transformed point cloud pairs in mismatching, thereby realizing self-supervision learning training; further, the loss function of the comparative learning strategy is as follows:
wherein, q is an anchor point,
for positive sample points (i.e. transform match points),
as a set of positive and negative sample points (i.e. transform mismatch points),
a temperature coefficient is expressed, which determines the attention degree of the contrast loss to the difficult negative samples; in the Loss function formula, contrast learning is considered as a classification problem, and model building is carried out by adopting Softmax Loss; based on the loss function, the transformed point cloud pair is used as a self-supervision training set of self-supervision training, then the initial self-supervision model is used for carrying out self-supervision training on the self-supervision training set by using contrast learning loss until the initial self-supervision model is trained until the contrast learning loss is converged, the initial self-supervision model is judged to be trained completely, and the trained initial self-supervision model is used as an extraction model of point cloud initial features, namely the trained self-supervision model; subsequently, extracting the characteristics of each point cloud in the original point cloud data by adopting a characteristic extraction network of the trained self-supervision model to obtain the initial characteristics of each point cloud; in the actual operation in the industrial detection field, the time and resources consumed for making a complete large-scale labeled data set are generally unacceptable, but the acquisition of unlabeled data is easy, and the data can be easily acquired in the actual operation process, however, most of the existing deep learning methods do not consider to utilize the unlabeled data, but simply discard the unlabeled data, so in the step S100, the unlabeled data is utilized, the pre-training of the feature extraction model and the extraction of point cloud features are carried out in combination with the self-supervision learning algorithm, and the original data do not need to be manually labeled in the process, and the self-supervision learning algorithm is used for carrying out self-supervision on the unlabeled data, so that the unlabeled data are better monitored and the self-supervision is carried out in the self-supervision processThe deep learning feature extractor is optimized, so that tasks such as downstream detection and segmentation benefit; moreover, unlike the two-dimensional image which usually adopts a pre-training model on a large-scale data set as initialization, the current three-dimensional point cloud data does not have a good pre-training model, and the initialization problem of a three-dimensional data feature extractor can be relieved to a certain extent by self-supervision training;
s200, model training step based on the over-point segmentation, specifically comprising:
s210, performing point cloud over-segmentation processing on the original point cloud data to obtain initial super points; distributing characteristics for the initial super points based on the point cloud initial characteristics to obtain initial super point characteristics; obtaining an initial labeling sample based on the initial super point feature and a mean value clustering algorithm; training an initial point cloud detection model based on the initial labeling sample;
as an embodiment of the present invention, the point cloud over-segmentation process includes: acquiring characteristic information of the original point cloud data; in this embodiment, the feature information is a local structure or color corresponding to the original point cloud data, wherein the color element is only used as a participating element of the feature information when the original point cloud data has a color; dividing the original point cloud data into a plurality of simple and meaningful sub-point cloud blocks according to the characteristic information; setting a plurality of sub-point cloud blocks as a plurality of initial super points respectively; in this embodiment, a plurality of the initial super points have the same semantic information and geometric structure, and adjacent initial super points have different geometric structures; the initial super-point is a result of clustering according to coordinates and colors (under the condition that the original point cloud data has colors) of each point and adjacent points in the original point cloud data, and the super-point enables the points with similar structures and colors to be divided together; the 'super point' is mainly used as a basic marking unit in the application, and in the actual operation process, all points in a super point area can be quickly marked by clicking the super point area corresponding to the super point once; in the embodiment, the process of point cloud over-segmentation is similar to the process of obtaining sub-pixel blocks after the two-dimensional image is over-segmented by a super-pixel finger; in the embodiment, point Cloud over-segmentation is an unsupervised process, and can be realized by using a VCSS algorithm or an SPG algorithm in a PCL (Point Cloud Library) Library constructed based on python; for the 3D UNet model in the embodiment, a VCSS algorithm is selected to generate the initial overtemperature, and the adjacency relation between each initial overtemperature is stored by using an octree; since the point cloud initial features can be obtained by the self-supervision model, the initial super points only divide the original point cloud data and do not change the properties of the points, so that the point cloud initial features of each initial super point are known;
as an embodiment of the present invention, since the number of the initial super points is several, when performing feature allocation, corresponding features are allocated to the several initial super points according to several point cloud initial features corresponding to the several initial super points, so as to finally obtain several initial super point features;
as an embodiment of the present invention, the obtaining an initial labeled sample based on the initial hyper-point feature and the mean clustering algorithm includes: setting a certain amount of clustering number and a certain amount of over-point extraction number according to the requirements in actual operation; clustering a plurality of initial over point characteristics by adopting the mean value clustering algorithm according to the clustering number to obtain a plurality of clustering clusters; after clustering, proportionally and randomly extracting a plurality of first initial super point features matched with the super point extraction quantity from each clustering cluster; setting a plurality of initial overtemperature points corresponding to the plurality of first initial overtemperature point characteristics as a plurality of overtemperature points to be marked respectively; marking the plurality of to-be-marked over points by professional markers respectively; enabling the marked multiple to-be-marked over points to be multiple initial marked samples; the initial labeling samples can form the initial labeling training data set/initial labeling data set;
as an embodiment of the present invention, an initial point cloud detection model is trained based on the initial labeling sample; in the application, the trained self-monitoring model is used for trunk parameter initialization, namely, an initial point cloud detection model is constructed in a mode of adding a classification layer in the trained self-monitoring model; when an initial point cloud detection model is trained on the basis of the initial labeling sample, fine tuning training is carried out on the trained self-supervision model added with the classification layer; the reason for adding the classification layer is that the trained self-monitoring model is only used as a feature extractor, the model needs to be adjusted and modified according to actual detection needs, the mode of adding the classification layer is only used as an example, and when other needs exist, the self-monitoring model is also modified for other purposes, so that the subsequent detection needs are met; it should be noted that, since the super point of the initial labeled sample is an aggregate of a group of points, during training, the class of the super point needs to be determined, and each point in the super point should be assigned with the class label that is the same as the class of the super point;
s300, model iteration step based on autonomous learning, specifically comprising:
s310, executing a loop iteration operation based on autonomous learning on the initial point cloud detection model to obtain a final point cloud detection model; the cycle iterative operation based on the autonomous learning comprises the following steps: calculating a high contribution degree overtime point in the unmarked overtime points of the initial overtime point based on an uncertainty active learning algorithm; marking the high-contribution-degree super point, and updating the initial marked sample based on the marked high-contribution-degree super point; iterating the initial point cloud detection model based on the updated initial labeling sample; a final point cloud detection model, namely a point cloud detection model put into final defect detection (industrial detection);
as an embodiment of the present invention, the unmarked super point is: the rest initial overtopping points except the overtopping point to be marked in the plurality of initial overtopping points;
as an embodiment of the invention, the loop iteration operation based on the autonomous learning means that the nodes with higher contribution degree are selected from the unmarked initial nodes in a loop, and after each selection, the selected nodes are added into the initial marking sample of the step S200 to fine-tune the initial point cloud detection model, so as to improve the detection performance of the initial point cloud detection model continuously;
as an embodiment of the present invention, the uncertainty active learning algorithm includes: calculating the region entropy, the region surface change rate and the region color change rate of each un-marked over point;
specifically, in the present embodiment, the entropy H of the region without the super point is described n Area surface change rate V n And rate of change of area color C n As an index for mainly measuring the uncertainty of a certain over point; firstly, confirming the over-point area without over-points; in the calculation region of entropy H n Then, acquiring a point information entropy in the over-point region, and based on the average calculation of the point information entropy, further acquiring the region entropy; specifically, the information entropy is a typical index for evaluating the uncertainty of the information, the larger the information entropy is, the higher the uncertainty is, in the application, the Softmax information entropy is adopted, and when the average calculation of the point information entropy is performed, a preset regional entropy H needs to be introduced n Calculating a formula; regional entropy H n The calculation formula is as follows:
wherein the region entropy H n The principle of the calculation formula is that firstly, a model obtained in the last uncertainty active learning algorithm (the initial point cloud detection model is adopted for the first time) is used for obtaining the soft maximum probability of all point clouds in the unmarked super points; then, a soft maximum probability P of the point cloud i is given i By averaging belonging to a region S of excess points n Computing the nth super-point region S by the information entropy of the middle point n Regional entropy H of (1) n ；
Specifically, the surface change rate V is calculated in the area n Then, acquiring the point surface change rate in the over point area, and further acquiring the area surface change rate based on the average calculation of the point surface change rate; in actual operation, the area surface change rate is used to evaluate the degree of surface change of a certain point in the point cloud with respect to its neighboring points, i.e. how much the point is geometrically different from the neighboring points, and the complex surface area and boundary position may represent a semantically uncertain part, which is called the surface change rate; wherein, the point is estimated by mainly utilizing principal component analysis of the neighborhood of the local point of the pointSampling local curved surface characteristics on a curved surface; when the average calculation of the point surface change rate is carried out, a preset area surface change rate calculation formula is introduced:
wherein the principle of the area surface change rate calculation formula is that
Representing the surface rate of change of point i, the region S of overtint n Surface change rate V of medium overall area n Surface rate of change for each point in the area
Average value of (d);
specifically, the color change rate C is calculated in the calculation region n Then, acquiring point color change data in the over point area, and based on average calculation of the point color change data, further obtaining the area color change rate; correspondingly, the color change rate C of the region n Only aiming at the available original point cloud data of the color features, the color change rate can be additionally added as a judgment factor of the over-point uncertainty; specifically, the color change rate is used for evaluating the color change degree of a certain point in the point cloud relative to its neighboring points, namely the color difference degree between the point and its neighboring points; introducing a region color change rate C during average calculation of point color change data n Calculating the formula:
wherein, the region color change rate C n The principle of the calculation formula is that for all points with color intensity value I in the point cloud, 1 norm color difference between a point I and the nearest k adjacent point is calculated through a pair of super point regions S n Averaging the color change rate values of all points in the region to generate the nth super-point region S n Color change rate of (C) n ；
Specifically, after the region entropy, the region surface change rate and the region color change rate are calculated, proportional weighting calculation is performed based on the region entropy, the region surface change rate and the region color change rate to obtain an uncertainty value of the unmarked overtop, the uncertainty value and the contribution degree of the unmarked overtop are in a proportional relation, and the larger the uncertainty value is, the higher the contribution degree is, the higher the value is; the contribution degree of each over point is the contribution degree of the over point to the performance improvement of the detection model, and in the application, the most valuable over points are extracted as far as possible; therefore, when the analysis is carried out, an active learning budget is set, and the active learning budget is the required over-point quantity limited during the model iterative training; the uncertainty values of the unmarked overtops are sequenced from large to small according to the uncertainty values of the unmarked overtops, and a sequencing result of the corresponding uncertainty values from large to small is obtained; for example, when the active learning budget is 5, selecting unmarked outliers corresponding to the first 5 uncertainty values in the ranking result as training samples of the initial point cloud detection model (i.e., the screened partial outliers/high contribution outliers);
as an embodiment of the present invention, subsequently, the high-contribution-degree over point is labeled by an expert annotator, and the labeled high-contribution-degree over point is added to the initial labeled sample to complete the update of the initial labeled sample; training/iteratively training/fine-tuning the initial point cloud detection model again based on the updated initial labeling sample;
as an embodiment of the present invention, the iterating the initial point cloud detection model based on the updated initial labeling sample includes: setting the expected performance of the model; the expected performance of the model is the preset detection performance of the point cloud detection model to be achieved; when the initial point cloud detection model is iterated, judging whether the iterated initial point cloud detection model meets the expected performance of the model or not, and judging whether the unmarked overtop exists or not; when the initial point cloud detection model meets the expected performance of the model, the initial point cloud detection model after current iteration is proved to have reached the expected desired detection performance without iteration, when the unmarked overtop does not exist, all the initial overtops are proved to be used in the model iteration process, the marking budget is proved to be exhausted, at the moment, the iteration can only be stopped, and the initial point cloud detection model after current iteration is used as a final model; setting the initial point cloud detection model as the final point cloud detection model when the initial point cloud detection model meets the expected performance of the model or the unmarked overtop does not exist; according to the method, the model can be continuously subjected to fine tuning based on the cyclic iteration operation of the autonomous learning, and the over-point selection and marking are continuously performed according to uncertainty, so that a new training set fine tuning model is generated until the model reaches the expected performance or the marking budget is exhausted;
s400, point cloud detection, specifically comprising:
s410, point cloud detection is carried out based on the final point cloud detection model;
as an optional implementation manner of the present invention, the final point cloud detection model may be applied in the field of industrial detection, for example, point cloud detection of a PCB, when the final point cloud detection model is applied in the point cloud detection of the PCB, the initial labeling sample for performing iteration on the final point cloud detection model is also a point cloud data sample of the PCB, the original point cloud data in the method is also three-dimensional point cloud data about the PCB, and finally, the point cloud detection may be performed on the target PCB through the final point cloud detection model after iteration, and accurate point cloud data of the target PCB is output; of course, it is conceivable that the method may also be applied to the detection of other target objects in industrial inspection, such as the detection of defects in a certain component; it should be noted that the method focuses on an integrated point cloud detection strategy based on self-supervision and active learning, so that the detection model can achieve the balance between cost and performance as much as possible, and the performance and precision of the final detection model are ensured to the greatest extent.
Example 2
The present embodiment provides a point cloud detection system based on self-supervision and active learning based on the same inventive concept as the point cloud detection method based on self-supervision and active learning described in embodiment 1, as shown in fig. 5, including: the system comprises a feature extraction module, an initial training module, a model iteration module and a point cloud detection module;
the characteristic extraction module is used for acquiring original point cloud data; the feature extraction module extracts point cloud initial features of the original point cloud data based on a self-supervision learning algorithm;
as an embodiment of the present invention, the feature extraction module includes: a point cloud obtaining sub-module and a self-supervision learning sub-module; the point cloud obtaining submodule is used for obtaining original point cloud data; the self-supervision learning sub-module is used for extracting point cloud initial features of the original point cloud data based on a self-supervision learning algorithm;
as an embodiment of the present invention, the self-supervised learning submodule includes: the system comprises a model setting unit, a data transformation processing unit, a self-supervision training unit and a feature extraction control unit;
as an embodiment of the present invention, the self-supervised learning algorithm includes: the model setting unit sets an initial self-monitoring model; the data conversion processing unit performs data conversion operation on the original point cloud data to obtain a conversion point cloud pair; the self-supervision training unit adopts the transformed point cloud pair and trains the initial self-supervision model based on a contrast learning strategy to obtain a trained self-supervision model; and the feature extraction control unit adopts the feature extraction network of the trained self-supervision model to extract features of the original point cloud data to obtain initial features of the point cloud.
As an embodiment of the present invention, the loss function of the comparison learning strategy is:
as an embodiment of the present invention, the self-monitoring training unit uses the transformed point cloud pair, and trains the initial self-monitoring model based on a contrast learning strategy to obtain a trained self-monitoring model, including: the self-supervision training unit takes the transformation point cloud pair as a self-supervision training set; the self-supervision training unit leads the initial self-supervision model to train on the self-supervision training set according to the comparison learning loss based on the loss function; and training the initial self-monitoring model to the time of convergence of the comparison learning loss to obtain the trained self-monitoring model.
The initial training module is used for carrying out point cloud over-segmentation processing on the original point cloud data to obtain an initial super point; the initial training module distributes characteristics for the initial super points based on the point cloud initial characteristics to obtain initial super point characteristics; the initial training module obtains an initial labeling sample based on the initial super point feature and a mean value clustering algorithm; the initial training module trains an initial point cloud detection model based on the initial labeling sample;
as an embodiment of the present invention, the initial training module includes: the system comprises an over-segmentation processing sub-module, a super-point feature distribution sub-module, a clustering processing sub-module and a detection model training sub-module;
the over-segmentation processing sub-module is used for carrying out point cloud over-segmentation processing on the original point cloud data to obtain an initial over-point; the super point feature distribution submodule is used for distributing features for the initial super points based on the point cloud initial features to obtain initial super point features; the clustering processing submodule is used for obtaining an initial labeling sample based on the initial super point feature and a mean clustering algorithm; the detection model training submodule is used for training an initial point cloud detection model based on the initial labeling sample;
as an embodiment of the present invention, the point cloud over-segmentation process includes: the over-segmentation processing sub-module acquires the characteristic information of the original point cloud data; the over-segmentation processing submodule divides the original point cloud data into a plurality of sub-point cloud blocks according to the characteristic information; and the over-segmentation processing sub-module sets the sub-point cloud blocks to be the initial super-points respectively.
As an embodiment of the present invention, the clustering sub-module obtains an initial labeling sample based on the initial hyper-point feature and the mean clustering algorithm, and includes: the clustering processing submodule sets the clustering number and the over-point extraction quantity; clustering the initial over point characteristics by the clustering processing sub-module according to the clustering number by adopting the mean clustering algorithm to obtain a clustering cluster; the clustering processing submodule extracts a first initial super point feature from the clustering cluster according to the super point extraction quantity; the clustering processing submodule sets the initial overtopping point corresponding to the first initial overtopping point characteristic as the overtopping point to be marked; the clustering processing submodule labels the to-be-labeled over points; and the clustering processing submodule leads the marked over point to be marked to be the initial marking sample.
The model iteration module is used for executing a cycle iteration operation based on autonomous learning on the initial point cloud detection model to obtain a final point cloud detection model; the cycle iterative operation based on the autonomous learning comprises the following steps: the model iteration module calculates a high-contribution-degree super point based on an uncertainty active learning algorithm and an unmarked super point in the initial super points; the model iteration module is used for marking the high-contribution-degree super point, and updating the initial marking sample based on the marked high-contribution-degree super point; the model iteration module iterates the initial point cloud detection model based on the updated initial labeling sample;
as an embodiment of the present invention, the model iteration module includes: an uncertainty calculation sub-module, a super-point complex labeling sub-module, a sample set updating sub-module and an iteration processing sub-module; the uncertainty calculation submodule is used for calculating a high-contribution-degree super point based on an uncertainty active learning algorithm and an unmarked super point in the initial super points; the super-point repeated marking submodule is used for marking the high-contribution super-point, and the sample set updating submodule updates the initial marking sample based on the marked high-contribution super-point; the iteration processing sub-module iterates the initial point cloud detection model based on the updated initial labeling sample;
as an embodiment of the present invention, the unmarked super point is: initial overtops except the overtops to be marked;
as an embodiment of the present invention, the uncertainty active learning algorithm includes: the uncertainty calculation submodule calculates the region entropy, the region surface change rate and the region color change rate of the unmarked overtop; the uncertainty calculation submodule carries out weighting calculation on the basis of the region entropy, the region surface change rate and the region color change rate to obtain an uncertainty value of the unmarked overtop, and the uncertainty value and the contribution degree of the unmarked overtop are in a direct proportion relation; the uncertainty calculation submodule sets active learning budget and sequences uncertainty values of the unmarked over points; and the uncertainty calculation submodule screens partial overtops from the unmarked overtops according to the sequencing result of the uncertainty value and the active learning budget to serve as the high-contribution-degree overtops.
As an embodiment of the present invention, the iteration processing sub-module iterates the initial point cloud detection model based on the updated initial labeling sample, including: the iteration processing submodule sets the expected performance of the model; when the initial point cloud detection model is iterated, the iterative processing sub-module judges whether the iterated initial point cloud detection model meets the expected performance of the model and judges whether the unmarked overtop exists; and when the initial point cloud detection model meets the expected performance of the model or the unmarked super points do not exist, the iteration processing sub-module sets the initial point cloud detection model as the final point cloud detection model.
As an embodiment of the present invention, the uncertainty calculation sub-module calculates the region entropy, the region surface change rate, and the region color change rate of the unlabeled super point, and includes: the uncertainty calculation submodule confirms the over point area without the over point; the uncertainty calculation submodule acquires a point information entropy in the over-point region, and the uncertainty calculation submodule obtains the region entropy based on average calculation of the point information entropy; the uncertainty calculation submodule acquires the point surface change rate in the over point area, and the uncertainty calculation submodule obtains the area surface change rate based on the average calculation of the point surface change rate; and the uncertainty calculation submodule acquires point color change data in the over point area, and the uncertainty calculation submodule obtains the area color change rate based on average calculation of the point color change data.
And the point cloud detection module is used for carrying out point cloud detection according to the final point cloud detection model.
Example 3
The present embodiments provide a computer-readable storage medium comprising:
the storage medium is used for storing computer software instructions for implementing the point cloud detection method based on self-supervision and active learning of the embodiment 1, and comprises a program for executing the point cloud detection method based on self-supervision and active learning; specifically, the executable program may be embedded in the point cloud detection system based on self-supervision and active learning described in embodiment 2, so that the point cloud detection system based on self-supervision and active learning may implement the point cloud detection method based on self-supervision and active learning described in embodiment 1 by executing the embedded executable program.
Furthermore, the computer-readable storage medium of the present embodiments may take any combination of one or more readable storage media, where a readable storage medium includes an electronic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof.
Compared with the prior art, the point cloud detection method, system and medium based on self-supervision and active learning can obtain better model initialization and initial sample selection based on a self-supervision learning algorithm, meanwhile, the method is combined with a high-value sample selection method based on uncertain active learning, the problem of difficulty in labeling in industrial detection is solved, further, higher detection precision is achieved under limited labeling data, the problem of contradiction between cost and performance in practical application is greatly relieved, the performance and precision of a final detection model are guaranteed to a great extent through continuous model iteration, the detection precision in final point cloud detection application is improved, and the method has higher applicability and application value.
It should be understood that, in various embodiments herein, the sequence numbers of the above-mentioned processes do not mean the execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments herein.
It should also be understood that, in the embodiments herein, the term "and/or" is only one kind of association relation describing an associated object, meaning that three kinds of relations may exist. For example, a and/or B, may represent: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.
It can be clearly understood by those skilled in the art that, for convenience and simplicity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided herein, it should be understood that the disclosed system, apparatus, and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may also be an electric, mechanical or other form of connection.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purposes of the embodiments herein.
In addition, functional units in the embodiments herein may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present invention may be implemented in a form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods described in the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.
The above description is only an embodiment of the present invention, and not intended to limit the scope of the present invention, and all modifications of equivalent structures and equivalent processes performed by the present specification and drawings, or directly or indirectly applied to other related technical fields, are included in the scope of the present invention.
Claims (10)
1. A point cloud detection method based on self-supervision and active learning is characterized by comprising the following steps:
a feature acquisition step based on self-supervision:
acquiring original point cloud data, wherein the original point cloud data is three-dimensional point cloud data;
extracting point cloud initial features of the original point cloud data based on a self-supervision learning algorithm;
model training step based on the super point segmentation:
performing point cloud over-segmentation processing on the original point cloud data to obtain an initial super point;
distributing characteristics for the initial super points based on the point cloud initial characteristics to obtain initial super point characteristics;
obtaining an initial labeling sample based on the initial super point feature and a mean value clustering algorithm;
training an initial point cloud detection model based on the initial labeling sample;
model iteration step based on autonomous learning:
executing a cyclic iteration operation based on autonomous learning on the initial point cloud detection model to obtain a final point cloud detection model;
the cycle iterative operation based on the autonomous learning comprises the following steps: calculating a high-contribution-degree overtime point in the unmarked overtime points of the initial overtime point based on an uncertainty active learning algorithm; marking the high-contribution-degree super point, and updating the initial marked sample based on the marked high-contribution-degree super point; iterating the initial point cloud detection model based on the updated initial labeling sample;
point cloud detection:
and carrying out point cloud detection based on the final point cloud detection model.
2. The point cloud detection method based on self-supervision and active learning according to claim 1, characterized in that:
the self-supervision learning algorithm comprises the following steps:
setting an initial self-monitoring model;
carrying out data conversion operation on the original point cloud data to obtain a conversion point cloud pair;
training the initial self-supervision model by adopting the transformation point cloud pair and based on a contrast learning strategy to obtain a trained self-supervision model;
and extracting the characteristics of the original point cloud data by adopting the characteristic extraction network of the trained self-supervision model to obtain the initial characteristics of the point cloud.
3. The point cloud detection method based on self-supervision and active learning according to claim 2, characterized in that:
the loss function of the comparative learning strategy is as follows:
in the loss function, q is the anchor point,
in order to be a positive sample point,
a set of positive and negative sample points,
is the temperature coefficient;
the training of the initial self-supervision model by adopting the transformation point cloud pair and based on a contrast learning strategy to obtain a trained self-supervision model comprises the following steps:
taking the transformation point cloud pair as an automatic supervision training set;
based on the loss function, the initial self-supervision model is trained on the self-supervision training set according to comparative learning loss;
and training the initial self-monitoring model to the time of convergence of the comparison learning loss to obtain the trained self-monitoring model.
4. The point cloud detection method based on self-supervision and active learning according to claim 1, characterized in that:
the point cloud over-segmentation processing comprises the following steps:
acquiring characteristic information of the original point cloud data;
dividing the original point cloud data into a plurality of sub-point cloud blocks according to the characteristic information;
and setting the sub point cloud blocks as the initial super points respectively.
5. The point cloud detection method based on self-supervision and active learning according to claim 1, characterized in that:
the obtaining of the initial labeling sample based on the initial hyper-point feature and the mean value clustering algorithm comprises:
setting the clustering number and the number of the over-point extractions;
clustering the initial over point characteristics by adopting the mean value clustering algorithm according to the clustering number to obtain a clustering cluster;
extracting first initial super point features from the cluster according to the super point extraction quantity;
setting the initial overtop corresponding to the first initial overtop feature as the overtop to be marked;
marking the over points to be marked;
and enabling the marked overtoints to be marked to be the initial marked sample.
6. The point cloud detection method based on self-supervision and active learning according to claim 5, characterized in that:
the unmarked super points are as follows: initial overtops except the overtops to be marked;
the uncertainty active learning algorithm comprises:
calculating the region entropy, the region surface change rate and the region color change rate of the unmarked super points;
performing weighted calculation based on the region entropy, the region surface change rate and the region color change rate to obtain an uncertainty value of the unmarked overtop, wherein the uncertainty value and the contribution degree of the unmarked overtop are in a direct proportion relation;
setting an active learning budget, and sequencing the uncertainty values of the unmarked over points;
and screening partial overtops from the unmarked overtops according to the sequencing result of the uncertainty value and the active learning budget to serve as the high-contribution-degree overtops.
7. The point cloud detection method based on self-supervision and active learning according to claim 6, characterized in that:
iterating the initial point cloud detection model based on the updated initial annotation sample, including:
setting the expected performance of the model;
when the initial point cloud detection model is iterated, judging whether the iterated initial point cloud detection model meets the expected performance of the model or not, and judging whether the unmarked overtop exists or not;
and when the initial point cloud detection model meets the expected performance of the model or the unmarked super points do not exist, setting the initial point cloud detection model as the final point cloud detection model.
8. The point cloud detection method based on self-supervision and active learning according to claim 6, characterized in that:
the calculating the region entropy, the region surface change rate and the region color change rate of the unmarked super point comprises the following steps:
confirming the over point area without over point marking;
acquiring a point information entropy in the over-point region, and obtaining the region entropy based on average calculation of the point information entropy;
obtaining the point surface change rate in the over point area, and obtaining the area surface change rate based on the average calculation of the point surface change rate;
and acquiring point color change data in the over point area, and obtaining the area color change rate based on average calculation of the point color change data.
9. A point cloud detection system based on self-supervision and active learning is characterized by comprising: the system comprises a feature extraction module, an initial training module, a model iteration module and a point cloud detection module;
the characteristic extraction module is used for acquiring original point cloud data; the feature extraction module extracts point cloud initial features of the original point cloud data based on a self-supervision learning algorithm; the original point cloud data is three-dimensional point cloud data;
the initial training module is used for carrying out point cloud over-segmentation processing on the original point cloud data to obtain an initial super point; the initial training module distributes characteristics for the initial super points based on the point cloud initial characteristics to obtain initial super point characteristics; the initial training module obtains an initial labeling sample based on the initial super point feature and a mean value clustering algorithm; the initial training module trains an initial point cloud detection model based on the initial labeling sample;
the model iteration module is used for executing a cycle iteration operation based on autonomous learning on the initial point cloud detection model to obtain a final point cloud detection model; the cycle iterative operation based on the autonomous learning comprises the following steps: the model iteration module calculates a high-contribution-degree super point based on an uncertainty active learning algorithm and an unmarked super point in the initial super points; the model iteration module is used for marking the high-contribution-degree super point, and updating the initial marking sample based on the marked high-contribution-degree super point; the model iteration module iterates the initial point cloud detection model based on the updated initial labeling sample;
and the point cloud detection module is used for carrying out point cloud detection according to the final point cloud detection model.
10. A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, carries out the steps of the method for self-supervised and active learning based point cloud detection of any one of claims 1~8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211671076.8A CN115641583B (en) | 2022-12-26 | 2022-12-26 | Point cloud detection method, system and medium based on self-supervision and active learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211671076.8A CN115641583B (en) | 2022-12-26 | 2022-12-26 | Point cloud detection method, system and medium based on self-supervision and active learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115641583A true CN115641583A (en) | 2023-01-24 |
| CN115641583B CN115641583B (en) | 2023-03-31 |
Family
ID=84950038
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211671076.8A Active CN115641583B (en) | 2022-12-26 | 2022-12-26 | Point cloud detection method, system and medium based on self-supervision and active learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115641583B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116664825A (en) * | 2023-06-26 | 2023-08-29 | 北京智源人工智能研究院 | Self-supervision contrast learning method and system for large-scene point cloud object detection |
| CN116824189A (en) * | 2023-07-03 | 2023-09-29 | 浙江大学 | Data point semantic clustering active learning method and device based on space structure diversity |
| CN117894015A (en) * | 2024-03-15 | 2024-04-16 | 浙江华是科技股份有限公司 | Point cloud annotation data optimization method and system |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108319957A (en) * | 2018-02-09 | 2018-07-24 | 深圳市唯特视科技有限公司 | A kind of large-scale point cloud semantic segmentation method based on overtrick figure |
| CN110827302A (en) * | 2019-11-14 | 2020-02-21 | 中南大学 | Point cloud target extraction method and device based on depth map convolutional network |
| CN111091062A (en) * | 2019-11-21 | 2020-05-01 | 东南大学 | A Robot Out-of-Order Object Sorting Method Based on 3D Vision Clustering and Matching |
| CN113191387A (en) * | 2021-03-27 | 2021-07-30 | 西北大学 | Cultural relic fragment point cloud classification method combining unsupervised learning and data self-enhancement |
| CN114882020A (en) * | 2022-07-06 | 2022-08-09 | 深圳市信润富联数字科技有限公司 | Method, device and equipment for detecting defects of product and computer readable medium |
| CN114926469A (en) * | 2022-04-26 | 2022-08-19 | 中南大学 | Semantic segmentation model training method, semantic segmentation method, storage medium and terminal |
-
2022
- 2022-12-26 CN CN202211671076.8A patent/CN115641583B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108319957A (en) * | 2018-02-09 | 2018-07-24 | 深圳市唯特视科技有限公司 | A kind of large-scale point cloud semantic segmentation method based on overtrick figure |
| CN110827302A (en) * | 2019-11-14 | 2020-02-21 | 中南大学 | Point cloud target extraction method and device based on depth map convolutional network |
| CN111091062A (en) * | 2019-11-21 | 2020-05-01 | 东南大学 | A Robot Out-of-Order Object Sorting Method Based on 3D Vision Clustering and Matching |
| CN113191387A (en) * | 2021-03-27 | 2021-07-30 | 西北大学 | Cultural relic fragment point cloud classification method combining unsupervised learning and data self-enhancement |
| CN114926469A (en) * | 2022-04-26 | 2022-08-19 | 中南大学 | Semantic segmentation model training method, semantic segmentation method, storage medium and terminal |
| CN114882020A (en) * | 2022-07-06 | 2022-08-09 | 深圳市信润富联数字科技有限公司 | Method, device and equipment for detecting defects of product and computer readable medium |
Non-Patent Citations (1)
| Title |
|---|
| 刘耿欣: ""基于全局与局部特征对比的点云自监督学习"" * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116664825A (en) * | 2023-06-26 | 2023-08-29 | 北京智源人工智能研究院 | Self-supervision contrast learning method and system for large-scene point cloud object detection |
| CN116824189A (en) * | 2023-07-03 | 2023-09-29 | 浙江大学 | Data point semantic clustering active learning method and device based on space structure diversity |
| CN116824189B (en) * | 2023-07-03 | 2025-09-02 | 浙江大学 | Active learning method and device for semantic clustering of data points based on spatial structure diversity |
| CN117894015A (en) * | 2024-03-15 | 2024-04-16 | 浙江华是科技股份有限公司 | Point cloud annotation data optimization method and system |
| CN117894015B (en) * | 2024-03-15 | 2024-05-24 | 浙江华是科技股份有限公司 | Point cloud annotation data optimization method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115641583B (en) | 2023-03-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ibrahim et al. | Image segmentation methods based on superpixel techniques: A survey | |
| CN115641583B (en) | Point cloud detection method, system and medium based on self-supervision and active learning | |
| CN112347970B (en) | Remote sensing image ground object identification method based on graph convolution neural network | |
| Li et al. | An overlapping-free leaf segmentation method for plant point clouds | |
| US20110216976A1 (en) | Updating Image Segmentation Following User Input | |
| JP2005129065A (en) | Object extraction based on color and visual texture | |
| CN106611423B (en) | SAR image segmentation method based on ridge ripple filter and deconvolution structural model | |
| CN107180426A (en) | Area of computer aided Lung neoplasm sorting technique based on transportable multiple-model integration | |
| CN102915541B (en) | Multi-scale image segmenting method | |
| CN108537102A (en) | High Resolution SAR image classification method based on sparse features and condition random field | |
| CN105118090A (en) | Adaptive point-cloud filtering method for complex terrain structure | |
| CN106611420A (en) | SAR image segmentation method based on deconvolution network and sketch direction constraint | |
| Rios et al. | Feature visualization for 3D point cloud autoencoders | |
| CN109712150A (en) | Optical microwave image co-registration method for reconstructing and device based on rarefaction representation | |
| CN106611421A (en) | SAR image segmentation method based on feature learning and sketch line constraint | |
| CN111915617A (en) | An anti-overlap segmentation method for single leaf of plant point cloud | |
| WO2015120519A1 (en) | Processing hyperspectral or multispectral image data | |
| CN110084820A (en) | Purple soil image adaptive division and extracting method based on improved FCM algorithm | |
| CN109345536B (en) | Image super-pixel segmentation method and device | |
| CN104036294A (en) | Adaptive Classification Method for Multispectral Remote Sensing Images Based on Spectral Labeling | |
| CN116681946A (en) | Hyperspectral classification method based on band erasure and contrast learning | |
| CN112766340A (en) | Depth capsule network image classification method and system based on adaptive spatial mode | |
| Chen et al. | Segmentation of overlapping cells in cervical cytology images: A survey | |
| Freire Santana et al. | Multispectral image unsupervised segmentation using watershed transformation and cross-entropy minimization in different land use | |
| CN116993947B (en) | Visual display method and system for three-dimensional scene |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |