CN120451166B - A method and system for automobile parts defect detection based on AMEF - Google Patents
A method and system for automobile parts defect detection based on AMEFInfo
- Publication number
- CN120451166B CN120451166B CN202510954041.2A CN202510954041A CN120451166B CN 120451166 B CN120451166 B CN 120451166B CN 202510954041 A CN202510954041 A CN 202510954041A CN 120451166 B CN120451166 B CN 120451166B
- Authority
- CN
- China
- Prior art keywords
- image
- module
- frequency
- defect detection
- automobile part
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Image Analysis (AREA)
Abstract
本发明公开了一种基于AMEF的汽车零部件缺陷检测方法及系统，搭建汽车零部件缺陷检测模型，采用训练集对汽车零部件缺陷检测模型进行训练并计算损失，得到训练后的汽车零部件缺陷检测模型，获取真实场景下的汽车零部件图像和注意力图像并输入至训练后的汽车零部件缺陷检测模型，汽车零部件缺陷检测模型的AMEF模块接收输入图像并进行特征提取与融合处理，输出融合特征，主干特征提取模块接收融合特征并处理，输出深层特征，分类模块对深层特征进行分类与识别，输出汽车零部件缺陷检测结果。该方法适用于复杂背景、同类型缺陷尺度多样情况下的汽车零部件缺陷检测，识别效果佳。
The present invention discloses an AMEF-based automotive parts defect detection method and system. The method involves building an automotive parts defect detection model, training the model using a training set and calculating the loss to obtain a trained model. Real-world automotive parts images and attention images are then acquired and input into the trained model. The AMEF module of the automotive parts defect detection model receives the input image, extracts and fuses features, and outputs fused features. A backbone feature extraction module receives and processes the fused features, outputting deep features. A classification module classifies and identifies the deep features and outputs automotive parts defect detection results. This method is suitable for automotive parts defect detection in complex backgrounds and with varying scales of the same type of defects, achieving excellent recognition results.
Description
Technical Field
The invention relates to the technical field of defect detection of automobile parts, in particular to a AMEF-based defect detection method and system for automobile parts.
Background
The defect detection of the automobile parts aims at finding out the material defects and manufacturing process defects of the parts, realizes efficient, accurate and automatic defect detection of the automobile parts, and is important for improving the quality of the whole automobile and guaranteeing the safety performance. In the actual production process, defects in the defect detection of automobile parts usually appear in a small part area of a product to be detected, the defects of the parts are often found manually according to subjective experience, the efficiency is low, and the false detection and omission are caused by the influence of personal factors such as fatigue, eyesight and the like of related technicians. In addition, the defect mode is often unpredictable, so that it is difficult to directly describe the defects, and the existing generalized defects still cannot cover all types, so that the existing method cannot identify the new defect types.
In the prior art, the classification of defects of automobile parts is usually realized by constructing and training a generator and a discriminator network, however, the method is limited by the scarcity of defect samples, and the classification of defect types cannot cover all defect types, and in addition, the problems of poor recognition effect of products to be detected due to large diversity difference of the same type of defect scale and complex background exist, so that improvement is needed.
Disclosure of Invention
Aiming at the problems, the application of the invention provides a method and a system for detecting defects of automobile parts based on AMEF, which are used for solving the problems that in the related technology, the defects are limited by scarcity of defect samples, all defect types cannot be covered for classifying defect types, and the identification effect of a product to be detected is poor due to the large diversity difference of complex background and similar defect scales.
On the one hand, the invention provides a AMEF-based automobile part defect detection method, which comprises the following steps:
S1, building a defect detection model of an automobile part, wherein the model comprises a AMEF module, a trunk feature extraction module, an intra-class consistency constraint module and a classification module, wherein the AMEF module is connected with the trunk feature extraction module, and the intra-class consistency constraint module and the classification module are respectively connected with the trunk feature extraction module;
S2, acquiring a plurality of original images containing defects of automobile parts and attention images corresponding to the original images to form a training set;
S3, training the automobile part defect detection model by adopting a training set, designing a loss function of the automobile part defect detection model, calculating loss, and selecting a parameter updating model when a loss value is not updated any more to obtain a trained automobile part defect detection model;
S4, acquiring an image of an automobile part and a corresponding attention image in a real scene, inputting the image and the image into a trained automobile part defect detection model for processing, enabling a AMEF module to receive the input image, performing feature extraction and fusion, outputting fusion features, enabling a trunk feature extraction module to receive and process the fusion features, outputting deep features, enabling a classification module to receive the deep features, classifying known defect types and identifying unknown defect types, and outputting an automobile part defect detection result.
Preferably, the AMEF module in S1 includes a spatial domain feature extraction module, a wavelet convolution transformation module, a frequency expert module, and a fusion module, where the wavelet convolution transformation module, the frequency expert module, and the fusion module are sequentially connected, and the spatial domain feature extraction module is connected with the fusion module.
Preferably, the specific procedure of S3 is as follows:
S31, randomly selecting a preset number of original images and attention images corresponding to the selected original images from a training set, and inputting the attention images into a model for detecting the defects of the automobile parts;
S32, AMEF module receives the input original image and attention image, performs feature extraction and fusion processing, and outputs fusion features;
S33, a trunk feature extraction module is used for receiving and processing the fusion features and outputting deep features;
s34, receiving deep features by the intra-class consistency constraint module, performing high-dimensional space mapping to obtain high-dimensional mapping features, and performing optimization distribution on the high-dimensional mapping features through consistency constraint to obtain high-dimensional mapping features of different classes;
s35, a classification module receives deep features and classifies known defect categories and identifies unknown defect categories to obtain defect detection results;
S36, designing a loss function of an automobile part defect detection model according to different types of high-dimensional mapping characteristics and defect detection results, and calculating loss;
S37, additionally selecting a preset number of original images and attention images corresponding to the selected original images from the training set, repeating the steps S31 to S36, and selecting a parameter updating model when the loss value is not updated any more to obtain a trained automobile part defect detection model.
Preferably, the specific procedure of S32 is as follows:
S321, a spatial domain feature extraction module receives and processes an input original image to obtain spatial domain features;
S322, receiving and processing an input original image by a wavelet convolution transformation module to obtain a frequency domain image;
s323, the frequency expert module receives and processes the attention image and the frequency domain image to obtain a frequency domain fusion characteristic;
S324, the fusion module receives the spatial domain feature and the frequency domain fusion feature and splices the spatial domain feature and the frequency domain fusion feature based on the channel dimension to obtain the fusion feature.
Preferably, S322 specifically includes the following:
s3221, performing scale transformation and feature extraction on an input original image through Haar wavelet to obtain a two-dimensional low-pass filter coefficient and a two-dimensional high-pass filter coefficient, and obtaining a two-dimensional wavelet filter through combining the low-pass filter coefficient and the high-pass filter coefficient;
S3222, a two-dimensional wavelet filter is applied to an input original image, and a low-frequency component corresponding to the input original image is extracted through convolution operation;
s3223, performing recursive wavelet transformation on the low-frequency components corresponding to the original image by adopting a two-dimensional wavelet filter, and extracting the low-frequency components and the high-frequency components of different layers in the low-frequency components layer by layer;
S3224, unifying the low-frequency components and the high-frequency components of different layers extracted layer by layer into the same size by an up-sampling method to obtain the low-frequency components and the high-frequency components of different layers with the same size;
S3225, adjusting characteristic distribution through 1X 1 convolution operation, and splicing the low-frequency components and the high-frequency components with the same size and different layers based on channel dimensions to obtain a frequency domain image.
Preferably, the frequency domain image in S3225 is specifically expressed as:
;
Wherein, the Is the weight of 1X 1 convolution kernel, the weight size is,Is a bias term of the size of,In order to input the number of channels,Is the input image is in positionA frequency domain image on the upper surface of the frame,Is the up-sampled pixel value.
Preferably, the frequency expert module includes a gating network and a plurality of frequency expert networks respectively connected to the gating network, and S323 specifically includes the following:
S3231, the gating network receives and processes the attention image and generates sparse weights;
s3232, adding random noise into the sparse weight to obtain the sparse weight added with the random noise;
S3233, respectively carrying out frequency decomposition and feature extraction on the input frequency domain image by a plurality of frequency expert networks to obtain a feature map processed by each expert network;
s3234, carrying out weighted combination on the feature map processed by each expert network according to the sparse weight of the random noise to obtain the frequency domain fusion feature.
Preferably, the frequency domain fusion feature in S3234 is specifically expressed as:
;
Wherein, the ;
;
In the formula,For the frequency domain fusion feature,Is the sparse gating weight after adding random noise,The sparse weights are represented by a set of weights,For gating weights, M is the total number of frequency expert networks,For selecting from M frequency expert networksA frequency expert network (f-expert) is provided,Is selected fromFrequency Expert Network (FEN)The output of the frequency expert network,Is an attention image.
Preferably, the loss function of the defect detection model of the automobile part in S36 is specifically:
;
Wherein, the ;
;
In the formula,Indicating the total loss of the automobile part defect detection model,Representing the loss of classification of a known class of defects,Indicating a loss of consistency within the class,Is the loss of load and,Representing the loss of constraint within the class,Representing the loss of constraint between classes,Is the firstThe number of times the frequency expert network is selected,Is the total number of inputs, M is the total number of frequency expert networks,、、AndRespectively, weight coefficients.
The invention also provides an automobile part defect detection system, which adopts the automobile part defect detection method based on AMEF to detect automobile part defects, wherein the automobile part defect detection system comprises an image acquisition module, an image processing module and a computer system, the automobile part defect detection model is arranged in the computer system, the image acquisition module is respectively connected with the image processing module and the computer system, and the image processing module is connected with the computer system, wherein:
The image acquisition module is used for acquiring an image of the automobile part to be detected and transmitting the image of the automobile part to be detected to the image processing module and the computer system;
The image processing module is used for processing the input image of the automobile part to be detected to obtain a corresponding attention image;
The computer system is used for receiving the image of the automobile part to be detected and the corresponding attention image, inputting the image into the automobile part defect detection model for processing, and outputting the detection result of the image of the automobile part to be detected.
According to the automobile part defect detection method and system based on AMEF, an automobile part defect detection model is built, a plurality of original images containing automobile part defects and attention images corresponding to the original images are obtained and form a training set, the training set is used for training the automobile part defect detection model and calculating loss, the trained automobile part defect detection model is obtained, an automobile part image and a corresponding attention image in a real scene are obtained and are input into the trained automobile part defect detection model to be processed, a AMEF module receives the input automobile part image and the corresponding attention image to be subjected to feature extraction and fusion processing, fusion features are output, a trunk feature extraction module receives and processes fusion features, deep features are output, a classification module classifies deep features into known defect categories and identifies unknown defect categories, and an automobile part defect detection result is output. The method is suitable for detecting the defects of the automobile parts under the conditions of complex background and large diversity difference of the same type of defect scales, and has good recognition effect.
Drawings
FIG. 1 is a flowchart of a method for detecting defects of automotive parts based on AMEF according to one embodiment of the present invention;
Fig. 2 is a schematic structural diagram of a defect detection model for an automobile part according to an embodiment of the invention.
Detailed Description
In order to make the technical scheme of the present invention better understood by those skilled in the art, the present invention will be further described in detail with reference to the accompanying drawings.
Referring to fig. 1, a method for detecting defects of automobile parts based on AMEF includes the following steps:
s1, building an automobile part defect detection model, wherein the model comprises a AMEF module, a trunk feature extraction module, an intra-class consistency constraint module and a classification module, the AMEF module is connected with the trunk feature extraction module, and the intra-class consistency constraint module and the classification module are respectively connected with the trunk feature extraction module.
Referring to fig. 2, the automobile part defect detection model includes a AMEF module (Attention-directed multi-frequency expert fusion network), a trunk feature extraction module, an intra-class consistency constraint module, and a classification module, wherein the AMEF module is connected with the trunk feature extraction module, and the intra-class consistency constraint module and the classification module are respectively connected with the trunk feature extraction module. The AMEF module is used for receiving an input original image and an attention image, carrying out feature extraction and fusion processing, outputting fusion features, the trunk feature extraction module is used for receiving and processing the fusion features, outputting deep features, the intra-class consistency constraint module is used for receiving the deep features and carrying out high-dimensional mapping to obtain high-dimensional mapping features, the classification module is used for receiving and processing the deep features, and outputting an automobile part defect detection result.
Further, AMEF modules comprise a spatial domain feature extraction module, a wavelet convolution transformation module, a frequency expert module and a fusion module, wherein the wavelet convolution transformation module, the frequency expert module and the fusion module are sequentially connected, and the spatial domain feature extraction module is connected with the fusion module.
S2, acquiring a plurality of original images containing defects of the automobile parts and attention images corresponding to the original images to form a training set.
Specifically, a plurality of original images containing automobile parts are obtained, the original images are divided into normal samples and defect samples of different categories, attention images corresponding to the normal samples or the defect samples of different categories are obtained through an existing abnormality detection method, and the normal samples, the defect samples of different categories and attention force diagrams corresponding to the normal samples and the defect samples of different categories are combined into a data set.
S3, training the automobile part defect detection model by adopting a training set, designing a loss function of the automobile part defect detection model, calculating loss, and selecting a parameter updating model when a loss value is not updated to obtain the trained automobile part defect detection model.
Further, in S3, the training set is adopted to train the automobile part defect detection model, and the specific process is as follows:
s31, randomly selecting a preset number of original images and attention image images corresponding to the selected original images from the training set, and inputting the attention image images into the automobile part defect detection model.
S32, AMEF module receives the input original image and attention image, performs feature extraction and fusion processing, and outputs fusion features.
Further, the specific process of S32 is as follows:
s321, receiving an input original image by a spatial domain feature extraction module And processing to obtain spatial domain features。
S322, the wavelet convolution transformation module receives the input original imageAnd processing to obtain frequency domain image。
Further, S322 specifically includes the following:
s3221, inputting original image by Haar wavelet pair Performing scale transformation and feature extraction to obtain two-dimensional low-pass filter coefficientsAnd high pass filter coefficientsBy combining low pass filter coefficientsAnd high pass filter coefficientsA two-dimensional wavelet filter is obtained.
The two-dimensional wavelet filter can be expressed specifically as:
(1)-1
(1)-2
(1)-3
(1)-4
Wherein, the Is a low-pass-low-pass filter,As a low-pass-high-pass filter,In the form of a high-pass-low-pass filter,In the form of a high-pass-high-pass filter,AndRepresenting the filter convolution kernel pixel location,For the corresponding row coordinate, the position of the row coordinate,Corresponding column coordinates.
S3222, applying a two-dimensional wavelet filter to an input original imageExtracting an input original image by a convolution operationCorresponding low frequency components and high frequency components.
Specifically, in the first pass wavelet transform, a two-dimensional wavelet filter is applied to an input imageThe low frequency component and the high frequency component are extracted by a convolution operation:
(2)-1
(2)-2
(2)-3
(2)-4
Wherein, the The input image is represented by a representation of the input image,For the low frequency components of the input image,、、For the high frequency components of the input image,For the pixel positions of the input image,Pixel locations that convolve the kernel for the filter.
S3223, the original imageThe corresponding low-frequency components are subjected to recursive wavelet transformation by adopting a two-dimensional wavelet filter, and the low-frequency components and the high-frequency components of different layers in the low-frequency components are extracted layer by layer;
Specifically, in order to capture multi-scale features, recursive wavelet transformation is repeatedly performed on low-frequency components of an input image, and low-frequency components and high-frequency components of different layers are extracted layer by layer, wherein the formula is as follows:
(3)-1
(3)-2
(3)-3
(3)-4
Wherein, the Representing the first obtained after low pass-low pass filteringThe low frequency component of the layer is used,Representing the first obtained after low-pass-high-pass filteringThe high-frequency component of the layer,Representing the first obtained after high-pass-low-pass filteringThe high-frequency component of the layer,Representing the first obtained after high pass-high pass filteringThe high-frequency component of the layer,Represent the firstA low-pass filter of the layer,Represent the firstA low-pass-high-pass filter of the layer,Represent the firstThe high-pass-low-pass filter of the layer,Represent the firstA high-pass-high-pass filter of the layer,Represent the firstLow frequency components of the layer.
S3224, unifying the low-frequency components and the high-frequency components of different layers extracted layer by layer into the same size by an up-sampling method to obtain the low-frequency components and the high-frequency components of different layers with the same size.
Specifically, for the frequency domain features with different sizes, the frequency domain features are unified into the same size by an up-sampling method, and the formula is as follows:
(4)
(4)-1
Wherein, the For up-sampling the pre-pixel values, i.e. the low frequency and high frequency components of different levels,For up-sampled pixel values, i.e. low frequency and high frequency components of uniform size,AndIs a function of the weight of the interpolation,Representing the height of the feature before upsampling,Representing the width of the feature before upsampling.
S3225, adjusting characteristic distribution through 1X 1 convolution operation, and splicing the low-frequency components and the high-frequency components with the same size and different layers based on channel dimensions to obtain a frequency domain image。
Further, the frequency domain image in S3225The concrete steps are as follows:
(5)
Wherein, the Is the weight of 1X 1 convolution kernel, the weight size is,Is a bias term of the size of,In order to input the number of channels,Is the input image is in positionA frequency domain image on the upper surface of the frame,I.e. the up-sampled pixel values.
S323, frequency expert module for receiving attention imageAnd frequency domain imageAnd processing to obtain frequency domain fusion characteristics。
Further, the frequency expert module includes a gating network and a plurality of frequency expert networks respectively connected to the gating network, and S323 specifically includes the following:
s3231, the gating network receives the attention image And processing to generate sparse weights.
(6)
Wherein, the Representing sparse weights for use in transforming frequency domain imagesAssigned to different frequency expert networks,,In order to be able to take an image of attention,,In order to gate the weights on the basis of the weight,For selecting from M frequency expert networksA frequency expert network (f-expert) is provided,In order to be of a batch size,Is the flattened image dimension.
S3232, adding random noise into the sparse weight to obtain the sparse weight added with the random noise.
Specifically, to enhance the exploratory capacity of the gating network, random noise is added to the sparse weights:
(7)
Wherein, the Representing the sparse weights after the addition of random noise,Representing random noise, obeying the mean value of 0 and the variance of 0Is used for the normal distribution of the (c),The size of the model is controlled through specific parameters, so that the influence of excessive noise on the stability of the model is avoided.
S3233, multiple frequency expert networks input frequency domain imagesAnd respectively carrying out frequency decomposition and feature extraction to obtain a feature map processed by each expert network.
For example, for the first of the K selected frequency expert networksPersonal frequency expert networkThe output is:
(8)
Wherein, the Represent the firstPersonal frequency expert networkFor the input frequency domain imageThe characteristic diagram obtained after the processing is used for processing the image,,In order to be of a batch size,The number of channels output by the wavelet convolution transformation module,AndIs the size of the frequency domain image.
S3234, weighting and combining the feature images processed by each frequency expert network according to the sparse weight of random noise to obtain a frequency domain fusion feature。
Further, the frequency domain fusion features in S3234The concrete steps are as follows:
(9)
In the formula, For the frequency domain fusion feature,Is sparse gating weight added with random noise, K is passingSelected from M frequency expert networksThe number of frequency expert networks for which the weights are most important,Is selected fromFrequency Expert Network (FEN)And an output of the frequency expert network.
S324, the fusion module receives the spatial domain featuresFeature fusion with frequency domainAnd splicing the two based on channel dimension to obtain fusion characteristics。
Further, fusion featuresCan be expressed as:
(10)
Aiming at the defect detection problem of automobile parts under a complex background, the invention provides a AMEF module which firstly uses wavelet transformation to acquire characteristic information of different scales and frequencies so as to reduce the influence of background noise, and then focuses on a foreground target area through an attention-guided frequency expert module so as to further improve the adaptability to the complex background.
S33, a trunk feature extraction module is used for receiving and processing the fusion features and outputting deep features.
Further, the backbone feature extraction module may be a ResNet-18 network, which fuses featuresInputting to ResNet-18 network for feature extraction to extract high-order semantic features, and finally outputting deep features。
S34, the intra-class consistency constraint module receives the deep features and performs high-dimensional space mapping to obtain high-dimensional mapping features, and the high-dimensional mapping features are optimally distributed through consistency constraint to obtain high-dimensional mapping features of different classes.
Specifically, the intra-class consistency constraint module maps deep features through random fourier featuresMapping to a high-dimensional space, wherein the mapping function is as follows:
, (11)
Wherein, the Representing the mapping result of the input deep features in the high-dimensional space, i.e. the high-dimensional mapping features,Is from normal distributionA random matrix of mid-samples,Is from uniform distributionThe offset vector of the mid-sample,Is the dimension of the high-dimensional space. The correlation of the captured features can be aided by the high-dimensional features after random fourier mapping.
By adopting the method, a preset number of original images are selected from the training set each time, corresponding deep features of each original image in the selected original images are obtained, and the intra-class consistency constraint module is used for carrying out high-dimensional space mapping on the deep features to obtain high-dimensional mapping features.
By the high-dimensional mapping method, the input deep features can be efficiently mapped to a high-dimensional space, so that nonlinear association between original features is measured in a method similar to a kernel. The method not only can enrich the expression capability of the features, but also can better capture fine granularity information, and provides powerful support for subsequent feature optimization.
The distribution is then further optimized for the extracted high-dimensional map features by consistency constraints. First, to reduce the degree of dispersion of the same-class defect samples in the high-dimensional feature space, the class consistency constraint module introduces a loss of intra-class consistency constraint. This loss makes the intra-class features tighter by minimizing the variability in the feature distribution between the same class samples. It is defined as:
(12)
Wherein, the (12)-1
In the formula,Representing the loss of constraint within the class,Is a sampleAndHigh-dimensional mapping features of (a)AndA covariance matrix between the two data sets,Representing the mean of the high-dimensional mapping features corresponding to all samples in each batch,The trace operation of the matrix is represented,The Frobenius norm square representing the matrix,Representation and sampleA collection of samples of the same class,Indicating the batch size.
By minimizing the bias of the covariance matrix, the loss constraint is more similar to the eigenvectors of the class samples, thereby reducing the feature distribution variability inside the class.
Second, to enhance the ability to distinguish between different classes of defects, the class consistency constraint module introduces an inter-class separation constraint penalty. The inter-class separation loss ensures more dispersion of the characteristics between classes by maximizing the characteristic distribution difference between the defect samples of different classes. The definition is as follows:
(13)
Wherein, the (13)-1
In the formula,Representing the loss of constraint between classes,Is a sampleAnd sampleHigh-dimensional feature vectors of (a)AndA covariance matrix between the two data sets,Representation and sampleSets of samples of different classes.
The inter-class separation constraint enhances feature distinguishability between classes by maximizing the Frobenius norm of the covariance matrix between the features of different classes of samples while reducing its trace value.
By means of intra-class constraint loss measurement and inter-class constraint loss measurement, compactness of similar features and separability of different similar features can be improved, and therefore classification accuracy is improved.
S35, the classification module receives the deep features, classifies the known defect types and identifies the unknown defect types, and obtains a defect detection result.
S36, designing a loss function of the automobile part defect detection model according to the high-dimensional mapping characteristics and the defect detection results of different categories, and calculating loss.
Further, the loss function of the defect detection model of the automobile part in S36 is specifically:
(14)
Wherein, the (14)-1
(14)-2
In the formula,Indicating the total loss of the automobile part defect detection model,Representing the loss of classification of a known class of defects,Indicating a loss of consistency within the class,Is the loss of load and,Representing the loss of constraint within the class,Representing the loss of constraint between classes,Is the firstThe number of times the frequency expert network is selected,Is the total number of inputs, M is the total number of frequency expert networks,、、AndRespectively, weight coefficients.
Specifically, the total loss design process of the automobile part defect detection model is as follows:
1) In order to make the selection of all frequency expert networks more uniform, the load loss is defined as follows:
;
Wherein, the Is the loss of load and,Is the firstThe number of times the frequency expert network is selected,Is the total input number, n=b number of batches,Is the total number of frequency expert networks,To lose weight.
2) The intra-class consistency loss function is designed according to the intra-class constraint loss and the inter-class constraint loss as follows:
;
Wherein, the Indicating a loss of consistency within the class,Indicating a loss in the class of the object,Indicating the loss between classes of the object,AndThe weight coefficients are used to balance intra-class and inter-class losses, respectively.
3) According to load lossWithin class consistency lossClassification loss for known class defectsTotal loss of design automobile parts defect detection model:
;
S37, additionally selecting a preset number of original images and attention images corresponding to the selected original images from the training set, repeating the steps S31 to S36, and selecting a parameter updating model when the loss value is not updated any more to obtain a trained automobile part defect detection model.
Specifically, after each training, calculating the total loss of the automobile part defect detection model through the loss function, and reversely propagating and optimizing parameters of the automobile part defect detection model until the loss value is not updated any more, so as to obtain the trained automobile part defect detection model.
S4, acquiring an image of an automobile part and a corresponding attention image in a real scene, inputting the image and the image into a trained automobile part defect detection model for processing, enabling a AMEF module to receive the input image, performing feature extraction and fusion, outputting fusion features, enabling a trunk feature extraction module to receive and process the fusion features, outputting deep features, enabling a classification module to receive the deep features, classifying known defect types and identifying unknown defect types, and outputting an automobile part defect detection result.
Specifically, in the application stage, the intra-class consistency constraint module is not involved in defect recognition any more, a plurality of known defect types are preset, the AMEF module receives an input automobile part image and a corresponding attention image, performs feature extraction and fusion processing, outputs fusion features, the main feature extraction module (specifically, resNet-18 network) performs deep feature extraction on the fusion features, outputs deep features, inputs the deep features into the classification module for processing, outputs an automobile part defect detection result, wherein the defect detection result can be divided into unknown type defects and known type defects, and when the known type defects are identified, the specific known defect types are also output.
The defect type probability vector output by the classification module is assumed to be:
;
Wherein, the Is the weight of the classification module and,Representing that it is of a presetThe predictive probability vector for each known defect class,Representing deep features of the fusion features of the automobile part image and the corresponding attention image in the real scene after the extraction of the main network,Representing the Softmax function.
In order to identify unknown defect categories, a rejection mechanism is designed, assuming that the highest value of the probability vector of the defect category is:
;
defining a threshold value for distinguishing between known defects and unknown defects If (if)Judging the defect type corresponding to the input image as unknown defect and labeling asWherein the unknown defects include unusual defects or defects which are difficult to collect, ifThe defect type judgment corresponding to the input image is judged to be the known type defect, and when the defect type is the known type defect, the classification module determines the known defect type label according to the maximum index:
;
Wherein, the A known defect class label representing the classification module prediction,Representing solving predictive probability vectorsIndexing of the maximum value.
In one embodiment, an automobile part defect detection system detects an automobile part defect by adopting the automobile part defect detection method based on AMEF, the automobile part defect detection system comprises an image acquisition module, an image processing module and a computer system, the automobile part defect detection model is arranged in the computer system, the image acquisition module is respectively connected with the image processing module and the computer system, and the image processing module is connected with the computer system, wherein:
The image acquisition module is used for acquiring an image of the automobile part to be detected and transmitting the image of the automobile part to be detected to the image processing module and the computer system;
The image processing module is used for processing the input image of the automobile part to be detected to obtain a corresponding attention image;
The computer system is used for receiving the image of the automobile part to be detected and the corresponding attention image, inputting the image into the automobile part defect detection model for processing, and outputting the detection result of the image of the automobile part to be detected.
For a specific limitation of the system of the method for detecting defects of an automobile part, reference may be made to the limitation of the method for detecting defects of an automobile part based on AMEF hereinabove, and the description thereof will not be repeated here.
Further, the effect verification is carried out on the automobile part defect detection method based on AMEF through a comparison test.
In a comparison experiment, firstly, an automobile part defect type data set is self-constructed, an experiment is carried out based on the existing open set identification method, and experimental results are shown in a table 1, and performances of different methods on Accuracy (ACC), F1 fraction and AUROC index are shown.
Table 1 performance data corresponding to different methods
As can be seen from experimental comparison, the automobile part defect detection method based on AMEF (corresponding to the method in the application in table 1) can obviously improve the performance of identifying the appearance defects of the automobile part, wherein the accuracy rate ACC reaches 98.35%, the F1 fraction reaches 81.36%, and the AUROC reaches 94.93%. The result shows that the method provided by the application can ensure high accuracy and balance the prediction performance among categories when processing the identification task of the appearance defects of the automobile parts, thereby obviously improving the F1 fraction and AUROC and having excellent comprehensive performance.
According to the automobile part defect detection method and system based on AMEF, an automobile part defect detection model is built, the model comprises a AMEF module, a trunk feature extraction module, an intra-class consistency constraint module and a classification module, a plurality of original images containing automobile part defects and attention images corresponding to the original images are obtained to form a training set, the training set is adopted to train the automobile part defect detection model, loss is calculated, the trained automobile part defect detection model is obtained, an automobile part image and a corresponding attention image in a real scene are obtained and are input into the trained automobile part defect detection model to be processed, the AMEF module receives the input automobile part image and a corresponding attention image to be processed in a feature extraction and fusion mode, fusion features are output, the trunk feature extraction module receives and processes fusion features, deep layer features are output, and the classification module classifies high-dimensional defect features into known defect types and identifies unknown defect types to output automobile part defect detection results. The method is suitable for detecting the defects of the automobile parts under the conditions of complex background and large diversity difference of the same type of defect scales, and has good recognition effect.
The automobile part defect detection method and system based on AMEF provided by the invention are described in detail. The principles and embodiments of the present invention have been described herein with reference to specific examples, the description of which is intended only to facilitate an understanding of the core concepts of the invention. It should be noted that it will be apparent to those skilled in the art that various modifications and adaptations of the invention can be made without departing from the principles of the invention and these modifications and adaptations are intended to be within the scope of the invention as defined in the following claims.
Claims (9)
1. A method for detecting defects of automobile parts based on AMEF, the method comprising:
S1, building a defect detection model of an automobile part, wherein the model comprises a AMEF module, a trunk feature extraction module, an intra-class consistency constraint module and a classification module, wherein the AMEF module is connected with the trunk feature extraction module, and the intra-class consistency constraint module and the classification module are respectively connected with the trunk feature extraction module;
S2, acquiring a plurality of original images containing defects of automobile parts and attention images corresponding to the original images to form a training set;
S3, training the automobile part defect detection model by adopting a training set, designing a loss function of the automobile part defect detection model, calculating loss, and selecting a parameter updating model when a loss value is not updated any more to obtain a trained automobile part defect detection model;
S4, acquiring an image of an automobile part and a corresponding attention image in a real scene, inputting the image and the image into a trained automobile part defect detection model for processing, enabling a AMEF module to receive the input image, performing feature extraction and fusion, outputting fusion features, enabling a trunk feature extraction module to receive and process the fusion features, outputting deep features, enabling a classification module to receive the deep features, classifying known defect types and identifying unknown defect types, and outputting an automobile part defect detection result;
The AMEF module in the S1 comprises a spatial domain feature extraction module, a wavelet convolution transformation module, a frequency expert module and a fusion module, wherein the wavelet convolution transformation module, the frequency expert module and the fusion module are sequentially connected, and the spatial domain feature extraction module is connected with the fusion module.
2. The AMEF-based automobile part defect detection method as claimed in claim 1, wherein the specific process of S3 is as follows:
S31, randomly selecting a preset number of original images and attention images corresponding to the selected original images from a training set, and inputting the attention images into a model for detecting the defects of the automobile parts;
S32, AMEF module receives the input original image and attention image, performs feature extraction and fusion processing, and outputs fusion features;
S33, a trunk feature extraction module is used for receiving and processing the fusion features and outputting deep features;
s34, receiving deep features by the intra-class consistency constraint module, performing high-dimensional space mapping to obtain high-dimensional mapping features, and performing optimization distribution on the high-dimensional mapping features through consistency constraint to obtain high-dimensional mapping features of different classes;
s35, a classification module receives deep features and classifies known defect categories and identifies unknown defect categories to obtain defect detection results;
S36, designing a loss function of an automobile part defect detection model according to different types of high-dimensional mapping characteristics and defect detection results, and calculating loss;
S37, additionally selecting a preset number of original images and attention images corresponding to the selected original images from the training set, repeating the steps S31 to S36, and selecting a parameter updating model when the loss value is not updated any more to obtain a trained automobile part defect detection model.
3. The method for detecting defects of automobile parts based on AMEF as claimed in claim 2, wherein the specific process of S32 is as follows:
S321, a spatial domain feature extraction module receives and processes an input original image to obtain spatial domain features;
S322, receiving and processing an input original image by a wavelet convolution transformation module to obtain a frequency domain image;
s323, the frequency expert module receives and processes the attention image and the frequency domain image to obtain a frequency domain fusion characteristic;
S324, the fusion module receives the spatial domain feature and the frequency domain fusion feature and splices the spatial domain feature and the frequency domain fusion feature based on the channel dimension to obtain the fusion feature.
4. The method for detecting defects of automotive parts based on AMEF as set forth in claim 3, wherein S322 specifically includes the following steps:
s3221, performing scale transformation and feature extraction on an input original image through Haar wavelet to obtain a two-dimensional low-pass filter coefficient and a two-dimensional high-pass filter coefficient, and obtaining a two-dimensional wavelet filter through combining the low-pass filter coefficient and the high-pass filter coefficient;
S3222, a two-dimensional wavelet filter is applied to an input original image, and a low-frequency component corresponding to the input original image is extracted through convolution operation;
s3223, performing recursive wavelet transformation on the low-frequency components corresponding to the original image by adopting a two-dimensional wavelet filter, and extracting the low-frequency components and the high-frequency components of different layers in the low-frequency components layer by layer;
S3224, unifying the low-frequency components and the high-frequency components of different layers extracted layer by layer into the same size by an up-sampling method to obtain the low-frequency components and the high-frequency components of different layers with the same size;
S3225, adjusting characteristic distribution through 1X 1 convolution operation, and splicing the low-frequency components and the high-frequency components with the same size and different layers based on channel dimensions to obtain a frequency domain image.
5. The method for detecting defects of automotive parts based on AMEF as set forth in claim 4, wherein the frequency domain image in S3225 is specifically expressed as:
;
Wherein, the Is the weight of 1X 1 convolution kernel, the weight size is,Is a bias term of the size of,In order to input the number of channels,Is the input image is in positionA frequency domain image on the upper surface of the frame,Is the up-sampled pixel value.
6. The method for detecting defects of automotive parts based on AMEF, wherein the frequency expert module comprises a gating network and a plurality of frequency expert networks respectively connected with the gating network, and S323 specifically comprises the following steps:
S3231, the gating network receives and processes the attention image and generates sparse weights;
s3232, adding random noise into the sparse weight to obtain the sparse weight added with the random noise;
S3233, respectively carrying out frequency decomposition and feature extraction on the input frequency domain image by a plurality of frequency expert networks to obtain a feature map processed by each expert network;
s3234, carrying out weighted combination on the feature map processed by each expert network according to the sparse weight of the random noise to obtain the frequency domain fusion feature.
7. The method for detecting defects of automotive parts based on AMEF as set forth in claim 6, wherein the frequency domain fusion characteristics in S3234 are specifically expressed as:
;
Wherein, the ;
;
In the formula,For the frequency domain fusion feature,Is the sparse gating weight after adding random noise,The sparse weights are represented by a set of weights,For gating weights, M is the total number of frequency expert networks,For selecting from M frequency expert networksA frequency expert network (f-expert) is provided,Is selected fromFrequency Expert Network (FEN)The output of the frequency expert network,Is an attention image.
8. The method for detecting defects of automobile parts based on AMEF as claimed in claim 2, wherein the loss function of the defect detection model of automobile parts in S36 is specifically:
;
Wherein, the ;
;
In the formula,Indicating the total loss of the automobile part defect detection model,Representing the loss of classification of a known class of defects,Indicating a loss of consistency within the class,Is the loss of load and,Representing the loss of constraint within the class,Representing the loss of constraint between classes,Is the firstThe number of times the frequency expert network is selected,Is the total number of inputs, M is the total number of frequency expert networks,、、AndRespectively, weight coefficients.
9. An automobile part defect detection system for detecting automobile part defects by adopting the automobile part defect detection method based on AMEF as claimed in any one of claims 1 to 8, which is characterized in that the automobile part defect detection system comprises an image acquisition module, an image processing module and a computer system, wherein the automobile part defect detection model is arranged in the computer system, the image acquisition module is respectively connected with the image processing module and the computer system, and the image processing module is connected with the computer system, wherein:
The image acquisition module is used for acquiring an image of the automobile part to be detected and transmitting the image of the automobile part to be detected to the image processing module and the computer system;
The image processing module is used for processing the input image of the automobile part to be detected to obtain a corresponding attention image;
The computer system is used for receiving the image of the automobile part to be detected and the corresponding attention image, inputting the image into the automobile part defect detection model for processing, and outputting the detection result of the image of the automobile part to be detected.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510954041.2A CN120451166B (en) | 2025-07-11 | 2025-07-11 | A method and system for automobile parts defect detection based on AMEF |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202510954041.2A CN120451166B (en) | 2025-07-11 | 2025-07-11 | A method and system for automobile parts defect detection based on AMEF |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN120451166A CN120451166A (en) | 2025-08-08 |
| CN120451166B true CN120451166B (en) | 2025-09-19 |
Family
ID=96624517
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202510954041.2A Active CN120451166B (en) | 2025-07-11 | 2025-07-11 | A method and system for automobile parts defect detection based on AMEF |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN120451166B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116129100A (en) * | 2022-12-08 | 2023-05-16 | 国能铁路装备有限责任公司 | Truck part positioning and fault identifying method, device, equipment and medium |
| CN116703928A (en) * | 2023-08-08 | 2023-09-05 | 宁德市天铭新能源汽车配件有限公司 | Automobile part production detection method and system based on machine learning |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10176382B1 (en) * | 2015-10-01 | 2019-01-08 | Hrl Laboratories, Llc | Method and apparatus for sparse associative recognition and recall for visual media reasoning |
| CN114663436A (en) * | 2022-05-25 | 2022-06-24 | 南京航空航天大学 | Cross-scale defect detection method based on deep learning |
| CN115661042A (en) * | 2022-09-29 | 2023-01-31 | 河北工业大学 | Hierarchical classification defect detection method based on attention mechanism guidance |
| US20240177465A1 (en) * | 2022-11-30 | 2024-05-30 | Guoqiang Xue | Geological target identification method and apparatus based on image information fusion |
| CN118864454B (en) * | 2024-09-25 | 2025-01-21 | 山东省计算中心（国家超级计算济南中心） | An unsupervised anomaly detection method and system based on memory expert guidance |
-
2025
- 2025-07-11 CN CN202510954041.2A patent/CN120451166B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116129100A (en) * | 2022-12-08 | 2023-05-16 | 国能铁路装备有限责任公司 | Truck part positioning and fault identifying method, device, equipment and medium |
| CN116703928A (en) * | 2023-08-08 | 2023-09-05 | 宁德市天铭新能源汽车配件有限公司 | Automobile part production detection method and system based on machine learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN120451166A (en) | 2025-08-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110991236B (en) | Image classification method and related device | |
| CN113762138B (en) | Identification method, device, computer equipment and storage medium for fake face pictures | |
| CN108765412B (en) | A method for classifying surface defects of strip steel | |
| CN109671070B (en) | Target detection method based on feature weighting and feature correlation fusion | |
| CN107992891B (en) | Multispectral remote sensing image change detection method based on spectral vector analysis | |
| CN113673556B (en) | A hyperspectral image classification method based on multi-scale dense convolutional network | |
| CN115829909A (en) | Forgery detection method based on feature enhancement and spectrum analysis | |
| CN118674716B (en) | Stem cell quality evaluation method and system | |
| Xing et al. | Investigation of a novel automatic micro image-based method for the recognition of animal fibers based on wavelet and Markov random field | |
| CN115131503A (en) | Health monitoring method and system for iris three-dimensional recognition | |
| CN119478554B (en) | A method and device for identifying defects in power insulators based on deep decision fusion | |
| CN112926602A (en) | Rock lithology analysis method, device and equipment based on visualization algorithm | |
| CN115798633A (en) | Raman spectrum data processing method based on lightweight convolutional neural network | |
| CN115410017A (en) | Seed mildew detection method, device, equipment and storage medium | |
| CN118277674B (en) | Personalized image content recommendation method based on big data analysis | |
| CN120451166B (en) | A method and system for automobile parts defect detection based on AMEF | |
| CN112949422A (en) | Hyperspectral target detection method based on self-supervision spectrum matching framework | |
| CN114897051B (en) | Citrus disease degree identification method, device, equipment and storage medium | |
| Jain et al. | Natural scene statistics and CNN based parallel network for image quality assessment | |
| CN116542848A (en) | Terahertz image super-resolution reconstruction method, system and application | |
| CN108898052A (en) | The detection method and equipment of man-made features in remote sensing images | |
| CN116363643A (en) | Method for identifying microscopic components of kerogen | |
| CN114187477A (en) | Small sample hyperspectral image classification method based on supervised self-contrast learning | |
| CN119888720B (en) | Multi-view quantitative evaluation method and device for tea leaves and storage medium | |
| CN119904452B (en) | Surface defect detection method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |