CN118506093A - Defect type identification method, model training method, device and storage medium - Google Patents
Defect type identification method, model training method, device and storage medium Download PDFInfo
- Publication number
- CN118506093A CN118506093A CN202410675448.7A CN202410675448A CN118506093A CN 118506093 A CN118506093 A CN 118506093A CN 202410675448 A CN202410675448 A CN 202410675448A CN 118506093 A CN118506093 A CN 118506093A
- Authority
- CN
- China
- Prior art keywords
- information
- image
- training
- detected
- shooting angle
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30141—Printed circuit board [PCB]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
The application relates to a defect type identification method, a model training method, a device and a storage medium, wherein the method comprises the following steps: the method comprises the steps of obtaining a device detection image of a device to be detected and device detection information of the device to be detected, wherein the device detection image is an image obtained based on a shooting result of the device to be detected, performing shooting angle identification according to the device detection information and the device detection image in a shooting angle classification layer of a defect type identification model to obtain shooting angle information, identifying a target defect type corresponding to the device to be detected from defect types corresponding to the shooting angle information according to the shooting angle information and the device detection image in a defect identification layer of the defect type identification model, namely identifying the shooting angle of the device to be detected first, and identifying the target defect type of the device to be detected from the defect types corresponding to the shooting angle, so that the identification accuracy of a defect product in the defect type identification process of the device to be detected is improved.
Description
Technical Field
The present application relates to the field of image recognition technologies, and in particular, to a defect type recognition method, a model training device, and a storage medium.
Background
The surface mount technology (Surface Mounted Technology, SMT) production line needs to check the appearance of the electronic component before and after reflow soldering to confirm the process. At present, an automatic optical detector (Auto Optical Inspection, AOI for short) is commonly used for detection in an SMT production line, relevant parameters such as relative position coordinates, length, width, height, offset angles, colors and the like of components to be checked are set in the AOI equipment, and then shooting comparison is carried out on the components to be checked.
From the current use situation, the AOI device can intercept most of poor patches and poor welding, but still some defective products are judged to be normal and flow to the post-processing procedure, after the post-processing procedure finds new defective products, corresponding adjustment is needed to program parameters so that the device can intercept the new defective products, but after the judgment parameters are adjusted, new problems such as misjudgment and the like may be caused, so how to accurately identify defective products in the welding stage becomes a problem to be solved urgently.
Disclosure of Invention
The application provides a defect type identification method, a model training method, a device and a storage medium, which are used for solving the technical problem of how to accurately identify a defect product.
In a first aspect, the present application provides a defect type identifying method, the method comprising:
Acquiring a device detection image of a device to be detected and device detection information of the device to be detected, wherein the device detection image is an image obtained based on a result of shooting the device to be detected, and the device detection information is material information and a circuit structure file of the device to be detected;
Inputting the device detection information and the device detection image into a shooting angle classification layer of a defect type identification model to carry out shooting angle identification, so as to obtain shooting angle information corresponding to the device to be detected;
Inputting the shooting angle information and the device detection image into a defect identification layer of the defect type identification model, and identifying the target defect type corresponding to the device to be detected from the defect types corresponding to the shooting angle information.
Optionally, the shooting angle classification layer includes a device modeling layer and a shooting angle identification layer, the step of inputting the device detection information and the device detection image into the shooting angle classification layer of the defect type identification model to perform shooting angle identification, and the step of obtaining shooting angle information corresponding to the device to be detected includes:
inputting the device detection information into the device modeling layer for three-dimensional modeling to obtain a model image corresponding to the device to be detected;
and inputting the model image and the device detection image into the shooting angle identification layer to identify the shooting angle, so as to obtain the shooting angle information.
Optionally, the step of inputting the device detection information into the device modeling layer to perform stereo modeling, and obtaining a model image corresponding to the device to be detected includes:
acquiring device packaging information corresponding to the device to be detected through the material information in the device detection information;
And carrying out three-dimensional image simulation on the device to be detected based on preset device standard information, a circuit structure file in the device detection information and the device packaging information, and generating the model image.
Optionally, the step of inputting the model image and the device detection image into the shooting angle identification layer to perform shooting angle identification, and obtaining the shooting angle information includes:
performing angle adjustment on the model image at least once, and matching the model image subjected to each angle adjustment with the device detection image to obtain an angle matching result;
and outputting current angle information of the model image as the shooting angle information under the condition that the angle matching result indicates that the model image after angle adjustment is matched with the device detection image.
Optionally, the step of acquiring the device detection image of the device to be detected and the device detection information of the device to be detected further includes:
acquiring a shooting result of the device to be detected and device detection information of the device to be detected;
and carrying out image enhancement on the shooting result of the device to be detected to obtain a device detection image.
In a second aspect, the present application provides a method for training a defect type identification model, the method comprising:
Acquiring a device training image of a preset device, device training information of the preset device and defect marking information corresponding to the device training image, wherein the device training image is an image obtained based on a result of shooting the preset device, and the device training information is material information and a circuit structure file of the preset device;
inputting the device training information and the device training image into a shooting angle classification layer of a model to be trained to perform shooting angle identification, so as to obtain shooting angle training information corresponding to the preset device;
Inputting the shooting angle training information and the device training image into a defect identification layer of the model to be trained, and identifying defect type training information corresponding to the preset device from defect types corresponding to the shooting angle training information;
comparing the defect type training information with the defect labeling information to obtain target loss information;
Training the model to be trained based on the target loss information to obtain the defect type identification model according to any one of the first aspect.
In a third aspect, the present application provides a defect type identifying apparatus, the apparatus comprising:
The device detection device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a device detection image of a device to be detected and device detection information of the device to be detected, the device detection image is an image obtained based on a result of shooting the device to be detected, and the device detection information is material information and a circuit structure file of the device to be detected;
The angle identification module is used for inputting the device detection information and the device detection image into a shooting angle classification layer of a defect type identification model to carry out shooting angle identification, so as to obtain shooting angle information corresponding to the device to be detected;
The type identification module is used for inputting the shooting angle information and the device detection image into a defect identification layer of the defect type identification model, and identifying the target defect type corresponding to the device to be detected from the defect types corresponding to the shooting angle information.
In a fourth aspect, the present application provides a defect type identification model training device, the device comprising:
The device training image is an image obtained based on a result of shooting the preset device, and the device training information is material information and a circuit structure file of the preset device;
The first recognition module is used for inputting the device training information and the device training image into a shooting angle classification layer of the model to be trained to recognize shooting angles, so as to obtain shooting angle training information corresponding to the preset device;
The second recognition module is used for inputting the shooting angle training information and the device training image into a defect recognition layer of the model to be trained, and recognizing defect type training information corresponding to the preset device from defect types corresponding to the shooting angle training information;
the comparison module is used for comparing the defect type training information with the defect marking information to obtain target loss information;
And the training module is used for training the model to be trained based on the target loss information to obtain a defect type identification model.
In a fifth aspect, the present application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory perform communication with each other through the communication bus;
A memory for storing a computer program;
And the processor is used for realizing the defect type identification method according to any embodiment of the first aspect or realizing the defect type identification model training method according to the second aspect when executing the program stored in the memory.
In a sixth aspect, the present application provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the defect type identification method according to any one of the embodiments of the first aspect or implements the defect type identification model training method according to the second aspect.
Compared with the prior art, the technical scheme provided by the embodiment of the application has the following advantages: according to the method provided by the embodiment of the application, the device detection image of the device to be detected and the device detection information of the device to be detected are obtained, wherein the device detection image is an image obtained based on the shooting result of the device to be detected, and the device detection information is the material information and the circuit structure file of the device to be detected; inputting the device detection information and the device detection image into a shooting angle classification layer of the defect type identification model to identify shooting angles, so as to obtain shooting angle information corresponding to the device to be detected; inputting shooting angle information and a device detection image into a defect identification layer of a defect type identification model, and identifying a target defect type corresponding to a device to be detected from defect types corresponding to the shooting angle information. According to the method, a device detection image of a device to be detected and device detection information of the device to be detected can be obtained, shooting angle identification is carried out in a shooting angle classification layer of a defect type identification model according to the device detection information and the device detection image to obtain shooting angle information, a target defect type corresponding to the device to be detected is identified in a defect identification layer of the defect type identification model according to the shooting angle information and the device detection image, namely, the shooting angle of the device to be detected is identified firstly, then the target defect type of the device to be detected is identified in the defect type corresponding to the shooting angle, so that the defect type can be classified based on spatial characteristics firstly, defects under the spatial characteristics are further identified, and accordingly the identification accuracy of defect products in the defect type identification process of the device to be detected can be improved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and together with the description, serve to explain the principles of the application.
In order to more clearly illustrate the embodiments of the application or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, and it will be obvious to a person skilled in the art that other drawings can be obtained from these drawings without inventive effort.
One or more embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings, in which like references indicate similar elements, and in which the figures of the drawings are not to be taken in a limiting sense, unless otherwise indicated.
FIG. 1 is a system architecture diagram of a defect type identification method according to an embodiment of the present application;
FIG. 2 is a flow chart illustrating a defect type identification method according to an embodiment of the present application;
FIG. 3 is a flowchart of a method for training a defect type recognition model according to an embodiment of the present application;
FIG. 4 is a schematic diagram of a defect type identifying apparatus according to an embodiment of the present application;
FIG. 5 is a schematic diagram of a training device for defect type identification model according to an embodiment of the present application;
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the application.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are some embodiments of the present application, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.
The following disclosure provides many different embodiments, or examples, for implementing different structures of the application. In order to simplify the present disclosure, components and arrangements of specific examples are described below. They are, of course, merely examples and are not intended to limit the application. Furthermore, the present application may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed.
In order to solve the technical problem of how to accurately identify defective products in the prior art, the application provides a defect type identification method, a model training method, a device and a storage medium, which can improve the accuracy of identifying defective products.
The first embodiment of the present application provides a defect type identification method, which can be applied to a system architecture shown in fig. 1, where the system architecture includes at least an image acquisition module 101 and an identification module 102, and the image acquisition module 101 and the identification module 102 establish a communication connection.
Next, based on the system architecture, the defect type identification method is described in detail, as shown in fig. 2, and the defect type identification method includes:
step 201, obtaining a device detection image of a device to be detected and device detection information of the device to be detected, wherein the device detection image is an image obtained based on a shooting result of the device to be detected, and the device detection information is material information and a circuit structure file of the device to be detected.
Because part of defects can only be shot at a specific angle, the device detection image of the device to be detected can be a plurality of detection images obtained by shooting the device to be detected at multiple angles, so that the defect is shot in the acquired device detection image under the condition that the device to be detected has the defect.
The device to be detected may be a chip component, such as a resistor, a capacitor, an inductor, a diode, a triode, a MOS transistor, a Ball grid array (Ball GRID ARRAY, BGA) device, a multi-PIN chip, or a socket component, such as an in-line discrete device, a dual in-line package (DIP package) chip, or various multi-PIN in-line interfaces. The embodiment of the application can detect common defect types in the welding process of the patch element, such as: many, few, wrong, many tin, few tin, empty welding, cold welding, refusing to weld, tin bridge, tin crack, golden finger tin pick up, float, turn to white, tombstoning, skew, extremely reverse, have the foreign matter etc. also can detect plug-in components device welding process defect, for example: kneeling, multi-tin, insufficient PTH solder filling, white spots, pull tips, dewetting and non-wetting, board deformation, tin connection, solder joint hollowness, tin overflow, tin balls, blow holes or pinholes, cold welding, soldering leakage, soldering flux residues and the like.
The device detection information is Material information of a device to be detected and a circuit structure file of the device to be detected, wherein the Material information is a Bill of materials (BOM), and the circuit structure file can be a structure file (Gerber) of a Pcb board.
The device detection image can be directly generated based on the shooting result of the image to be detected, or can be generated after image enhancement based on the shooting result.
In one embodiment, the step of acquiring the device inspection image of the device to be inspected and the device inspection information of the device to be inspected includes: acquiring a shooting result of a device to be detected and device detection information of the device to be detected; and carrying out image enhancement on the shooting result of the device to be detected, improving the definition of the shooting result, and taking the enhanced shooting result as a device detection image.
In this embodiment, the image enhancement is performed on the shooting result, so that the definition of the device detection image can be improved, and the defect type identification is more accurate.
Step 202, inputting the device detection information and the device detection image into a shooting angle classification layer of the defect type identification model to perform shooting angle identification, and obtaining shooting angle information corresponding to the device to be detected.
The defect type recognition model may be a two-layer recognition model, such as including a photographing angle classification layer and a defect recognition layer. The shooting angle classification layer can firstly identify shooting angle information of the device to be detected corresponding to the device detection image, and then identify the target defect type in the defect types corresponding to the shooting angle information based on the defect identification layer, so that accuracy of defect type identification is improved.
In one embodiment, the shooting angle classification layer includes a device modeling layer and a shooting angle identification layer.
Inputting the device detection information and the device detection image into a shooting angle classification layer of a defect type identification model for shooting angle identification, and obtaining shooting angle information corresponding to the device to be detected comprises the following steps: inputting the device detection information into a device modeling layer for three-dimensional modeling to obtain a model image corresponding to the device to be detected; and inputting the model image and the device detection image into a shooting angle identification layer to identify the shooting angle, so as to obtain shooting angle information.
In this embodiment, the device to be detected may be subjected to stereoscopic modeling in the device modeling layer based on the device detection information of the device to be detected, to obtain a model image of the device to be detected, such as a welded 3D image, and then the welded 3D image is adjusted to an angle consistent with the device detection image from the shooting angle identification layer, so as to implement shooting angle identification, and obtain shooting angle information.
The shooting angle information may be a shooting angle when the device detects an image, or may be a spatial position (for example, may be three-dimensional coordinate data, such as coordinate data of the device detected image in a spatial coordinate system or a polar coordinate system, etc.) representing the shooting angle, and the shooting angle information may include at least one shooting angle or at least one spatial position, without limitation.
Among the above mentioned defect types, the above mentioned defect types may be distinguished according to different shooting angles, for example, multiple pieces, fewer pieces, and mistakes may be classified into shooting angle information 1, where the shooting angle information 1 includes a spatial position 1, and the spatial position 1 may correspond to a top view shooting angle; the tin-less, the cold solder, the tin bridge and the tin crack can be classified into shooting angle information 2, the shooting angle information 2 comprises a space position 2, and the space position 2 can correspond to a side view shooting angle; the non-wetting, tin ball, tin connection and cold welding can be classified into shooting angle information 3, the shooting angle information 3 comprises a space position 2 and a space position 3, and the space position 3 can correspond to a microcosmic shooting angle; kneeling feet, pull tips, pinholes or blowholes, missing welds, etc. can be categorized as shooting angle information 4, and the shooting angle information 4 can include a spatial position 3 and a spatial position 1, i.e., a microscopic shooting angle and a top view shooting angle, which can both be represented by three-dimensional coordinates or polar coordinates. It should be noted that the classification of the shooting angle information 1 to 4 is merely illustrative, and each type may be classified in a generalized manner according to actual needs, and is not limited.
The shooting angle classification layer can identify the shooting angle, so that the analog image keeps an angle consistent with the device detection image, and an identification basis is provided for defect type identification.
In one embodiment, the step of inputting device detection information into a device modeling layer to perform stereoscopic modeling to obtain a model image corresponding to a device to be detected includes: acquiring device packaging information corresponding to a device to be detected through material information in the device detection information; based on preset device standard information, circuit structure files in device detection information and device packaging information, stereoscopic image simulation is carried out on the device to be detected, and a model image is generated.
In this embodiment, A structural file Gerber input system of A bill of materials BOM and A Pcb may be used, where the system may obtain packaging information (packaging) of A device to be detected according to A datA table (datasheet) corresponding to A material information index of the device to be detected on the BOM in the system, and then combine the Gerber information and device standard information (such as IPC-A-610 standard) to perform stereo image simulation on the device to be detected, construct A welding stereo image of the device to be detected, such as A welding 3D image, adjust the 3D image to an angle consistent with the device detection image, and record A spatial position (may be three-dimensional coordinate datA, for example, A spatial coordinate system, A polar coordinate system, etc.) corresponding to the angle as shooting angle information of the device detection image.
In one embodiment, the step of inputting the model image and the device detection image into the shooting angle identification layer to identify the shooting angle, and obtaining shooting angle information includes: performing angle adjustment on the model image at least once, and matching the model image subjected to each angle adjustment with a device detection image to obtain an angle matching result; and outputting the current angle information of the model image as shooting angle information under the condition that the angle matching result indicates that the model image after angle adjustment is matched with the device detection image.
In this embodiment, when the shooting angle is identified in the shooting angle identifying layer, the angle of the model image may be adjusted, and after each angle adjustment, the model image and the device detection image are matched to obtain an angle matching result, and when the angle matching result indicates that the model image after angle adjustment is matched with the device detection image, the current angle information of the model image is output as shooting angle information, and if the angle matching result indicates that the model image is not matched, the angle is continuously adjusted until the model image is matched.
Step 203, inputting the shooting angle information and the device detection image into a defect identification layer of the defect type identification model, and identifying the target defect type corresponding to the device to be detected from the defect types corresponding to the shooting angle information.
According to the method, the device detection image of the device to be detected and the device detection information of the device to be detected can be obtained, shooting angle identification is carried out in a shooting angle classification layer of the defect type identification model according to the device detection information and the device detection image to obtain shooting angle information, the target defect type corresponding to the device to be detected is identified from the defect types corresponding to the shooting angle information according to the shooting angle information and the device detection image in the defect identification layer of the defect type identification model, namely the shooting angle of the device to be detected is identified firstly, then the target defect type of the device to be detected is identified from the defect types corresponding to the shooting angle, so that the identification accuracy of defect products in the defect type identification process of the device to be detected is improved, defects can be classified through spatial characteristics, defect data identification is balanced when the defect identification types are increased, and the defect data of a certain type are prevented from being too much to cause the model training to have obvious deviation.
In one embodiment, before the step of inputting the device detection information and the device detection image into the shooting angle classification layer of the defect type identification model to perform shooting angle identification to obtain shooting angle information corresponding to the device to be detected, the method further includes: and obtaining a defect type identification model.
The defect type recognition model can be obtained based on training and learning of a model to be trained, and the model to be trained can be a deep learning system, such as a convolutional neural network, without limitation. The training process of the defect type recognition model is described in the following defect type recognition model training method, and will not be described herein.
Based on the same technical concept, a second embodiment of the present application provides a defect type identification model training method, as shown in fig. 3, where the method includes:
Step 301, obtaining a device training image of a preset device, device training information of the preset device and defect labeling information corresponding to the device training image, wherein the device training image is an image obtained based on a result of shooting the preset device, and the device training information is material information and a circuit structure file of the preset device.
The preset device may be a chip component, such as a resistor, a capacitor, an inductor, a diode, a triode, a MOS transistor, a BGA component, a multi-PIN chip, or a socket component, such as a direct-plug discrete device, a DIP packaged chip, various multi-PIN direct-plug interfaces, or the like. Based on a defect type identification model obtained through training of a preset device, defects of the device to be detected can be identified.
The device training image is an image obtained based on the shooting result of the preset device, and comprises a normal image and a defect image, and the marking of the device body, pins, bonding pads and the like can be realized through the defect marking information of the defect image. The type of the component can be identified by marking the component body, the conditions of welding spots and soldering tin can be detected by marking all pins, the specific shooting angle information of the component can be known by marking the body and all the pad areas, and when the welding state of each pin is marked in the training process of the model, the normal pins and the defective pins are marked so as to provide more shooting angle information. Wherein, the data quantity of the defect images of the components corresponding to different shooting angle information should be similar or the same.
Step 302, inputting the device training information and the device training image into a shooting angle classification layer of the model to be trained to perform shooting angle identification, and obtaining shooting angle training information corresponding to the preset device.
Step 303, inputting the shooting angle training information and the device training image into a defect identification layer of the model to be trained, and identifying the defect type training information corresponding to the preset device from the defect types corresponding to the shooting angle training information.
And step 304, comparing the defect type training information with the defect labeling information to obtain target loss information.
The target loss information can be a loss function, for example, by adopting cross entropy, the difference between defect type training information and defect labeling information can be sensitively measured, and the performance of the defect type identification model is improved.
And step 305, training the model to be trained based on the target loss information to obtain a defect type identification model.
In this embodiment, the method for training the defect type recognition model may train the model to be trained based on the device training image of the preset device, the device training information of the preset device, and the defect labeling information corresponding to the device training image, where the defect type recognition model obtained by training is a double-layer recognition model including a shooting angle classification layer and a defect recognition layer, and may perform defect type recognition on the device to be detected.
In the above embodiments, the defect type recognition model is trained and the defect type recognition model is used to perform defect recognition, when the defect is recognized, the shooting angle information is first recognized, and then the target defect type corresponding to the device to be detected is recognized from the defect types corresponding to the shooting angle information, so that the model can maintain the accuracy of defect recognition when the defect recognition types are increased.
The defect type recognition model has autonomous learning ability for new defective types through training of a large number of good product images (normal images) and existing defective product type images (defective images), and has new defective ability without adjusting judgment parameters. In the use process, the defect type identification model can be added with a new judgment standard through learning, and the new bad judgment is added without sacrificing the existing judgment standard, so that the coverage range of defect type identification can be improved, and the erroneous judgment of products can be avoided. In addition, the defect type identification model has universality for all products, and when different products are produced by a production line, the corresponding judging program does not need to be switched, so that the compatibility of defect identification is improved.
Based on the same technical concept, a third embodiment of the present application provides a defect type identifying apparatus, as shown in fig. 4, including:
A first obtaining module 401, configured to obtain a device detection image of a device to be detected and device detection information of the device to be detected, where the device detection image is an image obtained based on a result of photographing the device to be detected, and the device detection information is material information and a circuit structure file of the device to be detected;
The angle recognition module 402 is configured to input the device detection information and the device detection image into a shooting angle classification layer of a defect type recognition model to perform shooting angle recognition, so as to obtain shooting angle information corresponding to the device to be detected;
The type identifying module 403 is configured to input the shooting angle information and the device detection image into a defect identifying layer of the defect type identifying model, and identify a target defect type corresponding to the device to be detected from defect types corresponding to the shooting angle information.
The device can conduct defect recognition through a double-layer recognition model, firstly, the spatial characteristics are determined according to a target image of a target device under a target angle, at least one corresponding defect type is determined based on the spatial characteristics and a shooting angle classification layer, and then a specific target defect type of the target image is determined from at least one defect type based on the defect recognition layer and the target image, so that the recognition accuracy of a defect product is improved.
The embodiment of the application also provides a device for training the defect type recognition model, as shown in fig. 5, which comprises:
A second obtaining module 501, configured to obtain a device training image of a preset device, device training information of the preset device, and defect labeling information corresponding to the device training image, where the device training image is an image obtained based on a result of capturing the preset device, and the device training information is material information and a circuit structure file of the preset device;
The first recognition module 502 is configured to input the device training information and the device training image into a shooting angle classification layer of a model to be trained to perform shooting angle recognition, so as to obtain shooting angle training information corresponding to the preset device;
A second identifying module 503, configured to input the shooting angle training information and the device training image into a defect identifying layer of the model to be trained, and identify defect type training information corresponding to the preset device from defect types corresponding to the shooting angle training information;
a comparison module 504, configured to compare the defect type training information and the defect labeling information to obtain target loss information;
And the training module 505 is configured to train the model to be trained based on the target loss information, so as to obtain a defect type identification model.
As shown in fig. 6, an embodiment of the present application provides an electronic device including a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 perform communication with each other through the communication bus 114,
A memory 113 for storing a computer program;
In one embodiment of the present application, the processor 111 is configured to implement the defect type identification method or the defect type identification model training method provided in any one of the foregoing method embodiments when executing the program stored in the memory 113.
The communication bus mentioned by the above terminal may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the terminal and other devices.
The memory may include random access memory (Random Access Memory, RAM) or may include non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The processor may be a general-purpose processor, including a central processing unit (Central Processing Unit, abbreviated as CPU), a network processor (Network Processor, abbreviated as NP), etc.; but may also be a digital signal processor (DIGITAL SIGNAL Processing, DSP), application Specific Integrated Circuit (ASIC), field-Programmable gate array (FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components.
The embodiment of the application also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor implements the defect type identification method or the defect type identification model training method provided in any one of the method embodiments described above.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
From the above description of embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus a general purpose hardware platform, or may be implemented by hardware. Based on such understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the related art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the method described in the respective embodiments or some parts of the embodiments.
It is to be understood that the terminology used herein is for the purpose of describing particular example embodiments only, and is not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises," "comprising," "includes," "including," and "having" are inclusive and therefore specify the presence of stated features, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and/or groups thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring their performance in the particular order described or illustrated, unless an order of performance is explicitly stated. It should also be appreciated that additional or alternative steps may be used.
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application. In the description, suffixes such as "module", "part" or "unit" for representing elements are used only for facilitating the description of the present application, and have no specific meaning per se. Thus, "module," "component," or "unit" may be used in combination.
The foregoing is only a specific embodiment of the application to enable those skilled in the art to understand or practice the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method of defect type identification, the method comprising:
Acquiring a device detection image of a device to be detected and device detection information of the device to be detected, wherein the device detection image is an image obtained based on a result of shooting the device to be detected, and the device detection information is material information and a circuit structure file of the device to be detected;
Inputting the device detection information and the device detection image into a shooting angle classification layer of a defect type identification model to carry out shooting angle identification, so as to obtain shooting angle information corresponding to the device to be detected;
Inputting the shooting angle information and the device detection image into a defect identification layer of the defect type identification model, and identifying the target defect type corresponding to the device to be detected from the defect types corresponding to the shooting angle information.
2. The method according to claim 1, wherein the photographing angle classification layer includes a device modeling layer and a photographing angle recognition layer, the step of inputting the device detection information and the device detection image into the photographing angle classification layer of the defect type recognition model to perform photographing angle recognition, and the step of obtaining photographing angle information corresponding to the device to be detected includes:
inputting the device detection information into the device modeling layer for three-dimensional modeling to obtain a model image corresponding to the device to be detected;
and inputting the model image and the device detection image into the shooting angle identification layer to identify the shooting angle, so as to obtain the shooting angle information.
3. The method according to claim 2, wherein the step of inputting the device inspection information into the device modeling layer for stereoscopic modeling to obtain the model image corresponding to the device to be inspected includes:
acquiring device packaging information corresponding to the device to be detected through the material information in the device detection information;
And carrying out three-dimensional image simulation on the device to be detected based on preset device standard information, a circuit structure file in the device detection information and the device packaging information, and generating the model image.
4. The method according to claim 2, wherein the step of inputting the model image and the device detection image into the photographing angle recognition layer for photographing angle recognition, and obtaining the photographing angle information includes:
performing angle adjustment on the model image at least once, and matching the model image subjected to each angle adjustment with the device detection image to obtain an angle matching result;
and outputting current angle information of the model image as the shooting angle information under the condition that the angle matching result indicates that the model image after angle adjustment is matched with the device detection image.
5. The method according to claim 1, wherein the step of acquiring the device inspection image of the device to be inspected and the device inspection information of the device to be inspected includes:
acquiring a shooting result of the device to be detected and device detection information of the device to be detected;
and carrying out image enhancement on the shooting result of the device to be detected to obtain a device detection image.
6. A method for training a defect type recognition model, the method comprising:
Acquiring a device training image of a preset device, device training information of the preset device and defect marking information corresponding to the device training image, wherein the device training image is an image obtained based on a result of shooting the preset device, and the device training information is material information and a circuit structure file of the preset device;
inputting the device training information and the device training image into a shooting angle classification layer of a model to be trained to perform shooting angle identification, so as to obtain shooting angle training information corresponding to the preset device;
Inputting the shooting angle training information and the device training image into a defect identification layer of the model to be trained, and identifying defect type training information corresponding to the preset device from defect types corresponding to the shooting angle training information;
comparing the defect type training information with the defect labeling information to obtain target loss information;
training the model to be trained based on the target loss information to obtain the defect type identification model according to any one of claims 1 to 5.
7. A defect type identifying apparatus, the apparatus comprising:
The device detection device comprises a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used for acquiring a device detection image of a device to be detected and device detection information of the device to be detected, the device detection image is an image obtained based on a result of shooting the device to be detected, and the device detection information is material information and a circuit structure file of the device to be detected;
The angle identification module is used for inputting the device detection information and the device detection image into a shooting angle classification layer of a defect type identification model to carry out shooting angle identification, so as to obtain shooting angle information corresponding to the device to be detected;
The type identification module is used for inputting the shooting angle information and the device detection image into a defect identification layer of the defect type identification model, and identifying the target defect type corresponding to the device to be detected from the defect types corresponding to the shooting angle information.
8. A defect type recognition model training apparatus, the apparatus comprising:
The device training image is an image obtained based on a result of shooting the preset device, and the device training information is material information and a circuit structure file of the preset device;
The first recognition module is used for inputting the device training information and the device training image into a shooting angle classification layer of the model to be trained to recognize shooting angles, so as to obtain shooting angle training information corresponding to the preset device;
The second recognition module is used for inputting the shooting angle training information and the device training image into a defect recognition layer of the model to be trained, and recognizing defect type training information corresponding to the preset device from defect types corresponding to the shooting angle training information;
the comparison module is used for comparing the defect type training information with the defect marking information to obtain target loss information;
And the training module is used for training the model to be trained based on the target loss information to obtain a defect type identification model.
9. The electronic device is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
A memory for storing a computer program;
A processor for implementing the defect type identification method according to any one of claims 1 to 5 or the defect type identification model training method according to claim 6 when executing a program stored on a memory.
10. A computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the defect type recognition method of any one of claims 1-5 or implements the defect type recognition model training method of claim 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410675448.7A CN118506093A (en) | 2024-05-28 | 2024-05-28 | Defect type identification method, model training method, device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410675448.7A CN118506093A (en) | 2024-05-28 | 2024-05-28 | Defect type identification method, model training method, device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118506093A true CN118506093A (en) | 2024-08-16 |
Family
ID=92238637
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410675448.7A Pending CN118506093A (en) | 2024-05-28 | 2024-05-28 | Defect type identification method, model training method, device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118506093A (en) |
-
2024
- 2024-05-28 CN CN202410675448.7A patent/CN118506093A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN1318839C (en) | Automated Optical Inspection Method for Defective Components on Printed Circuit Boards | |
| KR101438157B1 (en) | Board inspection method | |
| Wu et al. | Feature-extraction-based inspection algorithm for IC solder joints | |
| JP4493421B2 (en) | Printed circuit board inspection apparatus, printed circuit board assembly inspection line system, and program | |
| US20120327215A1 (en) | High speed optical sensor inspection system | |
| JP6988499B2 (en) | Inspection management system, inspection management device, inspection management method | |
| JP5441990B2 (en) | Inspection method | |
| US20140125375A1 (en) | Board inspection apparatus system and board inspection method | |
| CN115136750B (en) | Data correlation between different machines in an electronic component production line | |
| JP2010271165A (en) | Inspection device for printed circuit board | |
| JP6988500B2 (en) | Inspection management system, inspection management device, inspection management method | |
| JP6277754B2 (en) | Quality control system and internal inspection device | |
| CN116912230B (en) | Patch welding quality detection method and device, electronic equipment and storage medium | |
| US8860456B2 (en) | Non-destructive tilt data measurement to detect defective bumps | |
| CN110132960B (en) | Detection method of circuit board assembly | |
| JP6387620B2 (en) | Quality control system | |
| CN111899248B (en) | Automatic PCB solder paste defect detection method based on machine learning | |
| US7551765B2 (en) | Electronic component detection system | |
| Han et al. | A template matching based method for surface-mount rectangular-pin-chip positioning and defect detection | |
| CN118506093A (en) | Defect type identification method, model training method, device and storage medium | |
| JP4333349B2 (en) | Mounting appearance inspection method and mounting appearance inspection apparatus | |
| KR100719712B1 (en) | Bump checking apparatus and method | |
| US20060034506A1 (en) | Machine vision analysis system and method | |
| JP2008191106A (en) | Soldering state inspection method | |
| JP6468002B2 (en) | Solder inspection apparatus and solder inspection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |