CN112881427B - Electronic component defect detection device and method based on visible light and infrared thermal imaging - Google Patents
Electronic component defect detection device and method based on visible light and infrared thermal imaging Download PDFInfo
- Publication number
- CN112881427B CN112881427B CN202110041762.6A CN202110041762A CN112881427B CN 112881427 B CN112881427 B CN 112881427B CN 202110041762 A CN202110041762 A CN 202110041762A CN 112881427 B CN112881427 B CN 112881427B
- Authority
- CN
- China
- Prior art keywords
- visible light
- camera system
- infrared
- defect
- infrared camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000007547 defect Effects 0.000 title claims abstract description 93
- 238000000034 method Methods 0.000 title claims abstract description 28
- 238000001931 thermography Methods 0.000 title claims abstract description 15
- 238000001514 detection method Methods 0.000 title claims description 65
- 238000012549 training Methods 0.000 claims description 15
- 230000002950 deficient Effects 0.000 claims description 13
- 238000013527 convolutional neural network Methods 0.000 claims description 9
- 238000013528 artificial neural network Methods 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 4
- 238000004140 cleaning Methods 0.000 claims description 3
- 238000012360 testing method Methods 0.000 claims description 3
- 238000005336 cracking Methods 0.000 claims 2
- 238000012216 screening Methods 0.000 claims 1
- 238000010438 heat treatment Methods 0.000 abstract description 3
- 239000012535 impurity Substances 0.000 abstract description 2
- 239000000047 product Substances 0.000 description 15
- 238000004519 manufacturing process Methods 0.000 description 9
- 230000003287 optical effect Effects 0.000 description 8
- 238000007650 screen-printing Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 5
- 238000012545 processing Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000003466 welding Methods 0.000 description 4
- 238000004804 winding Methods 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 3
- 206010061245 Internal injury Diseases 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 230000003203 everyday effect Effects 0.000 description 2
- 238000010147 laser engraving Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 239000003973 paint Substances 0.000 description 2
- 208000007443 Neurasthenia Diseases 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 206010003549 asthenia Diseases 0.000 description 1
- 239000011248 coating agent Substances 0.000 description 1
- 238000000576 coating method Methods 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 238000005530 etching Methods 0.000 description 1
- 239000003889 eye drop Substances 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000003331 infrared imaging Methods 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000004093 laser heating Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
- 210000003462 vein Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/95—Investigating the presence of flaws or contamination characterised by the material or shape of the object to be examined
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N25/00—Investigating or analyzing materials by the use of thermal means
- G01N25/72—Investigating presence of flaws
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8854—Grading and classifying of flaws
- G01N2021/8861—Determining coordinates of flaws
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8854—Grading and classifying of flaws
- G01N2021/8874—Taking dimensions of defect into account
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/8851—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges
- G01N2021/8887—Scan or image signal processing specially adapted therefor, e.g. for scan signal adjustment, for detecting different kinds of defects, for compensating for structures, markings, edges based on image processing techniques
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- General Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Immunology (AREA)
- Pathology (AREA)
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
Abstract
The invention relates to a device and a method for detecting defects of electronic components by visible light and infrared thermal imaging, wherein a laser heater, a visible light camera system and an infrared camera system are sequentially arranged above a workpiece conveyor belt according to the conveying direction; the infrared camera system is slidably arranged on the sliding rail; the infrared camera system and the visible light camera system are connected with the industrial personal computer; the visible light camera system is used for collecting visible light pictures and transmitting the visible light pictures to the industrial personal computer; the industrial personal computer controls the infrared camera system to move and take pictures. The invention adopts the visible light camera to identify and position the workpiece of the component, thereby effectively identifying and mixing the workpiece and product impurities. After the electronic component type identification and the foreign matter identification are completed, the heated electronic component enters an infrared camera shooting area. The laser is used for heating electronic components, in order to reduce the volume of the whole machine, the infrared heating area can be overlapped with the visible light shooting area, and the infrared filter is added to the visible light camera.
Description
Technical Field
The invention belongs to the technical field of electronic component detection, and particularly relates to a device and a method for detecting defects of electronic components by visible light and infrared thermal imaging, which are used for quality detection of surface defects and internal defects of electronic component products.
Background
The existing electronic component surface defect detection industry uses detection equipment for identifying simple defects and a manual auxiliary detection mode, and the method has the advantages of low automation degree, low efficiency and very high production cost, and the existing equipment has limited detection capability and cannot identify complex defects such as dark lines, cracks in background noise, internal injuries and the like.
In the field of electronic component production and processing, an automatic assembly line is basically realized at present. Taking precise winding inductance as an example, an electronic component is generally processed by the following steps: (1) Materials such as magnetic cores and the like are fed into a production line from an equipment inlet, and coil winding is completed on a precise winding machine sequentially through a conveyor belt; (2) Then the enameled wire is conveyed to laser paint removing equipment through a conveyor belt, and the surface coating of the enameled wire is removed; (3) Then the wire ends are sent to an automatic pin welding table through a conveyor belt to finish the welding of the paint removing wire ends and the welding points of the magnetic cores; (4) Then the mixture is sent to a laser screen printer through a conveyor belt to finish the etching of screen printing; (5) Finally, the sample is sent to a quality detection station through a conveyor belt, and a CCD device or a man-made part completes the detection procedure to confirm the integrity.
The precise winding inductor has the advantages that the knowledge in the processing process is subjected to mechanical conveying process of a conveying belt or a clamp for more than 5 times, the mechanical conveying efficiency is high, but the defects are obvious, namely, physical damage can be caused to components, and the indexes, the precision and the performance of products are affected. When the enamelled coil is extruded and pulled by mechanical movement in the production process, deformation and stretching of the coil and loosening of welding spots can be caused, and once the length of each coil is excessively uneven, the quality factor can not meet the product requirement. It can be seen that computing a tiny device defect may result in: (1) the basic parameter index of the product can not meet the requirement. (2) the uniformity of the index of the product is poor. (3) lifetime and power consumption of the product are also potentially problematic. So that the common component processing equipment has a special detection unit.
The current mainstream detection methods include two types according to defect complexity: one is a simple defect, which is automatically identified by the machine after image processing using a CCD imaging detection or infrared imaging unit. The method comprises the steps of carrying out image preprocessing on defects such as obvious defects and obvious cracks, and comparing and judging whether the defects exist or not through a template method, or separating the defect images through angular point feature detection and an SVM classifier. The implementation method is also used for most products on the market at present, and is characterized in that the method can realize complete automatic detection, but can only be effective for a few simple defects.
In addition, aiming at complex defects, such as fine dark veins on the surface of a component, cracks under a rough texture background, cracks on silk screen printing of fine cracks in a stain background and other complex defect characteristics, the current factory practice mainly displays the processed image on a display, and quality detection workers pick out defective products through naked eyes. For the detection of complex defects, the important problems currently exist are:
(1) The manual defect detection has high requirements on workers, a plurality of workers are required to be equipped on a product line every day to ensure the normal operation of the product line, and the payroll cost paid by enterprises is very high;
(2) The manual defect detection has high requirements on the serious degree of workers, and the problems of eye-drop, inattention, neurasthenia and the like can occur after the workers work for a long time every day, so that the probability of false division is greatly improved;
(3) If the complex defects have complex background noise through the existing automatic detection technology, the interference such as amplified porous texture characteristics, silk-screen pseudo defects and the like cannot be removed, and the detection rate of products is seriously affected.
(4) For defects such as dark lines, internal injuries and the like, the existing detection method cannot be effective.
Disclosure of Invention
Aiming at precise electronic components such as inductors and the like to be inspected, the invention provides a device and a method for detecting defects of the electronic components by using visible light and infrared thermal imaging. Defects such as scratches, broken corners, cracks, dark cracks and the like are also key bottlenecks for detecting defects of electronic components at present, are common problems in the industry, and have huge value. The invention adopts the infrared flaw detection technology and the visible light fusion computer vision technology, improves the detection function of defects such as scratches, broken corners, cracks, dark cracks and the like which are difficult to detect, and improves the defect detection rate of electronic components. The precise electronic components detected by the equipment do not need to be detected manually, the requirements of the product yield are met, the production efficiency is improved, and the production cost is reduced. Meanwhile, the selling price of the equipment is reduced through technology localization, and the technology upgrading in the field of defect detection of precise electronic components is further promoted.
The specific technical scheme is as follows:
the device for detecting the defects of the electronic components of the visible light and infrared thermal imaging is characterized in that a laser heater, a visible light camera system and an infrared camera system are sequentially arranged above a workpiece conveyor belt according to the conveying direction; the infrared camera system is slidably arranged on the sliding rail;
the infrared camera system and the visible light camera system are connected with the industrial personal computer;
the visible light camera system is used for collecting visible light pictures and transmitting the visible light pictures to the industrial personal computer; the industrial personal computer controls the infrared camera system to move and take pictures.
The sliding rail is arranged on the supporting structural member, the infrared camera system comprises an infrared camera and an optical lens, the infrared camera is fixed on the linear motor, and the linear motor is slidably arranged on the sliding rail; the linear motor is connected with a driver through a control flat cable, and the driver is connected with an industrial personal computer.
The visible light camera system comprises a visible light camera, a lens and a light supplementing lamp.
The method for detecting the defects of the electronic components by using visible light and infrared thermal imaging comprises the following steps:
the visible light camera system collects visible light pictures; after the picture is acquired, uploading the picture to an industrial personal computer, identifying the industrial personal computer by adopting a target identification neural network, judging whether the workpiece is a specified electronic component type, if the workpiece is the specified electronic component type, further calculating the coordinate position (x, y) of the component, providing the coordinate position to an infrared camera system, moving the infrared camera system to a proper position, shooting a surface image of the component heated by a laser heater, and detecting defects by using the surface image by a target detection unit; if the electronic component type is not specified, judging that other electronic components or foreign matters are provided for the sorting unit for cleaning;
after the target detection unit finishes classification and positioning of the images of the defective electronic components, the position information (x, y), the defect size (w, h) and the defect type c of the defective electronic components are sent to the sorting unit, the sorting unit receives the information and then makes a judgment to determine whether the electronic components at the designated positions are defective or not, and if yes, the electronic components are cleaned.
The target detection unit receives an image of the surface of the component heated by the laser heater 4 shot by the infrared camera system 2, performs image preprocessing on the image, and then sends the image into a defect target detection convolutional neural network, and the convolutional neural network outputs coordinate position information (x, y) of a workpiece, defect size (w, h) and defect type c;
the construction method of the convolutional neural network comprises the following steps:
s1, collecting more than 5000 infrared pictures of a defective workpiece and a normal workpiece respectively;
s2, manually marking the positions and the sizes of defects and defect categories, wherein the defect categories comprise seven categories of scratches, broken angles, cracks, dark cracks, silk screen printing, normal defects, other defects and the like;
s3, increasing the number of various samples to more than 5 ten thousand samples by data enhancement methods such as rotation, scaling, gray level adjustment, noise increase, random clipping and the like;
s4, checking the number of various samples, and guaranteeing the approaching of the number of various images through the enhancement method in the step S3;
s5, sending the preprocessed infrared image as training set and test set data into a target detection network for training, wherein the target detection network is not limited to SSD, YOLO, fasterCNN and other target detection networks;
s6, after training is completed, storing a training model;
s7, loading a training model for defect detection.
The invention adopts the visible light camera to identify and position the workpiece of the component, and can effectively identify and mix the workpiece and product impurities. After the electronic component type identification and the foreign matter identification are completed, the heated electronic component enters an infrared camera shooting area. According to the invention, the laser is adopted to heat the electronic components, so that the volume of the whole machine is reduced, the infrared heating area can be overlapped with the visible light shooting area, and the infrared filter is added to the visible light camera to avoid interference, so that the interference of infrared light to the visible light camera is reduced.
(1) According to the invention, through infrared thermal imaging, not only can the defect characteristics of scratches, angles of collapse, cracks and the like be obtained, but also the dark crack characteristics can be captured, and the target detection is carried out on the image defects through a neural network, so that the detection efficiency is greatly improved.
(2) Compared with a visible light image, the infrared thermal imaging picture adopted by the invention has the advantages that the noise interference is reduced, the surface screen printing texture can be pressed to a certain extent, and the interference of the noise and the screen printing on defect detection is avoided.
(3) The invention can not only distinguish product defects, but also identify defect types directly on the basis of defect detection through a multi-classification neural network, is convenient for subsequent defect root analysis, and can reduce interference of silk screen printing on defect detection accuracy by introducing silk screen defect types.
(4) The invention adopts the visible light camera for identifying and positioning a plurality of components, can inspect other workpieces and foreign matters, sends the infrared image into the neural network to detect the defects, reduces the calculated amount of the industrial personal computer by a high-speed moving shooting mode of the linear motor, and improves the detection efficiency of the workpieces.
The invention is fully automatic in detection, greatly improves the production efficiency, and can save a great amount of labor cost. The invention can detect defects which cannot be observed by eyes and the existing equipment, the classification is more accurate, the false-division rate is reduced, and the product yield is improved. And the defects can be qualitatively and quantitatively analyzed, the problem of the production line can be rapidly positioned and solved, and the productivity of the production line can be improved.
Drawings
FIG. 1 is a schematic view of the apparatus of the present invention;
FIG. 2 is a schematic diagram of an infrared camera system according to the present invention;
FIG. 3 is a schematic diagram of a detection flow of the present invention;
FIG. 4 is a schematic diagram of the detection of the present invention;
FIG. 5 is a second schematic diagram of the detection of the present invention.
Detailed Description
The specific technical scheme for implementing the invention is combined.
As shown in fig. 1, a laser heater 4, a visible light camera system 3 and an infrared camera system 2 are sequentially arranged above a workpiece conveyor belt 5 according to the conveying direction; the infrared camera system 2 is slidably arranged on the sliding rail 1;
the infrared camera system 2 and the visible light camera system 3 are connected with the industrial personal computer;
the visible light camera system 3 is used for collecting visible light pictures and transmitting the visible light pictures to the industrial personal computer; the industrial personal computer controls the infrared camera system 2 to move and take pictures.
As shown in fig. 2, the sliding rail 1 is mounted on the supporting structural member 21, the infrared camera system 2 comprises an infrared camera 26 and an optical lens 27, the infrared camera 26 is fixed on the linear motor 23, and the linear motor 23 is slidably mounted on the sliding rail 1; the linear motor 23 is connected with a driver 25 through a control flat cable 24, and the driver 25 is connected with an industrial personal computer.
The visible light camera system 3 includes a visible light camera, a lens, and a light supplement lamp.
As shown in fig. 3, the method for detecting defects of electronic components by visible light and infrared thermal imaging comprises the following steps:
the visible light camera system 3 collects visible light pictures; after the picture is acquired, uploading the picture to an industrial personal computer, identifying the industrial personal computer by adopting a target identification neural network, judging whether the workpiece is a specified electronic component type, if the workpiece is the specified electronic component type, further calculating the coordinate position (x, y) of the component, providing the coordinate position for an infrared camera system 2, moving the infrared camera system 2 to a proper position, shooting the surface image of the component heated by a laser heater 4, and detecting the defect by using the surface image by a target detection unit; if the electronic component type is not specified, judging that other electronic components or foreign matters are provided for the sorting unit for cleaning;
after the target detection unit finishes classification and positioning of the images of the defective electronic components, the position information (x, y), the defect size (w, h) and the defect type c of the defective electronic components are sent to the sorting unit, the sorting unit receives the information and then makes a judgment to determine whether the electronic components at the designated positions are defective or not, and if yes, the electronic components are cleaned.
The target detection unit receives an image of the surface of the component heated by the laser heater 4 shot by the infrared camera system 2, performs image preprocessing on the image, and then sends the image into a defect target detection convolutional neural network, and the convolutional neural network outputs coordinate position information (x, y) of a workpiece, defect size (w, h) and defect type c;
the construction method of the convolutional neural network comprises the following steps:
s1, collecting more than 5000 infrared pictures of a defective workpiece and a normal workpiece respectively;
s2, manually marking the positions and the sizes of defects and defect categories, wherein the defect categories comprise seven categories of scratches, broken angles, cracks, dark cracks, silk screen printing, normal defects, other defects and the like;
s3, increasing the number of various samples to more than 5 ten thousand samples by data enhancement methods such as rotation, scaling, gray level adjustment, noise increase, random clipping and the like;
s4, checking the number of various samples, and guaranteeing the approaching of the number of various images through the enhancement method in the step S3;
s5, sending the preprocessed infrared image as training set and test set data into a target detection network for training, wherein the target detection network is not limited to SSD, YOLO, fasterCNN and other target detection networks;
s6, after training is completed, storing a training model;
s7, loading a training model for defect detection.
In order to improve the detection efficiency of the system, the visible light image sensor and the linear motor 23 are added. The visible light image sensor is a visible light camera system 3, and is mainly used for detecting and positioning components, and the linear motor 23 is mainly used for driving the infrared image sensor unit, namely the infrared camera system 2 to move.
As shown in fig. 2, the slide rail 1 is fixed on the supporting structural member 21, the linear motor 23 is mounted on the slide rail 1, the infrared camera 26 and the optical lens 27 are mounted at the bottom of the linear motor 23 through the structural member, and the infrared camera 26 and the optical lens 27 are used for moving in a straight line when the linear motor 23 operates.
The power supply, the infrared camera signal wire and the linear motor control wire are connected with a driver 25 arranged on the supporting structural member through a control flat cable 24, the driver 25 is communicated with the industrial personal computer through a USB interface, the industrial personal computer drives the linear motor 23 to pull the infrared camera 26 and the optical lens 27 to a designated position through positioning information, and the imaging is carried out and then the imaging is carried out on the industrial personal computer for defect positioning and identification.
As shown in fig. 1, the visible light camera system 3 is in front of the infrared camera system 2, the center position of the visible light camera system 3 is away from the center position s of the infrared camera system 2, and is used for identifying components and performing optical measurement, positioning the center position (x, y) of the components, after the components are positioned, the conveyor belt moves at a constant speed v, after an interval t=y/v, the coordinate position of the components is (x, 0), the components reach the positions below the infrared camera 26 and the optical lens 27, the infrared camera 26 and the optical lens 27 are driven by the linear motor 23 to move to the (x, 0) position, the pictures of the components are shot, and model reasoning is sent.
The infrared camera system 2 can take a picture of only one component at a time, and can use a defect target recognition network to recognize the picture, so that the processing problem of simultaneous detection of a plurality of components is avoided. By introducing the linear motor 23 machine, the computational complexity is reduced.
As shown in fig. 4 and 5, the electronic component 6 is on the workpiece conveyor 5, the workpiece conveyor 5 rotates from right to left, the surface temperature of the electronic component 6 rises after laser heating by the laser heater 4, the response of the defect to the temperature is different from the defect-free part, and the response of the different defects to the temperature is also different. Therefore, there is a difference in defect feature map seen by the picture obtained by infrared thermal imaging, and defect detection and defect classification can be performed based on the feature.
The electronic component 6 will typically identify the chip type using a laser engraving technique, and laser engraving will form on the surface of the component, similar to the texture of scratch defects, which will reduce the accuracy of defect identification therebetween. In order to reduce the interference of the silk-screen marks on the accuracy, the silk-screen area is used as an independent labeling type, so that the temperature response difference caused by the difference of the silk-screen marks can be prevented, the situation that the silk-screen marks are mistakenly identified as crack curves can be avoided, and the samples identified as the silk-screen marks and the normal samples in actual use belong to the normal samples.
Claims (5)
1. The detection method of the electronic component defect detection device of visible light and infrared thermal imaging comprises the steps that a laser heater (4), a visible light camera system (3) and an infrared camera system (2) are sequentially arranged above a workpiece conveyor belt (5) according to the conveying direction; the infrared camera system (2) is slidably arranged on the sliding rail (1);
the infrared camera system (2) and the visible light camera system (3) are connected with the industrial personal computer;
the visible light camera system (3) is used for collecting visible light pictures and transmitting the visible light pictures to the industrial personal computer; the industrial personal computer controls the infrared camera system (2) to move and take a picture;
the detection method is characterized by comprising the following steps of:
the visible light camera system (3) collects visible light pictures; after the picture is acquired, uploading the picture to an industrial personal computer, identifying the picture by adopting a target identification neural network, judging whether the workpiece is a specified electronic component type, if the workpiece is the specified electronic component type, further calculating the coordinate position (x, y) of the component, providing the coordinate position for an infrared camera system (2), moving the infrared camera system (2) to a proper position, shooting a surface image of the component heated by a laser heater (4), and detecting defects by using the surface image by a target detection unit; if the electronic component type is not specified, judging that other electronic components or foreign matters are provided for the sorting unit for cleaning;
the target detection unit receives an image of the surface of the component heated by the laser heater (4) shot by the infrared camera system (2), performs image preprocessing on the image, and then sends the image into the defect target detection convolutional neural network, and the convolutional neural network outputs coordinate position information (x, y), defect size (w, h) and defect type c of the workpiece;
after the target detection unit finishes classification and positioning of the images of the defective electronic components, the position information (x, y), the defect size (w, h) and the defect type c of the defective electronic components are sent to the sorting unit, the sorting unit receives the information and then makes a judgment to determine whether the electronic components at the designated positions are defective or not, and if yes, the electronic components are cleaned.
2. The method for detecting the defect of the electronic component by using the visible light and infrared thermal imaging according to claim 1, wherein the method for constructing the convolutional neural network is as follows:
s1, collecting more than 5000 infrared pictures of a defective workpiece and a normal workpiece respectively;
s2, manually marking the positions, sizes and types of the defects;
s3, increasing the number of various samples to more than 5 ten thousand samples by a data enhancement method;
s4, checking the number of various samples, and guaranteeing the approaching of the number of various images through the enhancement method in the step S3;
s5, sending the preprocessed infrared image serving as training set and testing set data into a target detection network for training;
s6, after training is completed, storing a training model;
s7, loading a training model for defect detection.
3. The method of claim 2, wherein the defect types include scratches, chipping, cracking, dark cracking, silk screening, normal, and other seven types.
4. The method for detecting defects of electronic components by visible light and infrared thermal imaging according to claim 2, wherein the data enhancement method comprises rotation, scaling, gray scale adjustment, noise addition and random clipping.
5. The method of detecting defects of electronic components by visible light and infrared thermal imaging according to claim 2, wherein said target detection network comprises SSD, YOLO, fasterCNN.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110041762.6A CN112881427B (en) | 2021-01-13 | 2021-01-13 | Electronic component defect detection device and method based on visible light and infrared thermal imaging |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110041762.6A CN112881427B (en) | 2021-01-13 | 2021-01-13 | Electronic component defect detection device and method based on visible light and infrared thermal imaging |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112881427A CN112881427A (en) | 2021-06-01 |
| CN112881427B true CN112881427B (en) | 2023-06-13 |
Family
ID=76045297
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110041762.6A Active CN112881427B (en) | 2021-01-13 | 2021-01-13 | Electronic component defect detection device and method based on visible light and infrared thermal imaging |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112881427B (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20220015635A (en) * | 2020-07-31 | 2022-02-08 | 주식회사 엘지에너지솔루션 | Device for detecting foreign matter and method for detecting foreign matter |
| CN113267508A (en) * | 2021-06-18 | 2021-08-17 | 苏州鹰蚁视觉科技有限公司 | Electronic component defect detection equipment and detection method |
| CN113702441A (en) * | 2021-09-23 | 2021-11-26 | 合肥维信诺科技有限公司 | Fracture detection device and fracture detection method |
| CN114022435A (en) * | 2021-10-29 | 2022-02-08 | 中国船舶重工集团公司第七0四研究所 | Method for establishing pipeline leakage identification system based on machine vision |
| CN114585253A (en) * | 2022-03-15 | 2022-06-03 | 杭州测质成科技有限公司 | PCB component installation and rechecking system and method based on image recognition |
| CN114594138A (en) * | 2022-03-15 | 2022-06-07 | 东莞先导先进科技有限公司 | A semiconductor device substrate inspection system and method |
| CN115063347B (en) * | 2022-05-17 | 2025-04-25 | 厦门理工学院 | Laser paint removal detection method, device, equipment and readable storage medium |
| CN115082434B (en) * | 2022-07-21 | 2022-12-09 | 浙江华是科技股份有限公司 | Multi-source feature-based magnetic core defect detection model training method and system |
| CN116182932B (en) * | 2022-12-13 | 2025-09-30 | 哈尔滨市科佳通用机电股份有限公司 | Railway train bolt anti-loosening mark detection device and method |
| CN115950896A (en) * | 2022-12-19 | 2023-04-11 | 广东华中科技大学工业技术研究院 | Two-stage cooperative machine vision detection method for transparent component |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005016991A (en) * | 2003-06-24 | 2005-01-20 | Railway Technical Res Inst | Infrared structure diagnostic system |
| JP2007315964A (en) * | 2006-05-26 | 2007-12-06 | Hitachi Ltd | Underwater defect inspection apparatus and underwater defect inspection method |
| CN205607877U (en) * | 2015-12-14 | 2016-09-28 | 昆明理工大学 | Utilize image acquisition device of auxiliary laser point formation of image |
| CN207908384U (en) * | 2017-10-13 | 2018-09-25 | 成都精工华耀科技有限公司 | A kind of double light spectrum image-forming rails and fastener abnormal detector |
| CN108732507A (en) * | 2018-05-21 | 2018-11-02 | 中国科学技术大学 | A kind of lithium battery defect detecting device based on battery temperature field and visible images |
Family Cites Families (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6236044B1 (en) * | 1998-08-21 | 2001-05-22 | Trw Inc. | Method and apparatus for inspection of a substrate by use of a ring illuminator |
| US6605807B2 (en) * | 2000-06-05 | 2003-08-12 | The Boeing Company | Infrared crack detection apparatus and method |
| JP2007271507A (en) * | 2006-03-31 | 2007-10-18 | Dainippon Printing Co Ltd | Defect detection method, defect detection apparatus, and defect detection program |
| AU2007361989B2 (en) * | 2007-12-06 | 2013-10-03 | Lockheed Martin Corporation | Non-destructive inspection using laser- ultrasound and infrared thermography |
| CN101788500A (en) * | 2009-01-23 | 2010-07-28 | 宇创视觉科技股份有限公司 | Optical detection device for solar cell and method thereof |
| JP2011021999A (en) * | 2009-07-15 | 2011-02-03 | Kyodo Design & Planning Corp | Substrate inspecting apparatus |
| CN105466945B (en) * | 2015-12-30 | 2018-08-28 | 深圳市创科自动化控制技术有限公司 | A kind of infrared detection method and detection device of automatic positioning test position |
| US10564108B2 (en) * | 2017-07-03 | 2020-02-18 | Saudi Arabian Oil Company | Apparatus and method for nondestructively inspecting fiberglass and nonmetallic pipes |
| CN109060822B (en) * | 2018-07-17 | 2021-06-29 | 上海大学 | Image processing method and system for long pulse infrared nondestructive testing sequence |
| CN111103303A (en) * | 2018-10-27 | 2020-05-05 | 成都四加二驴途电子商务有限责任公司 | Automatic detection control system for micro precision bearing |
| CN110560377A (en) * | 2019-09-11 | 2019-12-13 | 河南奇测电子科技有限公司 | Cylindrical metal surface defect detection device based on machine vision |
| CN111141741B (en) * | 2019-12-09 | 2023-02-10 | 合肥中科迪宏自动化有限公司 | Detection apparatus capable of identifying label defects of printer |
| CN111795977A (en) * | 2020-06-08 | 2020-10-20 | 武汉大学 | Online real-time monitoring system of various monitoring equipment for metal additive manufacturing |
| CN111999308A (en) * | 2020-08-27 | 2020-11-27 | 银河水滴科技（北京）有限公司 | Defect detection control method, control device and control system |
-
2021
- 2021-01-13 CN CN202110041762.6A patent/CN112881427B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005016991A (en) * | 2003-06-24 | 2005-01-20 | Railway Technical Res Inst | Infrared structure diagnostic system |
| JP2007315964A (en) * | 2006-05-26 | 2007-12-06 | Hitachi Ltd | Underwater defect inspection apparatus and underwater defect inspection method |
| CN205607877U (en) * | 2015-12-14 | 2016-09-28 | 昆明理工大学 | Utilize image acquisition device of auxiliary laser point formation of image |
| CN207908384U (en) * | 2017-10-13 | 2018-09-25 | 成都精工华耀科技有限公司 | A kind of double light spectrum image-forming rails and fastener abnormal detector |
| CN108732507A (en) * | 2018-05-21 | 2018-11-02 | 中国科学技术大学 | A kind of lithium battery defect detecting device based on battery temperature field and visible images |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112881427A (en) | 2021-06-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112881427B (en) | Electronic component defect detection device and method based on visible light and infrared thermal imaging | |
| CN110827256B (en) | Optical and thermal infrared multi-stage imaging detection method and device for defects of transparent component | |
| RU2763417C2 (en) | System and related method for detecting small defects on/in glass sheet on process line | |
| CN103913468B (en) | Many defects of vision checkout equipment and the method for large-scale LCD glass substrate on production line | |
| KR101772673B1 (en) | Multiple optics vision inspection system | |
| CN101576508B (en) | Chip appearance defect automatic detection device and detection method | |
| CN102621156B (en) | Image-processing-based automatic micro part sorting system | |
| CN110216080A (en) | Quality monitoring system of PCB processing production line based on image contrast | |
| CN110044909B (en) | A device and method for detecting defects of motor rotor solder joints based on image processing | |
| CN109724984A (en) | Device and method for defect detection and identification based on deep learning algorithm | |
| CN107886500A (en) | A kind of production monitoring method and system based on machine vision and machine learning | |
| CN110567976B (en) | Screen printing defect detection device and detection method for mobile phone cover plate based on machine vision | |
| CN114252452A (en) | Apparatus and method for on-line detection of appearance defects and contour dimensions of a small rotary body | |
| CN111220544A (en) | Lens quality detection device and detection method | |
| CN104237252A (en) | Machine-vision-based method and device for intelligently detecting surface micro-defects of product | |
| CN109142509B (en) | Round steel magnetic powder flaw detection method and device | |
| CN106442553A (en) | Detection and recognition device and method for sprayed codes on cylindrical surfaces of copper rings | |
| CN212301356U (en) | A visual inspection device for wheel hub welds | |
| CN113894055A (en) | Hardware surface defect detection and classification system and method based on machine vision | |
| CN115388778B (en) | Computer screen quality control device | |
| CN111060518A (en) | A Stamping Part Defect Recognition Method Based on Instance Segmentation | |
| CN114113129B (en) | A system and method for identifying and capturing tiny defects in lenses | |
| CN109693140A (en) | A kind of intelligent flexible production line and its working method | |
| CN111307817B (en) | Online detection method and system for PCB production process of intelligent production line | |
| Fu et al. | Medicine glass bottle defect detection based on machine vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20240923 Address after: Room 1003, 10th floor, unit 2, building 7, No. 399, West Fucheng Avenue, high tech Zone, Chengdu, Sichuan 610000 Patentee after: Sichuan Yuran Electronic Technology Co.,Ltd. Country or region after: China Address before: 611130 No.2, 3 / F, building 4, no.232-302, Chang'an Road, Wenjiang District, Chengdu City, Sichuan Province Patentee before: Sichuan Yuran Zhihui Technology Co.,Ltd. Country or region before: China |
|
| TR01 | Transfer of patent right |