CN110827249A - Electronic equipment backboard appearance flaw detection method and equipment - Google Patents
Electronic equipment backboard appearance flaw detection method and equipment Download PDFInfo
- Publication number
- CN110827249A CN110827249A CN201911033637.XA CN201911033637A CN110827249A CN 110827249 A CN110827249 A CN 110827249A CN 201911033637 A CN201911033637 A CN 201911033637A CN 110827249 A CN110827249 A CN 110827249A
- Authority
- CN
- China
- Prior art keywords
- electronic device
- appearance
- image
- backboard
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Quality & Reliability (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
- Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
- Image Processing (AREA)
Abstract
The invention aims to provide a method and equipment for detecting appearance flaws of a backboard of electronic equipment, wherein the method comprises the steps of obtaining an appearance image of the electronic equipment; extracting a backboard appearance area image of the electronic equipment from the appearance image of the electronic equipment, and adjusting the backboard appearance area image into an image with the same length and width; inputting the adjusted backboard appearance area image into a model of combining the FPN network and the backbone network after training is finished; receiving a defect detection result of a backboard appearance area of the electronic device, which is output from a model of the FPN network combined with the backbone network, wherein the defect detection result comprises: the defect type of the back plate of the electronic equipment, the position of the defect in the back plate of the electronic equipment and the confidence coefficient of the defect detection result can accurately identify the defect difference of the appearance of the back plate of the electronic equipment of the second-hand electronic equipment such as a mobile phone.
Description
Technical Field
The invention relates to the field of computers, in particular to a method and equipment for detecting appearance flaws of a backboard of electronic equipment.
Background
At present, the defect detection of the appearance of the back plate of the electronic equipment such as a mobile phone and the like in the second-hand electronic equipment is mainly based on the traditional image algorithm and is carried out by the modes of color space transformation, filtering, feature point extraction and mode matching, and the defect of a certain area can only be detected based on the traditional detection method, but the definition of the defect cannot be distinguished.
Disclosure of Invention
The invention aims to provide a method and equipment for detecting appearance defects of a backboard of electronic equipment.
According to an aspect of the present invention, there is provided a method for detecting an appearance defect of a backplane of an electronic device, the method comprising:
acquiring an appearance image of the electronic equipment;
extracting a backboard appearance area image of the electronic equipment from the appearance image of the electronic equipment, and adjusting the backboard appearance area image into an image with the same length and width;
inputting the adjusted backboard appearance area image into a model of combining the FPN network and the backbone network after training is finished;
receiving a defect detection result of a backboard appearance area of the electronic device, which is output from a model of the FPN network combined with the backbone network, wherein the defect detection result comprises: the defect detection method includes the steps of detecting defects of a back plate of an electronic device, determining positions of the defects in the back plate of the electronic device and confidence of defect detection results.
Further, in the above method, extracting a backboard appearance area image of the electronic device from the appearance image of the electronic device includes:
and extracting a backboard appearance area image of the electronic equipment from the appearance image of the electronic equipment by adopting a Unet instance segmentation mode.
Further, in the above method, the front 2 layer of the backhaul network adopts res structure, and the back 2 layer of the network adopts an initiation structure.
Further, in the above method, after receiving the defect detection result of the backplane appearance area of the electronic device from the model of the FPN network in combination with the backbone network, the method further includes:
identifying whether a confidence level of the flaw detection result is greater than a first preset threshold,
and if the defect type is larger than the first preset threshold, outputting result information including the defect type of the backboard of the electronic equipment and the position of the defect in the backboard of the electronic equipment.
Further, in the above method, before inputting the image of the backplane appearance area into the model combining the FPN network and the backbone network, the method further includes:
presetting a model of combining an FPN network with a backbone network and initial model parameters thereof;
inputting the backboard appearance area image of the sample electronic equipment into a FPN network with current model parameters and combining with a backbone network model to obtain a flaw prediction result of the backboard of the sample electronic equipment, wherein the flaw prediction result comprises the following steps: the defect type of the back panel of the sample electronic device, the position of the defect in the back panel of the sample electronic device, and the confidence of the defect detection result;
calculating a difference between the flaw prediction result and a true flaw result of the sample electronic device based on a preset objective function, identifying whether the difference is greater than a second preset threshold,
if the difference value is larger than a second preset threshold value, a fourth step of executing from the second step again after updating the model parameters of the FPN network combined with the backbone network based on the difference value;
and if the difference is smaller than or equal to a second preset threshold, step five, taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
Further, in the above method, adjusting the image of the appearance area of the backplane to an image with the same length and width includes:
and scaling the length direction of the backboard appearance area image.
According to another aspect of the present invention, there is also provided an electronic device backplane appearance defect detecting apparatus, the apparatus comprising:
the device comprises a first device, a second device and a third device, wherein the first device is used for acquiring an appearance image of the electronic equipment;
the second device is used for extracting a backboard appearance area image of the electronic equipment from the appearance image of the electronic equipment and adjusting the backboard appearance area image into an image with the same length and width;
the third device is used for inputting the adjusted backboard appearance area image into a model of the FPN network combined with the backbone network after training is finished;
a fourth device, configured to receive, from the model combining the FPN network and the backbone network, an output defect detection result of a backplane appearance area of the electronic device, where the defect detection result includes: the defect detection method includes the steps of detecting defects of a back plate of an electronic device, determining positions of the defects in the back plate of the electronic device and confidence of defect detection results.
Further, in the foregoing device, the second means is configured to extract a backplane appearance area image of the electronic device from the appearance image of the electronic device by using a Unet instance division manner.
Further, in the above device, the front 2 layer of the backhaul network adopts a res structure, and the rear 2 layer of the network adopts an initiation structure.
Further, in the foregoing apparatus, the fourth device is further configured to identify whether a confidence of the defect detection result is greater than a first preset threshold, and if the confidence is greater than the first preset threshold, output result information including a defect type of a backplane of the electronic apparatus and a position of the defect in the backplane of the electronic apparatus.
Further, the above apparatus further includes a fifth device, including:
a fifth device, configured to preset a model of the FPN network combined with the backbone network and initial model parameters thereof;
a fifth second device, configured to input the backplane appearance area image of the sample electronic device into a model combining a back bone network and an FPN network with current model parameters, to obtain a defect prediction result of the backplane of the sample electronic device, where the defect prediction result includes: the defect type of the back panel of the sample electronic device, the position of the defect in the back panel of the sample electronic device, and the confidence of the defect detection result;
a fifth third means for calculating a difference between the defect prediction result and a true defect result of the sample electronic device based on a preset objective function, and identifying whether the difference is greater than a second preset threshold, if the difference is greater than the second preset threshold, executing a fifth fourth means for, based on the fifth fourth means, updating the model parameter of the FPN network combined with the backbone network based on the difference, and then starting execution from the fifth second means again;
and if the difference is smaller than or equal to a second preset threshold, executing a fifth device, and taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
Further, in the above device, the second means is configured to scale the length direction of the backboard appearance area image.
The present invention also provides a computing-based device, comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
acquiring an appearance image of the electronic equipment;
extracting a backboard appearance area image of the electronic equipment from the appearance image of the electronic equipment, and adjusting the backboard appearance area image into an image with the same length and width;
inputting the adjusted backboard appearance area image into a model of combining the FPN network and the backbone network after training is finished;
receiving a defect detection result of a backboard appearance area of the electronic device, which is output from a model of the FPN network combined with the backbone network, wherein the defect detection result comprises: the defect detection method includes the steps of detecting defects of a back plate of an electronic device, determining positions of the defects in the back plate of the electronic device and confidence of defect detection results.
The present invention also provides a computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
acquiring an appearance image of the electronic equipment;
extracting a backboard appearance area image of the electronic equipment from the appearance image of the electronic equipment, and adjusting the backboard appearance area image into an image with the same length and width;
inputting the adjusted backboard appearance area image into a model of combining the FPN network and the backbone network after training is finished;
receiving a defect detection result of a backboard appearance area of the electronic device, which is output from a model of the FPN network combined with the backbone network, wherein the defect detection result comprises: the defect detection method includes the steps of detecting defects of a back plate of an electronic device, determining positions of the defects in the back plate of the electronic device and confidence of defect detection results.
Compared with the prior art, the method has the advantages that the appearance image of the electronic equipment is obtained; extracting a backboard appearance area image of the electronic equipment from the appearance image of the electronic equipment, and adjusting the backboard appearance area image into an image with the same length and width; inputting the adjusted backboard appearance area image into a model of combining the FPN network and the backbone network after training is finished; receiving a defect detection result of a backboard appearance area of the electronic device, which is output from a model of the FPN network combined with the backbone network, wherein the defect detection result comprises: the defect type of the back plate of the electronic equipment, the position of the defect in the back plate of the electronic equipment and the confidence coefficient of the defect detection result can accurately identify the defect difference of the appearance of the back plate of the electronic equipment of the second-hand electronic equipment such as a mobile phone.
Drawings
Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings:
FIG. 1 is a flow chart illustrating a method for detecting appearance defects of a backplane of an electronic device according to an embodiment of the invention;
FIG. 2 is a diagram illustrating a defect detection result according to an embodiment of the invention;
fig. 3 is a schematic diagram illustrating a model of a FPN network combined with a backbone network according to an embodiment of the present invention.
The same or similar reference numbers in the drawings identify the same or similar elements.
Detailed Description
The present invention is described in further detail below with reference to the attached drawing figures.
In a typical configuration of the present application, the terminal, the device serving the network, and the trusted party each include one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.
The invention provides a method for detecting appearance flaws of a backboard of electronic equipment, which comprises the following steps:
step S1, acquiring an appearance image of the electronic equipment;
step S2, extracting a backplane appearance area image of the electronic device from the appearance image of the electronic device, and adjusting the backplane appearance area image to an image with the same length and width;
here, the rear panel appearance area of the electronic apparatus includes a rear surface area, in which a camera and the like are generally mounted, in addition to the front screen area and the side surface area where the electronics are disposed.
The aspect ratio of the backboard appearance area image is abnormal, so that subsequent model identification is facilitated, image loss is avoided, and the aspect ratio of the backboard appearance area image needs to be adjusted to 1: 1; step S3, inputting the adjusted backboard appearance area image into the model of the FPN network combined with the backbone network after the training is finished;
step S4, receiving a defect detection result of the backplane appearance area of the electronic device from the model of the FPN network combined with the backbone network, where the defect detection result includes: the defect detection method includes the steps of detecting defects of a back plate of an electronic device, determining positions of the defects in the back plate of the electronic device and confidence of defect detection results.
The model of the FPN network in combination with the backbone network can be as shown in fig. 3.
Here, the defect detection results of the backplane appearance area of the electronic device, which are received and output from the model of the FPN network combined with the backbone network, as shown in fig. 2, each defect detection result includes cls, x1, y1, x2, y2, score, where cls is a defect type, x1, y1, x2, and y2 are 4 coordinates of the location of the defect in the backplane appearance area image, and score is the confidence of the defect.
The invention mainly utilizes the improved characteristic pyramid (FPN) network and the deep learning model of the backbone network to accurately identify the defect difference of the appearance of the back plate of the electronic equipment of the second-hand electronic equipment such as a mobile phone and accurately distinguish the defect types.
In an embodiment of the method for detecting the appearance defects of the back panel of the electronic device, in step S2, the extracting the image of the appearance area of the back panel of the electronic device from the image of the appearance of the electronic device includes:
and extracting a backboard appearance area image of the electronic equipment from the appearance image of the electronic equipment by adopting a Unet instance segmentation mode.
Here, the backboard appearance area image can be obtained quickly and efficiently by the Unet instance segmentation.
In an embodiment of the method for detecting the appearance flaws of the electronic device backplane, the front 2 layers of the backbone network adopt a res structure, and the rear 2 layers of the network adopt an acceptance structure.
In an embodiment of the method for detecting the appearance defect of the back panel of the electronic device, in step S4, after receiving the defect detection result of the back panel appearance area of the electronic device from the model of the FPN network in combination with the backbone network, the method further includes:
identifying whether a confidence level of the flaw detection result is greater than a first preset threshold,
and if the defect type is larger than the first preset threshold, outputting result information including the defect type of the backboard of the electronic equipment and the position of the defect in the backboard of the electronic equipment.
Here, the defect types of the electronic device backplane may include: cracks, stent screen separation, deformation, chipping loss, large area paint drop, small area paint drop (deformation, indentation exposes color), indentation and no discoloration, deep scratch and different color from the surroundings, small dot and different color from the surroundings, chipping, and the like.
In this embodiment, by identifying the confidence of the flaw detection result, a reliable result can be screened from the flaw detection result and output.
In an embodiment of the method for detecting the appearance defects of the electronic device backplane, before the step S3 of inputting the backplane appearance region image into the model of the FPN network combined with the backbone network, the method further includes:
presetting a model of combining an FPN network with a backbone network and initial model parameters thereof;
inputting the backboard appearance area image of the sample electronic equipment into a FPN network with current model parameters and combining with a backbone network model to obtain a flaw prediction result of the backboard of the sample electronic equipment, wherein the flaw prediction result comprises the following steps: the defect type of the back panel of the sample electronic device, the position of the defect in the back panel of the sample electronic device, and the confidence of the defect detection result;
calculating a difference between the flaw prediction result and a true flaw result of the sample electronic device based on a preset objective function, identifying whether the difference is greater than a second preset threshold,
if the difference value is larger than a second preset threshold value, a fourth step of executing from the second step again after updating the model parameters of the FPN network combined with the backbone network based on the difference value;
and if the difference is smaller than or equal to a second preset threshold, step five, taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
And circularly training the model of the FPN network combined with the backbone network by identifying whether the difference value is greater than a second preset threshold, so as to obtain a reliable model.
In an embodiment of the method for detecting the appearance flaws of the back plate of the electronic device, adjusting the image of the appearance area of the back plate into an image with the same length and width includes:
and scaling the length direction of the backboard appearance area image.
Here, the backplate appearance region image may be scaled to 2048 × 2048 pixels to obtain an image with the backplate appearance region image adjusted to have the same length and width.
The invention provides an electronic equipment backboard appearance flaw detection device, which comprises:
the device comprises a first device, a second device and a third device, wherein the first device is used for acquiring an appearance image of the electronic equipment;
the second device is used for extracting a backboard appearance area image of the electronic equipment from the appearance image of the electronic equipment and adjusting the backboard appearance area image into an image with the same length and width;
here, the back panel appearance area of the electronic device includes a side area, in which a headphone hole, a speaker, a charging hole, and the like are generally installed, in addition to the electronic components disposed in the front screen area and the back area.
The aspect ratio of the backboard appearance area image is abnormal, so that subsequent model identification is facilitated, image loss is avoided, and the aspect ratio of the backboard appearance area image needs to be adjusted to 1: 1;
the third device is used for inputting the adjusted backboard appearance area image into a model of the FPN network combined with the backbone network after training is finished;
a fourth device, configured to receive, from the model combining the FPN network and the backbone network, an output defect detection result of a backplane appearance area of the electronic device, where the defect detection result includes: the defect detection method includes the steps of detecting defects of a back plate of an electronic device, determining positions of the defects in the back plate of the electronic device and confidence of defect detection results.
Here, the defect detection results of the backplane appearance area of the electronic device, which are output from the model of the FPN network combined with the backbone network, each defect detection result includes cls, x1, y1, x2, y2, and score, where cls is a defect type, x1, y1, x2, and y2 are 4 coordinates of the location of a defect in the backplane appearance area image, and score is a confidence of the defect.
The invention mainly utilizes the improved characteristic pyramid (FPN) network and the deep learning model of the backbone network to accurately identify the appearance difference of the back plate of the electronic equipment of the second-hand electronic equipment such as a mobile phone.
In an embodiment of the method for detecting the appearance flaws of the backplane of the electronic device, the second device is configured to extract the backplane appearance area image of the electronic device from the appearance image of the electronic device in a manner of Unet instance segmentation.
Here, the backboard appearance area image can be obtained quickly and efficiently by the Unet instance segmentation.
In an embodiment of the method for detecting the appearance flaws of the electronic device backplane, the front 2 layers of the backbone network adopt a res structure, and the rear 2 layers of the network adopt an acceptance structure.
In an embodiment of the method for detecting an appearance defect of a backplane of an electronic device, the fourth device is further configured to identify whether a confidence of the defect detection result is greater than a first preset threshold, and if the confidence is greater than the first preset threshold, output result information including a defect type of the backplane of the electronic device and a position of the defect in the backplane of the electronic device.
Here, the defect types of the electronic device backplane may sequentially include a shallow scratch, a hard scratch, and a chipping type, which sequentially increase in grade.
In this embodiment, by identifying the confidence of the flaw detection result, a reliable result can be screened from the flaw detection result and output.
In an embodiment of the method for detecting an appearance defect of a backplane of an electronic device, the method further includes a fifth apparatus including:
a fifth device, configured to preset a model of the FPN network combined with the backbone network and initial model parameters thereof;
a fifth second device, configured to input the backplane appearance area image of the sample electronic device into a model combining a back bone network and an FPN network with current model parameters, to obtain a defect prediction result of the backplane of the sample electronic device, where the defect prediction result includes: the defect type of the back panel of the sample electronic device, the position of the defect in the back panel of the sample electronic device, and the confidence of the defect detection result;
a fifth third means for calculating a difference between the defect prediction result and a true defect result of the sample electronic device based on a preset objective function, and identifying whether the difference is greater than a second preset threshold, if the difference is greater than the second preset threshold, executing a fifth fourth means for, based on the fifth fourth means, updating the model parameter of the FPN network combined with the backbone network based on the difference, and then starting execution from the fifth second means again;
and if the difference is smaller than or equal to a second preset threshold, executing a fifth device, and taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
And circularly training the model of the FPN network combined with the backbone network by identifying whether the difference value is greater than a second preset threshold, so as to obtain a reliable model.
In an embodiment of the method for detecting the appearance flaws of the back plate of the electronic device, the second device is configured to scale the length direction of the image of the appearance area of the back plate.
Here, the backplate appearance region image may be scaled to 2048 × 2048 pixels to obtain an image with the backplate appearance region image adjusted to have the same length and width.
The present invention also provides a computing-based device, comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
step S1, acquiring an appearance image of the electronic equipment;
step S2, extracting a backplane appearance area image of the electronic device from the appearance image of the electronic device, and adjusting the backplane appearance area image to an image with the same length and width;
step S3, inputting the adjusted backboard appearance area image into the model of the FPN network combined with the backbone network after the training is finished;
step S4, receiving a defect detection result of the backplane appearance area of the electronic device from the model of the FPN network combined with the backbone network, where the defect detection result includes: the defect detection method includes the steps of detecting defects of a back plate of an electronic device, determining positions of the defects in the back plate of the electronic device and confidence of defect detection results.
The present invention also provides a computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
step S1, acquiring an appearance image of the electronic equipment;
step S2, extracting a backplane appearance area image of the electronic device from the appearance image of the electronic device, and adjusting the backplane appearance area image to an image with the same length and width;
step S3, inputting the adjusted backboard appearance area image into the model of the FPN network combined with the backbone network after the training is finished;
step S4, receiving a defect detection result of the backplane appearance area of the electronic device from the model of the FPN network combined with the backbone network, where the defect detection result includes: the defect detection method includes the steps of detecting defects of a back plate of an electronic device, determining positions of the defects in the back plate of the electronic device and confidence of defect detection results.
For details of embodiments of each device and storage medium of the present invention, reference may be made to corresponding parts of each method embodiment, and details are not described herein again.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present application without departing from the spirit and scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is intended to include such modifications and variations as well.
It should be noted that the present invention may be implemented in software and/or in a combination of software and hardware, for example, as an Application Specific Integrated Circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the present invention may be executed by a processor to implement the steps or functions described above. Also, the software programs (including associated data structures) of the present invention can be stored in a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. Further, some of the steps or functions of the present invention may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
In addition, some of the present invention can be applied as a computer program product, such as computer program instructions, which when executed by a computer, can invoke or provide the method and/or technical solution according to the present invention through the operation of the computer. Program instructions which invoke the methods of the present invention may be stored on a fixed or removable recording medium and/or transmitted via a data stream on a broadcast or other signal-bearing medium and/or stored within a working memory of a computer device operating in accordance with the program instructions. An embodiment according to the invention herein comprises an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the apparatus to perform a method and/or solution according to embodiments of the invention as described above.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the apparatus claims may also be implemented by one unit or means in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Claims (14)
1. A method for detecting appearance defects of a backboard of an electronic device comprises the following steps:
acquiring an appearance image of the electronic equipment;
extracting a backboard appearance area image of the electronic equipment from the appearance image of the electronic equipment, and adjusting the backboard appearance area image into an image with the same length and width;
inputting the adjusted backboard appearance area image into a model of combining the FPN network and the backbone network after training is finished;
receiving a defect detection result of a backboard appearance area of the electronic device, which is output from a model of the FPN network combined with the backbone network, wherein the defect detection result comprises: the defect detection method includes the steps of detecting defects of a back plate of an electronic device, determining positions of the defects in the back plate of the electronic device and confidence of defect detection results.
2. The method of claim 1, wherein extracting a backplate appearance area image of the electronic device from the appearance image of the electronic device comprises:
and extracting a backboard appearance area image of the electronic equipment from the appearance image of the electronic equipment by adopting a Unet instance segmentation mode.
3. The method of claim 1, wherein the first 2 layers of the backhaul network adopt res structure, and the last 2 layers of the network adopt initiation structure.
4. The method of claim 1, wherein after receiving the output defect detection result of the backplane appearance area of the electronic device from the model of the FPN network in combination with the backbone network, further comprising:
identifying whether a confidence level of the flaw detection result is greater than a first preset threshold,
and if the defect type is larger than the first preset threshold, outputting result information including the defect type of the backboard of the electronic equipment and the position of the defect in the backboard of the electronic equipment.
5. The method of claim 1, wherein before inputting the backplane appearance region image into a model of FPN network combined with backbone network, further comprising:
presetting a model of combining an FPN network with a backbone network and initial model parameters thereof;
inputting the backboard appearance area image of the sample electronic equipment into a FPN network with current model parameters and combining with a backbone network model to obtain a flaw prediction result of the backboard of the sample electronic equipment, wherein the flaw prediction result comprises the following steps: the defect type of the back panel of the sample electronic device, the position of the defect in the back panel of the sample electronic device, and the confidence of the defect detection result;
calculating a difference between the flaw prediction result and a true flaw result of the sample electronic device based on a preset objective function, identifying whether the difference is greater than a second preset threshold,
if the difference value is larger than a second preset threshold value, a fourth step of executing from the second step again after updating the model parameters of the FPN network combined with the backbone network based on the difference value;
and if the difference is smaller than or equal to a second preset threshold, step five, taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
6. The method of claim 1, wherein adjusting the backplane appearance area image to an image of the same length and width comprises:
and scaling the length direction of the backboard appearance area image.
7. An electronic device backplate visual flaw detection apparatus, wherein the apparatus comprises:
the device comprises a first device, a second device and a third device, wherein the first device is used for acquiring an appearance image of the electronic equipment;
the second device is used for extracting a backboard appearance area image of the electronic equipment from the appearance image of the electronic equipment and adjusting the backboard appearance area image into an image with the same length and width;
the third device is used for inputting the adjusted backboard appearance area image into a model of the FPN network combined with the backbone network after training is finished;
a fourth device, configured to receive, from the model combining the FPN network and the backbone network, an output defect detection result of a backplane appearance area of the electronic device, where the defect detection result includes: the defect detection method includes the steps of detecting defects of a back plate of an electronic device, determining positions of the defects in the back plate of the electronic device and confidence of defect detection results.
8. The apparatus of claim 7, wherein the second means is configured to extract a backplane appearance area image of the electronic device from the appearance image of the electronic device by using a Unet instance segmentation method.
9. The apparatus of claim 7, wherein the front 2 layers of the backhaul network adopt res structure, and the back 2 layers of the network adopt initiation structure.
10. The apparatus of claim 7, wherein the fourth means is further configured to identify whether a confidence of the defect detection result is greater than a first preset threshold, and if so, output result information including a defect type of a backplane of the electronic device and a location of the defect in the backplane of the electronic device.
11. The apparatus of claim 7, further comprising a fifth apparatus comprising:
a fifth device, configured to preset a model of the FPN network combined with the backbone network and initial model parameters thereof;
a fifth second device, configured to input the backplane appearance area image of the sample electronic device into a model combining a back bone network and an FPN network with current model parameters, to obtain a defect prediction result of the backplane of the sample electronic device, where the defect prediction result includes: the defect type of the back panel of the sample electronic device, the position of the defect in the back panel of the sample electronic device, and the confidence of the defect detection result;
a fifth third means for calculating a difference between the defect prediction result and a true defect result of the sample electronic device based on a preset objective function, and identifying whether the difference is greater than a second preset threshold, if the difference is greater than the second preset threshold, executing a fifth fourth means for, based on the fifth fourth means, updating the model parameter of the FPN network combined with the backbone network based on the difference, and then starting execution from the fifth second means again;
and if the difference is smaller than or equal to a second preset threshold, executing a fifth device, and taking the model of the FPN network combined with the backbone network with the current model parameters as the model of the FPN network combined with the backbone network after the training is finished.
12. The apparatus of claim 7, wherein the second means is configured to scale a length direction of the backplane appearance area image.
13. A computing-based device, comprising:
a processor; and
a memory arranged to store computer executable instructions that, when executed, cause the processor to:
acquiring an appearance image of the electronic equipment;
extracting a backboard appearance area image of the electronic equipment from the appearance image of the electronic equipment, and adjusting the backboard appearance area image into an image with the same length and width;
inputting the adjusted backboard appearance area image into a model of combining the FPN network and the backbone network after training is finished;
receiving a defect detection result of a backboard appearance area of the electronic device, which is output from a model of the FPN network combined with the backbone network, wherein the defect detection result comprises: the defect detection method includes the steps of detecting defects of a back plate of an electronic device, determining positions of the defects in the back plate of the electronic device and confidence of defect detection results.
14. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions, when executed by a processor, cause the processor to:
acquiring an appearance image of the electronic equipment;
extracting a backboard appearance area image of the electronic equipment from the appearance image of the electronic equipment, and adjusting the backboard appearance area image into an image with the same length and width;
inputting the adjusted backboard appearance area image into a model of combining the FPN network and the backbone network after training is finished;
receiving a defect detection result of a backboard appearance area of the electronic device, which is output from a model of the FPN network combined with the backbone network, wherein the defect detection result comprises: the defect detection method includes the steps of detecting defects of a back plate of an electronic device, determining positions of the defects in the back plate of the electronic device and confidence of defect detection results.
Priority Applications (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911033637.XA CN110827249A (en) | 2019-10-28 | 2019-10-28 | Electronic equipment backboard appearance flaw detection method and equipment |
| JP2022502080A JP2022539912A (en) | 2019-10-28 | 2020-10-14 | Electronic device backplane appearance defect inspection method and apparatus |
| PCT/CN2020/120878 WO2021082921A1 (en) | 2019-10-28 | 2020-10-14 | Back cover appearance defect detection method for electronic apparatus, and apparatus |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911033637.XA CN110827249A (en) | 2019-10-28 | 2019-10-28 | Electronic equipment backboard appearance flaw detection method and equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110827249A true CN110827249A (en) | 2020-02-21 |
Family
ID=69550888
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911033637.XA Pending CN110827249A (en) | 2019-10-28 | 2019-10-28 | Electronic equipment backboard appearance flaw detection method and equipment |
Country Status (3)
| Country | Link |
|---|---|
| JP (1) | JP2022539912A (en) |
| CN (1) | CN110827249A (en) |
| WO (1) | WO2021082921A1 (en) |
Cited By (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112700428A (en) * | 2021-01-08 | 2021-04-23 | 北京网瑞达科技有限公司 | Method and device for identifying backboard element of switch |
| WO2021082918A1 (en) * | 2019-10-28 | 2021-05-06 | 上海悦易网络信息技术有限公司 | Screen appearance defect detection method and device |
| WO2021082921A1 (en) * | 2019-10-28 | 2021-05-06 | 上海悦易网络信息技术有限公司 | Back cover appearance defect detection method for electronic apparatus, and apparatus |
| CN115457007A (en) * | 2022-09-23 | 2022-12-09 | 章鱼博士智能技术(上海)有限公司 | A battery explosion-proof film detection method and equipment |
| US11798250B2 (en) | 2019-02-18 | 2023-10-24 | Ecoatm, Llc | Neural network based physical condition evaluation of electronic devices, and associated systems and methods |
| US11843206B2 (en) | 2019-02-12 | 2023-12-12 | Ecoatm, Llc | Connector carrier for electronic device kiosk |
| US11922467B2 (en) | 2020-08-17 | 2024-03-05 | ecoATM, Inc. | Evaluating an electronic device using optical character recognition |
| US11989710B2 (en) | 2018-12-19 | 2024-05-21 | Ecoatm, Llc | Systems and methods for vending and/or purchasing mobile phones and other electronic devices |
| US12033454B2 (en) | 2020-08-17 | 2024-07-09 | Ecoatm, Llc | Kiosk for evaluating and purchasing used electronic devices |
| US12271929B2 (en) | 2020-08-17 | 2025-04-08 | Ecoatm Llc | Evaluating an electronic device using a wireless charger |
| US12300059B2 (en) | 2019-02-12 | 2025-05-13 | Ecoatm, Llc | Kiosk for evaluating and purchasing used electronic devices |
| US12321965B2 (en) | 2020-08-25 | 2025-06-03 | Ecoatm, Llc | Evaluating and recycling electronic devices |
| US12322259B2 (en) | 2018-12-19 | 2025-06-03 | Ecoatm, Llc | Systems and methods for vending and/or purchasing mobile phones and other electronic devices |
| US12380420B2 (en) | 2019-12-18 | 2025-08-05 | Ecoatm, Llc | Systems and methods for vending and/or purchasing mobile phones and other electronic devices |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114677326B (en) * | 2022-02-15 | 2024-08-09 | 杭州印鸽科技有限公司 | Device, system and method for detecting surface flaws of shell of printing mobile phone based on machine vision and deep learning |
| CN116681428B (en) * | 2023-08-03 | 2023-09-29 | 天津奇立软件技术有限公司 | Intelligent recycling management system and method for electronic equipment |
| CN117288761B (en) * | 2023-11-27 | 2024-02-06 | 天津市海迅科技发展有限公司 | Flaw detection classification evaluation method and system based on test materials |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106875381A (en) * | 2017-01-17 | 2017-06-20 | 同济大学 | A kind of phone housing defect inspection method based on deep learning |
| US20170277979A1 (en) * | 2016-03-22 | 2017-09-28 | Inovision Software Solutions, Inc. | Identifying defect on specular surfaces |
| US20190056333A1 (en) * | 2017-08-16 | 2019-02-21 | Siemens Energy, Inc. | Method and system for detecting defects on surface of object |
| CN109711474A (en) * | 2018-12-24 | 2019-05-03 | 中山大学 | An Algorithm for Detection of Surface Defects of Aluminum Materials Based on Deep Learning |
| CN109859190A (en) * | 2019-01-31 | 2019-06-07 | 北京工业大学 | A kind of target area detection method based on deep learning |
| CN109886077A (en) * | 2018-12-28 | 2019-06-14 | 北京旷视科技有限公司 | Image recognition method, device, computer equipment and storage medium |
| CN110378420A (en) * | 2019-07-19 | 2019-10-25 | Oppo广东移动通信有限公司 | Image detection method and device and computer readable storage medium |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10332249B2 (en) * | 2016-03-07 | 2019-06-25 | Hyla, Inc. | Screen damage detection for devices |
| JP6936958B2 (en) * | 2017-11-08 | 2021-09-22 | オムロン株式会社 | Data generator, data generation method and data generation program |
| JP7087397B2 (en) * | 2018-01-17 | 2022-06-21 | 東京エレクトロン株式会社 | Substrate defect inspection equipment, substrate defect inspection method and storage medium |
| CN110084124B (en) * | 2019-03-28 | 2021-07-09 | 北京大学 | Feature Enhanced Object Detection Method Based on Feature Pyramid Network |
| CN110148130B (en) * | 2019-05-27 | 2022-02-11 | 北京百度网讯科技有限公司 | Method and device for detecting part defects |
| CN110363104B (en) * | 2019-06-24 | 2022-07-15 | 中国科学技术大学 | A kind of detection method of diesel black smoke vehicle |
| CN110675399A (en) * | 2019-10-28 | 2020-01-10 | 上海悦易网络信息技术有限公司 | Screen appearance flaw detection method and equipment |
| CN110796646A (en) * | 2019-10-28 | 2020-02-14 | 上海悦易网络信息技术有限公司 | Method and device for detecting defects of screen area of electronic device |
| CN110827249A (en) * | 2019-10-28 | 2020-02-21 | 上海悦易网络信息技术有限公司 | Electronic equipment backboard appearance flaw detection method and equipment |
-
2019
- 2019-10-28 CN CN201911033637.XA patent/CN110827249A/en active Pending
-
2020
- 2020-10-14 JP JP2022502080A patent/JP2022539912A/en active Pending
- 2020-10-14 WO PCT/CN2020/120878 patent/WO2021082921A1/en not_active Ceased
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170277979A1 (en) * | 2016-03-22 | 2017-09-28 | Inovision Software Solutions, Inc. | Identifying defect on specular surfaces |
| CN106875381A (en) * | 2017-01-17 | 2017-06-20 | 同济大学 | A kind of phone housing defect inspection method based on deep learning |
| US20190056333A1 (en) * | 2017-08-16 | 2019-02-21 | Siemens Energy, Inc. | Method and system for detecting defects on surface of object |
| CN109711474A (en) * | 2018-12-24 | 2019-05-03 | 中山大学 | An Algorithm for Detection of Surface Defects of Aluminum Materials Based on Deep Learning |
| CN109886077A (en) * | 2018-12-28 | 2019-06-14 | 北京旷视科技有限公司 | Image recognition method, device, computer equipment and storage medium |
| CN109859190A (en) * | 2019-01-31 | 2019-06-07 | 北京工业大学 | A kind of target area detection method based on deep learning |
| CN110378420A (en) * | 2019-07-19 | 2019-10-25 | Oppo广东移动通信有限公司 | Image detection method and device and computer readable storage medium |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US12322259B2 (en) | 2018-12-19 | 2025-06-03 | Ecoatm, Llc | Systems and methods for vending and/or purchasing mobile phones and other electronic devices |
| US11989710B2 (en) | 2018-12-19 | 2024-05-21 | Ecoatm, Llc | Systems and methods for vending and/or purchasing mobile phones and other electronic devices |
| US11843206B2 (en) | 2019-02-12 | 2023-12-12 | Ecoatm, Llc | Connector carrier for electronic device kiosk |
| US12300059B2 (en) | 2019-02-12 | 2025-05-13 | Ecoatm, Llc | Kiosk for evaluating and purchasing used electronic devices |
| US12223684B2 (en) | 2019-02-18 | 2025-02-11 | Ecoatm, Llc | Neural network based physical condition evaluation of electronic devices, and associated systems and methods |
| US11798250B2 (en) | 2019-02-18 | 2023-10-24 | Ecoatm, Llc | Neural network based physical condition evaluation of electronic devices, and associated systems and methods |
| WO2021082918A1 (en) * | 2019-10-28 | 2021-05-06 | 上海悦易网络信息技术有限公司 | Screen appearance defect detection method and device |
| WO2021082921A1 (en) * | 2019-10-28 | 2021-05-06 | 上海悦易网络信息技术有限公司 | Back cover appearance defect detection method for electronic apparatus, and apparatus |
| US12380420B2 (en) | 2019-12-18 | 2025-08-05 | Ecoatm, Llc | Systems and methods for vending and/or purchasing mobile phones and other electronic devices |
| US11922467B2 (en) | 2020-08-17 | 2024-03-05 | ecoATM, Inc. | Evaluating an electronic device using optical character recognition |
| US12271929B2 (en) | 2020-08-17 | 2025-04-08 | Ecoatm Llc | Evaluating an electronic device using a wireless charger |
| US12033454B2 (en) | 2020-08-17 | 2024-07-09 | Ecoatm, Llc | Kiosk for evaluating and purchasing used electronic devices |
| US12321965B2 (en) | 2020-08-25 | 2025-06-03 | Ecoatm, Llc | Evaluating and recycling electronic devices |
| CN112700428A (en) * | 2021-01-08 | 2021-04-23 | 北京网瑞达科技有限公司 | Method and device for identifying backboard element of switch |
| CN115457007A (en) * | 2022-09-23 | 2022-12-09 | 章鱼博士智能技术(上海)有限公司 | A battery explosion-proof film detection method and equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| JP2022539912A (en) | 2022-09-13 |
| WO2021082921A1 (en) | 2021-05-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110827249A (en) | Electronic equipment backboard appearance flaw detection method and equipment | |
| CN110827246A (en) | Electronic equipment frame appearance flaw detection method and equipment | |
| CN110675399A (en) | Screen appearance flaw detection method and equipment | |
| CN110796646A (en) | Method and device for detecting defects of screen area of electronic device | |
| CN110796647A (en) | Method and device for detecting defects of screen area of electronic device | |
| JP6868119B2 (en) | Holographic anti-counterfeit code inspection method and equipment | |
| US9697416B2 (en) | Object detection using cascaded convolutional neural networks | |
| CN110827244A (en) | Method and equipment for detecting appearance flaws of electronic equipment | |
| CN111175318A (en) | Screen scratch fragmentation detection method and equipment | |
| CN110796669A (en) | Vertical frame positioning method and equipment | |
| CN110175609B (en) | Interface element detection method, device and equipment | |
| CN110427859A (en) | A kind of method for detecting human face, device, electronic equipment and storage medium | |
| US8374454B2 (en) | Detection of objects using range information | |
| TW201447775A (en) | Method and system for recognizing information | |
| US8254690B2 (en) | Information processing apparatus, information processing method, and program | |
| CN111210473A (en) | Mobile phone contour positioning method and equipment | |
| CN107292318A (en) | Image significance object detection method based on center dark channel prior information | |
| CN110781823B (en) | Screen recording detection method and device, readable medium and electronic equipment | |
| CN110708568B (en) | Video content mutation detection method and device | |
| CN111401238A (en) | Method and device for detecting character close-up segments in video | |
| US11728914B2 (en) | Detection device, detection method, and program | |
| CN111062377A (en) | A problem number detection method, system, storage medium and electronic device | |
| CN113853243A (en) | Game prop classification and neural network training method and device | |
| CN118314108A (en) | Image difference detection method, image processor test method and image processor test device | |
| CN110827261A (en) | Image quality detection method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 1101-1103, No. 433, Songhu Road, Yangpu District, Shanghai Applicant after: Shanghai wanwansheng Environmental Protection Technology Group Co.,Ltd. Address before: Room 1101-1103, No. 433, Songhu Road, Yangpu District, Shanghai Applicant before: SHANGHAI YUEYI NETWORK INFORMATION TECHNOLOGY Co.,Ltd. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200221 |