KR102789530B1 - Methods of part recognition and parts-feeder using thereof - Google Patents

Abstract

In one embodiment of the present invention, a method for recognizing a part of a part alignment device is proposed. The method comprises the steps of: photographing parts distributed on the part alignment device; deriving information on a target part regarding its location, type, and alignment status from the photographed image using an object derivation algorithm; and labeling each of the derived pieces of information.

Description

{METHODS OF PART RECOGNITION AND PARTS-FEEDER USING THEREOF}

The present invention relates to a component recognition method for recognizing components and a component alignment device using the method.

A technology is needed to align parts in one direction before they are fed into an automated product assembly process in bulk form.

Conventional parts sorters are classified by their center of gravity as parts move along the guideline. However, there was a problem in which parts would get caught in the guideline or other device components as they moved, causing the device to stop operating. To solve this problem, there was a limitation that a worker had to be stationed at all times.

Figure 7 is an image showing a conventional parts aligner, and it is possible to see at a glance the principle by which parts are aligned and how they can become stuck and stop operation.

In addition, since conventional parts sorters, once designed, can only sort parts of a fixed shape, there was a limitation that in order to sort new parts, the design had to be tailored to the part. As a result, existing parts sorters could not be reused, which increased the cost of the product assembly process.

Therefore, there is a need for a component alignment technology that can solve the component jamming problem, reduce process delay time, and be applied to various types of components.

The purpose of the present invention is to solve the above-described problem, and to propose a method for recognizing a part using a vision algorithm and a part alignment device using the same.

However, the problems to be solved by the present invention are not limited to those mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

In order to solve the above-described problem, a method for recognizing a part of a part alignment device proposed in one embodiment of the present invention comprises the steps of: a part alignment device that places a plurality of parts on a part alignment unit including a plate structure, applies vibration to disperse them, and then supplies them to the outside through a robot arm; a step of photographing the parts dispersed on the part alignment unit; a step of deriving information on a target part regarding its location, type, and alignment state from the photographed image using an object derivation algorithm; and a step of labeling each of the derived information.

According to one embodiment, the object derivation algorithm can be applied differently depending on the type and shape of the part, and may utilize one or more of the YOLO (You Only Look Once) algorithm, the color derivation algorithm, and the template matching algorithm.

According to one embodiment, the step of deriving information of the target component may include a process of suggesting to the user a recommended order of the object derivation algorithm according to the type and shape of the component, and allowing the user to determine a priority order of application of the object derivation algorithm.

According to one embodiment, the deriving step may use a plurality of object deriving algorithms, and when a difference in result values between the plurality of object deriving algorithms is greater than a preset threshold value, the step may return to the step of applying vibration to the part alignment unit to redistribute the part, and then photographing the part.

According to one embodiment, the captured image may include images captured at multiple points in time, and the object detection algorithm may detect an object by applying a score vector-based weight to each of the images captured at multiple points in time and adding up the weights.

In one embodiment, the object extraction algorithm may adjust the intensity of light projected onto the back surface of the plate structure and derive information about the alignment state from the captured image.

According to one embodiment, the template matching algorithm uses a template of a pre-registered target part, and the template of the pre-registered target part may be learned by taking a plurality of images at various angles while the target part is stationary on the part alignment unit after being dispersed by vibration, and converting the images based on the base coordinate system of the part alignment unit.

In another embodiment of the present invention, a parts alignment device may include a sensor unit that photographs a target part on a parts alignment unit; a processor unit that derives and labels a target part from an image acquired from the sensor unit; a transmission unit that outputs the labeling data; a camera guideline that adjusts the position and angle of the sensor unit; and an input interface unit that inputs a user command for selecting the type of object derivation algorithm. In addition, the sensor unit may include a camera equipped with an image sensor.

In one embodiment, the sensor unit may move along the camera guideline, the position and angle of which are set to photograph the same area on the parts feeder.

According to one embodiment, the sensor unit may be a plurality of sensors and transmit multiple viewpoint images to the processor unit, and the processor unit may use the component recognition method.

According to one embodiment of the present invention, unlike conventional parts aligners that align parts by pushing them along guidelines, parts can be aligned one by one, thereby reducing the time that the process is delayed due to the jamming phenomenon of parts. Accordingly, it can contribute to increasing the productivity of products by reducing the waste of manpower in responding to the problem of parts jamming.

In addition, since the parts can be aligned simply by changing the software even if the type and shape of the parts change, it has the effect of reducing the cost consumption and production delay time due to equipment changes.

However, the effects of the present invention are not limited to the effects described above, and include all effects naturally implemented due to the various configurations proposed in the present invention.

FIG. 1 is a flowchart sequentially showing each step of a component recognition method of a component alignment device according to one embodiment of the present invention.
FIG. 2 is a structural diagram exemplarily showing one form of a parts alignment device according to one embodiment of the present invention.
FIG. 3 is a structural diagram exemplarily showing one form of a parts aligner of a parts aligning device according to one embodiment of the present invention.
FIG. 4 is a structural diagram exemplarily showing one form in which a split guide part is installed in a component alignment part according to one embodiment of the present invention.
FIG. 5 is a structural diagram exemplarily showing one form of a plurality of cameras and camera guide lines according to one embodiment of the present invention.
FIG. 6 is a drawing exemplarily showing one form of a parts alignment device according to one embodiment of the present invention.
Figure 7 is an image showing a conventional parts aligner.

The embodiments of the present invention are exemplified for the purpose of explaining the technical idea of the present invention. The scope of rights according to the present invention is not limited to the embodiments presented below or the specific description of these embodiments.

All technical and scientific terms used in this invention, unless otherwise defined, have the meaning commonly understood by a person of ordinary skill in the art to which this invention belongs. All terms used in this invention have been selected for the purpose of more clearly explaining this invention and have not been selected to limit the scope of rights according to this invention.

Expressions such as “including,” “comprising,” “having,” etc., used in the present invention should be understood as open-ended terms that imply the possibility of including other embodiments, unless otherwise stated in the phrase or sentence in which the expression is included.

The singular expressions described in the present invention may include plural meanings unless otherwise stated, and the same applies to the singular expressions described in the claims.

Hereinafter, embodiments of the present invention will be described with reference to the attached drawings. In addition, in the description of the embodiments below, duplicate descriptions of identical or corresponding components may be omitted. However, even if the description of a component is omitted, it is not intended that such a component is not included in any embodiment.

The present invention aims to provide a technology for improving product productivity by proposing a parts aligner that can solve the problem of jamming when aligning parts in a product assembly process and can be applied to various parts.

According to an embodiment of the present invention, a vision algorithm can be used to recognize the type and arrangement of parts in a more accurate form. For example, based on the recognized information, a robot arm can select a part that is properly arranged in the correct direction, grab it, move it to a desired point, put it down (pick and place), align it, or assemble it by inserting it into a specific point.

Hereinafter, the component recognition method of the component alignment device of the present invention, the component alignment device using the component recognition method, and the utilization thereof will be specifically described with reference to examples and drawings. However, the present invention is not limited to these examples and drawings.

According to one embodiment, the component alignment device may be a device that places a plurality of components on a component alignment device (100), applies vibrations to disperse and align them, and then supplies them to the outside through a robot arm (300). At this time, the component alignment device may include a component alignment unit including a plate structure.

According to one embodiment, the component recognition method of the component alignment device may include a step (S110) of photographing components (400) distributed in the component alignment unit; a step (S120) of deriving information of the target components regarding the location, type, and alignment state from the photographed image using an object derivation algorithm; and a step (S130) of labeling each of the derived information. The component recognition method may further include a step (S140) of additionally transmitting the labeled information to the outside after labeling. The transmitting may be transmitting the labeled information to a robot arm near the component alignment device.

The above plate structure may include an elastic cylindrical body (150), and the vibration may be a vibration in which a vibration unit directly applies physical vibration to the plate structure. In addition, the vibration may be due to air pressure generated by blowing air to the elastic cylindrical body using an air blower, but is not limited thereto.

The above-mentioned component alignment part may include a split guide part (110). The split guide part may be a means for dividing a space on the upper part of the elastic surface of the component alignment part.

The information of the above target component may include not only information about the appearance or arrangement of the component itself, but also information about whether the split guide part and the component are in contact. In addition, information about whether the edge of the component alignment part and the component are in contact may be included. In this way, if the component is in contact with the split guide part or the component alignment part, the picking operation of the robot arm described above may be difficult. Therefore, the component on which the robot arm performs the moving operation may be selected by considering the information about whether or not there is contact. Through this, the probability of the robot arm's failure to grasp the component can be reduced.

According to one embodiment, the component recognition method of the component alignment device may apply the object derivation algorithm differently depending on the type and shape of the component, and may utilize one or more of the YOLO (You Only Look Once) algorithm, the color derivation algorithm, and the template matching algorithm.

The above YOLO algorithm may be a single-step object derivation algorithm, which may have the characteristic of deriving objects quickly compared to a dual-step object derivation algorithm. In addition, the color derivation algorithm may be an algorithm that defines colors using an HSV color space. In addition, the template matching algorithm may be an algorithm that determines whether a template of a pre-registered target part matches the target part included in the input image. In addition, various vision algorithms that are not listed may be additionally used to recognize the part itself and the arrangement state and to select the part to be moved.

In one embodiment, the YOLO algorithm and the template matching algorithm may include a process of converting the image into a black and white image in advance. This may be implemented for the purpose of reducing the amount of data processing and increasing the work speed. The process may be particularly useful in a process of placing a large number of parts on a parts alignment unit and working on a large amount of data that must be processed simultaneously. In addition, the process of converting to a black and white image may be determined to be applicable depending on the type and shape of the part.

According to one embodiment, in a method for recognizing a part of a part sorting device, the step of deriving information of the target part may include a process of suggesting to the user a recommended order of the object derivation algorithm according to the type and shape of the part, and allowing the user to determine the priority order of application of the object derivation algorithm.

The above recommendation order may vary depending on the type and shape of the part. For example, when the shape of the product is complex and the color difference between parts is not distinct, the YOLO algorithm may be recommended first. Also, when the shape of the part is simple, the template matching algorithm may be recommended first. And when the color difference between parts is distinct, the color derivation algorithm may be recommended first.

The above algorithm does not stop at recommending only the optimal first algorithm, but can additionally recommend the second and third algorithms. At this time, the user can select one or more algorithms. If the user selects multiple algorithms among various recommended algorithms, the data processing may be slightly delayed, but more accurate information on the parts and their alignment status can be obtained. In this way, the process of selecting multiple algorithms at the same time can be used in a process where the post-process is more complex and the pre-process speed of the parts supply can be relatively slow.

The component sorting device may include an input interface. The input interface may include a display screen. The recommendation algorithm may be displayed on the screen of the input interface, and the user may determine the priority application order of the object derivation algorithm by referring to the recommendation order. However, it is not limited to the recommendation order, and the priority application order may be determined according to the user's judgment.

According to one embodiment, the deriving step of the component recognition method of the component alignment device may use a plurality of object deriving algorithms, and if a difference in result values between the plurality of object deriving algorithms is greater than a preset threshold value, the step of applying vibration to the plate structure to redistribute the component, and then returning to the step of photographing the component.

The above result value may be position information including coordinate values, but is not limited thereto. The above coordinate values may be values of vertical coordinates and horizontal coordinates respectively given with the center of the plate structure of the component alignment section as the center.

At this time, if the above coordinate values are derived using a single image obtained from a camera at a single location, an error value may be included in the location of the part due to the distortion of the camera. In this case, the error can be corrected by periodically performing a calibration task using software processing.

However, according to one example of the present invention, the correction of the error can be performed using multiple images taken by the camera at different locations. A method can be taken to check the distorted error value and correct it by synthesizing the image taken at location A and the image taken at location B. In this case, the camera can move along the camera guideline of a pre-set location.

In addition, the above preset threshold value may be input in advance by the user. The above threshold value may vary depending on the type and shape of the part, and may be determined through several experiments before the part alignment device is put into mass production.

If the difference in the above result values is greater than the preset threshold value, it can be determined that the reliability of the object extraction algorithm that uses the above-described captured image as an input image is not high. In order to use an image with high reliability, the plate structure can be vibrated to redistribute the components. An alarm can be sent to the user when the redistribution step is repeated a preset number of times. The user can set the number of times.

In addition, if the difference in the above result values is greater than a preset threshold value, the object extraction algorithm can be executed again to extract a highly reliable object. Even in the same image, objects can be extracted with different probabilities, and since there may be differences in the result values, only the object extraction algorithm can be executed again without a redistribution process.

According to one embodiment, in the component recognition method of the component aligner, the object derivation algorithm may be to adjust the intensity of light projected on the back surface of the plate structure and derive information about the alignment state from the captured image.

The above-described component alignment device may include a lighting unit (not shown) at the bottom of the component alignment unit. The light of the lighting unit may be directed from the bottom to the top, and may pass through the elastic surface of the plate structure to illuminate the component. The error of the object extraction algorithm may increase depending on the brightness of the light. In order to reduce the above-described error, the above-described component alignment device may include a sensor that detects light. The intensity and color of the light of the surrounding environment may be recognized through the sensor, and the lighting unit may be controlled based on the recognized information.

The above lighting unit can illuminate the entire component alignment unit. Also, it can illuminate only the area where the image is captured. Also, it can illuminate only one area among the areas divided by the above division guide unit. For example, when the above division guide unit is illuminated, a shadow may be created by the lighting, which may increase the error of the object extraction algorithm. Therefore, it is possible to illuminate only some areas where the shadow is minimized.

According to one embodiment, in the component recognition method of the component alignment device, the template matching algorithm uses a template of a pre-registered target component, and the pre-registered template of the target component may be learned by taking a plurality of images at various angles while the target component is stationary on the component alignment unit after being dispersed by vibration, and converting the images based on the base coordinate system of the component alignment unit.

Depending on the type and shape of the target part, the position and alignment state at which it stops on the part alignment unit may be different. In addition, depending on the shape of the target part, a groove or a protrusion may be formed on the part alignment unit, which may prevent the target part from rolling or being bounced off the part alignment unit. In order to reflect different characteristics of the stop shape of the target part on the part alignment unit, the template may be learned by photographing the target part in a stop state after being distributed on the part alignment unit.

Additionally, the template of the pre-registered target part may be obtained by learning multiple images of the target part at various angles taken before the target part is supplied to the part alignment unit.

The above image may be an image captured based on the coordinate system of the sensor unit, and the base coordinate system of the component alignment unit may be used as a reference for conversion into the template.

According to one embodiment, the component alignment device includes a sensor unit that photographs a target component on a component feeder; a processor unit that extracts and labels a target component from an image acquired from the sensor unit; a transmission unit that outputs the labeling data; a camera guideline (210) that adjusts the position and angle of the sensor unit; and an input interface unit that inputs a user command for selecting the type of object extraction algorithm; and the sensor unit may include a camera (200) equipped with an image sensor.

The above sensor unit may move along the camera guideline whose position and angle are set to capture the same area on the parts feeder. The camera guideline may have a different shape depending on the range of movement of the robot arm. For example, it may be formed in the shape of a hemisphere so as not to interfere with the movement of the robot arm, but is not limited thereto.

The position of the above sensor part can be readjusted according to the movement of the robot arm after the labeled data is transmitted to the robot arm. This reduces interference by the robot arm when photographing the target part, and shortens the time for extracting the next part to be gripped.

According to one embodiment, the component alignment device may have a plurality of sensor units that transmit multiple viewpoint images to a processor unit, and the processor unit may use the component recognition method.

Multiple viewpoint images can be captured through the above multiple sensor units. Each of the multiple viewpoint images can be analyzed through an object extraction algorithm, and each can be given a weight based on a score vector. An object can be derived by adding up the given weights.

Some of the above-described multiple sensor units may be configured to capture images to be input into an object extraction algorithm, and others may be configured to inspect the quality of a target component. The quality may be inspected using hybrid vision technology and photometric stereo technology. The hybrid vision technology may apply rule-based technology to regular defects, and deep learning technology to irregular defects. In addition, the photometric stereo technology may be applied when inspecting the quality of a product with severe reflection and curvature.

The above description is merely an illustrative description of the technical idea of the present invention, and those skilled in the art will appreciate that various modifications and variations may be made without departing from the essential characteristics of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to explain it, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present invention.

100: Parts Aligner
110: Split guide section
120: Cover
130: Housing
150: Elastic cuboid
200: Camera
210: Camera Guidelines
300: Robot Arm
310: Gripper
400: Parts

Claims (8)

In a component alignment device that places a plurality of components on a component alignment section including a plate structure, applies vibration to disperse them, and then supplies them to the outside,
A step of photographing parts distributed in the above parts alignment section;
A step of deriving information on the target part regarding its location, type, and alignment status using an object derivation algorithm from the above-described captured image;
A step of labeling each of the above derived information;
Including,
The above object derivation algorithm can be applied differently depending on the shape complexity and color difference of the part.
It uses one or more of the YOLO (You Only Once) algorithm, color derivation algorithm, and template matching algorithm.
If the difference in the results between the above multiple object derivation algorithms is greater than a preset threshold value,
The step of applying vibration to the above-mentioned component alignment section to redistribute the components and then returning to the step of photographing the components.
A method for recognizing parts in a parts alignment device.
delete In the first paragraph,
The step of deriving information on the above target part proposes to the user a recommended order of the object derivation algorithm according to the type and shape of the above part,
Including a process in which the user determines the priority of application of the above object derivation algorithm.
A method for recognizing parts in a parts alignment device.
delete In the first paragraph,
The above object extraction algorithm controls the intensity of light projected on the back surface of the plate structure and derives information about the alignment state from the captured image.
A method for recognizing parts in a parts alignment device.
In the first paragraph,
The above template matching algorithm uses a template of a pre-registered target part.
The template of the above pre-registered target part is learned by taking multiple images at various angles while the target part is stationary on the part alignment unit after being dispersed by vibration and converting the images based on the base coordinate system of the part alignment unit.
A method for recognizing parts in a parts alignment device.
A sensor section that photographs a target part on a parts alignment section;
A processor unit that extracts and labels a target part from an image acquired from the sensor unit;
A transmission unit for outputting the above labeling data;
Camera guidelines for adjusting the position and angle of the above sensor portion; and
An input interface section for entering a user command for selecting a type of object derivation algorithm;
The above sensor part includes a camera equipped with an image sensor,
The above processor unit uses the component recognition method of claim 1.
Parts alignment device.
In Article 7,
The above sensor section is a plurality of units and transmits multiple time point images to the processor section.
Parts alignment device.

Images