JPWO2006135040A1 - Image processing apparatus and image processing method for performing three-dimensional measurement - Google Patents

Abstract

  An image processing apparatus that allows a user to easily execute a setting for specifying a target portion of a three-dimensional measurement on an image of a photographed target object, with an optical axis directed in a vertical direction A first camera (C0) that is installed and generates a front-view image of the workpiece W; and a second camera (C1) that is installed with the optical axis directed obliquely and generates a perspective image of the workpiece W. It has an imaging part (1). At the time of setting prior to measurement, an object to be set is photographed by each camera (C0, C1), and a designated area for specifying a measurement target position to the user using a front view image from the first camera C0. Make settings for. At the time of measurement, a specified area is defined in the front view image from the first camera (C0) based on the setting, and processing for specifying the measurement target position in this area is performed. Furthermore, for the oblique image from the second camera (C1), a position corresponding to the position specified in the front view image is specified, and processing for calculating three-dimensional coordinates is performed.

Description

The present invention relates to an apparatus that performs three-dimensional measurement based on images photographed by a plurality of cameras arranged to photograph an object from different directions.
“Measurement” in this specification includes measurement for the purpose of inspection. That is, in general, based on the fact that some measurement is performed in the course of an inspection, the term “measurement” includes a case where the final output is, for example, only a pass / fail of the inspection and no measurement value is output.

  2. Description of the Related Art Conventionally, an apparatus for measuring the shape and surface pattern of an object by two-dimensional image processing has been widely used at various product manufacturing sites. In these two-dimensional image processing apparatuses, a surface representing the object (for example, the bottom surface of the object) or a measurement target surface of the object is photographed from a direction perpendicular to the surface, that is, a direction in which the surface is viewed from the front. Measurement is performed based on the obtained image.

  On the other hand, a three-dimensional measurement technique based on the principle of stereoscopic vision using a plurality of cameras is known. For example, Patent Document 1 describes a printed circuit board inspection apparatus using such a technique. . However, conventional three-dimensional measurement devices limit the types and properties of objects, and are configured as specialized devices for the purpose of printed circuit board inspection, for example. However, it has not been realized as a device having versatility that can be applied to the system and operability that can be used without special knowledge of three-dimensional measurement.

Japanese Published Patent Publication No. 2003-522347

  In such a situation, the inventors develop an image processing apparatus having a three-dimensional measurement function that is as versatile as a conventional two-dimensional image processing apparatus and is easy to use by a user. I made it. Under such a policy, the inventors set the object for specifying the target part of the three-dimensional measurement on the image on which the photographed object appears, with the object of various shapes as the measurement object. It was set as an issue to make it easy to do.

(1) In order to solve the above-described problem, a first image processing apparatus proposed in this specification is a first front-view image obtained based on an image obtained by capturing an object with a first camera. A process using an image and a second image based on an image taken by a second camera arranged to photograph an object from a direction different from the direction taken by the first camera, Setting means for allowing a user to set a designated area using a setting image that is a front view image obtained based on an image obtained by photographing a setting object by a first camera; For the first image, which is a front view image obtained based on an image obtained by photographing the object to be measured by the camera, a designated area is determined based on the setting, and the designated area is defined on the object within the designated area. A position specifying means for specifying a position; and a first A position in the second image corresponding to the position specified in the image is specified, and a three-dimensional coordinate is calculated using the specified position on the first image and the position on the second image 3 A dimension measuring means.

  Here, when the first camera is arranged so as to shoot the object from the direction in which the object is viewed in front, and the image itself captured by the first camera is a front-view image, the first camera performs measurement. An image obtained by photographing the target object may be used as it is as the first image, and further, for example, a process such as moving an image for correcting misalignment is performed on the image taken by the first camera. The applied image may be used as the first image. When the image obtained by photographing the object with the first camera is a perspective image, an image subjected to at least processing for converting the perspective image into a front view image is used as the first image.

  According to such an image processing apparatus, the user only needs to set the designated region for specifying the position on the target object for the front view image. Setting can be performed easily.

(2) The above image processing apparatus further includes conversion means for performing a conversion operation for converting a perspective image captured by a first camera arranged to capture an object from a perspective direction into a front view image. May be provided. The setting image in this case is obtained by the conversion means converting the image captured from the direction in which the first camera squints the setting object, and the first image is the first image. This is obtained by the conversion means converting the image taken from the direction in which the camera squints the object to be measured.

  According to the above configuration, even when the camera is not installed or cannot be installed in the direction of front view (for example, vertically above the object) with respect to the object, the region setting for the front view image can be performed. .

(3) Further, in an embodiment of the image processing apparatus provided with the conversion means, the scale information relating the dimension in the converted front view image and the actual dimension of the measurement target portion is matched with the scale information. The value of the scale reference height that is the height of the front view and the value of the allowable range of the front view height that is determined so as to include the scale reference height can be used. Furthermore, the two-dimensional image processing means for performing the two-dimensional image processing using the scale information for the first image, and the front view height indicated by the three-dimensional coordinates calculated by the three-dimensional measurement means is within the allowable range. Determining means for determining whether or not it is included.

  According to the above aspect, if the calculated three-dimensional coordinates are not within the allowable range of the front view height, the scale indicated by the scale information used by the two-dimensional image processing means and the actual scale of the first image It can be seen that there is a greater difference between them than expected.

(4) In another aspect of the image processing apparatus provided with the conversion unit, in the front view image converted by the conversion unit using the front view height indicated by the three-dimensional coordinates calculated by the three-dimensional measurement unit. Scale information calculating means for calculating scale information relating the dimensions and actual dimensions of the measurement target portion, and two-dimensional image processing means for performing two-dimensional image processing using the scale information for the first image. Further provided.

  According to the above aspect, since the two-dimensional image processing is performed using the scale information calculated from the actual measurement value of the front view height, it is possible to perform more accurate measurement on the target object.

(5) In another aspect of the image processing apparatus provided with the conversion means, the scale information relating the dimensions in the converted front view image and the actual dimensions of the measurement target portion, and the front view height matching the scale information A scale reference height value is available, and adjustment means for changing the scale reference height and the scale information consistently based on an operation by the user is further provided.

According to the above aspect, when the scale reference height is adjusted based on the operation by the user so as to substantially match the actual front view height of the target portion of the three-dimensional measurement, the measurement target is displayed on the second image. The range in which the location may appear is reduced. Therefore, if the position corresponding to the position specified on the first image is specified for such a small range, the correspondence of the measurement target position between the first image and the second image is determined. The possibility of erroneous attachment is reduced, and the time required for the calculation for specifying the corresponding position is also reduced.
In addition, when the scale information is correctly adjusted based on the operation by the user, when various two-dimensional image processes involving measurement of dimensions and areas are applied to the first image, the error included in the result is small. Become.

(6) In the image processing apparatus according to the above aspect (5), an image for editing a display image in which a scale figure indicating an actual dimension on a plane at a scale reference height is added to the first image. Editing means may be provided. In this case, when the edited image is displayed, the user can perform an adjustment operation so that the size relationship between the displayed scale graphic and the image of the measurement target portion is correct.

(7) Second configuration of an image processing apparatus that solves the problem of enabling a user to easily execute a setting for specifying a target portion of a three-dimensional measurement on an image in which a captured target object appears. In this specification, the first image, which is a front view image obtained based on an image captured from the direction in which the first camera squints the object, and a direction different from the direction in which the first camera captures An image processing apparatus that performs processing using a second image based on an image captured by a second camera arranged to capture an object from a direction in which the object is perspectively viewed Conversion means for performing a conversion operation for converting a perspective image captured by the arranged first camera into a front-view image, and the conversion means for converting an image obtained by capturing an object for setting by the first camera. Setting image obtained by On the first image obtained by converting the setting means for allowing the user to set the measurement target position and the image obtained by photographing the target object to be measured by the first camera. , Position specifying means for specifying a position on the object based on the setting, and a position in the second image corresponding to the position specified in the first image, and the specified first image An image processing apparatus including a three-dimensional measuring unit that calculates three-dimensional coordinates using the upper position and the position on the second image is proposed.

  According to such an image processing device, the user only has to make a setting for specifying the position on the object with respect to the front view image even though the image captured by the camera is a perspective image. It is possible to easily perform setting for specifying a target portion for three-dimensional measurement. Furthermore, the setting can be easily performed even when the camera is not installed or cannot be installed in the front view direction with respect to the object.

(8) In an aspect of the image processing apparatus having the second configuration described above, the scale information relating the dimensions in the front view image converted by the conversion means and the actual dimensions of the measurement target portion is matched with the scale information. The value of the scale reference height that is the front view height and the value of the allowable range of the front view height that is determined to include the scale reference height are available, and the first image is targeted. Determining whether the front view height indicated by the three-dimensional coordinates calculated by the two-dimensional image processing means that performs two-dimensional image processing using the scale information is included in the allowable range. Determination means is further provided.

  According to the above aspect, if the calculated three-dimensional coordinates are not within the allowable range of the front view height, the scale indicated by the scale information used by the two-dimensional image processing means and the actual scale of the first image It can be seen that there is a greater difference between them than expected.

(9) In another aspect of the image processing apparatus of the second configuration, the dimensions in the front view image converted by the conversion means using the front view height indicated by the three-dimensional coordinates calculated by the three-dimensional measurement means, Scale information calculating means for calculating scale information relating the actual dimensions of the measurement target portion, and two-dimensional image processing means for performing two-dimensional image processing using the scale information for the first image. Provided.

  According to the above aspect, since the two-dimensional image processing is performed using the scale information calculated from the actual measurement value of the front view height, it is possible to perform more accurate measurement on the target object.

(10) In another aspect of the image processing apparatus having the second configuration, the scale information relating the dimensions in the front view image converted by the conversion means and the actual dimensions of the measurement target portion, and the front matching the scale information The scale reference height value, which is the visual height, can be used, and adjustment means for changing the scale reference height and the scale information consistently based on an operation by the user is further provided.

In the above aspect, when the scale reference height is adjusted so as to substantially coincide with the actual front view height of the target portion of the three-dimensional measurement based on the operation by the user, the measurement target portion on the second image The range in which can appear is reduced. Therefore, if the position corresponding to the position specified on the first image is specified for such a small range, the correspondence of the measurement target position between the first image and the second image is determined. The possibility of erroneous attachment is reduced, and the time required for the calculation for specifying the corresponding position is also reduced.
In addition, when the scale information is correctly adjusted based on the operation by the user, when various two-dimensional image processes involving measurement of dimensions and areas are applied to the first image, the error included in the result is small. Become.

(11) In the image processing apparatus according to the above aspect (10), an image for editing a display image in which a scale figure indicating an actual dimension on a plane at a scale reference height is added to the first image. Editing means may be further provided. In this way, when the edited image is displayed, the user can perform an adjustment operation so that the relationship between the displayed scale figure and the image of the measurement target portion is correct.

(12) In the first image processing method proposed in this specification, a first image which is a front view image obtained based on an image obtained by photographing a target by the first camera, and a first camera A second image based on an image photographed by a second camera arranged to photograph an object from a direction different from the direction in which the first camera is used for setting. A setting step for displaying a setting image that is a front-view image obtained based on an image obtained by photographing the target object, and allowing the user to set a designated area using the setting image; For the first image, which is a front view image obtained based on an image obtained by photographing the object to be measured by the camera, a designated area is determined based on the setting, and the designated area is defined on the object within the designated area. Positioning step for identifying the position The position in the second image corresponding to the position specified in the first image is specified, and the three-dimensional coordinates are determined using the specified position on the first image and the position on the second image. And a three-dimensional measurement step for calculating.

(13) The second image processing method proposed in this specification is a front view obtained based on an image obtained by photographing a target by a first camera arranged so as to photograph the target from a perspective direction. A first image, which is an image, and a second image based on an image taken by a second camera arranged to photograph an object from a direction different from the direction taken by the first camera are used. In this process, the image obtained by photographing the setting object by the first camera is converted into the setting image by a conversion operation for converting the perspective image photographed by the first camera into the front view image. The setting step for setting the measurement target position using the setting image, and the conversion calculation, the first camera converts an image obtained by photographing the measurement target object into the first image. Based on the above settings on the image A position specifying step for specifying a position on the object, a position in the second image corresponding to the position specified in the first image, a position on the first image specified and a second position specified And a three-dimensional measurement step for calculating a three-dimensional coordinate using the position on the image.

  According to the first and second image processing methods, the user can easily perform setting for specifying the target portion of the three-dimensional measurement on the image on which the photographed target object appears.

It is a perspective view which shows the structure of the imaging part of the test | inspection apparatus with which this invention was applied with the example of installation. It is explanatory drawing which shows the example of the image imaged with each camera. It is a block diagram of an inspection device. It is a flowchart which shows the procedure concerning a test | inspection of IC. It is explanatory drawing which shows the example of a setting of a detection area. It is a flowchart which shows the procedure of a teaching process. It is explanatory drawing which shows the example of a setting of a positioning area | region. It is a flowchart which shows the detailed procedure of a lead test | inspection. It is explanatory drawing which shows the relationship between the positioning area | region with respect to the workpiece | work in the front view image at the time of a test | inspection, and a detection area. It is explanatory drawing which shows the relationship of the corresponding point between each image. It is explanatory drawing which shows the setting method of a search area | region. It is explanatory drawing which shows the relationship between the range of height and a search area | region. It is explanatory drawing which shows the example of model registration with respect to the test | inspection site | part on a workpiece | work. It is a flowchart which shows the procedure concerning inspection of a character key. It is explanatory drawing which shows the example which set the area | region and search area | region to be examined with respect to the image at the time of a test | inspection. It is explanatory drawing which shows the front view image and perspective view concerning a workpiece | work which have a circular display area. It is explanatory drawing which shows the designation | designated result of the display area and measurement object area | region with respect to a quality work. It is a flowchart which shows the procedure in the case of performing the workpiece | work height inspection of FIG. It is explanatory drawing which shows the example in which a measurement object area | region and a search area follow the position change of a workpiece | work. It is explanatory drawing which showed the example which sets the virtual camera for front view. It is explanatory drawing which shows the method for converting a perspective image into a front view image. It is a flowchart which shows the procedure of the setting performed prior to a test | inspection. It is a flowchart which shows an example of the procedure of the test | inspection accompanied by the conversion process to a front view image. It is a flowchart which shows an example of the procedure of the test | inspection accompanied by the conversion process to a front view image. It is a flowchart which shows an example of the procedure of the test | inspection accompanied by the conversion process to a front view image. It is a flowchart which shows the procedure of the setting including the change of scale reference height and scale information. It is explanatory drawing which shows the imaging | photography condition of the workpiece | work by a perspective camera. It is explanatory drawing which shows the content of the window displayed on a monitor. It is explanatory drawing which shows the change of the display content of the edited image accompanying the change of scale reference | standard height.

FIG. 1 shows a configuration of an imaging unit of an inspection apparatus (image processing apparatus) to which the present invention is applied, along with an installation example thereof.
This inspection apparatus has both three-dimensional and two-dimensional measurement processing functions, and an inspection object W (hereinafter referred to as “work W”) transported along a factory inspection line L is captured by the imaging unit 1. Imaging is performed in order, and measurement processing and discrimination processing according to various inspection purposes are executed.
The imaging unit 1 has a configuration in which two cameras C0 and C1 are incorporated in a housing 15, and is installed above the inspection line L. One camera C0 is installed with its optical axis oriented in the vertical direction (front view with respect to the workpiece W). The other camera C1 is installed with the optical axis tilted so that the field of view overlaps the camera C0. The imaging plane that defines the field of view of the camera C0 and the camera C1 is rectangular, and the camera C1 corresponds to the camera C0 in the lateral direction of the field of view of the camera C0 (the x-axis direction of the image A0 in FIG. 2). .) Are arranged in line.

  FIG. 2 shows an example of an image of the workpiece W generated by each camera C0, C1. In the figure, A0 is an image from the camera C0, and A1 is an image from the camera C1. In the figure illustrating the image of the work W in FIG. 2 and FIG. 5 and the like which will be described later, the work in each of the images A0 and A1 is also shown using W as in FIG. The horizontal direction (horizontal direction) of the image A0 is the x-axis direction, and the vertical direction (vertical direction) is the y-axis direction.

  Since the camera C0 is installed with the optical axis oriented in the vertical direction, the image A0 shows a state where the upper surface of the workpiece W is viewed from the front. On the other hand, since the camera C1 is installed with the optical axis directed obliquely, the work W in a state seen from the oblique appears in the image A1. Hereinafter, the image A0 from the camera C0 is referred to as “front-view image A0”, and the image A1 from the camera C1 is referred to as “perspective image A1”. The front-view image A0 corresponds to a “first image”, and the perspective image A1 corresponds to a “second image”. In this inspection apparatus, first, a measurement target position is specified using a front-view image A0 with a small image distortion (close to the plan view of the workpiece W), and then a measurement target on the front-view image A0 on the perspective image A1. The position corresponding to the position is specified.

  FIG. 3 is a block diagram showing the overall configuration of the inspection apparatus. In addition to the imaging unit 1, the inspection apparatus includes a main body unit 2, a monitor 3, a console 4 and the like. The main body 2 is provided with image input units 10 and 11 for the cameras C0 and C1, a camera drive unit 12, an arithmetic processing unit 20, an output unit 28, and the like.

  The camera drive unit 12 receives the detection signal from a workpiece detection sensor (not shown) and drives the cameras C0 and C1 simultaneously. The image signals generated by the cameras C0 and C1 are input to the respective image input units 10 and 11 and digitally converted. Thereby, digital images for measurement processing (front-view image A0 and perspective image A1) are generated for each camera.

  The arithmetic processing unit 20 is configured by a computer, and after performing measurement processing using the images of the cameras C0 and C1, the suitability of the work W is determined from the processing result. The output unit 28 is an output interface for outputting the results of measurement processing and determination processing to an external device such as a PLC.

In addition to the image memory 21 for storing the images A0 and A1, the arithmetic processing unit 20 includes an image processing unit 22, a measurement processing unit 23, a determination unit 24, a display control unit 25, a parameter calculation unit 26, and a parameter storage unit 27. Etc. are provided. Each unit other than the image memory 21 and the parameter storage unit 27 is a function set in the computer as the arithmetic processing unit 20 by a dedicated program. The image memory 21 and the parameter storage unit 27 are set in the memory (RAM, etc.) of this computer.
Although not shown in FIG. 3, the arithmetic processing unit 20 is also provided with a memory for registering information necessary for inspection (inspection area setting conditions, model images, and the like). Setting or changing the registration processing in the registration memory and the processing executed by each processing unit of the arithmetic processing unit 20 can be appropriately performed according to the operation of the console 4.

  The image processing unit 22 identifies the inspection target part of the workpiece W by binarization, edge extraction, pattern matching, and the like. The measurement processing unit 23 executes processing for measuring the position, size, and the like of the inspection target part specified by the image processing unit 22. The image processing unit 22 and the measurement processing unit 23 can execute processing of two-dimensional measurement and three-dimensional measurement.

The determination unit 24 determines the quality of the workpiece W by comparing the measurement result of the measurement processing unit 23 with a predetermined threshold value. The measurement results and determination results are output to the output unit 28 and the display control unit 25.
The display control unit 25 controls the display operation of the monitor 3 and can display the front view image A0 and the perspective image A1 generated by the image input units 10 and 11 in parallel on one screen. Furthermore, the processing results of the image processing unit 22, the measurement processing unit 23, and the determination unit 24 can be appropriately received and displayed together with the image.

The parameter storage unit 27 stores various coefficients used for calculation for three-dimensional measurement. The values of these coefficients are the relationship between the stereo coordinate system constituted by the cameras C0 and C1 and the spatial coordinate system representing the position in the actual space (the distance between the origins of each coordinate system, the stereo coordinate system for the spatial coordinate system). (Hereinafter, these coefficients are referred to as “parameters”). These parameters are calculated by the image processing unit 22 and the parameter calculation unit 26 and stored in the parameter storage unit 27 prior to the inspection. In the process of calculating this parameter, a calibration work having a plurality of feature points is used.
Further, the parameter storage unit 27 also registers parameters constituting a homography matrix of the arithmetic expression (1) described later.

  This inspection apparatus can assemble an inspection algorithm by presenting a plurality of types of inspection menus to a user and accepting a selection operation. In addition, an inspection by a two-dimensional measurement process and an inspection by a three-dimensional measurement process can be selected and executed according to the region to be inspected. In the inspection by the two-dimensional measurement processing, pattern matching processing, binarization processing, edge extraction processing, etc. are executed on the front-view image A0 from the camera C0, and the entire workpiece or the inspection target portion in the workpiece is specified. .

  Further, this inspection apparatus speeds up the three-dimensional measurement process by effectively using the front-view image A0 from the camera C0 for the inspection by the three-dimensional measurement process. This point will be described in detail later.

  FIG. 4 shows an inspection procedure performed when the workpiece W is an IC. This procedure is started upon receiving a detection signal from a workpiece detection sensor. In the first ST1 (ST is an abbreviation of “step”. The same applies to the following), the cameras C0 and C1 are simultaneously driven by the camera driving unit 12 to generate the front view image A0 and the perspective image A1.

  In the next ST2, an inspection for characters printed on the package portion of the IC is executed. In this inspection, two-dimensional image processing using only the front-view image A0 is executed. For example, a character print region is extracted by pattern matching processing, and the suitability of the character print state is determined from the degree of correlation at the time of matching and the matching position.

  In the next ST3, the coordinates of the tip position of each lead in the front-view image A0 are obtained by an edge detection method, and then the coordinates of the tip position of each corresponding lead in the perspective image A1 are obtained by an edge detection method. Then, the three-dimensional coordinates of the leading end of each lead are obtained from the coordinates of the leading end position of each lead in both images, and it is determined from the calculated value whether there is any abnormality such as floating or bending in each lead.

  When the inspections of ST2 and 3 are completed, the result of each inspection is output to an external device or the monitor 3 in ST4. Further, when the next IC is detected by the workpiece detection sensor, the process returns to ST1 and the same procedure as described above is executed.

  In this way, by picking up an image of the workpiece W once by the two cameras C0 and C1, the inspection by the two-dimensional measurement and the inspection by the three-dimensional measurement can be executed continuously. Since the front-view image A0 is used in the two-dimensional measurement process, it is possible to perform a highly accurate measurement process using an image with no character distortion.

  When performing the three-dimensional measurement process, the corresponding measurement target points are specified between the front-view image A0 and the perspective image A1, and the coordinates of the specified points are applied to an arithmetic expression based on the principle of triangulation. Thus, the three-dimensional coordinates are calculated.

  FIG. 5 shows a setting example of the detection area (designated area set by the user) in the front view image A0 and the detection area in the perspective image A1. The vertical direction of the image (the direction along the lead arrangement in the front-view image A0) is the y-direction, and the horizontal direction (the direction along the lead length in the front-view image A0) is the x-direction. In the front-view image A0, an individual detection area 7 is set for each lead 6, and one edge point corresponding to the tip of the lead is specified for each detection area 7.

  That is, here, since the cameras C0 and C1 are arranged in the x direction, parallax mainly occurs in the x direction. Therefore, by binarizing the image in the detection area 7 and projecting the obtained binary image along the y direction, the number of projected “bright” pixels or “dark” pixels is plotted with the x direction as the horizontal axis. Make a histogram with the vertical axis. Then, the x coordinate of the portion where the value of the histogram changes suddenly is set as the x coordinate of the tip of the lead 6. On the other hand, the y coordinate of the midpoint of the detection region 7 in the y direction is assigned to the y coordinate of the tip of the lead 6. The points indicated by the x-coordinate and y-coordinate thus obtained are called edge points.

  Here, the image in the detection area 7 is binarized. However, the present invention is not limited to this. The density of each pixel is integrated in the y direction while the image in the detection area 7 is a grayscale image, and the integration thus obtained is integrated. You may make it obtain | require x coordinate of the location (for example, location which crosses a threshold value) where the value of density distribution changes suddenly along a x direction.

  Thus, the position of the edge is detected along one specific direction. In the example of FIG. 5, the x direction is the edge detection direction. Here, after the detection area 7 is set in the front-view image A0, the edge detection direction is designated for the detection area 7. However, the present invention is not limited to this, and the detection area 7 having the inherent edge detection direction may be set. Alternatively, the edge detection direction of the detection area 7 may be specified first, and then detected in the front-view image A0. The area 7 may be set.

  Even in the perspective image A1, a detection region 8 is set for each lead 6. These detection areas 8 are specified in each detection area 7 of the front-view image A0 based on an arithmetic expression (formula (1) described later) for converting one point on one image into one point on the other image. The coordinates are set using the coordinates of the edge point and the height range specified by the user (the range that the height of the target portion of the three-dimensional measurement can take). The height here is the height in the vertical direction with respect to the mounting surface of the workpiece W, that is, the height in the front view direction, and is also referred to as the front view height. The height reference is not limited to the surface on which the workpiece W is placed, and can be taken at the position of the camera C0 or any other position. The height range specified by the user is a target range for three-dimensional measurement along the optical axis of the camera C0.

  Although FIG. 5 shows only the area setting for the right lead of the workpiece W, the same setting is performed for the left lead (the same applies to the following drawings).

FIG. 6 shows the procedure of teaching processing (setting processing) for IC lead inspection. This procedure is performed before starting the IC inspection of FIG.
In ST11, which is the first step in this procedure, the length of the lead 6, the pitch between the leads 6, and the like are input for the workpiece W to be inspected (IC in this example). The data input here is registered in the working memory and used in ST15 described later.

  In the next ST12, a non-defective work is set as an object for setting at the imaging target position, and this is imaged by the cameras C0 and C1. In the teaching process, it is sufficient if the front-view image A0 from the camera C0 is generated. However, here, the cameras C0 and C1 are simultaneously driven during the teaching process, and the generated two images are displayed on the monitor 3. They are displayed side by side.

In the next ST13, a positioning region designation operation is accepted using the front-view image A0 as the setting image. FIG. 7 shows a display example of the front-view image A0 at the time of this designation operation, and 9 in the figure is a positioning area.
This positioning region 9 is used to extract the end lead (in the illustrated example, the top end lead 6a, hereinafter referred to as the “lead lead 6a”) among the leads 6 arranged in a line. It is done. In the example of FIG. 7, a square-shaped region 9 that includes only the top lead 6a is set. The size of the positioning area 9 is adjusted so that the leading lead 6a can be imaged in the positioning area 9 even if the position of the workpiece W is shifted as much as expected. The positioning area 9 is set so that the top lead 6a is imaged in the lower half of the positioning area 9. As a result, it can be confirmed that the top lead 6 a is imaged in the lower half of the positioning area 9 because the lead is not imaged in the upper half of the positioning area 9.

  Returning to FIG. 6, when the positioning area 9 is designated, the head lead 6a is extracted from the positioning area 9 in the next ST14. In this extraction process, for example, by binarizing the image in the positioning area 9 and projecting the binarized image along the y-axis direction and the x-axis direction, the x-coordinate and y of the tip of the leading lead 6a are projected. Find the coordinates. Alternatively, the contour line of the lead 6a may be extracted by extracting the edge in the positioning region 9 and its concentration gradient direction, and the x and y coordinates of the tip of the lead 6a may be obtained.

  In ST15, the detection area 7 is set in each lead 6 based on the x-coordinate and y-coordinate of the tip of the lead 6a and the data input in ST11. Specifically, the length of the lead 6 and the pitch between the leads 6 on the image are calculated using the data input in ST11, the number of pixels of the camera C0, the magnification, and the like, and each detection region is based on the calculated value. The size of 7 and the interval between regions are determined. In this way, data necessary for setting the detection area 7 for each lead 6 including the lead 6a, that is, setting conditions, is created based on the position of the lead 6a.

  The reason why such a method can be taken is that the front-view image A0 that directly reflects the characteristics of the inspection target part of the workpiece W (the length of each lead is equal, the pitch between leads is equal, etc.) is used as it is. It is. Therefore, if the tip lead 6a can be extracted in the positioning region 9 set by the user, the detection regions 7 can be set for all the leads 6 without extracting other leads 6, and the processing efficiency is greatly increased. Can be improved.

  In ST16, the setting conditions (position and size of the positioning area) specified in ST13 and the setting conditions of the detection area 7 set in ST15 are registered in the registration memory. In ST17, the image in the positioning area 9 is registered as a model in the registration memory. As a result, a series of teaching processes is completed.

  A series of processing from ST11 to ST17 is performed by using the setting image, which is a front view image obtained based on an image obtained by photographing the setting object by the first camera, to the user about the designated area. This corresponds to setting (more generally speaking, allowing the user to set the measurement target position). In addition, the arithmetic processing unit 20 in which a teaching processing program operates serves as setting means for executing this series of processing.

  Before executing ST16, an image showing the detection area 7 set according to the created setting condition superimposed on the front-view image A0 is displayed on the monitor 3, and registration is performed in accordance with the user's confirmation operation. Is desirable. At this time, the position and size of the detection region 7 may be finely adjusted.

FIG. 8 shows a detailed procedure for IC lead inspection (ST3 in FIG. 4).
The processing from ST21 to ST24 of this procedure is performed on the front view image A0 that is an image obtained by photographing the measurement target object. First, in ST21, the positioning area 9 is set in the front view image A0 based on the setting conditions registered by teaching. In the next ST22, the image in the positioning area 9 is compared with the model registered in ST17 of the teaching process, and a deviation amount with respect to the model is extracted (for example, a pattern matching technique can be applied to this process). it can.).

  In ST23, the setting condition of the detection area 7 registered at teaching is adjusted based on the deviation amount extracted in ST22, and the detection area 7 of each lead is set as the designated area by the setting condition after the adjustment. According to the front-view image A0, since it is not necessary to consider the distortion of the workpiece W on the image, the shift amount of the positioning region 9 can be applied to each detection region 7 as it is, and the teaching process is performed on each lead 6. The detection area 7 can be set with the same positional relationship as the time.

FIG. 9 shows an example of a front-view image A0 at the time of inspection. Since the workpiece W in this example is shifted to the right side from the image A0 at the time of teaching shown in FIG. 7, the tip of the lead 6 is in a state of protruding from the positioning region 9. However, since the adjustment process described above is performed for the detection area 7, the detection area 7 is set to any lead 6 under the same conditions as shown in FIG. 4.
In this example, the position in the image of the detection area 7 (position with respect to the frame of the image) is adjusted in accordance with the displaced workpiece W. However, instead of this, even if the workpiece W is displaced, The entire contents of the image may be moved so as to always have a fixed positional relationship with respect to the image frame, and the detection area 7 may always be set at a fixed position with respect to the image frame.

When the detection area 7 is set for each lead in this way, in the next ST24, the x and y coordinates of the tip of the lead are obtained for each detection area 7.
A series of processes from ST21 to ST24 is performed on the first image, which is a front view image obtained based on an image obtained by photographing the object to be measured by the first camera, based on the setting. And specifying the position on the object in the designated area (more generally, specifying the position on the object based on the setting). In addition, the arithmetic processing unit 20 in which a program combined to execute the procedure for the lead inspection process operates as a position specifying unit that executes this series of processes.

  In the next ST25, a detection area 8 for detecting the tip position of each lead is set on the perspective image A1. In ST26, the same processing as ST24 is executed in the set detection area 8, and the x and y coordinates of the tip of the lead are calculated. The process of setting the detection area 8 and calculating the x and y coordinates corresponds to specifying the position in the second image corresponding to the position specified in the first image.

Thereafter, in ST27, three-dimensional coordinates are calculated for each tip using the coordinates calculated in ST24 and 26, respectively. This process corresponds to calculating a three-dimensional coordinate using the specified position on the first image and the position on the second image. An arithmetic processing unit 20 in which a program combined to execute a procedure for specifying a position in the second image and calculating three-dimensional coordinates operates as a three-dimensional measuring unit that executes this series of processing. .
In ST28, the quality of each lead tip is determined by comparing the calculated three-dimensional coordinates with a reference value registered in advance. For example, if there is a float at the tip of any lead, the Z coordinate representing the height of that tip exceeds the reference value, and the lead is determined to be defective.

Next, the setting of the detection area 8 in ST25 will be described in detail.
FIG. 10 shows a state in which a point P on the plane D at an arbitrary height in the space is imaged at points p0 and p1 on the imaging surfaces F0 and F1 of the cameras C0 and C1, respectively. In FIG. 10, X, Y, and Z are coordinate axes representing a three-dimensional space, and the plane D is parallel to the XY plane. A two-dimensional coordinate system with axes x0 and y0 is set on the imaging surface F0, and a two-dimensional coordinate system with axes x1 and y1 is set on the imaging surface F1. In FIG. 10, the point on the plane D that is imaged at the origin of both imaging surfaces is assumed to be P, but this is not limiting, and the position of the point P is arbitrary on the plane D.

The imaging position of the point P on the imaging plane F0 coordinates _{(x cam0, y cam0)} of (point p0), the imaging position of the point P on the imaging plane F1 coordinates (point _{p 1) (x cam1, y} cam1) Then, the relationship between the points p ₀ and p ₁ is expressed by the following equation (1).

  In Equation (1), HZ is a 3 × 3 homography that represents the relationship between the imaging position on the imaging surface F0 and the imaging position on the imaging surface F1 for a point on the plane D having a height Z. Is a matrix and λ is a constant. The matrix HZ can be obtained in advance by calibration using known coordinates on the plane D (see Non-Patent Document 1 below for details of calibration).

Nobuhiro Miichi, Toshikazu Wada, Takashi Matsuyama “Calibration of Projector-Camera System”, [Searched on June 1, 2005], Internet <URL: http: // vision. kuee.Kyoto-u.ac.jp/Research/Thesis/Thesis_PDF/Miichi_2002_P_147.pdf>

Therefore, when thinking edge points of the lead extracted in each detection region 7 of the front view image A0 and the point _{p 0,} was substituted for the coordinates (1) of the _{(x cam0, y cam0),} it was calculated (X _cam1 , y _cam1 ) can be considered to correspond to the position of the lead tip in the perspective image A1. However, assuming that the height of the lead tip changes, the height Z of the plane D also changes accordingly, the homography matrix HZ changes accordingly, and the values of (x _cam1 , y _cam1 ) also change. become.

In ST25, based on this principle, when the upper limit value of the assumed height range (target range of three-dimensional measurement along the optical axis of the camera C0) is the height Z of the plane D, the lower limit value is the height. By executing the equation (1) using the homography matrix HZ corresponding to the height Z for each of Z and Z, the two points e and f shown in FIG. 11 as (x _cam1 , y _cam1 ) Get the coordinates. Then, in the perspective image A1, as shown in FIG. 11, the line segment gh obtained by translating the line segment ef to each side in the direction perpendicular to the line segment ef by the half-value width k of the detection region 7 on the front view image A0 side. A line segment g′h ′ is set, and a rectangular area ghh′g ′ connecting these four points is set as a detection area 8.

  FIG. 12 shows the size of the detection area 8 when the height range that the lead 6 can take is 0 to 5 mm and when the height range is −15 to 15 mm for the perspective image A1 similar to FIG. 5. This is shown in contrast. As is clear from this example, the detection region 8 becomes smaller as the fluctuation range of the height range is smaller. In FIG. 12, the detection areas 8 are simplified and drawn in parallel with each other. However, in reality, the perspective image A1 is distorted so that a rectangular object is imaged in a trapezoid shape due to the effect of perspective. Each detection region 8 has a non-parallel arrangement such that the distance between the center lines (line segments ef) increases toward the right in the drawing. In each detection area 8, the binarized image for obtaining the edge tip position is projected in a direction perpendicular to the center line of the detection area 8.

  In the inspection apparatus described above, the detection region 7 corresponding to each lead 6 is set in order to specify the lead tip position in the reference image A0, but instead, the y-axis direction includes the lead tip. One long detection area may be set, and the tip position of each lead may be obtained individually in this area.

  Next, a case will be described in which the height of each key is inspected for a work (remote control, telephone, etc.) provided with push button type character keys. In this inspection, a method is used in which a model image is registered at the preparation stage of the inspection, and a region matching the model image is searched for in the front-view image A0 and the perspective image A1.

  In the preparation stage for inspection, as shown in FIG. 13, each key 60 is drawn on the key 60 using a front-view image A0 obtained by imaging a non-defective work W that is a setting object. An area 70 including the designated character is designated, and an image in the area 70 is registered as a model. In this process, the setting of the measurement target position is performed by using the setting image which is a front view image obtained based on an image obtained by photographing the setting object with the first camera. Equivalent to. In other words, the model image is only registered here, and the measurement target position is not directly specified. However, at the time of inspection, the region that matches the model image is set as the measurement target region, so the model image is registered. As a result, the measurement target position is indirectly set. Although illustration is omitted, it is also possible to designate an area in which an area that matches the model image is to be searched at the time of inspection in consideration of an assumed positional deviation amount of the workpiece W at the time of inspection. This area to be searched corresponds to the designated area.

  FIG. 14 shows an inspection procedure. First, in ST31, the cameras C0 and C1 are simultaneously driven to generate an image. In the next ST32, pattern matching processing using a model registered before the inspection is executed for the front-view image A0, and an area that most closely matches the model is specified as a measurement target area. The search for the measurement target area is performed only in the designated area when the designated area is set. The measurement target area specifying process is performed for each model corresponding to each key 60, but here, in order to simplify the description, only one model will be described.

When the measurement target region is specified, in the next ST33, the coordinates of the representative point of this region (for example, the center point of the region) are specified. This is equivalent to specifying the position on the object based on the setting performed using the setting image. A plurality of representative points can be specified (for example, points corresponding to a plurality of feature points on a predetermined model).
In ST34, a search area is set on the perspective image A1 based on the coordinates of the representative point. Also in this case, as in the case where the detection region 8 is set by the lead inspection described above, the homography matrix HZ is set with the upper limit value and the lower limit value of the height range specified in advance as the height Z, and By executing the calculation twice using the upper limit value and the lower limit value of the coordinates and the height range, a range where the representative point can exist on the image A1 is obtained, and an area in which the size of the model is added to the range is obtained. The search area.

  In ST35, pattern matching processing with the model is executed in the search area, and the measurement target area and the position of the representative point in the area are specified. Furthermore, in ST36, three-dimensional coordinates are calculated using the coordinates of the representative points of the measurement target region in the frontal and perspective images A0 and A1. In ST37, the Z coordinate of the calculated three-dimensional coordinates is compared with a predetermined threshold value to determine whether the key height is appropriate. In ST38, the determination result is output and the process is terminated.

  When the point corresponding to the feature point determined in the model is specified as the representative point on the reference image A0, the corresponding point on the model can be specified in ST36 as well. In ST35, the model should be imaged by the perspective camera C1 using a homography matrix corresponding to a predetermined height within a specified height range (for example, a standard height when the workpiece is normal). The measurement target region may be specified using the converted model. Conversely, the perspective image A1 may be converted into a front view image, and an area that matches the model may be specified on the converted image.

  FIG. 15 shows the measurement target area 71 specified for the key 60 on the front-view image A0 in the above inspection, the search area 80 on the perspective image A1 side set based on the position and size of the area 71, and A measurement target area 81 specified in the search area 80 is shown.

  In the procedure shown in FIG. 14, only the inspection by the three-dimensional measurement process is performed, but also in this inspection, the print state of the character of each key is inspected by the two-dimensional measurement process using the front-view image A0. be able to.

Next, a case where the work having a circular display area at the center is inspected for appropriateness of the height in the display area will be described. In this inspection, a method of extracting a part of the front-view image A0 as a model image and searching for a portion that matches the model image in the perspective image A1 is used every time shooting is performed.
FIG. 16 shows images A0 and A1 of the workpiece W generated by the cameras C0 and C1. In the figure, S is a display (printing) area of characters to be inspected. In the front view image A0, since the front image of the workpiece W appears, the outline 72 of the display area S is also circular. On the other hand, in the perspective image A1, the shape of the outline 72 of the display area S, the arrangement state of the characters in the display area S, and the like are distorted.

  Prior to this inspection, a non-defective model of the workpiece W, which is a setting object, is imaged, and the user is allowed to specify the radius of the display area S and the measurement target area on the obtained front-view image A0. At this time, the image processing unit 22 executes processing for obtaining the position of the center point of the display area S from the front-view image A0, and registers the relative position of the measurement target area with respect to the center point in the registration memory. In this process, the setting of the measurement target position is performed by using the setting image which is a front view image obtained based on an image obtained by photographing the setting object with the first camera. Equivalent to.

  FIG. 17 shows the display area S in the front view image A0 in an enlarged manner. In the figure, reference numeral 73 denotes a measurement target area designated by the user, and 74 denotes a center point of the display area S. The position of the center point 74 executes a process of extracting a circular pattern from the front-view image A0, and a contour that best matches a circle (position correction model) having a size designated by the user from the extraction result. The line 72 is specified and obtained.

FIG. 18 shows a height inspection procedure for the workpiece W of FIG.
In ST41, the cameras C0 and C1 are simultaneously driven to generate the front view image A0 and the perspective image A1. In ST42, the position of the center point 74 of the display area S is obtained from the front view image A0 by the above processing.

  In the next ST43, the measurement target region 73 is set based on the previously registered relative position with reference to the coordinates of the center point 74 obtained in ST42. In ST44, the image of the measurement target area 73 is registered as a model image for search, and the representative point position of the measurement target area 73 (for example, the center point position in the area) is also registered. This is equivalent to specifying the position on the object based on the setting performed using the setting image.

  In ST45, a search area 82 (shown in FIG. 19) is set on the perspective image A1. For the position and size of the search area 82, the coordinates of the representative points registered in ST44 are substituted into the equation (1), and the homography matrix HZ corresponding to the upper limit value and lower limit value of the height range specified in advance is used. Then, it is determined by executing equation (1).

  In ST46, the correlation matching process is executed in the search area 82 using the model image registered in ST44. Then, an area most similar to the registered image is specified, and this is set as a measurement target area on the perspective image A1 side. In ST47, the coordinates of the representative point are obtained for the measurement target region on the perspective image A1 side, and the three-dimensional coordinates are calculated using the coordinates and the coordinates of the representative point on the front view image A0 side. In subsequent ST48, the suitability of the obtained Z coordinate is determined. Then, the determination result in ST49 is output, and then the process is terminated.

  FIGS. 19 (1) and 19 (2) show the measurement target region 73 in the front view image A0, the search region 82 in the perspective image A1, and the measurement target region 83, respectively, for one set of the front view image A0 and the perspective image A1. Further, in the front view image A0, the outline 72 of the display area S is shown with a bold line, and the position where the center point 74 of the display area S is obtained is shown. The image of (1) and the image of (2) in FIG. 19 are different in the position of the workpiece W, but in both cases, the measurement target area 73 is set correctly so that the target character is contained therein. Yes.

According to the inspection procedure shown in FIG. 18, the center point 74 extraction process and the region 73 position adjustment process are performed using the front-view image A0. The measurement target region 73 can be set to an appropriate position based on the relative positional relationship with respect to the point 74. Also in the example of FIG. 19, the position of the workpiece W is different between the image of the example of (1) and the image of the example of (2), but the measurement target region 73 is set under the same conditions.
Furthermore, according to the inspection procedure shown in FIG. 18, a model image for search is acquired from each work W to be inspected, so that the surface pattern such as characters to be subjected to three-dimensional measurement is changed to the work W. Even when different, it is possible to perform three-dimensional measurement by applying the same procedure. Here, although the surface pattern may be different for each workpiece W, the shape of the workpiece W and the position of the measurement target area based on the shape of the workpiece W are common to all the workpieces W. An area is set.

Also in the procedure of FIG. 18, the registered model image for search is converted into a shape that should be imaged by the perspective camera C1 using the homography matrix Hz, and the measurement target region 83 is specified using the converted image. You may do it. Conversely, the perspective image A1 may be converted into a front view image, and the measurement target region 83 matching the model may be specified on the converted image.
Furthermore, the process of inspecting the printing state of the characters in the display area S using the front-view image A0 can also be incorporated into the procedure of FIG.

  By the way, in the inspection apparatus described above, one camera C0 is installed with the optical axis in the vertical direction in order to generate a front view image. However, even when the optical axis of the camera C0 is set obliquely, A front view image can be generated by converting the image generated in C0.

FIG. 20 shows a calibration method for performing the above conversion processing. In this method, the calibration work 75 is placed on a plane having an arbitrary height parallel to the placement surface of the work W, and the cameras C0 and C1 are arranged so as to capture the calibration work 75 from obliquely above.
In this example, a planar work having a configuration in which a plurality of circular patterns 76 are arranged at equal intervals on the upper surface is used as the calibration work 75.

  In the calibration process, a virtual camera C2 installed with the optical axis oriented in the vertical direction is assumed, and a virtual image obtained when the calibration work 75 is imaged by the camera C2 is assumed. Then, as shown in FIG. 21, a homography matrix for converting an image A0 generated by the camera C0 into an image B0 by the virtual camera C2 is obtained. For this purpose, first, center coordinates of each circular pattern are obtained on the images A0 and B0. For the image A0, coordinates (such as barycentric coordinates) of the center of the circular pattern distorted in an elliptical shape from the actual image are obtained. For the image B0, an arbitrary imaging magnification is set in the virtual camera C2, and the actual interval of the circular pattern on the calibration work 75 is converted into the interval d on the image using the magnification set in the virtual camera C2. Thus, the coordinates of the center of the circular pattern on the image are calculated. Then, based on the arrangement order of the circular patterns, the combination of the center positions of the corresponding circles between the images A0 and B0 is specified, and the homography matrix can be obtained by the least square method using the coordinates of the centers of these circles. it can.

  According to the camera arrangement in FIG. 20, the height of the imaging target portion in the optical axis direction of the virtual camera C <b> 2 that performs a front view with respect to the workpiece W is the “front view height”. The front view height is expressed with reference to an arbitrary plane (for example, a placement surface of the workpiece W) orthogonal to the optical axis direction of the camera C2. The reference plane is generally a horizontal plane, but is not limited thereto, and a plane or a vertical plane inclined with respect to the horizontal plane may be used as the reference plane.

  When the homography matrix is determined by the above calibration processing, the front view in which the scale has been determined by the conversion processing using the image A0 from the camera C0 for the plane at the front view height equivalent to the calibration work 75 is obtained. An image can be generated.

  However, even when a plane that is parallel to the calibration work 75 but has a front view height different from that of the calibration work 75 is to be imaged, a front view image can be obtained by the same conversion calculation. However, when the plane to be imaged is positioned closer to the camera C2 than the calibration work 75 as compared with the front view image when the plane at the same front view height as the calibration work 75 is converted, the enlarged image is displayed. Converted into a front-view image. Further, when the imaging target plane is located farther than the calibration work 75 when viewed from the camera C2, the imaging target plane is reduced as compared with the case where the imaging target plane is at the same front view height as the calibration work 75. Converted to a front view image.

  Therefore, if the front view height of the imaging target plane is known, the degree of enlargement / reduction of the image when converted into the front view image can be obtained by calculation. Therefore, it is possible to measure the dimensions of the imaging target plane whose height of front view is known on the converted front view image.

  Hereinafter, an inspection apparatus in which the cameras C0 and C1 are both arranged in a perspective view as shown in FIG. 20 will be described. The entire block configuration is the same as that shown in FIG. 3, but the computer as the arithmetic processing unit 20 has a function (conversion means) for converting a perspective image generated by the camera C0 into a front view image. ) Is set. Factors that determine the size of a front-view image obtained by the conversion (hereinafter referred to as a front-view converted image) when displayed on the screen of the monitor 3 include a virtual imaging magnification and a display magnification. Also, the front view conversion image for the setting work W corresponds to the setting image, and the front view conversion image for the measurement target work W corresponds to the first image.

The virtual imaging magnification is a ratio between the actual distance between two points on the workpiece W at the same front view height and the distance between the two points imaged on the virtual imaging plane of the virtual camera C2. Yes, it is represented by the distance on the virtual imaging plane when the actual distance is 1. The virtual imaging magnification changes depending on the front view height of the imaging target so that the imaging target decreases as the imaging target moves away from the virtual camera C2.
The virtual imaging magnification can be changed by adjusting the parameters for the front view conversion calculation on the assumption that the focal length of the virtual camera C2 is changed, but here, the focal length of the virtual camera C2 is fixed. Suppose there is.

  The display magnification is a ratio between the distance between two points on the virtual imaging plane and the distance between the two points displayed on the monitor. The display magnification can be changed by performing an operation for enlarging or reducing the image. The display magnification does not affect the scale used for measurement, and does not affect the ratio between the size of the scale figure, which will be described later, and the size of the displayed front view conversion image, so that the image can be easily observed on the display screen. Choose an appropriate value. For example, when the display magnification of the front-view conversion image is selected so that the size of the image of the workpiece W does not change much between the perspective image captured by the camera C0 and the front-view conversion image, one of these images is changed to the other. It is easy to grasp the image content when the display content is switched to.

As described above, the virtual imaging magnification used at the time of calibration can be applied to a plane at the same front view height as the calibration work 75 as it is.
If the front view height is specified, the corresponding virtual imaging magnification can be obtained by calculation. If the virtual imaging magnification is known, it is possible to correctly measure the dimensions of the measurement target portion at the front view height using the front view conversion image.
The front view height, which is the premise for this dimension measurement and virtual imaging magnification, is called the scale reference height. That is, it can be said that the scale reference height is the front view height of the measurement target portion that is assumed when the actual size of the measurement target portion is obtained from the dimensions in the front view conversion image.

  Information relating the dimensions in the front view conversion image and the actual dimensions of the measurement target portion is referred to as scale information. For example, an actual dimension corresponding to one pixel of the front view conversion image can be set as the scale information. The scale reference height and the scale information must be consistent with each other, and if one is changed, the other must be changed.

FIG. 22 shows the procedure of the setting process performed prior to the inspection. First, in ST51, the setting work W (target object for setting) is photographed by the camera C0. Next, in ST52, the photographed perspective image is converted into a front view image. The arithmetic processing unit 20 in which the conversion calculation program operates functions as a conversion means. The same applies to conversion from a perspective image to a front-view image in an inspection described later.
The front view conversion image obtained by the conversion process is used as a setting image.

  In ST53, scale reference height and scale information are set. Specifically, the user's input is received for the height in front of the measurement target part of the setting work, and the input value is set as the scale reference height. Further, the scale information is calculated from the scale reference height, and the scale reference height and the scale information are stored in the arithmetic processing unit 20.

  In ST54, the setting image is used to cause the user to make settings necessary for the measurement process. Specific examples of setting contents are the same as those performed for the lead inspection, the character key inspection, and the workpiece height inspection when the camera C0 is arranged in front view. In addition, it is also set what processing is performed and in what order at the time of measurement.

  In the above ST53, for example, a dimension representing the height of the measurement target location with respect to the placement surface of the workpiece W is input as the scale reference height. The values of the front view height and the scale reference height do not necessarily have to be expressed with reference to the mounting surface of the workpiece W in the inside of the apparatus. For example, the values are expressed as Z coordinate values in the coordinate system of FIG. May be. Or you may express by the other arbitrary coordinate systems which can mutually be coordinate-transformed.

  However, it is preferable that the scale reference height input by the user is a height that the user can naturally recognize as the height of the measurement target portion. Even if the user does not understand the details of the internal processing of the device, the input of the scale reference height is required even if the user does not understand the details of the internal processing of the device by setting the height of the measurement target location with respect to the mounting surface of the workpiece W as the reference height. It is easy to understand which dimensions are being measured.

  However, in some cases, the step of ST53 may be omitted, and some predetermined value such as the height of the calibration work 75 viewed from the front, for example, may be used as the scale reference height. In general, the value of the virtual imaging magnification is often small. In such a case, the error of the scale is relatively small even if the front view height of the measurement target portion is different from the scale reference height. Therefore, when high-precision measurement of dimensions and area is not required, for example, when performing two-dimensional image processing for the purpose of determining the presence / absence of dirt, chipping detection in the outer contour, character type determination, etc. In many cases, there is no problem even if the default value is used as the scale reference height without inputting the scale reference height each time depending on the difference in the front view height. Similarly, when a measurement target position for three-dimensional measurement is specified, it is often possible to use the default value as the scale reference height without any trouble.

FIG. 23 shows an example of an inspection procedure executed after the above setting process is completed.
First, in ST61, the work (object to be measured) W is photographed by the cameras C0 and C1 in accordance with a detection signal from the work detection sensor. In ST62, the first image is obtained by converting the perspective image captured by the camera C0 into a front-view image. In ST63, the position to be subjected to the three-dimensional measurement is specified on the first image. An example of this position specifying method is the same as described above for various inspections when the camera C0 is in the front view arrangement. In ST64, a position corresponding to the previously specified position on the first image is specified on the second image taken by the camera C1.

  In ST65, three-dimensional coordinates are calculated using the specified position on the first image and the position on the second image. In ST66, two-dimensional measurement is performed using the first image. Since the first image is a front-view converted image, various two-dimensional image processing techniques developed on the premise that the front-view image is a processing target can be applied. When performing two-dimensional measurement with measurement of a dimension or an area, the scale information set in the setting process of FIG. 22 is used. At this time, the arithmetic processing unit 20 in which the program for two-dimensional measurement operates functions as a two-dimensional image processing means. When the measurement of ST66 ends, in ST67, the quality determination process of the measurement results obtained in ST65 and 66 is executed. After this, the process returns to ST61 and waits for the next workpiece W to come.

  ST63 to ST66 in FIG. 23 correspond to steps for measurement processing, but there are various settings for the processing contents and order of these steps. For example, three-dimensional measurement may be performed for a plurality of points. In that case, it is conceivable to specify a plurality of positions in each of ST63 and ST64 and calculate the three-dimensional coordinates of the plurality of points in ST65, or to specify the position and calculate the three-dimensional coordinates for one point. By repeating the steps from ST63 to ST65 a plurality of times, the three-dimensional coordinates may be calculated one place at a time for each repetition. Any number of types of two-dimensional measurement in ST66 can be set, and the timing may be set to execute the two-dimensional measurement at any timing after ST62.

  FIG. 24 shows another example of the inspection procedure. When performing this inspection, it is assumed that the allowable range of the front view height is set by the user and stored in the arithmetic processing unit 20 in ST54 of FIG. The allowable range of the height in front view is determined so that the scale reference height is included in the range. The permissible range of the front view height is determined from the viewpoint that if the front view height of the measurement target location is within that range, the scale error of the front view conversion image falls within the expected range. It is different from the criterion for W quality.

  ST61 to ST67 have the same contents as the processes with the same reference numerals in FIG. The process of ST71 determines whether or not the front view height indicated by the three-dimensional coordinates calculated in ST65 is included in the set allowable range. An arithmetic processing unit 20 in which a program for performing this determination operates serves as a determination unit.

  If the calculated three-dimensional coordinates are not within the allowable range of the front view height, the process returns to ST61 through the notification process of ST72, and if within the allowable range, the process returns to ST61 without performing the notification process. When the process proceeds to ST72, it is notified that there is a difference larger than an assumed level between the scale indicated by the scale information used in the two-dimensional measurement and the actual scale of the first image. For example, a display to that effect or a warning sound is generated. As another example of the notification, a display indicating that an error caused by processing based on the first image having a scale error is included in the measurement value may be added to the display of the two-dimensional measurement result. Good.

  FIG. 25 shows still another example of the inspection procedure. The processing contents of ST61 to ST65 and ST67 are the same as the processing of the same reference numerals in FIG. In ST81, the scale information of the first image is calculated using the front view height indicated by the three-dimensional coordinates obtained in ST65 as the scale reference height. The arithmetic processing unit 20 in which the program for calculating the scale information operates works as scale information calculation means. In ST82, two-dimensional measurement is performed on the first image using the calculated scale information.

  Next, FIG. 26 is a modification of the procedure of the setting process shown in FIG. 22 so that the user can adjust the set value while confirming whether the scale reference height and the scale information are set appropriately. In this procedure, first, a setting work W is photographed by the camera C0 in ST91. Next, in ST92, the photographed perspective image is converted into a front view image. The obtained front view conversion image is used as a setting image. At this time, the monitor 3 displays a setting window 90 (shown in FIG. 28) including the front view converted image and the original image before conversion.

  Although details will be described later, at this stage, the front view height of the calibration work 75 input at the time of calibration is set as the initial value of the scale reference height. In the next ST94, the user is caused to perform an operation of changing the value of the scale reference height using the setting window 90, and the scale reference height and the scale information are changed according to the operation. This change operation is repeatedly executed until the user determines that the scale reference height is appropriate. When the change process is completed, the process proceeds to ST95, and the setting necessary for the measurement process is made to be performed by the user using the setting image, as in ST54 of FIG.

  FIG. 27 shows the shooting situation of the setting work W by the camera C0 during the above setting process (the camera C1 is also provided, but not shown). It is assumed that the setting workpiece W is a non-defective product selected from the workpieces W to be measured. The work W here has a cubic shape as a whole. Reference symbol T denotes the upper surface of the workpiece W. FIG. 27 shows virtual planes R0, R1, R2, and R3. R0 is a placement surface of the workpiece W, for example, the surface of the belt conveyor on which the workpiece W is placed. The planes R1, R2, and R3 are parallel to R0, and the heights from R0 are h1, h2, and h3 (h1 <h2 <h3), respectively, and the height h2 of the plane R2 is the height of the upper surface T of the workpiece W. Match. The calibration is performed by placing the calibration work 75 at the height of the plane R1, and the height h1 of the plane R1 is set as the initial value of the scale reference height.

  FIG. 28 shows the contents of the setting window 90 displayed on the monitor 3 in the imaging state of FIG. The window 90 includes an image 91 of the setting work W photographed by the camera C0, an edited image 92, a scale reference height adjustment bar 93 and an adjustment handle 94, a scale reference height numerical value display section 95, a scale reference height. A size confirmation button 96, a scale graphic dimension input unit 97, and a scale graphic type selection unit 98 are displayed. In addition to the front-view converted image of the work W, a scale graphic 99 is also displayed in the edited image 92. The arithmetic processing unit 20 in which a program for creating the contents of the edited image 92 operates functions as an image editing unit.

  In the image 91 photographed by the camera C0, the upper surface T of the work W is displayed in a trapezoidal shape, but in the edited image 92, it is converted into an original square and displayed. However, while the initial value of the scale reference height is set to h1 which is the front view height of the calibration work 75, the actual front view height of the upper surface T is h2, so that it is converted and displayed. There is an error in the scale of the upper surface T. That is, the upper surface T is displayed larger than when it is at the scale reference height.

  The scale figure 99 is a figure showing the actual dimensions on the plane at the scale reference height on the display screen of the monitor 3 based on the virtual imaging magnification at the scale reference height. As the type of the scale graphic 99, the selection unit 98 can select any one of square, circle, and grid. The dimension input to the dimension input unit 97 means the length of one side when the scale figure 99 is a square, the diameter when it is a circle, and the interval between grid lines when it is a grid. The scale figure 99 can be moved to an arbitrary position in the edited image 92 by a drag operation so that it can be easily compared with the front view converted image of the workpiece W.

  Here, a figure that is easy to compare with the shape of the upper surface T of the work W is selected as the scale image 99 (here, a square having the same shape as the upper surface T is selected), and the size of the upper surface T of the work W is input to the dimension input unit 97. When a value suitable for comparison is input (here, the same dimension as the length of the side of the upper surface T is input), the actual height of the upper surface T (measurement target location) of the workpiece W is higher than the scale reference height. If it is close to the virtual camera C2, the size of the upper surface T recognized in contrast to the scale figure 99 is larger than the actual size, and vice versa.

  Therefore, if the size of the measurement target location of the setting work W is known, a scale image 99 that is easy to compare with the shape of the workpiece W observed on the monitor 3 is selected, and the dimensions thereof are known to the measurement target location. When the ratio is set to a value suitable for comparison with the size, if the ratio between the size of the measurement target portion and the size of the scale figure 99 in the front view conversion image of the workpiece W is correct, the scale reference height at that time Corresponds to the front view height of the measurement target portion of the workpiece W (in this case, the upper surface T).

  When ST93 is executed for the first time in FIG. 26, after the initial value of the scale information is calculated from the initial value of the scale reference height, the scale figure 99 selected by the user is input to the dimension input unit 97 with the dimensions and A size corresponding to the initial value of the scale information is displayed on the edited image 92, and the user adjusts the scale reference height. Further, each time adjustment is performed, scale information is calculated from the adjusted scale reference height, and the size of the scale graphic 99 is changed based on the calculation result.

  The scale reference height can be adjusted by dragging the adjustment handle 94 along the adjustment bar 93. The adjustment reference height at the present time is displayed on the numerical display section 95 for the scale reference height. When the adjustment of the scale reference height is completed, the process proceeds from ST94 to ST95 in FIG. 26 by pressing the confirm button 96. At this time, the scale information is also determined. In this way, the scale reference height and the scale information are changed consistently. An arithmetic processing unit 20 in which a program for performing this series of processes operates functions as an adjusting unit.

  FIG. 29 shows a change in the display content of the edited image 92 shown in FIG. 28 accompanying a change in the scale reference height. The edited image 92a is a case where the scale reference height is the initial value h1. Now, it is assumed that the upper surface T is a square with a side of 100 mm, and a square with a side of 100 mm is displayed as the scale figure 99. The scale figure 99 displayed in this case is assumed to have a square with a side of 100 mm at the scale reference height (in this case, height h1). Corresponds to the figure that should be displayed when converted. Since the actual upper surface T is at the height of h2, it is displayed larger than the scale figure 99.

  When the scale reference height is adjusted to h2, as shown in the edited image 92b, an edited image in which the size of the upper surface T matches the size of the scale figure is obtained. Further, when the scale reference height is increased to h3, the scale figure 99 is displayed larger than the front view conversion image on the upper surface T as shown in the edited image 92c.

  With the above operation, the user can determine that the set value matches the front view height of the upper surface T when the scale reference height is set to h2. When the scale figure 99 is superimposed on the front view conversion image on the upper surface T by the drag operation, the size can be compared more accurately.

  In this way, the user adjusts the scale reference height until the ratio between the measurement target portion in the edited image and the scale image 99 is in a correct state, whereby the scale reference height is set to the front view height of the measurement target portion. Can be adjusted correctly. Thereby, scale information can also be set correctly.

  According to the above display, the user can directly confirm from the display of the edited image 92 that the front view height of the measurement target portion of the setting work W and the scale reference height are approximately the same. If the error about the coincidence in height is small, the corresponding position can be found even if the corresponding position search range in the height direction on the second image is small. In addition, when the error about the coincidence of the height is small, the error of the measurement result is small when two-dimensional measurement including dimension measurement and area measurement is performed using the first image.

  As described above, in the above example, the scale reference height is adjusted to match the front view height of the measurement target portion while viewing the edited image 92, but the user does not necessarily understand the concept of the scale reference height. Since it is not necessary, it is not always necessary to indicate to the user that the adjustment target is the scale reference height. For example, it may appear to the user to adjust the scale of the front view conversion image, and the reference height may actually be adjusted. Further, the scale adjustment processing of the front-view converted image may actually be performed, and the corresponding scale reference height may be calculated from the scale adjustment result.

  For example, instead of directly adjusting the scale reference height, the scale graphic 99 and the workpiece W are adjusted so that the ratio between the size of the measurement target portion and the size of the scale graphic 99 in the front view conversion image of the workpiece W is correct. Enlarge or reduce one of the front-view converted images, or enlarge or reduce both at different ratios, obtain scale information from these enlargement / reduction ratios, and calculate the corresponding scale reference height Also good.

  The method of finally obtaining the scale reference height while referring to the ratio between the size of the measurement target portion in the front-view converted image of the workpiece W and the size of the scale figure 99 is the front of the measurement target portion of the workpiece W. This can be used when the scale reference height is set while viewing the image displayed on the monitor 3 without knowing the visual height.

  By the way, the method of obtaining the other from one of the scale reference height and the scale information is not limited to the case where the front view image converted from the perspective image is a target for two-dimensional measurement, and the camera C0 is viewed from the front as shown in FIG. The present invention can also be applied to a case in which a front-view image obtained by arranging and photographing so as to perform two-dimensional measurement is a target for two-dimensional measurement. In this case as well, the advantage of ease of setting can be obtained.

  In addition, the scale information value is appropriate by comparing the front view height indicated by the result of the three-dimensional measurement with the scale reference height when the first camera is in the perspective view or the front view placement. An assessment can be made as to whether or not Thus, performing three-dimensional measurement contributes to verifying the accuracy of two-dimensional measurement.

  Needless to say, the method for determining whether or not the front view height indicated by the calculated three-dimensional coordinates is included in the allowable range described with reference to FIG. 24, the calculation described with reference to FIG. 25. The method for calculating the scale information using the front view height indicated by the three-dimensional coordinates, and the method for adjusting the scale reference height based on the operation by the user described with reference to FIGS. This can also be applied to the case where the camera C0 is arranged in the front view direction.

  As the scale reference height used when calculating the scale information from the scale reference height, a value input by the user may be used, or the front view height actually measured for the workpiece W by the apparatus itself may be used. It may be used. This front view height may be measured by a three-dimensional measurement function using the first and second cameras. However, a sensor for front view height measurement is provided and measured by this sensor. Also good. As a sensor for measuring the height of the front view, various known sensors such as a laser displacement meter of a triangulation system based on projection of a laser beam and reception of reflected light, a probe contact type displacement meter, and the like may be used. it can.

Based on the above disclosure, the image processing apparatus described below is also recognized.
(A) A first image that is a front view image obtained based on an image obtained by photographing the object by the first camera and a direction that is different from the direction in which the first camera is photographed. An image processing apparatus that performs processing using a second image based on an image captured by a second camera arranged,
Setting means for allowing a user to set a measurement target position using a setting image that is a front-view image obtained based on an image obtained by photographing an object for setting by a first camera;
Position specifying means for specifying a position on the object based on the setting with respect to the first image which is a front view image obtained based on an image obtained by photographing the object to be measured by the first camera. When,
A position in the second image corresponding to the position specified in the first image is specified, and a three-dimensional coordinate is determined using the specified position on the first image and the position on the second image. A three-dimensional measuring means for calculating,
Scale information that associates the dimensions in the front view image with the actual dimensions of the measurement target location and the scale reference height that is the front view height that matches the scale information are available.
Furthermore, two-dimensional image processing means for performing two-dimensional image processing on the first image using scale information;
An image processing apparatus.

(B) Furthermore, the value of the allowable range of the front view height determined to include the scale reference height is made available,
(A) The image processing apparatus according to (A), further comprising: a determination unit that determines whether or not a front view height indicated by the three-dimensional coordinates calculated by the three-dimensional measurement unit is included in the allowable range.

(C) The image processing apparatus according to (A), further comprising scale information calculation means for calculating scale information using a front view height indicated by the three-dimensional coordinates calculated by the three-dimensional measurement means.

(D) The image processing apparatus according to (A), further including an adjustment unit that changes the scale reference height and the scale information consistently based on an operation by the user.

(E) The image processing apparatus according to (D), further comprising image editing means for editing a display image obtained by adding a scale figure indicating an actual dimension on a plane at a scale reference height to the first image. .

The front view image used by the image processing apparatus of (A) may be an image taken with a front-view camera, or may be a front view-converted image taken with a perspective camera. According to this image processing apparatus, scale information and scale reference height that match each other can be used, and two-dimensional image processing using scale information is performed on an object to be measured, and three-dimensional measurement is also performed. . Therefore, when the difference between the front view height obtained by the three-dimensional measurement and the scale reference height is large, it can be seen that an error occurs in the result of the two-dimensional image processing using the scale information.
In addition, if the front view height of the measurement target location is set as the scale reference height and the scale information is calculated from the scale reference height, the type of the target object is changed and the height of the measurement target location changes. Even in this case, scale information can be set easily. The front view height for determining the scale information may be specified by the user or measured by the image processing apparatus itself.

  According to the image processing apparatus of (B), when the calculated three-dimensional coordinates are not within the allowable range of the front view height, the scale indicated by the scale information used by the two-dimensional image processing means, and the first image It can be seen that there is a larger difference from the expected scale than the actual scale.

  According to the image processing apparatus of (C), two-dimensional image processing is performed using the scale information calculated from the actual measurement value of the front view height, so that more accurate measurement can be performed on the object.

According to the image processing apparatus of (D), when the scale reference height is adjusted so as to substantially match the actual front view height of the target portion of the three-dimensional measurement based on the operation by the user, the second image The range in which the measurement target portion may appear is reduced. Therefore, if the position corresponding to the position specified on the first image is specified for such a small range, the correspondence of the measurement target position between the first image and the second image is determined. The possibility of erroneous attachment is reduced, and the time required for the calculation for specifying the corresponding position is also reduced.
In addition, when the scale information is correctly adjusted based on the operation by the user, when various two-dimensional image processes involving measurement of dimensions and areas are applied to the first image, the error included in the result is small. Become.

According to the image processing apparatus of (E), when the edited image is displayed, the user performs an adjustment operation so that the size relationship between the displayed scale graphic and the image of the measurement target portion is correct. Can do.

Claims (13)

The first camera is arranged so as to photograph the object from a direction different from the direction in which the first camera captures the first image obtained based on the image obtained by capturing the object. An image processing apparatus that performs processing using a second image based on an image captured by a second camera,
Setting means for allowing a user to set a specified area using a setting image that is a front-view image obtained based on an image obtained by photographing an object for setting by a first camera;
For the first image which is a front view image obtained based on an image obtained by photographing the target object to be measured by the first camera, a designated area is determined based on the setting, and the target area is within the designated area. Position specifying means for specifying a position on an object;
A position in the second image corresponding to the position specified in the first image is specified, and a three-dimensional coordinate is determined using the specified position on the first image and the position on the second image. Three-dimensional measuring means for calculating;
An image processing apparatus. The image processing apparatus further includes conversion means for performing a conversion operation for converting a perspective image captured by the first camera arranged to capture an object from a perspective direction into a front view image,
The setting image is obtained by the conversion means converting the image taken from the direction in which the first camera squints the setting object,
The image processing apparatus according to claim 1, wherein the first image is obtained by the conversion unit converting an image captured from a direction in which the first camera squints an object to be measured. Scale information relating the dimensions in the front view image converted by the conversion means and the actual dimensions of the measurement target location, the value of the scale reference height that is the height in front view that matches the scale information, and the scale reference height And the allowable value of the front view height determined to include
Two-dimensional image processing means for performing two-dimensional image processing on the first image using scale information;
The image processing apparatus according to claim 2, further comprising a determination unit that determines whether or not a front view height indicated by the three-dimensional coordinates calculated by the three-dimensional measurement unit is included in the allowable range. Using the front view height indicated by the three-dimensional coordinates calculated by the three-dimensional measurement means, the scale information relating the dimensions in the front view image converted by the conversion means and the actual dimensions of the measurement target portion is calculated. Scale information calculation means for
The image processing apparatus according to claim 2, further comprising: a two-dimensional image processing unit that performs two-dimensional image processing on the first image using scale information. Scale information relating the dimensions in the front view image converted by the conversion means and the actual dimensions of the measurement target portion, and the value of the scale reference height that is the height in front view that matches the scale information can be used. Has been
The image processing apparatus according to claim 2, further comprising an adjusting unit that consistently changes the scale reference height and the scale information based on an operation by a user.   6. The image processing apparatus according to claim 5, further comprising image editing means for editing an image for display in which a scale graphic indicating an actual dimension on a plane at a scale reference height is added to the first image. . The first image, which is a front view image obtained based on the image captured from the direction in which the first camera illuminates the object, and the object is captured from a direction different from the direction in which the first camera images. An image processing apparatus that performs processing using a second image based on an image captured by a second camera disposed in
Conversion means for performing a conversion operation for converting a perspective image captured by a first camera arranged to capture an object from a perspective direction into a front view image;
Setting means for allowing the user to set the measurement target position using the setting image obtained by the conversion means converting the image obtained by photographing the setting object by the first camera;
Position specifying means for specifying a position on the object based on the setting on the first image obtained by the conversion means converting an image obtained by photographing the object to be measured by the first camera; ,
A position in the second image corresponding to the position specified in the first image is specified, and a three-dimensional coordinate is determined using the specified position on the first image and the position on the second image. Three-dimensional measuring means for calculating;
An image processing apparatus. Scale information relating the dimensions in the front view image converted by the conversion means and the actual dimensions of the measurement target location, the value of the scale reference height that is the height in front view that matches the scale information, and the scale reference height And the allowable value of the front view height determined to include
Two-dimensional image processing means for performing two-dimensional image processing on the first image using scale information;
The image processing apparatus according to claim 7, further comprising: a determination unit that determines whether or not a front view height indicated by the three-dimensional coordinates calculated by the three-dimensional measurement unit is included in the allowable range. Using the front view height indicated by the three-dimensional coordinates calculated by the three-dimensional measurement means, the scale information relating the dimensions in the front view image converted by the conversion means and the actual dimensions of the measurement target portion is calculated. Scale information calculation means for
The image processing apparatus according to claim 7, further comprising two-dimensional image processing means for performing two-dimensional image processing on the first image using scale information. Scale information relating the dimensions in the front view image converted by the conversion means and the actual dimensions of the measurement target portion, and the value of the scale reference height that is the height in front view that matches the scale information can be used. Has been
The image processing apparatus according to claim 7, further comprising an adjusting unit that consistently changes the scale reference height and the scale information based on an operation by a user.   The image processing apparatus according to claim 10, further comprising an image editing unit that edits a display image in which a scale graphic indicating an actual dimension on a plane at a scale reference height is added to the first image. . The first camera is arranged so as to photograph the object from a direction different from the direction in which the first camera captures the first image obtained based on the image obtained by capturing the object. An image processing method for performing processing using a second image based on an image captured by a second camera,
The first camera displays a setting image that is a front-view image obtained based on an image obtained by photographing the setting object, and allows the user to set the designated area using the setting image. Configuration steps;
For the first image which is a front view image obtained based on an image obtained by photographing the target object to be measured by the first camera, a designated area is determined based on the setting, and the target area is within the designated area. A position identifying step for identifying a position on an object;
A position in the second image corresponding to the position specified in the first image is specified, and a three-dimensional coordinate is determined using the specified position on the first image and the position on the second image. A three-dimensional measurement step to calculate;
An image processing method comprising: A first image, which is a front view image obtained based on an image obtained by photographing a target object by a first camera arranged to shoot the target object from a perspective direction, and a direction in which the first camera takes a picture. An image processing method for performing processing using a second image based on an image captured by a second camera arranged to capture an object from a different direction,
By converting the perspective image captured by the first camera into a front-view image, an image obtained by capturing the setting object by the first camera is converted into a setting image, and the setting image is used by the user. Setting step for setting the measurement target position,
A position specifying step of converting an image obtained by photographing the object to be measured by the first camera into a first image by the conversion calculation and specifying a position on the object based on the setting on the first image. When,
A position in the second image corresponding to the position specified in the first image is specified, and a three-dimensional coordinate is determined using the specified position on the first image and the position on the second image. A three-dimensional measurement step to calculate;
An image processing method comprising:

Images