JP4961860B2 - Robot apparatus and control method of robot apparatus - Google Patents

Description

  The present invention relates to a robot apparatus that performs an operation according to an external environment such as object gripping, and a control method thereof.

There have been proposed robotic devices that perform actions according to the external environment, such as robotic devices that recognize objects existing in the work environment and perform gripping operations, and robotic devices that move autonomously in the work environment. Yes. For example, in Patent Document 1, three-dimensional data of a work environment is acquired using a stereo camera, the position and orientation of a gripping target object existing in the environment are recognized based on the acquired three-dimensional data, and a gripping operation for this is performed. Is disclosed.
Japanese Patent Application Laid-Open No. 2004-1122

  A conventionally proposed robot apparatus that performs an operation according to an external environment has an image of an object included in the three-dimensional data of the work environment acquired by a visual sensor such as a stereo camera included in the robot apparatus. There is a problem that it is not possible to identify whether the image is a part of the image or an image of an external object existing in the work environment.

  As an example of a robot apparatus in which the influence of this problem becomes significant, a robot apparatus that performs a gripping operation of a gripping target object by an arm while avoiding an obstacle existing in a work environment is assumed. When the robot device calculates the movement path of the arm relative to the object to be grasped based on the three-dimensional data of the external environment, if the image of a part of the body of the robot device is included in the three-dimensional data, Then, even though the arm part can be moved linearly with respect to the object to be grasped, a path that avoids a part of the body that has been recognized as an obstacle is generated. It may not be possible to execute. Further, if the arm part to be grasped is recognized as an obstacle, the arm part and the obstacle have already collided, and there is a possibility that the movement path of the arm part cannot be calculated.

  The present invention has been made in consideration of the above-described problems, and an object of the present invention is a robot apparatus that executes an operation according to an external environment, and the body of the body included in the three-dimensional data of the external environment. It is to make the image identifiable.

  A robot apparatus according to a first aspect of the present invention is a robot apparatus that performs an operation in accordance with an external environment, based on a visual sensor that visually recognizes the external environment, and information acquired by the visual sensor, The three-dimensional data creation unit that creates the three-dimensional data of the external environment, the determination unit that determines whether or not the body image of the robot apparatus is included in the three-dimensional data, and the determination unit And a region specifying unit that specifies a region occupied by the body image of the robot device in the three-dimensional data when it is determined that the body image of the robot device is included in the three-dimensional data. With such a configuration, an image of the body included in the three-dimensional data of the external environment can be identified. The body image here means an image of a part of the body. An environment restoration unit included in the lot device according to the first embodiment of the invention described below corresponds to the three-dimensional data creation unit. Moreover, the body estimation part in Embodiment 1 of invention corresponds to the said determination part and the said area | region specific part.

  Note that the specification by the region specifying unit may be performed by collating the three-dimensional data with the body model of the robot apparatus.

  The robot device according to the first aspect further includes a calculation unit that calculates the position and orientation of each unit constituting the robot device, and the region specifying unit includes the position calculated by the calculation unit and It is desirable that the region of the three-dimensional data to be collated with the body model is specified based on the posture. With such a configuration, the three-dimensional data and the body model can be efficiently collated.

  In addition, the robot device according to the first aspect preferably includes an operation planning unit that determines the operation content of the robot device based on the specification result by the region specifying unit and the three-dimensional data. With such a configuration, it is possible to improve the variety of operations of the robot apparatus, such as changing the operation content of the robot apparatus depending on whether or not the 3D data acquired by the visual sensor includes an image of its own body.

  The robot device according to the first aspect includes a correction unit that removes the body image of the robot device from the three-dimensional data, and the robot device based on the three-dimensional data corrected by the correction unit. An operation planning unit that determines the operation content may be provided. With such a configuration, it is possible to suppress malfunction of the robot apparatus due to the image of the body being included in the three-dimensional data acquired by the visual sensor.

  The method according to the second aspect of the present invention is a method for controlling a robot apparatus that performs an operation in accordance with an external environment. Specifically, first, it is determined whether or not the body image of the robot apparatus is included in the three-dimensional data of the external environment. Next, when it is determined that the body image of the robot apparatus is included in the three-dimensional data, an area occupied by the body image of the robot apparatus in the three-dimensional data is specified. Subsequently, the operation content of the robot apparatus is determined based on the detection result of the region and the three-dimensional data. By such a method, an image of the body of the self included in the three-dimensional data of the external environment is identified, and the operation content of the robot apparatus is determined depending on whether the image of the self body is included in the three-dimensional data of the external environment It is possible to improve the diversity of operation of the robot device.

  A method according to a third aspect of the present invention is a method for controlling a robot apparatus that performs an operation in accordance with an external environment. Specifically, the three-dimensional data of the external environment is acquired, and the body position and posture of the robot apparatus are calculated. Next, a processing target region is selected from the three-dimensional data based on the calculated body position and posture of the robot device, and the three-dimensional data is obtained by collating the selected processing target region with the body model of the robot device. In the data, a body region including an image of the body of the robot apparatus is detected. By such a method, it is possible to identify an image of the body included in the three-dimensional data of the external environment.

  According to the present invention, it is possible to provide a robot apparatus and a control method for the robot apparatus that can identify an image of the body included in three-dimensional data of the external environment.

  Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings. In the drawings, the same elements are denoted by the same reference numerals, and redundant description will be omitted as necessary for the sake of clarity. In the following embodiments of the invention, the present invention is applied to a robot having an arm portion for gripping an object.

Embodiment 1 of the Invention
An appearance of the robot 1 according to the present embodiment is shown in FIG. The head 10 of the robot 1 is provided with a visual sensor 101 for acquiring external three-dimensional point group data (hereinafter referred to as distance image data). The head 10 is connected to the body 11. An arm 12 is connected to the body 11. More specifically, the upper arm part 121 included in the arm part 12 is connected to the body part 11 via a shoulder joint mechanism (not shown), and the upper arm part 121 and the forearm part 123 are connected via an elbow joint part 122. A hand portion 124 is provided at the tip of the forearm portion 123. Further, the body portion 11 is provided with wheels 131 and 132 which are moving mechanisms of the robot 1.

  The robot 1 according to the present embodiment recognizes the position and orientation of the object 50 that is the gripping object using the distance image data acquired by the visual sensor 101. Next, the robot 1 performs a route plan for moving the arm unit 12 to a position where the recognized object 50 can be gripped, and grips the object 50 by the arm unit 12. When executing the route planning of the arm unit 12, it is determined whether or not an obstacle exists between the object 50 and the arm unit 12 based on the distance image data acquired by the visual sensor 101, and the obstacle is detected. The route to reach the object 50 by avoiding is determined. At this time, the arm 12 may be captured in the distance image data acquired by the visual sensor 101. In this case, if the arm portion 12 is recognized as an obstacle, an optimal route plan cannot be executed. Therefore, the robot 1 has a mechanism for not recognizing the arm portion 12 as an obstacle.

  In the following, regarding the gripping operation of the object 50 performed by the robot 1, a mechanism for not recognizing this as an obstacle when the arm 12 is imaged in the distance image data acquired by the visual sensor 101 will be described. This will be described in detail with reference to FIG. 2 and FIG. The block diagram of FIG. 2 shows the internal configuration of the main part of the robot 1 related to the process for gripping the object 50.

  In FIG. 2, the visual sensor unit 101 acquires three-dimensional point group data (distance image data) of the outside world of the robot 1 as described above. Specifically, distance image data is acquired by an active distance sensor such as a laser range finder. It should be noted that a plurality of cameras including an image sensor such as a CCD image sensor or a CMOS image sensor may be provided, and distance image data may be generated using image data photographed by the plurality of cameras. Specifically, corresponding points are detected from image data captured by a plurality of cameras, and the three-dimensional positions of the corresponding points are restored by stereo viewing. Here, the search for corresponding points in a plurality of captured images may be performed by applying a known method such as a gradient method or a correlation method using a time-space differential constraint formula for the plurality of captured images.

  The environment restoration unit 102 generates triangular polygons by connecting neighboring points of distance image data obtained by the visual sensor 101, and polygon data obtained by modeling the external environment of the robot 1 (hereinafter referred to as environment data). Is generated. Here, the reference coordinate system of the environment data may be a coordinate system fixed to the robot 1 or a coordinate system fixed in the environment where the robot 1 exists.

  In addition, when generating environmental data over a wide range, distance image data obtained by moving the visual sensor 101, that is, the head 10, by acquiring distance image data at a plurality of different viewpoints and integrating the distance image data at the plurality of viewpoints. Environmental data is generated based on the image data. Here, the integration of the distance image data obtained from a plurality of viewpoints is obtained by collecting odometry information, specifically, measurement information of a joint angle sensor (not shown) of the neck joint connecting the head 10 and the body 11. This is performed by aligning a plurality of distance image data based on the angle of the neck joint when the distance image data is acquired by the visual sensor 101. Alternatively, the distance image data may be aligned with each other by obtaining corresponding points from a plurality of distance image data. The environment data generated by the environment restoration unit 102 is stored in the environment data storage unit 103.

  The body estimation unit 104 performs processing for specifying an area in which the image of the arm unit 12 is included in the environmental data. Below, the process flow of the body estimation part 104 is demonstrated using the flowchart of FIG.

  In step S101, odometry information of the robot 1 is input from the control unit 107. Here, the odometry information is an encoder (not shown) provided in the robot 1 for detecting the position, angle, speed, angular velocity, and the like of the head 10, the arm 12, and the wheels 132 and 133 constituting the robot 1. And measurement information of an internal sensor such as a joint angle sensor (not shown).

  In step S102, the position and posture of the robot 1 are calculated using the body model of the robot 1 and the odometry information input from the control unit 107. Here, the body model is a geometric model of the body of the robot 1 and is represented by a joint and a link connecting the joints. The body model has the same degree of freedom and constraint conditions as the body of the robot 1.

  In step S103, based on the position and orientation of the robot 1 calculated in step S102, it is determined whether or not the image of the arm portion 12 exists in the visual field when the distance image data is acquired by the visual sensor 101. To do. The determination can be made by determining the intersection of the convex polygon group formed by the body model and the quadrangular pyramid polygon created by the angle of view of the visual sensor 101.

When it is determined that a part of the body of the robot 1 is present in the field of view, the body model of the robot 1 is matched with the environment data generated by the environment restoration unit 102 , and the arm 12 An area including the image is specified (step S104). Specifically, the area in which the image of the arm 12 is included in the environment data is specified by matching the three-dimensional shape data of the arm 12 stored in advance by the robot 1 with the environment data. . Here, the matching may be performed by employing a conventionally known image recognition method.

  At the beginning of step S104, the region including the image of the arm 12 in the environment data is roughly specified by the position and posture of the robot 1 specified using the odometry information, and this is used as the initial value of matching. It is desirable. It is also possible to scan the entire environment data without setting initial values at the beginning of step S104. However, by selecting the environment data area matched with the body model according to the position and posture of the robot 1 specified using the odometry information, the amount of calculation can be reduced by reducing the scanning area, and the matching process can be speeded up. It is effective in that it becomes possible.

  Returning to FIG. 2, the description will be continued. The correction unit 105 corrects the environmental data stored in the environmental data storage unit 103 according to the processing result of the body estimation unit 104. Specifically, the data of the area determined to include the image of the arm 12 in the environment data is deleted so that the arm 12 is not detected as an obstacle in the route planning unit 106 described later.

  The purpose of the environmental data correction by the correction unit 105 is to prevent the distance information regarding the arm unit 12 included in the environmental data from affecting the path of the arm unit 12 calculated by the path planning unit 106. The image of the arm 12 is removed from the environmental data. For this reason, the content of the data correction performed by the correction unit 105 may be suitable for the processing of the route planning unit 106. For example, the data of the area determined to include the image of the arm 12 in the environmental data is included. It may be deleted, may be replaced by peripheral distance data that does not include the image of the arm portion 12, or may be replaced by alternative distance data by interpolation using peripheral distance data that does not include the image of the arm portion 12. May be generated.

  The path planning unit 106 calculates the position and orientation of the object 50 by collating the environment data corrected by the correction unit 105 with the three-dimensional shape data of the object 50 stored in advance by the robot 1. Further, the path planning unit 106 calculates the position of the obstacle existing between the object 50 and the robot 1 using the environmental data. Further, the path planning unit 106 calculates a movement path of the arm unit 12 for avoiding the obstacle and gripping the object 50 based on the calculated position and posture of the object 50 and the position of the obstacle, The operation information of the arm unit 12 is output to the control unit 107.

  The control unit 107 includes an encoder (not shown) and a joint angle sensor (not shown) included in the robot 1 for detecting the position, angle, speed, angular velocity, and the like of the head 10, the arm 12, and the wheels 132 and 133. And the like, and control signals for driving the head 10, the arm 12, and the wheels 132 and 133 are output to the driving unit that drives them. Further, the control unit 107 outputs a control signal for operating the arm unit 12 according to the movement path of the arm unit 12 determined by the path planning unit 106 to the arm driving unit 108. The arm drive unit 108 is a drive circuit for operating the actuator of the arm unit 12.

  As described above, the robot 1 according to the present embodiment detects the image of the arm portion 12 existing in the environment data generated using the measurement data obtained by the visual sensor 101, and extracts the image of the arm portion 12 from the environment data. The environmental data is corrected by removing it. As a result, when determining the movement path of the arm 12 for holding the object 50 while avoiding the obstacle existing in the environment where the robot 1 is placed, the arm that is a part of the body of the robot 1 It is possible to prevent malfunction caused by recognizing 12 as an obstacle.

  Further, if the region where the arm portion 12 is present in the environmental data is simply identified only by the odometry information, the distance image data acquisition timing by the visual sensor 101 and the deviation of the odometry information acquisition timing of the arm portion 12 are caused. Thus, there is a possibility that the specific error of the region where the arm portion 12 exists increases. On the other hand, the robot 1 according to the present embodiment roughly specifies a region including the image of the arm 12 in the environment data based on the position and posture of the robot 1 specified using the odometry information. The body model of the robot 1 is matched with the environmental data, and an area in which the image of the arm 12 is included in the environmental data is specified. For this reason, compared with the case where only odometry information is used, the area | region where the image of the arm part 12 in environmental data is included can be specified in detail.

Other embodiments.
In the first embodiment of the invention, the configuration has been described in which the body estimation unit 104 detects a region where the arm 12 is present from the environment data obtained by converting the distance image data acquired by the visual sensor 101 into a polygon. However, the body estimation unit 104 may detect a region where the arm unit 12 exists from the distance image data that is the point cloud data acquired by the visual sensor 101.

  In the first embodiment of the invention, the body estimation unit 104 detects whether or not the image of the arm unit 12 is included in the environmental data. Other parts may be detected.

  In the first embodiment of the invention, the movement planning unit 106 determines the movement path of the arm unit 12 while avoiding the obstacle. However, when the obstacle is detected, the gripping operation is stopped. Alternatively, attention may be urged to the outside by an alarm sound or the like. That is, the subsequent operation according to the detection of the obstacle is arbitrary.

  In the first embodiment of the invention, the robot 1 includes the visual sensor 101 and autonomously acquires environmental data. However, the distance image data is acquired by a distance sensor or the like provided outside the robot 1, The distance image data acquired via the communication means may be transmitted to the robot 1.

  Further, the robot 1 according to the first embodiment of the invention has been described as performing the gripping operation of the object 50 while avoiding the obstacle. However, the present invention is not limited to a robot that performs such a gripping operation, but can be widely applied to a robot that performs an operation according to an external environment recognized by a visual sensor.

  Furthermore, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present invention described above.

It is a figure which shows the external appearance of the robot concerning Embodiment 1 of invention. It is a block diagram which shows the internal structure of the robot concerning Embodiment 1 of invention. It is a flowchart which shows the body estimation process which the robot concerning Embodiment 1 performs.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Robot 10 Head 101 Visual sensor 102 Environment restoration part 103 Environment data storage part 104 Body estimation part 105 Correction | amendment part 106 Path planning part 107 Control part 108 Arm drive part 11 Torso part 12 Arm part 121 Upper arm part 122 Elbow joint mechanism 123 Forearm Part 124 Hand part 50 Object

Claims (4)

A robot apparatus that performs an operation according to an external environment,
A visual sensor that visually recognizes the external environment;
A three-dimensional data creation unit that creates three-dimensional data of the external environment based on information acquired by the visual sensor;
A calculation unit that calculates the position and posture of the body of the robot apparatus using measurement information of at least one internal sensor that measures the state of the robot apparatus itself;
A determination unit that determines whether or not the image of the body of the robot apparatus is included in the three-dimensional data based on the position and orientation calculated by the calculation unit;
When it is determined by the determination unit that the body image of the robot apparatus is included in the three-dimensional data, the body image of the robot apparatus is included based on the position and orientation calculated by the calculation unit. As described above, the processing target region that is a partial region is selected from the three-dimensional data, and the selected processing target region and the body model of the robot apparatus are collated using an image recognition technique. An area specifying unit for specifying an area occupied by the body image of the robot apparatus in the data;
An operation planning unit for determining the operation content of the robot apparatus based on the identification result by the region identification unit and the three-dimensional data;
A robot apparatus comprising: A robot apparatus that performs an operation according to an external environment,
A visual sensor that visually recognizes the external environment;
A three-dimensional data creation unit that creates three-dimensional data of the external environment based on information acquired by the visual sensor;
A calculation unit that calculates the position and posture of the body of the robot apparatus using measurement information of at least one internal sensor that measures the state of the robot apparatus itself;
A determination unit that determines whether or not the image of the body of the robot apparatus is included in the three-dimensional data based on the position and orientation calculated by the calculation unit;
When it is determined by the determination unit that the body image of the robot apparatus is included in the three-dimensional data, the body image of the robot apparatus is included based on the position and orientation calculated by the calculation unit. As described above, the processing target region that is a partial region is selected from the three-dimensional data, and the selected processing target region and the body model of the robot apparatus are collated using an image recognition technique. An area specifying unit for specifying an area occupied by the body image of the robot apparatus in the data;
A correction unit that removes the body image of the robot apparatus from the three-dimensional data;
An operation planning unit for determining the operation content of the robot apparatus based on the three-dimensional data corrected by the correction unit;
A robot apparatus comprising: A control method of a robot apparatus that performs an operation according to an external environment,
Using the measurement information of at least one internal sensor that measures the state of the robot device itself, the body position and posture of the robot device are calculated,
Based on the calculated body position and posture of the robot apparatus, it is determined whether or not an image of the body of the robot apparatus is included in the three-dimensional data of the external environment,
When it is determined that the body image of the robot apparatus is included in the three-dimensional data, the body of the robot apparatus is selected from the three-dimensional data based on the calculated position and posture of the body of the robot apparatus. In the three-dimensional data by selecting a processing target region that is a partial region so as to include the image and collating the selected processing target region with the body model of the robot apparatus using an image recognition technique. Identify the area occupied by the body image of the robotic device,
A control method in which the operation content of the robot apparatus is determined based on the result of specifying the region and the three-dimensional data. A control method of a robot apparatus that performs an operation according to an external environment,
Acquire 3D data of external environment,
Using the measurement information of at least one internal sensor that measures the state of the robot device itself, the body position and posture of the robot device are calculated,
Based on the calculated body position and posture of the robot apparatus, it is determined whether or not the body image of the robot apparatus is included in the three-dimensional data,
When it is determined that the body image of the robot apparatus is included in the three-dimensional data, the body image of the robot apparatus is included based on the calculated position and posture of the body of the robot apparatus. To select a processing target area from the three-dimensional data,
By detecting the selected region to be processed and the body model of the robot apparatus, a body area including the body image of the robot apparatus is detected in the three-dimensional data.
Correcting the three-dimensional data according to the detection result of the body region,
A control method in which the operation content of the robot apparatus is determined based on corrected three-dimensional data.