JP6157436B2 - Alignment apparatus and program - Google Patents

Description

  The present invention relates to an alignment apparatus and a program for aligning a part model imitating a part shape of an object to an object region extracted from an image for estimating a posture of a person.

  When estimating the posture of a person on the image, a part model imitating the shape of the head, torso, upper arm, forearm, upper limb, and lower limb is aligned and applied to the human region extracted from the image. Conventionally, the positioning of the part model is performed by rotating and translating each part model by a plurality of small amounts on the image to generate a posture model of the entire person and then evaluating the overlap with the entire person area. It was realized repeatedly.

Japanese Patent Laid-Open No. 07-302341

  In the prior art as disclosed in Patent Document 1, the amount of calculation required for position model alignment performed for posture estimation is enormous, and it is difficult to realize real-time processing.

  One factor that increases the amount of calculation in the prior art is that a plurality of part models are aligned at once with respect to the entire person region. In this regard, if the part models are sequentially aligned with respect to the person region one by one, it is possible to impose a constraint condition on the arrangement of the parts to be subsequently aligned by the previously aligned parts. It can be effectively reduced. However, in the prior art, there is no mechanism for sequentially aligning the site models one by one.

  Another factor that increases the amount of computation in the prior art is to rotate and translate the part model by a small amount in multiple ways at each iteration to determine the amount of rotation and translation that maximizes the overlap with the human area. It is mentioned that it is determined by trial and error. It is possible to effectively reduce the number of iterations if trial and error can be eliminated by deterministically calculating the amount of rotation and the amount of parallel movement with the largest overlap with the human region with respect to the arrangement of the part model in each iteration. . However, there is no effective algorithm for calculating the deterministic rotation amount.

  The present invention has been made in view of the above problem, and a position model that imitates the shape of a part that is a part of an object can be accurately aligned with the object area extracted from the image with a small amount of calculation. An object of the present invention is to provide a matching device and a program.

  The alignment apparatus according to the present invention aligns a part model imitating the shape of a part constituting the predetermined object in a target object region in which the predetermined target is extracted in a two-dimensional or three-dimensional processing target space. An alignment device, the storage means for storing the shape of the part model and the principal axis of inertia, and the initial placement means for initially arranging the part model at a position where the part model and the object region generate an overlapping area; Estimating the inertial principal axis of the overlapping area, calculating the amount of rotation of the part model to bring the direction of the inertial principal axis of the part model closer to the inertial principal axis of the overlapping area, and increasing the overlap area A movement amount calculating means for calculating a movement amount; a layout updating means for rotating the part model by the rotation amount and translating the part model by the parallel movement amount; and the movement amount calculation. Having a repetition control means for repeated until the end conditions predetermined processing by means and the placement updating means is satisfied.

  In another alignment apparatus according to the present invention, the processing target space is three-dimensional, and the movement amount calculating means is a three-dimensional coordinate axis having two of the three principal axes of inertia of the three-dimensional part model. The region model in which the principal axis of inertia of the overlapping region is projected onto a plane of the street, and the component around the axis perpendicular to the plane of the rotation amount is set as the coordinate axis of the plane in each of the three planes Is calculated according to an angle formed by one of the principal axes of inertia and the principal axis of inertia of the overlapping region projected onto the plane.

  In still another alignment apparatus according to the present invention, the movement amount calculation means divides the overlap region into a plurality of partial regions arranged in the direction of the principal axis of the part model, and arranges the centers of gravity of the plurality of partial regions. And a straight line passing through the center of gravity of the overlapping region is estimated as the inertial principal axis of the overlapping region.

  In still another alignment apparatus according to the present invention, the movement amount calculation unit divides the overlapping region into two partial regions by a straight line or a plane orthogonal to the principal axis of inertia of the part model, and each of the partial regions and the A straight line passing through the center of gravity of each overlapping region is estimated as the inertial principal axis of the overlapping region.

  The program according to the present invention provides a predetermined part model imitating a shape of a part constituting the predetermined object in a target area in which the predetermined target is extracted in a two-dimensional or three-dimensional processing target space. A program for causing a computer to perform alignment processing, wherein the computer initially arranges the part model at a position where the part model and the object area generate an overlapping area, and the inertia main axis of the overlapping area To calculate the amount of rotation of the part model that brings the direction of the principal axis of inertia of the part model closer to the principal axis of inertia of the overlapping area, and the process of rotating the part model by the amount of rotation and increasing the overlapping area The amount of translation of the part model to be calculated is calculated, and a process for translating the part model by the amount of translation is satisfied in advance. In be repeated.

  ADVANTAGE OF THE INVENTION According to this invention, the position alignment apparatus which can align the site | part model imitating the shape of the site | part which is a part of target object with the small amount of calculations to the target object area extracted from the image with high precision is obtained.

1 is a schematic block configuration diagram of an attitude estimation apparatus according to an embodiment of the present invention. It is a functional block diagram of the outline of the attitude | position estimation apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows the state in which a site | part model exists in a world coordinate system. It is a mimetic diagram explaining the conditions about the height of the initial position of a part model. It is a schematic diagram explaining the calculation method of the parallel displacement by the mean shift method. It is a schematic diagram explaining the calculation method of the rotation amount of a local coordinate system by a movement amount calculation means. It is a flowchart which shows the operation | movement of the outline of the attitude | position estimation apparatus which concerns on 1st embodiment of this invention. It is a schematic diagram which shows the state in which a site | part model exists in a world coordinate system. It is the schematic diagram which represented the setting rule of the reference | standard axis | shaft for every rotating shaft, a division | segmentation plane, and a division | segmentation area | region in tabular form. It is a schematic diagram explaining how to obtain the amount of rotation when the x-axis is the rotation axis. It is a schematic diagram explaining how to obtain the amount of rotation when the y axis is the rotation axis. It is a schematic diagram explaining how to obtain the amount of rotation when the z axis is the rotation axis. It is a flowchart which shows the operation | movement of the outline of the attitude | position estimation apparatus which concerns on 2nd embodiment of this invention.

  Hereinafter, an attitude estimation apparatus according to an embodiment of the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings. The posture estimation apparatus according to the embodiment uses a person as an object for posture estimation, and aligns a part model imitating the shape of the part of the person with a person region extracted based on a monitoring image obtained by photographing the monitoring space. Then, the posture of the person is estimated based on the degree of coincidence and arrangement of the aligned part models.

[First embodiment]
At the time of posture estimation, the posture estimation device 1 according to the first embodiment of the present invention aligns a two-dimensional part model with a two-dimensional person region extracted from a monitoring image.

  FIG. 1 is a schematic block diagram of the posture estimation apparatus 1. The posture estimation apparatus 1 includes a photographing unit 2, a storage unit 3, an image processing unit 4, and an output unit 5. The photographing unit 2, the storage unit 3, and the output unit 5 are connected to the image processing unit 4.

  The imaging unit 2 is a monitoring camera that images the monitoring space, and inputs the captured monitoring image to the image processing unit 4.

  The storage unit 3 is a storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), and a hard disk, and programs and data used by the image processing unit 4, various data generated by the image processing unit 4, and the like. Remember. The storage unit 3 inputs and outputs these programs and data to and from the image processing unit 4.

  The image processing unit 4 includes a processor such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), an MCU (Micro Control Unit), and its peripheral circuits. For example, the image processing unit 4 is configured by a computer that operates based on a program stored in the storage unit 3. The image processing unit 4 operates as each unit to be described later, and estimates the posture of the person from the monitoring image and outputs it to the output unit 5.

  The output unit 5 is a display device or the like that shows the estimated person posture to the user.

  FIG. 2 is a schematic functional block diagram of the posture estimation apparatus 1. In FIG. 2, the photographing unit 2 and the output unit 5 are not shown. The image processing unit 4 operates as an object area extraction unit 40, an initial arrangement unit 41, a movement amount calculation unit 42, an arrangement update unit 43, an iterative control unit 44, and an attitude estimation unit 45. Among these, the initial arrangement means 41, the movement amount calculation means 42, the arrangement update means 43, and the iterative control means 44 function as the alignment apparatus 10 according to the present invention. The storage unit 3 functions as a storage unit 30 that stores information used in the alignment apparatus 10. Specifically, the storage unit 30 stores the part model information 300.

  The object area extracting unit 40 extracts a person area as an object area from the monitoring image by, for example, a difference process between the monitoring image and a background image prepared in advance. In the difference image obtained by extraction, for example, the person area is displayed by setting the pixel value in the person area to “1” and the pixel value outside the person area to “0”.

  The alignment apparatus 10 uses a difference image on which a person region is displayed as a two-dimensional processing target space, and aligns a part model with a target part in the person region in the processing target space. Hereinafter, the two-dimensional processing target space of the present embodiment is referred to as a processing target image. In the present embodiment, the human head and torso are combined as one target part, and the posture estimation apparatus 1 uses the registration apparatus 10 to convert an elliptical part model on the processing target image to the head and torso of the person region. Estimate the posture of the upper body by aligning with the part. In this alignment process, the part model is arranged at the initial position, and the arrangement update for rotating and translating the part model is repeatedly performed, so that the arrangement of the part model at the target part is converged. Accordingly, alignment is performed with the goal of matching one end of the major axis of the ellipse with the top of the head and the other end with the buttocks.

  The part model information 300 is information that defines a part model, and includes information indicating the shape and orientation of the part model. The part model is a two-dimensional shape that models the shape of a part to be aligned. The shape of the part model is basically not limited to axial symmetry, central symmetry, and the like, and can be an arbitrary two-dimensional shape that matches the target part. In the present embodiment, the part model is an ellipse.

  Here, the coordinate system of the processing target image is defined by a right-handed orthogonal coordinate system XY, for example. Note that the processing target image and the monitoring image can be basically the same coordinate system. The coordinate system fixed to the part model is defined as a local coordinate system, and is defined by, for example, a right-handed orthogonal coordinate system xy.

The local coordinate system is set so that an ellipse as the part model 60 is expressed by the following equation.
^{(X 2 / Rx 2) +} (y 2 / Ry 2) = 1

  Here, Rx <Ry. That is, the region model 60 is an ellipse having a major radius Ry in the y-axis direction and a minor radius Rx in the x-axis direction. The center c of the ellipse is the origin of the local coordinate system. Therefore, the y axis and the x axis are set as the principal axes of inertia of the part model 60, and the origin is set as the center of gravity of the part model 60.

  When the part model 60 has an arbitrary shape, the axes of the first principal component and the second principal component in the principal component analysis of the shape can be defined as the y axis and the x axis, respectively.

  In the present embodiment in which the processing target space is two-dimensional, the appearance of the target part in the person region changes depending on the positional relationship between the monitoring camera and the target part. In order to cope with this, site model information 300 representing a plurality (M types) of site models 60 having different shapes is prepared as candidates. For example, a plurality of ellipses with different combinations of Rx and Ry can be prepared in the part model information 300 as the part model 60. Alternatively, the image processing unit 4 generates a plurality of part models 60 according to the size of the person area extracted by the object area extraction unit 40 and stores the generated part models 60 in the storage unit 30.

  FIG. 3 is a schematic diagram showing a state in which the part model 60 exists in the coordinate system of the processing target image. The posture of the part model 60 is described by the coordinate axes and the origin of the local coordinate system described in the coordinate system of the processing target image. That is, the translation of the part model 60 is described by the movement of the origin (point c) of the local coordinate system in the coordinate system of the processing target image, and the rotation of the part model 60 is performed in the local coordinate system of the processing target image. It is described by changes in the orientation of the x-axis and y-axis.

  The initial arrangement means 41 determines an elliptical initial position arranged as a part model in the processing target image. For example, an upright posture of a person is considered as a basic posture, and the elliptical direction at the initial position is set so that the y-axis direction is the vertical direction in the monitoring image. In this embodiment, the processing target image has the same coordinate system as the monitoring image, and the direction of the y axis is set to be the direction of the Y axis. Further, the position of the ellipse can be set based on the position of the person area in the processing target image. For example, the X coordinate in the coordinate system of the processing target image at the center of the ellipse (that is, the origin of the local coordinate system) can be made to correspond to the position of the person region in the X axis direction. On the other hand, the Y coordinate in the coordinate system of the processing target image at the center of the ellipse is such that the ellipse and the person area overlap, and the top of the ellipse is slightly higher than the top of the person area. Set to.

  Incidentally, the reason for initial setting so that the ellipse and the person area have overlapping areas is that the movement amount calculation means 42 uses the center of gravity of the overlapping area and the amount of rotation and the parallel movement amount of the part model 60 as described later. Because it asks for. In addition, the reason why the part model 60 is initially set to a position higher than the person region, that is, the height of the person, is related to the position model 60 being aligned with the head and torso parts forming the upper body of the person in the present embodiment. ing. The convergence determination of the repetitive process of rotation / translation of the part model 60 in the alignment apparatus 10 is made based on the size of the overlapping area between the part model 60 and the person area. For example, the overlapping area is either the upper body or the lower body. It can be determined that it has converged regardless. Therefore, if the initial position of the part model 60 is set to the lower side of the person area, there is a high possibility that the elliptical vertex converges in a state where it does not coincide with the top of the head. In order to reduce this possibility, the top of the part model 60 is initially set higher than the top of the person region.

  FIG. 4 is a schematic diagram for explaining the above-described conditions for the height of the initial position of the part model 60. FIG. 4A shows a case where the initial setting is made so that the top of the part model 60 is positioned higher than the height of the person. State (a-1) shows the initial setting, state (a-2) shows the convergence process, and state (a-3) shows the final converged result. In the state (a-3), one end in the longitudinal direction of the ellipse is located at the top of the head and the other end is located at the buttocks, and the part model 60 is suitable for the head part and the body part which are the target parts in the human region 62. Are aligned. On the other hand, FIG. 4B shows a case where the initial setting is made so that the top of the part model 60 is lower than the height of the person, and the state (b-1) is the state (b-2) at the initial setting. Indicates the convergence process, and state (b-3) indicates the final converged result. In this case, the position of the apex of the part model 60 does not converge to the top of the head or the buttocks, and the position of the part model 60 converges to an unintended part of the person region 62.

  The movement amount calculation means 42 calculates the parallel movement amount of the origin of the local coordinate system of the part model and the rotation amount of the x axis or the y axis. In the present embodiment, the parallel movement amount is obtained by a mean shift method. FIG. 5 is a schematic diagram for explaining a method of calculating the parallel movement amount by the mean shift method. In the present embodiment, the movement amount calculation means 42 obtains, for example, a person area existing in the part model 60, that is, a center of gravity g of an overlapping area between the part model 60 and the person area 62. In FIG. 5, “●” is marked on the center of gravity g in the state of FIG. Then, the movement amount calculation means 42 calculates a vector from the ellipse center c where the origin of the local coordinate system is located toward the center of gravity g as a parallel movement amount. FIG. 5B shows a state in which the part model 60 has been moved by the amount of translation calculated in the state of FIG. 5A, and the “x” mark indicating the ellipse center c is the “●” mark. Moved to position.

  FIG. 6 is a schematic diagram for explaining a method of calculating the rotation amount of the local coordinate system by the movement amount calculation means 42. FIG. 6A shows an arrangement example of the person region 62 and the part model 60 in a state where the rotation amount is to be calculated. In the present embodiment, the rotation amount θ is determined so that the inertial principal axis of the part model 60 approaches the inertial principal axis of the overlapping region between the part model 60 and the person region 62. The movement amount calculating means 42 sets the inertial main axis of the part model as a reference axis for performing the rotation operation, estimates the inertial main axis of the overlapping region corresponding to the reference axis, and sets it as the target axis with respect to the reference axis . Then, the rotation amount θ of the part model 60 that brings the direction of the reference axis closer to the target axis is calculated.

  In the present embodiment, the reference axis L is the one of the two inertial main axes that are orthogonal to each other and has the minimum moment of inertia. In this case, in the part model 60 having an elliptical shape, the inertial principal axis serving as the reference axis L coincides with the major axis of the elliptical shape. Note that the y-axis of the local coordinate system is also set to coincide with the major axis.

  In this embodiment, the inertia main axis of the overlapping region is estimated, and an approximate straight line of the inertia main axis is set as the target axis. Specifically, the movement amount calculating means 42 obtains the center of gravity g of the overlapping area between the part model 60 and the person area 62. The overlapping area is divided into two areas by a dividing line P passing through the center of gravity g and orthogonal to the main axis L. The two areas will be referred to as divided areas A and B. The movement amount calculating means 42 obtains centroids (called partial centroids) ga and gb of the divided areas A and B, respectively. In FIG. 6B, the partial centroids ga and gb are indicated by “■” and “▲”, respectively. In this embodiment, a straight line passing through ga, g, and gb is used as an approximation of the inertial principal axis of the overlapping region. If the reference axis of the part model 60 is aligned with the direction of a straight line passing through the center of gravity of the two partial areas A and B arranged in the direction of the reference axis L of the part model 60, it can be expected that the overlapping area will increase. Therefore, the movement amount calculating means 42 calculates the angle between the straight line and the axis L as the rotation amount θ using the straight line as the target axis. Assuming that a straight line parallel to the straight line L and passing through the center of gravity g is L ′, the movement amount calculating means 42 includes an angle θa formed by the line segment connecting g and ga and the straight line L ′, and a line segment connecting g and gb. An angle θb formed by the straight line L ′ is obtained, and an average value of θa and θb is defined as a rotation amount θ.

  Here, θa and θb are averaged in consideration of reducing the influence of calculation error when the number of pixels in the overlapping region is small, but in principle, ga, g, and gb are arranged in a straight line. , Θa and θb are the vertical angles formed by the straight line and the straight line L ′ and are equal to each other, so that either one of θa and θb can be set as the rotation amount θ. In addition, it may be configured to prevent vibration and divergence due to excessive rotation, such as reducing the rotation amount according to the number of pixels in the overlapping area, in response to a decrease in the calculation accuracy of the barycentric coordinates when the number of pixels in the overlapping area is small. .

  FIG. 6C shows a state where the part model 60 is rotated by an angle θ and the direction of the main axis L is changed to the approximate direction of the main axis of the overlapping region. Here, a state where the part model 60 is rotated around the center of gravity g is shown.

  The arrangement updating unit 43 updates the arrangement of the part model 60 by translating the part model 60 by the parallel movement amount obtained by the movement amount calculating unit 42 and rotating the part model 60 by the rotation amount obtained by the movement amount calculating unit 42. That is, the origin of the local coordinate system is translated by the calculated parallel movement amount, and the local coordinate system is rotated by the calculated rotation amount. Here, if the rotation center is the origin of the local coordinate system (ellipse center c), the translation operation and the rotation operation become independent operations, and there is no processing order dependency. Updated to the same arrangement.

  The iterative control unit 44 determines whether or not the part model 60 satisfies the end condition of the alignment process every time the movement amount calculation unit 42 and the arrangement update unit 43 perform one arrangement update. For example, the end condition can be that the amount of parallel movement and the amount of rotation of one time converges to a predetermined threshold value or less. The iterative control unit 44 repeats the processing by the movement amount calculating unit 42 and the arrangement updating unit 43 until the end condition is satisfied. When the end condition is satisfied, the arrangement of the aligned part model 60 is added to the part model information 300. Remember me.

The end condition is not limited to the above example. For example, the end condition may be that the ratio R _IN of the area of the person region included in the part model 60 is equal to or greater than a predetermined value β. Here, the value of β is assumed to be a value that can be expected to be suitably aligned, for example, 0.95. Assuming that the iterative process falls into an infinite loop, it is also possible to set an end condition so as to prevent the iterative process from being aborted when a predetermined number of iterations is exceeded.

  The posture estimation means 45 estimates the posture of the person shown in the monitoring image based on the position of the aligned part model 60.

  FIG. 7 is a flowchart showing a schematic operation of the posture estimation apparatus 1 of the present embodiment. The posture estimation apparatus 1 is activated in a state where there is no person in the monitoring space, and in this state, the monitoring space is imaged by the imaging unit 2 to acquire a background image (step S2). Note that the background image can be appropriately updated during the operation of the posture estimation apparatus 1. The acquired background image is stored in the storage unit 3.

  The imaging unit 2 images the monitoring space at regular intervals, for example (step S4). The captured image is input to the image processing unit 4 as a monitoring image.

  In the image processing unit 4, the object region extraction unit 40 performs background difference processing on the monitoring image using the background image stored in the storage unit 3, and a person region is extracted as a processing target image in the alignment apparatus 10. A difference image is generated (step S6). The difference image is, for example, a binarized image in which the person area in the monitoring image has a pixel value “1” and the other area has a pixel value “0”.

  The alignment apparatus 10 performs alignment processing for each of the M part model information 300 stored in the storage unit 30. The image processing unit 4 sets an index I indicating the type of the part model information 300 to an initial value “1” (step S8), increments the index I by 1, and uses the alignment apparatus 10 for each of the M part model 60. Perform alignment processing.

  The alignment apparatus 10 reads the I-th part model information 300 from the storage unit 30 (step S10), and the initial placement means 41 converts the part model 60 represented by the I-th part model information 300 into the person area as described above. Arrangement is made at the initial position set on the basis (step S12), and the arrangement (steps S14 to S20) of repetitively updating the arrangement of the part model 60 and aligning with the target part of the person region is started. .

  In the iterative process, the movement amount calculation means 42 calculates the parallel movement amount and the rotation amount of the part model 60 as described above based on the overlapping region of the current part model 60 and the person region (step S14), and updates the arrangement. The means 43 updates the part model 60 to a new arrangement based on the calculation result of the movement amount calculating means 42 (step S16). Specifically, the orientation of the ellipse center c indicating the position of the part model 60 and the y-axis processing target image indicating the posture (orientation) of the part model 60 in the coordinate system is updated and stored in the part model information 300. .

  The iterative control means 44 performs a process for controlling the iterative process every time one update process by the arrangement updating means 43 is completed (steps S18 and S20). For example, the parallel movement amount and the rotation amount calculated by the movement amount calculation means 42 are respectively compared with threshold values for determining the convergence of iteration (step S18), and if either exceeds the threshold value, the iteration is repeated. Is repeated (in the case of “No” in step S20), and if both are equal to or less than the threshold value, the iteration is terminated as being converged (in the case of “Yes” in step S20). In addition, an upper limit may be set in advance as the number of iterations, and the iteration may be terminated when the number of iterations is exceeded.

When the iteration for the I-th part model 60 is completed, the iteration control unit 44 calculates the degree of coincidence of the part model 60 with the person region (step S22). For example, a value R _IN obtained by dividing the area S _{IN of the} person region included in the part model 60 by the area of the part model 60 can be used as the degree of coincidence. Further, the value R _OUT of the ratio of the remaining area (S−S _IN ) outside the part model 60 to the entire area S of the human region can be used as an index value indicating that the smaller the value, the higher the matching degree. . A value indicating the calculated degree of coincidence is stored in the part model information 300.

  The alignment apparatus 10 increments the index I (step S26) if the part model 60 for which the alignment process has been completed is not the last, that is, the Mth one (in the case of “No” in step S24). The alignment process (steps S10 to S22) for the part model 60 is performed.

  On the other hand, when the alignment process is completed for all M part body models 60 (in the case of “Yes” in step S24), the posture estimation unit 45 determines the degree of coincidence for each part model 60 stored in the part model information 300. Comparison is made, and a part model 60 most suitable for the target part of the person region is selected, and the posture of the person is estimated (step S28).

  Every time a monitoring image is obtained (step S4), the image processing unit 4 performs the processes of steps S6 to S28 described above.

  In this embodiment, the mean shift method is used for calculating the parallel movement amount. Thereby, since a suitable amount of parallel movement is uniquely determined based on the positional relationship between the part model 60 and the person area 62, for example, the part model with respect to the person area 62 rather than performing a certain amount of parallel movement. 60 alignments are performed efficiently.

  Furthermore, in the present embodiment, with respect to the calculation of the rotation amount, the rotation amount is determined so that the inertial main axis of the part model 60 is aligned with the straight line that estimates the inertial main axis of the overlapping region in each iterative process. In this method, similar to the mean shift method in translation, a suitable rotation amount is uniquely determined based on the positional relationship between the part model 60 and the person area 62, so that the position of the part model 60 with respect to the person area 62 is efficient. To be done.

  The method of estimating the inertial principal axis of the overlapping region between the part model 60 and the person region 62 may be a method other than the above embodiment.

  For example, a principal component analysis can be performed on the overlapping region to obtain the principal axis of inertia. Specifically, the first component of the principal component analysis obtained for the pixel distribution of the overlapping region in the processing target image becomes the inertial principal axis of the overlapping region. Therefore, the part model 60 is adapted so that the inertia principal axis of the minimum moment of inertia of the part model 60 adopted as the reference axis for alignment in the above embodiment matches the inertia principal axis of the minimum moment of inertia obtained by principal component analysis for the overlapping region. The amount of rotation can be calculated.

  In the above embodiment, when estimating the inertial principal axis of the overlap region, the overlap region is divided into two regions by the dividing line P orthogonal to the inertial principal axis L of the part model 60. It is also possible to estimate a straight line Q that is divided into three or more partial regions and that approximates the arrangement of the centroids of the plurality of partial regions and passes through the centroid g of the overlapping region as the inertial principal axis of the overlapping region. Specifically, the overlapping area is divided into three or more by a plurality of straight lines orthogonal to L, and the partial center of gravity of each divided area is obtained. A straight line that approximates the distribution of the partial centroids can be obtained by the least square method under the constraint that it passes through the centroid g of the overlapping region, and this can be used as the straight line Q. When obtaining a straight line approximating the distribution of partial centroids, a weighted least square method is used in which the size of the divided area corresponding to each partial centroid is considered as a weight.

  In the above embodiment, assuming that the upper body of the person is standing, the direction of the elliptical long axis (y-axis) of the part model 60 at the initial position is set to the Y-axis direction. As a result, the overlapping area is also likely to be relatively long, and the axis obtained for the overlapping area by the above-described method of dividing the overlapping area by the straight line P orthogonal to the y-axis is the same as the inertia main axis set for the part model 60. It can be expected that the moment approximates the principal axis of inertia, and the part model 60 can be suitably aligned to the intended person's head and torso.

  On the other hand, it is also assumed that the overlapping area does not become vertically long when the upper body of the person has a large inclination. In such a case, the axis of the overlapping area obtained by the above-described method of dividing the overlapping area by the straight line P orthogonal to the y-axis is oriented close to the inertial main axis where the moment of inertia is not minimum, and the part model 60 is in an unintended posture. May converge. In order to avoid this, a plurality of orientations in the initial arrangement are set for the same part model 60, the degree of coincidence is obtained by performing iterative processing for each posture, and the posture that gives the maximum degree of coincidence and the degree of coincidence are obtained as part model information. It is good also as a structure memorize | stored in 300 and utilized with the attitude | position estimation means 45. FIG.

[Second Embodiment]
In the following description of the second embodiment of the present invention, the same components as those in the first embodiment are denoted by the same reference numerals, explanations of common matters are omitted, and differences from the first embodiment are described. Mainly explained.

  At the time of posture estimation, the posture estimation device 1 according to the second embodiment aligns a three-dimensional part model with a three-dimensional human region obtained from a monitoring image. That is, the first embodiment is the alignment in the two-dimensional space, whereas the present embodiment is the basic difference in that the alignment is in the three-dimensional space. In the present embodiment, the coordinate system of the processing target space is defined as a world coordinate system, which is defined by a right-handed orthogonal coordinate system XYZ. A local coordinate system fixed to the part model 60 is defined by a right-handed orthogonal coordinate system xyz.

  The configuration of the present embodiment is basically the same as that shown in FIGS. 1 and 2 of the first embodiment, and these figures are incorporated.

  The photographing unit 2 is a plurality of cameras, and is arranged so that a three-dimensional shape of a person in the surveillance space can be obtained from a surveillance image photographed by them by a visual volume intersection method or the like. For example, each camera is equipped with a fisheye lens and is installed directly downward from the ceiling.

  The internal parameters and external parameters of the plurality of cameras are measured by calibration in advance and stored in the storage unit 3 as camera parameters. The position / orientation of each camera in the world coordinate system is represented by an external parameter, and the position of the world coordinate system XYZ is converted (projected) into the shooting plane coordinate system uv of each camera, or the shooting plane coordinate system uv of each camera. Can be converted (backprojected) into the world coordinate system XYZ.

  The object region extraction unit 40 performs background difference processing on the monitoring images of the cameras and generates a difference image representing the silhouette of the person. The object area extraction unit 40 backprojects the difference image onto the world coordinate system using the camera parameters stored in the storage unit 3. By this back projection, a viewing volume, which is a space of a cone with the camera position as a vertex, is defined. The object region extraction means 40 calculates a space (product space) where the viewing volumes of all the cameras intersect. This product space represents a three-dimensional schematic shape of a person, and this is a person area in this embodiment. In the first embodiment, the person area can be handled as a set of pixels. Similarly, in this embodiment, a person area can be handled as a set of voxels.

  In the present embodiment, a virtual space in which a three-dimensional person area set corresponding to the monitoring space is arranged is defined as the processing target space. The processing target space and the monitoring space are defined by a common world coordinate system XYZ.

  The alignment apparatus 10 estimates the posture of a person by aligning a three-dimensional part model with a target part in the person area in the processing target space in which the person area is arranged. In this alignment process, the part model is arranged at the initial position, and the arrangement update for rotating and translating the part model is repeatedly performed, so that the arrangement of the part model at the target part is converged. In this embodiment, as in the first embodiment, the head and torso portions of a person are combined into one target portion, and a flat ellipsoid (hereinafter simply referred to as an ellipsoid) portion model is aligned. It shall be. Therefore, alignment is performed with the goal of matching one end of the major axis of the ellipsoid with the top of the head and the other end with the buttocks.

The shape of the part model can basically be an arbitrary three-dimensional shape that matches the target part, but in this embodiment, it is an ellipsoid represented by the following expression in the local coordinate system.
^{(X 2 / Rx 2) +} (y 2 / Ry 2) + (z 2 / Rz 2) = 1

  Here, Rx <Ry <Rz. That is, the part model 60 is a three-axis unequal ellipsoid having a major radius Rz in the z-axis direction. Incidentally, the x-axis direction of the ellipsoid corresponds to the human front-rear direction, the y-axis direction corresponds to the torso side direction, and the z-axis direction corresponds to the height direction. The center c of the ellipsoid is the origin of the local coordinate system.

  The storage means 30 stores ellipsoidal part model information 300 as the part model 60. In this embodiment, unlike the first embodiment, alignment is performed in a three-dimensional space. Therefore, the shape of the target part corresponding to the posture of the person according to the projection of the three-dimensional object onto the two-dimensional image is obtained. No change will occur. Therefore, the part model 60 can be only one type representing the representative shapes of the person's head and torso parts.

  FIG. 8 is a schematic diagram showing a state in which the part model 60 exists in the world coordinate system. The posture of the part model 60 is described by the coordinate axes and the origin of the local coordinate system described in the world coordinate system. That is, the parallel movement of the part model 60 is described by the movement of the origin (point c) of the local coordinate system in the world coordinate system, and the rotation of the part model 60 is the x-axis, y-axis, It is described by a change in the orientation of the z-axis.

  The initial placement means 41 determines an initial position of an ellipsoid placed as a part model in the processing target space. For example, the upright posture of the person is considered as the basic posture, and the orientation of the ellipsoid at the initial position is set so that the direction of the z axis is the vertical direction in the processing target space, that is, the Z axis direction. Further, the x-axis and y-axis of the ellipsoid can be set according to the horizontal cross-sectional shape of the person area, that is, the shape along the XY plane. Specifically, the x-axis and y-axis of the part model can be set so as to be approximately in the short direction and the long direction of the person region, respectively.

  The position of the ellipsoid can be set based on the position of the person area in the processing target space. For example, the X and Y coordinates in the world coordinate system at the center of the ellipsoid (that is, the origin of the local coordinate system) can be made to correspond to the positions of the person region in the X and Y axis directions. On the other hand, the Z coordinate in the world coordinate system at the center of the ellipsoid is the same as in the first embodiment, and the ellipsoid and the person area overlap with each other, and the uppermost part of the ellipsoid is higher than the uppermost part of the person area. Also set it to a slightly higher position.

  The movement amount calculation means 42 calculates the parallel movement amount of the origin of the local coordinate system of the part model and the rotation amounts θx, θy, θz about the x axis, the y axis, and the z axis, respectively. The amount of translation is determined by the mean shift method.

  The calculation methods for the rotation amounts θx, θy, and θz are three-dimensional extensions of the calculation method for the rotation amount of the two-dimensional part model 60 described in the first embodiment. In the first embodiment, an axis orthogonal to the xy plane is a rotation axis, and the amount of rotation θ around the axis is obtained. At that time, an elliptical long axis representing the part model 60 is used as a reference axis, and the overlapping region is divided into two divided regions A and B by a straight line P passing through the center of gravity g of the overlapping region and orthogonal to the reference axis. The angle formed by the direction in which the center of gravity ga, gb of each of the divided areas A and B and the center of gravity g of the entire overlapping area are aligned and the direction of the reference axis was determined as θ. On the other hand, in this embodiment, each of the x, y, and z axes is selected as a rotation axis, and the amount of rotation in a plane (projection plane) orthogonal to the rotation axis is obtained. At this time, the elliptical long axis obtained by viewing the part model 60 that is an ellipsoid from the direction of the rotation axis in each of the projection planes is used as a reference axis, and the overlapping area is formed on a plane P that passes through the center of gravity g of the overlapping area and is orthogonal to the reference axis Are divided into divided areas A and B, and the directions in which the points ga ′, gb ′, and g ′ are arranged by projecting the centroids ga and gb of the divided areas A and B and the centroid g of the entire overlapping area onto the projection plane. And the angle between the direction of the reference axis and the rotation amount around the selected rotation axis. Here, in particular, as a difference from the first embodiment, in the present embodiment, the overlapping region and the divided region are three-dimensional regions, and are overlapped using the dividing plane P instead of the dividing line P of the first embodiment. It is mentioned that the area is divided, and the center of gravity g and the partial centers of gravity ga and gb are points defined in a three-dimensional space.

  FIG. 9 is a schematic diagram showing the setting rules for the reference axis, the division plane P, and the division areas A and B for each rotation axis when Rx <Ry <Rz. FIG. 10 is a schematic diagram for explaining how to obtain the rotation amount when the x axis is the rotation axis. FIG. 10A is a perspective view of the part model 60 and the division plane P, and FIG. It is a projection view from the x-axis direction. Similarly, FIG. 11 is a schematic diagram for explaining how to obtain the rotation amount when the y-axis is a rotation axis, and FIG. 10 to 12, the display of the person area is omitted, and the overlapping area between the part model 60 and the person area is hatched.

  For example, when the x axis is the rotation axis, as shown in FIG. 10, the movement amount calculating means 42 sets the z axis as the reference axis and sets a plane parallel to the xy plane as the division plane P at a position passing through the center of gravity g. Set. The movement amount calculating means 42 obtains centroids (partial centroids) ga and gb of the divided areas A and B set by the division plane P, and points ga ′ and gb ′ obtained by projecting the partial centroids ga and gb on the yz plane. Ask for. The movement amount calculation means 42 uses a straight line passing through ga ′, g ′, and gb ′ as an approximation of a straight line obtained by projecting the principal axis of inertia of the overlapping region onto the yz plane, and rotates the angle formed by the straight line and the reference z axis. Let it be the quantity θx. For example, let L ′ be a straight line parallel to the reference axis and passing through the center-of-gravity projection point g ′, and a line segment connecting the angles θa, g ′ and gb ′ formed by the line segment connecting g ′ and ga ′ and the straight line L ′. The angle θb formed by the straight line L ′ is obtained, and the average value of θa and θb is defined as the rotation amount θx.

  The calculation method of the rotation amount θy when the movement amount calculation means 42 uses the y axis as the rotation axis and the rotation amount θz when the z axis serves as the rotation axis are as inferred from the calculation method of the rotation amount θx described above. There is no explanation.

  Here, when the x-axis is the rotation axis and the y-axis is the rotation axis, the reference z-axis is the inertia main axis that gives the minimum moment of inertia of the part model 60. Common with form. On the other hand, the y-axis, which is the reference axis when the z-axis is the rotation axis, is the same as in the first embodiment in that it is an elliptical long axis of the cross section of the part model 60 on the xy plane. The axis differs from the first embodiment in that the axis is not the principal axis of inertia that gives the minimum moment of inertia of the part model 60. In the ellipsoid that is the part model 60, the z-axis is the inertia main axis with the minimum moment of inertia, the x-axis is the inertia main axis with the maximum moment of inertia, and the y-axis is the inertia main axis that gives an intermediate moment of inertia. Basically, the rotation amounts θx and θy around the x-axis and the y-axis determine the height direction of the person represented by the part model 60, and the rotation amount θz around the z-axis is before, after, or left and right of the person represented by the part model 60 Determine the direction.

  As described above, the movement amount calculation means 42 corresponds to the reference axis with one of the coordinate axes as a reference axis in each of three planes having two of the three principal axes of inertia of the three-dimensional region model 60 as the coordinate axes. Project the principal axis of inertia of the overlapping area estimated in this way onto the plane, and the component of the rotation amount around the axis perpendicular to the plane depends on the angle between the reference axis and the principal axis of inertia of the overlapping area projected onto the plane To calculate. Thereby, the rotation amount can be calculated deterministically with a small amount of calculation.

  Note that the modified example of the θ calculation method described in the first embodiment can also be applied to this embodiment.

  As described in the first embodiment, this embodiment also has an effect of efficiently searching for a suitable posture with respect to the person region 62 of the part model 60. By the way, it is possible to find a suitable position of the part model using the steepest descent method which is a conventional method for iterative calculation, but this method is based on six degrees of freedom combining translation and rotation in three-dimensional space. For each posture parameter of the part model, the correction amount of the parameter is obtained by obtaining the gradient of the degree of coincidence between the part model and the person region, and the cost of calculating the gradient is high. Since the present invention described in the present embodiment and the first embodiment does not require such high calculation cost gradient calculation, the alignment speed can be improved as compared with the conventional method.

  The arrangement updating unit 43 corrects the position / posture of the part model 60 according to the parallel movement amount and the rotation amount obtained by the movement amount calculation unit 42. Here, as described above, the rotation operation for aligning the principal axis of inertia of the part model 60 with the target axis set in the overlapping region as shown by the straight line ga-gb is the rotation amounts θx, θy, Decomposing into θz can cause errors. That is, even if the rotation amounts θx, θy, and θz are rotated about the x, y, and z axes, the inertia principal axes of the part model 60 generally do not completely coincide with the target axes of the overlapping regions. However, the posture of the part model 60 can be converged in a suitable direction by iterative processing. Note that there may be a difference in the final rotation result depending on the order of the axes to be rotated. Considering this point, the order of the axes to be rotated is kept constant during the iterative process. For example, the rotation is performed in the order of the x axis, the y axis, and the z axis. That is, it first rotates around the x-axis (the y-axis and z-axis of the local coordinate system rotate around the x-axis), and then rotates around the y-axis with respect to the resulting coordinate system (local coordinates The x-axis and z-axis of the system rotate around the y-axis), and the resulting coordinate system rotates around the z-axis (the x- and y-axes of the local coordinate system rotate around the z-axis) To do).

  As for the order of the translation operation and the rotation operation, if the rotation center is the origin (ellipsoid center c) of the local coordinate system, it is corrected to the same position and orientation regardless of which is executed first. .

  FIG. 13 is a flowchart showing a schematic operation of the posture estimation apparatus 1 of the present embodiment. Background image shooting (step S50) and shooting (step S52) are performed for each of the plurality of cameras. Those processes in each camera are basically the same as steps S2 and S4 in the first embodiment.

  In the human region extraction process (step S6) of the first embodiment, a difference image between the monitoring image and the background image is generated and set as a two-dimensional processing target space, whereas the human region extraction of this embodiment is performed. In the processing (step S54), a three-dimensional processing target space is set. Specifically, the object region extraction unit 40 generates a difference image for each camera from the background image captured in step S50 and stored in the storage unit 3, and the monitoring image captured in step S52. A difference image for each camera is back-projected to generate a three-dimensional human region by the view volume intersection method, and a three-dimensional processing target space in which the human region is arranged is set (step S54).

  The alignment apparatus 10 reads the part model information 300 from the storage unit 30 and places the part model 60 represented by the part model information 300 by the initial placement unit 41 at an initial position set based on the person area (step S56). Then, alignment processing (steps S58 to S64) for repetitively updating the arrangement of the part model 60 is performed.

  In the iterative process, the movement amount calculation means 42 calculates the parallel movement amount and the rotation amount of the part model 60 as described above based on the overlapping region of the current part model 60 and the person region (step S58), and updates the arrangement. The means 43 updates the center c and orientation of the part model 60 based on the calculation result of the movement amount calculating means 42 and stores it in the part model information 300 (step S60). In the present embodiment, the orientation of the part model 60 is changed about each axis of xyz by θx, θy, and θz calculated as rotation amounts.

  Each time the update processing unit 43 completes one update process, the iterative control unit 44 determines whether or not the end condition of the iterative process is satisfied (step S62). The processing (steps S58 and S60) by the updating unit 43 is repeated (in the case of “No” in step S64).

When the iterative process is completed (“Yes” in step S64), the iterative control unit 44 calculates the degree of coincidence of the part model 60 with the person region (step S66). For example, the value R _IN obtained by dividing the volume (number of voxels) _VIN of the person region included in the part model 60 by the volume (number of voxels) of the part model 60 can be used as the degree of coincidence. Further, the value R _OUT of the ratio of the remaining volume (V−V _IN ) outside the part model 60 to the entire volume V of the person region can be used as an index value indicating that the smaller the value, the higher the matching degree. . A value indicating the calculated degree of coincidence is stored in the part model information 300.

  The posture estimation means 45 estimates the posture of the person based on the arrangement of the part model 60 that matches the target part of the person region.

  Every time a monitoring image is obtained (step S52), the image processing unit 4 performs the processes of steps S54 to S66 described above.

  In each of the above embodiments, the posture of a person is estimated. However, the object is not limited to a person, but can be applied to various objects including a plurality of parts such as a dog, a cat, and a chair. Further, for example, it is possible to apply not only to posture estimation but also to position detection of a part, such as specifying “the person who possesses the eyelid” as the object and “鞄” as the part and specifying the position of the eyelid.

  DESCRIPTION OF SYMBOLS 1 Posture estimation apparatus, 2 Image pick-up part, 3 Storage part, 4 Image processing part, 5 Output part, 10 Positioning apparatus, 30 Storage means, 40 Object area extraction means, 41 Initial arrangement means, 42 Movement amount calculation means, 43 Arrangement updating means, 44 iteration control means, 45 posture estimation means, 60 part model, 62 person area.

Claims (5)

In a two-dimensional or three-dimensional processing target space, an alignment device that aligns a part model imitating the shape of a part constituting the predetermined object with an object region from which the predetermined object is extracted,
Storage means for storing the shape and inertial principal axis of the part model;
An initial placement means for initially placing the part model at a position where the part model and the object region generate an overlapping area;
Estimating the inertial principal axis of the overlapping area, calculating the amount of rotation of the part model to bring the direction of the inertial principal axis of the part model closer to the inertial principal axis of the overlapping area, and increasing the overlap area A movement amount calculating means for calculating a movement amount;
An arrangement updating means for rotating the part model by the rotation amount and translating by the translation amount;
Repetitive control means for repeating the processing by the movement amount calculating means and the arrangement updating means until a predetermined end condition is satisfied,
An alignment apparatus comprising: The processing target space is three-dimensional,
The movement amount calculating means projects the inertial principal axes of the overlap region onto three planes having two of the three principal axes of the three-dimensional part model as coordinate axes, and each of the three planes An angle between a component of the rotation amount around an axis orthogonal to the plane and an inertia main axis of the region model set on the coordinate axis of the plane and the inertia main axis of the overlapping region projected on the plane To calculate according to
The alignment apparatus according to claim 1.   The movement amount calculation means divides the overlapping region into a plurality of partial regions arranged in the direction of the principal axis of the part model, and is a straight line that approximates the arrangement of the centroids of the partial regions, and calculates the centroid of the overlapping region. The alignment apparatus according to claim 1, wherein a straight line passing through is estimated as an inertia main axis of the overlapping region.   The movement amount calculation means divides the overlapping area into two partial areas by a straight line or a plane orthogonal to the principal axis of inertia of the part model, and a straight line passing through the center of gravity of each of the partial areas and the overlapping area. The alignment apparatus according to claim 1, wherein the alignment apparatus estimates the main axis of inertia. A computer performs processing for aligning a predetermined part model imitating the shape of the part constituting the predetermined object with the target object region in which the predetermined target is extracted in a two-dimensional or three-dimensional processing target space A program for causing the computer to
Initially placing the part model at a position where the part model and the object region generate an overlapping region,
A process of estimating the inertia principal axis of the overlapping region, calculating the amount of rotation of the part model that brings the direction of the inertia principal axis of the region model closer to the inertia principal axis of the overlapping region, and rotating the part model by the amount of rotation; And calculating the amount of translation of the part model for increasing the overlapping region, and repeating the process of translating the part model by the amount of translation until a predetermined end condition is satisfied,
A program characterized by

Images