JP2511082B2 - Three-dimensional position measuring device for moving objects - Google Patents

Description

Description: TECHNICAL FIELD The present invention relates to a three-dimensional position measuring apparatus for a moving object, and more particularly to a measuring apparatus suitable for measuring the three-dimensional position of a plurality of objects.

[Conventional technology]

As a conventional example, there is Japanese Patent Application Laid-Open No. 62-159050 "Movement velocity measuring device".

This conventional example is a TV that captures images of moving objects and backgrounds.
It is composed of a camera (imaging device), a creating device that takes in an output video signal of the camera to obtain a trajectory of a moving object, and an image analyzing device that analyzes the created trajectory to obtain the velocity of the moving object.

In order to remove the background from the video signal of the TV camera, only the background is extracted and the background is removed from the video signal. Further, in order to obtain the velocity of the moving object, the starting point and the ending point of the moving object are obtained from the trajectory of the moving object, and the velocity is calculated from the starting point and the ending point.

[Problems to be solved by the invention]

JP-A-62-159050 is suitable for determining the velocity of a moving object when the moving object is single.

However, when there are a plurality of moving objects and each of them has its own movement, it is impossible to detect the moving speed of each of the plurality of moving objects. In addition, it is impossible to specify the three-dimensional position of each moving object instead of the moving speed.
In particular, when individual moving objects cannot be identified, from the viewpoint of image processing, the images in the depth direction overlap with each other, and the loci of movement are merged, or a plurality of moving objects within a short distance from each other are combined. When moving images, there is a problem that it is difficult to distinguish between images.

An object of the present invention is to provide a three-dimensional position measuring device capable of measuring the three-dimensional position of each moving object when there are a plurality of moving objects.

Incidentally, there is Japanese Patent Application No. 62-79656 as a prior application related to the present invention.

[Means for solving problems]

The above-mentioned object is to use a plurality of image pickup means, and take captured images of a plurality of moving objects momentarily from each image pickup means to analyze the trajectory of each moving object to measure the three-dimensional position of each moving object. In the three-dimensional position measuring apparatus, the identification number assigning means for assigning the identification number corresponding to the capture time (i = 1 to n) to each moving object image in each captured image captured for each image capturing means every moment. Next capture time i corresponding to a certain moving object image to which the corresponding identification number of capture time i is assigned
Capture time i + of the moving object image at +1 for each imaging means
A locus search means for searching among a plurality of moving object images to which an identification number corresponding to 1 is assigned, a two-dimensional locus of each moving object and a position of each imaging means obtained for each imaging means by the locus searching means. This is achieved by providing means for calculating the three-dimensional position of each moving object from the information.

[Action]

In order to obtain the trajectories of a large number of moving objects, the problem is how to associate the images of the moving objects before and after the movement. According to the present invention, an identification number is assigned according to the image capture time, and the image at the next time i + 1 of a moving object at time i is searched from the images assigned the identification number i + 1. Is shortened and the search accuracy is improved. Therefore, even if the trajectories of the moving objects intersect three-dimensionally, the three-dimensional trajectory of each moving object can be obtained with high accuracy.

〔Example〕

FIG. 1 shows an embodiment of the measuring device of the present invention. In the present embodiment, it is assumed that there are three sets of imaging systems # 1, # 2, and # 3, and a plurality of moving objects in a three-dimensional space are imaged from different three directions at the same timing. Three imaging systems # 1, # 2, #
3 has the same configuration and is as follows.

Each image pickup system includes an image pickup device 1 for capturing an image of a moving object 1, a positioning device 24 for the image pickup device, an A / D converter 2, an address generator 3,
It comprises an image memory 4.

An image analysis device 5, a display 6, a plotter 7, a printer 8, and a disk memory 9 are common to all imaging systems.

The operation is as follows. First, the moving object is imaged using the three imaging devices 1 at the same timing. The imaging signal of this moving object is an image of the moving object at the same time,
This signal is digitalized by the A / D converter 2, and the address generator 3 captures a moving object at an imaging timing (time) i (i).
= 1 to n), and an image is stored in the image memory 4 according to this address. Therefore, n
In the individual image memory 4, the picked-up images for each time are stored corresponding to the time. For example, the image at i = 1 is stored in the first memory of the memory 4.

Next, the momentary images of the moving object stored in the image memory 4 are sent to the image analysis device 5 through the line 25.
Further, information on the directional and optical positions of the image pickup device such as the angle, position, and focal length when the moving object is shot by the image pickup device 1 is also sent to the image analysis device 5 via the line 26.

Next, an image analysis method in the image analysis device 5 will be described. First, a method of obtaining a trajectory parameter, which is information on the movement of a moving object, from momentary images of the moving object captured by the three imaging devices 1 # 1 to # 3 will be described below by taking an image of one imaging device as an example. To do. In the above embodiment using three image pickup devices, the trajectory parameter is obtained for each image pickup device. The calculation method of these three locus parameters is the same method.

FIG. 2 shows, as an example, an image 27 at each time when measured from time i = 1 to i = 4 (that is, n = 4) when three spherical moving objects A, B, and C are present. . The result of obtaining the barycentric coordinate 28 for the moving object from time i = 1 to 4 is the third
Shown in the figure. This barycentric coordinate 28 can be calculated by image processing. The processing method is expressed by equations (1) and (2).

Here, (X _ik , Y _ik ) is the coordinate value of N _i pixels k constituting the image of the moving object at time i, and (X _gi ,
Y _gi ) is the barycentric coordinate of the moving object at time i. The movement information is obtained using the barycentric coordinates of the moving object at each time in FIG. For the sake of explanation, FIG. 4 (a) shows the positions of the barycentric coordinates of the moving object at each time in FIG. 3 collectively in one drawing. A method of obtaining the movement information will be described with reference to FIG. 4 (b) in the case where the respective barycentric coordinates have the relationship shown in FIG. 4 (a). Coordinate information Ai, Bi, Ci (i = 1 to 4) indicating the existence positions of three moving objects
Before associating the coordinate information of each time as the moving objects A, B, and C, there is no distinction other than the subscript i representing the time. Now, a case will be described as an example in which the moving object A is associated with an image of the moving object A, assuming that the moving object does not suddenly accelerate or change its direction.

(I) Time i = 2, which is the shortest distance from point A _{1 at} time i = 1
To search for points. C ₂ and A ₂ are listed as search candidates. Point C ₂
Search for a neighborhood (small radius ε) distant by a vector ▲ ▼ from. According to the result of this search, time i =
Point 3 does not exist in a circle with a small radius ε. Here, the distance from point C ₂ to vector ▲ ▼ is the point C ₂
From ▲ ▼ and the direction is ▲
It is said that it is the same as the vector direction of ▼. Therefore,
As shown in Fig. 5 (b), the vector ▲ from point C ₂
Position P ₁ of ▼ is ▲ ▼ on the straight direction of ▲ ▼
Position.

(Ii) Next, a point separated from the point A ₂ by the vector ▲ ▼
Search for a point at time i = 3 near P ₂ . As a result of this search, the point A ₃ is found in the circle with the small radius ε.

The same operation can be associated with the image of the moving object A by repeating the search with respect to the point P ₃ for i = 4. When n> 4, the same process may be repeated. By similarly searching for each of the moving objects B and C, the image B of the same moving object at each time can be obtained as B ₁ → B ₂ → B ₃ → B ₄ , and C C ₁ → C ₂ → C ₃ → C ₄ can be associated with the image of another moving object without error.

As described above, the trajectory parameter, which is information on the movement of the object, can be obtained from the image of the moving object captured by the imaging device 1 by the method described above.

Thus # 1, # 2, and # 3, respectively trajectory parameter _{# 1T (A 1 ~A 4,} B 1 ~B 4, C 1 ~C 4), # 2T (A 1 ~A 4, B 1
_{~B 4, C 1 ~C 4)} , # 3T (A 1 ~A 4, B 1 ~B 4, C 1 ~C 4) is obtained. These three trajectory parameters # 1T to # 3T are trajectory parameters in a two-dimensional image, and it is necessary to obtain the three-dimensional moving object position from this.

The method of calculating the three-dimensional position of this three-dimensional moving object will be described in detail below. First, from the three trajectory parameters # 1T to # 3T,
The trajectory parameters of the objects A, B, C are extracted.

Locus parameter # 1T for the object _{A (A 1 ~A 4),} # 2T (A 1 ~A 4) # 3T (A 1 ~A 4) trajectory parameters # 1T (B ₁ ~B ₄₎ for the object B, _{# 2T (B 1 ~B 4)} # 3T (B 1 ~B 4) trajectory parameters # 1T (C ₁ ~C ₄₎ for the object _{C, # 2T (C 1 ~C} 4) # 3T (C 1 ~C ₄ ) Next, for each object, the three-dimensional position is calculated from the trajectory parameters in # 1 to # 3 and the position information of the image pickup device. The method of triangulation is used as a method for obtaining this three-dimensional position.

Figure 5 shows the principle of triangulation. Imaging systems # 1 and # 2
In the coordinate values of the three-dimensional photographing an object P in P (x, y, z) , respectively on the two-dimensional projection plane _{29,30 (x a, y a)} , photographed as (x _b, y _b) However, a two-dimensional projection plane that is determined by the geometrical and optical position of the imaging device such as position, angle, focal length, etc.
When the positional relationship between 29, 30 and the object P is as shown in FIG. 5, P (x, y, z) is expressed by the following equation.

The three-dimensional coordinates like the above equation can be obtained when the same object is associated with a plurality of two-dimensional images. FIG. 6 shows the principle of this associating method when three image pickup devices are used. That is, the moving object P32 is photographed by three image pickup devices, and the two-dimensional images 29, 30, 31 are taken.
Respectively, and if the images of P on these two-dimensional images are P ₁ , P ₂ and P ₃ and the focal points are f ₁ , f ₂ and f ₃ , respectively, P ₁ is
A straight line P ₃ ′ obtained by projecting the straight lines P ₃ f ₃ and P ₂ f ₂ onto the two-dimensional image 29
It exists at the intersection of f ₃ ′ and P ₂ ′ f ₂ ′. Similarly, P
₂ is a straight line obtained by capturing the straight lines P ₁ f ₁ and P ₃ f ₃ into the two-dimensional image 30.
On the intersection of P ₁ ″ f ₁ ″ and P ₃ ″ f ₃ ″, P ₃ is a straight line P ₁ f ₁ and P ₂ f ₂
Are present on the intersections of the straight lines P ₁ f ₁ and P ₂ f ₂ projected on the two-dimensional image 31.

However, in actual measurement, the points corresponding to the geometrical and optical position errors of the image pickup device and the errors due to the image resolution do not always exist at each of the above-mentioned intersections, and they correspond within the error range. In addition, it is necessary to obtain the three-dimensional position. Seventh
FIG. 7A shows a two-dimensional image 29 of a moving object at a certain time (i = 4), and _{three three-} dimensional images 29 near the intersection b of the straight lines P ₂ ′ f ₂ ′ and P ₃ ′ f ₃ ′. It is assumed that the candidate points b ₁ , b ₂ , b ₃ are found. Now, assuming that the three-dimensional position of the moving object B is obtained, the movement information of the moving object B is as shown in FIG. 7 (b).
B _{1 to} B _5, and b _{2 in} FIG. 7 (a) corresponds to B ₄ ,
It can be confirmed that it is the object B.

If there is an error as described above, the imaging devices # 1 and #
The three-dimensional position of the moving object obtained by the principle of triangulation using the images # 2, # 2 and # 3, # 3 and # 1 respectively differs within the error range, so operations such as averaging are required. Becomes

In addition, as an example of the image pickup apparatus 1 used in the above-described embodiment,
Examples include imaging devices that use electromagnetic waves and ultrasonic waves such as television cameras and radar, and photographers.

Another embodiment is shown in FIG. The image memory 4 of the above-described embodiment is replaced with the image storage medium 41, and the others are the same. Examples of image storage media include the use of photographs, films, video recorders, optical discs and the like.

Next, FIG. 9 shows a flow chart of an embodiment in which the present invention is implemented by software. First, the measurement time n is input. Next, input the image of the imaging device, digitize it,
The image data is stored in the array corresponding to the time i. This process is repeated for the measurement time n, and the image data at each time is collected. After the collection is completed, the information on the geometrical and optical position of the image pickup device is fetched. Next, the momentary images of the moving objects stored in the array are analyzed to analyze the movement information of the objects.
Then, the three-dimensional position of the object is estimated from the geometrical and optical positional relationships such as the angle and focal length of the imaging device and the position of the object on the image. The estimated position is compared with the previously analyzed information on the movement of the object and checked to output the three-dimensional position.

FIG. 10 shows an embodiment using RGB (red, green, blue) color signals. A color image memory in which the image pickup signals of the three image pickup devices are made to correspond to the R (red), G (green), and B (blue) signals of the color image, respectively.
It is stored in 42 and the three-dimensional position is obtained by the image analysis device 5. In this embodiment, RGB signal 43 and color image memory 4
By using 2, the images from the three image pickup devices are
It can be processed by the / D converter 2, the address generator 3, and the color image memory 42.

FIG. 11 shows another embodiment. This embodiment comprises three image pickup devices 1, a positioning device 24 for the image pickup devices, an image analysis device 5, and a display 6, a plotter 7, a printer 8, a disk 9 for outputting and recording the analysis result. The three imaging devices 1 capture an image of a moving object. Image analysis device 5
Synchronizes with the image pickup apparatus 1 and takes in the image data 25 and the geometrical and optical position information 26 of the image pickup apparatus, analyzes the movement information of the object, estimates the three-dimensional position, and compares it with the movement information. , Find the three-dimensional position. In the present embodiment, the image analysis apparatus 5 outputs a three-dimensional position by performing analysis without delay in synchronism with the image input speed of the image pickup apparatus 1, so that a medium for storing the momentary image input by the image pickup apparatus 1 is stored. Is unnecessary.

Although the center of gravity is used as the trajectory parameter in the above embodiments, feature points other than the center of gravity may be used as the trajectory parameter.

〔The invention's effect〕

According to the present invention, the same moving object in each two-dimensional image estimated from the two-dimensional images at a certain time captured by a plurality of image capturing devices using information on the geometrical and optical positions of the respective image capturing devices. By using the movement of the moving object in the image considered to be, it is possible to confirm whether or not the moving object is the same even before and after the time. As a result, even when there are multiple moving objects and the images overlap in the depth direction, or when there are other moving objects at close positions, the moving object images can be accurately displayed without being confused with the images of other moving objects. Can measure three-dimensional position. In particular, it is effective for measurement when a plurality of moving objects make complicated three-dimensional movements.

[Brief description of drawings]

FIG. 1 is a diagram showing an embodiment of the measuring apparatus of the present invention, FIG. 2 is a view showing images taken by the imaging systems # 1, # 2, and # 3, and FIG. 3 is a barycentric position extracted from the images taken in FIG. Showing the movement of the 4th, 4th
The figure shows an example of calculation of trajectory parameters from the position of the center of gravity, FIG. 5 is an explanatory diagram of the principle of triangulation, and FIG. 6 is three-dimensional from trajectory parameters in three imaging systems # 1, # 2, and # 3 of the present invention. FIG. 7 is an explanatory view for specifying the position, FIG. 7 is another explanatory view for specifying the tertiary position, and FIG. 8 is another embodiment of the measuring device of the present invention.
FIG. 9 is an embodiment diagram of the processing flow of the present invention, and FIGS. 10 and 11 are other embodiments of the measuring apparatus of the present invention. # 1, # 2, # 3 ... First, second and third imaging systems.

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Kotaro Inoue 1168 Moriyama-cho, Hitachi, Hitachi, Ibaraki Energy Research Laboratory, Hitachi, Ltd. (56) References JP-A-60-15507 (JP, A) JP-A-61- 120913 (JP, A)

Claims (2)

(57) [Claims] 1. A moving object which uses a plurality of image pickup means and takes captured images of a plurality of moving objects from each image pickup means every moment to analyze the trajectory of each moving object to measure the three-dimensional position of each moving object. In the three-dimensional position measuring apparatus, the identification number assigning means for assigning the identification number corresponding to the capture time (i = 1 to n) to each moving object image in each captured image captured for each image capturing means every moment. Next capture time i + corresponding to a certain moving object image to which an identification number corresponding to capture time i is assigned
Captured time i + 1 for each image pickup means
Locus search means for searching among a plurality of moving object images to which the identification numbers corresponding to the moving object images are searched, and the two-dimensional locus of each moving object and the position information of each image pickup means obtained for each imaging means by the locus search means. And a means for calculating the three-dimensional position of each moving object from the above. 2. The locus search means obtains a position change from the position of time i-1 to the position of time i of the moving object image to be searched, and extends the position change to the position change from the position of time i. The three-dimensional position measuring apparatus for a moving object according to claim 1, wherein a moving object image in the vicinity of the position is identified as a moving object image at time i + 1 of the search target moving object image.