KR100933297B1 - 3D Coordinate Extraction Method Using Spherical Image from WPS / INS Device and Omnidirectional Camera - Google Patents

Abstract

The present invention relates to a method for extracting three-dimensional terrestrial coordinates from spherical images obtained from GPS / INS equipment and omnidirectional cameras. More particularly, the present invention relates to an image obtained by using an omnidirectional camera and GPS / INS equipment. Acquisition of known point coordinates in spherical images and images using location information and attitude information, and quickly obtain ground coordinates for a specific point in spherical images using spherical-plane transformation model, or conversely, A method for quickly obtaining coordinates in a spherical image.

Method for extracting three-dimensional ground coordinates from the spherical image obtained from the omni-directional camera of the present invention (S10) obtaining the surrounding images from the omnidirectional camera; (S20) obtaining location information and attitude information of the omnidirectional camera from a GPS / INS device; (S30) acquiring a spherical image from the surrounding images obtained from the omnidirectional camera by using the position information and the attitude information acquired from the GPS / INS equipment; (S40) extracting three-dimensional ground coordinates (X, Y, Z) for a specific point on the spherical image by using a [spherical-plane transformation model].

Description

Extraction method of 3-dimensional coordinate from aspherical image acquired by omnidirectical camera and GPS / INS}

The present invention relates to a method for extracting three-dimensional terrestrial coordinates from spherical images obtained from GPS / INS equipment and omnidirectional cameras. More particularly, the present invention relates to an image obtained by using an omnidirectional camera and GPS / INS equipment. Acquisition of known point coordinates in spherical images and images using location information and attitude information, and quickly obtain ground coordinates for a specific point in spherical images using spherical-plane transformation model, or conversely, A method for quickly obtaining coordinates in a spherical image.

There are two methods of extracting the ground coordinates of a specific point from an image using two images of the same region taken from different directions, and using one image and laser scanner data.

In the former case, two images obtained by the camera and the position and attitude information of the camera acquired by the GPS / INS mounted on the moving object (eg, airplane, car, etc.), such as the camera, are applied to the collinear condition equation. Extract the three-dimensional position of the point.

In the latter case, a single image obtained by the camera, position and attitude information of the camera acquired by GPS / INS, and distance information between the camera and the ground obtained by the laser scanner equipment are used to determine a specific point of the image. Extract the 3D position for.

The former method requires two images of the same area taken from different directions, and the latter method requires expensive laser scanner equipment.

For reference, GPS / INS equipment is a combination of GPS that determines three-dimensional position using satellite, INS including inertial measurement device that detects angular velocity motion and acceleration motion and navigation calculator, and various errors of INS sensor itself. And the quantization error and the cumulative error are corrected through GPS to provide absolute three-dimensional position information and attitude information of the moving object equipped with GPS / INS.

As described above, the present invention requires two or more images of the same area taken from different directions in order to obtain three-dimensional ground coordinates of a specific point in the image, or does not require an expensive laser scanner. An object of the present invention is to provide a method of extracting three-dimensional ground coordinates from a spherical image obtained from an omnidirectional camera capable of quickly and easily extracting three-dimensional ground coordinates of a specific point in a spherical image.

3D coordinate extraction method using a spherical image obtained from the GPS / INS equipment and the omnidirectional camera of the present invention for achieving the above object

(S10) acquiring surrounding images from the omnidirectional camera;

(S20) obtaining location information and attitude information of the omnidirectional camera from a GPS / INS device;

(S30) acquiring a spherical image from the surrounding images obtained from the omnidirectional camera by using the position information and the attitude information acquired from the GPS / INS equipment;

(S40) extracting three-dimensional ground coordinates (X, Y, Z) for a specific point on the spherical image by using a [spherical-plane transformation model].

And the (S40) step

(S41) By applying the Z0 value at the coordinates (X0, Y0, Z0) of the known point in the spherical image and the φ value and λ value, which are the pan and tilt angles in the spherical image at the specific point, are applied to the [Spherical-Plane Transformation Model]. Obtaining coordinates X and Y values for the point,

(S46) Assuming that the change of the feature from the known point to the specific point in the spherical image is changed only horizontally and vertically, the altitude value Z of the specific point is obtained using the image change characteristic from the known point to the specific point. Characterized in that it comprises a step,

The spherical-plane transformation model

X = N, cosφ, cosλ

Y = N, cosφ, sinλ

Z = N · sinφ,

Here, (X, Y, Z) is the three-dimensional ground coordinates of a specific point to be obtained,

(φ, λ) are pan and tilt values in the spherical image,

N is the sphere radius of the spherical image.

The method of extracting three-dimensional ground coordinates from the spherical image obtained from the omnidirectional camera according to the present invention having the above configuration has a single spherical image without the need for two or more images to extract the three-dimensional position information from the photographed image. It is sufficient, and the extraction of the three-dimensional positional information is also performed quickly.

Hereinafter, with reference to the drawings will be described the present invention in more detail.

1 is a flowchart illustrating a method of extracting 3D ground coordinates from a spherical image obtained from an omnidirectional camera according to the present invention. Obtaining the information and the attitude information (S20), obtaining the spherical image (S30), and extracting the three-dimensional ground coordinates of the specific point from the spherical image (S40).

In step S10, the surrounding image is acquired from the omnidirectional camera. The omnidirectional camera is a camera for capturing an image of the surrounding omnidirectional (front, rear, left and right directions) at a time, and as shown in FIG. 2, a plurality of camera modules are arranged in a spherical housing so as to face all directions. It is.

In step S20, position information and attitude information of the GPS / INS device are acquired from the GPS / INS device.

The omnidirectional camera and the GPS / INS equipment are mounted together in a moving object such as a car, and continuously acquire surrounding images, position information, and attitude information as the moving object moves. The location information and attitude information acquired by the GPS / INS device are the location information and the attitude information of the GPS / INS device itself. However, since the omnidirectional camera and the GPS / INS device are mounted on the same moving object, when the error is corrected, Posture information or moving object position information and posture information.

In step S30, the spherical image is acquired from the surrounding images photographed by the omnidirectional camera using the position information and the attitude information in the previous step.

Since a method for acquiring spherical images from surrounding images captured by the omnidirectional camera is a well-known technique, a detailed description thereof will be omitted.

A spherical image is generated for each position of the omnidirectional camera, and FIG. 2 is an example of a spherical image obtained from surrounding images taken from the omnidirectional camera, and is a spherical image in which the omnidirectional camera is located in the center of FIG.

In step S40, a three-dimensional ground coordinate of a specific point is extracted from the spherical image using the [spherical-plane conversion model].

[Sphere-Plane Transformation Model] shows the relationship when the sphere is unfolded as shown in Fig. 3, which is the radius (N), the pan value (φ) and the tilt value (λ), which are the coordinates of the sphere. It is a model for obtaining three-dimensional ground coordinates (X, Y, Z), which are rectangular coordinates.

And the [sphere-plane transformation model] representing these relations, that is, the relationship between the coordinates of the sphere and the rectangular coordinates of the plane, is expressed by the following equation.

X = N, cosφ, cosλ

Y = N, cosφ, sinλ

Z = N

or,

φ = arctan (Z / P)

λ = arctan (Y / Z)

Where P = (X ^ 2 + Y ^ 2) ^ (1/2).

In the spherical image, the radius N of the sphere is unknown. That is, it is not possible to determine whether to make the ground of the three-dimensional ground sphere, the building the sphere, or the sky the sphere in the spherical image photographed in three dimensions. In addition, in the present invention, since the N value is not directly used to extract the ground coordinates, it is not meaningful to set the spherical reference.

In extracting the ground coordinates from the spherical image, since N is not directly used, the equation of the [spherical-plane conversion model] is modified as follows.

Z = N · sinφ ☞ N = Z / sinφ

X = Z / sinφ · cosφ · cosλ

Y = Z / sin φ cos φ sin λ

As can be seen from the modified equation, the ground coordinates X and Y for a specific point in the spherical image are represented by the pan value φ, tilt value λ, and Z value of the specific point in the spherical image.

The pan value φ and the tilt value λ are values for the coordinates of the selected point (i.e. specific point) in the spherical image, and can be obtained immediately by selecting any point in the image. Therefore, the Z value of the selected point is known or specified. Assuming, we can get X and Y values.

In other words, if you already know the Z value of the selected point, use [Sphere-Plane Transformation Model] to find the X, Y value of the selected point.

If the Z value of the selected point is not known, the Z value of the selected point is assumed and then the X and Y values of the selected point are obtained using the [Spherical-Plane Transformation Model]. At this time, the Z value is assumed to be the Z value of the known point known in the spherical image, and the X and Y values obtained using the [Spherical-Plane Transformation Model] are moved to the vertical direction from the selected point (that is, the known point). X and Y of the point having the same altitude (Z) and are equal to each other. In this step, we want to find the X and Y values of the three-dimensional ground coordinates, and the Z values of the three-dimensional ground coordinates are obtained in the other steps. It can be.

And since the moving object is equipped with GPS / INS equipment, the altitude value of the camera can be obtained by using the position and attitude information acquired by the GPS / INS equipment.If the installation height of the camera is subtracted from the road surface, the Z value of the road surface is obtained. You can get it. Since the road surface Z value of this specific point is acquired at the same time as the image acquisition, it is always present and available in the spherical image, so it is based on the road surface of the path traveled by the moving object when obtaining ground coordinates X and Y values of the selected point. To estimate the Z value.

For reference, the coordinates X, Y, and Z obtained in this step S41 are coordinates based on the spherical image, and are not absolute ground coordinates based on the earth ellipsoid. However, when the rotation and movement transformation is performed using the absolute coordinates and attitude information of the camera obtained from GPS / INS, the coordinates based on the spherical image are easily converted to the absolute ground coordinates based on the ellipsoid of the earth. In addition, since rotation and movement transformations used herein correspond to well-known and common techniques well known in the art to which the present invention belongs, the coordinates X, Y, and Z obtained in this step (S41) and the absolute ground coordinates based on the ellipsoid of the earth are There is no problem even if identified. Therefore, in describing the present invention, the coordinates X, Y, and Z (particularly X, Y) obtained in step S41 have been expressed in ground coordinates.

In this way, after obtaining the ground coordinates X and Y values of a specific point (φ, λ) on the spherical image by using the known Z value of the known point in the spherical image (S41), the characteristics of the ground feature change are used. Obtain the ground coordinate Z of the specific point (S46).

Most of the land we live on is flat, and most of the features on the ground are vertical.

Therefore, after adding the assumption that the ground feature changes only horizontally and vertically, and then connecting the virtual horizontal line and vertical line continuous from the known point of the spherical image to a specific point, the specific point of the spherical image from the sum of the lengths of the vertical lines. We can get ground coordinate Z for.

For reference, the distinction between features in the image and the boundary between the horizontal and vertical portions in the same feature can be easily distinguished by the color and contrast of the image.

That is, when the bar and the image shown in FIG. 5 include roads, sidewalks, buildings and power poles on the sidewalks, and the user selects a specific point of the power pole,

It connects a virtual horizontal line and a vertical line to a specific point selected by using a point as a known point on the road surface where the camera-mounted moving object passed. That is, the first horizontal line r1 is connected from the known point to the end of the road surface, the first vertical line r2 is connected to the upper side of the sidewalk, and the second horizontal line r3 is connected from the upper side of the sidewalk to the lower side of the power pole, The second vertical line r4 is connected to the selected specific point.

That is, only the X and Y values of the ground coordinates and the X and Y values of the ground coordinates change on the horizontal line, only the Z values of the Z axis of the ground coordinates change on the vertical line, and the ground coordinates of the known point (assuming (X0, YO, Z0)) are already As you know, the ground coordinates (X1, Y1, Z0) of the point where the first horizontal line (r1) and the first vertical line (r2) meet, the ground coordinates of the point where the first vertical line (r2) and the second horizontal line (r3) meet ( X1, Y1, Z1), the ground coordinates (X2, Y2, Z1) at the point where the second horizontal line r3 and the second vertical line r4 meet, and the ground coordinates (X2, Y2, Z2) at the selected point are sequentially obtained. Can be.

And the known point may be preferable to reduce the error to select the nearest point to the selection point.

Through this process, the process of obtaining ground coordinates X, Y, and Z for any one point of the selected object on the spherical image is briefly described again.

After the omnidirectional camera and GPS / INS equipment are mounted on a mobile object such as a car, the moving object acquires the absolute coordinates and posture information of the surrounding images and the camera that captured the images in real time. Spherical images are obtained from these data acquired in real time. Here, the spherical image is acquired along with the absolute coordinates and the attitude information of the camera according to the position of the moving object at each moment.

After the spherical image is acquired, if you click (select) any point of interest in the spherical image, the coordinates of the clicked point (that is, the angle represented by pan and tilt) are extracted directly from the spherical image. The sphere radius N is also extracted directly from the known Z of the replacement point. In the spherical image, the known point becomes the ground of the moving path of the moving object equipped with GPS / INS equipment. That is, the absolute coordinates (Xa, Ya, Za) of the GPS / INS equipment are acquired by the GPS / INS equipment in real time, and the installation height H of the GPS / INS equipment is already known in the moving object, so that the ground (ie ) Z value is Za-H.

When you click (select) a point in the spherical image, the pan and tilt values of the spherical image and the Z value of the known point that replaces the spherical radius of the spherical image are extracted immediately. ] Is applied to obtain the X and Y values of the relative coordinates of the selected point in the spherical image.

As mentioned above, these X, Y, and Z are relative positions from the camera (i.e., X, Y, and Z are coordinates based on the spherical image, which indicates the direction and distance from the camera). Based on the absolute position of the camera obtained from the INS, it is obtained by the absolute ground coordinates of the selected point through a rotation transformation using the camera's detailed information.

After obtaining the absolute coordinate value of the clicked point in the spherical image, the coordinate values are obtained by using the image change characteristic of the location of various facilities of the image. As shown in Fig. 5, when the point to acquire the position is any point on the power pole, the known point in the spherical image is on the road which is the ground which is the path through which the moving object passed. In the drawing, the manhole cover on the road can be one of the known points. In the manhole cover of the known point, connect the virtual straight lines to a point on the power pole at the point where the position is to be acquired. The virtual straight lines are assumed to be changed only in the horizontal and vertical directions in the image feature. . This is an assumption that can be recognized because most of the features on the ground are installed vertically.

We can then find the change in the coordinates (X, Y, Z) of the horizontal and vertical lines, a virtual straight line connecting the known point on the manhole to the pick point on the pole. In other words, the Z value does not change on the horizontal line, only the Z value changes on the vertical line, and we finally obtain the position value that we want to obtain.

And, if the height r2 of India is known in Fig. 5, the Z value of the reference point applied to the [sphere-plane transformation model] is used instead of the Z value of the road surface, and r3 and r4 If you select the meeting point with a single click, you can obtain the three-dimensional coordinates of the power pole through the S41 process. Of course, the coordinates acquired at this time are the coordinates of the point where the power pole meets the sidewalk, and when selecting any point on the power pole as shown in FIG. 5, the height (Z value) of the selected point must be acquired through step S46. .

In the above description of the present invention, a method of extracting three-dimensional coordinates using spherical images obtained from a GPS / INS device and a omnidirectional camera having a specific procedure with reference to the accompanying drawings has been described. It is to be understood that such modifications and variations are intended to be included within the protection scope of the present invention.

1 is a procedure of a method for extracting three-dimensional ground coordinates from spherical images obtained from the omnidirectional camera according to the present invention.

2 is an example of a spherical image.

3 is a perspective view of the omnidirectional camera according to an example.

4 is an example of spreading a sphere in a plane to explain the [sphere-plane conversion model].

5 is a view for explaining a method of extracting the ground coordinate Z;

Claims (3)

(S10) acquiring surrounding images from the omnidirectional camera; (S20) obtaining location information and attitude information of the omnidirectional camera from a GPS / INS device; (S30) acquiring a spherical image from the surrounding images obtained from the omnidirectional camera by using the position information and the attitude information acquired from the GPS / INS equipment; (S40) extracting three-dimensional ground coordinates (X, Y, Z) for a specific point on the spherical image by using a [spherical-plane transformation model]. The step (S40) is (S41) By applying the Z0 value at the coordinates (X0, Y0, Z0) of the known point in the spherical image and the φ value and λ value, which are the pan and tilt angles in the spherical image at the specific point, are applied to the [Spherical-Plane Transformation Model]. Obtaining coordinates X and Y values for the point, (S46) Under the assumption that the change of the feature from the known point to the specific point in the spherical image is changed only horizontally and vertically, By using the image change characteristics (that is, the difference between the color and the contrast of the image) from the known point to the feature, the features are distinguished from each other in the image, and the boundaries of the horizontal and vertical parts in the same feature, And a step of obtaining an altitude value Z for a specific point of the spherical image from the sum of the lengths of the vertical lines after connecting the separated horizontal feature and the boundary from the known point of the image to the specific horizontal horizontal line and the vertical line. 3D coordinate extraction method using spherical images acquired from a GPS / INS device and an omnidirectional camera. delete The method of claim 1, wherein the [sphere-plane transformation model] X = N, cosφ, cosλ Y = N, cosφ, sinλ Z = N · sinφ, Here, (X, Y, Z) is the three-dimensional ground coordinates of a specific point to be obtained, (φ, λ) is the pan and tilt angle in the spherical image, N is a three-dimensional coordinate extraction method using a spherical image obtained from the GPS / INS equipment and the omnidirectional camera, characterized in that the sphere radius of the spherical image.

Images