KR100797391B1 - Camera position and attitude information correction method and device - Google Patents

Abstract

According to the present invention, in the operation of a camera made to obtain position and attitude information from a satellite navigation device and an inertial navigation device, it is possible to precisely correct the position and attitude information of a camera by analyzing image information obtained through a three-wire scanner attached to the camera. And a method and apparatus for enabling the same.

The present invention provides a method for calibrating position and attitude information of a camera, comprising: calculating a position of the camera using a satellite navigation apparatus; Calculating the pose of the camera using an inertial navigation device; Generating position and posture correction information of the camera by analyzing an image input through a three-wire scanner mounted to the camera; And feeding back the generated position and attitude correction information to the satellite navigation device and the inertial navigation device, respectively. The image input through the three-wire scanner is composed of a front scanner observation image, a vertical scanner observation image and a rear scanner observation image.

The present invention enables to accurately determine the exterior orientation parameters of a camera without using a ground control point for an image observed through a camera mounted on a vehicle such as a satellite or an aircraft.

In addition, the present invention has an effect that allows to determine the position and attitude information of the camera more precisely for a shorter time interval.

 Camera, three-wire scanner, position, posture, external expression

Description

Method and Apparatus for Correcting Position and Attitude Information of Camera {Method and Apparatus for Correcting Position And Attitude Information of Camera}

1 is a system configuration to which the image-based position and attitude correction apparatus according to the present invention is applied;

2 is a diagram showing a relationship between images acquired through a three-wire scanner;

3 is a flowchart illustrating a process of generating position and attitude correction information according to the present invention; And

Figure 4 is a detailed block diagram of a Kalman filter processing unit according to the present invention.

The present invention relates to a method and apparatus for calibrating position and attitude information of a camera. More specifically, the present invention relates to a position and attitude from a global navigation satellite system (GNSS) and an inertial navigation system (INS). A method and apparatus for precisely correcting the position and attitude of a camera by analyzing image information obtained through a three-line scanner attached to the camera in operating a camera made to obtain information. will be.

There are three ways to determine the position and attitude of observation devices such as satellites and aircraft or cameras mounted on them. One method is to determine the position and attitude information of the observation device by combining the information from satellite and inertial navigation devices. The other method is to determine the position and attitude of the observing device by analyzing signals from a plurality of GPS satellites using a GPS (Global Positioning System), which is a type of satellite navigation device, in the absence of an inertial navigation device. Another method is to generate 3D information of an object by analyzing an image input through a stereo camera equipped with a satellite navigation device and an inertial navigation device.

A technique for combining a satellite navigation device and an inertial navigation device to determine the position and attitude of a vehicle and indicating the current position of the vehicle through map matching on the result is disclosed in Korean Patent Application No. 1997-41019, "Hybrid Navigation System". Is disclosed.

Technology for determining position and attitude in mobile communication by combining information input from GPS sensor, inertial navigation system sensor and dead-reckoning (DR) sensor is disclosed in Korean Patent Application No. 2000-85817, "Moving using GPS and INS. Position and attitude determination system in a communication system.

A technique for acquiring three-dimensional position and attribute information of an object using a stereo image acquired through a digital video camera installed in a vehicle equipped with a satellite navigation device and an inertial navigation device is disclosed in Korean Patent Application No. 2001-41187, "3 Dimensional position measuring apparatus and method thereof.

Korean patent application No. 2002-62417, "Efficient Storage and Retrieval of Spatial Images," describes a technique for generating information by applying internal and external facial expressions to images acquired using a satellite navigation device, an inertial navigation device and a CCD camera. Spatial image information system and its retrieval method for support ".

A technique for predicting and checking the position and attitude of satellites by analyzing signals from a plurality of GPS satellites and information obtained through a star sensor (Star Sensor or Star Tracker) is disclosed in US Patent Publication No. 2002/0004691, "Attitude Determination and Alignment Using Electro-optical Sensors and Global Navugation Satellites.

A technique for determining the position and attitude of a vehicle by analyzing signals from a plurality of GPS satellites is disclosed in US Pat. No. 6580389, "Attitude Determination Using A Global Positioning System."

However, the conventional technique is a method that can determine the position and attitude of the observation apparatus using only the information obtained from the satellite navigation system or simultaneously using the satellite navigation system and the inertial navigation system, and the precision of the position information is the information obtained from the satellite navigation system. The attitude information is determined according to the accuracy of and the accuracy of the attitude information is determined by the accuracy of the information obtained from the inertial navigation apparatus.

Therefore, in the process of obtaining the position information, there is a problem in that the position information cannot be obtained for a time interval shorter than a predetermined time interval limited by the satellite navigation apparatus. For example, in the case of GPS, since GPS signals are typically provided at one second intervals, precise location information cannot be calculated for a shorter time interval.

In addition, even in the process of obtaining attitude information, there is a problem in that attitude information cannot be obtained for a time interval shorter than the time interval limited by the inertial navigation apparatus.

In addition, the prior art has a problem that the accuracy of the position and attitude depends only on the performance of the associated device, and does not utilize additional information from image information obtained from a camera, which is an observation device connected to the satellite navigation device and the inertial navigation device.

The present invention is to solve the above problems, a method for accurately determining the position and attitude information of the camera using the scan line image information obtained through the three-line scanner of the camera equipped with the satellite navigation device and the inertial navigation device And to provide a device.

Other objects and advantages of the present invention can be understood by the following description, and will be more clearly understood by the embodiments of the present invention. Also, it will be readily appreciated that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the claims.

According to an aspect of the present invention, there is provided a method of correcting a position and attitude information of a camera, the method comprising: calculating a position of the camera using a satellite navigation apparatus; Calculating the pose of the camera using an inertial navigation device; Generating position and posture correction information of the camera by analyzing an image input through a three-wire scanner mounted to the camera; And feeding back the generated position and attitude correction information to the satellite navigation device and the inertial navigation device, respectively. The image input through the three-wire scanner is composed of a front scanner observation image, a vertical scanner observation image and a rear scanner observation image.

In addition, the present invention in the position and attitude information correction device of the camera connected to the satellite navigation device and the inertial navigation device, by analyzing the image input through the three-wire scanner provided in the camera to generate the position and attitude correction information of the camera The position and attitude correction information is fed back to the satellite navigation device and the inertial navigation device, respectively, to improve the accuracy of the position and attitude information of the camera.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, whereby those skilled in the art may easily implement the technical idea of the present invention. There will be. In addition, in describing the present invention, when it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a system configuration to which the image-based position and posture correction device according to the present invention is applied.

 Referring to FIG. 1, a camera 100 having a three-wire scanner is mounted on an observation device such as a satellite or an aircraft. Therefore, the position and attitude information of the camera 100 is directly connected to the position and attitude information of the observation apparatus. Referring to FIG. 1, the three-wire scanner includes a vertical scanner 120 facing a direction perpendicular to the direction of travel of the observation device, and a front scanner facing an arbitrary direction between the direction of travel of the observation device and the direction in which the vertical scanner 120 faces. 110, and a rear scanner 130 facing an arbitrary direction between a direction opposite to the direction of travel of the observation apparatus and a direction toward the vertical scanner 120. Therefore, the angle between the vertical scanner 120 and the front scanner 110 and the angle between the vertical scanner 120 and the rear scanner 130 are 90 degrees or less. That is, as shown in FIG. 1, the angle between the front scanner 110 and the vertical scanner 120 is 90 degrees or less in the advancing direction, and the angle between the rear scanner 130 and the vertical scanner 120 is advanced. It becomes 90 degrees or less in the opposite direction to a direction.

The satellite navigation device 200 is connected to the camera 100 to determine the position of the camera 100. As an example of the satellite navigation apparatus 200, a GPS receiver may be used. The inertial navigation device 300 is connected to the camera 100 to determine the pose of the camera 100. The satellite navigation device 200 and the inertial navigation device 300 are physically fixed to the same frame as the camera 100. Since the satellite navigation device 200 and the inertial navigation device 300 may be conventional ones, a detailed description thereof will be omitted.

The image-based position and posture correcting apparatus 400 generates position and posture correction information of the camera 100 using the scanline image input through the three-line scanner of the camera 100. The generated position and attitude correction information is fed back to the satellite navigation device 200 and the inertial navigation device 300, respectively, and the position and attitude information of the camera 100 can be more precisely grasped.

2 shows a relationship between images acquired through a three-wire scanner.

The three-wire scanner is composed of the front scanner 110, the vertical scanner 120, the rear scanner 130 in accordance with the direction of the observation device. Accordingly, the scanline image is also generated by the front scanner observation image 520 generated through the front scanner 110, the vertical scanner observation image 510 generated by the vertical scanner 120, and the rear scanner 130 generated by the rear scanner 130. It consists of a scanner observation image (530).

Since three scanners are arranged based on the traveling direction of the viewing device, the ground surface or facility to be observed is first observed by the front scanner 110. After a predetermined time Δt ₁ , the observation target is observed by the vertical scanner 120, and again after a predetermined time Δt ₂ , the observation target is observed by the rear scanner 130.

When the observation device performs the constant velocity linear motion and the inertial motion with no change in posture, the observation time interval between the front scanner 110, the vertical scanner 120 and the rear scanner 130 is the same (that is, Δt _1). = Δt ₂ ), the positional deviation does not appear between the images observed through the three scanners 110, 120, and 130.

On the other hand, when the observation device does not perform constant velocity linear motion and inertial motion, position deviations appear between the images 510, 520, and 530 observed through the scanner as shown in FIG. 2. The image position deviation between the front scanner observing image 510 and the vertical scanner observing image 520 is (1,1) in pixel units, and the image position deviation between the vertical scanner observing image 520 and the rear scanner observing image 530 is (1,2) in pixel units. The image-based position and posture correcting apparatus 400 according to the present invention can accurately correct the position and posture information of the camera by analyzing the image position deviation.

3 is a detailed block diagram of the image-based position and posture correction device 400 according to the present invention.

As the observation apparatus equipped with the camera 100 having the three-wire scanner proceeds to the front, the front scanner observation image 510, the vertical scanner observation image 520, and the rear scanner observation image 530 are sequentially turned by the three-wire camera. Is generated. The generated observation images 510, 520, and 530 are input to the image-based position and posture correcting apparatus 400.

The forward scanner observation image 510 is temporarily stored in the first buffer memory 410. Thereafter, the image matching unit 430 performs image registration on the temporarily stored front scanner observation image 510 and the vertical scanner observation image 520 that is observed and input in real time through the vertical scanner 120 to perform two images. Image position deviations Δu ₁ and Δv _{1 in} pixel units are calculated between 510 and 520. Image matching may be performed by various known methods such as region-based matching or shape-based matching.

The same process applies to the images observed through the vertical scanner 120 and the rear scanner 130. That is, the vertical scanner observation image 520 is temporarily stored in the second buffer memory 420. Thereafter, the image matching unit 430 performs image registration on the temporarily stored vertical scanner observation image 520 and the rear scanner observation image 530 that is observed and input in real time through the rear scanner 130. Image position deviations Δu ₂ and Δv _{2 in} pixel units are calculated between 520 and 530. Image matching may be performed by various known methods such as region-based matching or shape-based matching.

The first image position deviation related information Δu ₁ , Δv ₁ , between the front scanner observed image 510 and the vertical scanner observed image 520. t _f , t _n ) and second image position deviation related information Δu ₂ , Δv ₂ , between the vertical scanner observed image 520 and the rear scanner observed image 530. t _n and t _r are input to the Kalman Filter processing unit 440.

Information related to the first image position deviation (Δu ₁ , Δv ₁ , t _f , t _n ) is the image position deviation Δu ₁ , Δv ₁ between the front scanner observation image 510 and the vertical scanner observation image 520, the time t _{f at} which the front scanner observation image 510 was generated, The vertical scanner observation image 520 includes a generated time t _n .

Second image position deviation related information (Δu ₂ , Δv ₂ , t _n , t _r ) is the image position deviation Δu ₂ , Δv ₂ between the vertical scanner observation image 520 and the rear scanner observation image 530, the time t _{n at} which the vertical scanner observation image 520 was generated, The rear scanner observation image 530 is composed of the generated time t _r .

The Kalman filter processing unit 440 may include first and second image position deviation related information, camera position information (x, y, z, t _n ) at a time t _n inputted from the satellite navigation apparatus 200, and an inertial navigation apparatus ( Position and attitude correction information (x ', y', z ', ω', κ ', φ', t _m from the camera attitude information (ω, κ, φ, t _n ) at the time t _n inputted from 300) ) Is estimated. The position and posture correction information includes the position and posture change of the camera locally calculated for each time interval at which image generation is performed. Here, the time t _m is an arbitrary time existing between the time t _f at which the front scanner observation image 510 is made and the time t _b at which the rear scanner observation image 530 is made. The time interval at which the image is generated may be shorter than the time interval limited by a conventional satellite navigation system or an inertial navigation system.

4 is a detailed block diagram of the Kalman filter processor 440 according to the present invention. The projection conversion unit 442 includes seven parameters (x, y, z, ω) defined by camera position information (x, y, z, t _n ) and camera pose information (ω, κ, φ, t _n ). , κ, φ, t _n ) are defined by seven parameters (Δu ₁ , Δv ₁ , Δu ₂ , Δv ₂ , defined by the first and second image position deviation related information. t _f , t _n , t _b ), the projection transformation is applied, and the result is applied to the parameters (x, y, z) representing the camera position information to calculate the parameters (δx, δy, δz) representing the real world position deviation.

The Kalman filter 444 is then optimized from parameters (x, y, z, ω, κ, φ, t _n ) representing position and attitude information and parameters (δx, δy, δz) representing real world position deviations. Calculate parameters (Δx, Δy, Δz, Δω, Δκ, Δφ) representing estimates for position and attitude deviations.

The deviation correction unit 446 uses the parameters (Δx, Δy, Δz, Δω, Δκ, Δφ) to indicate the estimates of the position and posture deviations. By correcting the deviations of, z, ω, κ, φ, t _n ), the position and posture correction information (x ', y', z ', ω', κ ', φ optimized for a given time t _m) ', t _m ) is calculated.

The estimated position and attitude correction information (x ', y', z ', ω', κ ', φ', t _m ) optimized for any time t _m existing between t _f and t _b is the position correction Information (x ', y', z ', t _m ) and posture correction information (ω', κ ', φ', t _m ). The separated position correction information and attitude correction information are fed back to the satellite navigation device 200 and the inertial navigation device 300, respectively, and used to precisely correct the position and attitude information of the camera.

The technique of improving the accuracy of position and posture information by feeding back the correction information obtained by analyzing the three-wire scanner image may be performed by an onboard processing approach as well as a postprocessing approach.

The instant processing approach observes images from a satellite or aircraft through a three-wire scanner, while feeding position and attitude correction information back to the satellite and inertial navigation units to improve the accuracy of the position and attitude information, and correct the corrected position and attitude. This means applying information (ie, recording) directly to an image obtained through a three-wire scanner.

The post-processing approach records the position and attitude information obtained from the satellite navigation system and the inertial navigation system and the image information obtained through the three-wire scanner as it is, and further improves the accuracy of the position and attitude information through a separate process. it means.

The position and posture correction method according to the present invention may be implemented as a program and stored in a computer-readable recording medium (CD-ROM, RAM, ROM, floppy disk, hard disk, magneto-optical disk, etc.).

Persons having ordinary skill in the art to which the present invention pertains can make various substitutions, modifications, and changes to the above-described contents of the present invention without departing from the technical spirit of the present invention. It is not limited by the embodiment and the accompanying drawings.

The present invention enables to accurately determine the exterior orientation parameters of a camera without using a ground control point for an image observed through a camera mounted on a vehicle such as a satellite or an aircraft.

In addition, the present invention has an effect that allows to determine the position and attitude information of the camera more precisely for a shorter time interval.

Claims (12)

In the camera position and attitude information correction method, Calculating the position of the camera using a satellite navigation system; Calculating the pose of the camera using an inertial navigation device; Generating position and posture correction information of the camera by analyzing an image input through a three-wire scanner mounted to the camera; And Feeding back the generated position and attitude correction information to the satellite navigation device and the inertial navigation device, respectively. How to calibrate camera position and attitude information. The method of claim 1, The image input through the three-wire scanner consists of a front scanner observation image, a vertical scanner observation image and a rear scanner observation image. How to calibrate camera position and attitude information. The method of claim 2, Generating the position and posture correction information, Calculating a first image position deviation by image registration of the front scanner observation image and the vertical scanner observation image; And And calculating a second image position deviation by image registration of the vertical scanner observation image and the rear scanner observation image. How to calibrate camera position and attitude information. The method of claim 3, Generating the position and posture correction information, At the time when the front scanner observation image is generated, at the time when the vertical scanner observation image is generated, at the time when the rear scanner observation image is generated, at the time when the first and second image position deviations are generated, and at the time when the vertical scanner observation image is generated. The method may further include applying location information and pose information of the camera to a Kalman filter. How to calibrate camera position and attitude information. The method of claim 3, The first and second image position deviations are calculated in pixel units. How to calibrate camera position and attitude information. The method of claim 1, The technique for improving the precision of the position and posture information of the camera is made of an instant processing approach or a post processing approach How to calibrate camera position and attitude information. In the position and attitude information correction device of the camera connected to the satellite navigation device and the inertial navigation device, Analyze the image input through the three-wire scanner provided in the camera to generate the position and attitude correction information of the camera, Feeding the generated position and attitude correction information to the satellite navigation device and the inertial navigation device, respectively, to improve the accuracy of the position and attitude information of the camera. Camera position and attitude information correction device. The method of claim 7, wherein The three-wire scanner includes a vertical scanner facing in a direction perpendicular to a traveling direction of the observation apparatus on which the camera is mounted; A front scanner facing in an arbitrary direction between the advancing direction of the viewing device and the direction the vertical scanner faces; And It consists of a rear scanner facing in any direction between the direction opposite to the direction of travel of the observation device and the direction toward the vertical scanner, The angle between the vertical scanner and the front scanner and the angle between the vertical scanner and the rear scanner are 90 degrees or less. Camera position and attitude information correction device. The method of claim 8, Image matching between the observed image of the front scanner stored in the first buffer memory and the image observed and input in real time through the vertical scanner, and in real time through the observed image of the vertical scanner stored in the second buffer memory and the rear scanner An image matching unit for image matching the observed and input image Camera position and attitude information correction device. The method of claim 9, And a Kalman filter processor for generating position and posture correction information by applying the image position deviation of the pixel unit calculated as a result of the image registration to the Kalman filter. Camera position and attitude information correction device. The method of claim 10, The Kalman filter processing unit, A projection conversion unit configured to project-transform the camera position and attitude information when the observation image of the vertical scanner is generated to the image position deviation related information including the image position deviation information and the information on the observation image generation point and calculate a real world position deviation ; A Kalman filter for calculating an optimized position and posture deviation from the camera position and posture information and the real world position deviation at the time when the observed image of the vertical scanner is generated; And By correcting the deviation of the camera position and attitude information at the time when the observed image of the vertical scanner is generated from the optimized position and attitude deviation, the time at which the observed image of the front scanner is generated and the rear scanner observed image are generated. A deviation correcting unit that is optimized for an arbitrary time between the estimated times and calculates estimated position and attitude correction information. Camera position and attitude information correction device. The method of claim 8, The observation device is any one of satellites and aircraft Camera position and attitude information correction device.

Images