JP2011069773A - Information terminal device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a highly reliable information terminal device that can estimate an orientation of a user and can control display information of a display section in accordance with the estimation result without providing a geomagnetism sensor. <P>SOLUTION: A subject 11 receives light from a light source and its entire circumference. An image capture section 12 captures an image of the subject 11. An estimation section 13 analyzes an image captured by the image capture section 12, extracts a characteristic amount depending on the direction of the light source, sets a reference orientation by estimating the direction of the light source at the image captured by the image capture section based on the characteristic amount and estimates another orientation based on the reference orientation. A control section 15 reads out display information from a memory 14, controls the display information in accordance with the orientation estimated by the estimation section 13 and displays the display information on the display section 16. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an information terminal device, and in particular, information capable of estimating the orientation of a user by analyzing an image acquired by an imaging unit and controlling display information on a display unit according to the estimation result. It relates to a terminal device.

  There are information terminal devices that can acquire azimuth information such as user position information and user orientation and receive geographical information related to the acquired position information and azimuth information. A method of acquiring position information and azimuth information with an information terminal device is known as described in Patent Documents 1-3.

  Patent Document 1 describes a global positioning system (GPS) device and a geomagnetic sensor as one function of an exterior panel that can be attached to and detached from an information terminal device.

  Patent Document 2 describes a mobile phone and a mobile phone system that can accurately know the relationship between true north and the orientation of the display screen of a mobile phone by performing declination correction and dip angle correction.

  Patent Document 3 describes a portable azimuth calculating device that can detect an event of a magnetic field change in an information terminal device and can accurately determine the azimuth by using correction data for a geomagnetic detection value by a geomagnetic sensor. ing.

JP 2008-295077 A JP 2006-94368 A JP 2008-292493 A

  The GPS device and the geomagnetic sensor disclosed in Patent Document 1 and the mobile phone disclosed in Patent Document 2 do not consider the influence of the magnetic field that the geomagnetic sensor receives from the information terminal device. Since the geomagnetic sensor that detects weak geomagnetism is easily affected by the magnetic field generated by the components of the information terminal device, the GPS device and geomagnetic sensor of Patent Document 1 and the mobile phone of Patent Document 2 acquire accurate azimuth information. There is a problem that it is difficult.

  Patent Document 3 proposes to introduce a correction value for a static magnetic field into a geomagnetic detection value by a geomagnetic sensor. However, this portable azimuth calculation device does not take into account dynamic magnetic field changes that occur during operation of the information terminal device. Accordingly, there is still a problem that it is difficult to obtain accurate azimuth information.

  As described above, in any of the prior arts described in Patent Documents 1-3, there is a problem that it is difficult to obtain accurate azimuth information and the reliability of the apparatus is low. In addition, since the prior art described in Patent Documents 1-3 is based on the premise that a geomagnetic sensor is used to acquire azimuth information, the space strength for mounting the geomagnetic sensor and the strength of the magnetic field due to the denseness of parts are reduced. There is also a problem that it is difficult to reduce the size of the apparatus as the necessity of protection against the increase arises.

  The object of the present invention is to solve the above-mentioned problems, and it is not necessary to mount a geomagnetic sensor. It is to provide an information terminal device with high accuracy.

  In order to achieve the above object, the present invention analyzes a subject that receives light from all around including a light source, an imaging unit that acquires an image of the subject, an image acquired by the imaging unit, and A feature amount that depends on the direction is extracted, the direction of the light source in the image acquired by the imaging unit is estimated based on the feature amount, and a reference orientation is set, and another orientation is estimated based on the reference orientation. A first feature is that the estimation unit is provided.

  In addition, the present invention has a second feature in that the subject transmits light from a light source in an arbitrary direction.

  The third feature of the present invention is that the subject reflects light from a light source in an arbitrary direction.

  According to a fourth aspect of the present invention, the subject transmits light from a light source on the opposite side to the imaging unit and reflects light from a light source on the same side as the imaging unit. There is.

  In addition, the present invention has a function of presenting whether the light source is on the opposite side or the same side as the imaging unit, depending on whether or not the color of the subject and the color of the light source are mixed. There is a fifth feature.

  In addition, the present invention has a sixth feature in that the subject has a function of highlighting a feature amount of a portion corresponding to a direction of a light source in an image acquired by the imaging unit.

  Further, the present invention is characterized in that the estimation unit has a function of estimating a direction in which an area including the largest maximum luminance value in the subject in the image acquired by the imaging unit exists as a light source direction. There are 7 features.

  Further, according to the present invention, the estimation unit calculates a sum of pixel values in a radial direction from the center of the subject to a point on the outer periphery in a straight line in the subject in the image acquired by the imaging unit. There is an eighth feature in that it has a function of estimating the direction of the light source as the direction of the light source.

  Further, the present invention provides a pixel having a pixel value equal to or greater than a value obtained by multiplying the maximum pixel value in the subject in the image acquired by the imaging unit by a predetermined threshold value for each pixel value. And assigning a negative correlation value, and calculating the sum of the correlation values in the radial direction from the center of the subject to one point on the outer periphery in a straight line within the subject of the image acquired by the imaging unit. The ninth feature is that it has a function of estimating the direction in which the feature value takes the minimum value as the direction of the light source.

  In the present invention, when the estimation unit calculates the sum of correlation values in each radial direction, the estimation unit holds the minimum value calculated so far and the radial direction, and the calculated total is the minimum value. When the value exceeds the value, the calculation process in the radial direction is stopped at that point, and the calculation process is reduced by proceeding to the calculation process in the next radial direction.

  In addition, the present invention is characterized in that when the known azimuth is input as a reference azimuth, the estimation unit has a function of setting another azimuth by equally dividing the area of the image acquired by the imaging unit. There is an eleventh feature.

  Further, the present invention has a function of identifying whether the light source is on the opposite side or on the same side as the imaging unit, based on whether the color of the subject and the color of the light source are mixed. The point has a twelfth feature.

  In the present invention, when the reference direction is unknown, the estimation unit calculates the direction of the celestial body in the user environment, sets the direction as the reference direction, and sets another direction based on the reference direction. A thirteenth feature is that it has a function of estimating.

  Further, according to the present invention, the estimation unit calculates a sun azimuth from position information and time information of an information terminal, sets the azimuth as an absolute reference azimuth, and estimates another azimuth based on the reference azimuth. There is a fourteenth feature in that it has a function to perform.

  The present invention further includes a memory that stores display information, a control unit that reads display information from the memory and controls display information in accordance with a reference orientation and other orientations estimated by the estimation unit, There is a fifteenth feature in that a display unit for displaying display information controlled by the control unit is provided.

  The present invention estimates the orientation of the image by analyzing the image acquired by the imaging unit. Thus, the direction of the user can be estimated from the estimated image orientation.

  Further, by controlling the information display on the display unit in accordance with the estimated change in the azimuth of the image, the user can control the information display by an operation of changing the orientation at the spatial position. As a result, the user can intuitively perform the information display operation associated with the position information and the orientation information while associating with the real space, and can provide a highly reliable information terminal device. it can.

It is a functional block diagram which shows one Embodiment of this invention. It is an external view (a front view and a side view) showing a configuration example when a mobile phone is assumed as the information terminal device. FIG. 3 is a diagram illustrating an example of an image acquired by a subject and an imaging unit in the configuration example of FIG. 2. It is explanatory drawing of the azimuth | direction calculation of the sun. It is a figure which shows the to-be-photographed object in the other structural example. It is a figure which shows the to-be-photographed object in another structural example. It is a figure which shows the to-be-photographed object in another structural example. It is a figure which shows the example of a display at the time of applying this invention to the display control of a map. It is a figure which shows the example of a display at the time of applying this invention to the addition of information to an image.

  The present invention will be described below with reference to the drawings. FIG. 1 is a functional block diagram showing an embodiment of the present invention.

  The information terminal device of this embodiment includes a subject 11, an imaging unit 12, an estimation unit 13, a memory 14, a control unit 15, and a display unit 16. The information terminal device is, for example, a mobile phone, but may be a combination of a PC and a camera.

  The subject 11 receives light from the entire circumference including the light source.

  The imaging unit 12 is configured by a digital camera installed in the information terminal device. When the information terminal device is a mobile phone, the built-in camera can be used as the imaging unit 12, and when the information terminal device is a combination of a PC and a camera, the imaging unit 12 is added to the PC.

  The imaging unit 12 acquires an image using the subject 11 as an imaging target. Therefore, the subject 11 is mounted so as to exist within the imaging range of the imaging unit 12. The subject 11 may be fixedly attached to the imaging unit 12 or detachably attached.

  At the time of preparation prior to azimuth estimation, the user wears the subject 11 on the photographing unit 12. The estimation unit 13 analyzes the image captured by the imaging unit 12, obtains the azimuth of the light source in the image from the feature quantity that depends on the direction of the light source, and sets the azimuth as the reference azimuth.

  Next, the direction estimation is performed with the subject 11 still attached to the photographing unit 12. The estimation unit 13 receives one image acquired by the imaging unit 12 as an input, and based on the image and the feature amount in the image captured by the imaging unit 12 at the time of preparation and the reference orientation set at the time of preparation, the imaging unit 12 Estimate the azimuth of one image acquired by.

  The memory 14 stores information such as character information and map information.

  The control unit 15 reads out information stored in the memory 14 and causes the display unit 16 to display the information. At this time, the control unit 15 controls the information display according to the direction estimated by the estimation unit 13. The display unit 16 displays information according to control by the control unit 15.

  FIG. 2 is an external view (a front view and a side view) showing a configuration example when a mobile phone is assumed as the information terminal device. FIG. 2 shows the approximate positions of the subject 11, the imaging unit 12, the estimation unit 13, the memory 14, the control unit 15, and the display unit 16 in the mobile phone.

  FIG. 3 is a diagram schematically illustrating an example of an image acquired by the subject 11 and the imaging unit 12 in the configuration example of FIG. As shown in FIG. 3A, the subject 11 is a cylindrical object and is mounted on the imaging unit 12 so as to surround it. The subject 11 is made of a translucent material and transmits light from the light source in any direction. The upper part of the subject 11 is covered with a black plate so as not to transmit light. The imaging unit 12 images the subject 11. The acquired image includes light transmitted by the subject 11. FIG. 3B shows an example of an image acquired by the imaging unit 12 when the light source is on the back side of the subject 11 as shown in FIG. 3A, and the portion corresponding to the direction of the light source becomes brighter. ing. As a result of automatically adjusting the luminance, other portions are darkened.

  Next, the operation in the information terminal device will be described.

  First, an operation during preparation prior to azimuth estimation will be described. At the time of preparation, the user wears the subject 11 on the imaging unit 12, and takes the subject 11 with the imaging unit 12 while holding the information terminal device horizontally (so that the imaging unit 12 captures the sky direction). The acquired image includes light from the light source, and this image is output to the estimation unit 13. The estimation unit 13 analyzes this image, obtains the azimuth of the light source in the image from the feature quantity depending on the direction of the light source, and sets the azimuth as the reference azimuth.

  The reference orientation can be set as follows. Hereinafter, a case where an absolute direction is set as the reference direction will be described. However, the reference orientation does not have to be an absolute orientation. For example, when the present invention is applied to an electronic compass for estimating the azimuth (absolute azimuth) that the user is facing, the reference azimuth needs to be an absolute azimuth, depending on the orientation of the user When applied to a user interface that simply controls the display of information on the display unit 16, the reference orientation may be a reference orientation in the circumferential direction of the user.

  When setting the reference azimuth, if the absolute azimuth of the light source is known, the absolute azimuth is manually set with respect to the direction of the light source in the image acquired by the imaging unit 12, and this is used as the reference azimuth. That's fine. When the absolute orientation of the light source is not known, the orientation of the light source (absolute orientation) is calculated, and the orientation is set with respect to the direction of the light source in the image acquired by the imaging unit 12, and this is set as the reference orientation. do it.

  The direction of the light source in the image acquired by the imaging unit 12 can be estimated based on the feature amount generated by the transmission of light from the light source from the image. For this purpose, first, a feature amount generated by transmission of light from the light source is extracted from the image acquired by the imaging unit 12.

  As the feature amount, a luminance distribution due to transmission of light from the light source can be used. For example, the barycentric position of an area composed of pixels having a predetermined brightness or higher in luminance due to light transmission from the light source can be obtained, and the direction of the light source can be estimated from the barycentric position. As shown in FIG. 3, when the subject 11 is a cylindrical translucent material, the direction from the center of the subject 11 to the center of gravity of the region in the image acquired by the imaging unit 12 directly matches the direction of the light source. .

  Further, in the subject 11 in the image acquired by the imaging unit 12, the sum of the pixel values in the radial direction from the center of the subject 11 to one point on the outer periphery in a straight line shape is calculated, and this can be used as a feature amount. it can. In this case, the direction in which the feature amount takes the maximum value can be estimated as the direction of the light source.

  In addition, in the subject 11 in the image acquired by the imaging unit 12, a negative correlation value is assigned to each pixel value to a pixel having a pixel value equal to or greater than a value obtained by multiplying the maximum pixel value by a predetermined threshold value. In the subject 11 in the image acquired by the imaging unit 12, the sum of the correlation values in the radial direction from the center of the subject 11 to one point on the outer periphery in a straight line can be calculated, and this can be used as a feature amount. . For example, if the maximum pixel value in the subject 11 is 200, a pixel having a value of 180 or more obtained by multiplying it by 0.9 is set to a negative correlation value (200-value of the pixel) for each pixel value. A feature amount is obtained based on the assigned correlation value. In this case, the direction of the light source can be estimated as the direction in which the sum of the correlation values in the radial direction from the center of the subject 11 in a straight line to one point on the outer periphery takes the minimum value. Also, when calculating the total of correlation values in each radial direction, the minimum value calculated so far and the radial direction are kept, and when the calculated total exceeds the minimum value, the diameter is The calculation amount can be reduced by stopping the calculation process in the direction and proceeding to the calculation process in the next radial direction.

  Hereinafter, calculation of the azimuth of the sun and setting of the reference azimuth when the light source is the sun will be described. In this case, first, the direction of the sun is calculated from the current time information and the position information of the information terminal device. The time information can be obtained from a clock built in the information terminal device. Further, the position information of the information terminal device can be acquired from a GPS device built in the information terminal device. The GPS device acquires position information based on a received signal from a GPS antenna installed in the information terminal device. The location information can also be obtained by using location information recorded in a wireless base station that is wirelessly connected by a communication antenna. The user manually sends a place name or address to the server and is stored in the server. It can also be obtained from a geographic information database. The solar azimuth calculation method will be described later.

  The estimation unit 13 sets the above-mentioned known or calculated orientation with respect to the direction of the sun (light source) in the image acquired by the imaging unit 12 estimated as described above, and uses this as the reference orientation . This completes the preparation operation prior to bearing estimation.

  Next, the operation at the time of azimuth estimation will be described.

  At the time of azimuth estimation, the estimation unit 13 extracts the feature amount from one image acquired by the imaging unit 12 and estimates the direction of the sun as in the preparation. Next, the direction of the sun estimated at the time of preparation is compared with the direction of the sun estimated this time to examine the correspondence between the two. By this comparison, the azimuth of the image acquired this time by the imaging unit 12 is obtained as the coordinate of the relative movement amount from the reference azimuth. The coordinates thus obtained are converted into azimuth information with reference to the standard azimuth and output to the control unit 15. Since the user normally operates the information terminal device in his / her forward direction, the user's direction can be estimated from the orientation of one image acquired by the imaging unit 12.

  Display information to be displayed on the display unit 16 is developed on the memory 14 in advance. The control unit 15 reads out the corresponding information on the memory in accordance with the azimuth estimated by the estimation unit 13 and its change, and sends it to the display unit 16.

  The display unit 16 displays the display information sent from the control unit 15. At this time, the control unit 15 controls display according to the azimuth estimated by the estimation unit 13 and its change. For example, if the user continuously changes the spatial orientation of the information terminal device while the imaging unit 12 is in operation, the control unit 15 changes the display information on the display unit 16 continuously. Let

  Next, a method for calculating the direction of the sun will be described later.

  FIG. 4 is an explanatory diagram for calculating the orientation of the sun. When the spherical trigonometry is applied, the solar altitude h and the solar azimuth angle a at the point where the latitude is L and the longitude φ are related by the following formulas (1) to (3).

  Here, H and d are hour angle and solar red longitude, respectively. If Japan standard time JST and standard meridian (135 degrees) are used, the hour angle H is given by the following equation (4).

Here, E _q is the difference (equal time difference) between the time according to the average solar time and the time according to the true solar time. The equation of time E _q is obtained by the following formula (5).

  Here, w = 2π / 365 (the leap year is w = 2π / 366), and J is (the total number of days from the first day minus 1). Therefore, the solar height h and the solar direction a are given by the following formulas (6) and (7).

  Here, α is the solar declination and is obtained by the following formula (8).

  In addition, the sun azimuth | direction a is 0 degree | times in the south, and represents the southwest direction by the positive and the southeast direction by the negative angle.

  FIG. 5 is a diagram schematically illustrating the subject 11 in another configuration example. As shown in the figure, the subject 11 is a spherical object and is mounted within the imaging range of the imaging unit 12 at a certain distance from the imaging unit 12. The subject 11 is made of a translucent material or an opaque material, and reflects light from the light source in any direction. Arbitrary means can be used to attach the subject 11 to the imaging unit 12, but it should not interfere with the light from the light source to the subject 11.

  In this case, the user captures the subject 11 with the imaging unit 12 while holding the information terminal device horizontally (so that the imaging unit 12 captures the direction of the ground). The image acquired by the imaging unit 12 includes light reflected by the subject 11. In the image acquired by the imaging unit 12, the portion corresponding to the direction of the light source is bright as in FIG. As a result of automatically adjusting the luminance, other portions are darkened. The functional blocks and operations of the information terminal device in this case are the same as those in FIG.

  FIG. 6 is a diagram schematically showing the subject 11 in still another configuration example. As shown in the figure, the subject 11 is a spherical object and is mounted within the imaging range of the imaging unit 12 at a certain distance from the imaging unit 12. The subject 11 is made of a translucent material and transmits and reflects light from the light source.

  In this case, the user captures the subject 11 with the imaging unit 12 while holding the information terminal device vertically (so that the imaging unit 12 captures the front of the user). The image acquired by the imaging unit 12 includes transmitted light or reflected light from the subject 11. When the light source is on the back side of the subject 11, the image acquired by the imaging unit 12 includes transmitted light from the light source. When the light source is on the near side of the subject 11, the image is acquired by the imaging unit 12. The image includes light reflected from the light source.

  In this case, it is not possible to identify whether the light source is on the back side or the near side of the subject 11 only from the bright spot on the image acquired by the imaging unit 12. This identification is made possible by applying a unique color scheme to the subject 11. That is, the reflected light includes only the light color of the light source, and the transmitted light includes a color in which the color scheme of the subject 11 and the light color of the light source are mixed. Therefore, in the image acquired by the imaging unit 12, the reflected light The difference in color occurs depending on whether it is transmitted light or transmitted light. The estimation unit 13 can identify whether the light source is on the rear side or the near side of the subject 11 from the difference in color. In the image acquired by the imaging unit 12, the portion corresponding to the direction of the light source is bright. As a result of automatically adjusting the luminance, other portions are darkened. The functional block and the operation of the information terminal device in this case are the same as those in FIG. 1, but the estimation unit 13 determines whether the color of the subject 11 and the color of the light source are mixed or not. It has a function of identifying whether it is on the opposite side or on the same side.

  FIG. 7 is a diagram schematically showing the subject 11 in still another configuration example. As shown in the figure, the subject 11 is a fisheye lens and is mounted so as to cover the imaging unit 12. The subject 11 is made of a transparent or translucent material, and light from the light source is incident on the imaging unit 12 regardless of the direction of the light source. The imaging unit 12 images the subject 11. The image acquired in this way includes light incident through the subject 11. In the image acquired by the imaging unit 12, the portion corresponding to the direction of the light source is bright. As a result of automatically adjusting the luminance, other portions are darkened. In this case, the functional block of the information terminal device and its operation are also the same as those in FIG.

  The subject 11 may be any object as long as a feature amount depending on the direction of the light source appears by the analysis of the image of the subject 11 acquired by the imaging unit 12, and thereby the direction of the light source can be estimated. Is optional. However, in order to improve the resolution of the estimated azimuth angle, the shape of the subject 11 is preferably a cylindrical shape or a spherical shape as shown in FIGS.

  It is also preferable that the subject 11 has a function of enhancing the pixel value of a portion depending on the direction of the light source in the image acquired by the imaging unit 12 and highlighting the feature amount of the portion. This function is realized, for example, by using a material excellent in reflection of light from the light source or giving a specific color to the subject 11.

  According to the present invention, the user can change the screen displayed on the display unit 16, the cursor position in the screen, or the like in an analog manner by changing the spatial orientation of the information terminal device in an arbitrary direction. Therefore, operations such as screen scrolling and cursor position can be performed intuitively. Further, for example, by displaying a map, an image, a web page, etc. on the display unit 16 and by directing the imaging unit 12 in an arbitrary direction, an image such as a character or a map can be rotated or scrolled according to the orientation, The viewpoint can be moved.

  FIG. 8 is a display example when the present invention is applied to map display control. Here, an example is shown in which the map is displayed such that the direction estimated by the estimation unit is always on the upper side of the display screen. The present invention can be applied to a map display application for performing such map display control.

  FIG. 9 is a display example when the present invention is applied to adding information to an image. Here, the mountain image acquired by the imaging unit is displayed on the display unit, and the name of the mountain is superimposed on this display. In this case, when the user changes the spatial orientation of the information terminal device, the image acquired by the imaging unit changes according to the change, a new orientation is estimated, and the superimposed information and its display position are changed. Be controlled. The present invention can also be applied to an information superimposing application for adding information to an object estimated to fall within the angle of view of an image from such position information and azimuth information.

  Although the embodiment has been described above, the present invention is not limited to the above embodiment, and includes various modifications. For example, in the above embodiment, the absolute azimuth is set as the reference azimuth. However, in the information display control that does not matter the absolute azimuth, the reference azimuth is simply set as the reference azimuth, and the circumference from the azimuth is set. The display can be controlled according to the direction deviation.

  Moreover, in the said embodiment, although the light source was the sun, other celestial bodies, such as the moon, can be used as a light source, and the other light source with the known direction with respect to an information terminal device can also be utilized.

  Furthermore, the present invention is also effective as an apparatus for estimating an absolute orientation that a user is facing, that is, an electronic compass. The electronic compass can be composed of the subject 11, the imaging unit 12, and the estimation unit 13 in FIG. 1, and the orientation estimated by the estimation unit 13 may be estimated as the absolute orientation toward the user.

11 ... Subject, 12 ... Imaging unit, 13 ... Estimating unit, 14 ... Memory, 15 ... Control unit, 16 ... Display unit

Claims (15)

A subject that receives light from all around, including the light source, and
An imaging unit for acquiring an image of the subject;
Analyzing the image acquired by the imaging unit, extracting a feature quantity depending on the direction of the light source, estimating the direction of the light source in the image acquired by the imaging unit based on the feature quantity, and determining a reference orientation An information terminal device comprising: an estimation unit that sets and estimates another direction based on the reference direction.   The information terminal device according to claim 1, wherein the subject transmits light from a light source in an arbitrary direction.   The information terminal device according to claim 1, wherein the subject reflects light from a light source in an arbitrary direction.   2. The information according to claim 1, wherein the subject transmits light from a light source on a side opposite to the imaging unit and reflects light from a light source on the same side as the imaging unit. Terminal device.   The object has a function of presenting whether the light source is on the opposite side or on the same side as the imaging unit, depending on whether or not the color of the subject and the color of the light source are mixed. 4. The information terminal device according to 4.   The information terminal device according to claim 1, wherein the subject has a function of highlighting a feature amount of a portion corresponding to a direction of a light source in an image acquired by the imaging unit.   The said estimation part is provided with the function to estimate the direction in which the area | region where the largest brightness value exists most in the said subject in the image acquired by the said imaging part exists as a light source direction. The information terminal device described.   The estimation unit calculates a sum of pixel values in a radial direction from the center of the subject to one point on the outer periphery in a straight line from the subject in the image acquired by the imaging unit as a feature amount, The information terminal device according to claim 1, further comprising a function of estimating a direction in which a feature amount takes a maximum value as a direction of a light source.   The estimation unit has a negative correlation value for each pixel value in a pixel having a pixel value equal to or greater than a value obtained by multiplying the maximum pixel value in the subject in the image acquired by the imaging unit by a predetermined threshold value. And calculating the sum of the correlation values in the radial direction from the center of the subject to one point on the outer periphery in a straight line within the subject of the image acquired by the imaging unit, and the feature amount The information terminal device according to claim 1, further comprising a function of estimating a direction having a minimum value as a direction of the light source.   When calculating the sum of correlation values in each radial direction, the estimation unit holds the minimum value calculated so far and the radial direction, and when the calculated total exceeds the minimum value, The information terminal device according to claim 9, further comprising a function of reducing a calculation amount by stopping calculation processing in the radial direction and proceeding to calculation processing in the next radial direction.   The estimation unit includes a function of setting another orientation by equally dividing an area of an image acquired by the imaging unit when a known orientation is input as a reference orientation. The information terminal device described in 1.   The estimation unit includes a function of identifying whether the light source is on the opposite side or on the same side as the imaging unit based on whether or not the color of the subject and the color of the light source are mixed. Item 6. The information terminal device according to Item 5.   The estimation unit has a function of calculating the azimuth of a celestial body in a user environment when a reference azimuth is unknown, setting the azimuth as a reference azimuth, and estimating another azimuth based on the reference azimuth. The information terminal device according to claim 1.   The estimation unit has a function of calculating the azimuth of the sun from position information and time information of the information terminal, setting the azimuth as an absolute reference azimuth, and estimating another azimuth based on the reference azimuth. The information terminal device according to claim 12, wherein A memory for storing display information;
A control unit that reads the display information from the memory and controls the display information according to the reference direction and the other direction estimated by the estimation unit;
The information terminal device according to any one of claims 1 to 13, further comprising a display unit that displays display information controlled by the control unit.