JP2014219621A - Display device and display control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable stereoscopic vision producing no uncomfortable feeling.SOLUTION: A head-mounted display 10 includes: a first display for displaying a left-eye image; a first lens unit capable of dynamically varying the focal distance in order to extend the left-eye image displayed on the first display and form a virtual image; a second display for displaying a right-eye image; a second lens unit capable of dynamically varying the focal distance in order to extend the right-eye image displayed on the second display and form a virtual image; and controlling means for controlling the focal distances of the first lens unit and second lens unit.

Description

  The present invention relates to a display device such as a head mounted display, and a display control program for controlling the display device.

  A head-mounted display (HMD (head mount display)) is known as a display device that displays game images, video content, and the like. The head-mounted display is a display device that can be recognized as one image (video) by providing a display at a position corresponding to each of the left and right eyeballs, for example, and displaying an image (video) on each display. The head-mounted display is a small device that can be mounted on the head (or in front of the eyes), but can provide a sense of viewing an image (video) displayed on a large-screen display device from a remote position.

  In addition, since the head-mounted display is provided with a display corresponding to each of the left and right eyeballs, the left-eye video and the right-eye video generated to display a stereoscopic image (3D image) are displayed on each display. Can be displayed. For example, the liquid crystal display device described in Patent Document 1 can form one image using a first liquid crystal display and a second liquid crystal display that display an image.

JP 2004-117894 A

  In the technique described in Patent Document 1, when a stereoscopic image (3D) is displayed on a head mounted display, a left-eye image and a right-eye image are generated, and different images are used for the left-eye. Each is displayed on the display for the right eye. In general, in order to display a stereoscopic image (3D), an image for the left eye and an image for the right eye are generated using parallax.

  As described above, the conventional head-mounted display uses the parallax to generate the left-eye video and the right-eye video and display them on the left-eye and right-eye displays, respectively, thereby realizing stereoscopic viewing. doing. However, in conventional head-mounted displays, the imaging distance (the position of the recognized display) is fixed, so natural three-dimensional expression is possible in a specific range with the imaging distance as the reference distance, but the reference distance If it is far from the above, there may be a sense of incongruity. In other words, since humans perform three-dimensional recognition by focusing on an object according to the distance to the object to be viewed in addition to the parallax, there may be a sense of incongruity when the imaging distance is fixed. It was.

  The present invention has been made in view of the above-described circumstances, and an object thereof is to provide a display device and a display control program capable of stereoscopic viewing without a sense of incongruity.

  In order to solve the above-described problem, the present invention forms a virtual image by enlarging a first display for displaying an image for the left eye and an image for the left eye displayed on the first display. A first lens unit capable of dynamically changing a focal length, a second display for displaying an image for the right eye, and a virtual image by enlarging the image for the right eye displayed on the second display And a control unit that controls the focal length of the first lens unit and the second lens unit.

  According to the present invention, stereoscopic viewing without a sense of incongruity is possible.

The figure which shows the external appearance structure of the head mounted display in this embodiment. The block diagram which shows the structure of the head mounted display and control part in this embodiment. The figure which shows the optical system for left eyes in this embodiment. The flowchart which shows operation | movement of the head mounted display in this embodiment. The figure for demonstrating the fluctuation | variation of the focal distance according to the position of the object (virtual space object) in the virtual space in this embodiment. The figure for demonstrating the fluctuation | variation of the focal distance according to the position of the object (virtual space object) in the virtual space in this embodiment.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a diagram illustrating an external configuration of a head mounted display 10 according to the present embodiment. A head-mounted display 10 shown in FIG. 1 is configured, for example, in a glasses shape, and includes a horizontally long display unit portion 14 positioned in front of eyes and mounting members 12a and 12b extending rearward from both sides of the display unit portion 14. Is provided. The head mounted display 10 can be mounted so that the display unit portion 14 is positioned in front of the eyes by locking the attachment members 12a and 12b to the upper part of the ear.

  A nose pad 18 is provided at the center of the inner surface 16 of the display unit 14. A display unit 20a for displaying an image (video) for the left eye is disposed on the left side of the nose pad 18, and a display for displaying an image (video) for the right eye is displayed on the right side of the nose pad 18. A unit 20b is arranged.

  A line-of-sight detection sensor 22a is provided above the display unit 20a, and a line-of-sight detection sensor 22b is provided above the display unit 20b. The line-of-sight detection sensors 22a and 22b are sensors for detecting the line of sight of the user wearing the head mounted display 10, and for example, take an image of an eyeball (pupil) that is close to the display units 20a and 20b.

  The head mounted display 10 is connected via a cable 19 to a control unit 30 that controls an image (video) displayed on the head mounted display 10. The control unit 30 displays a stereoscopic image (3D image) on the head mounted display 10 by individually displaying the left-eye image (video) and the right-eye image (video) on the display units 20a and 20b. .

  The head mounted display 10 is not limited to the eyeglass type as shown in FIG. 1, and may have another shape such as a hat type (helmet type).

FIG. 2 is a block diagram showing the configuration of the head mounted display 10 and the control unit 30 in the present embodiment.
As shown in FIG. 2, the head mounted display 10 includes display units 20a and 20b, line-of-sight detection sensors 22a and 22b, lens driving units 50a and 50b, and display driving units 52a and 52b. The display unit 20a includes an eyepiece unit 54a and a display 56a. The display unit 20b includes an eyepiece unit 54b and a display 56b.

  The eyepiece units 54a and 54b are disposed between the user's eyeball wearing the head mounted display 10 and the displays 56a and 56b (details are shown in FIG. 3), and are driven by the lens driving units 50a and 50b. The focal length can be changed dynamically without changing the position. The eyepiece units 54a and 54b can be realized by a unit that changes the thickness of the lens, such as a fluid lens, or a unit that changes the distance between lenses by combining a plurality of lenses.

  The lens driving units 50a and 50b drive the eyepiece units 54a and 54b under the control of the control unit 30 (processor 40) to change the focal lengths of the eyepiece units 54a and 54b.

  The displays 56a and 56b are constituted by, for example, a liquid crystal display (LCD (Liquid Crystal Display)), an organic EL (Electroluminescence) display, or the like.

  The display driving units 52a and 52b drive the displays 56a and 56b under the control of the image processing unit 46 to display an image (video). The display driving unit 52a displays an image (video) for the left eye on the display 56a, and the display driving unit 52b displays an image (video) for the right eye on the display 56b.

  The control unit 30 includes a processor 40, a memory 42, a storage device 44, an image processing unit 46, and an I / O interface 48.

  The processor 40 implements various functions by executing a program stored in the memory 42. The processor 40 is mutually connected to the memory 42, the storage device 44, the image processing unit 46, and the I / O interface 48 via a bus.

  The memory 42 temporarily stores programs processed by the processor 40 and data processed by the processor 40. Programs stored in the memory 42 include an image generation program 42a and a focus control program 42b.

  The image generation program 42a is a program for generating image data to be displayed on the display units 20a and 20b (displays 56a and 56b) based on the content data 44a. The image generation program 42a of the present embodiment realizes a function (real-time 3DCG generation function) for generating a stereoscopic image (3D image) representing a virtual space in real time using a CG (computer graphics) technique. The processor 40 executes the image generation program 42a to generate image data for the left eye and image data for the right eye, respectively, in order to display a stereoscopic image (3D image) on the head mounted display 10. The data is output to the processing unit 46.

  The focus control program 42b is a program for controlling the focus of the eyepiece units 54a and 54b. The focus control program 42b is for detecting the line of sight of the user wearing the head mounted display 10 based on the images photographed by the line-of-sight detection sensors 22a and 22b input via the I / O interface 48. The processing is executed by the processor 40. For example, the processor 40 extracts an eyeball (pupil) from images captured by the line-of-sight detection sensors 22a and 22b, and detects the line of sight by tracking the movement of the eyeball (pupil). Note that other detection methods can be used for the detection of the line of sight.

  Further, the focus control program 42b detects which object in the virtual space displayed by the image generation program 42a is based on the user's line of sight, and detects the position of the corresponding object in the virtual space. Accordingly, the processor 40 is caused to execute processing for changing the focal length of the displays 56a and 56b.

  The storage device 44 stores various programs and data for controlling the control unit 30 in a nonvolatile storage medium. Examples of programs stored in the storage device 44 include an image generation program and a focus control program. In addition, the storage device 44 stores content data 44a for generating a stereoscopic image (3D image) representing the virtual space by the real-time 3DCG generation function. The content data 44a is, for example, data for displaying a game screen by CG, and is data (CG data) for displaying objects such as various characters and structures in a virtual space (game space). The various objects are expressed in such a manner that the apparent distance from the user wearing the head mounted display 10 changes by moving the position in the virtual space (game space) by the arithmetic processing based on the content data 44a. be able to. That is, the position of the object in the virtual space (game space) can be determined by a calculation process based on the content data 44a.

The image processing unit 46 drives the display driving units 52a and 52b in accordance with the image data generated by the processor 40 under the control of the processor 40, and displays images (videos) on the displays 56a and 56b. .
The I / O interface 48 is connected to the line-of-sight detection sensors 22a and 22b and the lens driving units 50a and 50b.

  2 shows a configuration example in which the head-mounted display 10 and the control unit 30 are separated, but a configuration in which each function of the control unit 30 is mounted on the head-mounted display 10 (the head-mounted display 10 and the control unit 30). It is good also as a structure which united the part 30).

Next, an outline of the optical system of the head mounted display 10 in the present embodiment will be described.
FIG. 3 is a diagram showing an optical system for the left eye (eyepiece unit 54a, display 56a). As shown in FIG. 3, the head mounted display 10 has an eyepiece unit 54a disposed between the user's eyeball 60 and the display unit 20a when the head mounted display 10 is mounted. The eyeball 60 refracts light rays by the front lens 64 (and cornea) and forms an image on the retina 62. The user focuses on the object by adjusting the thickness of the crystalline lens 64 according to the object to be watched.

  The attachment positions of the eyepiece unit 54a and the display unit 20a are fixed. 3, the distance S0 between the retina 62 and the crystalline lens 64, the distance S1 between the eyeball 60 (the crystalline lens 64) and the eyepiece unit 54a, the distance S2 between the eyepiece unit 54a and the display unit 20a, and the retina 62 and display. The distance L (S0 + S1 + S2) from the unit 20a is fixed.

  Although the physical distance between the eyeball 60 and the display unit 20a is short, the head mounted display 10 appears at a position of a distance D (virtual viewing distance) from the crystalline lens 64 by adjusting the focal length of the eyepiece unit 54a. It can be recognized as if there is an upper display. Moreover, the head mounted display 10 in this embodiment changes the focal distance of the eyepiece unit 54a, 54b, for example, the virtual viewing distance (distance D) is several meters to 20 meters, and the display size is several tens inches to 750 inches. Can be changed to a degree.

  The head mounted display 10 in the present embodiment detects which of the objects displayed on the display unit 20a is being watched by the user based on the line of sight detected by the line of sight detection sensor 22a, and the distances S1, S2, L The focal length of the eyepiece unit 54a is changed according to the position in the virtual space of the object being watched. That is, the focal length of the eyepiece unit 54a is changed so that the apparent display is at the position in the virtual space of the object being watched.

  Note that the right-eye optical system (the eyepiece unit 54b and the display 56b) is also configured in the same manner as in FIG.

Next, the operation of the head mounted display 10 in this embodiment will be described with reference to the flowchart shown in FIG.
The processor 40 generates the image data for the left eye and the image data for the right eye for displaying the stereoscopic image by the real-time 3DCG generation function based on the content data 44a by executing the image generation program 42a. To do. Here, image data for recognizing a stereoscopic image is generated using parallax.

  The image processing unit 46 drives the display driving units 52a and 52b according to the left-eye and right-eye image data, respectively. The display driving unit 52a displays an image corresponding to the image data for the left eye on the display 56a, and the display driving unit 52b displays an image corresponding to the image data for the right eye on the display 56b.

  On the other hand, by executing the focus control program 42b, the processor 40 inputs data of images taken by the line-of-sight detection sensors 22a and 22b through the I / O interface 48, and extracts an eyeball (pupil) region from this data. Then, the line of sight is detected by tracking the movement of the eyeball (pupil) (step A1).

  Based on the line of sight, the processor 40 determines which of the objects in the virtual space displayed on the display 56b is being watched by the user (step A2). Since images (videos) displayed on the displays 56a and 56b are created by CG, the position of the object (virtual space object) in the virtual space can be determined by arithmetic processing based on the content data 44a. it can. Further, the processor 40 detects the position in the virtual space of the object being watched by the user (step A3).

  Next, the processor 40 drives the lens driving units 50a and 50b so as to focus on the position of the object (virtual space object) in the virtual space, thereby changing the focal length of the eyepiece units 54a and 54b ( Step A4). That is, the focal length is changed so that the apparent display position of the user matches the position of the object (virtual space object) that is being watched.

5 and 6 are diagrams for explaining the variation of the focal length according to the position of the object (virtual space object) in the virtual space.
FIG. 5 is a diagram illustrating an example in which the user is gazing at the object B existing at a distant position in the virtual space. In this case, the processor 40 controls the eyepiece unit 54a to be in a short focus state according to the position of the object B in the virtual space (apparent distance from the user), and the object B in the virtual space Adjust the position so that there is an apparent display.

  On the other hand, in order to focus on the actual display unit 20a, the user places the crystalline lens 64 of the eyeball 60 in the long focus state as the eyepiece unit 54a is brought into the short focus state. That is, the user adjusts the focus of the eyeball 60 in the same manner as when viewing a real object located far away.

  FIG. 6 is a diagram illustrating an example in which the user is gazing at the object A existing at a close position in the virtual space. In this case, the processor 40 controls the eyepiece unit 54a to be in a long focus state according to the position of the object A in the virtual space (apparent distance from the user), and the object A in the virtual space is controlled. Adjust the position so that there is an apparent display.

  On the other hand, in order to focus on the actual display unit 20a, the user places the crystalline lens 64 of the eyeball 60 in the short focus state as the eyepiece unit 54a is set in the long focus state. That is, the user adjusts the focus of the eyeball 60 in the same manner as when viewing an object that is close to the real object.

  5 and 6 describe the left-eye optical system (the eyepiece unit 54a and the display unit 20a), but the right-eye optical system (the eyepiece unit 54b and the display unit 20b) is also described. The illustration is omitted as the same control is performed.

  Thus, the apparent distance to the object by the user is changed by changing the focal length of the eyepiece unit 54a according to the position (the apparent distance from the user) of the object that the user is gazing at. The focus will be adjusted accordingly. Humans recognize three-dimensionally by focusing on an object according to the distance to the object to be visually recognized as well as parallax. For this reason, an image for recognizing a stereoscopic image using parallax is displayed on the display units 20a and 20b, and a stereoscopic image without a sense of incongruity is displayed to the user by adjusting the focus with the user's eyeball (crystal). Can be recognized.

  Further, the processor 40 uses the position in the virtual space of the object that the user is gazing at as a reference position, and images the other objects (objects) existing in the virtual space according to the distance in the depth direction from the reference position. Processing is performed (step A5). That is, the processor 40 gives blur in the image representing the virtual space according to the distance in the depth direction of the virtual space so that the amount of blur increases as the distance from the object being watched increases. Thereby, it is possible to express the focus of the camera.

  In the above description, a case where a stereoscopic image is displayed by the real-time 3DCG generation function has been described as an example. However, the content data 44a may be data for displaying a two-dimensional image. In this case, an image processing technique (2D3D conversion technique) for converting a two-dimensional image into a three-dimensional image is used for the two-dimensional image.

  In the 2D3D conversion technique, for example, composition identification for identifying a composition with respect to a two-dimensional image, motion identification for identifying movement of an object in an image that changes over time, and the like are combined to obtain the depth of each object in the image. A position in the direction is determined, and a parallax image for displaying a 3D image is generated based on the position. Hereinafter, as described above, it is determined which object in the parallax image (3D image) generated by the 2D3D conversion technique is being watched by the user, and the eyepiece is set according to the position of the object in the depth direction. Control the position of the lens unit.

  In the above description, an image is generated based on the content data 44a stored in the storage device 44 and displayed on the displays 56a and 56b of the head mounted display 10. However, external image (video) reproduction is performed. You may make it input the image data displayed on the displays 56a and 56b from an apparatus.

  Further, the present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  DESCRIPTION OF SYMBOLS 10 ... Head mounted display, 20a, 20b ... Display unit, 22a, 22b ... Eye-gaze detection sensor, 30 ... Control part, 40 ... Processor, 42a ... Image generation program, 42b ... Focus control program, 54a, 54b ... Eyepiece unit, 56a, 56b ... display.

Claims (5)

A first display for displaying an image for the left eye;
A first lens unit capable of dynamically changing a focal length for enlarging a left-eye image displayed on the first display to form a virtual image;
A second display for displaying an image for the right eye;
A second lens unit capable of dynamically changing a focal length for enlarging an image for the right eye displayed on the second display to form a virtual image;
A display device comprising control means for controlling a focal length of the first lens unit and the second lens unit. Line-of-sight detection means for detecting line-of-sight with respect to a virtual space object in an image representing the virtual space displayed on the first and second displays;
Detecting means for detecting a virtual space object indicated by the line of sight;
2. The display according to claim 1, wherein the control unit varies a focal length of the first and second lens units in accordance with a position in the virtual space of the virtual space object detected by the detection unit. apparatus. The line-of-sight detection means includes
A first line-of-sight detection sensor disposed in the vicinity of the first display;
The display device according to claim 2, further comprising a second line-of-sight detection sensor disposed in the vicinity of the second display.   2. The display according to claim 1, further comprising image generation means for generating an image to be displayed on the first and second displays based on CG (computer graphics) data representing an image representing a virtual space. apparatus. A first display for displaying an image for the left eye;
A first lens unit capable of dynamically changing a focal length for enlarging a left-eye image displayed on the first display to form a virtual image;
A second display for displaying an image for the right eye;
A computer for controlling a display device having a second lens unit capable of dynamically changing a focal length for enlarging a right-eye image displayed on the second display to form a virtual image;
Line-of-sight detection means for detecting line-of-sight with respect to a virtual space object in an image representing the virtual space displayed on the first and second displays;
Detecting means for detecting a virtual space object indicated by the line of sight;
A display control program for functioning as control means for changing the focal lengths of the first and second lens units according to the position of the virtual space object detected by the detection means in the virtual space.

Images