JPH06195440A - Head follow-up type display device - Google Patents

Abstract

PURPOSE:To allow an observer to simultaneously attain the action of one's head and the using of both of one's hands in an artifitial reality environment and to reduce an observer's burden caused by mounting. CONSTITUTION:This display device generates the stereoscopic image of an artifitial video environment made into a model in a real time. A display part 3 displays the stereoscopic image and a displacement detection part 25 optically detects the relative positional relation of the display part 3 with the head of the observer to output it as displacement data. An actuater part is controlled based on output data of the displacement detection part 25 so as to make the display part 3 move freely in a space and an image processing part 20 switches the stereoscopic image displayed by the display part 3 based on displacement data from the displacement detection part 25.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a display device used in virtual reality, remote reality, etc., and in particular, an image on the display device mounted on the head is accompanied by movement of the wearer's head. The present invention relates to a head-following display device for changing artificial reality.

[0002]

2. Description of the Related Art Conventionally, as a display device for artificial reality, for example, a head-following head mounted display (HMD) has been used.

This HMD is a device that projects an image on the retinas of both eyes of an observer by combining a liquid crystal display device such as an LCD and an optical system. Since it is easy to downsize, it can be worn on the head. Further, by attaching a sensor for detecting the movement of the head and outputting an image corresponding to the change in the position of the head, it can be used as a display device for artificial reality.

On the other hand, a head coupled display (HC
In D; Head-Coupled-Display), as shown in FIG.
Lens system 101 and two built-in compact monochrome CRTs 102
A display unit consisting of and is hung on an arm of a multi-degree-of-freedom link 104 with a built-in encoder so that it can be moved in a three-dimensional space, and a balancer of a counterweight 103 is attached to facilitate the movement of the display unit. Thus, it is possible to hold it by hand while looking at the display section and move it. Therefore, it can be used as a display device for artificial reality that provides a high quality image which is somewhat large.

[0005]

However, the above-mentioned H
In the MD, a display device having an LCD with a pixel number of 300,000 pixel level is colorized and is enlarged and displayed to have a viewing angle of about 90 °. We haven't reached the point where we can say that we can feel the presence of artificial reality. Further, the viewing angle of 120 ° or more, which is required to obtain a real sense of reality, has not been realized. Furthermore, although the size has been reduced at present, it is still large, and if it is worn for a long time, the weight of the observer is heavy.

On the other hand, since the HCD is composed of the link type arm with the balancer of the counterweight 103, when looking around the surrounding image space in the artificial reality image environment, the image display device is looked into. However, it was necessary to move the robot with both hands, and in some cases, to operate with both hands. Furthermore, depending on the application of artificial reality, it was essential to operate the object in the artificial image environment by hand, but this technology has a problem that both hands cannot always be used, and it is possible to cope with these. It wasn't something.

The present invention has been made in view of the above problems, and it is an object of the present invention to provide a high-definition high-quality image for a high-definition large-sized display device or a new-type display device such as a laser type retina direct scanning display device in the future. In a display device, it is possible to simultaneously move the observer's head and use both hands in an artificial reality environment, and further reduce the burden of wearing the observer.

[0008]

In order to achieve the above object, the head-following display device of the present invention is a display device for generating a stereoscopic image of a modeled artificial video environment in real time, Display means for displaying the stereoscopic image, relative position detecting means for detecting a relative positional relationship between the display means and an observer's head, and outputting as displacement data, and control based on output data of the relative position detecting means And a display control means for switching the stereoscopic image displayed by the display means based on the displacement data from the relative position detection means. .

[0009]

That is, the head-following display device of the present invention is a display device for generating a stereoscopic image of a modeled artificial video environment in real time, and the display means displays the stereoscopic image and the relative position. The detection means detects the relative positional relationship between the display means and the observer's head and outputs it as displacement data, and the actuator means is controlled based on the output data of the relative position detection means to move the display means in space. The display control means switches the stereoscopic image displayed by the display means based on the displacement data from the relative position detection means.

[0010]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a schematic view of a head following type display device of the present invention.

As shown in FIG. 1, the head-following display device of the present invention is divided into an arm portion 1 having a 4-axis joint structure and a head portion 2 having a 3-axis structure in the rotational direction. The head unit 2 has a total of 7 degrees of freedom of movement, and is constructed based on a vertical multi-joint type manipulator type robot. Further, the arm portion 1 further has a waist portion 1a that enables movement on a vertical axis, a shoulder portion 1b that moves in a plane including the vertical axis, an elbow portion 1c, and a wrist portion 1.
The triaxial portion other than the waist portion 1a operates in a space in the same plane. Further, the head portion 2 is based on the yaw shaft portion 2a that enables the rotation operation of the wrist portion 1d, and the roll shaft portion 2b that enables the rotation operation of two axes orthogonal to the wrist portion 1d axis. , And three orthogonal rotation axes of the pitch shaft portion 2c. The 7 axes are servo-controlled by a DC motor, an encoder, and a controller. Next, with reference to FIG. 2, the reference reflector 4 used in the head following display device of the present invention will be described.

The reference reflector 4 is attached to the observer's head and is used for detecting the movement of the observer's head. For example, the reference reflector 4 is attached to a part of the observer's glasses or a part of headphones. It can also be worn as a hair band. 2A is a view of the alignment marks 4b, 4c, 4d of the reference reflection plate 4 as seen from above, and FIG. 2B shows the alignment marks 4b, 4c, 4d and respective parts. Is a view of the alignment light of FIG. 6 from an oblique direction, where the longitudinal direction of the reference reflection plate 4 is X, the short direction is Y, and the directions orthogonal to the X and Y axes are the Z directions. The rotation directions around the axis are shown as α, β, and θ, respectively.

In FIGS. 2A and 2B, the rectangular specular reflectors 4b and 4d provided near both ends of the reference reflector 4 are used for alignment in the X, Y and θ directions. The reflection mark 4c provided on the lower side of the central portion is a reflection portion for detecting the α and β rotation directions. The other portion 4a is surface-treated so as to diffusely reflect light, and is also used for detection in the Z-axis direction. Hereinafter, the measurement principle for detecting the relative position between the reference reflector 4 and the observer's head will be described in detail.

First, the principle of distance displacement measurement for detecting displacement in the Z direction will be described with reference to FIG. This distance displacement measurement is performed by an active photoelectric sensor that uses the principle of triangulation.

In the figure, a light 57 emitted from a light emitting element 6 such as a semiconductor laser (LD) driven by an LD drive circuit 5 is a light projection designed so that light having a sufficiently sharp directivity can be obtained. After passing through the lens 7, the object to be measured is irradiated. Then, when the reflected light 58 from the object to be measured is focused on the position detecting element 9 by the light receiving lens 8,
The position of the light receiving spot on the position detecting element 9 changes according to the distance of the object to be measured.

The output of the position detecting element 9 is current-voltage converted and amplified by the IV amplifier 10. Since this output is non-linear with respect to the displacement amount in the Z direction, after amplification, the linearization circuit 11 corrects the linearity to obtain a Z-axis displacement signal output. Therefore, the distance to the object to be measured can be obtained by electrically detecting the light receiving spot position. As described above, in the present invention, in order to electrically detect the light receiving spot position, a one-dimensional position detecting element (PSD; Position Sensitive) is used as the position detecting element 9.
Detector) is used to measure the vertical change of a point on the plane of the DUT.

On the other hand, the surface of the object to be measured cannot be measured unless it is an object that diffusely reflects, but it is possible to measure even if there is some inclination of the surface, so diffused reflection around the center of the reference reflector 4 is performed. It is arranged so that the projection spot hits the portion 4a.

Next, referring to FIG. 4, the principle of regular reflection of light is used to detect displacements in the α and β rotation directions that are rotations of the X and Y axes, that is, inclinations of the X and Y planes. The principle of angular displacement measurement will be described.

In the figure, the light 59 emitted from the light emitting element 6 such as an LD driven by the LD drive circuit 5 passes through a light projecting lens 7 designed to obtain light having a sufficiently sharp directivity. The object to be measured is irradiated. Then, when the reflected light 60 from the object to be measured is focused on the position detecting element 41 by the light receiving lens 8, the light receiving spot on the position detecting element 41 is two-dimensional according to the inclination of the surface of the object to be measured. Changes to.

The output of the position detecting element 41 is current-voltage converted and amplified by the IV amplifier 42. The position of the light spot incident on the position detection element 41 is
It is divided by the resistance value (distance) to the electrodes at both ends of and is output as a position (coordinate) signal. Therefore, the arithmetic circuit 4
The difference between the voltage values at both ends is obtained from 3 to obtain the α-axis displacement signal output and the β-axis displacement signal output. Therefore, the tilt of the object to be measured can be obtained by electrically detecting the position of the light receiving spot. A two-dimensional P is used as the position detecting element 41 in order to electrically detect the light receiving spot position.
SD is used.

However, in this method, the light receiving spot position also changes when the object to be measured moves up and down, that is, in the Z-axis direction, so that only the angle component of FIG. 3 is detected. By subtracting the component in the Z-axis direction from the measurement value obtained by the distance displacement measuring method, only the tilt component of the surface (α and β directions) is obtained. Further, the light projection spot positions of the distance displacement measurement and the angular displacement measurement need to be arranged at a short distance in order to reduce an error.

Based on the principle as described above, in the present invention, a reflection portion mark 4c on the mirror surface is provided on the lower side of the central portion of the reference reflection plate 4 so that the projection spot hits it, and on the upper side thereof. It is arranged so that the projection spot of distance displacement measurement will hit. Next, referring to FIG. 5 to FIG.
The principle of displacement measurement for detecting displacement and rotational displacement in the X and Y planes will be described. First, FIG. 5 is a diagram showing the measurement principle of the displacement measurement.

In the figure, the light emitted from the illumination light source 15 is reflected by the half mirror 14, passes through the objective lens 16, and illuminates one surface around the rectangular reflecting portion which is the object to be measured with a uniform illuminance distribution. Is designed to Then, the reflection images of the alignment marks 4b and 4d from the object to be measured are divided by the objective lens 16 into 4-division PDs which are light receiving elements.
When the image is formed at 13, the position of the alignment mark image on the light receiving element changes two-dimensionally according to the position of the object to be measured. Therefore, this alignment mark image is divided into four PDs.
The position displacement of the object to be measured can be obtained by electrically detecting with 13. A signal processing circuit for measuring the displacement displacement is as shown in FIG.

As shown in the figure, the signal processing circuit is composed of operational amplifiers 17 to 19, 17 having a current-voltage converter, 18 having a summing amplifier, and 19 having a difference amplifier. Then, the alignment mark 4 on the four-division PD 13 is obtained by obtaining the difference between the respective electric signals when the four-division PD 13 is regarded as the vertical division and the horizontal division.
The displacements of positions b and 4d can be obtained. That is,
It can be seen that when the alignment marks 4b and 4d are located at the center of the four-divided PD 13, the outputs of the four light-receiving surfaces are the same, and the differential output of the vertical division and the horizontal division is zero. Then, when moving in the + Y direction, the Y-direction displacement signal of the differential output signal becomes + output, so that the displacement direction and the distance are known.

Further, in the present invention, as shown in FIG.
Based on the displacement direction, the alignment marks 4b and 4d of the rectangular reflecting portion are provided at both ends of the reference reflection plate 4, and the detection sides are arranged so that the alignment marks can be measured.

These two alignment marks 4b and 4d
, Respectively, and the displacement displacement is measured separately. That is, the mark 4b on one side measures the displacement displacement in the X and Y directions,
For the mark 4d on the other side, by measuring only in the Y direction, the inclination θ due to the amount of deviation in the Y direction on both sides and the known length L.
Can be asked. Therefore, as the distance between the measurement positions is increased, the θ detection accuracy is improved, and it is necessary to increase the base length L as much as possible.

Here, the structure in which the detection unit capable of measuring the relative displacement between the reference reflection plate 4 and the observer's head based on the above-described measurement principles is incorporated in the display device side is as shown in FIG. . As shown in the figure, when the observer looks into the display unit 3, the detection light strikes around the observer's forehead, and the case display side end of the display device is erroneous to the observer. A sponge rubber 31 is attached for protection in the event of a collision and for blocking diplomacy. In addition, the distance displacement detection unit 30, the angular displacement detection unit 32, and the position shift detection unit 33 are arranged at the positions shown in FIG.

Further, the manner in which the reference reflector 4 is attached to the headphones and the glasses is shown in FIG. 9 (a),
This is as shown in (b). As shown in the figure, the reference reflectors 4 are arranged around the respective foreheads, and the reference reflectors 4 are placed on the head by the observer, and are placed at the optimum observation position while looking into the display unit 3. So that this reference reflector 4
And the position of the head are adjusted. Next, FIG. 10 is a block diagram showing the configuration of the first embodiment employing the head-following display device of the present invention.

As shown in the figure, in this embodiment, the actuator unit 29, the display unit 3, the displacement detection unit 25, and the image processing unit 2 are used.
0, displacement data processing unit 21, actuator control unit 2
2. Central control processing unit 23. And
The image processing unit 20 includes a video signal processing unit 20a, a real-time image generation unit 20b, and a visual field coordinate conversion unit 20c. The displacement data processing unit 21 includes the displacement signal processing unit 2
1a and a head coordinate conversion unit 21b, and the actuator control unit 22 includes an actuator servo controller 22a, a motor position conversion unit 22b, and an arm head coordinate conversion unit 22c.

In such a configuration, when the observer moves to look around the artificial image environment, the reference reflector 4 located on the forehead also moves with the movement of the observer, so that the relative position detecting device 30, shown in FIG. With 32 and 33, the displacement signal associated with the movement can be detected. This displacement signal is signal-processed by the displacement signal processing unit 21a by a method based on the above-described measurement principle, and relative displacement data of the head position in space and its direction is output. Then, the head coordinate conversion unit 21b
Then, the observer's head position and direction are converted into absolute coordinates from the displacement data and output, and the values are also internally updated and stored.

The origin position by initialization of the actuator unit 29 performed prior to these processes is used as a reference point of absolute coordinates. Inside the actuator unit 29, the optimum position of each axis of the arm unit 1 and the head unit 2 is calculated with respect to the measured head position data, and the position is obtained.
The motor absolute position conversion unit 22b obtains the physical absolute coordinates of each motor for the obtained axis position data. By outputting the data as the position data of each servo motor controller unit, the actuator unit 29 can always control each motor according to the data, and each axis of the arm unit 1 and the head unit 2 with respect to the head position data. The servo can be moved to the optimum position. Further, in the image processing unit 20, with respect to the position data of the head, that is,
Control is performed to switch the image output to the display unit 3 to the image in the viewing direction in accordance with the coordinates and direction of the observer's head.

Then, the visual field coordinate conversion unit 20c converts the head position data into coordinates in the artificial image environment with respect to the head position and direction of the observer and outputs them. Here, similarly to the above-described initialization operation in the head coordinate conversion unit 21b, the origin position by the initialization of the actuator unit 29 performed prior to these processes is used as the reference point of the coordinates in the artificial video environment. .

Further, the real-time image generation unit 20b cuts out an image that an observer in the artificial image environment can see based on the visual field coordinate data, generates a display image space image thereof in real time, and the image signal processing unit 20a. Is output to the display unit 3. Further, the central control processing unit 23 transfers the measured head position data to the actuator control unit 22 and the image processing unit 20.

In each of the above operating states, the actuator section 29 moves in real time in response to the displacement signal so as to reduce the displacement to zero. That is, the servo control is always performed so that the displacement signal becomes zero. However, when there is a long time until the actuator unit 29 side is servo-controlled due to slight vibration due to operation delay or disturbance of the actuator unit 29, or small shake on the observer side, the display shake prevention mode is set. can do.

Then, in the central control processing unit 23, the displacement amount of the actuator unit 29 is the FO of the display unit.
When it is sufficiently small with respect to V (viewing angle) or viewing range, the head position data is observed without moving the display surface so as to be transferred to the image processing unit 20 without moving the display unit 3 by the actuator unit 29. The head position data of the coordinates and direction assuming that only the viewpoint of the user has moved is transferred. This is the same as assuming that the display device side has the same function as the camera shake prevention mechanism based on the image processing method used in recent video cameras and the like, and is a kind of display shake prevention function. That is, when the difference between the movement of the head and the movement of the actuator unit 29, that is, the delay is small, the displacement data value is transferred to the image processing unit 20 as the display shake data, and only the image processing is necessary. Responds by moving the actuator unit 29 as well.

The manner in which the observer sits on the rotating chair and uses the head-following display device according to the first embodiment is as shown in FIG. That is, FIG. 11 (a)
FIG. 11 is a side view of a state in which the observer and the head following type display device are standing upright, and FIG. 11 (b) shows a front position and a 90 ° position of the head following type display device. The arrangement of the joint part of the arm part and the display part is seen from the top,
FIG. 11C is a diagram showing a state in which the observer has rotated 180 ° and the head-following display device is on the back side. As shown in each figure, it is possible to create an artificial image environment space that surrounds the observer.

As described above, in the first embodiment,
Since the actuator unit 29 drives the movement of the display device, there is no burden for the observer to operate the display device, and it is possible to use both hands freely while looking into the display unit 3.

Since the observer's head movement is measured in a non-contact manner, there is no contact between the observer and the display device,
It is easy to handle hygienically even in an environment where many unspecified people use it. In addition, since the observer does not have a cable for connecting the device or the position sensor, it is very easy to handle the device when looking into or leaving the device. Further, since it is supported by the actuator unit 29, it stands by itself even when it is separated from the device, and therefore, it becomes easy to return to the original situation in the artificial image environment even when looking again.
Furthermore, even in an environment where a large number of unspecified people use it, handling such as mounting and demounting becomes easy, and the rotation efficiency in use can be improved. Needless to say, each configuration of this embodiment can be variously improved and changed.

For example, as shown in FIG. 12, it is possible to suspend the arm portion 1 from the ceiling, and by doing so, the space in front of the observer and the moving space can be made wider. Furthermore, the head displacement detector is not of an optical type, and by using a capacitance sensor or the like, instead of the reference reflection plate, by using a reference plate composed of a three-dimensional reference surface composed of surfaces in three directions, The same non-contact spatial coordinate displacement can be detected. Then, if the configuration is made more complicated, the position and direction of the face can be measured by incorporating image processing with a device that images the feature points of the face on the display unit side in three directions. It is possible to detect spatial coordinate displacement due to non-contact. in this case,
There is no need to attach anything to the observer side, which makes handling easier. Further, as in the present embodiment, the HMD is used as a display device.
Instead of this, you can also move a large monitor.
In this case, it is not necessary to move the display device to the roll axis or the pitch axis depending on the application, and the structure of the head part can be simplified. Further, the angular displacement detector on the head displacement detecting device side is also unnecessary. Next, a second embodiment employing the head-following display device of the present invention will be described. FIG. 13 shows the second
It is a block diagram which shows the structure of an Example.

In this figure, this embodiment is composed of a master side device and a slave side device, and the master side device is the first device.
It has substantially the same configuration as the apparatus of the embodiment. However, the image processing unit 20 includes a video signal receiving unit 24 having a function of receiving and displaying a video transmitted from the slave side.
Since the slave side actuator device 29b is the same device as the actuator device 29a of the master side device, the same as the actuator part 29a of the master side device by transferring and controlling the head position data of the master side device. Make a move.

Further, an image input unit 26 is installed at a position corresponding to the display unit 3 of the master side device, and the pupil distance of the display unit 3 of the master side device is set so that stereo input can be performed by two video cameras. Are arranged in parallel at the same distance as. Then, the stereo image input signal is sent to the video signal receiving section 24 of the master side device in real time and displayed, so that the observer on the master side receives the image sent from the image input section 26 which is the camera of the slave side device. Can be observed in real time on the display unit 3 on the master side.

Further, by moving the head, the display unit 3 of the master side device and the image input unit 26 of the slave side device are moved in the same direction as the face direction, so that the image on the slave side device can be viewed. , The observer can get the illusion that the camera position of the slave side device has a head. That is, it is possible to obtain a realistic sensation of being in the place where the slave device is placed. The slave device does not have to have the same size as that of the master device. For example, by reducing the size and weight of the image input unit 26, the load on the actuator unit 29b can be reduced and the slave unit can be downsized. The weight and size of can be reduced.

As described above, in the second embodiment,
Since the actuator part of the master side device and the slave side device and the control part thereof can be configured by the same thing, the efficiency of design and production is improved, and the control signal can also use the head position data of the displacement detection part as it is, so the control program is also It can operate with almost the same.

As described above in detail, in the head-following display device of the present invention, the head position detection sensor is attached to the display device side to reduce the burden of equipment such as the position sensor on the observer side and to reduce fatigue. Both hands can be used freely while operating the display device.

[0045]

According to the present invention, in a high-definition high-quality display device such as a high-definition large-sized display device or a new type display device such as a laser-type retinal direct scanning display device in the future, an observer in an artificial reality environment It is possible to provide a head-following display device that enables simultaneous movement of the head and use of both hands, and further reduces the burden of wearing on the observer.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a head following display device of the present invention.

2A and 2B are schematic views of a reference reflection plate 4. FIG.

FIG. 3 is a diagram for explaining the principle of distance displacement measurement for detecting displacement in the Z direction.

FIG. 4 is a view for explaining the principle of angular displacement measurement that detects displacement in the α and β rotation directions that are rotations of the X and Y axes, that is, the inclination of the X and Y planes, using the principle of regular reflection of light. FIG.

FIG. 5 is a diagram showing a measurement principle of position displacement measurement.

FIG. 6 is a diagram for explaining a signal processing method for measuring the displacement measurement.

FIG. 7 is a diagram showing a state in which a detection side is arranged so that alignment marks 4b and 4d of a reference reflection plate 4 can be measured.

FIG. 8 is a diagram showing a configuration in which a detector capable of measuring relative displacement between a reference reflector 4 and an observer's head is incorporated in a display device side.

9 (a) and 9 (b) are views showing a state in which the reference reflection plate 4 is attached to headphones and glasses.

FIG. 10 is a first example of the head-following display device of the present invention.
3 is a block diagram showing the configuration of the embodiment of FIG.

11A to 11C are diagrams showing a state in which an observer sits on a rotating chair and uses the head-following display device according to the first embodiment.

FIG. 12 is a diagram showing a state in which the arm unit 1 is suspended from the ceiling.

FIG. 13 is a block diagram showing a configuration of a second embodiment.

FIG. 14 is a diagram showing a configuration of a conventional HCD.

[Explanation of symbols]

1 ... Arm part, 2 ... Head part, 3 ... Display part, 4 ... Reflector plate, 5 ... LD drive circuit, 6 ... Light emitting element, 7 ... Light projecting lens, 8 ... Light receiving lens, 9 ... Position detecting element, 10 ... IV
Amplifier, 11 ... Linearizing circuit, 13 ... Quadrant PD,
14 ... Half mirror, 15 ... Illumination light source, 16 ... Objective lens, 17 ... Current-voltage converter, 18 ... Summing amplifier, 19 ...
Differential amplifier, 20 ... Image processing unit, 21 ... Displacement data processing unit, 22 ... Actuator control unit, 23 ... Central control processing unit, 24 ... Video signal receiving unit, 25 ... Displacement detection unit, 26 ...
Image input unit, 27 ... Actuator control unit, 28 ... Video signal transmission unit, 29 ... Actuator unit, 43 ... Arithmetic circuit.

Claims (1)

[Claims] 1. A display device for generating a stereoscopic image of a modeled artificial video environment in real time, the display device displaying the stereoscopic image, and the relative position of the display device and the observer's head. From the relative position detection means, relative position detection means for detecting the relationship and outputting as displacement data, actuator means controlled based on the output data of the relative position detection means, and movable for the display means in space. A head-following display device, comprising: a display control unit that switches the stereoscopic image displayed by the display unit based on the displacement data.