CN110010138A - Head-mounted display and its control method - Google Patents

Abstract

The present invention provides a kind of head-mounted display and its control method, and the smooth of degree that the people of the wearer and outside that can be realized head-mounted display can mutually confirm that both sides are intended to exchanges.Head-mounted display (10) has: radio reception portion (11), sound of the radio reception from specific direction；Speech detection portion (14), from by radio reception portion (11) radio reception to sound in detect voice；Headwork test section (15) detects the movement on the head of the wearer of head-mounted display (10)；External display (16), the information that display can confirm from external view；And response test section (17), in the case where being detected voice by speech detection portion (14) and being detected the movement of the head of wearer towards specific direction by headwork test section (15), it will indicate that wearer has answered the information of voice and shows on external display (16).

Description

Head mounted display and control method thereof

technical field

The present invention relates to a head mounted display and a control method thereof.

Background technique

A user wearing the head-mounted display (hereinafter, also referred to as a "wearer") can concentrate on video and voice, but on the other hand, it is difficult to communicate with an outside person.

Therefore, a technique for facilitating communication between a wearer of a head-mounted display and an external person has been conventionally proposed (refer to Patent Document 1).

Patent Document 1: International Publication No. 2014/156388

SUMMARY OF THE INVENTION

However, in the technique of Patent Document 1, although the communication between the wearer of the head-mounted display and the external person is facilitated to some extent, there is a problem that smooth communication cannot be achieved to the extent that the intentions of both parties are mutually confirmed.

Therefore, the present invention provides a head-mounted display and a control method thereof, which can realize smooth communication to such an extent that a wearer of the head-mounted display and an external person can mutually confirm the intentions of both parties.

A head-mounted display according to one aspect of the present invention is a head-mounted display that presents images, and includes a sound pickup unit that pickups sound from a specific direction; a voice detection unit that detects a sound from the sound picked up by the sound pickup unit; a head motion detection unit that detects the motion of the head of the wearer of the head mounted display; an external display that displays information that can be visually recognized from the outside; When the movement of the head of the wearer in the specific direction is detected by the head movement detection unit, information indicating that the wearer has responded to the voice is displayed on the external display.

A method for controlling a head-mounted display according to one aspect of the present invention is a method for controlling a head-mounted display that presents an image, including: a sound pickup step of picking up sound from a specific direction; and a voice detection step of picking up sound from the sound pickup step to A head movement detection step detects the movement of the head of the wearer of the head-mounted display; and a response detection step, after the speech is detected by the speech detection step, and the head is detected by the head When the movement detection step detects the movement of the head of the wearer in the specific direction, displaying information indicating that the wearer has responded to the voice on an external display that can display information that can be viewed from the outside Information for visual confirmation.

A head-mounted display according to one aspect of the present invention is a head-mounted display that presents images, and includes a sound pickup unit that pickups sound from a specific direction; a voice detection unit that detects a sound from the sound picked up by the sound pickup unit; and a body an action detection unit that detects the orientation of the body of the wearer of the head-mounted display; an external display that displays information that can be visually recognized from the outside; and a response detection unit that detects a voice by the voice detection unit and is When the body movement detection unit detects the movement of the wearer's body in the specific direction, the external display displays information indicating that the wearer has responded to the voice.

A method for controlling a head-mounted display according to one aspect of the present invention is a method for controlling a head-mounted display that presents an image, including: a sound pickup step of picking up sound from a specific direction; and a voice detection step of picking up sound from the sound pickup step to A body motion detection step detects the orientation of the body of the wearer of the head-mounted display; and a response detection step is performed when the voice is detected by the voice detection step, and the body motion detection step is performed by the body motion detection step. When it is detected that the wearer's body is facing the specific direction, information indicating that the wearer has responded to the voice is displayed on an external display that displays information that can be visually confirmed from the outside.

A head-mounted display according to one aspect of the present invention is a head-mounted display that presents images, and includes: a sound pickup unit that pickups sound from a specific direction; a voice detection unit that detects a sound from the sound picked up by the sound pickup unit; and a wrist. a motion detection unit that detects motion of a hand or arm of a wearer of the head-mounted display; an external display that displays information that can be visually recognized from the outside; and a response detection unit that detects a voice by the voice detection unit, and When a predetermined movement of the wearer's hand or arm is detected by the wrist movement detection unit, information indicating that the wearer has responded to the voice is displayed on the external display.

A method for controlling a head-mounted display according to one aspect of the present invention is a method for controlling a head-mounted display that presents an image, including: a sound pickup step of picking up sound from a specific direction; and a voice detection step of picking up sound from the sound pickup step to The voice is detected in the sound of the head-mounted display; the wrist motion detection step detects the motion of the hand or arm of the wearer of the head-mounted display; In the detection step, when a predetermined movement of the wearer's hand or arm is detected, information indicating that the wearer has responded to the voice is displayed on an external display that displays information that can be visually recognized from the outside .

In addition, these general or specific forms can be implemented by systems, methods, integrated circuits, computer programs, or storage media such as computer-readable CD-ROMs, and can also be implemented by systems, methods, integrated circuits, computer programs, and storage media. any combination of .

【Invention effect】

According to the present invention, a head-mounted display and a control method thereof are provided, which can realize smooth communication to such an extent that a wearer of the head-mounted display and an external person can mutually confirm their intentions.

Description of drawings

FIG. 1 is an external view of a head mounted display according to an embodiment.

FIG. 2 is a block diagram showing the configuration of the head mounted display shown in FIG. 1 .

3 is a flowchart showing an operation of the head mounted display according to the embodiment.

4 is a block diagram showing a configuration of a head-mounted display according to Modification 1 of the embodiment.

FIG. 5 is a flowchart showing an operation of the head mounted display according to Modification 1 of the embodiment.

6 is a block diagram showing a configuration of a head-mounted display according to Modification 2 of the embodiment.

FIG. 7 is a flowchart showing an operation of the head mounted display according to Modification 2 of the embodiment.

8 is a block diagram showing a configuration of a head-mounted display according to Modification 3 of the embodiment.

FIG. 9 is a flowchart showing the operation of the head mounted display in Modification 3 of the embodiment.

10A is an external view of a steering apparatus constituting a remote control system according to Modification 4 of the embodiment.

10B is an external view of an automobile constituting the remote control system of Modification 4 of the embodiment.

11 is a block diagram showing a main configuration of a remote control system according to Modification 4 of the embodiment.

12 is a block diagram showing a configuration of a head-mounted display according to Modification 5 of the embodiment.

FIG. 13 is a flowchart showing the operation of the head mounted display according to Modification 5 of the embodiment.

14 is a block diagram showing a configuration of a head-mounted display according to Modification 6 of the embodiment.

FIG. 15 is a flowchart showing the operation of the head mounted display according to Modification 6 of the embodiment.

16 is a diagram showing a display example on an external display of the head mounted display according to Modification 6 of the embodiment.

17 is a block diagram showing a configuration of a head-mounted display according to Modification 7 of the embodiment.

18 is a flowchart showing the operation of the head mounted display according to the seventh modification of the embodiment.

19 is a block diagram showing a configuration of a head-mounted display according to Modification 8 of the embodiment.

FIG. 20 is a flowchart showing the operation of the head mounted display in Modification 8 of the embodiment.

[Description of reference numerals]

10. 10a～10g head mounted display

11, 11a, 11b Radio section

12a～12d Microphone

13, 13a, 13b Directivity processing unit

14 Voice Detection Section

15 Head Movement Detection Section

15A Body Motion Detection Section

15B Wrist motion detection unit

16 External monitor

17, 17a～17c Response detection part

18 Speaker Recognition Section

19 Sound source detection section

20 Voice Overlap Section

30 Video reproduction section

31 Content Storage

34 Internal display

35 Headphones

36 speakers

37 Sound Amplifier

40 Remote control system

41 Driving gear

45 cars

46 bikes

Detailed ways

(Insights that form the basis of the present invention)

In recent years, head-mounted displays have become widespread as display devices that are worn on the head of a user and that allow the user to view video and audio. As the use of the head-mounted display, for example, airlines have also appeared to provide passengers with in-flight entertainment using the head-mounted display as an in-flight service on the plane.

Passengers wearing the head-mounted display can concentrate on video and audio, but on the other hand, there is a problem that it is difficult to obtain communication with external people. For example, even if a cabin crew member makes a sound to a passenger, there is no way to identify whether the passenger is noticing them.

Therefore, it is desirable for passengers to recognize a cabin crew member approaching themselves while wearing the head-mounted display, and to easily communicate with the cabin crew while viewing and listening to the content without removing the head-mounted display as much as possible. In addition, cabin crew members want to know if passengers wearing head-mounted displays are noticing them.

Here, the technique disclosed in Patent Document 1 is proposed as a technique for facilitating communication between the wearer of the head-mounted display and an external person. In this technique, after notification (notification by video and voice) to the wearer of the head-mounted display is performed, the meaning is displayed on the outside of the head-mounted display. With this display, an outside person can know that the notification to the wearer has been performed.

However, in the technique of Patent Document 1, the notification to the wearer of the head-mounted display is displayed on the outside of the head-mounted display, and it does not necessarily mean that the wearer of the head-mounted display notices the notification. Displayed on the outside of the head-mounted display. For example, when the wearer is notified by voice, the wearer does not notice the notification when the volume of the content reproduced by the head mounted display is high. In the technique of Patent Document 1, even in such a case, the notification to the wearer is displayed on the outside of the head mounted display. That is, in the technique of Patent Document 1, an outside person does not know whether or not the wearer notices the notification.

In addition, the technique of Patent Document 1 presupposes that the wearer of the head-mounted display must pay attention to the notification. In order to ensure this, when the notification to the wearer is based on a video, a video that is easily noticed by the wearer is displayed. Therefore, viewing and listening of the content on the head-mounted display is hindered. On the other hand, when the notification to the wearer is based on a voice, in order to provide a voice that the wearer can easily notice, processing such as temporarily stopping or reducing the voice of the content reproduced by the head mounted display is required. Therefore, the viewing and listening of the content of the head-mounted display is still disturbed.

Furthermore, in the technique of Patent Document 1, the intention of the wearer of the head-mounted display, that is, whether or not to communicate with an external person is not reflected in the display on the outside of the head-mounted display.

As described above, according to the technique of Patent Document 1, although the wearer of the head-mounted display can communicate with an external person to some extent, there is a problem that smooth communication cannot be achieved to the extent that the intentions of both parties are mutually confirmed.

Therefore, the present invention provides a head-mounted display and a control method thereof, which can realize smooth communication to such an extent that a wearer of the head-mounted display and an external person can mutually confirm the intentions of both parties.

Therefore, in the head-mounted display of one embodiment of the present invention, the position (sound coordinates) of an external person who emits a sound to the wearer of the head-mounted display is determined in advance. Then, when it is detected that the wearer of the head-mounted display has moved the head, body, hand or arm in the direction of the sound coordinates, it is determined that the wearer of the head-mounted display has reacted to the external sound, and This is displayed on the outside of the wearable display.

Thereby, the wearer of the head-mounted display can recognize the notification from the outside. Therefore, the wearer of the head-mounted display can perform notification without interfering with the content, that is, it is possible to perform a natural notification while viewing and listening to the content. In addition, the intention of the wearer of the head-mounted display, that is, whether or not he wants to communicate with the outside is reflected in the display on the outside of the head-mounted display.

That is, according to the present invention, a head-mounted display capable of realizing smooth communication to such an extent that the wearer of the head-mounted display and an external person can mutually confirm their intentions is realized.

More specifically, a head-mounted display according to one aspect of the present invention is a head-mounted display that presents images, and includes a sound pickup unit that pickups sound from a specific direction, and a voice detection unit that detects from the sound picked up by the sound pickup. a voice; a head motion detection unit that detects motion of the head of the wearer of the head-mounted display; an external display that displays information that can be visually recognized from the outside; and a response detection unit that detects when the head movement detection unit detects the movement of the wearer's head in the specific direction, displaying information indicating that the wearer has responded to the speech on the external monitor.

In this way, when it is detected that the head of the wearer of the head-mounted display is facing a specific direction in the sound source direction of a voice uttered by a person set outside, information indicating that the wearer responded to the voice is displayed on the external monitor. Therefore, when the wearer expresses the intention to respond to the voice of the external person, since the intention is displayed on the external display, smooth communication can be performed to the extent that the wearer and the external person mutually confirm their intentions.

Here, the head mounted display may further include a speaker recognition unit that determines whether or not the speech detected by the speech detection unit is the speech of a predetermined person, and the response detection unit determines whether the speech detected by the speech detection unit is the speech of a predetermined person. The speaker recognition unit determines that the voice is the voice of a predetermined person, and when the head movement detection unit detects the movement of the wearer's head in the specific direction, the voice The information is displayed on the external display.

Accordingly, when the wearer of the head-mounted display expresses an intention to respond to the voice from a predetermined person, the intention is displayed on the external display. Accordingly, it is suppressed that the wearer has responded to an unrelated person's voice which is noise affecting the wearer, and the response accuracy of the head mounted display is improved.

In addition, the head-mounted display may further include a sound source detection unit that detects a sound source, and the sound pickup unit collects sound in a direction of the sound source detected by the sound source detection unit as sound from the specific direction .

Thereby, since the direction of the sound source is detected by the sound source detection unit, it is not necessary to fix the voice of an outside person to the voice from a specific direction. That is, when the wearer responds to the voice from all directions, the meaning is displayed on the external display, and the convenience of the head-mounted display is improved.

In addition, the head-mounted display may further include a headphone for providing the wearer with a sound accompanying the video, and a voice superimposing unit, and the voice superimposing section causes the voice to be generated by the headphone. The voice detected by the detection unit is superimposed on the audio accompanying the video and output to the headphone.

In this way, since the voice of the external person is superimposed on the sound accompanying the video and output from the headphone, the wearer of the head-mounted display can listen to the voice of the content being reproduced while listening to the voice of the external person. voice. Therefore, smoother communication can be performed between the wearer and an external person.

Further, the voice superimposing unit superimposes the voice accompanying the video after performing sound image localization so that the voice detected by the voice detecting unit can be heard from a direction corresponding to the specific direction.

Thus, through sound image localization, since the voice of an outside person can be heard from the actual sound source direction or the corresponding sound source direction, the wearer of the head-mounted display can, as in the case where the head-mounted display is not worn, can be heard. It is easy to detect the direction of the external person making the speech.

In addition, the head-mounted display may further include a headphone for providing sound accompanying the video to the wearer, and when the response detection unit displays the information on the external display, the supply of sound may be reduced The volume of the sound to the headset.

As a result, when an outside person utters a voice toward the wearer, the volume of the sound to be provided to the headphone is lowered. Therefore, the wearer of the head-mounted display can hear the voice of an outside person while wearing the head-mounted display.

In addition, the response detection unit may perform the detection when a voice is detected by the voice detection unit, and a movement of the wearer's head in the specific direction is detected by the head movement detection unit. Control for changing the direction in which the sound pickup unit picks up sound so as to pick up the voice uttered by the wearer.

As a result, after the wearer responds to the voice from the outside person, the direction of the sound collection is changed so that the voice from the wearer is picked up, so that the voice from the wearer is continuously picked up. Therefore, smoother communication can be performed between the wearer and an external person.

In addition, another aspect of the present invention is a control method of a head-mounted display. The control method is a control method of a head-mounted display presenting an image, and includes: a sound collection step, which collects sound from a specific direction; a voice detection step, which detects a voice from the sound picked up by the sound collection step; head movement detection step of detecting the movement of the head of the wearer of the head-mounted display; and a response detection step of detecting the voice by the voice detecting step and detecting the movement of the wearer by the head movement detecting step When the head is moved in the specific direction, information indicating that the wearer has responded to the voice is displayed on an external display that displays information that can be visually recognized from the outside.

Further, a head-mounted display according to one aspect of the present invention is a head-mounted display that presents images, and includes a sound pickup unit that pickups sound from a specific direction, and a voice detection unit that detects a sound from the sound picked up by the sound pickup unit. a body motion detection unit that detects the orientation of the body of the wearer of the head-mounted display; an external display that displays information that can be visually recognized from the outside; When the body motion detection unit detects that the wearer's body faces the specific direction, information indicating that the wearer has responded to the voice is displayed on the external display.

As a result, when it is detected that the body of the wearer of the head-mounted display is facing a specific direction in the sound source direction of the voice uttered by a person set outside, information indicating that the wearer responded to the voice is displayed on the external display . Therefore, when the wearer expresses the intention to respond to the voice of the external person, since the intention is displayed on the external display, smooth communication can be performed to the extent that the wearer and the external person mutually confirm their intentions.

Moreover, another aspect of this invention is the control method of a head mounted display. The control method is a control method of a head-mounted display presenting an image, and includes: a sound collection step, which collects sound from a specific direction; a voice detection step, which detects a voice from the sound picked up by the sound collection step; and a body motion detection step, which includes: an action of detecting the orientation of the body of the wearer of the head-mounted display; and a response detection step in which a voice is detected by the voice detection step and the body orientation of the wearer is detected by the body motion detection step In the case of the specific direction, information indicating that the wearer responded to the voice is displayed on an external display, and the external display displays information that can be visually confirmed from the outside.

Further, a head-mounted display according to one aspect of the present invention is a head-mounted display that presents images, and includes a sound pickup unit that pickups sound from a specific direction, and a voice detection unit that detects a sound from the sound picked up by the sound pickup unit. a wrist motion detection unit that detects the motion of the hand or arm of the wearer of the head-mounted display; an external display that displays information that can be visually recognized from the outside; And when the predetermined movement of the wearer's hand or arm is detected by the wrist movement detection unit, information indicating that the wearer has responded to the voice is displayed on the external display.

Accordingly, when a predetermined movement of the hand or arm of the wearer of the head mounted display is detected, information indicating that the wearer responded to the voice is displayed on the external display. Therefore, when the wearer expresses the intention to respond to the voice of the external person, since the fact is displayed on the external display, smooth communication can be performed to the extent that the wearer and the external person mutually confirm their intentions.

Moreover, another aspect of this invention is the control method of a head mounted display. The control method is a control method of a head-mounted display presenting an image, and includes: a sound collection step, which collects sound from a specific direction; a voice detection step, which detects the voice from the sound picked up by the sound collection step; and a wrist motion detection step, which includes: detecting movement of the hand or arm of the wearer of the head-mounted display; and a response detection step, in which a voice is detected by the voice detection step and the wearer's hand or arm is detected by the wrist motion detection step In the case of a predetermined movement of the arm, information indicating that the wearer has responded to the voice is displayed on an external display that displays information that can be visually recognized from the outside.

Another aspect of the present invention is a program executed by a CPU, a processor, or a computer, or a storage medium such as a computer-readable disk or a semiconductor memory that stores the program. The program includes steps constituting the above-described control method of the head-mounted display.

Hereinafter, the embodiment will be described in detail with reference to the drawings. In addition, the embodiments described below all represent general or specific examples of the present invention. Numerical values, shapes, materials, components, arrangement positions of components, connection methods, steps, order of steps, etc. shown in the following embodiments are examples, and do not limit the gist of the present invention. In addition, among the components in the following embodiments, components that are not described in the independent claims representing the highest-level concept of the present invention will be described as arbitrary components. In addition, each figure is not necessarily a strictly shown figure. In each drawing, the same reference numerals are attached to substantially the same structures, and overlapping descriptions are omitted or simplified.

(Embodiment)

FIG. 1 is an external view of a head mounted display 10 according to the embodiment. This head-mounted display 10 is a head-mounted display capable of realizing smooth communication to the extent that the intention of both parties can be mutually confirmed with an external person when the wearer of the head-mounted display 10 wears the head mounted display 10 . The head-mounted display 10 includes microphones 12 a to 12 d , an external display 16 , and a headphone 35 as components visible from the outside.

The microphones 12 a to 12 d are microphones for detecting the direction of the sound source of the sound emitted from the front of the head-mounted display 10 , and are provided at four locations on the front of the head-mounted display 10 , at the top, bottom, left, and right. In addition, it is not necessary to provide four microphones on the head-mounted display 10, and for example, two microphones may be provided separately only in the up-down direction, or only in the left-right direction.

The external display 16 is a display that displays information that can be visually recognized from the outside, and is, for example, a lamp such as an LED or a two-dimensional display such as an LCD. In addition, the so-called "information that can be visually recognized from the outside" may be any of simple lighting of lamps, characters, symbols, icons, images, and a mixture of at least two of them.

The headphone 35 is a headphone that provides the wearer with sound accompanying the video provided by the head-mounted display 10 , and is an open-type or closed-type headphone. In the present embodiment, it is preferable that the headphones 35 are open-type headphones so as to output the sound accompanying the video reproduced by the head-mounted display 10 so that the voice of an external person can also be transmitted to the head-mounted display. The wearer of 10 heard.

FIG. 2 is a block diagram showing the configuration of the head mounted display 10 shown in FIG. 1 . The head mounted display 10 includes a sound pickup unit 11 , a voice detection unit 14 , a head movement detection unit 15 , an external display 16 , a response detection unit 17 , a video reproduction unit 30 , an internal display 34 , and a headphone 35 .

The sound pickup unit 11 is a processing unit that picks up sound from a specific direction, and includes four microphones 12 a to 12 d and a directivity processing unit 13 . The directivity processing unit 13 performs signal processing on the audio signals from the four microphones 12 a to 12 d to pick up only the audio from a specific direction. As signal processing, there is a delay beamforming method that emphasizes sound in a specific direction by staggering the delays of the audio signals from the four microphones 12a to 12d, or a delay type beamforming method that generates sound in a specific direction and other unnecessary directions. An adaptive beamforming method that removes the sounds of undesired directions from the sounds of a specific direction. Alternatively, there is a method of picking up sound from a specific direction using a single directional microphone. Alternatively, there is a method in which a plurality of unidirectional microphones are prepared to pick up sounds in different directions, and only a unidirectional microphone corresponding to a specific direction is selected.

In addition, the term "specific direction" refers to a case where the positional relationship between the wearer of the head-mounted display 10 and a person outside the person talking to the wearer is determined to a certain degree, for example, when the wearer is limited to wearing the head-mounted display 10 in an airplane. In the case of a passenger and a cabin crew member of the head-mounted display 10, a predetermined fixed direction (for example, the oblique forward-upward direction close to the aisle). However, the "specific direction" is not limited to a fixed direction, and may be a dynamically changing direction indicated by the outside.

In addition, the number of microphones constituting the sound pickup unit 11 is not limited to four. For example, the sound pickup section 11 may be constituted by three or more microphones as long as the microphones pick up sound in a specific direction.

The voice detection unit 14 is a processing unit that detects voice from the voice picked up by the sound pickup unit 11. For example, when the power in the frequency band of the voice in the signal output from the sound pickup unit 11 exceeds a certain time period and a threshold value, the sound is collected from the The signal output by the unit 11 is determined to be a voice, and the response detection unit 17 is notified of the detection of the voice.

The head motion detection unit 15 is a sensor that detects the motion of the head of the wearer of the head mounted display 10 and outputs a signal indicating the detection result to the response detection unit 17 , for example, a gyro sensor and an acceleration sensor. at least one of. In addition, the head movement detection unit 15 may detect the movement of the head by detecting the line of sight of the wearer using a camera or the like.

When the voice detection unit 14 detects a voice and the head movement detection unit 15 detects that the wearer's head moves in a specific direction in which the sound is picked up by the sound pickup unit 11, the response detection unit 17 will indicate the wearer's response to the sound. The information that the voice has been answered is displayed on the processing unit of the external display 16 . In addition, "the wearer's head faces a specific direction" means "the wearer's face faces a specific direction".

The internal display 34 is a display that presents images to the wearer of the head mounted display 10 , and is, for example, a 3D image panel.

The video reproduction unit 30 has a content storage unit 31 that stores content, reads out the content from the content storage unit 31 and reproduces the content in accordance with an instruction given by the wearer through a remote controller (not shown) or the like, and outputs a video signal to the internal display 34 The processing unit that outputs the audio signal accompanying the video to the headphone 35 is composed of, for example, a video decoder, a processor having a program, and the like.

Typically, the directivity processing unit 13 , the voice detection unit 14 , and the response detection unit 17 are composed of a system circuit composed of a ROM and RAM for holding programs, a processor for executing programs, and input/output circuits for connecting to peripheral circuits. Or a microcomputer is implemented in software. However, these can also be implemented in hardware by a dedicated logic circuit such as an ASIC or a gate array that implements respective processes.

Next, the operation of the head mounted display 10 of the present embodiment configured as above will be described.

FIG. 3 is a flowchart showing the operation of the head mounted display 10 according to the present embodiment (ie, the control method of the head mounted display 10 ). Here, it is assumed that a passenger on an airplane wears the head mounted display 10 in a state of sitting on a chair and reproduces content.

First, the sound pickup unit 11 picks up sound from a specific direction (sound pickup step S10). Specifically, the directivity processing unit 13 performs signal processing such as delay beamforming on the audio signals from the four microphones 12a to 12d, and picks up only the audio from a specific direction (here, the oblique front upward direction).

Next, the voice detection unit 14 detects a voice from the voice picked up by the voice pickup unit 11 (voice detection step S11 ). Specifically, the speech detection unit 14 determines that the signal output from the sound pickup unit 11 is speech when the power in the frequency band of speech exceeds a certain time and a threshold value, and detects The response detection unit 17 is notified of the voice status. As a result, the response detection unit 17 determines whether or not a voice from diagonally forward and upward is detected, that is, whether or not the cabin crew member is talking to the wearer of the head mounted display 10 .

When the voice detection unit 14 detects a voice (YES in S11 ), the response detection unit 17 proceeds to the processing after step S12 , otherwise (NO in S11 ), the processing ends.

The head movement detection unit 15 detects the movement of the head of the wearer of the head mounted display 10, and outputs a signal indicating the detection result to the response detection unit 17 (head movement detection step S12).

The response detection unit 17 analyzes the signal output from the head movement detection unit 15 to determine whether the wearer of the head mounted display 10 (here, the passenger) responded to the call from the cabin crew (S13).

Specifically, when the head movement detection unit 15 detects that the wearer's head moves in the specific direction in which the sound is picked up by the sound pickup unit 11 (Yes in S13 ), the response detection unit 17 determines that the wearer has a right to the sound. The call from the cabin crew is answered, and information indicating that the wearer has responded to the voice is displayed on the external display 16 (response detection step S14).

After that, the response detection unit 17 judges whether or not the end condition of the process is satisfied ( S15 ), and continues to display the information indicating that the wearer responded to the voice on the external display 16 until it is satisfied. In addition, the "end condition of processing" refers to the case where the wearer no longer responds to the voice, for example, the head movement detection unit 15 cannot detect the movement of the wearer's head in the specific direction in which the sound is picked up by the sound pickup unit 11 .

On the other hand, when the head movement detection unit 15 cannot detect the movement of the wearer's head in the specific direction in which the sound is picked up by the sound pickup unit 11 (NO in S13 ), the response detection unit 17 determines that the wearer is the wearer If the call from the cabin crew member is not answered, the process ends.

As described above, the head-mounted display 10 of the present embodiment is a head-mounted display that presents images, and includes a sound pickup unit 11 that picks up sound from a specific direction, and a voice detection unit 14 that detects a sound from the sound picked up by the sound pickup unit 11 . The head movement detection unit 15 detects the movement of the head of the wearer of the head mounted display 10; the external display 16 displays information that can be visually confirmed from the outside; When the head movement detection unit 15 detects a voice and the movement of the wearer's head in a specific direction is detected, information indicating that the wearer responded to the voice is displayed on the external display 16 .

In addition, the control method of the head-mounted display 10 of the present embodiment is a control method of the head-mounted display 10 that presents an image, and includes: a sound pickup step S10, which picks up sound from a specific direction; a voice detection step S11, which starts from a sound pickup step S10 The voice is detected in the sound received in the middle; the head movement detection step S12 detects the movement of the head of the wearer of the head-mounted display 10; When the movement of the wearer's head in a specific direction is detected in the part movement detection step S12, information indicating that the wearer has responded to the voice is displayed on the external display 16, and the external display 16 displays information that can be visually confirmed from the outside .

As a result, when it is detected that the face of the wearer of the head mounted display 10 faces a specific direction in the sound source direction of the voice uttered by a person set outside, information indicating that the wearer responded to the voice is displayed on the External Display 16. Therefore, when the wearer expresses an intention to respond to the voice of an external person, since the intention is displayed on the external display 16, smooth communication can be performed to such an extent that the wearer and the external person can mutually confirm their intentions.

In the present embodiment, in the flowchart of FIG. 3 , the head motion detection ( S12 ) is performed after the directivity processing ( S10 ) and the speech detection ( S11 ), but the order is not limited to this. The directionality processing ( S10 ), the voice detection ( S11 ), and the detection of the head movement ( S12 ) may be reversed in order, or may be performed in parallel.

(Variation 1)

Next, a head-mounted display according to Modification 1 of the embodiment will be described.

FIG. 4 is a block diagram showing the configuration of the head-mounted display 10a according to Modification 1 of the embodiment. The head-mounted display 10a is a head-mounted display having a function of recognizing a speaker, and includes a sound pickup unit 11, a speech detection unit 14, a speaker recognition unit 18, a head movement detection unit 15, an external display 16, a response detection unit 17a, Video reproduction unit 30 , internal display 34 , and headphones 35 .

The head-mounted display 10a of the present modification has basically the same configuration as the head-mounted display 10 of the embodiment, but includes the speaker recognition unit 18 as a new component and the response detection unit 17a of the modification This point is different from the embodiment. Hereinafter, the difference from the embodiment will be mainly described.

The speaker recognition unit 18 is a processing unit that determines whether or not the speech detected by the speech detection unit 14 is the speech of a predetermined person, and notifies the response detection unit 17a of the determination result. In this modification, the speaker recognition unit 18 includes a storage unit in which the voices of the cabin crew are registered in advance, and determines whether the voice detected by the voice detection unit 14 matches any of the voices registered in the storage unit.

For example, the speaker identification unit 18 uses i-vectors to extract the feature quantities of the speakers and identify the speakers (Non-Patent Document 1: Misaki Tsujikawa et al., "Study on Speaker Recognition Based on i-vectors for Short Vocabulary", letter Journal of Science and Technology, vol.115, no.99, SP2015-12, pp.65-70, June 2015). That is, in the speaker identification unit 18, the speech is converted into a multidimensional feature vector. The newly input speech is also converted into a feature vector, and it is determined whether or not the input speech is the speech of a cabin crew member according to whether it is close to any of the registered feature vectors.

In addition, the speaker identification unit 18 is typically implemented in software by a system circuit or a microcomputer including a ROM and RAM for holding a program, a processor for executing the program, and an input/output circuit for connecting to peripheral circuits. However, these can also be implemented in hardware by a dedicated logic circuit such as an ASIC or a gate array that implements respective processes.

The response detection unit 17a has basically the same function as the response detection unit 17 of the embodiment. In this modification, the speaker recognition unit 18 determines that the speech is the speech of a predetermined person, and the head movement detection unit When the movement of the wearer's head in a specific direction is detected, 15 displays information on the external display 16 that the wearer has responded to the voice.

FIG. 5 is a flowchart showing the operation of the head mounted display 10a (that is, the control method of the head mounted display 10a) according to this modification. This operation is basically the same as the flowchart shown in FIG. 3 of the embodiment, but differs from the embodiment in that step S20 is added. Hereinafter, the difference from the embodiment will be mainly described.

When a voice is detected by the voice detection unit 14 (YES in S11 ), the speaker recognition unit 18 determines whether or not the voice detected by the voice detection unit 14 is the voice of a predetermined person, and notifies the response of the determination result. Detection unit 17a (S20). Specifically, the speaker recognition unit 18 determines whether or not the feature vector of the speech input via the speech detection unit 14 is close to any of the feature vectors of the voices of the cabin crew members registered in advance, and notifies the response detection unit of the determination result. 17a.

When the response detection unit 17a receives notification from the speaker recognition unit 18 that the input voice is the voice of a predetermined person (here, a cabin crew member) (YES in S20), the process proceeds to step S12 and subsequent steps , otherwise (NO in S20), the process ends.

After step S12, the response detection unit 17a determines that the wearer is the wearer when the head movement detection unit 15 detects that the wearer's head moves in the specific direction in which the sound is picked up by the sound pickup unit 11 (YES in S13). It is the same as the embodiment in that the call from the cabin crew is answered, and the information indicating that the wearer has responded to the voice is displayed on the external display 16 (response detection step S14).

In this way, the head mounted display 10a of the present modification includes, in addition to the configuration of the embodiment, the speaker recognition unit 18 that determines whether or not the speech detected by the speech detection unit 14 belongs to a predetermined person. voice. The response detection unit 17a determines that the speech is a predetermined human speech by the speaker recognition unit 18 (S20) and the head movement detection unit 15 detects that the wearer's head is moving in a specific direction (S13) , and the external display 16 displays information indicating that the wearer has responded to the voice.

Thereby, when the wearer of the head mounted display 10 a has expressed the intention to respond to the voice from a predetermined person, the intention is displayed on the external display 16 . Therefore, it is possible to suppress erroneous recognition that the wearer has responded to a voice from an unrelated person that is noise that suddenly affects the wearer, thereby improving the response accuracy of the head mounted display 10a.

(Variation 2)

Next, a head-mounted display according to Modification 2 of the embodiment will be described.

FIG. 6 is a block diagram showing a configuration of a head-mounted display 10b according to Modification 2 of the embodiment. The head-mounted display 10b is a head-mounted display having a function of detecting a sound source, and includes a sound pickup unit 11a, a voice detection unit 14, a head movement detection unit 15, an external display 16, a response detection unit 17, a sound source detection unit 19, Video reproduction unit 30 , internal display 34 , and headphones 35 .

The head mounted display 10b of the present modification has basically the same configuration as the head mounted display 10 of the embodiment, but includes the sound source detection unit 19 as a new component and the sound pickup unit 11a of the modification The above is different from the embodiment. Hereinafter, the difference from the embodiment will be mainly described.

The sound source detection unit 19 is a processing unit that detects a sound source. Specifically, the sound source detection unit 19 uses the four microphones 12a to 12d to detect the direction or position (coordinates in the three-dimensional space) of the speech generated in the surroundings. For example, the sound source detection section 19 determines the direction of the sound source in the plane including the two microphones, based on the time difference between the same speech arriving at the two microphones provided at a distance, respectively. Alternatively, the sound source detection unit 19 uses two microphones arranged at a distance from each other as a pair, and determines the intersection of the two sound source directions as the position of the sound source by using two pairs of microphones arranged in different directions.

The sound source detection unit 19 is typically implemented in software by a system circuit or a microcomputer including a ROM and RAM holding a program, a processor for executing the program, and an input/output circuit for connecting to peripheral circuits. However, these can also be implemented in hardware by a dedicated logic circuit such as an ASIC or a gate array that implements respective processes.

The sound pickup unit 11 a picks up the sound in the direction of the sound source detected by the sound source detection unit 19 as the sound from a specific direction. Specifically, the directivity processing unit 13a constituting the sound pickup unit 11a performs signal processing on the sound signals from the four microphones 12a to 12d to pick up only the sound from the sound source direction detected by the sound source detection unit 19, Only the sound from the direction of the sound source detected by the sound source detection unit 19 is picked up. Signal processing is the same as that of the sound pickup unit 11 of the embodiment, for example.

FIG. 7 is a flowchart showing the operation of the head mounted display 10b (that is, the control method of the head mounted display 10b) according to this modification. This operation is basically the same as the flowchart shown in FIG. 3 of the embodiment, but differs from the embodiment in that step S22 is added. Hereinafter, the difference from the embodiment will be mainly described.

The sound source detection unit 19 detects a sound source ( S22 ) before the directivity processing ( S10 ) by the directivity processing unit 13 a . For example, the sound source detection unit 19 detects the position (coordinates in the three-dimensional space) of the sound source using the four microphones 12a to 12d. In addition, when the sound source cannot be detected (NO in S22), the process ends.

On the other hand, when the sound source is detected (YES in S22 ), the directivity processing unit 13 a of the sound pickup unit 11 a regards the sound in the direction of the position of the sound source detected by the sound source detection unit 19 as the sound from the sound source detection unit 19 . The sound in a specific direction is picked up (S10). Specifically, the directivity processing unit 13a performs signal processing on the sound signals from the four microphones 12a to 12d to pick up only the sound from the direction of the sound source detected by the sound source detection unit 19, so as to pick up only the sound from the direction of the sound source detected by the sound source detection unit 19. The sound in the direction of the sound source detected by the sound source detection unit 19 . The subsequent processing (S11-) is the same as that of the embodiment.

In this way, the head mounted display 10b of the present modification is provided with the sound source detection unit 19 for detecting the sound source in addition to the configuration of the embodiment. The sound pickup unit 11a picks up the sound in the direction of the sound source detected by the sound source detection unit 19 as the sound from a specific direction.

Thereby, since the sound source detection unit 19 detects the direction of the sound source, the external human voice does not need to be fixed to the voice from a specific direction. That is, when the wearer responds to the voice from all directions, the meaning is displayed on the external display 16, and the convenience of the head mounted display 10b is improved.

(Variation 3)

Next, a head-mounted display according to Modification 3 of the embodiment will be described.

FIG. 8 is a block diagram showing a configuration of a head-mounted display 10c according to Modification 3 of the embodiment. The head-mounted display 10c is a head-mounted display having a function of superimposing the voice of an external person on the sound accompanying video reproduction and outputting to the headphones 35, and includes a sound pickup unit 11a, a voice detection unit 14, and a sound source The detection unit 19 , the voice superimposition unit 20 , the head movement detection unit 15 , the external display 16 , the response detection unit 17 , the video reproduction unit 30 , the internal display 34 , and the headphone 35 .

The head-mounted display 10c of the present modification has basically the same configuration as the head-mounted display 10b of the modification 2 of the embodiment, but differs from the modification of the embodiment in that the voice superimposing unit 20 is provided as a new component. 2 different. Hereinafter, the difference from Modification 2 of the embodiment will be mainly described.

The voice superimposing unit 20 is a processing unit that superimposes the voice detected by the voice detecting unit 14 on the audio accompanying the video and outputs it to the headphone 35 . At this time, the voice superimposing section 20 performs sound image localization so that the voice detected by the voice detecting section 14 is heard from the direction corresponding to the specific direction, and on this basis is superimposed on the sound accompanying the image. Specifically, the speech superimposing unit 20 performs signal processing of sound image localization on the speech detected by the speech detecting unit 14 so that the speech is heard stereoscopically (ie, from the direction detected by the sound source detecting unit 19 ), and On this basis, the reproduction channel is allocated to the headphone 35 and output. For example, considering the shape of the human head (ie, using the head transfer function), the voice superimposing section 20 performs signal processing of sound image localization on the voice detected by the voice detecting section 14 even if the headphone 35 is 2ch , and can also hear the voice of an external person stereoscopically.

In addition, in this modification, since the voice of an external person is heard by the wearer of the head mounted display 10c, the headphone 35 may be a closed type headphone.

In addition, the speech superimposing unit 20 is typically implemented in software by a system circuit or a microcomputer including a ROM and RAM for holding a program, a processor for executing the program, and an input/output circuit for connecting to peripheral circuits. However, these can also be implemented in hardware by a dedicated logic circuit such as an ASIC or a gate array that implements respective processes.

FIG. 9 is a flowchart showing the operation of the head-mounted display 10c (that is, the control method of the head-mounted display 10c) according to this modification. This operation is basically the same as the flowchart shown in FIG. 7 of the second modification of the embodiment, but differs from the second modification of the embodiment in that steps S24 and S26 are added. Hereinafter, the difference from Modification 2 of the embodiment will be mainly described.

In the present modification, when a voice is detected by the voice detection unit 14 (YES in S11 ), the voice superimposition unit 20 performs signal processing of sound image localization on the voice detected by the voice detection unit 14 ( S24 ) to The voice is heard from the direction detected by the sound source detection unit 19, superimposed on the sound reproduced by the video reproduction unit 30, and output to the headphone 35 (S26).

The subsequent processing (S13-) is the same as that of the embodiment.

In this way, the head mounted display 10c of the present modification includes, in addition to the configuration of the embodiment, the speech superimposing unit 20 that superimposes the speech detected by the speech detecting unit 14 on the audio accompanying the video and outputs it to the headphones 35 .

As a result, since the voice of an outside person (eg, a cabin crew member) is output from the headphone 35 superimposed on the audio accompanying the video, the wearer of the head-mounted display 10c can view the voice of the content being reproduced while listening to the voice of the content being reproduced. The voices of people outside are also heard. Therefore, smoother communication can be performed between the wearer and an external person.

Further, the voice superimposing section 20 performs sound image localization so that the voice detected by the voice detecting section 14 is heard from a direction corresponding to a specific direction, and on this basis is superimposed on the sound accompanying the image.

Thus, through sound image localization, the wearer of the head-mounted display 10c hears the voice of an outside person (for example, a cabin crew member) from the actual sound source direction or the sound source direction corresponding to the sound source direction. Similarly, in the case of the display 10c, it is possible to easily detect the direction in which the outside person who uttered the voice is located.

In addition, in this modification, in the flowchart of FIG. 9, sound image localization superposition (S24) and reproduction (S26) are performed after the head motion detection (S12), but the order is not limited. The detection ( S12 ) of the head movement, the superposition of the sound image localization ( S24 ), and the reproduction ( S26 ) may be performed in reverse order or in parallel.

(Variation 4)

Next, an example in which the head-mounted display of the present invention is applied to a remote control system of an automobile will be described as a fourth modification of the embodiment. The remote control system consists of a driving device for remotely operating a car and a car, the car is connected with the driving device through wireless communication, and the driving device performs remote control.

10A is an external view of a steering device 41 constituting a remote control system 40 according to Modification 4 of the embodiment. FIG. 10B is an external view of a car 45 constituting the remote control system 40 according to Modification 4 of the embodiment.

As shown in FIG. 10A , the driving device 41 is installed in a house or the like, and is a device that simulates the driver's seat of an automobile 45 . The driving device 41 is constituted by a steering wheel, an accelerator, a brake, and the like for remotely operating the automobile 45 .

A display corresponding to the internal display 34 of the head mounted display of the embodiment (herein referred to as the internal display 34 ) is provided in front of the driving device 41 . In the present modification, the interior display 34 displays a video image transmitted from a camera installed in the car 45 to capture the image of the front of the car 45 . In addition, the driving device 41 is provided with a speaker 36 corresponding to a headphone of the head-mounted display of the embodiment. In the present modification, the speaker 36 reproduces the sound picked up by the microphones 12 a to 12 d mounted on the front of the automobile 45 using a multi-channel speaker.

As shown in FIG. 10B , the automobile 45 is remotely controlled by the driving device 41 and can be driven unmanned. Parts corresponding to the microphones 12 a to 12 d of the head mounted display according to the embodiment (referred to as microphones 12 a to 12 d here) are attached to four parts on the front surface of the automobile 45 . In addition, a device corresponding to the external display 16 of the head-mounted display of the embodiment (herein, referred to as the external display 16 ) is provided in front of the front windshield of the automobile 45 .

FIG. 11 is a block diagram showing a main configuration of a remote control system 40 according to this modification. Here, among the components of the remote control system 40, only the components of the part to which the head mounted display of the embodiment is applied are shown.

The remote control system 40 includes a sound pickup unit 11a, a speech detection unit 14, a sound source detection unit 19, a speech superimposition unit 20, a head movement detection unit 15, an external display 16, a response detection unit 17, a video reproduction unit 30, an internal display 34, and Speaker 36. This configuration corresponds to a configuration in which the headphones 35 in the head-mounted display 10 c of the modification 3 of the embodiment shown in FIG. 8 are replaced by the speakers 36 . However, in this modification, the video reproduction unit 30 is the above-described camera installed in the automobile 45 . In addition, the voice detection unit 14 detects sounds around the vehicle registered in advance as dangerous sounds such as the braking sound of the bicycle 46 shown in FIG. 10B . The head movement detection unit 15 is an acceleration sensor attached to the head of the remote operator who is operating the steering device 41 , or a camera that captures and detects movement of the remote operator's head. In addition, the sound pickup unit 11a, the video reproduction unit 30, and the external display 16 are typically installed in the automobile 45, and the other components are installed in the driving device 41 or installed in the remote operator.

As described above, since this remote control system 40 has the same configuration as the head mounted display 10c of the third modification of the embodiment as a basic function, the same effects as those of the third modification of the embodiment can be achieved. That is, the remote control system 40 is provided with the voice superimposing unit 20 that performs sound image localization of the sound around the vehicle, such as the braking sound of the bicycle 46 detected by the voice detecting unit 14, so as to obtain a sound image that is close to reality. The direction of the source direction hears a sound that predicts danger. Therefore, when the head of the remote operator faces the direction of the sound source of the sound around the vehicle, such as the braking sound of the bicycle 46, as shown in FIG. meaning of sound.

At this time, the external display 16 of the automobile 45 also sees other than the driver of the bicycle 46 . In order to visually indicate to the driver of the bicycle 46 a display indicating that the driver of the bicycle 46 is noticing the bicycle 46 , the display of the external display 16 is displayed in such a manner as to be displayed in a position close to the bicycle 46 , or in the direction of the bicycle 46 . Arrows or eyeball icons, or mirror display of images of bicycles 46 captured by cameras.

Thereby, the driver of the bicycle 46 can visually recognize that the vehicle 45 approaching the driver has noticed the brake sound of the bicycle 46 . Therefore, according to the remote control system 40 to which such a head-mounted display is applied, although the distance between the remote operator and the car and its surroundings is large, the remote operator and the person outside (here, the driver of the bicycle 46 ) smooth communication between them.

(Variation 5)

Next, a head-mounted display according to Modification 5 of the embodiment will be described.

FIG. 12 is a block diagram showing a configuration of a head mounted display 10d according to Modification 5 of the embodiment. The head-mounted display 10d is a head-mounted display having a function of automatically reducing the volume of sound supplied to the headset, and includes a sound pickup unit 11, a voice detection unit 14, a head movement detection unit 15, an external display 16, and a response detection unit. unit 17 b , video reproduction unit 30 , internal display 34 , headphones 35 , and harmony amplifier 37 .

The head-mounted display 10d of the present modification has basically the same configuration as the head-mounted display 10 of the embodiment, but includes the sound amplifier 37 as a new component and the response detection unit 17b of the modification Different from the implementation. Hereinafter, the difference from the embodiment will be mainly described.

The audio amplifier 37 is a volume controller that amplifies or attenuates the audio signal output from the video reproduction unit 30 and outputs it to the headphones 35 in accordance with a control instruction from the response detection unit 17b, and is composed of, for example, a variable gain amplifier.

The response detection unit 17b has basically the same function as the response detection unit 17 of the embodiment, and in this modification, when displaying information indicating that the wearer has responded to the voice on the external display 16, it reduces the amount of information provided to the headset. The sound amplifier 37 is controlled in such a manner that the volume of the sound of the headphones 35 is controlled.

FIG. 13 is a flowchart showing the operation of the head mounted display 10d (that is, the control method of the head mounted display 10d) according to this modification. This operation is basically the same as the flowchart shown in FIG. 3 of the embodiment, but differs from the embodiment in that step S28 is added. Hereinafter, the difference from the embodiment will be mainly described.

The response detection unit 17b controls the sound amplifier 37 to lower the head of the wearer when the head movement detection unit 15 detects the movement of the wearer's head in the specific direction in which the sound is picked up by the sound pickup unit 11 (YES in S13). In addition to the volume of the sound provided by the earphone 35 (S28), information indicating that the wearer responded to the voice is displayed on the external display 16 (response detection step S14).

In this way, the head mounted display 10d of the present modification includes the sound amplifier 37 for adjusting the volume of the sound output from the video reproduction unit 30 in addition to the configuration of the embodiment. The response detection unit 17b controls the sound amplifier 37 so as to reduce the volume of the sound to be supplied to the headset 35 when the information indicating that the wearer has responded to the voice is displayed on the external display 16 .

Thereby, when an outside person utters a voice toward the wearer, the volume of the sound provided to the headphone 35 is lowered. Therefore, the wearer of the head-mounted display 10d can listen to the voice of an external person while wearing the head-mounted display 10d.

(Variation 6)

Next, a head-mounted display according to Modification 6 of the embodiment will be described.

FIG. 14 is a block diagram showing a configuration of a head-mounted display 10e according to Modification 6 of the embodiment. The head-mounted display 10e is a head-mounted display having a function of collecting not only the external person but also the voice uttered by the wearer, and includes a sound collecting unit 11b, a voice detecting unit 14, a head movement detecting unit 15, an external display 16, The response detection unit 17 c , the video reproduction unit 30 , the internal display 34 , and the headphone 35 .

The head mounted display 10e of the present modification has basically the same configuration as the head mounted display 10 of the embodiment, but differs from the embodiment in that it includes the sound pickup unit 11b and the response detection unit 17c of the modification. Hereinafter, the difference from the embodiment will be mainly described.

The sound pickup unit 11b basically has the same function as the sound pickup unit 11 of the embodiment, but in this modification, it includes the directivity processing unit 13b of the modification. Similar to the directivity processing unit 13 of the embodiment, the directivity processing unit 13b has a function of performing signal processing on the sound signals from the four microphones 12a to 12d to pick up only the sound from a specific direction, but in this modification, It also has a function of changing the direction of sound pickup in accordance with a control instruction from the response detection unit 17c so as to pick up the voice uttered by the wearer. Specifically, the directivity processing unit 13b performs, as signal processing, a delay-type beamforming method or the like by shifting the delays of the audio signals from the four microphones 12a to 12d in accordance with a control instruction from the response detection unit 17c. Delay beamforming or the like that emphasizes sound from a predetermined specific direction or from the wearer's direction.

The response detection unit 17c has basically the same function as the response detection unit 17 of the embodiment, and in this modification, further performs control such that when the speech is detected by the speech detection unit 14 and the head motion detection unit 15 detects the speech When the wearer's head moves in a specific direction, the direction in which the sound pickup unit 11b picks up sound is changed so as to pick up the voice uttered by the wearer.

FIG. 15 is a flowchart showing the operation of the head mounted display 10e according to the present modification (ie, the control method of the head mounted display 10e). FIG. 16 is a diagram showing a display example of the external display 16 of the head mounted display 10e of the present modification.

The operation shown in FIG. 15 is basically the same as the flowchart shown in FIG. 3 of the embodiment, except that the display on the external display 16 ( S30 , S33 , S35 ) and the control of the sound pickup unit 11 b ( S31 , S32 ) are added. In this point, it is different from the embodiment. Hereinafter, the difference from the embodiment will be mainly described.

When the voice detection unit 14 detects a voice (YES in S11 ), the response detection unit 17 c displays an image indicating that the voice detected by the voice detection unit 14 is displayed on the external display 16 as in the display example of FIG. 16( b ). That is, an image showing the function of the microphones 12a to 12d when the cabin crew calls the wearer of the head mounted display 10e (here, an image showing the microphones and the like) (S30). As a result, the external display 16 changes from the normal display state (here, no display shown in FIG. 16( a )) to an image showing the functioning of the microphones 12 a to 12 d ( FIG. 16 ( b )). Therefore, the cabin crew can know that their call is detected by the microphones 12a to 12d of the head mounted display 10e.

Next, when the movement of the wearer's head toward the specific direction in which the sound is picked up by the sound pickup unit 11b is detected (YES in S13), the response detection unit 17c performs control to change the direction of sound pickup by the sound pickup unit 11b so that the wearer's sound is picked up. uttered voice (S31). As a result, the sound pickup unit 11b performs directivity processing to pick up the voice of the wearer (S32).

Then, the response detection unit 17c displays, on the external display 16, an image (a question mark here) indicating that the wearer has responded, as shown in the display example of FIG. 16(c) (S33). Thereby, the cabin crew can know that the wearer of the head mounted display 10e has noticed his own call.

After that, the response detection unit 17c judges whether or not the termination condition of the process is satisfied (S15), and if not (NO in S15), it judges whether or not the voice of the wearer is detected via the sound pickup unit 11b and the voice detection unit 14 (S34). ), and when the voice of the wearer cannot be detected (NO in S34), the response display (S33) and the termination determination (S15) are repeated.

On the other hand, when the voice of the wearer is detected (YES in S34 ), the response detection unit 17c displays on the external display 16 that the voice of the wearer is expressed as in the display example of FIG. 16( d ). , that is, an image (here, an exclamation mark) in which the wearer responded to the call of the cabin attendant ( S33 ). Thereby, the cabin crew can know that the wearer of the head-mounted display 10e has responded to his own call.

In addition, after the response detection unit 17c performs the display shown in FIG. 16( d ), a series of conversations in which the wearer of the head mounted display 10e is regarded as responding to the call from the cabin attendant ends, as shown in FIG. 16 . As in the display example of (e), the external display 16 is returned to the normal display state as in FIG. 16(a), and the process is terminated.

In this way, in the head mounted display 10e of the present modification, the response detection unit 17c detects a speech by the speech detection unit 14 and detects the movement of the wearer's head in a specific direction by the head movement detection unit 15 Next, control is performed to change the direction in which the sound pickup unit 11b picks up sound so as to pick up the voice uttered by the wearer.

Thereby, after the wearer turns his face toward the voice from the outside person, the sound pickup direction is changed so that the voice from the wearer is picked up, so that the voice from the wearer is picked up next. Therefore, smoother communication can be performed between the wearer and an external person.

(Variation 7)

Next, a head-mounted display according to Modification 7 of the embodiment will be described.

FIG. 17 is a block diagram showing a configuration of a head-mounted display 10f according to Modification 7 of the embodiment. The head-mounted display 10f is a head-mounted display that enables smooth communication to the extent that the wearer wears the head-mounted display 10f to the extent that the intentions of both parties are mutually confirmed with an external person, and includes a sound pickup unit 11, a voice detection unit 14, The body motion detection unit 15A, the external display 16 , the response detection unit 17 , the video reproduction unit 30 , the internal display 34 , and the headphone 35 .

The head-mounted display 10f of the present modification has basically the same configuration as the head-mounted display 10 of the embodiment, but is different from the embodiment in that a body motion detection unit 15A is provided as a new component instead of the head motion detection unit 15 . different. Hereinafter, the difference from the embodiment will be mainly described.

The body motion detection unit 15A detects the orientation of the body and determines the response to the voice. As a mechanism for detecting the orientation of the body, there are a mechanism for performing image recognition using an image of a camera facing the wearer of the head-mounted display 10f, and a mechanism for attaching an acceleration sensor or a gyro sensor to the body of the wearer of the head-mounted display 10f mechanism. In addition, when the remote controller is in hand, an acceleration sensor or a gyro sensor mounted in the remote controller can be used. The body motion detection unit 15A judges that the voice has been answered based on the fact that the body is oriented in a specific direction.

FIG. 18 is a flowchart showing the operation of the head mounted display 10f according to the present modification (ie, the control method of the head mounted display 10f). This operation is basically the same as the flowchart shown in FIG. 3 of the embodiment, but differs from the embodiment in that step S12 is changed to S12A. Hereinafter, the difference from the embodiment will be mainly described.

The response detection unit 17 proceeds to the processing after step S12A when the voice detection unit 14 detects a voice (YES in S11 ), and ends the processing otherwise (NO in S11 ).

The body motion detection unit 15A detects the body motion of the wearer of the head mounted display 10f, and outputs a signal indicating the detection result to the response detection unit 17 (body motion detection step S12A).

The response detection unit 17 analyzes the signal output from the body motion detection unit 15A to determine whether or not the wearer of the head mounted display 10 (here, the passenger) responded to the call from the cabin crew ( S13 ).

Specifically, when the body motion detection unit 15A detects that the wearer's body is facing the specific direction in which the sound picked up by the sound pickup unit 11 is detected (YES in S13 ), the response detection unit 17 determines that the wearer responds to the sound from the cabin attendant. The call is answered, and information indicating that the wearer has responded to the voice is displayed on the external display 16 (response detection step S14).

After that, the response detection unit 17 judges whether or not the end condition of the process is satisfied ( S15 ), and continues to display the information indicating that the wearer responded to the voice on the external display 16 until it is satisfied. In addition, the "end condition of the process" refers to the case where the wearer no longer responds to the voice, for example, the body movement detection unit 15A cannot detect the movement of the wearer's body in the specific direction picked up by the sound pickup unit 11 .

On the other hand, when the body motion detection unit 15A cannot detect the movement of the wearer's body in the specific direction in which the sound is picked up by the sound pickup unit 11 (NO in S13 ), the response detection unit 17 determines that the wearer does not respond to the The cabin crew responds to the call and ends the process.

As described above, the head-mounted display 10f of the present embodiment is a head-mounted display that presents images, and includes a sound pickup unit 11 that picks up sound from a specific direction, and a voice detection unit 14 that detects a sound from the sound picked up by the sound pickup unit 11. The body motion detection unit 15A detects the orientation of the body of the wearer of the head-mounted display 10f; the external display 16 displays information that can be visually confirmed from the outside; When the body motion detection unit 15A detects that the wearer's body is facing a specific direction, information indicating that the wearer responded to the voice is displayed on the external display 16 .

In addition, the control method of the head-mounted display 10f of the present modification is a control method of the head-mounted display 10f that presents an image, and includes: a sound pickup step S10, which picks up sound from a specific direction; a voice detection step S11, which starts from the sound pickup step S10 The voice is detected in the sound received in the middle; the body motion detection step S12A detects the orientation of the body of the wearer of the head-mounted display 10f; When it is detected in step S12 that the wearer's body is facing the specific direction, information indicating that the wearer has responded to the voice is displayed on the external display 16, and the external display 16 displays information that can be visually confirmed from the outside.

Accordingly, when it is detected that the body of the wearer of the head-mounted display 10f is oriented in a specific direction in the direction of the sound source of the voice uttered by a person set outside, information indicating that the wearer responded to the voice is displayed on the External Display 16. Therefore, when the wearer expresses an intention to respond to the voice of an external person, the fact is displayed on the external display 16, so that the wearer and the external person can communicate smoothly to the extent that the wearer and the external person can mutually confirm their intentions.

In addition, in this modification, in the flowchart of FIG. 18 , body motion detection ( S12A ) is performed after the directivity processing ( S10 ) and the voice detection ( S11 ), but the order is not limited to this. Directivity processing (S10), voice detection (S11), and body motion detection (S12A) may be performed in reverse order or in parallel.

(Variation 8)

Next, a head-mounted display according to Modification 8 of the embodiment will be described.

FIG. 19 is a block diagram showing a configuration of a head-mounted display 10 g according to Modification 8 of the embodiment. The head-mounted display 10g is a head-mounted display capable of realizing smooth communication to the extent that the wearer wears the head-mounted display 10g to the extent that the intentions of both parties are mutually confirmed with an external person, and includes a sound pickup unit 11, a voice detection unit 14, The wrist motion detection unit 15B, the external display 16 , the response detection unit 17 , the video reproduction unit 30 , the internal display 34 , and the headphone 35 .

The head-mounted display 10g of the present modification basically has the same configuration as the head-mounted display 10 of the embodiment, but includes a wrist motion detection unit 15B as a new component instead of the head motion detection unit 15, which is different from the embodiment. different. Hereinafter, the difference from the embodiment will be mainly described.

The wrist motion detection unit 15B detects the motion of the hand or the arm to determine the response to the voice. As a mechanism for detecting the motion of the hand or arm, there are a mechanism for performing image recognition using an image of a camera facing the wearer of the head-mounted display 10g, and an acceleration sensor or a gyro sensor is attached to the body of the wearer of the head-mounted display 10g institution. In addition, when the remote controller is in hand, an acceleration sensor or a gyro sensor mounted in the remote controller can be used. The response to the voice is determined by the wrist motion detection unit 15B performing a predetermined motion of the hand or arm (for example, raising the hand up).

FIG. 20 is a flowchart showing the operation of the head-mounted display 10g (that is, the control method of the head-mounted display 10g) according to this modification. This operation is basically the same as the flowchart shown in FIG. 3 of the embodiment, but differs from the embodiment in that step S12 is changed to S12B. Hereinafter, the difference from the embodiment will be mainly described.

When the voice detection unit 14 detects a voice (YES in S11), the response detection unit 17 proceeds to the processing after step S12B, otherwise (NO in S11), the processing ends.

The wrist motion detection unit 15B detects the motion of the wrist of the wearer of the head mounted display 10g, and outputs a signal indicating the detection result to the response detection unit 17 (wrist motion detection step S12B).

The response detection unit 17 analyzes the signal output from the wrist motion detection unit 15B to determine whether or not the wearer of the head mounted display 10g (here, the passenger) responded to the call from the cabin crew (S13).

Specifically, when the wrist motion detection unit 15B detects a predetermined motion of the wearer's hand or arm in the specific direction picked up by the sound pickup unit 11 (YES in S13 ), the response detection unit 17 determines that the wearer is wearing The wearer responds to the call from the cabin crew member, and information indicating that the wearer responded to the voice is displayed on the external display 16 (response detection step S14).

After that, the response detection unit 17 judges whether or not the end condition of the process is satisfied ( S15 ), and continues to display the information indicating that the wearer responded to the voice on the external display 16 until it is satisfied. In addition, the "end condition of the process" means that the wearer no longer responds to the voice, for example, the wrist movement detection unit 15B cannot detect a predetermined movement of the wearer's hand or arm.

On the other hand, when the wrist motion detection unit 15B cannot detect the predetermined motion of the wearer's hand or arm (NO in S13), the response detection unit 17 determines that the wearer does not respond to the call from the cabin crew member, End processing.

As described above, the head-mounted display 10 g of the present modification is a head-mounted display that presents images, and includes a sound pickup unit 11 that picks up sound from a specific direction, and a voice detection unit 14 that detects a sound from the sound picked up by the sound pickup unit 11 . The wrist motion detection unit 15B detects the motion of the hand or arm of the wearer of the head mounted display 10g; the external display 16 displays information that can be visually confirmed from the outside; When a predetermined movement of the wearer's hand or arm is detected by the wrist motion detection unit 15B, information indicating that the wearer has responded to the voice is displayed on the external display 16 .

In addition, the control method of the head-mounted display 10g of the present modification is a control method of the head-mounted display 10 that presents an image, and includes: a sound pickup step S10, which picks up sound from a specific direction; a voice detection step S11, which starts from a sound pickup step S10 The voice is detected in the sound received in the middle; the wrist motion detection step S12B detects the movement of the hand or arm of the wearer of the head-mounted display 10g; When a predetermined movement of the wearer's hand or arm is detected in the movement detection step S12B, information indicating that the wearer has responded to the voice is displayed on the external display 16 .

Accordingly, when it is detected that the hand or arm of the wearer of the head-mounted display 10g is facing a specific direction in the sound source direction of the voice uttered by a person set to the outside, a message indicating that the wearer responded to the voice is displayed. Information is displayed on the external display 16 . Therefore, when the wearer expresses the intention to respond to the voice of the external person, the fact is displayed on the external display 16, so that the wearer and the external person can communicate smoothly to the extent that the wearer and the external person can mutually confirm their intentions.

In addition, in this modification, in the flowchart of FIG. 20 , the wrist motion detection ( S12B ) is performed after the directivity processing ( S10 ) and the voice detection ( S11 ), but the order is not limited to this. The order of the directivity processing ( S10 ), the voice detection ( S11 ), and the detection of the wrist motion ( S12B ) may be reversed or performed in parallel.

The head-mounted display and the control method thereof of the present invention have been described above based on the embodiments and modifications, but the present invention is not limited to these embodiments and modifications. As long as it does not deviate from the gist of the present invention, various modifications conceived by those skilled in the art can be applied to the embodiments and modified examples, or other forms constructed by combining some of the components in the embodiments and modified examples are also included in the present invention. within the range.

For example, the sound source detection unit 19 of Modification 2, the speech superimposition unit 20 of Modification 3, and the sound amplifier 37 of Modification 5 may be added to the head mounted display 10a of Modification 1 of the embodiment, so that the response detection section includes function of these variants. Thereby, a high-function head-mounted display that obtains the effects of the various modifications is realized.

In addition, in the embodiment and the modified example, each component may be constituted by dedicated hardware, or may be realized by executing a software program suitable for each component. Each component can also be realized by a program execution unit such as a CPU or a processor reading and executing a software program stored in a storage medium such as a hard disk or a semiconductor memory. Here, the software for realizing the head-mounted display and the like of the embodiment and the modification are the following programs.

That is, this program is a program for causing a computer to execute a control method of a head-mounted display that presents a video. The control method is a control method of the head-mounted display 10 that presents images, and includes: a sound pickup step S10, picking up sound from a specific direction; a voice detection step S11, detecting a voice from the sound picked up in the sound pickup step S10; The motion detection step S12 detects the motion of the head of the wearer of the head-mounted display 10; and the response detection step S14, in which the voice is detected by the voice detection step S11 and the head of the wearer is detected by the head motion detection step S12 When the wearer moves in a specific direction, information indicating that the wearer has responded to the voice is displayed on the external display 16, and the external display 16 displays information that can be visually recognized from the outside.

In addition to the head movement detection step S12 of detecting the movement of the head of the wearer of the head-mounted display 10 , the program may use the body movement detection step S12A of detecting the orientation of the body of the wearer of the head-mounted display 10f , or a wrist motion detection step S12B for detecting a predetermined motion of the hand or arm of the wearer of the head mounted display 10g.

【Industrial Availability】

The present invention can be utilized as a head-mounted display and a remote control system, particularly as a head-mounted display and a remote control system that can realize smooth communication to the extent that the wearer of the head-mounted display and an external person can mutually confirm their intentions.

Claims (13)

1. a kind of head-mounted display that image is presented, wherein

Have:

Radio reception portion, sound of the radio reception from specific direction；

Speech detection portion, from by the radio reception portion radio reception to sound in detect voice；

Headwork test section detects the movement on the head of the wearer of the head-mounted display；

External display, the information that display can confirm from external view；And

Response test section is being detected voice by the speech detection portion and is being detected by the headwork test section described The head of wearer towards the specific direction movement in the case where, will indicate that the wearer answers the voice The information answered is shown in the external display.

2. head-mounted display according to claim 1, wherein

Be also equipped with speaker identification portion, the speaker identification portion determine the voice detected by the speech detection portion whether be The voice of pre-determined people,

The response test section by the speaker identification portion be determined as the voice be the voice of pre-determined people and by The headwork test section detect the head of the wearer towards the specific direction movement in the case where, will be described Information is shown in the external display.

3. head-mounted display according to claim 1, wherein

It is also equipped with the sound source probe portion of detection sound source,

The radio reception portion is using the sound in the direction of the sound source detected by the sound source probe portion as from the specific direction Sound carry out radio reception.

4. head-mounted display according to claim 1, wherein

Also include

Headphone will be supplied to the wearer with the sound of the image；And

Voice overlapping portion makes the voice detected by the speech detection portion be overlapped in the sound with the image and is output to The headphone.

5. head-mounted display according to claim 4, wherein

The voice overlapping portion so that the voice detected by the speech detection portion from side corresponding with the specific direction After carrying out Sound image localization to the mode being heard, it is overlapped in the sound with the image.

6. head-mounted display according to claim 1, wherein

It is also equipped with headphone, the headphone will be supplied to the wearer with the sound of the image,

When the information is displayed on the external display, the response test section reduction is supplied to the wear-type ear The volume of the sound of machine.

7. head-mounted display according to any one of claim 1 to 6, wherein

The response test section detecting voice by the speech detection portion and detects institute by the headwork test section State the head of wearer towards the specific direction movement in the case where, carry out the voice progress to issue to the wearer The control that the mode of radio reception changes the direction of the radio reception portion radio reception.

8. a kind of control method for the head-mounted display that image is presented, wherein

Include:

Radio reception step, sound of the radio reception from specific direction；

Speech detection step, from radio reception in the radio reception step to sound in detect voice；

Headwork detecting step detects the movement on the head of the wearer of the head-mounted display；And

Response detecting step is being detected voice by the speech detection step and is being detected by the headwork detecting step To the head of the wearer towards in the case where the movement of the specific direction, will indicate the wearer to the voice into The information of response of having gone is shown in external display, which shows the information that can be confirmed from external view.

9. a kind of storage medium is stored with the journey for making to include the steps that in control method described in computer perform claim requirement 8 Sequence.

10. a kind of head-mounted display that image is presented, wherein

Have:

Radio reception portion, sound of the radio reception from specific direction；

Speech detection portion, from by the radio reception portion radio reception to sound in detect voice；

Body action test section detects the direction of the body of the wearer of the head-mounted display；

External display, the information that display can confirm from external view；And

Response test section is being detected voice by the speech detection portion and is being detected by the body action test section described The body of wearer will indicate that the wearer has carried out the letter of response to the voice towards in the case where the specific direction Breath is shown in the external display.

11. a kind of control method for the head-mounted display that image is presented, wherein

Include:

Radio reception step, sound of the radio reception from specific direction；

Speech detection step, from by the radio reception step radio reception to sound in detect voice；

Body action detecting step detects the direction of the body of the wearer of the head-mounted display；And

Response detecting step is being detected voice by the speech detection step and is being detected by the body action detecting step To the body of the wearer towards in the case where the specific direction, it will indicate that the wearer answers the voice The information answered is shown in external display, which shows the information that can be confirmed from external view.

12. a kind of head-mounted display that image is presented, wherein

Have:

Radio reception portion, sound of the radio reception from specific direction；

Speech detection portion, from by the radio reception portion radio reception to sound in detect voice；

List action test section detects the movement of the hand or arm of the wearer of the head-mounted display；

External display, the information that display can confirm from external view；And

Response test section is being detected voice by the speech detection portion and is being detected by the list action test section described In the case where the compulsory exercise of the hand or arm of wearer, it will indicate that the wearer has carried out the information of response to the voice It is shown in the external display.

13. a kind of control method for the head-mounted display that image is presented, wherein

Include:

Radio reception step, sound of the radio reception from specific direction；

Speech detection step, from by the radio reception step radio reception to sound in detect voice；

List action detecting step detects the movement of the hand or arm of the wearer of the head-mounted display；And

Response detecting step is being detected sound by the speech detection step and is being detected by the list action test section In the case where the compulsory exercise of the hand or arm of the wearer, it will indicate that the wearer has carried out response to the voice Information is shown in external display, which shows the information that can be confirmed from external view.