JP2008021186A - Position notifying method with sound, and information processing system using the method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To correctly notify an operator of positions even if a plurality of sound users exist and to notify the operator of target positions even if a plurality of targets exist in notifying the operator of the target positions with sound. <P>SOLUTION: The user is stereophonically notified of the target position by processing for inputting coordinate data of one or more targets, virtual target conversion processing for converting input coordinate data into a virtual position of sound transmitted with sound, and processing for outputting sound from each sound source based on the virtual position. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a position notification method capable of indicating the position of a target as a sound source generation location by two-dimensional or three-dimensional stereophonic sound and allowing a user to recognize the location by hearing, and an information processing system using the method. .

  In general, a user interface of an information processing system uses a technique that places importance on vision through display display. For example, when selecting a folder, a directory, an application program or the like (hereinafter referred to as a folder or the like), the pointer is overlaid on the icon of the folder or the like and then clicked.

  However, taking such an interface that relies solely on vision is not only very inconvenient for visually impaired people, but also causes a lot of fatigue and an operational error for healthy people who are gazing at the display. It becomes.

  2. Description of the Related Art Conventionally, for the purpose of improving user operability and reducing operational errors, a technique for notifying a user of a position and other information by sound has been proposed. For example, in Patent Document 1, in order to guide the driver to the destination appropriately, the current position and traveling direction of the vehicle are acquired, and route guidance information is generated based on the current position and traveling direction and geographic information. When the route guidance information is a left turn, the driver can hear from the left side, and when the route guidance information is a right turn, the sound field is formed so that the driver can hear from the right side. Support devices have been proposed.

  Patent Document 2 discloses a technical idea of adding position information of a window to be reproduced to the sound so that the user can hear the sound and identify the window being reproduced. This document includes a window position monitoring unit that notifies a window ID and a new display position to a voice direction calculation unit when a change occurs, and a voice direction calculation unit that converts an audible direction according to the window display position. The voice status monitoring unit that calls the voice management record addition / deletion unit when a change occurs, and when voice occurs, the record is added to the voice management table, and when the voice disappears, the record is deleted A voice management record addition / deletion unit, a voice output command issuing unit that converts voice into a method that conforms to the format of the stereophonic sound system provided by the window system, and a stereophonic output unit that performs actual sound output An audio playback device for a window system is described.

  However, the technique described in Patent Document 1 always outputs a notification by voice at the center of the left and right sound sources to the driver.

On the other hand, the technique described in Patent Document 2 determines the volume of each sound source on the assumption that the face of the operator (user) is on the center line of the screen.
JP-A-9-35177 JP-A-11-126153

  However, if the volume of each sound source arranged three-dimensionally is determined based on the positional relationship of the target with the operator as a reference, inconvenience occurs depending on the use of the sound source.

  For example, when the operator is an airplane pilot, a controller who confirms the situation remotely cannot always recognize sound at the pilot's position. For example, even if the object position detected by the sensor is reported to the pilot by sound, if the monitor camera is taken from behind the pilot, the direction of the image and sound seen on the monitor screen will be Does not necessarily match.

  Also, when trying to recognize what is displayed on the screen with sound, if there are a plurality of targets on the screen, there is a problem that the sound source is complicated and confuses the operator.

  The present invention has been made in view of the above-described circumstances. When the target position is notified to the operator by sound, the position can be correctly transmitted even if there are a plurality of sound users, and the target is also detected. It is an object of the present invention to provide an acoustic position notification method that can be easily communicated to an operator even when there are a plurality of information, and an information processing system using the method.

  In order to solve the above-described problem, a position notification method using sound according to the present invention is a method of notifying a user of a target position using sound, and inputting coordinate data of one or more targets. And a virtual target conversion process for converting the input coordinate data into a virtual position of sound transmitted by sound, and a process for outputting sound from each sound source based on the virtual position.

  In the present invention, coordinate data of a target input from a sensor, an input device, or the like is coordinate-converted to a spatial position on a computer and output as sound.

  Preferably, sound equipment such as a speaker is arranged around the user (operator) so as to be notified as three-dimensional stereophonic sound.

  The virtual mentioned here is generally drawn by a computer processing device and displayed on a monitor screen, or a plane or three-dimensional spatial arrangement existing in the computer. A space is displayed.

  In addition, the information processing system generally means a system having an electronic calculation function equipped with a CPU, and includes a computer system called a personal computer.

  Furthermore, in the acoustic position notification method according to the present invention, when there are a plurality of the targets, a different sound source is assigned to each target or each target group classified under a predetermined condition, and the position is determined by the self. The controllable target that can control the sound is assigned a sound production method different from the uncontrollable target that cannot control its position.

  When targets to be pronounced are mixed, it is difficult for the user to recognize. At this time, only by changing the tone color or the pitch, the user's object marker is weak, and first, the sound production method is changed depending on whether or not the target can be controlled by the user.

  Here, the sound production method is not only a sound technology such as using an equalizer to make a special sound, but also sounding in response to the position of the target or in response to changes in the position of the target It is intended to include output methods such as whether to do.

  In particular, in the sound production method, the controllable target is sounded based on a change in the position of the target, and the uncontrollable target is sounded continuously or intermittently based on the position of the target. Good.

  Since the controllable target can control its position, it only needs to be pronounced when the position changes. If you want to recognize the position of a controllable target, you can easily recognize it by moving the target because it makes a sound. The uncontrollable target should be sounded continuously or intermittently because it cannot control its position. Thereby, even if there are a plurality of targets, it is possible to reliably and easily recognize a target position that can be controlled by itself.

  Preferably, the controllable target determines the direction and volume of stereophonic sound based on the position of the target in the movable range, and the uncontrollable target is based on the position of the controllable target, The direction and volume of the stereophonic sound should be determined.

  In particular, for a controllable target, it is only necessary to check its position when necessary. In this case, it is important to know which position is in the entire movable range. On the other hand, regarding the uncontrollable target, the user can easily access the uncontrollable target by notifying the direction of the controllable target by sound with reference to the position of the controllable target (that is, the user position).

  More preferably, when the controllable target falls within a predetermined range of the uncontrollable target, a sound or a message that can identify the uncontrollable target may be output. As a result, the user can know that the uncontrollable target has been accessed without mistake.

  In addition, the acoustic position notification method according to the present invention is characterized in that different sound generation timings are assigned to each target.

  If there are multiple targets, instead of sounding at the same time, it is possible to create the closest object by shifting the timing and, for example, sounding the target so that the user is wrapped in order from each direction. The recognition of the mark becomes easy and the user's attention can be urged.

  The acoustic position notification method according to the present invention includes a process of notifying the progress of movement up to the present time based on position information from a predetermined time before each target with a continuous sound or intermittent sound, and the movement progress. And a process of predicting a future movement path based on the prediction calculation process and a process of notifying a future movement direction or movement path based on the prediction calculation result.

  Thus, the user can take measures such as taking a workaround by predicting a future moving direction or route from the past moving route.

  An information processing system according to the present invention includes means for displaying a computer-executable menu as an icon, an input device for moving a pointer (including a cursor), means for setting the pointer as a controllable target, and a computer It is characterized by comprising means for setting the icon displayed on the screen as an uncontrollable target, and a sound output device for notifying the position of the pointer and the icon with stereophonic sound.

  In the present invention, the convenience as a user interface is improved by applying the above-described acoustic position notification method to the OS of the computer system.

  An information processing system according to the present invention is an information processing system having a plurality of user terminals and a server connected to each user terminal via a communication network, and the server is provided as a controllable icon for each user. A means for setting the selected icon as a controllable target and an icon other than a controllable icon as an uncontrollable target, and the controllable target and the uncontrollable target on each user terminal Means for displaying and outputting, wherein each user terminal calculates the position of the uncontrollable target with reference to the position of the controllable target of the user, and acoustically detects the position of the uncontrollable target And notifying means.

  In the present invention, the target coordinates sent from the server are corrected at each user terminal to calculate virtual target coordinates. At this time, the coordinates of the controllable target that can be controlled by itself (virtual target coordinates) are calculated, and then the coordinates of the uncontrollable target are calculated using this coordinate as a reference.

  According to the present invention, when the target position is notified to the operator by sound, the concept of the virtual target is adopted, so that even if there are a plurality of sound users or the sound data and the image data are communicated. Even when transmitting to a distant place, the position of the target can be correctly communicated in conjunction with the screen, and even when there are a plurality of targets, the position of the target can be clearly communicated to the operator in an easy-to-understand manner.

Embodiments of the present invention will be described below.
FIG. 1 is a block diagram of an information processing system according to the first embodiment. Here, the information processing system 1 is connected to an input device 70 such as a mouse connected to the processing device 10 via the processing device 10 such as a computer, the sound output device 60, and the communication means 2. Further, the signal acquired through the sensor 76 is converted into communication data by the sensor processing device 81 and sent to the processing device 10 via the communication means 2.

  The communication means 2 may be a wide area communication network such as the Internet, or a local or short distance communication means such as a LAN or USB. If necessary, the input device 70 and the sensor processing device 81 may communicate with the processing device 10 via different communication means.

  The sound output device 60 uses conventional technology for three-dimensional sound and can express in 3D such as vertical, horizontal, and depth, and uses a general reproduction device such as a speaker or headphones as a sound generation means.

  It is assumed that the user is in the same position with respect to the speakers of these playback devices, or is in a fixed position attached to the user like a headphone. Note that when the audience user moves, the position of the user is calculated by a sensor, and the moving position of the user is added to or subtracted from the localization of the sound output from the playback device to reproduce the stereophonic sound. An optimum localization may be represented. Since the balance of the sound reproduced from each speaker is lost as the user moves, the correction is made to give the user a sense of distance.

  On the other hand, the processing device 10 includes a transmission / reception unit 11 that transmits / receives data to / from the input device 70 or the sensor processing device 81 via the communication unit 2, a processing calculation unit 12 that processes received data, and a recording unit 14 that stores data. A display unit 16 that displays the processed data, a sound source unit 13 that acoustically processes screen information displayed on the display unit 16, and a transmission / reception unit 15 that outputs the processed data to the acoustic output device 60.

  Further, the processing operation unit 12 of the processing device 10 is an input signal processing means 22 for processing a signal received from the input device 70 or the sensor processing device 81, and a basic information registration for registering basic information related to a target or sound in the recording unit 14. Means 21; target coordinate conversion means 23 for converting the coordinates of the input signal into virtual target coordinates on the computer; target motion for executing processing relating to the movement of the target, such as the moving speed of the target and stop determination of the moving distance A calculation unit 24, a target image output unit 25 that performs video output processing, and a sound source unit control instruction unit 26 that outputs as three-dimensional sound based on virtual target coordinates are provided. Each means 21-26 is a function executable as a CPU program.

Next, the functional outline of each device will be described.
(1) Functions of the input device 70, the sensor 76, and the sensor processing device 81 The input device 70 is a mouse, a keyboard, a joystick, a touch pen, a game controller, etc., and a numerical value moved in the x, y, and z directions and a user Has a function of transmitting the on / off numerical value operated by the processor 10 to the processing device 10. By transmitting signals such as a moving distance, a moving direction, and a speed from the input device 70, a cursor in a space such as a monitor can be moved three-dimensionally, such as up, down, left, right, back and front.

  An example of transmission data of the input device 70 is shown in FIG. In addition to the position coordinates, it is composed of a selection flag indicating a movement stop state and a switch state.

  The sensor 76 is an infrared sensor, a sound wave radar (car, fighter, aircraft), an optical sensor, an odor sensor, etc. The signals detected by these sensors are processed by the sensor processing device 81, and the communication means 2 is And sent to the processing device 10. As described above, the sensor 76 and the sensor processing device 81 have a function of transmitting data such as the distance and angle of the target position to the processing device 10.

  FIG. 3 shows an example of transmission data of the sensor processing device 81. In addition to motion information such as target coordinates and movement speed, it consists of various data determined by the type of sensor, such as temperature, angle, color, and smell.

  A virtual target (for example, a cursor on a monitor generally used in an OS of a computer) can be moved by the computer, that is, the input device 70 of the processing device 10.

(2) Function of Processing Device 10 The position of a virtual target (for example, a cursor) is recorded by the input signal processing means 22 on the signal input from the input device 70.

  The target coordinate conversion means 23 of the processing device 10 converts the input target coordinates into virtual target coordinates on the computer.

  As an example of the conversion processing method, for example, the input device is converted into virtual target coordinates by the following equation (1).

Virtual target coordinates = virtual target coordinates + coordinates (x, y, z) (1)
About a sensor, it converts into a virtual target coordinate by (2) Formula.

Virtual target coordinates = target coordinates + (user position-sensor position) (2)
Then, the target motion calculation means 24 acquires the virtual target coordinates and the target attribute information transmitted from the input device 70 or the sensor processing device 81, and executes processing such as changing the target position. Further, the target motion calculation means 24 records whether or not another virtual target exists at or around the place where the cursor is moving or stopped, and passes this information to the acoustic unit 13. Specifically, it is whether or not a file or directory exists directly under the cursor. Based on the coordinates indicated by the target coordinate conversion means 23, the sound source instruction determination means 51 selects a sound source and hands over the coordinates and localization information to the sound source generation means 52. The sound source generating means 52 moves the sound localization as if the user moved by the stereophonic sound. When a virtual target is present around the moving cursor or at the same position, the sound output device 60 indicates the presence of the virtual target in three-dimensional sound.

  In addition, when the user selects (for example, clicks) with the input device 70, what the target indicates may be executed. For example, opening a directory. Then, in order to further indicate the cursor in the virtual space on the opened target, a record of the input coordinates is stored, the coordinate position information is handed over to the sound source generating means 52, and the sound is output through the sound output device 60. When the target is moved or dragged while being selected by the cursor, a sound indicating the processing status may be emitted.

  When the change in coordinate information is large (such as when the mouse is moved quickly), the pitch of the pronunciation may be changed or the volume may be changed. Also, when moved to the right or upward, the stereo output is also increased from the right to the upper, bright tone or pitch is increased, and when moved to the left or downward, the stereo output is also left. The sound is changed depending on the coordinate position, for example, a dark tone or a low tone is generated downward or downward. At the back and in front, the output side is instructed to change the pitch and tone.

  Further, the movement of the target may be previewed with sound, going back to the past or the future. The movement of the target from any time before may be read and the movement up to the present may be indicated continuously by sound, or conversely, the future time predicted from the past movement data, angle and speed The movement prediction ahead may be indicated by sound.

(3) Functions of the sound output device 60 The sound output device 60 is a surround device, an audio device, a stereophonic device, a virtual surround, or the like.
Based on the coordinate information (x, y, z) information, timbre, and pitch instructions delivered from the processing device 10, sound waves are generated from the sound source, and the sound is transmitted through the stereophonic amplifier, speaker unit, and headphone unit. Is generated.

Next, the operation of the information processing system 1 having the above configuration will be described.
First, in the processing apparatus 10, attribute information and basic information about sound are registered in advance as various kinds of information related to a target for sound output such as a cursor and a folder. The registration of this information may be performed via the input device 70 or may be performed via a dedicated input device (not shown) provided separately.

The target attribute information and the acoustic basic information input from the input device 70 are stored in the recording unit 14 by the basic information registration unit 21.
4 and 5 are table configuration examples of basic acoustic information and target attribute information stored in the recording unit 14, respectively.

  Here, the basic sound information is stored as a sound source table storing sound source identification information, an equalizer table storing equalizer parameter information, and an effector table storing sound effect methods.

  The target attribute information includes an attribute / state table for storing the relationship between operation on the target and pronunciation, and a depth table for storing the relationship between hierarchy and pronunciation when folders and the like have a hierarchical structure. A guide sound table for storing the relationship (effect information) between the moving speed of the target, the sound source, and the equalizer, and a pronunciation table for storing the effect information when the targets overlap each other or perform a specific movement are stored.

  When the information processing apparatus 1 receives a signal transmitted from the input device 70 via the communication unit 2 by the transmission / reception unit 11, the input signal processing unit 22 stores the coordinate data in the recording unit 14.

The user sets the control target device. For example, in the case of a general-purpose computer, a cursor may be set as a default. In the case of a game or the like, a tank, a helicopter, or a character is set by an application program.
The correction coordinates of the target are corrected based on the character or the like.

Hereinafter, the correction process will be described.
First, basic processing will be described. (Step 1) First, a virtual target and attribute processing routine is activated by a certain trigger. Examples of the trigger include moving a mouse with a hand, a sensor capturing a target, another person operating a target online, and a CPU operating a target. (Step 2) By this trigger, storage processing of changes in virtual coordinates and attributes is started. Specifically, the coordinate position of the target, attribute recording, cursor position recording, and the like are performed. (Step 3) Then, a difference from the coordinates of the past virtual target is calculated. If the difference is large based on this difference, the localization is determined based on the direction in which it is determined that the operation is fast and the movement is performed. Processing determination is performed such as whether there is another target between the target and the self position or the cursor. By performing such processing, for example, the user can notify the route from the past to the present by sound by designating the past number of seconds. (Step 4) Next, the sound localization and sound generation method are determined, and instructions are output to the sound source circuit.

The basic functions described above enable the following functions.
(1) Process the information sent from the actual input device, convert it appropriately into coordinates in 360 degree virtual space, calculate its moving distance, object, moving direction, angle, and previous coordinates The position is recorded virtually, the difference between the previous and current is taken, and the amount of change is measured. Then, in response to this, the pitch, tone color, volume, and localization (reproduction position, up / down / left / right depth) of the sound to be reproduced are changed, and the sound is reproduced with 360-degree stereophony.

(For example, if you move the mouse quickly from the upper right to the lower right, the stereo sound will play from the upper right to the lower right, with a quick sound called "Cuen")
When there is an obstacle in front of the car, when the distance is short, the sound becomes louder and the pitch is high. If it ’s a person, it prints “I am a person”, and if it ’s a dog, it prints “I am a dog”

(2) The condition is determined based on the type or attribute of the target indicated by the sensor, and the type or change (sound type, EQ, blinking, etc.) of the sound in the virtual space is given.

  A specific sound is produced and changed according to the sensor attributes such as whether it is a large object exceeding a specific range, a living object, a non-living object, a bright object, a small object, or a smell.

(3) The past movement is indicated by sound.
The change from the coordinate position 5 seconds ago to the current coordinate position can be moved with continuous sound in stereophonic sound. When you are moving, you can hear it relatively by adding the number of movement distances to the past coordinate position.
For example, in a computer game, in the case of a fighter aircraft, the current measurement data is acquired using the measurement data of an enemy aircraft (uncontrollable target) 0.5 seconds before as a virtual position. The difference is measured, and if it is close to its own machine (controllable target) that it is controlling, it is dangerous, so the sound is loud and a high pitch sound is produced. Then, a virtual point is connected from the position of the enemy aircraft 5 seconds ago to the current position with a point, and this moving distance is continuously reproduced with stereophonic sound.

(4) The future movement prediction is indicated by sound.
Judging from the movement data and attributes up to now, the virtual target coordinates up to a few seconds ahead of the movement are indicated by continuous pronunciation. When you are moving, you can listen relatively by adding your own moving distance to the future coordinate position.

(5) Hit determination The information sent from the input device 70 is processed and appropriately converted into coordinates in a 360-degree virtual space, and another virtual target existing in or near the place is converted. Detect and change the pitch and tone of the sound to be played, the volume, and the localization (playing position, vertical and horizontal depth) according to the type of target, the distance to the target, and the relationship. To play.

  A condition is determined based on the type and attribute of the target indicated by the sensor, and the type and change (sound type, EQ, blinking, etc.) of the sound in the virtual space are given.

  A specific sound is produced and changed according to the sensor attributes such as whether it is a large object exceeding a specific range, a living object, a non-living object, a bright object, a small object, or a smell.

  Also, when virtual targets approach each other or are included in the set range, the type or change of sound (sound type, EQ, blinking, etc.) in the virtual space is generated from the coordinates in the virtual space. ,give.

  When the car corner sensor reacts, a warning sound is generated from this direction, and the sound volume is changed from that distance or changed by EQ.

  For example, when the mouse is moved and the cursor moves, a suitable sound is produced when passing over any file. Alternatively, when the virtual coordinate position of the latest data overlaps the virtual coordinate position set in advance by a fighter or the like, an appropriate sound is output.

(6) Sound can be changed in the environment between the target and the target.
For example, there is an obstacle wall between the target (uncontrollable target) and oneself (controllable target), or the sound changes depending on the temperature change and the amount of oxygen and other transmission substances (during a fire) .

(Embodiment 1) Operation system In the operation system of a computer, the location of a file or directory is indicated by three-dimensional sound. For example, on the screen of an OS using a visual, the presence of a cursor indicated by an input device such as a mouse is virtually output from its position with stereophonic sound. Assume that the user is in front of the monitor. If the cursor is at the bottom left of the screen, the sound will come out from the bottom left. When the input device is moved to move to the upper right, the sound moves from the lower left to the upper right. In addition, when the concept of the back is added due to the 3D desktop, sound is emitted from the back and back localization. When any file or directory is indicated at or around the cursor, the details may be indicated by sound or voice. Using known stereophonic technology, it is realized with speakers and headphones. It can be used as an aid to use a personal computer (hereinafter also referred to as a personal computer or a PC) for people with visual handicap, but it does not use visual information as a support for normal use of a PC. Can be operated smoothly.

An example of a processing procedure for realizing the above function will be described with reference to FIG.
When activated by detecting a change in the mouse position, the mouse position coordinates are input. Note that the position and acoustic data illustrated in FIG. 7 are provided for each virtual target such as a cursor, and this data is sequentially updated. Then, a pronunciation is made to the user from a location corresponding to the cursor position (S101).

  Next, it is determined whether or not the file exists at or around the cursor position (S102). If the file exists, the file name is read out or notified with a sound corresponding to the file (S103). . Then, depending on whether it is clicked (S104) or double-clicked (S106), application processing such as reading a file name or executing a file is performed (S105, S107).

  If NO in step 102, it is next determined whether or not the directory exists at or around the cursor position (S108), and processing similar to the above file is executed on the directory, such as reading out the directory name. (S109 to S113). If necessary, the same processing is performed on the body target other than files and directories.

  Thereafter, processing related to the mouse is executed. Specifically, it is determined whether or not the button is still clicked (S114). If YES, it is determined whether or not it has further moved (S115). If YES, the sound localization is moved while sounding. (S116). If NO in step S115, the location of the cursor is displayed while sounding (S117).

  If NO in step S114, sound generation processing is executed with a pitch or volume corresponding to the speed (S118 to S121).

(Embodiment 2) In addition to the mobile device PC, the portable audio device can be used for music selection, use operation support, mobile phone and the like. When carrying these, it is possible to select the item by checking the position of the internal file information and hierarchy with sound without looking at the output display screen of the process. Functions, phone numbers, address books, content applications, etc. are located in the virtual space, and these can be confirmed and operated with 3D sound. Even if it is contained in clothes, you can operate it with just your hand without having to take it out.

(Embodiment 3) A game and this technology can be applied to search for enemies on the game by using only sound in a virtual space as a computer game without video. Past movement trajectories and future predictions can also be indicated by sound.

(Embodiment 4) Car, Aircraft In Embodiments 1 to 3 described above, the input device is moved spontaneously by the user. However, when this input device is a known sensor or radar wave, these input devices capture it. If the position information such as the target direction and distance is processed, and the sound source tone is changed according to the information and converted into stereophonic output, the target position is shown to the user in stereophony. be able to.

  For example, in the case of an automobile corner sensor or a radar obstacle detection device in front of the radar, the position of the obstacle existing in front of the automobile can be measured and measured. If these ideas are combined with this idea, a warning sound can be emitted with three-dimensional sound from the direction in which the obstacle exists to the driver when there is an obstacle. Even without visual confirmation, it can be recognized that there is an obstacle in the direction and distance where the sound is consciously generated.

  In a computer game, when a pilot of a fighter plane approaches an enemy aircraft or locks a missile, the warning sound is also three-dimensional sound from the approaching direction, so it can be identified by sound without visual recognition. Depending on whether it is a missile, an airplane, fast or slow, the tone is changed.

  In addition, when there is an obstacle behind the car driver when traveling backward, a sound is generated with a three-dimensional sound from the direction of the obstacle. Note that the driver's position in this case is a fixed position as seen from the speaker that generates sound, and correction and adjustment are performed when the driver's position changes or the height (ear position) changes. . The past movement trajectory of the object (target) and the future prediction can also be indicated by sound.

As described above, according to the present embodiment, it is possible for the user to operate by relying on the sound in the direction in which the cursor is to be advanced without looking at the monitor. In addition, the auditory support reduces the burden on the user to operate the input device.
In addition, by using stereophonic sound, the front, rear, left, right, back, front, etc. can be output with sound, so that the direction of sound generation can be consciously confirmed by the ear without actually viewing.

Next, a second embodiment will be described.
In the present embodiment, a case where a game is played through a communication network will be described. In FIG. 8, in the information processing system 1 according to the present embodiment, user terminals 200a and 200b are connected to a server 100 that executes a game application via a communication network 5 such as the Internet. The number of user terminals varies depending on the content of the game.

  The server 100 includes a transmission / reception unit 110 and a processing calculation unit 120 that exchange data with the communication network 5, and the processing calculation unit 120 receives data sent from a user terminal as a function executed by the CPU. Receiving processing means 121, target basic information registration means 122 for registering basic information about the target, application execution means 123 for executing the application program, and transmission processing means 122 for transmitting the execution result to the user terminal.

  On the other hand, the user terminals 200A and 200B include a transmission / reception unit 210 that exchanges data with the communication network 5, a processing calculation unit 220 that performs various calculation processes by the CPU, a storage unit 240 that stores data, and data such as game control commands. Input device 250, display device 270 for displaying game content, and sound output device 260 for realizing stereophonic sound.

  The processing operation unit 220 also includes input processing means 224 that processes data input from the input device 250, input data transmission means 221 that transmits input data to the server 100, and data such as games sent from the server 100. Received from the server 100 by the reception processing means 222 for receiving, the target attribute registering means 227 for each user for registering the attribute of the target in the storage unit 240, the screen display control means 228 for controlling the display range of the game screen, and the display range. Target coordinate conversion means 223 for converting the coordinates of the target, target video output means 225 for executing display output processing of the game content including the target, and stereo sound output processing based on the coordinates and attributes of the target The sound processing means 226 for performing Each means 221 to 228 can be realized by a program as a function of the CPU.

  Now, suppose a battle game between a fighter and a tank as a game, and consider a case in which the fighter is controlled by the user A via the terminal 200A and the tank is controlled by the user B via the terminal 200B.

  The user A and the user B access the server 100 from the respective terminals 200a and 200b, and set targets (whether they are fighters or tanks) that they operate. This setting information is stored in the target basic information table 141 by the target basic information registration unit 122 of the server 100.

  FIG. 9 is a data configuration example of the target basic information table 141. Each piece of user identification information is stored in association with operation target object-specific information.

  The application unit 123 of the server 100 is executed by a game start command from the user terminal 200a or 200b. As the game progresses, the state of each target (moving / moving direction, stopped, destroyed, attack) and the position of the target are sequentially stored in the basic information table.

  The transmission / reception processing means 124 transmits the data in the basic information table 141 to the user terminals 200a and 200b as needed. Note that the load on the server and the communication network can be reduced by performing the transmission when the state of the table changes.

  On the user terminal 200a side, first, when basic information data of a target in the server 100 sent from the server 100 is received by the reception processing means 222, a target (controllable by the user) can be controlled by the user-specific target attribute registration means 227. The target management table 241 is created separately for a target) and an uncontrollable target (uncontrollable target) and stored in the storage unit 240.

FIG. 10 is a data configuration example of the target management table 241.
Here, in the target management table 241 of the user A, a fighter plane operated by the user A is registered as a controllable target, and a tank operated by the user B is registered as an uncontrollable target.
On the other hand, in the target management table 241 on the terminal 200b of the user B, the tank operated by the user B is registered as a controllable target, and the fighter plane operated by the user A is registered as an uncontrollable target.

  Returning to the user terminal 200a again, the screen display control means 228 displays the fighter aircraft 301, which is a controllable target, at a position where the user A can easily control it. Normally, the controllable target is brought to a display position that is almost in the center. Then, the conversion value between the display position and the coordinates sent from the server 100 is stored in the target management table 241.

FIG. 11 is a flowchart showing the processing procedure of the screen display control means 228.
When the screen display control means 228 is activated, it first inputs the coordinates of the controllable target (S201), adds a correction value to it, and calculates the coordinates to be output to the display device 270. Then, the whole is displayed at the designated magnification, and the same correction value is added to the uncontrollable target, and displayed at the display position at the designated magnification. The calculation result is displayed on the display device 270 by the target image output means 225.

  On the other hand, the sound processing means 226 extracts the sound source of the tank 302 that is a controllable target based on the coordinates, and outputs sound from the position converted by the target coordinate conversion means. At this time, normally, sound is output so as to wrap the user A or the user A from the surroundings, and a sound effect is emitted from the target position by the operation of the user A.

  In addition, the tank position, which is an uncontrollable target, is also output as an image, and the sound processing means 226 outputs a stereophonic sound.

  As a result of the above processing, the display device 270 of the terminal 200a of the user A displays the screen as shown in FIG. 12A. As the sound effect, the position of the fighter 301 that is a controllable target, that is, the user position, and uncontrollable It is output from the position of the tank 302 as the target.

  On the other hand, on the display device 270 of the user B terminal 200b, as shown in FIG. 12B, the tank 302 that can be controlled by the user B is displayed at a substantially central position, and a sound effect is similarly generated based on this position. Note that the display direction and the display magnification may be changed as necessary. At this time, since the relative coordinate positions of the controllable target and the uncontrollable target change, the sound source direction of the three-dimensional sound also changes.

  Then, it is determined whether or not the corrected coordinate of the controllable target exceeds a predetermined range (S202). If the controllable target exceeds the predetermined range, the correction value ( Coordinate conversion data) is changed (S203).

  As described above, according to the present embodiment, the target coordinates managed by the server are replaced with virtual target coordinates that match each controllable target on each terminal, so that the user can focus on the target that can be controlled by the user. In addition, it is possible to know the positions of other targets with stereophonic sound, and a more realistic game becomes possible. Further, since the server manages each target coordinate without distinguishing between controllable targets and uncontrollable targets, the load on the server can be reduced.

  In addition, by restricting the display range of the controllable target to the vicinity of the center of the screen, etc., and by notifying the position of the uncontrollable target relative to the controllable target, the user can easily display the controllable target. The direction and distance can be grasped. Further, by notifying the position of the uncontrollable target without being limited to the display range on the display device, the user can recognize the position of the uncontrollable target without being restricted by the screen display.

  In the above description, the network game is taken as an example. However, an object (including odor) is detected by a sensor on the server side, and the coordinates of the object are sent to a plurality of terminals. Also applicable to cases where a target is identified (equivalent to a controllable target) and the relative position of other targets (equivalent to uncontrollable targets) is reported to the user by sound based on the identified target. it can.

Next, a third embodiment will be described.
In this embodiment, a case where a cursor is moved on a computer and a folder or an application program icon is selected will be described.

The cursor is set as a controllable target, and the other icons are set as uncontrollable targets.
As the folder arrangement, not only a two-dimensional plane but also a three-dimensional arrangement with a depth can be used. In such a state, there are a plurality of target coordinates to be sounded.

  The volume of each target is determined based on the distance between the cursor position and each target coordinate. Further, the position of the uncontrollable target at this time is a position centered on the cursor that is the controllable target.

For a target, that is, by separating the cursor position, not the target position on the screen, even if the cursor is in the right corner of the screen, if the folder is in the upper right, the sound of the folder You can hear it from directly above.
In this way, when the cursor is stationary, there is no sound from the cursor, but the direction in which the folder is moved can be accurately recognized.

  On the other hand, since the pronunciation from the cursor is made by moving the cursor, it is possible to know where it is on the screen.

  As described above, the target coordinate can be freely moved to the virtual position on the computer by the correction value, and the correction value is made variable depending on whether it is a controllable target or an uncontrollable target, and the pronunciation position coordinate of the target is determined. Furthermore, since sound is generated by a sound source determined by motion information such as target attributes, position, and speed, and a production method, position instruction processing by versatile sound can be performed.

  The present invention can be implemented with various modifications without departing from the gist of the present invention regardless of the above-described embodiments. For example, the position of the uncontrollable target may not be set as a relative position to the controllable target, but may be notified by sound according to the position of the uncontrollable target in the movable range, or relative to the controllable target. It may be possible to switch according to the user's selection whether to use a specific position or a movable range as a reference. Further, a means capable of adjusting in advance the display position of the controllable target and the reference position of the sound source may be provided.

It is a functional block diagram of the information processing system by the 1st Embodiment of this invention. It is an example of the transmission data of the input device 70 of FIG. It is an example of the transmission data of the sensor processing apparatus 81 of FIG. It is the acoustic basic information table preserve | saved at the recording part 14 of FIG. It is the target attribute information table preserve | saved at the recording part 14 of FIG. It is a flowchart which shows the cursor processing procedure of OS by the 1st Embodiment of this invention. It is the target information table preserve | saved at the recording part 14 of FIG. It is a functional block diagram of the information processing stem by the 2nd Embodiment of this invention. It is an example of a data structure of the target basic information table 141 of FIG. It is a data structural example of the target management table 241 of FIG. It is a flowchart which shows the process sequence of the screen display control means 228 of FIG. 12A is an explanatory diagram of a screen displayed on the display device 270 of the terminal 200a of the user A in FIG. 8, and FIG. 12B is a display device 270 of the terminal 200b of the user B in FIG. It is explanatory drawing of the screen displayed on top.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Information processing system 2 Communication means 5 Communication network 10 Processing apparatus 11, 15, 61, 75, 80, 82, 84 Transmission / reception part 12, 72, 78, 83 Processing calculating part 13 Sound source part 14 Recording part 16 Display part 21 Basic information Registration means 22 Input signal processing conversion means 23 Target coordinate conversion means 24 Target motion calculation means 25 Target video output means 26 Sound source control instruction means 41 Each means record DB
42 Sound waveform DB 43 Speech synthesis DB
44 Sound waveform recording DB 51 Sound source instruction determination unit 52 Sound source generation unit 62 D / A conversion unit 63 Audio amplifier for each channel 64 Speaker unit 65 Headphone unit 70 Input device 71 Input unit 73, 79 A / D conversion unit 76 Sensor 77 Sensor unit 85 Target type data 86 Target position coordinate data 87 Target brightness data 88 Target color data 89 Target angle data 90 Target temperature data 91 Target odor data 100 Server 110, 210 Transmission / reception unit 120, 220 Processing calculation unit 121 Reception processing means 122 Target basic information registration means 123 Application execution means 124 Transmission processing means 141 Target basic information table 200a, 200b User terminal 221 Input data transmission means 222 Reception processing means 223 Target coordinate conversion means 224 Input processing means 225 Marking footage Force means 226 Sound source unit control instruction means 227 User-specific target attribute registration means 228 Screen display control means 241 Target management table 250 Input device 260 Sound output device 270 Display device

Claims (9)

A method of notifying a user of a target position using sound,
A process of inputting coordinate data of one or more targets,
A virtual target conversion process for converting the input coordinate data into a virtual position of sound transmitted by sound;
Processing to output sound from each sound source based on the virtual position;
The position notification method by sound characterized by including. When there are a plurality of the targets, different sound sources are allocated for each target or for each target group classified under a predetermined condition, and
2. The position notification by sound according to claim 1, wherein the controllable target that can control its position is assigned a different sound production method than the uncontrollable target that cannot control its position. Method.   In the sound production method, the controllable target is pronounced based on a change in the position of the target, and the uncontrollable target is pronounced continuously or intermittently based on the position of the target. The position notification method by sound according to claim 2. The controllable target determines the direction and volume of the stereophony based on the position of the target in the movable range,
4. The position notification method by sound according to claim 2, wherein the uncontrollable target determines a direction and volume of stereophonic sound based on a position of the controllable target.   The sound or message capable of identifying the uncontrollable target is output when the controllable target enters a predetermined range of the uncontrollable target. The acoustic position notification method described.   6. The position notification method by sound according to any one of claims 1 to 5, wherein a different pronunciation timing is assigned to each target. A process of notifying the progress of movement up to the present time based on position information from a predetermined time before each target with a continuous sound or intermittent sound,
A process of predicting a future movement path based on the movement progress;
A process of notifying the future movement direction or movement path according to the prediction calculation result;
The position notification method by sound according to any one of claims 1 to 6, characterized by comprising: An information processing system for notifying the position of a target by sound using the position notification method according to any one of claims 2 to 5,
Means for displaying computer-executable menus as icons;
An input device for moving the pointer;
Means for setting the pointer as a controllable target;
Means for setting the icon displayed on the computer screen as an uncontrollable target;
An acoustic output device that acoustically informs the position of the pointer and the icon;
An information processing system comprising: An information processing system having a plurality of user terminals and a server connected to each user terminal via a communication network,
The server sets an icon selected for each user as a controllable icon as a controllable target, and sets an icon other than the controllable icon as an uncontrollable target; the controllable target; Means for displaying and outputting the uncontrollable target on each user terminal,
Each user terminal includes means for calculating the position of the uncontrollable target with reference to the position of the controllable target of the user, and means for notifying the position of the uncontrollable target by sound. An information processing system characterized by this.

Images