CN103000171B - The control method of music performance apparatus, emission control device and music performance apparatus - Google Patents

Abstract

The present invention provides a musical performance device capable of reducing the power consumption of the musical performance parts in which musical sounds are emitted based on the position coordinates of the light-emitting parts of the musical performance parts in an imaging space. The performance device (1) comprises: a rod part (10), which is held by a player, and has a light-emitting and extinguished marking part (15) at the front end; Shooting; and the central unit part (30), according to the position coordinates of the marking part (15) in the luminescence in the shooting space photographed by the camera unit part (20), the percussion instrument is pronounced, and the bar part (10) is used to detect the performance The situation of the start of the person's downward swinging action is a condition to make the marking part (15) emit light, and the situation of detecting the end of the corresponding action is a condition to make the marking part (15) go out.

Description

Performance device, lighting control device and control method for performance device

This application is based on and claims priority from prior Japanese Patent Application No. 2011-181795 filed on Aug. 23, 2011, the entire contents of which are hereby incorporated by reference.

technical field

The present invention relates to a performance device emitting electronic sounds and a lighting control device used in the performance device.

Background technique

Conventionally, performance devices have been proposed which, when detecting a player's performance movement, emit electronic sounds corresponding to the performance movement. For example, there is known a performance device (air drum) that emits a percussion sound only through a part on a stick. In this performance device, a sensor is set on a stick-shaped part, and the player uses his Hold and swing the part, whereby the sensor detects the performance action and emits a percussion sound.

According to such a performance device, since the musical sound of the musical instrument can be emitted without an actual musical instrument, it is possible to enjoy the performance without being restricted by the performance place or the performance space.

For such performance devices, for example, in FIG. 1 of JP Patent No. 3599115, a musical instrument game device constructed in the following manner is proposed. A composite image obtained by synthesizing the playing action and the virtual image representing the musical instrument set is displayed on the monitor, and a predetermined musical sound is emitted based on the position information of the rod-shaped member and the virtual musical instrument set.

However, in a performance device that captures a player and emits musical tones, it is necessary to be able to recognize the performance parts from the captured image. More specifically, it is necessary to specify the position coordinates of the part of the performance part that is in contact with the virtual musical instrument in the captured image.

In this regard, in the musical instrument game device described in JP Patent No. 3599115, a light (FIG. 4) is provided at the front end of a light pen (Ben Light: pen light) used by a player, and by determining the position coordinates of the light, it is distinguished. The part that comes into contact with the virtual instrument.

Therefore, in the performance part (light pen), a power source for turning on the lamp is required. However, in order to realize the above-mentioned features such as performance venues and performance space not being restricted, it is not necessary to supply power with cables, but to set power sources such as batteries inside the performance parts. In addition, the weight of the performance parts needs to be kept constant so that the player can maintain the performance, and the simple method of installing a large battery is not ideal in order to enable long-term use.

In this regard, in the musical instrument game device of Patent Document 1, further improvement is required from the viewpoint of reducing the power consumption of the performance parts without considering the lighting control of the lamp at all.

Contents of the invention

The present invention has been made in view of such a desire, and its object is to provide a performance device that can reduce the power consumption of a performance part in a performance device that emits musical sounds based on the position coordinates of the light emitting part of the performance part in the imaging space. and lighting controls.

In order to achieve the above object, a performance device according to an aspect of the present invention is characterized in that it includes: a performance part held by a player and having a light-emitting part that emits light and goes out; The image of the shooting space is photographed; the sounding device emits sound according to the position coordinates of the above-mentioned light-emitting part in the light-emitting in the above-mentioned shooting space captured by the above-mentioned shooting device; The start and end of the action; and the light-emitting extinguishing control unit, when the detection unit detects the start of the above-mentioned downward swing, executes the light-emitting control of making the above-mentioned light-emitting part emit light, and when the detection unit detects that the above-mentioned downward swing is completed, executes the light-emitting control that makes the above-mentioned swinging motion end. The extinguishing control of extinguishing the above-mentioned light-emitting part.

In addition, a light emission control device according to an aspect of the present invention is characterized by comprising: a detection unit that detects the start and end of an operation given to a member having a light emitting unit; and a control unit that responds to the detection of the start of the operation by the detection unit. , turning on the light emitting unit, and turning off the light emitting unit in response to the detection of the completion of the operation by the detection unit.

In addition, a control method of a performance device according to an aspect of the present invention, the performance device includes: a performance part, including a light-emitting part that emits light and goes out, and is held by a player; a photographing device; It includes: a photographing step of photographing, by the photographing device, an image of the photographing space in which the player holding the performance part exists; and a sounding step, according to the above-mentioned The position coordinates of the light-emitting part are used to make the above-mentioned sounding device sound; the detection step is to detect the start and end of the downward swing of the above-mentioned performance parts by the above-mentioned player; At the start of an operation, light emission control is performed to light the light emitting unit, and when the end of the swiping motion is detected in the detecting step, light-off control is performed to turn off the light emitting unit.

Description of drawings

1A and 1B are diagrams showing an outline of an embodiment of the musical performance device of the present invention.

Fig. 2 is a block diagram showing a hardware configuration of a lever constituting the musical instrument.

Fig. 3 is a perspective view of the above-mentioned rod.

Fig. 4 is a block diagram showing a hardware configuration of a camera unit constituting the musical performance device.

Fig. 5 is a block diagram showing a hardware configuration of a central unit constituting the musical performance device.

FIG. 6 is a flowchart showing the flow of processing of the above-mentioned lever unit.

FIG. 7 is a graph showing changes in output related to acceleration in the vertical direction of the motion sensor unit.

FIG. 8 is a flowchart showing the flow of processing of the above-mentioned lever unit.

FIG. 9 is a flowchart showing the flow of processing of the above-mentioned camera unit.

FIG. 10 is a flowchart showing the flow of processing of the central unit unit.

Detailed ways

Embodiments of the present invention are described below with reference to the drawings.

(Summary of Performance Device 1)

First, with reference to FIG. 1 , an overview of a musical performance device 1 as an embodiment of the present invention will be described.

As shown in FIG. 1A , the performance apparatus 1 of the present invention includes: rod portions 10A, 10B; a camera unit portion 20 ; and a center unit portion 30 . In order to realize a virtual drum performance using two sticks, the performance device 1 of the present embodiment has two sticks 10A, 10B. However, the number of rods is not limited thereto, and may be one, or three or more. In addition, below, when not distinguishing rod part 10A, 10B, both are collectively called "rod part 10."

The rod portion 10 is a rod-shaped member extending in the longitudinal direction, and corresponds to the musical performance part of the present invention. A player performs a performance motion by holding one end (root side) of the shaft portion 10 in his hand and swinging up and down centering on the wrist or the like. In order to detect such a player's performance movement, various sensors such as an acceleration sensor (a motion sensor unit 14 described later) are provided at the other end (the front end side) of the rod unit 10 . And, according to the performance movement detected by various sensors, the stick part 10 sends an event record (note on event) to the central unit part 30.

In addition, a marker portion 15 (refer to FIG. 2 ), which will be described later, is provided on the front end side of the shaft portion 10 , and the camera unit 20 can identify the front end of the shaft portion 10 at the time of shooting.

The camera unit 20 is an optical camera that captures a moving image (movie) of a player holding the stick 10 while playing at a predetermined frame rate, and corresponds to the imaging device of the present invention. The camera unit unit 20 specifies the position coordinates of the marker unit 15 during the light emission in the imaging space, and sends it to the center unit unit 30 .

When the center unit unit 30 receives an event record from the pole unit 10, it emits a predetermined musical sound based on the position coordinate data of the marker unit 15 at the time of reception. Specifically, the center unit unit 30 stores the position coordinate data of the virtual drum set D shown in FIG. 1B in association with the imaging space of the camera unit unit 20 . Then, based on the position coordinate data of the virtual drum set D and the position coordinate data of the marking part 15 at the time of receiving the event record, the musical instrument struck by the stick part 10 is specified, and a musical sound corresponding to the musical instrument is emitted.

In such a musical performance device 1 , the lever portion 10 of the present embodiment reduces power consumption, and performs light-emitting control and light-off control of the marking portion 15 . Specifically, in the performance device 1, it is necessary to emit a musical sound when the rod 10 strikes a virtual musical instrument, but generally in percussion instruments, the strike of the rod 10 is performed when the rod 10 is swung down, not on the rod 10. The upswing of part 10 is carried out.

Therefore, the shaft part 10 of this embodiment performs the light emission control of the marking part 15 on the condition that the start of the downswing movement is detected, and thereafter, the end of the downswing movement and the start of the upswing movement are detected. As a condition, the extinguishing control of the marking portion 15 is performed, thereby achieving reduction in power consumption. In addition, the light emission control refers to the control of making the marker 15 emit light and the control of maintaining the state of light emission. However, the state of emitting light includes not only a state of always emitting light but also a state of temporarily extinguishing it by blinking. In addition, the extinguishing control refers to the control to extinguish the light emission of the marking part 15 and the control to maintain the extinguished state.

Embodiments of the present invention will be specifically described below.

(Structure of the performance device 1)

First, with reference to FIGS. 2 to 5 , the structures of the rod portion 10 , the camera unit portion 20 and the center unit portion 30 constituting the musical performance device 1 of the present invention will be described. FIG. 2 is a block diagram showing a hardware configuration of the lever unit 10 . 3 is a perspective view of the pole unit 10 , FIG. 4 is a block diagram showing the hardware configuration of the camera unit unit 20 , and FIG. 5 is a block diagram showing the hardware configuration of the center unit unit 30 .

(Structure of rod portion 10)

As shown in Figure 2, the rod portion 10 includes a CPU11 (Central Processing Unit: Central Processing Unit), a ROM (Read Only Memory: Read Only Memory) 12, a RAM (Random Access Memory: Random Access Memory) 13, a motion sensor portion 14, Marking unit 15 and data communication unit 16 .

The CPU 11 executes overall control of the stick 10. For example, based on the sensor value output from the motion sensor unit 14, in addition to the detection of the posture of the stick 10, the impact detection, and the motion detection, light emission and extinction of the marker 15 are also performed. control. At this time, the CPU 11 reads the mark characteristic information from the ROM 12 , and performs light emission control of the mark portion 15 according to the mark characteristic information. Moreover, CPU11 performs communication control with the center unit part 30 via the data communication part 16. As shown in FIG.

The ROM 12 stores processing programs for various processing executed by the CPU 11 . In addition, the ROM 12 stores marker characteristic information used for light emission control of the marker portion 15 . Here, the camera unit 20 needs to distinguish the marking part 15 (first marking) of the shaft part 10A from the marking part 15 (second marking) of the shaft part 10B. The marker feature information is information for the camera unit 20 to distinguish between the first marker and the second marker. For example, in addition to the shape, size, hue, chroma, or luminance when emitting light, the flickering speed when emitting light, etc. may be used.

The CPU 11 of the lever unit 10A and the CPU 11 of the lever unit 10B read different marker characteristic information, respectively, and perform light emission control of the respective markers.

The RAM 13 stores values obtained or generated during processing, such as various sensor values output by the motion sensor unit 14 .

The motion sensor unit 14 is various sensors for detecting the state of the rod unit 10 and outputs predetermined sensor values. Here, as sensors constituting the motion sensor unit 14 , for example, an acceleration sensor, an angular velocity sensor, a magnetic sensor, and the like can be used.

As the acceleration sensor, a three-axis sensor that outputs accelerations generated in three axial directions of the X axis, the Y axis, and the Z axis can be used. In addition, regarding the X-axis, Y-axis, and Z-axis, as shown in FIG. 3 , the axis that coincides with the axis in the longitudinal direction of the rod portion 10 can be used as the Y-axis, and the axis parallel to the substrate (not shown) on which the acceleration sensor is arranged can be used as the Y-axis. Furthermore, the axis perpendicular to the Y axis is defined as the X axis, and the axis perpendicular to the X axis and the Y axis is defined as the Z axis. At this time, the acceleration sensor may obtain the acceleration of each component of the X-axis, the Y-axis, and the Z-axis, and may calculate a sensor synthesis value obtained by combining the respective accelerations. Here, the player holds one end (root side) of the shaft 10 and swings up and down around the wrist or the like, thereby causing the shaft 10 to rotate. Next, corresponding to this, when the rod portion 10 is at rest, the acceleration sensor calculates a value corresponding to the gravitational acceleration 1G as the sensor composite value, and when the rod portion 10 performs rotational motion, the acceleration sensor calculates a value greater than the gravitational acceleration 1G as Sensor composite value. In addition, the sensor composite value is obtained by calculating the square root of the sum of the squares of the accelerations of the X-axis, Y-axis, and Z-axis components, for example.

In addition, as the angle sensor, for example, a sensor having a gyroscope can be used. Here, as shown in FIG. 3 , the angular velocity sensor outputs a rotation angle 301 of the rod portion 10 in the Y-axis direction and a rotation angle 311 of the rod portion 10 in the X-axis direction.

Here, since the rotation angle 301 in the Y-axis direction is the rotation angle of the front-rear axis seen from the player when the player holds the shaft 10, it can be called a roll angle (roll angle). The roll angle is generated by the player holding the stick 10 and rotating it in the left-right direction with the wrist as the axis corresponding to the angle 302 indicating how much the X-Y plane is inclined with respect to the X-axis.

In addition, since the rotation angle 311 in the X-axis direction is the rotation angle of the horizontal axis seen from the player when the player holds the shaft 10 , it can be called a pitch angle (pitch angle). The pitch angle is generated by the player holding the stick 10 and rotating it vertically about the wrist as an axis corresponding to the angle 312 indicating how much the X-Y plane is inclined with respect to the Y axis.

In addition, although not shown, the angular velocity sensor may output together with the rotation angle in the Z-axis direction. At this time, the rotation angle in the Z-axis direction, which is the pitch angle generated by the player holding the stick part 10 and vibrating the wrist in the left-right direction, has basically the same property as the rotation angle 311 in the X-axis direction. .

In addition, as the magnetic sensor, a magnetic sensor capable of outputting magnetic sensor values in the three-axis directions of the X axis, the Y axis, and the Z axis shown in FIG. 3 can be used. From such a magnetic sensor, vectors indicating the north (magnetic north) of the magnet are output with respect to the X-axis direction, the Y-axis direction, and the Z-axis direction, respectively. Since the output components in each axis direction differ according to the posture (orientation) of the shaft 10, the CPU 11 can calculate the roll angle of the shaft 10, the rotation angles in the X-axis direction and the Z-axis direction based on these components.

The motion sensor section 14 (in detail, the CPU 11 that receives the sensor value from the motion sensor section 14) detects the state of the stick section 10 held by the player (may also be referred to as the player's playing state) using such various sensors. . As an example, the CPU 11 detects an impact timing (impact timing) of the stick 10 on a virtual musical instrument based on the acceleration (or sensor composite value) output by the acceleration sensor. In addition, as will be described later, the CPU 11 detects the downward swing motion and the upward swing motion of the shaft portion 10 based on the sensor values output from the respective sensors.

Returning to FIG. 2 , the marking part 15 is a luminous body such as an LED provided on the front end side of the rod part 10 , and lights up and goes out under the control of the CPU 11 . Specifically, the marking unit 15 emits light based on the marking characteristic information read by the CPU 11 from the ROM 12 . At this time, since the marker characteristic information of the rod 10A is different from the marker characteristic information of the rod 10B, the camera unit 20 can distinguish and obtain the position coordinates of the marker (first marker) of the rod 10A and the marker of the rod 10B. The position coordinates of the part (2nd mark).

The data communication unit 16 performs predetermined wireless communication at least between itself and the central unit unit 30 . The predetermined wireless communication may be performed by any method, but in this embodiment, wireless communication with the central unit unit 30 is performed by infrared communication. In addition, the data communication part 16 may be set to perform wireless communication with the camera unit part 20, and may also be set to perform wireless communication with the stick part 10A and with the stick part 10B.

(Structure of the camera unit part 20)

The description of the structure of the rod portion 10 is as above. Next, the configuration of the camera unit section 20 will be described with reference to FIG. 4 .

The camera unit unit 20 includes a CPU 21 , a ROM 22 , a RAM 23 , a marker detection unit 24 , and a data communication unit 25 .

The CPU 21 executes overall control of the camera unit section 20 . For example, control is performed to calculate the respective position coordinates of the markers 15 (the first marker and the second marker) of the rods 10A and 10B based on the position coordinate data and marker characteristic information of the markers 15 detected by the marker detector 24. data. Moreover, CPU21 performs communication control which transmits calculated position coordinate data etc. to the center unit part 30 via the data communication part 25.

The ROM 22 stores processing programs for various processing executed by the CPU 21 . The RAM 23 stores values obtained or generated during processing, such as position coordinate data of the marker portion 15 detected by the marker detection unit 24 . In addition, the RAM 23 also stores together the respective flag feature information of the pole parts 10A and 10B received from the center unit part 30 .

The mark detection unit 24 is, for example, an optical camera, and captures a moving image of a player holding the stick unit 10 and performing a performance movement at a predetermined frame rate. In addition, the marker detection unit 24 outputs the captured data for each frame to the CPU 21 . In addition, the camera unit 20 specifies the position coordinates of the marker 15 of the pole 10 in the imaging space, but the position coordinates of the marker 15 can be determined by the marker detection unit 24 or by the CPU 21 . Similarly, the marker feature information of the captured marker 15 may be information specified by the marker detection unit 24 or information specified by the CPU 21 .

The data communication unit 25 performs predetermined wireless communication (for example, infrared communication) at least between itself and the central unit unit 30 . In addition, the data communication unit 16 may perform wireless communication with the rod unit 10 .

(Structure of the central unit part 30)

The description of the structure of the camera unit section 20 is as above. Next, the configuration of the central unit unit 30 will be described with reference to FIG. 5 .

The central unit unit 30 includes a CPU 31 , a ROM 32 , a RAM 33 , a switch operation detection circuit 34 , a display circuit 35 , a sound source device 36 , and a data communication unit 37 .

The CPU 31 executes overall control of the central unit unit 30 . For example, based on the impact detection received from the stick unit 10 and the position coordinates of the marker unit 15 received from the camera unit unit 20 , control to emit a predetermined musical sound is performed. In addition, the CPU 31 performs communication control with the stick unit 10 via the data communication unit 37 and communication control with the camera unit unit 20 .

The ROM 32 stores processing programs for various processing executed by the CPU 31 . In addition, the ROM 32 stores waveform data of various timbres, such as wind instruments such as flutes, saxophones, and trumpets; keyboard instruments such as pianos; stringed instruments such as guitars; Waveform data of percussion instruments such as snare drums (snare), cymbals (cymbal), and gongs (tam: tam) are stored in association with positional coordinates and the like.

The CPU 31 reads the waveform data stored in the ROM 32 in association with the position coordinates of the marker 15 at the time of impact detection (that is, when the event record is received), thereby emitting musical sounds corresponding to the performance movements of the player.

The RAM 33 stores values obtained or formed in processing, such as the state of the lever 10 received from the lever 10 (hit detection, etc.), the position coordinates of the marker 15 received from the camera unit 20, and the like.

The switch operation detection circuit 34 is connected to the switch 341 and receives input information through the switch 341 . The input information includes, for example, changes in the volume of the uttered musical sound or the timbre of the uttered musical sound, switching of the display on the display device 351 , and the like.

In addition, the display circuit 35 is connected to the display device 351 and performs display control of the display device 351 .

The sound source device 36 reads waveform data from the ROM 32 in accordance with an instruction from the CPU 31 to generate musical sound data, converts the musical sound data into an analog signal, and emits musical sound from a speaker not shown.

In addition, the data communication unit 37 performs predetermined wireless communication (for example, infrared communication) with the stick unit 10 and with the camera unit unit 20 .

(processing of performance device 1)

In the above, the structure of the rod part 10, the camera unit part 20, and the center unit part 30 which comprise the musical performance apparatus 1 was demonstrated. Next, the processing of the performance device 1 will be described with reference to FIG. 6 to FIG. 10 . FIG. 6 is a flowchart showing processing of the lever unit 10 . FIG. 7 is a diagram showing the impact detection timing (event record formation timing) of the lever portion 10 . FIG. 8 is a flowchart showing a mark lighting process of the lever portion 10 . In addition, FIG. 9 is a flowchart showing processing of the camera unit section 20 . FIG. 10 is a flowchart showing processing of the central unit unit 30 .

(Handling of the rod portion 10)

As shown in FIG. 6 , the CPU 11 of the shaft 10 reads out the mark characteristic information stored in the ROM 12 (step S1 ). In this process, the CPU 11 of the lever portions 10A and 10B reads out different mark feature information. The reading of different marker characteristic information can be performed by any method. For example, the poles 10A and 10B may communicate directly or via the center unit 30 . Alternatively, one piece of marker feature information may be previously associated with each of the levers 10 , and the CPU 11 of the levers 10A, 10B may read out the unique (individual) marker feature information associated with each other.

When the CPU 11 reads the tag feature information, it stores the tag feature information in the RAM 13 and sends it to the central unit unit 30 via the data communication unit 16 (step S2). At this time, the CPU 11 transmits, to the center unit 30 , the marker feature information and the identification information (rod identification information) capable of distinguishing the rod portions 10A and 10B from each other in association with each other.

Next, the CPU 11 reads motion sensor information, that is, sensor values output by various sensors from the motion sensor unit 14, and stores them in the RAM 13 (step S3). Thereafter, the CPU 11 performs posture detection processing of the lever portion 10 based on the read motion sensor information (step S4 ). In the attitude detection process, the CPU 11 detects the attitude of the stick 10 , such as the inclination of the stick 10 , the displacement of the roll angle, and the pitch angle, based on the motion sensor information.

Next, the CPU 11 performs impact detection processing based on the motion sensor information (step S5). Here, when a player performs a performance using the stick portion 10 , generally the same motion as hitting an actual musical instrument (for example, a drum) is performed. In such a (playing) motion, the player first swings up the shaft 10 and then swings down toward the virtual musical instrument. Then, immediately before the stick 10 hits the virtual musical instrument, a force to stop the movement of the stick 10 acts. At this time, since the player assumes that the musical sound is produced at the moment when the player hits the stick portion 10 on the virtual musical instrument, it is expected that the musical sound can be produced at the timing assumed by the player. Therefore, in this embodiment, it is assumed that the musical sound is emitted at the moment when the player hits the stick portion on the surface of the virtual musical instrument or slightly before.

Here, referring to FIG. 7 , an example of the timing at which musical sounds are emitted using the lever portion 10 will be described. FIG. 7 is a graph showing changes in output related to acceleration in the vertical direction of the motion sensor unit 14 when performing a musical performance using the stick unit 10 . In addition, the acceleration in the vertical direction refers to the acceleration in the vertical direction with respect to the horizontal plane. It can also be calculated from the acceleration decomposition of the Y-axis component. Alternatively, the acceleration in the Z-axis direction (the acceleration in the X-axis direction according to the roll angle) may also be decomposed and calculated. In addition, in FIG. 7 , a positive acceleration represents an acceleration in the downward direction applied to the rod portion 10 , and a negative acceleration represents an acceleration in the upward direction applied to the rod portion 10 .

Even in a state where the shaft 10 is at rest (a portion indicated by a), gravitational acceleration acts on the shaft 10 . Accordingly, the motion sensor unit 14 of the stationary pole unit 10 detects a constant acceleration in the vertical upward direction, that is, in the negative direction, against the acceleration due to gravity. In addition, when the acceleration acting on the rod part 10 is 0, the rod part 10 is in a state of free fall.

When the player lifts up the shaft 10 with the upward swing in a stationary state, the player moves in the opposite direction with respect to the acceleration of gravity. Thus, the acceleration acting on the rod portion 10 increases in the negative direction. Afterwards, when the lifting speed is decreased to make the pole 10 stationary, the upward acceleration decreases, and the acceleration in the negative direction of the pole 10 detected by the motion sensor unit 14 decreases (the portion indicated by b). In addition, the acceleration at the time when the upward swing reaches the highest point is only the gravitational acceleration (the portion shown near the boundary between b and c).

When the shaft portion 10 reaches the apex due to the upward swing, the player performs a downward swing of the shaft portion 10 . In this downward swing, the shaft 10 moves in the downward direction. Therefore, the acceleration acting on the rod portion 10 increases in the positive direction compared to the acceleration in the negative direction detected against the gravitational acceleration. After that, since the player reduces the acceleration in the downward direction in order to strike, the acceleration acting on the shaft 10 increases in the negative direction. During this period, after the moment when the downward swing reaches the highest speed, the state where only the acceleration of gravity acts on the shaft 10 again (the portion indicated by c) is reached.

Thereafter, when the player further applies an acceleration in the upswing direction to the shaft 10 for striking, the applied acceleration increases in the negative direction. In addition, when the impact is completed, the shaft 10 stops again, and returns to the state of detecting the negative acceleration in the form opposite to the gravitational acceleration (the part indicated by d).

In the present embodiment, the moment when acceleration in the upward swing direction is applied after performing the downward swing is detected as the moment when the player hits the stick portion 10 on the surface of the virtual musical instrument. That is, in the portion indicated by d in FIG. 7 , point A, which is further increased by a predetermined value in the negative direction from the downward swing state, in other words, from when the applied acceleration is only the gravitational acceleration, is taken as the time of impact detection.

The time at which the impact is detected is regarded as the sounding time, and when it is determined that the sounding time has been reached, the CPU 11 of the lever 10 creates an event record and sends it to the central unit 30 . As a result, the central unit unit 30 executes sound generation processing to generate musical sounds.

Returning to FIG. 6 , in the impact detection process shown in step S5 , an event record is generated based on the motion sensor information (for example, the acceleration composite value of the acceleration sensor). In this case, the generated event log may also include the volume of the emitted musical sound. In addition, the volume of the musical sound may be obtained from, for example, the maximum value of the acceleration synthesis value.

Next, the CPU 11 performs a process of detecting information (hereinafter referred to as action information) indicating a predetermined action (action: action) of the player based on the motion sensor information, that is, an action detection process (step S6 ). Next, the CPU 11 transmits the information detected in the processes of steps S4 to S6 , that is, posture information, impact information, and movement information, to the center unit 30 via the data transmission unit 16 (step S7 ). At this time, the CPU 11 sends the posture information, the strike information, and the motion information to the center unit unit 30 in association with the stick identification information.

Next, the CPU 11 performs mark turning on and off processing (step S8 ), shifts to the processing of step S3 , and executes the subsequent processing repeatedly. Here, the marker lighting and extinguishing process will be described in detail with reference to FIG. 8 , and the CPU 11 controls the marker 15 to be turned on or off based on motion sensor information and the like.

(mark lighting and extinguishing processing)

Next, with reference to FIG. 8 , the marker turning on and off processing will be described.

First, the CPU 11 of the shaft 10 determines whether or not a downward swing has been detected based on motion sensor information, posture information, impact information, motion information, and the like (step S11 ). At this time, when the downward swing is detected, the CPU 11 performs a lighting process of the marker portion 15 (step S12 ), and ends the marker lighting and extinguishing process.

On the other hand, when the downward swing is not detected, the CPU 11 determines whether the upward swing is detected based on motion sensor information, posture information, impact information, motion information, and the like (step S13 ). At this time, when the upward swing is detected, the CPU 11 performs the extinguishing process of the marker 15 (step S14 ), and ends the marker lighting and extinguishing process. On the other hand, when the upward swing is not detected, the CPU 11 ends the mark lighting and extinguishing processing.

Here, the detection of the downward swing and the upward swing in step S11 and step S13 may be performed by any method, for example, the acceleration in the vertical direction of the shaft part 10 may be used. Next, the detection of the downward swing and the upward swing by the CPU 11 will be described by taking a case where the change in acceleration in the vertical direction of the motion sensor unit 14 shows a change as shown in FIG. 7 as an example.

The start of the upswing is taken as the moment of hit detection. That is, point A is assumed to be point A obtained by increasing the acceleration by a predetermined value in the negative direction from the state where the applied acceleration is only gravitational acceleration in the portion indicated by d in FIG. 7 . Of course, it is also possible to set a time interval between the start of the upswing motion and the hit detection time, instead of making the same time.

In addition, the start of the downward swing is point B, which is point B at which the applied acceleration increases by a predetermined value in the positive direction from the state of only the gravitational acceleration in the portion shown by c in FIG. 7 . Here, in general performance operations of percussion instruments, etc., the upward swing is always performed right away from the downward swing. Therefore, in the present embodiment, the end of the downward swing and the start of the upward swing are set at the same time. That is, the downward swing motion starts at the time of point B in FIG. 7 and ends at the time of point A. Of course, a time interval can also be set between the end of the downward swing and the start of the upward swing.

In addition, in this embodiment, the downward swing and the upward swing are detected based on the acceleration in the vertical direction detected by the motion sensor unit 14 (acceleration sensor). However, as another example, the posture information of the shaft 10 may be used in the detection of the downward swing and the upward swing. Here, the pose information can use the displacement of the pitch angle. For example, when the pitch angle is shifted in the downward direction, the CPU 11 detects the start of the downward swing. The CPU 11 detects the start of the upward swing when the pitch angle is shifted in the upward direction and when the pitch angle is displaced in the downward direction.

In addition, the detection of the downward swing and the upward swing may also be performed by the camera unit 20 . That is, it is also possible to judge the movement of the player's hand based on the moving image captured by the camera unit 20, and detect the downward swing and the upward swing. In this case, the stick part 10 can receive these detection information from the camera unit part 20 .

In this way, the detection of the swing-down motion and the swing-up motion in step S11 and step S13 can be performed by various methods.

In addition, since the lever portion 10 and the camera unit portion 20 are not synchronized, there are cases where the camera unit portion 20 cannot perform appropriate shooting according to the timing when the marker portion 15 is turned off. Therefore, in the lever portion 10 , the timing of turning off the marker portion 15 may be delayed by one frame of imaging by the camera unit portion 20 . In this way, in the camera unit 20 , the position coordinates of the marker 15 at the time of hitting can be specified regardless of the time shift caused by the asynchrony of the lever 10 and the camera unit 20 .

(Processing of the camera unit section 20)

As shown in FIG. 9 , the CPU 21 of the camera unit 20 performs marker detection condition acquisition processing (step S21 ). In this process, CPU21 acquires the marker detection condition information transmitted from the center unit part 30, and stores it in RAM23. In addition, the marker detection condition information is a condition for detecting each marker part 15 of the lever part 10A, 10B, and is created from marker characteristic information (refer step S31, step S32 of FIG. 10). Here, as described above, as the mark feature information, for example, the shape, size, hue, chroma, or brightness of the mark can be used. Next, the CPU 21 performs marker detection condition setting processing (step S22). In this process, the CPU 21 performs various settings of the marker detection unit 24 based on the marker detection condition information.

Next, the CPU 21 performs the first marker detection process (step S23) and the second marker detection process (step S24). In these processes, the CPU 21 acquires marks such as position coordinates, dimensions, and angles of the mark portion 15 (first mark) of the rod portion 10A and the mark portion 15 (second mark) of the rod portion 10B detected by the mark detection portion 24 . The detection information is stored in RAM23. At this time, the marker detection unit 24 detects marker detection information for the marker unit 15 that is emitting light.

Next, the CPU 21 transmits the marker detection information obtained in steps S23 and S24 to the central unit unit 30 via the data communication unit 25 (step S25), and proceeds to the processing of step S23.

(Processing of the central unit unit 30)

As shown in FIG. 10 , the CPU 31 of the center unit 30 receives the mark feature information from the rod 10 and stores it in the RAM 33 (step S31 ). Next, the CPU 31 generates marker detection condition information based on the marker characteristic information and the detection condition set via the switch 341, and transmits it to the camera unit 20 via the data communication unit 37 (step S32).

Next, the CPU 31 receives marker detection information for each of the first marker and the second marker from the camera unit 20, and stores them in the RAM 33 (step S33). In addition, the CPU 31 receives posture information, impact information, and movement associated with the stick identification information from the stick parts 10A and 10B, and stores them in the RAM 33 (step S34 ).

Next, the CPU 31 judges whether or not there is a hit (step S35). In this process, the CPU 31 judges the presence or absence of a hit based on whether or not an event record has been received from the stick 10 . At this time, when it is determined that there is a hit, the CPU 31 performs a hit process (step S36 ). In the striking process, CPU 31 reads waveform data corresponding to position coordinates, dimensions, angles, etc. included in the mark detection information from ROM 32 , and outputs it to sound source device 36 together with volume data included in the event log. Then, the sound source device 36 emits corresponding tones according to the received waveform data.

After step S36, or when it is determined as "No" in step S35, the CPU 31 determines whether there is a movement based on the movement information received from the lever 10 (step S35). At this time, when it is determined that there is an action, the CPU 31 performs action processing based on the received action information (step S38 ), and proceeds to the process of step S33 . On the other hand, when it is judged that there is no operation, the CPU 31 proceeds to the processing of step S33.

The configuration and processing of the musical performance device 1 according to the present embodiment have been described above. According to such performance device 1 , the occurrence of a situation (hit) in which the position coordinate data of the marker 15 is required is predicted, the marker 15 is turned on in advance, and when the situation ends, the lighting of the marker 15 is terminated. Thus, since the marker 15 is turned on only when the position coordinate data of the marker 15 is needed, the power consumption of the pole 10 can be reduced compared with the case of always turning on, and the length of the pole 10 can be realized. Time-driven and lightweight.

In addition, by turning on and off the marking portion 15 in conjunction with the situation (hit), a visual effect can be expected, and the performance of the performance using the stick portion 10 can be improved.

As mentioned above, although embodiment of this invention was described, embodiment is only an illustration, and does not limit the technical scope of this invention. The present invention can take other various embodiments, and various changes such as omissions and substitutions can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in this specification and the like, and are included in the invention described in the claims and their equivalents.

In the above-mentioned embodiment, the virtual drum set D (see FIG. 1 ) has been described as an example of a virtual percussion instrument. However, it is not limited thereto, and the present invention can be applied to other musical instruments such as a xylophone that emits musical tones by swinging the stem portion 10 downward.

In addition, in the above-mentioned embodiment, arbitrary processing among the processes performed by the lever unit 10, the camera unit unit 20, and the center unit unit 30 may be performed by other units (the lever unit 10, the camera unit unit 20, and the center unit unit 30). )conduct. For example, processing such as impact detection performed by the CPU 11 of the lever unit 10 may be performed by the center unit unit 30 .

In addition, in the above-mentioned embodiment, the light emission control of the marking part 15 which the lever part 10 has was demonstrated. However, it is not limited to the rod portion 10, and the light emission control of the present invention may be performed on other components having a light emitting portion. That is, the present invention can be applied to a light emission control device that detects the start and end of an action given to a component having a light emitting section, lights the light emitting section in response to the detection of the start of the action, and responds to the end of the action. The detection reacts by turning off the light emitting part. At this time, the CPU (detection unit) can detect the start and end of the operation based on values detected by various sensors (motion sensor unit). In addition, the CPU (control unit) can also perform lighting control that responds to the start and end of motion.

Claims (5)

1. a music performance apparatus, comprising:

Play parts, kept by player, the acceleration transducer of the illuminating part comprising luminescence and extinguishing and the acceleration detecting gravity direction;

Filming apparatus, takes the image in the shooting space kept existing for the above-mentioned player of above-mentioned performance parts;

Pronunciation device, pronounces according to the position coordinates of the above-mentioned illuminating part in the luminescence in the above-mentioned shooting space that above-mentioned filming apparatus photographs;

Detecting unit, when detecting that above-mentioned acceleration adds setting along gravity direction compared with acceleration of gravity, detect by brandishing work under the above-mentioned performance parts that above-mentioned player carries out, when detecting that above-mentioned acceleration adds setting along the direction contrary with gravity direction compared with acceleration of gravity, detect above-mentioned under brandish the end of work; And

Luminous extinguishing control module, when brandishing the beginning of work under above-mentioned detecting unit detection is above-mentioned, performs the light emitting control making above-mentioned light from light source, at the end of brandishing work, performs the extinguishing control that above-mentioned illuminating part is extinguished under above-mentioned detecting unit detection is above-mentioned.

2. music performance apparatus according to claim 1, wherein:

Above-mentioned performance parts, above-mentioned filming apparatus, above-mentioned pronunciation device are can the mode of radio communication connect.

3. an emission control device, comprising:

Functional unit, has the acceleration transducer of the acceleration of illuminating part and detection gravity direction;

Detecting unit, when detecting that above-mentioned acceleration adds setting along gravity direction compared with acceleration of gravity, the beginning of the action of brandishing work detected down, when detecting that above-mentioned acceleration adds setting along the direction contrary with gravity direction compared with acceleration of gravity, detect above-mentioned under brandish the end of work; And

Control module, reacts to the detection of brandishing the beginning of work under above-mentioned to above-mentioned detecting unit, above-mentioned illuminating part is lighted, and reacts to the detection of brandishing the end of work under above-mentioned to above-mentioned detecting unit, and above-mentioned illuminating part is extinguished.

4. a control method for music performance apparatus, this music performance apparatus comprises: play parts, and the acceleration transducer of the illuminating part comprising luminous and extinguishing and the acceleration detecting gravity direction, is kept by player; Filming apparatus; And pronunciation device,

The control method of this music performance apparatus comprises:

Shooting step, by above-mentioned filming apparatus, takes the image in the shooting space kept existing for the above-mentioned player of above-mentioned performance parts;

Pronunciation step, according to the position coordinates of the above-mentioned illuminating part in the luminescence in the above-mentioned shooting space that above-mentioned filming apparatus photographs, makes above-mentioned pronunciation device pronounce;

Detecting step, when detecting that above-mentioned acceleration adds setting along gravity direction compared with acceleration of gravity, the beginning of work is brandished under the above-mentioned performance parts that above-mentioned player carries out being detected, when detecting that above-mentioned acceleration adds setting along the direction contrary with gravity direction compared with acceleration of gravity, detect above-mentioned under brandish the end of work; And

Luminous extinguishing rate-determining steps, when detect in above-mentioned detecting step above-mentioned under brandish the beginning of work time, perform and make the light emitting control of above-mentioned light from light source, when detect in above-mentioned detecting step above-mentioned under brandish work at the end of, perform the extinguishing control that above-mentioned illuminating part is extinguished.

5. the control method of music performance apparatus according to claim 4, wherein:

Above-mentioned performance parts, above-mentioned filming apparatus, above-mentioned pronunciation device are can the mode of radio communication connect.