JP2001142463A - Method and device for analyzing musical sound waveform, and recording medium which records musical sound waveform analyzing program on it - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make the locus of the peak of frequency spectrum able to be tracked even in sound such as piano high tone which is percussive or pulsatile and is quickly attenuated. SOLUTION: The frequencies of base sound and its double sound are detected by analyzing the vicinity of the peak of the frequency spectrum of an original waveform employing a first STF analysis B. Then, the frequency spectrum of an FFT analysis D is filter processed by a frequency mask process C in accordance with the detected frequency. Then, secondary waveforms are generated from the filter processed frequency spectrum by an IFFT process E. Then, an envelope of the secondary waveforms is extracted by an envelope extracting process F, an inverse envelope is multiplied to the secondary waveforms by a normalize process G and normalized time waveforms are obtained. Then, the peak of the frequency spectrum of the normalized time waveforms is racked by a second SFT analysis H. Then, an envelope is given to the result of the peak tracking in an envelope giving process I and musical sound is synthesized in a synthesizing process J.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a musical tone waveform analyzing method, a musical tone waveform analyzing apparatus, and a musical tone waveform analyzing program suitable for tracking a locus of a peak point of amplitude data in a time series frequency spectrum of a musical tone to be analyzed. It relates to a recorded recording medium.

[0002]

2. Description of the Related Art Conventionally, tone analysis and (re) synthesis (Analysis & (Re) Syn) for analyzing the properties of musical tones and synthesizing musical tone waveforms
In the technical field of “thesis”, there are the following techniques for analyzing the frequency spectrum of a musical sound waveform. First, a frame is cut out by multiplying a time-series sampling waveform of a musical tone by a window function, and a fast Fourier transform (FFT) process is performed to obtain frequency spectrum analysis data. The analysis data of the frequency spectrum includes data on the frequency axis, that is, frequency data, amplitude data, and phase data, and detects all peak points (frequency values) that can be regarded as peaks of the amplitude data. Then, the window of the window function is moved to update the frame, the above processing is repeated, and the peak point of the amplitude data of the frequency spectrum in each frame is detected.

Here, the peak point in each frame is
Generally, a plurality of peaks are detected corresponding to a plurality of frequencies, and these peak points are detected corresponding to a fundamental tone of an original sampling waveform, a component corresponding to a harmonic thereof, a noise component, a side lobe of an FFT window function, and the like. Is done. Among these, the peak points corresponding to the fundamental tone and the overtone become components having strong periodicity in the frequency axis direction, and furthermore tend to form long trajectories that are continuous in the FFT frame in the time axis direction. Therefore, by tracking the locus of the peak point, a component having a strong periodicity (the frequency of a fundamental tone or an overtone) is extracted, and the extracted component is separated from other components having a strong randomness.

[0004] In this specification, the technique of tracking the locus of the peak point of the frequency spectrum as described above is referred to as a "spectral locus tracking method" or "STF (Spectral)".
Trajectory Finder).

When such a spectrum locus tracking method is used, when creating waveform data for a waveform memory readout type sound source (PCM sound source), a loop waveform (repetitive reading for a continuous sound), which is the most complicated operation, is required. It becomes easy to determine the reading section of the waveform data). Further, by editing the analysis result, there is an advantage that it becomes easy to add a subtle flavor to the input original sound or to add a new characteristic that was not present in the original sound.

[0006]

However, depending on the type of musical sound, the peak cannot be tracked well by the above-described spectrum trajectory tracking method. For example, when analyzing percussive or pulse-like fast-decay sounds, such as high-pitched sounds of a piano, such musical tones may have many random components in addition to harmonic components, and the amplitude level may suddenly increase after an attack. The trajectory of the peak point was not well tracked in the low attenuation portion, and it was difficult to separate the fundamental component having strong periodicity and its harmonic component from the noise component having strong randomness.

SUMMARY OF THE INVENTION It is an object of the present invention to be able to track the locus of the peak of the frequency spectrum even with a percussive or pulse-like fast-decay sound such as a treble sound of a piano.

[0008]

According to a first aspect of the present invention, there is provided a musical tone waveform analyzing method, comprising: inputting an original waveform of a musical tone to be analyzed, extracting an envelope of the original waveform, and determining a level of the original waveform based on the envelope. And the peak of the frequency spectrum is tracked with respect to the waveform whose level has been corrected.

According to the musical tone waveform analysis method of the present invention, the level of the original waveform is corrected based on the envelope of the original waveform. Can be standardized to about the level of the waveform of the attack portion, and the peak of the frequency spectrum is tracked with respect to the corrected waveform, so that the peak can be tracked even for the attenuated portion.

Note that the envelope of the original waveform is an envelope value with the peak level of the waveform amplitude being 1, and the level is corrected by multiplying the inverse envelope value by a reciprocal of the envelope value. Suitable to do. Further, by setting the reverse envelope value to 1 from the beginning of the musical tone to the peak of the amplitude, the characteristics of the attack portion can be left as it is, and a subtle tone of the attack portion can be reproduced.

According to a second aspect of the present invention, there is provided a musical tone waveform analyzing method.
Input the original waveform of the musical tone to be analyzed, analyze the vicinity of the peak of the original waveform, detect the frequency of the fundamental tone and the harmonics, and select the neighborhood of the frequency of the fundamental tone and the harmonics with respect to the time-series frequency spectrum of the original waveform. A secondary waveform from the filtered frequency spectrum, extract an envelope of the secondary waveform, correct the level of the secondary waveform based on the envelope, and correct the level. A peak of the frequency spectrum is tracked for the obtained secondary waveform, and a synthesized waveform is generated from the envelope and the frequency components near the fundamental tone and the overtone obtained by tracking the peak.

According to the musical tone waveform analysis method of the second aspect, the S / N ratio of the frequency components of the fundamental tone and the overtone is better in the secondary waveform than in the original waveform. Since the peak of the frequency spectrum is tracked in the same manner as in the first aspect, the tracking accuracy of the peak in the attenuated portion is further improved. Note that a composite waveform and a residual waveform obtained by subtracting the composite waveform from the original waveform may be output.

According to a third aspect of the present invention, there is provided a musical sound waveform analyzer.
Waveform input means for inputting the original waveform of the musical tone to be analyzed; frequency detecting means for analyzing the vicinity of the peak of the original waveform to detect the frequencies of the fundamental and harmonics; and Filter processing means for selecting the vicinity of the fundamental and harmonic frequencies detected by the frequency detection means, secondary waveform generation means for generating a secondary waveform from the frequency spectrum selected by the filter processing means, and secondary waveform generation Means for extracting the envelope of the secondary waveform generated by the means, level correction means for correcting the level of the secondary waveform based on the envelope extracted by the envelope extraction means, and level correction by the level correction means. The peak of the frequency spectrum is tracked with respect to the corrected secondary waveform, and the frequency components near the fundamental and harmonics and the envelope are used. A synthesized waveform generating means for generating a formed waveform, characterized by comprising a.

According to the musical sound waveform analyzer of the third aspect configured as described above, the same operation and effect as those of the second aspect can be obtained.

According to a fourth aspect of the present invention, there is provided a recording medium, comprising: a step of inputting an original waveform of a musical tone to be analyzed; a step of extracting an envelope of the original waveform; and a step of correcting the level of the original waveform based on the envelope. And a step of tracking the peak of the frequency spectrum with respect to the waveform whose level has been corrected. The tone waveform analysis program for executing by a computer is recorded. According to the execution of the musical tone waveform analysis program, the same operation and effect as those of the first aspect can be obtained.

[0016]

FIG. 2 is a block diagram of a musical tone waveform analyzer according to an embodiment of the present invention, and shows a case where the personal computer having a CPU 1 is used.
The program memory 2 is a hard disk drive or a CD-RO
This is an M device or other external storage device, and stores a tone waveform analysis program whose main part is shown in the flowchart of FIG. The data memory 3 is a RAM or the like, and stores tone waveform data to be analyzed, analysis results, and the like. The input device 4 and the performance operator 5 are a keyboard and the like, and the display 6 is a CRT or a liquid crystal display. The tone synthesizer 7 is a tone generator board or tone generator equipped with various LSI chips for synthesizing a tone based on the analysis result. Further, the network interface 8 is an interface for connecting to various networks 9 such as MIDI, LAN, and telephone line.

The CPU 1 performs a tone waveform analysis process, which will be described later, on the designated tone waveform data in the data memory 3 based on the tone waveform analysis program stored in, for example, a hard disk device of the program memory 2, and performs a Fourier analysis process. The peak point of the frequency spectrum in each frame is detected. Then, a peak point forming a locus between frames is extracted, and analysis data of the peak point in each frame forming the locus is obtained as an analysis result.

The CPU 1 converts frequency data, amplitude data, and phase data, which are analysis data of the peak points forming the trajectory, for each frame based on the trajectories of the peak points obtained as a result of the waveform analysis described later. By outputting to the tone synthesizing unit 7 in accordance with the frame time, the tone synthesizing unit 7 performs Fourier synthesis. Further, a difference between the original waveform data and the Fourier-combined waveform data is obtained as residual waveform data. The waveform data obtained by performing Fourier synthesis with the residual waveform data is re-synthesized by the tone synthesizer 7, converted from digital to analog, and generated as a tone signal. In addition, desired waveform data can be obtained by modifying the analysis data and the residual waveform data output to the tone synthesizer 7.

FIG. 1 is a functional block diagram showing the flow of various processes in the embodiment. The function of each block is shown in FIG.
Or the execution of the musical tone waveform analysis program shown in FIG. Next, FIG. 1 and FIGS.
The processing flow of the embodiment will be described with reference to FIG. It should be noted that the figure numbers added to the connector lines in FIG. 1 are the figure numbers of the drawings showing the output waveforms and the like. “STEP1 to STEP8” appended to each block in FIG. 1 correspond to each step in the flowchart in FIG.

In FIG. 1, an original waveform A is original waveform data designated as a musical tone waveform to be analyzed, and is read out from the data memory 3, for example. Here, FIG. 3 is a diagram showing a musical tone waveform (original waveform data) to be analyzed in the embodiment, which is a time waveform of a piano with a decay time of about 1.5 seconds for 83 keys (basic tone = 3135.96 Hz). As shown in the figure, this waveform abruptly attenuates from the attack portion, and the amplitude after about 0.3 seconds (not shown in the figure) is extremely attenuated as compared with the attack portion. This waveform data is time-series sampling data obtained by sampling and quantizing the amplitude of the musical tone waveform at a sampling frequency of 44.1 kHz.

In FIG. 1, in the first STF analysis B,
The peak locus of the frequency spectrum is tracked with respect to the original waveform of FIG. 3 by the spectrum locus tracking method, and the frequencies of the fundamental tone and the overtone corresponding to the tracked locus are obtained. Here, FIG. 4A is a diagram showing a distribution of peak points of a time-series frequency spectrum obtained by performing the FFT processing on the original waveform data, and FIG. 4A is approximately 1500 msec from the top. 1.5 seconds)
FIG. 4B shows the distribution of peak points for 25 msec in which the vicinity of the attack portion is enlarged in the time axis direction.

Although the distribution appears in a linear or irregular shape in the figure, each peak point corresponds to each dot forming the line or the irregular shape. Each peak point has amplitude data. As shown in the spectrum of FIG. 4A, only the peak corresponding to the fundamental tone appears in the latter half of the original waveform, but as in the enlarged spectrum of FIG. 4B, a peak appears up to the fourth harmonic in the vicinity of the attack portion. . Note that the pitches of the second harmonic, the third harmonic, and the fourth harmonic tend to shift from integer multiples of the fundamental tone to higher frequencies.

Therefore, in this embodiment, the first STF
By the analysis B, the peak locus is tracked in the vicinity of the peak of the original waveform data, that is, in the vicinity of the attack portion, and the frequencies of the fundamental tone and the overtone corresponding to the tracked locus are obtained. First, the four peak points indicated by circles in FIG. 4B are picked up, and the trajectories of the peak points are traced before (past) and after (future) temporally from the peak points. The tracking process of the locus of the peak point is performed, for example, as follows. A predetermined range is provided around the value of the amplitude data of the peak point picked up first, and the peak point having the amplitude data within the range is selected as a trajectory. Then, a predetermined range is provided centering on the largest amplitude data from among them, the same processing is performed, and this is repeated.

FIG. 5 is a diagram showing a result of tracking the locus of the peak point obtained in the first STF analysis B (referred to as "STF data"), from the fundamental tone to the fourth harmonic appearing linearly. Is detected. Then, from the detected frequency of the STF data, a filter coefficient in a frequency mask C process described below is determined, and the filter coefficient is output to the frequency mask C.

In the frequency masking process C, for example, a peak filter bank as shown in FIG. 6 is used, and the center frequency of each peak filter is determined from the locus data included in the STF data. The bandwidth is determined from the frequency of the fundamental tone, and in this embodiment, it is set to 1/8 of the fundamental tone. Then, the original waveform is subjected to Fourier analysis processing by the FFT analysis D over the entire time, and a peak filter is applied to the time-series frequency spectrum data, which is the analysis result of the FFT analysis D, by the frequency mask processing C. In the process of applying the peak filter, the frequency spectrum is multiplied by the amplitude (the maximum value is 1) shown in FIG. 6 for each FFT frame time.

An inverse Fourier transform is performed by IFFT processing E based on the frequency spectrum subjected to the peak filter as described above, and the time waveform shown in FIG. 7 is obtained as a secondary waveform. Here, when the secondary waveform of FIG. 7 is subtracted from the original waveform of FIG. 3, a residual waveform as shown in FIG. 8 is obtained. This residual waveform should have only the noise and the got of the attack part remaining. The residual waveform of FIG. 8 is reproduced, and if the pitch component can be heard, the first STF analysis B is redone. To perform the same processing.

Next, in FIG. 1, the envelope of the secondary waveform (time waveform) shown in FIG. 7 is extracted by an envelope extracting process F. FIG. 9 (A) shows the envelope of the secondary waveform of FIG. 7, and FIG. 9 (B) shows the vicinity of the origin enlarged in the time direction. Note that the actual envelope is shown in FIG.
As shown by the dashed line in (B), the tone rises from the time of generation of the musical tone, reaches the attack time, and then attenuates. In this embodiment, however, the envelope value is set to 1 at the time of the maximum amplitude attack.
In addition, as shown by the solid line in FIG. 9 (B), the portion from the beginning of the waveform to the time of the attack is set to a constant envelope value 1, and the shape of the envelope of the waveform after the time of the attack is unchanged.

Then, by the normalizing process G, the reciprocal of the envelope value (FIG. 9B) is multiplied by the secondary waveform of FIG. 7 to obtain a normalized time waveform as shown in FIG. . This means that the level of the secondary waveform has been corrected based on the envelope. Note that, as in this embodiment, the reverse envelope value is 1 from the beginning of the musical tone to the time of the attack (attack portion).
Therefore, even if the correction is made, the characteristics of the attack portion can be left as it is, and the subtle tone of the attack portion can be reproduced.

Next, the second STF analysis H shown in FIG.
The peak locus of the frequency spectrum is tracked by the spectrum locus tracking method with respect to the normalized time waveform, and the frequencies of the fundamental tone and the overtone corresponding to the tracked locus are obtained. Here, FIG. 11 shows the F time for the normalized time waveform.
It is a figure which shows the distribution of the peak point of the spectrum obtained by performing FT processing, and has shown the distribution of the peak point in about 920 msec from the head. Then, the peak locus is tracked with respect to the distribution of the peak points in FIG.
F data (frequency data, amplitude data, and phase data obtained as a result of tracking the locus of the peak point) is obtained. In this example, the trajectory from the fundamental tone to the fourth harmonic is successfully tracked as shown in the figure.

Here, the time waveform data may be synthesized by performing an inverse Fourier transform based on the STF data of FIG. 12, and an envelope may be applied thereto. However, since STF data is required when performing a waveform data creation operation such as determining a loop waveform readout section, in this embodiment, first, the envelope adding process I in FIG.
Envelope processing is performed in the STF data area, that is, in the time-frequency area (hereinafter, referred to as “STF area”), and the enveloped ST is processed as shown in FIG.
Obtain F data. Although the third overtone is a broken part, it is less than a few decibels and has almost no effect. Note that the peak amplitude data (level) changes by applying the envelope, but the change in the amplitude data is not shown in FIG.

On the other hand, an inverse Fourier transform is performed by the synthesizing process J based on the enveloped STF data to synthesize a time waveform as shown in FIG. Further, as shown by the inverter symbol and the adder symbol in FIG. 1, when the time waveform in FIG. 14 is subtracted from the original waveform in FIG. 3, a residual waveform as shown in FIG. 15 is obtained. As a result of reproducing and listening to the residual waveform data of FIG. 15,
It was found that the component having the pitch could be clearly separated until the end of the waveform.

As described above, since the original waveform is corrected by its envelope and the peak locus of the frequency spectrum is tracked by the spectrum locus tracking method, it is abruptly attenuated from the attack portion like a musical tone in a high frequency range of a piano. The peak trajectory can be tracked even with a simple waveform.

In the above embodiment, the original waveform is once subjected to a filtering process by the first STF analysis B, the frequency mask C, and the IFFT process to generate a secondary waveform, and the envelope of the secondary waveform is generated. Although the peak locus is tracked by performing the correction in (1), the peak locus may be tracked by directly correcting the original waveform with the envelope without filtering.

In the embodiment, the synthesis of the waveform and the generation of the musical tone are performed after the waveform analysis. However, the present invention relates to the analysis of the waveform. It is not limited. Further, it is not necessary to generate the residual waveform data.

Further, in the embodiment, the case where the musical tone waveform analysis program is stored in the program memory 2 in advance has been described. For example, a tone waveform analysis program is recorded on a CD-ROM, and the tone waveform analysis program is loaded from a CD-ROM device onto a hard disk. And CPU
The system 1 develops the musical tone waveform analysis program of the hard disk in a RAM or the like, and controls the operation of the musical tone waveform analysis based on the RAM program in the same manner as in the above embodiment. This allows the CPU to perform the same operation as when the tone waveform analysis program is stored in the program memory. This makes it possible to easily install, add, or upgrade the tone waveform analysis program. Alternatively, the tone waveform analysis program may be recorded on a floppy disk, a magnetic disk (MO), or the like, and supplied to the RAM or the hard disk.

The tone waveform analysis program may be downloaded using the network interface 7. At this time, for example, a connection is made to a network 9 such as a LAN (local area network), the Internet, or a telephone line, and the distribution of the musical tone waveform analysis program from the server computer via the network 9 is recorded on the hard disk. To complete the download. Further, a tone waveform analysis program may be executed through a network.

The present invention is not limited to the personal computer as in the above embodiment, but includes various electronic musical instruments, tone generators, sequencers, effectors, and various other devices.
It may be incorporated as a function or an application in a system or the like that connects each device using communication means such as IDI or various networks.

It should be noted that the medium storing the tone waveform analysis program as described in the above embodiment, that is, RO
A destination storage device such as M, RAM, hard disk, CD-ROM, magneto-optical disk, DVD (digital versatile disk), or network server computer is equivalent to a medium storing the tone waveform analysis program according to claim 4 of the present invention. I do.

[0039]

As described above, according to the first aspect of the present invention,
According to the musical tone waveform analysis method of the present invention or the execution of the musical tone waveform analysis program recorded on the recording medium of claim 4, the level of the original waveform is corrected based on the envelope of the original waveform. Since the level of the attenuated portion can be standardized to about the level of the waveform of the attack portion, and the peak of the frequency spectrum is tracked with respect to the corrected waveform, the peak of the attenuated portion can also be tracked.

According to the musical tone analyzing method of the present invention or the musical tone analyzing apparatus of the present invention, the secondary waveform has a better S / N ratio of the fundamental and harmonic frequency components than the original waveform.
Further, since the peak of the frequency spectrum is tracked with respect to the secondary waveform in the same manner as in the first aspect, the tracking accuracy of the peak is further improved for the attenuated portion.

[Brief description of the drawings]

FIG. 1 is a functional block diagram illustrating a flow of various processes according to an embodiment of the present invention.

FIG. 2 is a block diagram of a musical tone waveform analyzer according to the embodiment.

FIG. 3 is a diagram showing a musical tone waveform to be analyzed in the embodiment.

FIG. 4 is a diagram showing a distribution of peak points of a frequency spectrum of original waveform data in the embodiment.

FIG. 5 is a diagram showing a result (STF data) of tracking of a locus of a peak point in FIG. 4;

FIG. 6 is a diagram illustrating characteristics of a peak filter bank in frequency mask processing according to the embodiment.

FIG. 7 is a diagram showing a time waveform as a secondary waveform in the embodiment.

FIG. 8 is a diagram showing a residual waveform obtained by subtracting a secondary waveform from an original waveform in the embodiment.

FIG. 9 is a diagram illustrating an envelope of a secondary waveform in the embodiment.

FIG. 10 is a diagram showing a normalized time waveform in the embodiment.

FIG. 11 is a diagram illustrating a distribution of peak points of a frequency spectrum of a normalized time waveform in the embodiment.

FIG. 12 shows a result of tracking the locus of the peak point in FIG. 11 (ST
F data).

FIG. 13 is a diagram showing STF data obtained by applying an envelope to the STF data shown in FIG. 12;

14 is a diagram showing a time waveform synthesized from the enveloped STF data of FIG. 13;

15 is a diagram illustrating a residual waveform obtained by subtracting the time waveform of FIG. 14 from an original waveform.

FIG. 16 is a flowchart of a main part of a musical tone waveform analysis program in the embodiment.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... CPU, 2 ... Program memory, 3 ... Data memory, 7 ... Tone synthesis part, F ... Envelope extraction processing, G ...
Normalization processing

Claims (4)

[Claims] 1. An original waveform of a musical tone to be analyzed is input, an envelope of the original waveform is extracted, a level of the original waveform is corrected based on the envelope, and a frequency spectrum of the waveform whose level has been corrected is targeted. A musical sound waveform analysis method characterized by tracking a peak. 2. An original waveform of a musical tone to be analyzed is input, and the vicinity of a peak of the original waveform is analyzed to detect the frequencies of a fundamental tone and an overtone. Performing a filtering process to select the vicinity of the frequency, generating a secondary waveform from the filtered frequency spectrum, extracting an envelope of the secondary waveform, and correcting a level of the secondary waveform based on the envelope. Tracking the peak of the frequency spectrum with respect to the secondary waveform whose level has been corrected, and generating a composite waveform from the frequency components near the fundamental and harmonics obtained by tracking the peak and the envelope. Musical sound waveform analysis method. 3. A waveform input means for inputting an original waveform of a musical tone to be analyzed, a frequency detecting means for analyzing a vicinity of a peak of the original waveform to detect a frequency of a fundamental tone and a harmonic, and a time-series frequency of the original waveform. Filter processing means for selecting near the frequencies of the fundamental tone and harmonic detected by the frequency detection means for the spectrum; secondary waveform generation means for generating a secondary waveform from the frequency spectrum selected by the filter processing means; An envelope extracting means for extracting an envelope of the secondary waveform generated by the secondary waveform generating means; a level correcting means for correcting a level of the secondary waveform based on the envelope extracted by the envelope extracting means; The peak of the frequency spectrum is tracked with respect to the secondary waveform whose level has been corrected by the level correcting means, and the frequency components near the fundamental tone and the overtone and the A musical sound waveform analyzer comprising: a synthetic waveform generating means for generating a synthetic waveform from an envelope. 4. A step of inputting an original waveform of a musical tone to be analyzed, a step of extracting an envelope of the original waveform, a step of correcting a level of the original waveform based on the envelope, and a waveform whose level has been corrected. Recording a musical tone waveform analysis program for executing by a computer a step of tracing a peak of a frequency spectrum for the object.