CN117765902A - Method, apparatus, device, storage medium and program product for generating accompaniment of music - Google Patents

Abstract

The application discloses a method, a device, equipment, a storage medium and a program product for generating a musical accompaniment, and relates to the technical field of computers. The method comprises the following steps: obtaining music data; extracting music information from the music data to obtain beat data information, chord data information and melody data information; generating musical accompaniment data by taking melody data information as a generation limiting condition of musical accompaniment and through beat data information and chord data information; and rendering the audio data based on the music accompaniment data to obtain the music accompaniment corresponding to the music data. By the method, the music accompaniment data can be presented more carefully through the music parameters in the process of defining the music melody and bringing the beats and the chords into the accompaniment melody generation by carrying out quantization processing on the music beats and the music notes, so that the generation stability and the generation effect of the music accompaniment are improved. The method and the device can be applied to various scenes such as music accompaniment generation and the like.

Description

Music accompaniment generation method, device, equipment, storage medium and program product

Technical field

Embodiments of the present application relate to the field of computer technology, and in particular to a method, device, equipment, storage medium and program product for generating musical accompaniment.

Background technique

Artistic creation has always been considered the exclusive domain of artists. However, in recent years, with the development of computer technology, artistic creation has gradually broken down traditional barriers. With the help of computer technology, music works can be manually produced, making music types and music content have greater scope. room for improvement.

In related technologies, if you want to generate corresponding music accompaniment based on music, you usually use a method based on deep learning to generate accompaniment, and train the model with the help of sample music and corresponding sample accompaniment, so that the model learns the method of generating accompaniment; and then The music that needs to generate accompaniment is input into the trained model to obtain the music accompaniment.

The above process relies too much on the accurate correspondence between sample music and sample accompaniment. Although this method can obtain good music accompaniment to a certain extent, it ignores the music theory information of the music itself, which can easily lead to the style of the music accompaniment being different from that of the sample accompaniment. There are large gaps in the music itself, which affects the accuracy of music accompaniment in the field of music.

Contents of the invention

Embodiments of the present application provide a method, device, equipment, storage medium and program product for generating music accompaniment, which can incorporate beats and chords into the accompaniment melody generation process within the limits of the music melody by performing quantification processing on the music beat and music notes. , so that the music accompaniment data can be presented in more detail through music parameters, improving the stability and generation effect of the music accompaniment. The technical solution is as follows.

On the one hand, a method for generating musical accompaniment is provided, and the method includes:

Obtaining music data, the music data is divided into a plurality of music beats, and the music beats include music notes;

Extract music information from the music data to obtain beat data information, chord data information and melody data information. The beat data information is used to represent the changing speed of the beats of the multiple music pieces. The chord data information is used to describe the The music beat is a chord unit extracted in units, and the melody data information is used to describe note changes between multiple music notes;

The melody data information is used as a restriction condition for the generation of the music accompaniment, and the music accompaniment data is generated through the beat data information and the chord data information, and the melody data information is used to limit the music accompaniment with the music melody of the music data. The accompaniment melody of the music accompaniment, the music accompaniment data describes the accompaniment of the music data through music parameters;

Audio data rendering is performed based on the music accompaniment data to obtain the music accompaniment corresponding to the music data.

On the other hand, a device for generating musical accompaniment is provided, and the device includes:

A data acquisition module, used to obtain music data, the music data is divided into a plurality of music beats, and the music beats include music notes;

An information extraction module is used to extract music information from the music data to obtain beat data information, chord data information and melody data information. The beat data information is used to represent the changing speed of the beats of the multiple music pieces. The chord The data information is used to describe the chord units extracted in units of the beat of the music piece, and the melody data information is used to describe note changes between multiple music notes;

A data generation module, configured to use the melody data information as a generation restriction condition for the music accompaniment, and generate music accompaniment data based on the beat data information and the chord data information, where the melody data information is used to use the music accompaniment The music melody of the data defines the accompaniment melody of the music accompaniment, and the music accompaniment data describes the accompaniment situation of the music data through music parameters;

The accompaniment generation module is used for rendering audio data based on the music accompaniment data to obtain the music accompaniment corresponding to the music data.

On the other hand, a computer device is provided. The computer device includes a processor and a memory. The memory stores at least one instruction, at least a program, a code set or an instruction set. The at least one instruction, the at least A program, the code set or the instruction set is loaded and executed by the processor to implement the method for generating musical accompaniment as described in any of the above embodiments of the present application.

On the other hand, a computer-readable storage medium is provided. The storage medium stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code The set or instruction set is loaded and executed by the processor to implement the method for generating musical accompaniment as described in any of the above embodiments of the present application.

In another aspect, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method for generating musical accompaniment described in any of the above embodiments.

The beneficial effects brought by the technical solutions provided by the embodiments of this application at least include:

Extract music information from the acquired music data to obtain beat data information, chord data information and melody data information; use the melody data information generation constraints to generate music accompaniment data through beat data information and chord data information, and then render based on the music accompaniment data to obtain Musical accompaniment. In the process of generating music accompaniment data from music data, the music beat and music notes are quantified through the music information extraction process to improve the accuracy of the analysis of the music data, making the accompaniment data generation process more targeted; and then integrating the melody data information, beat data information and chord data information. Under the limitations of the music melody, the beat and chords are included in the accompaniment melody generation process, so that the music accompaniment data can be presented in more detail through music parameters, and then more detailed information can be obtained through the music accompaniment data rendering. Accurately presents the music accompaniment of the music data, improving the stability and effect of the music accompaniment generation.

Description of the drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without exerting creative efforts.

Figure 1 is a schematic diagram of the implementation environment provided by an exemplary embodiment of the present application;

Figure 2 is a flow chart of a method for generating musical accompaniment provided by an exemplary embodiment of the present application;

Figure 3 is a flow chart of a method for generating musical accompaniment provided by another exemplary embodiment of the present application;

Figure 4 is a flow chart of a method for generating musical accompaniment provided by yet another exemplary embodiment of the present application;

Figure 5 is a flow chart of a method for generating musical accompaniment provided by yet another exemplary embodiment of the present application;

Figure 6 is a schematic diagram of the overall framework of a method for generating musical accompaniment provided by an exemplary embodiment of the present application;

Figure 7 is an input schematic diagram of the accompaniment generation method provided by an exemplary embodiment of the present application;

Figure 8 is a processing flow chart of a method for generating musical accompaniment provided by an exemplary embodiment of the present application;

Figure 9 is a schematic diagram of piano musical notation provided by an exemplary embodiment of the present application;

Figure 10 is a structural block diagram of a device for generating musical accompaniment provided by an exemplary embodiment of the present application;

Figure 11 is a structural block diagram of a server provided by an exemplary embodiment of the present application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

First, a brief introduction will be given to the terms involved in the embodiments of this application.

Symbolic domain: In music theory, the symbolic domain refers to the range of symbols or values used by computers when processing music. That is, the symbolic domain refers to the digital music format that the music format read and processed by the computer during the generation process. In digital music formats, musical elements such as notes, pitch, and volume are usually represented in digital form. For example, digital music formats are implemented as Digital Music Interface (Musical Instrument Digital Interface, MIDI) format, Music Extensible Markup Language (MusicXML) format, etc. For example, MIDI uses numbers to represent notes, pitch, volume, and other musical parameters. In MIDI files, different values represent different notes, and the velocity of the notes can be represented by numbers. The symbol field here may include things such as note value (number representing the value of the note, such as: 4 represents a quarter note, 8 represents an eighth note, etc.), pitch (using numbers or codes to represent the pitch of the note, in MIDI Each note has a corresponding pitch value), volume (a numerical representation of the volume level of the note or instrument, a larger value usually indicates a higher volume), etc. These numbers or symbols constitute the symbolic field used by computers in the music generation process. Through these symbols, computers can read, process and generate music. In this context, the definition of the semiotic domain concerns the scope and manner in which musical elements are digitally represented.

Piano cover: refers to the adaptation and performance of the original song in audio format with vocal singing and complete accompaniment in the form of piano music.

In related technologies, if you want to generate corresponding music accompaniment based on music, you usually use a method based on deep learning to generate accompaniment, and train the model with the help of sample music and corresponding sample accompaniment, so that the model learns the method of generating accompaniment; and then The music that needs to generate accompaniment is input into the trained model to obtain the music accompaniment. The above process relies too much on the accurate correspondence between sample music and sample accompaniment. Although this method can obtain good music accompaniment to a certain extent, it ignores the music theory information of the music itself, which can easily lead to the style of the music accompaniment being different from that of the sample accompaniment. There are large gaps in the music itself, which affects the accuracy of music accompaniment in the field of music.

In the embodiment of this application, a method of generating musical accompaniment is introduced, which can perform quantitative processing on the music beat and musical notes through the music information extraction process, thereby improving the accuracy of the analysis of the music data; and then integrating multiple quantified information, in Under the limitation of the music melody, beats and chords are included in the accompaniment melody generation process, so that the music accompaniment data can be presented in more detail through music parameters, improving the stability and generation effect of the music accompaniment. The method of generating music accompaniment can be applied to the fields of music production, music creation, game production, advertising production, virtual reality (Virtual Reality, VR) and augmented reality (Augmented Reality, AR). The embodiments of this application are This is not limited.

It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application, All are authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of the relevant regions. For example, the music data, music information extraction and other contents involved in this application were all obtained with full authorization.

Secondly, the implementation environment involved in the embodiments of the present application is explained. The method for generating musical accompaniment provided by the embodiments of the present application can be implemented by the terminal alone, by the server, or by the terminal and the server through data interaction. This method The application examples do not limit this. Optionally, the method of generating a music accompaniment by the terminal and the server interactively is used as an example for description.

Schematically, please refer to Figure 1. This implementation environment involves a terminal 110 and a server 120. The terminal 110 and the server 120 are connected through a communication network 130.

In some embodiments, the terminal 110 has a music data collection function, and can obtain music data through recording, manual production, etc. For example: use a song as music data.

Optionally, the terminal 110 sends the music data to the server 120 through the communication network 130, and the server 120 obtains the music data. The music data is divided into a plurality of music beats, and the music beats include music notes.

In some embodiments, the server 120 extracts music information from the music data to obtain beat data information, chord data information and melody data information.

Among them, the beat data information is used to represent the changing speed of the beats of multiple music pieces, the chord data information is used to describe the chord units extracted in units of music beats, and the melody data information is used to describe the note changes between the notes of multiple music pieces.

In some embodiments, the server 120 uses melody data information as a restriction condition for the generation of music accompaniment, and generates music accompaniment data based on beat data information and chord data information.

Among them, the melody data information is used to define the accompaniment melody of the music accompaniment with the melody of the music data, and the music accompaniment data describes the accompaniment of the music data through music parameters. The melody data information, as the melody content of the overall music data that represents the music data, is conducive to the overall limiting effect in the music accompaniment generation process. Therefore, on the basis of the music melody limiting the accompaniment melody, the beat data information represents the beat speed and chord data information. Characterize the local music note conditions to generate more accurate music accompaniment data.

In some embodiments, the server 120 performs audio data rendering based on the music accompaniment data to obtain the music accompaniment corresponding to the music data.

Among them, the music accompaniment data, as the accompaniment described by music parameters, is the music data content displayed in the form of a symbolic domain. Therefore, if you need to obtain the music accompaniment corresponding to the music data, you need to perform audio data rendering on the music accompaniment data to obtain the playback Musical accompaniment. For example, specific audio data decoding software is used to analyze the music accompaniment data, thereby obtaining the music accompaniment that represents the audio data of the music accompaniment data.

Optionally, the server 120 sends the music accompaniment data to the terminal 110 through the communication network, so that the terminal 110 performs audio data rendering on the music accompaniment data to obtain the music accompaniment; or, the server performs audio data rendering on the music accompaniment data to obtain the music accompaniment data that is convenient for the terminal. The accompaniment data file is directly played, and the accompaniment data file is sent to the terminal 110 through the communication network. The terminal 110 downloads and plays the musical accompaniment based on the accompaniment data file.

It is worth noting that the above-mentioned terminals include but are not limited to mobile terminals such as mobile phones, tablet computers, portable laptops, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, etc., and can also be implemented as desktop computers, etc.; the above-mentioned servers can be independent A physical server can also be a server cluster or distributed system composed of multiple physical servers, or a cloud server.

In some embodiments, the above-mentioned server can also be implemented as a node in the blockchain system.

Combining the above noun introduction and application scenarios, the method for generating music accompaniment provided by this application will be described, taking the method applied to the server as an example. As shown in Figure 2, the method includes the following steps 210 to 240.

Step 210: Obtain music data.

Illustratively, music data refers to data expressed in audio form and is used to contain the actual audio content of the music.

Optionally, the music data can be implemented in Waveform Audio File Format (WAV), Moving Picture Experts Group Audio Layer III (MP3), Free Lossless Audio Codec , FLAC) and other formats, which contain sound wave signals, so the music data can be used to play music content.

For example: the music data is implemented as a vocal song, such as: album music sung by a singer, or a song sung by an individual, or a song hummed by an individual, etc.; the music data can also be implemented as a radio station program, TV program, news broadcast etc.; the music data can also be implemented as concert recording or concert recording; the music data can also be implemented as telephone recording, etc.

The music data is divided into a plurality of music beats, and the music beats include music notes.

Schematically, the beat of a piece of music is a basic organizational element in music. The beat of a piece of music determines the arrangement and emphasis of the notes of the piece in time. As a content that can express the sense of time in music, a piece of music usually alternates between strong beats and weak beats. composition. The strong and weak relationship between the beats of a piece of music creates the sense of rhythm of the piece of music. In different music styles and cultures, the use and emphasis of the beat of the piece of music may be different. The beat of the piece of music is an important means for creators to express musical emotions and convey musical information. one.

Schematically, the tempo of a piece of music is a time unit in the beats. Each beat has a strong beat (usually the first beat), and other beats are called weak beats. As a symbol representing the rhythm of music on the score, beats consist of two numbers. The upper number represents the number of beats in each section, and the lower number represents the duration of each beat. The duration represents the duration of the musical notes in the beat of the music. For example: 4/4 time means that there are 4 beats in each measure, and each beat is called a quarter note in time.

That is to say: the beat of music is a basic time unit in music, and the music notes included in it determine the music information within this time unit. Within a beat, musical notes of different durations can be accommodated, and the combination and arrangement of these notes form the rhythm of the music.

Step 220: Extract music information from the music data to obtain beat data information, chord data information and melody data information.

Illustratively, the music information extraction is used to perform quantization processing on the music data, so as to present the music data presented in audio form in the form of data.

Optionally, extracting music information from the music data can obtain beat data information that represents rhythm changes in the music data, chord data information that represents different chords in the music data, and melody data that represents overall melody changes in the music data. information.

That is, the beat data information is used to represent the changing speed of the beats of multiple music pieces, the chord data information is used to describe the chord units extracted in units of music beats, and the melody data information is used to describe the note changes between the notes of multiple music pieces.

Schematically, as the basic time unit in the music data, the music beat creates the basic rhythm structure of the music data and defines the ordering of the music notes on the time axis. When the beat of the music is represented by a beat (Time Signature), specify how many beats there are in each measure and the duration of each beat. When obtaining the beat data information, multiple music beats corresponding to the music data may be first determined, and then the beat data information may be determined based on the distribution changes of the multiple music beats.

Schematically, a chord is a collection of at least two notes played simultaneously, arranged in a fixed interval relationship. Chords are the basis of harmony in music and are important for creating and emphasizing harmony. Chords can contain musical notes such as fundamentals, thirds, and fifths, creating different harmonic textures. That is to say: chords focus on the combination of sounds that occur at the same time, and their emphasis is on expressing pitch relationships in the vertical direction. When obtaining the chord data information, the music notes in each music beat can be determined based on the music beat as a unit, and then the music audio is grouped into chords (or chord units).

Schematically, a melody is a series of organically connected notes, played in time sequence, forming a musical expression with melodic clues. Melody is a combination of pitch and duration. It is the easiest part to recognize and remember in music. It usually consists of main melody and sub-melody. That is to say: the melody focuses on the combination of pitch and duration in the horizontal direction. When acquiring the melody data information, focus on the pitch changes corresponding to the multiple music notes and the corresponding durations of the multiple music notes, thereby comprehensively determining the melody data information corresponding to the music data.

In a piece of music data, the three elements of beat, chord, and melody are often intertwined to create a complex and interesting musical experience. The beat of the music provides a time frame, the chords provide a harmonic foundation, and the melody gives guidance clues to the music, thereby presenting the music data in more detail.

With the help of the music information extraction process, the beat of the music is quantified into beat data information, the chords are quantified into chord data information, and the melody is quantified into melody data information, so that the music data can be analyzed based on multiple quantified information after information extraction.

Step 230: Use the melody data information as a restriction condition for the generation of music accompaniment, and generate music accompaniment data based on the beat data information and chord data information.

Schematically, after obtaining the melody data information, the overall melody situation of the music data is represented based on the melody data information. Therefore, the melody data information is used as the generation restriction condition to generate the music accompaniment corresponding to the music data within the music melody represented by the melody data information. .

That is, the melody data information is used to define the accompaniment melody of the music accompaniment based on the music melody of the music data.

Schematically, the music melody is used to represent a series of organically connected music notes, which are played in time order to form a musical expression with melody clues. It is the easiest part to identify and remember in the music data; melody data information refers to the representation of the melody. Numerical, symbolic or computer-readable form, which can include pitch sequences, timing information, note duration, etc., usually encoded in the form of numbers or symbols, and is used for computer analysis, music generation or other digital music processing tasks Information. That is to say: there is a corresponding relationship between the melody data information and the melody of the music piece, and the melody data information is the digital form of the music melody.

In the same way, the accompaniment melody is used to represent a series of organically connected musical notes, which are played in time order to form a musical accompaniment with melodic clues. Therefore, when the melody data information is used as a restriction condition for the generation of music accompaniment, the accompaniment melody corresponding to the music accompaniment is limited by the music melody corresponding to the melody data information.

Among them, the music accompaniment data describes the accompaniment of the music data through music parameters.

Schematically, there is a corresponding relationship between the music accompaniment data and the accompaniment melody, and the music accompaniment data is the digital form of the accompaniment melody; when generated through the beat data information and the chord data information, the music accompaniment data corresponding to the accompaniment melody is generated, and the music accompaniment data is generated. The data is a digital form corresponding to the accompaniment melody, and is generated by describing the accompaniment of the music data through numerous musical parameters.

Based on the fact that the music accompaniment is not only related to the melody corresponding to the music data, but also related to the chords and beats, when generating the music accompaniment data, it will not only be expressed through the pitch, duration, timbre and other music parameters related to the melody, but also through It is related to musical parameters such as pitch and harmony related to chords, and also related to music parameters such as volume and rhythm pattern related to beats. That is to say: the music parameters used to describe the accompaniment of the music data include pitch, timbre, volume, rhythm pattern, harmony part and other parameters related to the music accompaniment.

Step 240: Perform audio data rendering based on the music accompaniment data to obtain the music accompaniment corresponding to the music data.

Optionally, after obtaining the music accompaniment data, based on the fact that the music accompaniment data is a digital form corresponding to the music accompaniment, the music accompaniment data can be data decoded to present the music accompaniment data in audio form. That is, audio data rendering is performed on the music accompaniment data to obtain the music accompaniment corresponding to the music data.

Schematically, the music accompaniment data is implemented in the form of a MIDI file, a digital representation of the music accompaniment data. When it is necessary to obtain the music accompaniment corresponding to the music accompaniment data, the MIDI file contains music information such as notes, pitches, and durations, but it itself It does not contain sound waveforms, so the MIDI file needs to be converted into accompaniment data for rendering the music accompaniment. That is to say: first perform audio conversion based on the music accompaniment data to obtain the accompaniment audio; perform audio rendering on the accompaniment audio to obtain the music accompaniment.

Optionally, the method of performing audio conversion on the music accompaniment data and obtaining the accompaniment data includes the following steps.

(1) Software tools: Use specialized music production software or MIDI editor to convert MIDI files into accompaniment data. MIDI editors are implemented as professional sequencers (Ableton Live), Fruity Loops Studio (FL Studio), logic enhancers (Logic Pro), fluid synthesizers (Fluid Synth), Timidity++, etc.

(2) Virtual instruments or sound sources: Select or load virtual instruments or sound source plug-ins in the above software. These plug-ins act as virtual instruments, capable of generating corresponding audio waveforms based on information from MIDI files.

(3) Connect MIDI files: Import MIDI files into the software, usually through drag-and-drop files or the import function. MIDI files contain music notes, pitch, duration and other information.

(4) Assign timbre: Assign appropriate timbre (instrument) to MIDI files or notes, and determine which instrument to use to simulate the notes in the MIDI file. This process is usually done within the interface of a virtual instrument or sound module plug-in.

(5) Render to audio: Use the "export" or "render" function of the preset software to render the MIDI file into an audio file, usually WAV, MP3, Audio Interchange File Format (AIFF), etc.; preset The software will generate corresponding audio waveforms based on MIDI files.

(6) Adjust effects: You can add audio effects, reverberation, equalizer, etc. in the software as needed to adjust the final effect of the audio.

It is worth noting that in the above conversion process of converting MIDI files to accompaniment audio, the sound quality and expressiveness depend on the quality of the virtual instrument or sound source used. Some professional virtual instrument libraries provide high-quality timbres and realistic audio generation; with the above process, the music information in the MIDI file can be converted into a computer-recognizable audio format (accompaniment audio), so that it can be played on a computer or other audio device Turn on the accompaniment audio for the purpose of playing music.

Optionally, the audio-converted accompaniment audio is rendered to obtain the music accompaniment. For example: open the accompaniment audio through the terminal to play the music accompaniment, etc.

It is worth noting that the above are only illustrative examples, and the embodiments of the present application are not limited thereto.

To sum up, the music information is extracted from the acquired music data to obtain the beat data information, the chord data information and the melody data information; the music accompaniment data is generated through the beat data information and the chord data information based on the melody data information generation constraints, and then based on The music accompaniment data is rendered to obtain the music accompaniment. In the process of generating music accompaniment data from music data, the music beat and music notes are quantified through the music information extraction process to improve the accuracy of the analysis of the music data, making the accompaniment data generation process more targeted; and then integrating the melody data information, beat data information and chord data information. Under the limitations of the music melody, the beat and chords are included in the accompaniment melody generation process, so that the music accompaniment data can be presented in more detail through music parameters, and then more detailed information can be obtained through the music accompaniment data rendering. Accurately presents the music accompaniment of the music data, improving the stability and effect of the music accompaniment generation.

In an optional embodiment, beat data information is determined based on the number of beats of a plurality of music pieces in unit time, and melody data information and chord data information respectively corresponding to the music piece data are obtained through the melody track and the chord track. Schematically, as shown in Figure 3, the above step 220 shown in Figure 2 can also be implemented as the following steps 310 to 330.

Step 310: Determine the beat data information corresponding to the music data based on the number of multiple music beats in unit time.

Illustratively, the unit time is a preset unit duration. For example: the unit time is 1 minute; or the unit time is half a minute; or the unit time is 5 seconds, etc.

Optionally, Beats Per Minute (BPM) is used as the data beat information corresponding to the music data; BPM is a unit used to express rhythm speed in music and is used to represent how many beats there are per minute. In the process of determining BPM, you can use manual calculation or rely on tools such as music production software. For example: when using manual calculation, if you can feel the rhythm of the music, you can use a watch or timer to determine the number of beats in one minute. If you count 120 beats in one minute, the BPM is 120; or, use the music beat The detector determines the BPM and can display the BPM of a song, etc.

Step 320: Extract music information from the music data to determine the chord track and melody track corresponding to the music data.

Among them, the chord track is used to describe a sound group constructed in the form of a chord by multiple musical notes with a temporal relationship.

Schematically, the chord track (Chord Track) is a function in music production software, used to represent the chord progression in music.

Optionally, chord tracks are often split into bars to better organize and understand the structure of the music. The number of bars in a chord track depends on the structure and arrangement of the music; in music production software, chords can be created and edited on the chord track, with each bar typically corresponding to a bar in the music, helping to maintain consistency throughout the track. Consistent chord progressions make it easy to adjust and organize your music.

Among them, the melody track is used to describe the main melody composed of multiple music notes according to the time sequence relationship.

Schematically, in music production software, melody track (Melody Track) is a concept in music production, which refers to the audio track or track containing the main melody. The main melody is the most prominent and eye-catching melody line in music, usually performed by instruments or singing (such as vocal singing). In music production software, producers can use melody tracks to record, edit, and arrange main melodies.

Among them, the melody track is a track used to accommodate the main melody, and can record the most important and most memorable melody line in the music data. Because the main melody is one of the key elements in music that attracts the audience's attention. Melody tracks allow producers to precisely shape and control the core elements of music to create works with unique style and emotion.

Optionally, chord tracks are often split into bars as well. The number of bars in a melody track depends on the structure of the music, its arrangement, and the creative style of the producer.

Step 330: Obtain melody data information and chord data information respectively corresponding to the music data based on the melody track and the chord track.

In an optional embodiment, the number of bars in the melody track and the number of bars in the chord track are respectively determined to obtain a first number corresponding to the melody track and a second number corresponding to the chord track.

Among them, the first quantity is used to represent the number of bars in the melody track, that is: the melody track includes the first number of bars; the second quantity is used to represent the number of bars in the chord track, that is: the chord track includes the third number of bars. section.

Illustratively, the melody track and the chord track are usually divided into measures respectively; after dividing the melody track into measures, a first number of measures is obtained; after dividing the chord track into measures, a second number of measures is obtained.

Optionally, a music feature extraction model is obtained in advance. The music feature extraction model is a pre-trained model and is used to analyze music data such as music beat, chord, melody and other features. The music feature extraction model is used to extract measures from the melody track and the chord track respectively, thereby obtaining the first number of measures in the melody track and the second number of measures in the chord track.

Illustratively, the first quantity and the second quantity may be equal; or, the first quantity and the second quantity may be different, such as: the first quantity is smaller than the second quantity, or the second quantity is smaller than the first quantity, etc.

In an optional embodiment, the second number of chord tracks is adjusted based on the first number, and the adjusted chord track is obtained as the chord data information.

Optionally, in response to the second number being less than the first number, a completion operation is performed on the chord track based on the first number, and the completed chord track is obtained as the chord data information. The completed chord track is the adjusted chord track mentioned above.

Schematically, if the second number is smaller than the first number, it means that the number of bars obtained after dividing the chord track into bars is smaller, and the number of bars obtained after dividing the melody track into bars is greater; based on the melody track, it can be better It represents the overall effect of the music data, so based on the first number corresponding to the melody track, the second number is complemented, thereby obtaining the completed chord track as the chord data information.

Illustratively, during the completion operation, by copying the chord in the chord track in the previous section of the section that needs to be completed, the completed chord track is obtained as the chord data information.

In some embodiments, each bar in the completed chord track is detected. In order to avoid a long pause in the accompaniment, when there is a bar without any chords, the chord of the previous bar is copied; if the bar is In the first measure, copy the chord of the first measure that is not a full rest.

Optionally, in response to the second number being equal to the first number, using the chord track as chord data information; in response to the second number being greater than the first number, performing an alignment operation on the melody track and the chord track to obtain an aligned chord corresponding to the melody track tracks as chord data information, etc.

In an optional embodiment, based on the note pitches respectively corresponding to multiple music notes in the melody track, the melody track is subjected to melody transformation processing to obtain the updated melody track as melody data information.

Schematically, the melody track describes the note characteristics corresponding to multiple music notes. Pitch is a basic and important concept in music, which determines the relative height of the music notes.

Optionally, for some songs whose melody generally has a low pitch, in order to improve the listening experience and prevent conflict with the accompaniment, the melody can be selectively processed to a higher octave.

In some embodiments, a preset bass pitch and a preset treble pitch are obtained.

Among them, the preset bass pitch and the preset treble pitch are used to comprehensively define the melody transformation processing method for the melody track.

In some embodiments, the note pitches corresponding to the plurality of music notes in the melody track are compared with the preset bass pitch, and the note pitches corresponding to the multiple music notes are compared with the preset treble pitch to determine the bass. Note proportions and treble note proportions.

Among them, the bass note ratio is used to represent the proportion of bass notes accounting for multiple music notes, and the bass note is used to represent music notes lower than the preset bass pitch; the treble note ratio is used to represent the ratio of high pitch notes accounting for multiple music notes. , treble note is used to represent music notes that are higher than the preset treble pitch.

In some embodiments, the melody track is subjected to melody transformation processing based on the bass note ratio and the treble note ratio to obtain the updated melody track as melody data information.

Schematically, after obtaining the melody track, determine the note pitches corresponding to multiple music notes; obtain the preset bass pitch and the preset treble pitch, and compare the multiple music notes with the preset bass pitch and preset treble pitch respectively. High is compared to calculate the bass note ratio and the treble note ratio respectively.

For example: Take the preset bass pitch as 56 and the preset treble pitch as 70. After determining the note pitches corresponding to multiple music notes, compare the multiple note pitches with 56 and 70 respectively. If the music notes If the pitch of the corresponding note is lower than 56, it is considered that the note of the music is a bass note; if the pitch of the note corresponding to the note of the music is higher than 70, it is considered that the note of the music is a treble note; thus, the number of the first note of the bass note and the number of treble notes can be determined The number of second notes of the notes is then combined to determine the bass note ratio by combining the number of first notes and the number of music notes of the music notes, and the ratio of treble notes is determined by combining the number of second notes and the number of music notes.

Illustratively, when the bass note ratio and the treble note ratio meet the preset conditions at the same time, the melody track will not be adjusted.

For example: when the proportion of treble notes exceeds 10% and the proportion of bass notes is less than 20%, the melody track will not be processed an octave higher; when the above preset conditions are not met, the melody track will be adjusted.

In some embodiments, in order to make the music have a sense of completeness and ending, an ending may be added to each piece of music, with a length ranging from one measure to three measures. Because the melodies of different pieces of music have different ending measures, some end with a long note, some end with a long rest, and some do not have enough time to add an ending. Therefore, you need to use the melody track to determine in advance whether to add an ending measure to the accompaniment track.

Illustratively, when the last note of the ending measure of the melody track ends after the second beat and is not a long tone, a blank ending measure is added in advance when generating the accompaniment track, and the chords copy the previous measure. chord; thus when converting the texture of the ending, if the selected texture exceeds four beats, it will be extended to two to three measures when written to MIDI.

Optionally, melody data information and chord data information are obtained based on the melody track and chord track through the above process, and the melody data information and chord data information are used to express the music data in digital form by integrating the beat data information.

It is worth noting that the above are only illustrative examples, and the embodiments of the present application are not limited thereto.

To sum up, in the process of generating music accompaniment data from music data, the music beat and music notes are quantified through the music information extraction process to improve the accuracy of the analysis of the music data and make the accompaniment data generation process more targeted. ; Then, the melody data information, beat data information and chord data information are integrated, and the beat and chords are included in the accompaniment melody generation process under the limitation of the music melody, so that the music accompaniment data can be presented in more detail through the music parameters, and then through the music accompaniment Data rendering results in music accompaniment that can more accurately present music data, improving the stability and effect of music accompaniment generation.

In an optional embodiment, the accompaniment texture corresponding to multiple music pieces in the music data is first obtained, and then the corresponding music sections are adjusted according to the accompaniment texture, so as to generate a music accompaniment by combining the multiple adjusted music sections. data. Schematically, as shown in Figure 4, the above-mentioned step 230 shown in Figure 2 can also be implemented as the following steps 410 to 430.

Step 410: In the case of music bars as units, melody data information is used as a restriction condition for the generation of music accompaniment, and accompaniment textures corresponding to multiple music bars are determined based on beat data information and chord data information.

Among them, music sections are also called sections, which include a preset number of music beats. They are a basic organizational unit for detailed analysis of music data and are used to divide and organize music segments. Introducing sections to analyze music data is convenient for Improve your understanding of music sections.

The unit of music bar is used to indicate that each music bar of the music data is analyzed separately. For each section of the music, the melody data information is used as the restriction condition for the generation of the music accompaniment, so that the accompaniment texture is determined based on the beat data information and the chord data information.

Among them, the accompaniment texture is a sound element used to set off the main melody corresponding to the melody data information. Schematically, the accompaniment texture refers to the sound layer in the music that is responsible for supporting and setting off the main melody. This layer usually includes elements such as chords, bass, drum beats, and percussion. Their combination forms the basis and background of the music. The role of accompaniment texture in music is to provide support for the main melody and add layer and richness to the music. Reasonable use of accompaniment texture can make the music more complete and rich, and enhance the listener's musical experience. Different types of music may have different ways of constructing accompaniment textures to accommodate different styles and emotional expressions.

In some embodiments, accompaniment textures respectively corresponding to multiple sections of the music piece are determined, so that the music notes in the section of the music piece are adjusted through the accompaniment texture, thereby enriching the expression form of the section of the music piece.

Optionally, when determining the accompaniment texture, the melody, beat, and chords are comprehensively adjusted to determine an accompaniment texture that is convenient for improving the musical performance of the section of the piece of music.

In an optional embodiment, the texture determination rule is obtained in units of music bars.

Among them, the texture determination rule is used to determine the accompaniment texture corresponding to the music measure through the music information extraction results.

Schematically, the music information extraction result is used to represent the result extracted through the music information extraction process. The music information extraction result includes melody data information, beat data information and chord data information.

That is to say: the obtained texture determination rule is used to synthesize the melody data information, beat data information and chord data information within the music section to determine the accompaniment texture corresponding to the music section.

In an optional embodiment, under the limitation of texture determination rules, the note change amplitude of the music accompaniment is defined through melody data information, the rhythm changes of the music accompaniment are defined through beat data information, and the rhythm changes of the music accompaniment are defined through chord data information. The degree of note coordination determines the accompaniment texture corresponding to the measures of the music.

Schematically, the melody data information is closely related to the notes of the music. Therefore, the melody data information can be used to assist in determining the pitch and changes of the accompaniment notes in the music accompaniment. That is, the melody data information can be used to limit the range of note changes in the music accompaniment.

Optionally, the melody data information is used to define the range of note changes corresponding to the musical accompaniment through the pitch range, pitch density, and the lowest pitch value.

Schematically, the beat data information is closely related to the beat of the music composed of the notes of the music. Therefore, the beat data information can be used to assist in determining the note rhythm, note change rate, etc. of the accompaniment notes in the music accompaniment.

Optionally, the beat data information is used to determine the rhythm type corresponding to the music accompaniment based on the number of music beats per unit time. The rhythm types include slow, medium, fast, etc.

Schematically, the chord data information, as information determined by the chord content composed of at least two notes, is closely related to the music notes themselves. Therefore, the chord data information can be used to assist in determining the note form of the accompaniment notes in the music accompaniment, that is: determining the music accompaniment. The notes that should be used to more precisely improve the accuracy of the musical accompaniment.

Optionally, the chord data information is used to determine the degree of note coordination through the chord position auxiliary melody data information.

In some embodiments, each section of the music is analyzed to obtain an accompaniment texture corresponding to each section of the music.

Optionally, the classification of accompaniment textures is commonly seen in the combination of different parts (such as harmony, bass, percussion, etc.) in the music, and their roles in the music. Accompaniment textures include: harmonic accompaniment (based on chords, responsible for supporting the main melody), rhythm accompaniment (based on drums and percussion, responsible for providing a sense of rhythm), bass accompaniment (based on bass instruments, providing a stable sound for the music) Bass basics), melody accompaniment (accompanying the main melody, and setting off the main melody through harmony, timbre or musical decoration), etc.

That is to say: based on the melody, chords and beats, the accompaniment texture type corresponding to each music section is comprehensively determined, so that the corresponding music section can be adjusted through the accompaniment texture.

In some embodiments, when performing texture conversion in units of bars, each song is processed in a loop from front to back. In order to adapt to songs of different speeds, three sets of texture conversion programs were designed based on BPM. If the BPM does not exceed 90, it is judged as slow, if the BPM exceeds 90 but is lower than 120, it is judged as medium, and if the BPM reaches 120, it is judged as fast. The texture of slow songs has the richest rhythm types, while in most cases the minimum value of notes in medium-tempo songs is eighth notes, and in fast songs the minimum value is quarter notes.

After the speed judgment is completed, enter the melody judgment of this section. Calculate the melody's pitch range, note density, and lowest pitch, and choose a high or low, sparse or dense accompaniment texture. In each case, there is also a certain degree of randomness in selecting the accompaniment texture from the candidates that meet the conditions. For the first measure, you need to judge whether it is a weak starting measure and whether to add a prelude. The judgment rules are: non-softly started measures or the first note starts before the second beat will be treated as normal measures; weakly started measures but the first note appears before the third beat will not be accompanied by accompaniment; weakly started measures but the first note appears before On the third beat and beyond, it is randomly selected to have no accompaniment or to have a prelude. The prelude only selects the accompaniment texture based on the first chord.

After the accompaniment texture is selected, in order to avoid errors in the total duration of the notes in this section, which will lead to accumulation of deviations, you can set a limit on the number of beats in each section: within a limited number of cycles, determine the sum of the durations of all notes in the current section. Whether it is equal to 4 beats. If it exceeds, the last note or rest will be removed. If it is insufficient, the rest will be added. In order to avoid destroying the designed accompaniment texture, there are no such restrictions on the bars that require intro and ending bars.

For the last accompaniment measure, you can first judge the melody: if there is an empty measure, directly match the ending; the rest reaches three beats and the end time of the last note is on the first beat or before, or drag out the sound, add a beat of chords, and then match the ending; Add two beats of chords to the rest and then add the ending.

It is worth noting that the above are only illustrative examples, and the embodiments of the present application are not limited thereto.

Step 420: Adjust the note durations of the music notes in the corresponding music sections through the accompaniment texture to obtain adjustment sections corresponding to multiple music sections.

Schematically, after obtaining the accompaniment texture corresponding to each music section, the music notes within the music section can be adjusted through the accompaniment texture.

Optionally, the note duration corresponding to the music note is adjusted through the accompaniment texture, and the note duration is used to represent the duration of the music note.

Illustratively, the duration of the music notes is modified within the music bar, making the music notes shorter or longer. This process can be achieved by splitting the composition notes into shorter notes or by merging multiple composition notes into one longer note. For example: decompose a quarter note into two eighth notes, or merge two eighth notes into one quarter note, etc.

In addition, when modifying the duration of music notes, you can also use legato or staccato to adjust the note duration. That is: connecting adjacent music notes through legato to make them closer in time; using staccato to create short intervals between music notes, adding some sense of blankness.

Optionally, you can also add decorative sounds to the music notes in the music bars through the accompaniment texture, such as vibrato, portamento, legato, etc., which can make the original music notes more expressive. These decorative sounds can be added without changing Add musical variations to the basic melody.

Optionally, you can also introduce new rhythmic patterns into the music bars through the accompaniment texture, such as introducing more complex drum beats, emphasis on downbeats, or cross-rhythms. This process can be done through drum beats or other percussion in the accompaniment texture. Implement changes on the instrument.

Optionally, you can also use the accompaniment texture to emphasize the downbeat or other parts of the beat in the music bar to make the music accompaniment more dynamic.

Optionally, you can also use the accompaniment texture to speed up or slow down at the end of the music measure to create a gradual change, etc.

Schematically, the above content is determined based on the accompaniment texture corresponding to the music section, and the accompaniment texture is comprehensively determined with the help of texture determination rules, melody, chords and beats.

Among them, the adjusted section includes multiple adjusted musical notes.

Schematically, the music notes in the corresponding music sections are adjusted through the accompaniment texture to obtain multiple adjusted music notes, thereby determining the adjusted music notes in each music section, and each adjusted music note is Measures are called adjustment measures.

Step 430: Generate music accompaniment data by combining multiple adjusted measures.

Schematically, the timing relationship of multiple music sections within the music data is determined; multiple adjustment sections are combined according to the timing relationship to generate music accompaniment data.

Optionally, the adjustment measures are obtained by adjusting the music notes through the accompaniment texture, so the adjustment measures focus on expressing the digital information of the music accompaniment, and the music accompaniment data generated by multiple adjustment measures also focus on expressing the music accompaniment. digital information. For example: music accompaniment data is information expressed in the form of MIDI files.

It is worth noting that the above distances are only illustrative, and are not limited by the embodiments of the present application.

To sum up, in the process of generating music accompaniment data from music data, the music beat and music notes are quantified through the music information extraction process to improve the accuracy of the analysis of the music data and make the accompaniment data generation process more targeted. ; Then, the melody data information, beat data information and chord data information are integrated, and the beat and chords are included in the accompaniment melody generation process under the limitation of the music melody, so that the music accompaniment data can be presented in more detail through the music parameters, and then through the music accompaniment Data rendering results in music accompaniment that can more accurately present music data, improving the stability and effect of music accompaniment generation.

In an optional embodiment, when rendering audio data through music accompaniment data, in order to improve the melody accuracy of the music accompaniment data, the melody data information and the music accompaniment data can be mixed, and then through the mixing As a result, the audio data rendering process is performed to obtain the music result. Schematically, as shown in Figure 4, the embodiment shown in Figure 2 can also be implemented as the following steps 510 to 550; the step 240 shown in the above Figure 2 can also be implemented as the following steps 540 to 550.

Step 510: Obtain music data.

The music data is divided into a plurality of music beats, and the music beats include music notes.

Illustratively, step 510 has been described in step 210 above, and will not be described again here.

Step 520: Extract music information from the music data to obtain beat data information, chord data information and melody data information.

Among them, the beat data information is used to represent the changing speed of the beats of multiple music pieces, the chord data information is used to describe the chord units extracted in units of music beats, and the melody data information is used to describe the note changes between the notes of multiple music pieces.

Illustratively, step 520 has been described in the above-mentioned step 220 and step 310 to step 330, and will not be described again here.

Step 530: Use the melody data information as a restriction condition for the generation of music accompaniment, and generate music accompaniment data based on the beat data information and chord data information.

Among them, the melody data information is used to define the accompaniment melody of the music accompaniment with the melody of the music data, and the music accompaniment data describes the accompaniment of the music data through music parameters.

Illustratively, step 530 has been described in the above-mentioned step 230 and step 410 to step 430, and will not be described again here.

Step 540: Perform mixing processing on the melody data information and music accompaniment data to obtain a mixing result.

Schematically, in order to more appropriately display the melody of the music piece while reflecting the accompaniment, the melody data information corresponding to the music melody is mixed with the generated music accompaniment data.

Among them, mixing refers to the process of merging different audio tracks (such as melody and accompaniment) together to create the overall music. The purpose of mixing is to combine the individual tracks into a complete musical composition while maintaining audio clarity and balance. The purpose of mixing melody data information and music accompaniment data is to create an overall sense of music, so that the music melody data corresponding to the music data itself and the generated music accompaniment data become an organic whole, so that they can complement each other after mixing. They do not suppress each other and form audio effects such as equalization, compression, and reverberation.

In an optional embodiment, the melody data information and the music accompaniment data are mixed through a sound source reverberator to generate a mixing result.

Schematically, both melody data information and music accompaniment data are implemented as MIDI files, and the melody data information and music accompaniment data involved in the mixing process are merged together to create an overall MIDI file. This can be adjusted in terms of notes, volume, timbre, etc. to create more complex and rich musical compositions.

In some embodiments, the music melody track corresponding to the melody data information and the music accompaniment track corresponding to the music accompaniment data are determined; under the condition that the audio tracks are aligned, the music melody track and the music accompaniment track are mixed to generate a mix result.

Schematically, the melody data information represents the melody track of the music, which includes the performance of the music notes based on the melody data information; the music accompaniment data represents the music accompaniment track, which includes the performance of the music notes based on the music accompaniment data. Audio track alignment is used to represent the conditions under which the melody track of the song and the accompaniment track of the song are aligned, and the melody track and the accompaniment track of the song are mixed, thereby generating a mixing result.

Optionally, the mixing process can be implemented as the following steps.

(1) Import melody data information and music accompaniment data: Import the two information that need to be mixed into MIDI editing software or Digital Audio Workstation (Digital Audio Workstation, DAW), and ensure that they are correctly aligned on the timeline.

(2) Adjust the starting point: Ensure that the melody data information and the music accompaniment data start from the same starting point on the timeline so that they can be synchronized.

(3) Note editing: Edit the music notes in the melody data information and music accompaniment data. Such as deleting or adjusting some notes to make them blend into each other better. Also consider adjusting the duration of some notes to create richer chord or melody variations.

(4) Volume balance: Adjust the volume of the melody data information and the music accompaniment data to ensure that they can maintain balance after mixing. This can be achieved by adjusting the volume controllers on the channels corresponding to the melody data information and the music accompaniment data respectively.

(5) Timbre and expression: During the mixing process, you can adjust the timbre (instrument selection) and expression controller (such as volume, note length, timbre, etc.) on the channels corresponding to the melody data information and the music accompaniment data to Make the overall music more expressive.

(6) Rhythm and time sense: Adjust the rhythm and time sense of melody data information and music accompaniment data to make them better coordinated after mixing, such as: time offset of music notes, adding some artificial micro-time value changes, etc. .

(7) Effect processing: Apply some transformation effects, such as merge effects, delay effects, chorus effects, etc., to increase the complexity and layering of the mix.

(8) Mixing and exporting: In the final stage of the mixing process, after ensuring that the mixing effect of melody data information and music accompaniment data meets the preset requirements, export the mixing result after mixing, and the mixing result is implemented in the form of a MIDI file .

Step 550: Perform audio data rendering on the mixing result to obtain the music result corresponding to the music data.

Among them, the music results include music accompaniment as the accompaniment content.

Schematically, after the mixing result is obtained, the audio data is rendered on the mixing result to obtain a music result that expresses the complete music content. The music result is generated based on the music accompaniment data, so the music result includes the music accompaniment data corresponding to the music result. Music accompaniment, which is conducive to generating music accompaniment and displaying the content of the music more realistically through melody data information.

Optionally, the mixing data is implemented in the form of a MIDI file, a digital representation of the music data. When it is necessary to obtain the music result corresponding to the mixing data, the MIDI file contains music information such as notes, pitches, and durations, but it itself It does not contain sound waveforms, so the MIDI file needs to be converted into data for rendering the music result. That is to say: first perform audio conversion based on the mixing result to obtain the music result audio; perform audio rendering on the music result audio to obtain the music result.

Optionally, the audio-converted music composition result audio is rendered to obtain the music composition result. For example: open the music result audio through the terminal, thereby playing the music result with music accompaniment, etc.

It is worth noting that the above are only illustrative examples, and the embodiments of the present application are not limited thereto.

To sum up, in the process of generating music accompaniment data from music data, the music beat and music notes are quantified through the music information extraction process to improve the accuracy of the analysis of the music data and make the accompaniment data generation process more targeted. ; Then, the melody data information, beat data information and chord data information are integrated, and the beat and chords are included in the accompaniment melody generation process under the limitation of the music melody, so that the music accompaniment data can be presented in more detail through the music parameters, and then through the music accompaniment Data rendering results in music accompaniment that can more accurately present music data, improving the stability and effect of music accompaniment generation.

In an optional embodiment, the above music data can be implemented as any popular song, and the method of generating music accompaniment is applied to the accompaniment generation process of pop music. This method can also be called "a symbol of a popular song". "Automatic generation method of domain piano flipping", as shown in Figure 6, which is an overall framework diagram of the embodiment of the present application.

Illustratively, after the original music audio file 610 (i.e., the above-mentioned music data) is obtained, a preset algorithm (such as neural network transform) is first used to perform melody extraction, beat tracking, and chord extraction on the original music audio file 610 to obtain the music. Information, including melody notes after melody extraction 621, beat information after beat tracking 622, and chord sequence 623 after chord extraction.

The extracted melody note 621 is represented by a starting point, an offset point and a pitch; the melody note 621 is then aligned with the beat information 622 detected from the original music audio file 610 and quantized to sixteenth notes to obtain a melody MIDI file (i.e. the above melody data information).

In chord extraction, one beat is used as the minimum unit, including major and minor chords, and the chord sequence 623 is obtained.

The melody and chord sequence are synthesized into a MIDI file after deleting the measures without melody, with a time signature of 4/4.

In addition, the speed information of the original music audio file 610, that is, BPM, is further estimated from the beat information 622, and the average speed of the original music audio file 610 is slightly slowed down; secondly, the range and sparseness of the accompaniment are controlled based on the melody. At the same time, the accompaniment generation algorithm generates a texture that matches the melody based on the chord progression, type, etc., and outputs the piano accompaniment MIDI file (ie: music accompaniment data). Finally, the two-track MIDI of melody and accompaniment is rendered through a batch music production method to render the audio file of the piano rendition (that is, the music audio used to render the music accompaniment).

In some embodiments, an input example of the accompaniment generation algorithm is shown in Figure 7, which includes multiple notes 710, and the input file is in ".mid" format, that is: a MIDI file, which stores two tracks, a melody track and a chord track, including Time signature (both in 4/4 time) and key signature information. The chord preprocessing is specific pitches instead of chord markings such as "C" and "Am". The pitch range of the input chords is in the default range (starting from the small group). Only major and minor triads in position are considered, but the weaving The body includes transpositions, seventh tones, ninth tones, etc.

The accompaniment generation method is executed with the help of the accompaniment generation algorithm, as shown in Figure 8, which mainly includes the following five processes: (1) input reading 810: input MIDI and BPM reading; (2) preprocessing 820: chord track preprocessing; (3) Texture conversion 830; (4) Post-processing 840: melody and accompaniment post-processing; (5) Writing MIDI file 850.

(1)Input read 810

Schematically, the input MIDI file is read, the melody and chords are divided into two tracks and stored in two parts, and the estimated BPM of the original song is input as one of the conditions for selecting the accompaniment texture.

(2) Preprocessing 820

Calculate the number of bars in the melody track and the chord track respectively. If the number of bars in the chord track is less than the number of bars in the melody track, fill them in. The filled bars copy the chords of the previous bar. In addition, each measure is detected. In order to avoid a long pause in the accompaniment, when there is a measure without any chords, the algorithm will copy the chord of the previous measure; if the measure is the first measure, the algorithm will copy the first chord that does not follow. A chord that is a full rest measure.

For some songs whose melody is generally low-pitched, in order to improve the listening experience and prevent conflict with the accompaniment, the melody can be raised an octave. The specific judgment method is to use MIDI pitches 56 and 70 as thresholds to calculate the proportion of bass notes and the proportion of treble notes respectively. When the proportion of treble notes exceeds 10% and the proportion of bass notes is less than 20%, no higher octave processing is performed.

In order to make the music complete and complete, you can add an ending to each piece of music, with the length ranging from one measure to three measures. Because the melodies of different pieces of music have different ending measures, some end with a long note, some end with a long rest, and some do not have enough time to add an ending. Therefore, you need to determine in advance whether to add an ending measure to the accompaniment track. The specific judgment rule is: when the last note of the ending measure of the melody ends after the second beat, and it is not a long tone, a blank ending measure is added in advance, and the chords copy the chords of the previous section. When texturing the ending, if the selected texture exceeds four beats, it will be expanded to two or three bars when written to MIDI.

(3) Texture conversion 830

The texture conversion is performed in units of measures, and each song is processed from the beginning to the end in a loop. In order to adapt to songs of different speeds, three sets of texture conversion programs are designed according to BPM. BPM not exceeding 90 is judged as slow, BPM exceeding 90 but below 120 is judged as medium speed, and BPM reaching 120 is judged as fast. The texture of slow songs has the richest rhythm types, while the minimum note value of medium-speed songs is mostly eighth notes, and fast songs are quarter notes.

After the speed judgment is completed, enter the melody judgment of this section. Calculate the melody's pitch range, note density, and lowest pitch, and choose a high or low, sparse or dense accompaniment texture. In each case, there is also a certain degree of randomness in selecting textures from candidates that meet the conditions. For the first measure, you need to judge whether it is a weak starting measure and whether to add a prelude. The specific rules are: non-softly started measures or the first note starts before the second beat are treated as normal measures; weakly started measures but the first note appears before the third beat are not accompanied by accompaniment; weakly started measures but the first note appears before On the third beat and beyond, it is randomly selected to have no accompaniment or to have a prelude, and the prelude only selects the texture based on the first chord.

After the texture selection is completed, in order to avoid errors in the total duration of the notes in this section, which will lead to accumulation of deviations, a limit on the number of beats in each section is set: under a limited number of cycles, determine whether the sum of the durations of all notes in the current section is It is equal to 4 beats. If it exceeds, the last note or rest will be removed. If it is insufficient, the rest will be added. In order to avoid destroying the designed texture, there are no such restrictions on the bars that require intro and ending bars.

For the last accompaniment measure, the algorithm first determines the melody: if there is an empty measure, it will be directly paired with the outro; if the rest reaches three beats and the last note ends at the first beat or before, or the sound will be prolonged, a one-beat chord will be added and the outro will be added; Add two beats of chords to the rest and then add the ending.

In some embodiments, the texture types include columnar, semi-decomposed, and fully decomposed, and the rhythm patterns include quarter, 28, 16, syncopated, dotted, quintuplets, etc., and include arpeggios. and other decorative tones, with rich diversity. When the input is only major and minor chords, the seventh, ninth, thirteenth and chord inversions are selectively added according to the range and density of the melody to expand the chord types. Taking the song speed into consideration, various specially designed textures can cover various styles of songs, including slow lyrical songs, medium-tempo songs, and fast hot songs, while maintaining a soothing overall style. In addition, the texture of the prelude is also designed for the weak attack measure where the rest reaches three beats, and the texture of the prelude is randomly selected from the two cases of the weak attack measure without accompaniment. In order to make the music have a sense of ending, a variety of endings are also designed, which are selected based on the note duration, ending position and other information of the final section of the melody.

Texture conversion is divided into two conversion methods: function and file. Function conversion is simple and stable and can be applied to all input chord types, while file conversion is more conducive to expansion. Function conversion means that the relevant texture conversion function directly selects basic notes from the chord according to the various chord durations within the bar, and designs the pitch and duration according to the harmony, combines single notes and chords, and generates a complete texture. body section and return. File conversion refers to using digital score files, that is, ".mxl" files to store the texture of each measure, in two situations: major triad and minor triad, and the chord durations are four beats, three beats, two beats, and one beat. Each type of texture is divided into eight files. Each section of the score file stores a texture, which is a C or Cm chord and its variations. When used, the key is transposed according to the interval relationship between the root note of the current chord and the C4 pitch. As shown in FIG. 9 , a plurality of music notes 910 are shown. The plurality of music notes 910 represent the Cadd9 chord and a texture file example with a duration of 4 beats.

(4) Post-processing 840

After completing the texture conversion, the algorithm first inserts the estimated BPM of the original song into the current score. In addition to the first and last bars, it then adjusts the pitch of the accompaniment to obtain a better listening experience: a single note is an octave higher than G2, and the chord is mid-low. An octave above the pitch of A2. Accompaniment notes that are higher than the melody will be processed an octave lower. Secondly, compare the sounds with close starting points in the melody and accompaniment, and adjust the discordant major and minor second sounds and the identical sounds. In addition, for the same sound repeated in the accompaniment, a tuplet (excluding chords) is added. Finally, adjust the intensity of the melody and accompaniment for post-audio production.

(5)Write MIDI file 850

After completing all the above steps, save the melody and accompaniment as two MIDI files respectively to facilitate batch rendering of audio later.

It is worth noting that the above are only illustrative examples, and the embodiments of the present application are not limited thereto.

To sum up, in the process of generating music accompaniment data from music data, the music beat and music notes are quantified through the music information extraction process to improve the accuracy of the analysis of the music data and make the accompaniment data generation process more targeted. ; Then, the melody data information, beat data information and chord data information are integrated, and the beat and chords are included in the accompaniment melody generation process under the limitation of the music melody, so that the music accompaniment data can be presented in more detail through the music parameters, and then through the music accompaniment Data rendering results in music accompaniment that can more accurately present music data, improving the stability and effect of music accompaniment generation.

Figure 10 is a structural block diagram of a device for generating musical accompaniment provided by an exemplary embodiment of the present application. As shown in Figure 10, the device includes the following parts:

The data acquisition module 1010 is used to obtain music data, the music data is divided into a plurality of music beats, and the music beats include music notes;

The information extraction module 1020 is used to extract music information from the music data to obtain beat data information, chord data information and melody data information. The beat data information is used to represent the changing speed of the beats of the multiple music pieces. The chord data information is used to describe the chord units extracted in units of the beat of the music piece, and the melody data information is used to describe the note changes between multiple music notes;

The data generation module 1030 is configured to use the melody data information as a generation restriction condition for the music accompaniment, and generate music accompaniment data based on the beat data information and the chord data information, and the melody data information is used to use the melody data information as the generation restriction condition for the music accompaniment. The music melody of the music data defines the accompaniment melody of the music accompaniment, and the music accompaniment data describes the accompaniment of the music data through music parameters;

The accompaniment generation module 1040 is configured to perform audio data rendering based on the music accompaniment data to obtain the music accompaniment corresponding to the music data.

In an optional embodiment, the information extraction module 1020 is further configured to determine the beat data information corresponding to the music data based on the number of the multiple music beats in unit time; for the music data Extract music information, and determine the chord track and melody track corresponding to the music data. The chord track is used to describe the sound group composed of the multiple music notes in the form of chords, and the melody track is used to describe the multiple music notes. The main melody composed of the musical notes according to the time sequence relationship; the melody data information and the chord data information respectively corresponding to the music data are obtained based on the melody track and the chord track.

In an optional embodiment, the information extraction module 1020 is further configured to determine the number of bars in the melody track and the number of bars in the chord track, respectively, to obtain the first number corresponding to the melody track, and The second number corresponding to the chord track, the first number is used to represent the number of bars in the melody track, the second number is used to represent the number of bars in the chord track; with the first number as a reference, adjust the second number of the chord track to obtain the adjusted chord track as the chord data information; based on the note pitches respectively corresponding to the plurality of music notes in the melody track, The melody track undergoes melody transformation processing to obtain an updated melody track as the melody data information.

In an optional embodiment, the information extraction module 1020 is also used to obtain a preset bass pitch and a preset treble pitch, and the preset bass pitch and the preset treble pitch are used to comprehensively define the melody. The processing method of performing melody transformation processing on the track; comparing the note pitches respectively corresponding to the plurality of music notes in the melody track with the preset bass pitch, and comparing the note pitches respectively corresponding to the plurality of music notes in the melody track. The high pitch is compared with the preset treble pitch to determine the bass note ratio and the treble note ratio; the bass note ratio is used to represent the bass note ratio of the multiple music notes, and the bass note is used to represent the lower The music notes of the preset low pitch; the treble note ratio is used to represent the ratio of treble notes to the plurality of music notes, and the treble notes are used to represent music notes that are higher than the preset treble pitch; Perform melody transformation processing on the melody track based on the bass note ratio and the treble note ratio to obtain an updated melody track as the melody data information.

In an optional embodiment, the data generation module 1030 is also configured to use the melody data information as the generation restriction condition for the music accompaniment based on the beat data information in the case of music bars as the unit. and the chord data information to determine accompaniment textures corresponding to multiple music sections; wherein, one music section includes a preset number of music beats, and the accompaniment texture is used to set off the corresponding melody data information. The sound element of the main melody; the note duration of the musical notes in the corresponding music section is adjusted through the accompaniment texture to obtain adjustment sections corresponding to multiple music sections respectively, and the note duration is used to represent the musical note. Duration, the adjustment section includes a plurality of adjusted music notes; the music accompaniment data is generated by combining multiple adjustment sections.

In an optional embodiment, the data generation module 1030 is also configured to obtain texture determination rules based on the music piece section, and the texture determination rules are used to determine based on music piece information extraction results. The accompaniment texture corresponding to the music section, the music information extraction result includes the beat data information, the chord data information and the melody data information; under the constraints of the texture determination rules, by The melody data information defines the note change amplitude of the musical accompaniment, the beat data information defines the rhythm change of the musical accompaniment, and the chord data information defines the degree of coordination of the musical accompaniment with the musical accompaniment. The accompaniment texture corresponding to the plurality of music sections respectively.

In an optional embodiment, the melody data information is used to define the change amplitude of the notes corresponding to the musical accompaniment through the pitch range, pitch density and the lowest pitch value;

The beat data information is used to determine the rhythm type corresponding to the music accompaniment based on the number of music beats per unit time, and the rhythm type includes slow, medium and fast;

The chord data information is used to assist the melody data information in determining the degree of coordination of the notes through chord positions.

In an optional embodiment, the accompaniment generation module 1040 is also configured to perform mixing processing on the melody data information and the music accompaniment data to obtain a mixing result; and perform audio data rendering on the mixing result. , obtain the music result corresponding to the music data, and the music result uses the music accompaniment as the accompaniment content.

In an optional embodiment, the accompaniment generation module 1040 is also used to determine the music melody track corresponding to the melody data information, and the music accompaniment track corresponding to the music accompaniment data; under the condition that the audio tracks are aligned, Perform mixing processing on the music melody track and the music accompaniment track to generate the mixing result.

To sum up, in the process of generating music accompaniment data from music data, the music beat and music notes are quantified through the music information extraction process to improve the accuracy of the analysis of the music data and make the accompaniment data generation process more targeted. ; Then, the melody data information, beat data information and chord data information are integrated, and the beat and chords are included in the accompaniment melody generation process under the limitation of the music melody, so that the music accompaniment data can be presented in more detail through the music parameters, and then through the music accompaniment Data rendering results in music accompaniment that can more accurately present music data, improving the stability and effect of music accompaniment generation.

It should be noted that the device for generating music accompaniment provided in the above embodiments is only illustrated by taking the division of the above functional modules. In practical applications, the above function allocation can be completed by different functional modules as needed, that is, the device The internal structure is divided into different functional modules to complete all or part of the functions described above. In addition, the musical accompaniment generating device and the musical accompaniment generating method embodiment provided in the above embodiments belong to the same concept. The specific implementation process can be found in the method embodiment and will not be described again here.

Figure 11 shows a schematic structural diagram of a server provided by an exemplary embodiment of the present application. The server 1100 includes a central processing unit (Central Processing Unit, CPU) 1101, a system memory 1104 including a random access memory (Random Access Memory, RAM) 1102 and a read only memory (Read Only Memory, ROM) 1103, and a connection system memory 1104 and the system bus 1105 of the central processing unit 1101. Server 1100 also includes a mass storage device 1106 for storing operating system 1113, applications 1114, and other program modules 1115.

Mass storage device 1106 is connected to central processing unit 1101 through a mass storage controller (not shown) connected to system bus 1105 . Mass storage device 1106 and its associated computer-readable media provide non-volatile storage for server 1100 . That is, mass storage device 1106 may include computer-readable media (not shown) such as a hard disk or a Compact Disc Read Only Memory (CD-ROM) drive.

Without a loss of generality, computer-readable media may include computer storage media and communication media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. The above-mentioned system memory 1104 and mass storage device 1106 may be collectively referred to as memory.

According to various embodiments of the present application, the server 1100 may also run on a remote computer connected to a network through a network such as the Internet. That is, the server 1100 can be connected to the network 1112 through the network interface unit 1111 connected to the system bus 1105, or the network interface unit 1111 can also be used to connect to other types of networks or remote computer systems (not shown).

The above-mentioned memory also includes one or more programs. One or more programs are stored in the memory and configured to be executed by the CPU.

Embodiments of the present application also provide a computer device. The computer device includes a processor and a memory. The memory stores at least one instruction, at least a program, a code set or an instruction set. At least one instruction, at least a program, code. The set or instruction set is loaded and executed by the processor to implement the method for generating musical accompaniment provided by the above method embodiments.

Embodiments of the present application also provide a computer-readable storage medium, which stores at least one instruction, at least a program, a code set or an instruction set, at least one instruction, at least a program, a code set or a set of instructions. The instruction set is loaded and executed by the processor to implement the method for generating musical accompaniment provided by the above method embodiments.

Embodiments of the present application also provide a computer program product or computer program. The computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method for generating musical accompaniment described in any of the above embodiments.

The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection of the present application. within the range.

Claims (13)

1. A method of generating a musical accompaniment, the method comprising:

obtaining musical composition data, wherein the musical composition data is divided into a plurality of musical composition beats, and the musical composition beats comprise musical composition notes;

extracting music information from the music data to obtain beat data information, chord data information and melody data information, wherein the beat data information is used for representing the change speed of the music beats, the chord data information is used for describing chord units extracted by taking the music beats as units, and the melody data information is used for describing note change conditions among the music notes;

Generating musical accompaniment data by taking the melody data information as a generation limiting condition of the musical accompaniment and the beat data information and the chord data information, wherein the melody data information is used for limiting the accompaniment of the musical accompaniment by the musical melody of the musical data, and the musical accompaniment data describes the accompaniment condition of the musical data through musical parameters;

and rendering the audio data based on the music accompaniment data to obtain the music accompaniment corresponding to the music data.

2. The method according to claim 1, wherein the extracting of the musical composition information from the musical composition data to obtain beat data information, chord data information, and melody data information includes:

determining the beat data information corresponding to the music data based on the number of the plurality of music beats in unit time;

extracting the musical composition information from the musical composition data, and determining a chord track and a melody track corresponding to the musical composition data, wherein the chord track is used for describing a sound group constructed by the musical composition notes in a chord form, and the melody track is used for describing a main melody constructed by the musical composition notes according to a time sequence relation;

And acquiring the melody data information and the chord data information respectively corresponding to the music data based on the melody track and the chord track.

3. The method according to claim 2, wherein the acquiring the melody data information and the chord data information respectively corresponding to the music data based on the melody track and the chord track includes:

respectively determining the number of bars in the melody track and the number of bars in the chord track to obtain a first number corresponding to the melody track and a second number corresponding to the chord track, wherein the first number is used for representing the number of bars in the melody track, and the second number is used for representing the number of bars in the chord track;

adjusting the second number of the chord tracks by taking the first number as a reference to obtain the adjusted chord tracks as the chord data information;

and performing melody transformation processing on the melody track based on the note pitches corresponding to the plurality of musical notes in the melody track, so as to obtain updated melody track as the melody data information.

4. The method according to claim 3, wherein said performing a melody transformation process on said melody track based on the note pitches corresponding to said plurality of musical notes in said melody track, to obtain an updated melody track as said melody data information, comprises:

Acquiring a preset low pitch and a preset high pitch, wherein the preset low pitch and the preset high pitch are used for comprehensively limiting a processing mode for performing melody transformation processing on the melody track;

comparing the note pitches corresponding to the musical notes in the melody track with the preset low pitch, and comparing the note pitches corresponding to the musical notes with the preset high pitch to determine the bass note proportion and the treble note proportion; the bass note ratio is used for expressing the note ratio of the bass notes to the plurality of musical notes, and the bass notes are used for expressing the musical notes lower than the preset low pitch; the high pitch note ratio is used for representing the note ratio of the high pitch notes to the plurality of musical notes, and the high pitch notes are used for representing the musical notes with higher pitch than the preset high pitch;

and performing melody transformation processing on the melody track based on the bass note proportion and the treble note proportion to obtain an updated melody track as the melody data information.

5. The method according to any one of claims 1 to 4, wherein said generating musical accompaniment data with the melody data information as a generation restriction condition for the musical accompaniment, by the beat data information and the chord data information, includes:

Determining accompaniment textures corresponding to a plurality of musical composition bars, respectively, based on the beat data information and the chord data information, with the melody data information as a generation restriction condition for the musical composition accompaniment in units of musical composition bars; wherein, a music bar comprises a preset number of music beats, and the accompaniment fabric is a sound element for setting off the main melody corresponding to the melody data information;

the method comprises the steps that note values of musical notes in corresponding musical compositions are adjusted through the accompaniment fabric, adjustment bars corresponding to the musical compositions are obtained, the note values are used for representing duration time of the musical compositions, and the adjustment bars comprise a plurality of adjusted musical compositions;

the musical accompaniment data is generated in a plurality of adjustment bar combinations.

6. The method according to claim 5, wherein the determining, in the case of the musical composition measure as a generation constraint condition of the musical composition accompaniment with the melody data information, accompaniment textures corresponding to the plurality of musical composition measures, respectively, based on the beat data information and the chord data information, includes:

Acquiring a texture determination rule for determining the accompaniment texture corresponding to the musical composition bar from a musical composition information extraction result including the beat data information, the chord data information, and the melody data information, in a case where the musical composition bar is a unit;

and under the limitation of the texture determining rule, the note variation amplitude of the musical accompaniment is limited by the melody data information, the rhythm variation condition of the musical accompaniment is limited by the beat data information, the note coordination degree of the musical accompaniment is limited by the chord data information, and the accompaniment textures respectively corresponding to the plurality of musical bars are determined.

7. The method according to any one of claim 1 to 4, wherein,

the melody data information is used for limiting the note variation amplitude corresponding to the musical accompaniment through a pitch range, a pitch density and a pitch value of the lowest tone;

the beat data information is used for determining the rhythm type corresponding to the accompaniment of the music through the number of beats of the music in unit time, and the rhythm type comprises slow speed, medium speed and fast speed;

The chord data information is used to assist the melody data information in determining the degree of note coordination by chord location.

8. The method according to any one of claims 1 to 4, further comprising:

mixing the melody data information and the accompaniment data of the music to obtain a mixing result;

and rendering the audio data of the mixing result to obtain a music result corresponding to the music data, wherein the music result takes the music accompaniment as accompaniment content.

9. The method of claim 8, wherein the mixing the melody data information and the accompaniment data to obtain a mixing result comprises:

determining a melody track corresponding to the melody data information and a melody accompaniment track corresponding to the melody accompaniment data;

and under the condition of aligning the audio tracks, carrying out audio mixing processing on the melody track of the music and the accompaniment track of the music to generate the audio mixing result.

10. A musical accompaniment generating apparatus, the apparatus comprising:

a data acquisition module for acquiring musical composition data divided into a plurality of musical composition beats including musical composition notes therein;

The information extraction module is used for extracting the music data to obtain beat data information, chord data information and melody data information, wherein the beat data information is used for representing the change speed of the music beats, the chord data information is used for describing chord units extracted by taking the music beats as units, and the melody data information is used for describing note change conditions among the music notes;

a data generation module for generating musical accompaniment data by using the melody data information as a generation constraint condition of the musical accompaniment, the melody data information being used for defining an accompaniment melody of the musical accompaniment with a musical melody of the musical data, the musical accompaniment data describing accompaniment conditions of the musical data by musical parameters;

and the accompaniment generation module is used for rendering the audio data based on the accompaniment data of the music to obtain the accompaniment of the music corresponding to the music data.

11. A computer device comprising a processor and a memory, wherein the memory has stored therein at least one program that is loaded and executed by the processor to implement the method of generating a musical accompaniment as claimed in any one of claims 1 to 9.

12. A computer-readable storage medium, wherein at least one section of a program is stored in the storage medium, the at least one section of the program being loaded and executed by a processor to realize the method of generating a musical accompaniment according to any one of claims 1 to 9.

13. A computer program product comprising a computer program or instructions which, when executed by a processor, implement a method of generating a musical accompaniment as claimed in any one of claims 1 to 9.