CN110223712A - A kind of music emotion recognition method based on two-way convolution loop sparse network - Google Patents

Abstract

The invention discloses a music emotion recognition method based on a two-way convolutional cyclic sparse network. The present invention combines the convolutional neural network and the cyclic neural network to adaptively learn the emotional salience features containing time series information from the two-dimensional time-frequency representation (i.e. frequency map) of the original audio signal. Furthermore, the present invention proposes to use a weighted mixed binary representation to reduce the computational complexity of numerical real data by converting the regression prediction problem into a weighted combination of multiple binary classification problems. The experimental results show that the emotional saliency features containing temporal information extracted by the bidirectional convolutional sparse network show better predictive performance than the optimal features in MediaEval 2015; the proposed model is comparable to the current common music emotion recognition network Compared with the optimal method, the training time is reduced and the prediction accuracy is improved. Therefore, the method of the present invention effectively solves the problems of the accuracy and efficiency of music emotion recognition, and is superior to existing recognition methods.

Description

A Music Emotion Recognition Method Based on Bidirectional Convolutional Recurrent Sparse Network

technical field

The invention belongs to the field of machine learning and emotional computing, and in particular relates to a music emotion recognition method based on a two-way convolutional loop sparse network.

Background technique

With the development of multimedia technology, the explosive growth of the number of digital music from different media has drawn more and more attention to the research on fast and effective music query and retrieval methods. Since music can convey emotion-related information and emotion-based music information retrieval has high universality and user satisfaction, music information retrieval by identifying the emotion of music audio signals has become an important research trend. The core difficulty is how to further improve the accuracy and efficiency of music emotion recognition.

The goal of music emotion recognition is to learn its perceived emotional state by extracting and analyzing musical features such as tempo, timbre, and intensity. A large number of studies on music emotion recognition based on convolutional neural network (CNN) and recurrent neural network (RNN) have shown certain advantages. CNN can adaptively learn the characteristics of high-level invariant features from raw audio data to eliminate the dependence of the feature extraction process on human subjectivity or experience, and RNN can solve the problem of timing dependence of music information. The present invention adopts a music emotion recognition method based on two-way convolutional cyclic sparse network, which combines the characteristics of CNN self-adaptive learning of advanced invariant features and the ability of RNN to learn the temporal relationship of features, and is used for motivation (Arousal) and valence (Valence ) emotion value prediction, and then improve the accuracy of music emotion recognition.

Contents of the invention

The purpose of the present invention is to improve the accuracy and efficiency of music emotion recognition, providing a music emotion recognition method based on two-way convolution cyclic sparse network, which combines CNN and RNN to learn the emotion of timing information contained in the time-frequency graph Salient features, while using a weighted mixed binary representation to convert the regression problem into multiple binary classification problems, reducing the training time of the model and improving the prediction accuracy.

In order to achieve the above object, the present invention adopts following technical scheme to realize:

A music emotion recognition method based on a two-way convolutional cyclic sparse network. This method first converts the audio signal into a time-frequency graph; secondly, the internal fusion of the convolutional neural network and the cyclic neural network is used to establish an audio timing model to learn the internal timing. The emotional saliency feature of information, referred to as SII-ASF, is combined with weighted mixed binary representation to transform the regression problem into multiple binary classification problems to reduce computational complexity; finally, the continuous emotion recognition of music is carried out.

A further improvement of the present invention is to specifically include the following steps:

1) Time-frequency diagram conversion of audio signals: including time-frequency diagram conversion of audio files and dimensionality reduction processing of time-frequency diagrams, the specific steps are as follows,

1-1) Time-frequency map conversion of audio files: each time-domain audio file is divided into non-overlapping segments of fixed duration, and for each segment, a sliding window with fixed frame length and step size is set to convert it into a time-frequency map ;

1-2) Dimension reduction processing of time-frequency diagram: PCA whitening method is adopted, and 99% data difference retention is set to reduce the dimensionality of the frequency domain dimension of time-frequency diagram;

2) Establish an audio timing model to learn emotional salience features that contain timing information: combine CNN adaptive learning features and RNN's ability to process timing data to build a bidirectional convolutional cyclic sparse network, referred to as BCRSN; through CNN local interconnection and weight sharing The way to change the connection between the input layer and the hidden layer of the model is to use multiple convolution kernels to obtain the two-way convolution cycle feature map group, referred to as BCRFMs; replace each neuron in BCRFMs by the long short-term memory network (LSTM) module To consider the long-term dependencies between BCRFMs, the long-short-term memory network is referred to as LSTM;

3) The regression problem is transformed into a binary classification problem: including the representation and sparse processing of binary values, the specific steps are as follows,

3-1) Representation of binary values: Based on the representation method of numerical real data, the weighted mixed binary representation method converts the regression problem into a weighted combination of multiple binary classification problems to reduce the computational complexity of the model;

3-2) Sparse processing: use the consistency correlation coefficient as the loss function and add a penalty term to CCC as the objective function of the model to make BCRFMs as sparse as possible, and obtain SII-ASF, where the consistency correlation coefficient is referred to as CCC;

4) Continuous emotion recognition of music: According to the results of multiple binary classifications, the emotion recognition is first performed on the audio content of a segment, and then the continuous emotion recognition is performed on multiple audio segments of the complete music file.

A further improvement of the present invention is that the specific operation of the step 1-1) is: each time-domain audio file is divided into non-overlapping segments in units of 500 ms in duration, and for each segmented audio segment, a frame length of 60 ms is used and a sliding window with a step size of 10ms to convert it into a time-frequency plot.

The further improvement of the present invention is that the specific operation of the step 1-2) is: perform PCA whitening with 99% data difference retention, reduce the dimension of the frequency domain of the time-frequency map to 45 dimensions, and obtain a 45×45 size Time-frequency plots are used as input to the BCRSN model.

The further improvement of the present invention is that the specific operation of the step 2) is: use 64 3*1 convolution kernels with a step size of 2 to perform convolution operations on the time-frequency map in the time domain to obtain BCRFMs; neurons in BCRFMs There is a two-way loop according to the time sequence of the audio frame. The input of a neuron in a certain frame is the weighted sum of its corresponding convolution result and the neuron output of the previous/next frame; at the same time, the LSTM module is used to modify each of the BCRFMs. A neuron, through the input, output and forgetting threshold of the module to memorize a certain information of any length segment, and finally use the 3×1 downsampling operation to reduce the size of the feature map and enhance the robustness of the model.

A further improvement of the present invention is that the learning of BCRFMs in step 2) comprises the following steps:

(i) The connection between the input layer of the BCRSN model and the forward and reverse convolutional loop layers uses the convolution kernel as the medium, and the forward and reverse convolutional loop layers are set with the same number of neurons as the CNN convolutional layer and The arrangement makes the model have the ability to adaptively learn invariant features, and the convolution result of each neuron is calculated by formula (1):

In the formula, C _nt,k is the convolution result of the neuron at the kth feature map position (n,t), n=1,2,...,(N-1)/2, t=1,2 ..., T; is the two-dimensional feature matrix at the corresponding position (n, t) of the input layer, and W _k is the weight parameter of the kth convolution kernel;

(ii) There is a two-way cycle between neurons in BCRFMs according to the time sequence of audio frames, and the input of a neuron in a certain frame is the weighted sum of its corresponding convolution result and the neuron output of the previous/next frame;

For the feature map of the forward convolution loop layer, the input of each neuron is expressed by formula (2):

The output is expressed as formula (3):

FO _nt,k = σ(FI _nt,k +b _nt,k ) (3)

For the feature map of the reverse convolutional recurrent layer, the input of each neuron is expressed by formula (4):

The output is expressed as equation (5):

BO _nt,k = σ(BI _nt,k +b _nt,k ) (5)

In the formula Represents the output results of all neurons in the previous frame t-1/t+1 of the kth feature map; Represents the connection matrix of neurons in the process of forward propagation and backward propagation, and the weights are shared between each audio frame; b _nt,k is the network bias;

(iii) Use the LSTM module to modify each neuron in the BCRFMs, through the input, output and forgetting threshold of the module to remember a certain information of any length of time, in the forward and reverse convolution layer and the forward and reverse The downsampling operation is performed in the frequency domain between the pooling layers, and the largest feature in the downsampling area of 3×1 size is used to represent the features of the area in turn, reducing the size of the feature map.

The further improvement of the present invention is that the specific operation of said step 3-1) is: L+ ₁ neurons are set at the output layer of the BCRSN model, and the obtained prediction sequence is represented by O; wherein, O1 predicts the positive or negative of the true value, O ₂ ～ _OL+1 predicts the absolute value of the real value, and its range is (0,1); each neuron acts as a binary classifier, thereby reducing the computational complexity of the loss function to O((L+1) ×1 ² )=O(L+1), which makes the model converge faster.

A further improvement of the present invention is that in step 3-1), a weighted mixed binary value representation method is adopted, comprising the following steps:

(i) The new weighted mixed binary representation converts the numerical real data g into a mixed binary vector O ^* to reduce the computational complexity, each bit of the vector Calculated with formula (6):

where g ₁ =g; Determined by the positive or negative value of g ₁ , when g ₁ ≥ 0, When g ₁ <0,

(ii) Set the contribution weight of the output layer neuron O _i to the model loss function to control its convergence direction and improve the prediction accuracy, which is calculated by the following formula:

where δ( ) represents the calculation formula of the loss function, and λi _{represents the contribution of O i to} the segment loss function.

A further improvement of the present invention is that the step 3-2) is specifically operated as: using CCC as a loss function and adding the Lasso penalty item of the BCRFMs weight to the CCC as the objective function of the model to make the BCRFMs as sparse as possible, and obtain SII- ASF.

A further improvement of the present invention is that in step 3-2), CCC is used as a loss function so that the network can be trained more discriminatively; specifically, each song is divided into segments of fixed duration and the real data of each segment is transformed into For mixed binary vectors O ^* , the loss function solution consists of the following steps:

(i) Calculate the CCC of the predicted sequence O and the true sequence O ^* for each fragment, the predicted sequence f _s and the target sequence of the sequence sample s The CCC between is defined as:

where S _s represents the sum square error (SSE), Q _s represent the covariance, t represents the time index of each tag value, and N _s represents the length of the sequence s; based on this, the number of bits L+1 of the mixed binary vector is used as the sequence length of each segment and the contribution weight of each bit to the model loss function is considered , rewrite formula (7) to get the CCC of each fragment prediction sequence O and real sequence O ^* :

where O ^* , O represent the real and predicted mixed binary vectors of the segment, respectively, and λ=(λ ₁ ,λ ₂ ,...,λ _L+1 ) represents the set of contribution parameters of O to the segment loss function; therefore, The CCC solution of the regression prediction problem is transformed into the weighted sum of multiple binary classification accuracy rates, namely Thus defining:

(ii) Calculate the average CCC of each song, which is calculated from the CCC and the number of segments of each segment:

In the formula, N _s represents the length of each song, i.e. the number of segments;

Use Lasso regression to set the coefficients of some neurons to 0 to delete repeated related variables and many noise features, and select SII-ASF with stronger emotional significance; specifically, in the loss function On the basis of adding the Lasso penalty item of BCRFMs weight as the final objective function:

In the formula, β _F represents the parameter set of BCRFMs, akin, α _F and α _B are hyperparameters used to control the sparsity of feature maps. The larger the value of α, the higher the sparsity; minimize L to delete noise features, select emotionally salient features, and improve prediction accuracy.

The present invention has following beneficial technical effect:

A music emotion recognition method based on a two-way convolutional cyclic sparse network provided by the present invention first converts the audio signal into a time-frequency graph, and secondly adopts the internal integration of CNN and RNN to establish an audio timing model to learn SII-ASF, and at the same time combines The weighted mixed binary representation transforms the regression problem into multiple binary classification problems to reduce computational complexity, and finally performs continuous emotion recognition of music. Compared with the current common music emotion recognition network structure and optimal methods, the BCRSN model can significantly reduce the training time and improve the prediction accuracy, and the extracted SII-ASF features are better than the optimal features proposed by the participants in MediaEval 2015. excellent predictive performance.

Description of drawings

Fig. 1 is the flow chart of BCRSN system among the present invention;

Fig. 2 is the conversion process figure from numerical real data to mixed binary vector among the present invention;

Fig. 3 is a comparison chart of prediction performance and training time between the BCRSN model and CNN-based, BLSTM-based and stacked CNN-BLSTM-based models on the DEAM and MTurk music emotion recognition data sets in the present invention.

Detailed ways

The present invention will be described in further detail below in conjunction with the accompanying drawings.

With reference to Fig. 1, a kind of music emotion recognition method based on two-way convolution cyclic sparse network provided by the present invention first converts audio signal into a time-frequency graph; secondly adopts convolutional neural network (CNN) and cyclic neural network (RNN) internal The fusion method is used to build an audio timing model to learn the emotional saliency feature (SII-ASF) containing timing information, and at the same time combine the weighted mixed binary representation to convert the regression problem into multiple binary classification problems to reduce the computational complexity; finally, the music Continuous emotion recognition, which specifically includes the following steps:

1) Time-frequency diagram conversion of audio signals: including time-frequency diagram conversion of audio files and dimensionality reduction processing of time-frequency diagrams, the specific steps are as follows,

Step1 Time-frequency map conversion of audio files: each time-domain audio file is divided into non-overlapping segments of fixed duration, and for each segment, a sliding window with fixed frame length and step size is set to convert it into a time-frequency map;

Step2 Dimensionality reduction processing of time-frequency diagram: PCA whitening method is used to set a certain degree of data difference retention to reduce the dimensionality of the frequency domain dimension of the time-frequency diagram.

2) Establish an audio timing model to learn emotional salience features that contain timing information: Combine CNN adaptive learning features and RNN's ability to process timing data to build a bidirectional convolutional cyclic sparse network (BCRSN for short). Referring to Figure 1, the input two-dimensional time-frequency image is replaced by CNN local interconnection and weight sharing in the input layer and forward and reverse convolution loop layers (Forward/Backward 1c Layer) in each frame t _i between layers, and between audio frames Set up two-way loop transfer timing information to learn BCRFMs; at the same time, use LSTM network module to replace each neuron in BCRFMs, so that there is a long-term dependency between the features in BCRFMs.

3) The regression problem is converted into a binary classification problem: including the representation and sparse processing of weighted binary values, referring to Figure 1 and Figure 2, the specific steps are as follows,

Step1 Representation of weighted binary values: Based on the method of representing numerical real data, the weighted mixed binary representation converts the regression problem into a weighted combination of multiple binary classification problems to reduce the computational complexity;

Step2 sparse processing: use CCC as the loss function and add the Lasso penalty term (L1 regularization) of the weight of BCRFMs to CCC as the objective function of the model to make BCRFMs as sparse as possible and obtain SII-ASF.

4) Continuous emotion recognition of music: Input the audio time-frequency graph into the BCRSN model, first perform emotion recognition on the audio content of a single segment according to the results of multiple binary classifications, and then perform continuous emotion recognition on multiple audio segments of the complete music file identify.

Referring to Figure 3, on the DEAM and MTurk datasets, the BCRSN model in the present invention achieves the best performance in continuous emotion prediction in both Valence and Arousal dimensions compared with CNN-based, BLSTM-based and stacked CNN-BLSTM-based models.

Referring to Table 1, compared with the optimal algorithm of MediaEval 2015, the BCRSN model in the present invention can learn effective features from the original audio signal for the prediction target with the least prior knowledge, which is better than that of MediaEval2015 Top three best performing methods (BLSTM-RNN, BLSTM-ELM and deep LSTM-RNN).

Table 1: Comparison of the BCRSN model with the top three best-performing methods (BLSTM-RNN, BLSTM-ELM and deep LSTM-RNN) in MediaEval 2015 when the original audio signal is used as input in the present invention.

Note: N.S.-Not Significant means that the performance of this method has no significant difference compared with the BCRSN model, otherwise it means that there is a significant difference.

Referring to Table 2, the SII-ASF and SII-NASF obtained by the BCRSN model in the present invention when there is a Lasso penalty item and no Lasso penalty item are compared to the feature set (JUNLP, PKUAIPL, HKPOLYU, THU- HCSIL and IRIT-SAMOVA), both showed good predictive performance.

Table 2: Performance comparison between the SII-ASF and SII-NASF features extracted in the present invention and the features proposed by the competitors in MediaEval 2015 (JUNLP, PKUAIPL, HKPOLYU, THU-HCSIL and IRIT-SAMOVA).

Note: N.S.-Not Significant means that there is no significant difference between the performance of this feature and the SII-ASF ratio, otherwise it means that there is a significant difference.

Claims (10)

1. A music emotion recognition method based on two-way convolutional cyclic sparse network, characterized in that, the method first converts audio signal into a time-frequency graph; secondly, adopts the internal fusion of convolutional neural network and cyclic neural network to establish audio sequence The model is used to learn the emotional salience feature containing temporal information, referred to as SII-ASF, and combined with the weighted mixed binary representation to convert the regression problem into multiple binary classification problems to reduce the computational complexity; finally, the continuous emotion recognition of music is carried out. 2. a kind of music emotion recognition method based on two-way convolution circular sparse network according to claim 1, is characterized in that, specifically comprises the following steps: 1) Time-frequency diagram conversion of audio signals: including time-frequency diagram conversion of audio files and dimensionality reduction processing of time-frequency diagrams, the specific steps are as follows, 1-1) Time-frequency map conversion of audio files: each time-domain audio file is divided into non-overlapping segments of fixed duration, and for each segment, a sliding window with fixed frame length and step size is set to convert it into a time-frequency map ; 1-2) Dimension reduction processing of time-frequency diagram: PCA whitening method is adopted, and 99% data difference retention is set to reduce the dimensionality of the frequency domain dimension of time-frequency diagram; 2) Establish an audio timing model to learn emotional salience features that contain timing information: combine CNN adaptive learning features and RNN's ability to process timing data to build a bidirectional convolutional cyclic sparse network, referred to as BCRSN; through CNN local interconnection and weight sharing The way to change the connection between the input layer and the hidden layer of the model is to use multiple convolution kernels to obtain the two-way convolution cycle feature map group, referred to as BCRFMs; replace each neuron in BCRFMs by the long short-term memory network (LSTM) module To consider the long-term dependencies between BCRFMs, the long-short-term memory network is referred to as LSTM; 3) The regression problem is transformed into a binary classification problem: including the representation and sparse processing of binary values, the specific steps are as follows, 3-1) Representation of binary values: Based on the representation method of numerical real data, the weighted mixed binary representation method converts the regression problem into a weighted combination of multiple binary classification problems to reduce the computational complexity of the model; 3-2) Sparse processing: use the consistency correlation coefficient as the loss function and add a penalty term to CCC as the objective function of the model to make BCRFMs as sparse as possible, and obtain SII-ASF, where the consistency correlation coefficient is referred to as CCC; 4) Continuous emotion recognition of music: According to the results of multiple binary classifications, the emotion recognition is first performed on the audio content of a segment, and then the continuous emotion recognition is performed on multiple audio segments of the complete music file. 3. a kind of music emotion recognition method based on two-way convolution circular sparse network according to claim 2, is characterized in that, described step 1-1) concrete operation is: with the unit of duration 500ms each time domain audio frequency The file is divided into non-overlapping segments, and for each segmented audio segment, it is converted into a time-frequency map using a sliding window with a frame length of 60 ms and a step size of 10 ms. 4. a kind of music emotion recognition method based on two-way convolution circular sparse network according to claim 3, is characterized in that, described step 1-2) concrete operation is: carry out PCA with 99% data difference retention degree Whitening, reducing the dimension of the frequency domain of the time-frequency image to 45 dimensions, and obtaining a time-frequency image of 45×45 size as the input of the BCRSN model. 5. A kind of music emotion recognition method based on two-way convolution circular sparse network according to claim 4, it is characterized in that, described step 2) concrete operation is: use 64 volumes of 3 * 1 and step-size to be 2 The product kernel performs a convolution operation on the time-frequency map in the time domain to obtain BCRFMs; there is a two-way cycle between neurons in BCRFMs according to the time sequence of the audio frame, and the input of a neuron in a certain frame is the corresponding convolution result and the previous /The weighted sum of the neuron output of the next frame; at the same time, use the LSTM module to modify each neuron in the BCRFMs, and use the input, output and forgetting threshold of the module to memorize a certain information of any length segment, and finally use 3×1 The size downsampling operation reduces the feature map size and enhances the robustness of the model. 6. a kind of music emotion recognition method based on two-way convolution circular sparse network according to claim 5, is characterized in that, step 2) in the learning of BCRFMs, comprises the following steps: (i) The connection between the input layer of the BCRSN model and the forward and reverse convolutional loop layers uses the convolution kernel as the medium, and the forward and reverse convolutional loop layers are set with the same number of neurons as the CNN convolutional layer and The arrangement makes the model have the ability to adaptively learn invariant features, and the convolution result of each neuron is calculated by formula (1): In the formula, C _nt,k is the convolution result of the neuron at the kth feature map position (n,t), n=1,2,...,(N-1)/2, t=1,2 ..., T; is the two-dimensional feature matrix at the corresponding position (n, t) of the input layer, and W _k is the weight parameter of the kth convolution kernel; (ii) There is a two-way cycle between neurons in BCRFMs according to the time sequence of audio frames, and the input of a neuron in a certain frame is the weighted sum of its corresponding convolution result and the neuron output of the previous/next frame; For the feature map of the forward convolution loop layer, the input of each neuron is expressed by formula (2): The output is expressed as formula (3): FO _nt,k = σ(FI _nt,k +b _nt,k ) (3) For the feature map of the reverse convolutional recurrent layer, the input of each neuron is expressed by formula (4): The output is expressed as equation (5): BO _nt,k = σ(BI _nt,k +b _nt,k ) (5) In the formula Represents the output results of all neurons in the previous frame t-1/t+1 of the kth feature map; Represents the connection matrix of neurons in the process of forward propagation and backward propagation, and the weights are shared between each audio frame; b _nt,k is the network bias; (iii) Use the LSTM module to modify each neuron in the BCRFMs, through the input, output and forgetting threshold of the module to remember a certain information of any length of time, in the forward and reverse convolution layer and the forward and reverse The downsampling operation is performed in the frequency domain between the pooling layers, and the largest feature in the downsampling area of 3×1 size is used to represent the features of the area in turn, reducing the size of the feature map. 7. a kind of music emotion recognition method based on two-way convolution circular sparse network according to claim 6, is characterized in that, described step 3-1) concrete operation is: set L+1 neurons at BCRSN model output layer unit, the obtained prediction sequence is denoted by O; among them, O ₁ predicts the positive or negative of the real value, O ₂ ～O _L+1 predicts the absolute value of the real value, and its range is (0,1); each neuron acts as A binary classifier, thereby reducing the computational complexity of the loss function to O((L+1)×1 ² )=O(L+1), making the model converge faster. 8. a kind of music emotion recognition method based on two-way convolution circular sparse network according to claim 7, is characterized in that, in step 3-1), adopts weighted mixing binary value representation method, comprises the following steps: (i) The new weighted mixed binary representation converts the numerical real data g into a mixed binary vector O ^* to reduce the computational complexity, each bit of the vector Calculated with formula (6): where g ₁ =g; Determined by the positive or negative value of g ₁ , when g ₁ ≥ 0, When g ₁ <0, (ii) Set the contribution weight of the output layer neuron O _i to the model loss function to control its convergence direction and improve the prediction accuracy, which is calculated by the following formula: where δ( ) represents the calculation formula of the loss function, and λi _{represents the contribution of O i to} the segment loss function. 9. a kind of music emotion recognition method based on two-way convolution circular sparse network according to claim 8, is characterized in that, described step 3-2) concrete operation is: use CCC as loss function and increase BCRFMs in CCC The Lasso penalty term of the weight is used as the objective function of the model to make the BCRFMs as sparse as possible and obtain SII-ASF. 10. a kind of music emotion recognition method based on two-way convolution circular sparse network according to claim 9, is characterized in that, in step 3-2) with CCC as loss function so that network obtains more differentiated training; Specifically, each song is divided into segments of fixed duration and the real data of each segment is converted into a mixed binary vector O ^* , and the loss function solution includes the following steps: (i) Calculate the CCC between the predicted sequence O and the real sequence O ^* of each segment, and the CCC between the predicted sequence f _s and the target sequence f _s ^* of the sequence sample s is defined as: where S _s represents the sum square error (SSE), Q _s represent the covariance, t represents the time index of each tag value, and N _s represents the length of the sequence s; based on this, the number of bits L+1 of the mixed binary vector is used as the sequence length of each segment and the contribution weight of each bit to the model loss function is considered , rewrite formula (7) to get the CCC of each fragment prediction sequence O and real sequence O ^* : In the formula, O ^* , O represent the real and predicted mixed binary vectors of the segment, respectively, and λ=(λ ₁ ,λ ₂ ,...,λ _L+1 ) represents the set of contribution parameters of O to the segment loss function; therefore, The CCC solution of the regression prediction problem is transformed into the weighted sum of multiple binary classification accuracy rates, namely Thus defining: (ii) Calculate the average CCC of each song, which is calculated from the CCC and the number of segments of each segment: In the formula, N _s represents the length of each song, i.e. the number of segments; Use Lasso regression to set the coefficients of some neurons to 0 to delete repeated related variables and many noise features, and select SII-ASF with stronger emotional significance; specifically, in the loss function On the basis of adding the Lasso penalty item of BCRFMs weight as the final objective function: In the formula, β _F represents the parameter set of BCRFMs, akin, α _F and α _B are hyperparameters used to control the sparsity of feature maps. The larger the value of α, the higher the sparsity; minimize L to delete noise features, select emotionally salient features, and improve prediction accuracy.