CN110069756B - A Resource or Service Recommendation Method Considering User Evaluation - Google Patents

Abstract

A resource or service recommendation method considering user evaluation belongs to the field of computer artificial intelligence, and relates to a resource or service recommendation method considering user evaluation. The method includes: a user feature encoding module, a resource or service encoding module, an Encoder module, and a Decoder module. The principle of the present invention is a resource or service recommendation method that considers user evaluation. The application of multi-level coding method not only learns the original characteristics of resources or services, but also learns the role of user evaluation on resources or services. Through the attention mechanism, the relative importance of user characteristics and resources or services is obtained. The model complexity is reduced through the residual network, and the recommendation results of resources or services can be obtained faster and more accurately.

Description

A resource or service recommendation method considering user evaluation

technical field

The invention belongs to the field of computer artificial intelligence and relates to a resource or service recommendation method considering user evaluation.

Background technique

A resource or service is a platform-independent, low-coupling, self-contained, programmable-based application that uses open standards to describe, publish, discover, coordinate, and configure these applications for the development of distributed interoperable applications.

As the number of resources or services grows, a large number of services go unnoticed due to small visits. Resource or service recommendation is to solve the problem that services with a small amount of visits cannot be recommended to the point of being inaccessible and cold start. Therefore, the traditional recommendation algorithm cannot be applied to the recommendation of resources or services.

The particularity of resources or services lies in the fact that the user’s evaluation of the service or resource is changing dynamically and cannot be calculated simply by using data. The evaluation makes the recommendation of resources or services different from the recommendation of other things; through multi-level coding, resources or services can learn from user feedback and evaluation, which improves the accuracy of recommendation. The attention mechanism enables the neural network model to learn the importance of different user features in resource or service recommendation. The intermediate vector is obtained by weighted addition, and the final resource or service recommendation output is obtained by encoding and decoding the intermediate vector.

The BiLSTM model proposed in the prior art to learn the feature relationship between user features and resources or services simply uses user features and resource or service features, and does not take into account the particularity of resources or services, that is, the role of user evaluation for resource or service recommendations, nor does it take into account the importance of different user features for recommendations. Although this method is not perfect, it provides us with the idea of using deep neural networks to recommend resources or services.

Contents of the invention

A resource or service recommendation method considering user evaluation, the method comprising:

① The present invention proposes a multi-level coding method for user characteristics and resources or services. The grid adopts two coding methods of word-embedding and position-embedding for the semantic input of user characteristics, and adds the two coding results bitwise to obtain the final input vector of user characteristics; performs three coding methods of onehot-embedding, QOE-embedding, and QOS-embedding for resource or service data, and adds the three coding results bit by bit to obtain the final resource or service input vector.

②The present invention proposes an Encoder-Decoder structure based on the Mulit-Head Self-Attention mechanism. The input vector of user characteristics is used as the input of the Encoder, the input vector of resources or services is used as the input of the Decoder, and the output of the Encoder is also used as the input of the Decoder.

③The structure of the Encoder node is an Encoder Mulit-Head Self-Attention layer and a fully connected layer, and a residual structure is used in each layer; the Encoder end is composed of the same number of Encoder nodes in series.

④The structure of the Decoder node is a Decoder Mulit-Head Self-Attention layer, an Encoder-Decoder Self-Attention layer, and a fully connected layer, and a residual structure is used in each layer; the Decoder end is composed of the same number of Decoder nodes in series.

⑤After the output vectors of the Decoder end pass Softmax, the top K with the highest probability values are used as the recommendation results of this resource or service.

The principle of the present invention is a resource or service recommendation method that considers user evaluation. The application of the multi-level coding method not only learns the original characteristics of resources or services, but also learns the role of user evaluation on resources or services. Through the attention mechanism, the relative importance of user characteristics and resources or services is obtained. The model complexity is reduced through the residual network, and the recommendation results of resources or services can be obtained faster and more accurately.

In order to achieve the above object of the invention, the present invention adopts the following technical solutions:

A resource or service recommendation method considering user evaluation, comprising: a user feature encoding module, a resource or service encoding module, an Encoder module, and a Decoder module. The Encoder and Decoder modules use the Mulit-Head Self-Attention model proposed by the present invention.

User feature encoding module, which performs multi-level encoding of user feature sentences after word segmentation, where word-embedding uses the Skip-Gram form in word2vec, position-embedding uses cos and sin functions to obtain position encoding, and adds the two encoding methods bit by bit to obtain the input vector of user features.

Resource or service encoding module, this module performs multi-level encoding of the resource or service data corresponding to the user's feature statement, in which onehot-embedding encoding sets all resource or service information used by the user to 1 and other settings to 0, QOE-embedding uses softmax to obtain the probability of the importance degree of the current resource or service vector, QOS-embedding uses softmax to obtain the probability of the importance degree of the current resource or service vector, and multiplies the three encoding methods bit by bit to obtain the input vector of the final resource or service. The Encoder module takes the user feature vector as input to obtain the intermediate hidden vector. The structure of the Encoder node is an Encoder Mulit-Head Self-Attention layer and a fully connected layer, and a residual structure is used in each layer; the Encoder end is composed of the same number of Encoder nodes in series.

The Decoder module takes the resource or service vector and the intermediate hidden vector of the Encoder as input, and obtains the final output vector to obtain the recommendation result through softmax. The structure of the Decoder node is a Decoder Mulit-HeadSelf-Attention layer, an Encoder-Decoder Self-Attention layer, and a fully connected layer. A residual structure is used in each layer; the Decoder end is composed of the same number of Decoder nodes in series.

The resource or service recommendation method considering user evaluation proposed by the present invention learns the correlation between user characteristics and resources or services through the encoding and decoding model of the Mulit-Head Self-Attention mechanism, and reduces the model complexity through the residual network to improve the accuracy of recommendation.

Description of drawings

Figure 1. An overall framework for resource or service recommendation considering user evaluation;

Figure 2 Encoder node structure based on the Mulit-Head Self-Attention mechanism;

Figure 3 is based on the Decoder node structure of the Mulit-Head Self-Attention mechanism.

Detailed ways

Features and exemplary embodiments of various aspects of the invention are described in detail below:

The present invention takes user features and resource or service feature data as input, and generates recommendation results of resources or services through a neural network model based on Mulit-Head Self-Attention, which improves accuracy, reduces model complexity, and improves parallel computing capabilities. Figure 1 shows the overall framework of the present invention, Figure 2 shows the Encoder node structure, and Figure 3 shows the Decoder node structure.

User feature encoding module (1): This module converts the sentence input by the user into an input vector at the Encoder end. The specific method is to first perform word-embedding on the input sentence at the word level, and the word-embedding model uses Google's open-source word vector library. Do position-embedding on the input sentence, the specific formula is as follows:

Where pos is the position of the word in the sentence, k is the dimension of the word, d _model is the model constant, set to 100, and the PE result is the position encoding result of the kth position of the word pos.

Add the word-embedding vector calculated for each word to the position-embedding vector to obtain the input vector X _embedding of user features.

Resource or service encoding module (2): This module converts resource or service input into an input vector on the Decoder side. Use onehot-embedding according to the resource or service data corresponding to the user, that is, when there is a resource or service, set the coding position corresponding to the resource or service to 1; otherwise, set it to 0. QOE-embedding is the quality of user experience code, and uses the form of softmax to calculate the importance weighted score of all resources or services used by the user. The specific formula is as follows:

Where m is the serial number of the mth resource or service, QOE _m is the user quality score of the mth resource or service, and the importance weighted score of the resource or service is obtained through softmax Get the weighted scores of the importance of all resources or services to get the QOE-embedding code vector.

QOS-embedding is the quality of service code, and uses the form of softmax to calculate the importance weighted score of all resources or services used by the user. The specific formula is as follows:

Where p is the serial number of the pth resource or service, QOS _p is the service quality score of the pth resource or service, and the importance weighted score of the resource or service is obtained through softmax Get the weighted scores of the importance of all resources or services to get the QOS-embedding code vector.

Multiply the obtained onehot-embedding, QOE-embedding, and QOS-embedding bitwise to obtain the input vector Y _embedding of the resource or service.

Encoder module (3): This module takes the user feature input vector as the output, and outputs the intermediate vector to the Decoder module. Figure 2 shows the specific Encoder node structure. There are num _Encoder Encoder nodes on the Encoder side, which is set to 6. The specific Encoder node includes a Mulit-Head Self-Attention layer and a fully connected layer two-layer structure, and a residual network structure is used in each layer. In the Mulit-Head Self-Attention layer, there are num _Attention attention matrices, which are set to 6. Divide the user feature input vector into 6 short vectors, input each short vector into the corresponding attention matrix to obtain the corresponding short attention vector, splice the 6 short attention vectors together and obtain the final attention vector of the node through the residual network. Pass the attention vector through a fully connected layer with a residual network to get the output vector of the node. The output vector for the previous Encoder node is input to the next Encoder node as the input vector. The output vector of the last Encoder node is input to the Decoder end as the output vector of the Encoder end.

The specific formula of the Encoder Mulit-Head Self-Attention layer is as follows:

Among them, X _embedding ⁱ is the short vector after the i-th segmentation on the Encoder side, and i is the short vector number on the Encoder side and the corresponding attention matrix number.

In formula (5), is the Key matrix in the Encoder node, /> It is the obtained key vector in the Encoder node.

In formula (6), For the Query matrix in the Encoder node, /> For the obtained query vector in the Encoder node.

In formula (7), For the Value matrix in the Encoder node, /> For the obtained Encoder node value vector.

In formula (8), In order to obtain the attention short vector in the i-th attention matrix of the Encoder node, all the attention short vectors are spliced together to obtain the final attention vector Attention _E of the node.

In formula (9), add the obtained final attention vector Attention _E to the input vector X _embedding to obtain the output vector of this layer

The specific formula of the fully connected layer is as follows:

In formula (10), is the fully connected layer matrix, b ^x is the bias in the formula, and X _out is the output vector of the node. The output vector of each Encoder node is input to the next Encoder node as an input vector, and the output vector of the last Encoder node will be input to the Decoder as an intermediate vector.

All matrices and offsets in the Encoder end are input to the network using a truncated Gaussian distribution, and the training stops when the parameters converge or the maximum number of iterations is 50. All vector lengths in the Encoder side are set to 500.

Decoder module (4): This module takes the input vector of the resource or service and the output vector of the Encoder as input, and outputs the final hidden layer vector to obtain the final recommendation result. The Decoder side consists of num _Decoder Decoder nodes, set to 6. The structure of the Decoder node is a Decoder Mulit-Head Self-Attention layer, an Encoder-DecoderSelf-Attention layer, and a fully connected layer, and a residual structure is used in each layer; in the Decoder Mulit-Head Self-Attention layer, it consists of num _Attention attention matrices, which are set to 6. Divide the resource or service input vector into 6 short vectors bit by bit, input each short vector into the corresponding attention matrix to obtain the corresponding short attention vector, splicing the 6 short attention vectors together to obtain the final attention vector of the node through the residual network. Pass the attention vector through a fully connected layer with a residual network to get the output vector of the node. The output vector for the previous Decoder node is input to the next Decoder node as the input vector. The output vector of the last Decoder node is used as the final output vector, and the final classification vector is obtained through the softmax layer, and the top K resources or services with the largest values in the classification vector are used as the recommended resources or services for the user, and K is set to 6.

The specific formula of the Decoder Mulit-Head Self-Attention layer is as follows:

Where Y _embedding ^j is the short vector after the j-th segmentation on the Decoder side, and j is the sequence number of the short vector at the Decoder side and the corresponding attention matrix sequence number.

In formula (11), is the Key matrix in the Dncoder node, /> For the obtained key vector in the Dncoder node.

In formula (12), For the Query matrix in the Dncoder node, /> For the obtained query vector in the Dncoder node.

In formula (13), For the Value matrix in the Dncoder node, /> For the obtained value vector in the Dncoder node.

In formula (14), In order to obtain the attention short vector in the jth attention matrix of the Dncoder node, all the attention short vectors are spliced together to obtain the final attention vector Attention _D of the node.

In formula (15), add the obtained final attention vector Attention _D to the input vector Y _embedding to obtain the output vector of this layer

The specific formula of the Encoder-Decoder Self-Attention layer is as follows:

In formula (16), is the layer weight matrix, b ^y is the layer bias, /> is the final attention vector output vector of the previous layer, X _out is the output vector of the Encoder node, and Y _ED is the output vector of this layer.

The specific formula of the fully connected layer is as follows:

In formula (17), is the weight matrix of this layer, Y _ED is the output vector of the previous layer, b ^y is the bias of this layer, Y _out is the output vector of this layer.

The output vector of each Decoder node is used as the input vector of the next Decoder node, and the output vector Y _outfinally of the last Decoder node is the final output vector. Perform the softmax operation on the final output vector Y _outfinally , and use the top K resources or services with the largest values in the classification vector as the recommended resources or services for the user, and set K to 6.

All matrices and offsets in the Decoder end are initialized with a truncated Gaussian distribution, and the training ends when the parameters converge or the maximum number of iterations reaches 50. All vector lengths are set to 500 on the Decoder side. The length of the final output vector is the length of the number of resource or service data, which is set to 2000.

Claims (4)

1. A resource or service recommendation method considering user evaluation comprises four modules: the system comprises a user characteristic coding module (1), a resource or service coding module (2), an Encoder end (3) and a Decoder end (4);

(1) And a user characteristic coding module:

the module carries out multi-level coding on user characteristic sentences after word segmentation, wherein word-ebedding uses Skip-Gram forms in word2vec, position-ebedding uses cos and sin functions to obtain position codes, and the two coding modes are added according to bits to obtain user characteristic vectors;

(2) Resource or service encoding module:

the module carries out multi-level coding on the resource or service data corresponding to the user characteristic statement, wherein the onehot-embedding coding sets all the resource or service information used by the user to be 1 and the other is set to be 0, QOE-embedding uses softmax to obtain the importance degree probability of the current resource or service vector, QOS-embedding uses softmax to obtain the importance degree probability of the current resource or service vector, and the three coding modes are multiplied by bits to obtain the input vector of the final resource or service;

(3) The Encoder terminal:

the module takes the user characteristic vector as input to obtain an intermediate hidden vector; the structure of the Encoder node is an Encoder Mulit-Head Self-Attention layer and a full connection layer, and each layer uses a residual structure; the end of the Encoder consists of a plurality of identical Encoder nodes in series;

(4) The Decoder end:

the module takes the resource or service vector and the intermediate hidden vector of the Encoder end as input to obtain an output vector finally, and a recommendation result is obtained through softmax; the structure of the Decoder node is a Decoder Mulit-Head Self-attribute layer, an Encode-Decoder Self-attribute layer and a full connection layer, and each layer uses a residual structure; the Decoder end is formed by a plurality of same Decoder nodes in series;

step (4), the concrete steps are as follows:

the module takes the input vector of the resource or the service and the output vector of the Encoder end as inputs, and outputs the final hidden layer vector to obtain the final recommendation result; the Decoder end is formed by num _Decoder A Decoder node set to 6; the structure of the Decoder node is a Decoder Mulit-Head Self-attribute layer, an Encode-Decoder Self-attribute layer and a full connection layer, and each layer uses a residual structure; co-existence of num in the Decoder Mulit-Head Self-Attention layer _Attention A plurality of attention moment arrays, which are set to 6; dividing the input vector of the resource or service into 6 short vectors according to the bit, inputting each short vector into the corresponding attention moment arrayThe corresponding short attention vectors are obtained, the 6 short attention vectors are spliced together, and the final attention vector of the node is obtained through a residual error network; the attention vector is passed through a full connection layer with a residual error network to obtain an output vector of the node; the output vector of the former Decoder node is used as an input vector to be input to the latter Decoder node; the output vector of the last Decoder node is used as a final output vector, a final classification vector is obtained through a softmax layer, the first K resources or services with the largest numerical value in the classification vector are used as recommended resources or services of the user, and K is set to be 6;

the specific formula of the Decoder Mulit-Head Self-attribute layer is as follows:

wherein Y is _embedding ^j The j is the serial number of the short vector of the Decoder end and the serial number of the corresponding attention moment array;

in the formula (11) of the present invention,for the Key matrix in the Dncoder node,>obtaining a key vector at a Dncoder node;

in the formula (12) of the present invention,for the Query matrix at Dncoder node,>the query vector is obtained at the Dncoder node;

in the case of the formula (13),for Value matrix at Dncoder node,/->Obtaining a value vector of the node at the Dncoder;

in the case of the formula (14),to obtain the Attention short vector at the j-th Attention matrix of the Dncoder node, all the Attention short vectors are spliced to obtain the final Attention vector Attention of the node _D ；

In equation (15), the resulting final Attention vector Attention _D And input vector Y _embedding Adding bits to obtain the output vector of the layer

The specific formula of the Encoder-Decoder Self-attribute layer is shown below:

in the formula (16) of the present invention,for the layer weight matrix, b ^y Bias for the layer->Outputting a vector, X, for the final attention vector of the upper layer _out For the output vector of the Encoder node, Y _ED An output vector for the layer;

the specific formula of the full link layer is as follows:

in the formula (17) of the present invention,for the weight matrix of the layer, Y _ED B is the output vector of the upper layer ^y For the bias of the layer, Y _out An output vector for the layer;

the output vector of each Decoder node is used as the input vector of the next Decoder node, and the output vector Y of the last Decoder node _outfinally Is the final output vector; will finally output vector Y _outfinally Performing softmax operation, and setting the first K resources or services with the largest numerical value in the classification vector as recommended resources or services of the user, wherein K is set to be 6;

initializing all matrixes and bias in a Decoder end by adopting truncated Gaussian distribution, and ending training when parameters converge or reach the maximum iteration number of 50 times; all vector lengths in the Decoder end are set to 500; the final output vector length is the length of the number of resources or service data, and is set to 2000.

2. The resource or service recommendation method considering user evaluation according to claim 1, wherein step (1) is specifically as follows:

the module converts sentences input by a user into input vectors of an Encoder terminal; firstly, word-embedding is carried out on an input sentence in the word level, and a word-embedding model adopts a word vector library of an open source of Google company; the position-embedding is carried out on the input sentence, and the specific formula is as follows:

where pos is the position of the word in the sentence, k is the dimension of the word, d _model Setting the model constant as 100, wherein the PE result is the position coding result at the kth position of the pos word;

adding word-casting vectors obtained by calculating each word and position-casting vectors according to the bit to obtain an input vector X of the user characteristic _embedding 。

3. The resource or service recommendation method considering user evaluation according to claim 1, wherein step (2) is specifically as follows:

the module converts the resource or service input into an input vector of a Decoder terminal; setting the corresponding coding position of the resource or service to be 1 when a certain resource or service exists or setting the corresponding coding position to be 0 when the resource or service exists according to the fact that whether the onehot-sounding is adopted or not by using the resource or service data corresponding to the user; QOE-unbinding encodes the quality of experience of the user, and calculates importance weighted scores in all resources or services used by the user in the form of softmax, wherein a specific formula is as follows:

where m is the sequence number of the mth resource or service, QOE _m For the user quality score of the mth resource or service, the importance weighted score of the resource or service is obtained through softmaxObtaining importance weighted scores of all resources or services to obtain QOE-ebedding coding vectors;

QOS-unbinding is a quality of service code, and the importance weighted score in all resources or services used by the user is calculated in the form of softmax, and the specific formula is as follows:

where p is the sequence number of the p-th resource or service, QOS _p For the quality of service score of the p-th resource or service, the importance weighting score of the resource or service is obtained through softmaxObtaining importance weighted scores of all resources or services to obtain QOS-ebedding coding vectors;

the obtained onehot-email and QOE-email and QOS-email are bit-multiplied to obtain an input vector Y of a resource or service _embedding 。

4. The resource or service recommendation method considering user evaluation according to claim 1, wherein step (3) is specifically as follows:

the module takes the input vector of the user characteristic as output, and outputs an intermediate vector to a Decoder terminal; having num at the end of the Encoder _Encoder The number of the Encoder nodes is 6; the specific Encoder node comprises a Mulit-Head Self-attribute layer and a full-connection layer two-layer structure, and each layer adoptsResidual error network structure; co-production of num in the Mulit-Head Self-Attention layer _Attention A plurality of attention moment arrays, which are set to 6; dividing a user characteristic input vector into 6 vectors according to bits, inputting each vector into a corresponding attention matrix to obtain a corresponding short attention vector, splicing the 6 short attention vectors together, and obtaining a final attention vector of the node through a residual error network; the attention vector is passed through a full connection layer with a residual error network to obtain an output vector of the node; the output vector of the former Encoder node is input to the latter Encoder node as an input vector; the output vector of the last Encoder node is used as the output vector of the Encoder terminal to be input to the Decode terminal;

the specific formula of the Encode Mulit-Head Self-attribute layer is as follows:

wherein X is _embedding ⁱ For the short vector after the i-th segmentation of the Encoder end, i is the short vector sequence number of the Encoder end and the corresponding attention moment array sequence number;

in the formula (5) of the present invention,for the Key matrix in the Encoder node,>a key vector at the Encoder node is obtained;

in the formula (6) of the present invention,for the Query matrix at the Encoder node,>a query vector at the Encoder node is obtained;

in the formula (7) of the present invention,for Value matrix at the Encoder node, </i >>A value vector at the Encoder node is obtained;

in the formula (8) of the present invention,to obtain an Attention short vector at the ith Attention matrix of an Encoder node, all Attention short vectors are concatenated to obtain the final Attention vector Attention of the node _E ；

In equation (9), the resulting final Attention vector Attention _E And input vector X _embedding Adding bits to obtain the output vector of the layer

The specific formula of the full link layer is as follows:

in the formula (10) of the present invention,is a full connection layer matrix, b ^x For the bias in this formula, X _out An output vector for the node; the output vector of each Encoder node is input into the next Encoder node as an input vector, and the output vector of the last Encoder node is input into a Decoder terminal as an intermediate vector;

all matrixes and offsets in the Encoder end adopt truncated Gaussian distribution input networks, and training is stopped when parameters are converged or the maximum iteration number is 50 times; all vector lengths in the Encoder end are set to 500.