CN116050427B - Information generation method, training device, electronic equipment and storage medium - Google Patents

Abstract

The present disclosure provides an information generation method, a training method, a device, an electronic device and a storage medium, and relates to the field of artificial intelligence technology, especially to the technical fields of natural language processing and deep learning. The specific implementation plan is: perform semantic understanding of the query information to obtain understanding information, where the query information includes historical dialogue information, and the understanding information includes object actions and dialogue status; in response to detecting the auxiliary request instruction, based on the query information and understanding information, obtain Auxiliary request information; generate dialogue response information based on query information, understanding information and auxiliary request information.

Description

Information generation method, training method, device, electronic device and storage medium

Technical field

The present disclosure relates to the technical field of artificial intelligence, and in particular to the technical fields of natural language processing and deep learning. Specifically, it relates to an information generation method, training method, device, electronic device and storage medium.

Background technique

With the development of artificial intelligence technology, artificial intelligence technology can be used to realize task-oriented dialogue (TaskOriented Dialogue, TOD).

Task-based dialogue can mean that the system needs to go through limited dialogue rounds and access external databases to guide the user to complete the dialogue task and achieve the purpose of the dialogue. For example, the conversation task may include at least one of the following: a query task, a recommendation task, and a reservation task. The purpose of the conversation may include at least one of the following: checking the weather, recommending attractions, making hotel reservations, etc.

Contents of the invention

The present disclosure provides an information generation method, training method, device, electronic device and storage medium.

According to one aspect of the present disclosure, an information generation method is provided, including: performing semantic understanding on query information to obtain understanding information, wherein the above query information includes historical dialogue information, and the above understanding information includes object actions and dialogue states; in response to The auxiliary request instruction is detected, and the auxiliary request information is obtained based on the above query information and the above understanding information; and the dialogue response information is generated based on the above query information, the above understanding information, and the above auxiliary request information.

According to another aspect of the present disclosure, a training method for a pre-training model is provided, including: performing semantic understanding on the first sample query information to obtain the first sample understanding information, wherein the above-mentioned first sample query information includes The first sample historical dialogue information, the above-mentioned first sample understanding information includes the first sample object action and the first sample dialogue state; according to the above-mentioned first sample query information and the above-mentioned first sample understanding information, the first sample is obtained Sample auxiliary request information; generating first sample dialogue response information based on the above-mentioned first sample query information, the above-mentioned first sample understanding information and the above-mentioned first sample auxiliary request information; and, using the above-mentioned first sample query information, the above-mentioned first sample understanding information and the above-mentioned first sample dialogue response information to train a pre-trained dialogue generation model to obtain an information generation model.

According to another aspect of the present disclosure, an information generation device is provided, including: a first semantic understanding module for performing semantic understanding on query information to obtain understanding information, wherein the above query information includes historical conversation information, and the above understanding information Including object actions and dialogue states; a first obtaining module, in response to detecting the auxiliary request instruction, obtaining auxiliary request information based on the above query information and the above understanding information; and a first generation module, used according to the above query information, The above understanding information and the above auxiliary request information generate dialogue response information.

According to another aspect of the present disclosure, a training device for a pre-training model is provided, including: a second semantic understanding module for performing semantic understanding on the first sample query information to obtain the first sample understanding information, wherein, The above-mentioned first sample query information includes the first sample historical dialogue information, the above-mentioned first sample understanding information includes the first sample object action and the first sample dialogue state; the second acquisition module is used to obtain the first sample according to the above-mentioned first sample. This query information and the above-mentioned first sample understanding information are used to obtain the first sample auxiliary request information; the second generation module is used to obtain the above-mentioned first sample query information, the above-mentioned first sample understanding information and the above-mentioned first sample. Auxiliary request information to generate first sample dialogue response information; and, a training module for training pre-training dialogue generation using the above-mentioned first sample query information, the above-mentioned first sample understanding information and the above-mentioned first sample dialogue response information. Model, get the information to generate the model.

According to another aspect of the present disclosure, an electronic device is provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor. , the above-mentioned instructions are executed by the above-mentioned at least one processor, so that the above-mentioned at least one processor can execute the method according to the present disclosure.

According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to the present disclosure.

According to another aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a method according to the present disclosure.

It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become readily understood from the following description.

Description of the drawings

The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure. in:

Figure 1 schematically illustrates an exemplary system architecture in which an information generation method, a pre-training model training method and a device can be applied according to an embodiment of the present disclosure;

Figure 2 schematically shows a flow chart of an information generation method according to an embodiment of the present disclosure;

3A schematically illustrates an example schematic diagram of an information generation process according to an embodiment of the present disclosure;

3B schematically illustrates an example schematic diagram of an information generation process according to another embodiment of the present disclosure;

FIG. 4A schematically shows an example diagram of generating dialogue response information based on first fusion information according to an embodiment of the present disclosure;

FIG. 4B schematically shows an example diagram of generating dialogue response information based on first fusion information according to another embodiment of the present disclosure;

4C schematically illustrates an example diagram of generating dialogue response information based on first fusion information according to another embodiment of the present disclosure;

4D schematically illustrates an example diagram of generating dialogue response information based on first fusion information according to another embodiment of the present disclosure;

FIG. 5A schematically shows an example diagram of semantic understanding of query information and obtaining understanding information according to an embodiment of the present disclosure;

Figure 5B schematically shows an example diagram of performing semantic understanding on query information and obtaining understanding information according to another embodiment of the present disclosure;

Figure 6 schematically shows a flow chart of a training method for a pre-trained model according to an embodiment of the present disclosure;

FIG. 7A schematically shows an example schematic diagram of a method for generating a real corpus according to an embodiment of the present disclosure;

FIG. 7B schematically shows an example schematic diagram of a method for generating a simulation corpus according to an embodiment of the present disclosure;

Figure 8 schematically shows a block diagram of an information generation device according to an embodiment of the present disclosure;

Figure 9 schematically shows a block diagram of a training device for a pre-trained model according to an embodiment of the present disclosure; and

FIG. 10 schematically shows a block diagram of an electronic device suitable for implementing the information generation method and the training method of the pre-training model according to an embodiment of the present disclosure.

Detailed ways

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the present disclosure are included to facilitate understanding and should be considered to be exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.

To this end, embodiments of the present disclosure propose an information generation solution. For example, semantic understanding of query information is performed to obtain understanding information. Query information includes historical dialogue information, and understanding information includes object actions and dialogue status. In response to detecting the assistance request instruction, the assistance request information is obtained based on the query information and the understanding information. Conversation response information is generated based on query information, understanding information and auxiliary request information.

According to embodiments of the present disclosure, since the understanding information is obtained by semantic understanding of the query information, the understanding information for dialogue understanding can be obtained. Since the auxiliary request information is obtained based on the query information and understanding information in response to detecting the auxiliary request instruction, external knowledge can be effectively utilized. On this basis, the accuracy of the dialogue response information is improved by generating dialogue response information based on the query information, understanding information and auxiliary request information.

In the technical solution of this disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are in compliance with relevant laws and regulations and do not violate public order and good customs.

In the technical solution of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

FIG. 1 schematically illustrates an exemplary system architecture in which an information generation method, a pre-training model training method and a device can be applied according to an embodiment of the present disclosure.

It should be noted that Figure 1 is only an example of a system architecture to which embodiments of the present disclosure can be applied, to help those skilled in the art understand the technical content of the present disclosure, but does not mean that the embodiments of the present disclosure cannot be used in other applications. Device, system, environment or scenario. For example, in another embodiment, the exemplary system architecture in which the information generation method, the pre-training model training method and the device can be applied may include a terminal device, but the terminal device may implement the embodiments of the present disclosure without interacting with the server. Provided information generation methods, pre-training model training methods and devices.

As shown in FIG. 1 , the system architecture 100 according to this embodiment may include a first terminal device 101 , a second terminal device 102 , a third terminal device 103 , a network 104 and a server 105 . The network 104 is used as a medium for providing communication links between the first terminal device 101 , the second terminal device 102 , the third terminal device 103 and the server 105 . Network 104 may include various connection types. For example, at least one of wired and wireless communication links, and the like. The terminal device may include at least one of the first terminal device 101, the second terminal device 102, and the third terminal device 103.

The user may use at least one of the first terminal device 101, the second terminal device 102 and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages and the like. At least one of the first terminal device 101, the second terminal device 102 and the third terminal device 103 may be installed with various communication client applications. For example, at least one of a knowledge reading application, a web browser application, a search application, an instant messaging tool, an email client, and a social platform software.

The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having a display screen and supporting web browsing. For example, the electronic device may include at least one of a smartphone, a tablet computer, a laptop computer, a desktop computer, and the like.

The server 105 may be a server that provides various services. For example, the server 105 can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the problem between traditional physical hosts and VPS services (Virtual Private Server). , which has the disadvantages of difficult management and weak business scalability.

It should be noted that the information generation method provided by the embodiment of the present disclosure can generally be executed by one of the first terminal device 101, the second terminal device 102, and the third terminal device 103. Correspondingly, the information generation apparatus provided by the embodiment of the present disclosure may also be provided in one of the first terminal device 101, the second terminal device 102, and the third terminal device 103.

Alternatively, the information generation method provided by the embodiment of the present disclosure can generally also be executed by the server 105. Correspondingly, the information generation device provided by the embodiment of the present disclosure can generally be provided in the server 105. The information generation method provided by the embodiment of the present disclosure may also be performed by a server or servers that are different from the server 105 and can communicate with at least one of the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. Cluster execution. Correspondingly, the information generation device provided by the embodiment of the present disclosure may also be provided in a location different from the server 105 and capable of communicating with at least one of the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. of servers or server clusters.

It should be noted that the training method of the pre-trained model provided by the embodiment of the present disclosure can generally also be executed by the server 105. Accordingly, the training device for the pre-training model provided by the embodiment of the present disclosure may generally be installed in the server 105 . The training method of the pre-training model provided by the embodiment of the present disclosure can also be performed by a server different from the server 105 and capable of communicating with at least one of the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. Server or server cluster execution. Correspondingly, the training device of the pre-training model provided by the embodiment of the present disclosure can also be provided in a location different from the server 105 and can communicate with at least one of the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. One of the communicating servers or server clusters.

Alternatively, the training method of the pre-training model provided by the embodiment of the present disclosure may generally be executed by one of the first terminal device 101, the second terminal device 102, and the third terminal device 103. Correspondingly, the training device for the pre-training model provided by the embodiment of the present disclosure may also be provided in one of the first terminal device 101, the second terminal device 102, and the third terminal device 103.

It should be understood that the numbers of the first terminal device, the second terminal device, the third terminal device network and the servers in Figure 1 are only illustrative. According to implementation requirements, there may be any number of first terminal devices, second terminal devices, third terminal devices, networks and servers.

It should be noted that the sequence number of each operation in the following method is only used as a representation of the operation for the purpose of description, and should not be regarded as indicating the execution order of the respective operations. Unless explicitly stated, the methods need not be performed in exactly the order shown.

FIG. 2 schematically shows a flow chart of an information generation method according to an embodiment of the present disclosure.

As shown in Figure 2, the method 200 includes operations S210 to S230.

In operation S210, the query information is semantically understood to obtain understanding information, where the query information includes historical dialogue information, and the understanding information includes object actions and dialogue states.

In operation S220, in response to detecting the assistance request instruction, assistance request information is obtained based on the query information and the understanding information.

In operation S230, dialogue response information is generated based on the query information, understanding information, and assistance request information.

According to embodiments of the present disclosure, in response to receiving a dialogue instruction from an object, dialogue response information may be generated. In the process of generating dialogue response information, query information can be obtained according to dialogue instructions. Query information can be obtained through real-time collection. For example, it can be obtained by collecting the voice information of the object. Alternatively, query information can be obtained from a data source. Data sources may include at least one of the following: local databases, cloud databases, and network resources. For example, you can call the data interface and use the data interface to obtain query information from the data source. Alternatively, the query information may be received and sent from other terminal devices. The embodiment of the present disclosure does not limit the method of obtaining query information.

According to embodiments of the present disclosure, the query information may include historical conversation information. Historical conversation information can come from the interaction process between the system and the object. The historical dialogue information may include at least one of the following: object historical dialogue information as of the current system moment and system historical dialogue information. The object historical dialogue information may refer to user historical dialogue information. System history dialogue information may refer to system history reply information.

According to embodiments of the present disclosure, after obtaining the query information, the query information can be parsed into structured, machine-readable understanding information by performing semantic understanding on the query information. Semantic understanding methods may include at least one of the following: Natural Language Understanding (NLU), Dialog State Tracking (DST), Dialogue Policy Learning (DPL), and Natural Language Generation (NLG). ).

According to embodiments of the present disclosure, the understanding information may include at least one of the following: query domain (ie, domain), query intent (ie, intent), and query word slot (ie, slot). The query domain can refer to a semantic understanding scenario. Semantic understanding scenarios can have corresponding query intentions and query word slots. Semantic understanding scenarios may include at least one of the following: chat, weather, map, radio, translation, story, alarm clock, people, news, music, and movies. Query intent can refer to the purpose that an object expresses through interactive input. Query word slots can refer to restrictions attached to objects under query intent.

According to embodiments of the present disclosure, natural language understanding can be used to understand the query intention and query behavior of the object. Natural language understanding may include at least one of the following: word segmentation processing, part-of-speech tagging (POStagging) processing, named entity recognition (Named Entity Recognition, NER) processing, syntactic parsing processing, sentiment analysis processing, keywords and Summary extraction processing and text analysis processing.

According to embodiments of the present disclosure, dialogue state tracking can be used to extract entities and attributes from understood object semantic information to track the completion of the current task, so as to determine the dialogue state according to the completion. The conversation state may refer to a data structure including historical conversation information from time zero to the current time, query field, query intention and query word slot. The method of dialogue state tracking may include at least one of the following: rule-based dialogue state tracking, generative model-based dialogue state tracking, and discriminative model-based dialogue state tracking.

According to embodiments of the present disclosure, dialogue policy learning may be used to determine an object action corresponding to the dialogue state in the knowledge base according to the dialogue state. The object action may refer to the purpose that the object corresponding to the query intention expresses through interactive input. Natural language generation can be used to generate system responses in natural language.

According to embodiments of the present disclosure, after obtaining understanding information, the type of requested instruction may be determined. For example, the type of the requested instruction can be determined based on the object label information in the dialog instruction. The type of request instructions may include one of the following: non-assisted request instructions and assisted request instructions. Non-auxiliary request instructions may refer to request instructions that do not require calling external resources in the process of generating dialogue response information. In response to detecting the non-assisted request instruction, dialogue response information can be generated directly based on the query information and understanding information.

According to an embodiment of the present disclosure, the auxiliary request instruction may refer to a request instruction that requires calling external resources in the process of generating dialogue response information. In response to detecting the auxiliary request instruction, the auxiliary request information may be obtained based on the query information and the understanding information. The auxiliary request information may include interface request parameters corresponding to the external resource. On this basis, dialogue response information can be generated based on query information, understanding information and auxiliary request information.

According to embodiments of the present disclosure, since the understanding information is obtained by semantic understanding of the query information, the understanding information for dialogue understanding can be obtained by retaining the process of the understanding stage. Since the auxiliary request information is obtained based on the query information and understanding information in response to detecting the auxiliary request instruction, external knowledge can be effectively utilized. On this basis, the accuracy of the dialogue response information is improved by generating dialogue response information based on the query information, understanding information and auxiliary request information.

The information generation method 200 according to the embodiment of the present disclosure will be further described below with reference to FIGS. 3A, 3B, 4A, 4B, 4C, 4D, 5A and 5B.

According to an embodiment of the present disclosure, the query information further includes a query word slot.

According to embodiments of the present disclosure, a slot may refer to information that needs to be completed to convert the initial object intention into a clear object instruction during multiple rounds of dialogue. A slot may include at least one slot. At least one slot each has a corresponding slot filling method. The attributes of a slot can include one of the following: word slot and interface slot. Word slots can refer to slot filling methods that obtain information through keywords in user conversations. Interface slots can refer to slot filling methods that obtain information through other means. For example, a query slot can belong to a slot.

According to embodiments of the present disclosure, slot filling can be understood as a sequence tagging (Sequence Tagging) problem. Sequence labeling can refer to the process of given an input sequence and using a first predetermined model to label each position of the input sequence with a corresponding label, that is, assigning each word in the continuous sequence a corresponding semantic category label. The first predetermined model can be configured according to actual business requirements, and it only needs to be able to realize the function of sequence labeling, which is not limited here.

For example, the first predetermined model may include at least one of the following: a first predetermined model based on a generative model and a first predetermined model based on a discriminative model. The first predetermined model based on the generative model may include at least one of the following: a Hidden Markov Model (HMM) and a Hidden Vector State (HVS) model. The first predetermined model based on the discriminant model may include at least one of the following: a conditional random field (Conditional Random Field, CRF) model, a maximum entropy Markov model (Maximum Entropy Markov Model, MEMM), and a support vector machine (Support Vector Machine, SVM) model.

According to an embodiment of the present disclosure, operation S230 may include the following operations.

Determine auxiliary response information corresponding to the auxiliary request information from the data source. Conversation response information is generated based on query information, understanding information and auxiliary response information.

According to embodiments of the present disclosure, the data source may include at least one of the following: a database and a knowledge base. Knowledge base (Knowledge Base) can refer to a structured, easy-to-operate, easy-to-use, comprehensive and organized knowledge cluster in knowledge engineering. Knowledge base is the need to solve problems in a certain or certain fields. A knowledge base may include a collection of interconnected knowledge pieces that are stored, organized, managed, and used in computer memory using one or several knowledge representation methods. The knowledge pieces in the knowledge piece collection can include theoretical knowledge and factual data related to the domain. Alternatively, the knowledge pieces in the knowledge piece collection may also include heuristic knowledge obtained from expert experience, such as relevant definitions, theorems, algorithms and common sense knowledge in a certain field.

According to an embodiment of the present disclosure, after obtaining the assistance request information, the assistance response information corresponding to the assistance request information can be determined from the data source according to the assistance request information. For example, when the data source is a knowledge base, the auxiliary response information can be used to characterize the knowledge base results corresponding to the auxiliary request information. In this case, the dialogue response information can be generated based on the knowledge base results, query information, object actions and dialogue status.

According to an embodiment of the present disclosure, the assistance response information is determined from a data source according to the assistance request information, and the data source includes at least one of a database and a knowledge base, so that external knowledge can be effectively utilized. On this basis, since the dialogue response information is generated based on the query information, understanding information and auxiliary response information, the knowledge update can be completed without retraining the model.

FIG. 3A schematically illustrates an example schematic diagram of an information generation process according to an embodiment of the present disclosure.

As shown in Figure 3A, in 300A, semantic understanding can be performed on the query information 301 to obtain understanding information 302. In response to detecting the assistance request instruction, assistance request information 303 may be obtained based on the query information 301 and the understanding information 302 .

After obtaining the assistance request information 303, the assistance response information 304 corresponding to the assistance request information 303 may be determined from the data source. After obtaining the auxiliary response information 304, the dialogue response information 305 can be generated based on the query information 301, the understanding information 302 and the auxiliary response information 304.

According to embodiments of the present disclosure, generating dialogue response information based on query information, understanding information, and auxiliary response information may include the following operations.

The query information, understanding information and auxiliary response information are fused to obtain first fused information. Conversation response information is generated according to the first fusion information.

According to embodiments of the present disclosure, after obtaining the auxiliary response information, the query information, the understanding information and the auxiliary response information can be fused to obtain the first fused information. Fusion may include at least one of the following: splicing and addition. For example, the query information, understanding information and auxiliary response information can be spliced to obtain the first fusion information. Alternatively, the query information, understanding information and auxiliary response information can be added together to obtain the first fusion information. Alternatively, the query information, understanding information and auxiliary response information can be spliced and added to obtain the first fusion information.

According to embodiments of the present disclosure, after obtaining the first fusion information, the first fusion information can be input into the second predetermined model to obtain the dialogue response information. The second predetermined model may include at least one of the following: a recurrent neural network (Recurrent Neural Networks, RNN) model, a long short-term memory network (Long Short-Term Memory, LSTM) model, and a Transformer model. The second reservation model can be configured according to actual business requirements, and it only needs to be able to realize the function of generating dialogue response information, which is not limited here.

FIG. 3B schematically illustrates an example schematic diagram of an information generation process according to another embodiment of the present disclosure.

As shown in Figure 3B, in step 300B, the query information 306 can be semantically understood to obtain understanding information 307. In response to detecting the assistance request instruction, assistance request information 308 may be obtained based on the query information 306 and the understanding information 307 .

After obtaining the assistance request information 308, the assistance response information 309 corresponding to the assistance request information 308 may be determined from the data source. After obtaining the auxiliary response information 309, the query information 306, the understanding information 307 and the auxiliary response information 309 can be fused to obtain the first fusion information 310. After obtaining the first fusion information 310, the dialogue response information 311 can be generated according to the first fusion information 310.

According to an embodiment of the present disclosure, generating dialogue response information according to the first fusion information may include the following operations.

Encode the first fusion information to obtain first encoded information. Self-decode the first encoded information to obtain intermediate decoded information. Conversation response information is generated based on the first encoded information and the intermediate decoded information.

According to embodiments of the present disclosure, after obtaining the first fusion information, the first fusion information can be input into the third predetermined model to obtain the dialogue response information. The third reservation model can be configured according to actual business requirements, and it only needs to be able to realize the function of generating dialogue response information, which is not limited here. For example, the third predetermined model may include a BoB (BERT over BERT) model. The third predetermined model may include a first encoder, a first decoder and a second decoder. Multiple rounds of training may be performed on the first encoder, the first decoder and the second decoder until a predetermined condition is met. The trained first encoder, first decoder and second decoder are determined as a third predetermined model.

According to an embodiment of the present disclosure, the first encoder may include at least one of the following: a first bidirectional long short term memory network (Bidirectional Long Short Term Memory, BiLSTM), a first gated recurrent unit (GatedRecurrent Unit, GRU), a first Convolutional Neural Networks (CNN), the first long short-term memory network and the first recurrent neural network. The first encoder may include a first input layer and a first hidden layer. The first encoder may employ a linear transformation function. The first encoder may be used to encode the first fusion information to obtain the first encoded information. For example, the first input layer of the first encoder may be used to encode the first fusion information to obtain the first intermediate vector. The first intermediate vector is processed using the first hidden layer of the first encoder to obtain the first encoded information.

According to an embodiment of the present disclosure, the first decoder may include at least one of the following: a second bidirectional long short-term memory network, a second gated recurrent unit, a second convolutional neural network, a second long short-term memory network, and a second loop Neural Networks. The first decoder may include an autoregressive decoder. The first decoder may include a second hidden layer and a first output layer. The first decoder may employ a linear transformation function. The first decoder may be used to reconstruct the first encoded information to obtain intermediate decoded information to implement a response dialogue reply. For example, the second hidden layer of the first decoder can be used to self-decode the first encoded information to obtain the first auxiliary decoding information. The first auxiliary decoding information is processed using the first output layer of the first decoder to obtain intermediate decoding information.

According to an embodiment of the present disclosure, the second decoder may include at least one of the following: a third bidirectional long short-term memory network, a third gated recurrent unit, a third convolutional neural network, a third long short-term memory network, and a third loop Neural Networks. The second decoder may include a third hidden layer and a second output layer. The second decoder can use the Unlikelihood function. The second decoder can be used to reconstruct the first encoded information and the intermediate decoded information to generate dialogue response information to achieve consistent understanding. For example, the third hidden layer of the second decoder can be used to process the first encoded information and the intermediate decoded information to obtain the first auxiliary dialogue response information. The first auxiliary dialogue response information is processed using the second output layer of the second decoder to generate dialogue response information.

FIG. 4A schematically shows an example diagram of generating dialogue response information according to the first fusion information according to an embodiment of the present disclosure.

As shown in FIG. 4A, in step 400A, after obtaining the first fusion information 401, the first fusion information 401 may be encoded to obtain the first encoded information 402. After obtaining the first encoded information 402, the first encoded information 402 can be self-decoded to obtain intermediate decoded information 403. After obtaining the intermediate decoding information 403, the conversation response information 404 can be generated according to the first encoding information 402 and the intermediate decoding information 403.

According to an embodiment of the present disclosure, generating dialogue response information according to the first fusion information may include the following operations.

The first fusion information is encoded to obtain the second encoded information. Decode the second encoded information to obtain dialogue response information.

According to embodiments of the present disclosure, after obtaining the first fusion information, the first fusion information can be input into the fourth predetermined model to obtain the dialogue response information. The fourth predetermined model can be configured according to actual business requirements, and it only needs to be able to realize the function of generating dialogue response information, which is not limited here. For example, the fourth predetermined model may include a model based on the TransfOrmer-ED (ie, Transformer's Encoder and Decoder) structure. The fourth predetermined model may include a second encoder and a third decoder. Multiple rounds of training may be performed on the second encoder and the third decoder until predetermined conditions are met. The trained second encoder and third decoder are determined as a fourth predetermined model.

According to an embodiment of the present disclosure, the second encoder may include at least one of the following: a fourth bidirectional long short-term memory network, a fourth gated recurrent unit, a fourth convolutional neural network, a fourth long short-term memory network, and a fourth loop Neural Networks. The second encoder may include a second input layer and a fourth hidden layer. The second encoder may employ a linear transformation function. The second encoder may be used to encode the first fusion information to obtain the second encoded information. For example, the second input layer of the second encoder can be used to encode the first fusion information to obtain the second intermediate vector. The second intermediate vector is processed using the fourth hidden layer of the second encoder to obtain the second encoded information.

According to an embodiment of the present disclosure, the third decoder may include at least one of the following: a fifth bidirectional long short-term memory network, a fifth gated recurrent unit, a fifth convolutional neural network, a fifth long short-term memory network, and a fifth loop Neural Networks. The third decoder may include a fifth hidden layer and a third output layer. The third decoder may employ a linear transformation function. The third decoder can be used to reconstruct the first encoded information to obtain the dialogue response information, so as to realize the response dialogue reply. For example, the fifth hidden layer of the three-decoder can be used to decode the second encoded information to obtain the second auxiliary decoding information. The second auxiliary decoding information is processed using the third output layer of the third decoder to obtain the dialogue response information.

FIG. 4B schematically illustrates an example of generating dialogue response information according to first fusion information according to another embodiment of the present disclosure.

As shown in Figure 4B, in 400B, after obtaining the first fusion information 405, the first fusion information 405 can be encoded to obtain the second encoded information 406. After obtaining the second encoded information 406, the second encoded information 406 can be decoded to obtain the conversation response information 407.

According to an embodiment of the present disclosure, generating dialogue response information according to the first fusion information may include the following operations.

According to the first fusion information, at least one first candidate dialogue response information is generated. The at least one first candidate dialogue response information and the first fusion information are respectively fused to obtain at least one second fusion information. Conversation response information is determined from at least one first candidate conversation response information based on at least one second fusion information.

According to embodiments of the present disclosure, after obtaining the first fusion information, the first fusion information can be input into the fifth predetermined model to obtain the dialogue response information. The fifth reservation model can be configured according to actual business requirements, and it only needs to be able to realize the function of generating dialogue response information, which is not limited here. For example, the fifth predetermined model may include a model based on the Transformer-Dec (ie, Transformer's Decoder) structure. The fifth predetermined model may include a first language sub-model (Dialogue Model) and a first maximum mutual information scoring function sub-model (Maximum MutualInformationscoring fuunction, MMI Model). The first language sub-model may include a third encoder and a fourth decoder. Multiple rounds of training may be performed on the first language sub-model and the first maximum mutual information scoring function sub-model until predetermined conditions are met. The trained first language sub-model and the first maximum mutual information scoring function sub-model are determined as the fifth predetermined model.

According to an embodiment of the present disclosure, the third encoder may include at least one of the following: a sixth bidirectional long short-term memory network, a sixth gated recurrent unit, a sixth convolutional neural network, a sixth long short-term memory network, and a sixth loop Neural Networks. The third encoder may include a third input layer and a sixth hidden layer. The third encoder may be used to encode the first fusion information to obtain third encoded information. For example, the third input layer of the third encoder may be used to encode the first fusion information to obtain the third intermediate vector. The third intermediate vector is processed using the sixth hidden layer of the third encoder to obtain third encoded information.

According to an embodiment of the present disclosure, the fourth decoder may include at least one of the following: a seventh bidirectional long short-term memory network, a seventh gated recurrent unit, a seventh convolutional neural network, a seventh long short-term memory network, and a seventh loop Neural Networks. The fourth decoder may include a seventh hidden layer and a fourth output layer. The fourth decoder may be used to reconstruct the third encoded information and generate at least one first candidate dialogue response information. For example, the third encoded information can be decoded using the seventh hidden layer of the quad decoder to obtain the third auxiliary decoding information. The third auxiliary decoding information is processed using a fourth output layer of the fourth decoder to generate at least one first candidate dialogue response information.

According to embodiments of the present disclosure, after obtaining at least one first candidate dialogue response information, the at least one first candidate dialogue response information and the first fusion information may be respectively fused to obtain at least one second fusion information. Fusion may include at least one of the following: splicing and addition. After obtaining the at least one second fusion information, the first maximum mutual information scoring function sub-model may be used to determine the dialogue response information from the at least one first candidate dialogue response information based on the at least one second fusion information. For example, the first maximum mutual information scoring function sub-model can be used to separately process at least one second fusion information to obtain the first maximum mutual information score corresponding to each of the at least one first candidate dialogue response information. The dialogue response information is determined from the at least one first candidate dialogue response information according to the first maximum mutual information score respectively corresponding to the at least one first candidate dialogue response information.

FIG. 4C schematically illustrates an example of generating dialogue response information according to first fusion information according to another embodiment of the present disclosure.

As shown in Figure 4C, in 400C, after obtaining the first fusion information 408, at least one first candidate dialogue response information 409 can be generated according to the first fusion information 408. At least one first candidate dialogue response information 409 may include first candidate dialogue response information 4091, first candidate dialogue response information 409_2, ..., first candidate dialogue response information 409_m, ..., first candidate dialogue response information 409_M . M can be an integer greater than or equal to 1, m∈{1, 2,..., (M-1), M}.

After obtaining at least one first candidate dialogue response information 409, the first candidate dialogue response information 409_1, the first candidate dialogue response information 409_2, ..., the first candidate dialogue response information 409_m, ..., the first candidate dialogue response information 409_1, ..., the first candidate dialogue response information 409_m, ..., the first candidate dialogue response information 409_1, . The candidate dialogue response information 409_M and the first fusion information 408 are fused to obtain at least one second fusion information. The at least one second fusion information may include second fusion information 410_1, second fusion information 410_2, ..., second fusion information 410_m, ..., second fusion information 410_M.

After obtaining at least one second fusion information, the first candidate dialogue response may be obtained according to the second fusion information 4101, the second fusion information 410_2, ..., the second fusion information 410_m, ..., the second fusion information 410_M. The dialogue response information 411 is determined among the information 409_1, the first candidate dialogue response information 409_2,..., the first candidate dialogue response information 409_m,..., the first candidate dialogue response information 409_M.

According to an embodiment of the present disclosure, generating dialogue response information according to the first fusion information may include the following operations.

The at least one first latent variable information and the first fusion information are respectively fused to obtain at least one third fusion information. Generate at least one second candidate dialogue response information based on at least one third fusion information. The dialogue response information is determined from the at least one second candidate dialogue response information based on an evaluation value corresponding to the at least one second candidate dialogue response information.

According to embodiments of the present disclosure, after obtaining the first fusion information, the first fusion information can be input into the sixth predetermined model to obtain the dialogue response information. The sixth predetermined model can be configured according to actual business requirements, and it only needs to be able to realize the function of generating dialogue response information, which is not limited here. For example, the sixth predetermined model may include a model based on a UniLM-based (ie, Unified Language Model Pre-training for Natural Language Understanding and Generation) structure. The sixth predetermined model may include a fourth encoder, a fifth decoder and a first evaluator. Multiple rounds of training may be performed on the fourth encoder and the fifth decoder until predetermined conditions are met. The trained fourth encoder and fifth decoder are determined as a sixth predetermined model.

According to embodiments of the present disclosure, the first latent variable information corresponding to at least one dialogue round may be generated based on the dialogue content (ie, Djalogue Context) and dialogue response (ie, Response) corresponding to at least one dialogue round. The dialogue content and dialogue response respectively corresponding to at least one dialogue round can reflect the first latent variable information corresponding to the dialogue round. After obtaining at least one first latent variable information, at least one first latent variable information and the first fusion information may be respectively fused to obtain at least one third fusion information. Fusion may include at least one of the following: splicing and addition.

According to an embodiment of the present disclosure, the fourth encoder may include at least one of the following: an eighth bidirectional long short-term memory network, an eighth gated recurrent unit, an eighth convolutional neural network, an eighth long short-term memory network, and an eighth loop Neural Networks. The fourth encoder may include a fourth input layer and an eighth hidden layer. The fourth encoder may be used to separately encode at least one third piece of fusion information to obtain fourth coded information respectively corresponding to the at least one third piece of fusion information. For example, the fourth input layer of the fourth encoder may be used to encode the third fusion information to obtain the fourth intermediate vector. The fourth intermediate vector is processed using an eighth hidden layer of the fourth encoder to obtain fourth coding information respectively corresponding to at least one third fusion information.

According to an embodiment of the present disclosure, the fifth decoder may include at least one of the following: a ninth bidirectional long short-term memory network, a ninth gated recurrent unit, a ninth convolutional neural network, a ninth long short-term memory network, and a ninth loop Neural Networks. The fifth decoder may include a ninth hidden layer and a fifth output layer. The fifth decoder may be used to respectively reconstruct the fourth encoded information respectively corresponding to the at least one third fusion information, and generate at least one second candidate dialogue response information. For example, the ninth hidden layer of the fifth decoder may be used to decode the fourth encoded information respectively corresponding to the at least one third fusion information, to obtain the fourth auxiliary decoding information respectively corresponding to the at least one third fusion information. The fifth output layer of the fifth decoder is used to separately process the fourth auxiliary decoding information corresponding to the at least one third fusion information to generate at least one second candidate dialogue response information.

According to an embodiment of the present disclosure, after obtaining at least one second candidate dialogue response information, the first evaluator may be used to process the at least one second candidate dialogue response information to obtain an evaluation value corresponding to the at least one second candidate dialogue response information. . The first evaluator may be trained based on the NSP (Next Sentence Prediction) task and the MLM (Mask Language Model) task. After obtaining the evaluation values corresponding to the at least one second candidate dialogue response information, the dialogue response information may be determined from the at least one second candidate dialogue response information according to the evaluation values corresponding to the at least one second candidate dialogue response information.

FIG. 4D schematically illustrates an example diagram of generating dialogue response information based on first fusion information according to another embodiment of the present disclosure.

As shown in FIG. 4D, in 400D, after obtaining the first fusion information, at least one first latent variable information 412 may be determined. At least one first latent variable information 412 may include first latent variable information 412_1, first latent variable information 412_2, ..., first latent variable information 412_n, ..., first latent variable information 412_N. N can be an integer greater than or equal to 1, n∈{1, 2,..., (N-1), N}.

After obtaining at least one first latent variable information 412, the first latent variable information 412_1, the first latent variable information 412_2, ..., the first latent variable information 412_n, ..., the first latent variable information 412_N can be respectively Fusion with the first fusion information to obtain at least one third fusion information. The at least one third fusion information may include third fusion information 413_1, third fusion information 413_2, ..., third fusion information 413_n, ..., third fusion information 413_N.

After at least one third fusion information is obtained, at least one second candidate can be generated based on the third fusion information 4131, the third fusion information 413_2, ..., the third fusion information 413_n, ..., the third fusion information 413_N Conversation response message. The at least one second candidate dialogue response information may include second candidate dialogue response information 414_1, second candidate dialogue response information 414_2, . . . , second candidate dialogue response information 414_n, . . . , second candidate dialogue response information 414_N.

After obtaining at least one second candidate dialogue response information, the second candidate dialogue response information 414_1, the second candidate dialogue response information 414_2, ..., the second candidate dialogue response information 414_n, ..., the second candidate dialogue response information 414_1, the second candidate dialogue response information 414_n, ..., the second candidate dialogue response information 414_n, ... The respective evaluation values corresponding to the dialogue response information 414_N. At least one evaluation value may include evaluation value 415_1, evaluation value 415_2, ..., evaluation value 415_n, ..., evaluation value 415_N.

After obtaining evaluation values corresponding to at least one second candidate dialogue response information, the second candidate dialogue response may be obtained according to the evaluation value 415_1, the evaluation value 415_2, ..., the evaluation value 415_n, ..., the evaluation value 415_N. The dialogue response information 416 is determined among the information 414_1, the second candidate dialogue response information 414_2, . . . , the second candidate dialogue response information 414_n, . . . , the second candidate dialogue response information 414_N.

According to an embodiment of the present disclosure, operation S210 may include the following operations.

The at least one second latent variable information and the query information are respectively fused to obtain at least one fourth fusion information. Generate at least one first candidate understanding information based on at least one fourth fusion information. Understanding information is determined from the at least one first candidate understanding information based on an evaluation value corresponding to the at least one first candidate understanding information.

According to embodiments of the present disclosure, after obtaining the query information, the query information can be input into the seventh predetermined model to obtain understanding information. The seventh predetermined model can be configured according to actual business needs, as long as it can realize the function of determining and understanding information, and is not limited here. For example, the seventh predetermined model may include a fifth encoder, a sixth decoder, and a second evaluator. Multiple rounds of training may be performed on the fifth encoder and the sixth decoder until predetermined conditions are met. The trained fifth encoder and sixth decoder are determined as a seventh predetermined model.

According to embodiments of the present disclosure, second latent variable information corresponding to at least one dialogue round may be generated based on dialogue content and dialogue responses corresponding to at least one dialogue round. The dialogue content and dialogue response respectively corresponding to at least one dialogue round can reflect the second latent variable information corresponding to the dialogue round. After obtaining at least one second latent variable information, the at least one second latent variable information and the query information can be respectively fused to obtain at least one fourth fusion information. Fusion may include at least one of the following: splicing and addition.

According to an embodiment of the present disclosure, the fifth encoder may include at least one of the following: the fifth encoder may include a fifth input layer and a tenth hidden layer. The fifth encoder may be used to separately encode at least one piece of fourth fusion information to obtain fifth coded information respectively corresponding to at least one piece of fourth fusion information. For example, the fifth input layer of the fifth encoder may be used to encode the fourth fusion information to obtain the fifth intermediate vector. The fifth intermediate vector is processed using the tenth hidden layer of the fifth encoder to obtain fifth coding information respectively corresponding to at least one fourth fusion information.

According to embodiments of the present disclosure, the sixth decoder may include an eleventh hidden layer and a sixth output layer. The sixth decoder may be used to reconstruct the fifth encoded information respectively corresponding to the at least one fourth fusion information, and generate at least one first candidate understanding information. For example, the eleventh hidden layer of the sixth decoder can be used to decode the fifth coded information corresponding to the at least one fourth fusion information, and obtain the fifth auxiliary decoding information corresponding to the at least one fourth fusion information. The sixth output layer of the sixth decoder is used to separately process the fifth auxiliary decoding information corresponding to the at least one fourth fusion information to generate at least one first candidate understanding information.

According to embodiments of the present disclosure, after obtaining at least one first candidate understanding information, the second evaluator may be used to process the at least one first candidate understanding information to obtain evaluation values corresponding to the at least one first candidate understanding information. The second evaluator may be trained based on the NSP task and the MLM task. After obtaining the evaluation values respectively corresponding to the at least one first candidate understanding information, the understanding information may be determined from the at least one first candidate understanding information based on the evaluation values corresponding to the at least one first candidate understanding information.

FIG. 5A schematically shows an example of performing semantic understanding on query information and obtaining understanding information according to an embodiment of the present disclosure.

As shown in FIG. 5A, in 500A, after obtaining the query information, at least one second latent variable information 501 may be determined. The at least one second latent variable information 501 may include second latent variable information 501_1, second latent variable information 501_2, ..., second latent variable information 501_p, ..., second latent variable information 501_P. P can be an integer greater than or equal to 1, p∈{1, 2,..., (P-1), P}.

After obtaining at least one second latent variable information 501, the second latent variable information 501_1, the second latent variable information 501_2, ..., the second latent variable information 501_p, ..., the second latent variable information 501_P can be respectively Fusion with the query information to obtain at least one fourth fusion information. The at least one fourth fusion information may include fourth fusion information 502_1, fourth fusion information 502_2, ..., fourth fusion information 502_p, ..., fourth fusion information 502_P.

After obtaining at least one fourth fusion information, at least one first candidate can be generated based on the fourth fusion information 502_1, the fourth fusion information 502_2, ..., the fourth fusion information 502_p, ..., the fourth fusion information 502_P Understand the message. The at least one first candidate understanding information may include first candidate understanding information 503_1, first candidate understanding information 503_2, ..., first candidate understanding information 503_p, ..., first candidate understanding information 503_P.

After at least one first candidate understanding information is obtained, each of the first candidate understanding information 503_1, the first candidate understanding information 503_2, ..., the first candidate understanding information 503_p, ..., the first candidate understanding information 503_P may be determined. the corresponding evaluation value. At least one evaluation value may include evaluation value 504_1, evaluation value 504_2, ..., evaluation value 504_p, ..., evaluation value 504_P.

After obtaining an evaluation value corresponding to at least one first candidate understanding information, the first candidate understanding information 503_1, 504_2, ..., 504_p, ..., evaluation value 504_P may be obtained. The understanding information 505 is determined among the first candidate understanding information 503_2, ..., the first candidate understanding information 503_p, ..., and the first candidate understanding information 503_P.

According to an embodiment of the present disclosure, operation S210 may include the following operations.

Based on the query information, at least one second candidate understanding information is generated. The at least one second candidate understanding information and the query information are respectively fused to obtain at least one fifth fusion information. Understanding information is determined from at least one second candidate understanding information based on at least one fifth fusion information.

According to embodiments of the present disclosure, after obtaining the query information, the query information can be input into the eighth predetermined model to obtain understanding information. The eighth predetermined model can be configured according to actual business needs, as long as it can realize the function of determining and understanding information, and is not limited here. For example, the eighth predetermined model may include a model based on the Transformer-Dec structure. The eighth predetermined model may include a second language sub-model and a second maximum mutual information scoring function sub-model. The second language sub-model may include a sixth encoder and a seventh decoder. Multiple rounds of training may be performed on the second language sub-model and the second maximum mutual information scoring function sub-model until predetermined conditions are met. The trained second language sub-model and the second maximum mutual information scoring function sub-model are determined as the eighth predetermined model.

According to embodiments of the present disclosure, the sixth encoder may include a sixth input layer and a twelfth hidden layer. The sixth encoder may be used to encode the query information to obtain sixth encoded information. For example, the sixth input layer of the sixth encoder can be used to encode the query information to obtain a sixth intermediate vector. The sixth intermediate vector is processed using the twelfth hidden layer of the sixth encoder to obtain the sixth encoded information.

According to embodiments of the present disclosure, the seventh decoder may include a thirteenth hidden layer and a seventh output layer. The seventh decoder may be used to reconstruct the sixth encoded information and generate at least one second candidate understanding information. For example, the sixth encoded information can be decoded using the thirteenth hidden layer of the seven-decoder to obtain the sixth auxiliary decoding information. The sixth auxiliary decoding information is processed using a seventh output layer of the seventh decoder to generate at least one second candidate understanding information.

According to embodiments of the present disclosure, after obtaining at least one second candidate understanding information, the at least one second candidate understanding information and the query information may be respectively fused to obtain at least one fifth fusion information. Fusion may include at least one of the following: splicing and addition. After obtaining the at least one fifth fusion information, the second maximum mutual information scoring function sub-model may be used to determine the understanding information from the at least one second candidate understanding information based on the at least one fifth fusion information. For example, the second maximum mutual information scoring function sub-model can be used to separately process at least one fifth fusion information to obtain a second maximum mutual information score corresponding to each of at least one second candidate understanding information. The understanding information is determined from the at least one second candidate understanding information according to the second maximum mutual information score respectively corresponding to the at least one second candidate understanding information.

According to embodiments of the present disclosure, the historical dialogue information may include system historical dialogue information and object historical dialogue information. The system historical dialogue information may include a system identification and historical dialogue information corresponding to the system identification. The object historical dialogue information may include an object identification and historical dialogue information corresponding to the object identification. The understanding information may also include an action identifier corresponding to the object's action and a state identifier corresponding to the conversation state.

According to an embodiment of the present disclosure, as shown in Table 1 below, "system" may be used to characterize the system identifier. Use "user" to characterize the object identity. Use "user_action" to characterize the action identifier. Use "belief" to characterize status identification.

Table 1

According to the embodiments of the present disclosure, since the system identifier can be used to represent the system historical dialogue information, and the object identifier can be used to represent the object historical dialogue information, a clear role distinction between the system and the object in the multi-round dialogue is achieved. Therefore, The coherence of the information generation method is improved, thereby improving the accuracy of the dialogue response information.

FIG. 5B schematically shows an example diagram of performing semantic understanding on query information to obtain understanding information according to another embodiment of the present disclosure.

As shown in FIG. 5B, in step 500B, after obtaining the query information 506, at least one second candidate understanding information may be generated based on the query information 506. The at least one second candidate understanding information may include second candidate understanding information 507_1, second candidate understanding information 507_2, ..., second candidate understanding information 507_q, ..., second candidate understanding information 507_Q. Q can be an integer greater than or equal to 1, q∈{1, 2,…, (Q-1), Q}.

After obtaining at least one second candidate understanding information, the second candidate understanding information 507_1, the second candidate understanding information 507_2, ..., the second candidate understanding information 507_q, ..., the second candidate understanding information 507_Q and The query information 506 is fused to obtain at least one fifth fusion information. At least one fifth fusion information may include fifth fusion information 508_1, fifth fusion information 508_2, ..., fifth fusion information 508_q, ..., fifth fusion information 508_Q.

After obtaining at least one fifth fusion information, information may be understood from the second candidate according to the fifth fusion information 508_1, the fifth fusion information 508_2, ..., the fifth fusion information 508_q, ..., the fifth fusion information 508_Q. The understanding information 509 is determined among 507_1, second candidate understanding information 507_2, ..., second candidate understanding information 507_q, ..., and second candidate understanding information 507_Q.

According to an embodiment of the present disclosure, the information generation method 200 may further include the following operations.

In response to detecting the non-assisted request instruction, dialogue response information is generated directly based on the query information and understanding information.

According to an embodiment of the present disclosure, a non-assisted request instruction may refer to a request instruction that does not require calling external resources in the process of generating dialogue response information. When a non-auxiliary request instruction is detected, since there is no need to call external resources, there is no need to determine the auxiliary request information, and the auxiliary request information identifier may be NULL. In this case, dialogue response information can be generated directly based on the query information and understanding information.

According to embodiments of the present disclosure, Table 2 may be used to characterize an example schematic process for generating conversation response information in response to detecting a non-assisted request instruction. As shown in Table 2, the assistance request information may include an assistance request information identifier corresponding to the assistance request information, and "callapi" may be used to represent the assistance request information identifier. The auxiliary response information may include an auxiliary response information identifier related to the auxiliary response information, and "kb" may be used to represent the auxiliary response information identifier. The dialogue response information may include a system action, a system action identifier corresponding to the system action, a system reply, and a system reply identifier corresponding to the system reply. "system action" may be used to represent the system action identifier. Use "response" to represent the system reply identifier.

Table 2

According to embodiments of the present disclosure, Table 3 may be used to characterize an example schematic process for generating conversation response information in response to detecting an assistance request instruction.

table 3

The above are only exemplary embodiments, but are not limited thereto. Other information generation methods known in the art may also be included, as long as the accuracy of the dialogue response information can be improved.

Figure 6 schematically shows a flow chart of a training method of a pre-trained model according to an embodiment of the present disclosure.

As shown in Figure 6, the method 600 includes operations S610 to S640.

In operation S610, semantic understanding is performed on the first sample query information to obtain the first sample understanding information, where the first sample query information includes the first sample historical dialogue information, and the first sample understanding information includes the first sample This object's actions and first sample dialogue state.

In operation S620, first sample assistance request information is obtained according to the first sample query information and the first sample understanding information.

In operation S630, first sample dialogue response information is generated according to the first sample query information, the first sample understanding information, and the first sample assistance request information.

In operation S640, the pre-trained dialogue generation model is trained using the first sample query information, the first sample understanding information and the first sample dialogue response information to obtain an information generation model.

According to an embodiment of the present disclosure, after obtaining the information generation model, the information generation method 200 may be performed using the information generation model.

According to an embodiment of the present disclosure, for the first sample query information, the first sample understanding information, the first sample historical dialogue information, the first sample object action, the first sample dialogue status, and the first sample assistance request For descriptions of the information and the first sample dialogue response information, please refer to the relevant contents above regarding query information, understanding information, historical dialogue information, object actions, dialogue status, auxiliary request information, and dialogue response information, which will not be described again here.

According to embodiments of the present disclosure, the first training sample may be constructed based on the first sample historical dialogue information, the first sample object action, and the first sample dialogue state. For example, the first training sample may be "first sample historical dialogue information -> first sample object action + first sample dialogue state". The second training sample may be constructed based on the first sample historical dialogue information, the first sample object action, the first sample dialogue state, the first sample auxiliary request information, and the first sample dialogue response information. For example, the first training sample may be "first sample historical dialogue information + first sample object action + first sample dialogue state + first sample auxiliary request information -> first sample dialogue response information".

According to embodiments of the present disclosure, since the first sample understanding information is obtained by performing semantic understanding on the first sample query information, the first sample understanding information for dialogue understanding can be obtained. Since the first sample auxiliary request information is obtained based on the first sample query information and the first sample understanding information, external knowledge can be effectively utilized. In addition, the first sample dialogue response information is generated based on the first sample query information, the first sample understanding information and the first sample auxiliary request information. Therefore, the accuracy of the first sample dialogue response information is improved. sex. On this basis, by using the first sample query information, the first sample understanding information and the first sample dialogue response information to train the pre-trained dialogue generation model, an integrated information generation model is obtained. Since the information generation model integrates semantics Understanding and information generation can achieve sharing between different task parameters, thus improving the representation learning ability of the information generation model, thereby improving the information generation ability of the information generation model.

The training method 600 of the pre-trained model according to the embodiment of the present disclosure will be further described below with reference to FIG. 7A and FIG. 7B .

According to an embodiment of the present disclosure, the training method 600 of the pre-trained model may further include the following operations.

Obtain sample historical dialogue information from a corpus, where the corpus includes at least one of the following: a real corpus and a simulated corpus.

According to embodiments of the present disclosure, real corpora may be used to characterize data sets from publicly available data sets. Simulated corpora can be used to characterize artificially generated corpora. The original sample historical dialogue information can be obtained from the corpus. After obtaining the original sample historical dialogue information, the original sample historical dialogue information can be processed to obtain the sample historical dialogue information.

According to embodiments of the present disclosure, the real corpus may include a first real corpus and a second real corpus, where the second real corpus is obtained by translating the third real corpus, and the first real corpus and The languages of the second real corpus are the same.

According to embodiments of the present disclosure, the simulation corpus may be generated based on at least one of the following ways: generated based on predetermined text parameters and generated based on a generative adversarial network model processing predetermined random noise data.

According to embodiments of the present disclosure, the first real corpus and the third real corpus may be obtained from public data sets. The languages of the first real corpus and the third real corpus are different. After obtaining the third real corpus, translation can be performed on the third real corpus to obtain a second real corpus in the same language as the first real corpus.

According to embodiments of the present disclosure, a simulation corpus can be artificially generated using a program based on predetermined text parameters to expand the coverage of the simulation corpus. Alternatively, a simulation corpus can be obtained by inputting predetermined random noise data into a generative adversarial network model. Generative adversarial network models can include deep convolutional generative adversarial network models, bulldozer distance-based generative adversarial network models, or conditional generative adversarial network models, etc. A generative adversarial network model can include a generator and a discriminator. The generator and discriminator can include neural network models. The generator can be used to generate a simulated corpus, and by continuously training the generator, it can learn the data distribution of the simulated corpus, so that it can generate samples from scratch that are consistent with the data distribution of the simulated corpus and eliminate confusion as much as possible. discriminator. The discriminator can be used to distinguish between simulated corpus and real corpus.

According to embodiments of the present disclosure, the convergence condition of the generative adversarial network model may include generator convergence, both generator and discriminator convergence, or an iteration reaching a termination condition, and the iteration reaching the termination condition may include the number of iterations being equal to a predetermined number of iterations.

According to embodiments of the present disclosure, since the sample historical dialogue information is obtained from a corpus, which includes at least one of a real corpus and a simulated corpus, training expectations can be increased, thereby improving the cross-type dialogue of the information generation model. capabilities, thus improving the versatility of the information generation model.

FIG. 7A schematically shows an example schematic diagram of a method for generating a real corpus according to an embodiment of the present disclosure.

As shown in Figure 7A, in 700A, a third real corpus 701 can be obtained. After obtaining the third real corpus 701, the third real corpus 701 can be translated to obtain the second real corpus 702.

After obtaining the second real corpus 702, the first real corpus 703 that has the same language as the second real corpus 702 can be determined based on the second real corpus 702. After obtaining the first real corpus 703, the real corpus 704 may be determined based on the first real corpus 703 and the second real corpus 702.

FIG. 7B schematically shows an example schematic diagram of a method for generating a simulation corpus according to an embodiment of the present disclosure.

As shown in FIG. 7B, in 700B, a predetermined text parameter 705 and a predetermined random noise data 706 may be set in advance. A first simulation corpus 707 may be generated based on predetermined text parameters 705 . A second simulated corpus 708 may be generated based on the predetermined random noise data 706 .

After obtaining the first simulation corpus 707 and the second simulation corpus 708 , the simulation corpus 709 may be determined based on the first simulation corpus 707 and the second simulation corpus 708 .

According to an embodiment of the present disclosure, operation S640 may include the following operations.

Use positive samples and negative samples to train the dialogue generation model and obtain the information generation model.

According to an embodiment of the present disclosure, the positive sample may include first sample query information, first sample understanding information, first sample understanding label information, first sample dialogue response information, and first sample dialogue response label information.

According to an embodiment of the present disclosure, the negative sample may include first sample query information, first sample understanding information, second sample understanding label information, first sample dialogue response information, and second sample dialogue response label information.

According to embodiments of the present disclosure, the first sample understanding label information may be used to characterize normal label information. The first sample dialogue response tag information can be used to characterize normal tag information. At least one of the second sample understanding label information and the second sample dialogue response label information may be abnormal label information.

According to embodiments of the present disclosure, in contrastive learning, the sub-sample obtained by performing data enhancement on the parent sample is considered to be a positive sample for the parent sample. This is because the sub-sample and the parent sample have the same category and maintain the same semantic information with each other. . The parent sample may refer to the sample that is the target of data enhancement processing. For the same parent sample, multiple data enhancements can be performed on the parent sample to obtain multiple subsamples. Although it is for multiple subsamples of the same parent sample, there are subtle differences between the multiple subsamples, that is, the multiple subsamples are not completely consistent. Negative samples can refer to other samples that are of a different category than the parent sample. In this embodiment of the present disclosure, positive samples may include parent samples and positive samples obtained by performing data enhancement on the parent samples.

According to embodiments of the present disclosure, the information generation model is obtained by training the dialogue generation model using positive samples and negative samples, thereby improving the learning ability of the information generation model.

The above are only exemplary embodiments, but are not limited thereto. Other pre-training model training methods known in the art may also be included, as long as the information generation capability of the information generation model can be improved.

FIG. 8 schematically shows a block diagram of an information generating device according to an embodiment of the present disclosure.

As shown in FIG. 8 , the information generating device 800 may include a first semantic understanding module 810 , a first obtaining module 820 and a first generating module 830 .

The first semantic understanding module 810 is used to perform semantic understanding on the query information to obtain understanding information, where the query information includes historical dialogue information, and the understanding information includes object actions and dialogue states.

The first obtaining module 820 is configured to obtain the assistance request information based on the query information and understanding information in response to detecting the assistance request instruction.

The first generation module 830 is used to generate dialogue response information based on query information, understanding information and auxiliary request information.

According to an embodiment of the present disclosure, the first generating module 830 may include a first determining sub-module and a first generating sub-module.

The first determination sub-module is used to determine the assistance response information corresponding to the assistance request information from the data source.

The first generation sub-module is used to generate dialogue response information based on query information, understanding information and auxiliary response information.

According to embodiments of the present disclosure, the first generation sub-module may include a fusion unit and a generation unit.

The fusion unit is used to fuse the query information, understanding information and auxiliary response information to obtain the first fusion information.

A generating unit configured to generate dialogue response information based on the first fusion information.

According to embodiments of the present disclosure, the generation unit may include a first encoding sub-unit, a first decoding sub-unit, and a first generation sub-unit.

The first encoding subunit is used to encode the first fusion information to obtain the first encoded information.

The first decoding subunit is used to self-decode the first encoded information to obtain intermediate decoded information.

The first generating subunit is used to generate dialogue response information based on the first encoded information and the intermediate decoded information.

According to embodiments of the present disclosure, the generation unit may include a second encoding subunit and a second decoding subunit.

The second encoding subunit is used to encode the first fusion information to obtain the second encoded information.

The second decoding subunit is used to decode the second encoded information to obtain dialogue response information.

According to embodiments of the present disclosure, the generation unit may include a second generation subunit, a first fusion subunit, and a first determination subunit.

The second generating subunit is configured to generate at least one first candidate dialogue response information according to the first fusion information.

The first fusion subunit is used to fuse at least one first candidate dialogue response information and the first fusion information respectively to obtain at least one second fusion information.

The first determining subunit is configured to determine dialogue response information from at least one first candidate dialogue response information based on at least one second fusion information.

According to embodiments of the present disclosure, the generation unit may include a second fusion subunit, a third generation subunit, and a second determination subunit.

The second fusion subunit is used to fuse at least one first latent variable information and the first fusion information respectively to obtain at least one third fusion information.

The third generation subunit is configured to generate at least one second candidate dialogue response information based on at least one third fusion information.

The second determination subunit is configured to determine dialogue response information from at least one second candidate dialogue response information based on an evaluation value corresponding to the at least one second candidate dialogue response information.

According to an embodiment of the present disclosure, the first semantic understanding module 810 may include a first fusion sub-module, a first generating sub-module and a second determining sub-module.

The first fusion sub-module is used to fuse at least one second latent variable information and the query information respectively to obtain at least one fourth fusion information.

The first generation sub-module is used to generate at least one first candidate understanding information based on at least one fourth fusion information.

The second determination sub-module is used to determine understanding information from at least one first candidate understanding information according to the evaluation value corresponding to the at least one first candidate understanding information.

According to an embodiment of the present disclosure, the first semantic understanding module 810 may include a second generating sub-module, a second fusion sub-module and a third determining sub-module.

The second generation sub-module is used to generate at least one second candidate understanding information according to the query information.

The second fusion sub-module is used to fuse at least one second candidate understanding information and the query information respectively to obtain at least one fifth fusion information.

The third determination sub-module is used to determine understanding information from at least one second candidate understanding information according to at least one fifth fusion information.

According to an embodiment of the present disclosure, the information generation device 800 may further include a third generation module.

The third generation module is used to generate dialogue response information directly based on the query information and understanding information in response to detecting the non-auxiliary request instruction.

According to an embodiment of the present disclosure, the query information further includes a query word slot.

According to an embodiment of the present disclosure, the historical dialogue information includes system historical dialogue information and object historical dialogue information. The system historical dialogue information includes a system identification and historical dialogue information corresponding to the system identification. The object historical dialogue information includes an object identification and a corresponding object identification. historical conversation information.

According to an embodiment of the present disclosure, the understanding information further includes an action identifier corresponding to the object action and a state identifier corresponding to the conversation state.

Figure 9 schematically shows a block diagram of a training device for a pre-trained model according to an embodiment of the present disclosure.

As shown in FIG. 9 , the training device 900 of the pre-training model may include a second semantic understanding module 910 , a second acquisition module 920 , a second generation module 930 and a training module 940 .

The second semantic understanding module 910 is used to perform semantic understanding on the first sample query information to obtain the first sample understanding information, where the first sample query information includes the first sample historical dialogue information, and the first sample understanding information The information includes a first sample object action and a first sample conversation state.

The second obtaining module 920 is used to obtain the first sample assistance request information according to the first sample query information and the first sample understanding information.

The second generation module 930 is configured to generate first sample dialogue response information based on the first sample query information, the first sample understanding information, and the first sample assistance request information.

The training module 940 is used to train the pre-trained dialogue generation model using the first sample query information, the first sample understanding information and the first sample dialogue response information to obtain an information generation model.

According to an embodiment of the present disclosure, the training device 900 of the pre-trained model may further include an acquisition module.

The acquisition module is used to obtain sample historical dialogue information from the corpus, where the corpus includes at least one of the following: a real corpus and a simulated corpus.

According to an embodiment of the present disclosure, the real corpus includes a first real corpus and a second real corpus, where the second real corpus is obtained by translating the third real corpus, and the first real corpus and the third real corpus The languages of the two real corpora are the same.

According to an embodiment of the present disclosure, the simulation corpus is generated based on at least one of the following ways: generated based on predetermined text parameters and generated based on a generative adversarial network model processing predetermined random noise data.

According to embodiments of the present disclosure, the training module 940 may include a training sub-module.

The training submodule is used to train the dialogue generation model using positive samples and negative samples to obtain the information generation model.

According to an embodiment of the present disclosure, the positive sample includes first sample query information, first sample understanding information, first sample understanding label information, first sample dialogue response information, and first sample dialogue response label information.

According to an embodiment of the present disclosure, the negative sample includes first sample query information, first sample understanding information, second sample understanding label information, first sample dialogue response information, and second sample dialogue response label information. The second sample At least one of the understanding tag information and the second sample conversation response tag information is anomaly tag information.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

According to an embodiment of the present disclosure, an electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by at least one processor, and the instructions are processed by at least one processor. The processor executes, so that at least one processor can execute the method as described in the present disclosure.

According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions are used to cause a computer to perform a method as described in the present disclosure.

According to an embodiment of the present disclosure, a computer program product includes a computer program, and when executed by a processor, the computer program implements the method according to the present disclosure.

FIG. 10 schematically shows a block diagram of an electronic device suitable for implementing the information generation method and the training method of the pre-training model according to an embodiment of the present disclosure. Electronic devices are intended to refer to various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are examples only and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in FIG. 10 , the electronic device 1000 includes a computing unit 1001 that can perform calculations according to a computer program stored in a read-only memory (ROM) 1002 or loaded from a storage unit 1008 into a random access memory (RAM) 1003 . Perform various appropriate actions and processing. In the RAM 1003, various programs and data required for the operation of the electronic device 1000 can also be stored. Computing unit 1001, ROM 1002 and RAM 1003 are connected to each other via bus 1004. An input/output (I/O) interface 1005 is also connected to bus 1004.

Multiple components in the electronic device 1000 are connected to the I/O interface 1005, including: input unit 1006, such as a keyboard, mouse, etc.; output unit 1007, such as various types of displays, speakers, etc.; storage unit 1008, such as a magnetic disk, optical disk etc.; and communication unit 1009, such as network card, modem, wireless communication transceiver, etc. The communication unit 1009 allows the electronic device 1000 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunications networks.

Computing unit 1001 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processing processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as the information generation method and the training method of the pre-trained model. For example, in some embodiments, the information generation method and the training method of the pre-trained model may be implemented as a computer software program, which is tangibly embodied in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 1000 via the ROM 1002 and/or the communication unit 1009 . When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the information generation method and the training method of the pre-trained model described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform the information generation method and the training method of the pre-trained model in any other suitable manner (eg, by means of firmware).

Various implementations of the systems and techniques described above may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip implemented in a system (SOC), complex programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include implementation in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor The processor, which may be a special purpose or general purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device. An output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that the program codes, when executed by the processor or controller, cause the functions specified in the flowcharts and/or block diagrams/ The operation is implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and may be provided in any form, including Acoustic input, voice input or tactile input) to receive input from the user.

The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., A user's computer having a graphical user interface or web browser through which the user can interact with implementations of the systems and technologies described herein), or including such backend components, middleware components, or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communications network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

Computer systems may include clients and servers. Clients and servers are generally remote from each other and typically interact over a communications network. The relationship of client and server is created by computer programs running on corresponding computers and having a client-server relationship with each other. The server can be a cloud server, a distributed system server, or a server combined with a blockchain.

It should be understood that various forms of the process shown above may be used, with steps reordered, added or deleted. For example, each step described in the present disclosure can be executed in parallel, sequentially, or in a different order. As long as the desired results of the technical solution disclosed in the present disclosure can be achieved, there is no limitation here.

The above-mentioned specific embodiments do not constitute a limitation on the scope of the present disclosure. It will be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions are possible depending on design requirements and other factors. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this disclosure shall be included in the protection scope of this disclosure.

Claims (20)

1. An information generation method, comprising:

semantic understanding is carried out on query information and second hidden variable information corresponding to at least one dialogue round number respectively to obtain understanding information, wherein the query information comprises historical dialogue information, the understanding information comprises object actions and dialogue states, and the second hidden variable information is generated according to dialogue contents and dialogue responses corresponding to the at least one dialogue round number respectively;

Responding to the detection of an auxiliary request instruction, and obtaining auxiliary request information according to the query information and the understanding information; and

generating dialogue response information according to the query information, the understanding information and the auxiliary request information;

the semantic understanding of the query information and the second hidden variable information corresponding to at least one dialogue round number respectively is carried out to obtain understanding information, and the method comprises the following steps:

respectively fusing at least one second hidden variable information and the query information to obtain at least one fourth fused information;

generating at least one first candidate understanding information according to the at least one fourth fusion information; and

and determining the understanding information from the at least one first candidate understanding information according to the evaluation value corresponding to the at least one first candidate understanding information.

2. The method of claim 1, wherein the generating dialogue response information from the query information, the understanding information, and the auxiliary request information comprises:

determining auxiliary response information corresponding to the auxiliary request information from a data source; and

and generating the dialogue response information according to the query information, the understanding information and the auxiliary response information.

3. The method of claim 2, wherein the generating the dialogue response information from the query information, the understanding information, and the auxiliary response information comprises:

fusing the query information, the understanding information and the auxiliary response information to obtain first fused information; and

and generating the dialogue response information according to the first fusion information.

4. A method according to claim 3, wherein said generating said dialogue response information from said first fusion information comprises:

encoding the first fusion information to obtain first encoded information;

performing self-decoding on the first coding information to obtain intermediate decoding information; and

and generating the dialogue response information according to the first coding information and the intermediate decoding information.

5. A method according to claim 3, wherein said generating said dialogue response information from said first fusion information comprises:

encoding the first fusion information to obtain second encoded information; and

and decoding the second encoded information to obtain the dialogue response information.

6. A method according to claim 3, wherein said generating said dialogue response information from said first fusion information comprises:

Generating at least one first candidate dialogue response message according to the first fusion message;

respectively fusing the at least one first candidate dialogue response information and the first fusion information to obtain at least one second fusion information; and

and determining the dialogue response information from the at least one first candidate dialogue response information according to the at least one second fusion information.

7. A method according to claim 3, wherein said generating said dialogue response information from said first fusion information comprises:

respectively fusing at least one first hidden variable information and the first fusion information to obtain at least one third fusion information;

generating at least one second candidate dialogue response message according to the at least one third fusion message; and

and determining the dialogue response information from the at least one second candidate dialogue response information according to the evaluation value corresponding to the at least one second candidate dialogue response information.

8. The method as recited in claim 1, further comprising:

generating at least one piece of second candidate understanding information according to the query information;

respectively fusing the at least one second candidate understanding information and the query information to obtain at least one fifth fused information; and

And determining the understanding information from the at least one second candidate understanding information according to the at least one fifth fusion information.

9. The method of claim 1, further comprising:

and responding to the detection of the non-auxiliary request instruction, and generating the dialogue response information directly according to the query information and the understanding information.

10. The method of claim 1, wherein the query information further comprises a query word slot.

11. The method of any of claims 1-10, wherein the historical dialog information includes system historical dialog information and object historical dialog information, the system historical dialog information including a system identification and historical dialog information corresponding to the system identification, the object historical dialog information including an object identification and historical dialog information corresponding to the object identification;

the understanding information further includes an action identifier corresponding to the object action and a state identifier corresponding to the dialog state.

12. A training method of a pre-training model, comprising:

semantic understanding is carried out on the first sample query information and second sample hidden variable information corresponding to at least one dialogue round number respectively to obtain first sample understanding information, wherein the first sample query information comprises first sample historical dialogue information, the first sample understanding information comprises first sample object actions and first sample dialogue states, and the second sample hidden variable information is generated according to dialogue contents and dialogue responses corresponding to the at least one dialogue round number respectively;

Obtaining first sample auxiliary request information according to the first sample inquiry information and the first sample understanding information;

generating first sample opposite-speaking response information according to the first sample query information, the first sample understanding information and the first sample auxiliary request information; and

training a pre-training dialogue generation model by using the first sample query information, the first sample understanding information and the first sample opposite-speaking response information to obtain an information generation model;

the semantic understanding of the first sample query information and the second sample hidden variable information corresponding to at least one dialogue round number respectively is carried out to obtain first sample understanding information, and the semantic understanding method comprises the following steps:

respectively fusing at least one second sample hidden variable information and the first sample query information to obtain at least one fourth sample fusion information;

generating at least one first candidate sample understanding information according to the at least one fourth sample fusion information; and

and determining the first sample understanding information from the at least one first candidate sample understanding information according to the evaluation value corresponding to the at least one first candidate sample understanding information.

13. The method of claim 12, further comprising:

obtaining the sample historical dialog information from a corpus, wherein the corpus comprises at least one of: a real corpus and a simulated corpus.

14. The method of claim 13, wherein the real corpus comprises a first real corpus and a second real corpus, wherein the second real corpus is obtained by translating a third real corpus, and languages of the first real corpus and the second real corpus are the same;

the simulated corpus is generated based on at least one of: generated based on predetermined text parameters and generated based on generating the countermeasure network model processing predetermined random noise data.

15. The method of any of claims 12-14, wherein the training a pre-trained dialog generation model using the first sample query information, the first sample understanding information, and the first sample pair-call answer information to obtain an information generation model comprises:

training the dialogue generating model by using a positive sample and a negative sample to obtain the information generating model;

wherein the positive sample includes the first sample query information, the first sample understanding information, first sample understanding tag information, the first sample opposite-speaking reply information, and first sample opposite-speaking reply tag information;

Wherein the negative sample includes the first sample query information, the first sample understanding information, second sample understanding tag information, the first sample session response information, and second sample session response tag information, at least one of the second sample understanding tag information and the second sample session response tag information being abnormal tag information.

16. An information generating apparatus comprising:

the first semantic understanding module is used for carrying out semantic understanding on query information and second hidden variable information corresponding to at least one dialogue round number respectively to obtain understanding information, wherein the query information comprises historical dialogue information, the understanding information comprises object actions and dialogue states, and the second hidden variable information is generated according to dialogue contents and dialogue responses corresponding to the at least one dialogue round number respectively;

the first obtaining module is used for responding to the detection of the auxiliary request instruction and obtaining auxiliary request information according to the query information and the understanding information; and

the first generation module is used for generating dialogue response information according to the query information, the understanding information and the auxiliary request information;

The semantic understanding of the query information and the second hidden variable information corresponding to at least one dialogue round number respectively is carried out to obtain understanding information, and the method comprises the following steps:

respectively fusing at least one second hidden variable information and the query information to obtain at least one fourth fused information;

generating at least one first candidate understanding information according to the at least one fourth fusion information; and

and determining the understanding information from the at least one first candidate understanding information according to the evaluation value corresponding to the at least one first candidate understanding information.

17. A training device for a pre-training model, comprising:

the second semantic understanding module is used for carrying out semantic understanding on the first sample query information and second sample hidden variable information corresponding to at least one dialogue round number respectively to obtain first sample understanding information, wherein the first sample query information comprises first sample historical dialogue information, the first sample understanding information comprises first sample object actions and first sample dialogue states, and the second sample hidden variable information is generated according to dialogue contents and dialogue responses corresponding to the at least one dialogue round number respectively;

The second obtaining module is used for obtaining first sample auxiliary request information according to the first sample inquiry information and the first sample understanding information;

the second generation module is used for generating first sample opposite-speaking response information according to the first sample query information, the first sample understanding information and the first sample auxiliary request information; and

the training module is used for training a pre-training dialogue generation model by using the first sample query information, the first sample understanding information and the first sample opposite-speaking response information to obtain an information generation model;

the semantic understanding of the first sample query information and the second sample hidden variable information corresponding to at least one dialogue round number respectively is carried out to obtain first sample understanding information, and the semantic understanding method comprises the following steps:

respectively fusing at least one second sample hidden variable information and the first sample query information to obtain at least one fourth sample fusion information;

generating at least one first candidate sample understanding information according to the at least one fourth sample fusion information; and

and determining the first sample understanding information from the at least one first candidate sample understanding information according to the evaluation value corresponding to the at least one first candidate sample understanding information.

18. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 11 or claims 12 to 15.

19. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-11 or claims 12-15.

20. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 11 or claims 12 to 15.