CN106448670B - Conversational system is automatically replied based on deep learning and intensified learning - Google Patents
Conversational system is automatically replied based on deep learning and intensified learning Download PDFInfo
- Publication number
- CN106448670B CN106448670B CN201610920931.2A CN201610920931A CN106448670B CN 106448670 B CN106448670 B CN 106448670B CN 201610920931 A CN201610920931 A CN 201610920931A CN 106448670 B CN106448670 B CN 106448670B
- Authority
- CN
- China
- Prior art keywords
- user
- information
- learning
- conversational system
- portrait
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013135 deep learning Methods 0.000 title claims abstract description 26
- 238000004458 analytical method Methods 0.000 claims abstract description 11
- 238000005516 engineering process Methods 0.000 claims abstract description 11
- 230000002452 interceptive effect Effects 0.000 claims abstract description 8
- 230000008451 emotion Effects 0.000 claims abstract description 7
- 238000013528 artificial neural network Methods 0.000 claims description 15
- 238000000034 method Methods 0.000 claims description 10
- 230000002996 emotional effect Effects 0.000 claims description 9
- 238000012549 training Methods 0.000 claims description 8
- 230000000306 recurrent effect Effects 0.000 claims description 6
- 238000003062 neural network model Methods 0.000 claims description 2
- 238000000605 extraction Methods 0.000 description 7
- 230000036651 mood Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000013461 design Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000003796 beauty Effects 0.000 description 1
- 230000003542 behavioural effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000004064 recycling Methods 0.000 description 1
- 230000003014 reinforcing effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/226—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics
- G10L2015/227—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics of the speaker; Human-factor methodology
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/226—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics
- G10L2015/228—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics of application context
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses automatically reply conversational system based on deep learning and intensified learning, comprising: user interactive module receives the problem of user inputs in conversational system interface information；Session management module, records the active state of user, and active state includes dialog history information, user location information converting and user emotion change information；Customer analysis module analyzes the registration information and active state of user, draws a portrait for user, obtains user's portrait information；Session module, according to user in problem information, in conjunction with the portrait of user, the answer information generated by language model；Model learning module passes through intensified learning technology innovation language model according to the answer information that language model generates.Conversational system is automatically replied the present invention is based on deep learning and intensified learning, the dialog text that can be inputted according to user, intentions in information from the context, the personal characteristics of user and dialogue provides the dialogue answer for meeting user personality.
Description
Technical field
The present invention relates to artificial intelligence fields, more particularly to intelligent human-computer dialogue field.
Background technique
With the continuous rising of the continuous evolution and manual service cost of human society informationization, people increasingly wish
It is exchanged by natural language with computer, Intelligent dialogue robot system becomes the production being born under such historical background
Object can look after the mood of user, can mention to user especially it will be appreciated that user, can remember the dialog history of user
For the dialogue robot system of personalized service, just becoming direction and the emphasis of each major company and institutions for academic research research and development.
Implementation one needs to answer simultaneously careful design selection logic for the building of various problems is various in the prior art, needs
Put into huge manpower.If it is considered that the dependence between context, then the design of rule can be extremely complex, and is advised
Also there can be conflict between then.Implementation two needs to translate based on statistical system, but under the scene of dialogue, because replying
There is no the synonymies of semantic level between input, so that output can not input well and match.Particularly with more
The scene and system translation for taking turns dialogue are even more that there are basic differences, because also needing the information more consideration is given to context, such as
Space-time locating for the intention of user, hobby and user, the content that front is talked with, the switching etc. of conversation subject.
Therefore defect in the prior art be: existing interactive system implementation cannot according to the user's intention or
It is that the difference of context provides answer that is accurate and meeting user personality, cannot achieve intelligent chat.
Summary of the invention
Dialogue is automatically replied based on deep learning and intensified learning the technical problem to be solved in the present invention is to provide a kind of
System, the dialog text that can be inputted according to user, the intention in information from the context, the personal characteristics of user and dialogue provide
The dialogue for meeting user personality is answered.
In order to solve the above technical problems, present invention provide the technical scheme that
The present invention, which provides, a kind of automatically replies conversational system based on deep learning and intensified learning, comprising:
User interactive module, for receiving the problem of user inputs in conversational system interface information；
Session management module, for recording the active state of the user, the active state includes dialog history information,
User location information converting and user emotion change information；
Customer analysis module is drawn a portrait for analyzing the registration information and active state of the user for the user,
User's portrait information is obtained, the user draws a portrait information for describing the personal characteristics of the user, the note of the user
Volume information includes userspersonal information；
Session module, for passing through language mould in conjunction with the portrait of the user in described problem information according to the user
The answer information that type generates；
Model learning module, the answer information for being generated according to the language model, passes through intensified learning technology innovation
The language model.
The technical solution of the present invention is as follows: user interactive module, asked for receive that user inputs in conversational system interface
Inscribe information；Session management module, for recording the active state of the user, the active state includes dialog history information,
User location information converting and user emotion change information；Customer analysis module, for analyze the user registration information and
Active state is drawn a portrait for the user, obtains user's portrait information, it is described for describing that the user draws a portrait information
The personal characteristics of user, the registration information of the user include userspersonal information；
Session module, for passing through language mould in conjunction with the portrait of the user in described problem information according to the user
The answer information that type generates；Model learning module, the answer information for being generated according to the language model, passes through intensified learning
Language model described in technology innovation.
Carry out human-computer dialogue by the above-mentioned conversational system that automatically replies based on deep learning and intensified learning, can according to
The dialog text of family input, the intention in information from the context, the personal characteristics of user and dialogue, which provides, meets user personality
Dialogue answer, improve user experience.
Further, it in the session module, is generated and is replied by the language model, specifically:
The dialog history information of the problem of obtaining user information and user；
The problem of to the user information and the user dialog history information by encoder carry out information extraction, obtain
To information is extracted, the extraction information includes the subject information for including, speech performance information and mood letter in described problem information
Breath；
Described problem information is encoded by the encoder, exports the vector of a regular length, the vector
Indicate the user category information obtained according to the portrait of the user；
It is decoded in conjunction with the extraction information by decoder according to the vector of the regular length, generates and answer letter
Breath.
Further, the training process of the language model is divided into offline supervised learning stage and online unsupervised reinforcing
The study stage.
Further, in the model learning module, in the offline supervised learning stage, pass through Recognition with Recurrent Neural Network
The language model is established, specifically:
The problem of first layer input active user of the circulation neural network inputs information and the dialog history information；
The second layer of the circulation neural network is the problem of active user extracted by encoder inputs information
Subject information, speech performance information, emotional information and export a regular length vector；
The third layer of the circulation neural network exports the answer obtained by the information that decoder decodes the second layer
Information.
Further, in the model learning module, in the online unsupervised intensified learning stage, pass through intensified learning
Technology innovation language model:
According to the answer information that the language model generates, award coefficient is calculated；
It is carried out in calculating process by back-propagation algorithm, it will be in the award coefficient and the back-propagation algorithm
Derivative is multiplied, and generates new derivative, realizes the update of the language model.
Further, the problem of user inputs in conversational system interface information includes voice messaging, graphical information
And text information.
Further, user's portrait information uses keyword message, user to use in a search engine by user
The dialog text information of application program of mobile phone information and user in conversational system interface obtains；
The user includes in a search engine being drawn by the user in search using the acquisition pattern of keyword message
It holds up middle using the number of keyword, frequency of use and using the time；
The user includes using mobile phone application journey by the user using the acquisition pattern of application program of mobile phone information
The number of sequence, frequency of use and use the time.
Further, user's portrait information includes label information and profile information.
Further, the userspersonal information includes the gender of user, the age, address, speaking style, social relationships,
Emotional change mode and personal preference.
Further, the decoder is realized by GRU algorithm.
Detailed description of the invention
It, below will be to specific in order to illustrate more clearly of the specific embodiment of the invention or technical solution in the prior art
Embodiment or attached drawing needed to be used in the description of the prior art are briefly described.
Fig. 1 shows a kind of automatically replying based on deep learning and intensified learning provided by first embodiment of the invention
The schematic diagram of conversational system；
Fig. 2 shows a kind of automatically replying based on deep learning and intensified learning provided by first embodiment of the invention
The language model training system block diagram of conversational system；
Fig. 3 shows a kind of automatically replying based on deep learning and intensified learning provided by first embodiment of the invention
Schematic diagram is established in the language model training of conversational system；
Fig. 4 shows a kind of automatically replying based on deep learning and intensified learning provided by first embodiment of the invention
The GRU algorithm schematic diagram of conversational system；
Fig. 5 shows a kind of automatically replying based on deep learning and intensified learning provided by first embodiment of the invention
The intensified learning schematic diagram of conversational system.
Specific embodiment
It is described in detail below in conjunction with embodiment of the attached drawing to technical solution of the present invention.Following embodiment is only used for
Clearly illustrate technical solution of the present invention, therefore be intended only as example, and cannot be used as a limitation and limit protection of the invention
Range.
Embodiment one
Fig. 1 shows a kind of automatically replying based on deep learning and intensified learning provided by first embodiment of the invention
The schematic diagram of conversational system 10, as shown in Figure 1, a kind of automatic time based on deep learning and intensified learning that embodiment one provides
Multiple conversational system 10, comprising:
User interactive module 101, for receiving the problem of user inputs in conversational system interface information；
Session management module 102, for recording the active state of user, active state includes dialog history information, user
Evolution information and user emotion change information；
Customer analysis module 103 is drawn a portrait for user, is obtained for analyzing the registration information and active state of user
User's portrait information, user draw a portrait information for describing the personal characteristics of user, and the registration information of user includes individual subscriber letter
Breath；
Session module 104, for being generated in conjunction with the portrait of user by language model according to user in problem information
Answer information；
Model learning module 105, the answer information for being generated according to language model pass through intensified learning technology innovation language
Say model.
The technical solution of the present invention is as follows: user interactive module 101, inputted in conversational system interface for receiving user
Problem information；Session management module 102, for recording the active state of user, active state includes dialog history information, user
Evolution information and user emotion change information；Customer analysis module 103, for analyzing the registration information and moving type of user
State is drawn a portrait for user, obtains user's portrait information, and user's portrait information is used to describe the personal characteristics of user, user's
Registration information includes userspersonal information；
Session module 104, for being generated in conjunction with the portrait of user by language model according to user in problem information
Answer information；Model learning module 105, the answer information for being generated according to language model, passes through intensified learning technology innovation
Language model.
User interactive module 101 receives the problem of user's input；Session management module 102 obtains the ID of the user, user
The information such as the dialog history of time and nearest 5 wheel are putd question in current location；Session management module 102 is asked to customer analysis module 103
The classification information of user is sought, the information that session management module 102 obtains front is sent to session module 104, and session module makes
An answer is generated with trained language model；Session module 104 can request model learning module 105 to use intensified learning
Technology updates language model.
Human-computer dialogue is carried out by the above-mentioned conversational system 10 that automatically replies based on deep learning and intensified learning, it can basis
The dialog text of user's input, the intention in information from the context, the personal characteristics of user and dialogue, which provides, meets user
Property dialogue answer, improve user experience.
Wherein, the classification information of user is generated by customer analysis module 103, and customer analysis module 103 is used for off-line analysis
User, according to the registration information of user, Conversation History and from social networks of acquisitions such as external system such as microblogging etc.；It gives
User classifies and tagged, in this way generate dialogue when can using the classification of user as input information it
One, personalized reply is generated according to different classes of user to realize.
Specifically, registration information of the customer analysis module 103 from user, history chat record, historical activity information, history
Dialog information etc. is excavated out the age of user, gender, address, scope of activities, hobby, interest, the style of speech, emotional change mould
Formula, social relationships etc., draws a portrait to user.
Specifically, it in session module, is generated and is replied by language model, specifically:
The dialog history information of the problem of obtaining user information and user；
The problem of to user information and user dialog history information by encoder carry out information extraction, obtain extract letter
Breath, extracting information includes the subject information for including, speech performance information and emotional information in problem information；
Problem information is encoded by encoder, exports the vector of a regular length, vector is indicated according to user
The obtained user category information of portrait；
It is decoded in conjunction with information is extracted by decoder according to the vector of regular length, generates and answer information.
The semanteme in user session content information is analyzed by encoder, then the information such as mood are decoded by decoder,
The dialogue answer for meeting user personality is generated, the intention that trained language model in advance analyzes user session content, connection are passed through
Context information and the individual character of user are fastened, the dialogue answer for meeting user personality can be provided in this way.
Specifically, the training process of language model is divided into offline supervised learning stage and online unsupervised intensified learning rank
Section.
Specifically, in model learning module, in the offline supervised learning stage, language mould is established by Recognition with Recurrent Neural Network
Type, specifically:
The problem of recycling first layer input active user's input of neural network information and dialog history information；
The second layer of circulation neural network is that the theme of the problem of active user extracted by encoder inputs information is believed
Breath, speech performance information, emotional information and the vector for exporting a regular length；
The answer information that the third layer output of circulation neural network is obtained by the information that decoder decodes the second layer.
Language model generally passes through neural network, and the present invention is based on automatically replying pair for deep learning and intensified learning
Telephone system establishes language model using Recognition with Recurrent Neural Network, and Recognition with Recurrent Neural Network (RNN) really can be utilized fully on all
Context information predicts next word, i.e., establishing language model with Recognition with Recurrent Neural Network can be deduced with contextual information
The answer information of dialogue keeps human-computer dialogue more natural more acurrate.
As shown in Fig. 2, the problem of language model is made of two parts of encoder and decoder, and user proposes uses first
Encoder is encoded, and the n-dimensional vector of a regular length is exported, and is reused decoder and is decoded, and an answer is generated.
Specifically, as shown in figure 3, having used the problem of 4 encoders are come to user in language model of the invention and having gone through
History dialogue carries out information extraction, and indicates that the vector of user information is input in decoder together with one and be decoded, next life
At answer.
Specifically, four encoders are representation device, theme encoder, speech performance encoder and mood coding respectively
Device；Wherein:
Representation device (Question Encoder): for being encoded to the problem of user；Representation device uses
Multilayer GRU or LSTM are realized；
Theme encoder (Topic Encoder): for being encoded to dialog history and current problem, coding result side
Overweight extraction subject information；Theme encoder is realized using CNN+RNN mode；
Speech performance encoder (Speech Act Encoder): for being encoded to dialog history and current problem,
Coding result, which is laid particular emphasis on, extracts speech performance information, the implementation of speech performance encoder and the implementation of representation device
Similar, different places is to need in training pattern using the corpus for being labelled with speech performance；
Mood editing machine (Emotion Encoder), for being encoded to dialog history and current problem, coding result
Lay particular emphasis on extraction emotional information；The implementation of mood editing machine is similar with the implementation of representation device, different places
Only need in training pattern using the corpus for being labelled with mood.
Specifically, user information is indicated using a n-dimensional vector, each value dimension 0 or 1, class of subscriber can be used
The gender at family, age bracket, hobby, current time, the information codings such as place are into this vector, for example, if user is divided into 3
Class shows respectively 3 class of subscribers then can be indicated long with 3, such as 100,010,001.
Specifically, the output of above 4 encoders and user information vector are input, are decoded, and generate and answer, decoding
Device is achieved in that using GRU realization, but is different from traditional GRU, increases context input here, therefore become
C-GRU, specific calculating process are as shown in Figure 4.
Wherein, the calculation formula of output is as follows:
yt=Softmax(Whyht)
ht=ztοht-1+(1-zt)οgt
gt=tanh (Wxgxt+Whg(rtοht-1)+Wcg(utοc)+bg)
zt=σ (Wxzxt+Whzht-1+Wcz(utοc)+bz)
rt=σ (Wxrxt+Whrht-1+Wcr(utοc)+br)
ut=σ (α Whuht-1+Wcuc+bu)
Specifically, in model learning module, in the online unsupervised intensified learning stage, pass through intensified learning technology innovation language
Say model:
According to the answer information that language model generates, award coefficient is calculated；
It is carried out in calculating process by back-propagation algorithm, by award coefficient and the derivative phase in back-propagation algorithm
Multiply, generate new derivative, realizes the update of language model.
BP (Back Propagation, back-propagation algorithm) network is 1986 to be by Rumelhart and McCelland
First scientist group proposes, is a kind of Multi-layered Feedforward Networks by Back Propagation Algorithm training, is most widely used at present
One of neural network model.BP network can learn and store a large amount of input-output mode map relationship, without taking off in advance
Show that description is this to reflect.
The unsupervised language model in the intensified learning stage, trained can produce relatively good online, not have substantially
The answer of syntax error, but due to using MLE (maximum likelihood eapection) target, compare appearance
Some meaningless answers easily generated, so in this stage using the parameter of intensified learning amendment language model, such as Fig. 5 institute
Show, when conversational system receives user the problem of, and generates an answer, then use a bonus policy (Reward
Policy award coefficient r) is calculated, is multiplied as new derivative, so when carrying out BP (Back Propagation) with derivative
After continue to complete BP process.
Specifically, the problem of user inputs in conversational system interface information includes voice messaging, graphical information and text
Information.The dialog text information of user is inputted in conversational system interface, text information can be a Duan Yuyin, can be one
Shot image information, is also possible to text information, and system can all provide corresponding answer according to these information, and multiple input modes can be with
The different demands for meeting user, improve user experience.
Specifically, user's portrait information uses keyword message, user that mobile phone is used to answer in a search engine by user
It is obtained with the dialog text information of program information and user in conversational system interface；
It includes being used in a search engine by user that user uses the acquisition pattern of keyword message in a search engine
The number of keyword, frequency of use and use the time；
User using the acquisition pattern of application program of mobile phone information include by user using application program of mobile phone number,
Frequency of use and use the time.
The portrait information of user is the portrait information that description includes user personality, feature and behavioural characteristic, can pass through use
Family uses application program of mobile phone information and the user session content sample collected using keyword message, user in a search engine
This information acquisition, for example, user often searches for the keyword about cuisines in a search engine, then the user may be a beauty
Intelligent is eaten, it, can also be with then the user may be online shopping fan for example, application program of the user commonly using shopping class
By the user session content sample information collected, these information can make the text information of the previous user session every time,
It is also possible to some sample informations that we are collected into.By these information, the portrait information of user is obtained, it can be according to difference
The different characteristics of user provides more accurate answer.
Specifically, it includes by user in search engine that user uses the acquisition pattern of keyword message in a search engine
It is middle using the number of keyword, frequency of use and use the time；User includes using the acquisition pattern of application program of mobile phone information
Using the number of application program of mobile phone, frequency of use and the time is used by the user.
Automatically replying conversational system and can be had in several ways the present invention is based on deep learning and intensified learning
There is the information of user's personal characteristics, three kinds of acquisition modes is referred in the present invention, wherein the user uses in a search engine
Keyword message is that the number of keyword is used to obtain in a search engine according to the user, can count the user and search
Index holds up the middle number using keyword, when being greater than a certain threshold value, that is, determines that this keyword is that user passes through in a search engine
It is often used, equally, the number of application program of mobile phone can also be used to determine this when being greater than a certain threshold value with counting user
Application program is that the user is commonly used, and may determine that the personal characteristics of the user according to these information, so that it may according to
Different users provides different answers.
Specifically, user's portrait information includes label information and profile information.User's portrait information is for describing user
Personal touch, therefore can be showed by way of label or image, intuitive and convenient.Specifically, userspersonal information
Gender including user, age, address, speaking style, social relationships, emotional change mode and personal preference.
In conjunction with the personal information of user, more accurate answer can be provided, for example knows the gender of user, in dialog procedure
In will provide different answers according to different genders, for example user wonders place more joyful nearby, then system
It will be women according to user, the place for recommending some suitable women to play, such as KTV, market etc. will recommend if it is male
Such as gymnasium, the places such as electronic game city, therefore the personal information of user is obtained, it is available more accurately to answer.
Embodiment two
Conversational system 10 is automatically replied the present invention is based on deep learning and intensified learning, in conjunction with user in system interface
Conversation content specifically introduces system flow of the invention；
User: hello！
System: good afternoon, has anything that can help you
User: my computer booting is not shown.
System: sad for you.Do are you with Windows operating system
User: not being, linux system.
System: it there is there what miscue
User carries out human-computer dialogue in system interface, inputs the text information of user session, and such as " hello ", system first will
" hello " code conversion facilitates computer to be calculated at term vector, and then for category information of simply saying hello, system can basis
Preset mode provides corresponding answer, such as " good afternoon, has anything that can help you " or " good afternoon, may I ask assorted
Thing ", then user then engages in the dialogue, and " my computer booting is not shown ", equally, the words is first converted by system
Term vector information, then system calculates the intent information of user by perceptron according to the term vector information being converted into, it is intended that
Information be exactly system can analyze user's the words what is intended that, the words is intended that " display of user computer is not
Easily make, ask for help ", then system according to the user's intention, in conjunction with the portrait information of user, is deduced by language model
To the answer information of user, wherein the portrait information of user describes the information such as the personal touch of user, makes full use of so each
The different feature of user, provides the user with the answer of difference personalization, such as " sad for you.You are to use Windows operating system
", in language model, many data are store, according to the user's intention information and portrait information, it can be quickly in language
Some information accordingly about computer are found in model, then provide corresponding answer.User and system in the same way
It engages in the dialogue, above-mentioned conversation content will be obtained.
Therefore, conversational system is automatically replied based on deep learning and intensified learning through the invention, it can be according to user
The intention of conversation content expression, provides personalized answer.
Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；To the greatest extent
Pipe present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: its according to
So be possible to modify the technical solutions described in the foregoing embodiments, or to some or all of the technical features into
Row equivalent replacement；And these are modified or replaceed, various embodiments of the present invention technology that it does not separate the essence of the corresponding technical solution
The range of scheme should all cover within the scope of the claims and the description of the invention.
Claims (7)
1. automatically replying conversational system based on deep learning and intensified learning characterized by comprising
User interactive module, for receiving the problem of user inputs in conversational system interface information；
Session management module, for recording the active state of the user, the active state includes dialog history information, user
Evolution information and user emotion change information；
Customer analysis module is drawn a portrait for the user, is obtained for analyzing the registration information and active state of the user
User's portrait information, the user draw a portrait information for describing the personal characteristics of the user, the registration letter of the user
Breath includes userspersonal information；
Session module, for passing through language model life in conjunction with the portrait of the user in described problem information according to the user
At answer information；
Model learning module, the answer information for being generated according to the language model, by described in intensified learning technology innovation
Language model；
The training process of the language model is divided into offline supervised learning stage and online unsupervised intensified learning stage；
In the model learning module, in the offline supervised learning stage, the language is established by Recognition with Recurrent Neural Network
Model, specifically:
The problem of first layer input active user of the circulation neural network inputs information and the dialog history information；
The second layer of the circulation neural network is the master of the problem of active user extracted by encoder inputs information
Inscribe information, speech performance information, emotional information and the vector for exporting a regular length；
The third layer of the circulation neural network exports the answer information obtained by the information that decoder decodes the second layer.
2. automatically replying conversational system based on deep learning and intensified learning according to claim 1, which is characterized in that
In the model learning module, in the online unsupervised intensified learning stage, pass through intensified learning technology innovation language
Model:
According to the answer information that the language model generates, award coefficient is calculated；
It is carried out in calculating process by back-propagation algorithm, by the derivative in the award coefficient and the back-propagation algorithm
It is multiplied, generates new derivative, realize the update of the language model.
3. automatically replying conversational system based on deep learning and intensified learning according to claim 1, which is characterized in that
The problem of user inputs in conversational system interface information includes voice messaging, graphical information and text information.
4. automatically replying conversational system based on deep learning and intensified learning according to claim 1, which is characterized in that
User's portrait information uses keyword message, user that application program of mobile phone is used to believe in a search engine by user
The dialog text information of breath and user in conversational system interface obtains；
It includes passing through the user in a search engine that the user uses the acquisition pattern of keyword message in a search engine
Using the number of keyword, frequency of use and use the time；
The user includes using application program of mobile phone by the user using the acquisition pattern of application program of mobile phone information
Number, frequency of use and use the time.
5. automatically replying conversational system based on deep learning and intensified learning according to claim 1, which is characterized in that
User's portrait information includes label information and profile information.
6. automatically replying conversational system based on deep learning and intensified learning according to claim 1, which is characterized in that
The userspersonal information includes the gender of user, the age, address, speaking style, social relationships, emotional change mode and
Personal preference.
7. automatically replying conversational system based on deep learning and intensified learning according to claim 1, which is characterized in that
The decoder is realized by GRU algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610920931.2A CN106448670B (en) | 2016-10-21 | 2016-10-21 | Conversational system is automatically replied based on deep learning and intensified learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610920931.2A CN106448670B (en) | 2016-10-21 | 2016-10-21 | Conversational system is automatically replied based on deep learning and intensified learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106448670A CN106448670A (en) | 2017-02-22 |
| CN106448670B true CN106448670B (en) | 2019-11-19 |
Family
ID=58175697
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610920931.2A Active CN106448670B (en) | 2016-10-21 | 2016-10-21 | Conversational system is automatically replied based on deep learning and intensified learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106448670B (en) |
Families Citing this family (94)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106997375B (en) * | 2017-02-28 | 2020-08-18 | 浙江大学 | Customer service reply recommendation method based on deep learning |
| CN106991124A (en) * | 2017-03-02 | 2017-07-28 | 竹间智能科技（上海）有限公司 | Answer method and system based on how interactive situation editor |
| CN107066567B (en) * | 2017-04-05 | 2021-08-31 | 竹间智能科技（上海）有限公司 | Topic detection-based user portrait modeling method and system in text conversation |
| CN107025283A (en) * | 2017-04-05 | 2017-08-08 | 竹间智能科技（上海）有限公司 | The answer method and system of candidate answers sequence are carried out based on subscriber data |
| CN108733703A (en) * | 2017-04-20 | 2018-11-02 | 北京京东尚科信息技术有限公司 | The answer prediction technique and device of question answering system, electronic equipment, storage medium |
| US10679009B2 (en) * | 2017-04-21 | 2020-06-09 | Tata Consultancy Services Limited | System and method for belief based human-bot conversation |
| CN107180115A (en) * | 2017-06-28 | 2017-09-19 | 上海与德通讯技术有限公司 | The exchange method and system of robot |
| WO2019011824A1 (en) * | 2017-07-11 | 2019-01-17 | Koninklijke Philips N.V. | Multi-modal dialogue agent |
| CN109427334A (en) * | 2017-09-01 | 2019-03-05 | 王阅 | A kind of man-machine interaction method and system based on artificial intelligence |
| CN110069606A (en) * | 2017-10-26 | 2019-07-30 | 北京京东尚科信息技术有限公司 | Man-machine conversation's method, apparatus, electronic equipment and storage medium |
| CN107943998B (en) * | 2017-12-05 | 2021-05-11 | 竹间智能科技（上海）有限公司 | Man-machine conversation control system and method based on knowledge graph |
| CN108090443B (en) * | 2017-12-15 | 2020-09-22 | 华南理工大学 | Scene text detection method and system based on deep reinforcement learning |
| CN108038209A (en) * | 2017-12-18 | 2018-05-15 | 深圳前海微众银行股份有限公司 | Answer system of selection, device and computer-readable recording medium |
| CN108153879A (en) * | 2017-12-26 | 2018-06-12 | 爱因互动科技发展（北京）有限公司 | The method and device of recommendation information is provided a user by human-computer interaction |
| CN108182942B (en) * | 2017-12-28 | 2021-11-26 | 瑞芯微电子股份有限公司 | Method and device for supporting interaction of different virtual roles |
| CN108196862A (en) * | 2017-12-29 | 2018-06-22 | 北京声智科技有限公司 | The update device and method of speech ciphering equipment |
| CN108304489B (en) * | 2018-01-05 | 2021-12-28 | 广东工业大学 | Target-guided personalized dialogue method and system based on reinforcement learning network |
| CN108172209A (en) * | 2018-01-09 | 2018-06-15 | 上海大学 | Method for building voice idols |
| CN108256065B (en) * | 2018-01-16 | 2021-11-09 | 智言科技（深圳）有限公司 | Knowledge graph reasoning method based on relation detection and reinforcement learning |
| CN108563628A (en) * | 2018-03-07 | 2018-09-21 | 中山大学 | Talk with generation method based on the emotion of HRED and inside and outside memory network unit |
| CN108829797A (en) * | 2018-04-25 | 2018-11-16 | 苏州思必驰信息科技有限公司 | Multiple agent dialog strategy system constituting method and adaptive approach |
| JP6969491B2 (en) * | 2018-05-11 | 2021-11-24 | トヨタ自動車株式会社 | Voice dialogue system, voice dialogue method and program |
| EP3793783A1 (en) | 2018-05-18 | 2021-03-24 | Google LLC | System and methods for pixel based model predictive control |
| CN108681610B (en) * | 2018-05-28 | 2019-12-10 | 山东大学 | Generative multi-round chat dialogue method, system and computer-readable storage medium |
| CN108806671B (en) * | 2018-05-29 | 2019-06-28 | 杭州认识科技有限公司 | Semantic analysis, device and electronic equipment |
| CN108804611B (en) * | 2018-05-30 | 2021-11-19 | 浙江大学 | Dialog reply generation method and system based on self comment sequence learning |
| CN110619870B (en) * | 2018-06-04 | 2022-05-06 | 佛山市顺德区美的电热电器制造有限公司 | Man-machine conversation method and device, household appliance and computer storage medium |
| CN108959421B (en) * | 2018-06-08 | 2021-04-13 | 腾讯科技（深圳）有限公司 | Candidate reply evaluation device, query reply device, method thereof, and storage medium |
| CN108846073B (en) * | 2018-06-08 | 2022-02-15 | 合肥工业大学 | Personalized man-machine emotion conversation system |
| JP6964558B2 (en) * | 2018-06-22 | 2021-11-10 | 株式会社日立製作所 | Speech dialogue system and modeling device and its method |
| CN109002500B (en) * | 2018-06-29 | 2024-08-27 | 北京百度网讯科技有限公司 | Dialog generation method, apparatus, device and computer readable medium |
| CN109242109B (en) * | 2018-07-06 | 2022-05-10 | 网宿科技股份有限公司 | Deep model management method and server |
| CN110765338B (en) * | 2018-07-26 | 2024-11-08 | 北京搜狗科技发展有限公司 | A data processing method, a data processing device and a data processing device |
| CN109145101B (en) * | 2018-09-06 | 2021-05-25 | 北京京东尚科信息技术有限公司 | Man-machine conversation method, device and computer readable storage medium |
| US10909970B2 (en) * | 2018-09-19 | 2021-02-02 | Adobe Inc. | Utilizing a dynamic memory network to track digital dialog states and generate responses |
| CN109299237B (en) * | 2018-09-26 | 2020-06-16 | 苏州大学 | A Recurrent Network Human-Machine Dialogue Method Based on Actor-Critic Reinforcement Learning Algorithm |
| CN110970021B (en) * | 2018-09-30 | 2022-03-08 | 航天信息股份有限公司 | Question-answering control method, device and system |
| CN111048075A (en) * | 2018-10-11 | 2020-04-21 | 上海智臻智能网络科技股份有限公司 | Intelligent customer service system and intelligent customer service robot |
| CN111128135B (en) * | 2018-10-15 | 2022-09-27 | 珠海格力电器股份有限公司 | Voice communication method and device |
| CN109493166B (en) * | 2018-10-23 | 2021-12-28 | 深圳智能思创科技有限公司 | Construction method for task type dialogue system aiming at e-commerce shopping guide scene |
| CN109635080A (en) * | 2018-11-15 | 2019-04-16 | 上海指旺信息科技有限公司 | Acknowledgment strategy generation method and device |
| CN109347980B (en) * | 2018-11-23 | 2022-07-15 | 网易有道信息技术（北京）有限公司 | Method, medium, device and computing equipment for presenting and pushing information |
| CN111292733A (en) * | 2018-12-06 | 2020-06-16 | 阿里巴巴集团控股有限公司 | Voice interaction method and device |
| CN109670911A (en) * | 2018-12-14 | 2019-04-23 | 安徽仁昊智能科技有限公司 | A kind of online shopping platform automatically replies voice system |
| CN109635095A (en) * | 2018-12-17 | 2019-04-16 | 北京百度网讯科技有限公司 | Method and apparatus for optimizing dialog model |
| CN109785289B (en) * | 2018-12-18 | 2021-07-20 | 中国科学院深圳先进技术研究院 | A transmission line defect detection method, system and electronic device |
| CN109783704B (en) * | 2019-01-03 | 2021-02-02 | 中国科学院自动化研究所 | Man-machine hybrid response method, system and device |
| CN109800294B (en) * | 2019-01-08 | 2020-10-13 | 中国科学院自动化研究所 | Autonomous evolution intelligent dialogue method, system and device based on physical environment game |
| CN111552784A (en) * | 2019-02-12 | 2020-08-18 | 厦门邑通软件科技有限公司 | Man-machine conversation method based on ABC communication rule |
| CN109754810A (en) * | 2019-02-21 | 2019-05-14 | 珠海格力电器股份有限公司 | Voice control method and device, storage medium and air conditioner |
| CN110263131B (en) * | 2019-03-05 | 2023-07-04 | 腾讯科技（深圳）有限公司 | Reply information generation method, device and storage medium |
| CN109992669B (en) * | 2019-04-08 | 2020-12-15 | 浙江大学 | A Keyword Question Answering Method Based on Language Model and Reinforcement Learning |
| CN110417637A (en) * | 2019-04-26 | 2019-11-05 | 成海林 | AI artificial intelligence augmentative communication technology |
| CN110597968A (en) * | 2019-04-28 | 2019-12-20 | 河北省讯飞人工智能研究院 | Reply selection method and device |
| CN110211572B (en) * | 2019-05-14 | 2021-12-10 | 北京来也网络科技有限公司 | Dialogue control method and device based on reinforcement learning |
| CN110188177A (en) * | 2019-05-28 | 2019-08-30 | 北京搜狗科技发展有限公司 | Talk with generation method and device |
| CN118379989A (en) * | 2019-05-31 | 2024-07-23 | 华为技术有限公司 | Speech recognition method, apparatus, device and computer readable storage medium |
| CN110195660B (en) * | 2019-06-19 | 2020-04-21 | 南京航空航天大学 | Aero-engine control device based on deep Q-learning |
| CN110569344B (en) * | 2019-08-22 | 2023-06-02 | 创新先进技术有限公司 | Method and device for determining standard question corresponding to dialogue text |
| CN110688468B (en) * | 2019-08-28 | 2021-06-25 | 北京三快在线科技有限公司 | Method and device for outputting response message, electronic equipment and readable storage medium |
| US12014823B2 (en) | 2019-08-30 | 2024-06-18 | GE Precision Healthcare LLC | Methods and systems for computer-aided diagnosis with deep learning models |
| CN110674276B (en) * | 2019-09-23 | 2024-08-16 | 深圳前海微众银行股份有限公司 | Robot self-learning method, robot terminal, device and readable storage medium |
| CN110737764B (en) * | 2019-10-24 | 2023-07-07 | 西北工业大学 | Personalized dialogue content generation method |
| CN110851581B (en) * | 2019-11-19 | 2022-11-11 | 东软集团股份有限公司 | Model parameter determination method, device, equipment and storage medium |
| CN111177330A (en) * | 2019-11-20 | 2020-05-19 | 国网江苏省电力有限公司电力科学研究院 | A kind of personal intelligent assistant system and data processing method |
| CN111026843B (en) * | 2019-12-02 | 2023-03-14 | 北京智乐瑟维科技有限公司 | Artificial intelligent voice outbound method, system and storage medium |
| CN110674281B (en) * | 2019-12-05 | 2020-05-29 | 北京百度网讯科技有限公司 | Man-machine conversation and man-machine conversation model acquisition method, device and storage medium |
| CN111161708A (en) * | 2019-12-31 | 2020-05-15 | 中国银行股份有限公司 | Voice information processing method and device |
| CN111833854B (en) * | 2020-01-08 | 2024-07-16 | 北京嘀嘀无限科技发展有限公司 | Man-machine interaction method, terminal and computer readable storage medium |
| CN111259132A (en) * | 2020-01-16 | 2020-06-09 | 中国平安财产保险股份有限公司 | Method, apparatus, computer equipment and storage medium for speech recommendation |
| CN111314451A (en) * | 2020-02-07 | 2020-06-19 | 普强时代(珠海横琴)信息技术有限公司 | Language processing system based on cloud computing application |
| CN111324708A (en) * | 2020-02-07 | 2020-06-23 | 普强时代(珠海横琴)信息技术有限公司 | Natural language processing system based on human-computer interaction |
| JP7434978B2 (en) * | 2020-02-10 | 2024-02-21 | トヨタ自動車株式会社 | Information processing device, information processing method, and program |
| CN111522936B (en) * | 2020-04-24 | 2023-08-04 | 上海智臻智能网络科技股份有限公司 | A method, device, and electronic device for generating intelligent customer service dialogue reply including emotion |
| US11455340B2 (en) | 2020-06-25 | 2022-09-27 | Optum Services (Ireland) Limited | Predictive prompt generation by an automated prompt system |
| CN114005439B (en) * | 2020-07-27 | 2025-06-13 | 北京中科金得助智能科技有限公司 | Method, device and storage medium for determining speech technique |
| CN114065976A (en) * | 2020-07-29 | 2022-02-18 | 宝马股份公司 | Method and system for car service reservation based on natural language input |
| US11245648B1 (en) | 2020-07-31 | 2022-02-08 | International Business Machines Corporation | Cognitive management of context switching for multiple-round dialogues |
| CN112100354B (en) * | 2020-09-16 | 2023-07-25 | 北京奇艺世纪科技有限公司 | Man-machine conversation method, device, equipment and storage medium |
| CN112735374B (en) * | 2020-12-29 | 2023-01-06 | 北京三快在线科技有限公司 | Automatic voice interaction method and device |
| CN113781190A (en) * | 2021-01-13 | 2021-12-10 | 北京沃东天骏信息技术有限公司 | Bill data processing method, system, computer system and medium |
| CN115129829A (en) * | 2021-03-26 | 2022-09-30 | 阿里巴巴新加坡控股有限公司 | Question-answer calculation method, server and storage medium |
| CN112951215B (en) * | 2021-04-27 | 2024-05-07 | 平安科技（深圳）有限公司 | Voice intelligent customer service answering method and device and computer equipment |
| CN114297450A (en) * | 2021-12-31 | 2022-04-08 | 山东科技大学 | Deep learning-based dialogue system and dialogue method thereof |
| CN114297364A (en) * | 2021-12-31 | 2022-04-08 | 山东科技大学 | Automatic reply dialogue system based on deep learning and reinforcement learning |
| CN114579820A (en) * | 2022-03-21 | 2022-06-03 | 徐涛 | Method and device for extracting context features in man-machine conversation |
| CN115617973B (en) * | 2022-12-14 | 2023-03-21 | 安徽数分智能科技有限公司 | Information acquisition method based on intelligent data processing |
| CN116774891B (en) * | 2023-04-27 | 2024-08-02 | 北京鹅厂科技有限公司 | Method and device for applying artificial intelligence |
| CN116595141A (en) * | 2023-05-11 | 2023-08-15 | 中国平安财产保险股份有限公司 | Multi-round dialogue method, device, computer equipment and storage medium |
| CN116681087B (en) * | 2023-07-25 | 2023-10-10 | 云南师范大学 | An automatic question generation method based on multi-stage timing and semantic information enhancement |
| CN116932726B (en) * | 2023-08-04 | 2024-05-10 | 重庆邮电大学 | Open domain dialogue generation method based on controllable multi-space feature decoupling |
| CN117171443A (en) * | 2023-09-21 | 2023-12-05 | 支付宝(杭州)信息技术有限公司 | Model training method and device, content generation method and device |
| CN117235241A (en) * | 2023-11-15 | 2023-12-15 | 安徽省立医院(中国科学技术大学附属第一医院) | A human-computer interaction method for hypertension consultation and follow-up scenarios |
| CN118737151A (en) * | 2024-06-18 | 2024-10-01 | 中国长江三峡集团有限公司 | A method, device, equipment and storage medium for generating key points of conference conversation records |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20150065523A (en) * | 2013-12-05 | 2015-06-15 | 포항공과대학교 산학협력단 | Method and apparatus for providing counseling dialogue using counseling information |
| CN105138671A (en) * | 2015-09-07 | 2015-12-09 | 百度在线网络技术（北京）有限公司 | Human-computer interaction guiding method and device based on artificial intelligence |
| CN105183848A (en) * | 2015-09-07 | 2015-12-23 | 百度在线网络技术（北京）有限公司 | Human-computer chatting method and device based on artificial intelligence |
| WO2016037311A1 (en) * | 2014-09-09 | 2016-03-17 | Microsoft Technology Licensing, Llc | Variable-component deep neural network for robust speech recognition |
| CN105788593A (en) * | 2016-02-29 | 2016-07-20 | 中国科学院声学研究所 | Method and system for generating dialogue strategy |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6275806B1 (en) * | 1999-08-31 | 2001-08-14 | Andersen Consulting, Llp | System method and article of manufacture for detecting emotion in voice signals by utilizing statistics for voice signal parameters |
| US8793119B2 (en) * | 2009-07-13 | 2014-07-29 | At&T Intellectual Property I, L.P. | System and method for generating manually designed and automatically optimized spoken dialog systems |
-
2016
- 2016-10-21 CN CN201610920931.2A patent/CN106448670B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20150065523A (en) * | 2013-12-05 | 2015-06-15 | 포항공과대학교 산학협력단 | Method and apparatus for providing counseling dialogue using counseling information |
| WO2016037311A1 (en) * | 2014-09-09 | 2016-03-17 | Microsoft Technology Licensing, Llc | Variable-component deep neural network for robust speech recognition |
| CN105138671A (en) * | 2015-09-07 | 2015-12-09 | 百度在线网络技术（北京）有限公司 | Human-computer interaction guiding method and device based on artificial intelligence |
| CN105183848A (en) * | 2015-09-07 | 2015-12-23 | 百度在线网络技术（北京）有限公司 | Human-computer chatting method and device based on artificial intelligence |
| CN105788593A (en) * | 2016-02-29 | 2016-07-20 | 中国科学院声学研究所 | Method and system for generating dialogue strategy |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106448670A (en) | 2017-02-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106448670B (en) | Conversational system is automatically replied based on deep learning and intensified learning | |
| Shou et al. | Conversational emotion recognition studies based on graph convolutional neural networks and a dependent syntactic analysis | |
| CN111966800B (en) | Emotion dialogue generation method and device and emotion dialogue model training method and device | |
| Nie et al. | Long dialogue emotion detection based on commonsense knowledge graph guidance | |
| Mai et al. | Locally confined modality fusion network with a global perspective for multimodal human affective computing | |
| Nie et al. | I-GCN: Incremental graph convolution network for conversation emotion detection | |
| CN110347792A (en) | Talk with generation method and device, storage medium, electronic equipment | |
| CN108595436B (en) | Method and system for generating emotional dialogue content and storage medium | |
| CN108874972A (en) | A kind of more wheel emotion dialogue methods based on deep learning | |
| CN109460479A (en) | A kind of prediction technique based on reason map, device and system | |
| CN109844743A (en) | Response is generated in automatic chatting | |
| CN113987179A (en) | Dialogue emotion recognition network model, construction method, electronic device and storage medium based on knowledge enhancement and retrospective loss | |
| CN111145721A (en) | Personalized prompt language generation method, device and equipment | |
| CN112214585B (en) | Reply message generation method, system, computer device and storage medium | |
| CN114911932A (en) | Heterogeneous graph structure multi-conversation person emotion analysis method based on theme semantic enhancement | |
| Kao et al. | Model of multi-turn dialogue in emotional chatbot | |
| WO2021135457A1 (en) | Recurrent neural network-based emotion recognition method, apparatus, and storage medium | |
| CN109597876A (en) | A kind of more wheels dialogue answer preference pattern and its method based on intensified learning | |
| CN116991982B (en) | Interactive dialogue method, device, equipment and storage medium based on artificial intelligence | |
| CN114373443B (en) | Speech synthesis method and device, computing device, storage medium and program product | |
| CN112183106A (en) | Semantic understanding method and device based on phoneme association and deep learning | |
| CN114386426B (en) | Gold medal speaking skill recommendation method and device based on multivariate semantic fusion | |
| Ai et al. | A two-stage multimodal emotion recognition model based on graph contrastive learning | |
| CN113486174A (en) | Model training, reading understanding method and device, electronic equipment and storage medium | |
| CN114860908B (en) | A task-based dialogue state tracking method integrating slot association and semantic association |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |