DE102018127010A1 - Method, computer program and device for automated computer-based communication processing - Google Patents

Abstract

The invention relates to a method for automated, computer-based communication processing of visual, auditory and / or textual communication of at least one first participant with at least one computer-based communication assistant as the second participant. The invention also relates to a computer program, a computer-readable storage medium and an apparatus for performing the method.

Description

Field of the Invention

The invention relates to a method for automated, computer-based processing of a communication of at least one first communication subscriber, in particular a natural person, with at least one computer-based communication assistant as the second communication subscriber. The invention also relates to a computer program and a device for carrying out the method.

Background of the Invention

With the help of speech recognition systems, the possibility should be created that people (natural persons) can use language as an input form on machines. Speech recognition systems of this type generally serve as a human-machine interface, in which input to the machine can be made using natural language or reduced speech systems derived therefrom. The spoken language of the respective person or natural person is provided as an input parameter of the speech recognition machine, which then recognizes the individual spoken words with the aid of a provided context and generates an input to the machine in digital form. At the same time, the reverse is also conceivable, in which the machine generates corresponding system reactions, which are then output to the respective person with the aid of a speech output in the form of natural language.

From the DE 10 2009 034 069 A1 An operating device for a motor vehicle is known which has a display for displaying variable information. The direction of view of the motor vehicle driver can be determined with the aid of a line of sight detection unit, wherein the operating device can be operated with the aid of acoustic commands if the line of sight of the motor vehicle driver is directed at the display.

From the US 2007/0215745 A1 A system for monitoring aircraft activities is known, wherein the voice communication between the aircraft and the ATC (Air Traffic Controller) is recorded and analyzed using voice recognition. The speech information recognized from the speech analysis is then compared and correlated together with the data required for monitoring the airspace.

From the DE 10 2008 017 957 A1 a system for eavesdropping in the field of healthcare is known, in which voice communications between different people within a hospital are intercepted and analyzed for specific keywords. If the corresponding keywords are recognized, a measure can already be carried out automatically by the system before the persons concerned initiate this.

From the EP 2 341 494 a system is also known in which the voice communication between the pilot and the ATC is intercepted and analyzed. If aircraft IDs are recognized in voice communication, the relevant flying objects are automatically shown on the display of the controller.

From the EP 0 613 111 A1 An airport surveillance system is known in which various sensors for data acquisition are to be used. The voice commands between the ATC and the pilot are listened to and analyzed in order to be able to derive corresponding data at the airport for the aircraft concerned.

From the US 2005/0283475 A1 a method and a device for displaying information, in particular in call centers, are known. The voice communication between the call center agent and the caller is analyzed so that the agent can be provided with additional information quickly.

From the DE 196 19 015 A1 A method and an arrangement for traffic monitoring at airports is known in which the position of the aircraft is to be determined with the aid of direction-finding systems on the basis of the radio signals transmitted. In order to obtain a quick assignment of the position to the respective aircraft, it is proposed that the radio communication should be analyzed using speech technology and the aircraft identifier should be recognized, which results in a quick assignment between the aircraft and the detected parking position.

From the JP 06342061 Speech recognition is known, in which corresponding aircraft identifiers are also to be recorded and then used for differentiation and data selection.

From the DE 10 2007 018 327 C5 Finally, a store scale with a microphone device for listening in on a conversation between the seller and the customer is known, the recorded conversations being analyzed and further information being displayed when appropriate keywords are recognized.

From the US 2008/0091406 A1 , JP 2005-173134 A , DE 60 2004 002 230 T2 and US 2005/0137868 A1 Methods for speech recognition are known in which a content-related speech context is derived from the recognized speech information, in order to improve speech recognition based on the speech context.

Such speech recognition systems are very tied to the language context they can recognize and the device they are supposed to control. Speech recognition is often very static and cannot adapt to changing conditions in communication. For example, the communication in the known speech recognition systems generally consists only of speech commands and actions, it being predetermined from the start what should be recognized or output. So there is no way to process words or information that are not predefined by the system. More complex forms of communication, which include, for example, several simultaneous questions or commands, a change of subject, a free dialogue or even conversations with several participants, can certainly not be recognized with the conventional systems, let alone processed.

It is therefore an object of the present invention to provide an improved method and an improved device with which the recognition of communication of a voice-based human-machine interface can be improved. It is also an object of the present invention to propose an improved method and an improved speech recognition system which, in particular, enable dynamic recognition of the communication and / or flexible decision-making and do not remain purely static like the methods and systems known in the prior art.

The object is achieved according to the invention in the method and the device according to the independent claims. Advantageous refinements are the subject of the dependent claims.

Description of the invention

The inventor has recognized, among other things, that known speech recognition systems only enable the spoken words to be recognized, but not the entire communication environment. If you consider that only approx. 10% of a communication represents the pure spoken words, approx. 90% remain unused for recognizing the communication that is directed to the respective device. However, these unused components of the communication can contain a difference in the intention of the speaker, so that today's speech recognition systems semantically give the wrong or at least not the optimal response despite the correct recognition of the words.

According to claim 1, therefore, a method for, preferably automated, computer-based communication processing of a visual, auditory and / or textual communication of a first participant with at least one computer-based communication assistant is proposed as a second participant, wherein several participants can also be involved in the overall communication.

The first participant is preferably a natural person. It is also possible for several natural persons to communicate using a computer-based communication assistant. It is also conceivable that only one natural person is involved in the communication. As the term is used here, the participation of the “natural person” also includes the possibility that the person communicates with the communication assistant only indirectly, for example via communication means such as a telephone, a sound and / or video recording. However, the first participant can also be a computer, in particular another computer-based communication assistant. Of course, several computer-based communication assistants can also be present.

In a first procedural step, the computer-based communication assistant is initially provided with verbal and non-verbal communication components of a received visual, auditory and / or textual communication. In particular, the, preferably entire, visual, auditory and / or textual communication divided into a plurality of individual verbal and non-verbal communication components.

As the terms are used here, verbal communication components are in particular the words used in the communication, while the non-verbal communication components are those communication components that are not codified in the form of speech and in particular also convey information about a property of the participants or the communication can. Such non-verbal communication components can be, for example, emotions, gestures and facial expressions.

The reception of the visual, auditory and / or textual communication can take place in particular by digital means or include a conversion of the visual, auditory and / or textual communication into digital data recognizable by the communication assistant and sees for example in the case of visual and / or auditory communication a video or audio file or stream. However, textual communication (or a mixed form of audio and / or video) is also conceivable, with textual communication this being able to be made available, for example, as a text stream. Textual communication can be, for example, chat communication.

The verbal and non-verbal communication components for the computer-based communication assistant can be provided in various ways. In a preferred embodiment, the visual, auditory and / or textual communication is initially received by the computer-based communication assistant. Then both verbal and non-verbal communication components of the received visual, auditory and / or textual communication are recognized by the computer-based communication assistant and made available for further processing. This can take place, for example, with the aid of a communication recognition module of the communication assistant, which for this purpose analyzes the visual, auditory and / or textual communication and breaks it down or divides it into individual recognizable verbal and non-verbal communication components in accordance with the above statements.

In a further preferred embodiment it is provided that verbal and non-verbal communication components of a received visual, auditory and / or textual communication are provided to the computer-based communication assistant via an interface on the computer-based communication assistant which can be recognized by the computer-based communication assistant. The provision can also be made in that the computer-based communication assistant receives visual, auditory and / or textual communication via an interface, which according to the above statements already contains a plurality of individual verbal and non-verbal communication components or is divided into those by the computer-based communication assistant can be recognized. These embodiments have the advantage that the reception of the visual, auditory and / or textual communication or its subdivision into verbal and non-verbal communication components recognizable by the communication assistant can be implemented independently of the computer-based communication assistant, for example with the aid of an external camera, an external microphone or a smartphone. In addition to a constructive simplification of the communication assistant, this has the particular advantage that the method according to the invention can be implemented flexibly, in that the required technical components can be combined in a modular manner and, in particular, can be exchanged individually.

Of course, any combination of the aforementioned embodiments is also possible with regard to the provision of the verbal and non-verbal communication components.

The computer-based communication assistant also has at least one digital communication context that was previously made available to it. This digital communication context contains a plurality of communication contents to be recognized, each communication content each containing a combination of different concrete verbal and / or non-verbal communication components, and each communication content being assigned to at least one corresponding predetermined system reaction of the computer-based communication assistant. A communication content of the digital communication context thus corresponds to an n-tuple with n as the number of different communication components that were recognized, the n-tuple mapping to at least one predefined system reaction. The computer-based communication assistant preferably has several such digital communication contexts.

By comparing the recognized verbal and non-verbal communication components of the visual, auditory and / or textual communication with the communication content of the digital Communication context, the communication content (s) can be determined which contains a match, in particular the highest match, with the recognized verbal and non-verbal communication components of the communication to be recognized. If the corresponding communication content has been recognized, the system reaction assigned to this communication content can be determined and then carried out by the computer-based communication assistant.

It is of course also possible here to determine a plurality of communication contents of one or more communication contexts, each of which is assigned to a system reaction. In other words, the communication processing according to the invention can also lead to several system reactions. It is also possible for several communication contents of one or more communication contexts to be determined, which are assigned in combination to a common system reaction and in this way form an “overall decision”, so to speak. It is also possible that a communication content is ascertained which is assigned to several different system reactions. Finally, any combination of these variants is also possible. This has the advantage that complex communications, which for example can also include several dialogues, can be processed very precisely and / or very comprehensively with the method according to the invention.

Such a system reaction can include, for example, a visual, auditory and / or textual communication output in order to provide the subscriber communicating with the computer-based communication assistant or a system connected to the communication assistant (e.g. a computer) with appropriate feedback. However, it is also conceivable that the system reaction includes an internal system reaction, such as performing an action outside of the communication of the respective participant. This also includes operating a device or ordering an item and storing data. In preferred embodiments, the execution of the system reaction associated with the determined communication content by the computer-based communication assistant means that at least one or more further communication contexts are provided accordingly for the computer-based communication assistant. For this purpose, the recognized communication content can expediently refer directly to such a further communication context to be included. Another possibility is that, for example, the number and / or the degree of correspondence between the recognized verbal and non-verbal communication components and the communication components of the recognized communication content means that the communication with the initially provided communication context cannot or cannot be processed sufficiently or alternative processing is also possible, so that further communication context is required for clarification or for more precise communication processing. The further communication context is then integrated into the process in accordance with the above description. This solution, also referred to below as “context-in-context”, of the method according to the invention makes it possible to branch to systematic reactions to any communication contexts and thereby dynamically change the course of the communication. This increases the flexibility of the system, since communications with variable content, in particular content that is not predetermined by the system, can also be managed and processed reliably.

Finally, it is also conceivable that the system reaction includes doing nothing, which is also a reaction of the computer-based communication assistant.

With the aid of the present method, it is thus possible to take into account both verbal and non-verbal communication components when communicating with a computer-based communication assistant, as a result of which the recognition of what is meant by the natural person by means of the communication addressed to the communication assistant can be greatly improved. In addition, the structure of the provided digital communication context enables a modular and dynamic structure that can be easily supplemented, changed or exchanged, whereby in particular the communication contexts and system reactions can each be at least partially in the form of separate files or programs.

It also becomes possible to generate different system reactions for one and the same verbal content of a verbal communication component if the contents of the non-verbal communication components change. For example, it makes a difference whether the spoken words or sentences are pronounced quickly or slowly, loudly or quietly, so that although the recognized words of the verbal communication component always remain the same, the non-verbal communication components, such as mood or emotion, each have a different content Change what can be considered within the digital communication context. As a result, different system responses to changing non-verbal content can be achieved even with the same linguistic content Specify communication components. This ensures that the computer-based communication assistant reacts more humanly and reacts to the other components besides the verbal communication component.

In one embodiment, the computer-based communication assistant is provided with dynamically corresponding digital communication contexts before or during the communication, these having been selected beforehand as a function of at least one possible communication parameter from a multiplicity of digital communication contexts. Such a communication parameter can result, for example, from the type of communication established (for example, visually, auditory and / or textually). Such a communication parameter can also be based on the person or the location of the participant, or the time, the topic of the communication to be started, the number or concern of the participants and the like. Of course, a communication context can also be selected on the basis of several communication parameters, or several communication parameters can lead to the selection of several communication contexts, which are provided at the same time, for example, in order to cover a larger expected communication spectrum from the beginning. But the reverse case is also conceivable, in that communication contexts are excluded or not integrated from the outset, so that communication can be carried out very quickly and effectively. In principle, a communication parameter in the sense of the present invention can therefore be everything that is already established as a framework condition in advance of the communication or that changes or occurs dynamically during the communication (for example a catastrophe warning) and influences the communication and / or its targeted processing, but in particular is not part of the communication itself.

In a further alternative or additional embodiment it is provided that, depending on at least one recognized verbal and / or non-verbal communication component or its content of the received visual, auditory and / or textual communication, one or more digital communication contexts are selected from and a large number of digital communication contexts then be provided to the computer-based communication assistant accordingly. It is thus conceivable that, after the mood of the first participant has been recognized, that is to say in particular in the case of a natural person, appropriate digital communication contexts are formed in order to take the recognized mood and thus the recognized non-verbal communication component into account accordingly. If, for example, it has been recognized that the participant has a sad mood, then different digital communication contexts can be made available than in the case of a recognized happy mood. This allows the response of the computer-based communication assistant to be fine-tuned to the overall situation. Of course, it is also possible for one or more communication contexts to be assigned to certain keywords, which, provided the words have been recognized, are made available to the communication assistant. In this respect, this embodiment differs from the above communication parameter, because the verbal and / or non-verbal communication components used for the selection of the communication contexts can at the same time also represent a component of the communication itself. In this way, additional contexts can be integrated into the method at any time, which change the initially planned communication during the runtime and thereby contribute to the free modeling of a non-predetermined communication process. In particular, multilingual dialogues or multilingual communications can be recognized and processed in this way, which was previously not possible with conventional methods.

In a further development, so-called qualifiers are determined or defined for the individual verbal and non-verbal communication components, under which special contents of the communication or communication components can be categorized. The qualifiers make it possible to categorize the individual verbal and non-verbal communication components and process them separately.

For example, in the case of a verbal communication component, this can be assigned to a KeywordQualifier, a SemanticQualifier or an AssociationQualifier. A KeywordQualifier represents a category in which one or more keywords have been specified or recognized that require special processing. In this way, content can also be mapped that may not be 100% specified, but is processed in an abstract form using a corresponding keyword. A KeywordQualifier can, for example, comprise or consist of a keyword or several keywords, or a combination thereof, which, if recognized within a verbal communication component, cause one or more unrecognized or only partially recognized words (hereinafter also referred to as “wildcards “Designated), which precede or follow the keyword or keywords and / or which are arranged between them, are processed. In this way, the system according to the invention can also use words that it cannot recognize as such, but under certain circumstances for determining or Processing of the communication content would have to recognize, for example because these words themselves at least partially contain a, possibly separate, communication component, lead to a meaningful or purposeful further processing in the system reaction.

The semantic qualifier as a verbal communication component represents a category that includes a standardization of feeling, thinking and / or trading based on the recognized words. Similar to the KeywordQualifier, the SemanticQualifier can comprise or consist of two or more linguistic elements such as keywords or a combination thereof, between which wildcards, i.e. the words to be recognized or processed are arranged or which include the words to be recognized or processed. In this sense, the beginning and / or end of a sentence can also be such linguistic elements.

The semantic qualifier can, for example, include or consist of linguistic utterances that have a certain standardized meaning, which, if recognized within a verbal communication component, cause a, in particular likewise standardized, system reaction. This has the advantage that e.g. a previously unknown decision that was made within a communication can also be recognized: "Today we decide that we will make a purchase for the following reasons A, B and C." If "decide" and the element "sentence end" together are defined as semantic qualifiers, the system can recognize the decision, from "decide" to the end of the sentence, without having to recognize the content of the decision and still react to it, e.g. initiate a saving of the decision or further communication.

The AssociationQualifier as a further possible verbal communication component represents a connection to another communication, another communication context and / or another system reaction based on one or more recognized keywords or combinations thereof. For example, spoken words such as "This is how ... "Or" ... the same situation as ... "are suitable AssociationQualifiers. In this way, complex communications or communication content can be recognized more quickly and processed more effectively, and / or the most suitable system response can be selected on the basis of “experiences” with the system.

The qualifiers or categories can be recognized from the communication itself. But they also serve to define the digital communication context. If, for example, verbal communication components in the form of a keyword qualifier and a semantic qualifier were recognized, the comparison within the digital communication context is also carried out with these qualifiers, whereby these must be defined in the individual communication content to be recognized. Only if a corresponding communication content also contains a KeywordQualifier and a SemanticQualifier with the corresponding content, as recognized, can a matching be positive and the corresponding assigned system reaction can be selected. A special feature of the verbal qualifiers such as KeywordQualifier, SemanticQualifier and AssociationQualifier is that the key words themselves can also be part of the communication or part of it. In other words, the verbal qualifiers can perform a double function in the context of the determination or processing of the communication content, on the one hand they can be assigned to a communication component and at the same time, with the help of them, for example, that unknown or non-predefined linguistic statements (wildcards) can be assigned to a communication component are assignable. This can be the same communication component or different communication components.

For the non-verbal communication components, for example, a ComQualifier, a MemberQualifier, an EmotionQualifier, a MimicQualifier, a GestureQualifier, a BodyQualifier, a BioQualifier and a PropertyQualifier can be provided.

The ComQualifier can contain certain communication parameters of the communication, which can arise or change particularly dynamically during communication, such as changing location, changing participants, predetermined intention, predetermined goals, entertainment canceled, external events such as an earthquake warning or a trading stop of a share and the same. The MemberQualifier can contain properties of the participants, such as name and / or password, positions in the company and the like. This information can, for example, also be predetermined in the communication or the communication context itself. However, they can also result from the communication itself and be available for a login procedure, for example.

An EmotionQualifier contains at least one emotional parameter or characteristic of the first participant, which in this case is a natural person, and can express, for example, fear, anger, excitement, pity, etc. Each emotion that can be derived from a communication can be mapped as a non-verbal communication component via such an EmotionQualifier.

Using a MimicQualifier, visible properties of the face of the participant can be mapped, for example, if it is a visual communication. Visible movements of the face surface, such as smiles or other facial expressions, can be categorized using this MimicQualifier and used as a communication component. With the GestureQualifier, gestures of the first participant are recognized, such as pointing to something or a corresponding hand movement.

The BodyQualifier is a visible movement of the first participant, such as crossed arms, which is then listed as a non-verbal communication component in this category. With the BioQualifier, vital parameters and / or vital properties of the first participant, if present, are defined, while acoustic parameters and / or acoustic properties of the auditory communication are stored using the PropertyQualifier. This can be loud, quiet, fast, slow, etc., for example. The content of the PropertyQualifier can also be defined for "silence", if nothing has been said, which also represents a type of communication.

With the aid of the qualifiers according to the invention, a very differentiated decoding of communication content, which in particular leads to improved communication, can thus also be achieved in multifactorial "human" communications, which typically consist predominantly of body language elements such as facial expressions and gestures and only a small part of factual content or more precise decision making of the system. However, the process does not have to make a decision, it can only lead, moderate or log and primarily serves to dynamically control communication. Of course, the qualifiers are not limited to the above examples. Rather, the qualifiers are a general concept of the communication processing according to the invention, for which further design options in the form of suitable qualifiers are readily apparent to the person skilled in the art on the basis of the present description.

In a further advantageous embodiment, the recognized verbal and non-verbal communication components are stored in a database or in a data memory as a combination together with the system reaction carried out. This can also include the storage of at least parts or all of the original communication data or the original data of the communication recognition. Such stored combinations of the communication components can then be used, for example, as a communication context for visual, auditory and / or textual communications received in the future. This allows the context to be expanded dynamically regardless of the communication contexts provided, and the system can “learn”.

The object is also achieved with a computer program with program code means for carrying out the method described above and a computer-readable storage medium which includes the computer program.

Finally, the object is also achieved with a computer-based communication assistant for automated, computer-based communication processing of visual, auditory and / or textual communication of at least one first participant and the computer-based communication assistant as at least second participant, which is set up to carry out the method according to the invention.

Preferred and advantageous configurations of the computer program, the computer-readable storage medium and the computer-based communication assistant correspond, if applicable, to those of the method according to the invention. Features that are disclosed above and below for the method can therefore also relate to the computer program, the computer-readable storage medium and / or the computer-based communication assistant and vice versa.

Figure list

• 1 eine schematische Darstellung einer Kommunikation und einer Vorrichtung zu deren Verarbeitung;
• 2 eine schematische Darstellung eines digitalen Kommunikationskontextes;
• 3 eine schematische Darstellung einer weiteren Kommunikation und einer weiteren Vorrichtung zu deren Verarbeitung.

The invention is explained in more detail by way of example with reference to the accompanying figures. Show it:

• 1 a schematic representation of a communication and a device for processing it;
• 2nd a schematic representation of a digital communication context;
• 3rd a schematic representation of another communication and another device for processing it.

Detailed description of exemplary embodiments

1 shows schematically the essential elements that are involved in the inventive method for automated computer-based communication processing. In doing so, a natural person wants 1 in a visual, auditory and / or textual way with a computer-based communication assistant 10th communicate. Based on the communication carried out by the natural person 1 , the computer-based communication assistant 10th thereby a reaction towards the natural person 1 generate, which also represents a form of communication and thereby on the previously indicated communication of the natural person 1 responds.

The computer-based communication assistant 10th has a communication recognition module for this purpose 11 which receives visual, auditory and / or textual communication via an entrance. The communication detection module 11 now analyzes the visual, auditory and / or textual communication and breaks it down into recognizable verbal and non-verbal communication components 12th . These verbal and non-verbal communication components 12th are now an evaluation module 13 provided for further comparison with the context. The evaluation module 13 receives besides the verbal and non-verbal communication components 12th also a digital communication context 14 which is the evaluation module 13 from a database 15 provided. The evaluation module 13 also multiple communication contexts 14 be provided at the same time.

The evaluation module 13 now compares the verbal and non-verbal communication components received and their content with the communication content of the respective communication context 14 and tries that communication content of a communication context 14 to be determined in which the recognized verbal and non-verbal communication components have the greatest possible correspondence in the corresponding communication content.

Such communication content, as for example in 2nd of a digital communication context, shows communication content 20th in the form of an n-tuple, each containing a number n of verbal and non-verbal communication components B ₁ to B _n having. Every component of communication B ₁ to B _n corresponds to a verbal or non-verbal component of the communication and can thus represent a corresponding categorization of the respective component. Any verbal or non-verbal part of communication B ₁ to B _n contains a corresponding value or content that is compared with the recognized verbal and non-verbal communication components of the received visual, auditory and / or textual communication.

Each n-tuple 20 is a system reaction R _m assigned so that when identifying the corresponding communication content 20th , which corresponds to the respective n-tuple, then this communication content 20th assigned system reaction R _m is determined. Even if it is in the 2nd is not shown, any n-tuple 20th can of course also be assigned to several system reactions or preprocessed before the system reaction is called.

Coming back on 1 is then by the evaluation module 13 the determined system response to an execution module 16 transmitted, through which then the corresponding system reaction R _m is performed. This can be, for example, linguistic feedback from the natural person 1 represent or contain other reactions, such as ordering an article, calling up a file, such as another communication context, or simply doing nothing.

• („Hallo“, leise, freundlich) → (Reaktion „Hallo“ zurück)
• („Hallo“, laut, verärgert) → (Reaktion „Was kann ich für Sie tun“)

In the following, the difference to conventional language processing will be briefly illustrated using a simple context file. The natural person 1 opens the communication with the word "Hello" and expects a corresponding reaction from the computer-based communication assistant. In the digital communication context there are two different communication contents for the word "hello" 20th Are defined:

• ("Hello", quiet, friendly) → (reaction "Hello" back)
• ("Hello", loud, annoyed) → (reaction "What can I do for you")

That by the person 1 indexed communication does not differ in terms of the changed wording. However, it makes a difference to the system response whether the communication is quiet and in a friendly tone or whether the communication is visibly loud and annoyed. The parameter quiet or loud is used as a non-verbal communication component B _n recognized and sorted into the category "PropertyQualifier". The communication component regarding friendly or upset is defined as a category in the "MimicQualifier".

MimicQualifier(Eigenschaft)
    Wenn Eigenschaft(freundlich) Dann
                Speichern(Mimik, Eigenschaft)
    Wenn Eigenschaft(verärgert) Dann
                Speichern(Mimik, Eigenschaft)
PropertyQualifier(Eigenschaft)
    Wenn Eigenschaft(Leise) Dann
                Speichern(Property, Eigenschaft)
PropertyQualifier(Eigenschaft)
    Wenn Eigenschaft(laut) Dann
                Speichern(Property, Eigenschaft)
KeywordQualifier(Keyword)
    Wenn Keyword =„Hallo“ Dann
                Speichern(Keyword, „Hallo“)

A corresponding context processing could look like this:

MimicQualifier property
    If property (friendly) Then
                Save (facial expressions, property)
    If property (upset) Then
                Save (facial expressions, property)
PropertyQualifier
    If property (Quiet) Then
                Save (property)
PropertyQualifier
    If property (loud) Then
                Save (property)
KeywordQualifier (Keyword)
    If Keyword = "Hello" Then
                Save (keyword, "hello") 

Saving (facial expressions, properties), saving (property, properties) and saving (keyword, "hello") each correspond to a system reaction, in this case the call of a function in the computer system. There it would also be decided whether a system reaction in the form of the voice output "Hello" or "What can I do for you" should take place because one of the two possible three-way constellations ("Hello", quiet, friendly) or (" Hello ”, loud, upset) was recognized.

Opens up the person 1 Now a communication with the word "Hello" in a low tone and with a friendly facial expression, both the characteristic quiet and the characteristic friendly are recognized accordingly and sorted accordingly into the category as a communication component. Then the communication components of the word "Hello" as well as the components with the property "quiet" and the property "friendly" as communication content 20th in the form of an n-tuple in the digital communication context. If a sufficient match has been found, the system reaction assigned to this communication content, in this case a “hello”, is carried out.

However, the same word "hello" in a loud tone and with an angry facial expression leads to a different system reaction, namely a reaction in which the computer-based communication assistant responds in an exemplary neutral manner to the request and merely returns a "What can I do for you".

This form of breaking down the communication into verbal and non-verbal communication components and the fact that a large number of verbal and / or non-verbal communication components can be combined into a single communication content enables a modular structure and a very finely structured adjustment of the system to the communication presented to it .

A special feature of the qualifiers assigned to the verbal communication components is that they are spoken by the participant and are not delivered as part of speech recognition or are supplemented by the system during communication processing. In conventional speech processing, attempts are made to recognize predefined words or sentences (which of course is also possible with the method according to the invention) and to carry out an action based on the recognized. However, there is no possibility of recognizing words that are not defined but that should or should be recognized for a satisfactory system response. For example, a first participant during the communication could express the wish "I want to order the milk article and the lawn mower article". Is the word " Lawnmowers ”have not been defined in the conventional way, the request is not processed. This problem is solved by the KeywordQualifier according to the invention, whereby it should be assumed here by way of example that the preceding word “article” was defined as KeywordQualifier. This makes it possible with the aid of the communication processing according to the invention to identify and process the word "lawn mower" following the keyword qualifier as an unrecognized article, for example by searching for "lawn mower" in other communication contexts or a clarifying query from the system to the first participant he follows.

The SemanticQualifier, on the other hand, standardizes linguistic expressions with which a specific goal is to be achieved. For example, a SemanticQualifier can have the general structure <Start>, <Subject>, <End>, where <Start> declares the class, <Subject> the object name and <End> the method. For example, if the participant expresses the wish "I would like to call up information on the current budget", a correspondingly defined SemanticQualifier ("Information", "Budget", "call up") could, for example, be linked to a system response that uses the current budget as a speech output issues. "Current" is therefore not part of the SemanticQualifier, but was taken from the linguistic statement, the so-called text stream, which is included in the SemanticQualifier. It could have been said "old". Although "current" and "old" in this contextual content are the appropriate keywords to get the right budget from the system, they can still be used in other contextual content or contexts.

3rd finally shows a further exemplary communication in which, instead of a single device, a distributed system of hardware and software is set up for communication processing according to the invention. One of the advantages of this is that the individual components are interchangeable.

As for the 1 explained, first receives a communication recognition module 31 which, as shown here, can be a smartphone, for example, the visual, auditory and / or textual communication of a participant 300 and breaks them down into individual ones for a communication assistant 30th recognizable verbal and non-verbal communication components 32 . These communication components 32 are then evaluated using a suitable interface, for example via Bluetooth 33 the computer-based communication assistant 30th provided for context comparison. It is also possible for one and the same device to have a communication recognition module at the same time 31 and communication assistant 30th is. For this purpose, the modules of the communication assistant can be set up, for example, as applications that can be executed on the smartphone.

The evaluation module 33 also receives several digital communication contexts 34 that the evaluation module 33 from an external data storage 35 , here a cloud, for example. The communication contexts 34 can be used as modular context files from third parties 302 , for example suppliers and / or other service providers, in the data store 35 be made available for download. This has the advantage that the communication contexts 34 are available for a large number of participants and can be adapted at any time, for example to keep the content (such as prices) up to date or to make decisions of different companies comparable with a context.

In principle, any number of communication contexts 34 from any third party 302 started at the same time. For efficient communication processing, however, it will regularly be of advantage if the evaluation module 33 provided digital communication contexts 34 goal-oriented using a communication parameter 38 which, for example, in the participant's place 300 can be justified from the large number in the database 35 existing contexts 34 to be chosen. Alternatively, the participant 300 the communication contexts relevant to him 34 also book or call up specifically (not shown here).

The evaluation module 33 compares the verbal and non-verbal communication components received 32 or their content with the communication content 20th of the respective communication context 34 and tries that communication content 20th of a communication context 34 to determine the recognized verbal and non-verbal communication components 32 the greatest possible match in the corresponding communication content 20th exhibit. In this regard, refer to the descriptions 1 and 2nd referred.

Then the evaluation module 33 the determined system response to an execution module 36 transmitted, through which the corresponding system reaction is then carried out. This can, as in 3rd schematically indicated by the dashed arrows, for example a) feedback to the Attendees 300 for example by push notification or b ) feedback to the third party 302 such as ordering an item from the local supplier. Another peculiarity is that the system response too c ) the provision of further communication contexts 34 may include to process the communication.

Although it is not shown or mentioned in the figures and associated explanations for reasons of clarity, there may of course also be a plurality of communication assistants which process the communication, if necessary simultaneously.

The invention is not restricted to the exemplary embodiments by the description based on these. Rather, the invention encompasses every new feature and every combination of features, which in particular includes every combination of features in the patent claims, even if this feature or this combination itself is not explicitly specified in the patent claims or exemplary embodiments.

Reference list

1, 300

natural person

10, 30

computer-based communication assistant

11, 31

Communication detection module

12, 32

verbal and non-verbal communication components

13, 33

Evaluation module

14, 34

digital communication context

15, 35

Context database, data storage

16, 36

Execution module

20th

Communication content, n-tuple

302

Third party

B _n

concrete communication component of a communication content

R _m

concrete system reaction

QUOTES INCLUDE IN THE DESCRIPTION

This list of documents listed by the applicant has been generated automatically and is only included for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Patent literature cited

• DE 102009034069 A1 [0003]
• US 2007/0215745 A1 [0004]
• DE 102008017957 A1 [0005]
• EP 2341494 [0006]
• EP 0613111 A1 [0007]
• US 2005/0283475 A1 [0008]
• DE 19619015 A1 [0009]
• JP 06342061 [0010]
• DE 102007018327 C5 [0011]
• US 2008/0091406 A1 [0012]
• JP 2005173134 A [0012]
• DE 602004002230 T2 [0012]
• US 2005/0137868 A1 [0012]

Claims (12)

A method for automated, computer-based communication processing of visual, auditory and / or textual communication of at least one first participant (1, 300) with at least one computer-based communication assistant (10, 30) as second participant, the method comprising the following steps: - Providing verbal and non-verbal communication components (12, 32) of a received visual, auditory and / or textual communication to the computer-based communication assistant (10, 30), - providing at least one digital communication context (14, 34) of the computer-based communication assistant (10, 30), whereby the digital communication context (14, 34) contains a plurality of communication contents (20) to be recognized, each of which has a combination of concrete verbal and / or non-verbal communication components (12, 32) and each has at least one corresponding predefined system reaction (R _m ) of the computer-based communication assistant (10) are assigned, - determining at least one communication content (20) of the digital communication context (14, 34) to be recognized by comparing the verbal and non-verbal communication components (12, 32) of the received visual, auditory and / or textual communication with the verbal and non-verbal communication components (12, 32) of the communication content (20) of the communication context (14, 34) to be recognized, and - execution of the system reaction (R _m ) associated with the determined communication content (20) by the computer-based communication assistant (10, 30). The procedure after Claim 1 , characterized in that one or more digital communication contexts (14, 34) are selected as a function of at least one communication parameter and made available to the computer-based communication assistant (10, 30). The procedure after Claim 1 or 2nd , characterized in that the execution of the system reaction (R _m ) associated with the determined communication content (20) by the computer-based communication assistant (10, 30) comprises providing at least one further digital communication context (14, 34). The procedure according to one of the Claims 1 to 3rd , characterized in that visual, auditory and / or textual communication is received by the computer-based communication assistant (10, 30) and verbal and non-verbal communication components (12, 32) from the received visual, auditory and / or textual communication by the computer-based communication assistant (10, 30) are recognized and provided. The procedure according to one of the Claims 1 to 4th , characterized in that depending on at least one recognized verbal and / or non-verbal communication component (12, 32) of the received visual, auditory and / or textual communication one or more digital communication contexts (14, 34) are selected and the computer-based communication assistant (10, 30) can be provided. The procedure according to one of the Claims 1 to 5 , characterized in that one or more of the following elements from the received communication are recognized as verbal communication components (12, 32) and / or are contained in the communication content (20) provided: - Keywords, as words or words, in whole or in part (Wildcards) are recognized, - a KeywordQualifier, which comprises one or more predefined keywords, which cause one or more unrecognized or only partially recognized words to be placed before and / or after the keyword or the keywords or arranged between them, are processed; - a SemanticQualifier, which includes a standardization of perception, thinking and acting in relation to the recognized words and comprises two or more predefined linguistic elements that cause one or more unrecognized, only partially and / or fully recognized words (wildcards) to be used arranged between the linguistic elements, processed; - An AssociationQualifier, which contains a connection to another communication, a different communication context (14, 34) and / or another system reaction (R _m ). The procedure according to one of the Claims 1 to 6 , characterized in that one or more of the following elements from the received communication are recognized as non-verbal communication components (12, 32) and / or are contained in the communication content (20) provided: a ComQualifier that contains a communication parameter or a communication property of the communication per se; - a MemberQualifier, which contains a parameter and / or a property of the first participant (1, 300); - an EmotionQualifier, which contains an emotional parameter and / or an emotional characteristic of the first participant (1, 300); - a MimicQualifier, which contains a visible property of the face of the first participant (1, 300); - a gesture qualifier which contains a gesture of the first participant (1, 300); - A BodyQualifier, which contains a visible movement of the first participant (1, 300); - a BioQualifier, which contains a vital parameter and / or a vital characteristic of the first participant (1, 300); - a PropertyQualifier that contains an acoustic parameter and / or an acoustic property. The procedure according to one of the Claims 1 to 7 , characterized in that one or more recognized verbal and non-verbal communication components (12, 32) of the received visual, auditory and / or textual communication are stored in combination with the executed system reaction (R _m ) in a data memory (15, 35). The procedure after Claim 8 , characterized in that the stored combination of the communication components (12, 32) is used as a communication context (14, 34) of visual, auditory and / or textual communications received in the future. A computer program with program code means, set up to carry out the method according to one of the Claims 1 to 9 when the computer program is running on a data processing system. A computer-readable storage medium that the computer program according to Claim 10 includes. A computer-based communication assistant (10, 30) for automated, computer-based communication processing of visual, auditory and / or textual communication of at least one first participant (1, 300) and the computer-based communication assistant (10, 30) as at least second participant, characterized in that the computer-based communication assistant (10, 30) for performing the method according to one of the Claims 1 to 9 is set up.

Images