JP2019175432A - Dialogue control device, dialogue system, dialogue control method, and program - Google Patents

Abstract

An object of the present invention is to accurately grasp a user's preference and perform a dialogue that matches the user's preference. A robot includes a result of determining a response of a predetermined target to an utterance by the robot and a result of determining a response of a predetermined target to an utterance by a robot provided separately from the robot. Controlling utterance by at least one of a plurality of utterance devices including the robot 100A and the robot 100B based on the plurality of reaction determination units obtained by the reaction obtaining unit 110A that obtains a plurality of reaction determination results and the plurality of reaction determination units obtained by the reaction obtaining unit 110A And an utterance control unit 115A. [Selection diagram] FIG.

Description

   The present invention relates to a dialog control device, a dialog system, a dialog control method, and a program.

  The development of devices such as robots that communicate with people is underway, and friendliness is an important point for the spread of such devices such as robots. For example, Patent Literature 1 discloses a technique for learning user preferences through dialogue with the user and performing dialogue that matches the user preference.

JP 2006-071936 A

  In the technology disclosed in Patent Document 1, since there are a wide variety of combinations of information used for grasping user preferences, it is difficult to accurately grasp user preferences.

  The present invention has been made in view of the above-described circumstances, and has a dialog control device, a dialog system, and dialog control capable of accurately grasping a user's preference and performing a dialog suitable for the user's preference. An object is to provide a method and a program.

In order to achieve the above object, an aspect of the dialogue control apparatus according to the present invention is as follows.
A result of determining a response of a predetermined object to an utterance by the first utterance device, and a result of determining a response of the predetermined object to an utterance by a second utterance device provided separately from the first utterance device Reaction acquisition means for acquiring a plurality of reaction determination results;
Utterance control means for controlling the utterance by at least one of a plurality of utterance devices including the first and second utterance devices based on a plurality of reaction determination results acquired by the reaction acquisition means;
Comprising
It is characterized by that.

In order to achieve the above object, an aspect of the dialog system according to the present invention is as follows.
A dialogue system comprising a first utterance device and a second utterance device configured to be utterable, and a dialogue control device,
The dialogue control device comprises:
A result of determining a response of a predetermined object to an utterance by the first utterance device, and a result of determining a response of the predetermined object to an utterance by the second utterance device provided separately from the first utterance device; Reaction acquisition means for acquiring a plurality of reaction determination results including:
Utterance control means for controlling the utterance by at least one of a plurality of utterance devices including the first and second utterance devices based on a plurality of reaction determination results acquired by the reaction acquisition means;
Comprising
It is characterized by that.

In order to achieve the above object, an aspect of the dialogue control method according to the present invention is as follows.
A result of determining a response of a predetermined object to an utterance by the first utterance device, and a result of determining a response of the predetermined object to an utterance by a second utterance device provided separately from the first utterance device Processing for obtaining a plurality of reaction determination results;
A process for controlling the utterance by at least one of a plurality of utterance devices including the first and second utterance devices based on the plurality of obtained reaction determination results;
including,
It is characterized by that.

In order to achieve the above object, an aspect of the program according to the present invention is as follows:
Computer
A result of determining a response of a predetermined object to an utterance by the first utterance device, and a result of determining a response of the predetermined object to an utterance by a second utterance device provided separately from the first utterance device Reaction acquisition means for acquiring a plurality of reaction determination results;
Utterance control means for controlling the utterance by at least one of a plurality of utterance devices including the first and second utterance devices based on a plurality of reaction determination results acquired by the reaction acquisition means;
Function as
It is characterized by that.

  According to the present invention, it is possible to provide a dialog control device, a dialog system, a dialog control method, and a program capable of accurately grasping a user's preference and performing a dialog suitable for the user's preference.

It is a figure which shows the structure of the dialogue system which concerns on the 1st Embodiment of this invention. It is a front view of the robot which concerns on 1st Embodiment. It is a block diagram which shows the structure of the robot which concerns on 1st Embodiment. It is a figure which shows an example of the audio | voice reaction polarity determination table which concerns on 1st Embodiment. It is a flowchart which shows the flow of the dialogue control process which concerns on 1st Embodiment. It is a flowchart which shows the flow of the user specific process which concerns on 1st Embodiment. It is a flowchart which shows the flow of the audio | voice determination process which concerns on 1st Embodiment. It is a flowchart which shows the flow of the facial expression determination process which concerns on 1st Embodiment. It is a flowchart which shows the flow of the action determination process which concerns on 1st Embodiment. It is a flowchart which shows the flow of the preference determination process which concerns on 1st Embodiment. It is a block diagram which shows the structure of the dialogue system of 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
The dialogue system 1 according to the first embodiment of the present invention includes a plurality of robots 100. The plurality of robots 100 are arranged in a living space such as an office or a house of a predetermined target, and the plurality of robots 100 interact with the predetermined target. In the following description, an example in which two robots 100 interact with a predetermined target will be described. However, the conversation system 1 may be configured to include three or more robots 100.

  Here, the predetermined target is a user (user) who uses the dialog system, and typically includes the owner of the dialog system, the family or friends of the owner, and the like. In addition to the person, the predetermined target includes, for example, an animal kept as a pet and another robot different from the robot 100.

  As shown in FIG. 1, the dialog system 1 includes two robots 100 that can communicate with each other, and performs a dialog with a user USR. Here, for convenience of explanation, the robot 100 on the left side in FIG. 1 is referred to as a robot 100A, and the robot 100 on the right side in FIG. 1 is referred to as a robot 100B. When the robot 100A and the robot 100B are described without being distinguished from each other, any one of the robots or these may be collectively referred to as “robot 100”. The robot 100A and the robot 100B are arranged at different locations, and are provided at locations where the same predetermined target cannot recognize both the robot 100A and the utterance of the robot 100B. For example, the robot 100A is disposed in a predetermined target office, and the robot 100B is disposed in a predetermined target house apart from the office. Alternatively, the robot 100A is disposed at a facility where a predetermined object passes, and the robot 100B is disposed at another facility away from this facility where a predetermined object passes.

  As shown in FIG. 2, the robot 100 is a robot having a three-dimensional shape imitating a person in appearance. Further, the exterior of the robot 100 is formed using synthetic resin as a main material. The robot 100 includes a body part 101, a head part 102 connected to the upper part of the body part 101, a hand part 103 connected to each of the left and right sides of the body part 101, and two legs connected to the lower part from the body part 101. 104. The head 102 has a pair of left and right eyes 105, a mouth 106, and a pair of left and right ears 107. Note that the upper side, lower side, left side, and right side in FIG. 2 are the upper side, lower side, right side, and left side of the robot 100, respectively.

  Next, the configuration of the robot 100 will be described with reference to FIG. FIG. 3 is a block diagram showing the configurations of the robot 100A and the robot 100B, but the configurations of the robot 100A and the robot 100B are the same. First, the robot 100A will be taken up and its configuration will be described.

  As shown in FIG. 3, the robot 100A includes a control unit 110A, a storage unit 120A, an imaging unit 130A, an audio input unit 140A, an audio output unit 150A, a moving unit 160A, and a communication unit 170A. These units are electrically connected to each other via a bus line BL.

  The control unit 110A includes a computer having a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory), and controls the operation of the entire robot 100A. 110 A of control parts control operation | movement of each part of 100 A of robots, when CPU reads the control program memorize | stored in ROM and runs on RAM.

  The control unit 110A executes a control program to thereby execute a user detection unit 111A, a user identification unit 112A, a user information acquisition unit 113A, a speech recognition unit 114A, an utterance control unit 115A, a speech synthesis unit 116A, a reaction determination unit 117A, a preference It functions as the determination unit 118A.

  The user detection unit 111A detects the user USR existing around the robot 100A (for example, within a radius of 2 m from the robot 100A). For example, the user detection unit 111A controls the imaging unit 130A described later to image the surroundings of the robot 100A, and the user USR existing around the robot 100A is detected according to the detection of the movement of the object, the head, the face, and the like. Detect.

  The user specifying unit 112A specifies the user USR detected by the user detecting unit 111A. For example, the user specifying unit 112A extracts a face image corresponding to the face portion of the user USR from the image captured by the image capturing unit 130A. Then, the user specifying unit 112A detects a feature amount from the face image, calculates the similarity by collating it with face information indicating the feature amount of the face registered in a user information database of the storage unit 120A described later, The user USR is specified according to whether or not a predetermined standard is satisfied. In the user information database of the storage unit 120A, face information indicating the feature amount of each face of a predetermined plurality of users USR is stored. The user specifying unit 112A specifies which of the user USRs the user USR detected by the user detecting unit 111A is. The feature amount may be information that can identify the user USR. For example, the feature amount is information that shows numerically the appearance features such as the shape, size, and arrangement of each part included in the face such as the eyes, nose, and mouth. is there. In the following description, the user USR detected by the user detection unit 111A and specified by the user specification unit 112A is referred to as a target user. Thus, the user specifying unit 112A functions as a target specifying unit of the present invention.

  The user information acquisition unit 113A acquires user information representing the speech, appearance, behavior, and the like of the target user. In the present embodiment, the user information acquisition unit 113A controls, for example, the imaging unit 130A and the audio input unit 140A as user information, and includes image information including image data of a captured image capturing the target user and the target user. At least one of voice information including voice data of the emitted voice is acquired. As described above, the user information acquisition unit 113A functions as the acquisition unit of the present invention in cooperation with the imaging unit 130A and the voice input unit 140A.

  The voice recognition unit 114A performs voice recognition processing on the voice data included in the voice information acquired by the user information acquisition unit 113A, and converts the voice data into text data indicating the utterance content of the target user. For the speech recognition processing, for example, an acoustic model, a language model, and a word dictionary stored in the speech information DB (Data Base) 122A of the storage unit 120A are used. For example, the speech recognition unit 114A erases background noise from the speech data, refers to the acoustic model, identifies the phonemes included in the acquired speech data, refers to the word dictionary, and converts the phoneme string into words Generate multiple conversion candidates. Then, the speech recognition unit 114A refers to the language model, selects the most appropriate one from a plurality of conversion candidates, and outputs it as text data corresponding to the speech data.

  The utterance control unit 115A controls the utterance of the robot 100A. For example, the utterance control unit 115A refers to the utterance information stored in the utterance information DB 123A of the storage unit 120A, and extracts a plurality of utterance candidates according to the situation. Then, the utterance control unit 115A refers to the preference information included in the user information stored in the user information DB 121A, selects the utterance candidate that matches the preference of the target user from the plurality of extracted utterance candidates, and the utterance content of the robot 100A Determine as. Thus, the utterance control unit 115A functions as the utterance control means of the present invention.

  In addition, the utterance control unit 115A communicates with the robot 100B via the communication unit 170A and adjusts and determines the utterance content of the robot 100A as follows in cooperation with the utterance control unit 115B of the robot 100B.

  That is, the utterance control unit 115A cooperates with the utterance control unit 115B of the robot 100B to acquire, for example, an elapsed time since the robot 100B uttered, and the acquired elapsed time is within a predetermined elapsed time (for example, 72 hours). When the robot 100A speaks at some time, the utterance of the robot 100A is such that the topic uttered by the robot 100A is different from the topic uttered by the robot 100B within a predetermined elapsed time before the utterance of the robot 100A starts. Adjust the topic and decide. Such topic determination is similarly performed in the speech control unit 115B of the robot 100B. As described above, the topics uttered by the robot 100A and the robot 100B are determined as different topics, and the utterances of both the robots 100A and 100B are controlled by the determined topics.

  As will be described later, each of the robot 100A and the robot 100B determines the response of the target user to his / her utterance, and collects (stores) the target user's preference information based on the determination result. In this case, the robot 100A If the topics spoken by the robot 100B overlap or are always related, new preference information of the target user or preference information in a wider field cannot be collected. In addition, the target user also feels annoyed by hearing the utterances of overlapping topics. By determining the topics of the utterances of the robot 100A and the robot 100B as different topics, it is possible to collect more various types of preference information.

  On the other hand, when a predetermined elapsed time has elapsed since the robot 100B uttered, the utterance control unit 115A determines the utterance content independently without being limited to the utterance content of the robot 100B. That is, the topics (speech contents) spoken by the robots 100A and 100B are determined independently of each other (independently) without being linked to each other.

  The utterance control unit 115A generates and outputs text data indicating the content of the utterance determined in cooperation with the robot 100B.

  The speech synthesizer 116A generates speech data corresponding to text data indicating the utterance content of the robot 100A input from the utterance control unit 115A. The speech synthesis unit 116A generates speech data that reads out a character string indicated by the text data, using, for example, an acoustic model stored in the speech information DB 122A of the storage unit 120A. In addition, the voice synthesis unit 116A controls the voice output unit 150A to output the generated voice data as a voice.

  The response determination unit 117A determines the response of the target user to the utterance of the robot 100A. Thereby, the reaction with respect to the utterance of the robot 100A is determined for each target user specified by the user specifying unit 112A among the predetermined plurality of users USR. The reaction determination unit 117A includes a voice determination unit 117AA, a facial expression determination unit 117BA, and an action determination unit 117CA. The voice determination unit 117AA, the facial expression determination unit 117BA, and the behavior determination unit 117CA are positive responses based on the target user's voice, facial expression, and behavior, respectively, in response to the speech of the target robot 100A. "Negative" which is a negative reaction, and "Neutral" which is a neutral reaction which is neither positive nor negative.

  The voice determination unit 117AA determines the response of the target user to the utterance of the robot 100A based on the voice uttered by the target user after the robot 100A utters. The voice determination unit 117AA uses the voice recognition unit 114A to perform a voice recognition process on the voice acquired by the user information acquisition unit 113A after the utterance of the robot 100A. By classifying the reaction polarities into three polarities of “positive”, “negative”, and “neutral”, the response of the target user to the utterance of the robot 100A is determined. Thus, the voice determination unit 117AA functions as the voice determination unit of the present invention.

  The facial expression determination unit 117BA determines the response of the target user to the speech of the robot 100A based on the facial expression of the target user after the robot 100A speaks. The facial expression determination unit 117BA calculates a smile level indicating the smile level as an index for evaluating the facial expression of the target user. The facial expression determination unit 117BA extracts the target user's face image from the captured image acquired by the user information acquisition unit 113A after the utterance of the robot 100A, and detects the feature amount of the target user's face. The facial expression determination unit 117BA refers to the smile level information stored in the reaction determination information DB 124A of the storage unit 120A, and calculates the smile level of the target user based on the detected feature amount. The facial expression determination unit 117BA classifies the target user's facial expression into three facial expression reaction polarities of “positive”, “negative”, and “neutral” according to the calculated smile level, and the target user's response to the utterance of the robot 100A Determine. As described above, the facial expression determination unit 117BA functions as facial expression determination means of the present invention.

  The behavior determination unit 117CA determines the response of the target user to the utterance of the robot 100A based on the behavior of the target user after the robot 100A utters. The behavior determination unit 117CA detects the behavior of the target user from the captured image acquired by the user information acquisition unit 113A after the utterance of the robot 100A, and the behavior of the target user with the behavior reaction polarity “positive”, “negative”, “neutral”. By classifying into three polarities, the reaction of the target user to the utterance of the robot 100A is determined. Thus, the behavior determination unit 117CA functions as the behavior determination unit of the present invention.

  The preference determination unit 118A specifies a topic in the conversation between the target user and the robot 100A, and determines a preference level indicating the level of preference of the target user with respect to the specified topic based on each determination result by the reaction determination unit 117A. . Thereby, the preference degree is determined for each target user specified by the user specifying unit 112A among the predetermined plurality of users USR. Here, the preference is a hobby or preference related to various things regardless of tangible or intangible. For example, in addition to a hobby or preference related to food, sports, weather, etc., a preference for correspondence (utterance content) of the robot 100 Including. The preference determination unit 118A classifies the preference levels into four levels of “preference level A”, “preference level B”, “preference level C”, and “preference level D” in descending order of the preference of the target user with respect to the topic. Thus, the preference determination unit 118A functions as the specifying unit and the preference determination unit of the present invention.

  The user detection unit 111A, user identification unit 112A, user information acquisition unit 113A, speech recognition unit 114A, speech control unit 115A, speech synthesis unit 116A, reaction determination unit 117A, and preference determination unit 118A are each a single computer. The functions may be realized, or each function may be realized by a separate computer.

  The storage unit 120A includes a nonvolatile semiconductor memory, a hard disk drive, and the like in which stored contents can be rewritten, and stores various data necessary for the control unit 110A to control each unit of the robot 100A.

  The storage unit 120A has a plurality of databases that store various data. The storage unit 120A includes, for example, a user information DB 121A, a voice information DB 122A, an utterance information DB 123A, and a reaction determination information DB 124A. In addition, in the storage unit 120A, utterance history information including the utterance date and time of the utterance of the robot 100A, the topic uttered, and the like is stored for each user USR.

  The user information DB 121A accumulates and stores various types of information related to each of a plurality of registered users USR as user information. The user information includes, for example, user identification information (for example, ID of the user USR) assigned in advance to identify each of the plurality of users USR, face information indicating the feature amount of the face of the user USR, and the user USR for each topic. The preference information indicating the degree of preference of the user is included. Thus, the preference information of each of the plurality of users USR is stored so as to be able to identify which user USR belongs using the user identification information.

  The voice information DB 122A, as data used for the voice recognition process or the voice synthesis process, is, for example, an acoustic model that represents each feature (frequency characteristic) of a phoneme that is a minimum unit of sound that distinguishes meaning from other words, A word dictionary for associating features with words, and a language model representing word sequences and their connection probabilities are stored.

  The utterance information DB 123A stores utterance information indicating utterance candidates of the robot 100A. The utterance information corresponds to the state of dialogue with the target user, such as an utterance candidate when talking to the target user, an utterance candidate when responding to the utterance of the target user, an utterance candidate when talking to the robot 100B, etc. Various utterance candidates are included.

  The reaction determination information DB 124A stores reaction determination information used when the reaction determination unit 117A determines a target user's reaction to the utterance of the robot 100A. The reaction determination information DB 124A stores, for example, voice determination information used when the voice determination unit 117AA of the reaction determination unit 117A determines the reaction of the target user with respect to the utterance of the robot 100A as the reaction determination information. The voice determination information is stored, for example, in the form of a voice reaction polarity determination table shown in FIG. In the voice reaction polarity determination table, a voice reaction polarity, which will be described later, and a feature keyword are associated with each other. In addition, the reaction determination information DB 124A stores, for example, smile degree information used when the facial expression determination unit 117BA of the reaction determination unit 117A calculates the smile degree of the target user as the reaction determination information. The smile level information is information obtained by quantifying the smile level in the range of 0 to 100% according to the degree of change in the position of the corners of the eyes, the corners of the mouth, the size of the eyes and the mouth, for example.

  The imaging unit 130A is configured by a camera including an imaging element such as a lens, a CCD (Charge Coupled Device) image sensor, or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and images the surroundings of the robot 100A. The imaging unit 130A is installed, for example, in the upper front portion of the head 102, images the front of the head 102, generates digital image data, and outputs the digital image data. The camera is attached to a motor-driven base (such as a gimbal) that can be operated so as to change the direction in which the lens faces, and is configured to track the face of the user USR.

  The audio input unit 140A includes a microphone, an A / D (Analog to Digital) converter, and the like, for example, amplifies the sound collected by the microphone installed in the ear unit 107, and performs A / D conversion, encoding, and the like. The digital audio data (audio information) subjected to the signal processing is output to the control unit 110A.

  The audio output unit 150A includes a speaker, a D / A (Digital to Analog) converter, and the like, and decodes, D / A converts, amplifies signals, etc. into audio data supplied from the audio synthesis unit 116A of the control unit 110A. Processing is performed, and an analog audio signal is output from, for example, a speaker installed in the mouth portion 106.

  The robot 100A collects the voice of the target user with the microphone of the voice input unit 140A, and outputs the voice corresponding to the utterance content of the target user from the speaker of the voice output unit 150A under the control of the control unit 110A. Communicating by interacting with the target user. Thus, the robot 100A functions as the first utterance device of the present invention.

  The moving unit 160A is a part for moving the robot 100A. The moving unit 160A includes wheels provided at the bottoms of the left and right feet 104 of the robot 100A, a motor that rotationally drives the left and right wheels, and a drive circuit that drives and controls the motor. In accordance with the control signal received from the control unit 110A, the drive circuit supplies a drive pulse signal to the motor. The motor rotates the left and right wheels according to the driving pulse signal to move the robot 100A. Thus, the moving unit 160A functions as the moving unit of the present invention. Note that the number of motors is arbitrary as long as the left and right wheels are configured to rotate independently of each other and the robot 100A can travel forward, backward, turn, accelerate, and decelerate. For example, the left and right wheels may be driven by one motor by providing a connection mechanism or a steering mechanism. In addition, the number of drive circuits can be changed as appropriate in accordance with the number of motors.

  The communication unit 170A includes a wireless communication module and an antenna for performing communication using a wireless communication method, and performs wireless data communication with the robot 100B. As the wireless communication method, for example, Bluetooth (registered trademark), BLE (Bluetooth Low Energy), ZigBee (registered trademark), short-range wireless communication methods such as infrared communication, and wireless LAN communication methods such as WiFi (Wireless Fidelity) are appropriately used. Can be adopted. In the present embodiment, the robot 100A performs wireless data communication with the robot 100B via the communication unit 170A, so that the robot 100A and the robot 100B interact with the target user.

  Since the robot 100B is similar to the robot 100A, its configuration will be briefly described. Similar to the robot 100A, the robot 100B includes a control unit 110B, a storage unit 120B, an imaging unit 130B, an audio input unit 140B, an audio output unit 150B, a moving unit 160B, and a communication unit 170B. The control unit 110B controls the entire operation of the robot 100B and executes a control program, thereby performing a user detection unit 111B, a user identification unit 112B, a user information acquisition unit 113B, a voice recognition unit 114B, a speech control unit 115B, a voice synthesis. Functions as a unit 116B, a reaction determination unit 117B, and a preference determination unit 118B. The utterance control unit 115B refers to the preference information included in the user information stored in the user information DB 121B, selects an utterance candidate that matches the target user's preference from the extracted utterance candidates, and determines the utterance content of the robot 100B. To do. The robot 100A communicates with the robot 100A via the communication unit 170B, and acquires the elapsed time since the robot 100A spoke, for example, in cooperation with the speech control unit 115A of the robot 100A. When the acquired elapsed time is within the predetermined elapsed time, the speech control unit 115B determines that the topic spoken by the robot 100B is different from the topic spoken by the robot 100A within the predetermined elapsed time before the start of speech of the robot 100B. Then, the content of the utterance of the robot 100B is adjusted and determined. The response determination unit 117B determines the response of the target user to the utterance of the robot 100B. The reaction determination unit 117B includes a voice determination unit 117AB, a facial expression determination unit 117BB, and an action determination unit 117CB. The voice determination unit 117AB, the facial expression determination unit 117BB, and the behavior determination unit 117CB each react to the utterance of the target robot 100B based on the voice, facial expression, and behavior of the target user, “positive”, “negative”, “neutral”. It is determined by classifying into three polarities. The storage unit 120B has a plurality of databases that store various data. The storage unit 120B includes, for example, a user information DB 121B, a voice information DB 122B, an utterance information DB 123B, and a reaction determination information DB 124B. In addition, in the storage unit 120B, utterance history information including the utterance date and time of the utterance of the robot 100B and the uttered topic is stored for each user USR. The robot 100B collects the voice of the target user with the microphone of the voice input unit 140B, and outputs the voice corresponding to the utterance content of the target user from the speaker of the voice output unit 150B under the control of the control unit 110B. Communicating by interacting with the target user. Thus, the robot 100B functions as the second speech device of the present invention.

  Next, the dialogue control process executed by the robot 100 will be described with reference to the flowchart shown in FIG. The dialogue control process is a process for controlling the dialogue according to the preference of the target user. Here, the dialogue control process will be described by taking as an example a case where the control unit 110A of the robot 100A executes. The control unit 110A starts the dialogue control process when the user detection unit 111A detects the user USR around the robot 100A.

  110 A of control parts will perform a user specific process first, if a dialog control process is started (step S101). Here, the user specifying process will be described with reference to the flowchart shown in FIG. The user specifying process is a process of specifying a user existing around the robot 100A detected by the user detecting unit 111A.

  When the user specifying process is started, the control unit 110A first extracts the face image of the target user from the captured image acquired from the imaging unit 130A (step S201). For example, the control unit 110A (the user specifying unit 112A) detects a skin color area in the captured image, determines whether there is a part corresponding to a face part such as an eye, nose, or b in the skin color area. If it is determined that there is a part corresponding to the part, the skin color area is regarded as a face image and extracted.

  Subsequently, the control unit 110A searches for a registered user corresponding to the extracted face image (step S202). For example, the control unit 110A (user specifying unit 112A) detects a feature amount from the extracted face image, compares it with the face information stored in the user information DB 121A of the storage unit 120A, and the similarity is equal to or higher than a predetermined reference. Search for registered users.

  110 A of control parts identify the user USR which exists around the robot 100 according to the search result in step S202 (step S203). For example, the control unit 110A (user specifying unit 112A) has the highest similarity to the feature amount detected from the face image among the facial feature amounts of each of the plurality of users USR stored in the user information DB 121A. The user USR corresponding to the amount is specified as a target user existing around the robot 100A.

  After executing the process of step S203, the control unit 110A ends the user specifying process and returns the process to the dialog control process.

  Returning to FIG. 5, after executing the user specifying process (step S101), the control unit 110A establishes a communication connection with the robot 100B (another robot) (step S102). Here, establishment of communication connection refers to establishing a state in which data can be transmitted and received with each other by specifying a communication partner and performing a predetermined procedure. The control unit 110A controls the communication unit 170A and establishes a communication connection with the robot 100B by performing a predetermined procedure based on the communication method. When the robot 100A and the robot 100B perform data communication using the infrared communication method, it is not necessary to establish a communication connection in advance.

  Subsequently, the control unit 110A determines whether or not the target user identified in step S101 has spoken within a predetermined time (for example, within 20 seconds) shorter than the predetermined elapsed time (step S103). For example, the control unit 110A measures the elapsed time from the start of execution of this process using current time information measured by an RTC (Real Time Clock) attached to the CPU, and acquires the voice acquired by the user information acquisition unit 113A. Based on the information, the presence or absence of the target user's utterance is determined within a predetermined time.

  When it is determined that the target user has uttered within the predetermined time (step S103: YES), the control unit 110A (speech control unit 115A) determines that a dialogue with the target user is being executed, The content of the utterance as a response is determined in cooperation with the robot 100B (step S104). The control unit 110A (the utterance control unit 115A) refers to the utterance information DB 123A and the user information DB 121A in the storage unit 120A, corresponds to the utterance content of the target user, and conforms to the preference of the target user stored in the user information DB 121A. Determine the topic candidates. In this case, topics corresponding to the preference degrees A and B, which will be described later, are determined as topic candidates as topics that match the preferences of the target user.

  In this step S104, when there is one determined topic candidate, this is determined as the final topic. On the other hand, when a plurality of topic candidates are determined and the utterance history information is stored in the storage unit 120B of the robot 100B, the control unit 110A (the utterance control unit 115A) is stored in the storage unit via the communication unit 170A. The utterance history information stored in 120B is read, and the read utterance history information is a topic that is the same as or related to any of a plurality of topic candidates, and from the utterance date and time (ie, when the robot 100A starts utterance) ) Until the elapsed time is within a predetermined elapsed time (hereinafter referred to as “first comparison target topic”).

  When the control unit 110A (the utterance control unit 115A) determines that the first comparison target topic is present in the utterance history information, the control unit 110A matches or matches the first comparison target topic from a plurality of topic candidates. Exclude related ones and finally determine the topic. When there are a plurality of candidate topics remaining due to this exclusion, one topic selected at random is determined as the final topic.

  On the other hand, when a plurality of topic candidates are determined, it is determined that no utterance history information is stored in the storage unit 120B of the robot 100B, or that the first comparison target topic does not exist in the utterance history information. Then, one topic randomly selected from the plurality of determined topic candidates is determined as the final topic. The utterance control unit 115A outputs text data indicating the utterance content along the topic determined as described above.

  On the other hand, when it is determined that the target user does not speak within a predetermined time (step S103: NO), the control unit 110A (speech control unit 115A) determines the topic of the speech to be spoken to the target user (step S105). At this time, the control unit 110A (the utterance control unit 115A) refers to the utterance information DB 123A and the user information DB 121A in the storage unit 120A, and determines a plurality of topic candidates that match the preferences of the target user stored in the user information DB 121A. To do. In this case, topics corresponding to the preference degrees A and B, which will be described later, are determined as topic candidates as topics that match the preferences of the target user.

  In this step S105, when there is one determined topic candidate, this is determined as the final topic. On the other hand, when a plurality of topic candidates are determined, the final topic is selected from the plurality of topic candidates in the same manner as in step S104. Specifically, control unit 110A (speech control unit 115A) controls control unit 110A (utterance utterance) when utterance history information is stored in storage unit 120B of robot 100B when a plurality of topic candidates are determined. The control unit 115A) reads the utterance history information stored in the storage unit 120B via the communication unit 170A, and determines whether or not the first comparison target topic exists in the read utterance history information. .

  Then, when it is determined that the first comparison target topic exists in the utterance history information, the control unit 110A (the utterance control unit 115A) matches or relates to the first comparison target topic from a plurality of topic candidates. Exclude and finally determine the topic. When there are a plurality of candidate topics remaining due to this exclusion, one topic selected at random is determined as the final topic.

  On the other hand, when a plurality of topic candidates are determined, it is determined that no utterance history information is stored in the storage unit 120B of the robot 100B, or that the first comparison target topic does not exist in the utterance history information. Then, one topic randomly selected from the plurality of determined topic candidates is determined as the final topic.

  The action of speaking to the target user when the target user does not speak within a predetermined time is a trigger for a conversation between the target user and the robot 100A and the robot 100B, and encourages the target user to use the interactive system 1. To be implemented.

  After executing Step S104 or Step S105, the control unit 110A utters based on the utterance content along the determined topic (Step S106). Control unit 110A (voice synthesis unit 116A) generates voice data corresponding to text data indicating the utterance content of robot 100A input from utterance control unit 115A, controls voice output unit 150A, and controls voice based on the voice data. Is output.

  Steps S107 to S109 are processes for determining the target user's reaction to the utterance of the robot 100A in step S106.

  First, the control unit 110A (the voice determination unit 117AA of the reaction determination unit 117A) executes a voice determination process (step S107). Here, the voice determination process will be described with reference to the flowchart shown in FIG. The voice determination process is a process for determining the response of the target user to the utterance of the robot 100A based on the voice uttered from the target user after the utterance of the robot 100A.

  When the sound determination unit 117AA starts the sound determination process, it first determines whether or not the target user has spoken after the robot 100A uttered in step S106 (step S301). 110 A of control parts determine the presence or absence of an utterance of the target user with respect to the utterance of the robot 100A based on the audio | voice information which the user information acquisition part 113A acquired after the utterance of the robot 100A.

  If it is determined that the target user has spoken after the utterance of the robot 100A (step S301: YES), the voice determination unit 117AA extracts a feature keyword from the utterance of the target user with respect to the utterance of the robot 100A (step S302). The voice determination unit 117AA extracts, for example, a keyword related to emotion as a feature keyword that characterizes the content of the target user's utterance based on the text data indicating the content of the target user's utterance by the voice recognition unit 114.

  Subsequently, the voice determination unit 117AA determines the voice reaction polarity based on the feature keyword (step S303). For example, the voice determination unit 117AA refers to the voice reaction polarity determination table shown in FIG. 4 stored as the response determination information in the reaction determination information DB 124A of the storage unit 120A, and sets the voice response polarity associated with the extracted feature keyword. Judgment accordingly. For example, when the feature keyword is “like”, “fun” or the like, the voice determination unit 117AA determines the voice reaction polarity as “positive”.

  On the other hand, when it is determined that there is no utterance of the target user after the utterance of the robot 100A (step S301: NO), the voice determination unit 117AA does not clearly know the reaction to the utterance of the robot 100A, so the voice reaction polarity is set to “neutral”. Determination is made (step S304).

  After executing Step S303 or S304, the control unit 110 ends the voice determination process and returns the process to the dialog control process.

  Returning to FIG. 5, after executing the voice determination process (step S107), the control unit 110A (the facial expression determination unit 117BA of the reaction determination unit 117) executes the facial expression determination process (step S108). Here, the facial expression determination process will be described with reference to the flowchart shown in FIG. The facial expression determination process is a process for determining the response of the target user to the utterance of the robot 100A based on the facial expression of the target user.

  When the control unit 110A (the facial expression determination unit 117BA of the reaction determination unit 117A) starts the facial expression determination process, first, the facial image of the target user is obtained from the captured image acquired by the user information acquisition unit 113A after the utterance of the robot 100A in step S106. Extract (step S401).

  Subsequently, the facial expression determination unit 117BA calculates the smile level of the target user based on the face image extracted in step S401 (step S402). For example, the control unit 110 refers to the smile level information stored in the reaction determination information DB 124A, and determines the smile level of the target user from 0 to 0 based on a change in the position of the outer corner of the face image, a change in the size of the mouth, and the like. Calculated in the range of 100%.

  Next, the facial expression determination unit 117BA determines whether or not the smile level of the target user calculated in step S402 is 70% or more (step S403). When the smile level of the target user is 70% or more (step S403: YES), the control unit 110 determines that the facial expression reaction polarity is “positive” (step S405).

  When the smile level of the target user is not 70% or more (step S403: NO), the control unit 110A determines whether the smile level of the target user is 40% or more and less than 70% (step S404). When the smile level of the target user is 40% or more and less than 70% (step S404: YES), the control unit 110 determines that the facial expression reaction polarity is “neutral” (step S406).

  When the smile level of the target user is not 40% or more and less than 70% (step S404: NO), that is, when the smile level of the target user is less than 40%, the control unit 110 sets the facial expression reaction polarity to “negative”. Determination is made (step S407).

  After determining the facial expression reaction polarity of the target user in any of steps S405 to S407, control unit 110A ends the facial expression determination process and returns the process to the dialog control process.

  Returning to FIG. 5, after executing the facial expression determination process (step S108), the control unit 110A executes the behavior determination process (step S109). Here, the action determination process will be described with reference to the flowchart shown in FIG. The action determination process is a process of determining a response of the target user to the utterance of the robot 100A based on the action of the target user.

  When the control unit 110A (behavior determination unit 117CA of the reaction determination unit 117A) starts the behavior determination process, it first determines whether or not the target user is actively moving (step S501). The behavior determination unit 117CA determines based on the movement of the target user in the captured image acquired by the user information acquisition unit 113A after the utterance of the robot 100A in step S106. If it is determined that the target user is actively moving (step S501: YES), the behavior determination unit 117CA determines whether the target user's line of sight is facing the robot 100A (step S502). The behavior determination unit 117CA determines, for example, by specifying the direction of the line of sight of the target user from the position of the pupil in the eye region and the direction of the face in the captured image acquired by the user information acquisition unit 113A.

  When it is determined that the line of sight of the target user is facing the robot 100A (step S502: YES), the behavior determination unit 117CA determines that the behavior reaction polarity is “positive” (step S508). On the other hand, when it is determined that the line of sight of the target user is not facing the robot 100A (step S502: NO), the behavior determination unit 117CA determines that the behavior reaction polarity is “negative” (step S509).

  If it is determined in step S501 that the target user is not actively moving (step S501: NO), the behavior determination unit 117CA determines whether the target user has approached the robot 100A (step S503). The behavior determination unit 117CA determines, for example, according to a change in the size of the face image in the captured image acquired by the user information acquisition unit 113A.

  When it is determined that the target user has approached the robot 100A (step S503: YES), the behavior determination unit 117CA determines whether the target user's line of sight is facing the robot 100A (step S504). When it is determined that the line of sight of the target user is facing the robot 100A (step S504: YES), the behavior determination unit 117CA determines that the behavior response polarity is “positive” (step S508). On the other hand, when it is determined that the line of sight of the target user is not facing the robot 100A (step S504: NO), the behavior determination unit 117CA determines that the behavior reaction polarity is “negative” (step S509).

  If it is determined in step S503 that the target user has not approached the robot 100A (step S503: NO), the behavior determination unit 117CA determines whether the target user has moved away from the robot 100A (step S505). When it is determined that the target user has moved away from the robot 100A (step S505: YES), the behavior determination unit 117CA determines that the behavior reaction polarity is “negative” (step S509).

  On the other hand, when it is determined that the target user has not moved away from the robot 100A (step S505: NO), the behavior determination unit 117C determines whether or not the target user's face has been lost (step S506). When the target user cannot extract the target user's face image from the captured image, for example, by inverting the face direction, the behavior determination unit 117CA determines that the target user's face has been lost. If it is determined that the face portion of the target user has been lost (step S506: YES), the behavior determination unit 117CA determines that the behavior response polarity is “neutral” (step S510).

  When it is determined that the face portion of the target user has not been lost (step S506: NO), the behavior determination unit 117CA determines whether or not the target user's line of sight faces the robot 100A (step S507). If it is determined that the line of sight of the target user is facing the robot 100A (step S507: YES), the behavior determination unit 117CA determines that the behavior reaction polarity is “positive” (step S508). On the other hand, when it is determined that the line of sight of the target user is not facing the robot 100A (step S507: NO), the behavior determination unit 117CA determines that the behavior reaction polarity is “negative” (step S509).

  After determining the behavior response polarity of the target user in any one of steps S508 to S510, control unit 110 ends the behavior determination process and returns the process to the dialogue control process.

  Returning to FIG. 5, after performing the action determination process (step S109), the control unit 110A (preference determination unit 118A) executes the preference determination process (step S110). Here, the preference determination process will be described with reference to the flowchart shown in FIG. The preference determination process is a process of comprehensively determining the degree of preference of the target user with respect to the topic in the conversation between the target user and the robot 100 </ b> A using the determination results of the voice determination process, the expression determination process, and the action determination process.

  When the preference determination unit 118A starts the preference determination process, first, the preference determination unit 118A specifies a topic in the conversation between the target user and the robot 100A (step S601). In step S105 of the dialogue control process, the preference determination unit 118A, when talking to the target user when the target user has not spoken for a predetermined time, if the topic is set in advance, the topic stored in the RAM or the like With reference to the keyword, the topic in the dialogue between the target user and the robot 100A is specified. On the other hand, when a topic is not set in advance, the target user and the robot 100A are extracted by extracting a topic keyword from the speech of the target user based on text data indicating the content of the speech of the target user by the voice recognition unit 114A. Identify topics in the dialogue with. For example, the topic “baseball” is identified from the utterance of the target user “I like baseball”.

  Next, the preference determination unit 118A determines whether or not the voice response polarity determined in the voice determination process of FIG. 7 is “positive” (step S602), and when the voice response polarity is “positive” (step S602). Step S602: YES), the preference level is determined as “preference level A” (step S609).

  When the voice reaction polarity is not “positive” (step S602: NO), the preference determination unit 118A determines whether or not the voice reaction polarity is “negative” (step S603). When the voice reaction polarity is “negative” (step S603: YES), the preference determination unit 118A determines whether or not the facial expression reaction polarity determined in the facial expression determination process of FIG. S604). When the facial expression reaction polarity is “positive” (step S604: YES), the preference determination unit 118A determines the preference level as “preference level B” (step S610). On the other hand, when the facial expression reaction polarity is not “positive” (step S604: NO), the preference determination unit 118A determines the preference level as “preference level D” (step S612).

  In step S603, when the voice response polarity is not “negative” (step S603: NO), the preference determination unit 118A determines whether or not the behavior response polarity determined in the behavior determination process of FIG. 9 is “positive”. Determination is made (step S605). When the behavior reaction polarity is “positive” (step S605: YES), the preference determination unit 118A determines whether the facial expression reaction polarity is “positive” or “neutral” (step S606). When the facial expression reaction polarity is either “positive” or “neutral” (step S606: YES), the preference determination unit 118A determines the preference level as “preference level A” (step S609). On the other hand, when the facial expression reaction polarity is neither “positive” nor “neutral” (step S606: NO), that is, when the facial expression reaction polarity is “negative”, the preference determination unit 118A sets the preference level to “preference level”. C ”(step S611).

  If the behavior response polarity is not “positive” in step S605 (step S605: NO), the preference determination unit 118A determines whether the behavior response polarity is “neutral” (step S607), and the behavior response polarity. Is not “neutral” (step S607: NO), the preference determination unit 118A determines the preference level to be “preference level C” (step S611).

  On the other hand, when the action response polarity is “neutral” (step S607: YES), the preference determination unit 118A determines whether the facial expression reaction polarity is “positive” (step S608). When the facial expression reaction polarity is “positive” (step S608: YES), the preference determination unit 118A determines the preference level as “preference level B” (step S610), and the facial expression reaction polarity is not “positive” (step S610). Step S608: NO), the preference level is determined as “preference level D” (Step S612).

  In any of step S609 to step S612, after determining the preference level of the target user, the preference determination unit 118A ends the preference determination process and returns the process to the dialogue control process.

  Returning to FIG. 5, after performing the preference determination process (step S110), the control unit 110A reflects the preference determination result in the preference level information (step S111). The control unit 110A uses, as the preference determination result in the preference determination process, information in which the topic and the preference level in the conversation between the target user and the robot 100A are associated with each other in the preference information of the user information stored in the user information DB 121A. Add and update preference information. Thereby, the preference level information is updated for each user USR. The topic in the dialogue between the target user and the robot 100A is the topic indicated by the topic keyword stored in the RAM or the like. In addition, the control unit 110A controls the communication unit 170A to transmit information in which the topic and the preference level in the dialogue between the target user and the robot 100A are associated with the robot 100B. Similarly, the robot 100B that has received this information adds the preference information of the user information stored in the user information DB 121B and updates the preference information. Thereby, the robot 100A and the robot 100B can share the respective preference determination results. Note that the preference level initial value included in the preference level information stored in association with each of a plurality of topics is set to the preference level A. Thus, the control unit 110A (110B) and the communication unit 170A (170B) including the reaction determination unit 117A (117B), the preference determination unit 118A (118B) function as the reaction acquisition unit of the present invention.

  After executing the process of step S111, the control unit 110A determines whether or not the target user exists around the robot 100A (step S112). If it is determined that there is a target user around the robot 100A (step S112: YES), the control unit 110A determines that the conversation with the target user can be continued, and returns the process to step S103. In step S103 in the case where this step S112 is YES, it is determined whether or not the elapsed time from the completion of the utterance in step S106 is within a predetermined time.

  On the other hand, when it is determined that the target user does not exist around the robot 100A (step S112: NO), the control unit 110A determines that the conversation with the target user cannot be continued, and the robot 100B (another robot). The communication connection with is canceled (step S113). The control unit 110A controls the communication unit 170A to release a communication connection with the robot 100B by performing a predetermined procedure based on the communication method. Thereafter, the control unit 110A ends the dialogue control process.

  The above is the dialogue control process executed by the control unit 110A of the robot 100A, but the dialogue control process executed by the control unit 110B of the robot 100B is the same. As shown in FIG. 5, the control unit 110B starts the dialogue control process. The user specifying process is executed as shown in FIG.

  If it is determined in step S103 in FIG. 5 that the target user has spoken within a predetermined time (step S103: YES), the control unit 110B (speech control unit 115B) determines that an interaction with the target user is being performed. Then, the content of the utterance as a response to the utterance of the target user is determined (step S104). The control unit 110B (the utterance control unit 115B) refers to the utterance information DB 123B and the user information DB 121B of the storage unit 120B, and determines a topic candidate corresponding to the utterance content of the target user and suitable for the target user's preference. .

  In this step S104, when there is one determined topic candidate, this is determined as the final topic. On the other hand, when a plurality of topic candidates are determined and the utterance history information is stored in the storage unit 120A of the robot 100A, the control unit 110B (the utterance control unit 115B) is stored in the storage unit via the communication unit 170B. The utterance history information stored in 120A is read. Then, the control unit 110B (the utterance control unit 115B) has a topic that is the same as or related to any of a plurality of topic candidates in the read utterance history information, and the present (that is, the utterance start of the robot 100B) from the utterance date and time. It is determined whether or not there is a subject whose elapsed time is within a predetermined elapsed time (hereinafter referred to as “second comparison target topic”).

  When it is determined that the second comparison target topic exists, the control unit 110B (the utterance control unit 115B) excludes a plurality of topic candidates that match or relate to the second comparison target topic, and finally Determine the topic.

  On the other hand, when a plurality of topic candidates are determined, it is determined that no utterance history information is stored in the storage unit 120A of the robot 100A or that the second comparison target topic does not exist in the utterance history information. Then, one topic randomly selected from the plurality of determined topic candidates is determined as the final topic. The utterance control unit 115B outputs text data indicating the utterance content along the topic determined as described above.

  On the other hand, when it is determined that the target user does not speak within the predetermined time (step S103: NO), the control unit 110B (speech control unit 115B) determines the content of the utterance to be spoken to the target user (step S105). At this time, the control unit 110B (the utterance control unit 115B) refers to the utterance information DB 123B and the user information DB 121B in the storage unit 120B, and determines a plurality of topic candidates that match the preferences of the target user stored in the user information DB 121B. To do. In this case, topics corresponding to the preference degrees A and B are determined as topic candidates as topics that match the preferences of the target user.

  In this step S105, when there is one determined topic candidate, this is determined as the final topic. On the other hand, when a plurality of topic candidates are determined, the final topic is selected from the plurality of topic candidates in the same manner as in step S104. Specifically, the control unit 110B (speech control unit 115B) determines that the control unit 110B (speech utterance) is stored when utterance history information is stored in the storage unit 120A of the robot 100A when a plurality of topic candidates are determined. The control unit 115B) reads the utterance history information stored in the storage unit 120A via the communication unit 170B. Then, the control unit 110B (the utterance control unit 115B) determines whether or not the second comparison target topic exists in the read utterance history information.

  When it is determined that the second comparison target topic exists, the control unit 110B (the utterance control unit 115B) excludes a plurality of topic candidates that match or relate to the second comparison target topic, and finally selects the topic. decide.

  On the other hand, when a plurality of topic candidates are determined, it is determined that no utterance history information is stored in the storage unit 120A of the robot 100A or that the second comparison target topic does not exist in the utterance history information. Then, one topic randomly selected from the plurality of determined topic candidates is determined as the final topic.

  The control unit 110B utters based on the utterance content along the determined topic (step S106), and when the sound is output, the sound determination process shown in FIG. 7 for determining the reaction of the target user, shown in FIG. The facial expression determination process and the action determination process shown in FIG. 9 are executed. When the action determination process ends, the preference determination process shown in FIG. 10 is executed. The control unit 110B adds the preference determination result in the preference determination process to the preference information of the user information stored in the user information DB 121B, and updates the preference information. In addition, the control unit 110B controls the communication unit 170B to transmit information in which the topic and the preference level in the conversation between the target user and the robot 100B are associated with the robot 100A. Similarly, the robot 100A that has received this information adds the preference information of the user information stored in the user information DB 121A and updates the preference information. Thereby, the robot 100A and the robot 100B share each preference determination result.

  In the above-described first embodiment, when one of the robots 100A and 100B speaks and the other robot speaks within the predetermined elapsed time, the topic of the other robot speaks as the other robot. The topic is determined to be different from the topic spoken by one of the robots within a predetermined elapsed time before the utterance. In other cases, topics uttered by the robots 100A and 100B are determined independently of each other (independently from each other) without being linked to each other. Instead of this determination method, when the number of preference information of the target user stored in the user information DB 121A (DB 121B) is smaller than a predetermined threshold, the topics that the robots 100A and 100B speak are determined as different topics. However, the topics spoken by the robots 100A and 100B may be determined independently of each other when the predetermined threshold value is exceeded. That is, when the predetermined condition is satisfied, the topics uttered by the robots 100A and 100B are determined as different topics, and when the predetermined condition is not satisfied, the topics uttered by the robots 100A and 100B are mutually determined. It may be determined independently. Alternatively, the topics (speech contents) spoken by the robots 100A and 100B may be determined independently of each other without being linked to each other regardless of predetermined conditions.

(Second Embodiment)
In the embodiment described above, the robot 100A and the robot 100B have the functions of reaction determination and speech control, respectively, but these functions may exist separately from the robot 100A and the robot 100B. In this embodiment, an external server capable of communicating with the robot 100A and the robot 100B is provided, and the server performs processing for reaction determination and speech control between the robot 100A and the robot 100B.

  The dialogue system 1 according to the present embodiment includes a robot 100A, a robot 100B, and a server 200 as shown in FIG.

  Similar to the first embodiment, the robot 100A includes a control unit 110A, a storage unit 120A, an imaging unit 130A, an audio input unit 140A, an audio output unit 150A, a moving unit 160A, and a communication unit 170A. However, unlike the case of the first embodiment, the control unit 110A does not include the speech control unit 115A, the reaction determination unit 117A, and the preference determination unit 118A. Further, unlike the case of the first embodiment, the storage unit 120A does not include the user information DB 121A, the voice information DB 122A, the speech information DB 123A, and the reaction determination information DB 124A. The configuration of the robot 100B is the same as that of the robot 100A, and includes a control unit 110B, a storage unit 120B, an imaging unit 130B, a voice input unit 140B, a voice output unit 150B, a moving unit 160B, and a communication unit 170B. Control unit 110B does not include speech control unit 115B, reaction determination unit 117B, and preference determination unit 118B. The storage unit 120B does not include the user information DB 121B, the voice information DB 122B, the utterance information DB 123B, and the reaction determination information DB 124B.

  The server 200 includes a control unit 210, a storage unit 220, and a communication unit 270. The control unit 210 includes an utterance control unit 215, a reaction determination unit 217, and a preference determination unit 218. That is, instead of the robot 100A and the robot 100B, the server 200 performs various processes for controlling the utterances of the robot 100A and the robot 100B, determining the user's reaction, determining the user's preference, and the like. The storage unit 220 includes a user information DB 221, a voice information DB 222, an utterance information DB 223, and a reaction determination information DB 224. In other words, the database included in the robot 100A and the robot 100B is collected in the server 200. In addition, the storage unit 220 stores, for each user USR, utterance history information including the utterance date and time of the utterances of the robot 100A and the robot 100B and the uttered topics. The server 200 performs wireless data communication with the robot 100A and the robot 100B via the communication unit 270, the communication unit 170A of the robot 100A, and the communication unit 170B of the robot 100B. Thereby, the server 200 controls the robot 100A and the robot 100B to interact with the target user. Thus, the communication unit 270 functions as the first communication unit of the present invention. The communication units 170A and 170B function as the second communication unit of the present invention.

  Next, the dialogue control process in this embodiment will be described. Here, the dialogue control process of the robot 100A will be described as an example. The control unit 110A of the robot 100A starts the dialogue control process when the user detection unit 111A detects the user USR around the robot 100A.

  110 A of control parts will perform a user specific process first, if a dialog control process (refer FIG. 5) is started. The control unit 110A searches for a registered user corresponding to the face image extracted from the captured image acquired from the imaging unit 130A. The control unit 110A (user specifying unit 112A) accesses the user information DB 221 of the storage unit 220 of the server 200, and extracts the face image extracted from the captured image as the face image of each of a plurality of users stored in the user information DB 221. And the user USR is specified as the target user. Here, the control unit 210 functions as an object specifying unit of the present invention.

  When the control unit 210 of the server 200 that has received the information of the user USR determines that the target user has uttered within a predetermined time, the control unit 210 (the utterance control unit 215) is executing a dialog with the target user. The content of the utterance as a response to the utterance of the target user is determined. The control unit 210 (the utterance control unit 215) refers to the utterance information DB 223 and the user information DB 221 in the storage unit 220, and determines topic candidates that correspond to the utterance content of the target user and that match the preference of the target user. .

  When there is one determined topic candidate, this is determined as the final topic. On the other hand, when a plurality of topic candidates are determined and the utterance history information of the robot 100B is stored in the storage unit 220, the control unit 210 (the utterance control unit 215) stores the utterance stored in the storage unit 220. The history information is read, and it is determined whether or not the first comparison target topic exists in the read utterance history information.

  When the control unit 210 (the utterance control unit 215) determines that the first comparison target topic exists, the control unit 210 (utterance control unit 215) excludes from the plurality of topic candidates those that match or relate to the first comparison target topic, and finally Determine the topic.

  On the other hand, when a plurality of topic candidates are determined, the determination is made when no utterance history information of the robot 100B is stored, or when it is determined that the first comparison target topic does not exist in the utterance history information. One topic randomly selected from the plurality of topic candidates is determined as the final topic. The utterance control unit 215 outputs text data indicating the utterance content along the topic determined as described above.

  On the other hand, when it is determined that the target user does not speak within a predetermined time, the control unit 210 (speech control unit 215) determines the content of the utterance to be spoken to the target user. At this time, the utterance control unit 215 refers to the utterance information DB 223 and the user information DB 221 in the storage unit 220 and determines a plurality of topic candidates that match the preference of the target user stored in the user information DB 221.

  When there is one determined topic candidate, this is determined as the final topic. On the other hand, when a plurality of topic candidates are determined, a final topic is selected from the plurality of topic candidates. When a plurality of topic candidates are determined and the utterance history information of the robot 100B is stored, the control unit 210 (the utterance control unit 215) reads out the utterance history information and there is a first comparison target topic. It is determined whether or not.

  When the control unit 210 (the utterance control unit 215) determines that the first comparison target topic exists, the control unit 210 (utterance control unit 215) excludes a plurality of topic candidates that match or relate to the first comparison target topic, and finally selects the topic. decide.

  On the other hand, when a plurality of topic candidates are determined, the determination is made when no utterance history information of the robot 100B is stored, or when it is determined that the first comparison target topic does not exist in the utterance history information. One topic randomly selected from the plurality of topic candidates is determined as the final topic.

  The robot 100A receives the text data via the communication unit 170A and sends it to the speech synthesis unit 116A. The speech synthesis unit 116A accesses the speech information DB 222 of the storage unit 220 of the server 200, and generates speech data from the text data using an acoustic model or the like stored in the speech information DB 222. The voice synthesizer 116A controls the voice output unit 150A to output the generated voice data as a voice.

  Subsequently, a reaction determination process (see FIGS. 7 to 9) for determining the reaction of the target user with respect to the utterance of the robot 100A is executed.

  Control unit 210 (voice determination unit 217A of reaction determination unit 217) executes a voice determination process (see FIG. 7). The voice determination unit 217A determines the response of the target user to the utterance of the robot 100A based on the voice uttered by the target user after the robot 100A utters. When the target user speaks, the speech recognition unit 114A of the robot 100A accesses the speech information DB 222 of the storage unit 220 of the server 200 and generates text data from the speech data using an acoustic model or the like stored in the speech information DB 222. To do. The text data is transmitted to the server 200. Based on the text data received through the communication unit 270, the voice determination unit 217A determines the response of the target user to the utterances of the robot 100A and the robot 100B.

  After executing the voice determination process, the control unit 210 (the facial expression determination unit 217B of the reaction determination unit 217) executes the facial expression determination process (see FIG. 8). The facial expression determination unit 217B determines the response of the target user to the speech of the robot 100A based on the facial expression of the target user after the robot 100A speaks. When the user information acquisition unit 113A of the robot 100A acquires the user's captured image, the captured image is transmitted to the server 200 via the communication unit 170A. The facial expression determination unit 217B detects the feature amount of the target user's face detected from the captured image acquired via the communication unit 270 with reference to the smile degree information stored in the reaction determination information DB 224 of the storage unit 220. The smile level of the target user is calculated based on the feature amount. The facial expression determination unit 217B determines the response of the target user to the utterance of the robot 100A according to the calculated smile level.

  After executing the facial expression determination process, the control unit 210 executes an action determination process (see FIG. 9). The behavior determination unit 217C determines the response of the target user to the speech of the robot 100A based on the behavior of the target user after the robot 100A speaks. The behavior determination unit 217C determines the response of the target user to the utterance of the robot 100A based on the behavior of the target user detected from the captured image via the communication unit 270.

  After executing the behavior determination process, the control unit 210 (preference determination unit 218A) executes the preference determination process (see FIG. 10). The preference determination unit 218 identifies a topic in the conversation between the target user and the robot 100A, and determines a preference level indicating the level of preference of the target user with respect to the topic based on each determination result by the reaction determination unit 217.

  After executing the preference determination process, the control unit 210 reflects the preference determination result in the preference degree information. As the preference determination result in the preference determination process, the control unit 210 uses information in which the topic and the preference level in the conversation between the target user and the robot 100A are associated with each other as the preference level information of the user information stored in the user information DB 221. Add and update preference information. Thereby, preference information is updated for every user USR.

  A similar control process is performed for the robot 100B. In the first embodiment, the robot 100A updates the preference level information in the dialogue between the target user and the robot 100A, and transmits the preference information to the robot 100B. The preference level information stored in the DB 121B is updated. Thereby, the robot 100A and the robot 100B can share the respective preference determination results. On the other hand, in the present embodiment, the preference information of the robot 100A and the robot 100B is stored in the user information DB 221 of the server 200 for each user USR, so that it is not necessary to update the preference information of each other.

  In the above-described embodiment, the server 200 executes various processes such as control of the utterances of the robot 100A and the robot 100B, determination of user reaction, determination of user preference, and the like. However, the present invention is not limited to this, and the server 200 can select and execute an arbitrary process of the robot 100A and the robot 100B. For example, the control unit 210 of the server 200 may include only the utterance control unit 215, execute only the utterance control processing of the robot 100A and the robot 100B, and perform other processing by the robot 100A and the robot 100B. Further, the server may execute all the processes of user detection, user identification, user information acquisition, voice recognition, voice synthesis, speech control, reaction determination, and preference determination for the robot 100A and the robot 100B. In the present embodiment, the storage unit 220 of the server 200 includes a user information DB 221, a voice information DB 222, an utterance information DB 223, and a reaction determination information DB 224. However, the present invention is not limited to this, and the server 200 can include an arbitrary database. For example, in this embodiment, each of the robot 100A and the robot 100B may include the voice information DB 222 instead of the server 200. Further, not only the server 200 but also each of the robot 100A and the robot 100B may have face information that identifies a user in the user information DB 221. Thus, the robot 100A and the robot 100B do not need to access the server 200 when performing speech recognition, speech synthesis, or user identification.

  As described above, according to the present embodiment, the dialogue system 1 includes the robot 100A and the robot 100B, or includes the server 200 in addition to the robot 100A and the robot 100A (or the robot 100B or the server 200). Based on the result of determining the response of the target user to the utterance by the robot 100A (ie, the preference information of the target user) and the result of determining the response of the target user to the utterance by the robot 100B (ie, the preference information of the target user) The utterance by at least one of 100A and robot 100B is controlled. As a result, it is possible to accurately grasp the preferences of the target user and perform a dialogue that matches the preferences of the target user.

  In addition, the control unit 110A of the robot 100A (or the control unit 110B of the robot 100B or the server 200) determines the response of the target user to the utterance by the robot 100A and the determination result of the response of the target user to the utterance by the robot 100B. Based on the above, the utterance by at least one of the robot 100A and the robot 100B is controlled. Therefore, it is possible to efficiently grasp the preferences of the target user and accurately reflect the preferences of the target user in subsequent utterances.

  In addition, this invention is not limited to the said embodiment, A various deformation | transformation and application are possible. The embodiment described above may be modified as follows.

  In the above-described embodiment, the robot 100A and the robot 100B are provided in a place where mutual speech is not recognized by the target user. On the other hand, a modified example in which the robot 100A and the robot 100B are provided in a place where the mutual utterance is recognized by the target user will be described. In this case, the robot 100A and the robot 100B can interact with the target user at the same time. However, if the utterance times of the robot 100A and the robot 100B overlap or continue, it may not be possible to appropriately determine which utterance the target user has reacted to. Then, the target user's preference information cannot be appropriately acquired, and an appropriate response cannot be made. Therefore, the utterance control unit 115A sets the utterance start time of the robot 100A to the utterance control unit 115B of the robot 100B in order to prevent the utterance times of the robot 100A and the robot 100B from overlapping each other or from being continuous with each other. Decide in cooperation. The utterance control unit 115 </ b> A determines the utterance start time of the robot 100 </ b> A so that the utterance interval between the robot 100 </ b> A and the robot 100 </ b> B is equal to or longer than a predetermined time such as a time sufficient for determining the reaction of the target user. Further, the utterance control unit 115B of the robot 100B determines the utterance start time of the start robot 100B so that the robot 100B does not utter continuously during the utterance of the robot 100A and immediately after the end of the utterance. The utterance start timing of the robot 100A and the robot 100B may be determined by either one of the utterance control units 115A and 115B. When the server 200 controls the utterances of the robot 100A and the robot 100B, the utterance control unit 215 determines both utterance start times. Thereby, the utterances by the robot 100A and the robot 100B are not performed continuously, but are performed at timings different from each other by a predetermined time or more. Accordingly, it is possible to accurately grasp the target user's preference and perform a conversation that matches the target user's preference.

  Furthermore, in the above-described modification, the utterance control unit 115A may determine the topics uttered by the robot 100A and the robot 100B as different topics in cooperation with the utterance control unit 115B of the robot 100B. In this case, as in the first embodiment, when one of the robots 100A and 100B speaks and the other robot speaks within the predetermined elapsed time, the topic spoken by the other robot Is determined to be a topic different from the topic spoken by one robot within a predetermined elapsed time before the other robot speaks, otherwise the topics spoken by the robots 100A and 100B are not linked to each other, It may be determined independently of each other (independently of each other). Alternatively, in this case, when the number of target user's preference information stored in the user information DB 121A (DB 121B) is smaller than a predetermined threshold, the topics spoken by the robots 100A and 100B are determined as different topics. The topics spoken by the robots 100A and 100B when they are equal to or greater than a predetermined threshold may be determined independently of each other. Alternatively, regardless of the predetermined condition as described above, the topics (speech contents) spoken by the robots 100A and 100B may always be determined independently of each other without being linked to each other.

  Further, for example, a movement control unit that controls the moving unit 160A according to the utterance control of the utterance control unit 115A may be provided. For example, the movement control unit may control the moving unit 160A so that the robot 100A approaches the target user when the robot 100A starts speaking.

  For example, a master / slave method is adopted for a plurality of robots 100 constituting the dialogue system 1. For example, the robot 100 functioning as a master determines collectively the utterance content of the robot 100 functioning as a slave, May be instructed to speak based on the determined utterance content. In this case, the determination method of the robot 100 functioning as the master and the robot 100 functioning as the slave is arbitrary. For example, the robot that first detects and identifies the surrounding user USR functions as the master, and the other robot 100 May function as a slave. Further, for example, the robot 100 that is first turned on by the user USR may function as a master, and then the robot 100 that is turned on as a slave may function as a slave, or the user USR may perform a physical switch or the like. The robot 100 that functions as a master and the robot 100 that functions as a slave may be set.

  Further, the robot 100 functioning as a master and the robot 100 functioning as a slave may be fixed. In this case, some functions that can be executed by the robot 100 functioning as a slave may be omitted. For example, when speaking in accordance with an instruction from the robot 100 functioning as a master, the robot 100 functioning as a slave may not have a function corresponding to the speech control unit 115A or the like.

  In the above-described embodiment, an example in which the robot 100A and the robot 100B interact with the target user has been described. However, the robot 100A and the target user may be configured to interact with the target user. In this case, for example, as in the case where one robot 100 functions as the master described above, the content of the utterance of itself and the content of speech of other robots are determined in a lump, and the determined utterance content is changed by changing the voice color or the like. By sequentially outputting voices, it may be produced as if one robot 100 is speaking for another robot.

  In the above embodiment, the case where the dialog system 1 is a robot system including a plurality of robots 100 has been described as an example. However, the dialog system 1 includes a plurality of dialog devices including all or a part of the configuration included in the robot 100. It may be constituted by.

  In the above-described embodiment, the control program executed by the CPUs of the control units 110A and 110B is stored in advance in a ROM or the like. However, the present invention is not limited to this, and by implementing a control program for executing the above-described various processes in an electronic device such as an existing general-purpose computer, a framework, or a workstation, You may make it function as an apparatus equivalent to robot 100A, 100B concerning a form. For example, speech devices corresponding to the robots 100A and 100B include portable terminals having a voice assistant function, digital signage, and the like. Digital signage is a system that displays video and information on an electronic display device such as a display. Note that the utterance is not limited to outputting the sound through the speaker, but includes displaying the character on a display device. Therefore, portable terminals, digital signage, and the like that display utterances as characters are also included as utterance devices corresponding to the robots 100A and 100B.

  The method of providing such a program is arbitrary. For example, the program is stored and distributed on a computer-readable recording medium (flexible disc, CD (Compact Disc) -ROM, DVD (Digital Versatile Disc) -ROM) or the like. Alternatively, the program may be stored in a storage on a network such as the Internet and provided by downloading it.

  Further, when the above processing is executed by sharing an OS (Operating System) and an application program, or by cooperation between the OS and the application program, only the application program may be stored in a recording medium or storage. It is also possible to superimpose a program on a carrier wave and distribute it via a network. For example, the program may be posted on a bulletin board (BBS) on a network and distributed through the network. Then, the distributed processing may be executed by starting the distributed program and executing it in the same manner as other application programs under the control of the OS.

  The present invention is capable of various embodiments and modifications without departing from the broad spirit and scope of the present invention. The above-described embodiments are for explaining the present invention and do not limit the scope of the present invention. In other words, the scope of the present invention is shown not by the embodiments but by the claims. Various modifications made within the scope of the claims and within the scope of the meaning of the invention equivalent to the claims are considered to be within the scope of the present invention.

  The invention described in the scope of the claims of the present application will be appended below.

(Appendix)
(Appendix 1)
A result of determining a response of a predetermined object to an utterance by the first utterance device, and a result of determining a response of the predetermined object to an utterance by a second utterance device provided separately from the first utterance device Reaction acquisition means for acquiring a plurality of reaction determination results;
Utterance control means for controlling the utterance by at least one of a plurality of utterance devices including the first and second utterance devices based on a plurality of reaction determination results acquired by the reaction acquisition means;
Comprising
An interactive control device characterized by the above.

(Appendix 2)
The reaction acquisition means includes a place where the utterance is performed on the predetermined object by the first utterance device and a place where the utterance is performed on the predetermined object by the second utterance device. The response of the predetermined object to each of the utterances by the first and second utterance devices is determined when the predetermined object cannot recognize both of the utterances by the first and second utterance devices. Obtaining the plurality of reaction determination results including the results;
The dialogue control apparatus according to supplementary note 1, wherein:

(Appendix 3)
The utterance control means controls the utterances by the first and second utterance devices to be performed at timings different from each other at a predetermined time or more, without being performed continuously with each other;
The dialogue control apparatus according to supplementary note 1, wherein:

(Appendix 4)
The speech control means determines the topic of the speech by the first and second speech devices as different topics.
The dialog control apparatus according to appendix 1 or 2, characterized in that:

(Appendix 5)
The utterance control means determines the content of the utterance by the first and second utterance devices independently of each other;
The dialog control apparatus according to appendix 1 or 2, characterized in that:

(Appendix 6)
The plurality of response determination results determine the response of the predetermined target to the utterances by the first and second utterance devices based on at least one of a voice uttered by the predetermined target and a captured image of the predetermined target. Is the result of
6. The dialogue control device according to any one of appendices 1 to 5, characterized in that:

(Appendix 7)
An acquisition means for acquiring at least one of a sound uttered by the predetermined target and a captured image of the predetermined target;
The reaction acquisition means includes
Determining the reaction of the predetermined target with respect to the utterance by each of the first and second utterance devices based on at least one of the voice and the captured image acquired by the acquisition means; Get the response judgment result of
The dialogue control device according to any one of supplementary notes 1 to 6, characterized in that:

(Appendix 8)
The reaction acquisition means includes
Based on the voice acquired by the acquisition means, voice determination means for determining the content of the voice of the predetermined target for the utterance by each of the first and second utterance devices;
Facial expression determination means for determining the facial expression of the predetermined target for the utterance by each of the first and second utterance devices based on the captured image acquired by the acquisition means;
Action determining means for determining the action of the predetermined target for the utterance by each of the first and second utterance devices based on the captured image acquired by the acquiring means;
Having at least one of
Determining a response of the predetermined target to the utterance by each of the first and second utterance devices based on the determination result of at least one of the voice determination unit, the facial expression determination unit, and the action determination unit; To obtain the plurality of reaction determination results,
The dialog control device according to appendix 7, characterized by:

(Appendix 9)
The response acquisition means is a positive response, a negative response, a positive or negative response to the response of the predetermined target based on at least one of the voice, the facial expression, and the action of the predetermined target. Determining the response of the given subject by classifying it into a neutral response that is not
9. The dialogue control device according to appendix 8, wherein

(Appendix 10)
A topic in a dialogue with the predetermined target is specified based on at least one of the voice uttered by the predetermined target, the utterance by the first utterance device, and the utterance by the second utterance device. Specific means,
A preference determination unit that determines a preference level indicating a degree of preference of the predetermined target with respect to the topic identified by the identification unit based on the plurality of obtained reaction determination results;
Further comprising
The utterance control means controls the utterance by the at least one of the plurality of utterance devices based on the preference degree determined by the preference determination means;
The dialogue control device according to any one of appendices 7 to 9, characterized in that:

(Appendix 11)
The preference is a hobby or preference related to things regardless of tangible or intangible, and in addition to a hobby or preference related to food, sports, or the weather, to the utterance content of at least one of the first utterance device and the second utterance device. Including preferences,
Item 11. The dialog control device according to appendix 10, wherein

(Appendix 12)
The preference determination means determines the preference level in a plurality of stages in descending order of the preference of the predetermined target with respect to the topic,
The utterance control unit controls the utterance by the at least one of the plurality of utterance devices based on the information of the plurality of stages indicating the preference degree determined by the preference determination unit;
12. The dialogue control apparatus according to appendix 10 or 11, characterized by the above.

(Appendix 13)
The predetermined object includes a person or an animal or a robot,
13. The dialogue control device according to any one of appendices 1 to 12, characterized in that:

(Appendix 14)
A target specifying means for specifying the predetermined target from a plurality of different targets;
The reaction acquisition unit is configured to determine the response of the specified predetermined target with respect to the utterance by the first utterance device and the utterance by the second utterance device provided separately from the first utterance device. Obtaining a plurality of response determination results including a result of determining the response of the specified predetermined target,
The dialog control device according to any one of appendices 1 to 13, characterized in that:

(Appendix 15)
The dialogue control device according to any one of appendices 1 to 14, wherein the dialogue control device is provided in at least one of the first and second utterance devices.

(Appendix 16)
The dialogue control device according to any one of appendices 1 to 14, wherein the dialogue control device is provided separately from the first and second utterance devices.

(Appendix 17)
A dialogue system comprising a first utterance device and a second utterance device configured to be utterable, and a dialogue control device,
The dialogue control device comprises:
A result of determining a response of a predetermined object to an utterance by the first utterance device, and a result of determining a response of the predetermined object to an utterance by the second utterance device provided separately from the first utterance device; Reaction acquisition means for acquiring a plurality of reaction determination results including:
Utterance control means for controlling the utterance by at least one of a plurality of utterance devices including the first and second utterance devices based on a plurality of reaction determination results acquired by the reaction acquisition means;
Comprising
An interactive system characterized by that.

(Appendix 18)
Each of the first and second utterance devices includes:
Acquisition means for acquiring at least one of a sound uttered by the predetermined target and a captured image of the predetermined target;
First communication means,
The dialogue control device further includes second communication means for communicating with the first and second utterance devices via the first communication means,
The reaction acquisition means of the dialog control device comprises:
The first data which is at least one of the voice and the captured image acquired by the acquisition unit of the first utterance device is acquired via the first and second communication units, and the acquired The determination result of the response of the predetermined object to the utterance by the first utterance device by determining the reaction of the predetermined object to the utterance by the first utterance device based on the first data. Get the response judgment result,
The second data that is at least one of the voice and the captured image acquired by the acquisition unit of the second utterance device is acquired via the first and second communication units, and the acquired The determination result of the response of the predetermined object to the utterance by the second utterance device by determining the response of the predetermined object to the utterance by the second utterance device based on the second data. Get the response judgment result,
The utterance control means of the dialog control device is configured to pass through the second and first communication means based on the plurality of reaction determination results including the first and second reaction determination results acquired by the reaction acquisition means. Controlling the utterance by the first and second utterance devices;
18. The dialogue system according to appendix 17, characterized by that.

(Appendix 19)
A result of determining a response of a predetermined object to an utterance by the first utterance device, and a result of determining a response of the predetermined object to an utterance by a second utterance device provided separately from the first utterance device Processing for obtaining a plurality of reaction determination results;
A process for controlling the utterance by at least one of a plurality of utterance devices including the first and second utterance devices based on the plurality of obtained reaction determination results;
including,
A dialog control method characterized by the above.

(Appendix 20)
Computer
A result of determining a response of a predetermined object to an utterance by the first utterance device, and a result of determining a response of the predetermined object to an utterance by a second utterance device provided separately from the first utterance device Reaction acquisition means for acquiring a plurality of reaction determination results;
Utterance control means for controlling the utterance by at least one of a plurality of utterance devices including the first and second utterance devices based on a plurality of reaction determination results acquired by the reaction acquisition means;
Function as
A program characterized by that.

DESCRIPTION OF SYMBOLS 1 ... Dialog system, 100, 100A, 100B ... Robot, 101 ... Torso part, 102 ... Head part, 103 ... Hand part, 104 ... Foot part, 105 ... Eye part, 106 ... Mouth part, 107 ... Ear part, 110A, 110B, 210 ... control unit, 111A, 111B ... user detection unit, 112A, 112B ... user identification unit, 113A, 113B ... user information acquisition unit, 114A, 114B ... voice recognition unit, 115A, 115B, 215 ... utterance control unit, 116A, 116B: Speech synthesis unit, 117A, 117B, 217 ... Reaction determination unit, 117AA, 117AB, 217A ... Speech determination unit, 117BA, 117BB, 217B ... Facial expression determination unit, 117CA, 117CB, 217C ... Behavior determination unit, 118A, 118B, 218 ... Preference determination unit, 120A, 120B, 220 ... Storage unit, 121A, 21B, 221 ... user information DB, 122A, 122B, 222 ... voice information DB, 123A, 123B, 223 ... utterance information DB, 124A, 124B, 224 ... reaction determination information DB, 130A, 130B ... imaging unit, 140A, 140B ... Voice input unit, 150A, 150B ... voice output unit, 160A, 160B ... moving unit, 170A, 170B, 270 ... communication unit, 200 ... server, BL ... bus line, USR ... user

Claims (20)

A result of determining a response of a predetermined object to an utterance by the first utterance device, and a result of determining a response of the predetermined object to an utterance by a second utterance device provided separately from the first utterance device Reaction acquisition means for acquiring a plurality of reaction determination results;
Utterance control means for controlling the utterance by at least one of a plurality of utterance devices including the first and second utterance devices based on a plurality of reaction determination results acquired by the reaction acquisition means;
Comprising
An interactive control device characterized by the above. The reaction acquisition means includes a place where the utterance is performed on the predetermined object by the first utterance device and a place where the utterance is performed on the predetermined object by the second utterance device. The response of the predetermined object to each of the utterances by the first and second utterance devices is determined when the predetermined object cannot recognize both of the utterances by the first and second utterance devices. Obtaining the plurality of reaction determination results including the results;
The dialogue control apparatus according to claim 1, wherein The utterance control means controls the utterances by the first and second utterance devices to be performed at timings different from each other at a predetermined time or more, without being performed continuously with each other;
The dialogue control apparatus according to claim 1, wherein The speech control means determines the topic of the speech by the first and second speech devices as different topics.
The dialogue control apparatus according to claim 1 or 2, wherein The utterance control means determines the content of the utterance by the first and second utterance devices independently of each other;
The dialogue control apparatus according to claim 1 or 2, wherein The plurality of response determination results determine the response of the predetermined target to the utterances by the first and second utterance devices based on at least one of a voice uttered by the predetermined target and a captured image of the predetermined target. Is the result of
6. The dialogue control apparatus according to claim 1, wherein An acquisition means for acquiring at least one of a sound uttered by the predetermined target and a captured image of the predetermined target;
The reaction acquisition means includes
Determining the reaction of the predetermined target with respect to the utterance by each of the first and second utterance devices based on at least one of the voice and the captured image acquired by the acquisition means; Get the response judgment result of
The dialogue control device according to claim 1, wherein The reaction acquisition means includes
Based on the voice acquired by the acquisition means, voice determination means for determining the content of the voice of the predetermined target for the utterance by each of the first and second utterance devices;
Facial expression determination means for determining the facial expression of the predetermined target for the utterance by each of the first and second utterance devices based on the captured image acquired by the acquisition means;
Action determining means for determining the action of the predetermined target for the utterance by each of the first and second utterance devices based on the captured image acquired by the acquiring means;
Having at least one of
Determining a response of the predetermined target to the utterance by each of the first and second utterance devices based on the determination result of at least one of the voice determination unit, the facial expression determination unit, and the action determination unit; To obtain the plurality of reaction determination results,
The dialogue control apparatus according to claim 7. The response acquisition means is a positive response, a negative response, a positive or negative response to the response of the predetermined target based on at least one of the voice, the facial expression, and the action of the predetermined target. Determining the response of the given subject by classifying it into a neutral response that is not
The dialogue control apparatus according to claim 8, wherein: The topic in the dialogue with the predetermined target is specified based on at least one of the voice uttered by the predetermined target, the utterance by the first utterance device, and the utterance by the second utterance device. Specific means,
A preference determination unit that determines a preference level indicating a degree of preference of the predetermined target with respect to the topic identified by the identification unit based on the plurality of obtained reaction determination results;
Further comprising
The utterance control means controls the utterance by the at least one of the plurality of utterance devices based on the preference degree determined by the preference determination means;
10. The dialogue control apparatus according to claim 7, wherein the dialogue control apparatus is any one of claims 7 to 9. The preference is a hobby or preference regarding things regardless of tangible or intangible. In addition to a hobby or preference regarding food, sports, or weather, the preference is for at least one of the utterance contents of the first utterance device and the second utterance device. Including preferences,
The dialogue control apparatus according to claim 10. The preference determination means determines the preference level in a plurality of stages in descending order of the preference of the predetermined target with respect to the topic,
The utterance control unit controls the utterance by the at least one of the plurality of utterance devices based on the information of the plurality of stages indicating the preference degree determined by the preference determination unit;
12. The dialogue control apparatus according to claim 10 or 11, wherein: The predetermined object includes a person or an animal or a robot,
The dialogue control apparatus according to any one of claims 1 to 12, wherein A target specifying means for specifying the predetermined target from a plurality of different targets;
The reaction acquisition unit is configured to determine the response of the specified predetermined target with respect to the utterance by the first utterance device and the utterance by the second utterance device provided separately from the first utterance device. Obtaining a plurality of response determination results including a result of determining the response of the specified predetermined target,
The dialogue control device according to claim 1, wherein the dialogue control device is characterized in that:   15. The dialogue control device according to claim 1, wherein the dialogue control device is provided in at least one of the first and second utterance devices.   15. The dialogue control device according to claim 1, wherein the dialogue control device is provided separately from the first and second utterance devices. A dialogue system comprising a first utterance device and a second utterance device configured to be utterable, and a dialogue control device,
The dialogue control device comprises:
A result of determining a response of a predetermined object to an utterance by the first utterance device, and a result of determining a response of the predetermined object to an utterance by the second utterance device provided separately from the first utterance device; Reaction acquisition means for acquiring a plurality of reaction determination results including:
Utterance control means for controlling the utterance by at least one of a plurality of utterance devices including the first and second utterance devices based on a plurality of reaction determination results acquired by the reaction acquisition means;
Comprising
An interactive system characterized by that. Each of the first and second utterance devices includes:
An acquisition means for acquiring at least one of a sound uttered by the predetermined target and a captured image of the predetermined target;
First communication means,
The dialogue control device further includes second communication means for communicating with the first and second utterance devices via the first communication means,
The reaction acquisition means of the dialog control device comprises:
The first data which is at least one of the voice and the captured image acquired by the acquisition unit of the first utterance device is acquired via the first and second communication units, and the acquired The determination result of the response of the predetermined object to the utterance by the first utterance device by determining the reaction of the predetermined object to the utterance by the first utterance device based on the first data. Get the response judgment result,
The second data that is at least one of the voice and the captured image acquired by the acquisition unit of the second utterance device is acquired via the first and second communication units, and the acquired The determination result of the response of the predetermined object to the utterance by the second utterance device by determining the response of the predetermined object to the utterance by the second utterance device based on the second data. Get the response judgment result,
The utterance control means of the dialog control device is configured to pass through the second and first communication means based on the plurality of reaction determination results including the first and second reaction determination results acquired by the reaction acquisition means. Controlling the utterance by the first and second utterance devices;
The interactive system according to claim 17, wherein: A result of determining a response of a predetermined object to an utterance by the first utterance device, and a result of determining a response of the predetermined object to an utterance by a second utterance device provided separately from the first utterance device Processing for obtaining a plurality of reaction determination results;
A process for controlling the utterance by at least one of a plurality of utterance devices including the first and second utterance devices based on the plurality of obtained reaction determination results;
including,
A dialog control method characterized by the above. Computer
A result of determining a response of a predetermined object to an utterance by the first utterance device, and a result of determining a response of the predetermined object to an utterance by a second utterance device provided separately from the first utterance device Reaction acquisition means for acquiring a plurality of reaction determination results;
Utterance control means for controlling the utterance by at least one of a plurality of utterance devices including the first and second utterance devices based on a plurality of reaction determination results acquired by the reaction acquisition means;
Function as
A program characterized by that.

Images