JP7331933B2 - Language estimation device, language estimation method, and program - Google Patents

Description

The present invention relates to a language estimation device, a language estimation method, and a program.

In recent years, with the increase in the number of foreign immigrants and foreign tourists, the demand for communication via translators and interpreters is increasing.
Patent Literature 1 describes a voice communication device that outputs and inputs voice as a dedicated terminal for the owner of the ID card when an ID card with an individual identification number written therein is inserted. The ID card stores the owner's voice or a specific phrase in his or her native language. The owner-only terminal can be controlled by verifying the voice with the ID card inserted.

Patent Literature 2 describes a device for creating an export duty-free article purchase record slip when purchasing an article at a duty-free shop. In this device, a plurality of language-specific announcement information are associated with a country code and stored, and when the country code is read from the recording medium, the language-specific announcement information associated with the country code is obtained, The information to be notified is printed out using the language-specific notification information.

Patent Document 3 describes obtaining guidance in a language corresponding to the nationality by reading attribute data indicating the nationality of a recognition target person from a passport and translating a message into a language corresponding to the read nationality. ing.

JP-A-3-150927 JP 2017-4333 A JP 2019-40642 A

As internationalization progresses, the number of foreigners visiting Japan is also diversifying. As a result, the number of languages requiring translation increased, and it took time to identify the languages spoken by foreign speakers.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique for performing language estimation efficiently and accurately.

Each aspect of the present invention employs the following configurations in order to solve the above-described problems.

A first aspect relates to a language estimation device.
A first language estimation device according to a first aspect includes:
Acquisition means for acquiring nationality information;
selection means for selecting a language estimation engine using the obtained nationality information;
and identifying means for analyzing speech information of a speaker using the selected language estimation engine to identify the language used by the speaker.
A second language estimation device according to the first aspect includes:
Acquisition means for acquiring nationality information;
selection means for selecting a language candidate for language estimation using the acquired nationality information;
and identifying means for analyzing speech information of a speaker using a language estimation engine and identifying the language used by the speaker from the selected candidates.

A second aspect relates to at least one computer-implemented language estimation method.
A first language estimation method according to the second aspect includes:
A language estimation device
Get nationality information,
Selecting a language estimation engine using the acquired nationality information,
Using the selected language estimation engine, analyzing a speaker's voice information to identify the language spoken by the speaker.
A second language estimation method according to the second aspect includes:
A language estimation device
Get nationality information,
Selecting a language candidate for language estimation using the acquired nationality information,
using a language estimation engine to analyze speech information of a speaker to identify a language used by the speaker from the selected candidates;

As another aspect of the present invention, it may be a program that causes at least one computer to execute the method of the second aspect, or a computer-readable recording medium recording such a program. may This recording medium includes a non-transitory tangible medium.
The computer program includes computer program code which, when executed by a computer, causes the computer to implement the language estimation method on the language estimation device.

Any combination of the above constituent elements, and conversion of expressions of the present invention into methods, devices, systems, recording media, computer programs, etc. are also effective as aspects of the present invention.

In addition, the various constituent elements of the present invention do not necessarily have to exist independently of each other. A component may be part of another component, a part of a component may overlap a part of another component, and the like.

In addition, although the method and computer program of the present invention describe multiple procedures in order, the order of description does not limit the order in which the multiple procedures are performed. Therefore, when implementing the method and computer program of the present invention, the order of the plurality of procedures can be changed within a range that does not interfere with the content.

Furthermore, the multiple steps of the method and computer program of the present invention are not limited to being performed at different times. Therefore, the occurrence of another procedure during the execution of a certain procedure, or the overlap of some or all of the execution timing of one procedure with the execution timing of another procedure, and the like are acceptable.

According to each of the aspects described above, it is possible to provide a technique for performing language estimation efficiently and accurately.

1 is a block diagram showing a conceptual configuration example of a multilingual communication system according to an embodiment of the present invention; FIG. 1 is a functional block diagram logically showing the configuration of a language estimation device according to an embodiment of the present invention; FIG. It is a block diagram which illustrates the hardware constitutions of the computer which implement|achieves the language estimation apparatus which concerns on embodiment of this invention. It is a flow chart which shows an example of operation of the language estimation device of this embodiment. FIG. 4 is a diagram showing an example of the data structure of a country-specific language estimation engine table; 1 is a functional block diagram showing a logical configuration example of a language estimation device of this embodiment; FIG. It is a flow chart which shows an example of operation of the language estimation device of this embodiment. 1 is a functional block diagram showing a logical configuration example of a language estimation device of this embodiment; FIG. It is a figure which shows an example of the screen which an output part displays. It is a flow chart which shows an example of operation of a language estimation device of this embodiment.

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. In addition, in all the drawings, the same constituent elements are denoted by the same reference numerals, and the description thereof will be omitted as appropriate. In the following figures, the configurations of parts that are not related to the essence of the present invention are omitted and not shown.

In the embodiment, "acquisition" means that the own device goes to get data or information stored in another device or storage medium (active acquisition), and that the device is output from another device Including at least one of entering data or information (passive acquisition). Examples of active acquisition include requesting or interrogating other devices and receiving their replies, and accessing and reading other devices or storage media. Also, examples of passive acquisition include receiving information that is distributed (or sent, pushed, etc.). Furthermore, "acquisition" may be to select and acquire received data or information, or to select and receive distributed data or information.

(First embodiment)
<System overview>
FIG. 1 is a block diagram showing a conceptual configuration example of a multilingual communication system 1 according to an embodiment of the present invention. A multilingual communication system 1 includes a language estimation device 100 and a translation device 10 . The language estimation device 100 and the translation device 10 may be integrated, that is, physically the same hardware.

For example, the multilingual communication system 1 converts the first language used by the visitor (first speaker Ua) into language Estimate using the estimation device 100 . Then, the translation device 10 performs mutual translation between the identified first language La of the speaker Ua and the second language Lb used by the interlocutor (second speaker Ub), such as a window attendant.

The translation device 10 inputs the uttered voices of the first speaker Ua and the second speaker Ub via a voice input device such as the microphone 4 . In FIG. 1, both are provided with microphones 4, respectively, but the present invention is not limited to this. It may be one microphone 4 having directivity in at least two directions. In addition, in FIG. 1, both devices are provided with audio output devices such as the speaker 6, but at least one speaker 6 may be provided, and two or more speakers 6 may be provided. In yet another example, a mobile terminal may be used to replace the audio input/output device (microphone 4 and speaker 6).

Then, after recognizing that the uttered voice of the first speaker Ua input via the microphone 4 is in the first language La, the translation apparatus 10 converts the content indicated by this uttered voice into the voice of the first language La. The language is specified using recognition processing, and the content is translated into the second language Lb of the second speaker Ub and output as voice through the speaker 6 (the flow indicated by the dashed arrow in the figure). The translation device 10 recognizes the uttered voice of the second speaker Ub input via the microphone 4 in the second language Lb, and then translates it into the first language La of the first speaker Ua. audio output via (the flow indicated by the dashed-dotted arrow in the figure).

However, translation between languages is not limited to bidirectional, and may be unidirectional. Alternatively, an interpreter who can speak the estimated language may interpret without using the translation device 10 .
Furthermore, the language estimation device 100 may estimate not only the language of the speaker but also the dialect and accent of the speaker's place of residence.

<Example of functional configuration>
FIG. 2 is a functional block diagram logically showing the configuration of language estimation apparatus 100 according to the embodiment of the present invention. Language estimation device 100 includes acquisition unit 102 , selection unit 104 , and identification unit 106 .
Acquisition unit 102 acquires nationality information.
The selection unit 104 selects the language estimation engine 110 using the acquired nationality information.
The specifying unit 106 uses the selected language estimation engine 110 to analyze the speech information 30 of the speaker and specifies the language used by the speaker.

The obtaining unit 102 obtains nationality information from, for example, the passport 20 possessed by the passenger. As an example, the acquiring unit 102 reads nationality information recorded in an IC (Integrated Circuit) chip embedded in the passport 20 via an IC reader (not shown). In another example, the acquisition unit 102 acquires an image including the nationality mark written on the passport 20, and reads characters by processing this image with OCR (Optical Character Recognition). Moreover, even if the passport 20 does not include the notation of nationality, the nationality information may be acquired by reading the serial number including the nationality information described in the passport. In another example, a two-dimensional code recorded with nationality information written on the passport 20 is read by a bar code reader.

Furthermore, in another example, the nationality information is specified by acquiring an image of a design such as the cover of the passport 20 (different for each country). Specifically, the language estimation apparatus 100 registers in advance the feature values of the designs of the passports 20 of each country in the storage device 1040 of FIG. performs matching processing of the feature quantity by image processing of the photographed image of the passport 20 to specify the country and acquire it as nationality information.

Furthermore, the language estimation device 100 can be used by, for example, airport check-in counters, baggage check-in counters, information desks at airports and railways, staff at various counters, and crew members in airplanes, cabins, and passenger cars. It can also be applied to infer languages or dialects, accents, etc.

For example, the acquisition unit 102 acquires an image including descriptions such as the departure airport name described in the airline ticket possessed by the first speaker Ua, the boarding station name of the ticket for a vehicle such as a train, and the like. The characters may be read by processing the image with OCR. A country name or a prefecture name may be retrieved and acquired from the departure airport name or boarding station name. In the example of estimating dialects and accents, the language estimation engine 110 is prepared specifically for each prefecture name.

The language estimation engine 110 uses the spoken voice of the speaker to estimate what the language of the spoken voice is. In this embodiment, the language estimation engine 110 is prepared for each country and trained specifically for a plurality of languages used for each country.

<Hardware configuration example>
FIG. 3 is a block diagram illustrating the hardware configuration of computer 1000 that implements language estimation apparatus 100 shown in FIG. Computer 1000 has bus 1010 , processor 1020 , memory 1030 , storage device 1040 , input/output interface 1050 and network interface 1060 .

Bus 1010 is a data transmission path for processor 1020, memory 1030, storage device 1040, input/output interface 1050, and network interface 1060 to mutually transmit and receive data. However, the method of connecting processors 1020 and the like to each other is not limited to bus connection.

The processor 1020 is a processor realized by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

The memory 1030 is a main memory implemented by RAM (Random Access Memory) or the like.

The storage device 1040 is an auxiliary storage device such as a HDD (Hard Disk Drive), SSD (Solid State Drive), memory card, or ROM (Read Only Memory). The storage device 1040 stores program modules that implement each function of the computer 1000 . Each function corresponding to the program module is realized by the processor 1020 reading each program module into the memory 1030 and executing it. Storage device 1040 also stores language estimation engine 110 .

The program module may be recorded on a recording medium. The recording medium for recording the program module includes a non-transitory tangible medium usable by the computer 1000, and the program code readable by the computer 1000 (processor 1020) may be embedded in the medium.

The input/output interface 1050 is an interface for connecting the computer 1000 and various input/output devices.

Network interface 1060 is an interface for connecting computer 1000 to a communication network. This communication network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). A method for connecting the network interface 1060 to the communication network may be a wireless connection or a wired connection.

The computer 1000 is then connected to necessary devices (for example, the microphone 4 and the speaker 6) via the input/output interface 1050 or the network interface 1060. FIG.

A computer 1000 that implements the language estimation device 100 is, for example, a personal computer, a smart phone, a tablet terminal, or the like. Alternatively, computer 1000 that implements language estimation apparatus 100 may be a dedicated terminal device. As described above, language estimation apparatus 100 may be implemented by computer 1000 that is physically integrated with translation apparatus 10 . For example, language estimation device 100 is realized by installing an application program for realizing language estimation device 100 in computer 1000 and starting it.

In another example, the computer 1000 is a web server, and the user activates a browser on a user terminal such as a personal computer, a smartphone, or a tablet terminal, and accesses a web server that provides services of the language estimation apparatus 100 via a network such as the Internet. The functions of the language estimation device 100 may be used by accessing the page.

In yet another example, computer 1000 may be a server device of a system such as SaaS (Software as a Service) that provides services of language estimation device 100 . A user may access a server device from a user terminal such as a personal computer, a smart phone, or a tablet terminal via a network such as the Internet, and the language estimation device 100 may be realized by a program running on the server device.

<Operation example>
FIG. 4 is a flow chart showing an example of the operation of the language estimation device 100 of this embodiment.
First, the acquiring unit 102 acquires nationality information from the passport 20 of the first speaker Ua (step S101).

FIG. 5 is a diagram showing an example of the data structure of the country-specific language estimation engine table 112. As shown in FIG. The country-specific language estimation engine table 112 associates and stores language estimation engines specialized for each country. It should be noted that in FIG. 5, "America", "A", "English", etc. are described as being stored in the country-specific language estimation engine table 112 for easy understanding. Actually, the country-by-country language estimation engine table 112 stores information indicating a country, for example, information identifying a country and information identifying a language estimation engine in association with each other. Furthermore, the language to be estimated by each language estimation engine is not stored in the country-specific language estimation engine table 112 . FIG. 5 shows, for the sake of explanation, which language the language estimation engine is specialized for as an estimation target.

For example, USA is associated with a language estimation engine A that specializes in multiple languages such as English and Spanish. Switzerland is associated with a language estimation engine B that specializes in four languages: French, Italian, German and Romansh.

Then, the selecting unit 104 refers to the country-specific language estimation engine table 112, reads out the language estimation engine 110 associated with the country indicated by the nationality information acquired in step S101, and selects the language estimation engine 110 to be used by the specifying unit 106. is selected (step S103). As an example, if the country indicated by the nationality information acquired in step S101 is the United States, information indicating the country of the United States is selected from among a plurality of language estimation engines stored in advance in the country-specific language estimation engine table 112. An associated language estimation engine A is selected.

Then, the specifying unit 106 acquires the uttered voice of the first speaker Ua from the microphone 4, analyzes the voice information 30 of the uttered voice using the language estimation engine A selected in step S103, and obtains the first A first language La used by the speaker Ua is identified from a plurality of languages such as English and Spanish (step S105). Here, it is assumed that the speech information 30 of the uttered speech of the first speaker Ua is analyzed and identified as Spanish. Since language estimation engines specialized for each country are used in this manner, the number of languages to be analyzed for the speech information 30 can be narrowed down.

As described above, according to the present embodiment, the language estimation engine 110 that has been trained for each country using the nationality information acquired from the passport 20 or the like is used to narrow down the number of languages to be analyzed and perform language estimation. Therefore, language estimation processing can be performed efficiently and accurately.

(Second embodiment)
The language estimation apparatus 100 of this embodiment differs from the above embodiments in that it has a configuration for estimating languages using language estimation engines 110 for a plurality of countries.

<Example of functional configuration>
FIG. 6 is a functional block diagram showing a logical configuration example of the language estimation device 100 of this embodiment. Acquisition unit 102 is the same as in the above embodiment of FIG.
The selection unit 104 selects language candidates for language estimation using the nationality information acquired by the acquisition unit 102 . The specifying unit 106 uses one language estimation engine 110 for a plurality of countries, analyzes the voice information of the speaker, and specifies the language used by the speaker from the selected candidates.

<Operation example>
FIG. 7 is a flow chart showing an example of the operation of the language estimation device 100 of this embodiment.
The flowchart of FIG. 7 includes the same step S101 as the flowchart of the above embodiment of FIG. 4, and further includes steps S113 and S115.

The selection unit 104 selects language candidates for language estimation using the nationality information acquired in step S101 (step S113).

The language estimation engine 110 of this embodiment builds neural networks of all languages (eg, 50 languages) corresponding to all target countries (eg, over 100 countries) by deep learning. The input of this neural network is speech data, and the output is language.

In other words, the selection unit 104 narrows down the candidates by masking the language output from the neural network of the language estimation engine 110 by country. For example, an output language is associated with each country and stored in advance, and the language of the output associated with the country and the language of the output of the neural network are ANDed to narrow down the language candidates.

Then, using the language estimation engine 110 in which the language candidates are thus narrowed down, the identification unit 106 analyzes the speech information of the speaker and identifies the language used by the speaker (step S115).

According to this embodiment, the same effects as those of the above embodiment can be obtained.

(Third Embodiment)
FIG. 8 is a functional block diagram showing a logical configuration example of the language estimation device 100 of this embodiment. The language estimation apparatus 100 of this embodiment is the same as any of the above embodiments, except that it has a configuration for presenting the estimated language to the user.

<Example of functional configuration>
The language estimation device 100 of FIG. 8 includes an acquisition unit 102, a selection unit 104, a specification unit 106, and a language estimation engine 110, which are the same as the language estimation device 100 of the above embodiment of FIG. 2 or FIG. Further, an output unit 120 is provided.

The output unit 120 outputs voice or text using the specified language when the score indicating the reliability of the language estimation result using the voice information of the speaker is equal to or less than a first reference value. Here, the score indicating the reliability of the language estimation result may be, for example, the likelihood included in the result of speech recognition processing of the speech information of the speaker. If the score indicating the reliability of the language estimation result is lower than the first reference value, considering the possibility that the estimation result is wrong, outputs voice or characters using the language and allows the speaker or the attendant to select. be able to. The determination using the score may be performed by the identifying unit 106 . Moreover, when the score exceeds the first reference value, the specifying unit 106 may determine the language with the highest score.

The first reference value is a reference for determining whether or not the result of language estimation is reliable. Exceeding the reference value indicates that the result of language estimation is reliable.

The output unit 120 outputs voice to the speaker 6 or characters to the display device 122 . A question sentence such as "Do you speak Hindi?" may be output using the specified language.

The output unit 120 outputs voice or text using the language specified in order of score.
FIG. 9 is a diagram showing an example of a screen 300 displayed by the output unit 120. As shown in FIG. The screen 300 displays a plurality of languages specified by the specifying unit 106 as operation buttons 302 in order of score, and also displays a message 304 prompting the user to select a language. While displaying the screen 300, you may output a message with an audio|voice. Here, the multiple languages are preferably displayed in descending order of score.

In FIG. 9, the operation buttons 302 and the messages 304 are written in Japanese for the sake of simplification of explanation, but actually they are written in the specified language. After selecting the language by pressing the operation button 302, the OK button 306 can be pressed to confirm the language selection. Also, each time a language is selected with the operation button 302, the notation of the message 304 may be changed to the selected language.

In the example of FIG. 9, operation buttons are used as a GUI (Graphical User Interface), but check buttons, radio buttons, pull-down menus, drum rolls, etc. may be used in other examples. Alternatively, a UI for accepting a user's selection operation may simply be displayed in multiple languages and a question message for the user.

Furthermore, the output unit 120 does not output speech or characters using languages whose scores are equal to or lower than the second reference value. The second reference value is a value that is lower than the first reference value, and is a value that defines a range that is considered to have little reliability. As a result, it is possible to present the user with a language in which a certain degree of reliability is ensured.

Furthermore, when the score difference between the candidates is equal to or less than the third reference value, the output unit 120 uses the language of the candidate, in other words, uses the candidates whose accuracy of the language estimation result is Or you may output the language. This determination process may also be performed by the identifying unit 106 . Moreover, when the difference exceeds the third reference value, the identifying unit 106 may determine the language with the highest score.

The output unit 120 performs at least one of determination processing using the first reference value, the second reference value, and the third reference value, or a combination of at least any two, and the determination result can decide whether to provide speech or language output.

<Operation example>
FIG. 10 is a flowchart showing an operation example of the language estimation device 100 of this embodiment. The flow in FIG. 10 starts after step S105 in FIG. 4 or after step S115 in FIG. The output unit 120 determines whether or not the score indicating the reliability of the speech recognition result when analyzing the speech information of the speaker using the plurality of languages selected in step S103 is equal to or less than the first reference value ( step S201). If the score is not equal to or less than the first reference value (YES in step S201), step S203 is bypassed and this flow ends.

On the other hand, if the score is equal to or less than the first reference value (NO in step S201), the output unit 120 outputs the voice using the language from the speaker 6 or displays the characters using the language on the display device 122. (Step S203).

According to this embodiment, when the score describing the reliability of the speech recognition result of the speaker by the language estimation engine 110 is lower than the first reference value, the speech or characters using that language are output. As a result, the language estimation device 100 of this embodiment has the same effect as the above-described embodiment, and furthermore, when the reliability of the estimation result is low, the speaker or the respondent can confirm by voice or text, Appropriate language selection can be accepted.

Although the embodiments of the present invention have been described above with reference to the drawings, these are examples of the present invention, and various configurations other than those described above can also be adopted.
For example, when the nationality information acquired by the acquisition unit 102 indicates a predetermined country, the selection unit 104 does not select the language estimation engine 110, and the identification unit 106 is associated with the country in advance. Identify language.

Here, for countries such as Japan where only Japanese is spoken as a single language, the country and the language are stored in advance in the country-by-country language correspondence table in association with each other. Then, the selection unit 104 first refers to the country-specific language correspondence table, searches the country indicated by the nationality information acquired by the acquisition unit 102 in the country-specific language correspondence table, and if a country is found, selects the country. Get and print the associated language.

According to this configuration, the language estimation process using the language estimation engine 110 can be omitted for countries with a single language, so the load on the computer 1000 can be reduced, the accuracy is improved, and the efficiency is good.

Moreover, language estimation apparatus 100 may further include a second selection unit (not shown) that selects a translation engine corresponding to the language identified by identification unit 106 . The second selection unit notifies the translation device 10 of the multilingual communication system 1 of FIG. 1 of the selected translation engine information. The translation device 10 can use the notified translation engine to perform mutual translation between the first language La of the first speaker Ua and the second language Lb of the second speaker Ub.

Although the present invention has been described with reference to the embodiments and examples, the present invention is not limited to the above embodiments and examples. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.
In the present invention, acquisition and use of information relating to users shall be done legally.

Some or all of the above embodiments can also be described as the following additional remarks, but are not limited to the following.
Examples of reference forms are added below.
1. Acquisition means for acquiring nationality information;
selection means for selecting a language estimation engine using the obtained nationality information;
and identifying means for analyzing speech information of a speaker and identifying a language used by the speaker using the selected language estimation engine.
2. Acquisition means for acquiring nationality information;
selection means for selecting a language candidate for language estimation using the acquired nationality information;
A language estimating apparatus comprising: a specifying means for specifying a language used by the speaker from the selected candidates by analyzing voice information of the speaker using a language estimating engine.
3. the acquiring means acquires the nationality information from a passport;
1. or 2. The language estimation device according to .
4. output means for outputting voice or characters using the language specified by the specifying means when a score indicating reliability of the language estimation result using the voice information of the speaker is equal to or lower than a first reference value; prepare
1. to 3. The language estimation device according to any one of .
5. The output means outputs the voice or the characters in the order of the scores.
4. The language estimation device according to .
6. The output means further does not output the voice or the characters using the language for which the score is equal to or lower than a second reference value.
4. or 5. The language estimation device according to .
7. further comprising second selection means for selecting a translation engine corresponding to the language identified by the identification means;
1. to 6. The language estimation device according to any one of .
8. When the nationality information acquired by the acquiring means indicates a predetermined country, the selecting means does not select a language estimation engine, and the identifying means selects a language pre-associated with the country. identify the
1. to 7. The language estimation device according to any one of .

9. A language estimation device
Get nationality information,
Selecting a language estimation engine using the obtained nationality information,
A language estimation method, wherein the selected language estimation engine is used to analyze speech information of a speaker to identify the language used by the speaker.
10. A language estimation device
Get nationality information,
Selecting a language candidate for language estimation using the acquired nationality information,
A language estimation method, wherein a language estimation engine is used to analyze speech information of a speaker and specify a language used by the speaker from the selected candidates.
11. the language estimation device obtains the nationality information from a passport;
9. or 10. The language estimation method described in .
12. The language estimation device further
outputting voice or text using the identified language when a score indicating the reliability of the language estimation result using the voice information of the speaker is equal to or lower than a first reference value;
9. to 11. The language estimation method according to any one of.
13. The language estimation device further
outputting the voice or the text in order of the score;
12. The language estimation method described in .
14. The language estimation device further
Do not output the voice or the text using the language whose score is equal to or lower than a second reference value;
12. or 13. The language estimation method described in .
15. The language estimation device further
selecting a translation engine corresponding to the identified language;
9. to 14. The language estimation method according to any one of.
16. The language estimation device further
If the acquired nationality information indicates a predetermined country, specifying a language pre-associated with the country without selecting a language estimation engine;
9. to 15. The language estimation method according to any one of.

17. to the computer,
Procedures for obtaining nationality information,
a procedure for selecting a language estimation engine using the acquired nationality information;
A program for executing a procedure of analyzing speech information of a speaker and identifying the language used by the speaker, using the selected language estimation engine.
18. to the computer,
Procedures for obtaining nationality information,
a procedure of selecting a language candidate for language estimation using the acquired nationality information;
A program for using a language estimation engine to analyze voice information of a speaker and identify the language used by the speaker from the selected candidates.
19. for causing the computer to further execute a procedure for obtaining said nationality information from a passport;
17. or 18. program described in .
20. If the score indicating the reliability of the language estimation result using the voice information of the speaker is equal to or less than a first reference value, causing the computer to further execute a step of outputting voice or characters using the specified language. for,
17. to 19. A program according to any one of
21. for causing a computer to further execute a step of outputting the voice or the character in order of the scores;
20. program described in .
22. causing the computer to further execute a procedure of not outputting the voice or the text using the language whose score is equal to or less than a second reference value;
20. or 21. program described in .
23. for further causing the computer to perform a step of selecting a translation engine corresponding to the identified language;
17. to 22. A program according to any one of
24. causing the computer to further execute a procedure of specifying a language pre-associated with the country without selecting a language estimation engine if the obtained nationality information indicates a predetermined country;
17. to 23. The program according to any one of

This application claims priority based on Japanese Patent Application No. 2019-164404 filed on September 10, 2019, and the entire disclosure thereof is incorporated herein.

1 multilingual communication system 4 microphone 6 speaker 10 translation device 20 passport 30 voice information 100 language estimation device 102 acquisition unit 104 selection unit 106 identification unit 110 language estimation engine 112 country-specific language estimation engine table 120 output unit 122 display device 300 screen 302 Operation button 304 Message 306 OK button 1000 Computer 1010 Bus 1020 Processor 1030 Memory 1040 Storage device 1050 Input/output interface 1060 Network interface La First language Lb Second language Ua First speaker Ub Second speaker

Claims (10)

Acquisition means for acquiring nationality information;
selection means for selecting a language estimation engine using the acquired nationality information;
identifying means for identifying the language used by the speaker by analyzing the speech information of the speaker using the selected language estimation engine;
The language estimation engine is provided for each country, and the language estimation engine for each country estimates the language used by the speaker from among a plurality of languages used in the country,
The language estimation device, wherein the selection means selects the language estimation engine corresponding to the country indicated by the nationality information. Acquisition means for acquiring nationality information;
selection means for selecting a language candidate for language estimation using the acquired nationality information;
using a language estimation engine, identifying means for analyzing speech information of a speaker and identifying the language used by the speaker from the selected candidates ;
The language estimation engine is a neural network constructed for all languages corresponding to all target countries by deep learning,
a language output from the neural network is stored in association with each country indicated by the nationality information;
The selection means narrows down the candidates by masking the language output from the neural network of the language estimation engine with a language corresponding to the country of the speaker indicated by the nationality information,
The language estimation device, wherein the identifying means identifies the language of the speaker using the masked language estimation engine . the acquiring means acquires the nationality information from a passport;
3. The language estimation device according to claim 1 or 2. output means for outputting voice or characters using the language specified by the specifying means when a score indicating reliability of the language estimation result using the voice information of the speaker is equal to or lower than a first reference value; prepare
A language estimation device according to any one of claims 1 to 3. The output means outputs the voice or the characters in the order of the scores.
The language estimation device according to claim 4. The output means further does not output the voice or the characters using the language for which the score is equal to or lower than a second reference value.
A language estimation device according to claim 4 or 5. further comprising second selection means for selecting a translation engine corresponding to the language identified by the identification means;
A language estimation device according to any one of claims 1 to 6. When the nationality information acquired by the acquiring means indicates a predetermined country, the selecting means does not select a language estimation engine, and the identifying means selects a language pre-associated with the country. identify the
A language estimation device according to any one of claims 1 to 7. A language estimation device
Get nationality information,
Selecting a language estimation engine using the obtained nationality information,
using the selected language estimation engine to analyze the speech information of the speaker and identify the language used by the speaker ;
The language estimation engine is provided for each country, and the language estimation engine for each country estimates the language used by the speaker from among a plurality of languages used in the country,
the language estimation device,
A language estimation method, wherein the language estimation engine corresponding to the country indicated by the nationality information is selected . to the computer,
Procedures for obtaining nationality information,
a procedure for selecting a language estimation engine using the acquired nationality information;
using the selected language estimation engine to analyze speech information of a speaker and identify the language used by the speaker ;
The language estimation engine is provided for each country, and the language estimation engine for each country estimates the language used by the speaker from among a plurality of languages used in the country,
A program for causing the computer to further execute a procedure for selecting the language estimation engine corresponding to the country indicated by the nationality information .

Images