JP2009217689A - Information processor, information processing method, and program - Google Patents

Abstract

An information processing apparatus for generating a pair of sentences in a bilingual relationship from a comparable corpus without using a bilingual dictionary or the like.
A bilingual corpus storage unit stores a bilingual corpus related to a first language and a second language, and a comparable corpus storage unit stores a comparable corpus having a first language corpus and a second language corpus. The machine translation unit 13 translates each sentence of the corpus of the first language into the second language using the bilingual corpus, the second language sentence accumulation unit 15 that accumulates the translated second language sentence, and the second language Similar information calculation unit 16 that calculates similar information between a corpus sentence and an accumulated second language sentence, and using the similar information, a corpus sentence in the second language and a sentence that is highly similar to the sentence is accumulated. A bilingual information generating unit 17 that generates bilingual information for associating a sentence in the first language as a translation source of the second language sentence, and a bilingual information output unit 18 that outputs the bilingual information are provided.
[Selection] Figure 1

Description

  The present invention relates to an information processing apparatus that generates parallel translation information, which is information that associates sentences in a parallel translation relationship, using a comparable corpus.

Conventionally, a bilingual corpus has been used in research and development of corpus-based translation technology. In particular, in a statistical translation technique that has been actively researched and developed in recent years, a bilingual corpus associated with each sentence is indispensable. There have been many studies on sentence alignment in which the parallel corpus is automatically extracted from the comparable corpus (for example, see Non-Patent Document 1 and Non-Patent Document 2).
Masao Utiyama, Hitoshi Isahara, “Reliable Measurements for Aligning Japan-English News Articles and Sences”, Proc. of ACL-2003, p. 72-79 Xiaoyi Ma, “Campollion: A Robust Parallel Text Sentence Aligner”, Proc. of LREC 2006

  However, in the conventional method of extracting a bilingual corpus from a comparable corpus, a bilingual dictionary and a bilingual thesaurus are required to obtain high performance. Therefore, there is a problem that it cannot be applied to a new task or language pair for which no bilingual dictionary or bilingual thesaurus is prepared.

  The present invention has been made to solve this problem. An information processing apparatus and the like that can generate a pair of sentences in a bilingual relationship from a comparable corpus without using a bilingual dictionary or a bilingual thesaurus. The purpose is to provide.

  To achieve the above object, an information processing apparatus according to the present invention includes a bilingual corpus storage unit that stores a bilingual corpus that is a corpus that associates a sentence in a first language and a sentence in a second language that are in a bilingual relationship. A comparable corpus storage unit storing a comparable corpus having a language corpus and a second language corpus, and using the parallel corpus, each sentence included in the first language corpus of the comparable corpus A machine translation unit that performs machine translation into a second language, a second language sentence accumulation unit that accumulates sentences of the second language machine-translated by the machine translation unit, and a second language corpus that the comparable corpus has A similarity information calculation unit that calculates similarity information that is information related to the similarity between the sentence and the second language sentence accumulated by the second language sentence accumulation unit, and the similarity information Of the combination of the sentence included in the second language corpus of the comparable corpus and the sentence of the second language stored in the second language sentence storage unit, using the similar information calculated by the output unit A sentence included in the second language corpus of the comparable corpus, and a source of machine translation for the second language sentence accumulated by the second language sentence accumulation unit A bilingual information generating unit that generates bilingual information that is information for associating a sentence in the first language, and a bilingual information output unit that outputs the bilingual information generated by the bilingual information generating unit.

  With such a configuration, it is possible to generate bilingual information that is a pair of sentences in a bilingual relationship from a comparable corpus without using a bilingual dictionary or bilingual thesaurus.

  Further, in the information processing apparatus according to the present invention, the machine translation unit performs a process of machine-translating each sentence included in the second language corpus of the comparable corpus into the first language using the parallel corpus. A first language sentence storage unit that stores a first language sentence machine-translated by the machine translation unit, wherein the similarity information calculation unit includes a first language corpus of the comparable corpus Similar information that is information on the similarity between the sentence included in the first language sentence storage unit and the sentence in the first language stored in the first language sentence storage unit is also calculated, and the parallel information generation unit is calculated by the similarity information calculation unit Using the similar information, among the combinations of the sentence included in the corpus of the first language possessed by the comparable corpus and the sentence of the first language accumulated by the first language sentence accumulation unit, A second language that is a source of machine translation for a sentence included in the first language corpus of the comparable corpus and a sentence in the first language accumulated by the first language sentence accumulating unit The bilingual information, which is information for associating the sentence, may also be generated.

  With such a configuration, bilingual information can be generated from a comparable corpus using both machine translation from the first language to the second language and machine translation from the second language to the first language. More bilingual information can be generated than when only direction machine translation is used.

  In the information processing apparatus according to the present invention, the machine translation unit performs machine translation using the bilingual information and the bilingual corpus when the bilingual information generated by the bilingual information generation unit exists. And a series of processes for performing machine translation by the machine translation unit, calculation of similar information by the similar information calculation unit, and generation of bilingual information by the bilingual information generation unit may be repeatedly executed twice or more. Good.

  With such a configuration, it is considered that machine translation accuracy can be improved by performing machine translation using bilingual information, and as a result, accuracy of bilingual information generated from a comparable corpus is high. It is thought that it becomes.

  The information processing apparatus according to the present invention is included in a comparable corpus storage unit that stores a comparable corpus having a first language corpus and a second language corpus, and a first language corpus that the comparable corpus has. A machine translation unit that machine-translates each sentence into a second language, a second language sentence accumulation unit that accumulates sentences in the second language machine-translated by the machine translation unit, and a second language that the comparable corpus has A similarity information calculation unit that calculates similarity information that is information related to the similarity between the sentence included in the corpus and the sentence in the second language accumulated by the second language sentence accumulation unit, and the similarity calculated by the similarity information calculation unit Of the combinations of the sentence included in the second language corpus of the comparable corpus and the second language sentence accumulated by the second language sentence accumulation unit using the information, It becomes the source of machine translation for sentences included in the second language corpus of the comparable corpus, and sentences of the second language accumulated by the second language sentence accumulating unit, which are included in the combination having high similarity. A bilingual information generating unit that generates bilingual information that is information that associates a sentence in a first language; and a bilingual information output unit that outputs bilingual information generated by the bilingual information generating unit, When each sentence included in the first language corpus of the comparable corpus is translated for the first time, machine translation without using the parallel translation information is performed and included in the first language corpus of the comparable corpus When translating each sentence to be translated from the second time onward, machine translation using the parallel translation information is performed, machine translation by the machine translation unit, calculation of similar information by the similarity information calculation unit, Repeatedly executing the series of processing two or more times for performing the generation of parallel translation information by the information generating unit is intended.

  With such a configuration, it is possible to generate bilingual information that is a pair of sentences in a bilingual relationship from a comparable corpus without using a bilingual dictionary or bilingual thesaurus. In addition, in such a configuration, there is an advantage that it is not necessary to prepare a bilingual corpus because the bilingual corpus is not used in the first machine translation. Since the second and subsequent machine translations are performed using parallel translation information generated from the comparable corpus, the generated parallel translation information gradually becomes highly accurate as machine translation is repeatedly executed. It can be thought that it will become.

  According to the information processing apparatus and the like according to the present invention, it is possible to generate a sentence pair having a translation relation from a comparable corpus without using a bilingual dictionary or a bilingual thesaurus.

  Hereinafter, an information processing apparatus according to the present invention will be described using embodiments. In the following embodiments, components and steps denoted by the same reference numerals are the same or equivalent, and repetitive description may be omitted.

(Embodiment 1)
An information processing apparatus according to Embodiment 1 of the present invention will be described with reference to the drawings. The information processing apparatus according to the present embodiment generates a pair of sentences having a bilingual relationship from a comparable corpus.

  FIG. 1 is a block diagram showing a configuration of an information processing apparatus 1 according to the present embodiment. The information processing apparatus 1 according to the present embodiment includes a parallel corpus storage unit 11, a comparable corpus storage unit 12, a machine translation unit 13, a first language sentence storage unit 14, a second language sentence storage unit 15, A similar information calculation unit 16, a parallel translation information generation unit 17, a parallel translation information output unit 18, and a parallel translation information storage unit 19 are provided.

  The parallel corpus storage unit 11 stores a parallel corpus. The bilingual corpus is a corpus that associates a sentence in the first language and a sentence in the second language, which are in a bilingual relationship. This bilingual corpus may be generated manually, for example, or may be generated by other methods. Here, “corresponding the sentence in the first language and the sentence in the second language” means that it is only necessary to obtain the other information from the information in one of the sentence in the first language and the sentence in the second language. is there. Accordingly, the bilingual corpus may include information including a sentence in the first language and a sentence in the second language as a pair, or information that links the sentence in the first language and the sentence in the second language. Good. In the latter case, the bilingual corpus may be information associating, for example, a first language sentence and a pointer or address indicating the position where the second language sentence is stored. In the present embodiment, the former case will be described.

  The process in which the bilingual corpus is stored in the bilingual corpus storage unit 11 does not matter. For example, a bilingual corpus may be stored in the bilingual corpus storage unit 11 via a recording medium, and a bilingual corpus transmitted via a communication line or the like is stored in the bilingual corpus storage unit 11. Alternatively, the parallel corpus input via the input device may be stored in the parallel corpus storage unit 11. Storage in the bilingual corpus storage unit 11 may be temporary storage in a RAM or the like, or may be long-term storage. The bilingual corpus storage unit 11 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The comparable corpus storage unit 12 stores a comparable corpus having a first language corpus and a second language corpus. In the bilingual corpus, sentences in a bilingual relationship are paired, whereas in this comparable corpus, such sentence alignment does not have to be taken. The corpus of the first language and the corpus of the second language that the comparable corpus has may have a relationship in which a corpus of the first language translated into the second language becomes a corpus of the second language. Or it may not be. The comparable corpus may have, for example, a first language corpus that is a Japanese web encyclopedia and a second language corpus that is an English web encyclopedia. It may have a corpus of a certain first language and a corpus of a second language that is an English news article, or may be another. Needless to say, the first language and the second language are not limited as long as they are different languages. For example, the first language and the second language may be any two languages selected from Japanese, English, French, German, Italian, Spanish, Russian, Chinese, Korean, etc. Good.

  The bilingual corpus stored in the bilingual corpus storage unit 11 and the comparable corpus stored in the comparable corpus storage unit 12 are preferably corpuses in the same field. Also good. This field may be, for example, fields such as news, patents, economy, politics, and science.

  The process in which the comparable corpus is stored in the comparable corpus storage unit 12 does not matter. For example, the comparable corpus may be stored in the comparable corpus storage unit 12 via a recording medium, and the comparable corpus transmitted via a communication line or the like is stored in the comparable corpus storage unit 12. Alternatively, the comparable corpus input via the input device may be stored in the comparable corpus storage unit 12. The storage in the comparable corpus storage unit 12 may be temporary storage in a RAM or the like, or may be long-term storage. The compatible corpus storage unit 12 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  The machine translation unit 13 uses the bilingual corpus stored in the bilingual corpus storage unit 11 to process each sentence included in the first language corpus of the comparable corpus stored in the comparable corpus storage unit 12. Machine translation into two languages. In addition, the machine translation unit 13 uses the bilingual corpus stored in the bilingual corpus storage unit 11 to each sentence included in the second language corpus of the comparable corpus stored in the comparable corpus storage unit 12. You may also perform the process of machine-translating into the first language. In the present embodiment, a case will be described in which the machine translation unit 13 also performs a process of machine-translating each sentence included in the corpus of the second language into the first language. The machine translation unit 13 may perform machine translation using the bilingual information and the bilingual corpus when there is bilingual information generated by the bilingual information generating unit 17 described later.

  The machine translation by the machine translation unit 13 is a corpus-based translation system that can automatically perform system learning using a bilingual corpus. In a statistical translation system that is an example of corpus-based translation, for example, a translation model and a language model are automatically learned from a parallel corpus, and translation is performed using these models.

  A translation model is a translation unit of a source language (for example, a word, a morpheme, a phrase, or the like) included in a bilingual corpus, and a translation unit of a target language that has a bilingual relationship with the translation unit of the source language. Information having a translation unit of a target language, which is a translation unit included in the URL, and a probability relating to the translation unit of the source language and the translation unit of the target language. The probabilities regarding the source language translation unit and the target language translation unit include, for example, the probability of the target language translation unit when the source language translation unit is given, and the target language translation unit. Such as the probability of the source language translation unit.

  In addition, since the method of performing machine translation is already known, the detailed description of the process in which the machine translation part 13 performs machine translation is abbreviate | omitted.

  The first language sentence accumulation unit 14 accumulates the sentence in the first language machine-translated by the machine translation unit 13 in a recording medium. The recording medium is, for example, a semiconductor memory, an optical disk, a magnetic disk, or the like, and may be included in the first language sentence storage unit 14 or may exist outside the first language sentence storage unit 14. . Further, the recording medium may or may not temporarily store a sentence in the first language.

  The second language sentence accumulation unit 15 accumulates the second language sentence machine-translated by the machine translation unit 13 in a recording medium. The recording medium is, for example, a semiconductor memory, an optical disk, a magnetic disk, or the like, and may be included in the second language sentence storage unit 15 or may exist outside the second language sentence storage unit 15. . Further, this recording medium may or may not temporarily store a sentence in the second language.

  The similar information calculation unit 16 includes a sentence included in the corpus of the first language included in the comparable corpus stored in the comparable corpus storage unit 12, a sentence in the first language accumulated in the first language sentence accumulation unit 14, and Similarity information, which is information related to the similarity of, is calculated. The similar information calculation unit 16 also includes a sentence included in the second language corpus of the comparable corpus stored in the comparable corpus storage unit 12, and the second language sentence accumulated in the second language sentence accumulation unit 15. Similar information that is information on similarity to sentences is calculated. Here, the content of the similarity information is not limited as long as it is information that allows the similarity of two sentences to be calculated for the similarity information to be known. For example, the similarity information may indicate the similarity between the two sentences, or may indicate the dissimilarity (difference) between the two sentences.

  The similar information may be, for example, BLEU, WER (Word Error Rate), PER (Position independent word Error Rate), or other indicators. Since methods for calculating these BLEU, WER, PER, and the like are already known, a detailed description of the process in which the similar information calculation unit 16 calculates similar information is omitted.

  Note that similar information between the sentences included in the first language corpus of the comparable corpus and the sentences of the first language accumulated in the first language sentence accumulating unit 14 and the second language corpus of the comparable corpus Regarding the similar information between the sentence included and the sentence in the second language accumulated by the second language sentence accumulating unit 15, the similarity information calculating unit 16 may calculate all the similar information, or those sentences. Similar information may be calculated only for combinations that are considered to be highly similar among the combinations. In the latter case, the processing load for calculating similar information can be reduced. For example, when the corpus of the first language and the corpus of the second language that the comparable corpus has are news corpora, the similar information calculation unit 16 displays similar information between two sentences in the same period or the same field. It may be calculated. More specifically, the similar information calculation unit 16 includes a sentence included in the first language corpus and a sentence in the same period or the same field as the sentence (this sentence is included in the second language corpus). May be machine-translated and similar information with the sentence accumulated by the first language sentence accumulation unit 14 may be calculated. In addition, the similarity information calculation unit 16 may calculate, for example, similarity information between two sentences included in a similar sub-corpus among the corpus of the first language and the corpus of the second language that the comparable corpus has. More specifically, the similarity information calculation unit 16 includes a sentence included in a sub-corpus having a first language corpus and a sub-corpus similar to the sub-corpus (this sub-corpus is included in the second language corpus). Sentences included in a corpus) may be machine-translated, and similar information with sentences accumulated by the first language sentence accumulation unit 14 may be calculated. The similarity between the first language sub-corpus and the second language sub-corpus is obtained by, for example, extracting a feature term that is a term characterizing the sub-corpus from each sub-corpus, May be determined based on the degree of matching of the translated feature terms. The characteristic term may be, for example, a term having a high TF / IDF value.

  The bilingual information generation unit 17 uses the similarity information calculated by the similarity information calculation unit 16, the sentence included in the corpus of the first language included in the comparable corpus stored in the comparable corpus storage unit 12, and the first Among the combinations with the sentences of the first language accumulated by the language sentence accumulating unit 14, a combination having a high similarity between them is specified. The sentence included in the corpus of the first language included in the comparable corpus included in the specified combination and the sentence in the first language stored in the first language sentence storage unit 14 included in the specified combination. Bilingual information, which is information for associating a sentence in the second language that is the source of machine translation (the sentence in the second language is a sentence included in the corpus of the second language that the comparable corpus has), is generated.

  In addition, the bilingual information generation unit 17 uses the similar information calculated by the similar information calculation unit 16, a sentence included in the second language corpus of the comparable corpus stored in the comparable corpus storage unit 12, and Among the combinations with the second language sentence accumulated by the second language sentence accumulation unit 15, a combination having a high similarity between the two is specified. Then, the sentence included in the corpus of the second language included in the comparable corpus included in the specified combination and the sentence in the second language stored in the second language sentence storage unit 15 included in the specified combination. Thus, bilingual information is generated, which is information for associating a sentence in the first language that is the source of machine translation (the sentence in the first language is a sentence included in the corpus of the first language that the comparable corpus has).

  Here, the combination having a high similarity between the sentence A and the sentence B may be a combination in which the similarity between the sentence A and the sentence B is larger than a threshold, and the similarity between the sentence A and the sentence B A combination of a predetermined number selected from the higher one or a combination of a predetermined ratio selected from the higher similarity between the sentence A and the sentence B may be used. The term “greater than the threshold value” may or may not include the threshold value. The threshold value may be a preset value, for example, or may be determined according to the obtained similarity. In the latter case, for example, the threshold value may be a value obtained by multiplying the maximum value of similarity by a value smaller than 1 such as 0.9.

  In addition, among combinations of sentences included in the corpus of the second language possessed by the comparable corpus and sentences of the second language accumulated by the second language sentence accumulating unit 15, combinations having high similarity between them are, for example, Sentences in the second language accumulated by the second language sentence accumulation unit 15 that may be specified in the entire combination and stored on the basis of sentences contained in the second language corpus of the comparable corpus. Or a second language corpus having a comparable corpus having a high similarity to the sentence on the basis of the sentence of the second language accumulated by the second language sentence accumulating unit 15. It may be specified by searching for included sentences. The number of combinations specified in this way may be one, or two or more. Of the combinations of the sentences included in the first language corpus of the comparable corpus and the sentences of the first language accumulated by the first language sentence accumulating unit 14, the same applies to combinations having high similarity between them. To do.

  In addition, among combinations of sentences included in the corpus of the second language possessed by the comparable corpus and sentences of the second language accumulated by the second language sentence accumulating unit 15, combinations having high similarity between them are similarities. May not include the highest combination. Of the combinations of the sentences included in the first language corpus of the comparable corpus and the sentences of the first language accumulated by the first language sentence accumulating unit 14, the same applies to combinations having high similarity between them. To do. That is, a combination with high similarity may be selected in consideration of other factors. For example, the bilingual information generation unit 17 uses the similar information calculated by the similar information calculation unit 16 and the corpus sentence in the first language that the comparable corpus has (this is referred to as sentence A) and the comparable corpus has. Information related to similarity with a sentence in a second language corpus (this sentence is referred to as sentence B) (this information is referred to as “integrated similarity information”) is calculated, and the similarity indicated by the integrated similarity information is high. Bilingual information for associating sentence A and sentence B may be generated. “High similarity indicated by the integrated similarity information” is the same as the above “high similarity”. Here, as the integrated similarity information, the similarity information between the sentences included in the corpus of the first language possessed by the comparable corpus and the sentences of the first language accumulated by the first language sentence accumulating unit 14 or the comparable corpus has. When similar information between the sentence included in the corpus of the second language and the sentence of the second language accumulated by the second language sentence accumulation unit 15 is used, the processing is the same as described above. On the other hand, as the integrated similarity information, the sentence A is a second language sentence that has been machine-translated, the similar information between the sentence in the second language and the sentence B accumulated by the second language sentence accumulation unit 15, and the sentence B A machine-translated sentence in the first language, and the value of a function having as arguments the first language sentence accumulated by the first language sentence accumulation unit 14 and similar information between the sentence A may be used. The function may be, for example, a function that calculates the average of similar information as its argument, and is the maximum value of similar information as its argument (if the similar information indicates dissimilarity, it is the minimum value). Or other functions. When such integrated similar information is used, for example, a combination of a sentence included in the second language corpus of the comparable corpus and a sentence in the second language accumulated in the second language sentence accumulation unit 15 is used. Of these, a combination with high similarity between the two may not include the combination with the highest similarity, but even so, the combination as a whole is more similar than other combinations. The nature will be high.

  As described above, the parallel translation information is information that associates a sentence included in the corpus of the first language included in the comparable corpus with a sentence included in the corpus of the second language included in the comparable corpus. Here, “the sentence included in the corpus of the first language and the sentence included in the corpus of the second language” is associated with the sentence included in the corpus of the first language and the sentence included in the corpus of the second language. This means that it is only necessary to acquire the other information from the one information. Therefore, the bilingual information may be information including a sentence included in the corpus of the first language and a sentence included in the corpus of the second language as a pair. The sentence included in the corpus of the first language and the second language The information may be linked to a sentence included in the corpus. In the latter case, the bilingual corpus may be, for example, information that associates a sentence included in the first language corpus with a pointer or an address indicating a position where the sentence included in the second language corpus is stored. Good. In the present embodiment, the former case will be described.

  The parallel translation information output unit 18 outputs the parallel translation information generated by the parallel translation information generation unit 17. Here, the output may be, for example, display on a display device (for example, a CRT or a liquid crystal display), transmission via a communication line to a predetermined device, printing by a printer, or output to a recording medium. It may be accumulated or delivered to another component. In the present embodiment, the parallel translation information output unit 18 accumulates the parallel translation information in the parallel translation information storage unit 19. The bilingual information output unit 18 may or may not include an output device (for example, a display device or a printer). The bilingual information output unit 18 may be realized by hardware, or may be realized by software such as a driver that drives these devices.

  In the parallel translation information storage unit 19, the parallel translation information generated by the parallel translation information generation unit 17 is stored. Storage in the parallel translation information storage unit 19 may be temporary storage in a RAM or the like, or may be long-term storage. The parallel translation information storage unit 19 can be realized by a predetermined recording medium (for example, a semiconductor memory, a magnetic disk, an optical disk, etc.).

  A series of processes for machine translation by the machine translation unit 13, calculation of similar information by the similar information calculation unit 16, and generation of bilingual information by the bilingual information generation unit 17 are executed only once. It may be executed repeatedly twice or more. In the latter case, the machine translation unit 13 performs machine translation using the bilingual corpus stored in the bilingual corpus storage unit 11 and the bilingual information generated so far. As described above, by executing the series of processes repeatedly, the corpus learned by the machine translation unit 13 becomes larger than the bilingual corpus and more appropriate machine translation is performed. Will be able to generate. In the present embodiment, a case where the series of processing is repeatedly executed will be described. When a series of processes are repeatedly executed, the parallel translation information output unit 18 may output only the parallel translation information generated in the last series of processes.

  In addition, the bilingual corpus storage unit 11, the comparable corpus storage unit 12, the recording medium in which the first language sentence storage unit 14 stores sentences in the first language, and the second language sentence storage unit 15 in the second language sentence. Any two or more storage units or recording media among the recording medium for storing the information and the parallel translation information storage unit 19 may be realized by the same recording medium, or may be realized by separate recording media. Good. In the former case, for example, the area storing the bilingual corpus becomes the bilingual corpus storage unit 11, and the area storing the composable corpus becomes the comparable corpus storage unit 12.

Next, the operation of the information processing apparatus 1 according to the present embodiment will be described using the flowchart of FIG.
(Step S101) The machine translation unit 13 determines whether or not bilingual information is stored in the bilingual information storage unit 19. If it is stored, the process proceeds to step S102, and if not, the process proceeds to step S103.

  (Step S <b> 102) The machine translation unit 13 performs learning using the bilingual corpus stored in the bilingual corpus storage unit 11 and the bilingual information stored in the bilingual information storage unit 19. By this learning, a translation model corresponding to the bilingual corpus and the bilingual information is generated.

  (Step S103) The machine translation unit 13 performs learning using the bilingual corpus stored in the bilingual corpus storage unit 11. By this learning, a translation model corresponding to the bilingual corpus is generated.

  (Step S104) The machine translation unit 13 machine-translates each sentence included in the corpus of the first language included in the comparable corpus into a sentence of the second language. In this translation, the learned translation model is used.

  (Step S105) The second language sentence accumulation unit 15 accumulates the second language sentence translated by the machine translation unit 13 in a recording medium. Note that the processing in steps S104 and S105 may be repeatedly performed each time a sentence is machine-translated.

  (Step S106) The machine translation unit 13 machine translates each sentence included in the second language corpus of the comparable corpus into a first language sentence. In this translation, the learned translation model is used.

  (Step S107) The first language sentence storage unit 14 stores the sentence in the first language translated by the machine translation unit 13 in a recording medium. Note that the processing in steps S106 and S107 may be repeatedly performed each time a sentence is machine-translated.

  (Step S <b> 108) The similarity information calculation unit 16 calculates similarity information between the sentence included in the second language corpus of the comparable corpus and the second language sentence accumulated by the second language sentence accumulation unit 15. Further, the similarity information calculation unit 16 calculates similarity information between sentences included in the corpus of the first language included in the comparable corpus and sentences of the first language accumulated by the first language sentence accumulation unit 14. Details of this processing will be described later with reference to the flowchart of FIG.

  (Step S109) Using the similar information, the parallel translation information generation unit 17 associates the sentence included in the first language corpus of the comparable corpus with the sentence included in the second language corpus of the comparable corpus. Generate bilingual information. Details of this processing will be described later with reference to the flowchart of FIG.

  (Step S110) The parallel translation information output unit 18 outputs the parallel translation information generated by the parallel translation information generation unit 17. In the present embodiment, the parallel translation information output unit 18 accumulates the parallel translation information in the parallel translation information storage unit 19. Note that the processing in steps S109 and S110 may be repeatedly performed every time one piece of parallel translation information is generated.

  (Step S111) The machine translation unit 13 determines whether to end the machine translation process. If the process ends, the series of processes for generating the bilingual information ends. If not, the process returns to step S101. Note that whether or not to end may be determined by, for example, how many times the series of processes of steps S101 to S110 has been executed. For example, when the series of processes has been executed a preset number of times, it may be determined to end, and otherwise, it may be determined not to end.

  FIG. 3 is a flowchart showing details of the similarity information calculation process (the process of step S108) in the flowchart of FIG. Note that the flowchart of FIG. 3 shows an example of processing for calculating similar information, and it goes without saying that the similar information may be calculated by other methods.

  (Step S201) The similar information calculation unit 16 sets a counter i to 1.

  (Step S202) The similar information calculation unit 16 sets a counter j to 1.

  (Step S203) The similar information calculation unit 16 machine-translates the i-th sentence of the first language corpus of the comparable corpus, and the second language sentence accumulated by the second language sentence accumulation unit 15 Similar information with the j-th sentence of the corpus of the second language that the rubble corpus has is calculated. The sentence included in the corpus of the first language possessed by the comparable corpus can be matched with the sentence of the second language that has been machine-translated and accumulated by the second language sentence accumulating unit 15. Suppose that

  (Step S204) The similar information calculation unit 16 temporarily stores the similar information calculated in step S203 in a recording medium (not shown). It should be noted that when storing the similar information, it is preferable to know which two sentences are similar information. Therefore, for example, the similar information may be stored in association with the value of the counter i and the value of the counter j.

  (Step S205) The similar information calculation unit 16 increments the counter j by 1.

  (Step S206) The similar information calculation unit 16 determines whether or not the j-th sentence exists in the corpus of the second language that the comparable corpus has. And when it exists, it returns to step S203, and when that is not right, it progresses to step S207.

  (Step S207) The similar information calculation unit 16 increments the counter i by one.

  (Step S208) The similar information calculation unit 16 determines whether or not the i-th sentence exists in the corpus of the first language that the comparable corpus has. And when it exists, it returns to step S202, and when that is not right, it progresses to step S209.

  (Step S209) The similar information calculation unit 16 sets the counter j to 1.

  (Step S210) The similar information calculation unit 16 sets a counter i to 1.

  (Step S211) The similarity information calculation unit 16 machine-translates the i-th sentence of the corpus of the first language that the comparable corpus has and the j-th sentence of the corpus of the second language that the comparable corpus has. Similar information with the sentence of the first language stored by the language sentence storage unit 14 is calculated. The sentence included in the corpus of the second language possessed by the comparable corpus can be matched with the sentence of the first language that has been machine-translated and accumulated by the first language sentence accumulation unit 14. Suppose that

  (Step S212) The similar information calculation unit 16 temporarily stores the similar information calculated in step S211 in a recording medium (not shown). It should be noted that when storing the similar information, it is preferable to know which two sentences are similar information. Therefore, for example, the similar information may be stored in association with the value of the counter i and the value of the counter j.

  (Step S213) The similar information calculation unit 16 increments the counter i by 1.

  (Step S214) The similar information calculation unit 16 determines whether or not the i-th sentence exists in the corpus of the first language that the comparable corpus has. And when it exists, it returns to step S211, and when that is not right, it progresses to step S215.

  (Step S215) The similar information calculation unit 16 increments the counter j by 1.

  (Step S216) The similar information calculation unit 16 determines whether or not the j-th sentence exists in the second language corpus of the comparable corpus. And when it exists, it returns to step S210, and when that is not right, it returns to the flowchart of FIG.

  FIG. 4 is a flowchart showing details of the bilingual information generation process (the process of step S109) in the flowchart of FIG. Note that the flowchart of FIG. 4 shows an example of processing for generating parallel translation information, and it goes without saying that parallel translation information may be generated by other methods. In this flowchart, the case where the similar information is information indicating similarity is mainly described.

  (Step S301) The parallel translation information generation unit 17 deletes the similar information smaller than the threshold value from the similar information calculated by the similar information calculation unit 16. Note that if the counter value is associated with the similar information at the time of this deletion, the information is also deleted together.

  (Step S302) The parallel translation information generation unit 17 sets a counter i to 1.

  (Step S303) The parallel translation information generation unit 17 sorts the similar information regarding the i-th sentence of the corpus of the first language that the comparable corpus has in descending order. Note that the similar information regarding the i-th sentence of the first language corpus is similar information between the i-th sentence and a sentence included in the second language corpus. For example, in the flowchart of FIG. 3, when the similar information is stored in association with the values of the counters i and j, the parallel translation information generation unit 17 uses the similar information associated with the value of the counter i. Therefore, similar information that has not been deleted is sorted. If there is no similar information regarding the i-th sentence of the corpus of the first language that the comparable corpus has, for example, if all the similar information is deleted in step S301, the process may proceed to step S306.

  (Step S304) The parallel translation information generation unit 17 specifies similar information with high similarity in the sorted similar information. When the similar information is information indicating similarity, for example, a predetermined number of similar information may be specified from the top after sorting in descending order.

  (Step S305) The parallel translation information generation unit 17 includes the i-th sentence of the first language corpus of the comparable corpus and the sentences of the second language corpus that are associated with the i-th sentence by the identified similar information. The bilingual information that associates with is generated. The corpus sentence in the second language associated with the i-th sentence by the specified similar information, for example, in the flowchart of FIG. 3, the similar information is stored in association with the values of the counters i and j. In this case, it is the sentence of the second language corpus identified by the value of the counter j corresponding to the identified similar information, that is, the jth sentence of the corpus of the second language.

  (Step S306) The parallel translation information generation unit 17 increments the counter i by 1.

  (Step S307) The parallel translation information generation unit 17 determines whether or not the i-th sentence exists in the corpus of the first language that the comparable corpus has. If it exists, the process returns to step S303, and if not, the process returns to the flowchart of FIG.

  In the flowchart of FIG. 4, the process of deleting similar information with low similarity (the process of step S301) may not be performed. In addition, when there is duplication in the generated parallel translation information, that is, when there are two or more of the same translation information, only one of the two or more of the same translation information is left in order to eliminate the duplication. Other parallel translation information may be deleted. In the flowchart of FIG. 4, the case is described in which processing is performed on the basis of a corpus sentence in the first language possessed by the comparable corpus. May be.

Next, the operation of the information processing apparatus 1 according to this embodiment will be described using a specific example.
First, it is assumed that the process is started in a situation where the parallel translation information is not stored in the parallel translation information storage unit 19. Then, the machine translation unit 13 refers to the bilingual information storage unit 19 and determines that there is no bilingual information (step S101), reads the bilingual corpus stored in the bilingual corpus storage unit 11, learns it, and translates it. A model is generated (step S103). The machine translation unit 13 machine translates each sentence of the corpus of the first language possessed by the comparable corpus into a sentence of the second language using the generated translation model (step S104). Then, the first language corpus sentence number of the comparable corpus and the second language sentence obtained by machine translation of the first language sentence are paired and passed to the second language sentence storage unit 15. The second language sentence storage unit 15 stores the corpus sentence number of the first language and the sentence of the second language in association with each other in a recording medium (not shown) (step S105).

  Similarly, the machine translation unit 13 machine translates each sentence of the corpus of the second language possessed by the comparable corpus into a sentence of the first language using the generated translation model (step S106). Then, the second language corpus sentence number of the comparable corpus and the first language sentence obtained by machine translation of the second language sentence are paired and passed to the first language sentence storage unit 14. The first language sentence storage unit 14 stores the corpus sentence number of the second language and the sentence of the first language in association with each other in a recording medium (not shown) (step S107).

  Thereafter, the similar information calculation unit 16 performs a process of calculating similar information (step S108). Specifically, the similar information calculation unit 16 uses the second language sentence obtained by machine translation of the first sentence of the first language corpus of the comparable corpus, and the second language sentence storage unit 15 uses the second language. This is specified with reference to the recording medium storing the sentence. Further, the similar information calculation unit 16 specifies the first sentence of the corpus of the second language that the comparable corpus has. Then, the similar information calculation unit 16 calculates similar information of the two sentences specified (steps S201 to S203). Here, it is assumed that the value of the similar information is “0.01”. Then, the similar information calculation unit 16 temporarily stores the similar information in a recording medium (not shown) in association with the value of the counter i (= 1) and the value of the counter j (= 1) at that time ( Step S204). The first record in FIG. 5 shows similar information and the like temporarily stored as described above. In the table of FIG. 5, the sentence number in the corpus of the first language corresponds to the value of the counter i, and the sentence number in the corpus of the second language corresponds to the value of the counter j.

  Next, the similarity information calculation unit 16 uses the second language sentence obtained by machine translation of the first sentence of the first language corpus of the comparable corpus, and the second language sentence storage unit 15 uses the second language sentence. Is identified with reference to the recording medium storing the. Further, the similar information calculation unit 16 specifies the second sentence of the corpus of the second language that the comparable corpus has. Then, similar information of the two specified sentences is calculated (steps S205, S206, S201 to S203). Then, as shown by the second record in FIG. 5, the similar information calculation unit 16 stores the similar information in a recording medium (not shown) in association with the values of the counters i and j at that time ( Step S204). By repeating such processing, similar information is accumulated as shown in the table of FIG. 5 (steps S202 to S208).

  Next, the similar information calculation unit 16 specifies the first sentence of the corpus of the first language that the comparable corpus has. In addition, the similarity information calculation unit 16 uses the first language sentence obtained by machine translation of the first sentence of the second language corpus of the comparable corpus, and the first language sentence storage unit 14 uses the first language sentence. The identification is made with reference to the stored recording medium. Then, the similar information calculation unit 16 calculates the similar information of the two specified sentences (steps S209 to S211). Here, it is assumed that the value of the similar information is “0.03”. Then, the similar information calculation unit 16 temporarily stores the similar information in a recording medium (not shown) in association with the value of the counter i (= 1) and the value of the counter j (= 1) at that time ( Step S212). The first record in FIG. 6 shows similar information and the like temporarily stored as described above. Also in the table of FIG. 6, the sentence number in the first language corpus corresponds to the value of the counter i, and the sentence number in the second language corpus corresponds to the value of the counter j. Then, in the same manner as described above, by repeating the process of calculating and storing similar information, the similar information is stored as shown in the table of FIG. 6 (steps S210 to S216).

  Thereafter, the parallel translation information generation unit 17 performs processing for generating parallel translation information (step S109). Specifically, the parallel translation information generation unit 17 refers to the tables of FIGS. 5 and 6 and deletes records whose similarity information is smaller than a threshold value (step S301). Here, it is assumed that the threshold value is set to “0.5”. Then, the bilingual information generation unit 17 deletes, for example, the first record, the second record, the fourth record, and the like in the table shown in FIG.

  Next, the bilingual information generation unit 17 records from the tables shown in FIGS. 5 and 6 that the record relating to the first sentence of the first language corpus, that is, the sentence number in the first language corpus is “1”. Get a record. Then, the acquired records are sorted in descending order of the similar information (steps S302 and S303). Assume that the sorted records are as shown in FIG. In this specific example, it is assumed that a maximum of three pieces of parallel translation information is generated for each sentence of the first language corpus. Then, the parallel translation information generation unit 17 specifies the three records in descending order of the similar information in the record of FIG. 7 (step S304). Then, the number of the sentence in the corpus of the first language and the number of the sentence in the corpus of the second language are extracted from each record, and the sentence specified by the number of the sentence is read from the comparable corpus storage unit 12. Thus, parallel translation information is generated (step S305).

  Specifically, the bilingual information generating unit 17 translates bilingual information that associates the first sentence of the corpus of the first language with the 123rd sentence of the corpus of the second language, and the first translation of the corpus of the first language. Generates bilingual information that associates the sentence with the 222nd sentence of the corpus of the second language, and bilingual information that associates the first sentence of the corpus of the first language with the third sentence of the corpus of the second language To do. The parallel translation information may be temporarily stored in a recording medium (not shown), or may be accumulated in the parallel translation information storage unit 19 by the parallel translation information output unit 18.

  Next, the bilingual information generation unit 17 records from the tables shown in FIGS. 5 and 6 that the record related to the second sentence in the first language corpus, that is, the sentence number in the first language corpus is “2”. A certain record is acquired, and the acquired record is sorted in descending order of the similar information (steps S306, S307, S303). Similarly to the above description, the parallel translation information generation unit 17 specifies a record with large similar information and generates parallel translation information (steps S304 and S305). By repeating such processing (steps S303 to S307), parallel translation information is generated.

  The parallel translation information output unit 18 accumulates the generated parallel translation information in the parallel translation information storage unit 19 (step S110). Thereafter, the machine translation unit 13 refers to the bilingual information storage unit 19 and determines that the bilingual information exists because the stored bilingual information exists (step S101), and determines the bilingual information and the bilingual corpus. Learning is performed to generate a translation model (step S102). Then, similarly to the above description, processing for generating new parallel translation information is executed (steps S104 to S110). Then, a series of processes for generating the bilingual information is repeatedly executed a predetermined number of times.

  When the parallel translation information generated by this series of processes is accumulated in the parallel translation information storage section 19, the parallel translation information output section 18 deletes the parallel translation information generated by the previous series of processes and then creates a new parallel translation. Information may be accumulated. This is because the latest bilingual information is finally used as bilingual information, and the past bilingual information is unnecessary. When the bilingual information output unit 18 stores the latest bilingual information in the bilingual information storage unit 19 while leaving the past bilingual information, a flag or the like is provided so that the latest bilingual information can be distinguished from the past bilingual information. It is preferable to set and accumulate. Even when the machine translation unit 13 performs learning using the bilingual corpus and the bilingual information, it is preferable to perform learning using the latest bilingual information.

  Further, the specific data of FIGS. 5 to 7 used in this specific example is shown for explaining the details of the operation of the information processing apparatus 1 according to the present embodiment, and actually uses a corpus or the like. It is not calculated data.

[Experimental example]
Next, an experiment for evaluating the information processing apparatus 1 according to this embodiment will be described. In this experiment, the parallel corpus stored in the parallel corpus storage unit 11 and the comparable corpus stored in the comparable corpus storage unit 12 are the same. Then, learning is performed again using only the obtained bilingual information, and the system in which learning is performed using only the bilingual information is compared with the system in which learning is performed using only the bilingual corpus. The information processing apparatus 1 is evaluated according to the form. That is, the bilingual information obtained by this experiment is smaller in scale than the bilingual corpus stored in the bilingual corpus storage unit 11. And if the system that learned the bilingual information has the same performance as the system that learned the bilingual corpus, it can be seen that it is possible to reduce the size of the learning corpus without degrading the performance, As a result, the information processing apparatus 1 according to the present embodiment indirectly proves that appropriate parallel translation information can be extracted from the comparable corpus.

  In this experiment, the pharaoh training toolkit was used for learning the translation model. In addition, SRI language model tool kit was used for learning the language model. Moreover, the BLEU score was used as similar information.

[Experiment 1]
In Experiment 1, a bilingual corpus related to Japanese-English patents was used. This bilingual corpus was generated by the following procedure. First, prepare a Japanese-English patent corpus. For the Japanese-English patent corpus, please refer to the following document.

  Literature: Masao Utyyama, Hitachi Isahara, “A Japan-English Patent Parallel Corpus”, MT-Summi XI

  In the Japanese-English patent corpus, Japanese-English patents are associated with each article in advance, and only the abstract part is extracted from the Japanese-English patent corpus. If the number of sentences in the extracted Japanese and English abstracts is the same, each sentence is assumed to be translated in order, and each sentence is associated and registered in the bilingual corpus. On the other hand, if the number of sentences is different, the abstract is not used to generate a bilingual corpus. In this way, a bilingual corpus having 740,000 Japanese sentences and English sentences pairs was generated from the International Patent Classification (IPC) G06 publication.

  FIG. 8 is a graph showing the experimental results. In the graph of FIG. 8, the vertical axis represents a BLEU score, and this BLEU score is a BLEU score in units of test sets for a test set 500 sentence that is not used for learning. The horizontal axis represents the size of the learning set (translation information) selected based on the sentence-wise BLEU score (similar information), that is, the number of pairs of Japanese sentences and English sentences used in learning. . The broken line in the figure is the result of using all data (740,000 pairs) for learning.

  As can be seen from FIG. 8, even when about 300,000 pairs are deleted from the learning set, the same performance as when learning is performed using all the parallel corpora. Therefore, even if the size of the bilingual corpus is reduced by using the information processing apparatus 1 according to the present embodiment, it is understood that the performance does not deteriorate, so that the information processing apparatus 1 can extract appropriate bilingual information from the comparable corpus.

[Experiment 2]
In this experiment 2, a JENAD corpus having 150,000 pairs was used as a parallel corpus. For the JENAD AD corpus, refer to the aforementioned Non-Patent Document 1.

  FIG. 9 shows experimental results when the JENAD AD corpus is used. FIG. 9 also shows the BLEU score in units of test sets for the test set 500 sentences not used for learning, as in Experiment 1. In FIG. 9, the “baseline” in which the number of learning pairs is 150,000 is a result of using all of the JENAD AD corpus as they are. In FIG. 9, “baseline” in which the number of learning pairs is 100,000 and 50,000 is selected by using the sentence alignment score obtained by the method of Non-Patent Document 1 described above. It is a result at the time of using the learning pair after performing. In the “proposed method” in which the number of learning pairs is 100,000 or 50,000, the number of learning pairs is rounded down to 100,000 or 50,000 using the information processing apparatus 1 according to the present embodiment. It is a selection.

  As can be seen from FIG. 9, when 100,000 pairs are selected, the proposed method has a higher translation performance than the baseline. Even when 50,000 pairs are selected, the proposed method is substantially inferior to the baseline, although the performance is inferior to the baseline. Therefore, even if the size of the bilingual corpus is reduced using the information processing apparatus 1 according to the present embodiment, the performance does not substantially deteriorate, so that the information processing apparatus 1 can extract appropriate bilingual information from the comparable corpus. I understand.

  As described above, according to the information processing apparatus 1 according to the present embodiment, a bilingual associating a pair of sentences having a bilingual relationship from a comparable corpus without using a language resource such as a bilingual dictionary or a bilingual thesaurus. Information can be generated. The machine translation unit 13 repeatedly executes a series of processes of machine translation by the machine translation unit 13, calculation of similar information by the similarity information calculation unit 16, and generation of parallel translation information by the parallel translation information generation unit 17. 13 can perform learning using bilingual information, and as a result, the accuracy of machine translation can be improved. As a result, the accuracy of the generated parallel translation information is considered to be high.

  In this embodiment, the case where the parallel translation information output unit 18 accumulates the parallel translation information in the parallel translation information storage unit 19 is described. However, this is an example, and the parallel translation information output unit 18 outputs other than the accumulation. Needless to say, the processing may be performed. If the parallel translation information output unit 18 does not accumulate the parallel translation information in the parallel translation information storage unit 19, the information processing apparatus 1 may not include the parallel translation information storage unit 19.

  In the present embodiment, the case has been described in which the machine translation unit 13 performs machine translation for both the first language corpus sentence and the second language corpus sentence of the comparable corpus. The unit 13 may simply machine translate the corpus sentence of the first language into the sentence of the second language. In that case, since the translation into the sentence of the first language is not performed, the similar information calculation unit 16 and the parallel translation information generation unit 17 do not need to perform the process related to the sentence accumulated by the first language sentence accumulation unit 14. The information processing device 1 may not include the first language sentence storage unit 14.

  In the present embodiment, the case where the machine translation unit 13 is a corpus-based translation system that performs machine translation using a bilingual corpus has been described. However, this need not be the case for the first translation. For example, each time the machine translation unit 13 machine translates each sentence included in the first language corpus of the comparable corpus into the second language, or includes each sentence included in the second language corpus of the comparable corpus When a sentence is machine-translated into the first language, general-purpose machine translation that does not use a bilingual corpus or bilingual information (for example, rule-based machine translation may be used) is performed. After the first time, each sentence included in the corpus of the first language possessed by the comparable corpus is machine-translated into the second language, or each sentence contained in the corpus of the second language possessed by the comparable corpus is translated into the first language. For example, machine translation (this machine translation becomes a corpus-based translation) may be performed using parallel translation information generated by the previous machine translation. Therefore, the machine translation unit 13 may be capable of executing both rule-based machine translation and statistical machine translation, for example. In this way, a series of processes for machine translation, calculation of similar information, and generation of parallel translation information are repeatedly executed twice or more, so that the generated parallel translation information is gradually highly accurate. It can be thought that it will become.

  Moreover, although the case where the information processing apparatus 1 is a stand-alone has been described in the above embodiment, the information processing apparatus 1 may be a stand-alone apparatus or a server apparatus in a server / client system. In the latter case, the output unit may output information via a communication line.

  In the above embodiment, each process or each function may be realized by centralized processing by a single device or a single system, or may be distributedly processed by a plurality of devices or a plurality of systems. It may be realized by doing.

  In the above embodiment, information related to processing executed by each component, for example, information received, acquired, selected, generated, transmitted, or received by each component In addition, information such as threshold values, mathematical formulas, addresses, etc. used by each component in processing is retained temporarily or over a long period of time on a recording medium (not shown) even when not explicitly stated in the above description. It may be. Further, the storage of information in the recording medium (not shown) may be performed by each component or a storage unit (not shown). Further, reading of information from the recording medium (not shown) may be performed by each component or a reading unit (not shown).

  In the above embodiment, when information used by each component, for example, information such as a threshold value, an address, and various setting values used by each component may be changed by the user Even if it is not specified in the above description, the user may be able to change the information as appropriate, or it may not be. If the information can be changed by the user, the change is realized by, for example, a not-shown receiving unit that receives a change instruction from the user and a changing unit (not shown) that changes the information in accordance with the change instruction. May be. The change instruction received by the receiving unit (not shown) may be received from an input device, information received via a communication line, or information read from a predetermined recording medium, for example. .

  In the above embodiment, when two or more constituent elements included in the information processing apparatus 1 have communication devices, input devices, or the like, two or more constituent elements may physically have a single device. Or you may have separate devices.

  In the above embodiment, each component may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component can be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory. In addition, the software which implement | achieves the information processing apparatus 1 in the said embodiment is the following programs. That is, this program uses a parallel corpus stored in a parallel corpus storage unit in which a parallel corpus, which is a corpus that correlates a first language sentence and a second language sentence in a parallel translation relationship, is stored in the computer. Each sentence included in the first language corpus of the comparable corpus stored in the comparable corpus storage unit storing the comparable corpus having the first language corpus and the second language corpus is stored in the second language. A machine translation unit that performs machine translation into a language, a second language sentence accumulation unit that accumulates a second language sentence machine translated by the machine translation unit, and a sentence included in the second language corpus of the comparable corpus A similarity information calculation unit that calculates similarity information that is information related to similarity with sentences in the second language accumulated by the second language sentence accumulation unit; Using the similar information calculated by the information calculation unit, among the combinations of the sentence included in the second language corpus of the comparable corpus and the second language sentence accumulated by the second language sentence accumulation unit, Machine translation source for a sentence included in the second language corpus of the comparable corpus and a sentence in the second language accumulated by the second language sentence accumulating unit, included in a combination having high similarity between the two The bilingual information generating unit that generates bilingual information that is information for associating the sentence in the first language, and the bilingual information output unit that outputs the bilingual information generated by the bilingual information generating unit.

  In this program, the machine translation unit also performs a process of machine-translating each sentence included in the second language corpus of the comparable corpus into the first language using the bilingual corpus. The computer further functions as a first language sentence storage unit that stores sentences of the first language machine-translated by the machine translation unit, and the similarity information calculation unit is configured to use the corpus of the first language that the comparable corpus has. Similar information that is information on the similarity between the sentence included in the first language sentence storage unit and the sentence in the first language stored in the first language sentence storage unit is also calculated, and the parallel information generation unit is calculated by the similarity information calculation unit Using the similar information, the combination of the sentence included in the first language corpus of the comparable corpus and the sentence of the first language stored in the first language sentence storage unit is used. Machine translation of a sentence included in the first language corpus of the comparable corpus and a sentence in the first language accumulated by the first language sentence accumulating unit included in a combination having high similarity between the two. You may also generate parallel translation information, which is information for associating the original second language sentence.

  Further, in this program, the machine translation unit performs machine translation using the bilingual information and the bilingual corpus when the bilingual information generated by the bilingual information generation unit exists, A series of processes for machine translation by the machine translation unit, calculation of similar information by the similar information calculation unit, and generation of bilingual information by the bilingual information generation unit may be repeated twice or more.

  In addition, the other software which implement | achieves the information processing apparatus 1 in the said embodiment is the following programs. That is, this program stores a computer in a first language corpus of a comparable corpus stored in a comparable corpus storage unit in which a comparable corpus having a first language corpus and a second language corpus is stored. A machine translation unit that machine-translates each sentence included in the second language, a second language sentence accumulation unit that accumulates sentences in the second language machine-translated by the machine translation unit, and a second language sentence storage unit A similarity information calculation unit that calculates similarity information that is information related to the similarity between a sentence included in a bilingual corpus and a second language sentence accumulated by the second language sentence accumulation unit; and the similarity information calculation unit includes: Using the calculated similar information, the sentence included in the second language corpus of the comparable corpus and the second language sentence accumulated by the second language sentence accumulation unit Among the combinations of the two, the sentence included in the second language corpus of the comparable corpus and the sentence in the second language accumulated by the second language sentence accumulating unit A bilingual information generating unit that generates bilingual information that is information that associates a sentence in the first language that is the source of machine translation, and a bilingual information output unit that outputs bilingual information generated by the bilingual information generating unit, The machine translation unit performs machine translation without using the bilingual information when translating each sentence included in the corpus of the first language included in the comparable corpus for the first time. When each sentence included in a corpus of one language is translated from the second time onward, machine translation using the parallel translation information is performed, and machine translation by the machine translation unit and the similarity information calculation unit are performed. The calculation of the similarity information, repeatedly executes more than once a series of processing for the generation of parallel translation information by the parallel translation information generation unit is intended.

  In the program, the functions realized by the program do not include functions that can be realized only by hardware. For example, a function that can be realized only by hardware such as a modem or an interface card in an output unit that outputs information is not included in at least the function realized by the program.

  Further, this program may be executed by being downloaded from a server or the like, and a program recorded on a predetermined recording medium (for example, an optical disk such as a CD-ROM, a magnetic disk, a semiconductor memory, or the like) is read out. May be executed by Further, this program may be used as a program constituting a program product.

  Further, the computer that executes this program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  FIG. 10 is a schematic diagram illustrating an example of an external appearance of a computer that executes the program and realizes the information processing apparatus 1 according to the embodiment. The above-described embodiment can be realized by computer hardware and a computer program executed on the computer hardware.

  10, a computer system 900 includes a computer 901 including a CD-ROM (Compact Disk Read Only Memory) drive 905 and an FD (Floppy (registered trademark) Disk) drive 906, a keyboard 902, a mouse 903, a monitor 904, and the like. Is provided.

  FIG. 11 is a diagram showing an internal configuration of the computer system 900. In FIG. 11, in addition to the CD-ROM drive 905 and the FD drive 906, a computer 901 is connected to an MPU (Micro Processing Unit) 911, a ROM 912 for storing a program such as a bootup program, and the MPU 911. A RAM (Random Access Memory) 913 that temporarily stores program instructions and provides a temporary storage space, a hard disk 914 that stores application programs, system programs, and data, and an MPU 911 and a ROM 912 are interconnected. And a bus 915. The computer 901 may include a network card (not shown) that provides connection to the LAN.

  A program that causes the computer system 900 to execute the functions of the information processing apparatus 1 according to the above-described embodiment is stored in the CD-ROM 921 or the FD 922, inserted into the CD-ROM drive 905 or the FD drive 906, and stored in the hard disk 914. May be forwarded. Instead, the program may be transmitted to the computer 901 via a network (not shown) and stored in the hard disk 914. The program is loaded into the RAM 913 when executed. The program may be loaded directly from the CD-ROM 921, the FD 922, or the network.

  The program does not necessarily include an operating system (OS) or a third-party program that causes the computer 901 to execute the functions of the information processing apparatus 1 according to the above-described embodiment. The program may include only a part of an instruction that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 900 operates is well known and will not be described in detail.

  Further, the present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, according to the information processing apparatus and the like according to the present invention, it is possible to generate a pair of sentences having a translation relation from a comparable corpus without using a bilingual dictionary or a bilingual thesaurus. It is useful for generating parallel corpora used in statistical translation technology.

The block diagram which shows the structure of the information processing apparatus by Embodiment 1 of this invention. The flowchart which shows operation | movement of the information processing apparatus by the embodiment The flowchart which shows operation | movement of the information processing apparatus by the embodiment The flowchart which shows operation | movement of the information processing apparatus by the embodiment The figure which shows an example of the similar information calculated in the embodiment The figure which shows an example of the similar information calculated in the embodiment The figure which shows an example of the similar information after the sort in the embodiment The figure which shows the experimental result in the same embodiment The figure which shows the experimental result in the same embodiment Schematic diagram showing an example of the appearance of the computer system in the embodiment The figure which shows an example of a structure of the computer system in the embodiment

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Information processing apparatus 11 Bilingual corpus storage part 12 Comparable corpus storage part 13 Machine translation part 14 1st language sentence storage part 15 2nd language sentence storage part 16 Similar information calculation part 17 Bilingual information generation part 18 Bilingual information output part 19 Bilingual translation Information storage unit

Claims (8)

A bilingual corpus storage unit that stores a bilingual corpus that is a corpus that associates a sentence in a first language and a sentence in a second language in a bilingual relationship;
A comparable corpus storage unit that stores a comparable corpus having a first language corpus and a second language corpus;
A machine translation unit that machine-translates each sentence included in the corpus of the first language included in the comparable corpus into a second language using the bilingual corpus;
A second language sentence accumulating unit for accumulating sentences of the second language machine translated by the machine translation unit;
A similarity information calculation unit that calculates similarity information that is information about similarity between a sentence included in the corpus of the second language included in the comparable corpus and a sentence of the second language accumulated by the second language sentence accumulation unit; ,
Using the similar information calculated by the similar information calculation unit, a combination of a sentence included in the second language corpus of the comparable corpus and a second language sentence accumulated by the second language sentence accumulation unit Of these, machine translation is performed for a sentence included in the second language corpus of the comparable corpus and a sentence in the second language accumulated by the second language sentence accumulating unit, which are included in a combination having high similarity between the two. A bilingual information generating unit that generates bilingual information that is information that associates a sentence in the first language that is the source of
An information processing apparatus comprising: a parallel translation information output unit that outputs the parallel translation information generated by the parallel translation information generation unit. The machine translation unit also performs a process of machine-translating each sentence included in the second language corpus of the comparable corpus into the first language using the bilingual corpus,
A first language sentence accumulating unit for accumulating sentences in the first language machine translated by the machine translating unit;
The similarity information calculation unit obtains similarity information that is information related to the similarity between a sentence included in the first language corpus of the comparable corpus and a sentence in the first language accumulated in the first language sentence accumulation unit. Also calculate
The bilingual information generation unit uses the similarity information calculated by the similarity information calculation unit, and the sentence included in the corpus of the first language included in the comparable corpus and the first language sentence accumulated by the first language sentence accumulation unit. A sentence included in a corpus of a first language included in the comparable corpus and a first language accumulated by the first language sentence accumulating unit, which are included in a combination having high similarity between the sentences of a language The information processing apparatus according to claim 1, further comprising: bilingual information that is information for associating a sentence in a second language that is a source of machine translation with the sentence. The machine translation unit performs machine translation using the bilingual information and the bilingual corpus when the bilingual information generated by the bilingual information generation unit exists,
The series of processes for performing machine translation by the machine translation unit, calculation of similar information by the similarity information calculation unit, and generation of parallel translation information by the parallel translation information generation unit are repeatedly performed twice or more, or The information processing apparatus according to claim 2. A comparable corpus storage unit that stores a comparable corpus having a first language corpus and a second language corpus;
A machine translation unit that machine translates each sentence included in the first language corpus of the comparable corpus into a second language;
A second language sentence accumulating unit for accumulating sentences of the second language machine translated by the machine translation unit;
A similarity information calculation unit that calculates similarity information that is information about similarity between a sentence included in the corpus of the second language included in the comparable corpus and a sentence of the second language accumulated by the second language sentence accumulation unit; ,
Using the similar information calculated by the similar information calculation unit, a combination of a sentence included in the second language corpus of the comparable corpus and a second language sentence accumulated by the second language sentence accumulation unit Of these, machine translation is performed for a sentence included in the second language corpus of the comparable corpus and a sentence in the second language accumulated by the second language sentence accumulating unit, which are included in a combination having high similarity between the two. A bilingual information generating unit that generates bilingual information that is information that associates a sentence in the first language that is the source of
A bilingual information output unit that outputs the bilingual information generated by the bilingual information generating unit,
The machine translation unit performs machine translation without using the parallel translation information when the sentences included in the corpus of the first language included in the comparable corpus are translated for the first time. When translating each sentence contained in one language corpus from the second time onwards, perform machine translation using the bilingual information,
An information processing apparatus that repeatedly executes a series of processes for performing machine translation by the machine translation unit, calculation of similar information by the similarity information calculation unit, and generation of parallel translation information by the parallel translation information generation unit twice or more. A parallel corpus storage unit that stores a parallel corpus that correlates a sentence in a first language and a sentence in a second language that are in a parallel translation relationship, and a comparable corpus having a corpus of a first language and a corpus of a second language Processing method processed using a composable corpus storage unit, a machine translation unit, a second language sentence storage unit, a similar information calculation unit, a parallel translation information generation unit, and a parallel translation information output unit Because
A machine translation step in which the machine translation unit machine translates each sentence included in the corpus of the first language included in the comparable corpus into a second language using the bilingual corpus;
A second language sentence accumulation step in which the second language sentence accumulation unit accumulates sentences of the second language machine-translated in the machine translation step;
The similarity information calculation unit calculates similarity information that is information related to the similarity between the sentence included in the second language corpus of the comparable corpus and the sentence in the second language accumulated in the second language sentence accumulation step. A similar information calculation step to calculate,
The bilingual information generation unit uses the similarity information calculated in the similarity information calculation step, and the sentence included in the corpus of the second language included in the comparable corpus and the second language sentence accumulated in the second language sentence accumulation step. Of the combinations with language sentences, the sentences included in the second language corpus of the comparable corpus and the second language stored in the second language sentence accumulation unit A bilingual information generating step for generating bilingual information that is information for associating a sentence in a first language as a source of machine translation with the sentence of
An information processing method comprising: a parallel translation information output step in which the parallel translation information output unit outputs the parallel translation information generated in the parallel translation information generation step. A comparable corpus storage unit that stores a comparable corpus having a first language corpus and a second language corpus, a machine translation unit, a second language sentence storage unit, a similar information calculation unit, and bilingual information generation Information processing method that is processed using a translation part and a parallel translation information output part,
A machine translation step in which the machine translation unit machine translates each sentence included in the corpus of the first language included in the comparable corpus into a second language;
A second language sentence accumulation step in which the second language sentence accumulation unit accumulates sentences of the second language machine-translated in the machine translation step;
The similarity information calculation unit calculates similarity information that is information related to the similarity between the sentence included in the second language corpus of the comparable corpus and the sentence in the second language accumulated in the second language sentence accumulation step. A similar information calculation step to calculate,
The bilingual information generation unit uses the similarity information calculated in the similarity information calculation step, and the sentence included in the corpus of the second language included in the comparable corpus and the second language sentence accumulated in the second language sentence accumulation step. Of the combinations with language sentences, the sentences included in the corpus of the second language possessed by the comparable corpus included in the combination having high similarity between the two, and the second language accumulated in the second language sentence accumulation step A bilingual information generating step for generating bilingual information that is information for associating a sentence in a first language as a source of machine translation with the sentence of
The bilingual information output unit includes a bilingual information output step for outputting the bilingual information generated in the bilingual information generating step;
A series of processes of the machine translation step, the second language sentence accumulation step, the similarity information calculation step, and the parallel translation information generation step are repeatedly executed twice or more,
In the machine translation step, when each sentence included in the corpus of the first language included in the comparable corpus is translated for the first time, machine translation is performed without using the parallel translation information, and the comparable corpus includes An information processing method for performing machine translation using the parallel translation information when each sentence included in a corpus of one language is translated from the second time onward. Computer
A bilingual corpus stored in a bilingual corpus storage unit that stores a bilingual corpus that correlates a sentence in a first language and a sentence in a second language in a bilingual relationship is used. A machine translation unit that machine-translates each sentence included in the first language corpus of the comparable corpus stored in the comparable corpus storage unit that stores the comparable corpus having the language corpus into the second language; ,
A second language sentence accumulating unit for accumulating sentences of the second language machine translated by the machine translation unit;
A similarity information calculation unit that calculates similarity information that is information about similarity between a sentence included in the corpus of the second language included in the comparable corpus and a sentence of the second language accumulated by the second language sentence accumulation unit; ,
Using the similar information calculated by the similar information calculation unit, a combination of a sentence included in the second language corpus of the comparable corpus and a second language sentence accumulated by the second language sentence accumulation unit Of these, machine translation is performed for a sentence included in the second language corpus of the comparable corpus and a sentence in the second language accumulated by the second language sentence accumulating unit, which are included in a combination having high similarity between the two. A bilingual information generating unit that generates bilingual information that is information that associates a sentence in the first language that is the source of
The program for functioning as a parallel translation information output part which outputs the parallel translation information which the said parallel translation information generation part produced | generated. Computer
Each sentence included in the first language corpus of the comparable corpus stored in the comparable corpus storage unit storing the comparable corpus having the first language corpus and the second language corpus is stored in the second language. A machine translation section that machine translates into
A second language sentence accumulating unit for accumulating sentences of the second language machine translated by the machine translation unit;
A similarity information calculation unit that calculates similarity information that is information about similarity between a sentence included in the corpus of the second language included in the comparable corpus and a sentence of the second language accumulated by the second language sentence accumulation unit; ,
Using the similar information calculated by the similar information calculation unit, a combination of a sentence included in the second language corpus of the comparable corpus and a second language sentence accumulated by the second language sentence accumulation unit Of these, machine translation is performed for a sentence included in the second language corpus of the comparable corpus and a sentence in the second language accumulated by the second language sentence accumulating unit, which are included in a combination having high similarity between the two. A bilingual information generating unit that generates bilingual information that is information that associates a sentence in the first language that is the source of
Function as a parallel translation information output unit that outputs the parallel translation information generated by the parallel translation information generation unit,
The machine translation unit performs machine translation without using the bilingual information when translating each sentence included in the corpus of the first language included in the comparable corpus for the first time. When translating each sentence contained in one language corpus from the second time onwards, perform machine translation using the bilingual information,
A program that repeatedly executes a series of processes of performing machine translation by the machine translation unit, calculation of similar information by the similarity information calculation unit, and generation of parallel translation information by the parallel translation information generation unit twice or more.

Images