CN114357122A - Text processing method and device - Google Patents

Abstract

The present application provides a text processing method and device, wherein the text processing method includes: obtaining target spoken text; classifying the target spoken text to obtain a text type corresponding to the target spoken text; In the case of text type, select the corresponding written language rewriting model according to the standard text type; input the target spoken language text into the written language rewriting model for processing, and obtain the target written language text corresponding to the target spoken language text; wherein, the The written language rewriting model is obtained by training the written language text and the spoken language text obtained by back-translating and converting the written language text, so as to select a written language rewriting model suitable for the target spoken language text according to the text type of the target spoken language text to rewrite the written language, so that the written language can be rewritten. Rewriting is more targeted and improves the accuracy of written language rewriting.

Description

Text processing method and device

technical field

The present application relates to the field of artificial intelligence of information technology, and in particular, to a text processing method. The present application also relates to a text processing apparatus, a computing device, and a computer-readable storage medium.

Background technique

Artificial intelligence (AI) refers to the ability of an engineered (ie designed and manufactured) system to perceive its environment, and to acquire, process, apply and represent knowledge. The development status of key technologies in the field of artificial intelligence, including machine learning, knowledge graph, natural language processing, computer vision, human-computer interaction, biometric recognition, virtual reality/augmented reality and other key technologies. Natural Language Processing (NLP) refers to the use of computers to process the shape, sound, meaning and other information of natural language, that is, the input, output, recognition, analysis, understanding, generation, etc. NLP is an important direction in the field of computer science and artificial intelligence. It studies various theories and methods that can realize effective communication between humans and computers using natural language.

Text generation task is an important research direction in the field of natural language processing. Among them, machine translation, summary generation, text style transfer, etc. are important tasks in the field of natural text generation. As one of the important tasks in the field of natural text generation, oral text-written text rewriting has important applications in daily work and life. For example, in the analysis and application scenarios involving spoken text, such as the analysis of recorded texts, the minutes of conference voice texts, and the transcription of important written materials and documents, the quality of transcription from spoken text to written text is very important. However, due to the uneven quality of spoken texts and the large uncertainty and discontinuity of the resulting texts generated by text generation tasks, the task of transcribing spoken texts to written texts has always been a huge challenge.

SUMMARY OF THE INVENTION

In view of this, embodiments of the present application provide a text processing method, and the present application simultaneously relates to a text processing apparatus, a computing device, and a computer-readable storage medium, so as to solve the technical defects existing in the prior art.

According to a first aspect of the embodiments of the present application, a text processing method is provided, including:

obtain written text;

By performing back-translation processing on the written text, a back-translated written text corresponding to the written text is obtained;

The written text and the back-translated written text are respectively subjected to the conversion processing of the sentence constituent units to obtain the spoken text;

A sample corpus is constructed based on the written text and the correspondence between the back-translated written text and the spoken text.

According to a second aspect of the embodiments of the present application, a text processing apparatus is provided, including:

The acquisition module is configured to acquire the target spoken text;

a classification module, configured to classify the target spoken text to obtain a text type corresponding to the target spoken text;

a selection module, configured to select a corresponding written language rewriting model according to the standard text type when the text type is a standard text type;

a processing module, configured to input the target spoken text into the written language rewriting model for processing, and obtain a target written language text corresponding to the target spoken language text; wherein, the written language rewriting model is based on the written language text and the written language text It is obtained by training the spoken text obtained by back-translation and conversion processing.

According to a third aspect of the embodiments of the present application, a computing device is provided, including a memory, a processor, and computer instructions stored in the memory and executable on the processor, the processor implementing the computer instructions when the processor executes the computer instructions. Describe the steps of the text processing method.

According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, which stores computer instructions, and when the computer instructions are executed by a processor, implements the steps of the text processing method.

According to a fifth aspect of the embodiments of the present application, a chip is provided, which stores computer instructions, and when the computer instructions are executed by the chip, implements the steps of the text processing method or the spoken language generation method.

In the embodiment of the present application, by obtaining the target spoken text; classifying the target spoken text to obtain the text type corresponding to the target spoken text; and then in the case that the text type is a standard text type, according to the Selecting the corresponding written language rewriting model for the standard text type realizes selecting a written language rewriting model suitable for the target spoken language text according to the text type of the target spoken language text; then inputting the target spoken language text into the written language rewriting model for processing to obtain the target spoken language The target written language text corresponding to the text makes the written language rewriting more targeted and improves the accuracy of the written language rewriting. The written language rewriting model is obtained by training based on the written language text and the spoken language text obtained by back-translation and conversion processing of the written language text, and realizes the preprocessing of the written language text based on the back-translation and conversion processing, thereby providing model training. A large amount of spoken text-written text sample corpus simplifies the training difficulty of the model, and also avoids the time-consuming and laborious collection and processing of large amounts of text data, saving time and labor costs.

Description of drawings

1 is a structural block diagram of a computing device provided by an embodiment of the present application;

2 is a schematic diagram of a text processing method provided by an embodiment of the present application;

3 is a flowchart of a text processing method provided by an embodiment of the present application;

4 is a schematic diagram of constructing a sample corpus in a text processing method provided by an embodiment of the present application;

FIG. 5 is a processing flowchart of a text processing method applied to an actual scene provided by an embodiment of the present application;

FIG. 6 is a schematic structural diagram of a text processing apparatus provided by an embodiment of the present application.

Detailed ways

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar promotions without violating the connotation of the present application. Therefore, the present application is not limited by the specific implementation disclosed below.

The terminology used in one or more embodiments of the present application is for the purpose of describing a particular embodiment only, and is not intended to limit the one or more embodiments of the present application. As used in one or more embodiments of this application and the appended claims, the singular forms "a," "the," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or" as used in one or more embodiments of this application is meant to include any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, etc. may be used in one or more embodiments of the present application to describe various information, such information should not be limited by these terms. These terms are only used to distinguish the same type of information from each other. For example, the first could be termed the second, and similarly the second could be termed the first, without departing from the scope of one or more embodiments of the present application. Depending on the context, the word "if" as used herein may be interpreted as "in response to determining."

First, terminology related to one or more embodiments of the present invention is explained.

Seq2Seq (Sequence to Sequence) models: A family of machine learning methods for natural language processing, commonly used in applications such as machine translation, image description, dialogue models, and text summarization.

Transformer model: A deep learning model that uses an attention mechanism to differentially weight the importance of each part of the input data, and is widely used in various natural language processing tasks.

Text classification: refers to assigning texts to one or more categories in a given classification system.

Natural Language Generation (NLG): The part of natural language processing that generates natural language text from machine representation systems such as knowledge bases or logical forms.

Text style transfer: Transcribing text in one style to generate text in another style.

Abstract generation: Through technical solutions, the process of compressing, summarizing and summarizing long texts to form short texts with general meanings is realized.

Machine Translation: The process of converting one natural language (source language) into another natural language (target language) using a computer.

Entity: Refers to the entity word or phrase description with a specific meaning in the text.

Part-of-speech tagging: It is the process of determining the grammatical category of each word in a given sentence, determining its part-of-speech and marking it, which is also a very important basic work in natural language processing (NLP).

Syntactic analysis: It is one of the key underlying technologies in natural language processing (NLP), and its basic task is to determine the syntactic structure of a sentence or the dependencies between words in a sentence.

In the present application, a text processing method is provided, and the present application simultaneously relates to a text processing apparatus, a computing device and a computer-readable storage medium, which will be described in detail in the following embodiments.

FIG. 1 shows a structural block diagram of a computing device 100 provided according to an embodiment of the present application. Components of the computing device 100 include, but are not limited to, memory 110 and processor 120 . The processor 120 is connected with the memory 110 through the bus 130, and the database 150 is used for saving data.

Computing device 100 also includes access device 140 that enables computing device 100 to communicate via one or more networks 160 . Examples of such networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. Access device 140 may include one or more of any type of network interface (eg, network interface card (NIC)), wired or wireless, such as IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, World Interoperability for Microwave Access ( Wi-MAX) interface, Ethernet interface, Universal Serial Bus (USB) interface, cellular network interface, Bluetooth interface, Near Field Communication (NFC) interface, etc.

In one embodiment of the present application, the above-described components of the computing device 100 and other components not shown in FIG. 1 may also be connected to each other, eg, via a bus. It should be understood that the structural block diagram of the computing device shown in FIG. 1 is only for the purpose of example, rather than limiting the scope of the present application. Those skilled in the art can add or replace other components as required.

Computing device 100 may be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (eg, tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (eg, smart phones) ), wearable computing devices (eg, smart watches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. Computing device 100 may also be a mobile or stationary server.

In practical applications, since the oral text-written text rewriting has important applications in daily work and life, in the prior art, in order to realize the oral text-written text rewriting, an artificial method can be used to rewrite the oral text into the written text. You can also use the method of rule rewriting to rewrite and replace some of the processable oral expressions; in addition, you can directly use the method of text translation to translate the spoken text into written text.

Among them, the manual rewriting method consumes a lot of manpower, and the quality and results of the text transcription are not uniform; using the regular method for rewriting, only a limited number of colloquial expressions and fixed text forms can be processed. The processing complexity of logic rules is high; while the way of rewriting by text translation has high requirements on the magnitude of the text corpus and the quality of the text data. This method can achieve a certain degree of transcription effect, but it is not suitable for spoken language to the whole. Written transcription tasks.

Therefore, in order to achieve accurate rewriting of spoken text to written text, a pre-trained rewriting model can be used to rewrite the spoken text. However, due to the uneven quality of spoken texts, if a unified rewriting model is used to process these spoken texts, accurate rewriting effects may not be achieved. Therefore, there is an urgent need for an effective solution to solve the above problems.

Referring to FIG. 2, FIG. 2 shows a schematic diagram of a text processing method provided according to an embodiment of the present application. After acquiring the target spoken text, the target spoken text is input into the text classification model, which is obtained by training the initial text classification model through pre-collected text corpus semantic clarity annotation data. Further, the text classification model outputs the predicted text type corresponding to the target spoken text by performing text classification on the input target spoken text. Then, according to the predicted text type, determine the rewriting ambiguity corresponding to the target spoken text (i.e. the written rewriting model or the written language conversion model; if the predicted text type is a standard text type, the rewriting ambiguity is a written rewriting model; if the predicted text type is fuzzy text type, the rewriting is fuzzy as a written language conversion model). Among them, the rewriting fuzzy is obtained by training the model through the sample corpus constructed from the oral text-written text alignment data. Further, in the rewriting ambiguity, the written text corresponding to the target spoken text can be output by rewriting the input target spoken text in written language.

The embodiment of the present application classifies the target spoken text according to the quality of the target spoken text, and selects a rewriting model suitable for the target spoken text to rewrite it, thereby making the written rewriting more targeted and improving the accuracy of the written rewriting sex.

3 shows a flowchart of a text processing method provided according to an embodiment of the present application, which specifically includes the following steps:

Step 302: Obtain the target spoken text.

The target spoken text refers to the spoken text to be rewritten. In practical applications, the target spoken text can be spoken text in any field, such as spoken text in the medical field, oral text in the field of chemistry, oral text in the field of sales, spoken text in the field of daily life, spoken text in the field of tourism, etc. In addition, the text quantity of the target spoken text may be one or multiple.

This embodiment takes the acquired target spoken text TST as an example to describe the text processing method. For other processing procedures of the target spoken text, reference may be made to the same or similar descriptions in this embodiment, which are not limited herein.

Step 304: Classify the target spoken text to obtain a text type corresponding to the target spoken text.

Specifically, on the basis of the above-mentioned acquisition of the target spoken text, since the quality of the target spoken text to be rewritten in written language may vary, in this case, directly rewriting the target spoken text in written language may not guarantee the quality of the rewriting. , in order to ensure the effect of written language rewriting, the target oral text can be classified, so that corresponding rewriting measures can be implemented for different types of target oral text.

Further, the target spoken text is classified and processed to obtain the text type corresponding to the target spoken text, which is specifically implemented in the following manner:

Input the target spoken text into the text classification model for classification processing, and obtain the text type corresponding to the target spoken text; wherein, the training of the text classification model includes:

Obtain the sample spoken text and the semantic clarity label corresponding to the sample spoken text;

Construct training sample pairs based on sample spoken text and semantic clarity labels;

Model training is performed on the initial text classification model by training samples until a text classification model that satisfies the classification training stopping condition is obtained.

The text classification model refers to a pre-trained model for classifying the target spoken text. The text classification model can be a two-classification model. By identifying the text semantic clarity of the target spoken text, the target spoken text is classified into standard text types. or fuzzy text type. Among them, the standard text type means that the textual semantic expression of the target spoken text is relatively clear; the fuzzy text type means that the textual semantic expression of the target spoken text is relatively vague. The semantic clarity label refers to the label marked according to the semantic clarity of the sample spoken text.

In addition, considering that the target spoken text may also be text that does not contain semantic information, it is meaningless to rewrite this type of text in written language. Therefore, a three-classification model can be used to classify the target spoken text and classify the target spoken text into standard text types, fuzzy text types or invalid text types.

In practical applications, if the text classification model is a binary classification model, the semantic clarity labels include: standard text type and fuzzy text type. If the text classification model is a three-classification model, the semantic clarity labels include: standard text type, fuzzy text type, and invalid text type.

It should be noted that the semantic clarity label of the sample spoken text needs to be marked in advance, and then the sample spoken text and the semantic clarity label are used as the text corpus semantic clarity label data to train the initial text type model. Among them, the initial text classification model can be a text classification model to be trained constructed in advance through CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory Network), FastText, TextCNN, HAN model, etc.

During the specific implementation, in the training process, the sample spoken text can be classified and processed by the initial text classification model to output the predicted text type, and the predicted text type and the loss value of the sample text type recorded by the semantic clarity label corresponding to the sample spoken text can be calculated. , In practical applications, the loss function for calculating the loss value of the model can be a 0-1 loss function, an absolute value loss function, a squared loss function, a cross entropy loss function, etc. Here, the 0-1 loss function is taken as an example For an explanation, see Equation 1 below:

公式1

Formula 1

Among them, L represents the loss value, f(X) represents the predicted text type, and Y represents the sample text type. In this application, the selection of the loss function is not limited, and the actual application shall prevail.

After calculating the model loss value, you can reversely adjust the model parameters of the initial text classification model according to the model loss value, and sample the next batch of text corpus semantic clarity annotation data to continue training the initial text classification model until the classification training stop condition is reached , Specifically, the classification training stop condition may be that the model loss value is less than the preset threshold or the number of training iterations reaches the preset number of iterations, etc., which is not limited here.

To sum up, the semantic clarity classification of the target spoken text through the pre-trained text classification model can effectively identify the semantic quality of the target spoken text, so as to rationally rewrite different types of target spoken text, ensuring the accuracy of written rewriting. quality.

Step 306: In the case that the text type is a standard text type, select a corresponding written language rewriting model according to the standard text type.

Specifically, after the above-mentioned determination of the text type corresponding to the target spoken text, considering that the semantic expression of the target spoken text is relatively clear (that is, the text type is a standard text type), when rewriting it in written language, it is possible to ensure the semantics based on a more complex rewriting method. Therefore, for the target spoken text of standard text type, a written rewriting model that performs more complex rewriting can be selected for processing.

The written language rewriting model refers to the module used to rewrite the spoken text into the written language text. Specifically, the written language rewriting model may be constructed based on the Seq2Seq model, and both the encoder and the decoder in the Seq2Seq model may be constructed using the Transformer model.

Following the above example, the target spoken text TST is input into the text classification model, and the text type corresponding to the target spoken text TST output by the text classification model is obtained as the standard text type, then the target spoken text TST is input into the written language rewriting model, and the written language is obtained. Rewrite the target written language text TLT1 output by the model.

Step 308: Input the target spoken text into the written rewriting model for processing, and obtain the target written text corresponding to the target spoken text.

Among them, the written language rewriting model is obtained by training based on the written language text and the spoken language text obtained by back-translating and converting the written language text.

Specifically, on the basis of selecting the written language rewriting model above, the written language rewriting can be performed on the target spoken language text through the written language rewriting model, so as to obtain the target written language text generated after the rewriting.

During specific implementation, in order to ensure the accuracy of written language rewriting by the written language rewriting model and avoid the uncontrollable situation of the text generation result, a character-level mask operation can be used in the rewriting process. Through the character-level mask operation, it is ensured that all the generated results of written language rewriting mainly come from the input text. In this embodiment of the present application, the written language rewriting model includes an encoding layer and a decoding layer, and the target spoken language text is rewritten through the written language rewriting model. Specifically, It is implemented as follows:

Sentence processing is performed on the target spoken text to obtain the sentence sequence contained in the target spoken text;

The spoken sentence units in the sentence sequence are sequentially input into the coding layer of the written language rewriting model for encoding processing to obtain the sentence feature vector and the vocabulary vector corresponding to the spoken sentence unit, wherein the vocabulary vector is obtained by mapping the spoken sentence unit and the vocabulary;

Calculate the vector product between the sentence feature vector and the vocabulary vector, and input the vector product into the decoding layer of the written language rewriting model for decoding processing to obtain the target written language text corresponding to the target spoken language text.

Among them, the coding layer is a hierarchical structure in the text generation model, which is expressed by converting information into another form for processing inside the model. Correspondingly, the sentence feature vector specifically refers to a vector expression obtained after encoding the spoken sentence unit. The decoding layer specifically refers to the hierarchical structure used in the written language rewriting model to convert the sentence feature vector into a decoding vector. In practical applications, after the decoder outputs the decoding vector, the decoding vector is input to the output layer to obtain the target written language output by the output layer. text.

Vocabulary is a list of words. Specifically, the vocabulary list may be generated by counting the frequencies of words/characters that appear in the sample corpus during the training process of the written language rewriting model (for example, adding characters/words that appear in the training sample corpus with a frequency greater than a threshold to the vocabulary list) ), can also be carried by the model itself, and can also be generated in other ways. The sentence sequence refers to a sequence formed by arranging the spoken sentences contained in the target spoken text according to the sequence in the target spoken text. Correspondingly, the spoken sentence unit refers to the spoken sentence contained in the sentence sequence.

During specific implementation, mapping the spoken sentence unit to the vocabulary refers to matching the characters/words in the spoken sentence with the characters/words in the vocabulary; if the characters/words in the spoken sentence hit any character/word in the vocabulary, Then, the vector bit corresponding to the hit character/word in the vocabulary is set to 1, and the vector bit corresponding to the missed character/word in the vocabulary is set to 0, then the vocabulary vector can be obtained. For example, the vocabulary contains 5000 characters. By mapping the 4 characters in the spoken sentence 1 to the vocabulary, the first character maps the third character in the vocabulary, and the second character maps the vocabulary. The 6th character in the vocabulary, the 3rd character maps the 9th character in the vocabulary, and the 4th character maps the 5th character in the vocabulary, then the obtained vocabulary vector is 00101100100...0.

Further, the vector product between the vocabulary vector and the sentence feature vector is calculated, and then decoding is performed based on the vector product, which realizes the restriction and restriction on the decoded output by counting the input text characters/words. The above operations implemented through the vocabulary can also be referred to as character-level mask operations.

To sum up, the character-level mask operation ensures that the text characters generated by the written language rewriting model mainly come from the input text source, which greatly avoids the semantic deviation of the rewriting results of the written language rewriting model.

During specific implementation, the training of the written language rewriting model is specifically implemented through the following steps 30802 to 30810:

Step 30802: Obtain written text.

Written text refers to the text formed in the language that people use when writing and reading articles, and the text is the main component. The written text can be written text in any field, such as written text in the field of medicine, written text in the field of chemistry, written text in the field of sales, written text in the field of daily life, written text in the field of tourism, etc. In addition, the written text is also According to the different styles used, it can be divided into literary style, official document style, scientific and technological style, etc.

For example, the acquired written text is the written text LT in the literary style.

Step 30804: Obtain the back-translated written text corresponding to the written text by performing back-translation processing on the written text.

Specifically, on the basis of the above-mentioned acquisition of the written text, considering that the written text is simply converted to generate the corresponding spoken text, the expansion of the sample corpus may still be limited. The translation processing method expands the written text, and then performs text conversion on the expanded written text to convert it into the spoken text corresponding to the written text.

The back-translation processing refers to the process of translating the text in language A into language B, and then translating the text in language B back into language A. In practical applications, since the back-translated written text generated by the back-translation process can generate a text representation that is different from the original written text, the back-translated written text generated by the back-translation process can expand the written text.

Further, considering that the back-translated written text generated after the back-translation process may be quite different from the original written text, and may lose the meaning to be expressed by the original written text, in order to ensure that the key information in the back-translated written text and the written text remains unchanged. Change, the back-translated written text generated by the back-translation can be replaced by the key words in the written text, and the embodiment of the present application is implemented in the following manner:

Translate the written language text into the translated written language text corresponding to the preset language;

Back-translate the written text of the translation into the target language to which the written text belongs, and obtain the initial back-translated written text;

The back-translated written text is obtained by replacing the target key words corresponding to the key words in the initial back-translated written text by the key words in the written text.

The preset language can be any one or more languages such as English, French, Korean, German, etc., which is not limited here. Correspondingly, the target language refers to the language to which the characters in the written text belong.

In practical applications, the written text is first translated into other languages, that is, the translated written text. Then translate the translated written text back to the language to which the written text belongs to obtain the initial back-translated written text. The initial back-translated written text may be quite different from the written text due to the back-translation process, or even deviate from the expressed meaning of the written text. In order to keep the key information of the two texts unchanged, the corresponding words (ie target key words) in the initial back-translated written text can be replaced by the key words in the written text, so as to generate a return with the key information consistent with the written text. Translate written text.

Among them, the key words may be pre-selected words in the written text that are considered to be more important to the written text. In practical applications, the key words may be selected according to preset selection rules, and the preset selection rules may be based on parts of speech. It can also be selected according to the entity type of the word. In addition, key words may also be selected through a pre-established keyword database, and words in the keyword database included in the written language text may be used as key words and the like.

During specific implementation, the target keyword corresponding to the keyword in the initial back-translated written text is replaced by the keyword in the written text, and the target keyword corresponding to the keyword needs to be determined first. Specifically, the determination method can be various, for example, it can be determined according to the positional association between the keyword and the target keyword in the text sentence, or it can be determined by searching for the synonym corresponding to the keyword in the initial back-translated written text, The synonym is used as the target key word, and the words belonging to the same sentence component can also be determined as the target key word according to the sentence component of the key word in the text sentence (for example, the subject, predicate, object, attribute, adverbial or Complements, etc., are used as key words, and words with the same components in the initial back-translation written text are selected as target key words). In practical applications, the target keyword corresponding to the keyword may be determined in an appropriate manner according to the actual scene.

After the target key words corresponding to the key words are determined, the back-translated written text can be obtained by replacing the corresponding target key words in the initial back-translated written text by the key words.

Following the above example, on the basis of determining that the language to which the written text LT belongs is Chinese and the default language is German, translate the Chinese written text LT into German to obtain the German translated written text LT1, and then the German translated written text LT1 is translated into: Chinese, and the initial back-translation written text LT2 of Chinese is obtained. Assuming that the written language text LT contains a written language sentence S1, the written language sentence S1 is specifically "My hometown is Shanxi, where it is beautiful". The key word in the written language sentence S1 is the geographic location entity "Shanxi", and the written language sentence S11 corresponding to the written language sentence S1 in the initial back-translated written language text LT2 is "My hometown is Shaanxi, where it is very beautiful", the written language If the target key word corresponding to the key word in the written language sentence S11 corresponding to the sentence S1 is the geographic location entity "Shaanxi", the "Shaanxi" in the written language sentence S11 is replaced by "Shanxi", and the back-translated written language text LT3 is obtained. The written language text LT3 includes the written language sentence S12 after replacing the written language sentence S11 "My hometown is Shanxi, it is very beautiful".

To sum up, in the process of back-translation, the key words in the written text are used to replace the corresponding target key words in the back-translated written text, so as to ensure the back-translated written text and written text in the case of corpus expansion of the written text. Consistency of key information in Improved the accuracy of back-translating written text.

During the specific implementation, considering that accurately determining the target keyword corresponding to the keyword is crucial to maintaining the consistency of the back-translated written text and the text in the written text, in order to avoid determining the replacement of the wrong target keyword by the keyword, By adding position marks to the key words in the written language text, it can be ensured that the target key words corresponding to the key words can be accurately obtained and replaced. include:

Identify the key words whose part of speech is the preset part of speech in the written text by analyzing the part of speech of the written text;

Mark the position of key words in the written text;

Correspondingly, the target key words corresponding to the key words in the initial back-translated written text are replaced by the key words in the written text to obtain the back-translated written text, including:

Based on the position markers, the corresponding target key words in the initial back-translated written text are replaced by the key words to obtain the back-translated written text.

Specifically, the part-of-speech analysis on the written text may be by marking the words in the written text by part of speech to determine what part of speech the words in the written text are. Among them, the part-of-speech tagging method can be a rule-based part-of-speech tagging method, or a part-of-speech tagging method based on a statistical model. In addition, a part-of-speech tagging method based on a combination of statistical methods and rule methods can also be used. Correspondingly, part of speech refers to the characteristics of words as the basis for classifying parts of speech. Parts of speech can be noun part of speech, verb part of speech, adjective part of speech, numeral part of speech, etc. In practical applications, because written texts may belong to different fields, the words (ie, key words) that are considered important parts of speech in different fields may be different. Noun part-of-speech words are key words.

During specific implementation, the position of the key word is marked with a position, which may be marked with symbols such as curly brackets "{}", or an asterisk "*". In practical applications, the position marker can be added before and after the keyword. For example, the key word is: mobile phone, the position of the key word is marked by braces "{}", and the marked key word is {mobile phone}.

It should be noted that, before translating the written text, the key words in the written text are marked with positions. The position mark can still be retained in the initial back-translated written text obtained after back-translation of the written text, and the word marked by the position mark in the initial back-translated written text is the target key word. That is, the target keyword corresponding to the keyword can be accurately located by means of position marking, so that the target keyword can be accurately replaced. When there are multiple key words in a written sentence, the target key word corresponding to the key word can be determined according to the similarity between the word marked by the position mark and the key word, or the target key word corresponding to the key word can be determined according to the similarity of the word marked by the position mark in the sentence. components (such as subject, predicate, object and other sentence components), determine the marked words of the same sentence components as the target keyword corresponding to the keyword.

In addition, in order to facilitate subsequent text processing of the written text and the replaced text after the replacement, the position markers in the written text and the replaced text can be deleted, and the position markers in the replaced text can be deleted. Get back-translated written text.

Taking the written language sentence S1 in the written language text LT as an example, the part-of-speech analysis of the written language sentence S1 is carried out. In the written language sentence S1, the key words whose preset part-of-speech is identified as a noun include: "hometown" and "Shanxi", and the noun is identified as a noun. "Hometown" and "Shanxi" are marked by the position mark {}, and the marked written language sentence S1 is obtained, and the marked written language sentence S1 is "My {hometown} is {Shanxi}, it is beautiful there". And the written language sentence S11 corresponding to the marked written language sentence S1 in the initial back-translated written language text LT2 is "My {hometown} is {Shaanxi}, it is very beautiful there", then the "hometown" is used to mark the position in the written language sentence S11. The "hometown" of the target keyword corresponding to "{}" is replaced, and the "Shaanxi" of the target keyword corresponding to the position mark "{}" in the written language sentence S11 is replaced by "Shanxi", and the replaced written language sentence is obtained. S11 is "My {hometown} is {Shanxi}, it is very beautiful there", delete the position mark in the replaced written language sentence S11, and the deleted written language sentence S12 is "My hometown is Shaanxi, it is very beautiful there" .

To sum up, after the key words in the written text are marked before translation, the target key words are determined and replaced by the location marking, which improves the accuracy and efficiency of the replacement.

Step 30806: Perform the conversion processing of the sentence constituent units on the written text and the back-translated written text, respectively, to obtain the spoken text.

Specifically, on the basis of obtaining the back-translated written text corresponding to the written text by performing back-translation processing on the written text, in order to further expand the spoken text, the written text and the back-translated written text can be converted respectively, so as to obtain the corresponding back-translated written text. Spoken text.

During specific implementation, the oral text obtained after the conversion can be further screened to obtain the oral text with relatively accurate semantic expression, which further improves the accuracy of the oral conversion of the written text.

Optionally, the sentence composition unit includes at least one of the following: a clause unit, a word unit, a character unit, and a symbol unit.

In practical applications, written language text is usually composed of written language sentences, and written language sentences are usually composed of various sentence units. These sentence units include: clause units (clauses), word units (words), character units (words) and symbolic units (punctuation marks), etc. There may be differences between written and spoken expressions in each sentence composition unit. Therefore, the written sentence can be transformed for each sentence composition unit, so that the written sentence can be divided into clause units, word units, character units and symbol units. etc. are more characteristic of oral expression.

Among them, the clause unit refers to the clause in the written language sentence. For example, when the written language sentence is "the weather is sunny today, there are no clouds in the sky, it is suitable to go out to play", the written language sentence includes 3 clauses, among which, the clause 1 is: "the weather is fine today", clause 2 is: "there are no clouds in the sky", and clause 3 is: "suitable for going out to play", these three clauses are separated by commas in written language sentences. Correspondingly, word units refer to words in written sentences. A character unit refers to a character in a written language sentence, and the character can be understood as a word in English or a single character in Chinese, which is not limited here. Symbolic units refer to punctuation marks in written language sentences, such as commas, quotation marks, dashes, etc., which are not limited here. In specific implementation, the written language sentences in the written language text can be adjusted or rewritten at the clause level, and/or the written language sentences in the written language text can be adjusted or rewritten at the word level, and/or The written language sentences in the written language text are adjusted or rewritten at the character level, and/or the written language sentences in the written language text are adjusted or rewritten at the symbol level, etc., so that the expression of the written language text can be more characteristic of spoken language.

During specific implementation, since the written text and the back-translated written text are used as different written corpora to construct sample corpora, the written text and the back-translated written text need to be converted into sentence constituent units respectively to obtain the corresponding spoken text. , the embodiment of the present application is specifically realized in the following ways:

The written language text is converted into the sentence composition unit to obtain the first spoken language text corresponding to the written language text;

Perform the conversion processing of the sentence composition unit on the back-translated written text to obtain the second spoken text corresponding to the back-translated written text;

The first spoken text and the second spoken text are taken as the spoken text.

The first spoken text refers to the spoken text obtained by converting the written text. The second spoken text refers to the spoken text obtained by converting the back-translated written text.

Following the above example, the written language text LT is converted into sentence composition units to obtain the first spoken language text ST1 corresponding to the written language text LT, and the sentence composition unit conversion processing is performed on the back-translated written language text LT3 to obtain the back-translated written language text LT3 For the corresponding second spoken text ST2, the first spoken text ST1 and the second spoken text ST2 are taken as the spoken text.

To sum up, the corresponding spoken texts are obtained by converting the written text and the back-translated written texts into the sentence constituent units respectively, and the obtained two spoken texts are taken as the spoken texts, that is, two spoken texts are obtained. Augmentation of spoken text.

In practical applications, although there may be many differences between spoken and written expressions, these differences are not reflected in every sentence, but appear with a certain probability according to the speaker's expression habits. It is more in line with the characteristics of spoken language. The corresponding conversion processing probability can be set for each conversion processing strategy, and whether to execute the conversion processing strategy is determined according to the conversion processing probability. The specific implementation is as follows:

Determine the conversion processing probability corresponding to the conversion processing strategy of the written text to be processed;

Based on the conversion processing probability, determine the target conversion processing strategy to be executed in the conversion processing strategy;

By executing the target conversion processing strategy, the written text is converted into sentence constituent units, and the spoken text corresponding to the written text to be processed is obtained.

The conversion processing strategy refers to a preset method (strategy) for the conversion processing of the written text to be processed. Specifically, the conversion processing strategy may include at least one of the following: a clause conversion processing strategy (a strategy for processing written sentences with clause units), and a word conversion processing strategy (a strategy for performing word unit conversion processing on written sentences) , a character conversion processing strategy (a strategy for converting character units to written language sentences), and a symbol conversion processing strategy (a strategy for converting written language sentences to symbol units).

Wherein, the clause conversion processing strategy may be duplication processing of clauses (ie, copying the clause conversion processing strategy), out-of-order processing, and/or inversion processing, and the like. The word conversion processing strategy may be addition processing, repetition processing, and/or out-of-order processing of words. The character conversion processing strategy may be character out-of-order processing, etc. The symbol conversion processing strategy may be deleting symbol processing, adding symbol processing, and/or modifying symbol processing, and the like.

Specifically, the conversion processing probability corresponding to the conversion processing strategy refers to the probability of executing the conversion processing strategy. In practical applications, each conversion processing strategy may have a corresponding conversion processing probability. Further, based on the conversion processing probability, the target conversion processing strategy to be executed is determined in the conversion processing strategy. Taking the conversion processing strategy A as an example, the conversion processing probability corresponding to the conversion processing strategy A is 10%. Then you can set a value range, the value range is 1-100 (or 1-10, etc.), and set a value range within this value range with the same value probability as the conversion processing probability corresponding to the conversion processing strategy A, such as 1-10 (or 90-100), and then randomly generate any value in the range of 1-100. If the generated value is 9, and the value is between 1 and 10, it means that the value satisfies the value probability of 10%, that is, it satisfies the conversion processing probability corresponding to executing the conversion processing strategy A, then it is determined to execute the conversion processing strategy A. , and use the conversion processing strategy A as the target conversion processing strategy; if the generated value is 50, and the value is between 11-100, it means that the value does not meet the 10% probability of taking a value, that is, the conversion is not satisfied. The conversion processing probability corresponding to the processing strategy A, so it is determined not to execute the conversion processing strategy. Similarly, other conversion processing strategies can also be processed in the above manner.

Further, the determined target conversion processing strategy may be one or multiple. In the case where there are multiple target conversion processing strategies, a preset execution sequence may be adopted, and these target conversion processing strategies are sequentially executed to perform conversion processing on the written text to be processed.

It should be noted that since each conversion processing strategy has a corresponding conversion processing probability, and each conversion processing strategy itself has a certain degree of randomness, multiple conversion processing is performed on the same written text to be processed, and the final generation The spoken text is likely to be different. Therefore, in order to further expand the corpus, at least one written language text to be processed may be subjected to the conversion processing of the sentence composition unit multiple times, so as to obtain a plurality of spoken language texts corresponding to the written language text to be processed.

In addition, considering that a higher conversion processing probability is set for the conversion processing strategy, the complexity of the sample corpus will be increased. The higher the complexity of the sample corpus, the more complex the written language rewriting performed by the rewriting model obtained by training the sample corpus. Therefore, when there are multiple rewriting models for different text types, complex rewriting is not suitable for the target spoken text of ambiguous text type. Therefore, in the case of constructing the sample corpus of the rewriting model corresponding to the target spoken text of the fuzzy text type, a lower conversion processing probability can be set for the conversion processing strategy.

Following the above example, assuming that there are four conversion processing strategies for written text, the conversion processing probabilities corresponding to each conversion processing strategy for written text LT are determined to be: 2%, 6%, 0.8%, and 8%, respectively. Then for each conversion processing strategy, a value range can be set for its corresponding conversion processing probability, and a value range corresponding to the conversion processing probability can be set, by randomly generating a number, if the number is within the value range , then the conversion processing strategy corresponding to the conversion processing probability is determined as the target conversion processing strategy, and the target conversion processing strategy is executed to convert the written text LT to sentence constituent units to obtain the spoken text corresponding to the written text LT.

In summary, the corresponding conversion processing probability is set for each conversion strategy, that is, each conversion strategy is executed according to a certain execution probability, so there is no need to deliberately execute each conversion processing strategy, thus ensuring the naturalness and rationality of written language conversion. .

During specific implementation, since the back-translated written text is generated to expand the written text, it is necessary to perform oral language conversion on the written text and the back-translated written text respectively. Therefore, any written language text in the written language text and the back-translated written language text can be used as the written language text to be processed, and the sentence composition unit conversion processing is performed on the written language text to be processed. When the sentence composition unit is a clause unit, the specific This is achieved by performing steps 30806-2 to 30806-6 as follows:

Step 30806-2: Perform sentence recognition on the written language text to be processed to obtain written language sentences contained in the written language text to be processed.

Performing sentence recognition on the written text to be processed can be understood as performing sentence processing on the text to be processed. In practical applications, it is possible to identify the clause symbols (such as period, question mark, semicolon and other identifiers used for clause operations) contained in the written text to be processed, and to divide (identify) sentences by means of clause symbols, that is, At least one written language sentence contained in the written language text to be processed can be obtained.

Step 30806-4: Convert the written language sentence to the clause unit to obtain the converted written language sentence.

Further, the clause unit conversion is performed on each of the identified written language sentences, so as to obtain a converted written language sentence corresponding to each written language sentence.

Specifically, since there are various ways of converting the written language sentence contained in the written language text to be processed for the conversion processing of the clause unit, in this embodiment of the present application, the written language can be converted to the written language in the following two ways or a combination of the following two ways. Statements are converted, including:

Method 1: Sampling the written sentences according to the preset clause sampling rules to obtain the target clauses in the written sentences; converting the target clauses in the written sentences to obtain the converted written sentences.

In practical applications, since a written language statement may contain multiple clauses, and these clauses may not all have the expression difference between written language and spoken language, therefore, you can first select the clauses that need to be converted into clause units in these written language sentences, The selected clauses are then converted.

The preset clause sampling rule refers to a preset sampling rule for sampling clauses in written sentences. The preset clause sampling rules may be random sampling or sampling based on positions, for example, the sampling position is in written sentences. In addition, sampling can be performed according to the number of characters, such as a clause with a number of characters less than 5 in the sampling clause, which is not limited here. Correspondingly, the target clause refers to a clause obtained by sampling written language sentences through a preset clause sampling rule.

On the basis of obtaining the target clause, the target clause can be converted in the written language sentence. During the specific implementation, since there are various ways to convert the selected target clause, in order to increase the conversion The naturalness and richness of the following written language sentences can be transformed into the target clause by the following three transformation methods or any combination of the following three transformation methods, including:

Mode A: Duplicate the target clause to obtain the duplicate target clause, and insert the duplicate target clause into the written language statement according to the preset clause insertion position to obtain the converted written language statement.

In practical applications, due to oral expressions, sometimes there are some colloquial expressions that are not found in written sentences, such as: right, right, good, and so on. In order to make written language more in line with the characteristics of spoken language, some colloquial clauses can be added to written language sentences.

Specifically, the preset clause insertion position refers to the preset position where the target clause is inserted into the written language sentence, and the position can be set according to the actual spoken language characteristics. For example, the preset clause insertion position can be the sentence of the written language sentence. The beginning or the end of the sentence can also be before or after the position of the target clause in the written sentence.

Following the above example, suppose that the written text LT is used as the written text to be processed, and the written text LT is sentence-recognized to obtain n written sentences contained in the written text LT. These n written sentences are the written sentence S1 and the written sentence S2 respectively. ...Written language sentence Sn. Taking the written language sentence S1 as an example to illustrate, randomly sample the clauses of the written language sentence S1 "My hometown is Shanxi, where it is beautiful", and obtain the target clause in the written language sentence S1 as "My hometown is Shanxi". Copy the target clause in the written language sentence S1 to obtain the copy target clause "My hometown is Shanxi". If the insertion position of the preset clause is before the position of the target clause, the target clause will be copied "My hometown is Shanxi" is inserted into the written language sentence S1, and the converted written language sentence S13 is obtained as: "My hometown is Shanxi, my hometown is Shanxi, and it is beautiful there".

Mode B: delete the target clause in the written language statement; insert the target clause into the deleted written language statement according to the preset clause insertion rule, and obtain the converted written language statement.

In practical applications, because sometimes the order of expression of clauses is not concerned when expressing in spoken language, there may be situations in which the order of expression of clauses in spoken sentences is inconsistent with that of written sentences. In order to make the converted written language more in line with the characteristics of spoken language, some clauses of written language sentences can be adjusted in position.

Specifically, the preset clause insertion rule refers to a preset rule for inserting the target clause, and the rule can be set according to actual experience. For example, the preset clause insertion rule can be random insertion (that is, the preset clause Randomly inserted before or after any clause in a written statement), either after the first clause, at the end of the sentence, etc.

It should be noted that, since the conversion processing of method A and the conversion processing of method B can be selectively performed, the conversion processing of method B may be performed on the converted written language sentence obtained by method A, or the conversion processing of method B may be directly performed on the original sentence. The conversion processing of method B is performed on the written language sentence, and the conversion processing of method A can be performed on the converted written language sentence obtained by method B. In addition, other conversion processing can also be selectively executed and/or executed sequentially.

Following the above example, let’s take the written language sentence S1 as an example to illustrate, randomly sample the clauses of the written language sentence S1 “My hometown is Shanxi, it is beautiful there”, and obtain the target clause in the written language sentence S1 as “My hometown is Shanxi". The target clause is deleted in the written language sentence S1, and when the preset clause insertion rule is random insertion, the target clause "My hometown is Shanxi" is randomly inserted after any clause of the written language sentence S1, The converted written language sentence S13 is: "My hometown is Shanxi, it is beautiful there, my hometown is Shanxi".

Mode C: perform syntactic analysis on the target clause to obtain the syntactic structure corresponding to the target clause; and obtain the converted written language statement by converting the target clause according to the target syntactic structure corresponding to the syntactic structure.

In practical applications, although the grammatical structure (syntax structure) of the clauses is inconsistent, the meanings expressed are still the same. Therefore, the word order in the clauses may be inconsistent with the grammatical structure in the written sentences in spoken sentences. Therefore, in order to make the converted written language more in line with the characteristics of spoken language, the grammatical structure of some clauses of the written language sentence can be changed, such as: inversion processing.

Specifically, to syntactically analyze the sampled target clauses, a rule-based syntax analysis method or a statistics-based syntax analysis method can be used to obtain a syntax structure corresponding to the target clause, and the syntax structure can be a subject-predicate-object structure or an object-predicate structure. The structure of the master, etc., is not limited here. Correspondingly, the target syntactic structure refers to a preset syntactic structure corresponding to the syntactic structure of the target clause. During specific implementation, conversion can be performed between the syntactic structure and the target syntactic structure. For example, the syntactic structure is an active syntactic structure of subject-predicate-object, and its target syntactic structure can be a passive syntactic structure of object-predicate-subject.

Following the above example, we still randomly sample the clauses of the written language sentence S1 "My hometown is Shanxi, where it is beautiful", and obtain the target clause in the written language sentence S1 as "My hometown is Shanxi" as an example to illustrate. The syntactic structure of the target clause is a subject-predicate-object structure, and the target syntactic structure corresponding to the syntactic structure is an object-predicate-subject structure. Then the target clause is converted into an object-predicate subject structure, and the converted target clause becomes: "Shanxi is my hometown". Correspondingly, the converted written language sentence S13 is: "Shanxi is my hometown, it is beautiful there".

Method 2: Determine the clause position probability distribution corresponding to the preset clause contained in the preset clause set; determine the target preset clause and the corresponding target preset clause in the preset clause based on the clause position probability distribution. Clause adding position; the target preset clause is added to the written language sentence according to the clause adding position, and the converted written language sentence is obtained.

The preset clause set refers to a preset set containing at least one colloquial clause. Correspondingly, a preset clause refers to a clause contained in a set of preset clauses. Clause position probability distribution refers to the position probability of each preset clause obtained by pre-stating the occurrence positions of preset clauses in a certain spoken language corpus (such as the beginning of a sentence, the end of a sentence, or the position in a sentence). distributed. In practical applications, the frequency of occurrence of each preset clause at each position can be counted, and then the position probability distribution can be calculated according to the counted frequency.

It is assumed that the preset clause set includes three preset clauses, and the three preset clauses are preset clause 1, preset clause 2, and preset clause 3 respectively. According to the statistics of the spoken language corpus in the sales field, the preset clause 1 appears 60 times at the beginning of the sentence, the preset clause 2 appears at the end of the sentence 20 times, and the preset clause 3 appears at the beginning of the sentence 20 times. Then the probability of adding preset clause 1 to the beginning of the sentence is: 60/(60+20+20)=60%, and the probability of adding preset clause 2 to the end of the sentence is 20/(60+20+20)=20 %, the probability that the preset clause 3 is added to the beginning of the sentence is also 20/(60+20+20)=20%. The above three probabilities are the clause position probability distributions corresponding to the preset clauses.

Further, based on the probability distribution of the clause positions, the target preset clause can be determined in the preset clause, and the clause addition position corresponding to the target preset clause (the position where the target preset clause is added in the written language sentence) can be determined. ). During specific implementation, it is also possible to preset a value range, the value range is 1-100 (or 1-10, etc.), and within this value range, set the value range in which the value probability is the same as the clause position probability distribution, such as 1-60, 61-80 and 81-100, and then randomly generate any value in the range of 1-100. If the generated value is 9, and the value is between 1 and 60, it means that the value satisfies the 60% probability of taking the value, that is, the probability of adding the preset clause 1 to the beginning of the sentence is met, then determine the target preset The sentence is the pre-set clause 1 and the clauses corresponding to the target pre-set clause are added at the beginning of the sentence.

Still further, in the case where the target preset clause is "Duo Duo Duo", the target pre-set clause "Duo Duo Duo" is added to the sentence beginning of the written language sentence S1, and the converted written language sentence S13 is obtained as: "Yes, yes, my hometown is Shanxi, which is very beautiful."

Step 30806-6: Determine the spoken text based on the converted written sentence.

When there are a plurality of converted written language sentences, the converted written language sentences may be combined according to the arrangement order of the original written language sentences in the written language text to generate a spoken language text.

Following the above example, perform at least one conversion process on at least one written language sentence among the n written language sentences included in the written language text LT, and obtain n converted written language sentences as written language sentence S13, written language sentence S23, ..., written language sentence respectively. The sentence Sn3 combines the n converted written sentences to generate the spoken text ST1.

To sum up, by performing the conversion processing of copying clauses of clause units, out-of-order clauses and/or adding clauses to written sentences, the oral rewriting of written sentences is realized, and the converted written sentences are more in line with the characteristics of spoken language.

In addition, in the case that the sentence composition unit is a word unit, there are various conversion methods for converting the written language text to be processed for the clause unit. The first implementation provided by the embodiment of the present application is implemented in the following manner. :

Perform sentence recognition on the written language text to be processed, and obtain written language sentences contained in the written language text to be processed;

Determine the word position probability distribution corresponding to the preset words included in the preset word set;

Determine the target preset word and the word addition position corresponding to the target preset word in the preset words according to the word position probability distribution, and insert the target preset word into the written language sentence according to the word addition position to obtain the converted written language sentence;

The spoken text is determined based on the transformed written sentences.

In practical applications, some colloquial words will be randomly added in the oral expression. These colloquial words may include: conjunctions, modal particles or other colloquial words, such as: wow, actually, etc. These colloquial words Usually not present in written language sentences. In order to make the written language more in line with the characteristics of spoken language, some oral words can be added to the written language sentences.

Specifically, the preset word set refers to a preset set containing at least one colloquial word. Correspondingly, the preset words refer to the words contained in the preset word set. The word position probability distribution refers to the position probability distribution of each preset word obtained by pre-stating the occurrence positions of preset words (such as sentence beginning, sentence end, or sentence middle position) in a certain spoken language corpus. In practical applications, the frequency of occurrence of each preset word at each position can be counted, and then the position probability distribution can be calculated according to the counted frequency.

It is assumed that the preset word set includes two preset words, and the two preset words are preset word 1 and preset word 2 respectively. According to the statistics of the oral corpus in the sales field, the preset word 1 appears 80 times at the beginning of the sentence, and the preset word 2 appears 20 times at the end of the sentence, then the probability that the preset word 1 is added to the beginning of the sentence is: 80/ (80+20)=80%, the probability that the preset word 2 is added to the end of the sentence is 20%, and the above two probabilities are the word position probability distributions corresponding to the preset words.

Specifically, for the specific implementation of determining the target preset word and the word addition position corresponding to the target preset word (the position where the target preset word is added in the written sentence) in the preset words according to the word position probability distribution, refer to the above clause-based The position probability distribution can be implemented by determining the target preset clause and the specific implementation of the clause addition position corresponding to the target preset clause in the preset clause, and details are not described here.

On the basis of determining the target preset word and the word addition position corresponding to the target preset word, the target preset word can be added to the word addition position in the written language sentence, and the converted written language sentence is obtained, and further, based on the converted written language sentence For the specific implementation of determining the spoken text by the written language sentence, refer to the above-mentioned conversion processing part in the clause unit, and the specific implementation of the spoken text is determined based on the converted written language sentence, which is not limited here.

Following the above example, on the basis of the n written language sentences included in the written language text LT, taking the written language sentence S1 as an example for illustration, the preset word set contains two preset words, and the two preset words are respectively. It is preset word 1 and preset word 2, and the word position probability distribution of these two preset words is: the probability that preset word 1 is added to the beginning of a sentence is 80%, and the probability that preset word 2 is added to the beginning of a sentence is 20% %. It is assumed that the target preset word is determined as preset word 1 in the preset word set according to the word position probability distribution and the word addition position corresponding to the target preset word is the beginning of the sentence. When the preset word 1 is "actually", the preset word 1 is added to the sentence beginning of the written language sentence S1, and the converted written language sentence S13 is obtained as: "Actually, my hometown is Shanxi, which is very beautiful". Then the n converted written sentences are combined to generate spoken text ST1.

To sum up, by adding the words of word units to the written sentences, the oral rewriting of the written sentences is realized, so that the converted written sentences are more in line with the characteristics of spoken language.

During the specific implementation, after adding some colloquial words, the added words may be habitually repeated. Therefore, in order to make the converted written language more in line with the characteristics of spoken language, the words added in the written language sentence can be changed. Carry out the replication process, the embodiment of the present application is specifically implemented in the following ways:

Copy the target preset words added in the converted written language sentence to obtain the copied words;

Insert the copied word into the converted written language statement according to the preset word insertion rule to obtain the inserted written language statement;

The spoken text is determined based on the inserted written language sentence.

Specifically, the preset word insertion rule refers to a preset rule for inserting target preset words into written sentences, and the rule can be set according to the characteristics of actual spoken language. For example, the preset word insertion rule can be the target preset word insertion rule The positions before or after the target preset words are inserted may also be other positions in which the target preset words are inserted in the written language sentence, which is not limited herein.

Following the above example, on the basis of obtaining the converted written language sentence S13 as: "Actually, my hometown is Shanxi, and it is beautiful there", copy the target preset word "actually" to obtain the duplicated word "actually", and in the preset Assuming that the word insertion rule is to be inserted before the target preset word, add the copied word "actually" to the converted written language sentence S13 to obtain the inserted written language sentence S14, and the written language sentence S14 is "actually actually I am. His hometown is Shanxi, which is very beautiful." Then, the n inserted written sentences are combined to generate spoken text ST1.

To sum up, in the process of adding words of word units to written sentences, the added words are repeated, so that the converted written sentences are more in line with the characteristics of spoken language.

In the case that the sentence composition unit is a word unit, in addition to the conversion processing of the word unit, the second implementation provided by the embodiment of the present application is specifically implemented by the following processing methods:

Perform sentence recognition on the written text to be processed, and obtain written sentences contained in the written text;

According to the preset word sampling rules, the words in the written language sentence are sampled to obtain the target word in the written language sentence;

Delete the target word in the written language sentence, insert the target word into the preset insertion range corresponding to the target word in the deleted written language sentence, and obtain the converted written language sentence;

The spoken text is determined based on the transformed written sentences.

Since the order of expression of words is sometimes ignored in spoken language, the order of expression of words in spoken sentences may be inconsistent with the order of expression in written sentences. In order to make the converted written language more in line with the characteristics of spoken language, some words in the written language sentence can be adjusted in position.

Specifically, the preset word sampling rule refers to a preset sampling rule for sampling words to be out of order in written sentences. The preset word sampling rule may be random sampling or sampling according to a preset number of characters , such as randomly sampling words with three characters in written sentences, etc. Correspondingly, the target word refers to the word sampled in the written language sentence by the word sampling rule.

The preset insertion range refers to a preset range for performing insertion processing. The preset insertion range can be preset according to actual experience or oral expression habits. Specifically, the preset insertion range corresponding to the target word can be 3 characters before the position of the target word in the written language sentence to the position of the target word at the position of the written language sentence For the character interval of the next three characters, the character interval may be referred to as [-3, 3] for short. In addition, the preset insertion range may also be within the scope of the clause to which the word belongs. Further, the target word can be randomly inserted within the preset insertion range.

Following the above example, on the basis of the n written language sentences contained in the written language text LT, taking the written language sentence S1 as an example to illustrate, randomly sample words in the written language sentence S1 to obtain the target word "there", and in the written language sentence The target word is deleted in S1, and the deleted written language sentence S1 is "My hometown is Shanxi, it is very beautiful". And when the preset insertion range is "the scope of the clause to which the target word belongs", insert the target word into the preset insertion range of the written language sentence S1, and the converted written language sentence S13 is obtained as: "My hometown is Shanxi is very beautiful there." Then the n converted written sentences are combined to generate spoken text ST1.

To sum up, through the conversion of word units to written sentences, the converted written sentences are more in line with the characteristics of spoken language.

In the case where the unit of sentence is a character unit, considering that sometimes the expression order of characters is not strictly followed in the process of spoken language expression, the expression order of characters in spoken language sentences may be inconsistent with the expression order of characters in written language sentences. In order to make the converted written language more in line with the characteristics of spoken language, some characters of the written language sentence can be adjusted in position.

Perform sentence recognition on the written language text to be processed, and obtain written language sentences contained in the written language text to be processed;

According to the preset character sampling rule, character sampling is performed on the characters in the written language sentence to obtain the target character in the written language sentence;

Delete the target character in the written language sentence, and insert the target character into the preset character insertion range corresponding to the target character in the deleted written language sentence to obtain the converted written language sentence;

The spoken text is determined based on the transformed written sentences.

Specifically, the preset character sampling rule refers to a preset sampling rule for sampling characters to be out of order in written sentences. The preset character sampling rule may be random sampling or sampling according to preset character positions , such as random sampling of characters whose position is the 5th position in the written language sentence, etc., which is not limited here.

Correspondingly, the preset character insertion range corresponding to the target character may be a character interval from 3 characters before the position of the target character at the position of the written language sentence to 3 characters after the position of the target word at the position of the written language sentence, and the character interval may be abbreviated as [- 3, 3], in addition, the preset character insertion range may also be within the scope of the clause where the target character is located, etc., which is not limited here.

Following the above example, on the basis of the n written language sentences contained in the written language text LT, the written language sentence S1 is taken as an example for illustration, and the characters are randomly sampled in the written language sentence S1 to obtain the target character "美". The target character is deleted in S1, and the deleted written language sentence S1 is "My hometown is Shanxi, where it is very". And when the preset character insertion range corresponding to the target character is "the scope of the clause to which the target character belongs", insert the target character "美" into the preset character insertion range of the written language sentence S1 to obtain the converted written language. The sentence S13 is: "My hometown is Shanxi, where it is very beautiful". Then the n converted written sentences are combined to generate spoken text ST1.

To sum up, by shuffling the character units of written sentences, the converted written sentences are more in line with the characteristics of spoken language.

In the case where the unit of the sentence is a symbol unit, since there may not be a clear division of the disconnection or cohesion of the sentence in the spoken language, or the division may be arbitrary, the symbols appearing in the spoken language sentence may be inconsistent with the symbols appearing in the written language sentence. Case. In order to make the converted written language more in line with the characteristics of spoken language, the symbol unit conversion processing of the written language text to be processed can be carried out in the following two ways or a combination of the following two ways, including:

Conversion method 1: Perform sentence recognition on the written language text to be processed, and obtain the written language sentences contained in the written language text to be processed; perform symbol sampling on the written language sentences according to the preset symbol sampling rules, obtain the target punctuation marks in the written language sentences, and delete them in the written language sentences The target punctuation marks, the converted written sentences are obtained; the spoken text is determined based on the converted written sentences.

Specifically, the preset symbol sampling rule refers to a preset sampling rule for sampling symbols to be deleted in written language sentences. The preset symbol sampling rule may be random sampling or sampling according to a preset position, such as sampling the punctuation marks after the first clause in a written language sentence, which is not limited here. Correspondingly, the target punctuation marks refer to punctuation marks sampled from written language sentences according to the preset symbol sampling rule.

Following the above example, on the basis of the n written language sentences contained in the written language text LT, taking the written language sentence S1 as an example to illustrate, randomly sample the symbols of the written language sentence S1, and obtain the target punctuation mark in the written language sentence S1 as sub A comma after the "My hometown is Shanxi" clause. The target punctuation mark is deleted in the written language sentence S1, and the converted written language sentence S13 is obtained as: "My hometown is Shanxi where it is beautiful". Then the n converted written sentences are combined to generate spoken text ST1.

Conversion method 2: Perform sentence recognition on the written language text to be processed, and obtain the written language sentences contained in the written language text to be processed; perform symbol clause sampling on the written language sentence according to the preset symbol clause sampling rule, and obtain the target symbol clause in the written language sentence, Insert a preset punctuation mark in the target symbol clause to obtain the converted written language sentence; determine the spoken text based on the converted written language sentence.

Specifically, the preset symbol clause sampling rule refers to a preset sampling rule for sampling clauses with added symbols in written language sentences. The preset symbol clause sampling rule can be random sampling, or sampling according to the number of characters in the clause, such as sampling the clause with the largest number of characters in the written language sentence, which is not limited here. Correspondingly, the preset punctuation mark refers to the preset punctuation mark used for insertion. In practical applications, the preset punctuation mark can be randomly inserted into the target symbol clause, or the preset punctuation mark can be inserted according to a preset position. The punctuation marks are set, which is not limited here; the target symbol clause refers to the clause sampled in the written language sentence according to the preset symbol clause sampling rule.

Following the above example, on the basis of the n written language sentences contained in the written language text LT, taking the written language sentence S1 as an example to illustrate, in the case where the preset symbol clause sampling rule is the longest clause for sampling characters, Sentence S1 performs symbolic clause sampling, and the target symbolic clause in written language sentence S1 is obtained as "My hometown is Shanxi". When the preset punctuation mark is "!", the preset punctuation mark is inserted into the written language sentence S1, and the converted written language sentence S13 is obtained as: "My hometown is Shanxi, it is beautiful there". Then the n converted written sentences are combined to generate spoken text ST1.

To sum up, the written language sentence is rewritten in oral language by deleting symbols and adding symbols in the written language sentence, so that the converted written language sentence is more in line with the characteristics of spoken language.

Step 30808: Construct a sample corpus based on the written text and the correspondence between the back-translated written text and the spoken text.

Specifically, on the basis of obtaining the spoken text above, since the obtained spoken text is obtained by converting written text or back-translated written text, there is a correspondence between the spoken text and the written text or the back-translated written text. Based on this correspondence, a sample corpus of written-spoken text alignment can be generated.

Among them, the sample corpus refers to the pair of training samples used for model training. In practical applications, by generating a training sample pair of written text and spoken text, it can be used to train a written language rewriting model from spoken text to written text. In the case of training the written rewriting model, the spoken text in the sample corpus is used as a training sample, and the written text in the sample corpus is used as the sample label corresponding to the spoken text.

In practical applications, there may be some abnormal data in the spoken text obtained through the conversion process, and the existence of these abnormal data seriously affects the quality of the spoken text. Data cleaning, an embodiment of the present application, is specifically implemented in the following manner:

Identify abnormal information in spoken text;

Perform data cleaning on the spoken text according to the abnormal information to obtain the cleaned spoken text;

A sample corpus is constructed based on the correspondence between the written text and the back-translated written text and the cleaned spoken text.

The abnormal information may be typos, repeated punctuation, Chinese punctuation mixed with English punctuation, special symbols, stop words and other abnormal information. In addition, the abnormal information may also be semantically vague or unreasonable information, which is not limited here. In practical applications, abnormal information in spoken text can be identified by preset anomaly identification rules, or abnormal information in spoken text can be identified based on a pre-trained text cleaning model. During specific implementation, the text cleaning model may be a deep context model for grammar error correction to perform grammar detection.

Further, after identifying the abnormal information in the spoken text, in the case of multiple oral texts, the spoken text with abnormal information can be deleted directly, so as to obtain the oral text without abnormal information (that is, the cleaned spoken text). text). In addition, the abnormal information in the spoken text can also be deleted or corrected, so as to obtain the cleaned spoken text, which is not limited here. It should be noted that, if any spoken text is deleted, its corresponding written text or back-translated written text also needs to be deleted.

During specific implementation, considering that the written text that may be used also contains abnormal information, data cleaning can be performed on both the written text and the spoken text.

Following the above example, after converting the written text LT as the written text to be processed, the spoken text ST1 is obtained, and the back-translated written text LT3 is used as the text to be processed for conversion processing. On the basis of obtaining the spoken text ST2, the exception identification rules , recognize that the abnormal information in the spoken text ST1 is ",!", but the spoken text ST2 has no abnormality, then perform data cleaning on the spoken text ST1 according to the abnormal information, obtain the cleaned spoken text ST1, and directly use the spoken text ST2. As the cleaned spoken text ST2. Based on the correspondence between the written text LT and the cleaned spoken text ST1, a sample corpus pair 1 is constructed by combining the written text LT and the cleaned spoken text ST1. And based on the correspondence between the back-translated written text LT3 and the cleaned spoken text ST2, a sample corpus pair 2 is constructed from the written text LT3 and the cleaned spoken text ST2, and the sample corpus pair 1 and the sample corpus pair 2 are used as sample corpus. .

To sum up, by performing data cleaning on the converted spoken text, and constructing a sample corpus from the cleaned spoken text, the quality of the sample corpus is guaranteed, and the accuracy of model training is further increased.

Referring to FIG. 4 , FIG. 4 shows a schematic diagram of constructing a sample corpus in a text processing method provided by an embodiment of the present application. After the written text is obtained, in order to further expand the written text, the written text can be back-translated. Entity words are back-paired (replaced) to ensure that the obtained back-translation corpus is consistent with the key information in the written text. Then, the back-translation corpus and the written text are jointly used as data sources and input into the oral data generation module for oral conversion. The colloquialized data generation module includes converting the written language sentences in the data source at clause level, word level, character level and symbol level.

Among them, the conversion processing at the clause level includes the conversion processing of clause repetition, clause generation, and clause disorder at the clause level for the written language sentence; the conversion processing at the word level includes adding words and words to the written language sentence at the word level. Conversion processing such as repetition and word disorder; character-level conversion processing includes conversion processing such as character-level disorder processing for written sentences at the character level, and symbol-level conversion processing includes conversion processing such as symbol deletion and symbol insertion for written language sentences at the symbol level deal with.

After the oral language conversion of the data source is carried out through the oral data generation module, the initial oral text can be obtained, and the data of the initial oral text can be cleaned to remove abnormal information (that is, information containing wrong information, wrong data or wrong punctuation), The spoken text corresponding to the data source can be output.

The embodiment of the present application summarizes the text structure and syntactic-grammatical structure characteristics of the spoken text through research and analysis, performs back-translation processing based on the standard written text, and then performs conversion processing on the expanded written text after expanding the written text to generate a corresponding written text. Spoken expressions are used to expand the sample corpus of the written language rewriting model, which improves the efficiency and richness of obtaining the sample corpus of written language rewriting.

Step 30810: Train the initial written language rewriting model through the sample corpus, until a written language rewriting model that satisfies the second training stop condition is obtained.

Specifically, on the basis of the above construction of the sample corpus, the initial written language rewriting model can be trained through the sample corpus, and after the training is completed, a written language rewriting model that can be rewritten in written language can be generated.

The initial written language rewriting model may be a written language rewriting model to be trained constructed based on the Seq2Seq model, wherein both the encoder and the decoder in the Seq2Seq model may be constructed using the Transformer model. Correspondingly, the second training stop condition refers to a condition for stopping the model training of the initial written language rewriting model based on the sample corpus. The second training stop condition may be that the loss value between the predicted written text and the sample written text generated by rewriting the spoken text in the sample corpus through the model is smaller than the preset loss value, or the number of training iterations reaches the preset iteration The number of times, such as 5 times or 6 times, is not limited here. Correspondingly, the written language rewriting model can be understood as a model that has been trained to perform written language rewriting on spoken text.

During the specific implementation, the initial written language rewriting model is used to rewrite the input sample spoken text in written language and output the predicted written language text, and calculate the loss value of the predicted written language text and the sample written language text. In practical applications, the loss function of the loss value of the model is calculated. In practical applications, it can be a 0-1 loss function, an absolute value loss function, a squared loss function, a cross entropy loss function, etc. Here, the absolute value loss function is used as an example for explanation, see the following formula 2:

公式2

Formula 2

Among them, L represents the loss value, f(X) represents the predicted written text, and Y represents the sample written text. In this application, the selection of the loss function is not limited, and the actual application shall prevail.

After calculating the model loss value, you can reversely adjust the model parameters of the initial text classification model according to the model loss value, and sample the next batch of sample corpus to continue training the initial text classification model until the training stop condition is reached, and the training is complete. written language rewriting model.

During specific implementation, since the written language rewriting model uses abundant sample corpus for model training, the written language rewriting model can be used to deal with more complex sentence rewriting. Therefore, the written language rewriting model can be used for standard text type target spoken language The text undergoes relatively complex rewriting processing.

In addition, on the basis of obtaining the text type corresponding to the target spoken text, there is also a case where the text type is a fuzzy text type. In this case, in order to ensure the rationality and accuracy of the written language rewriting, the embodiments of the present application are specifically implemented in the following ways:

When the text type is fuzzy text type, select the corresponding written language conversion model according to the fuzzy text type;

Input the target spoken text into the written language conversion model for processing, and obtain the converted written text corresponding to the target spoken text;

The written language conversion model is obtained by training based on the written language text and the basic spoken language text obtained by converting the written language text.

Since the target spoken text is of fuzzy text type, it indicates that the textual semantic expression of the target spoken text is relatively vague. For the target spoken text of the fuzzy text type, it is necessary to input a written language conversion model that slightly rewrites the spoken text to rewrite the written language, so as to obtain the written language text corresponding to the target spoken language text (convert written language text). This is because the semantic expression of this type of target spoken text itself is relatively vague, and if complex rewriting is performed on it, its semantics may be more ambiguous or prone to deviation. Therefore, for the target spoken text of the fuzzy text type, the written language conversion model can be used to rewrite it with simple spoken words, mood words, etc.

Assuming that the text type corresponding to the target spoken text TST output by the text classification model is a fuzzy text type, input the target spoken text TST into the written language conversion model to obtain the target written language text TLT2 output by the written language conversion model.

To sum up, by slightly rewriting the target spoken text of the fuzzy text type through the written language conversion model, the rational rewriting of different types of oral text is realized, and the quality of written language rewriting is guaranteed.

During specific implementation, the training of the written language conversion model is specifically implemented through the following steps:

obtain written text;

Convert written text to sentence composition units to obtain basic spoken text;

Construct basic sample corpus based on the correspondence between written text and basic spoken text;

The initial written language conversion model is trained through the basic sample corpus until a written language conversion model that satisfies the first training stop condition is obtained.

Basic spoken text refers to the spoken text generated by converting the acquired written text. In practical applications, the specific implementation of acquiring written text, and performing sentence composition unit conversion processing on the written text, to obtain the basic spoken text, is the same as the above-mentioned acquisition of written language text, and the written language text and the back-translated written language text are respectively subjected to sentence composition unit conversion processing. , the specific implementation of obtaining the spoken text is similar, and the above-mentioned specific implementation may be referred to, which will not be repeated here.

Correspondingly, the basic sample corpus refers to a sample corpus constructed by using the spoken text as a training sample and the written text corresponding to the spoken text as a sample label. The first training stop condition refers to a condition for stopping the model training of the initial written language rewriting model based on the basic sample corpus. Similarly, the first training stop condition may be that the loss value between the predicted written language text generated by rewriting the spoken language text in the sample corpus through the model and the sample written language text is less than the preset loss value, or it may be that the number of training iterations reaches the preset loss value. Set the number of iterations, such as 5 times, or 6 times, etc., and there is no limit here. Correspondingly, the written language conversion model can be understood as a written language rewriting model based on the basic sample corpus training for the spoken text.

In the specific implementation, the initial written language conversion model is trained by using the basic sample corpus until the specific implementation of the written language conversion model that satisfies the first training stop condition is obtained. The specific implementation of the written language conversion model for the second training stop condition is similar, and the above-mentioned implementation can be referred to, which will not be repeated here.

It should be noted that the written language text in the basic sample corpus is not expanded by back-translation processing, and therefore, the basic sample corpus is relatively compact compared to the above constructed sample corpus. Therefore, the written language rewriting performed by the written language conversion model obtained by performing model training on the basic sample corpus is also simpler than the above-mentioned written language rewriting model.

To sum up, the written language conversion model is trained by the basic sample corpus, so that the written language conversion model can slightly rewrite the target spoken text of ambiguous text type, making the written language rewriting more reasonable.

In addition, there is a possibility that the text type is an invalid text type, and if the text type is an invalid text type, the target spoken text is deleted. Due to the target spoken text of an invalid text type, it indicates that the target spoken text is a spoken text that does not contain semantic information. Written rewriting of this type of spoken text will also result in no semantic information. Therefore, the target spoken text for invalid text types can be deleted directly, that is, without written rewriting.

To sum up, the target spoken text of an invalid text type is directly deleted, which avoids wasting computing resources to process the invalid spoken text. Thereby, the computational cost is saved.

In the text processing method provided by the embodiment of the present application, the target spoken text is obtained by obtaining the target spoken text; the target spoken text is classified and processed to obtain the text type corresponding to the target spoken text; and when the text type is a standard text type, selection is made according to the standard text type The corresponding written language rewriting model realizes the selection of a written language rewriting model suitable for the target spoken language text according to the text type of the target spoken language text; then the target spoken language text is input into the written language rewriting model for processing, and the target written language text corresponding to the target spoken language text is obtained, so that the written language can be rewritten. It is more targeted and the accuracy of written language rewriting has been improved. Among them, the written language rewriting model is based on the written language text and the spoken language text obtained by back-translation and conversion processing of the written language text, and realizes the preprocessing of the written language text based on the back-translation and conversion processing, thus providing a large amount of spoken language text for model training- The sample corpus of written text simplifies the training difficulty of the model, avoids the time-consuming and laborious collection and processing of a large amount of text data, and saves time and labor costs.

The text processing method is further described below by taking the application of the text processing method provided by the present application in an actual scene as an example with reference to FIG. 5 . 5 shows a processing flow chart of a text processing method applied to an actual scene provided by an embodiment of the present application, which specifically includes the following steps:

Step 502: Acquire written language text.

Specifically, the written text can be written text in any field, such as written text in the field of medicine, written text in the field of chemistry, written text in the field of sales, written text in the field of daily life, written text in the field of tourism, etc. make restrictions. And the number of the written text can be one or more, which is not limited here.

Taking the sales field as an example, obtain the written text T of the sales field.

Step 504 : by performing part-of-speech analysis on the written text, identify key words whose part of speech is a preset part of speech in the written text.

Perform part-of-speech analysis on each word contained in the written text T to obtain the part-of-speech of each word in the written text T. When the preset part of speech is a noun part of speech, the word of the noun part of speech in the written language text T is identified as a key word.

Step 506: Mark the positions of the key words in the written text.

Based on this, it is assumed that the key words identified in the written language text T are "computer" and "speed", and the written language sentence SS to which these key words belong in the written language text T is: "I use a computer, which is very fast and very convenient. ", the written text T is marked with an asterisk "*", and the marked written text T is obtained after the marking is completed. The written language sentence SS in the marked written language text T is changed to: "I use *computer*, *speed* is very fast, and it is very convenient".

Step 508: Translate the marked written text into the translated written text corresponding to the preset language.

Specifically, the preset language may be any one or more languages such as English, French, and Korean, which are not limited herein.

Based on this, when the preset language is English, the marked written text T is translated into English, and an English translated written text T1 corresponding to the marked written text T is obtained.

Step 510: Translate the written text of the translation into the target language to which the written text belongs to obtain the initial back-translated written text.

Specifically, since the target language to which the text content in the written text T belongs is Chinese, the English translated written text T1 is translated into Chinese, and the initial back-translated written text T2 corresponding to the English translated written text T1 is obtained, wherein the initial back-translated written text T2 is obtained. The written language sentence SS2 corresponding to the written language sentence SS in the text T2 is updated to: "I use *computer*, *efficiency* is very fast and very convenient".

Step 512: Replacing the target key words corresponding to the position marks in the initial back-translated written text with the key words to obtain the back-translated written text.

Specifically, the target keyword corresponding to the position marker refers to the word marked by the position marker in the initial back-translated written text, and the target keyword also corresponds to the keyword. In practical applications, combined with part-of-speech analysis of written texts, during the back-translation process, words with specific parts of speech in written sentences are marked and replaced, so as to ensure that the key information in the back-translated written text and the written text remains unchanged as much as possible.

Based on this, the target keywords corresponding to the marked positions in the initial back-translated written text T2 are "computer" and "efficiency"; the "computer" in the initial back-translated written text T2 is replaced by the keyword "computer", and The "efficiency" in the initial back-translated written text T2 is replaced by the keyword word "speed" to obtain a back-translated written text T3, wherein the written text SS3 corresponding to the written sentence SS in the back-translated written text T3 is updated to: " I use a computer to do the calculations, which is very fast and very convenient.”

Step 514: Take each written text in the written text and the back-translated written text as the written text to be processed, perform sentence recognition on each written text to be processed, and obtain written sentences contained in each written text to be processed.

Specifically, each written language text in the written language text T and the back-translated written language text T3 is regarded as the written text to be processed, and sentence recognition is performed on each of the to-be-processed texts to obtain written language sentences contained in each of the to-be-processed texts. Further, the following steps 516 to 522 are performed for each written sentence in each text to be processed.

Based on this, it is assumed that the written language text T is taken as the text to be processed, and the text to be processed T is sentence-recognized to obtain n written language sentences contained in the to-be-processed text T, and the n written language sentences are written language sentence 1, written language sentence 2, ..., written language sentence n, the following steps 516 to 522 are respectively executed for the n written language sentences.

Step 516 : Convert the written language sentence to the clause unit to obtain the converted A4-th written language sentence.

Specifically, the conversion processing of the clause unit is performed on any written language sentence, which is specifically implemented by executing the following steps 516-1 to 516-18 as follows:

Step 516-1: Determine the duplicate clause conversion processing probability corresponding to the duplicate clause conversion processing policy in each clause conversion processing policy of the written language sentence.

Wherein, the copy clause conversion processing strategy refers to a processing strategy for copying clauses in a written language sentence, and correspondingly, the copy clause conversion processing probability refers to a preset probability of executing the copy clause conversion processing strategy. The conversion processing probability of the copy clause can be preset according to actual experience or oral expression habits. For example, the conversion processing probability of the copy clause can be 10%, 20%, 30%, etc., which is not limited here. In the case where the probability of copy clause conversion processing is 10%, it indicates that there is a 10% probability to execute the copy clause conversion processing strategy for written language sentences.

Based on this, assuming that written language sentence 1 is the above written language sentence SS "I use a computer, it is very fast and very convenient", and the copy clause conversion processing probability corresponding to the copy clause conversion processing strategy is 10%. Statement 1 executes the replication clause transformation processing probability of the replication clause transformation processing strategy is 10%.

Step 516-2: Based on the copy clause conversion processing probability, determine whether to execute the copy clause conversion processing strategy for the written language sentence.

Specifically, if it is determined to execute the duplicated clause conversion processing strategy based on the duplicated clause conversion processing probability, the following step 516-3 is executed; if it is determined not to execute the duplicated clause newline processing strategy, the written language sentence is directly used as the A1 written language sentence , and execute the following step 516-5.

Step 516-3: In the case of determining to execute the duplicate clause conversion processing strategy, perform clause sampling on the written language sentence according to the first preset sampling rule to obtain the first target clause in the written language sentence.

Specifically, the first preset sampling rule refers to a preset sampling rule for sampling clauses to be copied in written language sentences. The first preset sampling rule may be random sampling, or sampling according to position, such as a clause whose sampling position is in the first position in a written language sentence, in addition, sampling may be carried out according to the number of characters, such as sampling sub-clauses Clauses with less than 5 characters in the sentence, etc. The first preset sampling rule may be the same as the preset clause sampling rule in the foregoing method embodiments, and may also be understood as one of the preset clause sampling rules in the foregoing method embodiments. Correspondingly, the first target clause refers to a clause sampled in a written language sentence according to the first preset sampling rule, and can also be understood as the target clause in the above method embodiments.

Based on this, when it is determined to execute the copy clause conversion processing strategy, the written language sentence 1 is randomly sampled, and the first target clause in the written language sentence 1 is obtained as "fast".

Step 516-4: Duplicate the first target clause to obtain the duplicate target clause, and insert the duplicate target clause into the written language sentence according to the preset clause insertion position to obtain the converted A1-th written language sentence.

Specifically, the preset clause insertion position refers to a preset position where the target first clause is inserted into a written sentence, and the position can be set according to the characteristics of actual spoken language. For example, the preset clause insertion position can be a written sentence. The beginning or end of the sentence can also be before or after the position of the first target clause in the written sentence, which is not limited here.

Based on this, the first target clause is "very fast", and the obtained replication target clause is also "very fast", and the insertion position of the preset clause is before the position of the first target clause. In this case, the copy target clause is inserted before the position of the first target clause in written language sentence 1, and the transformed written language sentence A1 is obtained as: "I use a computer, which is very fast, very fast, and very fast. convenient".

Step 516-5: Determine the added clause conversion processing probability corresponding to the added clause conversion processing policy in each clause conversion processing policy of the A1-th written language sentence.

The addition clause conversion processing strategy refers to a processing strategy for adding clauses to written language sentences. Correspondingly, the added clause conversion processing probability refers to the preset probability of executing the related processing of the added clause conversion processing strategy. The added clause conversion processing probability can also be preset according to actual experience or oral expression habits, such as adding The clause conversion processing probability can be 15%, 20%, etc., which is not limited here. In the case of adding clause conversion processing probability of 15%, it indicates that there is a 15% probability to execute the addition clause conversion processing strategy for written sentences.

Based on this, it is determined that the addition clause conversion processing probability corresponding to the addition clause conversion processing strategy of the A1th written language sentence is 15%.

Step 516-6: Based on the added clause conversion processing probability, determine whether to execute the addition clause conversion processing strategy for the A1 th written language sentence.

Specifically, based on the added clause conversion processing probability, it is determined whether to execute the added clause conversion processing strategy for the A1 written language sentence. The specific implementation of the strategy is similar. Refer to the above-mentioned conversion processing probability based on the duplicated clause to determine whether to execute the specific implementation of the duplicated clause conversion processing strategy for the written language sentence, which will not be repeated here.

In the specific implementation, if it is determined to execute the addition clause conversion processing strategy, the following step 516-7 is executed; if it is determined not to execute the addition clause conversion processing strategy, the A1th written language statement is directly used as the A2th written language statement, and the following step 516-7 is executed. Step 516-9.

Based on this, it is determined that the addition-clause conversion processing strategy is executed for the A1-th written language sentence, assuming that the addition-clause conversion processing probability is 15%.

Step 516-7: In the case where it is determined to execute the adding clause conversion processing strategy, determine the probability distribution of the clause positions corresponding to the preset clauses included in the preset clause set.

Specifically, the preset clause set includes three preset clauses, and the three preset clauses are preset clause 1, preset clause 2, and preset clause 3 respectively. The probability distribution of the clause positions corresponding to the three preset clauses is as follows: the probability of adding preset clause 1 to the beginning of the sentence is 60%, the probability of adding preset clause 2 to the end of the sentence is 20%, and the probability of adding preset clause 2 to the end of the sentence is 20%. The probability of 3 being added to the beginning of a sentence is 20%.

Step 516-8: Determine the target preset clause and the clause addition position corresponding to the target preset clause in the preset clause based on the clause position probability distribution, and add the target preset clause to the target preset clause according to the clause addition position. In the A1 written language sentence, the converted A2 written language sentence is obtained.

Specifically, based on the probability distribution of the clause positions, it is determined in the preset clause that the target preset clause is preset clause 1 and the addition position of the clause corresponding to preset clause 1 is the beginning of the sentence, then the preset clause 1 is added to the beginning of the sentence in the A1 written language sentence, and in the case where the preset clause 1 is "Duo Duo Du", the converted A2 written language sentence is obtained as "Yes Du Du, I use a computer, the speed is very fast, the speed It's fast and very convenient."

Step 516-9: Determine the out-of-order clause conversion processing probability corresponding to the out-of-order clause conversion processing policy in each clause conversion processing policy of the A2-th written language sentence.

The out-of-order clause conversion processing strategy refers to a processing strategy for out-of-order clauses in a written language sentence. Correspondingly, the out-of-order clause conversion processing probability refers to the preset probability of executing the relevant processing of the out-of-order clause conversion processing strategy. The out-of-order clause conversion processing probability can also be preset according to actual experience or oral expression habits. , for example, the out-of-order clause conversion processing probability can be 5%, 10%, etc., which is not limited here. In the case that the out-of-order clause conversion processing probability is 5%, it indicates that the out-of-order clause conversion processing strategy has a 5% probability for written sentences.

Based on this, it is determined that the out-of-order clause conversion processing probability corresponding to the out-of-order clause conversion processing strategy of the A2-th written language sentence is 5%.

Step 516-10: Based on the out-of-order clause conversion processing probability, determine whether to execute the out-of-order clause conversion processing strategy for the A2-th written language sentence.

Specifically, based on the out-of-order clause conversion processing probability, it is determined whether to execute the out-of-order clause conversion processing strategy for the A2 written language sentence. The specific implementation of the conversion processing strategy is similar. Referring to the above-mentioned conversion processing probability based on the copy clause, it is sufficient to determine whether to execute the specific implementation of the copy clause conversion processing strategy for the written language sentence, which will not be repeated here.

During specific implementation, if it is determined to execute the out-of-order clause conversion processing strategy, the following step 516-11 is executed; if it is determined not to execute the out-of-order clause conversion processing strategy, the A2 written language statement is directly used as the A3 written language statement, and the execution is executed. Steps 516-13 are described below.

Based on this, it is assumed that the out-of-order clause conversion processing strategy is determined for the A2-th written language sentence based on the assumption that the added clause conversion processing probability is 5%.

Step 516-11: If it is determined to execute the out-of-order clause conversion processing strategy, perform clause sampling on the A2-th written language sentence according to the second preset sampling rule to obtain the second target clause in the A2-th written language sentence.

Specifically, the second preset sampling rule refers to a preset sampling rule for sampling clauses to be out of order in written sentences, and the second preset sampling rule may be random sampling or sampling according to location , such as a clause whose sampling position is at the end of a written sentence. In addition, sampling can be performed according to the number of characters, such as a clause with less than 5 characters in the sampling clause, which is not limited here. In practical applications, the second preset sampling rule may be the same as the above-mentioned first preset sampling rule, or may be different from the above-mentioned first preset sampling rule, which is not limited herein. The second preset sampling rule may also be the same as the preset clause sampling rule in the foregoing method embodiments, or may be understood as one of the preset clause sampling rules in the foregoing method embodiments. Correspondingly, the second target clause refers to a clause sampled in a written language sentence according to the second preset sampling rule, and can also be understood as the target clause in the above method embodiments.

Based on this, when it is determined to execute the out-of-order clause conversion processing strategy, the clause sampling is performed on the A2-th written language sentence at random, and the second target clause in the A2-th written language sentence is obtained as "pair-to-pair".

Step 516-12: Delete the second target clause in the A2-th written language sentence, and insert the second target clause into the deleted A2-th written language sentence according to the preset clause insertion rule to obtain the converted A3-th written language sentence.

Specifically, delete the second target clause "Dai Du Du" in the A2 written language sentence, and obtain the deleted A2 written language sentence: "I use a computer, which is very fast, very fast, and very convenient". And the second target clause "Duo Duo" is randomly inserted into the deleted A2 written language sentence, and the converted A3 written language sentence is obtained as: "I use a computer, which is very fast, very fast, and very convenient. Yes, that's right."

Step 516-13: Determine the inversion clause conversion processing probability corresponding to the inversion clause conversion processing strategy in each clause conversion processing strategy of the A3-th written language sentence.

Among them, the inversion clause conversion processing strategy refers to a processing strategy for inverting the word order of clauses in a written sentence (for example, from a subject-predicate-object structure to an object-predicate-subject structure). In practical applications, although the word order of the clauses is inconsistent, the meanings expressed are still the same. Therefore, the word order in the clause may be inconsistent with the word order in the written sentence in a spoken sentence. In order to make the converted written language more in line with the characteristics of spoken language, the word order of some clauses of the written language sentence can be reversed. Correspondingly, the inverted clause conversion processing probability refers to the preset probability of executing the relevant processing of the inverted clause conversion processing strategy. The inverted clause conversion processing probability can also be preset according to actual experience or oral expression habits. , for example, the conversion processing probability of the inverted clause can be 3%, 5%, etc., which is not limited here. When the probability of inversion clause conversion processing is 3%, it indicates that there is a 3% probability of executing the inverted clause conversion processing strategy for written language sentences.

Based on this, it is determined that the inversion clause conversion processing probability corresponding to the inversion clause conversion processing strategy of the A3 written language sentence is 3%.

Step 516-14: Based on the inversion clause conversion processing probability, determine whether to execute the inversion clause conversion processing strategy for the A3-th written language sentence.

Specifically, based on the inversion clause conversion processing probability, determining whether to execute the inversion clause conversion processing strategy for the written language sentence A3 is the same as the above-mentioned conversion processing probability based on the inversion clause. The specific implementation of the clause conversion processing strategy is similar. Refer to the above-mentioned specific implementation of the inverted clause conversion processing strategy based on the inverted clause conversion processing probability to determine whether to execute the inverted clause conversion processing strategy for the written language sentence, which is not repeated here.

During specific implementation, if it is determined to execute the inversion clause conversion processing strategy, the following steps 516-15 are executed; if it is determined not to execute the inverted clause conversion processing strategy, the A3 written language statement is directly used as the A4 written language statement, and the execution Step 518 is described below.

Based on this, it is assumed that the out-of-order clause conversion processing strategy is determined for the A2-th written language sentence based on the assumption that the added clause conversion processing probability is 5%.

Step 516-15: In the case of determining to execute the inverted clause conversion processing strategy, perform clause sampling on the A3-th written language sentence according to the third preset sampling rule to obtain the third target clause in the A3-th written language sentence.

Specifically, the third preset sampling rule refers to a preset sampling rule for sampling clauses to be inverted in written sentences, and the third preset sampling rule may be random sampling or sampling based on location , such as a clause whose sampling position is at the beginning of a sentence in a written sentence. In addition, sampling can be performed according to the number of characters, such as a clause with more than 5 characters in the sampling clause, which is not limited here. In practical applications, the third preset sampling rule may be the same as the first preset sampling rule or the second preset sampling rule, or different from the first preset sampling rule or the second preset sampling rule. This does not limit. The third preset sampling rule may also be the same as the preset clause sampling rule in the foregoing method embodiments, or may be understood as one of the preset clause sampling rules in the foregoing method embodiments. Correspondingly, the third target clause refers to a clause sampled in a written language sentence according to the third preset sampling rule, and can also be understood as the target clause in the above method embodiment.

Based on this, in the case of determining the implementation of the inverted clause conversion processing strategy, the clause sampling is performed on the A3-th written language sentence at random, and the third target clause in the A3-th written language sentence is obtained as "I use a computer".

Step 516-17: Perform syntax analysis on the third target clause to obtain a syntax structure corresponding to the third target clause.

Specifically, the third target clause is syntactically analyzed to obtain a subject-predicate-object structure corresponding to the third target clause.

Step 516-18: Obtain the converted A4th written language sentence by converting the third target clause according to the target syntax structure corresponding to the syntax structure.

Specifically, when the target syntactic structure corresponding to the subject-verb-object structure is the object-verb-subject structure, the third target clause is converted according to the object-verb-subject structure, and the converted written language sentence A4 is obtained as: "The computer is used by me. It's very fast to use, it's very fast, and it's very convenient, right, right."

Step 518: Perform word unit conversion processing on the A4th written language sentence to obtain the B3th written language sentence.

Specifically, on the basis of the above-mentioned conversion processing of clause units on the written language sentence to obtain the A4 written language sentence, the conversion processing of the word unit is performed on the A4 written language sentence. Specifically, the following steps 518-1 to 518- are executed as follows: 12 to implement:

Step 518-1: Determine the added word conversion processing probability corresponding to the added word conversion processing strategy in each word conversion processing strategy of the A4th written language sentence.

Specifically, the added word conversion processing strategy refers to a processing strategy for adding words to written sentences. Correspondingly, the added word conversion processing probability refers to a preset probability of executing the added word conversion processing strategy. The added word conversion processing probability may be preset according to actual experience or oral expression habits. For example, the added word conversion processing probability may be 10%, 13%, etc., which is not limited here. In the case of adding word conversion processing probability of 10%, it indicates that there is a 10% probability to execute the adding word conversion processing strategy for written sentences.

Based on this, determine the written language sentence A4: "The computer is used by me, the speed is very fast, the speed is very fast, and it is very convenient, right, right, right". Add word transformation processing strategy corresponding to the addition word transformation processing strategy If the probability is 10%, the probability of the addition-word conversion processing for executing the addition-word conversion processing strategy for the A4 written language sentence is 10%.

Step 518-2: Based on the added word conversion processing probability, determine whether to execute the added word conversion processing strategy for the A4th written language sentence.

Specifically, based on the added word conversion processing probability, it is determined whether to execute the added word conversion processing strategy for the A4 written language sentence. The specific implementation method is similar, referring to the above-mentioned specific implementation method for determining whether to execute the copy clause conversion processing strategy for the written language sentence based on the conversion processing probability based on the copy clause, which will not be repeated here.

In specific implementation, if it is determined to execute the added word conversion processing strategy, the following step 518-3 is executed; if it is determined not to execute the added word conversion processing strategy, the A4th written language sentence is directly regarded as the B1th written language sentence, and the following step 518 is executed. -5.

Based on this, it is assumed that the addition-word conversion processing strategy is executed for the A4th written language sentence based on the assumption that the addition-word conversion processing probability is 10%.

Step 518-3: In the case where it is determined to execute the adding word conversion processing strategy, determine the word position probability distribution corresponding to the preset words included in the preset word set.

Specifically, the preset word set includes two preset words, and the two preset words are preset word 1 and preset word 2 respectively. According to the statistics of the spoken language corpus in the sales field, the probability distribution of the word positions corresponding to the two preset words is: the probability of adding the preset word 1 to the beginning of the sentence is 80%, and the probability of adding the preset word 2 to the end of the sentence is 20% %.

Step 518-4: Determine the target preset word and the word addition position corresponding to the target preset word in the preset words according to the word position probability distribution, and add the target preset word to the A4th written language sentence according to the word addition position to obtain a conversion After the B1 written language statement.

Specifically, based on the word position probability distribution, it is determined in the preset words that the target preset word is the preset word 1 and the preset addition position corresponding to the preset word 1 is the beginning of the sentence, then the preset word 1 is added to the At the beginning of the sentence in the written language sentence, when the preset word 1 is "Wowsai", the converted B1 written sentence is obtained as "The computer is used by me, wow, it is very fast, very fast, and very convenient, exactly".

Step 518-5: Determine the duplicated word conversion processing probability corresponding to the duplicated word conversion processing strategy in each word conversion processing strategy of the B1th written language sentence.

The copy word conversion processing strategy refers to a processing strategy for copying the target preset words added in the above step 518-4 in the written language sentence. Correspondingly, the copied word conversion processing probability refers to the preset probability of executing the relevant processing of the copied word conversion processing strategy. The copied word conversion processing probability can also be preset according to actual experience or oral expression habits, such as the copied word conversion processing. The probability can be 8%, 12%, etc., which is not limited here. Under the circumstance that the conversion processing probability of duplicated words is 8%, it indicates that there is an 8% probability of executing the duplicated word conversion processing strategy for written sentences.

Based on this, it is determined that the duplicated word conversion processing probability corresponding to the duplicated word conversion processing strategy of the B1st written language sentence is 8%.

Step 518-6: Based on the duplicated word conversion processing probability, determine whether to execute the duplicated word conversion processing strategy for the B1th written language sentence.

Specifically, based on the duplicated word conversion processing probability, the specific implementation method of determining whether to execute the duplicated word conversion processing strategy for the B1 written language sentence and the above-mentioned method of determining whether to execute the duplicated clause conversion processing strategy for the written language sentence based on the duplicated clause conversion processing probability The specific implementation method is similar, referring to the above-mentioned specific implementation method for determining whether to execute the copy clause conversion processing strategy for the written language sentence based on the conversion processing probability based on the copy clause, which will not be repeated here.

In the specific implementation, if it is determined to execute the duplicated word conversion processing strategy, the following step 518-7 is executed; if it is determined not to execute the duplicated word conversion processing strategy, the B1th written language sentence is directly regarded as the B2th written language sentence, and the following step 518 is executed. -8.

Based on this, it is assumed that the duplicated word conversion processing strategy is executed for the B1 th written language sentence based on the assumption that the duplicated word conversion processing probability is 8%.

Step 518-7: In the case of determining to execute the copy word conversion processing strategy, copy the target preset words added in the B1st written language sentence to obtain the copied words, and insert the copied words into the B1th written language according to the preset word insertion rules. In the sentence, the B2-th written language sentence after insertion is obtained.

Based on this, in the case of determining to execute the copy word conversion processing strategy, the target preset word "Wase" in the B1 written language sentence is copied, and the obtained copied word is also "Wase". When the preset word is inserted before the position of the target preset word, insert the copied word before the position of the target preset word in the B1 written language sentence, and the inserted B2 written language sentence is obtained as: "The computer is used by me , wow, wow, it's fast, it's fast, and it's very convenient, right, right."

Step 518-8: Determine the out-of-order word conversion processing probability corresponding to the out-of-order word conversion processing strategy in each word conversion processing strategy of the B2th written language sentence.

Among them, the out-of-order word conversion processing strategy refers to a processing strategy for out-of-order words in a written sentence. Correspondingly, the conversion processing probability of out-of-order words refers to the preset probability of executing the relevant processing of the out-of-order word conversion processing strategy. The conversion processing probability of ordinal words can be 6%, 9%, etc., which is not limited here. Under the circumstance that the conversion processing probability of out-of-order words is 6%, it indicates that there is a 6% probability of executing out-of-order word conversion processing strategies for written sentences.

Based on this, it is determined that the out-of-order word conversion processing probability corresponding to the out-of-order word conversion processing strategy of the B2th written language sentence is 6%.

Step 518-9: Based on the out-of-order word conversion processing probability, determine whether to execute the out-of-order word conversion processing strategy for the B2th written language sentence.

Specifically, based on the out-of-order word conversion processing probability, it is determined whether to execute the out-of-order word conversion processing strategy for the B2 written language sentence. The specific implementation of the strategy is similar. Refer to the above-mentioned conversion processing probability based on the duplicated clause to determine whether to execute the specific implementation of the duplicated clause conversion processing strategy for the written language sentence, which will not be repeated here.

During specific implementation, if it is determined to execute the out-of-order word conversion processing strategy, the following steps 518-10 are executed; if it is determined not to execute the out-of-order word conversion processing strategy, the B2th written language sentence is directly regarded as the B3th written language sentence, and the following steps are executed. Step 520.

Based on this, assuming that the out-of-order word conversion processing probability is 5%, it is determined that the out-of-order word conversion processing strategy is executed for the B2th written language sentence.

Step 518-10: In the case of determining to execute the out-of-order word conversion processing strategy, perform word sampling on the words in the B2th written language sentence according to the preset word sampling rule to obtain the target word in the B2th written language sentence. Based on this, in the case of determining to execute the out-of-order word conversion processing strategy, randomly sample words of 2 characters in the B2th written language sentence, and obtain the target word in the B2th written language sentence as "use".

Step 518-11: Delete the target word in the B2-th written language sentence, and insert the target word into the preset insertion range corresponding to the target word in the deleted B2-th written language sentence to obtain the converted B3-th written language sentence.

Specifically, delete the target word "use" in the B2 written language sentence, and obtain the deleted B2 written language sentence: "The computer is used by me, and the speed is very fast, and it is very convenient." The target word "calculation" is randomly inserted into the character interval [-3, 3] in the deleted B2 written language sentence, and the converted B3 written language sentence is obtained as: "The computer is used by me, wow, wow, very fast. It’s fast, it’s fast, and it’s very convenient, right, right.”

Step 520: Perform character unit conversion processing on the B3th written language sentence to obtain the Cth written language sentence.

Specifically, on the basis that the word unit conversion processing is performed on the written language sentence and the B3 written language sentence is obtained, the character unit conversion processing is performed on the B3 written language sentence. Specifically, the following steps 520-1 to 520-4 are executed as follows To implement:

Step 520-1: Determine the out-of-order character conversion processing probability corresponding to the out-of-order character conversion processing strategy of the B3th written language sentence.

Specifically, the out-of-order character conversion processing strategy refers to a processing strategy for out-of-order characters in a written sentence. Correspondingly, the out-of-order character conversion processing probability refers to the preset probability of executing the relevant processing of the out-of-order character conversion processing strategy. The out-of-order character conversion processing probability can also be preset according to actual experience or oral expression habits, such as random The conversion processing probability of ordinal characters can be 5%, 9%, etc., which is not limited here. When the probability of out-of-order character conversion processing is 5%, it indicates that there is a 6% probability of executing out-of-order character conversion processing strategy for written sentences.

Based on this, it is determined that the out-of-order character conversion processing probability corresponding to the out-of-order character conversion processing strategy of the B3 written language sentence is 5%.

Step 520-2: Based on the out-of-order character conversion processing probability, determine whether to execute the out-of-order character conversion processing strategy for the B3th written language sentence.

Specifically, based on the out-of-order character conversion processing probability, it is determined whether to execute the out-of-order character conversion processing strategy for the B3 written language sentence. The specific implementation of the strategy is similar. Refer to the above-mentioned conversion processing probability based on the duplicated clause to determine whether to execute the specific implementation of the duplicated clause conversion processing strategy for the written language sentence, which will not be repeated here.

During specific implementation, if it is determined to execute the out-of-order character conversion processing strategy, the following step 520-3 is executed; if it is determined not to execute the out-of-order character conversion processing strategy, the B3th written language statement is directly used as the Cth written language statement, and the following step 520-3 is executed. Step 522.

Based on this, assuming that the out-of-order character conversion processing probability is 5%, it is determined that the out-of-order character conversion processing strategy is executed for the B3th written language sentence.

Step 520-3: In the case of determining to execute the out-of-order character conversion processing strategy, character sampling is performed on the characters in the B3th written language sentence according to the preset character sampling rule to obtain the target character in the B3th written language sentence. Based on this, when it is determined to execute the out-of-order character conversion processing strategy, character sampling is performed on the B3th written language sentence at random, and the target character in the B3th written language sentence is obtained as "make".

Step 520-4: Delete the target character in the B3th written language sentence, and insert the target character into the preset character insertion range corresponding to the target character in the B3th written language sentence after deletion, to obtain the converted Cth written language sentence. Specifically, delete the target character "make" in the B3 written language sentence, and obtain the deleted B3 written language sentence: "The computer is used by me, wow, wow, very fast, very fast, and very convenient, yes Yep". Randomly insert the target character "make" into the scope of the clause to which the target character belongs in the deleted B3 written language sentence, and the converted C written language sentence is obtained as: "The computer is used by me, wow, wow, very fast. It’s fast, it’s fast, and it’s very convenient, right, right.”

Step 522: Perform symbol unit conversion processing on the C-th written language sentence to obtain the D2-th written language sentence.

Specifically, on the basis of performing the character unit conversion process on the written language sentence and obtaining the Cth written language sentence, the symbol unit conversion process is performed on the Cth written language sentence. Specifically, the following steps 522-1 to 522-8 are executed as follows To implement:

Step 522-1: Determine the deletion symbol conversion processing probability corresponding to the deletion symbol conversion processing strategy in the symbol conversion processing strategy of the Cth written language sentence.

Specifically, the deletion symbol conversion processing strategy refers to a processing strategy for performing symbol deletion on written language sentences. Correspondingly, the deletion symbol conversion processing probability refers to the preset probability of executing the relevant processing of the deletion symbol conversion processing strategy. The deletion symbol conversion processing probability can also be preset according to actual experience or oral expression habits, such as deletion symbol conversion processing. The probability can be 8%, 12%, etc., which is not limited here. Under the circumstance that the probability of deletion symbol conversion processing is 8%, it indicates that there is an 8% probability of executing the deletion symbol conversion processing strategy for written sentences.

Based on this, it is determined that the deletion symbol conversion processing probability corresponding to the deletion symbol conversion processing strategy in the symbol conversion processing strategy of the Cth written language sentence is 8%, and the probability of executing the deletion symbol conversion processing strategy for the Cth written language sentence is 8%.

Step 522-2: Based on the probability of the deletion symbol conversion processing, determine whether to execute the deletion symbol conversion processing strategy for the Cth written language sentence.

Specifically, based on the probability of the deletion symbol conversion processing, it is determined whether to execute the deletion symbol conversion processing strategy for the Cth written language sentence. The specific implementation method is similar, referring to the above-mentioned specific implementation method for determining whether to execute the copy clause conversion processing strategy for the written language sentence based on the conversion processing probability based on the copy clause, which will not be repeated here.

During specific implementation, if it is determined to execute the deletion symbol conversion processing strategy, the following step 522-3 is executed; if it is determined not to execute the deletion symbol conversion processing strategy, the Cth written language statement is directly regarded as the D1th written language statement, and the following step 522 is executed -5.

Based on this, it is assumed that the deletion symbol conversion processing strategy is executed for the C written language statement based on the deletion symbol conversion processing probability of 8%.

Step 522-3: In the case of determining to execute the delete symbol conversion processing strategy, perform symbol sampling on the Cth written language sentence according to the preset symbol sampling rule, and obtain the target punctuation mark in the Cth written language sentence. Based on this, in the case of determining the implementation of the deletion symbol conversion processing strategy, the Cth written language sentence is randomly sampled to obtain the target punctuation mark in the Cth written language sentence is the comma after the first "fast" clause.

Step 522-4: Delete the target punctuation in the C-th written language sentence to obtain the converted D1-th written language sentence.

Specifically, delete the comma after the first "very fast" clause in the C-th written language statement, and obtain the deleted D1-th written language statement: "The computer is used by me, wow, wow, the speed is very fast. , and it’s very convenient, right, right.”

Step 522-5: Determine the added symbol conversion processing probability corresponding to the added symbol conversion processing strategy in the symbol conversion processing strategy of the D1-th written language sentence.

The processing strategy for adding symbols conversion refers to a processing strategy for adding symbols to written language sentences. Correspondingly, the added symbol conversion processing probability refers to the preset probability of executing the related processing of the added symbol conversion processing strategy. The added symbol conversion processing probability can also be preset according to actual experience or oral expression habits, such as adding a symbol conversion processing. The probability can be 2%, 5%, etc., which is not limited here. In the case of adding symbol conversion processing probability of 2%, it indicates that there is a 2% probability to implement the adding symbol conversion processing strategy for written language sentences.

Based on this, it is determined that the added sign conversion processing probability corresponding to the added sign conversion processing strategy of the D1-th written language sentence is 2%.

Step 522 - 6 : Based on the probability of adding symbol conversion processing, determine whether to execute the adding symbol conversion processing strategy for the D1 th written language sentence.

Specifically, based on the added symbol conversion processing probability, it is determined whether to execute the added symbol conversion processing strategy for the D1 written language sentence. The specific implementation method is similar, referring to the above-mentioned specific implementation method for determining whether to execute the copy clause conversion processing strategy for the written language sentence based on the conversion processing probability based on the copy clause, which will not be repeated here.

In the specific implementation, if it is determined to execute the addition symbol conversion processing strategy, the following step 522-7 is executed; if it is determined not to execute the addition symbol conversion processing strategy, the D1-th written language statement is directly used as the D2-th written language statement, and the following step 524 is executed .

Based on this, it is assumed that based on the addition-symbol conversion processing probability of 2%, it is determined that the addition-symbol conversion processing strategy is executed for the D1 written language sentence.

Step 522-7: If it is determined to execute the addition symbol conversion processing strategy, perform symbol clause sampling on the D1 th written language sentence according to the preset symbol clause sampling rule to obtain the target symbol clause in the D1 th written language sentence.

Based on this, in the case of determining the implementation of the conversion processing strategy of adding symbols, the clauses with the largest number of sampled characters in the D1 written language sentence include "very fast and very fast" and "and it is very convenient", and then from these two sub-clauses Randomly select a clause in the sentence to obtain the target character clause, the target character clause is "very fast very fast".

Step 522-8: Randomly insert a preset punctuation mark into the target symbol clause, obtain the converted D2-th written language sentence, and use the D2-th written language sentence as the target spoken language sentence.

Specifically, when the preset punctuation symbol is "!", the preset punctuation symbol "!" is randomly inserted into the target symbol clause "speed is very fast", and the converted D2 written language sentence is obtained as: "computer I use it, wow, wow, very fast, very fast! And it's very convenient, right, right." Use the D2 written language sentence as the target spoken sentence.

Step 524: Combine the D2-th written language sentences corresponding to each written sentence in each of the texts to be processed to obtain a spoken text corresponding to each of the to-be-processed texts.

Specifically, after performing the above steps 516 to 522 for the n written sentences contained in the text to be processed, respectively, the target spoken sentence corresponding to each written sentence in the to-be-processed text T can be obtained. Combine the n written sentences in the sentence order in the text T to be processed to obtain the spoken text corresponding to the text T to be processed. Among them, the spoken text includes the target spoken sentence corresponding to the above written sentence 1: "The computer is used by me, wow, wow, very fast! And it is very convenient, right, right".

Step 526: Identify abnormal information in the spoken text.

Specifically, the abnormal information in the target spoken sentence corresponding to the written sentence 1 in the spoken text is identified as ",!"

Step 528: Perform data cleaning on the spoken text according to the abnormal information to obtain the cleaned spoken text.

In practical applications, data cleaning is performed on the spoken text according to the abnormal information, and the abnormal information identified in the spoken text can be filtered or adjusted.

Specifically, according to the abnormal information ",!", data cleaning is performed on the target spoken sentence corresponding to the written sentence 1 in the spoken text, and the target spoken sentence corresponding to the written sentence 1 in the cleaned spoken text is changed to: "The computer is used by me. , wow, wow, it's fast, fast! And it's very convenient, right, right."

Step 530: Construct a sample corpus based on the written text and the correspondence between the back-translated written text and the cleaned spoken text.

In practical applications, a corresponding cleaned spoken text can be obtained for each written text or back-translated written text. Therefore, a large number of written texts can be obtained, and the above steps 502 to 528 are respectively performed for each written text to obtain back-translated written texts corresponding to each written text and cleaned spoken texts corresponding to these written texts. A sample corpus pair is composed of the written language text and the cleaned spoken language text with the corresponding relationship, and the sample corpus pair is composed of the sample corpus.

Further, since various conversion processing strategies have their own conversion processing probabilities, it is not fixed whether various conversion processing strategies are executed each time the above steps 516 to 522 are executed. There is a high probability that the spoken texts generated in steps 516 to 522 are not the same. Based on this, the above-mentioned steps 516 to 522 can be performed for each text to be processed multiple times, so as to generate a variety of spoken texts for each text to be processed, so as to further expand the sample corpus.

Specifically, acquire m written language texts in the sales field (the m written language texts include the above written language text T), and perform back-translation processing from the above steps 504 to 512 on each written language text to obtain m back-translated written language texts , and then use the m written texts and m back-translated written texts as texts to be processed, and execute the above steps 514 to 528 to obtain m cleaned spoken texts corresponding to the m written texts, and m back-translated written texts Corresponding m cleaned spoken texts, m written texts are taken as training samples, and m corresponding m cleaned spoken texts are taken as sample labels, and m back-translated written texts are taken as training samples, and the m written texts are taken as training samples. The corresponding m cleaned spoken texts are used as sample labels to construct sample corpus.

Step 532 : Train the initial written language rewriting model through the sample corpus until a written language rewriting model that satisfies the second training stop condition is obtained.

Specifically, the initial written language rewriting model is trained through the sample corpus, and when the training meets the preset number of iterations of i, the training is stopped, and the written language rewriting model M1 is obtained.

In addition, the above steps 516 to 528 may be re-executed on the written language sentences contained in the above m written language texts on the basis of setting lower probability values for the various conversion processing probabilities in the above-mentioned steps 516 to 522 , obtain m cleaned oral texts SST corresponding to the m written texts, and then use the m written texts as training samples and the corresponding m cleaned oral texts SST as sample labels to construct a basic sample corpus . The above-mentioned initial written language rewriting model is then trained through the basic sample corpus until a written language conversion model M2 that satisfies the first training stop condition is obtained.

In specific implementation, the first training stop condition refers to the training stop condition for training the initial written language rewriting model by using the basic sample corpus. The first training stop condition may be the same as or different from the above-mentioned second training stop condition, which is not limited here. . The written language conversion model can be understood as a trained model that slightly rewrites the spoken text in written language. During specific implementation, since the written language conversion model uses simpler sample corpus than the written language rewriting model for model training, the written language conversion model can be used to deal with relatively minor sentence rewriting.

Step 534: Obtain the target spoken text.

Specifically, the target spoken text T4 is obtained. The target spoken text T4 can be any spoken text in the sales field.

Step 536: Perform text classification on the target spoken text through a text classification model to obtain the spoken text type.

Among them, the text classification model refers to a pre-trained model for classifying spoken text. In practical applications, the text classification model can be CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory) network), FastText, TextCNN, HAN model, etc., and there are no restrictions here.

In practical applications, a large number of spoken texts can be obtained, and these spoken texts can be marked and processed to obtain a text label corresponding to each spoken text. The text labels include: invalid text type, fuzzy text type, standard text type, etc. Here No restrictions. Then, a training sample is constructed by using the spoken text and the text label corresponding to the spoken text, and an initial text classification model is trained through the training sample, and the above-trained text classification model is obtained.

Specifically, the target spoken text T4 is subjected to text classification through a pre-trained text classification model, and a text type corresponding to the target spoken text T4 output by the text classification model is obtained.

Step 538: If the text type is an invalid text type, delete the target spoken text.

Specifically, if the text type is an invalid text type, the target spoken text T4 may be deleted.

Step 540: When the text is classified as a standard text type, input the target spoken text into the written language rewriting model for written language rewriting, and obtain the target written language text output by the written language rewriting model.

Specifically, assuming that the text type is a standard text type, input the target spoken text T4 into the written language rewriting model M1 for written language rewriting, and obtain the target written language text T5 output by the written language rewriting model M1.

Step 542: When the text type is the fuzzy text type, input the target spoken text into the written language conversion model for written language rewriting, and obtain the converted written language text output by the written language conversion model.

Specifically, assuming that the text type is a fuzzy text type, the target spoken text T4 is input into the written language conversion model M2 for written language rewriting, and the converted written language text T6 output by the written language conversion model M2 is obtained.

To sum up, in the text processing method provided by the embodiments of the present application, by performing back-translation processing on the written text to obtain the back-translated written text, the original written text can be expanded with the back-translated written text. On this basis, the written text is subjected to clause-level, word-level, character-level and symbol-level conversion processing with a preset conversion processing probability, thereby further expanding the spoken text corresponding to the written text. The expanded written text and spoken text are combined to generate sample corpus, which realizes the automatic generation of the sample corpus of the written language rewriting model, and enriches the sample corpus of the written language rewriting model, thereby improving the generation efficiency of the sample corpus, and indirectly By enriching the sample corpus, the rewriting accuracy of the written language rewriting model is improved.

Corresponding to the foregoing method embodiments, the present application further provides an embodiment of a text processing apparatus, and FIG. 6 shows a schematic structural diagram of a text processing apparatus provided by an embodiment of the present application. As shown in Figure 6, the device includes:

an obtaining module 602, configured to obtain the target spoken text;

The classification module 604 is configured to classify the target spoken text to obtain a text type corresponding to the target spoken text;

A selection module 606, configured to select a corresponding written language rewriting model according to the standard text type when the text type is a standard text type;

The processing module 608 is configured to input the target spoken language text into the written language rewriting model for processing, and obtain target written language text corresponding to the target spoken language text; wherein, the written language rewriting model is based on the written language text and the written language rewriting model. The text is back-translated and converted from the spoken text obtained by training.

Optionally, the text processing apparatus further includes:

a selection model module, configured to select a corresponding written language conversion model according to the fuzzy text type when the text type is a fuzzy text type;

Input the target spoken language text into the written language conversion model for processing, and obtain the converted written language text corresponding to the target spoken language text; wherein, the written language conversion model is obtained based on the written language text and the conversion processing of the written language text. The basic oral text training is obtained.

Optionally, the training of the written language conversion model is implemented by running the following modules:

a first obtaining module, configured to obtain written text;

a first conversion module, configured to perform conversion processing of sentence composition units on the written text to obtain basic spoken text;

a first building module, configured to construct a basic sample corpus based on the correspondence between the written text and the basic spoken text;

The first training module is configured to train an initial written language conversion model by using the basic sample corpus until the written language conversion model that satisfies the first training stopping condition is obtained.

Optionally, the classification module 604 is further configured to:

Inputting the target spoken text into a text classification model for classification processing to obtain a text type corresponding to the target spoken text; wherein, the training of the text classification model is achieved by running the following modules:

a sample acquisition module, configured to acquire the sample spoken text and the semantic clarity label corresponding to the sample spoken text;

a sample pair construction module configured to construct a training sample pair based on the sample spoken text and the semantic clarity label;

The model training module is configured to perform model training on the initial text classification model by using the training samples until the text classification model that satisfies the classification training stop condition is obtained.

Optionally, the processing module 608 is further configured to:

The target spoken text is subjected to sentence processing to obtain a sequence of sentences contained in the target spoken text;

The spoken sentence units in the sentence sequence are sequentially input into the coding layer of the written language rewriting model for encoding processing, and the sentence feature vector and vocabulary vector corresponding to the spoken sentence unit are obtained, wherein the vocabulary vector is determined by the The spoken sentence unit and the vocabulary are mapped to obtain;

Calculate the vector product between the sentence feature vector and the vocabulary vector, and input the vector product into the decoding layer of the written language rewriting model for decoding processing to obtain the target written language text corresponding to the target spoken language text.

Optionally, the training of the written language rewriting model is implemented by running the following modules:

a second obtaining module, configured to obtain written text;

a back-translation module, configured to obtain back-translated written text corresponding to the written text by performing back-translation processing on the written text;

The second conversion module is configured to perform conversion processing of sentence composition units on the written text and the back-translated written text, respectively, to obtain spoken text;

a second building module, configured to build a sample corpus based on the written text and the correspondence between the back-translated written text and the spoken text;

The second training module is configured to train the initial written language rewriting model through the sample corpus until a written language rewriting model that satisfies the second training stopping condition is obtained.

Optionally, the back translation module includes:

a translation submodule, configured to translate the written text into the translated written text corresponding to the preset language;

A back-translation sub-module, configured to back-translate the written text of the translation into the target language to which the written text belongs, and obtain the initial back-translated written text;

The replacement submodule is configured to replace the target key words corresponding to the key words in the initial back-translated written text by using the key words in the written text to obtain back-translated written text.

Optionally, the back translation module further includes:

A part-of-speech analysis submodule, configured to identify key words whose part-of-speech is a preset part of speech in the written-language text by performing part-of-speech analysis on the written language text;

a marking submodule, configured to mark the position of the key phrase in the written text;

Correspondingly, the replacement submodule is further configured as:

By using the key words, the target key words corresponding to the position marks in the initial back-translated written text are replaced to obtain back-translated written text.

Optionally, the sentence composition unit includes at least one of the following: a clause unit, a word unit, a character unit, and a symbol unit.

Optionally, when the statement composition unit is a clause unit, the second conversion module includes:

a first identification submodule, configured to perform sentence identification on the written language text to be processed, and obtain written language sentences contained in the written language text to be processed;

a clause conversion submodule, configured to perform conversion processing of clause units on the written language statement to obtain the converted written language statement;

The first determination module is configured to determine the spoken text based on the converted written sentence.

Optionally, the clause conversion submodule is further configured to:

Perform clause sampling on the written language statement according to the preset clause sampling rule to obtain the target clause in the written language statement; perform conversion processing on the target clause in the written language statement to obtain the converted written language statement ;and / or

Determine the clause position probability distribution corresponding to the preset clause included in the preset clause set; determine the target preset clause and the target preset sub-clause in the preset clause based on the clause position probability distribution The clause addition position corresponding to the sentence is added; the target preset clause is added to the written language sentence according to the clause addition position to obtain the converted written language sentence.

Optionally, the clause conversion submodule is further configured to:

Duplicating the target clause to obtain a duplicate target clause, and inserting the duplicate target clause into the written language statement according to a preset clause insertion position to obtain a converted written language statement; and/or,

The target clause is to be deleted in the written language statement; the target clause is inserted into the deleted written language statement according to the preset clause insertion rule to obtain the converted written language statement; and/or

Syntactic analysis is performed on the target clause to obtain a syntactic structure corresponding to the target clause; the converted written language sentence is obtained by converting the target clause according to the target syntactic structure corresponding to the syntactic structure.

Optionally, in the case that the sentence composition unit is a word unit, the second conversion module includes:

The second identification submodule is configured to perform sentence identification on the written language text to be processed, and obtain written language sentences contained in the written language text to be processed;

A determination distribution submodule, configured to determine the word position probability distribution corresponding to the preset words included in the preset word set;

A word adding sub-module is configured to determine a target preset word and a word addition position corresponding to the target preset word in the preset words according to the word position probability distribution, and add the word addition position according to the word addition position. Inserting target preset words into the written language sentence to obtain the converted written language sentence;

The second determination module is configured to determine the spoken text based on the converted written sentence.

Optionally, in the case that the sentence composition unit is a word unit, the second conversion module is further configured to:

Perform sentence recognition on the written language text to be processed to obtain written language sentences contained in the written language text;

Perform word sampling on the words in the written language sentence according to the preset word sampling rule to obtain the target word in the written language sentence;

Deleting the target word in the written language statement, and inserting the target word into the preset insertion range corresponding to the target word in the deleted written language statement to obtain a converted written language statement;

The spoken text is determined based on the transformed written sentences.

Optionally, the second conversion module is further configured as:

The copy word submodule is configured to copy the target preset words added in the converted written language sentence to obtain the copied words;

A word insertion submodule, configured to insert the copied word into the converted written language statement according to a preset word insertion rule, to obtain the inserted written language statement;

The third determination module is configured to determine the spoken text based on the inserted written sentence.

Optionally, in the case that the statement composition unit is a character unit, the second conversion module is further configured as:

Perform sentence recognition on the written language text to be processed to obtain written language sentences contained in the written language text to be processed;

According to the preset character sampling rule, character sampling is performed on the characters in the written language sentence to obtain the target character in the written language sentence;

Deleting the target character in the written language statement, and inserting the target character into the preset character insertion range corresponding to the target character in the deleted written language statement, to obtain the converted written language statement;

The spoken text is determined based on the transformed written sentences.

Optionally, in the case that the statement composition unit is a symbol unit, the second conversion module is further configured as:

Perform sentence recognition on the written language text to be processed to obtain written language sentences contained in the written language text to be processed; perform symbol sampling on the written language sentences according to preset symbol sampling rules to obtain target punctuation marks in the written language sentences, Deleting the target punctuation in the written sentence to obtain a converted written sentence; determining a spoken text based on the converted written sentence; and/or

Perform sentence recognition on the written language text to be processed to obtain written language sentences contained in the written language text to be processed; perform symbol clause sampling on the written language sentence according to a preset symbol clause sampling rule to obtain the written language sentence in the written language sentence. A target symbol clause, inserting a preset punctuation mark into the target symbol clause to obtain a converted written language sentence; determining the spoken text based on the converted written language sentence.

Optionally, the second conversion module is further configured as:

determining the conversion processing probability corresponding to the conversion processing strategy of the written text to be processed;

Based on the conversion processing probability, determining the target conversion processing strategy to be executed in the conversion processing strategy;

By executing the target conversion processing strategy, the written language text is subjected to sentence composition unit conversion processing, so as to obtain the spoken language text corresponding to the written language text to be processed.

Optionally, the text processing apparatus further includes:

an identification information module configured to identify abnormal information in the spoken text;

a cleaning module, configured to perform data cleaning on the spoken text according to the abnormal information to obtain cleaned spoken text;

A sample corpus construction module is configured to construct a sample corpus based on the written text and the correspondence between the back-translated written text and the cleaned spoken text.

Optionally, the second conversion module is further configured as:

performing conversion processing on the written language text of the sentence composition unit to obtain the first spoken language text corresponding to the written language text;

Performing sentence composition unit conversion processing on the back-translated written text to obtain a second spoken text corresponding to the back-translated written text;

The first spoken text and the second spoken text are used as the spoken text.

Optionally, the text processing apparatus further includes:

A deletion module, configured to delete the target spoken text when the text type is an invalid text type.

The text processing apparatus provided by the embodiment of the present application obtains the target spoken text by obtaining the target spoken text; classifying the target spoken text to obtain the text type corresponding to the target spoken text; and then in the case that the text type is a standard text type , select the corresponding written language rewriting model according to the standard text type, and realize the selection of the written language rewriting model suitable for the target spoken language text according to the text type of the target spoken language text; and then input the target spoken language text into the written language rewriting model for processing, and obtain The target written language text corresponding to the target spoken language text makes the written language rewriting more targeted and improves the accuracy of the written language rewriting. The written language rewriting model is obtained by training based on the written language text and the spoken language text obtained by back-translation and conversion processing of the written language text, and realizes the preprocessing of the written language text based on the back-translation and conversion processing, thereby providing model training. A large amount of spoken text-written text sample corpus simplifies the training difficulty of the model, and also avoids the time-consuming and laborious collection and processing of large amounts of text data, saving time and labor costs.

The above is a schematic solution of a text processing apparatus according to this embodiment. It should be noted that the technical solution of the text processing device and the technical solution of the above-mentioned text processing method belong to the same concept, and the details that are not described in detail in the technical solution of the text processing device can be referred to the description of the technical solution of the above-mentioned text processing method. .

It should be noted that each component in the device claim should be understood as a functional module that must be established to realize each step of the program flow or each step of the method, and each functional module is not an actual function division or separation limitation. A device claim defined by such a set of functional modules should be understood as a functional module architecture that mainly implements the solution through the computer program described in the specification, and should not be understood as a physical device that mainly implements the solution through hardware.

An embodiment of the present application further provides a computing device, including a memory, a processor, and computer instructions stored in the memory and executable on the processor, and the processor implements the text processing when executing the computer instructions steps of the method.

The above is a schematic solution of a computing device according to this embodiment. It should be noted that the technical solution of the computing device and the technical solution of the above text processing method belong to the same concept, and the details that are not described in detail in the technical solution of the computing device can be referred to the description of the technical solution of the above text processing method.

An embodiment of the present application further provides a computer-readable storage medium, which stores computer instructions, and when the computer instructions are executed by a processor, implements the steps of the aforementioned text processing method.

The above is a schematic solution of a computer-readable storage medium of this embodiment. It should be noted that the technical solution of the storage medium and the technical solution of the above-mentioned text processing method belong to the same concept, and the details not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the above-mentioned text processing method.

The embodiment of the present application discloses a chip, which stores computer instructions, and when the computer instructions are executed by a processor, implements the steps of the text processing method as described above.

The above is a schematic solution of a chip of this embodiment. It should be noted that the technical solution of the chip and the technical solution of the text processing method above belong to the same concept, and the details of the technical solution of the chip that are not described in detail can be referred to the description of the technical solution of the text processing method.

The foregoing describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The computer instructions include computer program code, which may be in source code form, object code form, an executable file, some intermediate form, or the like. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable media may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer-readable media Electric carrier signals and telecommunication signals are not included.

It should be noted that, for the convenience of description, the foregoing method embodiments are described as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described action sequence. Because in accordance with the present application, certain steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily all necessary for the present application.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

The preferred embodiments of the present application disclosed above are only provided to help illustrate the present application. Alternative embodiments are not intended to exhaust all details, nor do they limit the invention to only the described embodiments. Obviously, many modifications and variations are possible in light of the content of this application. The present application selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present application, so that those skilled in the art can well understand and utilize the present application. This application is to be limited only by the claims, along with their full scope and equivalents.

Claims (14)

1. a text processing method, is characterized in that, comprises: Get the target spoken text; classifying the target spoken text to obtain a text type corresponding to the target spoken text; In the case that the text type is a standard text type, selecting a corresponding written language rewriting model according to the standard text type; Inputting the target spoken text into the written rewriting model for processing to obtain a target written text corresponding to the target spoken text; The written language rewriting model is obtained by training based on the written language text and the spoken language text obtained by back-translating and converting the written language text. 2. The text processing method according to claim 1, wherein after obtaining the text type corresponding to the target spoken text, the method further comprises: In the case that the text type is a fuzzy text type, selecting a corresponding written language conversion model according to the fuzzy text type; Inputting the target spoken language text into the written language conversion model for processing to obtain the converted written language text corresponding to the target spoken language text; Wherein, the written language conversion model is obtained by training based on the written language text and the basic spoken language text obtained by converting the written language text. 3. The text processing method according to claim 2, wherein the training of the written language conversion model comprises: obtain written text; performing conversion processing on the written text to form units of sentences to obtain basic spoken text; constructing a basic sample corpus based on the correspondence between the written text and the basic spoken text; The initial written language conversion model is trained through the basic sample corpus until the written language conversion model that satisfies the second training stop condition is obtained. 4. The text processing method according to claim 1, wherein the classifying and processing the target spoken text to obtain a text type corresponding to the target spoken text, comprising: Inputting the target spoken text into a text classification model for classification processing to obtain a text type corresponding to the target spoken text; wherein, the training of the text classification model includes: Obtain the sample spoken text and the semantic clarity label corresponding to the sample spoken text; constructing a training sample pair based on the sample spoken text and the semantic intelligibility label; Model training is performed on the initial text classification model through the training samples until the text classification model that satisfies the classification training stop condition is obtained. 5. The text processing method according to claim 1, characterized in that, inputting the target spoken text into the written rewriting model for processing, and obtaining the target written text corresponding to the target spoken text, comprising: The target spoken text is subjected to sentence processing to obtain a sequence of sentences contained in the target spoken text; The spoken sentence units in the sentence sequence are sequentially input into the coding layer of the written language rewriting model for encoding processing, and the sentence feature vector and vocabulary vector corresponding to the spoken sentence unit are obtained, wherein the vocabulary vector is determined by the The spoken sentence unit and the vocabulary are mapped to obtain; Calculate the vector product between the sentence feature vector and the vocabulary vector, and input the vector product into the decoding layer of the written language rewriting model for decoding processing to obtain the target written language text corresponding to the target spoken language text. 6. The text processing method according to claim 1, wherein the written language rewriting model is obtained by training based on the written language text and the spoken language text obtained by back-translating and converting the written language text, comprising: obtain written text; By performing back-translation processing on the written text, a back-translated written text corresponding to the written text is obtained; The written text and the back-translated written text are respectively subjected to the conversion processing of the sentence constituent units to obtain the spoken text; constructing a sample corpus based on the written text and the correspondence between the back-translated written text and the spoken text; The initial written language rewriting model is trained through the sample corpus until the written language rewriting model that satisfies the second training stop condition is obtained. 7. The text processing method according to claim 6, wherein, by performing back-translation processing on the written text to obtain a back-translated written text corresponding to the written text, the method comprises: Translate the written language text into the translated written language text corresponding to the preset language; Back-translating the written text of the translation into the target language to which the written text belongs to obtain the initial back-translated written text; The back-translated written text is obtained by replacing the target key words corresponding to the key words in the initial back-translated written text with the key words in the written text. 8. The text processing method according to claim 7, wherein before translating the written text into the translated written text corresponding to a preset language, the method further comprises: By performing part-of-speech analysis on the written text, identifying key words whose part of speech is a preset part of speech in the written text; Mark the position of the key phrase in the written text; Correspondingly, the target key words corresponding to the key words in the initial back-translated written text are replaced by the key words in the written text to obtain the back-translated written text, including: Based on the position marker, the corresponding target key words in the initial back-translated written text are replaced by the key words to obtain back-translated written text. 9 . The text processing method according to claim 3 or 6 , wherein the sentence composition unit comprises at least one of the following: a clause unit, a word unit, a character unit and a symbol unit. 10 . 10 . The text processing method according to claim 6 , wherein any one of the written text and the back-translated written text is used as the written text to be processed, and the written text to be processed is composed of sentences. 11 . Unit conversion processing, including: determining the conversion processing probability corresponding to the conversion processing strategy of the written text to be processed; Based on the conversion processing probability, determining the target conversion processing strategy to be executed in the conversion processing strategy; By executing the target conversion processing strategy, the to-be-processed written text is subjected to sentence composition unit conversion processing, so as to obtain the spoken text corresponding to the to-be-processed written text. 11. The text processing method according to claim 1, wherein after obtaining the text type corresponding to the target spoken text, the method further comprises: In the case that the text type is an invalid text type, the target spoken text is deleted. 12. A text processing device, comprising: The acquisition module is configured to acquire the target spoken text; a classification module, configured to classify the target spoken text to obtain a text type corresponding to the target spoken text; a selection module, configured to select a corresponding written language rewriting model according to the standard text type when the text type is a standard text type; a processing module, configured to input the target spoken text into the written language rewriting model for processing, and obtain a target written language text corresponding to the target spoken language text; wherein, the written language rewriting model is based on the written language text and the written language text It is obtained by training the spoken text obtained by back-translation and conversion processing. 13. A computing device, comprising a memory, a processor, and computer instructions stored on the memory and running on the processor, wherein the processor implements any one of claims 1-11 when executing the computer instructions the steps of the method described in item. 14. A computer-readable storage medium storing computer instructions, wherein when the computer instructions are executed by a processor, the steps of the method according to any one of claims 1-11 are implemented.

Images