CN114328805A - Text processing method, system, storage medium and terminal equipment - Google Patents

Abstract

The embodiments of the present invention disclose a text processing method, a system, a storage medium and a terminal device, which are applied to the technical field of information processing based on artificial intelligence. The initial training model determined in the process of pre-training the cross-language summary model includes three branches of post-coding processing modules, corresponding to three different tasks, namely determining translation information, single-language summary information and cross-language summary information , and these three tasks share the same feature encoding module and the same feature extraction module, since the task of determining cross-language summary information can be the integration of the two sub-tasks of determining translation information and single-language summary information, in the training of cross-language summary model. In the process, the information of realizing these three tasks is used, and the overall task of determining cross-language summary information and its sub-tasks are considered, so that the cross-language summary model obtained by training is more accurate in extracting cross-language summary information.

Description

A text processing method, system, storage medium and terminal device

technical field

The present invention relates to the technical field of information processing based on artificial intelligence, in particular to a text processing method, system, storage medium and terminal device.

Background technique

Cross-language summarization technology is a task of summarizing the core information of the source language text and organizing it into summaries in the form of the target language. The research on cross-language summarization technology is of great significance for application scenarios such as cross-border e-commerce (assisting users to make decisions), public opinion analysis (helping analysts filter redundant information), and content recommendation (recommending foreign language news for users).

However, the effect of the existing cross-language summary information obtained during text processing is relatively poor. If the obtained cross-language summary information is further applied to various scenarios, it will cause a bad user experience.

SUMMARY OF THE INVENTION

The embodiments of the present invention provide a text processing method, a system, a storage medium and a terminal device, which realize cross-language abstract extraction using a relatively accurate cross-language abstract model.

One aspect of an embodiment of the present invention provides a text processing method, including:

Get the target object and call the pre-trained cross-language summarization model;

Extracting cross-language summary information of the target object by using the cross-language summary model;

Wherein, the cross-language summarization model is pre-trained by the following steps:

Determine the initial training model, the initial training model includes a feature extraction module, a feature encoding module and a post-coding processing module of three branches, the feature extraction module is used to extract the feature information of the sample object, and the feature encoding module is used for the sample object. The feature information of the object is encoded to obtain the encoded feature, and the encoding post-processing module of the first branch in the three branches is used to determine the translation information of the sample object according to the encoded feature. The three branches The post-encoding processing module in the second branch is used to determine the monolingual abstract information of the sample object according to the post-encoding features, and the post-encoding processing module in the third branch in the three branches is used for The post-encoding feature determines the cross-language summary information of the sample object;

determining a training sample, the training sample includes a plurality of first sample objects and their translation annotations, a plurality of second sample objects and their single-language abstract annotations, and a plurality of third sample objects and their cross-language abstract annotations;

The cross-language summarization model is trained according to the initial training model and training samples.

Another aspect of an embodiment of the present invention provides a text processing system, including:

The calling unit is used to obtain the target object and call the pre-trained cross-language summary model;

A summary extraction unit, configured to extract the cross-language summary information of the target object through the cross-language summary model;

The text processing system also includes:

The training unit is used to determine an initial training model, the initial training model includes a feature extraction module, a feature encoding module and a post-coding processing module of three branches, and the feature extraction module is used to extract the feature information of the sample object, and the feature encoding The module is used to encode the feature information of the sample object to obtain the encoded feature, and the post-encoded processing module of the first branch in the three branches is used to determine the translation information of the sample object according to the encoded feature, so The post-encoding processing module of the second branch in the three branches is used to determine the monolingual abstract information of the sample object according to the post-encoding features, and the post-encoding processing module of the third branch in the three branches for determining the cross-language summary information of the sample object according to the encoded features; determining a training sample, the training sample includes a plurality of first sample objects and their translation annotations, a plurality of second sample objects and their single-language abstracts Annotation and a plurality of third sample objects and their cross-language summary annotations; and the cross-language summary model is trained according to the initial training model and the training samples.

Another aspect of the embodiments of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a plurality of computer programs, and the computer programs are adapted to be loaded and executed by a processor as described in one aspect of the embodiments of the present invention the text processing method described.

Another aspect of the embodiments of the present invention further provides a terminal device, including a processor and a memory;

The memory is used for storing a plurality of computer programs, the computer programs are used for loading and executing the text processing method according to an aspect of the embodiment of the present invention by the processor; the processor is used for implementing the plurality of computers Individual computer programs within a program.

It can be seen that in the method of this embodiment, the text processing system will use a pre-trained cross-language summary model to extract the cross-language summary information of the target object, wherein the initial training model determined in the process of pre-training the cross-language summary model includes: The encoding post-processing modules of three branches correspond to three different tasks, namely, determining translation information, single-language summary information and cross-language summary information, and these three tasks share the same feature encoding module and the same feature extraction module. , since the task of determining cross-language summary information can be an integration of the two sub-tasks of determining translation information and single-language summary information, the information that implements these three tasks is utilized in the process of training the cross-language summary model, while taking into account the determination of cross-language summary information. The overall task of language summary information and its sub-tasks, so that even when the data set of cross-language summarization (ie, the third sample object in the training sample) is small, the cross-language summarization model obtained by training can extract cross-language summaries. It is also more accurate when summarizing information.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

1 is a schematic diagram of a text processing method provided by an embodiment of the present invention;

2 is a flowchart of a text processing method provided by an embodiment of the present invention;

3 is a schematic diagram of an initial training model determined in an embodiment of the present invention;

4 is a flowchart of a method for training a cross-language summary model in an embodiment of the present invention;

5 is a flowchart of a method for training a cross-language abstract model in an application embodiment of the present invention;

6 is a schematic diagram of an initial training model determined in an application embodiment of the present invention;

7 is a schematic diagram of a distributed system to which a text processing method is applied in another application embodiment of the present invention;

8 is a schematic diagram of a block structure in another application embodiment of the present invention;

9 is a schematic diagram of a logical structure of a text processing system provided by an embodiment of the present invention;

FIG. 10 is a schematic diagram of a logical structure of a terminal device according to an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The terms "first", "second", "third", "fourth", etc. (if present) in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to Describe a particular order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein can, for example, be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, eg, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed but may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

An embodiment of the present invention provides a text processing method, which is mainly applied to a text processing system to extract cross-language abstract information of a target object. As shown in FIG. 1 , the text processing system can extract cross-language abstract information through the following steps:

Get the target object and call the pre-trained cross-language summarization model;

Extracting cross-language summary information of the target object by using the cross-language summary model;

Wherein, the cross-language summarization model is pre-trained by the following steps:

Determine the initial training model, the initial training model includes a feature extraction module, a feature encoding module and a post-coding processing module of three branches, the feature extraction module is used to extract the feature information of the sample object, and the feature encoding module is used for the sample object. The feature information of the object is encoded to obtain the encoded feature, and the encoding post-processing module of the first branch in the three branches is used to determine the translation information of the sample object according to the encoded feature. The three branches The post-encoding processing module in the second branch is used to determine the monolingual abstract information of the sample object according to the post-encoding features, and the post-encoding processing module in the third branch in the three branches is used for The post-encoding feature determines the cross-language summary information of the sample object;

determining a training sample, the training sample includes a plurality of first sample objects and their translation annotations, a plurality of second sample objects and their single-language abstract annotations, and a plurality of third sample objects and their cross-language abstract annotations;

The cross-language summarization model is trained according to the initial training model and training samples.

In practical applications, the text processing system can be applied to an application terminal or server, such as a server implementing data recommendation, a dialogue summary server, or a search server.

It should be noted that the above-mentioned cross-language summarization model is a machine learning model based on artificial intelligence, wherein artificial intelligence (Artificial Intelligence, AI) is the use of digital computers or digital computer-controlled machines to simulate, extend and expand human intelligence. , the theory, method, technology and application system of perceiving the environment, acquiring knowledge and using knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making.

Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, including both hardware-level technology and software-level technology. The basic technologies of artificial intelligence generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation/interaction systems, and mechatronics. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning/deep learning.

Machine Learning (ML) is a multi-domain interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in how computers simulate or realize human learning behaviors to acquire new knowledge or skills, and to reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications are in all fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, teaching learning and other techniques.

In this way, the initial training model determined in the process of pre-training the cross-language summary model includes three branches of post-coding processing modules, corresponding to three different tasks, namely, determining translation information, single-language summary information, and cross-language summary information. Abstract information, and these three tasks share the same feature encoding module and the same feature extraction module, since the task of determining cross-language abstract information can be the integration of the two sub-tasks of determining translation information and single-language abstract information, when training cross-language abstract information In the process of the model, the information to achieve these three tasks is used, and the overall task of determining the cross-language summary information and the sub-tasks it includes are taken into account, so that the data set of the cross-language summary (that is, the third sample in the training sample) is immediately available. In the case of fewer objects), the trained cross-language summary model is also more accurate in extracting cross-language summary information.

An embodiment of the present invention provides a text processing method, mainly a method executed by a text processing system. The flowchart is shown in FIG. 2 , including:

In step 101, the target object is acquired, and the pre-trained cross-language summary model is invoked.

It can be understood that, in one case, the text processing system will provide a user interface, so that the user can input the target object through the user interface, thereby initiating the cross-language summarization process in this embodiment. In another case, the text processing system may actively take a text message as the target object, and initiate the cross-language summarization process of this embodiment. The target object may refer to text information in a certain language.

When the text processing system obtains the target object, it will call the cross-language summarization model preset in the system. The cross-language summarization model is a machine learning model, which can be trained by a certain method, and its operation logic is set in advance into a text processing system.

Step 102, extracting cross-language summary information of the target object through a cross-language summary model.

Wherein, the text processing system can pre-train the cross-language summarization model through the following steps before executing the above steps 101 and 102:

Step 201, determining an initial training model.

It can be understood that, when determining the initial training model, the text processing system will determine the multi-layer structure included in the initial training model and the initial values of the parameters in each layer mechanism. Among them, the parameters of the initial training model refer to the fixed parameters that are used in the calculation process of each layer structure in the initial training model and do not need to be assigned at any time, such as parameters such as parameter scale, number of network layers, and user vector length.

The structure of the initial training model can be shown in Figure 3. Specifically, the initial training model includes: a feature extraction module, a feature encoding module, and a post-coding processing module of three branches, and a feature extraction module for extracting the feature information of the sample object; The feature encoding module is used to encode the feature information of the sample object to obtain the encoded features; the post-encoding processing module of the first branch of the three branches is used to determine the translation information of the sample object according to the encoded features. The post-encoding processing module of the second branch is used to determine the monolingual abstract information of the sample object according to the post-encoding features, and the post-coding processing module of the third branch of the three branches is used to determine the cross-language of the sample object according to the post-encoding features Summary information.

Among them, the translation information refers to the text information of one language in the sample object corresponding to the text information of another language, and the single-language summary information refers to the summary information of the language corresponding to the text information of one language in the sample object. The summary information refers to the summary information of another language corresponding to the text information of one language in the sample object. For example, if the sample object is Chinese text information, the translation information may be the corresponding English text information, the single-language abstract information may be the corresponding Chinese abstract information, and the cross-language abstract information may be the corresponding English abstract information.

Further, as shown in FIG. 3 , the initial training model may also include attention models corresponding to the three branches respectively, specifically:

(1) If the translation information of the sample object includes multiple translation words, the multiple translation words are determined in sequence during the determination process. In this embodiment, when any translation word among the multiple translation words is determined, it is necessary to It is determined by combining the features of the determined translation words, so that in the process of determining multiple translation words, the relationship between adjacent translation words, that is, the context information of the translation words, is considered, so that the finally determined translation information is more accurate.

The attention module of the first branch is used to encode the determined translation word based on the attention mechanism to obtain the first historical encoded feature, and the post-encoded processing module of the first branch is used to obtain the first post-encoded feature according to the encoded feature and the first post-encoded feature. A historical post-encoding feature determines another translation word after the determined translation word.

(2) If the single-language summary information of the sample object includes multiple single-language words, the multiple single-language words are determined in sequence during the determination process. In this embodiment, when any one of the multiple single-language words is determined When a single-language word is used, it needs to be determined by combining the characteristics of the determined single-language word, so that in the process of determining multiple single-language words, the relationship between adjacent single-language words, that is, the context information of the single-language word is considered. , making the finalized single-language summary information more accurate.

The attention module of the second branch is used to encode the determined monolingual words based on the attention mechanism to obtain the second historical post-coding feature, and the post-coding processing module of the second branch is used to obtain the second post-coding feature according to the post-coding feature and The second historical post-encoding feature determines another monolingual word following the determined monolingual word.

(3) If the cross-language summary information of the sample object includes multiple cross-language words, the multiple cross-language words are determined in sequence during the determination process. In this embodiment, when any one of the multiple cross-language words is determined When cross-language words are used, they need to be determined by combining the characteristics of the determined cross-language words, so that in the process of determining multiple cross-language words, the relationship between adjacent cross-language words, that is, the context information of cross-language words is considered. , making the finalized cross-language summary information more accurate.

In the process of determining these cross-language words, the attention module of the third branch is summarized, and the identified cross-language words are encoded based on the attention mechanism to obtain the third historical post-encoding feature, and the post-encoding processing module of the third branch for determining another cross-language word subsequent to the determined cross-language word based on the post-encoded feature and a third historical post-encoded feature.

Among them, when the attention modules corresponding to the three branches are encoded based on the attention mechanism, they can be encoded based on a single-head self-attention mechanism (Self-Attention) or a multi-head self-attention mechanism (MultiHead). Among them, the attention mechanism mainly processes the target object through the attention function, and the essence of the attention function is the mapping from a query (query, Q) to a series of key (key, K)-value (value, V) pairs , the attention feature of the target object can be obtained through the attention function, which is used to represent the more important and noteworthy features of the target object. Specifically, when obtaining the above-mentioned first historically encoded features, the input of the attention function used is specifically the features of the determined translation words, and the obtained first historically encoded features are mainly used to characterize the determined translation words for comparison. The features of important translated words; when obtaining the above-mentioned second historical encoded features, the input of the attention function used is specifically the features of the determined monolingual words, and the obtained second historical encoded features are mainly used to represent the determined features. The characteristics of the more important monolingual words among the monolingual words of The feature is mainly used to characterize the features of the more important cross-language words among the determined cross-language words.

Among them, the similar single-head self-attention mechanism directly obtains the final attention feature based on its input (ie Q, K and V); the multi-head self-attention mechanism refers to the multiple linear transformation of its input (ie Q, K and V) first. , obtain the corresponding attention features based on the input after multiple transformations, and then synthesize the attention features obtained multiple times to determine the final attention features.

In a specific implementation process, the initial training model may specifically be a neural network in the form of a multi-task convolutional neural network (MTCNN), for example, an output network (ONet) of MTCNN.

Step 202: Determine a training sample, the training sample includes a plurality of first sample objects and their translation annotations, a plurality of second sample objects and their single-language abstract annotations, and a plurality of third sample objects and their cross-language abstract annotations.

Step 203, train a cross-language summarization model according to the initial training model and the training samples.

Specifically, as shown in Figure 4, the text processing system can train a cross-language summarization model through the following steps:

Step 2031: Determine the translation information of each first sample object, the single-language summary information of the second sample object, and the cross-language summary information of the third sample object through the initial training model.

Step 2032: Adjust the initial training model according to the translation information, single-language summary information and cross-language summary information determined by the initial training model, as well as the translation annotations, single-language summary annotations and cross-language summary annotations of the corresponding sample objects in the training samples, to obtain: The final trained model.

Specifically, the text processing system will first obtain the translation information, single-language summary information and cross-language summary information of each sample object obtained in the above step 2031, and the translation annotation, single-language summary annotation and cross-language summary information of the corresponding sample objects in the training sample. Abstract annotation, calculates the overall loss function related to the initial training model. The overall loss function is used to indicate that the translation information, single-language summary information and cross-language summary information of each sample object detected by the initial training model are different from the actual ones in the corresponding sample objects. Errors between translation information, single-language summary information, and cross-language summary information, such as the cross-entropy loss function, etc.; and then adjust the parameter values of the parameters in the initial training model according to the calculated overall loss function.

Among them, the process of training the training model is to reduce the value of the above error as much as possible. The training process is to continuously optimize the initial training model determined in the above step 201 through a series of mathematical optimization methods such as backpropagation derivation and gradient descent. The parameter value of the parameter, and minimize the calculated value of the above overall loss function.

In this embodiment, when calculating the overall loss function related to the initial training model, it can mainly include the following parts:

According to the translation information of the first sample object determined by the initial training model and the translation label of the corresponding first sample object in the training sample, the first loss function related to the post-coding processing module of the first branch is calculated; according to the initial training model The determined single-language summary information of the second sample object and the single-language summary annotation of the corresponding second sample object in the training sample are calculated, and the second loss function related to the post-coding processing module of the second branch is calculated; determined according to the initial training model The cross-language summary information of the third sample object and the cross-language summary annotation of the corresponding third sample object in the training sample, calculate the third loss function related to the encoding post-processing module of the third branch; The loss function and the third loss function calculate the overall loss function.

Wherein, when calculating the overall loss function, the first weight value and the second weight value may be determined first, and then the first product of the first weight value and the first loss function, the second weight value and the second loss function The product, and the sum of the third loss function are used as the overall loss function. The first weight value and the second weight value can be adjusted dynamically. Specifically, when the first weight value is adjusted, the adjusted first weight value is the first weight value before adjustment and the encoded first branch. The function calculation value between the total number of training steps of the processing module and the current count of the number of training steps; when adjusting the second weight value, the adjusted second weight value is the second weight value before adjustment, and the second branch after encoding The function computed between the total number of training steps for the processing module and the current count of training steps.

It should be noted that the above steps 2031 to 2032 are one-time adjustments to the parameter values in the initial training model based on the translation information, single-language summary information and cross-language summary information determined by the initial training model. The above steps 2031 to 2032 are continuously executed in a loop until the adjustment of the parameter value satisfies a certain stopping condition.

Therefore, after executing steps 2031 to 2032 in the above embodiment, the text processing system also needs to judge whether the current adjustment of the parameter value satisfies the preset stopping condition, and if so, the parameter value adjusted in the above step 2032 is used as the final training result. The parameter values in the training model of ; if not satisfied, return to the above steps 2031 to 2032 for the initial training model after adjusting the parameter values. Wherein, the preset stop condition includes but is not limited to any one of the following conditions: the difference between the parameter value currently adjusted and the parameter value adjusted last time is less than a threshold, that is, the adjusted parameter value has reached convergence; The adjustment times are equal to the preset times and so on.

Step 2033, it is determined that the pre-trained cross-language abstract model may include a feature extraction module, a feature encoding module and a post-encoding processing module of the third branch in the final training model.

It can be seen that in the method of this embodiment, the text processing system will use a pre-trained cross-language summary model to extract the cross-language summary information of the target object, wherein the initial training model determined in the process of pre-training the cross-language summary model includes: The encoding post-processing modules of three branches correspond to three different tasks, namely, determining translation information, single-language summary information and cross-language summary information, and these three tasks share the same feature encoding module and the same feature extraction module. , since the task of determining cross-language summary information can be an integration of the two sub-tasks of determining translation information and single-language summary information, the information that implements these three tasks is utilized in the process of training the cross-language summary model, while taking into account the determination of cross-language summary information. The overall task of language summary information and its sub-tasks, so that even when the data set of cross-language summarization (ie, the third sample object in the training sample) is small, the cross-language summarization model obtained by training can extract cross-language summaries. It is also more accurate when summarizing information.

The text processing method of the present invention is described with a specific application example below. The method of this embodiment may include the following two parts:

(1) As shown in Figure 5, the training of the cross-language summary model can be realized by the following steps:

Step 301: Determine the training sample, which may specifically include three parts, the first part of the training sample includes a plurality of first sample objects and their translation annotations, the second part of the training sample includes a plurality of second sample objects and their single-language abstract annotations, The third part of the training samples includes multiple third sample objects and their cross-language summary annotations.

Step 302, determine the initial training model, whose structure is shown in Figure 6, and may include: a feature extraction module, the feature encoding module is specifically an encoder (encoder), and the post-coding processing modules of the three branches specifically include a decoder (decoder) respectively. ) and probabilistic outputs (such as the softmax function), and an attention module with three branches.

Among them, the feature extraction module is used to extract the feature information of each part of the sample object, which can be embedded features (Embedding), such as token embedding features (token embeddings) and positional embedded features (positionalembeddings).

The encoder is used to encode the embedded features output by the feature extraction module to obtain encoded features.

具体地可以采用如下公式1-1来表示，其中，基于注意力机制具体为多头自注意机制(在其它实施例中可以采用单头自注意机制)，其输入为已确定的翻译词Y_mt,<t的嵌入特征y1；第一支路的解码器MT用于根据第一历史编码后特征

及针对第一样本对象得到的编码后特征

确定一个交互表示特征O_mt,t，具体可以通过如下公式1-2来表示，其中，可以在前馈神经网络(Feedforward Neural Network，FFN)中采用多头自注意机制来确定交互表示特征O_mt,t，这里t为某一个翻译词；第一支路的概率输出用于通过softmax函数输出第一样本对象的翻译信息(包括多个翻译词)的概率p_mt，具体可以通过如下公式1-3来表示，其中，W_mt和b_mt是需要学习的参数，需要先确定相应的初始值，X_mt表示第一样本对象的嵌入特征，Y_mt,t表示已确定的翻译词Y_mt,<t之后另一翻译词的概率。The attention module of the first branch is used to encode the determined translation word Y _mt,<t based on the attention mechanism to obtain the first historical encoded feature

Specifically, it can be represented by the following formula 1-1, wherein, the attention-based mechanism is specifically a multi-head self-attention mechanism (in other embodiments, a single-head self-attention mechanism can be used), and its input is the determined translation word Y _mt, The embedded feature y1 of _<t ; the decoder MT of the first branch is used to encode the feature according to the first history

and the encoded features obtained for the first sample object

Determining an interactive representation feature O _mt,t can be specifically represented by the following formula 1-2, wherein a multi-head self-attention mechanism can be used in a feedforward neural network (Feedforward Neural Network, FFN) to determine the interactive representation feature O _{mt, t} , where t is a certain translation word; the probability output of the first branch is used to output the probability p _mt of the translation information (including multiple translation words) of the first sample object through the softmax function, which can be specified by the following formula 1- 3 to represent, in which, W _mt and b _mt are parameters that need to be learned, and the corresponding initial values need to be determined first, X _mt represents the embedded feature of the first sample object, Y _{mt, t} represents the determined translation word Y _mt, The probability of another translated word after _<t .

具体地可以采用如下公式2-1来表示，其中，基于注意力机制具体为多头自注意机制，其输入为已确定的单语言词Y_ms,<t的嵌入特征y2；第二支路的解码器MS用于根据第二历史编码后特征

及针对第二样本对象得到的编码后特征

确定一个交互表示特征O_ms,t，具体可以通过如下公式2-2来表示，其中，可以在FFN网络中采用多头自注意机制来确定交互表示特征O_ms,t，这里t为某一个单语言词；第二支路的概率输出用于通过softmax函数输出第二样本对象的单语言摘要信息(包括多个单语言词)的概率p_mt，具体可以通过如下公式2-3来表示，其中，W_ms和b_ms是需要学习的参数，需要先确定相应的初始值，X_ms表示第二样本对象的嵌入特征，Y_ms,t表示已确定的单语言词Y_ms,<t之后另一单语言词的概率。The attention module of the second branch is used to encode the determined monolingual word Y _ms,<t based on the attention mechanism to obtain the second historical encoded feature

Specifically, it can be represented by the following formula 2-1, wherein, the attention-based mechanism is specifically a multi-head self-attention mechanism, and its input is the embedded feature y2 of the determined single-language word Y _ms,<t ; the decoding of the second branch The device MS is used to encode the features according to the second history

and the encoded features obtained for the second sample object

Determining an interactive representation feature O _ms,t can be expressed by the following formula 2-2, in which the multi-head self-attention mechanism can be used in the FFN network to determine the interactive representation feature O _ms,t , where t is a certain single language word; the probability output of the second branch is used to output the probability p _mt of the monolingual summary information (including multiple monolingual words) of the second sample object through the softmax function, which can be specifically expressed by the following formula 2-3, wherein, W _ms and b _ms are parameters that need to be learned, and the corresponding initial values need to be determined first, X _ms represents the embedded feature of the second sample object, Y _{ms, t} represents the determined single-language word Y _{ms, after < t} another single Probability of language words.

具体地可以采用如下公式3-1来表示，其中，基于注意力机制具体为多头自注意机制，其输入为已确定的跨语言词Y_cls,<t的嵌入特征y3；第三支路的解码器CLS用于根据第三历史编码后特征

及针对第三样本对象得到的编码后特征

确定一个交互表示特征O_cls,t，具体可以通过如下公式3-2来表示，其中，可以在FFN网络中采用多头自注意机制来确定交互表示特征O_cls,t，这里t为某一个跨语言词；第三支路的概率输出用于通过softmax函数输出第三样本对象的跨语言摘要信息(包括多个跨语言词)的概率p_cls，具体可以通过如下公式3-3来表示，其中，W_cls和b_cls是需要学习的参数，需要先确定相应的初始值，X_cls表示第三样本对象的嵌入特征，Y_cls,t表示已确定的跨语言词Y_cls,<t之后另一跨语言词的概率。The attention module of the third branch is used to encode the determined cross-language word Y _cls,<t based on the attention mechanism to obtain the third historical encoded feature

Specifically, it can be represented by the following formula 3-1, wherein, the attention-based mechanism is specifically a multi-head self-attention mechanism, and its input is the embedded feature y3 of the determined cross-language word Y _cls,<t ; the decoding of the third branch The device CLS is used to encode the features according to the third history

and the encoded features obtained for the third sample object

Determining an interactive representation feature O _cls,t can be specifically represented by the following formula 3-2, in which the multi-head self-attention mechanism can be used in the FFN network to determine the interactive representation feature O _cls,t , where t is a certain cross-language word; the probability output of the third branch is used to output the probability p _cls of the cross-language summary information (including multiple cross-language words) of the third sample object through the softmax function, which can be specifically expressed by the following formula 3-3, wherein, W _cls and b _cls are parameters that need to be learned, and the corresponding initial values need to be determined first, X _cls represents the embedded feature of the third sample object, Y _{cls, t} represents the determined cross-language word Y _{cls, <t} after another cross Probability of language words.

Step 303: Determine the translation information of the first sample object, the single-language summary information of the second sample object, and the cross-language summary information of the third sample object through the initial training model.

Step 304: Calculate the first loss function according to the translation information of the first sample object determined by the initial training model and the translation labels in the training sample, which can be specifically expressed by the following formula 4-1; the second sample object determined according to the initial training model The second loss function is calculated based on the single-language summary information and the single-language summary annotations in the training samples, which can be specifically expressed by the following formula 4-2; the cross-language summary information of the third sample object determined according to the initial training model and the cross-language summary information of the training sample The language summary annotation calculates the third loss function, which can be specifically expressed by the following formula 4-3.

Step 305: Calculate the overall loss function related to the initial training model according to the first loss function L _MT , the second loss function L _MS and the third loss function L _CLS . Specifically, it can be expressed by the following formula 5:

L=L _CLS + αL _MT + βL _MS (5)

Among them, α and β are the first weight value and the second weight value respectively, which can be adjusted dynamically. Specifically, when adjusting the first weight value, the following formula 6 can be used to adjust, and when adjusting the second weight value, The adjustment is made by the following formula 7:

α2=max(0,α1-d1), d1=α1*t2/T3 (6)

β2=max(0,β1-d2), d2=β1*t2/T4 (7)

Among them, α1 and α2 are the first weight values before and after adjustment, β1 and β2 are the second weight values before and after adjustment, t2 is the current count of training steps, T3 and T4 are the single-language summarization task and cross-language summarization, respectively The total number of training steps for the task. Among them, the number of training steps needs to be counted from 1 during the training process, and as the number of training steps increases, the first weight value and the second weight value will gradually decrease until the current count of the number of training steps and The total number of training steps is equal, that is, when t2=T3, the first weight value and the second weight value are both zero, so that in the process of training the cross-language summary model, while using the information of the translation task and the single-language summary task, it can also Making the cross-lingual summarization task dominant makes the resulting cross-language summarization model more accurate.

In step 306, the parameter values of the parameters in the initial training model are adjusted according to the overall loss function, and the final training model has been obtained.

Step 307, determine whether the parameter value adjusted in step 306 satisfies the preset stop condition, if so, take the parameter value adjusted in the above step 306 as the parameter value in the training model obtained by the final training, and continue to perform step 308; If it is not satisfied, return to the above-mentioned step 303.

In step 308, it is determined that the pre-trained cross-language abstract model may include the feature extraction module, the encoder and the decoder CLS of the third branch, the probability output and the attention module in the final training model, and the cross-language abstract obtained by training is used. Models are preset into the text processing system.

(2) When the text processing system initiates the process of obtaining the cross-language summary information of the target object, the pre-trained cross-language summary model can be called, and the cross-language summary information of the target object can be directly obtained through the cross-language summary model.

In the actual test, two cross-language summarization models are obtained by training using the existing method and the method of the embodiment of the present invention, one of which is a cross-language summarization model from Chinese to English, and the other is a cross-language summarization model from English to Chinese. The language summary model, after obtaining the cross-language summary information of the corresponding sample objects by using two cross-language summary models respectively, the matching rate between the calculated cross-language summary information and the actual cross-language summary information of the sample object (referred to as the correct matching rate) is as follows: 1, it can be seen that the cross-language summary information obtained by the cross-language summary model trained by the method of the embodiment of the present invention is more accurate:

Table 1

It can be seen that, in the process of training the cross-language summarization model in this embodiment, the realization information of three tasks (that is, the cross-language translation task, the single-language summarization task and the cross-language summarization task) is synthesized, which enhances the performance of the cross-language summarization model obtained by training. Performance, in particular, the trained cross-language summarization model is also more accurate in scenarios where training samples annotated by cross-language summaries are scarce.

The text processing method in the present invention is described below with another specific application example. The text processing system in the embodiment of the present invention is mainly a distributed system 100, and the distributed system may include a client 300 and a plurality of nodes 200 (access Any computing device in the network, such as a server, a user terminal), the client 300 and the node 200 are connected in the form of network communication.

Taking the distributed system as the blockchain system as an example, referring to FIG. 7 is an optional schematic structural diagram of the distributed system 100 provided in the embodiment of the present invention applied to the blockchain system. Any form of computing device (such as server, user terminal) and client 300 are formed, and a peer-to-peer (P2P, Peer To Peer) network is formed between nodes. The P2P protocol is a transmission control protocol (TCP, Transmission Control Protocol) Application layer protocol on top of the protocol. In a distributed system, any machine such as a server and a terminal can join to become a node, and a node includes a hardware layer, a middle layer, an operating system layer and an application layer.

Referring to the functions of each node in the blockchain system shown in Figure 7, the involved functions include:

1) Routing, a basic function that a node has to support communication between nodes.

In addition to the routing function, a node can also have the following functions:

2) Application, used to deploy in the blockchain, implement specific business according to actual business needs, record data related to the realization of functions to form record data, carry a digital signature in the record data to indicate the source of the task data, and send the record data To other nodes in the blockchain system, for other nodes to add the record data to the temporary block when verifying the source and integrity of the record data successfully.

For example, the business implemented by the application includes the code for implementing the cross-language summarization function, and the cross-language summarization function mainly includes:

Obtain the target object, call the pre-trained cross-language summary model; extract the cross-language summary information of the target object through the cross-language summary model; wherein, pre-train the cross-language summary model through the following steps: determining an initial training model, The initial training model includes a feature extraction module, a feature encoding module, and a post-coding processing module of three branches. The feature extraction module is used to extract the feature information of the sample object, and the feature encoding module is used to perform the feature information of the sample object. The encoded feature is obtained by encoding, and the encoding post-processing module of the first branch in the three branches is used to determine the translation information of the sample object according to the encoded feature, and the second branch in the three branches. The post-coding processing module is used to determine the monolingual abstract information of the sample object according to the post-coding feature, and the post-coding processing module of the third branch in the three branches is used to determine the sample based on the post-coding feature. Cross-language summary information of objects; determine training samples, the training samples include multiple first sample objects and their translation annotations, multiple second sample objects and their single-language summary annotations, and multiple third sample objects and their translation annotations. Cross-language summary annotation; train the cross-language summary model according to the initial training model and training samples.

3) Blockchain, including a series of blocks (Blocks) that follow each other in chronological order. Once a new block is added to the blockchain, it will not be removed. The block records the blockchain system. The record data submitted by the middle node.

8 is an optional schematic diagram of a block structure (Block Structure) provided by an embodiment of the present invention, and each block includes a hash value (the hash value of this block) of the transaction record stored in this block, and The hash value of the previous block, each block is connected by the hash value to form a blockchain. In addition, the block may also include information such as a timestamp when the block was generated. Blockchain (Blockchain) is essentially a decentralized database, which is a series of data blocks associated with cryptographic methods. Each data block contains relevant information to verify the validity of its information. (anti-counterfeiting) and generate the next block.

An embodiment of the present invention further provides a text processing system, the schematic diagram of which is shown in FIG. 9 , and may specifically include:

The calling unit 10 is used to obtain the target object and call the pre-trained cross-language summary model;

The abstract extraction unit 11 is configured to extract the cross-language abstract information of the target object through the object cross-language abstract model invoked by the invoking unit 10;

The text processing system also includes:

The training unit 12 is used to determine an initial training model, the initial training model includes a feature extraction module, a feature encoding module and a post-coding processing module of three branches, and the feature extraction module is used to extract the feature information of the sample object, the feature The encoding module is used to encode the feature information of the sample object to obtain the encoded feature, and the encoding post-processing module of the first branch in the three branches is used to determine the translation information of the sample object according to the encoded feature, The post-encoding processing module of the second branch of the three branches is used to determine the monolingual abstract information of the sample object according to the post-encoding characteristics, and the post-encoding processing of the third branch of the three branches The module is used to determine the cross-language summary information of the sample object according to the encoded features; determine the training sample, the training sample includes a plurality of first sample objects and their translation annotations, a plurality of second sample objects and their single language Abstract annotations and a plurality of third sample objects and their cross-language abstract annotations; the cross-language abstract model is trained according to the initial training model and the training samples. Then, the calling unit 10 will call the cross-language summary model trained by the training unit 12 .

The initial training model determined by the training unit 12 further includes attention modules corresponding to the three branches respectively; specifically, the translation information of the sample object includes a plurality of translation words, and the attention of the first branch The module is used to encode the determined translation word based on the attention mechanism to obtain the first historical post-coding feature, and the post-coding processing module of the first branch is used to obtain the post-coding feature according to the post-coding feature and the first historical post-coding feature. The feature determines another translation word after the determined translation word; the single-language summary information of the sample object includes a plurality of single-language words, and the attention module of the second branch is used to analyze the determined single-language word. The language word is encoded based on the attention mechanism to obtain the second historical post-encoding feature, and the post-encoding processing module of the second branch is used to determine the determined single feature according to the post-encoding feature and the second historical post-encoding feature. Another single language word after the language word; the cross-language summary information of the sample object includes a plurality of cross-language words, and the attention module of the third branch is used to perform attention-based attention on the determined cross-language words The encoding of the mechanism obtains the third historical post-encoding feature, and the post-encoding processing module of the third branch is used to determine another cross-language after the determined cross-language word according to the post-encoding feature and the third historical post-encoding feature word.

The attention modules corresponding to the three branches are used for encoding based on a single-head self-attention mechanism or a multi-head self-attention mechanism.

Further, when training the cross-language summary model according to the initial training model and the training samples, the training unit 12 is specifically configured to use the initial training model to respectively determine the translation information of each first sample object, the second The single-language summary information of the sample object and the cross-language summary information of the third sample object; the translation information, the single-language summary information and the cross-language summary information determined according to the initial training model, and the translation of the corresponding sample object in the training sample Labeling, single-language abstract labeling and cross-language abstract labeling, and adjusting the initial training model to obtain the final training model; it is determined that the pre-trained cross-language abstract model includes the feature extraction module and the feature encoding module in the final training model And the encoding post-processing module of the third branch.

Wherein, the training unit 12 determines the translation information, single-language summary information and cross-language summary information according to the initial training model, and the translation annotation, single-language summary annotation and cross-language summary annotation of corresponding sample objects in the training sample, When adjusting the initial training model, it is specifically used for the translation information, single-language summary information and cross-language summary information determined according to the initial training model, and the translation annotation, single-language summary annotation and cross-language summary information of the corresponding sample objects in the training sample Summary annotation, calculate the overall loss function related to the initial training model; adjust the parameter values of the parameters in the initial training model according to the overall loss function.

Wherein, the training unit 12 determines the translation information, single-language summary information and cross-language summary information according to the initial training model, and the translation annotation, single-language summary annotation and cross-language summary annotation of corresponding sample objects in the training sample, When calculating the overall loss function related to the initial training model, it is specifically used to calculate the translation information of the first sample object determined according to the initial training model and the translation annotation of the corresponding first sample object in the training sample, and calculate and The first loss function related to the encoding post-processing module of the first branch; according to the single-language summary information of the second sample object determined by the initial training model and the single-language summary annotation of the corresponding second sample object in the training sample, calculate The second loss function related to the post-coding processing module of the second branch; the cross-language summary information of the third sample object determined according to the initial training model and the cross-language summary annotation of the corresponding third sample object in the training sample , calculating a third loss function related to the post-encoding processing module of the third branch; calculating the overall loss function according to the first loss function, the second loss function and the third loss function.

Wherein, the training unit 12 calculates the overall loss function according to the first loss function, the second loss function and the third loss function, and is specifically used to determine the first weight value and the second weight value; The first product of the value and the first loss function, the second product of the second weight value and the second loss function, and the sum of the third loss function are used as the overall loss function. In this case, the text processing system may further include an adjustment unit 13 for determining the first weight value after adjustment, the first weight value before adjustment, the total number of training steps of the post-encoding processing module of the first branch, and The function calculation value between the current counts of the number of training steps; the second weight value after the adjustment is determined to be the second weight value before the adjustment, the number of training steps and the total number of training steps of the post-coding processing module of the second branch and The function evaluates between the current count of training steps.

Further, the training unit 12 is also used to adjust the parameter value of the parameter in the initial training model when the number of times of adjustment is equal to the preset number of times, or if the difference between the currently adjusted parameter value and the last adjusted parameter value is less than When a threshold value is reached, the adjustment of the parameter value is stopped.

It can be seen that, in the text processing system of this embodiment, the abstract extraction unit 11 will use a pre-trained cross-language abstract model to extract the cross-language abstract information of the target object, wherein the initial training determined in the process of pre-training the cross-lingual abstract model The model includes three branches of encoding post-processing modules, corresponding to three different tasks, namely determining translation information, single-language summary information and cross-language summary information, and these three tasks share the same feature encoding module and the same The feature extraction module, since the task of determining cross-language summary information can be an integration of the two sub-tasks of determining translation information and single-language summary information, uses the information to achieve these three tasks in the process of training the cross-language summary model, while considering When it comes to determining the overall task of cross-language summarization information and its sub-tasks, even when the data set of cross-language summarization (that is, the third sample object in the training sample) is small, the cross-language summarization model obtained by training is It is also more accurate when extracting cross-language summary information.

An embodiment of the present invention also provides a terminal device, the schematic diagram of which is shown in FIG. 10 , the terminal device may have relatively large differences due to different configurations or performance, and may include one or more central processing units (central processing units, CPUs) ) 20 (eg, one or more processors) and memory 21, one or more storage media 22 (eg, one or more mass storage devices) storing application programs 221 or data 222. Wherein, the memory 21 and the storage medium 22 may be short-term storage or persistent storage. The program stored in the storage medium 22 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the terminal device. Furthermore, the central processing unit 20 may be configured to communicate with the storage medium 22 to execute a series of instruction operations in the storage medium 22 on the terminal device.

Specifically, the application program 221 stored in the storage medium 22 includes a cross-language abstract application program, and the program may include the calling unit 10, the abstract extraction unit 11, the training unit 12 and the adjustment unit 13 in the above-mentioned text processing system. This is not repeated here. Furthermore, the central processing unit 20 may be configured to communicate with the storage medium 22 to execute a series of operations corresponding to the application program of the cross-language abstract stored in the storage medium 22 on the terminal device.

The terminal device may also include one or more power supplies 23, one or more wired or wireless network interfaces 24, one or more input and output interfaces 25, and/or, one or more operating systems 223, such as WindowsServerTM, Mac OS XTM , UnixTM, LinuxTM, FreeBSDTM and so on.

The steps performed by the text processing system described in the above method embodiments may be based on the structure of the terminal device shown in FIG. 10 .

Another aspect of the embodiments of the present invention further provides a computer-readable storage medium, where the computer-readable storage medium stores a plurality of computer programs, and the computer programs are adapted to be loaded by a processor and execute the text processing system as described above. text processing methods.

Another aspect of the embodiments of the present invention further provides a terminal device, including a processor and a memory; the memory is used to store a plurality of computer programs, and the computer programs are used to be loaded and executed by the processor as executed by the above text processing system The text processing method; the processor is used to implement each computer program in the plurality of computer programs.

According to one aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the text processing methods provided in the various optional implementations described above.

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium can include: Read only memory (ROM), random access memory (RAM), magnetic or optical disk, etc.

The text processing method, system, storage medium, and terminal device provided by the embodiments of the present invention have been described above in detail. The principles and implementations of the present invention are described with specific examples in this paper. The descriptions of the above embodiments are only It is used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific embodiments and application scope. The contents of the description should not be construed as limiting the present invention.

Claims (12)

1. A method of text processing, comprising:

acquiring a target object, and calling a pre-trained cross-language abstract model;

extracting cross-language abstract information of the target object through the cross-language abstract model;

wherein the cross-language abstract model is pre-trained by:

determining an initial training model, wherein the initial training model comprises a feature extraction module, a feature coding module and a coding post-processing module of three branches, the feature extraction module is used for extracting feature information of a sample object, the feature coding module is used for coding the feature information of the sample object to obtain coded features, the coding post-processing module of a first branch in the three branches is used for determining translation information of the sample object according to the coded features, the coding post-processing module of a second branch in the three branches is used for determining single-language abstract information of the sample object according to the coded features, and the coding post-processing module of a third branch in the three branches is used for determining cross-language abstract information of the sample object according to the coded features;

determining a training sample, wherein the training sample comprises a plurality of first sample objects and translation labels thereof, a plurality of second sample objects and single-language abstract labels thereof, and a plurality of third sample objects and cross-language abstract labels thereof;

and training the cross-language abstract model according to the initial training model and the training samples.

2. The method of claim 1, wherein the initial training model further comprises attention modules corresponding to the three branches, respectively;

the translation information of the sample object comprises a plurality of translation words, the attention module of the first branch is used for encoding the determined translation words based on an attention mechanism to obtain first historical encoded features, and the post-encoding processing module of the first branch is used for determining another translation word after the determined translation words according to the encoded features and the first historical encoded features;

the single-language abstract information of the sample object comprises a plurality of single-language words, the attention module of the second branch is used for coding the determined single-language words based on an attention mechanism to obtain second historical coded features, and the coded post-processing module of the second branch is used for determining another single-language word behind the determined single-language word according to the coded features and the second historical coded features;

the cross-language abstract information of the sample object comprises a plurality of cross-language words, the attention module of the third branch is used for coding the determined cross-language words based on an attention mechanism to obtain features after third history coding, and the coded post-processing module of the third branch is used for determining another cross-language word after the determined cross-language word according to the coded features and the features after third history coding.

3. The method of claim 2, wherein the attention modules corresponding to the three branches are used for encoding based on a single-head self-attention mechanism or based on a multi-head self-attention mechanism.

4. The method according to any one of claims 1 to 3, wherein the training of the cross-language digest model based on the initial training model and training samples specifically comprises:

respectively determining translation information of the first sample object, monolingual abstract information of the second sample object and cross-language abstract information of the third sample object through the initial training model;

adjusting the initial training model according to the translation information, the single-language abstract information and the cross-language abstract information determined by the initial training model, and the translation labels, the single-language abstract labels and the cross-language abstract labels of the corresponding sample objects in the training sample to obtain a final training model;

and determining that the pre-trained cross-language abstract model comprises a feature extraction module, a feature coding module and a coding post-processing module of a third branch in the final training model.

5. The method of claim 4, wherein the adjusting the initial training model based on the translation information, the monolingual abstract information, and the cross-language abstract information determined by the initial training model and the translation labels, the monolingual abstract labels, and the cross-language abstract labels of the corresponding sample objects in the training sample comprises:

calculating an overall loss function related to the initial training model according to the translation information, the single-language abstract information and the cross-language abstract information determined by the initial training model, and the translation label, the single-language abstract label and the cross-language abstract label of the corresponding sample object in the training sample;

and adjusting parameter values of parameters in the initial training model according to the overall loss function.

6. The method according to claim 5, wherein the calculating an overall loss function associated with the initial training model according to the translation information, the single-language abstract information and the cross-language abstract information determined by the initial training model and the translation labels, the single-language abstract labels and the cross-language abstract labels of the corresponding sample objects in the training sample comprises:

calculating a first loss function related to a coding post-processing module of the first branch according to the translation information of the first sample object determined by the initial training model and the translation label of the corresponding first sample object in the training sample;

calculating a second loss function related to the coding post-processing module of the second branch according to the monolingual abstract information of the second sample object determined by the initial training model and the monolingual abstract label of the corresponding second sample object in the training sample;

calculating a third loss function related to the coded post-processing module of the third branch according to the cross-language abstract information of the third sample object determined by the initial training model and the cross-language abstract label of the corresponding third sample object in the training sample;

and calculating the overall loss function according to the first loss function, the second loss function and the third loss function.

7. The method of claim 6, wherein said calculating the overall loss function from the first, second, and third loss functions comprises:

determining a first weight value and a second weight value;

taking a first product of the first weight value and a first loss function, a second product of a second weight value and a second loss function, and a sum of the third loss function as the overall loss function.

8. The method of claim 7, further comprising:

determining the adjusted first weight value as a function calculation value among the first weight value before adjustment, the total training step number of the coding post-processing module of the first branch and the current count of the training step number;

and determining the adjusted second weight value as a function calculation value between the second weight value before adjustment, the training step number, the total training step number of the coding post-processing module of the second branch and the current count of the training step number.

9. The method of claim 5, wherein the adjusting of the parameter value is stopped when the number of times of adjusting the parameter value of the parameter in the initial training model is equal to a preset number of times or if the difference between the currently adjusted parameter value and the last adjusted parameter value is less than a threshold value.

10. A text processing system, comprising:

the calling unit is used for acquiring a target object and calling a pre-trained cross-language abstract model;

the abstract extracting unit is used for extracting cross-language abstract information of the target object through the cross-language abstract model;

the text processing system further includes:

the training unit is used for determining an initial training model, the initial training model comprises a feature extraction module, a feature coding module and three branch coding post-processing modules, the feature extraction module is used for extracting feature information of a sample object, the feature coding module is used for coding the feature information of the sample object to obtain coded features, the coding post-processing module of a first branch in the three branches is used for determining translation information of the sample object according to the coded features, the coding post-processing module of a second branch in the three branches is used for determining single-language abstract information of the sample object according to the coded features, and the coding post-processing module of a third branch in the three branches is used for determining cross-language abstract information of the sample object according to the coded features; determining a training sample, wherein the training sample comprises a plurality of first sample objects and translation labels thereof, a plurality of second sample objects and single-language abstract labels thereof, and a plurality of third sample objects and cross-language abstract labels thereof; and training the cross-language abstract model according to the initial training model and the training samples.

11. A computer-readable storage medium, characterized in that it stores a plurality of computer programs adapted to be loaded by a processor and to execute the text processing method according to any one of claims 1 to 9.

12. A terminal device comprising a processor and a memory;

the memory is used for storing a plurality of computer programs, and the computer programs are used for being loaded by the processor and executing the text processing method according to any one of claims 1 to 9; the processor is configured to implement each of the plurality of computer programs.

Images