CN114254085A - Semantic annotation method and device and vehicle-mounted terminal equipment - Google Patents

Abstract

The embodiment of the invention discloses a semantic annotation method, a semantic annotation device and vehicle-mounted terminal equipment. The method comprises the following steps: obtaining a corpus to be labeled; marking the linguistic data to be marked according to the selectable marking quantity of the linguistic data to be marked; and according to the marking result, selecting a matched semantic marking strategy to carry out semantic marking on the corpus to be marked. According to the technical scheme, different semantic annotation strategies are adopted for semantic annotation according to the complexity of the linguistic data to be annotated, so that the human resource cost investment of semantic annotation is saved, and the efficiency of semantic annotation is improved.

Description

Semantic annotation method, device and vehicle terminal equipment

technical field

Embodiments of the present invention relate to the technical field of information processing, and in particular, to a semantic labeling method, device, and vehicle-mounted terminal device.

Background technique

In the field of intelligent dialogue, NLU (Natural Language Understanding) semantic understanding usually refers to the representation of unstructured text information with structured semantics. The structured semantics include: domain, intent ), attribute slot (slot).

At present, there are two methods of NLU semantic annotation: manual annotation and machine annotation. Since the complexity of the corpus is different, the complexity of semantic annotation is also different, which may lead to the problem of inaccurate semantic annotation. Therefore, how to improve the quality of NLU semantic annotation and reduce the cost of manual annotation is an urgent problem to be solved.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a semantic labeling method, an apparatus, and a vehicle-mounted terminal device, so as to improve the efficiency of NLU semantic labeling and reduce manual labeling costs.

In a first aspect, an embodiment of the present invention provides a semantic annotation method, including:

Get the corpus to be labeled;

Marking the to-be-annotated corpus according to the optional label quantity of the to-be-labeled corpus;

According to the labeling result, a matching semantic labeling strategy is selected to perform semantic labeling on the corpus to be labelled.

In a second aspect, an embodiment of the present invention further provides a semantic annotation device, including:

The corpus acquisition module is set to acquire the corpus to be marked;

A corpus marking module, configured to mark the corpus to be labeled according to the optional labeling quantity of the corpus to be labeled;

The semantic labeling module is configured to select a matching semantic labeling strategy to perform semantic labeling on the to-be-labeled corpus according to the labeling result.

In a third aspect, an embodiment of the present invention further provides an in-vehicle terminal device, including a memory, a processor, and a computer program stored in the memory and running on the processor, where the processor executes the program to achieve the The semantic labeling method described in any embodiment of the invention is disclosed.

In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the semantic annotation method according to any embodiment of the present invention.

In the technical solution provided by the embodiment of the present invention, when the corpus to be labeled is obtained, the corpus to be labeled is marked according to the number of optional labels of the corpus to be labeled, and then, according to the labeling result, a semantic labeling strategy matching the labeling result is selected to be labeled. The corpus is semantically annotated. In the above technical solution, different semantic labeling strategies are used for semantic labeling according to the number of optional labels of the corpus to be labelled, thereby saving the human resource cost of semantic labeling and improving the efficiency of semantic labeling.

Description of drawings

1 is a flowchart of a semantic labeling method in Embodiment 1 of the present invention;

2 is a flowchart of a semantic labeling method in Embodiment 2 of the present invention;

3 is a flowchart of a semantic labeling method in Embodiment 3 of the present invention;

4 is a schematic structural diagram of a semantic labeling device in Embodiment 4 of the present invention;

FIG. 5 is a schematic diagram of a hardware structure of a vehicle-mounted terminal device in Embodiment 5 of the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention. In addition, it should be noted that, for the convenience of description, the drawings only show some but not all structures related to the present invention.

Before discussing the exemplary embodiments in greater detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart depicts various operations (or steps) as sequential processing, many of the operations may be performed in parallel, concurrently, or concurrently. In addition, the order of operations can be rearranged. The process may be terminated when its operation is complete, but may also have additional steps not included in the figures. The processing may correspond to a method, function, procedure, subroutine, subroutine, or the like.

Example 1

1 is a flowchart of a semantic labeling method provided in Embodiment 1 of the present invention, which can be applied to the situation where different semantic labeling strategies are used for labeling according to the difficulty of semantic labeling of the corpus to be labelled. It is executed by the provided semantic labeling device, which can be implemented in software and/or hardware, and can generally be integrated in vehicle-mounted terminal equipment.

As shown in Figure 1, the method of this embodiment specifically includes:

S110, acquiring the corpus to be marked.

Corpus refers to language materials, for example, "I want to navigate to XX Square", "I want to see the nearest KFC", and so on.

Annotation refers to processing the corpus, and marking various annotation information that can represent the characteristics of the language material on the corresponding language components, so as to facilitate the recognition and reading of the computer. For example, the NLU model can be used to annotate the corpus, and the domain, intent and attribute slot information of the corpus can be marked.

The corpus to be labeled refers to the corpus that needs to be labeled. Optionally, the corpus to be annotated is a corpus that needs to be annotated and applied in the dialogue scene.

S120: Mark the corpus to be labeled according to the optional labeled quantity of the corpus to be labeled.

The number of optional annotations, which refers to the number of optional semantic annotation results. Some corpora are clearly expressed and easy for semantic annotation, and their annotation complexity is low. Usually, only one definite semantic annotation result can be obtained. At this time, the number of optional annotations for the corpus to be annotated is one; some corpora have polysemy. In this case, different people may have different understandings of the intention to be expressed, and then multiple semantic annotation results may appear when semantically annotating it. The annotation complexity is high, making the corresponding corpus to be annotated. The number of selected labels is multiple.

Marking the corpus to be annotated according to the number of optional annotations refers to classifying the corpus to be annotated according to the obtained number of optional annotations of the corpus to be annotated. The classification mark can be used to mark the corpus to be annotated to distinguish the number of optional annotations. For example, corresponding to the corpus to be annotated with one optional annotation quantity, this kind of simple corpus can be marked as the first type of corpus, which can be marked by marking "0"; corpus, such complex corpus can be marked as the second type of corpus, which can be marked by marking "1".

Exemplarily, according to the optional labeling quantity of the corpus to be labeled, the corpus to be labeled is marked, including:

Input the corpus to be labeled into the corpus classification model obtained by pre-training, and obtain the domain classification result of the corpus to be labeled;

If the number of domain classification results is one, mark the corpus to be labeled as the first type of corpus;

If the number of domain classification results is multiple, the corpus to be labeled is marked as the second type of corpus.

Among them, the labeling complexity of the second type of corpus is greater than that of the first type of corpus.

The corpus classification model refers to the model used to classify the input corpus to be labeled. The corpus to be labeled is input into the corpus classification model, and the corpus classification model can output the domain classification result of the corpus to be labeled.

Input the corpus to be labeled into the corpus classification model obtained by pre-training, and then the domain classification result of the corpus to be labeled can be obtained. Among them, the output of the corpus classification model may only include the classification results of various fields. For example, input the corpus to be labeled "find a nearest route from Tianjin Station to Tianjin West Station" into the pre-trained corpus classification model, and the labeled corpus can be obtained. The domain classification result of the corpus is "map"; the output of the corpus classification model can also include the domain classification result and the number of domain classification results. For example, input the corpus to be labeled "Look at the scenic spots in Chongqing" into the pre-trained corpus classification Model, the domain classification results of the annotated corpus can be obtained as "map", "navigation" and "scenic spots", and the number of domain classification results is three.

After acquiring the corpus to be annotated, the domain information of the corpus to be annotated is analyzed as the result of the domain classification of the corpus to be annotated. In view of the fact that the corpus to be labeled may be monogrammatic or polygrammatic, the number of domain classification results can be one or more. The number of domain classification results of the corpus to be labeled is counted, and the corpus to be labeled is marked according to the statistical results.

The number of domain classification results is one corpus to be labeled, indicating that this corpus to be labeled can be directly semantically labeled, and it is not easy to produce ambiguity. It can be regarded as a simple corpus with low labeling complexity, so it can be marked as the first. Class corpus, which can be marked by marking "0"; the number of domain classification results is multiple to-be-labeled corpus, indicating that this to-be-labeled corpus has polysemy, and different people may have different understandings, so it can be identified as It is a complex corpus with a high degree of labeling complexity, so it can be marked as the second type of corpus, which can be marked by marking "1".

For example, when the corpus to be labeled is "I want to navigate to XX Square", after semantic analysis of the corpus to be labeled, it can be determined that the domain information of the corpus to be labeled is "navigation", that is, the corpus to be labeled The domain classification result is "navigation", and the number of domain classification results is one, then this corpus to be labeled can be marked as the first type of corpus. For another example, when the corpus to be labeled is "I want to see the nearest KFC", the corpus to be labeled can be understood as "I want to navigate to the nearest KFC", "I want to see the specific location of the nearest KFC" or "I want to see what the nearest KFCs are", and after semantic analysis of the corpus to be labeled, it can be determined that the domain classification result of the corpus to be labeled is "navigation", "map" or "restaurant", the domain The number of classification results is multiple, so that the to-be-labeled corpus can be marked as the second type of corpus.

S130. According to the labeling result, select a matching semantic labeling strategy to perform semantic labeling on the corpus to be labelled.

The semantic tagging strategy refers to the specific scheme used when semantically tagging the corpus to be tagged. Different semantic labeling strategies can be used to label the corpus to be labelled with different numbers of optional labels.

Optionally, the labor cost for a semantic annotation strategy with one optional annotation is lower than that for a semantic annotation strategy with multiple optional annotations.

For a simple corpus with one optional annotation, machine annotation can be used to avoid waste of human resource costs and reduce the efficiency of semantic annotation. For complex corpus with multiple optional annotations, manual annotation can be used to improve the quality and accuracy of semantic annotation.

According to the labeling result of the corpus to be labeled, the number of optional labels of the corpus to be labeled is distinguished, and then a semantic labeling strategy matching the number of optional labels to be labeled is selected for semantic labeling of the corpus to be labeled.

Since the semantic annotation of a simple corpus with one optional annotation can meet the requirements of semantic annotation accuracy without high human resource cost investment, and the complex corpus with multiple optional annotations will have polysemy. In this case, higher human resource costs are required. Therefore, it is possible to design a semantic annotation strategy and rationally allocate the investment direction of human resources, so that more human resources can be invested in the semantic annotation of complex corpus, thereby saving the investment of human resources.

Exemplarily, according to the labeling result, a matching semantic labeling strategy is selected to perform semantic labeling on the corpus to be labelled, including:

If the corpus to be labeled is marked as the first type of corpus, the preset NLU model is used to semantically label the corpus to be labeled, and the semantic target annotation result of the corpus to be labeled is obtained.

The NLU model refers to the natural language understanding model, which covers Chinese automatic word segmentation, part-of-speech tagging, syntactic analysis, natural language generation, information retrieval, information extraction, machine translation, text entailment and other fields. The input corpus or the result of speech recognition is processed to extract the user's dialogue intention and the information transmitted by the user. As a result of the output of the NLU model, a semantic annotation result can be obtained.

Optionally, the NLU model can be used to obtain the semantic annotation results of the corpus to be annotated represented by the domain, intent and attribute slot. Among them, domain refers to the same type of data or resources, and services provided around these data or resources, such as "restaurant", "hotel", "air ticket", "map", "navigation", etc.; intent, refers to It is the operation of the field data. For example, in the field of air tickets, there can be intentions such as "purchase tickets" and "refunds"; attribute slots are used to store the attributes of the field, such as "time" in the field of air tickets. , "Origin", "Destination" and other attributes. This embodiment does not specifically limit the adopted preset NLU model.

In this embodiment, the pre-set NLU model can be used to semantically label the corpus to be labeled as the first type of corpus, that is, the corpus of lower complexity with the optional label quantity is one, so as to obtain the corpus to be labeled. Semantic object annotation results.

In an optional embodiment, if the corpus to be marked is marked as the first type of corpus, the preset NLU model is used to semantically mark the corpus to be marked, and the semantic pre-marking result of the corpus to be marked is obtained; if the semantic pre-marking result Through manual verification, the semantic pre-labeling result is used as the semantic target labeling result of the corpus to be labelled.

In this embodiment, the pre-set NLU model can be used to semantically label the corpus to be labeled as the first type of corpus, that is, the corpus with a lower complexity that can be optionally labeled as one, to obtain the semantics of the corpus to be labeled Pre-annotated results. Then the quality inspector can manually check the obtained semantic pre-labeling result. If the semantic pre-labeling result is correct, it is determined that the semantic pre-labeling result is the semantic target labeling result of the corpus to be labeled. If the semantic pre-labeling result is wrong, the quality The inspector can manually modify the semantic pre-labeling results, and then use the modified semantic pre-labeling results as the semantic target labeling results of the corpus to be labeled. Using the NLU model to replace manual semantic annotation saves a lot of human resource costs and improves the efficiency of semantic annotation. By manually checking the semantic pre-annotation results obtained by the NLU model, the accuracy and quality of the semantic annotation results are also guaranteed. .

For example, when the to-be-labeled corpus marked as the first type of corpus is "I want to navigate to XX Square", then the preset NLU model can be used to semantically label the to-be-labeled corpus to obtain the semantic prediction of the to-be-labeled corpus. Then the quality inspector manually checks the obtained semantic pre-labeling results. If the domain information in the semantic pre-labeling results is "navigation" and the semantic pre-labeling results pass manual verification, it is determined that the semantic pre-labeling results are to be labeled. The semantic target labeling result of the corpus. If the domain information in the semantic pre-labeling result is not "navigation", such as "scenic spots", and the semantic pre-labeling result has not passed manual verification, the quality inspector can use the semantic pre-labeling result. Domain The information is manually modified to "navigation", and then the modified semantic pre-labeling result is used as the semantic target labeling result of the corpus to be labelled.

Exemplarily, according to the labeling result, a matching semantic labeling strategy is selected to perform semantic labeling on the corpus to be labelled, including:

If the to-be-labeled corpus is marked as the second type of corpus, multiple semantic pending labeling results of the to-be-labeled corpus are obtained; among the multiple semantically pending labeling results, the semantic target labeling results of the to-be-labeled corpus are determined.

Among them, the semantic undetermined annotation result may be the semantic annotation result corresponding to the corpus to be annotated obtained through computer technology, for example, it may be a plurality of semantic annotation results obtained through corpus slogan software, or it may be combined with historical semantic annotation data. The determined multiple semantic annotation results may also be multiple semantic annotation results determined based on big data analysis, or multiple semantic annotation results obtained based on multiple annotation models or software at the same time, etc.; semantic pending annotation results, also It may be a semantic annotation result corresponding to the corpus to be annotated obtained by manual annotation means, for example, it may be a plurality of semantic annotation results obtained by simultaneous semantic annotation performed by multiple annotators with annotation experience.

When the corpus to be labeled is marked as the second type of corpus, semantic labeling is performed on the corpus with a high degree of labeling complexity. This kind of corpus to be labeled may be a pun or a triad, so you can choose at least four The annotator also performs semantic annotation on the to-be-annotated corpus to avoid the situation where multiple semantically-to-be-determined annotation results obtained after semantic annotation of the same to-be-annotated corpus are different.

Among the multiple semantic pending annotation results obtained, one semantic pending annotation result is selected as the semantic target annotation result of the to-be-annotated corpus according to the set rule. The semantic target annotation results of the corpus.

Further, among the multiple semantic pending annotation results, determine the semantic target annotation results of the corpus to be annotated, including:

Count the domain information in each semantic pending annotation result; if there is only one domain information with the largest proportion, the domain information with the largest proportion will be used as the target domain information of the corpus to be annotated, and will match the target domain information. A semantic pending annotation result is used as the semantic target annotation result of the corpus to be annotated;

If there are more than one domain information with the largest proportion, the intent information in each semantic pending annotation result is counted;

If there is only one intent information with the largest proportion, a semantic pending annotation result that matches the domain information with the largest proportion and the intent information with the largest proportion will be used as the semantic target annotation result of the corpus to be annotated;

If there are more than one intent information with the largest proportion, the attribute slot information in each semantic pending annotation result will be counted, and will be compared with the domain information with the largest proportion, the intent information with the largest proportion, and the A semantic pending annotation result matching the attribute slot information is used as the semantic target annotation result of the corpus to be annotated.

Count the domain information in each semantic pending annotation result. If there is only one domain information with the largest proportion, the principle of minority obeying the majority is adopted, and the domain information with the largest proportion is selected as the target domain information of the corpus to be annotated. Finally, a semantic pending annotation result matching the determined target domain information is used as the semantic target annotation result of the to-be-annotated corpus; if there are multiple domain information with the largest proportion, the intent information in each semantic pending annotation result is further counted: If there is only one intent information with the largest proportion of the number, a semantic pending annotation result matching the domain information with the largest proportion and the intent information with the largest proportion will be used as the semantic target annotation result of the corpus to be annotated; There are more than the largest intention information, and then further statistics the attribute slot information in each semantic pending annotation result, and compares it with the domain information that accounts for the largest number, the intent information that accounts for the largest number, and the attribute slot that accounts for the largest amount. A semantic pending annotation result of the information matching is used as the semantic target annotation result of the corpus to be annotated.

Furthermore, in the case of using multiple manual annotations at the same time, it is possible to avoid disputes and discussions due to the polysemy of the corpus to be annotated, save the time for semantic annotation, and obtain multiple semantic pending annotation results at the same time. The annotation results are repeatedly modified. For example, when the domain information of a corpus to be annotated is marked as A by the semantic annotator, it is revised to B when the quality inspector conducts manual verification. The domain information in the labeling result will be changed back to A or to C, so that the domain information of a corpus to be labelled is modified by the quality inspector to the correct semantic labeling result and then corrected by the semantic labeler. This results in repeated modification of the corpus to be labeled and disagreements in determining the semantic annotation results. Therefore, multiple manual semantic annotations are performed at the same time, and the principle of minority obeying the majority is used to determine the semantic annotation results, and the machine learning model can freeze the final semantic annotation results. , the semantic annotation results can be determined more conveniently, and the accuracy and efficiency of the semantic annotation results can be improved.

For the corpus to be labeled with polysemy, among the domain information of the multiple semantic undetermined labeling results obtained by statistics, if there is only one domain information with the largest proportion, the principle of minority obeying the majority is adopted to determine the corpus to be labeled. The target domain information of the target domain, and then a semantic pending annotation result matching the target domain information is used as the semantic target annotation result of the to-be-annotated corpus, which can not only avoid the obtained multiple semantic pending annotation results from being different, but also can quickly determine and freeze the semantics. Label the results.

For example, when the annotated corpus marked as the second type of corpus is "I want to find KFC", it is assumed that the following four semantic undetermined annotation results are obtained:

The domain information in the four semantic pending annotation results obtained by statistics, among them, the domain information in two pending annotation results is determined to be "map", and the domain information in the other two pending annotation results are "navigation" and "restaurant" respectively. ", according to the principle of minority obeying the majority, the number of "maps" is the largest, and there is only one "map" in the field information with the largest number, so "map" is selected as the target field information of the corpus to be marked "I want to find KFC", Then, a semantic undetermined annotation result matching the target domain information of the "map" is used as the semantic object annotation result of the to-be-annotated corpus. Among them, any semantic pending annotation result that matches the target domain information of the "map" can be used as the semantic target annotation result of the to-be-annotated corpus, or the number of intent information and attribute slot information in each semantic pending annotation result can be accounted for A semantic undetermined annotation result that matches the target domain information of the "map" is selected as the semantic object annotation result of the to-be-annotated corpus.

It is worth pointing out that there may be situations in which the amount of multiple domain information accounts for the same and the largest amount in the multiple semantic pending annotation results, that is, it is impossible to uniquely determine a domain information with the largest amount. In this case, one of the multiple domain information with the same proportion and the largest number can be randomly selected as the target domain information corresponding to the corpus to be annotated, and then a semantic pending annotation result matching the target domain information can be used as the pending annotation. The semantic target annotation results of the corpus can also be further judged according to the intent information and attribute slot information in the semantic pending annotation results. The target domain information corresponding to the corpus to be annotated is determined in the information, and then a semantic pending annotation result matching the target domain information is used as the semantic target annotation result of the corpus to be annotated.

For example, when the labeled corpus marked as the second type of corpus is "check the nearby train station", it is assumed that the following five semantic undetermined annotation results are obtained:

Label serial number corpus to be labeled Domain information in the annotation results one Check out the nearest train station map two Check out the nearest train station navigation three Check out the nearest train station map Four Check out the nearest train station TRAIN STATION five Check out the nearest train station TRAIN STATION

The field information in the five semantic undetermined annotation results obtained by statistics, among which two annotators each marked the to-be-annotated corpus "check the nearby train station" as two fields of "map" and "train station", so the number of There are two areas of information with the largest proportion. In this case, optionally, one domain information can be randomly selected as the target of the corpus to be labeled "check the nearby train stations" from the two domain information with the largest proportion of "map" and "train station" Domain information, and then any semantic pending annotation result that matches the target domain information can be used as the semantic target annotation result of the corpus to be annotated; it can also be further selected according to the intent information in the semantic pending annotation result. If further analysis determines the proportion of data If there is only one largest intent information, a semantic pending annotation result that matches the domain information with the largest proportion and the intent information with the largest proportion can be used as the semantic target annotation result of the corpus to be annotated, assuming the following:

Among them, the intent information in the two annotation results is "ride by car", and the intent information in the other two annotation results are "planned path" and "display location" respectively. According to the principle of minority obeying the majority, the intent information is "ride by car". The number of "car" is the largest and only one exists, so a semantic pending annotation result matching the domain information "train station" with the largest proportion and the intent information "ride" with the largest proportion is selected as the to-be-annotated corpus" Check out the semantic object annotation results for "Nearby Train Station".

It is worth pointing out that if there are multiple intent information with the largest number, for example, when the corpus to be annotated is labeled as “map” and “train station”, two annotators will each label the results. The intent information is "ride" and "path planning", then further statistics the attribute slot information in each semantic pending annotation result: if the attributes corresponding to the two intent information of "ride" and "path planning" If the proportion of the slot information is the same and the largest, then the attribute slot information with the largest proportion can be arbitrarily selected by random selection, and a semantic pending annotation result matching the attribute slot information can be used as the semantic target of the corpus to be annotated. Annotation results; if the proportion of attribute slot information corresponding to the two intent information of "ride" and "path planning" is different, the domain information with the largest proportion, the intent information with the largest proportion, and A semantic pending annotation result matched with the largest number of attribute slot information is used as the semantic target annotation result of the to-be-annotated corpus.

In the technical solution provided by the embodiment of the present invention, when the corpus to be labeled is acquired, the corpus to be labeled can be labeled according to the number of optional labels of the corpus to be labeled, and then, according to the labeling result, a semantic labeling strategy that matches the labeling result is selected to be treated. The annotation corpus is semantically annotated. Different semantic annotation strategies are matched according to the corpus to be annotated with different optional annotation quantities, which saves the cost of human resources, improves the efficiency of semantic annotation and the accuracy and quality of semantic annotation results.

Embodiment 2

FIG. 2 is a flowchart of a semantic labeling method provided in Embodiment 2 of the present invention. This embodiment is embodied on the basis of the above-mentioned embodiment, wherein, after inputting the corpus to be labeled into the pre-trained corpus classification model, the obtained Before the domain classification result of the corpus to be annotated, it can also include:

Obtain at least two sample corpora with domain classification results;

Corresponding the sample corpus and the domain classification results attached to the sample corpus as a set of training sample data;

At least two sets of training sample data are used to train the machine learning model to generate a corpus classification model. .

As shown in Figure 2, the method of this embodiment specifically includes:

S210: Acquire at least two sample corpora with field classification results.

For example, the sample corpus is "I want to listen to Zhou Moumou's song", and its accompanying field classification result is "music"; for another example, the sample corpus is "I want to watch KFC", and its accompanying field classification result is "map, Navigation, restaurants"; etc.

S220. Corresponding the sample corpus and the domain classification result attached to the sample corpus as a set of training sample data.

According to the example in step S210, the sample corpus "I want to listen to Zhou XX's song" and the domain classification result "music" can be used as a set of training sample data, and the sample corpus "I want to watch KFC" and the domain classification result " Map, Navigation, Restaurant” corresponds to a set of training sample data.

S230. Use at least two sets of training sample data to train the machine learning model to generate a corpus classification model.

At least two sets of training sample data are used to train the machine learning model to generate a corpus classification model. The machine learning model may be an NLU model, and a corpus classification model is generated by training the NLU model. The machine learning model can also be other models that can implement corpus classification, for example, the RCNN (Recurrent ConvolutionalNeural Networks, recurrent convolutional neural network) model, BERT (Bidirectional EncoderRepresentations from Transformer, Transformer-based bidirectional encoding representation) model, etc. The embodiment does not specifically limit this.

The machine learning model is trained and learned, so that it has the ability to classify the field of the labeled corpus. The sample corpus and its accompanying field classification results are used as a set of training sample data, and at least two sets of training sample data are used to train and learn the machine learning model to generate a corpus classification model.

For example, when training a corpus classification model, two sample corpora with domain classification results can be input: "I want to listen to Zhou Moumou's song - Category: Music" and "I want to navigate to Wudaokou - Category: Navigation ”, take these two sample corpora and their accompanying domain classification results as two sets of training sample data, in addition, you can also input some sample corpora with polysemy and accompanying domain classification results: “Look at Zhou Moumou— —Category: music, video, encyclopedia” and “I want to see KFC-map, navigation, restaurant”, take these two sample corpora and their accompanying domain classification results as another two sets of training sample data, and then use these two sets of The training sample data is used to train the machine learning model. When the training is completed, the model can classify the domain of the corpus of "I want to listen + [a singer]'s song" as "music", and "I want to see the navigation to The domain classification of the corpus of "+[a place name]" is judged as "navigation"; the domain classification of the corpus of "Look at + [a singer]" is judged to be "music", "video" and "encyclopedia", and the "I" The domain classification of the corpus that I want to see + [restaurant name]" is judged as "map", "navigation" and "restaurant".

S240. Obtain the corpus to be marked.

S250, input the corpus to be labeled into the corpus classification model obtained by training, and obtain the domain classification result of the corpus to be labeled.

Use the corpus classification model generated by training to determine the domain classification results of the corpus to be labeled, and then mark the corpus to be labeled according to the number of domain classification results to distinguish the number of optional labels for the corpus to be labeled, and then select different semantics in a targeted manner The annotation strategy processes different corpora, concentrates more human resources in the semantic annotation of complex samples with more than one optional annotation, saves the investment of human resources cost, and improves the accuracy and quality of semantic annotation. Moreover, in the above technical solution, the way of distinguishing the corpus to be marked is simple and easy to implement, with high accuracy, and no labor cost is required.

S260: Determine whether the number of field classification results is one, if yes, execute S270, otherwise, execute S2100.

S270, marking the corpus to be marked as the first type of corpus.

When the field classification result of the corpus to be labeled is one, the number of optional annotations of the corpus to be labeled is one, the complexity of the corpus to be labeled is relatively simple, and the corpus to be labeled is marked as the first type of corpus.

S280. Use a preset NLU model to perform semantic annotation on the corpus to be labeled, and obtain a semantic pre-labeling result of the corpus to be labeled.

S290. If the semantic pre-labeling result passes manual inspection, the semantic pre-labeling result is used as the semantic target labeling result of the corpus to be labelled.

When the pre-labeled corpus is semantically labeled using the preset NLU model, and the semantic pre-labeling result of the to-be-labeled corpus is obtained, the domain information in the obtained pre-labeling result can be manually checked, compared to the way of manual labeling. The annotation corpus is semantically annotated, and the semantic pre-annotation results obtained by the NLU model are manually checked, which can save time and human resource costs, and make it easier to ensure the accuracy and quality of the semantic annotation results. Optionally, if the semantic pre-labeling result does not pass manual verification, manually modify the semantic pre-labeling result, and then use the modified semantic pre-labeling result as the semantic target labeling result of the corpus to be labelled.

S2100. Mark the corpus to be marked as the second type of corpus.

When there are multiple domain classification results of the corpus to be labeled, the number of optional annotations of the corpus to be labeled is more than one, and the complexity of the corpus to be labeled is relatively complex, and the corpus to be labeled is marked as the second type of corpus.

S2110. Acquire multiple semantic pending annotation results of the corpus to be annotated.

S2120. Among the multiple semantic pending annotation results, determine the semantic target annotation results of the corpus to be annotated.

For details that are not described in this embodiment, please refer to the foregoing embodiments, which will not be repeated here.

In the above technical solution, by training the model, a corpus classification model capable of classifying the corpus to be labeled can be obtained, and then the corpus to be labeled is input into the corpus classification model obtained after training, and the domain classification result of the corpus to be labeled can be obtained. , mark the corpus to be labeled according to the number of domain classification results, and then select a matching semantic labeling strategy to perform semantic labeling on the corpus to be labeled according to the labeling results of the corpus to be labeled. In the above technical solution, the corpus to be labeled is divided into two categories according to the number of domain classification results, and different semantic labeling strategies are used to label the two kinds of corpus respectively, so as to avoid the manual labeling method for all the corpus to be labeled. Semantic annotation saves the cost of human resources, improves the efficiency of semantic annotation and the accuracy of semantic annotation results, and ensures the quality of semantic annotation results.

Embodiment 3

Fig. 3 is a flowchart of a semantic labeling method provided in Embodiment 3 of the present invention, and this embodiment provides a specific implementation manner.

As shown in Figure 3, the method of this embodiment specifically includes:

S310. Acquire the corpus to be marked.

S320. Input the corpus to be labeled into the corpus classification model obtained by pre-training, and obtain the domain classification result of the corpus to be labeled.

Input the corpus to be labeled into the pre-trained corpus classification model, and the corpus classification model outputs the domain classification result of the corpus to be labeled. The number of domain classification results can identify the labeling complexity of the corpus to be labeled. Optionally, when the number of domain classification results is 1, it can be determined that the labeling complexity of the corpus to be labeled is low. When the number of domain classification results When there are more than one, it can be determined that the annotation complexity of the corpus to be annotated is relatively high.

S330. Determine whether the number of field classification results is one, if so, execute S340, otherwise, execute S390.

S340, marking the corpus to be marked as 0.

Among them, the corpus to be annotated marked as 0 belongs to the first type of corpus, that is, the corpus of lower complexity with the number of optional annotations marked as one.

S350. Use a preset NLU model to perform semantic annotation on the corpus to be labeled, and obtain a semantic pre-labeling result of the corpus to be labeled.

Since the marking complexity of the corpus to be annotated marked as 0 is low, the semantic annotation can be completed with the help of the model, and then the accuracy and quality of the semantic annotation results can be achieved by manual verification. Therefore, the pre-labeled corpus marked as 0 is semantically labeled using the preset NLU model, the semantic pre-labeling result of the corpus to be labeled is obtained, and manual verification is performed.

S360: Determine whether the semantic pre-labeling result has passed the manual inspection, if yes, execute S370, and if not, execute S380.

S370. Use the semantic pre-labeling result as the semantic target labeling result of the corpus to be labelled.

S380. Obtain the artificially modified semantic labeling result as the semantic target labeling result of the corpus to be labelled.

S390, marking the corpus to be marked as 1.

Among them, the corpus to be annotated marked as 1 belongs to the second type of corpus, that is, the corpus with a relatively high degree of complexity with multiple optional annotations.

S3100. Acquire multiple semantic pending annotation results of the corpus to be annotated.

Due to the high degree of marking complexity of the corpus to be marked marked as 1, there will be polysemy, ambiguity will easily occur in semantic marking, and multiple semantic marking results may be obtained, so multiple manual semantics can be used at the same time. Labeled strategy. Therefore, multi-artificial semantic annotation is performed on the corpus to be annotated marked as 1, and multiple semantic undetermined annotation results of the corpus to be annotated are obtained.

S3110. Count the domain information in each semantic pending annotation result.

Count the proportion of the domain information in each semantic pending annotation result in the obtained multiple semantic pending annotation results.

S3120: Determine whether there is only one domain information with the largest proportion of the number, if yes, execute S3130, otherwise, execute S3140.

S3130. Use the domain information with the largest proportion as the target domain information of the corpus to be labeled, and use a semantic pending labeling result matching the target domain information as the semantic target labeling result of the corpus to be labeled.

Count the domain information in each semantic pending annotation result. If there is only one domain information with the largest proportion, adopt the principle of minority obeying the majority, and select the domain information with the largest proportion as the target domain information of the corpus to be annotated. Freeze the determined target domain information to avoid ambiguity when determining the semantic annotation results of the corpus to be annotated and the situation where the annotator and the quality inspector repeatedly modify the same corpus to be annotated.

A semantic pending annotation result matched according to the determined target domain information is used as the semantic target annotation result of the corpus to be annotated, which can quickly lock the correct semantic annotation result of the to-be-annotated semantics and improve the efficiency of semantic annotation.

S3140. Count the intent information in each semantic pending annotation result.

If there are more than one domain information with the largest proportion, the intent information in each semantic pending annotation result is further counted.

S3150: Determine whether there is only one intent information with the largest proportion, if so, execute S3160; otherwise, execute S3170.

S3160. Use a semantic pending annotation result matching the domain information with the largest quantity and the intention information with the largest quantity as the semantic target annotation result of the corpus to be annotated.

If it is judged that there is one intent information with the largest proportion, according to the principle that the minority obeys the majority, a semantic pending annotation result that matches the domain information with the largest proportion and the intent information with the largest proportion is used as the semantic target of the corpus to be annotated. Label the results.

S3170: Count the attribute slot information in each semantic pending annotation result, and use a semantic pending annotation result matching the domain information with the largest proportion, the intent information with the largest proportion, and the attribute slot information with the largest proportion as the semantic undetermined annotation result. The semantic target annotation results of the corpus to be annotated.

If it is judged that there are more than one intent information with the largest proportion, then the attribute slot information in each semantic pending annotation result will be further counted, and will be compared with the domain information with the largest proportion, the intent information with the largest proportion, and the number of A semantic pending annotation result matching the attribute slot information with the largest proportion is used as the semantic target annotation result of the corpus to be annotated. Among them, if there are more than one attribute slot information with the largest proportion, then one attribute slot information with the largest proportion can be arbitrarily selected by random selection, and a semantic pending annotation result matching the attribute slot information is used as the pending annotation. The semantic target annotation results of the corpus.

For details that are not described in this embodiment, please refer to the foregoing embodiments, which will not be repeated here.

In the above technical solution, the corpus to be labeled is input into a pre-trained corpus classification model, the number of domain classification results of the corpus to be labeled can be judged by the corpus classification model, and the input corpus to be labeled is divided into simple corpus marked as 0 and complex corpus marked as 1, in order to distinguish the labeling complexity of the corpus to be marked according to the marks 0 and 1, and use the preset NLU model for the simple corpus marked as 0 to semantically mark the corpus to be marked. Complex corpus uses multi-manual semantic annotation, rationally allocates manpower, saves the cost of human resources, and improves the accuracy of semantic annotation results.

Embodiment 4

4 is a schematic structural diagram of a semantic labeling device provided in Embodiment 4 of the present invention, which can be applied to the situation where different semantic labeling strategies are used for labeling according to the difficulty of semantic labeling of the corpus to be labelled. The device can use software and/or or hardware, and can generally be integrated in, for example, a processor of an in-vehicle terminal device.

As shown in FIG. 4 , the semantic labeling device specifically includes: a corpus acquisition module 410, a corpus labeling module 420, and a semantic labeling module 430. in,

The corpus acquisition module 410 is configured to acquire the corpus to be marked;

The corpus marking module 420 is configured to mark the corpus to be labeled according to the optional labeling quantity of the corpus to be labeled;

The semantic labeling module 430 is configured to select a matching semantic labeling strategy to perform semantic labeling on the corpus to be labelled according to the labeling result.

The semantic labeling device provided in this embodiment realizes matching different semantic labeling strategies for corpus to be labelled with different complexity, saves the cost input of human resources, improves the efficiency of semantic labeling and the accuracy of semantic labeling results. quality.

Further, the corpus marking module includes:

a domain classification result determination unit, configured to input the to-be-labeled corpus into a pre-trained corpus classification model to obtain the domain classification result of the to-be-labeled corpus;

The first corpus marking unit is configured to mark the corpus to be marked as the first type of corpus if the number of the field classification results is one;

The second corpus marking unit is configured to mark the to-be-labeled corpus as the second type of corpus if the number of the domain classification results is multiple.

Further, the above-mentioned device further includes: a corpus classification model generation module, configured to input the to-be-labeled corpus into a pre-trained corpus classification model, and obtain an additional domain classification result before obtaining the domain classification result of the to-be-labeled corpus. at least two sample corpora;

Corresponding to the sample corpus and the domain classification result attached to the sample corpus as a set of training sample data;

At least two sets of training sample data are used to train the machine learning model to generate a corpus classification model.

In an optional implementation manner, the semantic annotation module is specifically set as:

If the to-be-labeled corpus is marked as the first type of corpus, a preset natural language understanding NLU model is used to semantically label the to-be-labeled corpus, and a semantic target annotation result of the to-be-labeled corpus is obtained.

In another optional implementation manner, the semantic annotation module is specifically set as:

If the to-be-labeled corpus is marked as the second type of corpus, then multiple semantic pending labeling results of the to-be-labeled corpus are obtained;

Among the plurality of semantic pending annotation results, the semantic target annotation results of the to-be-annotated corpus are determined.

Further, the semantic annotation module is specifically set as:

Count the domain information in each of the semantic pending annotation results;

If there is only one domain information with the largest proportion, the domain information with the largest proportion is used as the target domain information of the to-be-annotated corpus, and a piece of the semantic to-be-determined information matching the target domain information is annotated The result is used as the semantic target labeling result of the corpus to be labelled;

If there are more than one domain information with the largest proportion, count the intent information in each of the semantic pending annotation results;

If there is only one piece of intent information with the largest proportion of the quantity, the one piece of the semantic pending annotation result that matches the domain information with the largest proportion of the quantity and the intent information with the largest proportion of the quantity is used as the corpus to be annotated. Semantic target annotation results;

If there are more than one intent information with the largest proportion, the attribute slot information in each of the semantic pending annotation results will be counted, and the domain information with the largest proportion of the number and the intent information with the largest proportion will be counted. and a piece of the semantic pending annotation result that matches the attribute slot information with the largest proportion as the semantic target annotation result of the to-be-annotated corpus.

The above-mentioned semantic labeling device can execute the semantic labeling method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the semantic labeling method.

Embodiment 5

FIG. 5 is a schematic diagram of the hardware structure of a vehicle-mounted terminal device according to Embodiment 5 of the present invention. As shown in FIG. 5 , the vehicle-mounted terminal device includes:

One or more processors 510, one processor 510 is taken as an example in FIG. 5;

memory 520;

The in-vehicle terminal device may further include: an input device 530 and an output device 540 .

The processor 510, the memory 520, the input device 530 and the output device 540 in the in-vehicle terminal device may be connected by a bus or in other ways, and the connection by a bus is taken as an example in FIG. 5 .

As a non-transitory computer-readable storage medium, the memory 520 can be used to store software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to a semantic labeling method in the embodiments of the present invention (for example, the accompanying drawings). The corpus acquisition module 410, the corpus tagging module 420 and the semantic tagging module 430 shown in 4). The processor 510 executes various functional applications and data processing of the in-vehicle terminal device by running the software programs, instructions and modules stored in the memory 520, that is, to implement a semantic labeling method of the above method embodiments.

The memory 520 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the computer device, and the like. Additionally, memory 520 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, memory 520 may optionally include memory located remotely from processor 510, which may be connected to the terminal device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The input device 530 may be used to receive input numerical or character information, and generate key signal input related to user setting and function control of the in-vehicle terminal device. The output device 540 may include a display device such as a display screen.

Embodiment 6

Embodiment 6 of the present invention also provides a storage medium containing computer-executable instructions, the computer-executable instructions being used to execute a semantic labeling method when executed by a computer processor, the method comprising: acquiring a corpus to be labeled; The optional label quantity of the to-be-labeled corpus is to mark the to-be-labeled corpus; according to the labeling result, a matching semantic labeling strategy is selected to perform semantic labeling on the to-be-labeled corpus.

Optionally, when executed by a computer processor, the computer-executable instructions may also be used to execute the technical solution of a semantic labeling method provided by any embodiment of the present invention.

From the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be realized by software and necessary general-purpose hardware, and of course can also be realized by hardware, but in many cases the former is a better embodiment . Based on such understanding, the technical solutions of the present invention can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products can be stored in a computer-readable storage medium, such as a floppy disk of a computer , read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), flash memory (FLASH), hard disk or CD, etc., including several instructions to make a computer device (which can be a personal computer, A server, or a network device, etc.) executes the methods described in the various embodiments of the present invention.

It is worth noting that, in the above embodiments of the semantic labeling device, the units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; The specific names of the functional units are only for the convenience of distinguishing from each other, and are not used to limit the protection scope of the present invention.

Note that the above are only preferred embodiments of the present invention and applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions can be made to those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and can also include more other equivalent embodiments without departing from the concept of the present invention. The scope is determined by the scope of the appended claims.

Claims (11)

1. a semantic labeling method, is characterized in that, comprises: Get the corpus to be labeled; Marking the to-be-annotated corpus according to the optional label quantity of the to-be-labeled corpus; According to the labeling result, a matching semantic labeling strategy is selected to perform semantic labeling on the corpus to be labelled. 2. The method according to claim 1, wherein, according to the optional labeling quantity of the corpus to be labeled, the corpus to be labeled is marked, comprising: Inputting the to-be-labeled corpus into a pre-trained corpus classification model to obtain a domain classification result of the to-be-labeled corpus; If the number of the domain classification results is one, marking the to-be-labeled corpus as the first type of corpus; If the number of the domain classification results is multiple, the to-be-labeled corpus is marked as the second type of corpus. 3. The method according to claim 2, characterized in that, before the corpus to be marked is input into a corpus classification model obtained by pre-training, and before the domain classification result of the corpus to be marked is obtained, it also comprises: Obtain at least two sample corpora with domain classification results; Corresponding the sample corpus and the domain classification result attached to the sample corpus as a set of training sample data; At least two sets of training sample data are used to train the machine learning model to generate a corpus classification model. 4. The method according to claim 2, wherein, according to the labeling result, selecting a matching semantic labeling strategy to perform semantic labeling on the to-be-labeled corpus, comprising: If the to-be-labeled corpus is marked as the first type of corpus, a preset natural language understanding NLU model is used to semantically label the to-be-labeled corpus, and a semantic target labeling result of the to-be-labeled corpus is obtained. 5. The method according to claim 2, wherein, according to the labeling result, selecting a matching semantic labeling strategy to perform semantic labeling on the to-be-labeled corpus, comprising: If the to-be-annotated corpus is marked as the second type of corpus, acquiring multiple semantic pending annotation results of the to-be-annotated corpus; Among the plurality of semantic pending annotation results, a semantic target annotation result of the to-be-annotated corpus is determined. 6. The method according to claim 5, wherein, among the plurality of semantic pending annotation results, determining the semantic target annotation results of the to-be-annotated corpus, comprising: Count the domain information in each of the semantic pending annotation results; If there is only one domain information with the largest proportion, the domain information with the largest proportion is used as the target domain information of the to-be-annotated corpus, and a piece of the semantic to-be-determined information matching the target domain information is annotated The result is used as the semantic target labeling result of the corpus to be labelled; If there is more than one domain information with the largest proportion, count the intent information in each of the semantic pending annotation results; If there is only one piece of intent information with the largest proportion of the quantity, the one piece of the semantic pending annotation result that matches the domain information with the largest proportion of the quantity and the intent information with the largest proportion of the quantity is used as the corpus to be annotated. Semantic target annotation results; If there are more than one intent information with the largest proportion, the attribute slot information in each of the semantic pending annotation results will be counted, and the domain information with the largest proportion of the number and the intent information with the largest proportion will be counted. and a piece of the semantic pending annotation result that matches the attribute slot information with the largest proportion as the semantic target annotation result of the to-be-annotated corpus. 7. A semantic labeling device, comprising: The corpus acquisition module is set to acquire the corpus to be marked; a corpus marking module, configured to mark the to-be-labeled corpus according to the optional label quantity of the to-be-labeled corpus; The semantic labeling module is configured to select a matching semantic labeling strategy to perform semantic labeling on the to-be-labeled corpus according to the labeling result. 8. The apparatus according to claim 7, wherein the corpus marking module comprises: a domain classification result determination unit, configured to input the to-be-labeled corpus into a pre-trained corpus classification model to obtain a domain classification result of the to-be-labeled corpus; a first corpus marking unit, configured to mark the to-be-labeled corpus as the first type of corpus if the number of the field classification results is one; The second corpus marking unit is configured to mark the to-be-labeled corpus as the second type of corpus if the number of the domain classification results is multiple; wherein, the labeling complexity of the second type of corpus is greater than that of the first type of corpus. A class of corpus. 9. The device according to claim 8, wherein the semantic labeling module is specifically set as: If the to-be-labeled corpus is marked as the first type of corpus, a preset natural language understanding NLU model is used to semantically label the to-be-labeled corpus, and a semantic target labeling result of the to-be-labeled corpus is obtained. 10. The device according to claim 8, wherein the semantic labeling module is specifically set to: If the to-be-annotated corpus is marked as the second type of corpus, acquiring multiple semantic pending annotation results of the to-be-annotated corpus; Among the plurality of semantic pending annotation results, a semantic target annotation result of the to-be-annotated corpus is determined. 11. An in-vehicle terminal device, comprising a memory, a processor and a computer program stored in the memory and running on the processor, characterized in that, when the processor executes the program, the program as claimed in claims 1 to 6 is implemented. any of the methods described.

Images