CN104142917A - A method and system for constructing a hierarchical semantic tree for language understanding - Google Patents

Abstract

A method and system for constructing a hierarchical semantic tree for language understanding. The method mainly includes the following steps: segmenting sentences and loading a semantic knowledge base; The hierarchy of nodes; generate a special node for the punctuation at the end of the sentence as the root node of the semantic tree; merge it according to the above-mentioned generated node information, identify the semantic edge block of the sentence, and hang the 0-level semantic edge as a child node on the root Node; loop through each of its child nodes until there are no low-level semantic edges, and hang on the child nodes as leaf nodes. In the absence of syntactic resources, the scheme only uses semantic information and word positions and collocations to obtain semantic structure trees, enabling computers to enter the deep semantic level of natural language, complete various processing of natural language on the basis of understanding, and realize It is the first step of natural language semantic understanding, which can be used for information retrieval, automatic summarization, machine translation, text classification and information filtering, etc.

Description

A method and system for constructing a hierarchical semantic tree for language understanding

technical field

The invention relates to the field of natural language processing, in particular to a method and a system for constructing a hierarchical semantic tree obtained by using semantic knowledge and the position and collocation of words.

Background technique

With the development of electronic information technology, digital information resources are widely used more and more. This requires that machines can also understand natural language, and complete various processing of natural language on the basis of "understanding", such as information retrieval, automatic summarization, machine translation, text classification, and information filtering. It can be seen that enabling computers to enter the semantic depth of natural language is a condition for achieving the above purpose. In order for a machine to understand the meaning of natural language, it must first understand the structure of natural language sentences. Sentence structure is a basic structure of natural language, generally including grammatical structure and semantic structure. In order to better describe the semantics of a sentence, using a sentence structure tree is a simple, clear and effective way. There are mainly two types of sentence structure trees: one is a syntactic structure tree, and the other is a semantic structure tree. The syntactic structure tree mainly includes phrase structure tree, dependency tree, etc. Its automatic construction is mainly based on syntactic annotation, and it is realized by a statistical method. The construction of such a syntactic structure tree does not use or seldom uses the semantic knowledge of words.

The construction of the semantic structure tree must use semantic knowledge. The construction of the semantic tree is under the guidance of the HNC (Concept Hierarchy Network) theory, in the absence of syntactic resources, only using semantic knowledge and word positions and collocations, so that the computer It can enter into the deep semantics of natural language, perform various processing of natural language on the basis of understanding, realize the first step of natural language semantic understanding, and create new information for subsequent applications in information retrieval, machine translation, information filtering, text classification, etc. condition.

In the Chinese patent document CN1606004A, a method and device for identifying semantic structures from text are disclosed, forming at least two candidate semantic structures, determining a semantic score for each candidate semantic structure based on the likelihood of the semantic structure, and also based on the word The position in the text and the position in the semantic structure of the semantic entities formed from the word determine a syntactic score for each semantic structure, the syntactic score and the semantic score are combined to select a semantic structure for at least a portion of the text. The schema of the entity is defined in this scheme, which includes semantic type and probability, Markov probability, and semantic rules. The acquisition of these semantic content requires training of large-scale data, which is highly dependent on the domain of text. Due to the complexity of the task, The effect obtained may not be ideal, and all subsequent operations depend on the result of this step, and the effect will be greatly reduced.

Contents of the invention

The technical problem to be solved by the present invention is that the method for identifying semantic structure in the prior art needs to train large-scale data and has a strong dependence on the domain of text, so a method and system for constructing a hierarchical semantic tree without training is proposed.

In order to solve the above technical problems, the present invention provides a method and system for constructing a hierarchical semantic tree for language understanding, comprising the following steps:

S1. Input the sentence to be processed, perform word segmentation on the sentence to be processed, and load the semantic knowledge of the word after word segmentation;

S2. According to the word segmentation result, identify the semantic node of the sentence;

S3. Using semantic knowledge and word positions and collocations to obtain the level of semantic nodes;

S4. Identify semantic edges at different levels in the sentence;

S5. Generate a hierarchical semantic tree according to the semantic edges of each level.

Preferably, in the step S1, when segmenting the sentence to be processed, the sentence to be processed is segmented according to the domain dictionary and the general dictionary.

Preferably, the semantic knowledge includes generalized concept classes of words and their subclasses, and the generalized concept classes of words include dynamic, static, objects, people, attributes, and logic.

Preferably, the process of "identifying the semantic node of the sentence according to the word segmentation result" in the step S2 includes:

For the word after word segmentation, if there is a logical concept in the semantic knowledge of the word, the word is marked as L, and if there is a dynamic concept in the semantic knowledge of the word, it is marked as V;

For all words marked as L or V, perform LV exclusion processing;

Mark all L tags according to their concept categories, and judge whether they have post-marks. If there are post-marks, mark the post-marked words as L1H, and generate semantic nodes according to all the above tags.

Preferably, the process of "identifying the semantic node of the sentence based on the word segmentation result" in step S2 further includes: taking the semantic node generated by the end punctuation as the root node.

Preferably, the process of "using semantic knowledge and word positions and collocations to obtain the hierarchy of semantic nodes" in the step S3 includes:

The default level of all L marks and v marks is recorded as 0, and when two of the above marks are adjacent, the level of the second mark is reduced by one level to -1.

Preferably, the process of "identifying semantic edges at different levels in the sentence" in step S4 includes

For all the semantic nodes marked V, perform core verb recognition and generate language chunks;

Generate chunks for all semantic nodes marked L;

Semantic edges are generated based on chunks.

Preferably, the process of performing core verb recognition includes:

exclude words that cannot form core verbs;

The rest of the words are given different weights according to the composition and the characteristics of the words themselves, and the core verbs are selected according to the ranking results and position information of the weights.

Preferably, the process of generating a hierarchical semantic tree according to the semantic edges of each level includes:

Select the root node;

Hang the high-level language blocks on the root node according to the order of the level, as child nodes;

Traverse all child nodes, and use all language blocks within the scope of each child node as child nodes of the child node until no new child nodes are generated.

A hierarchical semantic tree construction system corresponding to the hierarchical semantic tree construction method, comprising:

Preprocessing unit: input the sentence to be processed, segment the sentence to be processed, and load the semantic knowledge of the word after word segmentation;

The first sequence generation unit: according to the word segmentation result, identify the semantic node of the sentence; use the semantic knowledge and word position and collocation to obtain the level of the semantic node;

The second sequence generation unit: identify semantic edges at different levels in the sentence;

Hierarchical semantic tree generation unit: generate a hierarchical semantic tree according to the semantic edges of each level.

The above technical solution of the present invention has the following advantages compared with the prior art,

(1) The hierarchical semantic tree construction method described in this embodiment mainly includes the process of preprocessing, node recognition, semantic edge recognition, and semantic tree generation, and the hierarchical semantic tree can be obtained by outputting the above structure tree. In the scheme of constructing the hierarchical semantic tree in this embodiment, the analysis of the sentences is realized by using the rule method. In this solution, the scheduling of rules at different levels and stages is controlled through the identification of nodes and their levels, and the identification of semantic edges and their levels. Under the guidance of this principle, it is first necessary to classify the rules hierarchically. Each type of rule is only called in a fixed analysis level, and each rule only focuses on the analysis of linguistic phenomena in adjacent sentence strings, and does not need to take into account the judgment of the overall situation. , but to solve the rule compatibility problem through scheduling.

(2) The hierarchical semantic tree construction method in the present invention, in the absence of syntactic resources, only uses semantic information and word positions and the semantic structure tree obtained by collocation, so that the computer can enter the semantic deep layer of natural language, and in the process of understanding Based on the completion of various processing of natural language, the first step of semantic understanding of natural language is realized. Constructing a semantic tree can be widely used in the field of natural language processing, such as information retrieval, automatic summarization, machine translation, text classification, and information filtering. The semantic tree construction method in this embodiment has been applied to Chinese-English machine translation of patent documents, which significantly improves the readability and accuracy of patent document translations.

Description of drawings

In order to make the content of the present invention more easily understood, the present invention will be described in further detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein

Fig. 1 is the flow chart of the hierarchical semantic tree construction method of the present invention;

The node generation flowchart of Fig. 2 hierarchical semantic tree construction method of the present invention;

Fig. 3 is a flow chart of generating semantic edges of the hierarchical semantic tree construction method of the present invention;

Fig. 4, the result schematic diagram of an application example of the hierarchical semantic tree construction method of Fig. 5 of the present invention;

Fig. 6 is a structural block diagram of the hierarchical semantic tree construction system of the present invention.

Detailed ways

Example 1:

Provide a kind of hierarchical semantic tree construction method and system for language comprehension in the present embodiment, semantic tree is the semantic structure tree, is for a sentence in natural language, refers to the feature word block (core verb) in a sentence lexical chunk) and the semantic relationship between other chunks determined by it. For example, the characteristic chunk V in a sentence is a verb expressing action, which determines that there must be actor chunks, object chunks, and content chunks in this sentence. Only in this way can the semantics of the sentence be complete. Although one of the latter three can be omitted in a certain context, these four chunks are the necessary components for the complete semantics of a sentence, and they are also called main chunks. In contrast, auxiliary language chunks are not necessary components for the establishment of a sentence, but mainly express the way, means, way, condition, time, etc. of an action. Both the main chunk and the auxiliary chunk can be prompted by certain logical concepts, so it is possible to use the LV (logical concept and dynamic concept) criterion to identify the semantic structure of the sentence. The method for constructing a hierarchical semantic tree in this embodiment is to use the LV criterion to identify the main chunk and the auxiliary chunk of a sentence. This scheme can automatically divide sentences and be used in language translation, which can greatly improve machine translation. readability and accuracy.

The method for constructing a hierarchical semantic tree in this embodiment mainly includes: the sentence to be processed S110 undergoes preprocessing S120, node identification S130, semantic edge identification S140, and semantic tree generation S150 to obtain a semantic tree S160, as shown in the flowchart in Figure 1 As shown, it specifically includes the following steps:

S1. Input the sentence to be processed, perform word segmentation on the sentence to be processed, and load the semantic knowledge of the word after the word segmentation. When segmenting the sentence to be processed, the sentence to be processed is segmented according to the domain dictionary and the general dictionary.

S2. Identify the semantic nodes of the sentence according to the word segmentation result. It mainly includes the following process: for the word after word segmentation, if there is a function word meaning item in the semantic knowledge of the word, mark the word as L, if there is a verb meaning item in the semantic knowledge of the word, mark it as V; for all the words marked as L or V Words, perform LV exclusion processing; mark all L tags according to their concept categories, and judge whether they have post-marks, if there are post-marks, also mark the post-marked words, and generate semantic nodes according to all the above-mentioned tags.

The specific way of the above process is as follows:

LV recognition is carried out for each word. If there is a function word meaning item in the semantic knowledge of the word, the word is marked as L; if there is a verb meaning item in the semantic knowledge of the word, the word is marked as V. The semantic knowledge includes generalized concept categories of words and their subcategories (ie, concept categories), and the concept generalized concept categories of words include dynamic, static, objects, people, attributes and logic.

For all words marked as L or V, carry out LV exclusion processing, if there are words like "the" and "a" in front of the word, then cancel its L and V marks; if there are words like "the" behind the word words, cancel their L and V marks;

For all L labels, if the concept category of the node is l1, then its label is changed to L1; judge whether it has a post-label, in "when...time", "time" is the post-label of "when", and the post-label word, generate a tag labeled L1H; if the concept category of the node is l0, its tag is modified to L0.

Put all L tags (including L0, L1 and L1H) and V tags with position information to generate a semantic node and record it into a queue, which is called the first sequence. If more than one semantic node is generated on a word, they are all recorded in the first sequence.

S3. Using semantic knowledge and word positions to obtain the hierarchy of semantic nodes. First, the default levels of all L and v marks are recorded as 0, and when two of the above-mentioned marks are adjacent, the level of the second mark is reduced by one layer. details as follows:

For all semantic nodes in the first sequence, LV level identification is performed, and the default levels of all L marks and V marks are recorded as 0;

When two Ls are adjacent, that is, when L1L2 appears, the level of L2 is reduced by 1;

When L and V are adjacent, that is, when L1V2 appears, the level of V2 is reduced by 1;

When L and V are adjacent, that is, when V1L2 appears, the level of L2 is reduced by 1;

For period punctuation marks, a semantic node is generated, which is marked as SST, and recorded in the first sequence.

S4. Identify semantic edges at different levels in the sentence. Including: firstly, for all the semantic nodes marked V, the core verbs are identified to generate language chunks; then, for all the semantic nodes marked L, language chunks are generated; thus, semantic edges are generated according to the language chunks.

The specific method is as follows:

Generate a queue, called the second sequence;

For all the semantic nodes marked as V in the first sequence, carry out EG recognition, generate a language block, which is marked as CHK_EG, and add the language block to the second sequence;

For all semantic nodes marked as L in the first sequence, perform the following processing:

For all semantic nodes marked as L1, generate a language block, its mark is CHK_ABK, and its starting position is the starting position of the L1 node; judge whether there is L1H after the node, if so, the end position of the language block is L1H End position; if there is no L1H thereafter, then the end position of the language block is the starting position pos-1 of the next semantic node marked as L, the language block level is the level of the semantic node, and the language block is added to the second sequence;

For all semantic nodes marked as L0, generate a language block, its mark is CHK_L0, its starting position is the starting position of L0, its end position is the end position of L0, the level of language block is the level of semantic nodes, and the language block level is the level of semantic nodes. block joins the second sequence;

For all semantic nodes marked as L0, generate a language block, its mark is CHK_GBK, its starting position is the end position pos+1 of L0, and its end position is the next word block (its mark is CHK_EG or CHK_ABK or The initial position pos-1 of CHK_L0), the language chunk level is the level of semantic nodes, and the language chunk is added to the second sequence;

For the semantic node marked as SST in the first sequence, generate a chunk whose mark is CHK_SST, and add it to the second sequence. The chunks CHK_SST, CHK_ABK, CHK_EG, CHK_L0 obtained in this process are semantic edges.

In the above process, EG recognition refers to the recognition of core verbs. It mainly judges the weight of each dynamic concept as EG by designing a series of orderly weights. The process includes: first, exclude words that cannot form core verbs, Preliminarily exclude the words that may constitute EG in the sentence, including dynamic concept and static concept, logical concept, attribute and category, and different dynamic concept and category. Then, the rest of the words are given different weights according to the collocation and the characteristics of the words themselves, and the core verbs are selected according to the ranking results of the weights and the position information. That is to generate EG for all the remaining candidate words after exclusion, and assign different weights according to their composition or the characteristics of the words themselves, and comprehensively consider the weight sorting results and position information to select a suitable word as the EG of the sentence.

S5. Generate a hierarchical semantic tree according to the semantic edges of each level. First, select the root node; then, hang the high-level language blocks on the root node according to the order of the level as child nodes; finally, traverse all child nodes, and use all language blocks within the range of each child node as The child nodes of this child node until no new leaf nodes are generated.

The method for constructing a hierarchical semantic tree described in this embodiment mainly includes the following steps: carrying out word segmentation to a sentence and loading a semantic knowledge base; identifying all nodes and their levels of the sentence according to the LV rules and language rules; as the root node of the semantic tree; according to the node information generated above, it is merged, the semantic edge block of the statement is identified, and the 0-level semantic edge block is hung on the root node as a child node; each child node is traversed until there is no The low-level semantic edge block is attached to the child node as a leaf node. The hierarchical semantic tree can be obtained by outputting the above structure tree. In the scheme of constructing the hierarchical semantic tree in this embodiment, the analysis of the sentences is realized by using the rule method. One reason rule systems are questioned is that if the rule description is too simple, the results produced by the rules are either contradictory or insufficient to analyze the sentence. If you want to rely entirely on the rules to give accurate analysis results, you need each rule to be able to describe complex language phenomena, which makes the generalization of the rules poor, writing requires a lot of labor, and is not feasible. In order to solve this contradiction, this scheme controls the scheduling of rules at different levels and stages through the identification of nodes and their levels, and the identification of semantic edges and their levels. Under the guidance of this principle, it is first necessary to classify the rules hierarchically. Each type of rule is only called in a fixed analysis level, and each rule only focuses on the analysis of linguistic phenomena in adjacent sentence strings, and does not need to take into account the judgment of the overall situation. , but to solve the rule compatibility problem through scheduling. There are two strategies to be solved in this embodiment: first, avoid greedy matching of rules, make rule calling hierarchical, and call corresponding rules based on activation information at each level; The results generated by the rules are selected and synthesized. In this way, it not only reduces the rules that need to be matched, but also reduces the impact of the contradictions caused by different rules on the final analysis, so as to strengthen the control of rule invocation, and also makes it possible to build a rule-based hierarchical semantic tree.

The above construction of semantic tree is a semantic structure tree obtained under the guidance of concept hierarchy network theory and only using semantic information and language rules in the absence of syntactic resources, so that the computer can enter the deep semantic layer of natural language Various processing of natural language is completed on the computer, realizing the first step of semantic understanding of natural language. Constructing a semantic tree can be widely used in the field of natural language processing, such as information retrieval, automatic summarization, machine translation, text classification, and information filtering. The semantic tree construction method in this embodiment has been applied to Chinese-English machine translation of patent documents, which significantly improves the readability and accuracy of patent document translations.

Example 2:

Provide a concrete hierarchical semantic tree construction method in the present embodiment, the basic process of this scheme is also as shown in Figure 1, the hierarchical semantic tree construction method 100 starts in step S110 in the present embodiment and promptly imports the statement to be processed, then in step S110 In S120, the sentence to be processed is preprocessed, that is, the sentence to be processed is segmented according to the domain dictionary and the general dictionary, and the semantic knowledge of the word is loaded. The semantic knowledge mainly includes the generalized concept class of the word, namely V (dynamic), G (static), and W (thing), P (person), U (attribute), L (logic) six generalized concept categories and some subcategories under their overall planning; secondly, in step S130, identify the semantic nodes of the sentence and perform Distinguish, the first step is the result after word segmentation, adopt LV rule to identify all semantic nodes, the second step is to utilize semantic knowledge and word position, compare and judge the hierarchy of node; Again, identify the difference of this sentence in step S140 The semantic edge of the level, the recognition result of the semantic node at the clause level is recognized as the semantic edge of the clause level, and the recognition result of the semantic node at the language block level is recognized as the semantic edge of the language block level; then, in step S150 A hierarchical semantic tree is generated in the step S160, and the hierarchical semantic tree of the sentence to be processed is output in step S160 according to the recognition result of the semantic edge and according to the scheduling.

FIG. 2 is a schematic diagram illustrating node identification 300 . As shown in FIG. 2 , the entry S310 of node identification is the word segmentation result of the corpus to be processed. In step S311, words and punctuation are treated differently. For words, it is necessary to load semantic knowledge such as concept categories for each word. Semantic knowledge simply includes the following two aspects: word attribute, which includes generalized concept class GCC, concept category CC, LV attribute LV, morpheme QH, whether it is a pure V verb CHUNV; sentence class attribute, which includes generalized action sentence GXGY, subject block Quantity GBK_NUM, block expansion sentence EPER, GBK2 prototype sentence GBK2_YT, passive voice ALL_PASS, bidirectional relational sentence R0, comparison judgment sentence JD0. In particular, the classification and description of concept categories are shown in the table below:

The basic format of the knowledge base style is as follows:

word form

$Feature[Value]$

For example:

semiconductor element

$GCC[W]CC[pw]$

express

$CC[v]SC_GXY[GX]EPER[Y]GBK_NUM[3;4]SC_GBK1_PP[Y]$

Among them, GCC[W] indicates that the conceptual category of the entry ("semiconductor element") is object W, CC[pw] indicates that the concept category is artificial PW; CC[v] indicates that the entry ("representation") The concept category is a verb, SC_GXY[GX] means a generalized action sentence, EPER[Y] means a block extension sentence, GBK_NUM[3; 4] means a sentence with three main blocks or four main blocks, SC_GBK1_PP[Y] means that GBK1 must be person or living being.

For punctuation, a period generates a special semantic node, labeled SST, as the root node.

In step S330, "LV" recognition is carried out for each word, if there is a logic l concept in the semantic knowledge of the word, a semantic node is generated, and the word is marked as L, and if there is a dynamic v concept in the semantic knowledge of the word, etc., generate semantic node, the word is marked as V. At the same time, the words marked as V and L are respectively excluded by corresponding several disambiguation rules. For all words marked as V, the following two rules can be used as an example to carry out concurrent exclusion processing: for words marked as V, if there are words such as "of" and "a" in front of the word, then cancel its V mark; if there is such word as " of " behind this word, then cancel its V mark.

In step S340, for all L marks, if the concept category of the word is l1, then its mark is modified to L1; judge whether it has a back mark, if there is a back mark, generate a mark as L1H for the words of the back mark mark. For example, in Chinese "when...when", wherein, the concept category of "when" is l1, then its mark can be modified as L1, and "time" is the back mark of "when", and "time" is marked as L1H . If the concept category of the word is l0, then its mark is modified to L0, such as the word "把" in Chinese.

Step S350 is to identify all nodes.

In step S360, LV level identification is performed on all nodes, that is, the LEVEL information of the nodes is distinguished. For all semantic nodes in the first sequence, LV level recognition is performed, which includes the following operations: the default levels of all L marks and V marks are recorded as 0; when two Ls are adjacent, that is, when L1 and L2 appear, L2 minus 1, such as "take down the math book on the shelf", where "put" and "zai" are two adjacent L concepts, at this time, "put" is L1, and its level is 0 ; and "in" is L2, and its level is -1; when L and V are adjacent, that is, when L1 and V2 appear, the level of V2 is reduced by 1, such as "take down the math book on the shelf", Among them, "put" and "located" are two adjacent concepts of L and V. At this time, "put" is L1, and its level is 0; while "located" is V2, and its level is -1; when V and L When adjacent, that is, when V1 and L2 appear, the level of L2 is reduced by 1, such as "application and user-related modules", where "application" and "and" are two adjacent concepts of V and L, at this time, " Apply" is V1 and its level is 0, while "AND" is L2 and its level is -1.

In step S370, the result that obtains is exactly all nodes of this sentence that distinguishes LEVEL information, and writes down the first sequence, is referred to as the first sequence: put all L marks (comprising L0, L1 and L1H) and V marks, with The above position information in the sentence, as a semantic node, is recorded in the first sequence; if more than one semantic node is generated on a word, it is recorded in the first sequence; for punctuation marks, the generated semantic node SST is also recorded together first sequence.

FIG. 3 is a schematic diagram illustrating semantic edge identification 400 . As shown in Figure 4, the entry of semantic edge recognition is all nodes and their level LEVEL information.

First generate a queue, called the second sequence.

In step S410, perform EG recognition on all the semantic nodes marked as V in the first sequence, generate language chunks marked as CHK_EG, and add the language chunks to the second sequence.

For example, "the present invention can quickly access various devices that are docked with the electronic device 10." "Access, docking" is a semantic node marked V, and the two semantic nodes are weighted and downweighted by language rules, and "access" is passed The words "can, fast" are weighted, and "docking" is lowered by the "of" immediately after it. In this sentence, "visit" has a higher weight and is selected as the EG of the clause, marked as CHK_EG .

In step S420, perform the following processing on all semantic nodes marked as L in the first sequence:

For all semantic nodes marked as L1, generate a language block, its mark is CHK_ABK, and its starting position is the starting position of the L1 node; judge whether there is L1H after this node, if so, the end position of the language block is L1H If there is no L1H thereafter, the end position of the chunk is the starting position pos-1 of the next semantic node marked L, the chunk level is the level of the semantic node, and the chunk is added to the second sequence .

The following example illustrates the generation of CHK_ABK at the clause level: such as "the memory 130 can be separated in different ways.", wherein "with" is a semantic node marked L1, and there is no semantic node marked L1, L1H thereafter, then A block marked as CHK_ABK can be generated, its starting position is the starting position of the "with" semantic node, and the ending position is the starting position of CHK_EG and does not include this position, that is, the CHK_ABK block of this sentence is "with different mode"; as "the present invention removes weeds with a blade in a spiral rolling manner.", wherein "use" is a semantic node marked as L1, followed by a semantic node marked as L1 "with", then a mark can be generated The CHK_ABK language block, its starting position is the starting position of the "use" semantic node, and the ending position is the starting position of "yi" does not include this position, that is, the first CHK_ABK language block of the sentence is "use the blade" , as above, "in a spiral scrolling manner" is also a CHK_ABK of the sentence; another example is "presenting media content on the electronic device 10", wherein "in" is a semantic node marked L1, followed by a semantic node marked L1H If the node is “upper”, a chunk marked as CHK_ABK can be generated, its starting position is the starting position of the “with” semantic node, and the ending position is the position of the “inside” semantic node, that is, the CHK_ABK chunk of the sentence is "on electronic device 10". Both L1 and L1H in the above three examples are at the clause level, and their level is 0 by default, and the level of CHK_ABK is also 0. The following example illustrates the generation of CHK_ABK inside the language block. In the sentence "the user has the right to access the media content presented by the operating system 137." In the sentence, "access" is the CHK_EG of the sentence, and "the media content presented by the operating system 137" is A CHK_GBK language block, which is degenerated from the sentence "present media content through the operating system 137", wherein "present" is the V semantic node of the CHK_GBK language block, and CHK_EG can be generated, and its level is -1; where " "Pass" is a semantic node marked as L1, and its level is -1, then "pass through the operating system 137" can generate a language block marked as CHK_ABK. Similarly, the level of CHK_ABK generated inside the GBK chunk is -1.

For all semantic nodes marked as L0, generate a language block, its mark is CHK_L0, its starting position is the starting position of L0, its end position is the end position of L0, the level of language block is the level of semantic nodes, and the language block level is the level of semantic nodes. Blocks are added to the second sequence. The following example illustrates the generation of CHK_L0 at the clause level. In the sentence "the user enters the user name and/or password combination into the user interface 150 and/or the authentication device 70.", "by" is marked as L0, and its level information is 0, then it will generate a CHK_L0 block whose start position and end position are both L0; the following example illustrates the generation of CHK_L0 at the block level. ” is marked as L0, and its level information is -1, then it will generate a block marked as CHK_L0, and the start position and end position are both L0.

For the semantic node marked as SST in the first sequence, generate a chunk whose mark is CHK_SST, and add it to the second sequence.

In step S430, a language chunk is generated by using the relationship between all chunks CHK_L0 and CHK_ABK and CHK_EG, its mark is CHK_GBK, its starting position is the end position pos+1 of CHK_L0, and its ending position is the next The initial position pos-1 of a language chunk (its mark is CHK_ABK or CHK_EG), the language chunk level is the level of semantic nodes, and the language chunk is added to the second sequence. As in the above example "the user enters the user name and/or password combination into the user interface 150 and/or the authentication device 70.", "will" generate the language block CHK_L0, and "input to" generates the language block CHK_EG, then the "user", "User name and/or password combination" and "user interface 150 and/or authentication device 70" are CHK_GBK chunks.

In step S440, the obtained CHK_EG, CHK_ABK, CHK_L0, and CHK_SST are all semantic edges.

Determine to take SST as the root node, the first level CHK_EG, CHK_L0, CHK_ABK, CHK_GBK are its sub-nodes and hang under it, the sub-nodes of the CHK_EG, CHK_L0, CHK_ABK, CHK_GBK go sub-nodes of the second level are hung under it, with And so on until all are leaf nodes.

Example 3:

A specific application example is given in this embodiment, and FIG. 4 and FIG. 5 are schematic diagrams illustrating construction results of a hierarchical semantic tree of an example sentence. As shown in Figure 4, the sentence to be processed is "the web browser uses the Uniform Resource Locator to send the HTML request to the server controlled by the system.", and the semantic tree structure at the clause level is: GBK1 "web browser"+ABK" Use uniform resource locator "+L0" to send "+GBK2" HTML request "+EG" to "+GBK3" server controlled by the system", wherein CHK_SST (period) block is used as the root node. The semantic nodes of the first level are L1 (use), L0 (will), V (send to), and all three levels are 0; the semantic edges of the first level are CHK_ABK (use uniform resource locator), CHK_L0 (will ), CHK_EG (sent to), CHK_GBK (web browser, HTML request, remote server), the six language block levels are all 0, and it hangs out as a child node of the root node. According to the action of CHK_EG "send to", the semantic role of CHK_GBK can be determined as follows: "web browser" is the actor GBK1, "HTML request" is the content GBK2, and "remote server" is the target GBK3. The semantic relationship at the block level of GBK1 and GBK2 is relatively simple. Although "browser" is the semantic center of "web browser", "request" is the semantic center of "HTML request", and "server" is the semantic center of "remote server", But because there is no semantic edge at the chunk level, the words of the chunk are hung out as leaf nodes. Semantic tree structure at chunk level in GBK3: L0 "+GBK3 "server" controlled by +GBK2 "system" + EG "+CHK_L1", where GBK3 chunk is the root node. The semantic nodes of the second level are L0(by), V(control), L1(of), and all three levels are -1; the semantic edges of the second level are CHK_L0(by), CHK_EG(control), CHK_L1( ), CHK_GBK (system, server), five language block levels are all -1, and they hang out as child nodes. According to the concept that "control" of CHK_EG represents a generalized role, the semantic role of CHK_GBK can be determined as follows: "system" is the actor GBK1, and "server" is the content GBK2. The semantic tree established by the statement in this embodiment is shown in FIG. 4 and FIG. 5 .

Example 4:

In this embodiment, a system for implementing the method for constructing a hierarchical semantic tree described in the foregoing embodiments is provided. The hierarchical semantic tree construction system 500 in this embodiment has a structural block diagram as shown in FIG. 6 , including

Preprocessing unit S520: input the sentence to be processed, perform word segmentation on the sentence to be processed, and load the semantic knowledge of the word after word segmentation;

The first sequence generating unit S530: according to the word segmentation result, identify the semantic node of the sentence; use the semantic knowledge and word position to obtain the level of the semantic node;

The second sequence generating unit S540: identifying semantic edges of different levels in the sentence;

Hierarchical semantic tree generation unit S550: Generate a hierarchical semantic tree according to the semantic edges of each hierarchy.

In addition, during implementation, it also includes an input sentence unit and a hierarchical semantic tree output unit S560.

Preferably, in the preprocessing unit S520, when segmenting the sentence to be processed, the sentence to be processed is segmented according to the domain dictionary and the general dictionary. In this embodiment, the semantic knowledge includes generalized concept classes of words and their subclasses, and the generalized concept classes of words include dynamic, static, objects, people, attributes, and logic.

Preferably, the first sequence generating unit S530 includes:

The first subunit: For the word after word segmentation, if there is a logical concept in the semantic knowledge of the word,

Mark the word as L, if there is a dynamic concept in the semantic knowledge of the word, mark it as V;

The second subunit: carry out LV exclusion processing for all words marked as L or V;

The third subunit: mark all L tags according to their concept categories, and judge whether they have post-marks. If there are post-marks, mark the post-marked words as L1H, and generate semantic nodes according to all the above-mentioned tags.

It also includes a fourth subunit: taking the end-of-sentence punctuation generation semantic node as the root node.

The first sequence generating unit S530 also includes:

The fifth subunit: the default levels of all L marks and v marks are recorded as 0, and when two of the above marks are adjacent, the level of the second mark is reduced by one level to -1.

The second sequence generation unit S540 includes:

Core verb identification unit: for all semantic nodes marked V, perform core verb identification and generate language chunks;

Chunk generation unit: generate chunks for all semantic nodes marked as L;

Semantic edge generation unit: Generate semantic edges according to language chunks.

In the core verb identification unit, the core verb identification is performed, which also includes:

Exclude subunits: Exclude words that cannot form core verbs;

Selecting subunits: The rest of the words are given different weights according to the composition and the characteristics of the words themselves, and the core verbs are selected according to the ranking results and position information of the weights.

Hierarchical semantic tree generation unit S550, including:

Root node subunit: select the root node;

Sub-node sub-unit: link the high-level language blocks to the root node according to the order of the level as sub-nodes;

Traversing sub-units: Traversing all sub-nodes, taking all the blocks within the scope of each sub-node as the sub-nodes of the sub-node, until no new sub-nodes are generated.

FIG. 6 is a schematic diagram illustrating a hierarchical semantic tree construction system 500 in an embodiment of the present invention. The hierarchical semantic tree construction device 500 includes five units: a preprocessing unit S520, a first sequence generation unit S530, a second sequence generation unit S540, a hierarchical semantic tree generation unit S550 and a hierarchical semantic tree output unit S560. Step S510 represents the input of a sentence, which generally refers to a complete sentence, rather than sentence groups or chapters. The preprocessing unit S520 includes word segmentation processing for sentences, processing of special punctuation such as paired brackets, quotation marks, book title numbers, loading semantic knowledge base, binding numbers and English abbreviations appearing in sentences and loading their semantic information, Effective punctuation such as commas, colons, commas, periods, etc. is processed and its semantic information is loaded, and disambiguation rules are used to disambiguate words of the same category. The main purpose of the operation of the preprocessing unit is to eliminate interference and make the subsequent recognition steps more concise. easy. The first sequence generation unit S530 mainly adopts the LV principle to process all words containing the concept of l or v to identify all semantic nodes L/V, and utilizes the positional relationship presented by the LV semantic nodes to distinguish their levels. The default is 0, It represents the first level, and the second level is -1; the semantic nodes of the punctuation class are identified according to the semantic information of effective punctuation such as commas, colons, commas, and full stops. The second sequence generating unit S540 mainly identifies node edges CHK_EG, CHK_L0, CHK_ABK, CHK_GBK and their levels according to all semantic nodes L/V/SST and their levels. The hierarchical semantic tree generating unit S550 mainly analyzes the internal structure of CHK_GBK, and identifies the parallel semantic structure, the semantic structure of the above-mentioned degraded sentence and others according to the internal combinations of the chunk. In particular, it should be noted that the recognition of degraded sentences is similar to that at the clause level, the difference is that the level information of CHK_ABK, CHK_L0, and CHK_EG is -1. The hierarchical semantic tree output unit S560 mainly outputs the hierarchical semantic tree according to the result of the hierarchical semantic tree generating unit to obtain the hierarchical semantic tree, specifically including: determining that SST is the root node, and the first-level CHK_EG, CHK_L0, CHK_ABK, and CHK_GBK are its child nodes And hang under it, the child nodes of CHK_EG, CHK_L0, CHK_ABK, CHK_GBK go sub-nodes of the second level hang under it, and so on, until all are leaf nodes.

Apparently, the above-mentioned embodiments are only examples for clear description, rather than limiting the implementation. For those of ordinary skill in the art, other changes or changes in different forms can be made on the basis of the above description. It is not necessary and impossible to exhaustively list all the implementation manners here. And the obvious changes or changes derived therefrom are still within the scope of protection of the present invention.

Claims (10)

1. for Layer semantics tree constructing method and the system of language understanding, it is characterized in that, comprise the steps:

S1, input pending statement, pending statement is carried out to participle, and load the semantic knowledge of word after participle;

S2, according to word segmentation result, identify the semantic node of this statement;

S3, utilize semantic knowledge and word position and collocation to obtain the level of semantic node;

S4, identify the semantic limit of different levels in this statement;

S5, generate level semantic tree according to the semantic limit of each level.

2. Layer semantics tree constructing method according to claim 1, is characterized in that, comprising: in described step S1, when pending statement is carried out to participle, according to field dictionary and universaling dictionary, pending statement is carried out to participle.

3. Layer semantics tree constructing method according to claim 1 and 2, is characterized in that, described semantic knowledge comprises generalized concept class and the subclass thereof of word, and the generalized concept class of described word comprises dynamically, static state, thing, people, attribute, logic.

4. according to the arbitrary described Layer semantics tree constructing method of claim 1-3, it is characterized in that, the process that " according to word segmentation result, identifies the semantic node of this statement " in described step S2, comprising:

For the word after participle, if having logical concept in the semantic knowledge of word, this word is labeled as to L, if having dynamic concept in the semantic knowledge of word, be labeled as V;

To all words that are labeled as L or V, carry out LV and get rid of processing;

All L marks are carried out to mark according to its concept classification, and judge whether it has rear mark, if there is rear mark, the word of rear mark is labeled as to L1H, according to above-mentioned all mark generative semantics nodes.

5. according to the arbitrary described Layer semantics tree constructing method of claim 1-4, it is characterized in that, the process that " according to word segmentation result, identifies the semantic node of this statement " in described step S2, also comprises: using end of the sentence punctuate generative semantics node as root node.

6. according to the arbitrary described Layer semantics tree constructing method of claim 1-5, it is characterized in that, the process of " utilizing semantic knowledge and word position and collocation to obtain the level of semantic node " in described step S3, comprising:

The acquiescence level of all L marks and v mark is all designated as 0, and in the time occurring that two above-mentioned marks are adjacent, it is-1 that the level of second mark reduces one deck.

7. according to the arbitrary described Layer semantics tree constructing method of claim 1-6, it is characterized in that, the process of " identifying the semantic limit of different levels in this statement " in described step S4, comprises

To all semantic nodes that are labeled as V, carry out the identification of core verb, generate language piece;

To all semantic nodes that are labeled as L, generate language piece;

According to language piece generative semantics limit.

8. according to the arbitrary described Layer semantics tree constructing method of claim 1-7, it is characterized in that, described in carry out the identification of core verb process comprise:

Eliminating can not form the word of core verb;

Remaining word is given different weights according to the feature forming and word itself has, and selects core verb according to the ranking results of weights and positional information.

9. according to the arbitrary described Layer semantics tree constructing method of claim 1-8, it is characterized in that, the described process that generates level semantic tree according to the semantic limit of each level, comprising:

Select root node;

Language piece high level, according to the order in this level, be suspended on root node, as child node;

Travel through all child nodes, all language pieces within the scope of each child node are as the child node of this child node, until do not have new child node to produce.

10. the Layer semantics tree constructing system that the Layer semantics tree constructing method described in claim 1-9 is corresponding, is characterized in that, comprising:

Pretreatment unit: input pending statement, pending statement is carried out to participle, and load the semantic knowledge of word after participle;

First ray generation unit: according to word segmentation result, identify the semantic node of this statement; Utilize semantic knowledge and word position and collocation to obtain the level of semantic node;

The second sequence generating unit: the semantic limit of identifying different levels in this statement;

Layer semantics tree generation unit: generate level semantic tree according to the semantic limit of each level.