KR20130099327A - Apparatus for extracting information from open domains and method for the same - Google Patents

Abstract

Disclosed is an apparatus and method for extracting domain information according to an embodiment of the present invention. The domain information extracting apparatus according to an embodiment of the present invention includes a sentence separator for separating sentences from a text document in a specific domain received, and a morpheme analyzer for extracting the top N sentences by performing morphological analysis on the separated sentences. A dependency parser configured to analyze syntax dependences between the sentences of the extracted N sentences and generate upper M dependent syntax analysis results, and predetermine statistics on the N sentences and the M dependent syntax analysis results The semantic domain is determined by applying a model to determine a semantic factor of each verb in a sentence, and a semantic domain determination unit generating a frame by collecting the factors whose semantic ranges are determined for each verb and applying the ontology rule predefined to the generated frame. Assigns it as an instance of a specific class, and assigns the assigned instance to a particular class of domain ontology. To map the is configured to include parts of the ontology mapping.

Description

Apparatus and method for extracting open domain information {APPARATUS FOR EXTRACTING INFORMATION FROM OPEN DOMAINS AND METHOD FOR THE SAME}

The present invention relates to an apparatus and method for extracting open domain information, and more particularly, to an apparatus and method for extracting information from an unstructured web document in a domain.

Today, with the growth of the Internet, various information is provided through web sites. The current web requires users to follow a link to the site and to the destination of their choice. However, it is more efficient to query a large number of web pages than to read them all. To query, you need to extract the information contained in a web page and convert it into structured or semi-structured data.

Currently, there are many web information extraction tools, which are classified into automatic extraction tools and manual extraction tools. If a web page consists of data structured according to a given schema, it can be automatically extracted, but many web pages are a form of unstructured data without a defined schema. In the case of such unstructured data, the user must specify the schema of the data to be extracted. Extraction rules are required to extract the data of the schema specified by the user.

Most of the conventional methods for extracting information from an open domain web page use a method of extracting information dependent on a specific domain, and thus there is a problem that porting to another domain is not easy. Prior patent, "System and Method for Extracting Domain-Specific Information from Unstructured Web Documents" (Application No. 10-2005-0063896), learns information extraction rules from unstructured web documents containing unstructured data classified by domain. In this paper, a method of extracting information for each domain for automatically extracting key information from a web document of a specific domain is disclosed. However, since it uses rule-based information extraction method that is dependent on each domain to extract information by domain, porting to other domains is not easy.

In addition, the preceding paper, "Fine-grained named entity recognition and relation extraction for question answering" (Changki Lee, Yi-Gyu Hwang, Myung-Gil Jang. SIGIR 2007, 799-800), is a statistics-based entity for domain information extraction. A method of performing name recognition and extracting the relationship between two entity names. However, since the relation tag used for extracting a relationship is domain dependent, there is a problem in that the cost of porting the domain is high.

An object of the present invention for solving the above problems is to provide an information extraction apparatus that does not depend on a specific domain.

Another object of the present invention for solving the above problems is to provide an information extraction method that does not depend on a specific domain.

The present invention for achieving the above object, a sentence separator for separating sentences from the input text document, a morphological analysis unit for extracting the top N sentences by performing a morphological analysis on the separated sentences, the extracted N pieces A dependency parsing unit for generating syntax M-dependent syntax analysis results by analyzing syntax dependences between sentences of sentences, and applying the statistical model to the N sentences and the M dependent syntax analysis results by applying a predetermined statistical model. A semantic determining unit for determining a semantic range of the argument of each verb, and collecting a factor whose semantic is determined for each verb to generate a frame, and applying the generated ontology rule as an instance of a specific class. An ontology mapping unit for allocating and mapping the allocated instance to a specific class of a domain ontology It provides a domain information extraction apparatus including.

In accordance with another aspect of the present invention, there is provided a method of separating sentences from an input text document, extracting upper N sentences by performing a morphological analysis on the separated sentences, and extracting sentences from the extracted N sentences. Analyzing the syntax-dependency relations between the phrases and generating the top M dependent syntax analysis results, and applying the statistical model to the N sentences and the M dependent syntax analysis results by applying a predetermined statistical model Determining a symbol, collecting a factor whose semantic is determined for each term, and constructing a frame; allocating the configured frame as an instance of a specific class by applying a predefined ontology rule; assigning the assigned instance to a domain Domain information extraction method comprising the step of mapping to a specific class of ontology The ball.

In the case of using the method and apparatus for extracting domain information according to the present invention as described above, general information is extracted using dependency syntax analysis and semantic domain determination technology that is not dependent on a specific domain, and the extracted information is general ontology. By mapping to, domain-dependent part is mapped from general ontology to domain ontology to minimize the domain-dependent part, which makes it easy to port to another domain.

1 is a block diagram showing a configuration of an apparatus for extracting domain information according to an embodiment of the present invention.
2 is a conceptual diagram illustrating an example of a frame generated by the domain information extracting apparatus according to an embodiment of the present invention.
3 is a conceptual diagram illustrating a frame mapped to a domain ontology according to an embodiment of the present invention.
4 is a flowchart illustrating a process for performing a domain information extraction method according to an embodiment of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. Like reference numerals are used for like elements in describing each drawing.

The terms first, second, A, B, etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and are to be interpreted as either ideal or overly formal in the sense of the present application Do not.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate the understanding of the present invention, the same reference numerals are used for the same constituent elements in the drawings and redundant explanations for the same constituent elements are omitted.

1 is a block diagram showing a configuration of an apparatus for extracting domain information according to an embodiment of the present invention.

Referring to FIG. 1, the apparatus for extracting domain information according to an embodiment of the present invention includes a sentence separator 110, a morpheme analyzer 120, a dependency syntax analyzer 130, a semantic domain determiner 140, and an ontology. It may be configured to include a mapping unit 150.

In addition, referring to FIG. 1, components of a domain information extracting apparatus according to an embodiment of the present invention and connection relations between the components may be described as follows.

The sentence separator 110 may be a part for separating a sentence from the received text document 160. For example, the sentence separator 110 may receive a text document 160 extracted from an arbitrary domain, separate each sentence in the input document, and transmit the separated sentence to the morpheme analyzer 120.

The morpheme analysis unit 120 may be a part for extracting the top N sentences (for example, 10 sentences) by performing a morpheme analysis on the sentences received from the sentence separator 110. For example, the morpheme analysis unit 120 may extract the top 10 morphologically analyzed sentences from the received sentences and transfer the extracted sentence to the dependency syntax analyzer 130.

The dependency syntax analysis unit 130 may be a portion that generates syntax M dependency syntax analysis results by analyzing syntax dependences between the phrases of the sentences with respect to the N sentences extracted by the morpheme analysis unit 120.

That is, the dependent syntax analysis unit 130 may perform dependency syntax analysis on the N sentences that have been morphologically analyzed and given to each word in the sentence, and determine whether an object or an adverb is determined and stored in a tree form. In addition, the dependency parsing results may be ranked to generate upper M dependency parsing results, and may be transmitted to the semantic determining unit 140.

The semantic region determination unit 140 applies a predetermined statistical model to the N sentences extracted by the morpheme analysis unit 120 and the M dependent syntax analysis results generated by the dependency syntax analysis unit 130, thereby applying a factor of each verb in the sentence. Determining the semantic region of the, and may be a part for generating a frame by collecting the factors determined the semantic region for each term.

That is, the semantic domain determiner 140 uses the morphologically analyzed N sentences and M dependent syntax analysis results as features to conditional random fields (CRF) or structural support vector machines (Structural Support). Statistical models such as Vector Machine and Structural SVM can be used to determine the meaning of a verb in a sentence, and the semantic role of the argument of each verb, for example, actor, object, place, reason And so on.

For example, the result of the dependency syntax analysis of "Cheol eats at home" is <head, withdraw, modifier, dependent syntax>, <eat, rice, object>, <eat, home , Adverb>, and the result of semantic decision is <eat.01 (first meaning of "eat"), withdrawal (argument), actor (meaning)>, <eat.01, rice, object>, <eat .01, home, place>.

In addition, the semantic domain determination unit 140 may form a frame by collecting arguments for which a semantic role is determined for each term. In this case, the name of the frame may be “words” + “sense numbers of words”, the slot names of the frames may be semantic, and the element values of the frames may be configured to be arguments of the words. .

2 is an example of a frame generated by the domain information extracting apparatus according to an embodiment of the present invention, and shows a frame 200 generated for “he is eating at home”. Referring to FIG. 2, the frame name is “Eat. 01” 210, and the name of each component of the frame may be an actor 220, an object 230, and a place 240. The frame 200 may be configured such that the values of the elements are “retract” 221, “bob” 231, and “house” 241, respectively.

Meanwhile, the ontology mapping unit 150 allocates a frame generated by the semantic domain determining unit 140 as an instance of a specific class by applying a predefined ontology 151 rule, and assigns the allocated instance of the domain ontology 152. It may be a part mapping to a specific class.

For example, the “eat. 01” frame 200 of FIG. 2 may be mapped to an instance of a corresponding class of the predefined ontology 151. For example, if a particular class of general ontology 151 is defined as "event"-> "behavior"-> ("eat", "drink", ...), "eat.01"- The "eat.01" frame 200 above may be mapped to an instance of the "eat" class of the ontology 151 using the "eat" rule.

In addition, the domain ontology 152 is a modification or reduction of the general ontology 151 for a specific domain or application, and may use an rule to map an instance assigned to the general ontology class to a specific class of the domain ontology.

3 is a conceptual diagram illustrating a frame mapped to a domain ontology according to an embodiment of the present invention.

Referring to FIG. 3, the "rising. 02" frame 300 is a frame generated as a result of the semantic determination of "the stock has risen 50 points", and the component is the object 310, the degree 320, and the component. The values of may be "share prices" 311 and "50 points" 321, respectively. In this case, the "rise. 02" frame 300 may be mapped to the "event"->"state"->"rising" class of the general ontology 151, and the "share price rise" of the domain ontology 152. Can be mapped to a class.

4 is a flowchart illustrating a process for performing a domain information extraction method according to an embodiment of the present invention.

4, the domain information extraction method according to an embodiment of the present invention is a sentence separation step (S410), morphological analysis step (S420), dependency syntax analysis step (S430), semantic domain determination step (S440) and ontology It may be configured to include a mapping step (S450).

In addition, referring to Figure 4, each step of the domain information extraction method according to an embodiment of the present invention can be described as follows.

The sentence separating step S410 may be a step of receiving a text document and separating each sentence in the received document.

The morpheme analysis step S420 may be a step of extracting the top N sentences by performing a morpheme analysis on the sentences separated in the sentence separation step S410. For example, among the sentences separated in the sentence separation step S410, the top ten stemmed sentences may be extracted.

The dependency syntax analysis step S430 may be a step of analyzing the syntax-dependence relationship between the phrases of the sentences with respect to the N sentences extracted in the morphological analysis step S420 to generate the upper M dependent syntax analysis results.

That is, the dependent syntax analysis is performed on the N sentences that have been morphologically analyzed, and given to each word in the sentence, whether the object or the adverb is determined, and the result may be stored in the form of a tree. Also, the dependency parsing result may be ranked to generate the top M dependency parsing results.

In the semantic domain determination step (S440), the N sentences extracted in the morphological analysis step (S420) and the M dependent syntax analysis results generated in the dependency syntax analysis step (S430) are applied to each verb in the sentence by applying a predetermined statistical model. Determining the semantic range of the argument, it may be a step of generating a frame by collecting the factors whose semantic range is determined for each term

In other words, the results of the analysis of N stems and M dependent syntax analysis are used as features, and statistics such as Conditional Random Fields (CRF) or Structural Support Vector Machines (Structural SVM) The model may be applied to determine the meaning of a verb in a sentence, and a semantic role of an argument of each verb, for example, an actor, an object, a place, a reason, and the like.

In addition, with respect to the semantic domain determination result generated as described above, a frame may be configured by collecting arguments in which a semantic role is determined for each term. In this case, the name of the frame may be “words” + “sense numbers of words”, the slot names of the frames may be semantic, and the element values of the frames may be configured to be arguments of the words. .

The ontology mapping step (S450) is a part of assigning the frame generated in the semantic decision step (S440) as an instance of a specific class by applying a predefined general ontology rule, and mapping the assigned instance to a specific class of the domain ontology. Can be.

For example, the “eat. 01” frame 200 of FIG. 2 may be mapped to an instance of a corresponding class of a predefined generic ontology. Also, for example, if a particular class of generic ontology is defined as "event"-> "behavior"-> ("eat", "drink", ...), "eat.01"-> The “eat. 01” frame 200 of FIG. 2 may be mapped to an instance of the “eat” class of a generic ontology using the “eat” rule.

On the other hand, the domain ontology is a modification or reduction of the general ontology for a specific domain or application, and an instance assigned to the general ontology class can be mapped to a specific class of the domain ontology using a rule.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the present invention as defined by the following claims It can be understood that

Claims (1)

A sentence separator for separating a sentence from a text document in a specific domain received;
A morpheme analysis unit configured to extract a top N sentences by performing a morpheme analysis on the separated sentences;
A dependency syntax analysis unit configured to analyze syntax dependences between the sentences of the extracted N sentences and generate upper M dependent syntax analysis results;
Meaning of determining the semantic range of the factor of each word in a sentence by applying a predetermined statistical model to the N sentences and the M dependent syntax analysis results, and generating a frame by collecting the factors whose semantic range is determined for each of the words. Inverse determining unit; And
And an ontology mapping unit configured to allocate the generated frame as an instance of a specific class by applying a predefined ontology rule and to map the allocated instance to a specific class of a domain ontology.

Images