CN103026356B - Semantic content is searched for - Google Patents

Abstract

One or more techniques and/or systems are disclosed that provide document retrieval in which a user can identify key attributes of desired potential target documents (eg, have specific semantic content to the user). Additionally, related documents that include the desired semantic content can be retrieved. Additionally, users can provide feedback on retrieved documents, eg, based on key semantic concepts found in the documents, and the input can be used to update the taxonomy. For example, this process can be repeated to improve the retrieval and accuracy of documents found through machine learning techniques.

Description

Semantic Content Search

Background technique

Document retrieval in an enterprise environment is a very important problem, especially when it is expected to find critical information in a timely manner. For example, in a medical setting, it might be useful to find relevant alternatives to a case a doctor is currently working on (eg, to find patterns and/or specific treatment options). As an example, a doctor might be interested in finding a previous patient who was both a smoker and an aspirin allergy. In general, document searching involves keyword searching, where related terms that can be found in documents are entered into a search engine, and those documents that include the keyword are retrieved. Document retrieval can be performed on enterprise databases (such as hospitals), distributed databases, and on-line resources (such as the Internet).

Contents of the invention

This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. implementation. This Summary is not intended to identify key factors or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

Currently, free text searches such as by keyword often result in too few or too many results. As an example, when an Internet-based search engine is used to perform a keyword search, millions of results may be returned. Reviewing the results returned from a keyword search can be time-consuming and often frustrating because relevant information is often missing from the returned documents. For example, a user may be looking for documents with a word or phrase with a certain meaning, but a search engine may return irrelevant documents containing the same keyword but with a different meaning. As another example, English words and phrases often use the same word with completely different meanings (eg, "bass" could mean fish, musical instrument, or shoe). However, users want to quickly retrieve critical information from large databases.

Accordingly, disclosed herein is one or more techniques and/or systems that provide document retrieval in which a user can identify key attributes of desired potential target documents (such as having specific semantic content to the user). Additionally, relevant documents containing desired semantic content can be retrieved. Additionally, users can provide feedback on retrieved documents, eg, based on key semantic concepts found in the documents, and the input can be used to update the taxonomy. For example, the process can be repeated to improve the retrieval and accuracy of documents discovered through machine learning techniques.

In one embodiment of document search by semantic content, an end-user selection of a desired first portion of an initial document from a database comprising potential target documents is received, wherein the initial document contains metadata describing attributes of components of the initial document. data labels, and the selected first part contains the initial document composition with the semantic content desired by the user. The initial document is run through one or more trained classifiers along with the selected first part to identify from the database documents having a second part containing the same semantic content as the first part (e.g., selected by the user). The first potential target document.

In this embodiment, if the second part does not have the same semantic content as the first part, the end user's selection of the third part of the first potential target document is received, wherein the third part contains the same semantic content as the first part. In addition, the first potential target document is passed through one or more trained classifiers together with the selected third part to identify a second potential target document from the database, wherein the second potential target document has the same as the third part The fourth part of the same semantic content.

To the accomplishment of the above and related ends, the following description and drawings set forth certain illustrative aspects and implementations. These are indicative of but a few of the various ways in which one or more aspects may be employed. Other aspects, advantages and novel features of the invention will become apparent from the following "Detailed Description" read in conjunction with the accompanying drawings.

Description of drawings

FIG. 1 is a flowchart providing an exemplary method for document search by semantic content.

FIG. 2 is a flowchart illustrating an exemplary embodiment of an implementation of a method of document search by semantic content.

3 is an illustration of an example graph that can be used to visually identify classifier accuracy.

4 is a component diagram of an exemplary system for searching by semantic content.

5 is a component diagram illustrating an example implementation in which one or more systems and/or techniques described herein may be implemented.

6 is an illustration of an example computer-readable medium embodying processor-executable instructions configured to embody one or more of the principles set forth herein.

Fig. 7 illustrates an example computing environment in which one or more of the principles set forth herein may be implemented.

detailed description

The claimed subject matter is now described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of claimed subject matter. It may be evident, however, that claimed subject matter may be practiced without these specific details. In other instances, various structures and devices are shown in block diagram form in order to facilitate describing the claimed subject matter.

FIG. 1 is a flowchart of an exemplary method 100 of providing document search by semantic content. Exemplary method 100 begins at 102 and involves receiving an end-user selection of a desired first portion of an initial document from a database containing potential target documents at 104 . Here, the initial document contains meta-tags describing attributes of components of the initial document, such as words, and the selected first part contains components of the initial document with desired semantic content.

For example, a document can be parsed to determine the grammatical structure of the document. That is, a document may contain a sequence of tokens such as words, and the one or more sequences may be tokenized into individual constituents and these type) is tagged. Additionally, certain types of parsing may allow determination of the specific context of each corresponding component (eg, medical terminology, engineering terminology, etc.). In one embodiment, documents in the database may have been previously parsed for the user's intended use, and each corresponding component (such as a word, text block, etc.) tagged.

Additionally, in one embodiment, the user may select a portion of the document that contains the desired semantic content. That is, for example, a user may select a portion of a radiology report that identifies recommended follow-up within three weeks. In this example, the recommendation to return the patient for a follow-up visit within three weeks contains semantic content because it has specific meaning in the context of the surrounding words and radiology report. For example, tracking recommendations can be written in several different ways, but the meaning of the words is very specific. As another example, the diagnosis portion of a physician's report may include phrases such as "The diagnosis is...," "I believe the patient has...," "Test results indicate...," or various other variations. Again, however, the significance of the diagnosis is very specific.

In one embodiment, an end user, such as a physician or some other end user who wishes to identify a number of documents in the database with particular semantic content, may select an initial document. For example, an end user may be tasked with identifying customer report documents in an enterprise database in which complaints involving adverse response times are identified. In this example, adverse reaction times include semantic meanings that can be written in a number of different ways. In the past, an IT professional could be called in to set up a search, train a classifier, run tests, reset and tune the system to identify the required documents. In this embodiment, an end user such as an administrative assistant, customer service representative, or other, for example, may select the desired semantic content to identify the desired document.

At 106, the initial document having the selected first portion is passed through the one or more trained classifiers to identify a first potential target document from the database. In this embodiment, the first potential target document has a second portion having the same semantic content as the first portion selected by the end user. For example, one or more classifiers can be trained to identify words in documents using techniques such as Hidden Markov Models (HMM), Support Vector Machines (SVM), Conditional Random Fields (CRF), Statistical language models, etc.

Here, for example, one or more classifiers may attempt to identify target documents from the database that have the same semantic content that the end user identified in the original document. Also, in this example, the one or more classifiers may attempt to highlight a portion (second portion) of the target document that contains the same semantic content as the portion highlighted by the end user (first portion). In this way, the one or more classifiers are used to find documents that have the content desired by the end user, for example, may or may not use the same words but have the same meaning.

At 108 of the exemplary method 100, if the second portion does not have the same semantic content as the first portion, at 110 an end-user selection of a third portion of the first potential target document is received, wherein the third portion contains the same Same semantic content as the first part. For example, the portion of the target document highlighted by one or more classifiers may not have the same desired semantic content as the portion selected by the end user from the original document. That is, the classifier may have misclassified the semantic content and selected content from the retrieved target documents that did not match the content desired by the end user.

As an example, when one or more classifiers return misclassified content, the user can review the target document and select the content (third part) that corresponds to their desired semantic content (eg, the same as the first part in the original document). At 112, the first potential target document having the selected third portion may be passed through one or more trained classifiers to identify a second potential target document from the database. Here, the second potential target document contains a fourth section with the same semantic content as the third section.

As an example, after determining that one or more classifiers did not identify the correct content, an end user reviews the first document returned by the one or more classifiers, they highlight the correct content and rerun it through the one or more classifiers multiple classifiers. The one or more classifiers may then return another document from the database with the highlighted portion having the semantic content desired by the end user. In one embodiment, the steps at 108 may be iterated, for example, until the desired semantic content is retrieved from the target document in the database by one or more classifiers. Thus, in this example, the one or more classifiers can also be trained to identify the end user's desired semantic content.

Exemplary method 100 ends at 114 after target documents having desired semantic content are identified.

FIG. 2 is a flowchart illustrating an exemplary embodiment 200 of an implementation of a method of document search by semantic content. The exemplary embodiment 200 begins at 202 and involves performing a keyword search of a collection of documents at 204 . For example, a collection of documents may include an enterprise database, a collection of distributed databases, or documents from the Internet. In this embodiment, for example, a keyword search may be performed that contains words that may be found in documents containing the semantic content desired by the end user.

As an example, a hospital administrator may wish to identify patients for whom a physician has recommended a follow-up visit, such as whether a follow-up visit was performed and/or what the outcome of the follow-up visit was. In this embodiment, the hospital stores potentially millions of documents, and performing only a semantic content search on a document set of this size can also be burdensome. Accordingly, the manager may conduct a keyword search of the collection, such as using the words "track," "review," "return," and certain other words of similar meaning. The results of this keyword search can be used to populate a target database containing potential target documents 250 that can be searched with semantic content.

At 206 , in the exemplary embodiment 200 , the end user may select desired text (eg, contain desired semantic content) in the initial document 252 . As an example, a hospital administrator may identify a document that contains a doctor's recommendation that "patient return for a checkup within a month." In this example, the administrator may select this portion of the physician's report as the desired semantic content to include for its semantic content search. At 208 , the initial document with the selected text is passed through classifier 254 .

In one embodiment, passing the documents through the classifier 254 includes instructing the classifier to look for the selected type of text in the documents in the target document database 250 . Multiple trained classifiers 254 may be used in parallel to retrieve potential target documents 256 from the database. In one embodiment, the end user identifies which of a plurality of classifiers has the desired accuracy for retrieving potential target documents 256 for desired semantic content. In this embodiment, for example, the identified classifier can be used to retrieve the target document from the database for the desired semantic content, so that the retrieval can be performed faster and with fewer errors.

In another embodiment, a second classifier may be used to cross-validate potential target documents retrieved by the first classifier. Additionally, a combination of two or more classifiers can be identified that has a desired accuracy rate for retrieving potential target documents for desired semantic content. In this embodiment, the identified combination of classifiers can then be used to retrieve documents from the database for the desired semantic content.

FIG. 3 is an illustration of an example chart 300 that may be used to visually identify classifier accuracy. For example, a graph such as example 300 can be displayed to an end user as the end user trains the classifier to retrieve documents containing what they want, so the end user can see how the classifier is performing. In embodiment 300 , accuracy is represented by vertical line 310 and classifier is represented along horizontal line 312 . In this example, four classifiers 302-308 are used to identify target documents.

Based on whether the classifier correctly identified the content and/or document, an accuracy rate for each respective classifier may be determined 310 . In this example, it can be seen that classifier C2 304 has the highest accuracy. Thus, a user may determine to only utilize C2 304 to perform target document retrieval for specific semantic content. As an example, different semantic content and/or different types of documents may produce different classifier accuracy results. Thus, in one example, classifier C2 304 may not yield the same accuracy for different content and/or documents.

Returning to FIG. 2 , potential target documents 256 are retrieved from database 250 . For example, at 210, it is determined whether the potential target document satisfies the selection criteria such that the document contains selected content corresponding to the desired semantic content selected by the end user. If the second part known in the potential target document 256 has the same semantic content as the first part selected by the end user in the initial document, the end user can indicate that the classifier has correctly identified the desired semantic content in the target document , and at 212, the classifier 254 can be updated with this information.

That is, in one embodiment, user input from potential target documents returned by one or more classifiers may be used to update the one or more classifiers. In this way, for example, a classifier is trained to identify appropriate documents containing the desired semantic content. Additionally, after the classifiers have been updated with indications that the retrieval was correct, the one or more classifiers may be run against the database again to select a third potential target document 256 . This process can be iterated over the database to retrieve multiple appropriate target documents.

However, at 210, if the retrieved target document 256 does not identify content having the same semantic meaning as the content selected by the end user, then at 214, the user may select a portion of the returned document that contains the desired semantic content. For example, the end user may have selected a social security number (SSN) in the initial document for the classifier to retrieve the SSN from the document in the database 250 . However, after passing the initial document through the classifier, potential target document 256 identifies a phone number. In this example, the end user then points out that the classifiers (like one or more of them) incorrectly identified the semantic content, and then highlights the correct information, namely the SSN in the document.

At 208 , the corrected document may be passed through classifier 254 again to retrieve second potential target document 256 . In one embodiment, the portion of the potential target document 256 identified by the classifier 254 (the second portion) may not have the same semantic content as the first portion, and the potential target document may not contain the same user-selected portion of the initial document (Part 1) has the same semantic content as the content. That is, the returned documents may not have matching content for the user to select to rerun through the classifier.

In an alternative embodiment, the end user may indicate that the first potential target document does not contain content having the same semantic content as the first portion, and at 212 the classifier may be updated with this information. Additionally, the one or more classifiers may then be run against the database 250 to select another potential target document 256 (a third potential document). As an example, at 210, if the document satisfies the selection criteria, the classifiers may again be updated in order to train them to identify appropriate semantic content.

In one embodiment, end users may provide input to classifier training for use in updating classifiers (eg, making them more accurate). For example, for each document, an end user can indicate that one or more classifiers retrieved from the database contain the desired semantic content. Additionally, an end user may indicate that one or more classifiers retrieved documents from the database that do not contain desired semantic content. Alternatively, the end user may provide one or more classifiers with selected portions of documents retrieved from the database, wherein the selected portions contain desired semantic content.

In this embodiment 200, for example, when retrieving a target document, a threshold can be set for the classifier. In this embodiment, multiple documents may be passed through one or more classifiers until a desired threshold is reached, which may be determined at 216 . The desired thresholds may comprise different criteria, which may be selected by the end user, or set automatically (eg, default).

In one embodiment, multiple documents may be passed through one or more classifiers until a desired document selection accuracy is achieved for desired semantic content. For example, users may expect document retrieval to be 100 percent accurate when the information is critical; or they may be satisfied with 90 percent accuracy when accuracy is not so important.

Furthermore, in another embodiment, multiple documents may be passed through one or more classifiers until a desired number of correct documents are retrieved without retrieving incorrect documents. For example, a user may be satisfied that a document retrieval returns a hundred correct documents without any errors, and may then let the retriever run unsupervised. In yet another embodiment, multiple documents may be passed through one or more classifiers until the desired number of documents are retrieved from the database. For example, an end user may only need a thousand documents for their desired purpose, and may run a document retrieval until that number is reached.

At 216 , if the required threshold is not met, then at 218 the classifier 254 is run against the database 250 to select another potential target document 256 . In addition, each potential target document that is selected and satisfies the selection criteria is stored in the target document database 258 . At 216 , if the threshold is met, then example embodiment 200 ends at 220 .

In one aspect, the one or more classifiers can be a decision engine that calculates the probability that a potential target document in the database contains the desired semantic content. In one embodiment, the classifier may be provided (eg, by an end user or by default) with a classification threshold that determines whether a document should be presented for input by the end user. That is, for example, where a classifier is presented with documents, a semi-supervised type of training may only occur if the classifier is not confident about the classification results.

As an example, in an SVM, a decision about a document (eg, "yes", the document contains semantic content, or "no", the document does not contain semantic content) can be plotted as a matrix to determine which side of the decision matrix it falls on. In this example, there may be gaps between the "yes" and "no" parts of the matrix where the SVM is non-deterministic about the decision. In this embodiment, for example, this blank may contain a threshold where the classifier presents the document to the end user as to whether or not the input document contains the desired semantic content. That is, the desired threshold may include an uncertainty threshold for the classifier model, and suspect documents are presented to the user only if the classifier model's calculation falls within this threshold.

For example, a system can be devised for finding documents that contain certain semantic content. FIG. 4 is a component diagram of an exemplary system 400 for searching by semantic content. The memory component 402 stores a database 450 containing a plurality of potential target documents. Processor component 404 is operatively coupled to memory component 402 and configured to execute instructions of one or more classifiers 410 . In one embodiment, memory component 402 and processor component 404 may be disposed on the same computing device. In other implementations, these components may be located separately and/or may reside on the same computing device with other components in the exemplary system 400 .

The end user input receiving component 406 receives input from an end user 452 for a document, such as a document retrieved from a database 450, and/or an initial document that may be used as a seed for a database, for example. The input provided by the end user 452 may comprise a desired portion of the first document selected by the end user from the database 450, wherein the selected portion comprises a document composition having the desired semantic content of the initial document. For example, end-user 452 may provide an initial document as a seed for database 450 by selecting a portion of the initial document that has semantic content desired by user 452 (eg, text literals in the document).

Additionally, the input provided by the end user 452 may include an end user indication that the second document retrieved from the database 450 contains the same semantic content as the selected desired portion of the first document. For example, an end user may pass an initial document with selected desired semantic content through one or more classifiers 410 to retrieve a second document. In this example, user 452 may review the second document to determine that it does have the same semantic content as selected in the initial document. The user can then indicate that the retrieved document is correct, eg by input.

Additionally, the input provided by the end user 452 may include an end user indication that the second document retrieved from the database does not contain the same semantic content as the selected desired portion of the first document. For example, an end user may pass an initial document with selected desired semantic content through one or more classifiers 410 to retrieve a second document. In this example, user 452 may review the second document to determine that it does not have the same semantic content as selected in the initial document (eg, or that the content selected by the classifier was incorrect). The user may then indicate, for example, by inputting that the retrieved document is incorrect.

One or more classifier components 410 may be operatively coupled to the processor component 404 and the memory component 402 . The one or more classifier components 410 are used to identify a second document from the database, wherein the second document contains a target portion having the same semantic content as the selected desired portion in the first document. For example, an end user can seed the database 450 by passing an initial document through one or more classifier components 410 so that the classifier components 410 can be trained to retrieve only those target documents from the database 450 that contain the desired semantic content .

Classification update component 408 can be operatively coupled with end user input receiving component 406 and used to update one or more classifier components 410 with end user input to identify desired semantic content. For example, user input such as described above is used to train one or more classifier components so that they are more accurate at identifying target documents in database 450 that contain the desired semantic content.

FIG. 5 is a component diagram illustrating one embodiment 500 in which one or more systems and/or techniques described herein may be implemented. Database population component 520 is used to populate database 450 with potential target documents using a keyword search of document collection 560 . For example, using the database population component 520, a document collection containing millions of potential target documents can be reduced to a few thousand by keyword searching.

Document presentation component 522 presents documents from database 450 to end user 452 for identification of desired semantic content. For example, potential target documents can be retrieved from database 450 using a trained classifier, and document presentation component 522 can present them to end user 452 to determine whether they satisfy desired semantic content selection criteria. In one embodiment, the document presentation component 522 may utilize a computer-based display, such as a monitor, and the end user 452 may simply review the content on the display.

Database indexing component 524 can be used to tag documents in database 450 that contain desired semantic content. For example, when a classifier component retrieves a target document and determines that it contains desired semantic content, database indexing component 524 can append metadata to the document indicating that it has the desired semantic content. Additionally, in one embodiment, the database indexing component 524 can attach metadata to portions of the target document that contain the desired semantic content. For example, identification of this content may facilitate end user information collection.

In one embodiment, the database may include one or more sample documents (eg, seed documents) containing desired semantic content (eg, what the end user is searching for). In this embodiment, the one or more sample documents may be used to train one or more classifiers 410 to identify potential target documents having the same semantic content as the desired semantic content. In another embodiment, the desired portion of the document may be text, where the document components are made up of words (eg, a text-based literal document). As an example, text may contain numbers, symbols, or other text-based elements combined into strings.

Yet another embodiment involves a computer-readable medium comprising processor-executable instructions configured to implement one or more of the techniques presented herein. An exemplary computer-readable medium that can be designed in these ways is shown in FIG. 6, where an implementation 600 includes a computer-readable medium 608 (such as a CD-R, DVD-R, or hard drive platters). This computer readable data 606 in turn includes a set of computer instructions 604 configured to operate according to one or more of the principles set forth herein. In one such implementation 602 , processor-executable instructions 604 may be configured to perform a method, such as, for example, exemplary method 200 in FIG. 1 . In another such embodiment, processor-executable instructions 604 may be configured to implement a system, such as, for example, exemplary system 400 in FIG. 4 . Those of ordinary skill in the art can devise many such computer-readable media that can be configured to operate in accordance with the techniques described herein.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. The specific features and acts described above are disclosed as example forms of implementing the claims.

As used in this application, the terms "component," "module," "system," "interface," etc. are generally intended to refer to a computer-related entity, which may be hardware, a combination of hardware and software, software, or an software. For example, a component may be, but is not limited to being limited to, a process running on a process, a processor, an object, an executable, a thread of execution, a program, and/or a computer. As an illustration, both an application running on a controller and a controller can be a component. One or more components can reside within a process and/or thread of execution, and a component can be localized on one computer and/or distributed between two or more computers.

Furthermore, the claimed subject matter can be implemented as a method, apparatus or article of manufacture using standard programming and/or engineering techniques resulting in software, firmware, hardware or any combination thereof controlling a computer to implement the disclosed subject matter. The term "article of manufacture" as used herein is intended to cover a computer program accessible from any computer-readable device, carrier, or media. Of course, those skilled in the art will recognize many modifications may be made to this configuration without departing from the scope or spirit of the claimed subject matter.

FIG. 7 and the following discussion provide a brief general description of a suitable computing environment for implementing implementations of one or more of the principles set forth herein. The operating environment of FIG. 7 is only one example of a suitable operating environment and is not intended to suggest any limitation as to the scope of use or functionality of the operating environment. Example computing environments include, but are not limited to, personal computers, server computers, handheld or laptop devices, mobile devices (such as mobile phones, personal digital assistants (PDAs), media players, etc.), multiprocessor systems , consumer electronics, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

Although not required, various embodiments are described in the general context of "computer readable instructions" being executed by one or more computing devices. Computer readable instructions may be distributed via computer readable media (discussed below). Computer readable instructions may be implemented as program modules, functions, objects, application programming interfaces (APIs), data structures, etc., that perform particular tasks or implement particular abstract data types. Generally, the functions of the computer readable instructions may be combined or distributed among various environments as desired.

FIG. 7 illustrates an example of a system 710 including a computing device 712 configured to implement one or more implementations provided herein. In one configuration, computing device 712 includes at least one processing unit 716 and memory 718 . Depending on the exact configuration and type of computing device, memory 718 may be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.), or some combination of the two. This configuration is shown by dashed line 714 in FIG. 7 .

In other implementations, device 712 may include additional features and/or functionality. For example, device 712 may also include additional storage (eg, removable and/or non-removable) including, but not limited to, magnetic storage, optical storage, and the like. Such additional storage is illustrated by storage 720 in FIG. 7 . In one embodiment, computer readable instructions for implementing one or more embodiments described herein may be in storage 720 . Storage 720 may also store other computer readable instructions implementing an operating system, application programs, and the like. Computer readable instructions may be loaded in memory 718 for execution, such as by processing unit 716 .

The term "computer-readable medium" as used herein includes computer storage media. Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions or other data. Memory 718 and storage 720 are examples of computer storage media. Computer storage media including, but not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage, cassette tape, magnetic tape, magnetic disk storage or other magnetic storage device, or any other medium that can be used to store the required information and be accessed by device 712. Any such computer storage media may be part of device 712 .

The device 712 may also include a communication connection 726 that allows the device 712 to communicate with other devices. Communications connections 726 may include, but are not limited to, modems, network interface cards (NICs), integrated network interfaces, radio frequency transmitters/receivers, infrared ports, USB connections, or connections for connecting computing device 712 to other computing devices. other interfaces. Communications connection 726 may include a wired connection or a wireless connection. Communication connection 726 may send and/or receive communication media.

The term "computer-readable medium" may include communication media. Communication media typically embodies computer readable instructions or other data in a "modulated data signal" such as a carrier wave or other transport mechanism and includes any information delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.

Devices 712 may include input devices 724 such as keyboards, mice, pens, voice input devices, touch input devices, infrared cameras, video input devices, and/or any other input devices. Also included in device 712 are output devices 722, such as one or more displays, speakers, printers, and/or any other output devices. Input device 724 and output device 722 may be connected to device 712 via a wired connection, a wireless connection, or any combination thereof. In one embodiment, an input device or an output device from another computing device may be used as an input device 724 or an output device 722 of the computing environment 712 .

Components of computing environment 712 may be connected by various interconnects, such as a bus. Such interconnections may include Peripheral Component Interconnect (PCI) such as PCI Express, Universal Serial Bus (USB), FireWire (IEEE1394), and optical bus structures, among others. In another implementation, the components of computing environment 712 may be interconnected by a network. For example, memory 718 may comprise multiple physical memory units located in different physical locations interconnected by a network.

Those skilled in the art will realize that storage devices utilized to store computer readable instructions can be distributed across a network. For example, computing device 730 accessible via network 728 may store computer readable instructions to implement one or more embodiments provided herein. Computing device 712 may access computing device 730 and download some or all of the computer readable instructions for execution. Alternatively, computing device 712 may download pieces of computer readable instructions as needed, or some instructions may be executed in computing device 712 and some instructions may be executed in computing device 730 .

Various operations of various embodiments are provided herein. In one embodiment, one or more of the operations described may constitute computer-readable instructions stored on one or more computer-readable media that, if executed by a computing device, cause the computing device to perform the described operations. operation. The order in which some or all operations are described should not be construed as to imply that these operations are necessarily order dependent. Those skilled in the art having the benefit of this description will appreciate the ordering of substitutions. Additionally, it should be understood that not all operations necessarily occur in every implementation presented herein.

Additionally, use of the word "exemplary" herein is intended to be an example, instance, or illustration. Any aspect or design described herein as "exemplary" is not necessarily to be construed as advantageous over other aspects or designs. Rather, use of the word "exemplary" is intended to present various concepts in a concrete manner. As used in this application, the term "or" means an inclusive "or" rather than an exclusive "or". That is, unless specified otherwise or clear from context, "X employs A or B" means any naturally inclusive permutation. That is, if X uses A; X uses B; or X uses both A and B, then "X uses A or B" is satisfied in any of the above cases. Additionally, the articles "a" and "an" as used in this application and in the appended claims may generally be construed to mean "one or more" unless specified otherwise or clear from context that singular form.

Also, while the invention has been shown and described with reference to one or more implementations, various equivalents and modifications will occur to those skilled in the art upon the reading and understanding of this specification and the annexed drawings. The present invention includes all such modifications and substitutions and is limited only by the appended claims. In particular, for the various functions performed by the above-described components (eg, elements, resources, etc.), unless otherwise indicated, the terminology used to describe these components is intended to correspond to the implementation described herein in the exemplary implementation of the invention. Any components that are functionally equivalent (eg, functionally equivalent) to the functional components shown in , even if those components are not structurally equivalent to the disclosed structures. Additionally, although a particular feature of the invention may be disclosed with respect to only one of several implementations, that feature may be advantageously combined with one or more other features of other implementations as required for any given or particular application. combined. Furthermore, to the extent the terms "comprising", "having", "containing" and "with" and variations thereof are used in the specification or claims, these terms are intended to include in a manner similar to the term "comprising". sexual.

Claims (8)

1. for carrying out a computer based method for document searching based on semantic content, including:

Receive to the required Part I of the original document from the data base including potential target document Whole user selects, and described original document includes the first number describing the attribute of each ingredient of described original document According to label, described Part I includes required semantic content；

The processor using computer makes the described original document including selected Part I through one or many The individual grader trained includes the first potential target document of Part II with mark from described data base； And

In response to determining that described Part II does not have required semantic content, then:

Reception includes the final of the Part III of required semantic content to described first potential target document User selects；And

Make the described first potential target document including selected Part III through the one or more The grader of training includes having the of the Part IV of required semantic content with mark from described data base Two potential target documents；And

At least one in following action:

Receiving the end user to respective document to input, described input includes one in the following Or multiple:

The one or more grader from described database retrieval to document include required language The instruction of justice content；

The one or more grader from described database retrieval to document do not include required The instruction of semantic content；And

The one or more grader from described database retrieval to the selected portion of document, Wherein this selected portion includes required semantic content；

Make multiple document through the one or more grader until for required semantic content Till reaching required document choice accuracy；

Make the plurality of document through the one or more grader until there is no retrieval error literary composition Till retrieving the correct document of requirement in the case of Dang；

Make the plurality of document through the one or more grader until from described database retrieval Document to requirement；

The second grader is used to confirm the document retrieved by the first grader；Or

Mark has required accuracy rate, two or more graders combination of search file, and profit With the combined retrieval document identified to find required semantic content.

2. the method for claim 1, it is characterised in that also include by one group of document is performed pass Keyword search uses potential target document to fill described data base.

3. the method for claim 1, it is characterised in that in response to determining that described Part II has Required semantic content, described method includes:

Described data base is run the one or more grader to select the 3rd potential target document.

4. the method for claim 1, it is characterised in that if described Part II does not has and institute State the identical semantic content of Part I and described first potential target document does not includes having and described The content of a part of identical semantic content, the most described method includes:

Receive described first potential target document and do not include that there is the semantic content identical with described Part I Content end user instruction；And

Described data base is run the one or more grader to select the 3rd potential target document.

5. the method for claim 1, it is characterised in that include utilizing the one or more point User's input of the potential target document that class device is returned updates the one or more grader.

6. the method for claim 1, it is characterised in that include making the plurality of document through described One or more graders, until it reaches till required threshold value.

7. the method for claim 1, it is characterised in that include only providing at document classification being in institute When needing the result in threshold value, just present potential target document to described end user.

8. method as claimed in claim 7, it is characterised in that required threshold value includes that sorter model is not Definitiveness threshold value.