JP2020140435A

JP2020140435A - Sentence retrieval system, sentence retrieval method and sentence retrieval program

Info

Publication number: JP2020140435A
Application number: JP2019035349A
Authority: JP
Inventors: 英一郎神田; Eiichiro Kanda; 柏原　直樹; Naoki Kashiwabara; 直樹柏原
Original assignee: Kawasaki Gakuen Educational Foundation
Current assignee: Kawasaki Gakuen Educational Foundation
Priority date: 2019-02-28
Filing date: 2019-02-28
Publication date: 2020-09-03
Anticipated expiration: 2039-02-28
Also published as: JP7338848B2

Abstract

To efficiently narrow down a sentence, to prevent the leakage of a sentence matched to a retrieval request, and/to enable the evaluation of a retrieval result so as to eliminate problems of a conventional sentence retrieval method.SOLUTION: An information retrieval system for retrieving a sentence matched to a retrieval request includes: a sentence set reception unit 1 for receiving a sentence set including a plurality of sentences; a user retrieval request reception unit 2 for receiving a user's retrieval request; a word vector conversion unit 3, 12 for respectively converting each sentence and the retrieval request into a word vector; a sentence vector conversion unit 4 for respectively changing the word vector derived from each sentence and the word vector derived from the retrieval request into a sentence vector by a learned neural network; a similarity calculation unit 5 for calculating similarity between the sentence vector derived from each sentence and a sentence vector derived from the retrieval request; and a retrieval result presentation unit 6 for presenting the similarity between the sentence and the retrieval request.SELECTED DRAWING: Figure 1

Description

本発明は、文章検索システム、文章検索方法及び文章検索プログラムに関し、特に臨床疑問に関する文章検索システム、文章検索方法及び文章検索プログラムに関する。 The present invention relates to a text search system, a text search method and a text search program, and more particularly to a text search system, a text search method and a text search program relating to clinical questions.

（臨床疑問に関する従来の問題点）
医学におけるガイドラインでは、臨床疑問(clinical question, CQ)に関係するエビデンス(医学論文)をできるだけ集め、システマティックレビュー(systematic review, SR)を行った後、その結果に基づいてステートメントがまとめられる。
SRの手順は、まず一次スクリーニングとして、（１）CQに関係する論文をMEDLINEなどのデータベースから幅広く集めた文献リストを作成し、（２）そのアブストラクトをもとにCQに関与する論文を選択する。次に、二次スクリーニングとして、選択された論文を読み、基準にあった論文を選択する。その後、選択された各論文の治療効果やバイアスを評価し、エビデンス総体の評価を行う。これまでSRは、主として人の手作業で行われてきた。
一次スクリーニングでは、対象、曝露因子、介入だけでなく、害と不利益を含んだ幅広いアウトカムを拾うため検索式にアウトカムは含めず検索を行う。その結果、リストに挙げられた論文数は、非常に多く、論文の候補は数千になることもある。このリストから、タイトルとアブストラクトに基づいて、CQに関係した論文を選別するには、労力と時間がかかるばかりでなく、見逃してしまう可能が高い。このスクリーニングでの作業が問題となっている。
また、これまで報告されているSRに関係したソフトでは、論文の分類を行うが文献の選択を行うことができないものや、PubMed unique identifier という論文の番号を入力しなくてはならないものがある。これらは、SRの作業の効率化に役立たないため、一般的に使用されていない。 (Conventional problems related to clinical questions)
Medical guidelines collect as much evidence (medical papers) as possible related to clinical questions (CQs), conduct a systematic review (SR), and then formulate statements based on the results.
In the SR procedure, first, as a primary screening, (1) create a literature list that widely collects papers related to CQ from databases such as MEDLINE, and (2) select papers related to CQ based on the abstract. .. Next, as a secondary screening, the selected papers are read and the papers that meet the criteria are selected. After that, the therapeutic effect and bias of each selected paper are evaluated, and the total evidence is evaluated. Until now, SR has been performed mainly by human hands.
In the primary screening, the search formula does not include outcomes to pick up a wide range of outcomes that include not only subjects, exposure factors, and interventions, but also harms and disadvantages. As a result, the number of papers on the list is very large, and the number of paper candidates can be thousands. Selecting CQ-related papers from this list based on title and abstract is not only laborious and time-consuming, but also likely to be overlooked. The work in this screening is a problem.
In addition, some SR-related software that has been reported so far cannot classify articles but cannot select documents, and some require you to enter the number of the article called PubMed unique identifier. These are not commonly used because they do not help improve the efficiency of SR work.

（従来の文献検索）
検索語や検索文によって指定されるユーザの検索要求に適合する文献の検出や適合する順に文章を並び替える情報検索方法として、以下が知られている。 (Conventional literature search)
The following are known as information retrieval methods for detecting documents that match the user's search request specified by a search term or a search sentence and rearranging the sentences in the order of matching.

「TensorFlowを使った機械学習を論文抽出に適用、ヒントは大学入試問題対策の裏ワザ（非特許文献１）」では、システマティックレビューを対象として、テキストマイニングと機械学習を用いて文献のスクリーニングを実施した内容が開示されている。 In "Machine learning using TensorFlow is applied to dissertation extraction, hints are behind the scenes of measures for university entrance exams (Non-Patent Document 1)", literature screening is carried out using text mining and machine learning for systematic reviews. The contents of the study are disclosed.

特許文献１では、「コンピュータを用いて生物医学分野におけるキーワードを抽出するシステムであって、生物医学分野における文献データを蓄積した文献データベースと、生物医学分野における用語の基本的カテゴリを示す概念語の入力を受け付ける概念語入力部と、前記文献データ中で、前記入力された概念語と共起する共起語を検索し、その共起件数を計数する共起解析部と、検索された各共起語について、前記概念語との関連の強さを計算し、共起語、その共起件数及び関連の強さを含んだ共起リストを生成する共起リスト作成部と、生成した共起リストに含まれる各共起語について、前記文献データ中で、当該共起語と共起する単語を検索し、その共起件数を計数するキーワード計算部と、前記検索された単語のうち前記共起件数が一定数以上であるものをキーワードとして抽出するキーワード抽出部とを含んだシステム。」を開示している。 In Patent Document 1, "a system for extracting keywords in the biomedical field using a computer, a literature database accumulating literature data in the biomedical field, and conceptual words indicating basic categories of terms in the biomedical field. A co-occurrence analysis unit that searches for co-occurrence words that co-occur with the input concept word in the literature data and counts the number of co-occurrence cases, and each of the searched co-occurrence analysis units. For co-occurrence words, a co-occurrence list creation unit that calculates the strength of association with the concept word and generates a co-occurrence list including the co-occurrence words, the number of co-occurrence cases thereof, and the strength of association, and the generated co-occurrence For each co-occurrence word included in the list, a keyword calculation unit that searches for words that co-occur with the co-occurrence word in the literature data and counts the number of co-occurrence cases, and the co-occurrence of the searched words. A system that includes a keyword extraction unit that extracts keywords with a certain number of cases or more. "

「N-gram 全文検索と概念検索を融合した文書検索方式の検討（非特許文献２）」は、「文書検索において、文書全体からキーワードの存在を検証する“全文検索”と、文書内容とキーワードとの類似性を検証する“概念検索”とを融合することで、互いの欠点を補完するとともに検索精度の向上を図る技術が開示されている。そして概念検索においては、形態素解析に基づいて文書ベクトルと検索クエリ（検索者が入力した単語，フレーズ，文）ベクトルとの類似性・一致度を計算すること」を開示している。 "Examination of a document search method that combines N-gram full-text search and concept search (Non-Patent Document 2)" is "In document search," full-text search "that verifies the existence of keywords from the entire document, and document content and keywords. A technology is disclosed that complements each other's shortcomings and improves search accuracy by fusing with "concept search" that verifies the similarity with the document. And in concept search, documents are based on morphological analysis. "Calculating the degree of similarity / matching between a vector and a search query (word, phrase, sentence entered by a searcher) vector" is disclosed.

「隣接情報を用いた類似文書検索とリランキング（非特許文献３）」は、「特許調査や文献検索を対象として、クエリ文書（検索者が指定した文書）と意味的に類似した文書を検索する技術」を開示している。 "Similar document search and re-ranking using adjacent information (non-patent document 3)" is a search for documents that are semantically similar to the query document (document specified by the searcher) for patent search and document search. "Technology to do" is disclosed.

「ランキング結果を自由に閲覧するための再ランキングインターフェース（非特許文献４）」は、「様々な検索結果（ランキング結果）をユーザのインターラクションに応じて、すなわち気になった単語を追加選択したり単語の重要度を指定したりすることで、検索結果を再ランキングするためのシステム」を開示している。 The "re-ranking interface for freely viewing ranking results (Non-Patent Document 4)" selects "various search results (ranking results) according to the user's interaction, that is, additional words of interest. It discloses a system for re-ranking search results by specifying the importance of words or words.

上記の文献は、いずれも本発明の文章検索方法の特徴的な構成である効率的な文章の絞り込み工程、検索要求に適合した文章の漏れを防止する工程及び検索結果の評価工程を開示又は示唆をしていない。 All of the above documents disclose or suggest an efficient sentence narrowing process, a process of preventing omission of a sentence conforming to a search request, and a process of evaluating search results, which are characteristic structures of the sentence search method of the present invention. Not doing.

特開２００８−２１０２８号公報Japanese Unexamined Patent Publication No. 2008-21028

https://www.atmarkit.co.jp/ait/articles/1712/21/news028.htmlhttps://www.atmarkit.co.jp/ait/articles/1712/21/news028.html 亀代泰三他，情報処理学会第67 回全国大会Taizo Kameshiro et al., IPSJ 67th National Convention 古川修平他，DEIM Forrum 2009A9-Shuhei Furukawa et al., DEIM Forrum 2009 A9- 山本岳洋他，WISS 2009Takehiro Yamamoto et al., WISS 2009

従来の文章検索方法の問題を解決すべく、効率的な文章の絞り込み、検索要求に適合した文章の漏れを防止する及び／又は検索結果の評価が可能な文章検索システムを提供することを課題とする。 In order to solve the problem of the conventional sentence search method, the problem is to provide a sentence search system that can efficiently narrow down sentences, prevent omission of sentences that meet the search request, and / or evaluate the search results. To do.

本発明の文章検索システム等は、以下の構成、工程又は手段を含むことにより、本発明の課題のいずれか１を解決することを見出して本発明を完成した。 The text search system and the like of the present invention have completed the present invention by finding that any one of the problems of the present invention can be solved by including the following configuration, process or means.

本発明は以下の通りである。
１．検索要求に適合する文章を検索する情報検索システムであって、
（１）複数の文章を含む文章集合を受け付ける文章集合受信部、
（２）ユーザの検索要求を受け付けるユーザ検索要求受信部、
（３）該各文章及び該検索要求をそれぞれ単語ベクトルに変換する単語ベクトル変換部、
（４）各文章由来の単語ベクトル及び検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する文章ベクトル変換部、
（５）各文章由来の文章ベクトルと検索要求由来の文章ベクトルとの間の類似度を計算する類似度計算部、
（６）該文章と該検索要求との間の類似度を提示する検索結果提示部、
を含む情報検索システム。
２．さらに、
（１１）前記文章を選択するための検索語を受け付ける検索語受信部、
（１２）該検索語を検索語由来の単語ベクトルに変換する検索語由来の単語ベクトル変換部、
（１３）前記（３）の各文章由来の単語ベクトルが該検索語由来の単語ベクトルと一致又は類似した場合には、前記（４）の各文章由来の単語ベクトルに選択する文章選択部、
を含む前項１に記載の情報検索システム。
３．さらに、
（７）前記ユーザによる前記（６）で提示した文章から検索要求に適合する文章又は適合しない文章の選択結果を受け付けるユーザ文章判定部、
を含む前項１又は２に記載の情報検索システム。
４．さらに、２回目のスクリーニングとして、
（３−１）前記（７）でユーザが選択した文章を除く文章集合の各文章並びに前記（７）で検索要求に適合した文章を追加した前記（３）の検索要求をそれぞれ単語ベクトルに変換する単語ベクトル変換部２、
（４−１）（３−１）の該文章由来の単語ベクトル及び該検索要求に適合した文章を追加した検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する文章ベクトル変換部２、
（５−１）（４−１）の該文章ベクトルと該検索要求に適合した文章を追加した検索要求由来の文章ベクトルとの間の類似度を計算する類似度計算部２、
（６−１）該文章と該検索要求に適合した文章を追加した検索要求との間の類似度を提示する検索結果提示部２、
を含む前項３に記載の情報検索システム。
５．さらに、
（７−１）前記ユーザによる前記（６−１）で提示した文章から検索要求に適合する文章又は適合しない文章の選択結果を受け付けるユーザ文章判定部２、
を含む前項４に記載の情報検索システム。
６．さらに、３回目のスクリーニングとして、
（３−２）前記（７）及び（７−１）でユーザが選択した文章を除く文章集合の各文章並びに前記（７）及び（７−１）で検索要求に適合した文章又は前記（７−１）で検索要求に適合した文章を追加した前記（３）の検索要求をそれぞれ単語ベクトルに変換する単語ベクトル変換部３、
（４−２）（３−２）の該各文章由来の単語ベクトル及び該検索要求に適合した文章を追加した検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する文章ベクトル変換部３、
（５−２）（４−２）の該各文章由来の文章ベクトルと該検索要求に適合した文章を追加した検索要求由来の文章ベクトルとの間の類似度を計算する類似度計算部３、
（６−２）該文章と該検索要求に適合した文章を追加した検索要求との間の類似度を提示する検索結果提示部３
（７−２）該ユーザによる（６−２）で提示した文章から検索要求に適合する文章又は適合しない文章の選択結果を受け付けるユーザ文章判定部３、
を含む前項５に記載の情報検索システム。
７．さらに、
（８）前記（５）、（５−１）又は（５−２）の各文章の標準偏差、又は、前記（７）、（７−１）又は（７−２）のユーザが選択した各文章の標準偏差を計算する標準偏差計算部、
（９）該標準偏差を提示する標準偏差結果提示部、
を含む前項１〜６のいずれか１に記載の情報検索システム。
８．さらに、
（１０）各スクリーニング段階での標準偏差を比較する標準偏差比較部、
を含む前項７に記載の情報検索システム。
９．検索要求に適合する文章を検索する情報検索システムであって、
（１）複数の文章を含む文章集合を受け付ける文章集合受信部、
（２）ユーザの検索要求を受け付けるユーザ検索要求受信部、
（３）該各文章及び該検索要求をそれぞれ単語ベクトルに変換する単語ベクトル変換部、
（４）各文章由来の単語ベクトル及び検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する文章ベクトル変換部、
（５）各文章由来の文章ベクトルと検索要求由来の文章ベクトルとの間の類似度を計算する類似度計算部、
（６）該文章と該検索要求との間の類似度を提示する検索結果提示部、
（７）該ユーザによる（６）で提示した文章から検索要求に適合する文章の選択結果を受信するユーザ文章判定部、
（８）（６）で提示した文章又は（７）で該検索要求に適合した文章の標準偏差を計算する標準偏差計算部、
（９）該標準偏差を提示する標準偏差結果提示部、
を含む情報検索システム。
１０．さらに、
（１４）複数のユーザの前記（７）で検索要求に適合した文章を提示するユーザ検索結果比較部、
を有する前項３〜９のいずれか１に記載の情報検索システム。
１１．前記検索要求が臨床疑問である前項１〜１０のいずれか１に記載の情報検索システム。
１２．さらに、以下のいずれかの１以上を有する前項１〜１１のいずれか１に記載の情報検索システム
（１）検索結果出力部
（２）文章出力部
１３．検索要求に適合する文章を検索する情報検索方法であって、
（１）複数の文章を含む文章集合を受け付ける工程、
（２）ユーザの検索要求を受け付ける工程、
（３）該各文章及び該検索要求をそれぞれ単語ベクトルに変換する工程、
（４）各文章由来の単語ベクトル及び検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する工程、
（５）各文章由来の文章ベクトルと検索要求由来の文章ベクトルとの間の類似度を計算する工程、
（６）該文章と該検索要求との間の類似度を提示する工程、
を含む情報検索方法。
１４．さらに、
（１１）前記文章を選択するための検索語を受け付ける工程、
（１２）該検索語を検索語由来の単語ベクトルに変換する検索語由来の単語ベクトルに変換する工程、
（１３）前記（３）の各文章由来の単語ベクトルが該検索語由来の単語ベクトルと一致又は類似した場合には、前記（４）の各文章由来の単語ベクトルに選択する工程、
を含む前項１３に記載の情報検索方法。
１５．さらに、
（７）前記ユーザによる前記（６）で提示した文章から検索要求に適合する文章又は適合しない文章の選択結果を受け付ける工程、
を含む前項１３又は１４に記載の情報検索方法。
１６．さらに、２回目のスクリーニングとして、
（３−１）前記（７）でユーザが選択した文章を除く文章集合の各文章並びに前記（７）で検索要求に適合した文章を追加した前記（３）の検索要求をそれぞれ単語ベクトルに変換する工程、
（４−１）（３−１）の該文章由来の単語ベクトル及び該検索要求に適合した文章を追加した検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する工程、
（５−１）（４−１）の該文章ベクトルと該検索要求に適合した文章を追加した検索要求由来の文章ベクトルとの間の類似度を計算する工程、
（６−１）該文章と該検索要求に適合した文章を追加した検索要求との間の類似度を提示する工程、
を含む前項１５に記載の情報検索方法。
１７．さらに、
（７−１）前記ユーザによる前記（６−１）で提示した文章から検索要求に適合する文章又は適合しない文章の選択結果を受け付ける工程、
を含む前項１６に記載の情報検索方法。
１８．さらに、３回目のスクリーニングとして、
（３−２）前記（７）及び（７−１）でユーザが選択した文章を除く文章集合の各文章並びに前記（７）及び（７−１）で検索要求に適合した文章又は前記（７−１）で検索要求に適合した文章を追加した前記（３）の検索要求をそれぞれ単語ベクトルに変換する工程、
（４−２）（３−２）の該各文章由来の単語ベクトル及び該検索要求に適合した文章を追加した検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する工程、
（５−２）（４−２）の該各文章由来の文章ベクトルと該検索要求に適合した文章を追加した検索要求由来の文章ベクトルとの間の類似度を計算する工程、
（６−２）該文章と該検索要求に適合した文章を追加した検索要求との間の類似度を提示する工程、
（７−２）該ユーザによる（６−２）で提示した文章から検索要求に適合する文章又は適合しない文章の選択結果を受け付ける工程、
を含む前項１７に記載の情報検索方法。
１９．さらに、
（８）前記（５）、（５−１）又は（５−２）の各文章の標準偏差、又は、前記（７）、（７−１）又は（７−２）のユーザが選択した各文章の標準偏差を計算する工程、
（９）該標準偏差を提示する工程、
を含む前項１３〜１８のいずれか１に記載の情報検索方法。
２０．さらに、
（１０）各スクリーニング段階での標準偏差を比較する工程、
を含む前項１９に記載の情報検索方法。
２１．検索要求に適合する文章を検索する情報検索方法であって、
（１）複数の文章を含む文章集合を受け付ける工程、
（２）ユーザの検索要求を受け付ける工程、
（３）該各文章及び該検索要求をそれぞれ単語ベクトルに変換する工程、
（４）各文章由来の単語ベクトル及び検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する工程、
（５）各文章由来の文章ベクトルと検索要求由来の文章ベクトルとの間の類似度を計算する工程、
（６）該文章と該検索要求との間の類似度を提示する工程、
（７）該ユーザによる（６）で提示した文章から検索要求に適合する文章の選択結果を受信する工程、
（８）（６）で提示した文章又は（７）で該検索要求に適合した文章の標準偏差を計算する工程、
（９）該標準偏差を提示する工程、
を含む情報検索方法。
２２．さらに、
（１４）複数のユーザの前記（７）で検索要求に適合した文章を提示する工程、
を有する前項１５〜２１のいずれか１に記載の情報検索方法。
２３．前記検索要求が臨床疑問である前項１３〜２２のいずれか１に記載の情報検索方法。
２４．さらに、以下のいずれかの１以上を有する前項１３〜２３のいずれか１に記載の情報検索方法。
（１）検索結果提示工程
（２）文章出力工程
２５．検索要求に適合する文章を検索するプログラムであって、
（１）複数の文章を含む文章集合を受け付ける文章集合受信手段、
（２）ユーザの検索要求を受け付けるユーザ検索要求受信手段、
（３）該各文章及び該検索要求をそれぞれ単語ベクトルに変換する単語ベクトル変換手段、
（４）各文章由来の単語ベクトル及び検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する文章ベクトル変換手段、
（５）各文章由来の文章ベクトルと検索要求由来の文章ベクトルとの間の類似度を計算する類似度計算手段、
（６）該文章と該検索要求との間の類似度を提示する検索結果提示手段、
を含む情報検索プログラム。
２６．さらに、
（１１）前記文章を選択するための検索語を受け付ける検索語受信手段、
（１２）該検索語を検索語由来の単語ベクトルに変換する検索語由来の単語ベクトル変換手段、
（１３）前記（３）の各文章由来の単語ベクトルが該検索語由来の単語ベクトルと一致又は類似した場合には、前記（４）の各文章由来の単語ベクトルに選択する文章選択手段、
を含む前項２５に記載の情報検索プログラム。
２７．さらに、
（７）前記ユーザによる前記（６）で提示した文章から検索要求に適合する文章又は適合しない文章の選択結果を受け付けるユーザ文章判定手段、
を含む前項２５又は２６に記載の情報検索プログラム。
２８．さらに、２回目のスクリーニングとして、
（３−１）前記（７）でユーザが選択した文章を除く文章集合の各文章並びに前記（７）で検索要求に適合した文章を追加した前記（３）の検索要求をそれぞれ単語ベクトルに変換する単語ベクトル変換手段、
（４−１）（３−１）の該文章由来の単語ベクトル及び該検索要求に適合した文章を追加した検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する文章ベクトル変換手段、
（５−１）（４−１）の該文章ベクトルと該検索要求に適合した文章を追加した検索要求由来の文章ベクトルとの間の類似度を計算する類似度計算手段、
（６−１）該文章と該検索要求に適合した文章を追加した検索要求との間の類似度を提示する検索結果提示手段、
を含む前項２７に記載の情報検索プログラム。
２９．さらに、
（７−１）前記ユーザによる前記（６−１）で提示した文章から検索要求に適合する文章又は適合しない文章の選択結果を受け付けるユーザ文章判定手段、
を含む前項２８に記載の情報検索プログラム。
３０．さらに、３回目のスクリーニングとして、
（３−２）前記（７）及び（７−１）でユーザが選択した文章を除く文章集合の各文章並びに前記（７）及び（７−１）で検索要求に適合した文章又は前記（７−１）で検索要求に適合した文章を追加した前記（３）の検索要求をそれぞれ単語ベクトルに変換する単語ベクトル変換手段、
（４−２）（３−２）の該各文章由来の単語ベクトル及び該検索要求に適合した文章を追加した検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する文章ベクトル変換手段、
（５−２）（４−２）の該各文章由来の文章ベクトルと該検索要求に適合した文章を追加した検索要求由来の文章ベクトルとの間の類似度を計算する類似度計算手段、
（６−２）該文章と該検索要求に適合した文章を追加した検索要求との間の類似度を提示する検索結果提示手段、
（７−２）該ユーザによる（６−２）で提示した文章から検索要求に適合する文章又は適合しない文章の選択結果を受け付けるユーザ文章判定手段、
を含む前項２９に記載の情報検索プログラム。
３１．さらに、
（８）前記（５）、（５−１）又は（５−２）の各文章の標準偏差、又は、前記（７）、（７−１）又は（７−２）のユーザが選択した各文章の標準偏差を計算する標準偏差計算手段、
（９）該標準偏差を提示する標準偏差結果提示手段、
を含む前項２５〜３０のいずれか１に記載の情報検索プログラム。
３２．さらに、
（１０）各スクリーニング段階での標準偏差を比較する標準偏差比較手段、
を含む前項３１に記載の情報検索プログラム。
３３．検索要求に適合する文章を検索するプログラムであって、
（１）複数の文章を含む文章集合を受け付ける文章集合受信手段、
（２）ユーザの検索要求を受け付けるユーザ検索要求受信手段、
（３）該各文章及び該検索要求をそれぞれ単語ベクトルに変換する単語ベクトル変換手段、
（４）各文章由来の単語ベクトル及び検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する文章ベクトル変換手段、
（５）各文章由来の文章ベクトルと検索要求由来の文章ベクトルとの間の類似度を計算する類似度計算手段、
（６）該文章と該検索要求との間の類似度を提示する検索結果提示手段、
（７）該ユーザによる（６）で提示した文章から検索要求に適合する文章の選択結果を受信するユーザ文章判定手段、
（８）（６）で提示した文章又は（７）で該検索要求に適合した文章の標準偏差を計算する標準偏差計算手段、
（９）該標準偏差を提示する標準偏差結果提示手段、
を含む情報検索プログラム。
３４．さらに、
（１４）複数のユーザの前記（７）で検索要求に適合した文章を提示するユーザ検索結果比較手段、
を有する前項２７〜３３のいずれか１に記載の情報検索プログラム。
３５．前記検索要求が臨床疑問である前項２５〜３４のいずれか１に記載のプログラム。
３６．さらに、以下のいずれかの１以上を有する前項２５〜３５のいずれか１に記載のプログラム。
（１）検索結果出力手段
（２）文章出力手段 The present invention is as follows.
1. 1. An information retrieval system that searches for sentences that meet the search requirements.
(1) A sentence set receiver that accepts a sentence set containing multiple sentences,
(2) User search request receiver that accepts user search requests,
(3) A word vector conversion unit that converts each sentence and the search request into a word vector.
(4) A sentence vector conversion unit that changes a word vector derived from each sentence and a word vector derived from a search request into a sentence vector by a trained neural network.
(5) Similarity calculation unit that calculates the similarity between the sentence vector derived from each sentence and the sentence vector derived from the search request.
(6) A search result presentation unit that presents the degree of similarity between the sentence and the search request,
Information retrieval system including.
2. 2. further,
(11) A search term receiving unit that accepts a search term for selecting the sentence.
(12) A word vector conversion unit derived from a search term that converts the search term into a word vector derived from the search term.
(13) When the word vector derived from each sentence of the above (3) matches or is similar to the word vector derived from the search word, the sentence selection unit selects the word vector derived from each sentence of the above (4).
The information retrieval system according to item 1 above.
3. 3. further,
(7) A user sentence judgment unit that accepts a selection result of a sentence that matches or does not match the search request from the sentence presented in (6) by the user.
The information retrieval system according to the preceding paragraph 1 or 2, including the above.
4. In addition, as a second screening,
(3-1) Convert each sentence of the sentence set excluding the sentence selected by the user in the above (7) and the search request of the above (3) to which the sentence matching the search request is added in the above (7) into a word vector. Word vector conversion unit 2,
(4-1) Sentence vector conversion that changes the word vector derived from the sentence in (3-1) and the word vector derived from the search request to which a sentence matching the search request is added into a sentence vector by a trained neural network. Part 2,
(5-1) Similarity calculation unit 2 for calculating the similarity between the sentence vector of (4-1) and the sentence vector derived from the search request to which the sentence matching the search request is added.
(6-1) Search result presentation unit 2, which presents the degree of similarity between the sentence and the search request to which a sentence matching the search request is added.
The information retrieval system according to item 3 above.
5. further,
(7-1) User sentence determination unit 2, which accepts the selection result of a sentence that matches or does not match the search request from the sentence presented in (6-1) by the user.
The information retrieval system according to item 4 above.
6. Furthermore, as the third screening,
(3-2) Each sentence of the sentence set excluding the sentence selected by the user in (7) and (7-1) above, and the sentence matching the search request in (7) and (7-1) above or the above (7). -1) The word vector conversion unit 3, which converts each of the search requests in (3) above into a word vector, to which sentences matching the search request are added.
(4-2) A sentence vector that changes the word vector derived from each sentence in (3-2) and the word vector derived from the search request to which a sentence matching the search request is added into a sentence vector by a trained neural network. Conversion unit 3,
(5-2) Similarity calculation unit 3 for calculating the similarity between the sentence vector derived from each sentence in (4-2) and the sentence vector derived from the search request to which a sentence matching the search request is added.
(6-2) Search result presentation unit 3 that presents the degree of similarity between the sentence and the search request to which a sentence matching the search request is added.
(7-2) User sentence determination unit 3, which accepts the selection result of a sentence that matches or does not match the search request from the sentence presented in (6-2) by the user.
The information retrieval system according to item 5 above.
7. further,
(8) The standard deviation of each sentence of (5), (5-1) or (5-2), or each selected by the user of (7), (7-1) or (7-2). Standard deviation calculation unit that calculates the standard deviation of sentences,
(9) Standard deviation result presentation unit that presents the standard deviation,
The information retrieval system according to any one of items 1 to 6 above.
8. further,
(10) Standard deviation comparison unit that compares the standard deviations at each screening stage,
The information retrieval system according to item 7 above.
9. An information retrieval system that searches for sentences that meet the search requirements.
(1) A sentence set receiver that accepts a sentence set containing multiple sentences,
(2) User search request receiver that accepts user search requests,
(3) A word vector conversion unit that converts each sentence and the search request into a word vector.
(4) A sentence vector conversion unit that changes a word vector derived from each sentence and a word vector derived from a search request into a sentence vector by a trained neural network.
(5) Similarity calculation unit that calculates the similarity between the sentence vector derived from each sentence and the sentence vector derived from the search request.
(6) A search result presentation unit that presents the degree of similarity between the sentence and the search request,
(7) A user sentence judgment unit that receives a selection result of a sentence matching the search request from the sentence presented in (6) by the user.
(8) The standard deviation calculation unit that calculates the standard deviation of the sentence presented in (6) or the sentence matching the search request in (7).
(9) Standard deviation result presentation unit that presents the standard deviation,
Information retrieval system including.
10. further,
(14) A user search result comparison unit that presents sentences matching the search request in the above (7) of a plurality of users.
The information retrieval system according to any one of items 3 to 9 above.
11. The information retrieval system according to any one of the preceding items 1 to 10, wherein the search request is a clinical question.
12. Further, the information retrieval system according to any one of the above items 1 to 11 having any one or more of the following (1) search result output unit (2) text output unit 13. An information retrieval method that searches for sentences that meet the search request.
(1) A process of accepting a sentence set containing a plurality of sentences,
(2) The process of accepting user search requests,
(3) A step of converting each sentence and the search request into a word vector.
(4) A process of changing a word vector derived from each sentence and a word vector derived from a search request into a sentence vector by a trained neural network.
(5) The process of calculating the similarity between the sentence vector derived from each sentence and the sentence vector derived from the search request.
(6) A step of presenting the degree of similarity between the sentence and the search request,
Information retrieval method including.
14. further,
(11) A process of accepting a search term for selecting the sentence,
(12) Converting the search term into a word vector derived from the search term A step of converting the search term into a word vector derived from the search term.
(13) When the word vector derived from each sentence of the above (3) matches or is similar to the word vector derived from the search word, the step of selecting the word vector derived from each sentence of the above (4).
The information retrieval method according to the preceding paragraph 13, including.
15. further,
(7) A process of accepting a selection result of a sentence that matches or does not match the search request from the sentence presented in (6) above by the user.
The information retrieval method according to the preceding paragraph 13 or 14, including.
16. In addition, as a second screening,
(3-1) Each sentence of the sentence set excluding the sentence selected by the user in the above (7) and the search request of the above (3) to which the sentence matching the search request is added in the above (7) are converted into word vectors. Process to do,
(4-1) The step of changing the word vector derived from the sentence and the word vector derived from the search request to which the sentence matching the search request is added to the sentence vector by the learned neural network, respectively.
(5-1) The step of calculating the similarity between the sentence vector of (4-1) and the sentence vector derived from the search request to which the sentence matching the search request is added.
(6-1) A step of presenting the degree of similarity between the sentence and a search request to which a sentence matching the search request is added.
The information retrieval method according to the preceding paragraph 15, including the above.
17. further,
(7-1) A step of accepting a selection result of a sentence that matches or does not match the search request from the sentence presented in (6-1) by the user.
The information retrieval method according to the preceding paragraph 16, including.
18. Furthermore, as the third screening,
(3-2) Each sentence of the sentence set excluding the sentence selected by the user in (7) and (7-1) above, and the sentence matching the search request in (7) and (7-1) above or the above (7). The process of converting the search request of (3) above into a word vector by adding a sentence matching the search request in -1).
(4-2) The step of changing the word vector derived from each sentence and the word vector derived from the search request to which the sentence matching the search request is added to the sentence vector by the learned neural network, respectively.
(5-2) The step of calculating the similarity between the sentence vector derived from each sentence in (4-2) and the sentence vector derived from the search request to which a sentence matching the search request is added.
(6-2) A step of presenting the degree of similarity between the sentence and a search request to which a sentence matching the search request is added.
(7-2) A process of accepting a selection result of a sentence that matches or does not match the search request from the sentence presented in (6-2) by the user.
The information retrieval method according to the preceding paragraph 17, including.
19. further,
(8) The standard deviation of each sentence of (5), (5-1) or (5-2), or each selected by the user of (7), (7-1) or (7-2). The process of calculating the standard deviation of a sentence,
(9) Step of presenting the standard deviation,
The information retrieval method according to any one of items 13 to 18 above.
20. further,
(10) Step of comparing standard deviations at each screening stage,
The information retrieval method according to the preceding paragraph 19, including.
21. An information retrieval method that searches for sentences that meet the search request.
(1) A process of accepting a sentence set containing a plurality of sentences,
(2) The process of accepting user search requests,
(3) A step of converting each sentence and the search request into a word vector.
(4) A process of changing a word vector derived from each sentence and a word vector derived from a search request into a sentence vector by a trained neural network.
(5) The process of calculating the similarity between the sentence vector derived from each sentence and the sentence vector derived from the search request.
(6) A step of presenting the degree of similarity between the sentence and the search request,
(7) A step of receiving a selection result of a sentence matching the search request from the sentence presented in (6) by the user.
(8) The step of calculating the standard deviation of the sentence presented in (6) or the sentence matching the search request in (7).
(9) Step of presenting the standard deviation,
Information retrieval method including.
22. further,
(14) A step of presenting a sentence matching the search request in (7) above of a plurality of users.
The information retrieval method according to any one of items 15 to 21 above.
23. The information retrieval method according to any one of items 13 to 22 above, wherein the search request is a clinical question.
24. The information retrieval method according to any one of items 13 to 23 above, which has any one or more of the following.
(1) Search result presentation process (2) Sentence output process 25. A program that searches for sentences that meet your search requirements.
(1) Sentence set receiving means for accepting a sentence set containing a plurality of sentences,
(2) User search request receiving means for accepting user search requests,
(3) A word vector conversion means for converting each sentence and the search request into a word vector.
(4) A sentence vector conversion means for changing a word vector derived from each sentence and a word vector derived from a search request into a sentence vector by a trained neural network.
(5) Similarity calculation means for calculating the similarity between the sentence vector derived from each sentence and the sentence vector derived from the search request.
(6) Search result presenting means for presenting the degree of similarity between the sentence and the search request,
Information retrieval program including.
26. further,
(11) Search term receiving means for accepting search terms for selecting the sentence,
(12) A word vector conversion means derived from a search term that converts the search term into a word vector derived from the search term.
(13) When the word vector derived from each sentence of (3) above matches or is similar to the word vector derived from the search word, the sentence selection means for selecting the word vector derived from each sentence of (4) above.
25. The information retrieval program according to the preceding paragraph 25.
27. further,
(7) A user sentence determination means that accepts a selection result of a sentence that matches or does not match the search request from the sentence presented in (6) above by the user.
The information retrieval program according to the preceding paragraph 25 or 26, which includes.
28. In addition, as a second screening,
(3-1) Convert each sentence of the sentence set excluding the sentence selected by the user in the above (7) and the search request of the above (3) to which the sentence matching the search request is added in the above (7) into a word vector. Word vector conversion means,
(4-1) Sentence vector conversion that changes the word vector derived from the sentence in (3-1) and the word vector derived from the search request to which a sentence matching the search request is added into a sentence vector by a trained neural network. means,
(5-1) A similarity calculation means for calculating the similarity between the sentence vector of (4-1) and the sentence vector derived from the search request to which a sentence matching the search request is added.
(6-1) A search result presenting means for presenting the degree of similarity between the sentence and a search request to which a sentence matching the search request is added.
The information retrieval program according to item 27 above.
29. further,
(7-1) A user sentence determination means that accepts a selection result of a sentence that matches or does not match the search request from the sentence presented in (6-1) by the user.
28. The information retrieval program according to the preceding paragraph 28.
30. Furthermore, as the third screening,
(3-2) Each sentence of the sentence set excluding the sentence selected by the user in (7) and (7-1) above, and the sentence matching the search request in (7) and (7-1) above or the above (7). A word vector conversion means for converting each of the search requests in (3) above into a word vector by adding a sentence suitable for the search request in -1).
(4-2) A sentence vector that changes the word vector derived from each sentence in (3-2) and the word vector derived from the search request to which a sentence matching the search request is added into a sentence vector by a trained neural network. Conversion means,
(5-2) A similarity calculation means for calculating the similarity between the sentence vector derived from each sentence in (4-2) and the sentence vector derived from the search request to which a sentence matching the search request is added.
(6-2) A search result presenting means for presenting the degree of similarity between the sentence and a search request to which a sentence matching the search request is added.
(7-2) A user sentence determination means that accepts a selection result of a sentence that matches or does not match the search request from the sentence presented in (6-2) by the user.
29. The information retrieval program according to the preceding paragraph 29.
31. further,
(8) The standard deviation of each sentence of (5), (5-1) or (5-2), or each selected by the user of (7), (7-1) or (7-2). Standard deviation calculation method for calculating the standard deviation of sentences,
(9) Standard deviation result presenting means for presenting the standard deviation,
The information retrieval program according to any one of the preceding paragraphs 25 to 30, including.
32. further,
(10) Standard deviation comparison means for comparing the standard deviations at each screening stage,
The information retrieval program according to the preceding paragraph 31, which includes.
33. A program that searches for sentences that meet your search requirements.
(1) Sentence set receiving means for accepting a sentence set containing a plurality of sentences,
(2) User search request receiving means for accepting user search requests,
(3) A word vector conversion means for converting each sentence and the search request into a word vector.
(4) A sentence vector conversion means for changing a word vector derived from each sentence and a word vector derived from a search request into a sentence vector by a trained neural network.
(5) Similarity calculation means for calculating the similarity between the sentence vector derived from each sentence and the sentence vector derived from the search request.
(6) Search result presenting means for presenting the degree of similarity between the sentence and the search request,
(7) A user sentence determination means for receiving the selection result of a sentence matching the search request from the sentence presented in (6) by the user.
(8) A standard deviation calculation means for calculating the standard deviation of the sentence presented in (6) or the sentence conforming to the search request in (7).
(9) Standard deviation result presenting means for presenting the standard deviation,
Information retrieval program including.
34. further,
(14) A user search result comparison means for presenting sentences matching the search request in the above (7) of a plurality of users.
The information retrieval program according to any one of items 27 to 33 above.
35. The program according to any one of items 25 to 34 above, wherein the search request is a clinical question.
36. The program according to any one of items 25 to 35 above, further comprising any one or more of the following.
(1) Search result output means (2) Sentence output means

本発明の文章検索システム等は、以下のいずれか１以上の効果を有する。
（１）効率的な文章の絞り込み
（２）検索要求に適合した文章の漏れを防止
（３）検索結果の評価
（４）検索結果の出力
（５）文章の出力 The text search system and the like of the present invention have any one or more of the following effects.
(1) Efficient narrowing down of sentences (2) Preventing omission of sentences that meet search requests (3) Evaluation of search results (4) Output of search results (5) Output of sentences

本発明の情報検索システムの概要図（矢印は、データの流れを示す）。Schematic diagram of the information retrieval system of the present invention (arrows indicate data flow). 本発明の検索要求に適合する文章を検索する情報検索システムの流れ図。The flow chart of the information retrieval system which searches the sentence which meets the search request of this invention. 本発明の情報検索システムのユーザ操作画面の例。An example of a user operation screen of the information retrieval system of the present invention.

本発明の検索要求に適合する文章を検索する情報検索システム（情報検索装置を含む）、情報検索方法及び情報検索プログラムは図１及び図２を用いて下記で説明する。なお、本発明の情報検索システムは、図１及び図２の構成に限定されない。また、図１及び図２の構成は、あくまで一例であり、一つの機能をさらに分割し、複数の機能部をまとめて一つの機能部として構成してもよく、さらに一部の機能をインターネット回線を通じて日本国外に設置しても良い。 An information retrieval system (including an information retrieval device), an information retrieval method, and an information retrieval program for searching sentences that meet the search requirements of the present invention will be described below with reference to FIGS. 1 and 2. The information retrieval system of the present invention is not limited to the configurations shown in FIGS. 1 and 2. Further, the configurations of FIGS. 1 and 2 are merely examples, and one function may be further divided and a plurality of functional units may be collectively configured as one functional unit, and some functions may be further configured as an Internet line. It may be installed outside Japan through.

（複数の文章を含む文章集合を受け付ける文章集合受信部）
複数の文章を含む文章集合を受け付ける文章集合受信部１は、ユーザが選択した複数の文章を含む文章集合を各データベースから入手（ダウンロード）できれば、特に限定されない。加えて、文章集合受信部は、公知のプログラミング言語（例、Python等）により文章集合を各文章に分割する。
本発明での「ユーザ」は、本発明のシステムの利用者を意味するが、１人に限定される必要はない。複数のユーザA,B,Cが同じ文章集合を使用して文章を検索することができ、さらに選択した文章を比較評価することができる。
本発明での「文章（文献を含む）」は、複数の文を有すれば特に限定されないが、例えば、論文、学会発表、特許文献、書籍、インターネット情報等を意味する。加えて、文章は、全体だけでなく一部（例、要約等）のみを対象としても良い。
本発明での「文章集合」は、複数の文章を含めば特に限定されないが、例えば、論文データベース（Pubmed）、各国の特許データ（JPO）、学会予稿集、電子ジャーナル、電子書籍、WEBサイト等に収録された電子情報からなる文章の集合を意味する。 (Sentence set receiver that accepts a sentence set containing multiple sentences)
The sentence set receiving unit 1 that accepts a sentence set including a plurality of sentences is not particularly limited as long as a sentence set including a plurality of sentences selected by the user can be obtained (downloaded) from each database. In addition, the sentence set receiving unit divides the sentence set into each sentence by a known programming language (eg, Python, etc.).
The "user" in the present invention means a user of the system of the present invention, but is not limited to one person. Multiple users A, B, and C can search for sentences using the same sentence set, and can compare and evaluate selected sentences.
The "text (including literature)" in the present invention is not particularly limited as long as it has a plurality of texts, but means, for example, a paper, a conference presentation, a patent literature, a book, Internet information, or the like. In addition, the text may cover only a part (eg, summary, etc.) as well as the whole.
The "sentence set" in the present invention is not particularly limited as long as a plurality of sentences are included, but for example, a dissertation database (Pubmed), patent data (JPO) of each country, academic proceedings, electronic journals, electronic books, websites, etc. It means a set of sentences consisting of electronic information recorded in.

（ユーザの検索要求を受け付けるユーザ検索要求受信部）
ユーザの検索要求を受け付けるユーザ検索要求受信部２では、ユーザが作成した（指定した）検索要求を受け付ける。これにより、本発明のシステムが検索要求を認識する。
検索要求（クエリ）は、検索文だけでなく、検索語、複数の文章でも良い。例えば、検索文として、臨床疑問を例示することができる。 (User search request receiver that accepts user search requests)
The user search request receiving unit 2 that accepts the user's search request receives the (designated) search request created by the user. As a result, the system of the present invention recognizes the search request.
The search request (query) may be not only a search sentence but also a search term and a plurality of sentences. For example, a clinical question can be exemplified as a search sentence.

（単語ベクトル変換部）
単語ベクトル変換部３では、文章集合受信部１で受け付けた複数の文章を含む文章集合の一部又は全部並びにユーザ検索要求受信部２で受け付けた検索要求の一部又は全部をそれぞれ単語ベクトルに変換する。単語ベクトル変換部３は、単語列をその単語列の意味を表す単語ベクトルに変換することで、単語の意味を数値化する。単語ベクトル変換部３は、形態素解析部等により生成された単語列を、単語ベクトル列に変換する。
単語ベクトル変換部３は、例えば、既存のPythonパッケージ「genism」、「NaturalLanguage Tool Kit」等を使用してベクトル化する。別例として、Ｗｏｒｄ２Ｖｅｃ［Ｍｉｋｏｌｏｖ１３］（単語をベクトル化して表現する定量化手法）などを用いて、単語列を単語ベクトル列に変換しても良い。
例えば、単語ベクトルは単語の種類及びその出現数を示すことができる（例、単語が３０個の場合は、３０次元のベクトルになり、ベクトルの長さが出現数に比例する）。
他の例として、各単語に特徴となるベクトルをそれぞれ与えることも可能である。 (Word vector conversion part)
The word vector conversion unit 3 converts a part or all of a sentence set including a plurality of sentences received by the sentence set receiving unit 1 and a part or all of the search request received by the user search request receiving unit 2 into word vectors. To do. The word vector conversion unit 3 digitizes the meaning of a word by converting the word string into a word vector representing the meaning of the word string. The word vector conversion unit 3 converts the word string generated by the morphological analysis unit or the like into a word vector sequence.
The word vector conversion unit 3 vectorizes using, for example, the existing Python packages “genism”, “Natural Language Tool Kit”, and the like. As another example, a word string may be converted into a word vector string by using Word2Vec [Mikolov13] (a quantification method for expressing words by vectorizing them).
For example, the word vector can indicate the type of word and the number of occurrences thereof (for example, when there are 30 words, it becomes a 30-dimensional vector, and the length of the vector is proportional to the number of occurrences).
As another example, it is also possible to give each word a characteristic vector.

（文章ベクトル変換部）
文章ベクトル変換部４では、各文章由来の単語ベクトル及び検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更できれば特に限定されない。学習済みのニューラルネットワークは、例えば、文献IDと前後の単語から対象単語を推測するニューラルネットワーク(Doc2Vec)を使用できる。
より詳しくは、文章としてPubmedで検索できる論文の要約を使用する場合は、訓練データとして、Pubmedで検索できる各論文の要約集（文章集合）を使用する。そして、学習の例として、冠詞や代名詞などの頻用される単語や共通して多かった語の上位３％をストップワード辞書として使用して単語を削除する、検索要求に重要な単語又は不適節な単語（特徴語）に重み付をする（TF-IDF）、（MTF-IDF）等を行う。 (Sentence vector conversion part)
The sentence vector conversion unit 4 is not particularly limited as long as the word vector derived from each sentence and the word vector derived from the search request can be changed into sentence vectors by the trained neural network. As the trained neural network, for example, a neural network (Doc2Vec) that infers the target word from the document ID and the surrounding words can be used.
More specifically, when using the abstracts of articles that can be searched by Pubmed as sentences, use the abstracts (sentence sets) of each article that can be searched by Pubmed as training data. Then, as an example of learning, the top 3% of frequently used words such as articles and pronouns and common words are used as a stop word dictionary to delete words, which are important words or inappropriate words for search requests. Weight words (characteristic words) (TF-IDF), (MTF-IDF), etc.

（類似度計算部）
類似度計算部５では、文章ベクトル間又は単語ベクトル間の類似度を計算できれば特に限定されないが、例えば、自体公知の測定方法であるコサイン類似度、ユーグリッド距離、標準化ユークリッド距離、マハラノビス距離、マンハッタン距離、チェビシェフ距離、ミンコフスキー距離、ロジスティック回帰、ピアソンの相関係数、Jaccard係数、Dice係数、Simpson係数等を使用することができる。 (Similarity calculation unit)
The similarity calculation unit 5 is not particularly limited as long as it can calculate the similarity between sentence vectors or word vectors, but for example, cosine similarity, Euclidean distance, standardized Euclidean distance, Maharanobis distance, and Manhattan, which are known measurement methods themselves. Distance, Chebyshev distance, Minkowski distance, logistic regression, Pearson correlation coefficient, Jaccard coefficient, Dice coefficient, Simpson coefficient, etc. can be used.

（類似度を提示する検索結果提示部）
類似度を提示する検索結果提示部６では、各文章と検索要求との間の類似度をユーザ等に提示することができれば特に限定されない。例えば、検索要求との類似度が高い文章順にランキング形式での提示が好ましい。加えて、類似度が高い文章の上位１〜５０、１〜３０、１〜２０等のみを提示しても良い。
また、類似度を提示する検索結果提示部６は類似度の閾値設定部１５を含んでも良い。類似度閾値設定部１５は、予め設定した閾値以上又は以下の文章を提示することができる。 (Search result presentation section that presents the degree of similarity)
The search result presentation unit 6 that presents the similarity is not particularly limited as long as the similarity between each sentence and the search request can be presented to the user or the like. For example, it is preferable to present in a ranking format in the order of sentences having a high degree of similarity to the search request. In addition, only the top 1-50, 1-30, 1-20, etc. of sentences with high similarity may be presented.
Further, the search result presenting unit 6 for presenting the similarity may include the threshold setting unit 15 for the similarity. The similarity threshold setting unit 15 can present sentences equal to or less than a preset threshold.

（ユーザ文章判定部）
ユーザ文章判定部７では、類似度を提示する検索結果提示部６で提示された文章から検索要求に適合する文章又は適合しない文章のユーザによる選択結果を受け付ける。これにより、本発明のシステムでは、１回目のスクリーニングで選択された文章（検索要求に適合した文章及び検索要求に適合しなかった文章）を認識する。 (User sentence judgment unit)
The user sentence determination unit 7 accepts a selection result by the user of a sentence that matches the search request or a sentence that does not match the search request from the sentences presented by the search result presentation unit 6 that presents the similarity. As a result, the system of the present invention recognizes the sentences (sentences that meet the search request and sentences that do not meet the search request) selected in the first screening.

（２回目のスクリーニング）
本発明のシステムでは、検索要求（SD：SearchDemand）に適合した文章の漏れを防止するために、必要に応じて、２回目のスクリーニングを行う。２回目のスクリーニングの特徴として、ユーザが１回目のスクリーニングで選択した１又は２以上の文章（M1：検索要求に適合した文章MD1、又は検索要求に適合した文章MD1＋検索要求に適合しなかった文章NMD1（M1=MD1+NMD1））を除いた文章集合（X−M1）及び１回目のスクリーニングで検索要求に適合した文章（MD1）を追加した検索要求（MD1＋SD）を使用する。本発明のシステムの特徴として、１回目のスクリーニングで検索要求に適合した文章（MD1）を追加した検索要求（MD1＋SD）を使用することにより、２回目のスクリーニングでは検索要求と類似した文章をユーザに提供することができる。これにより、ユーザが１回目のスクリーニングでは確認していない類似度が高い文章を効率的に選択することができる。
例えば、以下により２回目のスクリーニングを実施することができる。
〇単語ベクトル変換部３−２
１回目のスクリーニングでユーザが選択した文章を除く文章集合の各文章並びに１回目のスクリーニングで検索要求に適合した文章を追加した検索要求をそれぞれ単語ベクトルに変換する。
なお、単語ベクトル変換部３−２は、単語ベクトル変換部３に内蔵されていても良いし、外部に接続されていても良い。
〇文章ベクトル変換部４−２
上記文章由来の単語ベクトル及び１回目のスクリーニングで検索要求に適合した文章を追加した検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する。
なお、文章ベクトル変換部４−２は、文章ベクトル変換部４に内蔵されていても良いし、外部に接続されていても良い。
〇類似度計算部５−２
上記文章ベクトルと１回目のスクリーニングで検索要求に適合した文章を追加した検索要求由来の文章ベクトルとの間の類似度を計算する。
なお、類似度計算部５−２は、類似度計算部５に内蔵されていても良いし、外部に接続されていても良い。
〇類似度を提示する検索結果提示部６−２
上記文章と１回目のスクリーニングで検索要求に適合した文章を追加した検索要求との間の類似度を提示する。
なお、類似度を提示する検索結果提示部６−２は、類似度を提示する検索結果提示部６に内蔵されていても良いし、外部に接続されていても良い。
〇ユーザ文章判定部７−２
上記に提示した文章から検索要求に適合する文章又は適合しない文章のユーザによる選択結果を受け付ける。
なお、ユーザ文章判定部７−２は、ユーザ文章判定部７内蔵されていても良いし、外部に接続されていても良い。 (Second screening)
In the system of the present invention, in order to prevent omission of sentences conforming to the search request (SD: SearchDemand), a second screening is performed as necessary. As a feature of the second screening, one or more sentences selected by the user in the first screening (M1: sentence MD1 conforming to the search request, or sentence MD1 conforming to the search request + sentence not conforming to the search request) Use the sentence set (X-M1) excluding NMD1 (M1 = MD1 + NMD1) and the search request (MD1 + SD) with the sentence (MD1) matching the search request added in the first screening. As a feature of the system of the present invention, by using a search request (MD1 + SD) in which a sentence (MD1) matching the search request is added in the first screening, a sentence similar to the search request is sent to the user in the second screening. Can be provided. As a result, it is possible to efficiently select sentences having a high degree of similarity that the user has not confirmed in the first screening.
For example, a second screening can be performed as follows.
〇 Word vector conversion unit 3-2
Each sentence of the sentence set excluding the sentence selected by the user in the first screening and the search request to which the sentence matching the search request is added in the first screening are converted into word vectors.
The word vector conversion unit 3-2 may be built in the word vector conversion unit 3 or may be connected to the outside.
〇Sentence vector conversion unit 4-2
The word vector derived from the above sentence and the word vector derived from the search request to which the sentence matching the search request is added in the first screening are changed to the sentence vector by the learned neural network.
The sentence vector conversion unit 4-2 may be built in the sentence vector conversion unit 4 or may be connected to the outside.
〇Similarity calculation unit 5-2
The similarity between the above sentence vector and the sentence vector derived from the search request to which the sentence matching the search request is added in the first screening is calculated.
The similarity calculation unit 5-2 may be built in the similarity calculation unit 5 or may be connected to the outside.
〇 Search result presentation unit 6-2 that presents the degree of similarity
The degree of similarity between the above sentence and the search request to which the sentence matching the search request is added in the first screening is presented.
The search result presentation unit 6-2 that presents the similarity may be built in the search result presentation unit 6 that presents the similarity, or may be connected to the outside.
〇User sentence judgment unit 7-2
From the sentences presented above, the user's selection result of sentences that meet or do not match the search request is accepted.
The user sentence determination unit 7-2 may be built in the user sentence determination unit 7 or may be connected to the outside.

（３回目のスクリーニング）
本発明のシステムでは、検索要求（SD）に適合した文章の漏れを防止するために、必要に応じて、３回目のスクリーニングを行う。３回目のスクリーニングの特徴として、ユーザが１回目のスクリーニングで選択した１又は２以上の文章及び２回目のスクリーニングで選択した１又は２以上の文章（M２：検索要求に適合した文章MD２、又は検索要求に適合した文章MD２＋検索要求に適合しなかった文章NMD２（M2=MD2+NMD2））を除いた文章集合（X−M1−M2）並びに１回目及び2回目のスクリーニングで検索要求に適合した文章（MD2）を追加した検索要求（MD1＋MD2＋SD）又は2回目のスクリーニングで検索要求に適合した文章（MD2）を追加した検索要求（MD2＋SD）を使用する。
本発明のシステムの特徴として、好ましくは、１回目及び2回目のスクリーニングで検索要求に適合した文章（MD2）を追加した検索要求（MD1＋MD2＋SD）を使用することにより、３回目のスクリーニングでは検索要求と類似した文章をユーザに提供することができる。これにより、ユーザが１回目及び２回目又は２回目のスクリーニングでは確認していない類似度が高い文章を効率的に選択することができる。
例えば、以下により３回目のスクリーニングを実施することができる。
〇単語ベクトル変換部３−３
１回目及び２回目のスクリーニングでユーザが選択した文章を除く文章集合の各文章並びに１回目及び２回目のスクリーニングで検索要求に適合した文章又は２回目のスクリーニングで検索要求に適合した文章を追加した検索要求をそれぞれ単語ベクトルに変換する。
なお、単語ベクトル変換部３−３は、単語ベクトル変換部３に内蔵されていても良いし、外部に接続されていても良い。
〇文章ベクトル変換部４−３
上記文章由来の単語ベクトル並びに１回目及び２回目のスクリーニングで検索要求に適合した文章又は２回目のスクリーニングで検索要求に適合した文章を追加した検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する。
なお、文章ベクトル変換部４−３は、文章ベクトル変換部４に内蔵されていても良いし、外部に接続されていても良い。
〇類似度計算部５−３
上記文章ベクトル並びに１回目及び２回目のスクリーニングで検索要求に適合した文章又は２回目のスクリーニングで検索要求に適合した文章を追加した検索要求由来の文章ベクトルとの間の類似度を計算する。
なお、類似度計算部５−３は、類似度計算部５に内蔵されていても良いし、外部に接続されていても良い。
〇類似度を提示する検索結果提示部６−３
上記文章と１回目及び２回目のスクリーニングで検索要求に適合した文章又は２回目のスクリーニングで検索要求に適合した文章を追加した検索要求との間の類似度を提示する。
なお、類似度を提示する検索結果提示部６−３は、類似度を提示する検索結果提示部６に内蔵されていても良いし、外部に接続されていても良い。
〇ユーザ文章判定部７−３
上記に提示した文章から検索要求に適合する又は適合しない文章のユーザによる選択結果を受け付ける。
なお、ユーザ文章判定部７−３は、ユーザ文章判定部７内蔵されていても良いし、外部に接続されていても良い。 (Third screening)
In the system of the present invention, a third screening is performed as necessary in order to prevent omission of sentences conforming to the search request (SD). As a feature of the third screening, one or more sentences selected by the user in the first screening and one or more sentences selected in the second screening (M2: sentence MD2 suitable for the search request, or search Sentences that meet the requirements MD2 + sentences that do not meet the search requirements NMD2 (M2 = MD2 + NMD2)) is excluded from the sentence set (X-M1-M2) and sentences that meet the search requirements in the first and second screenings. Use the search request (MD1 + MD2 + SD) with (MD2) added or the search request (MD2 + SD) with the sentence (MD2) that matches the search request in the second screening.
As a feature of the system of the present invention, preferably, by using a search request (MD1 + MD2 + SD) in which a sentence (MD2) suitable for the search request is added in the first and second screenings, the search request is displayed in the third screening. Similar sentences can be provided to the user. This makes it possible for the user to efficiently select sentences with high similarity that have not been confirmed in the first and second or second screenings.
For example, a third screening can be performed by:
〇 Word vector conversion unit 3-3
Added sentences in the sentence set excluding sentences selected by the user in the first and second screenings, sentences matching the search request in the first and second screenings, or sentences matching the search request in the second screening. Convert each search request into a word vector.
The word vector conversion unit 3-3 may be built in the word vector conversion unit 3 or may be connected to the outside.
〇Sentence vector conversion unit 4-3
The word vector derived from the above sentence and the word vector derived from the search request obtained by adding the sentence matching the search request in the first and second screenings or the sentence matching the search request in the second screening are obtained by the trained neural network. Change to a sentence vector.
The sentence vector conversion unit 4-3 may be built in the sentence vector conversion unit 4 or may be connected to the outside.
〇Similarity calculation unit 5-3
The similarity between the above sentence vector and the sentence vector derived from the search request to which the sentence conforming to the search request in the first and second screenings or the sentence conforming to the search request in the second screening is added is calculated.
The similarity calculation unit 5-3 may be built in the similarity calculation unit 5 or may be connected to the outside.
〇 Search result presentation unit 6-3 that presents the degree of similarity
The degree of similarity between the above sentence and the sentence that matches the search request in the first and second screenings or the search request that adds the sentence that matches the search request in the second screening is presented.
The search result presentation unit 6-3 that presents the similarity may be built in the search result presentation unit 6 that presents the similarity, or may be connected to the outside.
〇User sentence judgment unit 7-3
Accepts user selection results of sentences that meet or do not meet the search request from the sentences presented above.
The user sentence determination unit 7-3 may be built in the user sentence determination unit 7 or may be connected to the outside.

（４回目以降のスクリーニング）
本発明のシステムの４回目以降のスクリーニングは、上記３回目のスクリーニングと同様に行うことができる。本発明のシステムは、必要に応じて、５回目、６回目、７回目、８回目のスクリーニングを実施することができる。
４回目のスクリーニングの特徴として、ユーザが１回目のスクリーニングで選択した１又は２以上文章、２回目のスクリーニングで選択した１又は２以上文章並びに３回目のスクリーニングで選択した１又は２以上文章（M３：検索要求に適合した文章MD３、又は検索要求に適合した文章MD３＋検索要求に適合しなかった文章NMD３、すなわち、M3=MD3+NMD3）を除いた文章集合（X−M1−M2−M3）並びに１回目、２回目及び３回目のスクリーニングで検索要求に適合した文章（MD1＋MD2＋MD3）、３回目のスクリーニングで検索要求に適合した文章（MD3）、１回目及び３回目のスクリーニングで検索要求に適合した文章（MD1＋MD3）又は２回目及び３回目のスクリーニングで検索要求に適合した文章（MD2＋MD3）を追加した検索要求（MD1＋MD2＋MD3＋SD、MD3＋SD、MD1＋MD3＋SD又はMD2＋MD3＋SD）を使用する。 (Screening after the 4th time)
The fourth and subsequent screenings of the system of the present invention can be performed in the same manner as the third screening. The system of the present invention can perform the fifth, sixth, seventh, and eighth screenings as needed.
As a feature of the fourth screening, one or two or more sentences selected by the user in the first screening, one or two or more sentences selected in the second screening, and one or two or more sentences selected in the third screening (M3). : Sentence MD3 conforming to the search request, or sentence MD3 conforming to the search request + sentence NMD3 not conforming to the search request, that is, a sentence set (X-M1-M2-M3) excluding M3 = MD3 + NMD3) Sentences that meet the search request in the first, second, and third screenings (MD1 + MD2 + MD3), sentences that meet the search request in the third screening (MD3), and sentences that meet the search request in the first and third screenings. Use (MD1 + MD3) or a search request (MD1 + MD2 + MD3 + SD, MD3 + SD, MD1 + MD3 + SD or MD2 + MD3 + SD) to which sentences (MD2 + MD3) matching the search request are added in the second and third screenings.

（標準偏差計算部）
本発明のシステムでは、検索結果の客観的な評価ために、必要に応じて、標準偏差を算出する。
標準偏差計算部８では、各文章（文献）の標準偏差を算出する。標準偏差は、自体公知の方法を使用することができる。例えば、以下を例示することができるが特に限定されない。
〇１回目のスクリーニング
各文献（全部、一部、類似度上位の文献：Ｘ）nと検索要求（SD）との類似度xiを基にして類似度平均xを算出する。該文献の類似度xiと類似度平均xの差分を求め、該差分を２乗し、その総和を算出する。偏差の和S（式１）を各文献数ｎで割って、分散を算出して、該分散の平方根を算出して標準偏差（式２）とする。
〇２回目のスクリーニング
文献（ユーザが１回目のスクリーニングで選択した１又は２以上文章を除いた文章集合（X−M1））ｎと１回目のスクリーニングで検索要求に適合した文章（MD1）を追加した検索要求（MD1＋SD）を用いて、上記と同様に標準偏差を算出する。
〇３回目のスクリーニング
文献（X−M1−M2）nと１回目と2回目のスクリーニングで検索要求に適合した文章（MD1＋MD2）を追加した検索要求（MD1＋MD2＋SD）又は2回目のスクリーニングで検索要求に適合した文章（MD2）を追加した検索要求（MD2＋SD）を用いて、上記と同様に標準偏差を算出する。
〇４回目のスクリーニング
文献（X−M1−M2−M3）nと１回目、２回目及び３回目のスクリーニングで検索要求に適合した文章（MD1＋MD2＋MD3）、３回目のスクリーニングで検索要求に適合した文章（MD3）、１回目及び３回目のスクリーニングで検索要求に適合した文章（MD1＋MD3）又は２回目及び３回目のスクリーニングで検索要求に適合した文章（MD2＋MD3）を追加した検索要求（MD1＋MD2＋MD3＋SD、MD3＋SD、MD1＋MD3＋SD又はMD2＋MD3＋SD）を用いて、上記と同様に標準偏差を算出する。 (Standard deviation calculation unit)
In the system of the present invention, the standard deviation is calculated as necessary for the objective evaluation of the search result.
The standard deviation calculation unit 8 calculates the standard deviation of each sentence (literature). For the standard deviation, a method known per se can be used. For example, the following can be exemplified, but the present invention is not particularly limited.
〇 First screening Calculate the average similarity x based on the similarity xi between each document (all, part, documents with high similarity: X) n and the search request (SD). The difference between the similarity xi and the average similarity x of the document is obtained, the difference is squared, and the sum is calculated. Divide the sum of deviations S (expression 1) by the number of documents n to calculate the variance, and calculate the square root of the variance to obtain the standard deviation (expression 2).
〇 Second screening document (sentence set excluding one or more sentences selected by the user in the first screening (X-M1)) n and sentences matching the search request (MD1) in the first screening are added. The standard deviation is calculated in the same manner as above using the search request (MD1 + SD).
〇Third screening document (X-M1-M2) n and search request (MD1 + MD2 + SD) with sentences (MD1 + MD2) that match the search request in the first and second screenings or match the search request in the second screening The standard deviation is calculated in the same manner as above by using the search request (MD2 + SD) to which the written sentence (MD2) is added.
〇 4th screening literature (X-M1-M2-M3) n and sentences that meet the search request in the 1st, 2nd and 3rd screening (MD1 + MD2 + MD3) and sentences that meet the search request in the 3rd screening (MD1 + MD2 + MD3) MD3) Search request (MD1 + MD2 + MD3 + SD, MD3 + SD, MD1 + MD3 + SD or MD2 + MD3 + SD) is used to calculate the standard deviation in the same way as above.

（標準偏差結果提示部）
標準偏差結果提示部９は、標準偏差計算部８で計算した各スクリーニング段階の標準偏差を提示することができれば特に限定されない。標準偏差結果提示部９は、他の提示部に内蔵されていても良いし、外部に接続されていても良い。 (Standard deviation result presentation section)
The standard deviation result presenting unit 9 is not particularly limited as long as it can present the standard deviation of each screening stage calculated by the standard deviation calculation unit 8. The standard deviation result presentation unit 9 may be built in another presentation unit or may be connected to the outside.

（標準偏差比較部）
標準偏差比較部１０は、各スクリーニング段階での標準偏差を比較して、例えば、以下のような場合には、ユーザに警告又は助言する機能を有する。
（１）各スクリーニング段階の標準偏差が閾値（ユーザが自由に設定できる設定値）と比較して低い値で一定している場合、スクリーニングを完了しても良いとの助言をする。
（２）各スクリーニング段階の標準偏差が閾値と比較して高い値で一定している場合、検索要求、選択した文章集合、及び／又は検索語の再設定の警告をする。
（３）各スクリーニング段階の標準偏差が下降している場合、スクリーニングを完了しても良いとの助言をする。
（４）各スクリーニング段階の標準偏差が上昇している場合、検索要求、選択した文章集合、及び／又は検索語の再設定の警告をする。 (Standard deviation comparison section)
The standard deviation comparison unit 10 has a function of comparing the standard deviations at each screening stage and warning or giving advice to the user in the following cases, for example.
(1) If the standard deviation of each screening stage is constant at a low value compared to the threshold value (set value that can be freely set by the user), it is advised that the screening may be completed.
(2) If the standard deviation of each screening stage is constant at a high value compared to the threshold value, a search request, a selected sentence set, and / or a warning for resetting the search term is issued.
(3) If the standard deviation of each screening stage is decreasing, we advise that the screening may be completed.
(4) If the standard deviation of each screening stage is increasing, a search request, a selected sentence set, and / or a warning for resetting the search term is issued.

本発明のシステムでは、効率的な文章の絞り込みのために、必要に応じて、検索語により文章集合から文章を絞り込む。
（検索語受信部）
検索語受信部１１では、ユーザが選択した文章集合から文章を絞り込むための検索語を受け付ける。これにより、本発明のシステムは検索語を認識する。なお、検索語は、特に限定されないが、例えば、キーワードを利用することができる。キーワードとして、論文登録番号（例、PMID : PubMed登録番号）、タイトル、発行日、抄録、著者のフルネーム、著者、著者の所属機関、MeSH用語で付与された文献の主題、出版物名（巻、号、ページ、発行日）、言語等を例示することができるが特に限定されない。 In the system of the present invention, in order to efficiently narrow down sentences, sentences are narrowed down from a sentence set by search terms as needed.
(Search term receiver)
The search term receiving unit 11 accepts search terms for narrowing down sentences from a sentence set selected by the user. Thereby, the system of the present invention recognizes the search term. The search term is not particularly limited, but for example, a keyword can be used. Keywords include article registration number (eg PMID: PubMed registration number), title, publication date, abstract, author's full name, author, author's affiliation, subject of literature given in MeSH terminology, publication name (volume, volume, Issue, page, issue date), language, etc. can be exemplified, but are not particularly limited.

（検索語由来の単語ベクトル変換部）
検索語由来の単語ベクトル変換部１２は、検索語を検索語由来の単語ベクトルに変換する。
検索語由来の単語ベクトル変換部１２は、単語ベクトル変換部３に内蔵されていても良いし、外部に接続されていても良い。 (Word vector conversion part derived from search term)
The word vector conversion unit 12 derived from the search term converts the search term into a word vector derived from the search term.
The word vector conversion unit 12 derived from the search term may be built in the word vector conversion unit 3 or may be connected to the outside.

（文章選択部）
文章選択部１３は、各文章由来の単語ベクトルが検索語由来の単語ベクトルと一致又は類似した場合には、以後のスクリーニングに使用するように設定することができる。一致は、単語ベクトルの方向が一致したことを意味し、類似度はコサイン類似度、ユーグリッド距離、標準化ユークリッド距離、マハラノビス距離、マンハッタン距離、チェビシェフ距離、ミンコフスキー距離、ロジスティック回帰、ピアソンの相関係数、Jaccard係数、Dice係数、Simpson係数等から算出できる。なお、一致又は設定した類似度では選択された文章が少ない場合には、設定した類似度の幅を広げることにより選択した文章を増やすことができる。同様に、設定した類似度の幅を狭くすることにより選択した文章を減らすことができる。 (Sentence selection section)
The sentence selection unit 13 can be set to be used for subsequent screening when the word vector derived from each sentence matches or is similar to the word vector derived from the search word. Match means that the directions of the word vectors match, and similarity means cosine similarity, Euclidean distance, standardized Euclidean distance, Maharanobis distance, Manhattan distance, Chebyshev distance, Minkowski distance, logistic regression, Pearson's correlation coefficient. , Jaccard coefficient, Dice coefficient, Simpson coefficient, etc. If the number of selected sentences is small with the matching or set similarity, the number of selected sentences can be increased by widening the range of the set similarity. Similarly, the number of selected sentences can be reduced by narrowing the set range of similarity.

（ユーザ検索結果比較部）
ユーザ検索結果比較部１４は、複数のユーザが検索要求に適合したと判断した文章を提示することができる。例えば、ユーザAとユーザBが同じ文章集合（例、Pubmed）及び同じ検索要求を使用した場合において、一致した文章（論文）と一致していない文章をユーザに提示することができる。これにより、各ユーザは他のユーザの検索結果と比較して最終評価とすることができる。 (User search result comparison section)
The user search result comparison unit 14 can present sentences that are determined by a plurality of users to meet the search request. For example, when user A and user B use the same sentence set (eg Pubmed) and the same search request, it is possible to present to the user a sentence that matches and does not match the sentence (paper). As a result, each user can make a final evaluation by comparing with the search results of other users.

（検索結果出力部）
検索結果出力部１６は、検索結果のデータを出力することができる。これにより、ユーザは出力されたデータ（電子データ、例えば、CSVファイル等）を自由に加工することができる。検索結果は、検索要求の類似度順のリストだけでなく、論文名、著者、タイトル名、雑誌名等の各項目のリストも含んでもよい。 (Search result output section)
The search result output unit 16 can output the search result data. As a result, the user can freely process the output data (electronic data, for example, CSV file, etc.). The search result may include not only a list of search requests in order of similarity, but also a list of each item such as a paper title, an author, a title name, and a journal name.

（文章出力部）
文章出力部１７は、ユーザが所望の文献（文章）を選択することにより、該文献のデータベースから該文献を入手（ダウンロード）することができる。 (Sentence output section)
The sentence output unit 17 can obtain (download) the document from the database of the document by selecting a desired document (text) by the user.

本発明のシステムをコンピュータによって実現する場合、装置及びその各部が有する機能の処理内容はプログラムによって記述される。該プログラムは、ハードディスク装置等に格納されており、実行時には必要なプログラムやデータがＲＡＭに読み込まれる。その読み込まれたプログラムがＣＰＵにより実行されることにより、コンピュータ上で各処理内容が実現される。 When the system of the present invention is realized by a computer, the processing contents of the functions of the device and each part thereof are described by a program. The program is stored in a hard disk device or the like, and necessary programs and data are read into the RAM at the time of execution. When the read program is executed by the CPU, each processing content is realized on the computer.

（文章検索方法）
本発明の文章検索方法の工程の概要は以下の通りである。
（１）複数の文章を含む文章集合を受け付ける工程、
（２）ユーザの検索要求を受け付ける工程、
（３）該各文章及び該検索要求をそれぞれ単語ベクトルに変換する工程、
（４）各文章由来の単語ベクトル及び検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する工程、
（５）各文章由来の文章ベクトルと検索要求由来の文章ベクトルとの間の類似度を計算する工程、
（６）該文章と該検索要求との間の類似度を提示する工程。
本発明の別の文章検索方法の工程の概要は以下の通りである。
（１）複数の文章を含む文章集合を受け付ける工程、
（２）ユーザの検索要求を受け付ける工程、
（３）該各文章及び該検索要求をそれぞれ単語ベクトルに変換する工程、
（４）各文章由来の単語ベクトル及び検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する工程、
（５）各文章由来の文章ベクトルと検索要求由来の文章ベクトルとの間の類似度を計算する工程、
（６）該文章と該検索要求との間の類似度を提示する工程、
（７）該ユーザによる（６）で提示した文章から検索要求に適合する文章の選択結果を受信する工程、
（８）（６）で提示した文章又は（７）で検索要求に適合した文章の標準偏差を計算する工程、
（９）該標準偏差を提示する工程、
なお、本発明の文章検索方法の用語は、必要に応じて、上記で説明した本発明のシステムの用語を参照することができる。 (Sentence search method)
The outline of the process of the text search method of the present invention is as follows.
(1) A process of accepting a sentence set containing a plurality of sentences,
(2) The process of accepting user search requests,
(3) A step of converting each sentence and the search request into a word vector.
(4) A process of changing a word vector derived from each sentence and a word vector derived from a search request into a sentence vector by a trained neural network.
(5) The process of calculating the similarity between the sentence vector derived from each sentence and the sentence vector derived from the search request.
(6) A step of presenting the degree of similarity between the sentence and the search request.
The outline of the process of another text search method of the present invention is as follows.
(1) A process of accepting a sentence set containing a plurality of sentences,
(2) The process of accepting user search requests,
(3) A step of converting each sentence and the search request into a word vector.
(4) A process of changing a word vector derived from each sentence and a word vector derived from a search request into a sentence vector by a trained neural network.
(5) The process of calculating the similarity between the sentence vector derived from each sentence and the sentence vector derived from the search request.
(6) A step of presenting the degree of similarity between the sentence and the search request,
(7) A step of receiving a selection result of a sentence matching the search request from the sentence presented in (6) by the user.
(8) The process of calculating the standard deviation of the sentence presented in (6) or the sentence matching the search request in (7).
(9) Step of presenting the standard deviation,
As the term of the text search method of the present invention, the term of the system of the present invention described above can be referred to as necessary.

（文章検索プログラム）
本発明の文章検索プログラムの手段の概要は以下の通りである。
（１）複数の文章を含む文章集合を受け付ける文章集合受信手段、
（２）ユーザの検索要求を受け付けるユーザ検索要求受信手段、
（３）該各文章及び該検索要求をそれぞれ単語ベクトルに変換する単語ベクトル変換手段、
（４）各文章由来の単語ベクトル及び検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する文章ベクトル変換手段、
（５）各文章由来の文章ベクトルと検索要求由来の文章ベクトルとの間の類似度を計算する類似度計算手段、
（６）該文章と該検索要求との間の類似度を提示する検索結果提示手段、
本発明の別の文章検索プログラムの手段の概要は以下の通りである。
（１）複数の文章を含む文章集合を受け付ける文章集合受信手段、
（２）ユーザの検索要求を受け付けるユーザ検索要求受信手段、
（３）該各文章及び該検索要求をそれぞれ単語ベクトルに変換する単語ベクトル変換手段、
（４）各文章由来の単語ベクトル及び検索要求由来の単語ベクトルをそれぞれ学習済みのニューラルネットワークにより文章ベクトルに変更する文章ベクトル変換手段、
（５）各文章由来の文章ベクトルと検索要求由来の文章ベクトルとの間の類似度を計算する類似度計算手段、
（６）該文章と該検索要求との間の類似度を提示する検索結果提示手段、
（７）該ユーザによる（６）で提示した文章から検索要求に適合する文章の選択結果を受信するユーザ文章判定手段、
（８）（６）で提示した文章又は（７）で検索要求に適合した文章の標準偏差を計算する標準偏差計算手段、
（９）該標準偏差を提示する標準偏差結果提示手段、
なお、本発明の文章検索プログラムの用語は、必要に応じて、上記で説明した本発明のシステムの用語を参照することができる。 (Sentence search program)
The outline of the means of the text search program of the present invention is as follows.
(1) Sentence set receiving means for accepting a sentence set containing a plurality of sentences,
(2) User search request receiving means for accepting user search requests,
(3) A word vector conversion means for converting each sentence and the search request into a word vector.
(4) A sentence vector conversion means for changing a word vector derived from each sentence and a word vector derived from a search request into a sentence vector by a trained neural network.
(5) Similarity calculation means for calculating the similarity between the sentence vector derived from each sentence and the sentence vector derived from the search request.
(6) Search result presenting means for presenting the degree of similarity between the sentence and the search request,
The outline of the means of another text search program of the present invention is as follows.
(1) Sentence set receiving means for accepting a sentence set containing a plurality of sentences,
(2) User search request receiving means for accepting user search requests,
(3) A word vector conversion means for converting each sentence and the search request into a word vector.
(4) A sentence vector conversion means for changing a word vector derived from each sentence and a word vector derived from a search request into a sentence vector by a trained neural network.
(5) Similarity calculation means for calculating the similarity between the sentence vector derived from each sentence and the sentence vector derived from the search request.
(6) Search result presenting means for presenting the degree of similarity between the sentence and the search request,
(7) A user sentence determination means for receiving the selection result of a sentence matching the search request from the sentence presented in (6) by the user.
(8) A standard deviation calculation means for calculating the standard deviation of the sentence presented in (6) or the sentence matching the search request in (7).
(9) Standard deviation result presenting means for presenting the standard deviation,
As the term of the text search program of the present invention, the term of the system of the present invention described above can be referred to as necessary.

本発明のシステム、方法及びプログラムにおける各処理は、上記記載に従って時系列に実行されるのみならず、処理を実行する装置の処理能力あるいは必要に応じて並列的にあるいは個別に実行されてもよい。また、本発明のシステムの各機能は必要に応じ、併合・分割しても構わない。その他、本発明の趣旨を逸脱しない範囲で適宜変更が可能である。
本発明のシステムでは、WEBブラウザとWEBサーバーをリンクさせることにより、ユーザはWEBブラウザから必要な入力を行うことにより、本発明のシステムを稼働させるWEBサーバーから検索結果を得ることができる。加えて、本発明のシステムではWEBアプリとして実装することもでき、ユーザはコンピュータ言語の知識がなくても本発明のシステムを利用することができる。例えば、ホストコンピュータが文献検索システムのプログラムを保有しており、ユーザは、ホストコンピュータにアクセスする。
また、本発明の文献検索プログラムをコンピュータにダウンロードして、ユーザがインターネット回線に接続しているコンピュータで検索することもできる。
さらに、本発明のシステムでは、検索結果のファイルをテキストファイルとしてダウンロードすることができる機能を付与しても良い。 Each process in the system, method and program of the present invention is not only executed in chronological order according to the above description, but may also be executed in parallel or individually as required by the processing capacity of the device that executes the process. .. In addition, each function of the system of the present invention may be merged or divided as necessary. In addition, changes can be made as appropriate without departing from the spirit of the present invention.
In the system of the present invention, by linking the WEB browser and the WEB server, the user can obtain the search result from the WEB server that operates the system of the present invention by inputting necessary input from the WEB browser. In addition, the system of the present invention can be implemented as a WEB application, and the user can use the system of the present invention without knowledge of a computer language. For example, the host computer owns the program of the document search system, and the user accesses the host computer.
It is also possible to download the document search program of the present invention to a computer and search on a computer connected to the Internet line by the user.
Further, the system of the present invention may be provided with a function that allows the search result file to be downloaded as a text file.

（本発明のシステム、方法及びプログラムの用途）
本発明のシステム、方法及びプログラムは、上記説明したように、システマティックレビューに適しているので、臨床疑問（特に臨床疑問のガイドライン作成）、臨床研究のガイドライン作成、メタ解析等に好適に利用することができる。しかしながら、他の用途である特許文献検索、論文検索、インターネット情報からの目的の情報の入手、電子書籍検索等にも利用することができる。 (Use of the system, method and program of the present invention)
As explained above, the system, method and program of the present invention are suitable for systematic review, and therefore, they should be suitably used for clinical questions (particularly clinical question guideline preparation), clinical research guideline preparation, meta-analysis, etc. Can be done. However, it can also be used for other purposes such as patent document search, article search, acquisition of target information from Internet information, and electronic book search.

（ユーザの本発明のシステムのWEBブラウザでの使用例）
ユーザの本発明のシステムのWEBブラウザでの使用例を図２により説明する。
「ログイン画面」によりログインする。必要に応じてユーザ特有のパスワード入力を要求する。
「トップ画面」の「クリニカルクエスチョン」から、評価するCQを選択又は入力する。
「トップ画面」の「新規登録」ボタンを押し、使用する文章集合（例、PubMed）を登録する。
「トップ画面」の「文章集合一覧」から、評価する文章集合としてPubMedを選択して、「シミュレーション開始」ボタンを押す。
「シミュレーション実行画面」からシミュレーション条件（検索要求、又は検索要求及び検索語）を入力し、「シミュレーション実行」ボタンを押す。
「シミュレーション実行画面」のシミュレーション実行結果一覧から、評価する結果を選択して、抽出論文の詳細を確認する。
「シミュレーション実行画面」を確認して、「レビュー開始」ボタンを押す。担当評価又は最終評価を選択する。
「レビュー画面」担当者は各論文アブストラクトを評価し、「保存」、「破棄」又は「保留」ボタンのいずれかを押す。
「レビュー画面」各担当の評価完了後は、担当者2名で最終評価を実施する。評価方法は各担当操作と同じである。
「レビュー画面」全件の最終評価が完了したら、評価状況を「評価完了」に選択する。
「トップ画面」評価済みのPubMedを選択して、「更新」ボタンを押す。
「PubMed更新画面」進捗を「評価完了」にして、「更新」ボタンを押す。 (Example of using the system of the present invention by a user in a WEB browser)
An example of using the system of the present invention by a user in a WEB browser will be described with reference to FIG.
Log in using the "login screen". Prompt user-specific password input as needed.
Select or enter the CQ to evaluate from "Clinical Question" on the "Top Screen".
Click the "New Registration" button on the "Top Screen" to register the text set (eg PubMed) to use.
Select PubMed as the sentence set to be evaluated from the "Sentence set list" on the "Top screen", and press the "Start simulation" button.
Enter the simulation conditions (search request or search request and search term) from the "simulation execution screen" and press the "simulation execution" button.
Select the evaluation result from the simulation execution result list on the "Simulation execution screen" and check the details of the extracted paper.
Check the "Simulation execution screen" and press the "Start review" button. Select the responsible evaluation or the final evaluation.
The person in charge of the "review screen" evaluates each paper abstract and presses either the "save", "destroy" or "hold" button.
"Review screen" After the evaluation of each person in charge is completed, the final evaluation will be carried out by two people in charge. The evaluation method is the same as each operation in charge.
"Review screen" When the final evaluation of all cases is completed, select the evaluation status as "Evaluation completed".
"Top screen" Select the evaluated PubMed and press the "Update" button.
Set the progress of "PubMed update screen" to "evaluation completed" and press the "update" button.

（本発明のシステムの利用結果）
本システムを使用したCKD（Chronic Didney Disease）ガイドライン作成を従来の方法と比較した。
従来のPubMed＋EXCELを用いたCKDガイドライン作成には約１週間必要であった。また、１５本の論文を得ることができた。
本発明のシステムのパイロット版を使用してCKDガイドライン作成には２日間で終了した。６９本の論文を検索結果として得ることができた。
以上により、本システムは、効率的な文章の絞り込み、検索要求に適合した文章の漏れを防止しかつ検索結果の評価を行うことができることを確認した。 (Results of using the system of the present invention)
We compared the creation of CKD (Chronic Didney Disease) guidelines using this system with the conventional method.
It took about one week to create CKD guidelines using the conventional PubMed + EXCEL. In addition, 15 papers were obtained.
The creation of CKD guidelines using the pilot version of the system of the present invention was completed in 2 days. We were able to obtain 69 papers as search results.
From the above, it was confirmed that this system can efficiently narrow down sentences, prevent omission of sentences that meet the search request, and evaluate the search results.

Claims

An information retrieval system that searches for sentences that meet the search requirements.
(1) A sentence set receiver that accepts a sentence set containing multiple sentences,
(2) User search request receiver that accepts user search requests,
(3) A word vector conversion unit that converts each sentence and the search request into a word vector.
(4) A sentence vector conversion unit that changes a word vector derived from each sentence and a word vector derived from a search request into a sentence vector by a trained neural network.
(5) Similarity calculation unit that calculates the similarity between the sentence vector derived from each sentence and the sentence vector derived from the search request.
(6) A search result presentation unit that presents the degree of similarity between the sentence and the search request,
Information retrieval system including.

further,
(11) A search term receiving unit that accepts a search term for selecting the sentence.
(12) A word vector conversion unit derived from a search term that converts the search term into a word vector derived from the search term.
(13) When the word vector derived from each sentence of the above (3) matches or is similar to the word vector derived from the search word, the sentence selection unit selects the word vector derived from each sentence of the above (4).
The information retrieval system according to claim 1.

further,
(7) A user sentence judgment unit that accepts a selection result of a sentence that matches or does not match the search request from the sentence presented in (6) by the user.
The information retrieval system according to claim 1 or 2.

In addition, as a second screening,
(3-1) Convert each sentence of the sentence set excluding the sentence selected by the user in the above (7) and the search request of the above (3) to which the sentence matching the search request is added in the above (7) into a word vector. Word vector conversion unit 2,
(4-1) Sentence vector conversion that changes the word vector derived from the sentence in (3-1) and the word vector derived from the search request to which a sentence matching the search request is added into a sentence vector by a trained neural network. Part 2,
(5-1) Similarity calculation unit 2 for calculating the similarity between the sentence vector of (4-1) and the sentence vector derived from the search request to which the sentence matching the search request is added.
(6-1) Search result presentation unit 2, which presents the degree of similarity between the sentence and the search request to which a sentence matching the search request is added.
The information retrieval system according to claim 3.

further,
(7-1) User sentence determination unit 2, which accepts the selection result of a sentence that matches or does not match the search request from the sentence presented in (6-1) by the user.
The information retrieval system according to claim 4.

Furthermore, as the third screening,
(3-2) Each sentence of the sentence set excluding the sentence selected by the user in (7) and (7-1) above, and the sentence matching the search request in (7) and (7-1) above or the above (7). -1) The word vector conversion unit 3, which converts each of the search requests in (3) above into a word vector, to which sentences matching the search request are added.
(4-2) A sentence vector that changes the word vector derived from each sentence in (3-2) and the word vector derived from the search request to which a sentence matching the search request is added into a sentence vector by a trained neural network. Conversion unit 3,
(5-2) Similarity calculation unit 3 for calculating the similarity between the sentence vector derived from each sentence in (4-2) and the sentence vector derived from the search request to which a sentence matching the search request is added.
(6-2) Search result presentation unit 3, which presents the degree of similarity between the sentence and the search request to which a sentence matching the search request is added.
(7-2) User sentence determination unit 3, which accepts the selection result of a sentence that matches or does not match the search request from the sentence presented in (6-2) by the user.
The information retrieval system according to claim 5.

further,
(8) The standard deviation of each sentence of (5), (5-1) or (5-2), or each selected by the user of (7), (7-1) or (7-2). Standard deviation calculation unit that calculates the standard deviation of sentences,
(9) Standard deviation result presentation unit that presents the standard deviation,
The information retrieval system according to any one of claims 1 to 6.

further,
(10) Standard deviation comparison unit that compares the standard deviations at each screening stage,
The information retrieval system according to claim 7.

An information retrieval system that searches for sentences that meet the search requirements.
(1) A sentence set receiver that accepts a sentence set containing multiple sentences,
(2) User search request receiver that accepts user search requests,
(3) A word vector conversion unit that converts each sentence and the search request into a word vector.
(4) A sentence vector conversion unit that changes a word vector derived from each sentence and a word vector derived from a search request into a sentence vector by a trained neural network.
(5) Similarity calculation unit that calculates the similarity between the sentence vector derived from each sentence and the sentence vector derived from the search request.
(6) A search result presentation unit that presents the degree of similarity between the sentence and the search request,
(7) A user sentence judgment unit that receives a selection result of a sentence matching the search request from the sentence presented in (6) by the user.
(8) The standard deviation calculation unit that calculates the standard deviation of the sentence presented in (6) or the sentence matching the search request in (7).
(9) Standard deviation result presentation unit that presents the standard deviation,
Information retrieval system including.

further,
(14) A user search result comparison unit that presents sentences matching the search request in the above (7) of a plurality of users.
The information retrieval system according to any one of claims 3 to 9.

The information retrieval system according to any one of claims 1 to 10, wherein the retrieval request is a clinical question.

The information retrieval system according to any one of claims 1 to 11, further comprising any one or more of the following.
(1) Search result output section (2) Text output section

An information retrieval method that searches for sentences that meet the search request.
(1) A process of accepting a sentence set containing a plurality of sentences,
(2) The process of accepting user search requests,
(3) A step of converting each sentence and the search request into a word vector.
(4) A process of changing a word vector derived from each sentence and a word vector derived from a search request into a sentence vector by a trained neural network.
(5) The process of calculating the similarity between the sentence vector derived from each sentence and the sentence vector derived from the search request.
(6) A step of presenting the degree of similarity between the sentence and the search request,
Information retrieval method including.

An information retrieval method that searches for sentences that meet the search request.
(1) A process of accepting a sentence set containing a plurality of sentences,
(2) The process of accepting user search requests,
(3) A step of converting each sentence and the search request into a word vector.
(4) A process of changing a word vector derived from each sentence and a word vector derived from a search request into a sentence vector by a trained neural network.
(5) The process of calculating the similarity between the sentence vector derived from each sentence and the sentence vector derived from the search request.
(6) A step of presenting the degree of similarity between the sentence and the search request,
(7) A step of receiving a selection result of a sentence matching the search request from the sentence presented in (6) by the user.
(8) The step of calculating the standard deviation of the sentence presented in (6) or the sentence matching the search request in (7).
(9) Step of presenting the standard deviation,
Information retrieval method including.

A program that searches for sentences that meet your search requirements.
(1) Sentence set receiving means for accepting a sentence set containing a plurality of sentences,
(2) User search request receiving means for accepting user search requests,
(3) A word vector conversion means for converting each sentence and the search request into a word vector.
(4) A sentence vector conversion means for changing a word vector derived from each sentence and a word vector derived from a search request into a sentence vector by a trained neural network.
(5) Similarity calculation means for calculating the similarity between the sentence vector derived from each sentence and the sentence vector derived from the search request.
(6) Search result presenting means for presenting the degree of similarity between the sentence and the search request,
Information retrieval program including.

A program that searches for sentences that meet your search requirements.
(1) Sentence set receiving means for accepting a sentence set containing a plurality of sentences,
(2) User search request receiving means for accepting user search requests,
(3) A word vector conversion means for converting each sentence and the search request into a word vector.
(4) A sentence vector conversion means for changing a word vector derived from each sentence and a word vector derived from a search request into a sentence vector by a trained neural network.
(5) Similarity calculation means for calculating the similarity between the sentence vector derived from each sentence and the sentence vector derived from the search request.
(6) Search result presenting means for presenting the degree of similarity between the sentence and the search request,
(7) A user sentence determination means for receiving the selection result of a sentence matching the search request from the sentence presented in (6) by the user.
(8) A standard deviation calculation means for calculating the standard deviation of the sentence presented in (6) or the sentence matching the search request in (7).
(9) Standard deviation result presenting means for presenting the standard deviation,
Information retrieval program including.