CN119557462A - Information retrieval method, device, equipment, readable storage medium and program product - Google Patents
Information retrieval method, device, equipment, readable storage medium and program product Download PDFInfo
- Publication number
- CN119557462A CN119557462A CN202411757566.9A CN202411757566A CN119557462A CN 119557462 A CN119557462 A CN 119557462A CN 202411757566 A CN202411757566 A CN 202411757566A CN 119557462 A CN119557462 A CN 119557462A
- Authority
- CN
- China
- Prior art keywords
- information
- topic
- metadata
- prompt
- records
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/38—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/332—Query formulation
- G06F16/3329—Natural language query formulation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/335—Filtering based on additional data, e.g. user or group profiles
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Human Computer Interaction (AREA)
- Artificial Intelligence (AREA)
- Library & Information Science (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本申请涉及一种信息检索方法、装置、设备、可读存储介质和程序产品。所述方法包括：在接收到用户输入的信息查询语句的情况下，从信息查询语句中抽取出多个元数据；将多个元数据作为过滤条件，在对话记录数据库中进行检索得到多个目标话题片段；每个目标话题片段包括多个对话记录以及对应的话题；基于多个目标话题片段，确定信息查询提示词；基于信息查询提示词和大语言模型，得到大语言模型生成的信息检索结果。采用本方法能够提高检索结果的准确性。
The present application relates to an information retrieval method, device, equipment, readable storage medium and program product. The method comprises: extracting multiple metadata from the information query statement when receiving an information query statement input by a user; using the multiple metadata as filtering conditions, searching a conversation record database to obtain multiple target topic segments; each target topic segment includes multiple conversation records and corresponding topics; determining information query prompt words based on the multiple target topic segments; and obtaining information retrieval results generated by the large language model based on the information query prompt words and the large language model. The use of this method can improve the accuracy of the retrieval results.
Description
Technical Field
The present application relates to the field of artificial intelligence technology, and in particular, to an information retrieval method, apparatus, device, readable storage medium, and program product.
Background
Conventional chat software typically has a history dialogue search function that is implemented mainly by keyword retrieval. The user needs to enter a keyword and the system then returns a chat log that is related to the keyword. However, this method requires a great deal of effort for the user to recall and search keywords, and particularly when an accurate vocabulary cannot be recalled, there is a problem in that the retrieval result is inaccurate.
Disclosure of Invention
In view of the foregoing, it is desirable to provide an information retrieval method, apparatus, computer device, computer-readable storage medium, and computer program product that can improve the accuracy of the retrieval result.
In a first aspect, the present application provides an information retrieval method, including:
Under the condition that an information inquiry statement input by a user is received, extracting a plurality of metadata from the information inquiry statement;
searching in a dialogue record database by taking a plurality of metadata as filtering conditions to obtain a plurality of target topic fragments, wherein each target topic fragment comprises a plurality of dialogue records and corresponding topics;
determining information inquiry prompt words based on a plurality of target topic segments;
And obtaining an information retrieval result generated by the large language model based on the information query prompt word and the large language model.
In one embodiment, extracting a plurality of metadata from an information query statement includes:
The method comprises the steps of obtaining preset prompt words, wherein the preset prompt words comprise a requirement extraction template, texts to be extracted and information types to be extracted;
taking the information inquiry statement as a text to be extracted in preset prompt words to obtain combined prompt words;
Based on the combined hint words, a plurality of metadata are generated through a large language model.
In one embodiment, the plurality of metadata includes topic data, key query information, a dialogue object and a start-stop time point, and the searching in the dialogue record database by using the plurality of metadata as a filtering condition to obtain a plurality of target topic segments includes:
And searching a plurality of topic segments in the dialogue record database to obtain a plurality of target topic segments, wherein the relevance of the target topic segments and the metadata meets the preset relevance condition.
In one embodiment, determining information query terms based on a plurality of target topic segments includes:
The method comprises the steps of obtaining a prompt word template, wherein the prompt word template comprises a requirement template, reference information and requirement information;
and taking the target topic fragments as reference information of the prompt word template, and taking the key query information in the metadata as the requirement information of the prompt word template to obtain the information query prompt word.
In one embodiment, the step of constructing the dialogue record database includes:
Acquiring a plurality of continuous history dialogue records;
Topic separation is carried out on a plurality of continuous historical dialogue records to obtain a plurality of topic segments, wherein each topic segment comprises a plurality of dialogue records and corresponding topics;
a dialog records database is constructed based on the plurality of topic segments.
In one embodiment, topic separation is performed on a plurality of continuous historical conversational records to obtain a plurality of topic segments, including:
Respectively carrying out semantic analysis on a plurality of continuous history dialogue records to obtain topics corresponding to the plurality of continuous history dialogue records;
And taking a plurality of continuous dialogue records belonging to the same topic in the plurality of continuous historical dialogue records as a topic segment to obtain a plurality of topic segments.
In a second aspect, the present application also provides an information retrieval apparatus, including:
the extraction module is used for extracting a plurality of metadata from the information inquiry statement under the condition that the information inquiry statement input by a user is received;
the system comprises a search module, a search module and a search module, wherein the search module is used for searching in a dialogue record database by taking a plurality of metadata as filtering conditions to obtain a plurality of target topic fragments;
the determining module is used for determining information inquiry prompt words based on a plurality of target topic segments;
and the generation module is used for obtaining the information retrieval result generated by the large language model based on the information query prompt word and the large language model.
In a third aspect, the present application also provides a computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
Under the condition that an information inquiry statement input by a user is received, extracting a plurality of metadata from the information inquiry statement;
searching in a dialogue record database by taking a plurality of metadata as filtering conditions to obtain a plurality of target topic fragments, wherein each target topic fragment comprises a plurality of dialogue records and corresponding topics;
determining information inquiry prompt words based on a plurality of target topic segments;
And obtaining an information retrieval result generated by the large language model based on the information query prompt word and the large language model.
In a fourth aspect, the present application also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
Under the condition that an information inquiry statement input by a user is received, extracting a plurality of metadata from the information inquiry statement;
searching in a dialogue record database by taking a plurality of metadata as filtering conditions to obtain a plurality of target topic fragments, wherein each target topic fragment comprises a plurality of dialogue records and corresponding topics;
determining information inquiry prompt words based on a plurality of target topic segments;
And obtaining an information retrieval result generated by the large language model based on the information query prompt word and the large language model.
In a fifth aspect, the application also provides a computer program product comprising a computer program which, when executed by a processor, performs the steps of:
Under the condition that an information inquiry statement input by a user is received, extracting a plurality of metadata from the information inquiry statement;
searching in a dialogue record database by taking a plurality of metadata as filtering conditions to obtain a plurality of target topic fragments, wherein each target topic fragment comprises a plurality of dialogue records and corresponding topics;
determining information inquiry prompt words based on a plurality of target topic segments;
And obtaining an information retrieval result generated by the large language model based on the information query prompt word and the large language model.
The information retrieval method, the device, the computer equipment, the computer readable storage medium and the computer program product are beneficial to accurately understanding the query requirement of the user by extracting a plurality of metadata from the information query statement under the condition of receiving the information query statement input by the user, take the plurality of metadata as filtering conditions, help to quickly retrieve a plurality of target topic fragments related to the user requirement from a dialogue record database, improve the retrieval efficiency, and further determine an information query prompt word based on the plurality of target topic fragments, wherein the information query prompt word comprises dialogue records and corresponding topics, and takes the information query prompt word as an input prompt of a large language model.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related art, the drawings that are needed in the description of the embodiments of the present application or the related technologies will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other related drawings may be obtained according to these drawings without inventive effort to those of ordinary skill in the art.
FIG. 1 is an application environment diagram of an information retrieval method in one embodiment;
FIG. 2 is a flow diagram of a method of information retrieval in one embodiment;
FIG. 3 is a general flow diagram of a method of information retrieval in one embodiment;
FIG. 4 is a block diagram of an information retrieval device in one embodiment;
fig. 5 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
The information retrieval method provided by the embodiment of the application can be applied to an application environment shown in figure 1. Wherein the terminal 102 communicates with the server 104 via a network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104 or may be located on a cloud or other network server. The embodiment is illustrated by applying the method to the terminal 102, and it is understood that the method may also be applied to the server 104, and may also be applied to a system including the terminal 102 and the server 104, and implemented through interaction between the terminal 102 and the server 104. The terminal 102 extracts a plurality of metadata from the information query sentence under the condition of receiving the information query sentence input by the user, searches in the dialogue record database by taking the plurality of metadata as filtering conditions to obtain a plurality of target topic segments, wherein each target topic segment comprises a plurality of dialogue records and corresponding topics, determines an information query prompt word based on the plurality of target topic segments, and obtains an information search result generated by the large language model based on the information query prompt word and the large language model. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, internet of things devices and portable wearable devices, where the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart vehicle devices, projection devices, and the like. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The head-mounted device may be a Virtual Reality (VR) device, an augmented Reality (Augmented Reality, AR) device, smart glasses, or the like. The server 104 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing cloud computing services.
In an exemplary embodiment, as shown in fig. 2, an information retrieval method is provided, and the method is applied to the terminal 102 in fig. 1 for illustration, and includes the following steps 202 to 208. Wherein:
step 202, extracting a plurality of metadata from the information query sentence when the information query sentence input by the user is received.
Wherein, the information query statement refers to a statement input by a user for information query. The information query statement may take a variety of forms, e.g., a phonetic form, a text form, etc. The terminal is provided with a man-machine interaction interface, and a user inputs an information inquiry statement through the man-machine interaction interface. According to the scene to which the information retrieval method is applied, the information query statement can be mainly used for information query under various scenes. The application scene comprises, but is not limited to, instant messaging software, a social network platform, an online customer service system and the like.
For example, in an instant messaging scenario, the information query statement may be used to search for historical dialog content in the instant messaging software to obtain specific information or a reminder, and so on.
Metadata refers to data that can describe data attributes and features in an information query statement, and may be, for example, key attribute words, key feature words, and the like of the information query statement. For example, the Inquiry statement is that about the last week, I and Li Ming chat about what credentials I need to carry about, I and Li Ming, what credentials to do with, etc.
In step 204, the plurality of metadata are used as filtering conditions, and are searched in a dialogue record database to obtain a plurality of target topic segments, wherein each target topic segment comprises a plurality of dialogue records and corresponding topics.
The dialogue record database refers to a pre-constructed database, and is used for storing historical dialogue records and topics of the dialogue records.
The terminal uses the metadata as filtering conditions, and can retrieve dialogue records matched with the metadata and topics of the dialogue records from a dialogue record database.
Topics refer to topics of a conversation record, and one conversation record can correspond to one topic, and multiple conversation records can correspond to one topic, so that the conversation records in the target topic segment can comprise conversation record contexts corresponding to topics.
For example, each target topic segment includes a topic that includes a plurality of conversation records, and the target topic segment may be represented as topicA: [ userA: content1, userB: content2. Where topicA denotes topics, userA and userB denote dialogue objects, content1 denotes a dialogue record of userA, and content2 denotes a dialogue record of userB.
Step 206, determining information query terms based on the plurality of target topic segments.
The information inquiry prompt word can be used as an input prompt of a large language model, the plurality of target topic fragments comprise dialogue records related to metadata and corresponding topics, the plurality of target topic fragments can be used as references of information inquiry, and the information inquiry prompt word is constructed based on the target topic fragments. For example, multiple target topic segments and a prompt word template for an information query are assembled into an information query prompt word.
And step 208, obtaining an information retrieval result generated by the large language model based on the information query prompt word and the large language model.
The large language model (Large Language Model, LLM) is an artificial intelligent model capable of understanding and generating natural language, and has high language understanding and generating capability. The information query Prompt word is used as a Prompt (Prompt word) of the large language model, and can guide the large language model to generate information retrieval results related to the information query statement of the user.
In the information retrieval method, under the condition that an information query statement input by a user is received, a plurality of metadata are extracted from the information query statement, so that the user's query requirement can be accurately understood, the plurality of metadata are used as filtering conditions, a plurality of target topic segments related to the user requirement can be quickly retrieved from a dialogue record database, the retrieval efficiency is improved, in addition, an information query prompt word is determined based on the plurality of target topic segments, the dialogue record and the corresponding topic are contained in the information query prompt word, the information query prompt word is used as an input prompt of a large language model, the language understanding and generating capability of the large language model are utilized, the user's retrieval intention can be accurately understood, the information retrieval result which is more accurate and more in line with the user requirement can be generated, and the accuracy of the information retrieval result is improved.
In an exemplary embodiment, extracting a plurality of metadata from an information query sentence comprises obtaining a preset prompt word, wherein the preset prompt word comprises an extraction requirement template, a text to be extracted and a type of information to be extracted, taking the information query sentence as the text to be extracted in the preset prompt word to obtain a combined prompt word, and generating the plurality of metadata through a large language model based on the combined prompt word.
For metadata extraction, a large language model method may be used. In order to ensure that the extracted metadata better meets the query requirement of the user, a proper prompt word can be constructed based on the information query statement.
The preset prompting word refers to a prompting word template for information extraction. The preset prompting words can comprise a required template, a text to be extracted and an information type to be extracted. The extraction requirement template refers to a template indicating that the large language model finishes the extraction requirement. For example, the extraction requirement template can be that you are good at extracting key information in the text, please extract the sentence [ information type to be extracted ], and the text to be extracted is [ text to be extracted ].
The text to be extracted is a text to be filled in a preset prompt word for extraction, for example, an information inquiry sentence can be used as the text to be extracted.
The type of information to be extracted refers to the type of information to be extracted from the text, and the type can be specified in preset prompt words according to requirements. For example, the types of information that need to be extracted include start time, end time, dialog people, topics, key information points, and so on.
The terminal takes the information inquiry statement as a text to be extracted in the preset prompt words, and assembles the text to be extracted, the extraction requirement template and the information type to be extracted into combined prompt words. For example, the combined prompt may be that you are good at extracting key information in the text, please extract the beginning time, ending time, dialogue character, topic and key information points from the sentence, that you want to extract the text is about the last week, i am Li Ming chat to the things about putting cold and leaving Thailand travel, and he reminds i what credentials are needed?
The terminal takes the combined prompt words as the prompt of the large language model, and the generation result of the large language model is a plurality of metadata. For example, the plurality of metadata may be:
"Start_time" 2024-01-01"," end_time "2024-01-07", "dialog character people" Li Ming "," topic "show" discussion of cold-free vacation travel in Thailand "," key information point key_point "Li Ming" remind me of what credentials are needed.
In this embodiment, the information query sentence is used as the text to be extracted in the preset prompt words, the extraction requirement template and the information type to be extracted are designed in the preset prompt words in advance, the combined prompt words are finally assembled into combined prompt words, the combined prompt words are used for the prompt of the large language model,
The method for analyzing the metadata comprises the steps of obtaining a preset Prompt word, wherein the preset Prompt word comprises a required template, texts required to be extracted and information types required to be extracted, taking an information query statement as the texts required to be extracted in the preset Prompt word to obtain combined Prompt words, and generating a plurality of metadata through a large language model based on the combined Prompt words. This helps to enhance the user experience, enabling the user to more quickly find the desired information.
In one embodiment, the plurality of metadata comprises topic data, key query information, dialogue objects and start and stop time points, and the plurality of metadata is used as a filtering condition to search in the dialogue record database to obtain a plurality of target topic segments.
Taking the extracted metadata including topic data, key query information, dialogue objects and start and stop time points as examples, the topic data, the key query information, the dialogue objects and the start and stop time points are used as filtering conditions to search the dialogue record database. The retrieval method can adopt a retrieval method based on keywords and a retrieval method based on semantic vectors, such as a retrieval mode of RRF (Reciprocal Rank Fusion, inverse ranking fusion) in ELASTICSEARCH (an open-source distributed search and analysis engine), and the TopN target topic fragments most relevant to the query are retrieved from a database.
The preset relevance condition may include at least one of a semantic relevance condition and a keyword similarity condition. The dialogue record database comprises a plurality of topic segments. For example, semantic matching (SEMANTIC MATCHING) is adopted, semantic similarity between topics of each topic segment and topic data of metadata is compared, semantic similarity between dialogue records in each topic segment and key query information, dialogue objects and start and stop time points is compared, and the most relevant TopN target topic segments in a plurality of topic segments are determined through the similarity. For example, through the similarity of starting and ending time points, the P topic fragments with the most relevant time points are screened, the Q topic fragments with the most relevant topics are screened out of the P topic fragments, the M topic fragments with the most relevant dialogue objects are screened out of the Q topic fragments, and the N target topic fragments with the most relevant key query information are screened out of the M topic fragments. Or the target topic segments can also be screened by a weight dividing mode.
Illustratively, the TopN target topic segments retrieved may be:
[{topicA: [userA: content1, userB: content2...], topicB: [userA: content3...]...}]。
in this embodiment, by combining topic data, key query information, dialogue objects and start and stop time points as filtering conditions in an online query manner, a keyword-based search method and a semantic vector-based search method are used, so that TopN target topic segments most relevant to the query can be quickly searched out from a large number of dialogue records, which is helpful for improving the search efficiency and reducing the time and effort required by a user to search information.
In one embodiment, determining the information query prompt word based on a plurality of target topic segments comprises obtaining a prompt word template, wherein the prompt word template comprises a requirement template, reference information and requirement information, the target topic segments are used as the reference information of the prompt word template, and key query information in the metadata is used as the requirement information of the prompt word template to obtain the information query prompt word.
The target topic segments can be used for constructing information inquiry prompt words, so that the large language model is guided to generate information inquiry results meeting the inquiry requirements of users.
The hint word template is a template for generating a hint word of a query result. The hint word templates include generating a requirement template, reference information, and requirement information. Wherein, the generation requirement template refers to a template indicating that the large language model finishes generating the search result. For example, the requirement template may be generated by you being an assistant willing to answer a question, please answer the following question based on the reference information [ requirement information ], the reference information is as follows [ reference information ].
The terminal takes a plurality of target topic fragments as reference information of the prompt word template, takes key query information in a plurality of metadata as demand information of the prompt word template, and assembles the demand information and the generated demand template into the information query prompt word. For example, key query information may be Li Ming to remind me of which certificates should be carried, and the information query prompt may be you are an assistant willing to answer questions, please answer questions based on reference information Li Ming to remind me of which certificates should be carried:
[{topicA: [userA: content1, userB: content2...], topicB: [userA: content3...]...}]。
after the information inquiry prompt word is input into the large language model, the result generated by the large language model can be Li Ming to remind you to bring an identity card, passport.
In this embodiment, metadata of the target topic segments and the information query sentence are spliced through the Prompt through the target topic segments searched online to obtain the information query Prompt word, which is used for guiding the large language model to generate the final information query result, so that more accurate and relevant answers can be generated, the accuracy of the answers can be improved, and confusion and misunderstanding of users can be reduced.
In one embodiment, the conversation record database is constructed by acquiring a plurality of continuous historical conversation records, performing topic separation on the plurality of continuous historical conversation records to obtain a plurality of topic segments, wherein each topic segment comprises a plurality of conversation records and corresponding topics, and constructing the conversation record database based on the plurality of topic segments.
Wherein, the history dialogue record refers to dialogue information of the recorded user at the history moment. The conversation records of the user often spread around topics, so that a plurality of continuous historical conversation records can be separated according to corresponding topics to obtain a plurality of topic segments, each topic segment is provided with a plurality of conversation records, and the plurality of conversation records are continuous conversations spread around one topic.
Adding a plurality of topic segments to the database results in a dialogue record database. In some embodiments, the content stored in the session record database may increase as the session record increases.
In this embodiment, the historical dialogue records of the user are divided according to topics to obtain a plurality of topic segments, so that the plurality of dialogue records in each topic segment are expanded around the topics and have semantic relevance, and according to the topic division mode, the dialogue records meeting the user requirements can be quickly searched, and the accuracy of the search result is improved.
In one embodiment, topic separation is performed on a plurality of continuous history conversational records to obtain a plurality of topic segments, which includes respectively performing semantic analysis on the plurality of continuous history conversational records to obtain topics corresponding to the plurality of continuous history conversational records, and taking a plurality of continuous conversational records belonging to the same topic in the plurality of continuous history conversational records as a topic segment to obtain a plurality of topic segments.
Since the dialogue records of the same topic have semantic relevance, semantic analysis can be performed on a plurality of continuous history dialogue records, topics corresponding to the continuous history dialogue records can be identified, and a plurality of continuous dialogue records belonging to the same topic are used as a topic segment to obtain a plurality of topic segments. Illustratively, a Bert model based on supervised learning may be employed for semantic analysis of historical dialog records.
For example, the historical dialog records before topic separation may be:
[userA: cotent1, userB: content2, userA: content3...];
The historical dialog records before topic separation may be:
[{topicA: [userA: content1, userB: content2...], topicB: [userA: content3...]...}]。
In this embodiment, the historical dialog records are semantically analyzed, so that the historical dialog records are semantically analyzed and divided into a plurality of topic segments. Each topic segment contains a series of continuous chat messages which are related semantically, and the topic separation method is beneficial to storing the historical dialogue records according to topics and is beneficial to quickly retrieving content meeting the requirements of users.
To describe the information retrieval method and effect in this embodiment in detail, the following description will explain one of the most detailed embodiments:
The information retrieval method can be applied to various application scenes, such as scenes of instant messaging software, social network platforms, online customer service systems and the like. Taking instant messaging software as an example for illustration, fig. 3 is a flow chart of an information retrieval method in an embodiment.
The information inquiry statement input by the user and received by the terminal may be that about the last week, me and Li Ming chat to travel in thailand about cold-releasing false, he reminds me of which credentials are needed?
And step 1, topic separation. The terminal automatically preprocesses the history dialogue record of the user. Semantic analysis is carried out on the historical dialogue record by adopting a Bert model based on supervised learning, and a plurality of continuous historical dialogue records are divided into a plurality of topic segments. Each topic segment contains a series of consecutive dialog records that are semantically related. Examples are as follows:
prior to separation [ userA: cotent1, userB:content 2, userA:content 3
After separation [ (topicA: [ userA: content1, userB: content2. ], topicB: [ userA: content3. ] ]
And 2, analyzing metadata. When a user inputs an information Query statement Query, the terminal firstly analyzes the Query by using a large language model, and extracts metadata including topic data, key Query information, dialogue objects and start and stop time points. This step is accomplished by designing an appropriate promt to ensure that the large language model is able to accurately understand the query intent of the user. Examples are as follows:
The original Query statement, "about the last week, i and Li Ming chat about cold-free, fake going out of Thailand travel, he reminds that i need to carry what credentials?
The example of "you are good at extracting key information in text, please extract the beginning time, ending time, dialogue characters, topics and key information points from this sentence, i want to extract text that is about the last week, i and Li Ming chat to something about putting cold and going out of thailand travel, he reminds i need to carry what credentials.
The parsed information may be:
{ "Start_time": "2024-01-01", "end_time": "2024-01-07", "people": "Li Ming", "topic": "discussion of cold-free vacation to travel in Thailand", "key_Point": "Li Ming" remind me of which credentials should be carried.
And 3, searching. And the terminal searches in the chat record database according to the extracted metadata by combining topic data, key query information, dialogue objects and starting and ending time points as filtering conditions. Meanwhile, the TopN topic segments most relevant to the query are retrieved from the database using keyword-based retrieval and semantic vector-based retrieval methods, such as RRF retrieval in ELASTICSEARCH. Examples are as follows:
[{topicA: [userA: content1, userB: content2...], topicB: [userA: content3...]...}]。
And 4, generating a result by the large model. Combining TopN topic segments retrieved in step 3 with the key query information extracted in step 2 to form a new input, and guiding the large language model to generate a reply for the user query by designing a proper promt. Based on this information, the large language model generates an accurate, relevant answer, which is presented to the user. The simplet is exemplified as follows:
"you are an assistant willing to answer the question, please answer the following questions based on the reference information, li Ming remind me of which certificates should all be carried:
[{topicA: [userA: content1, userB: content2...], topicB: [userA: content3...]...}]"。
the final large model will generate an answer, for example, "Li Ming remind you to bring an identification card, passport.
According to the information retrieval method, under the condition that the information query statement input by the user is received, the plurality of metadata are extracted from the information query statement, the plurality of metadata are used as filtering conditions, the dialogue record database is searched to obtain the plurality of target topic segments, and because each target topic segment comprises a plurality of dialogue records and corresponding topics, the information query prompt word is determined based on the plurality of target topic segments, the dialogue records and the corresponding topics are contained in the information query prompt word, the information query prompt word is used as an input prompt of a large language model, the language understanding and generating capability of the large language model are utilized, the retrieval requirement of the user is accurately understood, the information retrieval result meeting the user requirement is generated, and the accuracy of the information retrieval result is improved. In addition, the dialogue record is divided into a plurality of semantically related topic segments through offline topic separation, so that chat content can be understood more accurately, and a more accurate basis is provided for subsequent retrieval. This helps to improve the accuracy of the search and reduce the interference of irrelevant content. The online Query metadata analysis can more accurately understand the Query requirement of the user, and through a large language model and a Prompt design, the topic data, the key Query information, the dialogue objects, the start-stop time points and other key information are extracted, so that the Query intention of the user can be more accurately understood, the user experience can be improved, and the user can more quickly find the required information. By online query, combining topic data, key query information, dialogue objects and start and stop time points as filtering conditions, and using a keyword-based retrieval method and a semantic vector-based retrieval method, topN topic segments most relevant to the query can be rapidly retrieved from a large number of dialogue records, which is helpful for improving the retrieval efficiency and reducing the time and effort required by users to search information. And after the query result and the query content are spliced through the Prompt, a large language model is called to generate a final reply result, and more accurate and relevant answers can be generated, so that the accuracy of the answers is improved, and confusion and misunderstanding of users are reduced.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides an information retrieval device for realizing the above related information retrieval method. The implementation of the solution provided by the device is similar to the implementation described in the above method, so the specific limitation of one or more embodiments of the information retrieval device provided below may refer to the limitation of the information retrieval method described above, and will not be repeated here.
In one exemplary embodiment, as shown in FIG. 4, an information retrieval apparatus 400 is provided, comprising an extraction module 420, a retrieval module 440, a determination module 460, and a generation module 480, wherein:
An extraction module 420, configured to extract a plurality of metadata from the information query sentence when the information query sentence input by the user is received;
The retrieval module 440 is configured to retrieve a plurality of metadata as filtering conditions in a dialogue record database to obtain a plurality of target topic segments, where each target topic segment includes a plurality of dialogue records and corresponding topics;
A determining module 460, configured to determine an information query term based on a plurality of target topic segments;
The generating module 480 is configured to obtain an information retrieval result generated by the large language model based on the information query prompt word and the large language model.
According to the information retrieval device, under the condition that the information query statement input by the user is received, the plurality of metadata are extracted from the information query statement, the plurality of metadata are used as filtering conditions, the dialogue record database is searched to obtain the plurality of target topic segments, and because each target topic segment comprises a plurality of dialogue records and corresponding topics, the information query prompt word is determined based on the plurality of target topic segments, the dialogue records and the corresponding topics are contained in the information query prompt word, the information query prompt word is used as an input prompt of a large language model, and the language understanding and generating capability of the large language model are utilized, so that the retrieval requirement of the user can be accurately understood, the information retrieval result meeting the user requirement can be generated, and the accuracy of the information retrieval result is improved.
In one embodiment, the extracting module 420 is further configured to extract a plurality of metadata from the information query sentence, obtain a preset prompting word, where the preset prompting word includes a requirement template, a text to be extracted, and a type of information to be extracted, use the information query sentence as the text to be extracted in the preset prompting word, obtain a combined prompting word, and generate a plurality of metadata through a large language model based on the combined prompting word.
In one embodiment, the plurality of metadata includes topic data, key query information, a dialogue object and a start-stop time point, and the search module 440 is further configured to search the plurality of topic segments in the dialogue record database to obtain a plurality of target topic segments having relevance to the plurality of metadata satisfying a preset relevance condition by using the plurality of metadata as a filtering condition and searching the dialogue record database to obtain a plurality of target topic segments.
In one embodiment, the determining module 460 is further configured to obtain a prompt word template, where the prompt word template includes generating a requirement template, reference information, and requirement information, and obtain the information query prompt word by using the multiple target topic segments as the reference information of the prompt word template and using the key query information in the multiple metadata as the requirement information of the prompt word template.
In one embodiment, in terms of construction of the dialogue record database, the retrieval module 440 is further configured to obtain a plurality of continuous history dialogue records, perform topic separation on the plurality of continuous history dialogue records to obtain a plurality of topic segments, each topic segment includes a plurality of dialogue records and corresponding topics, and construct the dialogue record database based on the plurality of topic segments.
In one embodiment, the topic separation is performed on a plurality of continuous history conversational records to obtain a plurality of topic segments, and the retrieval module 440 is further configured to perform semantic analysis on the plurality of continuous history conversational records to obtain topics corresponding to the plurality of continuous history conversational records, and use a plurality of continuous conversational records belonging to the same topic in the plurality of continuous history conversational records as a topic segment to obtain a plurality of topic segments.
The respective modules in the above-described information retrieval apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one exemplary embodiment, a computer device is provided, which may be a terminal, and an internal structure diagram thereof may be as shown in fig. 5. The computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input means. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and the external device. The Communication interface of the computer device is used for conducting wired or wireless Communication with an external terminal, and the wireless Communication can be realized through WIFI, a mobile cellular network, near field Communication (NEAR FIELD Communication) or other technologies. The computer program is executed by a processor to implement an information retrieval method. The display unit of the computer device is used for forming a visual picture, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be a key, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.
It will be appreciated by those skilled in the art that the structure shown in FIG. 5 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In one exemplary embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of:
Under the condition that an information inquiry statement input by a user is received, extracting a plurality of metadata from the information inquiry statement, searching in a dialogue record database by taking the plurality of metadata as a filtering condition to obtain a plurality of target topic fragments, wherein each target topic fragment comprises a plurality of dialogue records and corresponding topics, determining an information inquiry prompt word based on the plurality of target topic fragments, and obtaining an information search result generated by a large language model based on the information inquiry prompt word and the large language model.
In one embodiment, the processor when executing the computer program further performs the steps of:
The method comprises the steps of obtaining preset prompt words, wherein the preset prompt words comprise a required template, texts to be extracted and information types to be extracted, taking information inquiry sentences as the texts to be extracted in the preset prompt words to obtain combined prompt words, and generating a plurality of metadata through a large language model based on the combined prompt words.
In one embodiment, the processor when executing the computer program further performs the steps of:
The plurality of metadata comprise topic data, key query information, dialogue objects and starting and ending time points, and the plurality of topic fragments in the dialogue record database are searched to obtain a plurality of target topic fragments, wherein the relevance of the target topic fragments and the plurality of metadata meets the preset relevance condition.
In one embodiment, the processor when executing the computer program further performs the steps of:
The method comprises the steps of obtaining a prompt word template, wherein the prompt word template comprises a requirement template, reference information and requirement information, taking a plurality of target topic fragments as the reference information of the prompt word template, and taking key query information in a plurality of metadata as the requirement information of the prompt word template to obtain an information query prompt word.
In one embodiment, the processor when executing the computer program further performs the steps of:
the method comprises the steps of obtaining a plurality of continuous historical dialogue records, carrying out topic separation on the plurality of continuous historical dialogue records to obtain a plurality of topic fragments, wherein each topic fragment comprises a plurality of dialogue records and corresponding topics, and constructing a dialogue record database based on the plurality of topic fragments.
In one embodiment, the processor when executing the computer program further performs the steps of:
The method comprises the steps of respectively carrying out semantic analysis on a plurality of continuous history dialogue records to obtain topics corresponding to the plurality of continuous history dialogue records, taking a plurality of continuous dialogue records belonging to the same topic in the plurality of continuous history dialogue records as a topic segment, and obtaining a plurality of topic segments.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:
Under the condition that an information inquiry statement input by a user is received, extracting a plurality of metadata from the information inquiry statement, searching in a dialogue record database by taking the plurality of metadata as a filtering condition to obtain a plurality of target topic fragments, wherein each target topic fragment comprises a plurality of dialogue records and corresponding topics, determining an information inquiry prompt word based on the plurality of target topic fragments, and obtaining an information search result generated by a large language model based on the information inquiry prompt word and the large language model.
In one embodiment, the computer program when executed by the processor further performs the steps of:
The method comprises the steps of obtaining preset prompt words, wherein the preset prompt words comprise a required template, texts to be extracted and information types to be extracted, taking information inquiry sentences as the texts to be extracted in the preset prompt words to obtain combined prompt words, and generating a plurality of metadata through a large language model based on the combined prompt words.
In one embodiment, the computer program when executed by the processor further performs the steps of:
The plurality of metadata comprise topic data, key query information, dialogue objects and starting and ending time points, and the plurality of topic fragments in the dialogue record database are searched to obtain a plurality of target topic fragments, wherein the relevance of the target topic fragments and the plurality of metadata meets the preset relevance condition.
In one embodiment, the computer program when executed by the processor further performs the steps of:
The method comprises the steps of obtaining a prompt word template, wherein the prompt word template comprises a requirement template, reference information and requirement information, taking a plurality of target topic fragments as the reference information of the prompt word template, and taking key query information in a plurality of metadata as the requirement information of the prompt word template to obtain an information query prompt word.
In one embodiment, the computer program when executed by the processor further performs the steps of:
the method comprises the steps of obtaining a plurality of continuous historical dialogue records, carrying out topic separation on the plurality of continuous historical dialogue records to obtain a plurality of topic fragments, wherein each topic fragment comprises a plurality of dialogue records and corresponding topics, and constructing a dialogue record database based on the plurality of topic fragments.
In one embodiment, the computer program when executed by the processor further performs the steps of:
The method comprises the steps of respectively carrying out semantic analysis on a plurality of continuous history dialogue records to obtain topics corresponding to the plurality of continuous history dialogue records, taking a plurality of continuous dialogue records belonging to the same topic in the plurality of continuous history dialogue records as a topic segment, and obtaining a plurality of topic segments.
In one embodiment, a computer program product is provided comprising a computer program which, when executed by a processor, performs the steps of:
Under the condition that an information inquiry statement input by a user is received, extracting a plurality of metadata from the information inquiry statement, searching in a dialogue record database by taking the plurality of metadata as a filtering condition to obtain a plurality of target topic fragments, wherein each target topic fragment comprises a plurality of dialogue records and corresponding topics, determining an information inquiry prompt word based on the plurality of target topic fragments, and obtaining an information search result generated by a large language model based on the information inquiry prompt word and the large language model.
In one embodiment, the computer program when executed by the processor further performs the steps of:
The method comprises the steps of obtaining preset prompt words, wherein the preset prompt words comprise a required template, texts to be extracted and information types to be extracted, taking information inquiry sentences as the texts to be extracted in the preset prompt words to obtain combined prompt words, and generating a plurality of metadata through a large language model based on the combined prompt words.
In one embodiment, the computer program when executed by the processor further performs the steps of:
The plurality of metadata comprise topic data, key query information, dialogue objects and starting and ending time points, and the plurality of topic fragments in the dialogue record database are searched to obtain a plurality of target topic fragments, wherein the relevance of the target topic fragments and the plurality of metadata meets the preset relevance condition.
In one embodiment, the computer program when executed by the processor further performs the steps of:
The method comprises the steps of obtaining a prompt word template, wherein the prompt word template comprises a requirement template, reference information and requirement information, taking a plurality of target topic fragments as the reference information of the prompt word template, and taking key query information in a plurality of metadata as the requirement information of the prompt word template to obtain an information query prompt word.
In one embodiment, the computer program when executed by the processor further performs the steps of:
the method comprises the steps of obtaining a plurality of continuous historical dialogue records, carrying out topic separation on the plurality of continuous historical dialogue records to obtain a plurality of topic fragments, wherein each topic fragment comprises a plurality of dialogue records and corresponding topics, and constructing a dialogue record database based on the plurality of topic fragments.
In one embodiment, the computer program when executed by the processor further performs the steps of:
The method comprises the steps of respectively carrying out semantic analysis on a plurality of continuous history dialogue records to obtain topics corresponding to the plurality of continuous history dialogue records, taking a plurality of continuous dialogue records belonging to the same topic in the plurality of continuous history dialogue records as a topic segment, and obtaining a plurality of topic segments.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are both information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data are required to meet the related regulations.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile memory and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (RESISTIVE RANDOM ACCESS MEMORY, reRAM), magneto-resistive Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (PHASE CHANGE Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in various forms such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computation, an artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) processor, or the like, but is not limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the present application.
The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411757566.9A CN119557462A (en) | 2024-12-03 | 2024-12-03 | Information retrieval method, device, equipment, readable storage medium and program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411757566.9A CN119557462A (en) | 2024-12-03 | 2024-12-03 | Information retrieval method, device, equipment, readable storage medium and program product |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN119557462A true CN119557462A (en) | 2025-03-04 |
Family
ID=94747260
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411757566.9A Pending CN119557462A (en) | 2024-12-03 | 2024-12-03 | Information retrieval method, device, equipment, readable storage medium and program product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119557462A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN120448511A (en) * | 2025-07-10 | 2025-08-08 | 北京澜舟科技有限公司 | Agent dialogue generation method and storage medium |
-
2024
- 2024-12-03 CN CN202411757566.9A patent/CN119557462A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN120448511A (en) * | 2025-07-10 | 2025-08-08 | 北京澜舟科技有限公司 | Agent dialogue generation method and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US12381922B2 (en) | Data modeling for virtual collaboration environment | |
| JP6361351B2 (en) | Method, program and computing system for ranking spoken words | |
| JP2019501466A (en) | Method and system for search engine selection and optimization | |
| CN113806588B (en) | Method and device for searching videos | |
| CN118657222B (en) | Data question-answering method, data question-answering model training method, device and equipment | |
| CN119557462A (en) | Information retrieval method, device, equipment, readable storage medium and program product | |
| CN119066172A (en) | Question and answer processing method, device, computer equipment, readable storage medium and program product | |
| CN114490996B (en) | Intention recognition method and device, computer equipment and storage medium | |
| CN118153564B (en) | Script processing method, device, computer equipment, storage medium and program product | |
| CN114461749A (en) | Data processing method, device, electronic device and medium for dialogue content | |
| CN118708688A (en) | Dialogue processing method, dialogue method, device and electronic device based on large model | |
| US20230259541A1 (en) | Intelligent Assistant System for Conversational Job Search | |
| CN115203513B (en) | Search statement recommendation method, device, equipment and medium | |
| US11789944B2 (en) | User-specific computer interaction recall | |
| CN113535125A (en) | Method and device for generating financial demand items | |
| KR102120748B1 (en) | Method and computer readable recording medium for providing bookmark search service stored with hierachical dialogue flow management model based on context | |
| CN119046446B (en) | Method, device, computer equipment and readable storage medium for processing graphic question and answer | |
| CN115269785B (en) | Search method, device, computer equipment and storage medium | |
| CN119474364A (en) | Topic detection method, device, computer equipment, readable storage medium and program product | |
| US20150340026A1 (en) | Extracting candidate answers for a knowledge base from conversational sources | |
| CN119415662A (en) | Data processing method, device and computer equipment | |
| CN119520682A (en) | Service recommendation method, system, device, server and storage medium | |
| CN118916449A (en) | Dialog generation method, device, equipment and medium | |
| CN119202142A (en) | User question response method, system, computer device and readable storage medium | |
| CN119669414A (en) | Human-computer dialogue method, device, computer equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |