CN111797214A - Question screening method, device, computer equipment and medium based on FAQ database - Google Patents

Abstract

The embodiments of the present application belong to the field of artificial intelligence, and relate to a question screening method, device, computer equipment and medium based on a FAQ database, including analyzing a question sentence input by a user, performing word segmentation processing on the question sentence, and determining the weight of each word after the word segmentation. , query the candidate questions corresponding to the question statement, score and screen the candidate questions queried in the FAQ database according to the weight of each word, and then calculate the similarity value between the question statement and the query result according to the similarity algorithm model. The query results whose similarity value is not within the preset range are filtered, and finally, a classification algorithm is used to calculate the filtered query results, and a preset number of questions with the highest similarity to the question statement are determined. In addition, this application also involves blockchain technology, and the question statement input by the user can be stored in the blockchain. This application can improve the accuracy of question screening and recommend high-quality questions for users.

Description

Question screening method, device, computer equipment and medium based on FAQ database

technical field

The present application relates to the field of artificial intelligence technology, and in particular, to a method, device, computer equipment and medium for question screening based on FAQ database.

Background technique

In the Frequently Asked Question (FAQ) dialogue, the FAQ system is pre-built with a question-and-answer library that includes a large number of question-and-answer pairs. When a question raised by a user is received, the FAQ system can search in the above-mentioned question-and-answer library based on the question raised by the user. To the question that matches the question asked by the user, and return the question and the answer to the question determined by the FAQ system to the user.

At present, the question answering systems in the industry mostly use direct matching or direct word segmentation to find FAQ standard questions. This method has defects in the accuracy of finding standard questions, similar questions, and related questions, and it is difficult to find the semantically similar questions. In addition, through the limited configuration of the FAQ, it is difficult to match the corresponding problems of unlimited user descriptions of the problems, and it is difficult for customers High-quality questions are recommended. Therefore, the traditional FAQ needs to maintain a huge knowledge base, and there are still problems with low precision in question screening.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present application is to propose a question screening method, device, computer equipment and medium based on a FAQ database, the main purpose of which is to quickly and accurately screen out questions matching the questions for users.

In order to solve the above-mentioned technical problems, the embodiment of the present application provides a problem screening method based on FAQ database, which adopts the following technical solutions:

Analyzing the problem sentence input by the user, and performing word segmentation processing on the problem sentence;

After the word segmentation, the word frequency of each word in the FAQ database is counted, the weight of each word is determined, and the weight of each word and the word segmentation result are stored in the FAQ database;

Query the candidate questions corresponding to the question statement in the FAQ database, and score the candidate questions according to the weights of the words, and screen out the candidate questions with a score greater than or equal to a preset score as the query result;

Calculate the similarity value between the question statement and the query result according to the similarity algorithm model, and filter the query results whose similarity value is not within the preset range;

A classification algorithm is used to calculate the filtered query results, and to determine a preset number of questions with the highest similarity to the input question sentence.

In order to solve the above-mentioned technical problems, the embodiment of the present application also provides a problem screening device based on a FAQ database, which adopts the following technical solutions:

The word segmentation module is used to analyze the problem sentence input by the user, and perform word segmentation processing on the problem sentence;

The processing module is used to count the word frequency of each word in the FAQ database after word segmentation, determine the weight of each word, and store the weight of each word and the word segmentation result in the FAQ database;

A query scoring module, used for querying the candidate questions corresponding to the question statement in the FAQ database, and scoring the candidate questions according to the weight of each word;

The screening module is used to filter out the questions whose score is greater than or equal to the preset score as the query result;

a similarity calculation module, configured to calculate the similarity value between the question statement and the query result according to the similarity algorithm model, and filter the query results whose similarity value is not within a preset range;

A classification calculation module for calculating the filtered query results using a classification algorithm; and

A determination module, configured to determine a preset number of questions with the highest similarity to the input question sentence.

In order to solve the above-mentioned technical problems, the embodiment of the present application also provides a computer device, which adopts the following technical solutions:

The computer device includes a memory and a processor, the memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the processor implements the steps of the FAQ database-based question screening method as described above.

In order to solve the above technical problems, the embodiments of the present application also provide a computer-readable storage medium, which adopts the following technical solutions:

The computer-readable storage medium has computer-readable instructions stored thereon, and when the computer-readable instructions are executed by the processor, implements the steps of the above-mentioned FAQ database-based question screening method.

Compared with the prior art, the embodiments of the present application mainly have the following beneficial effects:

The present invention performs word segmentation processing on the question statement by analyzing the question statement input by the user; after the word segmentation, the word frequency of each word in the FAQ database is counted, the weight of each word is determined, and the weight of each word and the word segmentation result are stored in the FAQ database; query The candidate questions corresponding to the question sentences in the FAQ database, and the candidate questions are scored according to the weight of each word, and the candidate questions with a score greater than or equal to the preset score are screened out as the query result, which can be preliminarily screened. Obtain relatively accurate query results; calculate the similarity value between the question statement and the query result according to the similarity algorithm model, and filter the query results whose similarity value is not within the preset range, which can realize further filtering of the results of the preliminary screening. Similarity analysis to find out the questions that are actually similar to the questions input by the user; the classification algorithm is used to calculate the filtered query results to determine the preset number of questions with the highest similarity to the input question statement, which can further improve the screening process. The accuracy of the questions asked, recommends high-quality questions for users.

Description of drawings

In order to illustrate the solutions in the present application more clearly, the following will briefly introduce the accompanying drawings used in the description of the embodiments of the present application. For those of ordinary skill, other drawings can also be obtained from these drawings without any creative effort.

FIG. 1 is an exemplary system architecture diagram to which the present application can be applied;

Fig. 2 is a flow chart of an embodiment of the method for screening questions based on FAQ database according to the present application;

Fig. 3 is a flow chart of a specific implementation manner of step S203 in Fig. 2;

Fig. 4 is a flow chart of a specific implementation manner of step S204 in Fig. 2;

Fig. 5 is a flow chart of a specific implementation manner of step S205 in Fig. 2;

6 is a schematic structural diagram of an embodiment of a question screening device based on a FAQ database according to the present application;

FIG. 7 is a schematic structural diagram of an embodiment of a computer device according to the present application.

Detailed ways

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field of this application; the terms used herein in the specification of the application are for the purpose of describing specific embodiments only It is not intended to limit the application; the terms "comprising" and "having" and any variations thereof in the description and claims of this application and the above description of the drawings are intended to cover non-exclusive inclusion. The terms "first", "second" and the like in the description and claims of the present application or the above drawings are used to distinguish different objects, rather than to describe a specific order.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor a separate or alternative embodiment that is mutually exclusive of other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

In order to make those skilled in the art better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings.

In order to solve the problem of low accuracy in question screening of FAQ, the present application provides a problem screening method based on FAQ database, which involves artificial intelligence semantic analysis, and can be applied to the system architecture 100 shown in FIG. 1 . The system architecture 100 can Including terminal equipment 101 , 102 , 103 , network 104 and server 105 . The network 104 is a medium used to provide a communication link between the terminal devices 101 , 102 , 103 and the server 105 . The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages and the like. Various communication client applications may be installed on the terminal devices 101 , 102 and 103 , such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, and the like.

The terminal devices 101, 102, and 103 may be various electronic devices that have a display screen and support web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III, moving image Expert Compression Standard Audio Layer 3), MP4 (Moving PictureExperts Group Audio Layer IV, Moving Picture Experts Group Audio Layer 4) Players, Laptops and Desktops, etc.

The server 105 may be a server that provides various services, such as a background server that provides support for the pages displayed on the terminal devices 101 , 102 , and 103 .

It should be noted that the FAQ database-based question screening method provided by the embodiments of the present application is generally performed by a server/terminal device, and accordingly, the FAQ database-based question screening apparatus is generally set in the server/terminal device.

It should be understood that the numbers of terminal devices, networks and servers in FIG. 1 are merely illustrative. There can be any number of terminal devices, networks and servers according to implementation needs.

Please refer to FIG. 2. FIG. 2 shows a flowchart of an embodiment of a method for question screening based on a FAQ database provided by an embodiment of the present invention. The method is applied to the server of FIG. 1 as an example for description. The described question screening method based on FAQ database includes the following steps:

Step S201, analyze the question sentence input by the user, and perform word segmentation processing on the question sentence.

It should be noted that, the question sentence input by the user may be input through audio, or may be input through text, which is not limited herein. Further, under the situation that the information input by the user is obtained as an audio file, voice recognition is performed on the audio file input by the user, the obtained voice recognition result is converted into text data, and the text data is subjected to word segmentation to obtain the corresponding word segmentation. result.

In this embodiment, the problem sentence input by the user is received, and the problem sentence is analyzed. Specifically, the problem sentence can be semantically analyzed, and the semantic analysis includes word segmentation, part-of-speech analysis, named entity recognition, stop word removal, etc. , based on semantics, the word segmentation controller is used to process the word segmentation of the problem sentence.

It should be emphasized that, in order to further ensure the privacy and security of the above-mentioned user-input question statement, the above-mentioned user-input question statement may also be stored in a node of a blockchain.

The blockchain referred to in this application is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information to verify its Validity of information (anti-counterfeiting) and generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

In step S202, the word frequency of each word in the FAQ database after word segmentation is counted, the weight of each word is determined, and the weight of each word and the word segmentation result are stored in the FAQ database.

It should be noted that the FAQ database is pre-established, and a search engine is built in the FAQ database. In the FAQ database, the query, analysis and exploration functions of the search engine are used to retrieve the question statements input by the user, and the query and the The query result corresponding to the question statement entered by the user. Search engines in this application include but are not limited to Elastic Search.

In this embodiment, the question and answer history data of each user may be collected periodically, and the FAQ database may be updated by using the collected question and answer history data of each user, wherein the question and answer history data of the user may include: questions answered by the user, Asked questions, browsed questions, queried questions, etc., can also use web crawlers from Google, Baidu, Yahoo and other search engines to crawl question-and-answer pairs that match the questions raised by users to update the FAQ database.

Wherein, to determine the weight of each word, the distribution of each word in the FAQ database can be examined based on the characteristics of statistical distribution, and the weight of the word is set according to the distribution characteristics such as word frequency. Specifically, each word after word segmentation is obtained, and the word frequency of each word in the FAQ database is counted, and the weight value of each word is: word weight value=word frequency*100.

Here, the weights of the words and the word segmentation results are stored in the database for the user to perform training and learning in the process of inputting questions. The FAQ database is constantly updated, and the weights of words are also constantly updated. In this embodiment, the word segmentation result can be updated to the FAQ database by training the model.

Step S203 , query the candidate questions corresponding to the question sentences in the FAQ database, and score the candidate questions according to the weight of each word, and select the candidate questions with a score greater than or equal to a preset score as the query result.

In this embodiment, by using the query function of the search engine built on the FAQ database to query the FAQ database, the candidate questions corresponding to the question sentences input by the user are queried, and the candidate questions are searched through functions such as analysis of the search engine. Scoring is performed, and candidate questions with a score greater than or equal to a preset score are screened out as query results.

In this embodiment, the filtered query results may further be sorted according to the scores.

In this embodiment, by screening out candidate questions with a score greater than or equal to a preset score, the queried candidate questions are preliminarily screened, so that a relatively accurate query result can be obtained.

Step S204: Calculate the similarity value between the question statement and the query result according to the similarity algorithm model, and filter the query results whose similarity value is not within the preset range.

In this embodiment, in order to calculate the similarity between the query result and the user input question, a variety of similarity algorithm models are used for similarity calculation, including: Jaccard similarity algorithm, BM25 algorithm, cosine similarity (cosine) algorithm and edit distance ( Edit_distance), specifically, the Jaccard similarity algorithm, used to calculate the similarity between samples, the larger the calculated Jaccard coefficient value, the higher the sample similarity; the BM25 algorithm is an algorithm based on a probability retrieval model, using to evaluate the correlation between search terms and documents; cosine similarity, by measuring the cosine value of the angle between two vectors to measure the similarity between them, in information retrieval, each term is given a different dimension, One dimension is represented by a vector, and the values on each dimension correspond to the frequency of the term appearing in the document, and the cosine similarity can give the similarity of two documents in terms of their subject; edit distance, referring to two words Between strings, the minimum number of editing operations required to convert from one to the other, describes how similar two strings are, and the greater the distance between them, the more different they are.

It should be noted that these four similarity models are used in combination according to different weights in different scenarios. For example, calculate the similarity Sim(A, B) between the first question A and the second question B in scene A , assuming that the weight of the Jaccard similarity algorithm in this scenario is 0.3, the weight of the BM25 algorithm is 0.2, the weight of the cosine similarity is 0.2, and the weight of the edit distance similarity is 0.3, then Sim(A, B)=Jaccard*0.3+BM25*0.2 +cosine*0.2+Edit_distance*0.3.

Further, in some optional implementations of this embodiment, after screening out candidate questions with a score greater than or equal to a preset score and sorting them, a preset number of query results that are sorted earlier may be selected to use a similarity algorithm to conduct the query. Similarity calculation, which can improve the screening efficiency and ensure the efficient return of user questions.

In this embodiment, the similarity calculation is performed on the query results obtained from the primary screening, and the query results whose similarity value is not within the preset range are filtered out, that is, the query results are filtered twice, and the semantics of the question sentence input by the user are further found out. Similar question.

In step S205, a classification algorithm is used to calculate the filtered query results, and a preset number of questions with the highest similarity to the question sentence is determined.

Specifically, in the filtered query results, the classification algorithm is implemented by using the combination of FastText (Fast Text Classification Algorithm) and textCNN (Convolutional Neural Network Text Classification Algorithm), and the problem with the user input is obtained in the FAQ database. Questions about the preset number of sentences with the highest similarity. It should be noted that in text classification tasks, FastText can often achieve accuracy comparable to that of deep networks, but is many orders of magnitude faster than deep networks in training time, and using textCNN to apply convolutional neural networks to text classification tasks , using multi-dimensional kernels to extract key information in sentences.

In this embodiment, a classification algorithm is performed on the filtered query results, and the preset number of questions with the highest similarity to the question statement is determined. This is the third screening, which can maximize the effectiveness of the user's question. accurate return.

In this embodiment, by analyzing the question sentence input by the user, the word segmentation process is performed on the question sentence, and the weight of each word after the word segmentation is determined, and the candidate questions corresponding to the question sentence are inquired. The candidate questions are scored and screened; then the similarity value between the question statement and the filtered query results is calculated according to the similarity algorithm model, and the query results whose similarity value is not within the preset range are filtered; The query results are calculated, so as to determine the preset number of questions with the highest similarity to the input question statement, and realize multiple screening of the queried questions, which can improve the accuracy of screening questions and recommend high-quality questions for users. question.

In some optional implementations of this embodiment, as shown in FIG. 3 , in step S203 , query the FAQ database for candidate questions corresponding to the question statement, and score the candidate questions according to the weight of each word, which specifically includes the following steps :

Step S301, extracting the keywords in the question sentence according to the weight of each word.

Keywords are an important part of the question and can help understand the user's questioning intent. In this embodiment, the question sentence input by the user has been subjected to word segmentation processing in step S202, and based on the word segmentation result and the weight of each word, a word with a weight greater than a preset weight value is determined as the keyword of the question sentence.

Step S302, determining the expanded word of the keyword.

In this embodiment, the expansion word can be obtained through the associated vocabulary, the word with high correlation with the keyword in the question sentence is obtained through the associated vocabulary, and the keyword is replaced with the word; it can also be obtained through the knowledge map of the FAQ database, The knowledge graph can be pre-built, and the pre-built knowledge graph can be used to automatically generate expanded words according to the hyponyms or related synonymous attributes of keywords. It should be noted that the knowledge graph can continuously collect data for training according to the user's query, so that the scale of the expanded word becomes larger and more accurate. In this embodiment, by forming keywords into expansion words, the search range can be expanded, thereby improving the recall rate of user questions.

Step S303, generating candidate questions corresponding to the question sentences according to the keywords and expansion words.

In this embodiment, the query, analysis and exploration functions of the search engine are used to obtain candidate questions corresponding to the question sentences input by the user in the FAQ database through keywords and expansion words, and expansion words can increase the number of candidate questions.

For example, the question sentence input by the user is: where is the most lively in Shenzhen, the extracted keywords are <shenzhen> <where> <most lively>, and the candidate questions generated according to the keywords and expansion words are as follows:

A. Where is Shenzhen the most prosperous?

B. Shenzhen is the most prosperous place.

In step S304, the candidate questions are scored according to the weight of the words.

In this embodiment, scoring the candidate questions may be to perform word segmentation processing on the generated candidate questions, and perform superposition calculation according to the weight of each word after word segmentation, so as to score according to the total weight of the candidate questions. The weights of the words and/or extended words are superimposed, and the scores are scored according to the results. It should be noted that this application is not limited to using the weighted superposition calculation to score similar problems.

In this embodiment, the keyword of the question sentence input by the user is extracted, the expansion word of the keyword is determined, and the query is performed according to the keyword and the expansion word, so as to generate a candidate question corresponding to the question sentence, which can expand the search range, thereby improving the The recall rate of user questions.

In some optional implementations, as shown in FIG. 4 , step S204 specifically includes the following steps:

Step S401: Calculate the Jaccard similarity, BM25 similarity, cosine similarity, and edit distance similarity between the question sentence and the query result, respectively.

_In order to describe the basic principles of the embodiments _of the present invention more conveniently, T1 and T2 are used to represent the question statement and the query result input by the user respectively, then the Jaccard similarity algorithm is used to calculate the _similarity between the question statement _T1 and the query result T2 Degree value S ₁ :

It should be noted that, before using the Jaccard similarity algorithm to calculate the similarity between two texts, word segmentation needs to be performed first, and the word segmentation process is the same as step S201.

For example, the texts _of the question statement T1 and the query result T2 are _:

T ₁ : Where is the most lively in Shenzhen;

T ₂ : Where is Shenzhen the most prosperous;

The word segmentation results are:

T ₁ = [Shenzhen, where is the most lively];

T ₂ = [Shenzhen, where is the most prosperous];

but

The BM25 algorithm is used to calculate the similarity value S ₂ between the question statement T ₁ and the query result T ₂ , and the algorithm formula used is as follows:

Among them, _qi represents a morpheme after T ₁ parsing, for example, for Chinese, we can analyze the word segmentation of T ₁ as a morpheme analysis, and each word is regarded as a morpheme _qi ; dl is the document length of the query result T ₂ , avgdl is the average length of all documents; k ₁ , b are adjustment factors, k ₁ =2, b = 0.75; f _i is the frequency of occurrence of qi in T ₂ ; _n is the number of documents containing qi _.

The cosine similarity algorithm is used to calculate the similarity value S ₃ between the question sentence T ₁ and the query result T ₂ , and the algorithm formula used is as follows:

Among them, x _i represents the TF-IDF weight _of the ith participle in the question sentence T1, _yi represents the TF-IDF weight of the ith participle in the query result T2, and TF _- IDF (term frequency-inverse document frequency) is A commonly used weighting technique for information retrieval and data mining. TF means Term Frequency, and IDF means Inverse Document Frequency. After the sentence segmentation, use TF-IDF to calculate the weight of the words in the sentence for sentence selection. After the word is selected, the similarity measurement of the cosine angle of the space vector will not be affected by the index scale, and the cosine value falls in the interval [0,1 ], the larger the value, the smaller the difference.

The similarity value S ₄ between the question statement T ₁ and the query result T ₂ is calculated using the edit distance similarity, and the algorithm formula used is as follows:

Among them, editsim(w _1i , w _2j ) is the similarity between the i-th string w _1i of the question statement T ₁ and the j-th string w _2j of the query result T ₂ , where i, j, n, m All are positive integers, and 1≤i≤n, 1≤j≤m; the calculation formula of editsim(w _1i ,w _2j ) is as follows:

Among them, editdis(w _1i .length, w _2j .length) is the length between the i-th string w _1i of the problem statement T ₁ and the j-th string w _2j of the query result T ₂ , and w _1i .length is The length of the i-th string of the problem statement T ₁ ; w _2j .length is the length of the j-th string of the query result T ₂ .

Step S402, the calculated Jaccard similarity, BM25 similarity, cosine similarity and edit distance similarity are weighted and summed according to their respective weight values to obtain the similarity value between the question sentence and the query result.

In this embodiment, the similarity value Sim(T ₁ , T ₂ ) between the question statement and the query result adopts the following calculation formula:

Sim(T ₁ , T ₂ )=α×S ₁ +β×S ₂ +γ×S ₃ +ω×S ₄ ;

Among them, α is the weight value of Jaccard similarity, β is the weight value of BM25 similarity, γ is the weight value of cosine similarity, ω is the weight value of edit distance similarity; α+β+γ+ω=1, α ≥0, β≥0, γ≥0, ω≥0.

It should be noted that in different scenarios, the values of α, β, γ and ω are different, and can be set accordingly according to the actual application.

Step S403, filtering the query results whose similarity value is not within the preset range.

Specifically, the calculated similarity value is compared with the preset similarity threshold, and the query results whose similarity value is greater than or equal to the preset similarity threshold are retained; the query results whose similarity value is less than the preset similarity threshold are removed. .

In this embodiment, by calculating the similarity value between the question sentence input by the user and the query result, and filtering the query results whose similarity value is not within the preset range, it is possible to effectively find out the similarities and more from a large number of similar questions. high question.

In some optional implementations, as shown in FIG. 5 , step S205 specifically includes the following steps:

In step S501, the question sentence input by the user and the word vector of the query result are obtained respectively through FastText.

The FastText model is an existing open source word vector and text classification model in the field of natural language processing. It takes each word represented in the form of a vector and the N-Gram feature corresponding to each word as input, and outputs the label corresponding to the text. In its output, there is an output by-product—the embedding vector corresponding to each word, that is, the word vector in this embodiment. Among them, the embedding vector refers to the vector that has undergone dimensionality reduction; the N-Gram feature refers to the word feature used to evaluate the degree of difference between words. In this embodiment, each word represented in the form of a vector and the N-Gram feature corresponding to each word are used as the input of the FastText model, and the word vector corresponding to each word can be obtained.

Step S502, the word vector is input into the textCNN model, after the operation of the convolution layer and the pooling layer, the similarity matrix between the question sentence and the query result is constructed, and the similarity value between the question sentence and the query result is output through the full connection layer.

Specifically, the word vector of the question sentence input by the user and the word vector of the query result are respectively connected to obtain the corresponding sentence vector. The user-input question sentence vector and the query result vector respectively pass through the convolution layer to obtain the sequence information of the sentence, and pass through the pooling layer. The transformation layer compresses the dimension of the sentence vector, and constructs the similarity matrix between the user input question and the query result. After the full connection layer, the user input question sentence vector and the query result vector are converted into a vector, and the vector is input into the logistic regression model. , the similarity value between the user input question statement and the query result can be obtained.

It should be noted that the logistic regression model is used to compress the result of the input vector into [0, 1] for output.

For example, suppose that after the fully connected layer, the obtained vector is X(x ₁ ,x ₂ ,x ₃ ,...,x _n ), and this vector is used as the input of the logistic regression model. The principles are as follows:

further,

Among them, ω ₁ , ω ₂ , ..., ω _n are the weights of the input vectors x ₁ , x ₂ , ..., x _n respectively.

In this embodiment, by combining the FastText model and the textCNN model to calculate the similarity value between the question statement and the query result, the calculation efficiency can be improved, and the user's question can be returned accurately to the maximum extent.

In some optional implementation manners, after querying the candidate questions corresponding to the question statement in the FAQ database in step S203, the following steps are further included: judging the number of questions in the candidate questions, and performing corresponding operations according to the judgment result.

Specifically, if the number of questions is zero, the classification algorithm is directly used for calculation; if the number of questions is less than or equal to the preset threshold, the candidate questions are directly returned to the customer; if the number of questions is greater than the preset threshold, the The weights are used to score the candidate questions, and the candidate questions with a score greater than or equal to the preset score are screened out as query results and sorted, and then steps S204 and S205 are executed; When the question is asked, it is directly calculated by the classification algorithm, and the question of the same category as the question is returned to the user to avoid the recall rate being zero; when the number of questions is less than or equal to the preset threshold, the candidate questions are directly returned to the customer, which can improve the query efficiency; When the number of questions is greater than the preset threshold, by performing multiple calculations and screening on the queried questions, the accuracy of the question screening can be improved, and high-quality questions can be recommended for users.

Those of ordinary skill in the art can understand that the realization of all or part of the processes in the methods of the above embodiments can be accomplished by instructing relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the program is During execution, it may include the processes of the embodiments of the above-mentioned methods. The aforementioned storage medium may be a non-volatile storage medium such as a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM), or a random access memory (Random Access Memory, RAM).

It should be understood that although the various steps in the flowchart of the accompanying drawings are sequentially shown in the order indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order and may be performed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but may be executed at different times, and the execution sequence is also It does not have to be performed sequentially, but may be performed alternately or alternately with other steps or at least a portion of sub-steps or stages of other steps.

In one embodiment, as shown in FIG. 6 , a problem screening apparatus based on a FAQ database is provided, and the problem screening apparatus based on the FAQ database corresponds one-to-one with the problem screening method based on the FAQ database in the above embodiment. The question screening device based on the FAQ database includes:

The word segmentation module 601 is used to analyze the question sentence input by the user, and perform word segmentation processing on the question sentence;

The processing module 602 is used to count the word frequency of each word in the FAQ database after word segmentation, determine the weight of each word, and store the weight of each word and the word segmentation result in the FAQ database;

The query scoring module 603 is used to query the candidate questions corresponding to the question statement in the FAQ database, and score the candidate questions according to the weight of each word;

A screening module 604, configured to screen out questions with a score greater than or equal to a preset score as a query result;

The similarity calculation module 605 is configured to calculate the similarity value between the question statement and the query result according to the similarity algorithm model, and filter the query results whose similarity value is not within the preset range;

A classification calculation module 606, used to calculate the filtered query results by using a classification algorithm; and

The determining module 607 is configured to determine a preset number of questions with the highest similarity to the input question sentence.

It should be emphasized that, in order to further ensure the privacy and security of the above-mentioned user-input question statement, the above-mentioned user-input question statement may also be stored in a node of a blockchain.

In some optional implementations of this embodiment, the query scoring module 603 includes:

The extraction unit is used for extracting the keywords in the question sentence according to the weight of each word;

expansion unit, used to determine the expansion word of the keyword;

The query scoring unit is used to generate candidate questions corresponding to the question sentences according to the keywords and expansion words, and score the candidate questions according to the weight of the words.

In some optional implementations of this embodiment, the similarity calculation module 605 includes:

The calculation unit is used to calculate the Jaccard similarity, BM25 similarity, cosine similarity and edit distance similarity between the question statement and the query result respectively, and calculate the calculated Jaccard similarity, BM25 similarity, cosine similarity and edit distance. The distance similarity is weighted and summed according to their respective weight values to obtain the similarity value between the question statement and the query result;

The filtering unit is used to filter the query results whose similarity value is not within the preset range.

Specifically, the filtering unit is configured to compare the similarity value with the preset similarity threshold, retain the query results whose similarity value is greater than or equal to the preset similarity threshold according to the comparison result, and store the similarity value smaller than the preset similarity threshold according to the comparison result. The query results are removed.

In some optional implementations of this embodiment, the classification calculation module 606 includes:

The obtaining unit is used to obtain the word vector of the question statement and the query result respectively through FastText;

The processing unit is used to input the word vector into the textCNN model. After the operation of the convolution layer and the pooling layer, the similarity matrix between the question sentence and the query result is constructed, and the similarity between the question sentence and the query result is output through the fully connected layer. value.

The above-mentioned question screening device based on the FAQ database analyzes the question sentence input by the user, performs word segmentation processing on the question sentence, and determines the weight of each word after the word segmentation, inquires about the candidate questions corresponding to the question sentence, and determines the number of words in the question sentence according to the weight of each word. The candidate questions queried in the FAQ database are scored and screened; then the similarity value between the question statement and the filtered query results is calculated according to the similarity algorithm model, and the query results whose similarity value is not within the preset range are filtered; finally The classification algorithm is used to calculate the filtered query results, so as to determine the preset number of questions with the highest similarity to the input question statement, and to screen the queried questions multiple times, which can improve the accuracy of the screening questions. Users recommend high-quality questions.

To solve the above technical problems, the embodiments of the present application also provide computer equipment. For details, please refer to FIG. 7 , which is a block diagram of the basic structure of a computer device according to this embodiment.

The computer device 7 includes a memory 71 , a processor 72 , and a network interface 73 that communicate with each other through a system bus. It should be pointed out that only the computer device 7 with components 71-73 is shown in the figure, but it should be understood that it is not required to implement all of the shown components, and more or less components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculation and/or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, special-purpose Integrated circuit (ApplicationSpecific Integrated Circuit, ASIC), programmable gate array (Field-Programmable GateArray, FPGA), digital processor (Digital Signal Processor, DSP), embedded equipment, etc.

The computer equipment may be a desktop computer, a notebook computer, a palmtop computer, a cloud server and other computing equipment. The computer device can perform human-computer interaction with the user through a keyboard, a mouse, a remote control, a touch pad or a voice control device.

The memory 71 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static Random Access Memory (SRAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Programmable Read Only Memory (PROM), Magnetic Memory, Magnetic Disk, Optical Disk, etc. In some embodiments, the memory 71 may be an internal storage unit of the computer device 7 , such as a hard disk or a memory of the computer device 7 . In other embodiments, the memory 71 may also be an external storage device of the computer device 7 , such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, flash memory card (FlashCard) and so on. Of course, the memory 71 may also include both the internal storage unit of the computer device 7 and its external storage device. In this embodiment, the memory 71 is generally used to store the operating system and various application software installed on the computer device 7 , such as computer-readable instructions for a question screening method based on a FAQ database. In addition, the memory 71 can also be used to temporarily store various types of data that have been output or will be output.

In some embodiments, the processor 72 may be a central processing unit (Central Processing Unit, CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. This processor 72 is typically used to control the overall operation of the computer device 7 . In this embodiment, the processor 72 is configured to execute computer-readable instructions stored in the memory 71 or process data, for example, computer-readable instructions for executing the problem screening method based on the FAQ database.

The network interface 73 may include a wireless network interface or a wired network interface, and the network interface 73 is generally used to establish a communication connection between the computer device 7 and other electronic devices.

In this embodiment, when the processor executes the computer-readable instructions stored in the memory, the steps of the question screening method based on the FAQ database in the above-mentioned embodiment can be implemented, and the queried questions can be calculated and screened multiple times to improve the accuracy of screening questions. , which recommends high-quality questions for users.

The present application also provides another embodiment, that is, to provide a computer-readable storage medium, where the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to The at least one processor is made to execute the steps of the above-mentioned question screening method based on the FAQ database, so as to perform multiple calculation and screening on the queried questions, improve the accuracy of the screening questions, and recommend high-quality questions for users.

From the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform, and of course hardware can also be used, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or in a part that contributes to the prior art, and the computer software product is stored in a storage medium (such as ROM/RAM, magnetic disk, CD-ROM), including several instructions to make a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of this application.

Obviously, the above-described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The accompanying drawings show the preferred embodiments of the present application, but do not limit the scope of the patent of the present application. This application may be embodied in many different forms, rather these embodiments are provided so that a thorough and complete understanding of the disclosure of this application is provided. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or perform equivalent replacements for some of the technical features. . Any equivalent structure made by using the contents of the description and drawings of the present application, which is directly or indirectly used in other related technical fields, is also within the scope of protection of the patent of the present application.

Claims (10)

1. A problem screening method based on an FAQ database is characterized by comprising the following steps:

analyzing a question sentence input by a user, and performing word segmentation processing on the question sentence;

counting the word frequency of each word in the FAQ database after word segmentation, determining the weight of each word, and storing the weight of each word and the word segmentation result into the FAQ database;

inquiring candidate questions corresponding to the question sentences in the FAQ database, scoring the candidate questions according to the weight of each word, and screening out the candidate questions with the score larger than or equal to a preset score as inquiry results;

calculating a similarity value between the question statement and the query result according to a similarity algorithm model, and filtering the query result of which the similarity value is not in a preset range;

and calculating the filtered query result by adopting a classification algorithm, and determining the problems with the highest similarity to the input problem sentences and the preset number.

2. The method as claimed in claim 1, wherein the step of querying candidate questions corresponding to the question sentences in the FAQ database and scoring the queried questions according to the weight of each term comprises:

extracting keywords in the question sentences according to the weight of each word;

determining an expansion word of the keyword;

generating a candidate question corresponding to the question sentence according to the keyword and the expansion word;

the candidate questions are scored according to the weight of the word.

3. The method for screening questions based on the FAQ database according to claim 1, wherein the step of calculating the similarity value between the question statement and the query result according to a similarity algorithm model, and filtering the query result whose similarity value is not within a preset range specifically comprises:

respectively calculating the similarity of Jaccard, the similarity of BM25, the similarity of cosine and the similarity of edit distance between the question sentence and the query result;

weighting and summing the calculated Jaccard similarity, BM25 similarity, cosine similarity and edit distance similarity according to respective weight values to obtain a similarity value between the question statement and the query result;

and filtering the query results of which the similarity values are not in a preset range.

4. The FAQ-database-based question screening method according to claim 3, wherein the step of filtering the query results whose similarity values are not within a preset range specifically comprises:

comparing the similarity value with a preset similarity threshold value;

and reserving the query results with the similarity values larger than or equal to a preset similarity threshold according to the comparison results, and removing the query results with the similarity values smaller than the preset similarity threshold.

5. The FAQ-database-based question screening method according to claim 1, wherein the step of calculating the filtered query result by using a classification algorithm and determining the question with the highest similarity to the input question sentence comprises:

respectively acquiring word vectors of the question sentences and the query results through FastText;

and inputting the word vector into a textCNN model, constructing a similarity matrix of the question statement and the query result after the operation of a convolution layer and a pooling layer, and outputting the similarity of the question statement and the query result through a full connection layer.

6. The FAQ-database-based question screening method according to claim 1, further comprising, after the step of querying candidate questions in the FAQ database corresponding to the question sentences:

and judging the number of the problems in the candidate problems, and executing corresponding operation according to the judgment result.

7. The method as claimed in claim 6, wherein the step of determining the number of questions in the candidate questions and performing corresponding operations according to the determination result specifically comprises:

if the number of the problems is zero, directly adopting a classification algorithm to calculate;

if the number of the problems is less than or equal to a preset threshold value, directly returning the candidate problems to the client;

and if the number of the problems is larger than a preset threshold value, scoring the candidate problems according to the weight of each word.

8. An FAQ database-based question screening apparatus, comprising:

the word segmentation module is used for analyzing question sentences input by a user and carrying out word segmentation processing on the question sentences;

the processing module is used for counting the word frequency of each word in the FAQ database after word segmentation, determining the weight of each word and storing the weight of each word and the word segmentation result into the FAQ database;

the query scoring module is used for querying candidate questions corresponding to the question sentences in the FAQ database and scoring the candidate questions according to the weight of each word;

the screening module is used for screening out the problems with the scores larger than or equal to the preset scores as query results;

the similarity calculation module is used for calculating a similarity value between the question statement and the query result according to a similarity calculation model and filtering the query result of which the similarity value is not in a preset range;

the classification calculation module is used for calculating the filtered query result by adopting a classification algorithm; and

and the determining module is used for determining the preset number of questions with the highest similarity to the input question sentences.

9. A computer device comprising a memory having computer readable instructions stored therein and a processor that when executed implements the steps of the FAQ database-based question screening method of any one of claims 1 to 7.

10. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon computer readable instructions, which when executed by a processor, implement the steps of the FAQ database-based question screening method according to any one of claims 1 to 7.

Images