CN120561225A

CN120561225A - Query reply method, device, storage medium and equipment

Info

Publication number: CN120561225A
Application number: CN202510506741.5A
Authority: CN
Inventors: 欧阳亚; 陈鸿基; 李俊龙; 熊思伟; 付鹏宇; 丁振兰; 彭艳华; 刘永恒
Original assignee: Peng Cheng Laboratory
Current assignee: Peng Cheng Laboratory
Priority date: 2025-04-22
Filing date: 2025-04-22
Publication date: 2025-08-29

Abstract

The embodiment of the present application provides a query reply method, apparatus, storage medium and equipment, including: receiving query information sent by an object, determining the target identity authority of the object, and determining the query vector of the query information; based on the query type, query vector, business text vector, business image vector, business operation vector and interface call vector corresponding to each business node of the query information, determining at least one target business node that matches the query information from each business node; according to the target identity authority of the object, filtering out target data that meets the target identity authority from the business text, business image, business operation information and interface call information corresponding to the target business node; generating reply content according to the target data, and sending the reply content to the object. Replying based on the identity of the object is not limited to text-modal reply content, and increases the richness of application scenarios.

Description

Query reply method, device, storage medium and equipment

Technical Field

The present application relates to the field of artificial intelligence technologies, and in particular, to a query reply method, a device, a storage medium, and a device.

Background

The technical evolution of the current knowledge question-answering system has formed an infrastructure taking a static knowledge base as a core and text interaction as a main body, and derives typical technical routes of template matching, search question-answering, generation question-answering and the like, but the technical evolution gradually exposes systematic defects in complex application scenes. Firstly, the output form of a single text mode is difficult to support the high-order requirements of multi-element collaborative presentation such as operation guidance, technical document analysis and the like, and an object cannot acquire key information such as an interface screenshot, an operation line and the like through pure text interaction. And cannot differentially present content according to object identity (e.g., developer, average user). Therefore, a query recovery method is needed to solve the above-mentioned problems.

Disclosure of Invention

The application mainly aims to provide a query reply method, a query reply device, a storage medium and query reply equipment, which can reply contents which are not limited to text modes according to object identities, and increase the application scene richness of a knowledge question-answering system.

In a first aspect, an embodiment of the present application provides a query reply method, including:

Receiving inquiry information sent by an object, determining target identity authority of the object, and determining an inquiry vector of the inquiry information;

Determining at least one target service node matched with the query information from each service node based on the query type, the query vector, the service text vector, the service image vector, the service operation vector and the interface call vector of the query information;

Screening out target data conforming to the target identity authority from service text, service image, service operation information and interface call information corresponding to the target service node according to the target identity authority of the object;

and generating reply content according to the target data, and sending the reply content to the object.

In a second aspect, an embodiment of the present application provides a query reply device, including:

The receiving unit is used for receiving inquiry information sent by an object, determining the target identity authority of the object and determining an inquiry vector of the inquiry information;

A determining unit, configured to determine, from each service node, at least one target service node that matches the query information based on a query type, a query vector, a service text vector, a service image vector, a service operation vector, and an interface call vector of the query information;

the screening unit is used for screening out target data conforming to the target identity authority from service text, service image, service operation information and interface calling information corresponding to the target service node according to the target identity authority of the object;

and the generating unit is used for generating reply content according to the target data and sending the reply content to the object.

In a third aspect, embodiments of the present application provide a storage medium storing a plurality of instructions adapted to be loaded by a processor to perform a query reply method as defined in any one of the above.

In a fourth aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements a query reply method as in any one of the above when the computer program is executed.

In the embodiment of the application, the target identity authority of an object is determined by receiving query information sent by the object, and a query vector of the query information is determined, at least one target service node matched with the query information is determined from each service node based on the query type, the query vector, the service text vector, the service image vector, the service operation vector and the interface call vector of the query information, the target data conforming to the target identity authority is screened out from the service text, the service image, the service operation information and the interface call information corresponding to the target service node according to the target identity authority of the object, reply content is generated according to the target data, and the reply content is sent to the object.

Additional features and advantages of the disclosure will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the disclosure. The objectives and other advantages of the disclosure will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

In order to more clearly illustrate the embodiments of the application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present description, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of a scenario of a query reply system according to an embodiment of the present application.

Fig. 2 is a flowchart of a query reply method according to an embodiment of the present application.

FIG. 3 is a schematic diagram of a graphical interface according to an embodiment of the present application

Fig. 4 is a schematic diagram of a service output processing flow provided in an embodiment of the present application.

Fig. 5 is a flow chart illustrating the extraction of different modal vectors according to an embodiment of the present application.

Fig. 6 is a schematic diagram of reply content for an administrator according to an embodiment of the present application.

Fig. 7 is a schematic diagram of reply content for a common object according to an embodiment of the present application.

Fig. 8 is a schematic structural diagram of a query reply device according to an embodiment of the present application.

Fig. 9 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed Description

In order that those skilled in the art will better understand the solution of the present application, a technical solution of an embodiment of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiment of the present application, and it is apparent that the described embodiment is only a part of the embodiment of the present application, not all the embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to fall within the scope of the application.

It should be noted that, in some of the processes described in the specification, claims and drawings above, a plurality of steps appearing in a particular order are included, but it should be clearly understood that the steps may be performed out of order or performed in parallel, the step numbers are merely used to distinguish between the different steps, and the numbers themselves do not represent any order of execution. Furthermore, the description of "first," "second," or "object" and the like herein is for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order.

Referring to fig. 1, fig. 1 is a schematic diagram of a scenario of a query reply system according to an embodiment of the present application. It includes a terminal 140, the internet 130, a gateway 120, a server 110, etc.

The terminal 140 includes, but is not limited to, a pre-configured notebook computer, or an electronic device with data reporting capability, such as a tablet computer, a desktop computer, etc. In addition, the device can be a single device or a set of a plurality of devices. The terminal 140 may communicate with the internet 130 in a wired or wireless manner, exchanging data.

The terminal 140 refers to a computer system capable of reporting query information to the server 110. The server 110 has higher requirements in terms of stability, security, performance, etc. than a general terminal. The server 110 may be one high-performance computer in a network platform, a cluster of multiple high-performance computers, a portion of one high-performance computer (e.g., a virtual machine), a combination of portions of multiple high-performance computers (e.g., virtual machines), etc.

Gateway 120 is also known as an intersubnetwork connector, protocol converter. The gateway implements network interconnection on the transport layer, and is a computer system or device that acts as a translation. The gateway is a translator between two systems using different communication protocols, data formats or languages, and even architectures that are quite different. At the same time, the gateway may also provide filtering and security functions. The query information sent by the terminal 140 to the server 110 is to be sent to the corresponding server 110 through the gateway 120. The reply content sent by the server 110 to the terminal 140 is also sent to the corresponding terminal 140 through the gateway 120.

The query reply method of the embodiments of the present disclosure may be implemented at the server 110.

It should be noted that, the schematic view of the scenario of the query reply system shown in fig. 1 is only an example, and the query reply system and the scenario described in the embodiment of the present application are for more clearly describing the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided by the embodiment of the present application, and as one of ordinary skill in the art can know, along with the evolution of the image processing technology and the appearance of the new service scenario, the technical solution provided by the embodiment of the present application is also applicable to similar technical problems.

In this embodiment, description will be made from the viewpoint of a query reply device which may be integrated in a computer apparatus having a storage unit and a microprocessor mounted thereon and having an arithmetic capability.

Referring to fig. 2, fig. 2 is a flowchart of a query reply method according to an embodiment of the application. The query reply method comprises the following steps:

In step 201, query information sent by an object is received, target identity authority of the object is determined, and a query vector of the query information is determined.

Wherein when an object sends query information, information such as account login information, a verification Token (Token) and the like before the object initiates the query information can be mapped to predefined identity rights (such as a system administrator, a developer, a tester, a common business object, an external cooperation object and the like).

Specifically, the query vector is a query vector generated by inputting query information into a query feature extraction model (e.g., deepSeek model) after domain tuning, and thus the query feature extraction model. Taking the DeepSeek model as an example, query information q, query vector vq=deepsek (q). The generated JSON format query vector is recorded as follows:

{

"query" how to query contract status,

"queryVector":[0.12,-0.34,0.56,...]

}

In the above example, the query information query is "how to query the contract status? after feature extraction by the query feature extraction model, the corresponding query vector queryVector is" [0.12, -0.34,0.56, ]. The numerical values in the query vectors reflect semantic features of the query information and are used for carrying out similarity calculation on the vectors corresponding to the service nodes.

In step 202, at least one target service node matching the query information is determined from each service node based on the query type of the query information, the query vector, the service text vector, the service image vector, the service operation vector, and the interface call vector corresponding to each service node.

In the embodiment of the application, a vector database is configured in advance, and the vector database comprises each service node and a multi-mode vector corresponding to the service node. The multi-modal vector specifically comprises four modes, namely a business text vector, a business image vector, a business operation vector and an interface call vector. The service text vector is a numerical vector obtained by performing a series of preprocessing and model calculation on text contents (such as titles, descriptions, operation guides, function descriptions and the like) in the service node. The service image vector is a vector representation obtained by extracting features of images (such as operation interface screenshot, service flow chart and the like) related to the service node. The business operation vector is a vector generated by using a Natural Language Processing (NLP) technology, deepSeek and other models for operation flow texts related to the business node (such as texts describing operation steps, such as clicking a query contract button, inputting a user name and a password). The interface call vector is a vector obtained by vectorizing API call information (such as description of an API interface, a call method, parameters and the like) related to the service node. The query types comprise a single-step query type and a multi-step query type, and the specific query types can judge the query types through a trained intention classification model so as to judge the query intention of the object.

Specifically, to determine the reply content of the query information, it is necessary to comprehensively consider whether the query type of the query information is a single-step query or a multi-step query, and determine at least one target service node matching the query information from each service node by combining the multi-modal vector corresponding to each service node. Since the single-step query type is a query for only the content of a certain service node, and the multi-step query type is a query for the content of a plurality of service nodes, at least one matching target service node can be determined herein.

In some embodiments, the determining, based on the query type, the query vector, the service text vector, the service image vector, the service operation vector, and the interface call vector of the query information, at least one target service node matching the query information from each service node includes:

(1) Determining a target vector type corresponding to the target identity authority based on the mapping relation between the identity authority and the vector type;

(2) Calculating the similarity of the query vector and the vector of the target vector type in each service node, and determining the maximum similarity as the similarity of the service node and the query vector;

(3) Screening candidate service nodes with the maximum similarity from each service node;

(4) And when the query type of the query information is single-step query, determining the candidate service node as a target service node.

The method comprises the steps of determining a target vector type corresponding to the target identity authority based on a mapping relation between the identity authority and the vector type, determining the similarity between a vector of the target vector type in each service node and a query vector after determining the target vector type, wherein the similarity between the vectors can be calculated by referring to the following formula:

S_Modality＝cos(v_q,v_Modality);

V _Modality is a vector of the target vector type in the service node, cos (v _q,v_Modality) is a cosine similarity between the query vector and the vector of the target vector type, and a final similarity calculation result S _Modality is obtained.

After obtaining the similarity between the vector of the target vector type and the query vector in each service node, since there may be multiple types of vectors of the target vector type, for each service node, the maximum similarity in the similarity corresponding to the target vector type is determined as the similarity between the affiliated service node and the query vector. The similarity of the service node to the query vector can be calculated with reference to the following formula:

S_total＝max(S_text,S_img,S_op,S_api);

Wherein S _text is the similarity between the service text vector and the query vector in the service node, S _img is the similarity between the service image vector and the query vector in the service node, S _op is the similarity between the service operation vector and the query vector in the service node, and S _api is the similarity between the interface call vector and the query vector in the service node. max (S _text,S_img,S_op,S_api) is the maximum similarity selected from the plurality of similarities, as the similarity between the service node and the query vector S _total.

After the similarity between each service node and the query vector is obtained, the service node with the maximum similarity is determined as the candidate service node, and since the single-step query is the query type aiming at only one service node, when the query type of the query information is the single-step query, the candidate service node with the maximum similarity is determined as the target service node.

Therefore, the target vector type is determined through the mapping relation between the identity authority and the vector type, and the vector similarity between the query vector and the target vector type is calculated, so that the service node with the highest matching degree with the query intention of the user can be accurately screened out. For example, the ordinary user pays more attention to the operation flow, the system can determine the service operation vector as the target vector type according to the authority of the user to match, the matching degree of the query result and the user demand is improved, a large amount of irrelevant information is prevented from being returned, and the user can quickly acquire the required content. The requirements of different users are met, and personalized service is realized, namely, the identity authority of the users is considered, and the users with different authorities see information with different vector types. For example, a developer may need detailed information related to the interface call vector, and an ordinary user only needs to operate the basic operation guide corresponding to the vector and the text vector. The personalized processing mode can better meet the differentiated requirements of various users and improve the user experience.

In some embodiments, the method further comprises:

(1) When the query type of the query information is multi-step query, determining an upward backtracking service node from the candidate service node based on a parent-child relationship with the candidate service node, and determining a downward backtracking service node from the candidate service node;

(2) Constructing a candidate service node backtracking chain according to the upward backtracking service node, the candidate service node and the downward backtracking service node;

(3) And when the number of the candidate service node backtracking chains is one, determining each service node in the candidate service node backtracking chains as a target service node.

When the query type is multi-step query, it is stated that the object desires to query the service text in the multi-step service node, and because of the father-son relationship between the service nodes, the upward backtracking service node can be determined by respectively backtracking from the candidate service node according to the father-son relationship between the service nodes, and the downward backtracking service node can be determined by downward backtracking, wherein the upward is the father relationship direction, and the downward is the son relationship direction. And constructing a complete candidate service node backtracking chain without disconnection according to the upward backtracking service node, the candidate service node and the downward backtracking service node obtained by downward backtracking determined by upward backtracking. And when the number of candidate service node backtracking is only one, each service node in the candidate service node backtracking chain is directly determined to be the target service node.

For example, the candidate service node is a service node a, a downward backtracking service node is determined to be a parent node B of the service node a in the upward backtracking process, a downward backtracking service node is determined to be a child node C of the service node a and a child node D of the child node C in the downward backtracking process, only one constructed candidate service node backtracking chain exists, namely the parent node B-service node a-child node C-child node D, and the parent node B, the service node a, the child node C and the child node D are determined to be target service nodes.

In this way, the parent-child relationships between the service nodes reflect the logical order and dependencies in the service flow. By backtracking up and down, the system is able to mine all information about candidate service nodes deeply along this relationship. For example, in an e-commerce system, when an object inquires about how to process a refund of a customer, a candidate service node may be an "audit refund application", and forward steps such as "customer initiates refund request" may be found by upward backtracking, and subsequent steps such as "refund to account" may be found by downward backtracking, so that the whole flow of refund service is fully covered. The backtracking chain constructed according to the parent-child relationship can ensure that the sequence of the service nodes accords with the actual service logic, so that the information provided for the object has consistency and accuracy. Information jump or deletion is avoided, and the objects can understand and execute business operations according to the correct sequence.

In some embodiments, the method further comprises:

(1) When the number of the candidate service node backtracking chains is not one, determining a first weight value of each upward backtracking service node and a second weight value of each downward backtracking service node included in each candidate service node backtracking chain;

(2) Determining the upward backtracking similarity of each candidate service node backtracking chain based on a first weight value of each upward backtracking service node included in each candidate service node backtracking chain and the similarity with the query vector;

(3) Determining the backward tracing similarity of each candidate service node tracing chain based on the second weight value of each backward tracing service node included in each candidate service node tracing chain and the similarity with the query vector;

(4) Determining the sum value of the upward backtracking similarity of each candidate service node backtracking chain and the corresponding downward backtracking similarity to obtain the total similarity of each candidate service node backtracking chain;

(5) And screening a target service node backtracking chain with the maximum total similarity from each candidate service node backtracking chain, and determining service nodes included in the target service node backtracking chain as target service nodes.

When the number of the constructed candidate service node backtracking chains is not one, a first weight value of each upward backtracking service node and a second weight value of each downward backtracking service node in the candidate service node backtracking chains need to be determined, wherein the weight values can be preset according to service rules, for example, service nodes closer to a service end node are more important, and default is 1. In this way, a first weight value w _upi of each traceback service node and a second weight value w _downj of each traceback service node are determined.

Specifically, the upward backtracking similarity of the backtracking chain of each candidate service node can be calculated by the following formula:

S_up＝λ_up(∑_iw_upiS(v_q,v_upi));

Wherein v _upi is a vector of the ith upward backtracking service node, S (v _q,v_upi) is a similarity between the ith upward backtracking service node and the query vector v _q, Σ _iw_upiS(v_q,v_upi) is a sum of products of the similarity between each upward backtracking service node and the query vector v _q and the corresponding first weight value w _upi, and λ _up is a weight coefficient. Thus, the upward backtracking similarity of each candidate service node backtracking chain is obtained.

The backward tracing similarity of the backward tracing chain of each candidate service node can be calculated by the following formula:

S_diwn＝λ_down(∑_jw_downjS(v_q,v_downj));

Wherein v _downj is the vector of the j-th downstream traceback service node, S (v _q,v_upi) is the similarity between the j-th downstream traceback service node and the query vector v _q, Σ _jw_downjS(v_q,v_downj) is the sum of products of the similarity between each downstream traceback service node and the query vector v _q and the corresponding second weight value w _downj, and λ _down is the weight coefficient. Thus, the backward tracing similarity of each candidate service node tracing chain is obtained.

Finally, the total similarity of each candidate service node backtracking chain can be calculated by the following formula:

S_combined＝S_up+S_down=

λ_up(∑_iw_upiS(v_q,v_upi))+λ_down(∑_jw_downjS(v_q,v_downj));

Specifically, for each candidate service node backtracking chain, a sum of the corresponding upward backtracking similarity and the corresponding downward backtracking similarity is calculated to obtain a total similarity S _combined.

After the total similarity of each candidate service node backtracking chain is obtained, determining the candidate service node backtracking chain with the maximum total similarity as a target service node backtracking chain, and determining the service nodes included in the target service node backtracking chain as target service nodes.

Therefore, the correlation between a plurality of service nodes on the backtracking chain and the query can be comprehensively considered by setting weight values for the upward backtracking service node and the downward backtracking service node and combining the weight values with the similarity between the service nodes and the query vector to calculate the total similarity. For example, in a complex production and manufacturing process query, not only directly related operation nodes but also upstream and downstream association nodes are considered, so that the finally determined target service node can reflect the user demand more comprehensively and accurately, and the unilateral judgment based on a single node or simple association is avoided. The weight is set according to the service rule, for example, the weight of the service node closer to the service end node defaults to 1, and the node which has important significance for service completion is highlighted. In the process flow query of e-commerce order processing, the weight of nodes close to the flow end points such as 'confirmation of receipt', 'payment completion', and the like is higher, the influence of the similarity of the nodes on the result is larger when the total similarity is calculated, the system is ensured to be capable of preferentially returning core service nodes closely related to the user expectation, and the quality of the query result is improved.

In some embodiments, before determining at least one target service node matching the query information from each service node based on the query type, the query vector, the service text vector, the service image vector, the service operation vector, and the interface call vector of the query information, the method further includes:

(1) Acquiring a service code, and performing code conversion on the service code through an abstract syntax tree algorithm to obtain a conversion code;

(2) Responding to the recording control operation, recording service operation flow and interface calling information, generating a recording code, and writing the recording code into a conversion code of a target service node controlled by the control operation;

(3) Storing the service image of each service node under the corresponding service node in the conversion code;

(4) Integrating the service nodes in the conversion codes to obtain a simplified service tree, wherein the simplified service tree comprises service text, service images, service operation flow and interface call information of each service node;

(5) Carrying out semantic annotation on the service text of each service node to obtain a semantic label of each service node;

(6) And extracting the semantic tags, the service images, the service operation flows and the interface call information of each service node to obtain service text vectors, service image vectors, service operation vectors and interface call vectors corresponding to each service node.

The method can 1) acquire an HTML document of a service, namely a service code, by adopting an HTTP/HTTPS protocol, support AJAX and JavaScript rendering, and ensure that dynamic content is completely grabbed. The captured HTML document is used for subsequent analysis and is not directly used as knowledge graph data.

Contract viewing details pages of the data analyst platform are, for example:

The HTML label is converted into a tree structure by utilizing an Abstract Syntax Tree (AST) algorithm, the conversion result is stored in a special database table such as MongoDB in a nested JSON form, wherein the conversion result comprises label names, attributes, text contents and hierarchical relations of all nodes, and meanwhile, keywords and description information in the < meta > label are extracted. The JSON data (i.e. the conversion code) generated in the whole analysis process is used as the basis for the subsequent information integration and knowledge graph generation.

For example:

And a hidden door is arranged at the rear end and used for controlling whether to start the recording function, and after the hidden door is opened, an administrator account can intentionally record the key operation flow. In the recording process, the system captures the operation steps (such as recording XPath when clicking a 'query contract' button), records XPath, service operation flow, API interface calling information and other information, and adds the information into a corresponding service node JSON in a structured data format, and the system only records the API interface calling related to object interaction and automatically ignores WebSocket connection and polling APIs so as to reduce redundant data. For example:

When the system detects a version update or is operating for the first time, a page screenshot (i.e., a business image) is automatically captured and recorded under the corresponding business node in the translation code, replacing the old screenshot (if any). This information is also stored in JSON data, consistent with the operation records and API information, and no longer depends on the original HTML data.

And integrating service nodes in the conversion codes, extracting key information from the JSON data according to a predefined rule, namely automatically classifying the service nodes to the page head if node labels are < title > or < meta > and contain descriptive information, classifying the service nodes if the id or class of a certain < div > node contains keywords (such as content, main and policy), and integrating all the key information meeting the conditions into the corresponding service nodes when traversing the complete DOM JSON by using depth-first search (DFS) to generate a reduced service JSON tree.

For example, the extraction result for the contracted query page metadata is as follows:

batch information, etc.,

}

Here, the service node 7 acts as an aggregation point carrying page key information.

And finally, respectively extracting the semantic label, the service image, the service operation flow and the interface call information of each service node to obtain a service text vector, a service image vector, a service operation vector and an interface call vector corresponding to each service node, wherein all vectors of all service nodes form a vector database.

Therefore, the corresponding service text vector, service image vector, service operation vector and interface call vector are obtained by respectively extracting the semantic label, service image, service operation flow and interface call information of each service node. The vector representations convert the business information of different modes into a computable numerical form, so that the system can perform intelligent retrieval based on the vector similarity, and a more accurate and comprehensive query result is provided for a user. For example, when processing the inquiry about business operation by the user, the system can consider various information such as text, image, operation flow and the like at the same time, so that the accuracy and the relevance of the inquiry are improved.

In some embodiments, the semantic labeling of the service text of each service node to obtain a semantic label of each service node includes:

(1.1) performing word segmentation processing on the service text of each service node to obtain a plurality of word segments of each service node;

(1.2) matching each word segment with a preset business word, and determining the matching degree of each word segment;

(1.3) determining the word segmentation or word segmentation combination with the matching degree being in the preset number as the semantic label of the service node, and obtaining the semantic label of each service node.

The extracted service text (including title, description and functional description) is preprocessed by spaCy or NLTK to complete word segmentation, part-of-speech tagging and named entity recognition. And then automatically generating semantic tags by adopting a business keyword matching rule, wherein the business keyword matching rule is that a predefined business keyword library (comprising preset business words such as contracts, queries, registration and the like) is established, and the segmentation is matched to obtain the matching degree of each segmentation. When words with higher matching degree (for example, sorting according to each word segmentation or word segmentation combination with more matching degree) in the keyword library appear in the text, and the words with the preset number of word segmentation or word segmentation combination are taken, the semantic tags are preliminarily generated.

For example, if the terms "contract" and "query" frequently appear in the text, a "contract query" candidate tag is generated. After the candidate tags are determined (e.g., "contract query", "user registration", etc.) they are stored in the form of JSON sub-objects in the metadata (meta) of the corresponding service node for exact matching in subsequent retrieval and version control.

For example {

"id":"node_7",

"tag":"div",

"class":"content",

"PermissionTags" [ "developer" ]// only developer roles are visible

"Child": [ { "id": "node_8", "tag": "p", "content": "the page details how to query contract details and status. "}],

"metaData":{

"Title": "contract query help",

"Description" how to query contract details and status ",

"Keyword" means [ "contract", "query", "detail", "status" ]

},

"SemanticLabels" [ "contract query", "contract details", "State query ]

}

Wherein, keywords is a word segmentation, semanticLabels is a semantic tag.

Therefore, the core content of the service node can be accurately extracted by segmenting the service text and determining the semantic tags by matching with the preset service word. When a user inputs a query, the system can quickly locate relevant service nodes according to the semantic tags. For example, the user inquires about the contract inquiry flow, the system can quickly screen out the matched nodes from a large number of service nodes by virtue of the semantic label of contract inquiry, so that the inquiry response speed and accuracy are improved, and the time cost for the user to acquire information is reduced.

In some embodiments, the simplified service JSON tree may be utilized, and the system automatically generates a tree-like knowledge graph and stores the tree-like knowledge graph in a graph database (e.g., neo4 j), so as to ensure that all service nodes and key information are included. The administrator can edit the nodes (such as modifying URL, describing, screenshot, etc.) through the right key menu, and support the operations of adding, modifying, deleting and merging, thus ensuring clear structure and accurate data.

As shown in fig. 3, fig. 3 is a schematic diagram of a graphical interface provided by an embodiment of the present application, in which clicking a certain service node displays information such as metaData (extra metadata information related to the service node) back. The graphical interface schematic diagram comprises three pieces of key information of a top navigation bar, namely a data list, a contract list and an experiment creation. For the service main content area, the three pieces of key information and corresponding service nodes, such as sorting, screening and clicking details in a contract list, are respectively displayed.

Therefore, referring to fig. 4, fig. 4 is a schematic diagram of a service output processing flow according to an embodiment of the present application. The method comprises the steps of obtaining an HTML document to support AJAX/JS rendering, using the HTML document as original data for subsequent analysis, utilizing an AST algorithm to analyze the HTML document to generate a complete DOM JSON, storing databases such as DOM JSONs to MongDB, extracting tag keywords and descriptions, screening service related nodes such as titles, main contents, buttons and the like, generating a simplified service JSON tree to serve as a knowledge map data base, automatically generating a tree-shaped knowledge map storage map database Neo4j, and displaying and managing a visual interface to support editing, merging and deleting operations.

In some embodiments, the extracting the vector of the semantic label, the service image, the service operation flow and the interface call information of each service node to obtain a service text vector, a service image vector, a service operation vector and an interface call vector corresponding to each service node includes:

Inputting semantic labels of each service node into a trained text feature extraction model, and extracting service text vectors corresponding to each service node;

(1.2) inputting the service image of each service node into a trained image feature extraction model, and extracting a service image vector corresponding to each service node;

(1.3) inputting the business operation flow of each business node into a trained business operation feature extraction model, and extracting a business operation vector corresponding to each business node;

And (1.4) inputting the interface call information of each service node into the trained interface call feature extraction model, and extracting the interface call vector corresponding to each service node.

The semantic labels in the service nodes are preprocessed again, and the preprocessing steps comprise cleaning (special characters and redundant blank spaces), word segmentation, standardization (unified case) and word filtering disabling. The preprocessed text data is input into a DeepSeek model (a trained text feature extraction model) subjected to field fine tuning, and the whole semantic representation is directly output to generate high-dimensional text embedding. The generated vector is normalized and stored in a vector database { "ID": "vec_123456", "vector": [0.12, -0.45,0.67, ], jsonid: "node_7" }, its unique ID is stored in the metadata.textEmbedding field of the service node,

For example:

"metaData":{

"semanticLabels" [ "contract query", "status query" ],

"textEmbedding":"vec_123456"

}

TextEmbedding is a service text vector, and "vec_123456" is a service text vector corresponding to [ "contract query", "state query" ]. The DeepSeek model loads a pre-training model (for example DeepSeek-R1) under the HuggingFace Transformers framework, performs fine adjustment by using in-field labeling data, adopts a cross entropy loss function and an Adam optimizer in the training process, and dynamically adjusts training rounds by combining with an early stopping strategy so as to enhance the adaptability of the model to professional terms and operation description.

For the service image vector, the screenshot stored in the service node is subjected to image feature extraction by utilizing a CLIP model (trained image feature extraction model), the image vector is generated, the formula is vimg =clip (I), in order to facilitate the subsequent association of the image vector with the service node, the system stores an image embedded record in a vector database, and simultaneously, the corresponding screenshot path and vector ID are recorded in the service node JSON.

Vector database structures such as:

{

"id" vec_img_123456 "# is a unique field in JSON

Vector, [0.12,0.34, ], 0.98, # CLIP-generated vector

"metadata":{

"data_type":"image"

"Screenshot _path": "data/screenshots/1.Png", # screenshot path or field name in JSON

}

For the business operation vector, NLP and DeepSeek (trained business operation feature extraction model) can be used to generate an embedding, the specific formula is Vop=deep (O), wherein O represents the business operation flow, and then an association index is established in the vector database.

For the interface call vector, NLP and DeepSeek (input to the trained interface call feature extraction model) can be used to generate the embedding, and the specific formula is Vapi =deep Seek (A), wherein A represents interface call information.

Referring to fig. 5, fig. 5 is a flow chart illustrating the extraction of different modal vectors according to an embodiment of the application. Inputting corrected tree-like knowledge graph (namely, each service node comprises service text, service image, service operation flow and service interface call information), preprocessing the text comprising cleaning, analyzing, standardizing and stopping word filtering aiming at the service text, generating corresponding service text vector through DeepSeek, storing the corresponding service text vector into a vector database, recording ID textEmbedding, acquiring screenshot information (service image) aiming at the service image, generating service image vector by using a CLIP model, storing the service image vector into the vector database, recording ID imageEmbedding, preprocessing the description text of the service operation flow aiming at the service operation flow, generating service operation vector through DeepSeek, storing the service operation vector into the vector database, recording ID operationEmbedding, preprocessing the description of the service interface call information aiming at the service interface call information, generating interface call vector through DeepSeek, storing the interface call vector into the vector database, and recording ID apiEmbedding.

In step 203, according to the target identity authority of the object, the target data conforming to the target identity authority is screened out from the service text, the service image, the service operation information and the interface call information corresponding to the target service node.

The reply content conforming to the identity authority of the object is generated according to the target identity authority of the object, so that the target data conforming to the target identity authority is screened out from the service text, the service image, the service operation information and the interface call information corresponding to the target service node.

For example, the identity authority of the common object does not need to know conditions such as interface call, so that the receiving call information does not need to be the target data of the target identity authority of the common object, and the identity authority of the developer needs to know all contents of the service node, so that the service text, the service image, the service operation information and the interface call information are all used as the target data of the target identity authority of the developer. Thus, the target data is screened from the target service nodes according to the target identity authority.

In step 204, reply content is generated from the target data and sent to the object.

For the query type of single-step query, the NLG module is utilized to automatically abstract and semantically integrate text information (such as title, description and operation description) in the candidate nodes, so as to generate a logically consistent and content-rich answer text. For multi-step inquiry, the text content is reorganized according to the operation dependency sequence, so that the natural semantic connection among the steps is ensured, and meanwhile, the image vector records are required to be extracted from candidate nodes and combined with the original screenshot (stored in a local path) to be fused with the text abstract. In the answer interface, the relevant screenshot is displayed in a carousel or click-amplification mode, and interface information of each operation step is intuitively reflected. For multi-step inquiry, the image display is arranged according to the sequence of the operation flow, so that the object can clearly see the visual feedback of the whole flow.

Referring to fig. 6 and fig. 7, fig. 6 is a schematic diagram of reply content for an administrator according to an embodiment of the present application, and fig. 7 is a schematic diagram of reply content for a common object according to an embodiment of the present application. The front end displays the combined answer result using a markdown format.

For the problem how to check the contract, the reply content of the administrator comprises two steps, wherein step 1 is related description of logging in the system, the API call is:/API/login (post) through the account of the administrator, screenshot is [ click to check original figure ], step 2 is related description of entering contract management, and detailed contract data is acquired through navigation to a contract management module. The detailed log includes operation time, call parameters, and the like. And for the common object, information such as API call and log is not displayed.

According to the method, the system and the device, target identity authority of an object is determined by receiving query information sent by the object, query vectors of the query information are determined, query types of the query information, the query vectors, service text vectors corresponding to each service node, service image vectors, service operation vectors and interface call vectors are based on the query types of the query information, the query vectors, the service operation vectors corresponding to each service node, at least one target service node matched with the query information is determined from each service node, target data conforming to the target identity authority is screened out from service text, service images, service operation information and interface call information corresponding to the target service node according to the target identity authority of the object, reply content is generated according to the target data, and the reply content is sent to the object.

The specific implementation of the above steps can be referred to the previous embodiments, and will not be repeated here.

In order to facilitate better implementation of the query reply method provided by the embodiment of the application, the embodiment of the application also provides a device based on the query reply method. Where the meaning of a noun is the same as in the query reply method described above, specific implementation details may be referred to the description in the method embodiment.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a query reply device according to an embodiment of the application, where the query reply device is applied to a computer device. The query reply device may include a receiving unit 601, a determining unit 602, a screening unit 603, and a generating unit 604.

A receiving unit 601, configured to receive query information sent by an object, determine a target identity authority of the object, and determine a query vector of the query information;

A determining unit 602, configured to determine, from each service node, at least one target service node that matches the query information, based on a query type, a query vector, a service text vector, a service image vector, a service operation vector, and an interface call vector of the query information, where the service text vector, the service image vector, the service operation vector, and the interface call vector correspond to each service node;

a screening unit 603, configured to screen, according to the target identity authority of the object, target data that meets the target identity authority from service text, service image, service operation information and interface call information corresponding to the target service node;

and the generating unit 604 is configured to generate reply content according to the target data, and send the reply content to the object.

In some embodiments, the determining unit 602 includes:

the first determining subunit is used for determining a target vector type corresponding to the target identity authority based on the mapping relation between the identity authority and the vector type;

A calculating subunit, configured to calculate a similarity between the query vector and a vector of a target vector type in each service node, and determine a maximum similarity as a similarity between the service node and the query vector;

a screening subunit, configured to screen out candidate service nodes with the greatest similarity from each service node;

And the second determining subunit is used for determining the candidate service node as a target service node when the query type of the query information is single-step query.

In some embodiments, the determining unit 602 further comprises:

the backtracking subunit is used for determining an upward backtracking service node from the candidate service node and determining a downward backtracking service node based on a father-son relationship with the candidate service node when the query type of the query information is multi-step query;

the construction subunit is used for constructing a candidate service node backtracking chain according to the upward backtracking service node, the candidate service node and the downward backtracking service node;

and the third determining subunit is used for determining each service node in the candidate service node backtracking chain as a target service node when the number of the candidate service node backtracking chains is one.

In some embodiments, the determining unit 602 further comprises:

A fourth determining subunit, configured to determine, when the number of the candidate service node traceback chains is not one, a first weight value of each traceback service node included in each candidate service node traceback chain, and a second weight value of each traceback service node;

a fifth determining subunit, configured to determine an upward backtracking similarity of each candidate service node backtracking chain based on a first weight value of each upward backtracking service node included in each candidate service node backtracking chain and a similarity with the query vector;

a sixth determining subunit, configured to determine a backward tracing similarity of each candidate service node traceback chain based on a second weight value of each backward traceback service node included in each candidate service node traceback chain and a similarity with the query vector;

A seventh determining subunit, configured to determine a sum of an upward trace-back similarity of each candidate service node trace-back chain and a corresponding downward trace-back similarity, to obtain a total similarity of each candidate service node trace-back chain;

and the eighth determining subunit is configured to screen a target service node traceback chain with the maximum total similarity from each candidate service node traceback chain, and determine a service node included in the target service node traceback chain as a target service node.

In some embodiments, the apparatus further comprises:

the conversion unit is used for obtaining service codes, and performing code conversion on the service codes through an abstract syntax tree algorithm to obtain conversion codes;

The generating unit is used for responding to the recording control operation, recording the service operation flow and interface call information, generating a recording code and writing the recording code into a conversion code of a target service node controlled by the control operation;

The storage unit is used for storing the service image of each service node to the position below the corresponding service node in the conversion code;

the integration unit is used for integrating the service nodes in the conversion codes to obtain a simplified service tree, wherein the simplified service tree comprises service text, service image, service operation flow and interface call information of each service node;

the semantic annotation unit is used for carrying out semantic annotation on the service text of each service node to obtain a semantic label of each service node;

And the vector extraction unit is used for extracting the vectors of the semantic labels, the service images, the service operation flows and the interface call information of each service node respectively to obtain the service text vectors, the service image vectors, the service operation vectors and the interface call vectors corresponding to each service node.

In some embodiments, the semantic annotation unit comprises:

the word segmentation subunit is used for carrying out word segmentation processing on the service text of each service node to obtain a plurality of word segments of each service node;

The matching subunit is used for matching each word segment with a preset business word and determining the matching degree of each word segment;

And the ninth determining subunit is used for determining the word segmentation or word segmentation combination with the matching degree being in the preset number as the semantic label of the service node to obtain the semantic label of each service node.

In some embodiments, the vector extraction unit comprises:

the first input subunit is used for inputting the semantic label of each service node into the trained text feature extraction model and extracting the service text vector corresponding to each service node;

The second input subunit is used for inputting the service image of each service node into the trained image feature extraction model and extracting the service image vector corresponding to each service node;

the third input subunit is used for inputting the service operation flow of each service node into the trained service operation feature extraction model and extracting the service operation vector corresponding to each service node;

And the fourth input subunit is used for inputting the interface call information of each service node into the trained interface call feature extraction model and extracting the interface call vector corresponding to each service node.

The specific implementation of each unit can be referred to the previous embodiments, and will not be repeated here.

From the foregoing, it can be seen that, in the embodiment of the present application, the receiving unit 601 receives the query information sent by the object, determines the target identity authority of the object, and determines the query vector of the query information, the determining unit 602 determines at least one target service node matching with the query information from each service node based on the query type, the query vector, the service text vector, the service image vector, the service operation vector and the interface call vector of the query information, the screening unit 603 screens out the target data corresponding to the target identity authority from the service text, the service image, the service operation information and the interface call information corresponding to the target service node according to the target identity authority of the object, and the generating unit 604 generates reply content according to the target data and sends the reply content to the object. Compared with the prior art, the knowledge question-answering system can only answer questions in the output form of a text mode, and can not present contents according to the difference of object identities, the embodiment of the application determines the target service joints matched with query information through the similarity of the query vector and the multi-mode vector corresponding to each service joint, and screens out target data conforming to the target identity authority from the multi-mode data based on the target identity authority of the object, thereby generating reply contents conforming to the target identity authority according to the target data, realizing reply according to the object identity, being not only limited to the reply contents of the text mode, and expanding the application scene richness of the knowledge question-answering system.

Referring to fig. 9, fig. 9 is a block diagram of a portion of a computer device 1000 embodying an embodiment of the present disclosure. The computer device 1000 may vary considerably in configuration or performance and may include one or more central processing units (Central Processing Units, simply CPU) 622 (e.g., one or more processors) and memory 632, one or more storage mediums 630 (e.g., one or more mass storage devices) that store applications 642 or data 644. Wherein memory 632 and storage medium 630 may be transitory or persistent storage. The program stored in the storage medium 630 may include one or more modules (not shown), each of which may include a series of instruction operations in the server 600. Still further, the central processor 622 may be configured to communicate with a storage medium 630 and execute a series of instruction operations in the storage medium 630 on the server 600.

The computer device 1000 may also include one or more power supplies 626, one or more wired or wireless network interfaces 650, one or more input/output interfaces 658, and/or one or more operating systems 641, such as Windows ServerTM, mac OS XTM, unixTM, linuxTM, freeBSDTM, and the like.

The central processor 622 in the computer device 1000 may be used to perform the query reply method of embodiments of the present disclosure, for example:

The embodiments of the present disclosure also provide a computer readable storage medium storing a program code for executing the query reply method of the foregoing embodiments.

The disclosed embodiments also provide a computer program product comprising a computer program. The processor of the computer device reads the computer program and executes it, causing the computer device to execute the query reply method as described above. For example:

Furthermore, the terms "comprises," "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed or inherent to such process, method, article, or apparatus.

It should be understood that in the present application, "at least one (item)" means one or more, and "a plurality" means two or more. "and/or" is used to describe an association relationship of an associated object, and indicates that three relationships may exist, for example, "a and/or B" may indicate that only a exists, only B exists, and three cases of a and B exist simultaneously, where a and B may be singular or plural. The character "/" generally indicates that the context-dependent object is an "or" relationship. "at least one of" or the like means any combination of these items, including any combination of single item(s) or plural items(s). For example, at least one of a, b or c may represent a, b, c, "a and b", "a and c", "b and c", or "a and b and c", wherein a, b, c may be single or plural.

It should be understood that in the description of the embodiments of the present application, plural (or multiple) means two or more, and that greater than, less than, exceeding, etc. are understood to not include the present number, and that greater than, less than, within, etc. are understood to include the present number.

In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of elements is merely a logical functional division, and there may be additional divisions of actual implementation, e.g., multiple elements or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.

The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of the embodiments of the present application. The storage medium includes various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory RAM), a magnetic disk, or an optical disk.

It should also be appreciated that the various embodiments provided by the embodiments of the present application may be arbitrarily combined to achieve different technical effects.

In the present embodiment, the term "module" or "unit" refers to a computer program or a part of a computer program having a predetermined function and working together with other relevant parts to achieve a predetermined object, and may be implemented in whole or in part by using software, hardware (such as a processing circuit or a memory), or a combination thereof. Also, a processor (or multiple processors or memories) may be used to implement one or more modules or units. Furthermore, each module or unit may be part of an overall module or unit that incorporates the functionality of the module or unit.

The embodiments of the present application have been described in detail, but the present application is not limited to the above embodiments, and those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present application, and the equivalent modifications or substitutions are included in the scope of the present application as defined in the appended claims.

Claims

1. A method of query reply, comprising:

2. The query reply method according to claim 1, wherein the determining at least one target service node matching the query information from each service node based on a query type, a query vector, a service text vector, a service image vector, a service operation vector, and an interface call vector of the query information includes:

Determining a target vector type corresponding to the target identity authority based on the mapping relation between the identity authority and the vector type;

Calculating the similarity of the query vector and the vector of the target vector type in each service node, and determining the maximum similarity as the similarity of the service node and the query vector;

screening candidate service nodes with the maximum similarity from each service node;

And when the query type of the query information is single-step query, determining the candidate service node as a target service node.

3. The query reply method of claim 2, further comprising:

when the query type of the query information is multi-step query, determining an upward backtracking service node from the candidate service node based on a parent-child relationship with the candidate service node, and determining a downward backtracking service node from the candidate service node;

Constructing a candidate service node backtracking chain according to the upward backtracking service node, the candidate service node and the downward backtracking service node;

And when the number of the candidate service node backtracking chains is one, determining each service node in the candidate service node backtracking chains as a target service node.

4. The query reply method of claim 3, further comprising:

When the number of the candidate service node backtracking chains is not one, determining a first weight value of each upward backtracking service node and a second weight value of each downward backtracking service node included in each candidate service node backtracking chain;

determining the upward backtracking similarity of each candidate service node backtracking chain based on a first weight value of each upward backtracking service node included in each candidate service node backtracking chain and the similarity with the query vector;

determining the backward tracing similarity of each candidate service node tracing chain based on the second weight value of each backward tracing service node included in each candidate service node tracing chain and the similarity with the query vector;

Determining the sum value of the upward backtracking similarity of each candidate service node backtracking chain and the corresponding downward backtracking similarity to obtain the total similarity of each candidate service node backtracking chain;

And screening a target service node backtracking chain with the maximum total similarity from each candidate service node backtracking chain, and determining service nodes included in the target service node backtracking chain as target service nodes.

5. The query reply method according to claim 1, further comprising, before determining at least one target service node matching the query information from each of the service nodes, based on the query type, query vector, service text vector, service image vector, service operation vector, and interface call vector of the query information, each service node:

Acquiring a service code, and performing code conversion on the service code through an abstract syntax tree algorithm to obtain a conversion code;

Responding to the recording control operation, recording service operation flow and interface calling information, generating a recording code, and writing the recording code into a conversion code of a target service node controlled by the control operation;

Storing the service image of each service node under the corresponding service node in the conversion code;

Integrating the service nodes in the conversion codes to obtain a simplified service tree, wherein the simplified service tree comprises service text, service images, service operation flow and interface call information of each service node;

carrying out semantic annotation on the service text of each service node to obtain a semantic label of each service node;

And extracting the semantic tags, the service images, the service operation flows and the interface call information of each service node to obtain service text vectors, service image vectors, service operation vectors and interface call vectors corresponding to each service node.

6. The query reply method according to claim 5, wherein said semantically labeling the service text of each service node to obtain a semantic label of each service node includes:

performing word segmentation processing on the service text of each service node to obtain a plurality of word segments of each service node;

matching each word segment with a preset business word, and determining the matching degree of each word segment;

And determining the word segmentation or word segmentation combination with the matching degree in the preset number as the semantic label of the service node to obtain the semantic label of each service node.

7. The query reply method according to claim 5, wherein the extracting the vector of the semantic tag, the service image, the service operation flow, and the interface call information of each service node to obtain the service text vector, the service image vector, the service operation vector, and the interface call vector corresponding to each service node includes:

inputting the semantic label of each service node into a trained text feature extraction model, and extracting a service text vector corresponding to each service node;

inputting the service image of each service node into a trained image feature extraction model, and extracting a service image vector corresponding to each service node;

inputting the business operation flow of each business node into a trained business operation feature extraction model, and extracting a business operation vector corresponding to each business node;

And inputting the interface calling information of each service node into the trained interface calling feature extraction model, and extracting the interface calling vector corresponding to each service node.

8. A query reply device, comprising:

9. A computer readable storage medium storing a plurality of instructions adapted to be loaded by a processor to perform the query reply method of any one of claims 1 to 7.

10. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the query reply method of any of claims 1 to 7 when executing the computer program.