CN107291817B - Method and device for constructing longitudinal search engine - Google Patents

Abstract

The invention discloses a method and a device for constructing a longitudinal search engine, which belong to the technical field of Internet, and comprise the following steps: acquiring search data when a user browses a webpage, wherein the search data at least comprises: each webpage link group comprises a plurality of webpage links; determining first weight values of a plurality of webpage links in each webpage link group; merging the same webpage links in each webpage link group to determine a second weight value of each merged webpage link; obtaining a third weight value of each group of webpage connection group according to the second weight value of each webpage link; and sorting the webpage link groups from large to small according to the third weight value. The method provided by the invention improves the searching efficiency and has the technical effect of improving the searching experience of the user.

Description

A method and device for constructing a vertical search engine

technical field

The invention belongs to the field of Internet technology, and in particular relates to a method and device for constructing a vertical search engine.

Background technique

The core of the existing web search engine, like the page-rank algorithm used by Google, first finds all web pages related to the search keyword, and then sorts the result set based on the references between web pages.

However, using the above existing search engine, when the user inputs several keywords in the search engine, the result list returned by the search engine is the result of sorting the web pages.

In this way, in the specific search process, the user completes the search process by sequentially clicking the web page links in the result list, which has the technical defect of low search efficiency.

SUMMARY OF THE INVENTION

The present invention provides a method and device for constructing a vertical search engine, which is used to solve the problem of low search efficiency caused by the user's specific search process in the prior art, which is to complete the search process by sequentially clicking on web page links in the result list. technical defects.

According to an aspect of the embodiments of the present invention, the embodiments of the present invention provide a method for constructing a vertical search engine, including:

Obtaining search data when the user browses a web page, the search data at least includes: several web page link groups, each of which includes several web page links;

determining the first weight value of the several webpage links in each of the webpage link groups;

combining the same webpage links in each of the webpage link groups to determine the second weight value of each of the combined webpage links;

obtaining a third weight value of each group of the webpage link groups according to the second weight value of each of the webpage links;

The plurality of webpage link groups are sorted from large to small according to the third weight value.

Further, the method further includes: outputting the plurality of the webpage link groups after sorting in a sorted order.

Further, the method further includes: the search data further includes several keyword groups, each of which corresponds to each of the webpage link groups.

Further, the method further includes: the several webpage link groups are three groups, including a first webpage link group, a second webpage link group, and a third webpage link group;

The number of the several webpage links in the first webpage link group is n1;

The number of the several webpage links in the second webpage link group is n2;

The number of the several webpage links in the third webpage link group is n3;

The determining the first weight value of the several webpage links in each of the webpage link groups includes:

For the first webpage link group, the first weight value of each webpage link is 1/n1;

For the second webpage link group, the first weight value of each webpage link is 1/n2;

For the third webpage link group, the first weight value of each webpage link is 1/n3;

Wherein, the n1, the n2 and the n3 are all positive integers.

Further, the method further includes: merging the same webpage links in each of the webpage link groups to determine the second weight value of each of the combined webpage links includes:

Among the plurality of web page links, the first weight value of each identical web page link in the corresponding web page link group is added, as the second weight value of the same web page link after merging ;

Among the several webpage links, the first weight of each different webpage link in the corresponding webpage link group is used as the second weight of the different webpage link.

Further, the method further includes: obtaining the third weight value of each group of the webpage link groups according to the second weight value of each of the webpage links includes:

The second weight values of the several web page links in each of the web page link groups are respectively added together as a third weight value of the web page link group.

According to yet another aspect of the embodiment of the present invention, the embodiment of the present invention further provides an apparatus for constructing a vertical search engine, the apparatus comprising: a search data acquisition module for acquiring search data when a user browses a web page, the search The data at least includes: several web page link groups, each of which includes several web page links; a first weight value determination module for determining the number of web page links in each of the web page link groups. a first weight value; a second weight value determination module, configured to combine the same webpage links in each of the webpage link groups to determine the second weight value of each of the combined webpage links; The three-weight value determination module is used for obtaining the third weight value of each group of the webpage link groups according to the second weight value of each of the webpage links; the sorting module is used for sorting a number of The web page link groups are sorted in descending order.

Optionally, the apparatus further includes: an output module configured to output the plurality of the web page link groups after sorting in a sorted order.

One or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages:

The present invention first obtains the web page link search data when the user uses the search engine, in the first aspect, calculates the first weight value of each web page link, in the second aspect, combines the same web page links to calculate the second weight value of the same web page, In the third aspect, the third weight value of the webpage link group is calculated according to the second weight value of the same webpage; in the fourth aspect, the webpage link group is sorted from large to small according to the third weight value of the webpage link group. In this way, in the specific search process, when the user enters a keyword to search, the result list fed back to the user is the result that has been arranged, that is, the web page link group corresponding to the keyword input by the user. The web link group is already sorted from large to small. The meaning of sorting the webpage link groups according to the third weight value is that the webpage link group containing more important webpage links has higher importance; the webpage link group included in more webpage link groups has higher importance. According to the feedback results, users do not need to click the web links in turn to complete the search process. The feedback results provide users with experience information for reference, so that users can complete the search process faster based on experience information, improve search efficiency, and improve User search experience.

The above description is only an overview of the technical solutions of the present invention, in order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand , the following specific embodiments of the present invention are given.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

1 is a schematic flowchart of a method for constructing a vertical search engine according to an embodiment of the present invention;

2 is a structural block diagram of an apparatus for constructing a vertical search engine provided by another embodiment of the present invention;

3 a is a schematic diagram of a complete search process definition 1 in a method for constructing a vertical search engine;

Figure 3b is a schematic diagram of a complete search process definition 2 in a method for constructing a vertical search engine;

4a is a schematic diagram of a search path definition 1 in two different search processes of a method for constructing a vertical search engine;

4b is a schematic diagram of search path definition 2 in two different search processes of a method for constructing a vertical search engine;

FIG. 5 is a schematic diagram of an example of operation data searched by a user in a method for constructing a vertical search engine.

Detailed ways

The method and device for constructing a vertical search engine provided by the embodiments of the present invention are used to solve the specific search process of users in the prior art. The search process is completed by sequentially clicking web page links in the result list, resulting in low search efficiency. The technical effect of improving the search efficiency and the user's search experience has been achieved.

The technical scheme in the embodiment of the present invention, the general idea is as follows:

A method for constructing a vertical search engine, the method comprising:

Obtaining search data when the user browses a web page, the search data at least includes: several web page link groups, each of which includes several web page links;

determining the first weight value of the several webpage links in each of the webpage link groups;

combining the same webpage links in each of the webpage link groups to determine the second weight value of each of the combined webpage links;

obtaining a third weight value of each group of the webpage link groups according to the second weight value of each of the webpage links;

The plurality of webpage link groups are sorted from large to small according to the third weight value.

In the embodiment of the present invention, by acquiring the web page link search data when the user uses the search engine, the first weight value of each web page link is calculated first, the second weight value of the same web page link is calculated by combining the same web page links, and the third weight value is calculated according to the The third weight value of the webpage link group is calculated from the second weight value of the same webpage link; fourthly, the webpage link group is sorted from large to small according to the third weight value of the webpage link group. In the specific search process, when the user enters a keyword to search, the result list fed back to the user is the result that has been arranged, that is, the web page link group corresponding to the keyword input by the user, the web page at this time. Link groups have been sorted from largest to smallest. The meaning of sorting the webpage link groups according to the third weight value is that the webpage link group containing more important webpage links has higher importance; the webpage link group included in more webpage link groups has higher importance. According to the feedback results, users do not need to click the web links in turn to complete the search process. The feedback results provide users with experience information for reference, so that users can complete the search process faster based on experience information, improve search efficiency, and improve The technical effect of the user's search experience effect.

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The term "and/or" in this article is only an association relationship to describe the associated objects, indicating that there can be three kinds of relationships, for example, A and/or B, it can mean that A exists alone, A and B exist at the same time, and A and B exist independently B these three cases. In addition, the character "/" in this document generally indicates that the related objects are an "or" relationship.

An embodiment of the present invention provides a method for constructing a vertical search engine. The method can be applied to the field of Internet technology. Please refer to FIG. 1. The method includes but is not limited to the following steps:

Step S101: Obtain search data when a user browses a web page, the search data at least includes: several web page link groups, each of which includes several web page links;

Further, the search data further includes several keyword groups, and each of the keyword groups corresponds to each of the web page link groups.

Specifically, the web page link group includes web page links that the user clicks in sequence in a complete search process, and the web page link group is obtained when the user enters a keyword to search. A web link group is all web links clicked during a complete search. The user enters a keyword to search, which may be several keywords on the Google homepage or several keywords on the Baidu homepage, and the user clicks the web page links in sequence in the result list returned by the search engine.

The search data when the user browses the webpage is obtained, and the search data also includes several keyword groups, and each keyword group corresponds to each webpage link group. This data can be obtained by a search engine, for example, in the server of the Baidu search website to record each search process (starting from the user accessing the search engine with a browser until the user closes the browser, all search keyword submissions and webpage jumping operations The process may be referred to as a search process) of the user's operation data, including input keywords and/or clicked web page links. These data are stored in the Baidu search engine server in the form of log files. The actual log files may contain more information, but to obtain the search data when users browse web pages, only the required information can be selected. The data extraction process can use ETL Tool complete. After the data extraction is completed, the search data obtained when the user browses the web page can be in the form of key-value pairs (search keywords and search links), such as <"Health", ("Netease Health", "Health News Network")>, At this time, the keyword group is: "health", and the web page link group is: "Netease Health", "Health News Network". Each keyword group corresponds to each web link group means: as a keyword group "health", after entering "health" in the search engine, the user clicks in sequence in the results returned by the search engine: "Netease Health", "Health News" "Net" two webpage links; at this time, "Netease Health" and "Health News Network" are a webpage link group corresponding to the keyword group "health".

Referring to FIG. 3a and FIG. 3b, a complete search process has two definitions, and FIG. 3a and FIG. 3b are diagrams of the search process under the two definitions respectively.

Definition 1: The search process refers to the entire process in which a user submits a search keyword group to a search engine server and clicks a web link to jump. When the user submits a new search keyword group again, a new search process begins. In this case, the webpage link group refers to: all webpage links clicked by the user in sequence during a complete search process.

As shown in Figure 3a, a search process starts when a user submits a search keyword group once. It should be noted that the webpage jumping in FIG. 3a refers to the operation that the user clicks on the webpage link in the result list returned by the search engine to jump. The first search process means that the user accurately gives the search keyword group and finds the desired web page link through one submission.

For example: the user enters "outdoor sports equipment" as a keyword group to search, and clicks two web links in turn in the result list returned by the search engine server: "Chinese outdoor equipment", "mountain outdoor" (for the convenience of description, in the manual The specific link address is omitted, and the link title is used to represent the link itself. For example, "China Outdoor Equipment" is the link title, and its corresponding specific link address is: http://www.papbout.com/); then the user again Entering "movie 2016" as a new keyword group to submit a search, the process of the user inputting the keyword group "outdoor sports equipment" and clicking two webpage links is defined as a complete search process. At the same time, the two web links clicked by the user are for the keyword group "outdoor sports equipment"; at this time, the two web links clicked by the user in turn: "Chinese outdoor equipment" and "mountain outdoor" are a web page link group.

Definition 2: The search process refers to the process of all search keyword group submissions and web page jump operations starting from the user accessing the search engine with a browser until the user closes the browser. In this case, the webpage link group refers to: all webpage links clicked by the user in sequence during a complete search process.

As shown in Fig. 3b, the user starts from opening the browser until closing the browser, and the process of submitting all search keyword groups to the search engine server and the corresponding web page jump request process is a search process. The second type of search process means that the user can only find the desired information after modifying the search keyword group for many times.

For example: when a user uses a browser to access a search engine, the first keyword group entered is "outdoor sports", the search engine server returns a list of results, and at this time, two webpage links of "Sina Sports" and "Phoenix Sports" are clicked in turn; The user updates the keyword group for the second time, and the keyword group entered at this time is "health", and the corresponding two webpage links of "Netease Health" and "Health News" are clicked in turn, and then the user closes the browser. Then, the process that the user inputs the keyword group "outdoor sports" and the corresponding two web page links clicked, and the process of inputting the keyword group "health" and the corresponding clicked two web page links is defined as a complete search process. At the same time, when the user enters the keyword group "outdoor sports", the corresponding 2 webpage links are clicked: "Sina Sports" and "Phoenix Sports"; the user enters the keyword group "health", and the corresponding 2 webpage links clicked: "Netease Health" In the process of "Sina Sports", "Phoenix Sports", "Netease Health", and "Health News", the four webpage links are a webpage link group.

The above-mentioned two definition criteria can be a relationship between two alternatives. For the set of input data, either the first criterion is used for processing, or the second criterion is used. It is up to the implementer of the search engine to decide.

Referring to Fig. 4a and Fig. 4b, a search path refers to a sequence of webpage jumps (all webpage links clicked) performed by a user in a search process. The two search paths correspond to the above two search processes, respectively, and FIG. 4a and FIG. 4b are respectively the search paths in two different search processes.

Figure 4a corresponds to the search process of Definition 1 above. After the user submits the search keyword group, a web page jump operation is performed, that is, several web page links are clicked in the result list returned by the search engine, and the sequence of these web page links is the search path. In this case, a web page link group refers to: all web page links included in a search path.

Fig. 4b corresponds to the search process of Definition 2 above. When the user submits the search keyword group multiple times and performs multiple webpage jump operations, the sequence of all webpage links is the search path. In this case, a web page link group refers to: all web page links included in a search path.

"Several" in several webpage link groups is a positive integer, that is, several webpage link groups can be one, two, three, four, etc. When the search data includes three webpage link groups, then there is the first A webpage link group, a second webpage link group, and a third webpage link group. The fact that the search data includes at least several web page link groups means that the search data may also include information such as keyword groups corresponding to each web page link group. As in the above example, when the keyword group corresponding to each web page link group is included, the search data is like <"Health", ("Netease Health", "Health News Network")>, namely "NetEase Health" and "Health News Network" The keyword group corresponding to a web page link group formed by "health" is "health".

The fact that the search data includes at least several web page link groups means that the search data may also include a search keyword group. According to the above definition: in a search path, all web links included are: web link group.

Among them, the key-value pair of the search keyword group and the search path webpage link group is <K, SP>, K is the keyword group, and SP is the webpage link group corresponding to the keyword group, that is, each search path corresponds to one input key-value pair .

Please refer to Figure 5. Figure 5 is the operation data of the user's search. After the user opens the browser, the user submits two search keyword groups to the search engine server, the first time is "health", the second time is "exercise" and "" guide". Correspondingly, after the first submission, I clicked the two webpage links of "Netease Health" and "Health News" in turn, and the second time was "Healthy Exercise Guidance". Then, the user closes the browser.

According to the definition of the first search process above, there are 2 pieces of input data corresponding to the example in Figure 5, which are <{"Health"}, {"Netease Health", "Health News"}> and <{"Exercise Guidance"}> "}, {"Healthy Exercise Guide"}>. Among them, <{"Health"}, {"NetEase Health", "Health News Network"}> is an input key-value pair, among which "NetEase Health" and "Health News Network" are a web page link group.

According to the definition of the second search process above, the input data corresponding to the example in Figure 5 is 1, which is <{"Health", "Exercise guidance"}, {"Netease Health", "Health News Network", "Healthy Exercise" guidance"}>. Among them, {"Netease Health", "Health Newspaper", "Healthy Exercise Guide"} is a web link group.

Step S102: determining the first weight value of the several webpage links in each of the webpage link groups;

Further, the several webpage link groups are three groups, including a first webpage link group, a second webpage link group, and a third webpage link group;

The number of the several webpage links in the first webpage link group is n1;

The number of the several webpage links in the second webpage link group is n2;

The number of the several webpage links in the third webpage link group is n3;

The determining the first weight value of the several webpage links in each of the webpage link groups includes:

For the first webpage link group, the first weight value of each webpage link is 1/n1;

For the second webpage link group, the first weight value of each webpage link is 1/n2;

For the third webpage link group, the first weight value of each webpage link is 1/n3;

Wherein, the n1, the n2 and the n3 are all positive integers.

Specifically, the several webpage link groups may be: two, three, four, five, etc. webpage link groups. When several webpage link groups are three groups at that time, the three groups can be respectively: the first webpage link group, the second webpage link group, and the third webpage link group; wherein, each of the several webpage links in the first webpage link group The number is n1; the number of several webpage links in the second webpage link group is n2; the number of several webpage links in the third webpage link group is n3; n1, n2 and n3 are all positive integers; each webpage link The initial weight value of the group can be set to 1. For example (this column will also be quoted in step S103): the first webpage link group is ("Netease Health", "Health News"), the corresponding keyword group is: "health", and the first webpage link group includes 2 webpage links, namely "Netease Health" and "Health News"; the second webpage link group is ("Netease Health", "Sohu Health", "Baidu Health"), and the corresponding keyword group is: "Health" ”, the second webpage link group includes 3 webpage links, namely “NetEase Health”, “Sohu Health”, and “Baidu Health”; the third webpage link group is (“NetEase Health”, “Sohu Health”, “Phoenix Health” "), the corresponding keyword group is: "health", the third web page link group includes 3 web page links, namely "Netease Health", "Sohu Health", "Phoenix Health";

The determining the first weight value of the several webpage links in each of the webpage link groups includes: for the first webpage link group, the first weight value of each webpage link is 1/n1;

For the second web page link group, the first weight value of each web page link is 1/n2; for the third web page link group, the first weight value of each web page link is 1/n3; as above Example: n1=2 of the first webpage link group, since the initial weight value of each webpage link group can be set to 1, the first weight value of the webpage link "NetEase Health" in the first webpage link group is 1/2 ; n2=3 of the second webpage link group, and the first weight value of "Sohu Health" in the second webpage link group is 1/3.

The steps of this embodiment may be implemented by using the MapReduce computing framework of Google, which is a highly parallelized computing framework. That is, using multiple computers to deal with the same problem at the same time, in which each computer is divided into a small part of the whole problem, if there are thousands of machines calculating at the same time, the whole process is equivalent to being divided into thousands of parts, which will speed up calculation process. When a parallel framework allows parallel computing using a large number of computers at the same time, it is commonly used to call it highly parallelized. The MapReduce computing framework divides the computing process of the entire problem into a Map stage and a Reduce stage (the following will introduce the steps to implement the present invention in detail according to the Map stage and the Reduce stage. For example, the Map1 stage is the first stage of the Map, and the Reduce2 stage is the The second stage of Reduce), where the output of the Map stage is the input of the Reduce stage, and it can also contain multiple Map stages and multiple Reduce stages. Hadoop is open source software that implements the MapReduce computing framework. Steps S102 , S103 , S104 , and S105 in the embodiments of the present invention can all be implemented using the MapReduce computing framework and directly using Hadoop software.

For example, in the algorithm pseudocode shown below, in the Map stage (step S102 is completed), the first weight value of all webpage links in each webpage link group is calculated for each piece of search data in the input data set. The input data set received at this stage is {<K _i , SP _i >}, where <K _i , SP _i > is the ith input data (i is a positive integer) in the input data set, and K _i is the keyword group , SP _i is a web page link group, for example: the keyword group of the first input data is K ₁ , and SP ₁ is the corresponding web page link group. In the algorithm, p _j ( _j is a positive integer) refers to the web page link, for example: p ₁ is the first web page link, in the second to fourth lines of the algorithm, each web page included in the search path (that is, in the web page link group) is analyzed. For link p _j , the first weight value w of the web page link is calculated, and the function length(SP _i ) is used to obtain the number of web page links included in SP _i . If a search path contains n webpage links (n is a positive integer), the initial weight of each webpage link is 1/n. The output of this stage is a set of key-value pairs, in which the key-value pair data contains the first weight value of the web page connection, the form is similar to <"Health"|"Netease Health", 0.5>, the key is the web page link, the value is the first weight value of the corresponding web page link; wherein, it is assumed that 0.5 is the first weight value of "web page health".

Map phase 1

Line 1: input:{<K _i ,SP _i >}

Line 2: for all p _j in SP _i do

Line 3:

Line 4: output{<K _i |p _j ,w>}

Line 5: end for

Reduce phase 1

Line 6: input:{<K _i |p _j ,w>}

Line 7:

Line 8: output{<K _i |p _j ,W _j >}

Reduce phase 2

Line 9: input:{<K _i ,SP _i >},{<K _i |p _j ,W _j >}

Line 10:

Line 11: output{<K _i ,SP _i |SPW _i >}

Step S103: merging the same webpage links in each of the webpage link groups to determine the second weight value of each of the combined webpage links;

Further, in the several webpage links, the first weight value of each identical webpage link in the corresponding webpage link group is added as the first weight value of the same webpage link after merging. Two weight values;

Among the several webpage links, the first weight of each different webpage link in the corresponding webpage link group is used as the second weight of the different webpage link.

Specifically, the same web page link in the web page link group refers to the same web page link included in each web page link group. "NetEase Health" and "NetEase Health" in the second webpage link group and "NetEase Health" in the third webpage link group are all the same webpage links. Link the same webpage links included in the first, second, and third webpage link groups. Merging is: merging the first weight values of “NetEase Health” in the three web page link groups. Since the initial weight value of each web page link group can be set to 1, at this time, the “NetEase Health” web page link The second weight value is 1/2+1/3+1/3=7/6; "Sohu Health" in the second web link group and "Sohu Health" in the third web link group are also the same web link , the second weight value of "Sohu Health" after the merger is 1/3+1/3=2/3. The different webpage links in the page link group refer to the different webpage links included in each webpage link group. For example, in the above example, the “Health News” in the first webpage link group and the second webpage link group The "Baidu Health" in the third webpage link group and the "Phoenix Health" in the third webpage link group are different webpage links. The second weight value of "Health News" in the first webpage link group is equal to its first weight value. , that is, the second weight value of "Health News Network" is equal to 1/2.

When using MapReduce to calculate the framework, in the stage of Reduce 1: completing step S103, that is, adding the first weight values of all web page links under the same K _i |p _j ; wherein, the same K _i |p _j refers to: At the same time, both K _i and p _j are the same. (Refer to line 7 of the algorithm pseudocode shown above, where W _j is the second weight value of web page links, and j is a positive integer), and calculate the second weight value W _j of all web page links. In this stage, the output of the Map stage is used as the input of the Reduce1 stage, and then the same webpage links in all the webpage link groups are merged, and the first weight value of the same webpage link is added as the second link of the same webpage link after the merger. For the weight value, the first weight of each different webpage link in the corresponding webpage link group is taken as the second weight of the different webpage link. The output value at this stage is still a set of key-value pairs, in which the form of each key-value pair data is similar to <"Health" | "Netease Health", 7/6>, "Netease Health" is a web page link, 7/6 It is the second weight value of "NetEase Health".

Because the same web page links are merged, the first weight value of each same web page link in the corresponding web page link group is added during merging, so the higher weight value means that the user pays high attention when searching. The web page link can feed back the high price information in the user's historical experience information to the user, so that the user can complete the search faster, and achieve the technical effect of improving the search efficiency.

Step S104: obtaining a third weight value of each group of the webpage link groups according to the second weight value of each of the webpage links;

Further, the second weight values of the several web page links in each of the web page link groups are added together as the third weight value of the web page link group.

Specifically, the third weight value of the webpage link group is a value obtained by adding the second weight values of several webpage links included in the webpage link group. As in the above example (this column will also be quoted in step S105): the second weight value of the webpage link "NetEase Health" in the first webpage link group is 7/6, and the second weight value of the webpage link of "Health News Network" is 1/2, at this time, the third weight value of the first webpage link group is equal to the sum of the second weight values of "NetEase Health" and "Health News", that is, the third weight value of the first webpage link group is: 7/ 6+1/2=5/3.

When using MapReduce to calculate the framework, in the Reduce 2 stage: Step S104 is completed, that is, adding the second weight values of all web page links in each web page link group to calculate the third weight value of all web page link groups. In the calculation of the 10th line of the above algorithm pseudocode, for the keyword K _i , the third weight value SPW i of the corresponding web page link group SP _i is the third weight value SPW _i of all web page links corresponding to SP _i of K _i |p _j The two weight values W _j are added together to obtain. In this stage, the output of the Reduce 1 stage is used as the input in the Reduce 2 stage again. According to the second weight value of each webpage link output in the Reduce 1 stage, the third weight value of all webpage connection groups is calculated, and a set of key-value pairs is output. The key-value pair data contains the calculated value of the third weight value of the webpage link group, in the form of <"Health"|("Netease Health", "Health News Network")|5/3>, the key is the webpage link group, The value is the third weight value of the corresponding webpage link group.

Step S105: Sort a plurality of the webpage link groups from large to small according to the third weight value.

Further, outputting the plurality of the web page link groups after the sorting is performed according to the sorting order.

Specifically, the same webpage link group means that the webpage links included in the webpage link group are all the same, and the webpage link groups all correspond to the same keyword group. For example: Suppose the fourth webpage link group includes three webpage links: "NetEase Health", "Sohu Health" and "Phoenix Health", because the third webpage link group in the above example only includes: "NetEase Health", "Phoenix Health" Sohu Health” and “Phoenix Health” are three webpage links, and the keyword groups corresponding to the fourth webpage link group and the third webpage link group are all “health”, that is, the webpage links clicked in turn when the user enters the keyword group “health” , so the fourth webpage link group and the third webpage link group are the same webpage link group. When in the first, second, third, and fourth webpage link groups, only the third and fourth webpage link groups are the same webpage link group, if the third weight value of the third webpage link group is 1.2, the fourth webpage link group The third weight value of the group is also 1.2. Finally, the two web page link groups are merged into one link group when sorting, and the third weight value of the web page link group used in sorting is also 1.2; The size is sorted from largest to smallest. Because the third and fourth webpage link groups are the same webpage link group, the third and fourth webpage links are combined into one webpage link group to participate in the sorting. The webpage link group includes: "Netease Health", "Sohu Health" ” and “Phoenix Health” three webpage links, the third weight value of the webpage link group is: 1.2, the third weight value of the combined webpage link group is compared with the third weight value of each different webpage link group. The size comparison is to arrange the third weight value of the combined webpage link group and the third weight value of the first and second webpage link groups in descending order.

When outputting according to the third weight value of the webpage link group, the results can be arranged in descending order: (“NetEase Health”, “Sohu Health”, “Phoenix Health”), (“NetEase Health”, “Health News Network” ”), (“NetEase Health”, “Sohu Health”, “Baidu Health”), and the corresponding keyword groups are: “Health”. At this time, the result can be output alone, or the result can be output together with the result list returned by the original search engine. For example, it can be output together with the result list returned by the search engine when the user enters a keyword on the Google homepage. output. When a user uses this method, the search process can be as follows: submit search keywords to the search engine as usual, and then the search engine returns two results: web page ranking and ranking according to the third weight value of the web page link group by this method. Wherein, the ranking result of the web pages returned by the search engine is consistent with the ranking result when only the search engine is used. The result of using this method to arrange the third weight value of the webpage link group in descending order is to provide the user with a recommended/optional search path in the form of a sequence of multiple webpage links. Because this method is used to arrange the output according to the third weight value of the webpage link group from large to small, the user's experience information is fed back to the user, so that the user can complete the search faster, and achieves the technical effect of improving the search efficiency. .

When using MapReduce to calculate the frame, the output result in line 11 of the above algorithm pseudocode is: set {<K _i , SP _i | SPW _i >} (step S105 is completed). All webpage link groups under the same search keyword group are sorted according to their third weight value. When the user starts a new search process, if the submitted search keyword group is exactly the same as a certain keyword group in the output result, the sorted result of all webpage link groups under the keyword group is output.

The method provided by the present invention can be combined with the big data method to collect and analyze massive user search data, so the analysis result is: the more search paths selected by the user, the higher the ranking of the webpage link group. In the method provided by the present invention, the output arrangement result provides reference information for a large number of users. The top-ranked web page link group has more reference value and can represent the choices of most users. Therefore, the ranking result provided by the method provided by the present invention can meet the needs of most users. Therefore, the method provided by the present invention can improve the search experience of most users, and allow most users to quickly find a series of web pages needed according to the sorting results of the web page link groups, so as to achieve the technical effect of improving search efficiency and user search experience. .

Based on the same inventive concept, the embodiments of the present invention also provide apparatuses corresponding to the foregoing method embodiments, as follows:

Another embodiment of the present invention provides an apparatus, please refer to FIG. 2 , the apparatus includes:

A search data acquisition module 201, configured to acquire search data when a user browses a web page, the search data at least includes: several web page link groups, and each of the web page link groups includes several web page links;

a first weight value determination module 202, configured to determine the first weight value of the several webpage links in each of the webpage link groups;

The second weight value determination module 203 is configured to combine the same webpage links in each of the webpage link groups to determine the second weight value of each of the combined webpage links;

A third weight value determining module 204, configured to obtain a third weight value of each group of the webpage link groups according to the second weight value of each of the webpage links;

The sorting module 205 is configured to sort several of the webpage link groups from large to small according to the third weight value.

In the embodiment of the present invention, the apparatus further includes: an output module, configured to output the plurality of the web page link groups after sorting in a sorted order.

In the embodiment of the present invention, the several webpage link groups are three groups, including a first webpage link group, a second webpage link group, and a third webpage link group; the several webpages in the first webpage link group The number of links is n1; the number of the several webpage links in the second webpage link group is n2; the number of the several webpage links in the third webpage link group is n3; the determining The first weight value of the several webpage links in each of the webpage link groups includes: for the first webpage link group, the first weight value of each webpage link is 1/n1; for the first webpage link group, the first weight value of each webpage link is 1/n1; In the second webpage link group, the first weight value of each webpage link is 1/n2; for the third webpage link group, the first weight value of each webpage link is 1/n3; wherein, the n1 , the n2 and the n3 are all positive integers.

In this embodiment of the present invention, the second weight value determination module further includes:

The first sub-module is used to add the first weight values of each identical web page link in the corresponding web page link group among the several web page links, as the same after the combination. The second weight value of the webpage link;

The second sub-module is configured to use, among the several webpage links, the first weight of each different webpage link in the corresponding webpage link group as the second weight of the different webpage links .

In this embodiment of the present invention, the third weight value determination module further includes:

The third sub-module is configured to respectively add the second weight values of the plurality of the webpage links in each of the webpage link groups as the third weight value of the webpage link group.

Since the device introduced in another embodiment of the present invention is the device used to implement the method embodiment of the present invention, based on the method introduced in the first embodiment of the present invention, those skilled in the art can understand the specific structure and deformation of the device, Therefore, it is not repeated here. All devices used in the methods of the embodiments of the present invention belong to the scope of protection of the present invention.

The technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages:

By obtaining the web page link search data when the user uses the search engine, the first weight value of each web page link is calculated, the second weight value of the same web page is calculated by combining the same web page links, and the third weight value of the same web page is calculated according to the second The weight value calculates the third weight value of the webpage link group; fourthly, the webpage link group is sorted from large to small according to the third weight value of the webpage link group, and the repeated webpage link group is only arranged once during sorting. In the specific search process, when the user enters a keyword group to search, the result list fed back to the user is the result that has been arranged, that is, the web page link group corresponding to the keyword group input by the user, the web page at this time. Link groups have been sorted from largest to smallest. The meaning of sorting the webpage link groups according to the third weight value is that the webpage link group containing more important webpage links has higher importance; the webpage link group included in more webpage link groups has higher importance. According to the feedback results, users do not need to click the web links in turn to complete the search process. The feedback results provide users with experience information for reference, so that users can complete the search process faster based on experience information, improve search efficiency, and improve User search experience.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

Although preferred embodiments of the present invention have been described, additional changes and modifications to these embodiments may occur to those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be construed to include the preferred embodiment and all changes and modifications that fall within the scope of the present invention.

Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, provided that these modifications and variations of the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims (6)

1. a construction method of vertical search engine, is characterized in that, described method comprises: The search data when the user browses the web page is obtained, the search data at least includes: several web page link groups, each of which includes several web page links; wherein, the search data further includes: several keyword groups, each One of the keyword groups corresponds to each of the webpage link groups; determining a first weight value of the several webpage links in each of the webpage link groups; Among the plurality of web page links, the first weight value of each identical web page link in the corresponding web page link group is added, as the second weight value of the same web page link after merging ; obtaining a third weight value of each group of the webpage link groups according to the second weight value of each of the webpage links; Sorting a plurality of the webpage link groups from large to small according to the third weight value; The plurality of the web page link groups described after the sorting are output according to the sorting order. 2. The method of claim 1, wherein: The several webpage link groups are three groups, including a first webpage link group, a second webpage link group, and a third webpage link group; The number of the several webpage links in the first webpage link group is n1; The number of the several webpage links in the second webpage link group is n2; The number of the several webpage links in the third webpage link group is n3; The determining the first weight value of the several webpage links in each of the webpage link groups includes: For the first webpage link group, the first weight value of each webpage link is 1/n1; For the second webpage link group, the first weight value of each webpage link is 1/n2; For the third webpage link group, the first weight value of each webpage link is 1/n3; Wherein, the n1, the n2 and the n3 are all positive integers. 3. The method of claim 2, wherein: Among the several webpage links, the first weight of each different webpage link in the corresponding webpage link group is used as the second weight of the different webpage link. 4. The method of claim 3, wherein the obtaining the third weight value of each group of the webpage link groups according to the second weight value of each of the webpage links comprises: The second weight values of the several web page links in each of the web page link groups are respectively added together as a third weight value of the web page link group. 5. A device for constructing a vertical search engine, wherein the device comprises: A search data acquisition module, used for acquiring search data when a user browses a web page, the search data at least includes: several web page link groups, each of which includes several web page links; wherein, the search data further includes : several keyword groups, each of which corresponds to each of the web page link groups; a first weight value determination module, configured to determine the first weight value of the several webpage links in each of the webpage link groups; The second weight value determination module is configured to add the first weight values of each identical web page link in the corresponding web page link group among the several web page links, as the combined identical web page link. the second weight value of the webpage link; a third weight value determining module, configured to obtain the third weight value of each group of the webpage link groups according to the second weight value of each of the webpage links; The sorting module sorts several of the webpage link groups from large to small according to the third weight value. 6. The apparatus of claim 5, further comprising: The output module is used for outputting the plurality of the webpage link groups after sorting according to the sorting order.

Images