CN114417031A - Search method and device and search tag expansion method and device - Google Patents

Abstract

The present disclosure provides a search method and device, which relate to the technical fields of deep learning, image processing, and speech recognition. The specific implementation scheme is: obtaining the search corpus; in response to determining that the preset mapping relationship database does not include the search corpus, comparing the similarity between the search corpus and the reference corpus in the mapping relationship database to obtain a first similarity value, and the mapping relationship database It is used to characterize the correspondence between the reference corpus and the reference image cluster, and the reference image cluster includes at least one reference image; compare the similarity between the historical image and the reference image cluster in the mapping relationship library, and obtain a second similarity value; The first similarity and the second similarity determine historical images related to the search corpus. This implementation improves the recall rate of user searches.

Description

Search method and device, search tag extension method and device

technical field

The present disclosure relates to the field of computer technologies, in particular to the technical fields of deep learning, image processing, and speech recognition, and in particular, to a search method and apparatus, a search tag extension method and apparatus, electronic equipment, computer-readable media, and computer program products.

Background technique

In traditional technology, search relies on the richness and fine-grainedness of tags. For example, when a user searches for "cat", the model needs to cover the tag "cat". When the user needs to search for "Persian cat", the model also needs to cover "cat". ' sub-tab "Persian cat". On the other hand, the multi-label model has a high cost for label expansion, and the process cycle of data labeling, sorting, and effect testing is long, and the cost of brushing the library is high.

SUMMARY OF THE INVENTION

Provided are a search method and apparatus, a search tag extension method and apparatus, an electronic device, a computer-readable medium, and a computer program product.

According to a first aspect, a search method is provided, the method comprising: acquiring a search corpus; in response to determining that the preset mapping relation library does not include the search corpus, comparing the similarity between the search corpus and a reference corpus in the mapping relation library , obtain the first similarity value, the mapping relationship library is used to represent the corresponding relationship between the benchmark corpus and the benchmark image cluster, and the benchmark image cluster includes at least one benchmark image; By comparison, a second similarity value is obtained; based on the first similarity and the second similarity, a historical image related to the search corpus is determined.

According to a second aspect, a search tag extension method is provided, the method comprising: acquiring an image to be signed; in response to determining that the image to be signed is not included in a preset mapping relationship library, comparing the image to be signed with a reference in the mapping relationship library The similarity of the image clusters is compared to obtain a mapping similarity value. The mapping relationship library is used to represent the corresponding relationship between the reference corpus and the reference image cluster. The reference image cluster includes at least one reference image; based on the mapping similarity value, the image to be signed is determined. Corresponding reference corpus; add the corresponding relationship between the image to be signed and the reference corpus into the mapping relationship library.

According to a third aspect, a search device is provided, the device comprising: a corpus acquisition unit, configured to acquire a search corpus; a first comparison unit, configured to respond to determining that the preset mapping relation library does not include the search corpus, Comparing the similarity between the search corpus and the reference corpus in the mapping relationship library, to obtain a first similarity value, the mapping relationship library is used to represent the correspondence between the reference corpus and the reference image cluster, and the reference image cluster includes at least one reference image; The second comparison unit is configured to compare the similarity between the historical image and the reference image cluster in the mapping relationship library to obtain a second similarity value; the image determination unit is configured to be based on the first similarity and the second similarity, Identify historical images relevant to the search corpus.

According to a fourth aspect, a search tag extension device is provided, the device includes: an image acquisition unit configured to acquire an image to be signed; a comparison unit configured to respond to determining that the preset mapping relationship library does not include the image to be signed Image, compare the similarity between the to-be-signed image and the benchmark image cluster in the mapping relationship library to obtain the mapping similarity value, the mapping relationship library is used to represent the corresponding relationship between the benchmark corpus and the benchmark image cluster, and the benchmark image cluster includes at least one a reference image; a corpus determination unit configured to determine a reference corpus corresponding to the image to be signed based on the mapping similarity value; a mapping unit configured to add the correspondence between the to-be-signed image and the reference corpus into the mapping relation library.

According to a fifth aspect, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor. The at least one processor executes to enable the at least one processor to perform a method as described in any implementation of the first aspect or the second aspect.

According to a sixth aspect, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform a method as described in any implementation of the first aspect or the second aspect.

According to a seventh aspect, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method as described in any one of the implementations of the first aspect or the second aspect.

In the search method and device provided by the embodiments of the present disclosure, first, the search corpus is obtained; secondly, in response to determining that the preset mapping relationship database does not include the search corpus, the similarity between the search corpus and the reference corpus in the mapping relationship database is compared. , obtain the first similarity value, the mapping relationship library is used to represent the corresponding relationship between the benchmark corpus and the benchmark image cluster, and the benchmark image cluster includes at least one benchmark image; The similarity is compared to obtain a second similarity value; finally, based on the first similarity and the second similarity, historical images related to the search corpus are determined. Therefore, through the low-cost and fast customized mapping relationship between user search corpus and user pictures, all user search scenarios can be quickly and automatically covered, and the recall rate of user search can be improved while ensuring the search accuracy.

It should be understood that what is described in this section is not intended to identify key or critical features of embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become readily understood from the following description.

Description of drawings

The accompanying drawings are used for better understanding of the present solution, and do not constitute a limitation to the present disclosure. in:

1 is a flow chart of one embodiment of a search method according to the present disclosure;

FIG. 2 is a schematic diagram of searching using a search corpus and a mapping relation library in the present disclosure;

FIG. 3 is a flowchart of one embodiment of a search tag extension method according to the present disclosure;

4 is a schematic structural diagram of an embodiment of a search apparatus according to the present disclosure;

5 is a schematic structural diagram of an embodiment of a search tag extension apparatus according to the present disclosure;

FIG. 6 is a block diagram of an electronic device used to implement the search method or the search tag extension method according to an embodiment of the present disclosure.

Detailed ways

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding and should be considered as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.

FIG. 1 shows a process 100 according to an embodiment of the search method of the present disclosure, and the above-mentioned search method includes the following steps:

Step 101, acquiring search corpus.

In this embodiment, the acquired search corpus is a corpus corresponding to at least one modal information, wherein the modal information is represented in various forms in different applications or websites. For example, the modal information may include: text, video, picture, audio One or more of the modal information can be obtained by analyzing different modal information, for example, entity and relationship extraction is performed on the text on the webpage to obtain the search corpus corresponding to the text. For another example, a video segment is extracted from an introduction video on an application to obtain a search corpus corresponding to the video. For another example, for the search field, search all the questions input to the search engine, and obtain the search corpus corresponding to the search engine.

Step 102, in response to determining that the preset mapping relation library does not include the search corpus, compare the similarity between the search corpus and the reference corpus in the mapping relation library to obtain a first similarity value.

In this embodiment, the preset mapping relationship library is used to represent the correspondence between the reference corpus and the reference image cluster, and the reference image cluster includes at least one reference image. The mapping relation library may include multiple types of benchmark corpus, one benchmark corpus in each type of benchmark corpus corresponds to one benchmark image cluster, and each benchmark image cluster includes at least one benchmark image. For example, when the search corpus belongs to the dog category, the dog category includes many Subcategories, such as Teddy, correspond to benchmark image clusters that include multiple Teddy pictures. By mapping benchmark image clusters to benchmark corpus, the diversity effect of search can be guaranteed.

In this embodiment, the preset mapping relation library can select each reference corpus and the reference image cluster corresponding to each reference corpus by manual annotation, and the modalities of each reference corpus can include: text, video, picture, audio one or more. The reference image cluster is at least one image corresponding to each reference corpus. As shown in Figure 2, the reference corpus in the mapping relation database is a text modality, and the horses in the text modality correspond to multiple reference images related to horses. Optionally, the mapping relationship library can also crawl the reference image of the reference corpus through a web crawler to obtain a reference image cluster, and the process of the mapping relationship library can be automated by crawling the reference image through the web crawler.

In this embodiment, when the search corpus is the same as the reference corpus in the mapping relationship database, it is determined that the mapping relationship database includes the search corpus, and when the mapping relationship database includes the search corpus, the reference image cluster corresponding to the search corpus is directly output.

Optionally, when the similarity between the search corpus and the reference corpus in the mapping relationship library is greater than a preset threshold (eg, 90%), it is determined that the mapping relationship library includes the search corpus.

Optionally, the above-mentioned determining that the preset mapping relationship library does not include the search corpus includes: comparing the similarity between the search corpus and each reference corpus in the mapping relationship library, and in response to the similarity between the search corpus and the reference corpus being greater than a preset threshold value. , obtain the reference image cluster related to the reference corpus; detect whether the search corpus and any one of the reference images in the reference image cluster belong to the same category, and if they belong to the same category, determine that the mapping relation library includes the search corpus.

In this embodiment, comparing the similarity between the search corpus and the reference corpus in the mapping relationship database to obtain the first similarity value includes: calculating the similarity between the search corpus and the reference corpus to obtain the first similarity value, wherein , there are many ways to calculate similarity, such as Euclidean distance algorithm, included angle cosine algorithm, etc.

In this embodiment, when the search corpus is a text modality, a model based on deep learning can also be used, such as BERT (Bidirectional Encoder Representations from Transformers), Ernie (continuous pre-training framework for language understanding) ) etc. to calculate the similarity between the search corpus and the reference corpus to obtain a first similarity value.

In this embodiment, since there may be at least one reference corpus in the mapping relationship database, and the reference corpus for similarity comparison with the search corpus is at least one reference corpus in the mapping relationship database, the first similarity value has at least one , each similarity value is the similarity between the search corpus and a reference corpus.

Step 103 , compare the similarity between the historical image and the reference image cluster in the mapping relationship database to obtain a second similarity value.

In this embodiment, the historical images may be a large number of historical images collected at the target site in a preset time period. For example, the historical images are images corresponding to the questions searched by the user in the past three months obtained through a search engine. Optionally, after a large number of historical images are collected, the obtained historical images can be clustered by a clustering algorithm, and different types of historical images can be obtained. Types of historical images may be classified based on functions or attributes or characteristics of historical images. For example, the types of historical images include: introductory, detailed, interrogative.

In this embodiment, there is at least one second similarity value, the number of second similarity values is the same as the number of reference image clusters, and each reference image cluster corresponds to one second similarity value,

In this embodiment, the execution subject on which the search method runs may compare the similarity between the historical image and each reference image cluster in all reference image clusters, and obtain the second similarity value corresponding to each reference image cluster.

Optionally, the execution subject on which the search method runs may compare the similarity between the historical image and any one of the reference image clusters in each reference image cluster, and obtain the second similarity value corresponding to each reference image cluster.

Optionally, after obtaining the historical image, the execution subject on which the search method operates can also match the type of the historical image with different types of reference image clusters in the mapping relationship library, and determine the reference image corresponding to the type of the historical image. , calculate the similarity between the historical image and each reference image in this type of reference image, calculate the average of the similarity, and obtain the second similarity value.

Step 104: Determine historical images related to the search corpus based on the first similarity and the second similarity.

In this embodiment, a reference corpus in the mapping relationship database corresponds to a reference image cluster, that is, the reference corpus and the reference image cluster are in a corresponding relationship, and the first similarity value of the reference corpus can be associated with the reference image cluster. The second similarity value is weighted and summed to obtain the similarity value of the corresponding relationship. By sorting the similarity values of all the corresponding relationships in the mapping relationship database, the corresponding relationship with the highest similarity with the search corpus is determined, which will be similar to the corresponding relationship. The historical image with the highest degree is used as the historical image related to the search corpus.

Optionally, determining the historical images related to the search corpus based on the first similarity and the second similarity includes: for each corresponding relationship in the mapping relationship database, the first similarity of the reference corpus in the corresponding relationship can be performed. The degree value is multiplied by the second similarity value of the reference image cluster of the corresponding relationship to obtain the similarity value of the corresponding relationship. By sorting the similarity values of all correspondences in the mapping relationship database, the correspondence with the highest similarity to the search corpus is determined, and the historical image with the highest similarity with the correspondence is used as the historical image related to the search corpus.

In some optional implementations of this embodiment, determining the historical images related to the search corpus based on the first similarity and the second similarity includes: determining the first similarity and the first similarity corresponding to each corresponding relationship in the mapping relationship database. Two degrees of similarity; for each corresponding relationship, multiply the first similarity and the second similarity of the corresponding relationship to obtain the similarity of the corresponding relationship; sort the similarities of all the corresponding relationships in ascending or descending order; Sort results in descending order to identify historical images relevant to the search corpus.

In this optional implementation, based on the corresponding relationship of the top set positions (for example, the top 5 positions) in the descending order, the historical images of the corresponding relationship of the top set positions can be determined, and the historical images of the top set positions can be selected from the historical images of the top set positions. Historical images most relevant to the search corpus.

In this optional implementation manner, based on the corresponding relationship between the last set bits (for example, the last 5 digits) in the ascending order, the historical image of the corresponding relationship of the last set position can be determined, and the historical image of the last set position can be selected from the historical images of the last set position. Historical images most relevant to the search corpus.

In this optional implementation manner, multiply the first similarity and the second similarity of the corresponding relationship to obtain the similarity of the corresponding relationship; sort the similarities of all the corresponding relationships in ascending or descending order; arranging based on the ascending or descending order The arrangement results of the search corpus determine the historical images related to the search corpus, which provides a reliable implementation method for selecting the historical images of the search corpus, and ensures the accuracy of the correspondence between the historical images and the search corpus.

The search method provided by the embodiments of the present disclosure can improve the accuracy and recall of search scenarios of online disks and albums.

In the search method provided by the embodiments of the present disclosure, first, the search corpus is acquired; secondly, in response to determining that the preset mapping relation library does not include the search corpus, the similarity between the search corpus and the reference corpus in the mapping relation library is compared, and the result is obtained: The first similarity value, the mapping relationship library is used to represent the corresponding relationship between the reference corpus and the reference image cluster, and the reference image cluster includes at least one reference image; again, the similarity between the historical image and the reference image cluster in the mapping relationship library is calculated. By comparison, a second similarity value is obtained; finally, based on the first similarity and the second similarity, historical images related to the search corpus are determined. Therefore, through the low-cost and fast customized mapping relationship between user search corpus and user pictures, all user search scenarios can be quickly and automatically covered, and the recall rate of user search can be improved while ensuring the search accuracy.

In another embodiment of the present disclosure, the above method further includes: establishing a correspondence between the reference corpus, the search corpus, and the historical images related to the search corpus.

In this embodiment, after the correspondence between the reference corpus, the search corpus, and the historical images related to the search corpus is established, the correspondence between the reference corpus, the search corpus, and the historical images related to the search corpus can be stored in real time, and the When a user searches, based on the corresponding relationship, a reliable search result is provided for the user, and the search experience of the user is improved.

In some optional implementations of this embodiment, the similarity between the historical image and the reference image cluster in the mapping relationship library is compared to obtain the second similarity value, including: extracting the first image feature of the historical image; extracting the reference image The second image feature of the cluster; the similarity between the first image feature and the second image feature is compared to obtain a second similarity value.

In this optional implementation manner, the first image feature and the second image feature may be obtained through a feature extractor, wherein the feature extractor may use a model based on deep learning, such as CNN (Convolutional Neural Network), Transformer, and the like.

In this optional implementation manner, the second image feature may be obtained by extracting a feature of any image in the reference image cluster by a feature extractor.

In the method for obtaining the second similarity value provided in this optional implementation, the second similarity value is calculated by using the first image feature and the second image feature, which provides a method for calculating the similarity between the search corpus and the reference image. reliable way.

In some optional implementations of this embodiment, the above-mentioned extracting the second image features of the reference image cluster includes: acquiring the image features of each reference image in the reference image cluster; averaging the image features of all the reference images to obtain the second image feature. image features.

In this optional implementation manner, after acquiring the image features of each reference image in the reference image cluster, the image features of all reference images are averaged, so that the obtained second image features have the features of all reference images, ensuring that the second image feature has the features of all reference images. The reliability of the image features obtained.

In some optional implementations of this embodiment, the above-mentioned comparison of the similarity between the historical image and the reference image cluster in the mapping relationship library to obtain the second similarity value includes: extracting the first image feature of the historical image; obtaining the reference The image features of each reference image in the image cluster; the similarity between the first image feature and the image features of each reference image is calculated to obtain a similarity sequence; the second image feature is obtained based on the similarity sequence; the first image feature and the second image feature are obtained. The image features are compared for similarity to obtain a second similarity value.

In this optional implementation manner, obtaining the second image feature based on the similarity sequence includes: selecting the image feature of the reference image with the greatest similarity in the similarity sequence as the second image feature. Optionally, obtaining the second image feature based on the similarity sequence includes: sorting each similarity in the similarity sequence in ascending order, and calculating an average value of the image features of the reference image set after selecting to obtain the second image feature.

Optionally, the above-mentioned comparison of the similarity between the historical image and the reference image cluster in the mapping relationship library to obtain the second similarity value includes: extracting the first image feature of the historical image; obtaining the image of each reference image in the reference image cluster. feature; calculate the similarity between the first image feature and the image features of each reference image to obtain a similarity sequence; and obtain a second similarity value based on the similarity sequence.

In this optional implementation, the similarity sequence includes at least one similarity value between the first image feature and the image feature of the reference image, and the average similarity score of the TOPK in the similarity sequence can be taken as the second similarity value, where, when When K=1, the maximum similarity value in the similarity sequence is taken; when K=the number of reference images in the reference image cluster, the average of all the similarities in the similarity sequence can be taken to obtain the second similarity value.

In the method for obtaining the second similarity value provided in this optional implementation, the second image feature is calculated based on the similarity sequence, and then the second similarity value is calculated by using the first image feature and the second image feature, which is used for the search corpus and benchmark. Similarity calculation between images provides another reliable way.

As shown in Fig. 2, through the preset text similarity retriever, the first similarity value between the search corpus and the reference corpus in the preset mapping relation database can be obtained, which is denoted as St.

The second similarity value between the user's historical image and the reference image can also be obtained by calculation, denoted as Si. Specifically, the user's historical image is extracted by a preset feature extractor to obtain the first image feature (N dimension, N>1). The reference image cluster in the mapping relation library obtains the second image feature (N dimension, N>1) through the preset feature extractor. Calculate the similarity between the first image feature and the second image feature to obtain a second similarity value.

The second similarity value is obtained by calculating the similarity between each historical image of the user and the benchmark image cluster of each benchmark corpus. The second similarity can be recorded as the similarity between the historical image and the benchmark corpus. The historical image and benchmark The similarity value between image clusters can be calculated using different strategies according to different scenarios. There are two ways to calculate the similarity between the historical image and the reference image cluster:

1) Taking an average value of the features of the reference image cluster as the second image feature of the reference image cluster, calculating the similarity between the first image feature and the second image feature to obtain a second similarity value.

2) Calculate the similarity between the first image feature and the image feature of each benchmark image in the benchmark image cluster, and take the average similarity score of TOPK, indicating: when K=1, take the maximum similarity, and K=the benchmark image in the benchmark image cluster. number, and then take the average similarity of all the similarities to obtain the second similarity value.

The similarity between the historical images and the search corpus is obtained through the formula S=S _i *S _t , so that for the user: inputting the search corpus, a series of historical images that can be sorted according to the similarity can be obtained.

FIG. 3 shows a flow 300 of an embodiment of a search tag extension method according to the present disclosure. The above-mentioned search tag extension method includes the following steps:

Step 301, acquiring an image to be signed.

In this embodiment, the image to be signed is an image added to a preset mapping relationship library. By adding an image to the mapping relationship library, the richness of the images in the mapping relationship library can be increased.

In this embodiment, the preset mapping relationship library is used to represent the correspondence between the reference corpus and the reference image cluster, and the reference image cluster includes at least one reference image. The mapping relation library includes multiple types of reference corpus, one reference corpus in each type of reference corpus corresponds to one reference image cluster, and each reference image cluster includes at least one reference image.

In this embodiment, the preset mapping relation library can select each reference corpus and the reference image cluster corresponding to each reference corpus by manual annotation, and the modalities of each reference corpus can include: text, video, picture, audio one or more. The reference image cluster is at least one image corresponding to each reference corpus. As shown in Figure 2, the reference corpus in the mapping relationship database is a text mode, and the text horse corresponds to a plurality of reference images related to horses. Optionally, the mapping relationship library can also crawl the reference image of the reference corpus through a web crawler to obtain a reference image cluster, and the process of the mapping relationship library can be automated by crawling the reference image through the web crawler.

Step 302, in response to determining that the preset mapping relationship library does not include the image to be signed, compare the similarity between the image to be signed and the reference image cluster in the mapping relationship library to obtain a mapping similarity value.

In this embodiment, the mapping relationship library is used to represent the correspondence between the reference corpus and the reference image cluster, and the reference image cluster includes at least one reference image.

In this embodiment, before adding the to-be-signed image to the mapping relationship database, the to-be-signed image can be compared with each reference image in the mapping relationship database, and when the to-be-signed image is different from each reference image, the mapping is determined. Images to be signed off are not included in the relationship library.

In this embodiment, there are multiple mapping similarity values, and each mapping similarity value corresponds to a reference image cluster. Since the reference image cluster and the reference corpus are based on the corresponding relationship, each mapping similarity value corresponds to a reference corpus. correspond. The above-mentioned comparing the similarity between the image to be signed and the reference image clusters in the mapping relationship library, and obtaining the mapping similarity value includes: calculating the similarity between the image to be signed and any one of the reference image clusters in each reference image cluster in the mapping relationship library. , get the mapping similarity value of each benchmark image cluster. It should be noted that the mapping similarity value in this embodiment is similar to the calculation method of the second similarity value in the embodiment shown in FIG. 1 , and the above operations and descriptions for the second similarity value are also applicable to this embodiment. The mapping similarity in , will not be repeated here.

In some optional implementations of this embodiment, the similarity between the image to be signed and the reference image cluster in the mapping relationship library is compared to obtain the mapping similarity value, including: extracting the image features to be signed of the image to be signed; extracting The benchmark image features of the benchmark image cluster; the similarity between the to-be-signed image features and the benchmark image features is compared to obtain the mapping similarity.

In this optional implementation manner, the features of the image to be signed may be obtained through a feature extractor, wherein the feature extractor may use a model based on deep learning, such as CNN, Transformer, and the like.

In this optional implementation manner, the feature of the image to be signed may be obtained by extracting the feature of any image in the reference image cluster by the feature extractor.

The method for obtaining the mapping similarity value provided in this optional implementation mode calculates the mapping similarity value through the features of the image to be signed and the feature of the reference image, which provides a reliable method for calculating the similarity between the image to be signed and the reference image. Way.

In some optional implementations of this embodiment, the above-mentioned extraction of the reference image features of the reference image cluster includes: acquiring the image features of each reference image in the reference image cluster; averaging the image features of all reference images to obtain the reference image features .

In this optional implementation, after the image features of each reference image in the reference image cluster are acquired, the image features of all reference images are averaged, so that the obtained reference image features have the features of all reference images, ensuring that the reference images characteristic obtained reliability.

In some optional implementations of this embodiment, the similarity between the image to be signed and the reference image cluster in the mapping relationship library is compared to obtain the mapping similarity value, including: extracting the image features to be signed of the image to be signed; obtaining The image features of each reference image in the reference image cluster; the similarity between the image features to be signed and the image features of each reference image is calculated to obtain a similarity sequence; based on the similarity sequence, a mapping similarity value is obtained.

In this optional implementation, the similarity sequence includes a similarity value between the image feature to be signed and the image feature of the reference image, the similarity value is at least one, and the average similarity score of TOPK in the similarity sequence can be taken as the mapping similarity where, when K=1, the maximum value in the similarity sequence is taken as the mapping similarity value; when K=the number of reference images in the reference image cluster, the average of all the similarities in the similarity sequence can be obtained to obtain Map similarity values.

The method for obtaining the reference similarity value provided in this optional implementation mode calculates the reference similarity value based on the similarity sequence, which provides another reliable method for calculating the similarity between the image to be signed and the reference image.

Optionally, the above-mentioned comparison of the similarity between the image to be signed and the reference image cluster in the mapping relationship library to obtain the mapping similarity value includes: extracting the image feature to be signed of the image to be signed; Image features; calculate the similarity between the image features to be signed and the image features of each reference image, and obtain a similarity sequence; based on the similarity sequence, obtain the reference image features; compare the similarity between the features of the image to be signed and the reference image features, and get the mapping similarity degree value.

Optionally, obtaining the reference image feature based on the similarity sequence includes: selecting the image feature of the reference image with the greatest similarity in the similarity sequence as the reference image feature. Optionally, obtaining the reference image features based on the similarity sequence includes: sorting each similarity in the similarity sequence in ascending order, and calculating the average value of the image features of the reference image set after selecting to obtain the reference image features.

Step 303 , based on the mapping similarity value, determine the reference corpus corresponding to the image to be signed.

In this embodiment, after the mapping similarity value of each benchmark image cluster is obtained, the benchmark image cluster corresponding to the maximum value among all the mapped similarity values can be determined, and further, the benchmark corpus corresponding to the benchmark image cluster is used as the to-be-signed The benchmark corpus corresponding to the image.

Step 304, adding the corresponding relationship between the image to be signed and the reference corpus into the mapping relationship database.

In this embodiment, the corresponding relationship between the image to be signed and the reference corpus is added to the mapping relationship database, which can enrich the reference images in the mapping relationship database. Further, the correspondence between the image to be signed and the reference image cluster corresponding to the reference corpus can also be added to the mapping relationship library.

In the search label extension method provided in this embodiment, an image to be signed is obtained, and when the image to be signed is not included in the preset mapping relation library, the similarity between the image to be signed and the reference image cluster in the mapping relation library is compared to obtain a mapping Similarity value; based on the mapping similarity value, determine the reference corpus corresponding to the signed image, and add the corresponding relationship between the image to be signed and the reference corpus into the mapping relationship library, which provides a reliable means for expanding the mapping relationship library and ensures that The reliability of tag generation, easy to expand and search tags, shorten the development cycle and reduce the iteration cost.

In another embodiment of the present disclosure, the above-mentioned search tag extension method further includes: acquiring a search corpus, and in response to determining that the search corpus is not included in the mapping relationship database, comparing the similarity between the search corpus and the reference corpus in the mapping relationship database, The first similarity value is obtained, and the mapping relationship library is used to represent the corresponding relationship between the reference corpus and the reference image cluster, and the reference image cluster includes at least one reference image; the similarity between the historical image and the reference image cluster in the mapping relationship library is compared. , obtain a second similarity value; based on the first similarity and the second similarity, determine the historical images related to the search corpus.

Further referring to FIG. 4 , as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a search apparatus, which corresponds to the method embodiment shown in FIG. 1 .

As shown in FIG. 4 , the search apparatus 400 provided in this embodiment includes: a corpus acquisition unit 401 , a first comparison unit 402 , a second comparison unit 403 , and an image determination unit 404 . The above-mentioned corpus acquisition unit 401 may be configured to acquire search corpus. The above-mentioned first comparison unit 402 can be configured to, in response to determining that the preset mapping relationship library does not include the search corpus, compare the search corpus with the reference corpus in the mapping relationship library for similarity to obtain a first similarity value, mapping The relation library is used to represent the correspondence between the reference corpus and the reference image cluster, and the reference image cluster includes at least one reference image. The above-mentioned second comparison unit 403 may be configured to compare the similarity between the historical image and the reference image cluster in the mapping relationship database, and obtain a second similarity value. The above-mentioned image determination unit 404 may be configured to determine historical images related to the search corpus based on the first similarity and the second similarity.

In this embodiment, in the search device 400: the specific processing of the corpus acquisition unit 401, the first comparison unit 402, the second comparison unit 403, and the image determination unit 404 and the technical effects brought by them can be implemented with reference to FIG. 1 respectively. The related descriptions of step 101 , step 102 , step 103 , and step 104 in the example will not be repeated here.

In some optional implementations of this embodiment, the foregoing apparatus 400 further includes: a relationship establishing unit (not shown in the figure). Wherein, the above-mentioned relationship establishing unit may be configured to establish a corresponding relationship between the reference corpus, the search corpus, and the historical images related to the search corpus.

In some optional implementations of this embodiment, the above-mentioned second comparison unit 403 includes: a first extraction module (not shown in the figure), a second extraction module (not shown in the figure), a first comparison module ( not shown in the figure). The above-mentioned first extraction module may be configured to extract the first image feature of the historical image. The above-mentioned second extraction module may be configured to extract the second image features of the reference image cluster. The above-mentioned first comparison module may be configured to compare the similarity between the first image feature and the second image feature to obtain a second similarity value.

In some optional implementations of this embodiment, the above-mentioned second extraction module includes: an acquisition sub-module (not shown in the figure) and an average sub-module (not shown in the figure). Wherein, the above-mentioned obtaining sub-module may be configured to obtain the image features of each reference image in the reference image cluster. The above-mentioned averaging sub-module may be configured to average the image features of all the reference images to obtain the second image feature.

In some optional implementations of this embodiment, the above-mentioned second comparison unit 403 includes: a feature extraction module (not shown in the figure), a feature acquisition module (not shown in the figure), and a first calculation module (not shown in the figure) not shown), a feature obtaining module (not shown in the figure), a second comparison module (not shown in the figure). Wherein, the above-mentioned feature extraction module may be configured to extract the first image feature of the historical image. The above feature acquisition module may be configured to acquire image features of each reference image in the reference image cluster. The above-mentioned first calculation module may be configured to calculate the similarity between the first image feature and the image features of each reference image to obtain a similarity sequence. The above feature obtaining module may be configured to obtain the second image feature based on the similarity sequence. The above-mentioned second comparison module may be configured to compare the similarity between the first image feature and the second image feature to obtain a second similarity value.

In some optional implementations of this embodiment, the image determination unit 404 includes: a similarity determination module (not shown in the figure), a multiplication module (not shown in the figure), and an arrangement module (not shown in the figure) ), an image determination module (not shown in the figure). Wherein, the similarity determination module is configured to determine the first similarity and the second similarity corresponding to each corresponding relationship in the mapping relationship library; the multiplication module is configured to, for each corresponding relationship, determine the first similarity of the corresponding relationship The degree of similarity and the second degree of similarity are multiplied to obtain the degree of similarity of the corresponding relationship; the arrangement module is configured to arrange the similarity of all the corresponding relationships in ascending or descending order; the image determining module is configured to be based on the ascending or descending order. As a result, historical images relevant to the search corpus are determined.

In the search apparatus provided by the embodiments of the present disclosure, firstly, the corpus acquisition unit 401 acquires the search corpus; secondly, the first comparison unit 402, in response to determining that the preset mapping relation library does not include the search corpus, compares the search corpus with the mapping relation library The reference corpus is compared to obtain the first similarity value, and the mapping relationship library is used to represent the correspondence between the reference corpus and the reference image cluster, and the reference image cluster includes at least one reference image; again, the second comparison unit 403 will The similarity between the historical image and the reference image cluster in the mapping relationship library is compared to obtain a second similarity value; finally, the image determination unit 404 determines the historical image related to the search corpus based on the first similarity and the second similarity. Therefore, through the low-cost and fast customized mapping relationship between user search corpus and user pictures, all user search scenarios can be quickly and automatically covered, and the recall rate of user search can be improved while ensuring the search accuracy.

Further referring to FIG. 5 , as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a search tag extension apparatus, which corresponds to the method embodiment shown in FIG. 3 .

As shown in FIG. 5 , the search tag extension apparatus 500 provided in this embodiment includes: an image acquisition unit 501 , a comparison unit 502 , a corpus determination unit 503 , and a mapping unit 504 . The above-mentioned image acquisition unit 501 may be configured to acquire an image to be signed. The above-mentioned comparison unit 502 may be configured to, in response to determining that the preset mapping relationship library does not include the image to be signed, compare the similarity between the image to be signed and the reference image cluster in the mapping relationship library to obtain a mapping similarity value, and map the image to be signed. The relation library is used to represent the correspondence between the reference corpus and the reference image cluster, and the reference image cluster includes at least one reference image. The above-mentioned corpus determination unit 503 may be configured to determine the reference corpus corresponding to the image to be signed based on the mapping similarity value. The above-mentioned mapping unit may be configured to add the corresponding relationship between the image to be signed and the reference corpus into the mapping relationship database.

In this embodiment, in the search tag extension device 500 : the specific processing of the image acquisition unit 501 , the comparison unit 502 , the corpus determination unit 503 , and the mapping unit 504 and the technical effects brought about by the image acquisition unit 501 , and the technical effects brought about by them can be referred to in the corresponding embodiment of FIG. 3 respectively. The related descriptions of step 301 , step 302 , step 303 and step 304 of the above will not be repeated here.

In some optional implementations of this embodiment, the comparison unit 502 includes: a label extraction module (not shown in the figure), a reference extraction module (not shown in the figure), and a mapping comparison module (not shown in the figure) . Wherein, the above-mentioned label extraction module may be configured to extract the image features to be signed of the to-be-signed image. The above-mentioned reference extraction module may be configured to extract reference image features of reference image clusters. The above-mentioned mapping comparison module may be configured to compare the similarity between the features of the to-be-signed image and the reference image to obtain the mapping similarity.

In some optional implementations of this embodiment, the above-mentioned reference extraction module includes: an acquisition sub-module (not shown in the figure) and an average sub-module (not shown in the figure). Wherein, the above-mentioned obtaining sub-module may be configured to obtain the image features of each reference image in the reference image cluster. The above-mentioned averaging sub-module may be configured to average the image features of all the reference images to obtain the reference image features.

In some optional implementations of this embodiment, the above comparison unit 502 includes: an extraction module to be signed (not shown in the figure), an acquisition module (not shown in the figure), and a calculation module to be signed (not shown in the figure) ), mapping to obtain a module (not shown in the figure). The above-mentioned to-be-signed extraction module may be configured to extract the to-be-signed image features of the to-be-signed image. The above obtaining module may be configured to obtain image features of each reference image in the reference image cluster. The above-mentioned calculation module to be signed may be configured to calculate the similarity between the image features to be signed and the image features of each reference image to obtain a similarity sequence. The above-mentioned mapping obtaining module may be configured to obtain the reference similarity value based on the similarity sequence.

In the search tag extension device provided by the embodiment of the present disclosure, the image acquisition unit 501 acquires the image to be signed, and the comparison unit 502 compares the image to be signed with the benchmark in the mapping relation library when the preset mapping relation library does not include the to-be-signed image. The similarity of the image clusters is compared to obtain the mapping similarity value; the corpus determination unit 503 determines the reference corpus corresponding to the signed image based on the mapping similarity value, and the mapping unit 504 adds the correspondence between the image to be signed and the reference corpus to the mapping relationship In the library, a reliable means is provided for extending the mapping relationship library, which ensures the reliability of label generation.

In the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved are all in compliance with relevant laws and regulations, and do not violate public order and good customs.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

FIG. 6 shows a schematic block diagram of an example electronic device 600 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in FIG. 6 , the device 600 includes a computing unit 601 that can be executed according to a computer program stored in a read only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603 Various appropriate actions and handling. In the RAM 603, various programs and data necessary for the operation of the device 600 can also be stored. The computing unit 601 , the ROM 602 , and the RAM 603 are connected to each other through a bus 604 . An input/output (I/O) interface 605 is also connected to bus 604 .

Various components in the device 600 are connected to the I/O interface 605, including: an input unit 606, such as a keyboard, mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, an optical disk, etc. ; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 609 allows the device 600 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks.

Computing unit 601 may be various general-purpose and/or special-purpose processing components with processing and computing capabilities. Some examples of computing units 601 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various specialized artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as a search method or a search tag expansion method. For example, in some embodiments, the search method or the search tag extension method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 608 . In some embodiments, part or all of the computer program may be loaded and/or installed on device 600 via ROM 602 and/or communication unit 609 . When the computer program is loaded into RAM 603 and executed by computing unit 601, one or more steps of the above-described search method or search tag expansion method may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the search method or the search tag expansion method by any other suitable means (eg, by means of firmware).

Various implementations of the systems and techniques described herein above may be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips system (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpretable on a programmable system including at least one programmable processor that The processor, which may be a special purpose or general-purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device an output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable search device or search tag extension device, such that the program codes, when executed by the processor or controller, cause all of the flowcharts and/or block diagrams The specified function/operation is implemented. The program code may execute entirely on the machine, partly on the machine, partly on the machine and partly on a remote machine as a stand-alone software package or entirely on the remote machine or server.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with the instruction execution system, apparatus or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optics, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (eg, visual feedback, auditory feedback, or tactile feedback); and can be in any form (including acoustic input, voice input, or tactile input) to receive input from the user.

The systems and techniques described herein may be implemented on a computing system that includes back-end components (eg, as a data server), or a computing system that includes middleware components (eg, an application server), or a computing system that includes front-end components (eg, a user's computer having a graphical user interface or web browser through which a user may interact with implementations of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

A computer system can include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a distributed system server, or a server combined with blockchain.

It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, no limitation is imposed herein.

The above-mentioned specific embodiments do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure should be included within the protection scope of the present disclosure.

Claims (23)

1. A search method, the method comprising: Get search corpus; In response to determining that the search corpus is not included in the preset mapping relationship library, the similarity is compared between the search corpus and the reference corpus in the mapping relationship library to obtain a first similarity value, and the mapping relationship library uses For characterizing the correspondence between the reference corpus and the reference image cluster, the reference image cluster includes at least one reference image; Compare the similarity between the historical image and the reference image cluster in the mapping relationship library to obtain a second similarity value; Based on the first similarity and the second similarity, historical images related to the search corpus are determined. 2. The method of claim 1, further comprising: A correspondence between the reference corpus, the search corpus, and historical images related to the search corpus is established. 3. The method according to claim 1 or 2, wherein the comparison of the similarity between the historical image and the reference image cluster in the mapping relationship library to obtain the second similarity value comprises: extracting the first image feature of the historical image; extracting second image features of the reference image cluster; The similarity is compared between the first image feature and the second image feature to obtain a second similarity value. 4. The method according to claim 3, wherein the extracting the second image feature of the reference image cluster comprises: Obtain the image features of each benchmark image in the benchmark image cluster; The image features of all the reference images are averaged to obtain the second image feature. 5. The method according to claim 1 or 2, wherein the comparison of the similarity between the historical image and the reference image cluster in the mapping relationship library to obtain the second similarity value comprises: extracting the first image feature of the historical image; Obtain the image features of each benchmark image in the benchmark image cluster; Calculate the similarity between the first image feature and the image features of each reference image to obtain a similarity sequence; Based on the similarity sequence, obtain a second image feature; The similarity is compared between the first image feature and the second image feature to obtain a second similarity value. 6. The method according to claim 1, wherein the determining of historical images related to the search corpus based on the first similarity and the second similarity comprises: determining the first similarity and the second similarity corresponding to each corresponding relationship in the mapping relationship library; For each corresponding relationship, multiply the first similarity and the second similarity of the corresponding relationship to obtain the similarity of the corresponding relationship; Sort the similarity of all correspondences in ascending or descending order; Based on the ranking results in the ascending or descending order, historical images related to the search corpus are determined. 7. A search tag extension method, the method comprising: Get the image to be signed; In response to determining that the image to be signed is not included in the preset mapping relationship library, the similarity between the image to be signed and the reference image cluster in the mapping relationship library is compared to obtain a mapping similarity value, and the mapping relationship The library is used to characterize the correspondence between the reference corpus and the reference image cluster, and the reference image cluster includes at least one reference image; Based on the mapping similarity value, determine the reference corpus corresponding to the image to be signed; The corresponding relationship between the to-be-signed image and the reference corpus is added to the mapping relationship library. 8. The method according to claim 7, wherein the similarity comparison between the to-be-signed image and the reference image cluster in the mapping relationship library to obtain a mapping similarity value comprises: extracting to-be-signed image features of the to-be-signed image; extracting reference image features of the reference image cluster; The similarity is compared between the feature of the image to be signed and the feature of the reference image to obtain the similarity of the map. 9. The method according to claim 8, wherein the extracting the reference image features of the reference image cluster comprises: Obtain the image features of each benchmark image in the benchmark image cluster; The image features of all the reference images are averaged to obtain the reference image features. 10. The method according to claim 7, wherein the similarity comparison between the to-be-signed image and the reference image cluster in the mapping relationship library to obtain a mapping similarity value comprises: extracting to-be-signed image features of the to-be-signed image; Obtain the image features of each benchmark image in the benchmark image cluster; Calculate the similarity between the image features to be signed and the image features of each reference image to obtain a similarity sequence; Based on the similarity sequence, a mapping similarity value is obtained. 11. A search device, the device comprising: a corpus acquisition unit, configured to acquire search corpus; A first comparison unit, configured to, in response to determining that the search corpus is not included in the preset mapping relationship database, compare the similarity between the search corpus and the reference corpus in the mapping relationship database to obtain a first similarity degree value, the mapping relationship library is used to represent the correspondence between the reference corpus and the reference image cluster, and the reference image cluster includes at least one reference image; The second comparison unit is configured to compare the similarity between the historical image and the reference image cluster in the mapping relationship library to obtain a second similarity value; An image determination unit configured to determine historical images related to the search corpus based on the first similarity and the second similarity. 12. The apparatus of claim 11, further comprising: A relationship establishing unit is configured to establish a correspondence between the reference corpus, the search corpus, and historical images related to the search corpus. 13. The apparatus of claim 11 or 12, wherein the second comparison unit comprises: a first extraction module configured to extract a first image feature of the historical image; a second extraction module configured to extract second image features of the reference image cluster; The first comparison module is configured to compare the similarity between the first image feature and the second image feature to obtain a second similarity value. 14. The apparatus of claim 13, wherein the second extraction module comprises: an acquisition submodule, configured to acquire the image features of each benchmark image in the benchmark image cluster; The mean sub-module is configured to average the image features of all the reference images to obtain the second image feature. 15. The apparatus of claim 11 or 12, wherein the second comparison unit comprises: a feature extraction module configured to extract a first image feature of the historical image; a feature acquisition module, configured to acquire image features of each reference image in the reference image cluster; a first calculation module, configured to calculate the similarity between the first image feature and the image features of each reference image to obtain a similarity sequence; a feature obtaining module configured to obtain a second image feature based on the similarity sequence; The second comparison module is configured to compare the similarity between the first image feature and the second image feature to obtain a second similarity value. 16. The apparatus of claim 11, wherein the image determination unit comprises: a similarity determination module, configured to determine the first similarity and the second similarity corresponding to each corresponding relationship in the mapping relationship library; a multiplying module, configured to multiply the first similarity and the second similarity of the corresponding relationship for each corresponding relationship to obtain the similarity of the corresponding relationship; an arrangement module, configured to sort the similarity of all correspondences in ascending or descending order; The image determination module is configured to determine historical images related to the search corpus based on the ascending or descending ranking results. 17. An apparatus for expanding search tags, the apparatus comprising: an image acquisition unit, configured to acquire an image to be signed; The comparison unit is configured to, in response to determining that the preset mapping relationship library does not include the image to be signed, compare the similarity between the image to be signed and the reference image cluster in the mapping relationship library to obtain the mapping similarity value, the mapping relationship library is used to represent the correspondence between the reference corpus and the reference image cluster, and the reference image cluster includes at least one reference image; a corpus determination unit, configured to determine a reference corpus corresponding to the image to be signed based on the mapping similarity value; The mapping unit is configured to add the corresponding relationship between the image to be signed and the reference corpus into the mapping relationship library. 18. The apparatus of claim 17, wherein the comparison unit comprises: The label extraction module is configured to extract the to-be-signed image features of the to-be-signed image; a benchmark extraction module configured to extract benchmark image features of the cluster of benchmark images; The mapping comparison module is configured to compare the similarity between the features of the to-be-signed image and the reference image to obtain the mapping similarity. 19. The apparatus of claim 18, wherein the benchmark extraction module comprises: an acquisition submodule, configured to acquire the image features of each benchmark image in the benchmark image cluster; The mean value sub-module is configured to average the image features of all the reference images to obtain the reference image features. 20. The apparatus of claim 17, wherein the comparison unit comprises: a to-be-signed extraction module, configured to extract the to-be-signed image features of the to-be-signed image; an acquisition module, configured to acquire image features of each benchmark image in the benchmark image cluster; A to-be-signed calculation module, configured to calculate the similarity between the to-be-signed image feature and the image features of each reference image to obtain a similarity sequence; The mapping obtaining module is configured to obtain a mapping similarity value based on the similarity sequence. 21. An electronic device, characterized in that, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the execution of any of claims 1-10 Methods. 22. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method of any one of claims 1-10. 23. A computer program product comprising a computer program which, when executed by a processor, implements the method of any of claims 1-10.

Images