CN111382295B - A method and device for sorting image search results - Google Patents

Abstract

The embodiment of the application discloses a method and a device for ordering image search results, wherein when the image search results of keywords are acquired, a first image set which belongs to a first image category is identified from images of the image search results according to an image classification model, and aesthetic scores corresponding to the images in the first image set are determined through an aesthetic evaluation model corresponding to the first image category. And when the images in the image search results are ranked, determining the ranking positions of the images in the first image set in the image search results by combining the aesthetic scores corresponding to the images in the first image set. Because the aesthetic score of one category can show the aesthetic feeling of the image to the user under the category, the image with better aesthetic feeling can be preferentially displayed to the user because the user has relatively high possibility of selecting and viewing the image with higher aesthetic score, and the image with higher aesthetic score has higher possibility of meeting the image searching requirement, so that the image searching experience is improved.

Description

A method and device for sorting image search results

Technical Field

The present application relates to the field of image search, and in particular to a method and device for sorting image search results.

Background technique

In existing image searches, for image search results corresponding to keywords, a relevance evaluation method is generally used to sort images in the image search results.

However, image search is different from other types of search. Images with high relevance to keywords are not necessarily images that meet user needs. Therefore, according to the existing relevance evaluation method, the ranking results obtained are difficult to meet the user's image search needs, and the user search experience is not good.

Therefore, improving the ranking effect of image search results is an urgent problem that needs to be solved.

Summary of the invention

In order to solve the above technical problems, the present application provides a method and device for sorting image search results, so that images with higher aesthetic scores can be displayed to users first, thereby improving the user's image search experience.

The embodiments of the present application disclose the following technical solutions:

In a first aspect, an embodiment of the present application provides a method for sorting image search results, the method comprising:

Obtaining image search results corresponding to the keyword, wherein the image search results include multiple images;

identifying a first image set from the image search results according to an image classification model; the images in the first image set belong to a first image category;

Determining an aesthetic score corresponding to each image in the first image set by using an aesthetic evaluation model corresponding to the first image category;

When ranking the images in the picture search results, the ranking positions of the images in the first image set in the image search results are determined in combination with the aesthetic scores corresponding to the images in the first image set.

Optionally, identifying a first image set from the image search results according to an image classification model includes:

identifying the first image set and the second image set from the image search results according to the image classification model; the images in the second image set belong to a second image category different from the first image category;

The method further comprises:

Determining an aesthetic score corresponding to each image in the second image set by using an aesthetic evaluation model corresponding to the second image category;

When ranking the images in the picture search results, the ranking positions of the images in the second image set in the image search results are determined in combination with the aesthetic scores corresponding to the images in the second image set.

Optionally, the first image category is correlated with the type to which the keyword belongs.

Optionally, if the first image category is a person category, the method further includes:

Determining a face analysis score corresponding to each image in the first image set according to a face analysis model;

determining, according to the face analysis scores corresponding to the images in the first image set and the aesthetic scores corresponding to the images in the first image set, the quality scores corresponding to the images in the first image set;

The determining the ranking positions of the images in the first image set in the image search results by combining the aesthetic scores corresponding to the images in the first image set comprises:

The ranking positions of the images in the first image set in the image search results are determined in combination with the quality scores corresponding to the images in the first image set.

Optionally, the image search result is a plurality of images obtained by searching the keyword and whose relevance to the keyword meets a preset condition.

Optionally, if the keyword is input by the target user, the method further includes:

Determining the image click feature of the target user according to the image search behavior of the target user;

After determining the ranking result of the images in the image search results, adjusting the ranking result according to the image click feature, so as to advance the ranking position of the images in the image search results that meet the image click feature;

The image search results are displayed according to the adjusted ranking results.

In a second aspect, an embodiment of the present application provides a device for sorting image search results, the device comprising an acquisition unit, an identification unit, a determination unit, and a sorting unit:

The acquisition unit is used to acquire image search results corresponding to the keyword, and the image search results include multiple images;

The recognition unit is used to identify a first image set from the image search results according to an image classification model; the images in the first image set belong to a first image category;

The determining unit is configured to determine the aesthetic scores corresponding to the images in the first image set by using the aesthetic evaluation model corresponding to the first image category;

The sorting unit is used to determine the sorting positions of the images in the first image set in the image search results in combination with the aesthetic scores corresponding to the images in the first image set when sorting the images in the image search results.

Optionally, the recognition unit is further used to identify the first image set and the second image set from the image search results according to the image classification model; the images in the second image set belong to a second image category different from the first image category;

The determining unit is further configured to determine the aesthetic scores corresponding to the images in the second image set by using the aesthetic evaluation model corresponding to the second image category;

The sorting unit is further configured to determine, when sorting the images in the picture search results, the sorting positions of the images in the second image set in the image search results in combination with the aesthetic scores corresponding to the images in the second image set.

Optionally, the first image category is correlated with the type to which the keyword belongs.

Optionally, if the first image category is a person category, the device further includes a face analysis score determination unit and a quality score determination unit:

The face analysis score determination unit is used to determine the face analysis score corresponding to each image in the first image set according to the face analysis model;

The quality score determination unit is used to determine the quality score corresponding to each image in the first image set according to the face analysis score corresponding to each image in the first image set and the aesthetic score corresponding to each image in the first image set;

The ranking unit is further configured to determine the ranking positions of the images in the first image set in the image search results in combination with the quality scores corresponding to the images in the first image set.

Optionally, the image search result is a plurality of images obtained by searching the keyword and whose relevance to the keyword meets a preset condition.

Optionally, if the keyword is input by the target user, the device further includes an image click feature determination unit, an adjustment unit, and a display unit:

The image click feature determination unit is used to determine the image click feature of the target user according to the image search behavior of the target user;

The adjusting unit is used to adjust the ranking result according to the image click feature after determining the ranking result of the images in the image search result, so as to advance the ranking position of the images that meet the image click feature in the image search result;

The display unit is used to display the image search results according to the adjusted ranking results.

In a third aspect, an embodiment of the present application provides a device for ranking image search results, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the following operations:

Obtaining image search results corresponding to the keyword, wherein the image search results include multiple images;

identifying a first image set from the image search results according to an image classification model; the images in the first image set belong to a first image category;

Determining an aesthetic score corresponding to each image in the first image set by using an aesthetic evaluation model corresponding to the first image category;

When ranking the images in the picture search results, the ranking positions of the images in the first image set in the image search results are determined in combination with the aesthetic scores corresponding to the images in the first image set.

In a fourth aspect, an embodiment of the present application provides a machine-readable medium having instructions stored thereon, which, when executed by one or more processors, enables the device to perform a method for sorting image search results as described in one or more of the first aspect.

It can be seen from the above technical solution that when obtaining image search results corresponding to a keyword and including multiple images, a first image set that all belongs to a first image category is identified from the images in the image search results according to the image classification model, and the aesthetic scores corresponding to each image in the first image set are determined by the aesthetic evaluation model corresponding to the first image category. When sorting the images in the picture search results, the sorting positions of the images in the first image set in the image search results are determined in combination with the aesthetic scores corresponding to each image in the first image set. For images belonging to an image category, the aesthetic score of this category can reflect the aesthetic feeling that the image brings to the user under this category, and the possibility that the user chooses to view images with better aesthetic feeling is relatively high. Therefore, the sorting position of images with higher aesthetic scores can be advanced, and the sorting position of images with lower aesthetic scores can be postponed, so that images with higher aesthetic scores can be displayed to users first. Such images are more likely to meet the image search needs of users, thereby improving the image search experience of users.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

FIG1 is a flow chart of a method for sorting image search results provided by an embodiment of the present application;

FIG2 is a device structure diagram of a device for sorting image search results provided by an embodiment of the present application;

FIG3 is a structural diagram of a device for sorting image search results provided in an embodiment of the present application;

FIG4 is a structural diagram of a server provided in an embodiment of the present application.

Detailed ways

The embodiments of the present application are described below in conjunction with the accompanying drawings.

In existing image searches, for image search results corresponding to keywords, images in the image search results are generally ranked using a relevance evaluation method. Images with high relevance to keywords are ranked higher and are displayed to users first.

However, images are different from search items such as files and pages. The aesthetic perception of images directly affects whether users are willing to click to view them. For example, if an image in the image search results is highly correlated with keywords, but the aesthetic perception is not satisfactory, users are unlikely to click to view this image. It can be seen that images with high correlation with keywords are not necessarily images that meet user needs. Improving the ranking effect of image search results is an issue that needs to be urgently addressed.

To this end, an embodiment of the present application provides a method for sorting image search results, which determines the aesthetic score of an image of a certain image category in the image search results through an image classification model and an aesthetic evaluation model, and adjusts the sorting position of such images in the image search results in combination with the aesthetic score. The sorting method can be applied to a processing device, which can be a terminal, a computer, a server, etc. Among them, the image classification model and the aesthetic evaluation model can be configured in one processing device, or they can be configured in different processing devices respectively.

For images belonging to a certain image category, the aesthetic score of this category can reflect the aesthetic feeling that the image brings to the user under this category, and the user is relatively more likely to choose to view images with better aesthetic feeling. Therefore, the sorting position of images with higher aesthetic scores can be advanced, and the sorting position of images with lower aesthetic scores can be pushed back, so that images with higher aesthetic scores can be displayed to users first. Such images are more likely to meet the user's image search needs, thereby improving the user's image search experience.

Next, a method for sorting image search results provided by an embodiment of the present application is described in conjunction with FIG. 1 . The method includes:

S101: Obtain image search results corresponding to a keyword.

The image search result is an image related to the keyword obtained by the search engine according to the keyword search, and the image search result includes multiple images.

In the embodiment of the present application, the aesthetic score of an image of a certain image category in the image search results can be determined by the image classification model and the aesthetic evaluation model, and the ranking position of such images in the image search results can be adjusted in combination with the aesthetic score. In order to improve the efficiency of adjusting the ranking position in combination with the aesthetic score, the images for which the aesthetic score needs to be determined can be selected in a targeted manner.

In a possible implementation manner, the image search result is a plurality of images obtained by searching the keyword and whose relevance to the keyword meets a preset condition.

That is, in this possible implementation, the image search result may be a portion of all images obtained by searching based on the keyword, such as the top N images with the highest relevance.

The correlation between these images and the keywords is relatively high, which results in the ranking position originally obtained based on the correlation being relatively high. By determining the aesthetic score of these images and combining the adjusted ranking position with the aesthetic score, the possibility of a high ranking position is greater, making it easier to display them to users first, further increasing the possibility of users clicking to view these images.

S102: Identify a first image set from the image search results according to an image classification model.

The image classification model may be pre-trained and at least be able to realize the function of identifying images of a certain image category from images.

In an embodiment of the present application, a first image set belonging to a first image category is identified from image search results according to an image classification model, and the first image set includes at least one image in the image search results.

The embodiment of the present application does not limit the division method or division granularity of image categories. For example, image categories can be divided according to different objects displayed in the image, such as people, objects, materials, scenery, etc. The first image category can be any one of the above categories. In other words, the embodiment of the present application can identify any image category in the image search results, and adjust the sorting position by determining the aesthetic score of this type of image.

In a possible implementation, the first image category is correlated with the type to which the keyword belongs. That is, when performing image recognition on the image search results corresponding to the keyword, the image classification model can be used to specifically identify images whose image types are correlated with the type of the keyword from the image search results.

The relevance mentioned here can be understood as the relationship between the type to which the keyword belongs and the image type, such as similarity or sameness. For example, the image type includes four types: person type, object type, material type, and landscape type. If the keyword is "Andy Lau" and the type to which the keyword belongs is determined to be person, then the person type in the image type is relevant to the type "person" to which the keyword belongs, and the other three image types are not relevant to the type "person" to which the keyword belongs.

Therefore, in the present implementation, the images included in the identified first image set have a correlation between the image type and the keyword type, so that the images in the first image set are more consistent with the image search needs of users through keywords. Therefore, subsequent aesthetic scoring and adjustment of the sorting position of such images are more likely to meet the image search needs of users.

It should be noted that, since the embodiment of the present application needs to determine the corresponding aesthetic evaluation model according to the image category for aesthetic scoring, different image categories should have different aesthetic evaluation methods. The different aesthetic evaluation methods here may refer to completely different or partially different aesthetic evaluation methods.

For example, in human images, images with whiter, more three-dimensional facial structures, better body proportions, etc. can get higher aesthetic scores, while in landscape images, images with higher color saturation and better composition ratios can get higher aesthetic scores. Therefore, when classifying images, it is necessary to take the above into account and determine that different image categories obtained by classification have different aesthetic evaluation methods.

S103: Determine the aesthetic score corresponding to each image in the first image set by using the aesthetic evaluation model corresponding to the first image category.

The aesthetic evaluation model in this step is specifically used to score the aesthetic direction of images of the first image category. The aesthetic evaluation model can be pre-trained, for example, pre-trained by a deep neural network.

The aesthetic evaluation model can score the aesthetic direction based on the aesthetic characteristics corresponding to the first image category by analyzing features in the image related to the aesthetic characteristics. The higher the aesthetic score of an image, the better the aesthetic feeling brought by the image to the user.

By using the aesthetic evaluation model corresponding to the first image category, corresponding aesthetic scores can be determined for the images in the first image set, wherein one image has one corresponding aesthetic score.

Optionally, the aesthetic score can be a score from 0-1.

S104: When sorting the images in the picture search results, determine the sorting positions of the images in the first image set in the image search results in combination with the aesthetic scores corresponding to the images in the first image set.

In an embodiment of the present application, for the images in the first image set, the sorting position of the images in the picture search results can be determined solely based on the aesthetic score, or the sorting position can be determined based on the correlation between the images and keywords in the image search results, and then the original sorting position of the images in the first image set can be adjusted according to the aesthetic score, for example, the sorting position of images with higher aesthetic scores can be advanced, and the sorting position of images with lower aesthetic scores can be moved back.

Regardless of which of the above methods is used in combination with the aesthetic score to determine the ranking position of the images in the first image set in the image search results, at least some images with higher aesthetic scores can be ranked earlier, and the forward display position allows users to view such images more quickly.

It can be seen that when obtaining image search results corresponding to keywords and including multiple images, a first image set that all belongs to the first image category is identified from the images in the image search results according to the image classification model, and the aesthetic scores corresponding to the images in the first image set are determined by the aesthetic evaluation model corresponding to the first image category. When sorting the images in the picture search results, the sorting positions of the images in the first image set in the image search results are determined in combination with the aesthetic scores corresponding to the images in the first image set. For images belonging to an image category, the aesthetic score of this category can reflect the aesthetic feeling that the image brings to the user under this category, and the possibility that the user chooses to view images with better aesthetic feeling is relatively high. Therefore, the sorting position of images with higher aesthetic scores can be advanced, and the sorting position of images with lower aesthetic scores can be postponed, so that images with higher aesthetic scores can be displayed to users first. Such images are more likely to meet the image search needs of users, thereby improving the image search experience of users.

In S102, the image classification model can identify images belonging to the first image category from the image search results. In addition, the image classification model can also identify images of other image categories, such as a second image category different from the first image category. In other words, the image classification model can identify images of multiple image categories from the image search results and form a corresponding image set.

Therefore, in a possible implementation, S102 may be:

The first image set and the second image set are identified from the image search results according to the image classification model.

The images in the second image set belong to a second image category different from the first image category. For example, if the first image category is people and the second image category is landscape, then all images in the first image set are people images and all images in the second image category are landscape images.

It should be noted that some images may have multiple image categories. For example, an image a that includes a person in the wild may have an image category of both a person and a landscape. Therefore, when multiple image sets of different image categories, such as a first image set and a second image set, are identified from the image search results through an image classification model, one or some images in the image search results may be in both the first image set and the second image set. For example, for image a in the above example, if the first image category is a person and the second image category is a landscape, image a may be in both the first image set and the second image set. In this case, image a may have an aesthetic score corresponding to the person category and an aesthetic score corresponding to the landscape category. When determining the ranking position of image a in the image search results in combination with the aesthetic score, the aesthetic score of image a corresponding to the person category and the aesthetic score of image a corresponding to the landscape category can be considered simultaneously to determine the ranking position of image a. For example, different weights are set to integrate multiple aesthetic scores to obtain a total aesthetic score, and the ranking position of image a is removed by the total aesthetic score; or, the ranking position of image a can be determined based on the higher aesthetic score of image a corresponding to the person category and the aesthetic score of image a corresponding to the landscape category; or, if a certain image type of image a, such as the person category, is correlated with the type of the keyword, the aesthetic score of the person category can be used to determine the ranking position of image a.

After determining the first image set and the second image set, it is necessary to score the images in the second image set in terms of aesthetic direction. Based on the embodiment corresponding to FIG1 , the method further includes:

S201: Determine an aesthetic score corresponding to each image in the second image set by using an aesthetic evaluation model corresponding to the second image category.

The aesthetic evaluation model in this step is specifically used to score the aesthetic direction of the images of the second image category. The aesthetic evaluation model can be pre-trained, for example, pre-trained by a deep neural network.

The aesthetic evaluation model can score the aesthetic direction based on the aesthetic characteristics corresponding to the second image category by analyzing features in the image related to the aesthetic characteristics. The higher the aesthetic score of an image, the better the aesthetic feeling brought by the image to the user.

By using the aesthetic evaluation model corresponding to the second image category, corresponding aesthetic scores can be determined for the images in the second image set, wherein one image has one corresponding aesthetic score.

Optionally, the aesthetic score can be a score from 0-1.

S202: When sorting the images in the picture search results, determine the sorting positions of the images in the second image set in the image search results in combination with the aesthetic scores corresponding to the images in the second image set.

In an embodiment of the present application, for the images in the second image set, the sorting position of the image in the picture search results can be determined solely based on the aesthetic score, or the sorting position can be determined based on the correlation between the image and the keyword in the image search results, and then the original sorting position of the image in the second image set can be adjusted according to the aesthetic score, for example, the sorting position of the image with a higher aesthetic score can be advanced, and the sorting position of the image with a lower aesthetic score can be moved back.

Regardless of which of the above methods is used in combination with the aesthetic score to determine the ranking position of the images in the second image set in the image search results, at least some images with higher aesthetic scores can be ranked earlier, and the forward display position allows users to view such images more quickly.

In summary, due to the large differences between images of different image types, users have different aesthetic feelings or attractiveness evaluation standards for images of different image types. For example, for material and character types, there are great differences in aesthetic evaluation standards in terms of saturation and image composition. Therefore, in the embodiments of the present application, when scoring images in terms of aesthetic direction, different aesthetic scoring models can be used for scoring images of different image categories, and different image categories can have different scoring standards and methods, which improves the accuracy of aesthetic feelings that can be reflected by aesthetic scoring, and the image ranking position determined in this way can improve the user's image search experience.

For images whose image category is people, the embodiment of the present application further provides a method for further optimizing the sorting position.

If the first image category is a person category, in a possible implementation, based on the embodiment corresponding to FIG1 , the method further includes:

S301: Determine the face analysis score corresponding to each image in the first image set according to the face analysis model.

The face analysis model is used to analyze faces in images, which may include face detection and recognition, and the number of faces. Through the above face analysis process, face analysis scores can be performed for the images in the first image set based on the analysis results. One image corresponds to one face analysis score.

It should be noted that the face analysis model can implement the above-mentioned face analysis function through one model or through multiple models. For example, the face analysis model can be divided into a face detection model and a face recognition model. Among them, the face detection model can detect the position and number of faces in the image, and the face recognition model can identify the person corresponding to the face in the image, etc.

For example, if the keyword is "actor A", the face analysis model can identify the face information carried by each image in the image search results corresponding to the keyword, such as whether the person in the image is actor A, how many people there are, etc.

The facial analysis score can reflect whether the face identified and detected in the image is related to the person corresponding to the keyword. For example, in the image corresponding to "Actor A" in the above example, if an image only includes one face and it is actor A, the facial analysis score is relatively high. If an image includes multiple faces and only one face is actor A, the facial analysis score is relatively low.

From the explanation of face analysis scoring in the previous paragraph, it can be made clear that the face analysis scoring reflects the degree of association between the face contained in the person image and the person corresponding to the keyword, or whether the image is more focused on displaying the person corresponding to the keyword.

The higher the degree of correlation between the face contained in an image and the task corresponding to the keyword, for example, when it only contains the face of the person corresponding to the keyword and does not contain the faces of other people, the higher the face analysis score of this image will be.

It can be seen that the face analysis score can also affect whether the user chooses to click on the image. The person corresponding to the keyword can reflect the user's search needs, so the user is more likely to click on the image with a higher degree of association with the person corresponding to the keyword from the image search results. In other words, the higher the face analysis score of an image, the more likely the user is to click on the image.

S302: Determine a quality score corresponding to each image in the first image set according to the face analysis score corresponding to each image in the first image set and the aesthetic score corresponding to each image in the first image set.

Since each image in the first image set already has its own corresponding aesthetic score and face analysis score, the aesthetic score can reflect the aesthetic feeling brought by the image to the user, and the face analysis score can reflect the degree of association between the face contained in the image and the person corresponding to the keyword. Both scores can be used to adjust the sorting position of the image. For the convenience of calculation, these two scores can be combined to obtain the quality score corresponding to the image. In the first image set, one image has a corresponding quality score.

When calculating the quality score by combining the aesthetic score and the facial analysis score, the aesthetic score and the facial analysis score may have different or the same weight coefficients. The embodiment of the present application does not limit how to set the weight coefficient. For example, the weight coefficient may be pre-set or adjusted according to different application scenarios.

After the quality score is determined, in this embodiment, a possible implementation of S104 may be as shown in S303:

S303: Determine the ranking positions of the images in the first image set in the image search results in combination with the quality scores corresponding to the images in the first image set.

In an embodiment of the present application, for images in the first image set, the sorting position of the image in the picture search results can be determined only based on the quality score, or the sorting position can be determined based on the correlation between the image and the keyword in the image search results, and then the original sorting position of the image in the first image set can be adjusted according to the quality score, for example, the sorting position of images with higher quality scores can be advanced, and the sorting position of images with lower quality scores can be moved back.

Regardless of which of the above methods is used in combination with the quality score to determine the ranking position of the images in the first image set in the image search results, at least some images with higher quality scores can be ranked earlier. The forward display position allows users to view such images more quickly, thereby improving the display effect of the image search results.

Since different users have different image search behaviors, for the same keyword, different users may click on different images. In response to this phenomenon, the embodiment of the present application provides a personalized sorting method based on the user's image search behavior, and adjusts the image search results corresponding to the user in a targeted manner by analyzing the user's image click characteristics.

In a possible implementation, based on any one of the above embodiments, if the keyword is input by the target user, the method further includes:

S401: Determine the image click feature of the target user according to the image search behavior of the target user.

The image click characteristics of the target user can reflect the characteristics of the target user's image click behavior when performing image search, for example, reflecting which type of image the user is more interested in and is more inclined to click to select.

Optionally, the image search behavior data of the target user can be modeled and analyzed through deep learning methods, mainly to learn the tendency and internal rules of the target user's image clicks. For example, through analysis, it is found that the image click characteristics of the target user for images include relevance and freshness, that is, the target user is more inclined to choose images with high relevance and relatively freshness.

The types here and the aforementioned image categories may be divided in the same manner or in different manners, and this application does not limit this.

S402: After determining the ranking results of the images in the image search results, adjusting the ranking results according to the image click feature, so as to advance the ranking positions of the images in the image search results that meet the image click feature.

S403: Displaying the image search results according to the adjusted ranking results.

It should be noted that the “determining the ranking result of the image in the image search result” mentioned in S402 refers to determining the ranking position of the image in the image collection in combination with the aesthetic score.

For example, the ranking position of the image in the first image set is determined in combination with the aesthetic score; or,

For example, the ranking position of the image in the first image set is determined in combination with the aesthetic score, and the ranking position of the image in the second image set is determined in combination with the aesthetic score; or,

For example, the ranking positions of the images in the first image set in the image search results are determined in combination with the quality scores corresponding to the images in the first image set.

The reason why the determined image click feature is used to adjust the sorting result after the sorting result of the image in the image search results is that the image click feature can more directly reflect the characteristics of the user's image search needs. If the image sorting position is adjusted by the image click feature too early, it will result in most images displayed to the user that meet the characteristics of the user's image search needs, making the images in the image search results too single, and it is not easy for users to access other types of images, making it difficult to provide users with a diverse search experience.

After determining the ranking results of the images in the image search results, the determined image click features are used to adjust the ranking results, so that images that meet the user's image search needs in the diverse image search results can be displayed at a higher position, and other types of images with high aesthetic scores or quality scores will also be displayed to the user. This plays a role in giving priority to displaying images that meet the user's image search needs while maintaining the diversity of image search results, thereby further improving the user's image search experience.

Based on the method for sorting image search results provided in the embodiment corresponding to FIG. 1 , the present application embodiment provides a device for sorting image search results. Referring to FIG. 2 , the device includes an acquisition unit 201, an identification unit 202, a determination unit 203, and a sorting unit 204:

The acquisition unit 201 is used to acquire image search results corresponding to a keyword, where the image search results include multiple images;

The identification unit 202 is used to identify a first image set from the image search results according to an image classification model; the images in the first image set belong to a first image category;

The determining unit 203 is configured to determine the aesthetic scores corresponding to the images in the first image set by using the aesthetic evaluation model corresponding to the first image category;

The ranking unit 204 is configured to determine the ranking positions of the images in the first image set in the image search results in combination with the aesthetic scores corresponding to the images in the first image set when ranking the images in the image search results.

Optionally, the recognition unit is further used to identify the first image set and the second image set from the image search results according to the image classification model; the images in the second image set belong to a second image category different from the first image category;

The determining unit is further configured to determine the aesthetic scores corresponding to the images in the second image set by using the aesthetic evaluation model corresponding to the second image category;

The sorting unit is further configured to determine, when sorting the images in the picture search results, the sorting positions of the images in the second image set in the image search results in combination with the aesthetic scores corresponding to the images in the second image set.

Optionally, the first image category is correlated with the type to which the keyword belongs.

Optionally, if the first image category is a person category, the device further includes a face analysis score determination unit and a quality score determination unit:

The face analysis score determination unit is used to determine the face analysis score corresponding to each image in the first image set according to the face analysis model;

The quality score determination unit is used to determine the quality score corresponding to each image in the first image set according to the face analysis score corresponding to each image in the first image set and the aesthetic score corresponding to each image in the first image set;

The ranking unit is further configured to determine the ranking positions of the images in the first image set in the image search results in combination with the quality scores corresponding to the images in the first image set.

Optionally, the image search result is a plurality of images obtained by searching the keyword and whose relevance to the keyword meets a preset condition.

Optionally, if the keyword is input by the target user, the device further includes an image click feature determination unit, an adjustment unit, and a display unit:

The image click feature determination unit is used to determine the image click feature of the target user according to the image search behavior of the target user;

The adjusting unit is used to adjust the ranking result according to the image click feature after determining the ranking result of the images in the image search result, so as to advance the ranking position of the images that meet the image click feature in the image search result;

The display unit is used to display the image search results according to the adjusted ranking results.

It can be seen that when obtaining image search results corresponding to keywords and including multiple images, a first image set that all belongs to the first image category is identified from the images in the image search results according to the image classification model, and the aesthetic scores corresponding to the images in the first image set are determined by the aesthetic evaluation model corresponding to the first image category. When sorting the images in the picture search results, the sorting positions of the images in the first image set in the image search results are determined in combination with the aesthetic scores corresponding to the images in the first image set. For images belonging to an image category, the aesthetic score of this category can reflect the aesthetic feeling that the image brings to the user under this category, and the possibility that the user chooses to view images with better aesthetic feeling is relatively high. Therefore, the sorting position of images with higher aesthetic scores can be advanced, and the sorting position of images with lower aesthetic scores can be postponed, so that images with higher aesthetic scores can be displayed to users first. Such images are more likely to meet the image search needs of users, thereby improving the image search experience of users.

Based on the aforementioned method and apparatus for judging the quality of a traffic channel, this embodiment provides a device for judging the quality of a traffic channel, which may be a terminal device, and FIG3 is a block diagram of a terminal device 300 according to an exemplary embodiment. For example, the terminal device 300 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.

3 , the terminal device 300 may include one or more of the following components: a processing component 302 , a memory 304 , a power component 306 , a multimedia component 308 , an audio component 310 , an input/output (I/O) interface 312 , a sensor component 314 , and a communication component 316 .

The processing component 302 generally controls the overall operation of the terminal device 300, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 302 may include one or more processors 320 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 302 may include one or more modules to facilitate the interaction between the processing component 302 and other components. For example, the processing component 302 may include a multimedia module to facilitate the interaction between the multimedia component 308 and the processing component 302.

The memory 304 is configured to store various types of data to support operations on the terminal device 300. Examples of such data include instructions for any application or method operating on the device 300, contact data, phone book data, messages, pictures, videos, etc. The memory 304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

The power supply component 306 provides power to various components of the terminal device 300. The power supply component 306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 300.

The multimedia component 308 includes a screen that provides an output interface between the terminal device 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 308 includes a front camera and/or a rear camera. When the terminal device 300 is in an operating mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

The audio component 310 is configured to output and/or input audio signals. For example, the audio component 310 includes a microphone (MIC), and when the device 300 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 304 or sent via the communication component 316. In some embodiments, the audio component 310 also includes a speaker for outputting audio signals.

I/O interface 312 provides an interface between processing component 302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

The sensor assembly 314 includes one or more sensors for providing various aspects of status assessment for the terminal device 300. For example, the sensor assembly 314 can detect the open/closed state of the terminal device 300, the relative positioning of the components, such as the display and keypad of the terminal device 300, and the sensor assembly 314 can also detect the position change of the terminal device 300 or a component of the terminal device 300, the presence or absence of contact between the user and the terminal device 300, the orientation or acceleration/deceleration of the terminal device 300, and the temperature change of the terminal device 300. The sensor assembly 314 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 314 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 314 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 316 is configured to facilitate wired or wireless communication between the terminal device 300 and other devices. The terminal device 300 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 316 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 316 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, the terminal device 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 304 including instructions, and the instructions can be executed by a processor 320 of a terminal device 300 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

FIG4 is a schematic diagram of the structure of a server in an embodiment of the present invention. The server 400 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 422 (for example, one or more processors) and memory 432, and one or more storage media 430 (for example, one or more mass storage devices) storing application programs 442 or data 444. Among them, the memory 432 and the storage medium 430 may be temporary storage or permanent storage. The program stored in the storage medium 430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 422 may be configured to communicate with the storage medium 430 to execute a series of instruction operations in the storage medium 430 on the server 400.

The server 400 may also include one or more power supplies 426, one or more wired or wireless network interfaces 450, one or more input and output interfaces 458, one or more keyboards 456, and/or, one or more operating systems 441, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to execute a method for determining the quality of a traffic channel, the method comprising:

Obtaining image search results corresponding to the keyword, wherein the image search results include multiple images;

identifying a first image set from the image search results according to an image classification model; the images in the first image set belong to a first image category;

Determining an aesthetic score corresponding to each image in the first image set by using an aesthetic evaluation model corresponding to the first image category;

When ranking the images in the picture search results, the ranking positions of the images in the first image set in the image search results are determined in combination with the aesthetic scores corresponding to the images in the first image set.

A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the above program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the above storage medium can be at least one of the following media: read-only memory (ROM), RAM, magnetic disk or optical disk, etc., which can store program codes.

It should be noted that each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, in which the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims (8)

1. A method of ranking image search results, the method comprising:

acquiring an image search result corresponding to a keyword, wherein the image search result comprises a plurality of images which are obtained according to the keyword search and have correlation with the keyword meeting a preset condition;

Identifying a first set of images from the image search results according to an image classification model; the images in the first image set belong to a first image category;

Determining aesthetic scores corresponding to the images in the first image set through aesthetic evaluation models corresponding to the first image types;

When images in the image search results are ranked, determining initial ranking positions of the images in the first image set in the image search results according to the correlation between the images in the image search results and the keywords, adjusting the initial ranking positions by combining aesthetic scores corresponding to the images in the first image set, and determining final ranking positions of the images in the first image set in the image search results;

If the keyword is input by a target user, determining an image click feature of the target user according to the image searching behavior of the target user; the image clicking feature is used for representing the image clicking behavior characteristics of the target user when image searching is carried out, and comprises correlation and freshness;

After determining the sorting result of the images in the image search result, adjusting the sorting result according to the image clicking feature so as to advance the sorting position of the images conforming to the image clicking feature in the image search result;

displaying the image search result according to the adjusted sorting result;

Wherein identifying a first set of images from the image search results according to an image classification model comprises:

identifying the first and second image sets from the image search results according to the image classification model; the images in the second image set belong to a second image category different from the first image category;

And if the first image which is simultaneously in the first image set and the second image set exists and the image category of the first image has correlation with the type of the keyword, adopting the aesthetic score of the first image category as the aesthetic score of the first image.

2. The method according to claim 1, wherein the method further comprises:

determining aesthetic scores corresponding to the images in the second image set through aesthetic evaluation models corresponding to the second image categories;

And when the images in the image search results are ranked, determining the ranking positions of the images in the second image set in the image search results by combining the aesthetic scores corresponding to the images in the second image set.

3. The method of claim 1, wherein if the first image category is a person category, the method further comprises:

determining face analysis scores corresponding to the images in the first image set according to the face analysis model;

Determining the quality scores corresponding to the images in the first image set according to the face analysis scores corresponding to the images in the first image set and the aesthetic scores corresponding to the images in the first image set;

The determining, in combination with the aesthetic scores corresponding to the images in the first image set, the ranking positions of the images in the first image set in the image search results includes:

And determining the ordering positions of the images in the first image set in the image search results by combining the quality scores corresponding to the images in the first image set.

4. An image search result sorting device, characterized in that the device comprises an acquisition unit, an identification unit, a determination unit and a sorting unit:

The acquisition unit is used for acquiring an image search result corresponding to the keyword, wherein the image search result comprises a plurality of images which are obtained according to the keyword search and have the correlation with the keyword meeting the preset condition;

the identification unit is used for identifying a first image set from the image search results according to an image classification model; the images in the first image set belong to a first image category;

The determining unit is used for determining aesthetic scores corresponding to the images in the first image set through the aesthetic evaluation models corresponding to the first image categories;

The sorting unit is used for determining an initial sorting position of the images in the first image set in the image search result according to the correlation between the images in the image search result and the keywords when the images in the image search result are sorted, adjusting the initial sorting position by combining the aesthetic scores corresponding to the images in the first image set, and determining a final sorting position of the images in the first image set in the image search result;

if the keyword is input by the target user, the device further comprises an image click feature determining unit, an adjusting unit and a display unit:

The image click feature determining unit is used for determining the image click feature of the target user according to the image searching behavior of the target user; the image clicking feature is used for representing the image clicking behavior characteristics of the target user when image searching is carried out, and comprises correlation and freshness;

the adjusting unit is used for adjusting the sorting result according to the image clicking characteristics after determining the sorting result of the images in the image searching result so as to advance the sorting position of the images conforming to the image clicking characteristics in the image searching result;

The display unit is used for displaying the image search results according to the adjusted sorting results;

The identification unit is specifically configured to:

identifying the first and second image sets from the image search results according to the image classification model; the images in the second image set belong to a second image category different from the first image category;

And if the first image which is simultaneously in the first image set and the second image set exists and the image category of the first image has correlation with the type of the keyword, adopting the aesthetic score of the first image category as the aesthetic score of the first image.

5. The apparatus of claim 4, wherein the device comprises a plurality of sensors,

The determining unit is further configured to determine aesthetic scores corresponding to the images in the second image set according to the aesthetic evaluation models corresponding to the second image categories;

the ranking unit is further configured to determine, when ranking the images in the image search result, a ranking position of the images in the second image set in the image search result in combination with aesthetic scores corresponding to the images in the second image set.

6. The apparatus according to claim 4, wherein if the first image category is a person, the apparatus further comprises a face analysis score determination unit and a quality score determination unit:

The face analysis score determining unit is used for determining face analysis scores corresponding to the images in the first image set according to a face analysis model;

The quality score determining unit is used for determining the quality scores corresponding to the images in the first image set according to the face analysis scores corresponding to the images in the first image set and the aesthetic scores corresponding to the images in the first image set;

the ranking unit is further configured to determine a ranking position of the images in the first image set in the image search result in combination with the quality scores corresponding to the images in the first image set.

7. An apparatus for ranking image search results, comprising a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by one or more processors, the one or more programs comprising instructions for:

acquiring an image search result corresponding to a keyword, wherein the image search result comprises a plurality of images which are obtained according to the keyword search and have correlation with the keyword meeting a preset condition;

Identifying a first set of images from the image search results according to an image classification model; the images in the first image set belong to a first image category;

Determining aesthetic scores corresponding to the images in the first image set through aesthetic evaluation models corresponding to the first image types;

When images in the image search results are ranked, determining initial ranking positions of the images in the first image set in the image search results according to the correlation between the images in the image search results and the keywords, adjusting the initial ranking positions by combining aesthetic scores corresponding to the images in the first image set, and determining final ranking positions of the images in the first image set in the image search results;

If the keyword is input by a target user, determining an image click feature of the target user according to the image searching behavior of the target user; the image clicking feature is used for representing the image clicking behavior characteristics of the target user when image searching is carried out, and comprises correlation and freshness;

After determining the sorting result of the images in the image search result, adjusting the sorting result according to the image clicking feature so as to advance the sorting position of the images conforming to the image clicking feature in the image search result;

displaying the image search result according to the adjusted sorting result;

Wherein identifying a first set of images from the image search results according to an image classification model comprises:

identifying the first and second image sets from the image search results according to the image classification model; the images in the second image set belong to a second image category different from the first image category;

And if the first image which is simultaneously in the first image set and the second image set exists and the image category of the first image has correlation with the type of the keyword, adopting the aesthetic score of the first image category as the aesthetic score of the first image.

8. A machine readable medium having instructions stored thereon, which when executed by one or more processors, cause an apparatus to perform a method of ranking image search results as recited in one or more of claims 1-3.