CN103440262A - Image Retrieval System and Method Based on Correlation Feedback and Bag-of-Features - Google Patents

Abstract

The invention discloses an image retrieval system and method based on relevant feedback and Bag-of-Features, the system includes: the characteristic extraction module is used for preprocessing the images and extracting the local characteristics of each image; the characteristic dictionary generating module is used for finding out key characteristics from the whole image database and forming a dictionary set; the frequency characteristic vector generating module is used for constructing a characteristic vector for each image; the characteristic weighting module is used for generating weight for each key characteristic in the dictionary, and then multiplying the weight by the corresponding component in the frequency characteristic vector to construct a weighted characteristic vector for each image; the similarity measurement module is used for calculating the similarity between the two images; and the related feedback module is used for enabling the user to participate in the retrieval process. By implementing the invention, the efficiency and the accuracy of image retrieval are improved.

Description

Image Retrieval System and Method Based on Correlation Feedback and Bag-of-Features

technical field

The invention relates to the field of computers, in particular to an image retrieval method based on correlation feedback and Bag-of-Features.

Background technique

With the development of information technology, there are more and more images in various fields. How to quickly and accurately retrieve relevant images from a large amount of image data has gradually become a hot spot of attention. In recent years, large-scale image retrieval has received increasing attention in the industrial application and academic research circles, and various image retrieval methods have been proposed, including text-based image retrieval, classification-based image retrieval, and so on.

Text-based image retrieval follows the traditional text retrieval technology, avoids the analysis of image visualization elements, but indexes images from image name, image size, compression type, author, year, etc., and generally uses keywords to query images, or It is to browse and find images under a specific category in the form of a hierarchical directory. In addition, the theme of the page where the image is located, the file name of the image, the text content closely surrounding the image, and the link address of the image are all used as a sentence for image analysis, and the characteristics of the image in it are inferred based on the text analysis results.

Classification-based image retrieval is a technique that utilizes image classification for retrieval. The technique requires explicit classification of images in a database and selection of some of the most representative images for each category. When the user inputs a query image, the system measures the similarity between the image and the representative images of each category in the database to determine the category to which the query image belongs, and then returns all images of the category to the user as retrieval results.

Although the text-based image retrieval method is convenient and quick, and the desired image can be quickly queried according to the keyword, this retrieval method is completely separated from the visual content of the image, and only relies on the associated keyword, which requires text identification for all images in advance. , and the accuracy of identification directly affects the accuracy of retrieval. However, classification-based image retrieval methods rely on the classification of images in the database, and currently do not classify various images accurately and effectively. The disadvantages of these traditional image retrieval methods are becoming more and more prominent, and people urgently need to update more effective methods, and the research on image retrieval methods continues to move forward.

Due to various drawbacks in the traditional image retrieval methods, it has gradually been unable to meet people's needs. A different solution has been proposed: content-based image retrieval methods. Content-based image retrieval is to use the visual features of images to retrieve images. It extracts the low-level features of images, including color, shape, texture, etc., and then compares the low-level features of the query image with the features in the database to find out the relevant features. Images with similar query features are returned to the user.

Although the content-based image retrieval method breaks away from the image text annotation and retrieves the visual content of the image, this method only extracts low-level features such as color, shape, and texture of the image, and cannot represent the high-level semantic content of the image. Therefore, the retrieval results More often than not.

Contents of the invention

The present invention provides an image retrieval method based on correlation feedback and Bag-of-Features, adopts the Bag-of-Features method, regards each image as a collection of some local features, and selects the key points in the image library features, and then represent each image as a vector based on the key feature set, so that the purpose of image retrieval can be achieved through the comparison of vectors, which is concise and effective. At the same time, in order to take into account the high-level semantics of the image, combined with the method of relevant feedback, let the user participate in the retrieval process, the user marks the positive correlation and negative correlation of each retrieval result, and the system readjusts the retrieval parameters according to the user's feedback, so as to iterate , and finally get the user-satisfied retrieval results.

Correspondingly, an embodiment of the present invention provides an image retrieval system based on relevant feedback and Bag-of-Features, including:

The feature extraction module is used to preprocess the image and extract the local features of each image;

The feature dictionary generation module is used to find key features from the entire image database and form a dictionary set;

A frequency eigenvector generation module is used to construct a eigenvector for each image;

The feature weighting module is used to generate a weight for each key feature in the dictionary, and then multiplies the weight by the corresponding component in the frequency feature vector to construct a weighted feature vector for each image;

Similarity measurement module for calculating the similarity between two images;

The relevant feedback module is used to allow users to participate in the retrieval process. After the user enters the query conditions, the retrieval system returns the query results, and then the user screens the query results, and marks the useful ones as positive correlations, and the useless ones as negative correlations. According to the user's feedback, readjust the query conditions for retrieval, and repeat this cycle until the user is satisfied with the result.

Correspondingly, the embodiment of the present invention also provides an image retrieval method based on relevant feedback and Bag-of-Features, including:

Step 1, perform feature extraction on each image, find out local features, and express it with SIFT operator;

Step 2. Collect the local features of all the images together, and use the K-means clustering method to generate a specified number of key features to form a feature dictionary;

Step 3. For each image, assign each local feature to the key feature of the nearest neighbor in turn, indicating the frequency of the key feature, so that based on the feature dictionary, a frequency feature vector can be generated for each image;

Step 4, count the number of images in which each key feature appears, that is, how many images it has appeared in, and then divide it by the total number of images to obtain the IDF value of the key feature as the weight of the key feature;

Step 5. For each image, multiply each component in its feature vector by the weight of the corresponding key feature to obtain a weighted feature vector;

Step 6, calculate the similarity between the vector of the query image and the image vector in the data, and sort the output in order of similarity from high to low;

Step 7. The user checks the search results, if the requirements are met, then end; otherwise, go to step 8;

Step 8. If the user is not satisfied with the search result, he will mark the search result with positive correlation and negative correlation, and then re-input it to the retrieval system;

Step 9. The system readjusts the retrieval conditions according to the user's feedback, and proceeds to step 6.

The present invention has the following beneficial effects. The present invention perfectly combines the Bag-of-Features and related feedback methods to improve the efficiency and accuracy of image retrieval. First, the Bag-of-Features method is used to represent the image as a collection of features, and then as a feature vector. This quantization method makes image representation and similarity calculation convenient and simple. At the same time, the way of relevant feedback also allows users to fully participate in the retrieval process, avoiding the loss of high-level semantics of images. The combination of Bag-of-Features and relevant feedback greatly simplifies the image representation and similarity comparison, and also takes into account the low-level visual features and high-level semantic content of the image, making image retrieval simpler and more accurate.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is a schematic structural diagram of an image retrieval system based on correlation feedback and Bag-of-Features in an embodiment of the present invention;

FIG. 2 is a flowchart of an image retrieval method based on relevant feedback and Bag-of-Features in an embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

The present invention provides an image retrieval system and method based on correlation feedback and Bag-of-Features, adopts the Bag-of-Features method, regards each image as a collection of some local features, and selects the The key features, and then represent each image as a vector based on the key feature set, so that the purpose of image retrieval can be achieved by comparing the vectors, which is simple and effective. At the same time, in order to take into account the high-level semantics of the image, combined with the method of relevant feedback, let the user participate in the retrieval process, the user marks the positive correlation and negative correlation of each retrieval result, and the system readjusts the retrieval parameters according to the user's feedback, so as to iterate , and finally get the user-satisfied retrieval results.

Figure 1 shows an image retrieval system based on correlation feedback and Bag-of-Features in an embodiment of the present invention, which mainly includes a feature extraction module, a feature dictionary generation module, a frequency feature vector generation module, a feature weighting module, and a similarity measure modules, related feedback modules, etc.

The feature extraction module is to preprocess the image and extract the local features of each image, so that each image can be represented as a collection of local features without considering the positional relationship of the local features. At the same time, it is also necessary to quantify each local feature, and use the SIFT operator to extract local features and express them as 128-dimensional vectors.

The feature dictionary generation module is responsible for finding key features from the entire image database and forming a dictionary set. Because each image has many local features, the number of all local features in the entire database is very large, so it is necessary to use K-means clustering to find representative features and form a feature word list from these key features.

The frequency feature vector generation module is to construct a feature vector for each image. A feature vector can be generated for each image based on the feature dictionary, and each component of the vector represents the frequency of the corresponding key feature. Because the key features in the dictionary are obtained by clustering, it is impossible to find the key features completely in each image. We assign the local features in the image to the nearest key feature, indicating that the key feature appears once. Through the frequency feature vector generation module, each image is represented by a vector with the same dimension.

The feature weighting module generates a weight for each key feature in the dictionary, and then multiplies the weight by the corresponding component in the frequency feature vector to construct a weighted feature vector for each image. The importance of each key feature in the dictionary is different in the image library. The IDF algorithm can be used to calculate the IDF value of each key feature and use it as the weight of the key feature. Then calculate the weighted feature vector of each image.

The similarity measurement module is to calculate the similarity between two images. The images in the database have been quantized and represented by vectors with the same dimension, so the similarity between corresponding images is measured by calculating the distance between vectors, and sorted in descending order of similarity.

The relevant feedback module is to allow users to participate in the retrieval process. After the user enters the query conditions, the retrieval system returns the query results, and then the user filters the query results, and marks the useful ones as positive correlations, and the useless ones as negative correlations, and the system re-adjusts the query conditions for retrieval based on user feedback. Loop until the user is satisfied with the result.

Fig. 2 shows a flow chart of an image retrieval method based on relevant feedback and Bag-of-Features in an embodiment of the present invention, and the specific steps are as follows:

The first step is to perform feature extraction on each image, find out local features, and express them with SIFT operator.

In the second step, the local features of all images are gathered together, and K-means clustering is used to generate a specified number of key features to form a feature dictionary.

In the third step, for each image, each local feature is assigned to the nearest neighbor key feature in turn, indicating the frequency of key features. In this way, based on the feature dictionary, a frequency feature vector can be generated for each image.

Step 4, combined with step 3, count the number of images in which each key feature appears, that is, how many images it has appeared in, and then divide it by the total number of images to obtain the IDF value of the key feature as the weight of the key feature.

Step 5: For each image, multiply each component in its feature vector by the weight of the corresponding key feature to obtain a weighted feature vector.

Step 6: Calculate the similarity between the vector of the query image and the image vector in the data, and sort the output in order of similarity from high to low.

In step 7, the user checks the search results, if the requirements are met, then end; otherwise, go to step 8.

Step 8, if the user is not satisfied with the search result, he will mark the search result with positive correlation and negative correlation, and then input it to the retrieval system again.

In step 9, the system readjusts the retrieval conditions according to the user's feedback, and then enters step 6.

To sum up, the present invention perfectly combines Bag-of-Features and related feedback methods to improve the efficiency and accuracy of image retrieval. First, the Bag-of-Features method is used to represent the image as a collection of features, and then as a feature vector. This quantization method makes image representation and similarity calculation convenient and simple. At the same time, the way of relevant feedback also allows users to fully participate in the retrieval process, avoiding the loss of high-level semantics of images. The combination of Bag-of-Features and relevant feedback greatly simplifies the image representation and similarity comparison, and also takes into account the low-level visual features and high-level semantic content of the image, making image retrieval simpler and more accurate.

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above-mentioned embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium can include: Read-only memory (ROM, Read Only Memory), random access memory (RAM, Random AccessMemory), disk or CD-ROM, etc.

The image retrieval system and method based on the relevant feedback mode and Bag-of-Features provided by the embodiments of the present invention have been introduced in detail above. In this paper, specific examples have been used to illustrate the principle and implementation of the present invention. The above embodiments The description is only used to help understand the method of the present invention and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation and scope of application. In summary, As stated above, the content of this specification should not be construed as limiting the present invention.

Claims (2)

1. An image retrieval system based on relevant feedback and Bag-of-Features, characterized in that, comprising: The feature extraction module is used to preprocess the image and extract the local features of each image; The feature dictionary generation module is used to find key features from the entire image database and form a dictionary set; A frequency eigenvector generation module is used to construct a eigenvector for each image; The feature weighting module is used to generate a weight for each key feature in the dictionary, and then multiplies the weight by the corresponding component in the frequency feature vector to construct a weighted feature vector for each image; Similarity measurement module for calculating the similarity between two images; The relevant feedback module is used to allow users to participate in the retrieval process. After the user enters the query conditions, the retrieval system returns the query results, and then the user screens the query results, and marks the useful ones as positive correlations, and the useless ones as negative correlations. According to the user's feedback, readjust the query conditions for retrieval, and repeat this cycle until the user is satisfied with the result. 2. An image retrieval method based on relevant feedback and Bag-of-Features, characterized in that, comprising: Step 1, perform feature extraction on each image, find out local features, and express it with SIFT operator; Step 2. Collect the local features of all the images together, and use the K-means clustering method to generate a specified number of key features to form a feature dictionary; Step 3. For each image, assign each local feature to the key feature of the nearest neighbor in turn, indicating the frequency of the key feature, so that based on the feature dictionary, a frequency feature vector can be generated for each image; Step 4, count the number of images in which each key feature appears, that is, how many images it has appeared in, and then divide it by the total number of images to obtain the IDF value of the key feature as the weight of the key feature; Step 5. For each image, multiply each component in its feature vector by the weight of the corresponding key feature to obtain a weighted feature vector; Step 6, calculate the similarity between the vector of the query image and the image vector in the data, and sort the output in order of similarity from high to low; Step 7. The user checks the search results, if the requirements are met, then end; otherwise, go to step 8; Step 8. If the user is not satisfied with the search result, he will mark the search result with positive correlation and negative correlation, and then re-input it to the retrieval system; Step 9. The system readjusts the retrieval conditions according to the user's feedback, and proceeds to step 6.

Images