CN114626483B - A method and device for generating a landmark image - Google Patents
A method and device for generating a landmark image Download PDFInfo
- Publication number
- CN114626483B CN114626483B CN202210333684.1A CN202210333684A CN114626483B CN 114626483 B CN114626483 B CN 114626483B CN 202210333684 A CN202210333684 A CN 202210333684A CN 114626483 B CN114626483 B CN 114626483B
- Authority
- CN
- China
- Prior art keywords
- landmark
- image
- target
- images
- classification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本发明实施例提供了一种地标图像生成方法及装置，涉及数据处理技术领域，上述方法包括：获得目标地标的各地标图像；按照图像采集视角对地标图像进行分类；选择各分类中地标图像的数量小于预设图像数量的分类，作为孤立视角对应的目标分类；基于上述目标分类中包括的地标图像进行图像增广处理，生成上述目标地标的新地标图像。应用本发明实施例提供的方案，能够生成地标的图像。
The embodiment of the present invention provides a method and device for generating a landmark image, which relates to the field of data processing technology. The method includes: obtaining landmark images of a target landmark; classifying the landmark images according to the image acquisition perspective; selecting a category in which the number of landmark images in each category is less than the number of preset images as the target category corresponding to the isolated perspective; performing image augmentation processing based on the landmark images included in the target category to generate a new landmark image of the target landmark. Applying the solution provided by the embodiment of the present invention, an image of a landmark can be generated.
Description
Technical Field
The present invention relates to the field of data processing technologies, and in particular, to a landmark image generating method and apparatus.
Background
When a user browses pictures or views videos, places which are unknown to the user may appear in the picture content. To assist the user in learning the location in the screen content, landmark recognition may be performed based on the image, thereby allowing the user to quickly learn the location appearing in the screen content.
In the prior art, landmark images of all the landmarks are usually collected in advance and called as existing images, so that when landmark identification is performed based on the images, candidate landmarks corresponding to the existing images matched with the images to be identified can be determined first, and then the landmarks corresponding to the images to be identified are determined from the candidate landmarks according to the number of the matched images corresponding to the candidate landmarks, so that the landmark identification is completed. The greater the number of matching images corresponding to each candidate landmark, the greater the likelihood that the landmark is considered to be the landmark corresponding to the image to be identified.
However, when landmark images of some landmarks are collected, the difficulty is high, the number of collected landmark images is small, so that when images are matched, the number of matched images corresponding to the landmarks is small, and therefore, the situation of landmark identification errors can occur, and the accuracy of landmark identification is low.
Disclosure of Invention
The embodiment of the invention aims to provide a landmark image generation method and device for generating an image of a landmark. The specific technical scheme is as follows:
in a first aspect, an embodiment of the present invention provides a landmark image generating method, including:
Obtaining each landmark image of a target landmark;
classifying landmark images according to image acquisition visual angles;
selecting the classification of which the number of landmark images in each classification is smaller than the number of preset images as a target classification corresponding to an isolated view angle;
And performing image augmentation processing based on landmark images included in the target classification, and generating a new landmark image of the target landmark.
In one embodiment of the present invention, the classifying the landmark image according to the image acquisition view angle includes:
Obtaining global image features of each landmark image;
clustering the obtained global image features to obtain feature clusters;
and dividing landmark images corresponding to the global image features belonging to the same feature cluster into classifications corresponding to image acquisition visual angles.
In one embodiment of the present invention, the generating a new landmark image of the target landmark based on the image augmentation processing of the landmark image included in the target classification includes:
Determining the target number of images to be generated according to the number of landmark images included in the target classification and the preset image number;
And performing image augmentation processing based on landmark images included in the target classification, and generating new landmark images of the target number of target landmarks.
In one embodiment of the present invention, the generating a new landmark image of the target landmark based on the image augmentation processing of the landmark image included in the target classification includes:
And performing target processing on the landmark images included in the target classification, and generating a new landmark image of the target landmark based on the processed images, wherein the target processing comprises at least one of random clipping processing, random affine processing and random color parameter adjustment.
In a second aspect, an embodiment of the present invention provides a landmark image generating apparatus, including:
the image acquisition module is used for acquiring each landmark image of the target landmark;
the image classification module is used for classifying landmark images according to the image acquisition visual angles;
The target classification determining module is used for selecting the classification that the number of landmark images in each classification is smaller than the number of preset images as the target classification corresponding to the isolated view angle;
And the image generation module is used for carrying out image augmentation processing based on the landmark images included in the target classification and generating a new landmark image of the target landmark.
In one embodiment of the present invention, the image classification module includes:
The image feature obtaining sub-module is used for obtaining global image features of all landmark images;
the feature clustering sub-module is used for clustering the obtained global image features to obtain feature clusters;
And the image classification sub-module is used for dividing landmark images corresponding to the global image features belonging to the same feature cluster into classifications corresponding to one image acquisition view angle.
In one embodiment of the invention, the image generation module is specifically configured to determine the number of objects to be generated according to the number of landmark images included in the object classification and the preset number of images, and perform image augmentation processing based on the landmark images included in the object classification to generate new landmark images of the number of objects.
In one embodiment of the invention, the image generation module is specifically configured to perform target processing on landmark images included in the target classification, and generate a new landmark image of the target landmark based on the processed images, where the target processing includes at least one of random clipping processing, random affine processing, and random color parameter adjustment.
In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;
A memory for storing a computer program;
and the processor is used for realizing the landmark image generation method according to the first aspect when executing the program stored in the memory.
In a fourth aspect, an embodiment of the present invention provides a computer readable storage medium, in which a computer program is stored, the computer program implementing the landmark image generation method according to the first aspect, when executed by a processor.
In a fifth aspect, embodiments of the present invention provide a computer program product comprising instructions which, when run on a computer, cause the computer to perform the landmark image generation method as described in the first aspect above.
In the above, when the landmark image is generated by applying the scheme provided by the embodiment of the invention, each landmark image of the target landmark is divided into each category according to the image acquisition view angle, and the target category corresponding to the isolated view angle is determined based on the number of the landmark images in each category, so that the new landmark image of the target landmark can be generated by performing image augmentation processing based on the landmark images included in the target category, and the image number of the target landmark is effectively improved.
In addition, it can be seen that when the scheme provided by the embodiment of the invention is applied to landmark image generation, instead of directly performing image augmentation processing on each landmark image of a target landmark to generate a new landmark image, only the landmark images included in the target classification are subjected to image augmentation processing to generate the new landmark image, so that the newly generated landmark image belongs to the target classification, and the target classification corresponds to an isolated view angle, and the number of landmark images belonging to the isolated view angle is increased, and the number of landmark images belonging to each image acquisition view angle is relatively balanced because the number of images belonging to other image acquisition view angles is unchanged. On the basis, after the landmark images of the isolated view angles are added, when the landmark recognition is carried out on the image to be recognized, even if the image to be recognized is actually the image belonging to the isolated view angles, the situation that the landmark images belonging to the isolated view angles are fewer and the landmark recognition to the image to be recognized belongs to errors is avoided, so that the accuracy of the landmark recognition can be improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
Fig. 1 is a flowchart of a first landmark image generating method according to an embodiment of the present invention;
Fig. 2 is a flowchart of a second landmark image generating method according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a first landmark image generating apparatus according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a first landmark image generating apparatus according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The following description of the technical solutions according to the embodiments of the present invention will be given with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Because of the greater difficulty in collecting landmark images of some landmarks, the number of collected landmark images is less, and therefore, the embodiment of the invention provides a landmark image generation method, device, equipment and storage medium for generating landmark images. The following description will be made separately.
In one embodiment of the present invention, there is provided a landmark image generation method, including:
Obtaining each landmark image of a target landmark;
classifying landmark images according to image acquisition visual angles;
selecting the classification of which the number of landmark images in each classification is smaller than the number of preset images as a target classification corresponding to an isolated view angle;
And performing image augmentation processing based on the landmark images included in the target classification, and generating a new landmark image of the target landmark.
In the above, when the scheme provided by the embodiment of the invention is applied to generate images, each landmark image of the target landmark is divided into each category according to the image acquisition view angle, and the target category corresponding to the isolated view angle is determined based on the number of landmark images in each category, so that the image augmentation processing is performed based on the landmark images included in the target category, a new landmark image of the target landmark can be generated, and the image number of the target landmark is effectively increased.
In addition, it can be seen that when the scheme provided by the embodiment of the invention is applied to landmark image generation, instead of directly performing image augmentation processing on each landmark image of a target landmark to generate a new landmark image, only the landmark images included in the target classification are subjected to image augmentation processing to generate the new landmark image, so that the newly generated landmark image belongs to the target classification, and the target classification corresponds to an isolated view angle, and the number of landmark images belonging to the isolated view angle is increased, and the number of landmark images belonging to each image acquisition view angle is relatively balanced because the number of images belonging to other image acquisition view angles is unchanged. On the basis, after the landmark images of the isolated view angles are added, when the landmark recognition is carried out on the image to be recognized, even if the image to be recognized is actually the image belonging to the isolated view angles, the situation that the landmark images belonging to the isolated view angles are fewer and the landmark recognition to the image to be recognized belongs to errors is avoided, so that the accuracy of the landmark recognition can be improved.
The landmark image generation method provided by the embodiment of the invention is described in detail below through a specific embodiment.
Referring to fig. 1, fig. 1 is a flowchart of a first landmark image generating method according to an embodiment of the present invention, where the method includes the following steps S101 to S104.
And step S101, obtaining each landmark image of the target landmark.
The target landmark can be a marked building of a city, and can be any type of area or place, such as a natural landscape, an artificial building, a historical remains, and the like.
Specifically, each landmark image of the target landmark may be collected in advance and stored in a pre-established image database, so that each landmark image of the target landmark may be read from the image database. The landmark images stored in the image database in advance may be obtained by photographing at the target landmark by a worker using an image acquisition device such as a camera, a video camera, or the like, or may be obtained by extracting an image in a web page containing the target landmark information using a web crawler engine. The web page containing the target landmark information may be a travel propaganda web page of the target landmark, etc.
Step S102, classifying landmark images according to image acquisition view angles.
The image acquisition view angle is a lens view angle of the image acquisition device for acquiring the landmark image. It can be understood that the physical areas covered by different image acquisition viewing angles are also different, and thus the landmark images acquired by the image acquisition devices are also different.
The physical area covered by the image acquisition view angle may include all physical areas occupied by the target landmark, or may include a part of representative physical areas of the target landmark. For example, if the target landmark is an ancient building group a, the image acquisition view angle may cover all the occupied areas of the building group, or may cover only the area where the gate a is located, the area where the palace B is located, etc. in the building group, and if the target landmark is a natural scenic spot B, the image acquisition view angle may cover the area where the river c is located, the area where the lake d is located, etc. in the natural scenic spot.
Because a plurality of image acquisition view angles exist in the target landmark, landmark images acquired at the same image acquisition view angle tend to have smaller differences, and landmark images acquired at different image acquisition view angles tend to have larger differences, the landmark images belonging to the same image acquisition view angle are classified into the same category based on the image acquisition view angle to which the landmark images belong in the embodiment of the invention.
Specifically, landmark images may be classified by image acquisition perspectives in the following manner.
In one embodiment, the shooting location of the landmark image may be acquired, a preset area to which the shooting location belongs is determined as an area covered by the image acquisition view angle, then the landmark images with the same area covered by the image acquisition view angle are determined as images belonging to the same image acquisition view angle, and further the images belonging to the same image acquisition view angle are classified into the same category. For example, if the coordinates of the shooting locations of the landmark image P and the landmark image Q are (x 1, y 1) and (x 2, y 2) respectively, and the coordinates belong to the preset area where the palace b is located, the area where the palace b is located may be determined to be the area covered by the image acquisition view angles of the landmark image P and the landmark image Q, and further the landmark image P and the landmark image Q may be determined to be images belonging to the same image acquisition view angle, and the landmark image P and the landmark image Q may be classified into the same category.
How to acquire the shooting location of the above-described landmark image is described below by way of example in two ways.
In the first mode, the shooting location recorded in the image file of the landmark image may be read. The landmark image acquired by the image acquisition device is used, and the attribute information of the image file contains various information of the landmark image, such as image resolution, image size, shooting time and the like. In some cases, when an image is acquired by using an image acquisition device having a satellite positioning function, the location where the device was located when the landmark image was created is also recorded in the attribute information of the image file of the obtained landmark image as the shooting location of the landmark image. In this case, the shooting location in the attribute information of the image file of the landmark image can be read.
In the second mode, when the staff uses the image acquisition device to acquire the landmark images, the shooting locations of the landmark images can be manually recorded and stored, so that the electronic device can read the shooting location information to obtain the shooting locations of the landmark images.
In another embodiment, global image features of each landmark image may be obtained first, the obtained global image features are clustered to obtain feature clusters, and landmark images corresponding to the global image features belonging to the same feature cluster are classified into classifications corresponding to image acquisition viewing angles, where the specific implementation is shown in steps S202-S204 in the embodiment shown in fig. 2, and the description is omitted here.
Step S103, selecting the classification that the number of landmark images in each classification is smaller than the number of preset images as the target classification corresponding to the isolated view angle.
The isolated view angles are image acquisition view angles with larger difference between the corresponding landmark image number and other image acquisition view angles. When images of the target landmark are collected in advance, some image collection view angles belonging to the target landmark can cause less landmark images belonging to the image collection view angles than other image collection view angles due to the fact that the image collection difficulty is high and/or the awareness of the area covered by the image collection view angles is low. For example, the target landmark C is a mountain, the area covered by the image acquisition view angle C is a cliff area of the landmark, and since the difficulty in collecting the landmark images belonging to the image acquisition view angle C is high, only 1 existing landmark image belonging to the image acquisition view angle C is provided, and the number of images of other image acquisition view angles is 5, the difference between the number of landmark images belonging to the image acquisition view angle C and the number of images of other image acquisition view angles is large, and the image acquisition view angle C can be determined as an isolated view angle.
Since each classification obtained by division corresponds to each image acquisition view angle, the number of landmark images in each classification is the number of landmark images belonging to each image acquisition view angle. Therefore, according to the number of landmark images included in each image acquisition view angle, the isolated view angles with larger differences between the number of corresponding landmark images and other image acquisition view angles can be determined, and then the target classification corresponding to the isolated view angles can be determined.
Specifically, the preset number of images may be set by a worker according to the number of landmark images included in each category of the target landmark. For example, the landmark images of the target landmark C are classified into 10 classifications, wherein 8 classifications include 5 landmark images and 2 classifications include 2 landmark images and 3 landmark images, respectively, and the preset number of images may be 4, so that the classifications including 2 landmark images and 3 landmark images are determined as the target classifications corresponding to the isolated view angles. Therefore, the classification that the number of landmark images is smaller than the number of preset images is used as the target classification corresponding to the isolated view angle, and the target classification corresponding to the isolated view angle can be accurately determined.
In one embodiment of the present invention, this step may also be implemented by:
The average value of the number of landmark images in each category is calculated first, and the category with the number of landmark images smaller than the average value in each category is selected as the target category corresponding to the isolated view angle.
In this way, the classification of which the number of the included landmark images is smaller than the average value of the number of the landmark images in each classification is taken as the target classification corresponding to the isolated view angle, and after the image augmentation processing is performed on the landmark images belonging to the isolated view angle to generate new landmark images, the number of the images belonging to the isolated view angle can be made to approach to the average value, so that the number of the images included in each image acquisition view angle of the target landmark is relatively balanced. Similarly, the median, mode, and the like of the number of landmark images in each category may be calculated, and a category having a number of landmark images smaller than the above-described number in each category may be selected as the target category corresponding to the isolated view angle.
Step S104, performing image augmentation processing based on the landmark images included in the target classification, and generating a new landmark image of the target landmark.
As can be seen from the foregoing step S103, since the target class corresponding to the isolated view angle has been determined, the landmark image included in the target class can be subjected to the image augmentation process to generate a new landmark image of the target landmark.
Specifically, a new landmark image of the target landmark may be generated in the following manner.
In the first embodiment, the landmark image included in the target classification may be subjected to random cropping processing, and the processed image is used as a new landmark image of the target landmark.
For example, a clipping size smaller than the original size of the landmark image included in the target classification is randomly determined, and an image having the clipping size is clipped from the landmark image, thereby obtaining a new landmark image.
In a second embodiment, a random affine process may be performed on landmark images included in the target classification, and the processed images may be used as new landmark images of the target landmarks.
The random affine processing is performed on the landmark images included in the object classification, including performing random scaling, translation, rotation, reflection, miscut, or the like on the landmark images.
In a third embodiment, the landmark images included in the target classification may be subjected to random color parameter adjustment, and the adjusted images are used as new landmark images of the target landmarks.
The color parameters may include brightness, contrast, chromaticity, saturation, etc., and a new image may be generated by randomly adjusting the color parameters of the landmark image. For example, a floating value may be set in advance, the original color parameter of the landmark image may be randomly adjusted within the range of the floating value, and the adjusted image may be used as a new landmark image of the target landmark.
Of course, the new landmark image of the target landmark may be generated by overlapping at least two of the above three embodiments, and the number and the order of overlapping in the above embodiments are not limited in the embodiment of the present invention. For example, after performing the random affine processing on the landmark image included in the target classification, the image obtained after the processing may be further subjected to random color parameter transformation, and the processed image may be used as a new landmark image of the target landmark.
By performing at least one of a plurality of target processes on the landmark images included in the target classification, a new landmark image of a more varied target landmark can be generated based on the processed images.
In one embodiment of the present invention, a new landmark image of the target landmark may be generated by the following steps a and B.
And A, determining the target number of the images to be generated according to the number of the landmark images included in the target classification and the preset image number.
Specifically, a difference between the number of preset images and the number of landmark images included in the target classification may be calculated, and the difference may be determined as the target number of images to be generated. For example, if the number of the preset images is 3 and the number of the landmark images included in the target class is 1, the difference between the number of the preset images and the number of the landmark images included in the target class is 3-1=2, that is, the target number of the images to be generated is 2. Therefore, after the new landmark images are generated, the number of landmark images included in the classification corresponding to the isolated view angle reaches the preset number of images, and the number of landmark images included in each classification is more balanced.
And B, performing image augmentation processing based on landmark images included in the target classification, and generating new landmark images of a target number of target landmarks.
The above step B may refer to the image augmentation method in the foregoing embodiment, and only the number of the generated new landmark images is the target number, which is not described herein. In this way, aiming at the target classification corresponding to the isolated view angle, after the image augmentation processing is carried out on the landmark images included in the target classification, new landmark images of the target number of target landmarks can be obtained, and the number of landmark images included in the target classification corresponding to the isolated view angle is increased, so that the number of images belonging to each image acquisition view angle is more balanced.
In another embodiment of the present invention, a product of an average value of the number of landmark images in each of the classifications and a preset coefficient may be calculated, a difference between the product and the number of landmark images included in the target classification is determined as a target number of images to be generated, and image augmentation processing is performed based on the landmark images included in the target classification to generate new landmark images of the target number of target landmarks.
For example, when the preset coefficient is1, if the average value of the number of landmark images in each category is 5 and the number of landmark images included in the target category is 2, the target number of images to be generated is 5-2=3. When the preset coefficient is 2, if the average value of the number of landmark images in each category is 5 and the number of landmark images in the target category is 2, the target number of the images to be generated is 5*2-2=8. Because the number of the preset images is an average value, and twice the number of the preset images is the maximum value of the number of landmark images in each category, the difference value is determined to be the target number of images to be generated, so that the number of landmark images included in the category corresponding to the isolated view angle reaches the maximum value after the new landmark image is generated, and the number of landmark images included in each category is more balanced.
From the above, through the steps S101 to S104, the image augmentation process is performed on the landmark images included in the target classification corresponding to the isolated view angle, so that new images of the target landmarks are generated, and the number of images of the target landmarks is increased.
The method and the device for generating the landmark images by using the scheme provided by the embodiment of the invention are described below, and influence on landmark recognition accuracy after the map images are expanded is realized.
It can be understood that, for a landmark, an image to be identified at a certain image acquisition view angle of the landmark has high similarity with an existing image belonging to the image acquisition view angle, and the images are often matched with each other. Therefore, if there are fewer existing images belonging to a certain image acquisition view angle of the landmark, when landmark identification is performed on the images to be identified located in the image acquisition view angle of the landmark, the number of the matching images corresponding to the landmark is fewer, and error in landmark identification on the images to be identified is easy to occur.
The effect of the number of images of a certain image acquisition view on landmark recognition accuracy is specifically described below by way of example:
When the landmark identification is carried out on the image to be identified, the landmark to which the image to be identified belongs can be judged according to the confidence score of each landmark for the image to be identified, and the landmark with the highest confidence score is used as the landmark to which the image to be identified belongs. The confidence score is an average value of similarity of the matching images (number of matching images of the landmark/total number of existing images of the landmark+1).
For example, landmark recognition is described below by taking an example of an image of the image S to be recognized that actually belongs to the image acquisition view angle a of the landmark a.
If landmark a has 20 existing images, the image acquisition view angle a comprises 1 existing image, the similarity between the existing image and the image to be identified S is 80%, and landmark B has 20 existing images, the image acquisition view angle B comprises 10 existing images, and the similarity between the existing images and the image to be identified S is 60%. When the landmark is identified, if the landmark images with the similarity of more than 50% with the images to be identified are the matched images, the number of the matched images of the landmark A and the images to be identified is 1, the confidence score of the landmark A for the images to be identified is (1/20+1) ×80% = 0.84, the number of the matched images of the landmark B and the images to be identified is 6, and the confidence score of the landmark B for the landmarks to which the images to be identified belong is (10/20+1) ×60% = 0.9. It can be seen that the above confidence score of landmark B is higher than landmark a, so the image to be identified will be determined to belong to landmark B, whereas from the foregoing description it can be seen that the image to be identified S actually belongs to landmark a, and that the landmark identification is incorrect.
As can be seen from the analysis of the above process, the existing images included in the image acquisition view angle a of the landmark a have higher similarity with the image S, but because the number of the existing images included in the image acquisition view angle a is too small, the confidence score of the landmark a is smaller than that of the landmark B, and thus the landmark identification of the image S to be identified is wrong.
When the scheme provided by the embodiment of the invention is applied to landmark image generation, since the number of existing images corresponding to the image acquisition view angle a is 1, the image acquisition view angle a can be determined to be an isolated view angle by setting the preset image number and the like, so that image augmentation processing is performed based on the landmark image corresponding to the image acquisition view angle a, for example, 4 new landmark images are generated based on the existing landmark image of the image acquisition view angle a. After the image augmentation process is performed, the landmark image corresponding to the isolated view angle a is augmented, when landmark recognition is performed based on the augmented landmark image, the number of matched images of the landmark a and the image to be recognized is 5, the confidence score of the landmark a for the image to be recognized is (5/20+1) ×80% =1, and is greater than the confidence score of the landmark B by 0.9, so that the image to be recognized is determined to belong to the landmark a, and as can be seen from the foregoing description, the image S to be recognized actually belongs to the landmark a, and the visible landmark recognition is correct.
In the above, when the landmark image is generated by applying the scheme provided by the embodiment of the invention, each landmark image of the target landmark is divided into each category according to the image acquisition view angle, and the target category corresponding to the isolated view angle is determined based on the number of the landmark images in each category, so that the new landmark image of the target landmark can be generated by performing image augmentation processing based on the landmark images included in the target category, and the image number of the target landmark is effectively improved.
In addition, it can be seen that when the scheme provided by the embodiment of the invention is applied to landmark image generation, instead of directly performing image augmentation processing on each landmark image of a target landmark to generate a new landmark image, only the landmark images included in the target classification are subjected to image augmentation processing to generate the new landmark image, so that the newly generated landmark image belongs to the target classification, and the target classification corresponds to an isolated view angle, and the number of landmark images belonging to the isolated view angle is increased, and the number of landmark images belonging to each image acquisition view angle is relatively balanced because the number of images belonging to other image acquisition view angles is unchanged. On the basis, after the landmark images of the isolated view angles are added, when the landmark recognition is carried out on the image to be recognized, even if the image to be recognized is actually the image belonging to the isolated view angles, the situation that the landmark images belonging to the isolated view angles are fewer and the landmark recognition to the image to be recognized belongs to errors is avoided, so that the accuracy of the landmark recognition can be improved.
Since the image classification can be implemented in various ways in the process of generating a new landmark image, the landmark image generation process will be described below in connection with one of the image classification ways.
Referring to fig. 2, fig. 2 is a flowchart of a second landmark image generating method according to an embodiment of the present invention, where the method includes the following steps S201 to S206.
Step S201, each landmark image of the target landmark is obtained.
The step S201 is the same as the step S101 in the embodiment shown in fig. 1, and will not be repeated here.
Step S202, global image characteristics of each landmark image are obtained.
Because the global image features of landmark images belonging to the same image acquisition view angle tend to have higher similarity, the landmark images can be divided into classifications corresponding to different image acquisition view angles based on the global image features of the landmark images of the target landmark.
Specifically, the global image feature of each landmark image can be obtained in the following manner.
In a first embodiment, the global image features extracted from the pre-trained landmark recognition model may be obtained. When the landmark recognition model performs landmark recognition on a landmark image, global image features of the landmark image are extracted through a feature extraction layer, then the global image features are used as input features of a full connection layer, and finally a classification result of the landmark image is output by the full connection layer. From the above, it is possible to obtain all the global image features extracted by the landmark recognition model in the feature extraction stage, and then determine the global image features corresponding to the landmark image as the global image features of the landmark image.
In the second embodiment, an image feature database storing the global image features of the landmark images existing in the respective landmarks may be established in advance, and in this case, the global image features may be read from the image feature database established in advance.
In the third embodiment, the global image features may be extracted based on feature extraction algorithms such as an edge extraction operator, a texture feature extraction algorithm, a convolutional neural network algorithm, and the like.
And step S203, clustering the obtained global image features to obtain feature clusters.
Specifically, the manner of obtaining the feature clusters is described below by taking three clustering manners as examples.
In a first embodiment, the obtained global image features may be clustered using a hierarchical clustering algorithm, for example, a BIRCH (Balanced Iterative Reducing and Clustering Using Hierarchies, balanced iteration protocol and clustering algorithm using hierarchical method), a CURE (Clustering Using Representative, clustering algorithm using feature points), or a Chameleon (Chameleon) algorithm, where the above feature clusters are generated after the clustering is completed.
In a second embodiment, a K-Means algorithm may be used to cluster the obtained global image features, and the feature clusters are generated after the clustering is completed.
In a third embodiment, the obtained global image features may be clustered using DBSCAN (Density-Based Spatial Clustering of Applications with Noise, a clustering method based on noise), values of epsilon-neighborhood and min_samples (minimum sample number) are preset, and the feature clusters are generated after the clustering is completed.
Step S204, dividing landmark images corresponding to global image features belonging to the same feature cluster into categories corresponding to image acquisition view angles.
Because the global image features included in the same feature cluster are similar features, the global image features correspond to landmark images, namely similar landmark images, and the similar landmark images often belong to the same image acquisition view angle, the feature cluster can be considered to correspond to the image acquisition view angle, and then the landmark images corresponding to the global image features belonging to the same feature cluster can be classified into classifications corresponding to one image acquisition view angle.
Step S205, selecting the classification that the number of landmark images in each classification is smaller than the number of preset images as the target classification corresponding to the isolated view angle.
And S206, performing image augmentation processing based on the landmark images included in the target classification, and generating a new landmark image of the target landmark.
Steps S205 and S206 are the same as steps S103 and S104 in the embodiment shown in fig. 1, and are not described here again.
Therefore, the global image features of the landmark images belonging to the same image acquisition view angle are often high in similarity, so that the global image features of the landmark images can be clustered, the global image features with high similarity are generated into a feature cluster as a result of clustering, and at the moment, the landmark images corresponding to the global image features under the feature cluster can be understood as images belonging to the same image acquisition view angle. Therefore, landmark images corresponding to global image features belonging to the same feature cluster are divided into classifications corresponding to one image acquisition view angle, and each landmark image can be accurately classified according to the image acquisition view angle.
In one embodiment of the invention, after a new landmark image of a target landmark is generated, global image features of the new landmark image can be obtained, and then the landmark image of the target landmark and the global image features of the new landmark image are stored in an image feature database, so that when the existing landmark recognition scheme adopts a feature comparison mode to recognize the landmark, the image features of the existing landmark image can be directly read from the image feature database, and the execution of the landmark recognition scheme is facilitated. The specific obtaining manner of the global image feature of the new landmark image may refer to the global image feature obtaining manner mentioned in the foregoing step S202, which is not described herein.
Corresponding to the landmark image generation method, the embodiment of the invention also provides a landmark image generation device.
Referring to fig. 3, fig. 3 is a schematic structural diagram of a first landmark image generating apparatus according to an embodiment of the present invention, where the apparatus includes the following modules 301-304.
An image obtaining module 301, configured to obtain each landmark image of the target landmark;
The image classification module 302 is configured to classify landmark images according to an image acquisition view angle;
A target classification determining module 303, configured to select, as a target classification corresponding to an isolated view angle, a classification in which the number of landmark images in each classification is smaller than the number of preset images;
the image generating module 304 is configured to perform image augmentation processing based on the landmark images included in the target classification, and generate a new landmark image of the target landmark.
In the above, when the landmark image is generated by applying the scheme provided by the embodiment of the invention, each landmark image of the target landmark is divided into each category according to the image acquisition view angle, and the target category corresponding to the isolated view angle is determined based on the number of the landmark images in each category, so that the new landmark image of the target landmark can be generated by performing image augmentation processing based on the landmark images included in the target category, and the image number of the target landmark is effectively improved.
In addition, it can be seen that when the scheme provided by the embodiment of the invention is applied to landmark image generation, instead of directly performing image augmentation processing on each landmark image of a target landmark to generate a new landmark image, only the landmark images included in the target classification are subjected to image augmentation processing to generate the new landmark image, so that the newly generated landmark image belongs to the target classification, and the target classification corresponds to an isolated view angle, and the number of landmark images belonging to the isolated view angle is increased, and the number of landmark images belonging to each image acquisition view angle is relatively balanced because the number of images belonging to other image acquisition view angles is unchanged. On the basis, after the landmark images of the isolated view angles are added, when the landmark recognition is carried out on the image to be recognized, even if the image to be recognized is actually the image belonging to the isolated view angles, the situation that the landmark images belonging to the isolated view angles are fewer and the landmark recognition to the image to be recognized belongs to errors is avoided, so that the accuracy of the landmark recognition can be improved.
In one embodiment of the present invention, the image generating module 304 is specifically configured to determine, according to the number of landmark images included in the target classification and the preset number of images, a target number of images to be generated, and perform image augmentation processing based on the landmark images included in the target classification to generate new landmark images of the target number of the target landmarks.
In this way, aiming at the target classification corresponding to the isolated view angle, after the image augmentation processing is carried out on the landmark images included in the target classification, new landmark images of the target number of target landmarks can be obtained, and the number of landmark images included in the target classification corresponding to the isolated view angle is increased, so that the number of images belonging to each image acquisition view angle is more balanced.
In one embodiment of the present invention, the image generating module 304 is specifically configured to perform a target process on the landmark image included in the target classification, and generate a new landmark image of the target landmark based on the processed image, where the target process includes at least one of a random clipping process, a random affine process, and a random color parameter adjustment.
By performing at least one of a plurality of target processes on the landmark images included in the target classification, a new landmark image of a more varied target landmark can be generated based on the processed images.
Referring to fig. 4, fig. 4 is a schematic structural diagram of a second landmark image generating apparatus according to an embodiment of the present invention, where the apparatus includes the following modules 401-406.
An image obtaining module 401, configured to obtain each landmark image of the target landmark;
An image feature obtaining sub-module 402, configured to obtain global image features of each landmark image;
A feature clustering sub-module 403, configured to cluster the obtained global image features to obtain a feature cluster;
The image classification sub-module 404 is configured to divide landmark images corresponding to global image features belonging to the same feature cluster into classifications corresponding to one image acquisition view angle.
The target classification determining module 405 is configured to select, as a target classification corresponding to the isolated view angle, a classification in which the number of landmark images in each classification is smaller than the number of preset images.
An image generating module 406, configured to perform image augmentation processing based on the landmark images included in the target classification, and generate a new landmark image of the target landmark.
Therefore, the global image features of the landmark images belonging to the same image acquisition view angle are often high in similarity, so that the global image features of the landmark images can be clustered, the global image features with high similarity are generated into a feature cluster as a result of clustering, and at the moment, the landmark images corresponding to the global image features under the feature cluster can be understood as images belonging to the same image acquisition view angle. Therefore, landmark images corresponding to global image features belonging to the same feature cluster are divided into classifications corresponding to one image acquisition view angle, and each landmark image can be accurately classified according to the image acquisition view angle.
The embodiment of the invention also provides an electronic device, as shown in fig. 5, which comprises a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502 and the memory 503 complete communication with each other through the communication bus 504,
A memory 503 for storing a computer program;
the processor 501 is configured to implement the landmark image generating method provided in the above method embodiment when executing the program stored in the memory 503.
The communication bus mentioned by the above terminal may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, abbreviated as PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, abbreviated as EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the terminal and other devices.
The memory may include random access memory (Random Access Memory, RAM) or may include non-volatile memory (non-volatile memory), such as at least one disk memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The Processor may be a general-purpose Processor, including a central processing unit (Central Processing Unit, CPU), a network Processor (Network Processor, NP), a digital signal Processor (DIGITAL SIGNAL Processor, DSP), an Application-specific integrated Circuit (ASIC), a Field-Programmable gate array (FPGA) or other Programmable logic device, discrete gate or transistor logic device, or discrete hardware components.
In yet another embodiment of the present invention, there is further provided a computer readable storage medium having stored therein a computer program which, when executed by a processor, implements the landmark image generating method according to any one of the above embodiments.
In yet another embodiment of the present invention, a computer program product containing instructions that, when run on a computer, cause the computer to perform the landmark image generation method as set forth in any one of the above embodiments is also provided.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk Solid STATE DISK (SSD)), etc.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for apparatus, electronic devices, computer readable storage medium embodiments, since they are substantially similar to method embodiments, the description is relatively simple, and relevant references are made to the partial description of method embodiments.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims (10)
1. A method of landmark image generation, the method comprising:
Obtaining each landmark image of a target landmark;
classifying landmark images according to image acquisition visual angles;
selecting the classification of which the number of landmark images in each classification is smaller than the number of preset images as a target classification corresponding to an isolated view angle;
And performing image augmentation processing based on landmark images included in the target classification to generate a new landmark image of the target landmark, wherein the landmark image of the target landmark is used for matching with an image to be identified, and determining the possibility that the target landmark is the landmark corresponding to the image to be identified according to the number of the matched images and a first similarity, and the first similarity is the similarity between the matched image and the landmark image of the target landmark.
2. The method of claim 1, wherein classifying the landmark images by image acquisition view angle comprises:
Obtaining global image features of each landmark image;
clustering the obtained global image features to obtain feature clusters;
and dividing landmark images corresponding to the global image features belonging to the same feature cluster into classifications corresponding to image acquisition visual angles.
3. The method according to claim 1 or 2, wherein the performing image augmentation processing based on the landmark image included in the target classification to generate a new landmark image of the target landmark includes:
Determining the target number of images to be generated according to the number of landmark images included in the target classification and the preset image number;
And performing image augmentation processing based on landmark images included in the target classification, and generating new landmark images of the target number of target landmarks.
4. The method according to claim 1 or 2, wherein the performing image augmentation processing based on the landmark image included in the target classification to generate a new landmark image of the target landmark includes:
And performing target processing on the landmark images included in the target classification, and generating a new landmark image of the target landmark based on the processed images, wherein the target processing comprises at least one of random clipping processing, random affine processing and random color parameter adjustment.
5. A landmark image generation apparatus, the apparatus comprising:
the image acquisition module is used for acquiring each landmark image of the target landmark;
the image classification module is used for classifying landmark images according to the image acquisition visual angles;
The target classification determining module is used for selecting the classification that the number of landmark images in each classification is smaller than the number of preset images as the target classification corresponding to the isolated view angle;
The image generation module is used for carrying out image augmentation processing on the basis of landmark images included in the target classification to generate a new landmark image of the target landmark, wherein the landmark image of the target landmark is used for being matched with an image to be identified, the possibility that the target landmark is the landmark corresponding to the image to be identified is determined according to the number of the matched images and first similarity, and the first similarity is the similarity between the matched image and the landmark image of the target landmark.
6. The apparatus of claim 5, wherein the image classification module comprises:
The image feature obtaining sub-module is used for obtaining global image features of all landmark images;
the feature clustering sub-module is used for clustering the obtained global image features to obtain feature clusters;
And the image classification sub-module is used for dividing landmark images corresponding to the global image features belonging to the same feature cluster into classifications corresponding to one image acquisition view angle.
7. The apparatus of claim 5 or 6, wherein the device comprises a plurality of sensors,
The image generation module is specifically configured to determine the number of objects to be generated according to the number of landmark images included in the object classification and the preset number of images, and perform image augmentation processing based on the landmark images included in the object classification to generate new landmark images of the number of objects.
8. The apparatus of claim 5 or 6, wherein the device comprises a plurality of sensors,
The image generation module is specifically configured to perform target processing on landmark images included in the target classification, and generate a new landmark image of the target landmark based on the processed images, where the target processing includes at least one of random clipping processing, random affine processing, and random color parameter adjustment.
9. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
A memory for storing a computer program;
a processor for carrying out the method steps of any one of claims 1-4 when executing a program stored on a memory.
10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the method steps of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210333684.1A CN114626483B (en) | 2022-03-30 | 2022-03-30 | A method and device for generating a landmark image |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210333684.1A CN114626483B (en) | 2022-03-30 | 2022-03-30 | A method and device for generating a landmark image |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114626483A CN114626483A (en) | 2022-06-14 |
| CN114626483B true CN114626483B (en) | 2025-04-25 |
Family
ID=81906206
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210333684.1A Active CN114626483B (en) | 2022-03-30 | 2022-03-30 | A method and device for generating a landmark image |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114626483B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117809062B (en) * | 2024-03-01 | 2024-06-07 | 腾讯科技（深圳）有限公司 | Landmark identification method, apparatus, device, storage medium and computer program product |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109145937A (en) * | 2018-06-25 | 2019-01-04 | 北京达佳互联信息技术有限公司 | A kind of method and device of model training |
| CN111539451A (en) * | 2020-03-26 | 2020-08-14 | 平安科技（深圳）有限公司 | Sample data optimization method, device, equipment and storage medium |
Family Cites Families (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8676001B2 (en) * | 2008-05-12 | 2014-03-18 | Google Inc. | Automatic discovery of popular landmarks |
| US9740710B2 (en) * | 2014-09-02 | 2017-08-22 | Elekta Inc. | Systems and methods for segmenting medical images based on anatomical landmark-based features |
| CN109579825B (en) * | 2018-11-26 | 2022-08-19 | 江苏科技大学 | Robot positioning system and method based on binocular vision and convolutional neural network |
| CN110443281B (en) * | 2019-07-05 | 2023-09-26 | 重庆信科设计有限公司 | Text classification self-adaptive oversampling method based on HDBSCAN (high-density binary-coded decimal) clustering |
| CN111079733A (en) * | 2019-12-13 | 2020-04-28 | 哈尔滨工程大学 | Data augmentation method based on underwater imaging environment model |
| US11151732B2 (en) * | 2020-01-16 | 2021-10-19 | Siemens Healthcare Gmbh | Motion correction of angiography images for 3D reconstruction of coronary arteries |
| CN112613397B (en) * | 2020-12-21 | 2022-11-29 | 中国人民解放军战略支援部队航天工程大学 | Construction method of training sample set for multi-view optical satellite remote sensing image target recognition |
| CN114022706A (en) * | 2021-10-29 | 2022-02-08 | 际络科技(上海)有限公司 | Method, device and equipment for optimizing image classification model and storage medium |
-
2022
- 2022-03-30 CN CN202210333684.1A patent/CN114626483B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109145937A (en) * | 2018-06-25 | 2019-01-04 | 北京达佳互联信息技术有限公司 | A kind of method and device of model training |
| CN111539451A (en) * | 2020-03-26 | 2020-08-14 | 平安科技（深圳）有限公司 | Sample data optimization method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114626483A (en) | 2022-06-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113128610B (en) | Industrial part pose estimation method and system | |
| CN112541395A (en) | Target detection and tracking method and device, storage medium and electronic device | |
| CN111222395A (en) | Target detection method and device and electronic equipment | |
| CN110909724B (en) | A thumbnail generation method for multi-target images | |
| CN104616247B (en) | A kind of method for map splicing of being taken photo by plane based on super-pixel SIFT | |
| CN112581483B (en) | Self-learning-based plant leaf vein segmentation method and device | |
| CN110636248B (en) | Target tracking method and device | |
| CN108629286A (en) | A kind of remote sensing airport target detection method based on the notable model of subjective perception | |
| CN113627481A (en) | Multi-model combined unmanned aerial vehicle garbage classification method for smart gardens | |
| WO2010043954A1 (en) | Method, apparatus and computer program product for providing pattern detection with unknown noise levels | |
| CN119048344B (en) | Remote sensing image stitching method, device, computer equipment and medium | |
| TWI803243B (en) | Method for expanding images, computer device and storage medium | |
| CN107609562B (en) | Scale space feature detection method based on SIFT algorithm | |
| CN116433887B (en) | Building rapid positioning method based on artificial intelligence | |
| CN114626483B (en) | A method and device for generating a landmark image | |
| CN111275616B (en) | Low-altitude aerial image splicing method and device | |
| CN114140581B (en) | Automatic modeling method, device, computer equipment and storage medium | |
| CN111209819A (en) | Rotation-invariant face detection method, system equipment and readable storage medium | |
| CN119359937A (en) | Building boundary recognition method based on real-scene 3D technology and multi-source fusion data | |
| CN114896134B (en) | Metamorphic testing method, device and equipment for target detection model | |
| CN116612382A (en) | Urban remote sensing image target detection method and device | |
| CN116309821A (en) | A UAV localization method based on heterogeneous image registration | |
| CN115017968A (en) | A classification and extraction method of complex buildings based on U-Net model | |
| CN115690631A (en) | Unmanned aerial vehicle target identification method based on deep learning | |
| Li et al. | Reliable and fast mapping of keypoints on large-size remote sensing images by use of multiresolution and global information |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |