CN112733578A - Vehicle weight identification method and system - Google Patents

Abstract

The embodiment of the invention provides a vehicle weight recognition method and a vehicle weight recognition system, which adopt a method of combining the overall characteristics of a vehicle with the characteristics of salient regions, consider the salient regions of lamps, logos, windows, labels, pendants and the like of the vehicle, determine whether any two vehicle images to be compared belong to the same vehicle, can improve the effect of vehicle weight recognition and ensure the accuracy of vehicle weight recognition.

Description

Vehicle re-identification method and system

technical field

The present invention relates to the technical field of vehicle re-identification, and more particularly, to a vehicle re-identification method and system.

Background technique

With the increasing number of vehicles, it becomes increasingly difficult to manage vehicles. In some specific cases, it is necessary to track the trajectory of one or more vehicles traveling in the traffic flow. In specific implementation, the vehicle re-identification technology can be used to identify and search for the target vehicle from surveillance videos captured by multiple cameras at different positions, so as to realize the track tracking of the target vehicle. Among them, the vehicle re-identification technology refers to identifying the same target vehicle from surveillance videos collected at different times and at different locations.

In the actual road monitoring application scenario, affected by factors such as the light of the shooting environment, the angle of the camera, and the motion blur of the vehicle, the characteristics of the same vehicle in different shooting images may vary greatly; The differences in features exhibited in the captured images may be small. These problems bring great challenges to vehicle re-identification technology.

For the problem of vehicle re-identification, the existing technical solutions are mainly based on conventional neural network structures such as Convolutional Neural Networks (CNN), Deep Belief Networks (DBN) and other conventional neural network structures combined with different loss functions. After re-identifying the model, there is still room for improvement in model recognition accuracy.

SUMMARY OF THE INVENTION

To overcome the above problems or at least partially solve the above problems, embodiments of the present invention provide a vehicle re-identification method and system.

In a first aspect, an embodiment of the present invention provides a vehicle re-identification method, including:

Input any two vehicle images to be compared into the vehicle overall feature comparison module, and the vehicle overall feature comparison module determines the overall vehicle in the any two vehicle images to be compared based on the overall feature extraction model. similarity;

If the overall similarity is less than the first threshold, input the any two vehicle images to be compared into the salient region feature comparison module, and the salient region feature comparison module extracts the model based on the salient region features , determine the similarity of the salient regions of the vehicles in the any two vehicle images to be compared;

If the similarity of the saliency region is greater than or equal to the second threshold, it is determined that the two images of the vehicles to be compared belong to the same vehicle;

Wherein, the overall feature extraction model is constructed based on the first convolutional neural network, and is obtained by training the first-type sample vehicle images and the overall features of the first-type sample vehicles in the first-type sample vehicle images; the saliency The regional feature extraction model is constructed based on the second convolutional neural network and the attention mechanism network, and is trained by the second type of sample vehicle images.

Preferably, the saliency region similarity of the vehicles in the arbitrary two vehicle images to be compared is determined by the salient region feature comparison module based on the salient region feature extraction model, which specifically includes:

Based on the saliency region feature extraction model, respectively extract the salient region features of the vehicle in each of the two to-be-compared vehicle images in the to-be-compared vehicle image;

Calculate the cosine distance between the salient region features of the vehicles in the arbitrary two vehicle images to be compared, and use the cosine distance as the salient region similarity.

Preferably, the salient region features of the vehicle in each of the two to-be-compared vehicle images are respectively extracted based on the saliency region feature extraction model, specifically including:

For each image of the vehicle to be compared in any two images of the vehicle to be compared, based on the second convolutional neural network, extract the vehicle feature in the image of the vehicle to be compared, and input the vehicle feature into to the attention mechanism network;

Based on the attention mechanism network, a set of saliency region mask matrices of vehicles in the vehicle image to be compared is determined, and a set of saliency regions of vehicles in the vehicle image to be compared is determined; wherein the saliency The region mask matrix set includes a plurality of saliency region mask matrices, the saliency region set includes a plurality of saliency regions, and the saliency region mask matrix is in one-to-one correspondence with the saliency regions;

The saliency area set of the vehicle in the vehicle image to be compared is input to the second convolutional neural network, and the saliency area feature of each saliency area in the saliency area set is extracted.

Preferably, the determining the salient region mask matrix set of the vehicle in the vehicle image to be compared based on the attention mechanism network specifically includes:

Based on the parameter matrix of the attention mechanism network, the vehicle feature, and the mapping transformation function of the vehicle feature, a set of salient region mask matrices of the vehicle in the vehicle image to be compared is determined.

Preferably, each type of saliency region in the saliency region set corresponds to a preset weight; accordingly,

The determining of the set of saliency regions of the vehicle in the vehicle image to be compared specifically includes:

Each saliency region in the saliency region set is determined based on each saliency region mask matrix in the saliency region mask matrix set and a preset weight corresponding to each type of saliency region mask matrix.

Preferably, the calculating the similarity of the saliency area between the salient area features of the vehicle in any two images of the vehicle to be compared, specifically includes:

respectively calculating the local similarity between the saliency area features of each saliency area corresponding to the any two vehicle images to be compared;

The saliency region similarity is calculated based on all local similarities and preset weights corresponding to each type of saliency region.

Preferably, the set of mask matrices of the saliency region of the vehicle in the vehicle image to be compared is specifically determined by the following formula:

M=h(f(X))⊙F;

Wherein, M is the saliency area mask matrix set of the vehicle in the vehicle image to be compared, X is the image of the vehicle to be compared, f(X) is the vehicle feature in the image of the vehicle to be compared, F is the parameter matrix of the attention mechanism network, and h(f(X)) is the mapping transformation function of f(X).

In a second aspect, an embodiment of the present invention provides a vehicle re-identification system, including: an overall similarity determination subsystem, a saliency area similarity determination subsystem, and a vehicle judgment subsystem. in,

The overall similarity determination subsystem is used to input any two vehicle images to be compared into the vehicle overall feature comparison module, and the vehicle overall feature comparison module determines the arbitrary two to be compared based on the overall feature extraction model. Compare the overall similarity of vehicles in vehicle images;

The saliency area similarity determination subsystem is configured to input the any two vehicle images to be compared to the saliency area feature comparison module if the overall similarity is less than the first threshold, and the saliency area feature The comparison module determines the similarity of the salient regions of the vehicles in the any two vehicle images to be compared based on the salient region feature extraction model;

The vehicle judging subsystem is configured to judge that any two vehicle images to be compared belong to the same vehicle if the similarity of the salient region is greater than or equal to a second threshold;

Wherein, the overall feature extraction model is constructed based on the first convolutional neural network, and is obtained by training the first-type sample vehicle images and the overall features of the first-type sample vehicles in the first-type sample vehicle images; the saliency The regional feature extraction model is constructed based on the second convolutional neural network and the attention mechanism network, and is trained by the second type of sample vehicle images.

In a third aspect, an embodiment of the present invention provides an electronic device, including:

at least one processor, at least one memory, a communication interface, and a bus; wherein,

The processor, the memory, and the communication interface communicate with each other through the bus;

The memory stores program instructions executable by the processor, and the processor invokes the program instructions to execute the vehicle re-identification method provided by the first aspect.

In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to execute the vehicle provided in the first aspect Re-identification method.

The vehicle re-identification method and system provided by the embodiments of the present invention adopt a method of combining the overall characteristics of the vehicle with the characteristics of the salient area, and consider the salient areas of the vehicle, such as lights, vehicle logos, windows, stickers, and pendants. , to determine whether any two images of vehicles to be compared belong to the same vehicle, which can improve the effect of vehicle re-identification and ensure the accuracy of vehicle re-identification.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative efforts.

FIG. 1 is a schematic flowchart of a vehicle re-identification method according to an embodiment of the present invention;

2 is a schematic flowchart of the training and use of an overall feature extraction model in a vehicle overall feature comparison module according to an embodiment of the present invention;

3 is a schematic flowchart of training and use of a salient area feature extraction model in a salient area feature comparison module in a vehicle re-identification method provided by an embodiment of the present invention;

4 is a schematic overall flow diagram of a vehicle re-identification method provided by an embodiment of the present invention;

5 is a schematic structural diagram of a vehicle re-identification system according to an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Due to the influence of factors such as the light of the shooting environment, the angle of the camera, and the motion blur of the vehicle in the actual road monitoring application scenario, the characteristics shown in the same vehicle and different shooting images may vary greatly; The differences in features exhibited in the captured images may be small. The vehicle re-identification methods used in the prior art are all realized by constructing a model through a convolutional neural network, while the vehicle re-identification methods in the prior art can only realize the identification through the entire vehicle image, and cannot guarantee the accuracy and accuracy of the identification. . To this end, an embodiment of the present invention provides a vehicle re-identification method, the main goal of which is to reduce the influence of factors such as the light of the shooting environment, the angle of the camera, the interior decoration of the vehicle, and the motion blur of the vehicle, so as to improve the performance in practical application scenarios. Vehicle re-identification accuracy. For some models, although the overall features are similar, there are certain differences in the car logo area and lamp design; the same model, the window decorations, stickers, pendants, etc. of different cars are also different. In the embodiment of the present invention, salient areas such as lights, car logos, car windows, stickers, and pendants of the vehicle are fully utilized to accurately re-identify the vehicle and improve the recognition effect.

As shown in FIG. 1 , the vehicle re-identification method provided by the embodiment of the present invention includes:

S1, input any two vehicle images to be compared into the vehicle overall feature comparison module, and the vehicle overall feature comparison module determines the vehicle in the any two vehicle images to be compared based on the overall feature extraction model. the overall similarity;

S2, if the overall similarity is less than the first threshold, input any two vehicle images to be compared into a saliency area feature comparison module, and the saliency area feature comparison module is based on the saliency area feature extracting a model to determine the similarity of the salient regions of the vehicles in the any two images of the vehicles to be compared;

S3, if the similarity of the salient region is greater than or equal to a second threshold, determine that the two images of the vehicles to be compared belong to the same vehicle;

Wherein, the overall feature extraction model is constructed based on the first convolutional neural network, and is obtained by training the first-type sample vehicle images and the overall features of the first-type sample vehicles in the first-type sample vehicle images; the saliency The regional feature extraction model is constructed based on the second convolutional neural network and the attention mechanism network, and is trained by the second type of sample vehicle images.

Specifically, a vehicle re-identification method provided in the embodiment of the present invention is aimed at a set of vehicle images to be compared that need to be re-identified, and the same vehicle or different vehicles are identified from the set of images, that is, the judgment Whether the set of images belong to the same vehicle or to different vehicles. The execution body may be a local server or a cloud server, and the local server may be a computer or other device capable of executing the vehicle re-identification method provided in the embodiment of the present invention, which is not specifically limited in the embodiment of the present invention.

First, step S1 is performed, for any two vehicle images to be compared in a group of vehicle images to be compared that need to be re-identified, input any two images of the vehicle to be compared into the vehicle overall feature comparison module , the overall similarity of vehicles in any two vehicle images to be compared is determined by the vehicle overall feature comparison module based on the overall feature extraction model. The vehicle overall feature comparison module includes an overall feature extraction model, and the overall feature extraction model is constructed based on the first convolutional neural network. The first convolutional neural network may be a convolutional neural network (Convolutional Neural Networks, CNN) such as GoogLeNet, ResNet, and VGG. The overall feature extraction model can be obtained by training the first-type sample vehicle images and the overall features of the first-type sample vehicles in the first-type sample vehicle images, combined with the two loss functions of ArcLoss and TripletLoss. The first-type sample vehicle images refer to training The sample vehicle images used by the overall feature extraction model, the first-type sample vehicle images include the first-type sample vehicles, and the overall characteristics of the first-type sample vehicles are predetermined. The first-type sample vehicle images and the overall features of the first-type sample vehicles in the first-type sample vehicle images constitute the first-type training samples to train the overall feature extraction model.

When training the overall feature extraction model, although the accuracy of the model trained based on the TripletLoss loss function is high, the model convergence is slow. In order to reduce the model training time, the model is initially fitted based on the ArcLoss classification loss function, and the pre-trained model is obtained. . Then, based on the obtained pre-training model, combined with the TripletLoss loss function, the pre-training model is trained to obtain the overall feature extraction model.

FIG. 2 is a schematic flowchart of the training and use of the overall feature extraction model in the vehicle overall feature comparison module in the vehicle re-identification method provided in the embodiment of the present invention. Figure 2 includes two parts: model training and model use. For the model training part, the first type of training samples are input to the first convolutional neural network, and the two loss functions of ArcLoss and TripletLoss are used successively to train the first convolutional neural network. , the first convolutional neural network after training is obtained, which is the overall feature extraction model. For the model usage part, input any two images X and Y to be compared to the overall feature extraction model, and the overall feature extraction model determines the overall characteristics of the vehicle in each image to be compared.

The vehicle overall feature comparison module determines the overall similarity of the vehicles in any two to-be-compared vehicle images X and Y according to the overall characteristics of the vehicles in each to-be-compared image. The method for calculating the overall similarity may specifically be determined by calculating the cosine distance between the overall features of the vehicles in X and Y, the Pearson correlation coefficient, the Jaccard similarity coefficient, and the like.

After determining the overall similarity, the vehicle overall feature comparison module determines the magnitude relationship between the overall similarity and the _first threshold D1. The first threshold D ₁ refers to the minimum overall similarity when the two vehicle images belong to the same vehicle. The execution subject of the vehicle re-identification method provided in the embodiment of the present invention may also be executed by other third-party devices. This is not specifically limited in this embodiment of the present invention.

Then step S2 is executed to determine the magnitude relationship between the overall similarity of the vehicles in any two images X and Y of the vehicles to be compared obtained in step S1 and the _first threshold D1. The first threshold D ₁ refers to the minimum overall similarity when the two vehicle images belong to the same vehicle. When the overall similarity of the vehicles in any two images of vehicles to be compared X and Y is greater than or equal to the first threshold D ₁ , it means that the overall similarity of the vehicles in any of the two images of vehicles to be compared X and Y is high, belonging to the same vehicle.

For vehicle images with small differences in overall features, there is a high probability of misjudgment only by comparing the overall features. Although some models have similar overall characteristics, there are certain differences in the brand logo area and lamp design; for the same model, there are also differences in the window decorations, labels, and pendants of different cars. In order to further reduce the misrecognition rate, this solution makes full use of salient areas such as lights, logos, windows, stickers, and pendants of vehicles to assist in verifying vehicle recognition results and improve the recognition effect. When the overall similarity of the vehicles in any two vehicle images X and Y to be compared is less than the _first threshold D1, further judgment is made. Input any two vehicle images X and Y to be compared to the saliency area feature comparison module, and the salient area feature comparison module determines the vehicle in any two vehicle images to be compared based on the salient area feature extraction model. salient region similarity. Among them, the salient region feature comparison module includes a salient region feature extraction model, and the salient region feature extraction model is constructed based on the second convolutional neural network and the attention mechanism network. The second convolutional neural network may be a convolutional neural network (Convolutional Neural Networks, CNN) such as GoogLeNet, ResNet, and VGG. The saliency region feature extraction model can be obtained by training the second-class sample vehicle images combined with loss functions such as ArcLoss or TripletLoss. The second type of sample vehicle image refers to the sample vehicle image used for training the saliency region feature extraction model. The second type of sample vehicle image contains the second type of sample vehicle, and the salient region feature of the second type of sample vehicle is predetermined. . The second type of samples The vehicle images constitute the second type of training samples to train the saliency region feature extraction model.

FIG. 3 is a schematic flowchart of the training and use of the salient area feature extraction model in the salient area feature comparison module in the vehicle re-identification method provided in the embodiment of the present invention. Figure 3 includes two parts: model training and model use. For the model training part, the second type of training samples are input to the second convolutional neural network, and the second convolutional neural network performs feature extraction on the second type of training samples, and the extracted After the feature is mapped and transformed, it is used as the input of the attention mechanism network for parameter calculation. The attention mechanism network outputs the saliency region mask matrix set, which acts on the second type of training samples to obtain the saliency region set, and then the saliency region set is input. In the second convolutional neural network, the salient region features are obtained by combining the fully connected layer. The salient region features are combined with loss functions such as TripletLoss or ArcLoss to train the salient region feature extraction model, and the parameter matrix of the salient region feature extraction model is obtained. The saliency region feature extraction model obtains salient region features that are beneficial to vehicle recognition based on the attention mechanism, and can achieve end-to-end training without adding additional saliency region annotation data.

For the model usage part, input any two images X and Y to be compared to the saliency region feature extraction model, and the saliency region feature extraction model determines the salient region features of the vehicle in each image to be compared. The salient region feature comparison module determines the similarity of the salient regions of the vehicles in any two images X and Y of the vehicles to be compared according to the salient region features of the vehicles in each image to be compared. The method for calculating the similarity of the saliency regions can be specifically determined by calculating the cosine distance, the Pearson correlation coefficient, the Jaccard similarity coefficient, and the like between the salient region features of the vehicles in X and Y.

After the salient region feature comparison module determines the salient region similarity, it determines the magnitude relationship between the salient region similarity and the second threshold D ₂ . Wherein, the second threshold D ₂ refers to the minimum saliency area similarity when the two vehicle images belong to the same vehicle. It should be noted that, the judgment process may also be performed by the execution subject of the vehicle re-identification method provided in the embodiment of the present invention, or may be performed by other third-party devices. This is not specifically limited in this embodiment of the present invention.

Finally, step S3 is performed, that is, when the similarity of the saliency area is greater than or equal to the second threshold, it is determined that any two vehicle images X and Y to be compared belong to the same vehicle; when the similarity of the saliency area is less than the second threshold, it is determined that any The two vehicle images X and Y to be compared belong to two different vehicles.

FIG. 4 is a schematic diagram of the overall flow of the vehicle re-identification method provided in the embodiment of the present invention. In FIG. 4, any two images of vehicles to be compared are input into the vehicle overall feature comparison module, and the overall feature comparison is performed. , and the size relationship between the overall similarity and the first threshold D ₁ is determined by the vehicle overall feature comparison module. If the overall similarity is greater than or equal to D ₁ , it is determined that any two images of the vehicles to be compared belong to the same vehicle, otherwise, Input any two vehicle images to be compared to the saliency area feature comparison module, and compare the salient area features, and the saliency area feature comparison module determines the saliency of the vehicle in any two vehicle images to be compared. The size relationship between the regional similarity and the second threshold D ₂ , if the salient regional similarity is greater than or equal to D ₂ , any two images of vehicles to be compared belong to the same vehicle, otherwise they belong to different vehicles.

The vehicle re-identification method provided in the embodiment of the present invention adopts the method of combining the overall characteristics of the vehicle with the characteristics of the salient area, and considers the salient areas of the vehicle, such as lights, vehicle logos, windows, stickers, and pendants, and determines any salient area. Whether the two vehicle images to be compared belong to the same vehicle can improve the effect of vehicle re-identification and ensure the accuracy of vehicle re-identification.

On the basis of the above embodiment, in the vehicle re-identification method provided in the embodiment of the present invention, the salient region feature comparison module determines the salient region feature extraction model based on the salient region feature extraction model to determine which of the two vehicle images to be compared is The similarity of the saliency region of the vehicle, including:

Based on the saliency region feature extraction model, respectively extract the salient region features of the vehicle in each of the two to-be-compared vehicle images in the to-be-compared vehicle image;

Calculate the cosine distance between the salient region features of the vehicles in the arbitrary two vehicle images to be compared, and use the cosine distance as the salient region similarity.

Specifically, in the embodiment of the present invention, any two vehicle images X and Y to be compared are respectively input into the salient region feature extraction model, and the salient region features of the vehicles in X and Y are respectively extracted by the salient region feature extraction model. , and the saliency region similarity between the salient region features of vehicles in X and Y is determined by calculating the cosine distance.

The formula for calculating the cosine distance is as follows:

Among them, d represents the cosine distance, and θ represents the angle between X and Y in the vector space.

In the embodiment of the present invention, the salient region features are extracted by the salient region feature extraction model, and the salient region similarity is determined by calculating the cosine distance between the salient region features, the calculation process is simple, and the calculated salient region similarity more accurate and reliable.

On the basis of the above embodiment, in the vehicle re-identification method provided in the embodiment of the present invention, the vehicle to be compared is extracted from each of the two images of the vehicle to be compared based on the feature extraction model of the salient region. The salient region features of the vehicle in the image, including:

For each image of the vehicle to be compared in any two images of the vehicle to be compared, based on the second convolutional neural network, extract the vehicle feature in the image of the vehicle to be compared, and input the vehicle feature into to the attention mechanism network;

Based on the attention mechanism network, a set of saliency region mask matrices of vehicles in the vehicle image to be compared is determined, and a set of saliency regions of vehicles in the vehicle image to be compared is determined; wherein the saliency The region mask matrix set includes a plurality of saliency region mask matrices, the saliency region set includes a plurality of saliency regions, and the saliency region mask matrix is in one-to-one correspondence with the saliency regions;

The saliency area set of the vehicle in the vehicle image to be compared is input to the second convolutional neural network, and the saliency area feature of each saliency area in the saliency area set is extracted.

Specifically, in the embodiment of the present invention, when extracting the salient region features of the vehicle in each of the two images of the vehicle to be compared among any two images of the vehicle to be compared, the Each vehicle image to be compared in the vehicle image is input to the saliency region feature extraction model, and the vehicle image X to be compared is taken as an example for description. The vehicle feature f(X) in the vehicle image X to be compared is extracted by the second convolutional neural network, and the vehicle feature f(X) is input to the attention mechanism network. Determine the saliency area mask matrix set M=[M ₁ , M ₂ ,..., M _n ] of the vehicle in the vehicle image X to be compared by the attention mechanism network, where n is the vehicle image X to be compared. The number of vehicle saliency regions, and the value of n can be set according to the model training effect in specific business application scenarios. M ₁ is the saliency region mask matrix corresponding to the first saliency region, M ₂ is the saliency region mask matrix corresponding to the second saliency region, and similarly M _n is the saliency region corresponding to the nth saliency region Sexual area mask matrix. The obtained saliency area mask matrix set M is applied to the vehicle image X to be compared, and then the saliency area set of the vehicle in the vehicle image X to be compared can be determined, that is, the saliency area mask matrix set M and the to-be-compared vehicle image X can be determined. The vehicle image X is convolved.

Then, the saliency region set of the vehicle in the vehicle image X to be compared is input into the second convolutional neural network, and the saliency region feature of each saliency region in the saliency region set is extracted.

In the embodiment of the present invention, the salient region feature is determined through the second convolutional neural network and the attention mechanism network, so that the extraction of the salient region feature is more convenient and feasible.

On the basis of the above-mentioned embodiment, in the vehicle re-identification method provided in the embodiment of the present invention, the set of salient region mask matrices of the vehicle in the vehicle image to be compared is determined based on the attention mechanism network, specifically include:

Based on the parameter matrix of the attention mechanism network, the vehicle feature, and the mapping transformation function of the vehicle feature, a set of salient region mask matrices of the vehicle in the vehicle image to be compared is determined.

Specifically, in the embodiment of the present invention, when determining the saliency region mask matrix set, it can be specifically determined according to the parameter matrix of the attention mechanism network, the vehicle feature, and the mapping transformation function of the vehicle feature. Among them, the parameter matrix of the attention mechanism network is determined during the training of the salient region feature extraction model, the vehicle feature is determined by the second convolutional neural network, and the mapping transformation function of the vehicle feature is a predetermined function, such as logistic function, sigmoid function Wait.

On the basis of the above embodiment, the mask matrix set of the saliency area of the vehicle in the vehicle image to be compared can be specifically determined by the following formula:

M=h(f(X))⊙F;

Wherein, M is the saliency area mask matrix set of the vehicle in the vehicle image to be compared, X is the image of the vehicle to be compared, f(X) is the vehicle feature in the image of the vehicle to be compared, F is the parameter matrix of the attention mechanism network, and h(f(X)) is the mapping transformation function of f(X). When the mapping transformation function is specifically a logistic function, there are:

where k is a constant.

On the basis of the above-mentioned embodiment, in the vehicle re-identification method provided in the embodiment of the present invention, each salient area in the saliency area set corresponds to a preset weight respectively;

Correspondingly, the determining the set of saliency regions of the vehicle in the vehicle image to be compared specifically includes:

Each saliency region in the saliency region set is determined based on each saliency region mask matrix in the saliency region mask matrix set and a preset weight corresponding to each saliency region mask matrix.

Specifically, in the embodiment of the present invention, since different categories of saliency regions such as lights, car logos, windows, stickers, and pendants have certain differences in their contribution to the vehicle re-identification task, the salient regions in the embodiments of the present invention have certain differences in their contribution. Each type of saliency region in the saliency region set corresponds to a preset weight respectively.

Therefore, when determining the saliency area set of the vehicle in the vehicle image to be compared, it can be specifically based on each saliency area mask matrix in the saliency area mask matrix set and the corresponding prediction of each type of saliency area mask matrix. Set the weights to determine each saliency region in the saliency region set. The preset weight is determined during training of the salient region feature extraction model and is a constant.

Each saliency region in the saliency region set can be specifically determined by the following formula:

X _iatt =α _i (X⊙Μ _i ),i∈n;

Wherein, _Xiatt is the ith salient region in the salient region set, and α _i is the preset weight corresponding to the category to which the ith salient region belongs.

During the training process of the saliency region feature extraction model, the n saliency regions [X _1att , X _2att ,...,X _natt ] of the extracted second-class sample vehicle images are input into the second convolutional neural network for The salient region feature extraction is based on the extracted salient region features and the preset weights corresponding to different categories of salient regions, combined with loss functions such as TripletLoss or ArcLoss for model training. The training process includes two parts: the parameter matrix of the second convolutional neural network and the parameter matrix of the attention mechanism network (including the preset weights corresponding to different categories of salient regions). The parameter matrix in the feature extraction model of the saliency region and the preset weights corresponding to the saliency regions of different categories.

In the embodiment of the present invention, when the saliency region feature extraction model is constructed and trained, the contribution of different categories of saliency regions in the vehicle re-identification task can be adaptively obtained, that is, the preset weight, and the multi-saliency with weight is adopted. The regional feature obtains the similarity of the saliency region, and realizes the end-to-end region feature extraction of the saliency region without adding other input data annotations.

On the basis of the above embodiment, in the vehicle re-identification method provided in the embodiment of the present invention, the calculation of the salient area similarity between the salient area features of the vehicle in the arbitrary two vehicle images to be compared, specifically include:

respectively calculating the local similarity between the saliency area features of each saliency area corresponding to the any two vehicle images to be compared;

The saliency region similarity is calculated based on all local similarities and preset weights corresponding to each type of saliency region.

Specifically, when calculating the similarity of saliency regions in the embodiment of the present invention, the local similarities may be calculated separately, and then the saliency regions are calculated according to all the local similarities and the preset weights corresponding to each type of saliency region similarity. The obtained saliency region similarity is a comprehensive saliency region similarity, as follows:

Among them, L is the _salient region similarity, and Li is the local similarity.

As shown in FIG. 5 , an embodiment of the present invention provides a vehicle re-identification system, including: an overall similarity determination subsystem 51 , a salient region similarity determination subsystem 52 , and a vehicle judgment subsystem 53 . in,

The overall similarity determination subsystem 51 is used to input any two vehicle images to be compared into the vehicle overall feature comparison module, and the vehicle overall feature comparison module determines the arbitrary two based on the overall feature extraction model. The overall similarity of the vehicles in the vehicle images to be compared;

The saliency area similarity determination subsystem 52 is configured to input the any two vehicle images to be compared to the saliency area feature comparison module if the overall similarity is less than the first threshold, and the saliency area The feature comparison module determines the similarity of the salient regions of the vehicles in the arbitrary two vehicle images to be compared based on the salient region feature extraction model;

The vehicle judging subsystem 53 is configured to judge that the any two vehicle images to be compared belong to the same vehicle if the similarity of the salient region is greater than or equal to the second threshold;

Wherein, the overall feature extraction model is constructed based on the first convolutional neural network, and is obtained by training the first-type sample vehicle images and the overall features of the first-type sample vehicles in the first-type sample vehicle images; the saliency The regional feature extraction model is constructed based on the second convolutional neural network and the attention mechanism network, and is trained by the second type of sample vehicle images.

Specifically, the functions of the subsystems in the vehicle re-identification system provided in the embodiments of the present invention correspond one-to-one with the operation procedures of the steps in the above method embodiments, and the achieved effects are also the same. For details, refer to the above methods Examples, which are not repeated in this embodiment of the present invention.

As shown in FIG. 6 , on the basis of the foregoing embodiment, an embodiment of the present invention provides an electronic device, including: a processor (processor) 601 , a memory (memory) 602 , a communications interface (Communications Interface) 603 and a bus 604 ;in,

The processor 601 , the memory 602 and the communication interface 603 communicate with each other through the bus 604 . The memory 602 stores program instructions that can be executed by the processor 601, and the processor 601 is used to call the program instructions in the memory 602 to execute the methods provided by the above method embodiments, for example, including: combining any two The vehicle images to be compared are all input into the vehicle overall feature comparison module, and the vehicle overall feature comparison module determines the overall similarity of the vehicles in the any two vehicle images to be compared based on the overall feature extraction model; if If the overall similarity is less than the first threshold, input the any two vehicle images to be compared into the salient region feature comparison module, and the salient region feature comparison module extracts the model based on the salient region feature, Determine the similarity of the saliency area of the vehicle in the any two images of the vehicle to be compared; if the similarity of the salient area is greater than or equal to the second threshold, it is determined that the two images of the vehicle to be compared belong to the same vehicle.

The logic instructions in the memory 602 may be implemented in the form of software functional units and may be stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other media that can store program codes .

On the basis of the foregoing embodiments, an embodiment of the present invention provides a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions cause the computer to execute the foregoing The method provided by each method embodiment, for example, includes: inputting any two images of the vehicle to be compared into the vehicle overall feature comparison module, and the vehicle overall feature comparison module determines the overall feature extraction model based on the overall feature extraction model. The overall similarity of the vehicle in any two images of the vehicle to be compared; if the overall similarity is less than the first threshold, input the two images of the vehicle to be compared into the saliency area feature comparison module, and the The salient region feature comparison module determines the similarity of the salient region of the vehicle in the arbitrary two images of the vehicles to be compared based on the feature extraction model of the salient region; if the similarity of the salient region is greater than or equal to the second threshold, Then, it is determined that the two images of the vehicles to be compared belong to the same vehicle.

The device embodiments described above are only illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed over multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

From the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and certainly can also be implemented by hardware. Based on this understanding, the above-mentioned technical solutions can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic A disc, an optical disc, etc., includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to perform the methods described in various embodiments or some parts of the embodiments.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be The technical solutions described in the foregoing embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims (10)

1. a vehicle re-identification method, is characterized in that, comprises: Input any two vehicle images to be compared into the vehicle overall feature comparison module, and the vehicle overall feature comparison module determines the overall vehicle in the any two vehicle images to be compared based on the overall feature extraction model. similarity; If the overall similarity is less than the first threshold, input the any two vehicle images to be compared into the salient region feature comparison module, and the salient region feature comparison module extracts the model based on the salient region features , determine the similarity of the salient regions of the vehicles in the any two vehicle images to be compared; If the similarity of the saliency region is greater than or equal to the second threshold, it is determined that the two images of the vehicles to be compared belong to the same vehicle; Wherein, the overall feature extraction model is constructed based on the first convolutional neural network, and is obtained by training the first-type sample vehicle images and the overall features of the first-type sample vehicles in the first-type sample vehicle images; the saliency The regional feature extraction model is constructed based on the second convolutional neural network and the attention mechanism network, and is trained by the second type of sample vehicle images. 2 . The vehicle re-identification method according to claim 1 , wherein the salient region feature comparison module determines, based on the salient region feature extraction model, which of the two vehicle images to be compared is determined. 3 . The similarity of the saliency region of the vehicle, including: Based on the saliency region feature extraction model, respectively extract the salient region features of the vehicle in each of the two to-be-compared vehicle images in the to-be-compared vehicle image; Calculate the cosine distance between the salient region features of the vehicles in the arbitrary two vehicle images to be compared, and use the cosine distance as the salient region similarity. 3 . The vehicle re-identification method according to claim 2 , wherein the extraction model based on the salient region feature extraction model respectively extracts the image of each vehicle to be compared in the arbitrary two images of the vehicle to be compared. 4 . The salient regional features of the vehicle, including: For each image of the vehicle to be compared in any two images of the vehicle to be compared, based on the second convolutional neural network, extract the vehicle feature in the image of the vehicle to be compared, and input the vehicle feature into to the attention mechanism network; Based on the attention mechanism network, a set of saliency region mask matrices of vehicles in the vehicle image to be compared is determined, and a set of saliency regions of vehicles in the vehicle image to be compared is determined; wherein the saliency The region mask matrix set includes a plurality of saliency region mask matrices, the saliency region set includes a plurality of saliency regions, and the saliency region mask matrix is in one-to-one correspondence with the saliency regions; The saliency area set of the vehicle in the vehicle image to be compared is input to the second convolutional neural network, and the saliency area feature of each saliency area in the saliency area set is extracted. 4 . The vehicle re-identification method according to claim 3 , wherein determining the saliency region mask matrix set of the vehicle in the vehicle image to be compared based on the attention mechanism network, specifically comprising: 5 . Based on the parameter matrix of the attention mechanism network, the vehicle feature, and the mapping transformation function of the vehicle feature, a set of salient region mask matrices of the vehicle in the vehicle image to be compared is determined. 5 . The vehicle re-identification method according to claim 4 , wherein each type of saliency area in the saliency area set corresponds to a preset weight; accordingly, The determining of the set of saliency regions of the vehicle in the vehicle image to be compared specifically includes: Each saliency region in the saliency region set is determined based on each saliency region mask matrix in the saliency region mask matrix set and a preset weight corresponding to each type of saliency region mask matrix. 6 . The vehicle re-identification method according to claim 5 , wherein the calculating the similarity of the salient regions between the salient region features of the vehicles in the any two images of the vehicles to be compared, specifically comprises: 6 . respectively calculating the local similarity between the salient region features of each salient region corresponding to the any two vehicle images to be compared; The saliency region similarity is calculated based on all local similarities and preset weights corresponding to each type of saliency region. 7. The vehicle re-identification method according to claim 4, wherein the set of salient region mask matrices of the vehicle in the vehicle image to be compared is specifically determined by the following formula: M=h(f(X))⊙F; Wherein, M is the saliency area mask matrix set of the vehicle in the vehicle image to be compared, X is the image of the vehicle to be compared, f(X) is the vehicle feature in the image of the vehicle to be compared, F is the parameter matrix of the attention mechanism network, and h(f(X)) is the mapping transformation function of f(X). 8. A vehicle re-identification system, comprising: The overall similarity determination subsystem is used to input any two vehicle images to be compared into the vehicle overall feature comparison module, and the vehicle overall feature comparison module determines the arbitrary two based on the overall feature extraction model. The overall similarity of the vehicles in the vehicle images to be compared; The saliency area similarity determination subsystem is used for inputting the any two vehicle images to be compared to the saliency area feature comparison module if the overall similarity is less than the first threshold, and the saliency area is determined by the saliency area The feature comparison module determines the similarity of the salient regions of the vehicles in the arbitrary two vehicle images to be compared based on the salient region feature extraction model; a vehicle judging module, configured to judge that any two vehicle images to be compared belong to the same vehicle if the similarity of the salient region is greater than or equal to a second threshold; Wherein, the overall feature extraction model is constructed based on the first convolutional neural network, and is obtained by training the first-type sample vehicle images and the overall features of the first-type sample vehicles in the first-type sample vehicle images; the saliency The regional feature extraction model is constructed based on the second convolutional neural network and the attention mechanism network, and is obtained by training the second type of sample vehicle images. 9. An electronic device, comprising: a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the program as in claims 1-7 when the processor executes the program The steps of any one of the vehicle re-identification methods. 10. A non-transitory computer-readable storage medium on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the vehicle re-identification method according to any one of claims 1-7 is implemented A step of.

Images