CN111899193B - Criminal investigation photographing system and method based on low-illumination image enhancement algorithm - Google Patents

Abstract

The present invention belongs to the field of image enhancement technology, and discloses a criminal investigation photography system and method based on a low-light image enhancement algorithm. The photography end uses a photography device to acquire image data and capture objects, and transmits the acquired image data to the enhancement end; the enhancement end uses a low-light image enhancement model based on an enhancement network module to generate an adversarial network to enhance the transmitted low-light image, and uses an adversarial network algorithm to perform image recognition, and labels the types that have not been recognized; at the same time, similar labels that have been recognized in the storage background are extracted for auxiliary recognition; the images that have completed recognition and enhancement are transmitted to the photography end and the storage background for output and storage respectively. The present invention has the best low-light image enhancement effect, high enhancement efficiency, and a system that can be applied to various complex scenes, and selects the algorithm technology that is most suitable for criminal investigation photography.

Description

A criminal investigation photography system and method based on low-light image enhancement algorithm

Technical Field

The present invention belongs to the technical field of image enhancement, and in particular relates to a criminal investigation photography system and method based on a low-illumination image enhancement algorithm.

Background technique

At present, as an important way in criminal investigation, the mastery of criminal investigation photography technology is very important for grasping clues of criminal cases. Criminal investigation photography technicians need to use professional equipment to accurately and objectively record the actual situation. These images provide scientific images and evidence for solving the case. At the same time, the famous "Skynet Monitoring System" also combines photography with recognition technology to complete related criminal tasks.

Prior Art 1 is an earlier article that uses deep learning methods to complete low-light enhancement tasks. It proves that stacked sparse denoising autoencoders trained on synthetic data can enhance and denoise low-light noisy images. The model training is based on image patches and uses sparsity regularized reconstruction loss as the loss function. The main contributions are as follows: (1) A training data generation method (i.e., gamma correction and adding Gaussian noise) is proposed to simulate low-light environments. (2) Two types of network structures are explored: (a) LLNet, which learns contrast enhancement and denoising simultaneously; (b) S-LLNet, which uses two modules to perform contrast enhancement and denoising in stages. (3) Experiments are conducted on real low-light images to prove the effectiveness of the model trained with synthetic data. (4) The network weights are visualized to provide insights about the learned features.

In the first approach, which uses LLNet for both operations, the autoencoder module consists of multiple layers of hidden units, where the encoder is trained via unsupervised learning and the decoder weights are transferred from the encoder, followed by error fine-tuning via back-propagation.

In method b, the comparative experiments are conducted in stages using two modules, LLNet with simultaneous contrast enhancement and denoising modules, and S-LLNet with sequential contrast enhancement and denoising modules.

Prior art 2 introduced CNN. The traditional multi-scale Retinex (MSR) method can be regarded as a feed-forward convolutional neural network with different Gaussian convolution kernels, and a detailed demonstration was given. Then, following the process of MSR, they proposed MSR-net, which directly learns the end-to-end mapping of dark images to bright images. The training data used high-quality images adjusted with PS and corresponding synthetic low-light images (randomly reducing brightness and contrast, and gamma correction). The loss function is the F-norm square of the error matrix with regularization, that is, the sum of squared errors. MSR-net includes three modules: multi-scale logarithmic transformation, convolution difference, and color restoration.

The existing technology 3 actually focuses on single image contrast enhancement (SICE), which targets the low contrast problem under underexposure and overexposure. Its main contributions are as follows: (1) A multi-exposure image dataset is constructed, including low-contrast images of different exposures and corresponding high-quality reference images. (2) A two-stage enhancement model is proposed. In the first stage, the weighted least squares (WLE) filtering method is used to decompose the original image into low-frequency components and high-frequency components, and then the two components are enhanced separately; in the second stage, the enhanced low-frequency and high-frequency components are fused, and then enhanced again to output the result. Since the enhancement result of the single-stage CNN is not satisfactory and there is a color cast phenomenon, this may be because the single-stage CNN is difficult to balance the enhancement effect of the smooth component and the texture component of the image, so it is designed into a two-stage network structure, in which the Decomposition step of the first stage of the model adopts the traditional method, and the Retinex-Net introduced later is implemented using CNN.

With the development of image recognition technology, face, body shape and other factors have been widely used as labels for identity recognition. Face recognition and other technologies provide a lot of convenience to the daily life of the present invention. In criminal investigation, the identity of the criminal is identified by face, body shape and other factors as identification features, and object identification is used to search for details that are easily overlooked by the human eye, thereby introducing image recognition technology in the field of criminal investigation photography. As an important way in criminal investigation, the mastery of criminal investigation photography technology is very important for grasping clues in criminal cases. Criminal investigation photography technicians need to use professional equipment to accurately and objectively record the actual situation. These images provide scientific images and evidence for solving the case. At the same time, the famous "Skynet Monitoring System" also combines photography with recognition technology to complete related criminal tasks.

However, the complexity of the display scene poses a huge "challenge" to the combination of criminal investigation photography and image recognition technology. The scene of criminal investigation may appear in various places, and the various noises brought by the diversity of places increase the difficulty of generating high-precision images. Therefore, enhancing the analysis of pictures with noise is the focus of current research because light and shadow have the widest impact on pictures and the greatest impact on picture quality. In addition, crime scenes in the field of criminal investigation appear more in places with few people and dark places, and the proportion of scenes appearing at night is much higher than that appearing during the day. However, in the current research results, there is no perfect means to deal with the above problems. Therefore, it is of great theoretical significance and practical application value to study the combination of low-light image enhancement and criminal investigation photography.

Although the above methods have partially achieved the enhancement of low-light images, it was found in the comparison of subsequent experimental links that their experimental results were far from satisfactory.

Through the above analysis, the problems and defects of the prior art are as follows:

(1) Existing low-light image enhancement algorithms do not address the impact of various types of noise generated by various complex scenes on the quality of generated images.

(2) Existing low-light image enhancement algorithms cannot effectively solve light and shadow problems.

(3) The existing technology lacks a systematic modeling method and cannot establish a low-light image enhancement system that can be applied to various scenarios.

The difficulty of solving the above problems and defects is:

(1) Complex scenes and the presence of multiple types of noise place high demands on the robustness (adaptability) of the enhancement module.

(2) The existence of light and shadow problems is difficult to solve through traditional brightness enhancement methods.

(3) Systematic modeling methods place high demands on enhancing the adaptability of algorithms.

The significance of solving the above problems and defects is:

(1) The scenes captured by criminal investigation photography are complex and diverse. Solving the first type of defects mentioned above can help criminal investigation photography technology adapt to various types of environments.

(2) Most scenes used in criminal investigation photography, such as dark corners and underground parking lots, have common lighting and shadow problems. Solving the second type of defect can help criminal investigation photography play an effective role in such scenes.

(3) A systematic modeling approach can help criminal investigation photography technology effectively connect to various types of applications.

Summary of the invention

In view of the problems existing in the prior art, the present invention provides a criminal investigation photography system based on a low-illumination image enhancement algorithm.

The present invention is implemented as follows: a criminal investigation photography system based on a low-light image enhancement algorithm, the criminal investigation photography system based on a low-light image enhancement algorithm comprises: a photography end, an enhancement end and a storage background;

The photographing end is connected to the enhancing end; it includes a photographing module and an output module; the photographing module is used to acquire image data and capture objects, and transmit the acquired image data to the enhancing end; the output module is used to output the enhanced image;

The enhancement end is connected to the photographing end and the storage background, and includes an enhancement module, a recognition module and a transmission module;

The enhancement module is used to enhance the transmitted low-light image by using a low-light image enhancement model based on the enhancement network module to generate an adversarial network; the recognition module is used to perform image recognition using an adversarial network algorithm; it is also used to extract similar tags that have been recognized in the background for auxiliary recognition, and it can also label types that have not been recognized; the transmission module is used to transmit the image that has completed recognition and enhancement to the output module in the camera end, and is also used to transmit the enhancement and recognition results and tags that have not been stored in the storage background to the background for storage;

The storage backend is connected to the enhancement end and is used to store enhancement, recognition results and recognition category labels.

Another object of the present invention is to provide a criminal investigation photography method based on a low illumination image enhancement algorithm applied to the criminal investigation photography system based on the low illumination image enhancement algorithm, the criminal investigation photography method based on the low illumination image enhancement algorithm comprising:

Step 1: The photographing end uses a photographing device to acquire image data and capture an object, and transmits the acquired image data to the enhancing end;

Step 2: The enhancement end uses the low-light image enhancement model based on the enhancement network module to generate an adversarial network to enhance the transmitted low-light image, and uses the adversarial network algorithm to perform image recognition and label the types that have not been recognized; at the same time, similar labels that have been recognized in the storage background are extracted for auxiliary recognition;

Step three: transmit the enhanced image to the camera end and storage background for output and storage respectively.

Furthermore, in step 1, special settings of the hardware device functions will be made according to the different hardware devices at the photography end, such as the different characteristics of different cameras and the different objects to be captured. Its capture and transmission technology has been widely used in various types of smart devices.

The capture technology generally follows the following process:

Generate camera controller (various types of photography control systems) as the operation engine interface.

Initialize the camera device engine.

Perform asynchronous operations to start controlling the entire camera.

The asynchronous operation starts the camera and calls the camera observation system.

Start preparing for image capture using the camera observation system and set the engine to idle state.

When you need to take a picture, call the camera control system, execute the engineering commands in the engine, and then perform the operation.

Perform asynchronous image capture operations. After completion, call the image capture completion instruction in the camera engine to complete the operation.

Furthermore, in step 3, image transmission has been widely used in various types of smart devices. In the process of image transmission and storage, the present invention needs to complete image digitization processing to store the digital data of the image.

The general process of digital processing is as follows:

Sampling: The essence of sampling is how many points are used to describe an image. The quality of the sampling result is measured by the image resolution mentioned above.

Quantization: Quantization refers to the range of values used to represent each point after image sampling. The result of quantization is the total number of colors that the image can accommodate, which reflects the quality of sampling.

Compression coding: The amount of image data obtained after digitization is very huge, and coding technology must be used to compress its information. In a sense, coding compression technology is the key to achieving image transmission and storage.

The general compression coding method in the prior art is hybrid coding, that is, combining transform coding, motion estimation and motion compensation, and entropy coding to perform compression coding.

Transform coding: Eliminate intra-frame redundancy of images.

Motion estimation and motion compensation: remove inter-frame redundancy in images.

Entropy coding: further improves compression efficiency.

1) Transform coding

The function of transform coding is to transform the image signal described in the spatial domain into the frequency domain, and then encode the transformed coefficients. Generally speaking, images have strong spatial correlation, and transforming them into the frequency domain can achieve decorrelation and energy concentration.

2) Entropy Coding

Entropy coding is named because the average code length after coding is close to the entropy value of the source. Entropy coding is often implemented using variable length coding (VLC). Its basic principle is to assign short codes to symbols with a high probability of appearing in the source, and assign long codes to symbols with a low probability of appearing, thereby obtaining a shorter average code length statistically.

3) Motion Estimation and Motion Compensation

Motion estimation and motion compensation are effective means to eliminate the temporal correlation of image sequences.

Motion estimation technology generally divides the current input image into several small non-overlapping image sub-blocks. For example, if the size of a frame image is 1280x720, it is first divided into 40x45 non-overlapping image blocks of size 16x16 in a grid-like form, and then a most similar image block is found for each image block within a certain search window of the previous image or the next image. This search process is called motion estimation. By calculating the position information between the most similar image block and the image block, a motion vector can be obtained. In this way, during the encoding process, the block in the current image can be subtracted from the most similar image block pointed to by the motion vector of the reference image to obtain a residual image block. Since each pixel value in the residual image block is very small, a higher compression ratio can be obtained in compression coding. This subtraction process is called motion compensation.

Further, in step 2, the method for constructing a low-light image enhancement model based on an enhancement network module generating an adversarial network includes:

(1) Obtain high-quality image pairs as training datasets to train the generator network G;

(2) Randomly sample m low-light image pairs from the training dataset Where m represents the size of the training batch, I _x represents the low-light image, I _y represents the real-light image, and I _adv represents the input of the discriminator;

(3) The input of the discriminant network is fixed to I _adv = {0,0,…,0}, with a length of m;

(4) Minimize the overall loss of the generator network:

Loss _gen =ω _a L _a +ω _adv L _adv +ω _con L _con +ω _tv L _tv +ω _col L _col ;

(5) Train the discriminator network and randomly initialize the input of the discriminator network to I _adv = {1, 0, …, 0}, with a length of m;

(6) Randomly sample m low-light images from the training dataset

(7) Maximize the overall loss of the discriminator network:

Furthermore, the low-light image enhancement model based on the enhanced network module generating the adversarial network includes:

The low-light image enhancement model includes a generator, a discriminator and a loss function added with an enhancement network;

The generator with the enhanced network added is based on a fully convolutional network and consists of two parts: a plurality of residual blocks and a convolutional block; it is used to convert the input image as a whole into a similar picture in a new space;

The discriminator is used to simultaneously receive the image generated by the generator and the real image, and generate a true or false prediction value;

The loss function is: Loss = ω _a L _a + ω _adv L _adv + ω _con L _con + ω _tv L _tv + ω _col L _col ;

Among them, _La , _Ladv , _Lcon , _Ltv , _Lcol represent attention loss, adversarial loss, content loss, total variation loss, and color loss respectively, and _ωa , _ωadv , _ωcon , _ωtv , _ωcol represent the corresponding weights of their losses respectively.

Further, the enhanced network module includes:

The enhanced network module includes two 3x3 convolutional layers and a feature transformation layer; it is used to overcome the disadvantage of low contrast of the image and improve the details to enhance the image effect;

The first layer of the 3x3 convolutional layer is used for feature extraction, realizing the conversion from RGB channels to multiple features;

The first convolutional layer is followed by a feature transformation layer, which is a residual module; the feature transformation layer is used to perform complex feature transformation by connecting multiple residual units;

The feature transformation layer is followed by two convolution layers, and the convolution layers are used to restore the RGB image and realize the conversion of multiple features into RGB images; instance normalization and ReLU activation are performed after each convolution.

Furthermore, the enhanced network module includes four loss functions:

1) Content loss:

The content loss is defined based on the activation map produced by the ReLU layer of the pre-trained VGG-19 network; the content loss is the Euclidean distance between the feature representations of the enhanced image and the target image:

Among them, φ _i is the feature map obtained by the VGG-19 network after the i-th convolutional layer;

2) Total variation loss:

Among them, C, H, and W are the number of channels, height, and width of the enhanced image I _e , respectively. They are respectively to enhance the gradient of the image in the opposite direction of x and y;

3) Color loss:

_Lcolor ＝||δ(G( _Ix ))-δ( _Iy )|| ² ;

Among them, δ represents the Gaussian blur function, which is used to remove local details of the image;

4) Fighting Losses

Among them, D represents the discriminant network, G represents the generative network, I _x and I _y represent low-light images and natural-light images, respectively.

Further, the generator with the enhanced network added includes:

The generator consists of 1 convolution block, 4 residual blocks and 2 convolution blocks in sequence;

The four residual blocks are used to keep the height/width constant;

The generator performs instance regularization and ReLU activation after each convolution;

The last convolutional layer of the generator is tanh activated, and ReLU activation is used after each convolutional layer.

Furthermore, the discriminator includes 5 convolutional layers, 1 fully connected layer and 1 softmax layer;

The size of the convolution kernel of the convolution layer is reduced from 11 to 3, and the number of feature channels is increased from 3 to 192; it is used to gradually extract input features;

The fully connected layer and the softmax layer are used to predict the possibility that the extracted feature map comes from the generator or the real picture, and the result is a (Batch, P _true , P _false ) 3-tuple, where the values of P _true and P _false are both in the range of [0, 1].

Another object of the present invention is to provide a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and when the computer program is executed by the processor, the processor performs the following steps:

The photographing end uses photographic equipment to acquire image data and capture objects, and transmits the acquired image data to the enhancing end;

The enhancement end uses the low-light image enhancement model built based on the enhancement network module to generate adversarial networks to enhance the transmitted low-light images, and uses the adversarial network algorithm to perform image recognition and label the types that have not been recognized. At the same time, similar labels that have been recognized in the storage background are extracted for auxiliary recognition.

The images that have completed recognition and enhancement are transmitted to the camera end and the storage background for output and storage respectively.

Another object of the present invention is to provide a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the following steps:

The photographing end uses photographic equipment to acquire image data and capture objects, and transmits the acquired image data to the enhancing end;

The enhancement end uses the low-light image enhancement model built based on the enhancement network module to generate adversarial networks to enhance the transmitted low-light images, and uses the adversarial network algorithm to perform image recognition and label the types that have not been recognized. At the same time, similar labels that have been recognized in the storage background are extracted for auxiliary recognition.

The image after recognition enhancement is transmitted to the camera end and storage background for output and storage respectively

Combining all the above technical solutions, the advantages and positive effects of the present invention are as follows: the criminal investigation photography system based on the low-light image enhancement algorithm of the enhancement network module generative adversarial network of the present invention has the best low-light image enhancement effect for the enhancement network module generative adversarial network at present. The present invention provides a systematic criminal investigation photography effect enhancement system. Application of the low-light image enhancement algorithm based on the generative adversarial network of the present invention in the field of criminal investigation

The low-illumination image enhancement effect of the present invention is the best, the enhancement efficiency is high, the system can be applied to various complex scenes, and the algorithm technology most suitable for criminal investigation photography is selected.

The technical effects or experimental effects of the comparison include:

The Low Light Pairing Dataset (LOL) contains 500 low/normal light image pairs. It is a dataset of image pairs used for low-light enhanced shooting in real scenes. Most low-light images are collected by changing the exposure time and ISO, and the image pairs are aligned using a three-step method. The dataset contains images captured from various scenes, such as houses, campuses, clubs, and streets. Based on this dataset, the present invention produced 11,592 training set photos and 360 test set photos.

On the LOL dataset, the present invention compares the three methods of HE, SRIE, and DSLR. Since some methods cannot achieve denoising function, the present invention combines BM3D method for denoising, and then produces the final result. The quantitative results are shown in Table 3.

Table 3 Experimental results of the method of the present invention with HE, SRIE, and DSLR on the LOL dataset

The method of the present invention is the same as HE, SRIE, and DSLR in the visual effect of the LOL dataset. The qualitative results are shown in Figure 7 (a) low-light picture 1, 7 (b) prior art HE, 7 (c) prior art SRIE, 7 (d) prior art DSLR, 7 (e) the present invention, and 7 (f) the real photo after processing by the present invention.

As shown in Figure 8(a) low-light picture 2, Figure 8(b) prior art HE, Figure 8(c) prior art SRIE, Figure 8(d) prior art DSLR, Figure 8(e) the present invention, and Figure 8(f) real photo 2 after processing by the present invention.

By comparing the subjective visual effects of each algorithm in Figures 7 and 8, it can be seen that the HE algorithm has more content distortion and color distortion on the LOL dataset. For example, when enhancing the first picture, obvious background-irrelevant artifacts appear on the wall in the HE picture, and the overall image color is gray. When enhancing the second picture, the dark floor is obviously inconsistent with the light wooden floor in the real photo; the picture enhanced by the SRIE algorithm has more brightness distortion. It can be seen from the first and second pictures that its improvement in the brightness of low-light pictures is not enough to distinguish the image content; the DSLR algorithm can improve the brightness of the picture and retain certain content, but there are still problems of overexposure and local noise. For example, in the two enhanced pictures, the white wardrobe door in the first picture is obviously brighter, and the floor and wall brightness of the second picture are too high. At the same time, after enlarging the picture, it can be found that there is certain noise in the local area and the details are not smooth enough; while the picture enhanced by the method of the present invention has moderate brightness, avoiding the problem of overexposure, while the content is kept intact and less detail information is lost.

As can be seen from Table 3, the method of the present invention outperforms other methods in both indicators, indicating that the method of the present invention also has excellent performance and good robustness on real data sets. Compared with the traditional methods HE and SRIE, the present invention improves PSNR by 36.11% and 77.27% respectively, and improves SSIM by 74.06% and 86.76% respectively; compared with the current deep learning method DSLR, the present invention also improves PSNR by 9.69% and SSIM by 0.04%. It can be seen that the method of the present invention has a greatly improved effect compared with the traditional method, and also has certain advantages over the deep learning method.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

FIG1 is a schematic diagram of the structure of a criminal investigation photography system based on a low-light image enhancement algorithm provided by an embodiment of the present invention.

FIG. 2 is a flow chart of a criminal investigation photography method based on a low-light image enhancement algorithm provided in an embodiment of the present invention.

FIG3 is a schematic diagram of a generator network structure provided in an embodiment of the present invention.

FIG. 4 is a schematic diagram of a discriminator network structure provided by an embodiment of the present invention.

FIG5 is a diagram of a residual module network structure according to an embodiment of the present invention.

FIG. 6 is a diagram of an enhanced branch network structure provided by an embodiment of the present invention.

Figure 7(a) is a low-light picture 1 provided by an embodiment of the present invention, 7(b) is a prior art HE, 7(c) is a prior art SRIE, 7(d) is a prior art DSLR, 7(e) is the present invention, and 7(f) is a real photo 1 after being processed by the present invention.

FIG8(a) is a second low-light picture provided by an embodiment of the present invention, FIG8(b) is HE in the prior art, FIG8(c) is SRIE in the prior art, FIG8(d) is DSLR in the prior art, FIG8(e) is the present invention, and FIG8(f) is a second real photo processed by the present invention.

Detailed ways

In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

In view of the problems existing in the prior art, the present invention provides a criminal investigation photography system based on a low-illumination image enhancement algorithm. The present invention is described in detail below in conjunction with the accompanying drawings.

As shown in FIG1 , the criminal investigation photography system based on the low-light image enhancement algorithm provided by the embodiment of the present invention includes: a photography end, an enhancement end and a storage background;

The photographing end is connected to the enhancing end; it includes a photographing module and an output module; the photographing module is used to acquire image data and capture objects, and transmit the acquired image data to the enhancing end; the output module is used to output the enhanced image;

The enhancement end is connected to the photographing end and the storage background, and includes an enhancement module, a recognition module and a transmission module;

The enhancement module is used to enhance the transmitted low-light image by using a low-light image enhancement model based on the enhancement network module to generate an adversarial network; the recognition module is used to perform image recognition using an adversarial network algorithm; it is also used to extract similar tags that have been recognized in the background for auxiliary recognition, and it can also label types that have not been recognized; the transmission module is used to transmit the image that has completed recognition and enhancement to the output module in the camera end, and is also used to transmit the enhancement and recognition results and tags that have not been stored in the storage background to the background for storage;

The storage backend is connected to the enhancement end and is used to store enhancement, recognition results and recognition category labels.

As shown in FIG2 , the criminal investigation photography method based on the low-light image enhancement algorithm provided by the embodiment of the present invention includes:

S101, the photographing end uses a photographing device to acquire image data and capture an object, and transmits the acquired image data to the enhancing end;

S102, the enhancement end uses the low-light image enhancement model based on the enhancement network module to generate an adversarial network to enhance the transmitted low-light image, and uses the adversarial network algorithm to perform image recognition, and labels the types that have not been recognized; at the same time, similar labels that have been recognized in the storage background are extracted for auxiliary recognition;

S103, transmitting the image after recognition enhancement to the camera end and the storage background for output and storage respectively.

In step S101, special settings of the hardware device functions are performed according to the different hardware devices at the photography end, such as the different characteristics of different cameras, and the different objects to be captured. Its capture and transmission technology has been widely used in various types of smart devices.

The capture technology generally follows the following process:

Generate camera controller (various types of photography control systems) as the operation engine interface.

Initialize the camera device engine.

Perform asynchronous operations to start controlling the entire camera.

The asynchronous operation starts the camera and calls the camera observation system.

Start preparing for image capture using the camera observation system and set the engine to idle state.

When you need to take a picture, call the camera control system, execute the engineering commands in the engine, and then perform the operation.

Perform asynchronous image capture operations. After completion, call the image capture completion instruction in the camera engine to complete the operation.

In step S103, image transmission has been widely used in various types of intelligent devices. In the process of image transmission and storage, the present invention needs to complete image digitization processing to store the digital data of the image.

The general process of digital processing is as follows:

The essence of sampling is how many points are used to describe an image, and the quality of the sampling result is measured by the image resolution mentioned above.

Quantization Quantization refers to the range of values used to represent each point after the image is sampled. The result of quantization is the total number of colors that the image can accommodate, which reflects the quality of sampling.

The amount of image data obtained after compression coding is very huge, and coding technology must be used to compress its information. In a certain sense, coding compression technology is the key to realizing image transmission and storage.

In step S102, the method for constructing a low-light image enhancement model based on an enhancement network module generating an adversarial network provided by an embodiment of the present invention includes:

(1) Obtain high-quality image pairs as training datasets to train the generator network G;

(2) Randomly sample m low-light image pairs from the training dataset Where m represents the size of the training batch, I _x represents the low-light image, I _y represents the real-light image, and I _adv represents the input of the discriminator;

(3) The input of the discriminant network is fixed to I _adv = {0,0,…,0}, with a length of m;

(4) Minimize the overall loss of the generator network:

Loss _gen =ω _a L _a +ω _adv L _adv +ω _con L _con +ω _tv L _tv +ω _col L _col ;

(5) Train the discriminator network and randomly initialize the input of the discriminator network to I _adv = {1, 0, …, 0}, with a length of m;

(6) Randomly sample m low-light images from the training dataset

(7) Maximize the overall loss of the discriminator network:

The low-light image enhancement model based on the enhancement network module generative adversarial network provided by the embodiment of the present invention includes:

The low-light image enhancement model includes a generator, a discriminator and a loss function added with an enhancement network;

The generator with the enhanced network added is based on a fully convolutional network and consists of two parts: a plurality of residual blocks and a convolutional block; it is used to convert the input image as a whole into a similar picture in a new space;

The discriminator is used to simultaneously receive the image generated by the generator and the real image, and generate a true or false prediction value;

The loss function is: Loss = ω _a L _a + ω _adv L _adv + ω _con L _con + ω _tv L _tv + ω _col L _col ;

Among them, _La , _Ladv , _Lcon , _Ltv , _Lcol represent attention loss, adversarial loss, content loss, total variation loss, and color loss respectively, and _ωa , _ωadv , _ωcon , _ωtv , _ωcol represent the corresponding weights of their losses respectively.

The enhanced network module provided in the embodiment of the present invention includes:

The enhanced network module includes two 3x3 convolutional layers and a feature transformation layer; it is used to overcome the disadvantage of low contrast of the image and improve the details to enhance the image effect;

The first layer of the 3x3 convolutional layer is used for feature extraction, realizing from RGB channels to multiple features;

The first convolutional layer is followed by a feature transformation layer, which is a residual module; the feature transformation layer is used to perform complex feature transformation by connecting multiple residual units;

The feature transformation layer is followed by two convolution layers, and the convolution layers are used to restore the RGB image and realize the conversion of multiple features into RGB images; instance normalization and ReLU activation are performed after each convolution.

The enhanced network module provided in the embodiment of the present invention includes four loss functions:

1) Content loss:

The content loss is defined based on the activation map produced by the ReLU layer of the pre-trained VGG-19 network; the content loss is the Euclidean distance between the feature representations of the enhanced image and the target image:

Among them, φ _i is the feature map obtained by the VGG-19 network after the i-th convolutional layer;

2) Total variation loss:

Among them, C, H, and W are the number of channels, height, and width of the enhanced image I _e , respectively. They are respectively to enhance the gradient of the image in the opposite direction of x and y;

3) Color loss:

_Lcolor ＝||δ(G( _Ix ))-δ( _Iy )|| ² ;

Among them, δ represents the Gaussian blur function, which is used to remove local details of the image;

4) Fighting Losses

Among them, D represents the discriminant network, G represents the generative network, I _x and I _y represent low-light images and natural-light images, respectively.

The generator with an enhanced network provided in an embodiment of the present invention includes:

The generator consists of 1 convolution block, 4 residual blocks and 2 convolution blocks in sequence;

The four residual blocks are used to keep the height/width constant;

The generator performs instance regularization and ReLU activation after each convolution;

The last convolutional layer of the generator is tanh activated, and ReLU activation is used after each convolutional layer.

The discriminator provided in the embodiment of the present invention includes 5 convolutional layers, 1 fully connected layer and 1 softmax layer;

The size of the convolution kernel of the convolution layer is reduced from 11 to 3, and the number of feature channels is increased from 3 to 192; it is used to gradually extract input features;

The fully connected layer and the softmax layer are used to predict the possibility that the extracted feature map comes from the generator or the real picture, and the result is a (Batch, P _true , P _false ) 3-tuple, where the values of P _true and P _false are both in the range of [0, 1].

The technical effects of the present invention are further described below in conjunction with specific embodiments.

Embodiment 1:

4.1 Low-light image enhancement algorithm framework based on enhanced network module generative adversarial network

(1) Network structure

The network structure of low-light image enhancement is shown in Figures 3 and 4. In order to enhance the transmission of low-light position information in the network flow of low-light images, the generator adds an enhancement network. The enhancement network converts the input image as a whole into a similar picture in the new space. The attention network predicts the position mask of the low-light, which is the same size as the input image, and each pixel is a probability value between 0 and 1. Finally, the present invention combines the input image, the attention map and the converted input to form the final enhanced image. The discriminator can receive the picture generated by the generator and the real picture at the same time, and finally generate a true or false prediction value.

(i) The network structure details of the enhanced network are shown in Table 1. It is based on a fully convolutional network (FCN) and exploits the properties of convolutional neural networks, such as translation invariance and parameter sharing. The network consists of two parts: multiple residual blocks and convolutional blocks. There is one convolutional block at the beginning. The middle part contains 4 residual blocks, keeping the height/width constant, and performing instance normalization and ReLU activation after each convolution. Finally, there are 2 convolutional blocks. In addition to the residual block, the last convolution layer is tanh activated, and there are only ReLU activations after each convolution layer.

Table 1 Network structure details of the enhanced network

Convolutional Layer enter Convolution kernel/step size Output Conv0 I(100x100x3) 9x9x64/1 100x100x64 Res1 Conv0 3x3x64/1 100x100x64 Res2 Res1 3x3x64/1 100x100x64 Res3 Res2 3x3x64/1 100x100x64 Res4 Res3 3x3x64/1 100x100x64 Conv1 Res4 3x3x64/1 100x100x64 Conv2 Conv1 9x9x64/1 100x100x3

(ii) The details of the network structure of the discriminator network are shown in Table 2, which includes 5 convolutional layers, 1 fully connected layer and 1 softmax layer. Multiple convolutional layers are used to gradually extract input features. The size of the convolution kernel is reduced from 11 to 3, and the number of feature channels is increased from 3 to 192. For low-light images, due to the uneven distribution of light and the influence of noise, the image presents a large area of darkness and low light, resulting in a single local feature. At the beginning, a large receptive field is conducive to obtaining more information from the local feature map. As the number of channels increases, the features are gradually enriched. At this time, a small receptive field is conducive to extracting more detailed features of the image. The fully connected layer and the softmax layer are used to predict the possibility of the extracted feature map coming from the generator or the real picture. The result is a (Batch, P _true , P _false ) 3-tuple, and the values of P _true and P _false are both in the range of [0,1].

Table 2 Network structure details of the discriminator network

Convolutional Layer enter Convolution kernel/step size Output Conv0 I(100x100x3) 11x11x48/4 25x25x48 Conv1 Conv0 5x5x128/2 13x13x128 Conv2 Conv1 3x3x192/1 13x13x192 Conv3 Conv2 3x3x192/1 13x13x192 Conv4 Conv3 3x3x128/2 7x7x128 Fc Conv4 6272x1024 batch x1024 softmax Fc 1024x2 batch x2

(2) Loss Function

Since the input and target photos cannot be closely matched (i.e., pixel-to-pixel), i.e., different optical elements and sensors will cause specific local nonlinear distortions and aberrations, there will be a non-constant offset in the number of pixels between each image pair even after precise alignment. Therefore, standard per-pixel losses, except for perceptual quality indicators, are not applicable to the case of the present invention. In order to improve the overall image quality qualitatively and quantitatively, the present invention further proposes a new loss function:

Loss＝ _{ωaLa + ωadvLadv + ωconLcon + ωtvLtv + ωcolLcol} ( _4-1 )

Among them, _La , _Ladv , _Lcon , _Ltv , _Lcol represent attention loss, adversarial loss, content loss, total variation loss, and color loss respectively, and _ωa , _ωadv , _ωcon , _ωtv , _ωcol represent the corresponding weights of their losses respectively.

(3) Algorithm flow chart

After the training data is generated, the present invention uses high-quality image pairs to repeatedly train the GAN network of the present invention. In the stage of training the discriminator, the present invention randomly confuses a batch of generated samples with a batch of real samples, generates a batch of confused samples and re-inputs them as the discriminator. The discriminator attempts to identify real and fake images, so the discriminator is trained, which is equivalent to the process of maximizing the discriminant loss. The process of training the generator network is to minimize formula 3-1 to ensure that the generated image has the least loss in all aspects compared to the real image, and the generated effect is realistic.

To express the entire algorithm flow more concisely and clearly, G and D represent the generator network and the discriminator network respectively, and the size of a batch during training is m. For specific details, refer to the following algorithm flow:

4.2 Enhanced Network Module

This module is used as a backbone network to overcome the disadvantages of low contrast in images and improve details to achieve the effect of enhancing images. Considering that there is less feature information in low-light areas of low-light images and there may be noise interference, the present invention uses residual connections to construct basic residual modules to deepen the number of network layers, improve the network's modeling capabilities for low-light image enhancement, and avoid feature loss due to network deepening. Therefore, the present invention uses a residual module as the feature transformation layer in the enhancement network, as shown in Figure 5.

As shown in Figure 5, the module contains 2 layers of 3x3 ordinary convolutions, followed by instance normalization and ReLU activation after each convolution, and finally added to the input to get the final output result. Residual networks have proven their effectiveness in pixel-level tasks in many fields, such as object detection, semantics, and image segmentation. This operation uses residual connections, which can train deeper networks and avoid feature loss.

By introducing the residual module, the enhanced network structure is shown in Figure 6.

As shown in Figure 6, the first layer is used for feature extraction to realize the conversion from RGB channels to multiple features. The last two layers are used to restore RGB images and realize the conversion of multiple features into RGB images. The orange blocks in the figure represent residual units. By connecting multiple residual units to perform complex feature transformations, the network's modeling ability for enhanced low-light images is improved.

In order to constrain the enhancement network to improve the overall perceived quality of the image, the following four loss functions are designed:

(i) Content loss

The present invention defines a content loss based on the activation maps produced by the ReLU layers of a pre-trained VGG-19 network. This loss encourages them to have similar feature representations, and instead of measuring per-pixel differences between images, it encompasses aspects of content and perceptual quality. In the case of the present invention, it is used to preserve image semantics, as none of the other losses take this into account. Let φ _i be the feature map obtained by the VGG-19 network after the i-th convolutional layer, then the content loss of the present invention is defined as the Euclidean distance between the feature representations of the enhanced image and the target image

(ii) Total variation loss

This loss can enhance the spatial smoothness of the image and operate on the pixels of the generated combined image to force the generated image to have spatial continuity, thereby avoiding significant differences between adjacent pixels and preventing the appearance of checkerboard patterns in the image.

Here C, H, and W are the number of channels, height, and width of the enhanced image I _e , respectively. They enhance the image's gradient in the opposite direction of x and y. Since their weights are relatively low, they can remove noise without damaging the high-frequency part of the image.

(iii) Color loss

In order to avoid color distortion and evaluate the color difference between the enhanced image and the target image, the present invention introduces color loss. It first applies Gaussian blur to the image and then calculates their Euclidean distance. The color loss is defined as follows:

L _color =||δ(G(I _x ))-δ(I _y )|| ² (3-4)

Here δ represents a Gaussian blur function, which is used to remove local details of the image but retain its global information, such as color.

(v) Fighting Losses

This loss encourages the generative network to convert low-light images into natural images, and promotes the generator to learn the characteristics of natural images, including texture, contrast, etc. At the same time, the present invention uses gradient penalty to stabilize the training of the discriminator, and the adversarial loss is defined as follows:

Here D represents the discriminant network, G represents the generative network, I _x and I _y represent low-light images and natural-light images, respectively.

4.3 Criminal Investigation Photography System Model Based on Low-Illumination Image Enhancement Algorithm Generative Adversarial Network Enhancement Network Module

The present invention divides the entire criminal investigation photography system into three main parts: a photography end, an enhancement end and a storage background.

On the photography side, it is divided into two parts, namely the photography module and the output module. The photography module performs the functions of photo shooting and object capture, and is connected to the enhancement segment. After shooting, the photo is transmitted to the enhancement side for further processing. The output module is connected to the enhancement side and is responsible for outputting the enhanced image.

The enhancement end is composed of an enhancement module, a recognition module and a transmission module. The enhancement module contains the enhancement algorithm described above in the present invention, and is responsible for enhancing the low-light image transmitted. The recognition module uses an adversarial network algorithm for image recognition, and is connected to the storage background. It can extract similar tags that have been identified in the background for auxiliary recognition, and can also label types that have not been identified. The transmission module is responsible for transmitting the image that has completed the recognition enhancement to the output module in the photography end, and is also responsible for transmitting the enhancement, recognition results and tags that have not been stored in the storage background to the background for storage.

The storage backend is responsible for saving enhancement, recognition results and recognition category labels.

The present invention is further described below in conjunction with specific experimental results.

The method of the present invention is the same as HE, SRIE, and DSLR in the visual effect of the LOL dataset. The qualitative results are shown in Figure 7 (a) low-light picture 1, 7 (b) prior art HE, 7 (c) prior art SRIE, 7 (d) prior art DSLR, 7 (e) the present invention, and 7 (f) the real photo after processing by the present invention.

As shown in Figure 8(a) low-light picture 2, Figure 8(b) prior art HE, Figure 8(c) prior art SRIE, Figure 8(d) prior art DSLR, Figure 8(e) the present invention, and Figure 8(f) real photo 2 after processing by the present invention.

The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims (7)

1. A criminal investigation photography method based on a low-light image enhancement algorithm, characterized in that the criminal investigation photography method based on a low-light image enhancement algorithm comprises: Using photographic equipment to acquire image data and capture objects, and transmitting the acquired image data to the enhancement end; The enhancement end uses the low-light image enhancement model based on the enhancement network module to generate an adversarial network to enhance the transmitted low-light image, and uses the adversarial network algorithm to perform image recognition and label the types that have not been recognized; at the same time, similar labels that have been recognized in the storage background are extracted for auxiliary recognition; The enhanced recognition images are transmitted to the camera end and the storage backend for output and storage respectively; The capturing method comprises: Generate various types of photography control systems as operation engine interfaces; Initialize the camera engine; Perform asynchronous operations to start controlling the camera as a whole; Asynchronous operation starts the camera and calls the camera at the same time; Use the camera to capture images and set the engine to idle state; After taking a picture, call the camera control system to execute the engineering commands in the engine; Perform asynchronous image capture operations, and after completion, call the image capture completion instruction in the camera engine to complete the operation; The method for constructing a low-light image enhancement model based on an enhancement network module generating an adversarial network comprises: (1) Obtain high-quality image pairs as training datasets to train the generator network G; (2) Randomly sample m low-light image pairs from the training dataset Among them, I _x represents the low-light image, I _y represents the real-light image, and I _adv represents the input of the discriminator; (3) The input of the discriminant network is fixed to I _adv = {0,0,…,0}, with a length of m; (4) Minimize the overall loss of the generator network: Loss _gen =ω _a L _a +ω _adv L _adv +ω _con L _con +ω _tv L _tv +ω _col L _col ; (5) Train the discriminator network and randomly initialize the input of the discriminator network to I _adv = {1, 0, …, 0}, with a length of m; (6) Randomly sample m low-light images from the training dataset (7) Maximize the overall loss of the discriminator network: The low-light image enhancement model based on the enhanced network module generating the adversarial network includes: The low-light image enhancement model includes a generator, a discriminator and a loss function added with an enhancement network; The generator with the enhanced network added is based on a fully convolutional network and consists of two parts: a plurality of residual blocks and a convolutional block; it is used to convert the input image as a whole into a similar picture in a new space; The discriminator is used to simultaneously receive the image generated by the generator and the real image, and generate a true or false prediction value; The loss function is: Loss = ω _a L _a + ω _adv L _adv + ω _con L _con + ω _tv L _tv + ω _col L _col ; Among them, _La , _Ladv , _Lcon , _Ltv , _Lcol represent attention loss, adversarial loss, content loss, total variation loss, color loss respectively, _ωa , _ωadv , _ωcon , _ωtv , _ωcol represent the corresponding weights of their losses respectively; The enhancement network module includes two 3x3 convolutional layers and a feature transformation layer; it is used to overcome the disadvantage of low contrast of the image and improve the details to enhance the image effect; The first layer of the 3x3 convolutional layer is used for feature extraction, realizing from RGB channels to multiple features; The first convolutional layer is followed by a feature transformation layer, which is a residual module; the feature transformation layer is used to perform complex feature transformation by connecting multiple residual units; The feature transformation layer is followed by two convolution layers, and the convolution layers are used to restore the RGB image and realize the conversion of multiple features into RGB images; instance normalization and ReLU activation are performed after each convolution. 2. The criminal investigation photography method based on the low-light image enhancement algorithm according to claim 1, characterized in that the enhancement network module includes four loss functions: 1) Content loss: The content loss is defined based on the activation map produced by the ReLU layer of the pre-trained VGG-19 network; the content loss is the Euclidean distance between the feature representations of the enhanced image and the target image: Among them, φ _j is the feature map obtained by the VGG-19 network after the jth convolutional layer; 2) Total variation loss: Among them, C, H, and W are the number of channels, height, and width of the enhanced image I _e , respectively. They are respectively to enhance the gradient of the image in the opposite direction of x and y; 3) Color loss: _Lcolor ＝||δ(G( _Ix ))-δ( _Iy )|| ² ; Among them, δ represents the Gaussian blur function, which is used to remove local details of the image; 4) Fighting Losses Among them, D represents the discriminant network, G represents the generative network, I _x and I _y represent low-light images and natural-light images, respectively. 3. The criminal investigation photography method based on the low-light image enhancement algorithm according to claim 1, characterized in that the generator with the enhancement network added comprises: The generator consists of 1 convolution block, 4 residual blocks and 2 convolution blocks in sequence; The four residual blocks are used to keep the height/width constant; The generator performs instance regularization and ReLU activation after each convolution; The last convolutional layer of the generator is tanh activated, and each convolutional layer is followed by ReLU activation; The discriminator includes 5 convolutional layers, 1 fully connected layer and 1 softmax layer; The size of the convolution kernel of the convolution layer is reduced from 11 to 3, and the number of feature channels is increased from 3 to 192; it is used to gradually extract input features; The fully connected layer and the softmax layer are used to predict the possibility that the extracted feature map comes from the generator or the real picture, and the result is a (Batch, P _true , P _false ) 3-tuple, where the values of P _true and P _false are both in the range of [0, 1]. 4. The criminal investigation photography method based on low-light image enhancement algorithm according to claim 1, wherein the image digital processing method comprises: Sampling: The quality of the sampling result is measured by the image resolution; Quantization: the range of values used to represent each point after the image is sampled; the result of quantization is the total number of colors contained in the image, reflecting the quality of sampling; Compression coding: Use coding technology to compress the amount of information. 5. A criminal investigation photography system based on a low-light image enhancement algorithm using the criminal investigation photography method based on a low-light image enhancement algorithm according to any one of claims 1 to 4, characterized in that the criminal investigation photography system based on a low-light image enhancement algorithm comprises: The photographing end is connected to the enhancing end; it includes a photographing module and an output module; the photographing module is used to acquire image data and capture objects, and transmit the acquired image data to the enhancing end; the output module is used to output the enhanced image; The enhancement end is connected with the photographing end and the storage backend, and includes an enhancement module, a recognition module and a transmission module; The enhancement module is used to enhance the transmitted low-light image by using a low-light image enhancement model based on the enhancement network module to generate an adversarial network; the recognition module is used to perform image recognition using an adversarial network algorithm; it is also used to extract similar tags that have been recognized in the background for auxiliary recognition, and it can also label types that have not been recognized; the transmission module is used to transmit the image that has completed recognition and enhancement to the output module in the camera end, and is also used to transmit the enhancement and recognition results and tags that have not been stored in the storage background to the background for storage; The storage backend is connected to the enhancement end and is used to store enhancement, recognition results and recognition category labels. 6. A computer device, characterized in that the computer device comprises a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the criminal investigation photography method based on the low-light image enhancement algorithm according to any one of claims 1 to 4, comprising the following steps: The photographing end uses photographic equipment to acquire image data and capture objects, and transmits the acquired image data to the enhancing end; The enhancement end uses the low-light image enhancement model built based on the enhancement network module to generate adversarial networks to enhance the transmitted low-light images, and uses adversarial network algorithms to perform image recognition and label types that have not been recognized. At the same time, similar labels that have been recognized in the storage background are extracted for auxiliary recognition. The images that have completed recognition and enhancement are transmitted to the camera end and the storage background for output and storage respectively. 7. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the criminal investigation photography method based on the low-light image enhancement algorithm according to any one of claims 1 to 4, comprising the following steps: The photographing end uses photographic equipment to acquire image data and capture objects, and transmits the acquired image data to the enhancing end; The enhancement end uses the low-light image enhancement model built based on the enhancement network module to generate adversarial networks to enhance the transmitted low-light images, and uses adversarial network algorithms to perform image recognition and label types that have not been recognized. At the same time, similar labels that have been recognized in the storage background are extracted for auxiliary recognition. The images that have completed recognition and enhancement are transmitted to the camera end and the storage background for output and storage respectively.