CN113365014A - Parallel compressed sensing GPU (graphics processing Unit) acceleration real-time imaging system and method - Google Patents

Abstract

The invention discloses a parallel compressed sensing GPU accelerated real-time imaging system and method. The system includes an optical unit (I), an electrical unit (II) and a back-end data pipeline processing unit (III) for parallel pipeline processing; wherein, The optical unit (I) is used for collecting optical signals to obtain a target image signal, which is segmented and modulated and then sent to the electrical unit (II); the electrical unit (II) is used for parallel complementary compressed sensing imaging, Completing the parallel complementary measurement, sending the low-resolution image data to the back-end data pipeline processing unit (III); the back-end data pipeline processing unit (III) is used to implement high-resolution image reconstruction by using a GPU-accelerated compressed sensing high-speed reconstruction algorithm with real-time display. The invention adopts the parallel pipeline scheme of the optical unit, the electrical unit and the GPU accelerated reconstruction component, and realizes the real-time acquisition of low-resolution images, real-time reconstruction and display of high-resolution images by compressed sensing.

Description

A parallel compressed sensing GPU accelerated real-time imaging system and method

technical field

The invention relates to the field of imaging technology, in particular to a parallel compressed sensing GPU accelerated real-time imaging system and method, which is different from the imaging method of the traditional direct measurement computational imaging system.

Background technique

The imaging detection of optical signals is an important means for humans to perceive the surrounding environment and understand the world. It is no exaggeration to say that without the birth and development of imaging technology, there would be no modern photoelectric detection technology. The era of digitization and the intelligent Internet of Things (AIoT) has driven higher and higher requirements for the temporal and spatial resolution of detectors, the scale of detection data has surged, and the imaging methods and photoelectric detection technologies have been continuously improved and improved in various aspects. It has also experienced a huge leap, rapidly advancing the process of human perception of the world.

On the one hand, on the demand side of the application scene, the pixel scale of the detection scene increases, and the demand for higher temporal resolution and spatial resolution has never stopped. On the other hand, the cost of higher resolution detectors in the non-visible light band is still high, and the cost of visible light detectors is getting higher and higher as the resolution increases. Existing resolution detectors cannot realize real-time imaging of massive pixel-scale scenes in the direct measurement method of "what you see is what you get", while the existing technical indicator equipment needs to further achieve higher temporal resolution and control resolution in the traditional way. It is more difficult. How to deal with the contradictions between imaging pixel scale, temporal resolution, spatial resolution required data rate, real-time display requirements and transmission bandwidth, detector cost, and storage resources have become an important direction and difficulty in imaging system technology research. Compressed sensing imaging is motivated by reducing the sampling amount and reducing the detector resolution requirements, and creatively applies the new signal processing system compressed sensing theory to the imaging field. Among them, the single-pixel camera is the most representative new imaging system for compressed sensing imaging. It adopts the indirect measurement method of point detectors without spatial resolution through modulation sampling, and performs compression and sampling together. The number of samples required by the Sterling-Shannon sampling law is used to measure the signal, and then the reconstruction algorithm of the back-end data processing will pay the imaging target with spatial resolution, which realizes the detection with the point detector without spatial resolution. , low transmission bandwidth, and can ensure the final imaging target of maintaining spatial resolution image acquisition.

However, with the further improvement of detection resolution requirements, the optical acquisition front-end is also limited by the modulation frequency of the existing spatial optical modulation front-end. At the same time, the compressed sensing imaging model based on the compressed sensing imaging theory is also faced with the difficulty of constructing the measurement matrix in large-scale scenarios. The complexity of the reconstruction algorithm is rapidly increasing, and the traditional serial single-pixel camera architecture cannot meet the demand, and new performance improvement strategies are urgently needed.

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the shortcomings of the traditional imaging system using conventional detectors and transmission bandwidth in high-resolution fast acquisition and real-time display, and to expand the single-pixel camera by adopting divide-and-conquer processing and parallelization system ideas, thereby providing a parallel compression Perceptual GPU-accelerated real-time imaging systems and methods.

In order to achieve the above object, Embodiment 1 of the present invention proposes a parallel compressed sensing GPU-accelerated real-time imaging system, the system includes an optical unit, an electrical unit and a back-end data pipeline processing unit for parallel pipeline processing; wherein,

The optical unit is used to collect the optical signal, obtain the target image signal, and send it to the electrical unit after segmentation and modulation;

The electrical unit is used to perform parallel complementary compressed sensing imaging, complete parallel complementary measurement, and send the low-resolution image data to the back-end data pipeline processing unit;

The back-end data pipeline processing unit is used for realizing high-resolution image reconstruction and real-time display by using a GPU-accelerated compressed sensing high-speed reconstruction algorithm.

As an improvement of the above system, the optical unit includes: a field diaphragm and an imaging objective lens (1), a spatial light modulator and a condensing light-collecting component; wherein,

The field diaphragm and the imaging objective lens are used to collect the light signal transmitted, reflected or radiated by the target, and image the light signal on the spatial light modulator;

The spatial light modulator is used to perform segmental parallel random modulation on the target image signal, and reflect light at different positions to the converging light-receiving component;

The condensing light-collecting component is used for transmitting the condensed and collected light to the electrical unit.

As an improvement of the above system, the electrical unit includes: a photoelectric array detector, a random number generator, a signal synchronization control module, and a data acquisition buffer;

The photoelectric array detector is used to detect the segmented optical signals in parallel, convert them into electrical signals, and output them to the data acquisition buffer component;

The data acquisition cache component is used to instantly and continuously transmit the low-resolution image to the back-end data pipeline processing unit;

The random number generator is used to control the spatial light modulator to perform piecewise parallel random modulation on the optical signal; it is also used to generate randomly distributed speckles of the binary optical signal;

The signal synchronization control module is used to control the photoelectric array detector, the random number generator and the data acquisition buffer through the signal synchronization.

As an improvement of the above system, the back-end data pipeline processing unit includes: a data sharing service component, a high-resolution image reconstruction and display component, and a general-purpose computing GPU-accelerated compressed sensing high-speed reconstruction component; wherein,

the data sharing service component for receiving and storing low-resolution image data;

The high-resolution image reconstruction and display component is used to continuously process the low-resolution image data of the data sharing service component, and the general-purpose computing GPU-accelerated compressed sensing high-speed reconstruction component is called by the function to quickly reconstruct the high-resolution image and display it in real time;

The general-purpose computing GPU-accelerated compressed sensing high-speed reconstruction component is used for realizing high-resolution image reconstruction by using a GPU-accelerated compressed sensing high-speed reconstruction algorithm by utilizing low-resolution images, block random matrices and sparse bases.

As an improvement of the above system, the spatial light modulator adopts a digital micromirror device; the light-converging component includes a condensing lens and a diaphragm; wherein, the condensing lens is used to reflect the spatial light modulator to the photoelectric array Detector; the diaphragm for eliminating stray light.

Embodiment 2 of the present invention proposes a parallel compressed sensing GPU accelerated real-time imaging method, which is implemented based on the above-mentioned system system, and the method includes:

The field diaphragm and the imaging objective lens collect the light signal transmitted, reflected or radiated by the target, and image it to the spatial light modulator. The spatial light modulator performs segmental parallel random modulation on the target image signal under the control of the random number generator. Reflect the light at different positions to the condensing light-collecting part, and the condensing light-collecting part transmits the collected light to the photoelectric array detector;

The signal synchronization control module controls the photoelectric array detector, the random number generator and the data acquisition buffer through the signal synchronization, performs optical parallel complementary compressed sensing imaging, completes the parallel complementary measurement, and records the low-resolution image data to the data acquisition buffer;

The data acquisition cache instantly and continuously transmits low-resolution image data to the data sharing service component, and the high-resolution image reconstruction and display component continuously processes the low-resolution image data of the data sharing service component, and invokes general-purpose computing GPU to accelerate compressed sensing high-speed The reconstruction component quickly reconstructs high-resolution images and displays them in real time.

As an improvement of the above method, the optical parallel complementary compressed sensing imaging, to complete the parallel complementary measurement, specifically includes:

The N×N effective imaging area on the spatial light modulator is divided into several blocks, the size of each block is C×C, and the image is parallelized on the N/C×N/C pixels of the photoelectric array detector, then the size of each block is C×C. The obtained corresponding i-th observation vector column Y ⁽ⁱ⁾ satisfies the following formula:

Y ⁽ⁱ⁾ = Φ ⁽ⁱ⁾ (x ⁽ⁱ⁾ )+E ⁽ⁱ⁾

Wherein, X ⁽ⁱ⁾ represents the i-th sub-block in the spatial light modulator in a column-first manner, and Φ ⁽ⁱ⁾ is the projection operator of the sub-scene image of the i-th sub-block;

为由在空间光调制器上形成的目标场景组成块中的其中一块，并且设

为分别显示在空间光调制器上的互补测量二值模式，

为光电阵列探测器采集到的互补压缩测量，

是互补压缩测量差分矢量，则在光电阵列探测器上获得的每个测量值如下所示：Assume

is one of the blocks formed by the target scene formed on the spatial light modulator, and set

are the complementary measured binary patterns displayed on the spatial light modulator, respectively,

Complementary compression measurements acquired for the photoelectric array detector,

is the complementary compressed measurement difference vector, then each measurement obtained on the photoelectric array detector is as follows:

Among them, e ₁ and e ₂ represent two random noises collected by complementary compression measurement;

One measurement of the complementary matrix gives Δy as:

表示元素乘积。in,

Represents the element-wise product.

As an improvement of the above method, the high-resolution image reconstruction and display component continuously processes the low-resolution image data of the data sharing service component; specifically, it includes:

The high-resolution image reconstruction and display component arranges the parallel blocks in the measurement data of the data sharing service component in column-major order, and the complementary measurement vector in each block consists of the measurement values of the specific block distributed in the corresponding position of each low-resolution image. composed, in the sub-measurement matrix, the complementary measurement vector of each block is synchronized with each mask vector, and the compressed block observation vector of each original image block is reconstructed.

As an improvement of the above method, the high-resolution image is rapidly reconstructed and displayed in real time, which specifically includes:

According to the set compression ratio, reconstruction sub-block size and measurement times, generate a full sampling measurement matrix, extract from it to generate a sub-sampling measurement matrix, and convert it to CUDA sparse matrix format;

Combine multiple frames of low-resolution images into one frame; obtain complementary measurements through differential calculation, divide each frame into blocks, and reset each block by column vector;

Each image is reconstructed according to the CUDA sparse matrix format and GPU kernel function definition, and multiple blocks are stitched into one or more high-resolution images and displayed in real time.

Compared with the prior art, the advantages of the present invention are:

1. The present invention adopts the parallel compressed sensing theory, and the parallel compressed sensing adopts the idea of algorithm division (Divide and Conquer) to decompose the compressed sensing measurement and reconstruction problem of the complete signal into a plurality of independent sub-signal compression measurement and reconstruction problems, and finally the reconstruction is obtained. The original signal is obtained by combining multiple original sub-signals, so as to reduce the construction complexity of the measurement matrix, the memory space requirements of the measurement matrix and the complexity of the reconstruction algorithm, and improve the parallelism of the system;

2. The present invention uses low-resolution images to reconstruct high-resolution images, and solves the problem of lack of high-resolution mid-infrared detectors, single-photon detectors, and terahertz detectors at this stage;

3. The present invention adopts binary random modulation, gives full play to the turnover frequency of the spatial light modulator, and improves the sampling speed by combining the sub-sampling advantage of the compressed sensing theory and the parallel sampling advantage of parallel compressed sensing;

4. The present invention adopts the GPU-accelerated compressed sensing reconstruction algorithm, and combines the merged block reconstruction strategy and the frame splicing strategy to improve the reconstruction speed;

5. The present invention adopts the parallel pipeline scheme of the optical unit, the electrical unit and the GPU accelerated reconstruction component to realize the real-time acquisition, real-time reconstruction and display of compressed sensing.

Description of drawings

1 is a schematic structural diagram of a parallel compressed sensing GPU-accelerated imaging system according to Embodiment 1 of the present invention;

FIG. 2(a) is a schematic diagram of a process of combining sub-measurement signals and corresponding sub-measurement matrices according to Embodiment 2 of the present invention;

Fig. 2(b) is the corresponding sub-measurement matrix construction process of Fig. 2(a);

3 is a schematic diagram of a GPU-accelerated parallel reconstruction process according to Embodiment 2 of the present invention;

FIG. 4 is a comparison diagram of processing time between the GPU reconstruction algorithm and the method of the present invention.

reference number

Ⅰ. Optical unit Ⅱ. Electrical unit

Ⅲ. Back-end data pipeline processing unit

1. Field diaphragm and imaging objective lens 2. Spatial light modulator

3. Converging light-collecting components 4. Photoelectric array detector

5. Random number generator 6. Signal synchronization control module

7. Data collection cache 8. Data sharing service components

9. High-resolution image reconstruction and display components

10. General computing GPU-accelerated compressed sensing high-speed reconstruction components

Detailed ways

The invention provides a parallel compressed sensing GPU accelerated real-time imaging system and method,

The technical solutions of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

Example 1

As shown in FIG. 1 , Embodiment 1 of the present invention provides a parallel compressed sensing GPU-accelerated real-time imaging system.

The block-parallel compressed sensing imaging system accelerated by the general computing GPU reconstruction algorithm of the present invention utilizes the compressed sensing (Compressed Sensing, CS) principle, which is a brand-new signal processing proposed by Donoho, Tao and Candès et al. The system realizes the compressed sampling of the signal with the sub-sampling measurement quantity and the sampling method of random modulation of the signal, and perfectly restores the original signal through a mathematical algorithm at the receiving end. Parallel compressive sensing uses the algorithm of division and system (Divide and Conquer) to decompose the compressive sensing measurement and reconstruction problem of the complete signal into multiple independent sub-signal compression measurement and reconstruction problems, and finally merge the reconstructed original sub-signals to obtain the original In order to reduce the complexity of measurement matrix construction, measurement matrix memory space requirements and reconstruction algorithm complexity, and improve the parallelism of the system.

并行压缩感知将进行信号向量

分块。The complete mathematical model of compressed sensing measurement can be expressed as

Parallel compressed sensing will do the signal vector

Block.

的第i个元素，(·)^T是转置运算符。将该信号分为互不重叠的M段，即where x _i represents the signal

The ith element of , ( ) ^T is the transpose operator. The signal is divided into M segments that do not overlap each other, namely

的第j个长度为l_j的分段，则有

where x[j] represents the signal

the j-th segment of length l _j , then we have

The parallel test signal measurement matrix is expressed in the form of a diagonal matrix:

The measurement process is expressed in matrix form as:

The parallel compressed sensing imaging process can be modeled as parallel measurement and parallel reconstruction of optical signals using a diagonalized measurement matrix. Among them, in the process of parallel compression and sampling, the present invention adopts the complementary compression measurement method; in the process of parallel reconstruction, the present invention combines several sub-measurement signals and corresponding sub-measurement matrices, and then uses GPU acceleration algorithm for reconstruction. Parallel measurement, GPU-accelerated parallel reconstruction, and display form a pipeline.

1, the present invention provides a parallel compressed sensing GPU accelerated real-time imaging system and method, including an optical unit I, an electrical unit II, and a back-end data pipeline processing unit III; wherein, the optical unit I at least includes a field diaphragm and a Imaging objective lens 1, spatial light modulator 2, converging light-receiving component 3; electrical unit II at least includes photoelectric array detector 4, random number generator 5, signal synchronization control module 6, data acquisition buffer component 7; back-end data pipeline The processing unit III includes a data sharing service component 8, a high-resolution image reconstruction and display component 9, and a general-purpose computing GPU-accelerated compressed sensing high-speed reconstruction component 10;

In the optical unit I, the light signal transmitted, reflected or radiated by the target is collected by the field diaphragm and the imaging objective lens 1, and imaged on the spatial light modulator 2; The image signal is subjected to segmental parallel random modulation, and the light at different positions is reflected to the converging light-receiving part 3; the light collected by the converging is transmitted to the photoelectric array detector 4 of the electrical unit II;

In the electrical unit II, the low-resolution photoelectric array detector 4 detects the segmented optical signals collected by the converging light-receiving component 3 in parallel, converts them into electrical signals and outputs them, and records the low-resolution images to the data acquisition buffer. Component 7; the random number generator 5 controls the spatial light modulator 2 to perform segmental parallel random modulation on the optical signal; the signal synchronization module 6 controls and coordinates the camera modulation and acquisition and recording, including the optical unit and electrical The work control of each component of the unit and the synchronizing pulse trigger signal, ensure the synchronization between the spatial light modulator 2 and the random number generator 5, control and coordinate the photoelectric array detector 4 of the camera, data recording collection and the random number generation The random number generation rhythm of the controller 5; the data acquisition and cache component 7 instantly and continuously provides the output low-resolution image to the data sharing service component 8 of the back-end data pipeline processing unit III;

In the back-end data pipeline processing unit III, the high-resolution image reconstruction and display component 9 continuously processes the low-resolution image data of the data sharing service component 8, and invokes the general computing GPU-accelerated compressed sensing high-speed reconstruction component. 10. Quickly reconstruct a high-resolution image; the general-purpose computing GPU-accelerated compressed sensing high-speed reconstruction component 10 utilizes low-resolution images, block random matrices, and sparse bases to achieve high-resolution image reconstruction using a GPU-accelerated compressed sensing high-speed reconstruction algorithm; wherein, the The data acquisition and cache component 7, the data sharing service component 8, and the high-resolution image reconstruction and display component 9 constitute a parallel processing pipeline of modulation acquisition, compression transmission, and high-speed reconstruction.

The above is a description of the overall structure of the present invention, and the specific implementation of each component is further described below.

The field diaphragm and the imaging objective lens 1 collect the light signal transmitted, reflected or radiated by the target;

The spatial light modulator 2 contains many independent units, which are arranged in a one-dimensional or two-dimensional array in space, and each unit can independently receive the control of an optical signal or an electrical signal, and change its own optical properties according to the signal. , thereby modulating the light waves that illuminate it. Such devices can change the amplitude or intensity, phase, polarization state, and wavelength of a spatial light distribution, or convert incoherent light into coherent light, under the control of a time-varying electrical drive signal or other signal. Due to its nature, it can be used as a structural unit or key device in systems such as real-time optical information processing, optical computing, and optical neural networks. It can be divided into transmission type and reflection type. There are many types, mainly digital micromirrors. Device (Digital Micro-mirror Device, DMD), liquid crystal light valve realization. In this embodiment, the spatial light modulator is a digital micromirror device, and in other embodiments, it may also be other types of spatial light modulators.

The DMD used in this embodiment is an array containing a large number of micromirrors mounted on hinges (the mainstream DMD is composed of an array of 1024×768), and the size of each mirror is 13.68 μm×13.68 μm, and can be used for each mirror. The light on each pixel is independently controlled. By electronically addressing the memory cells under each mirror with binary signals, each mirror can be flipped to both sides (+12° and -12° in this embodiment) under the action of static electricity. These states are denoted as 1 and 0, corresponding to "on" and "off", respectively. When the lenses are not working, they are in a "parked" state of 0°.

The condensing light-receiving member 3 includes a condensing lens and a diaphragm. The converging lens reflects the spatial light modulator 2 to the photoelectric array detector; the diaphragm is used to eliminate stray light.

The photoelectric array detector 4 adopts a low-cost conventional array detector, which can be adjusted according to the response range of the wavelength band, including the visible light band and the non-visible light band. In this embodiment, the photoelectric array detector adopts an industrial camera CCD. The invention uses low-resolution images to reconstruct high-resolution images, and solves the problem of lack of high-resolution mid-infrared detectors, single-photon detectors, and terahertz detectors at the present stage.

The random number generator 5 is used to generate randomly distributed speckles of the binary optical signal.

The signal synchronization control module 6 ensures the synchronization between the spatial light modulator 2 and the random number generator 5, and controls and coordinates the data recording acquisition of the photoelectric array detector 4 of the camera and the random number of the random number generator 5. Generate beats.

The data sharing service component 8 acts as a "product" buffer synchronization amount in the data acquisition and cache component 7, and the high-resolution image reconstruction and display component 9 runs asynchronously to form a parallel pipeline in a "producer-consumer" mode.

The high-resolution image reconstruction and display component 9 calls the general computing GPU-accelerated compressed sensing high-speed reconstruction component 10 to perform reconstruction through a function, and the reconstruction module and the display module are executed in an asynchronous parallel manner.

The general computing GPU accelerated compressed sensing high-speed reconstruction component 10 adopts any of the following general computing GPGPU forms: desktop graphics card GPU, server GPU, data center GPU, and GPU mounted on mobile devices. The compressed sensing algorithm used by the general computing GPU-accelerated compressed sensing high-speed reconstruction component 10 adopts TV algorithm, least squares method, matching tracking algorithm MP, orthogonal matching tracking algorithm OMP, basis tracking algorithm BP, and TwIST algorithm; sparse basis adopts discrete Any of cosine transform basis, wavelet basis, Fourier transform basis, gradient basis, and gabor transform basis; directly reconstruct the original signal without using sparse basis.

Example 2

Embodiment 2 of the present invention provides a parallel compressed sensing GPU accelerated real-time imaging method, which is performed based on the system of Embodiment 1, and the specific steps are as follows:

Step 1) The steps of optical signal acquisition:

The light signal transmitted, reflected or radiated by the target is collected by the field diaphragm and the imaging objective lens 1, and imaged on the spatial light modulator 2; the spatial light modulator 2 performs segmented parallel on the target image signal Random modulation to reflect light at different positions to the converging light-collecting component 3; the light collected by the converging is transmitted to the photoelectric array detector 4 of the electrical unit II;

Step 2) Steps of Optical Parallel Complementary Compressed Sensing Imaging

The random number generator 5 controls the spatial light modulator 2 to perform segmental parallel random modulation on the optical signal; the signal synchronization module 6 controls and coordinates the camera modulation and acquisition and recording, including the components of the optical unit and the electrical unit. The work control and synchronization pulse start signal, ensure the synchronization between the spatial light modulator 2 and the random number generator 5, and control and coordinate the data recording acquisition of the photoelectric array detector 4 of the camera and the random number generator 5. Random number generation beats. The specific parallel method is as follows.

The N×N effective imaging area on the spatial light modulator 2 is divided into several blocks (the size of each block is C×C), which are imaged on the N/C×N/C pixels of the two-dimensional photoelectric array detector 4 in parallel. Each detector in the photoelectric array detector 4 is equivalent to a photodiode in the SPC (Single Pixel Camera) system, so the model is expressed as

Y ⁽ⁱ⁾ = Φ ⁽ⁱ⁾ (X ⁽ⁱ⁾ )+E ⁽ⁱ⁾ (5)

where X ⁽ⁱ⁾ represents the i-th block in the spatial light modulator 2 in a column-major manner, Y ⁽ⁱ⁾ represents the corresponding i-th observation vector column obtained on the photoelectric array detector 4, and Φ ⁽ⁱ⁾ is the spatial The projection operator of the sub-scene image of the ith sub-block of light modulator 2.

是由在空间光调制器2上形成的目标场景组成块中的其中一块，并且设

是分别显示在空间光调制器2上的互补测量二值模式。

为光电阵列探测器4采集到的互补压缩测量，

是互补压缩测量差分矢量。在光电阵列探测器4上获得的每个测量值如下所示：Assume

is one of the target scene constituent blocks formed on the spatial light modulator 2, and set

are the complementary measurement binary patterns displayed on the spatial light modulator 2, respectively.

for the complementary compression measurements collected by the photoelectric array detector 4,

is the complementary compression measurement difference vector. Each measurement obtained on Photo Array Detector 4 is as follows:

Finally, one measurement of the complementary matrix is given as

表示元素乘积。in the formula

Represents the element-wise product.

In the signal synchronization control module 6, the photodetector array detection 4, the random number generator 5 and the data acquisition buffer part 7 are controlled synchronously by the signal, and the parallel complementary measurement is completed with reference to the above steps, and the complementary measurement vector and the measured value are recorded in the data acquisition buffer part. 7.

Step 3) The sub-measurement signal and the corresponding sub-measurement matrix are combined

Figure 2 is a schematic diagram of the process, and Figure 2(a) depicts the compression ratio of multiple blocks of 4x4 (4x4 pixels as an element block) and 2x2 (8x8 pixels as a combined block) The construction of the measurement matrix; Figure 2(b) shows the construction process of the corresponding sub-measurement matrix.

以逐块和逐列的方式依次并行取自多个SPC块。最后，掩模向量化后合并到整个测量矩阵中。The effective imaging area of the spatial light modulator 2 is divided into several element blocks B×B of the same size ((C×C) is the basic element block, which is composed of an integer multiple of the element block), and each element block is equivalent to a photoelectric array detection. 4 parallel SPC reconstruction blocks. Second, each photoelectric array detection 4 parallel SPC encoded block mask (C × C) on the spatial light modulator 2 is generated, and complementary positive and negative measurements are performed to improve CS imaging quality. unit measurement vector corresponding to the measurement matrix

Taken in parallel from multiple SPC blocks sequentially in a block-by-block and column-by-column manner. Finally, the mask vectors are quantized and merged into the entire measurement matrix.

每一次行向量调制采样得到一张低分辨图像。In the designed measurement matrix, the observation vectors of all blocks are synchronized with each mask vector. First, the parallel blocks in the observations are arranged in column-major order. Second, the column vector of observations for each block is derived from a linear combination of the blocks corresponding to the measurement sequence. That is, each low-resolution image consists of the projection results of all element blocks, and the observation vector in each block consists of the observations of a specific block distributed at the corresponding position of each low-resolution image. Block _i is the i-th compressed block observation value vector used to reconstruct the i-th original image block, and M _j represents the j-th measurement value corresponding to the j-th encoding mode. Block _i and M _j accurately indicate the correspondence between the i-th compressed block and the j-th measurement observation, where

A low-resolution image is obtained for each row vector modulation sample.

Step 4) GPU-accelerated parallel reconstruction process

个像素，使得每个像素对应空间光调制器2上C×C个微镜区域大小，则每个区域图像将从m(m≤C×C)次测量值中恢复。1)在GPU上准备分块测量矩阵。生成尺寸为

的全采样分块投影矩阵和尺寸为

的欠采样分块测量矩阵，并将其转换为CUDA稀疏矩阵格式。2)在GPU上准备分块观测值向量组。为了获得更高的重建效率，需要将相应的低分辨率观测值加载到主机中，并将多帧合并为一帧记为N×N，下一步是将观测值与正负互补测量图像进行差分，将每一帧分为

块，并按列将每一帧置为列向量，大小为

3)用GPU加速模块重建每个块，并将所有重建的高分辨率块拼接成一个或多个高分辨率的完整图像帧。FIG. 3 is a schematic diagram of the process, using the spatial light modulator 2 . It is assumed that the effective imaging area of the spatial light modulator 2 has the resolution of N×N micromirrors, and the corresponding effective imaging area of the photoelectric array detector 4 has

pixels, so that each pixel corresponds to the size of C×C micromirror areas on the spatial light modulator 2 , then the image of each area will be recovered from m (m≦C×C) times of measurement values. 1) Prepare the block measurement matrix on the GPU. The resulting size is

The fully sampled block projection matrix and dimensions are

The undersampled block measurement matrix and convert it to CUDA sparse matrix format. 2) Prepare the block observation vector group on the GPU. In order to obtain higher reconstruction efficiency, the corresponding low-resolution observations need to be loaded into the host, and the multiple frames are merged into one frame and denoted as N×N. The next step is to differentiate the observations with the positive and negative complementary measurement images. , divide each frame into

blocks, and place each frame by column as a column vector of size

3) Reconstruct each block with a GPU-accelerated module and stitch all reconstructed high-resolution blocks into one or more high-resolution full image frames.

Step 5) The pipeline process of low-resolution image acquisition, high-resolution reconstruction and display

In the back-end data pipeline processing unit III, the high-resolution image reconstruction and display component 9 continuously processes the low-resolution image data of the data sharing service component 8, and invokes the general computing GPU-accelerated compressed sensing high-speed reconstruction component. 10. Quickly reconstruct a high-resolution image; the general-purpose computing GPU-accelerated compressed sensing high-speed reconstruction component 10 utilizes low-resolution images, block random matrices, and sparse bases to achieve high-resolution image reconstruction using a GPU-accelerated compressed sensing high-speed reconstruction algorithm; wherein, the The data acquisition and cache component 7, the data sharing service component 8, and the high-resolution image reconstruction and display component 9 constitute a parallel processing pipeline of modulation acquisition, compression transmission, and high-speed reconstruction.

FIG. 4 is a time comparison diagram of image processing using the method of the present invention and the prior art.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the embodiments, those of ordinary skill in the art should understand that any modification or equivalent replacement of the technical solutions of the present invention will not depart from the spirit and scope of the technical solutions of the present invention, and should be included in the present invention. within the scope of the claims.

Claims (9)

1. A parallel compressed sensing GPU accelerated real-time imaging system is characterized in that the system comprises an optical unit (I), an electrical unit (II) and a back-end data pipeline processing unit (III) which are processed in a parallel pipeline mode; wherein,

the optical unit (I) is used for collecting optical signals to obtain target image signals, and the target image signals are sent to the electrical unit (II) after being segmented and modulated;

the electrical unit (II) is used for performing parallel complementary compressed sensing imaging, completing parallel complementary measurement and sending low-resolution image data to the back-end data pipeline processing unit (III);

and the back-end data pipeline processing unit (III) is used for realizing high-resolution image reconstruction and real-time display by adopting a GPU accelerated compressed sensing high-speed reconstruction algorithm.

2. Parallel compressed-sensing GPU accelerated real-time imaging system according to claim 1, characterized in that said optical unit (I) comprises: the field diaphragm, the imaging objective lens (1), the spatial light modulator (2) and the light converging and collecting component (3); wherein,

the field diaphragm and the imaging objective lens (1) are used for collecting optical signals transmitted, reflected or radiated by a target and imaging the optical signals onto the spatial light modulator (2);

the spatial light modulator (2) is used for carrying out segmented parallel random modulation on a target image signal and reflecting light at different positions to the convergent light-receiving component (3);

the light-converging and light-receiving component (3) is used for transmitting the light converged and collected to the electrical unit (II).

3. Parallel compressed-aware GPU-accelerated real-time imaging system according to claim 2, wherein said electrical unit (II) comprises: the device comprises a photoelectric array detector (4), a random number generator (5), a signal synchronization control module (6) and a data acquisition cache (7);

the photoelectric array detector (4) is used for detecting segmented optical signals in parallel, converting the segmented optical signals into electric signals and outputting the electric signals to the data acquisition buffer component (7);

the data acquisition buffer component (7) is used for instantly and continuously transmitting the low-resolution image to the back-end data pipeline processing unit (III);

the random number generator (5) is used for controlling the spatial light modulator (2) to perform segmented parallel random modulation on the optical signals; the speckle filter is also used for generating binary optical signals which are randomly distributed;

and the signal synchronization control module (6) is used for synchronously controlling the photoelectric array detector (4), the random number generator (5) and the data acquisition buffer (7) through signals.

4. The parallel compressed-aware GPU-accelerated real-time imaging system of claim 3, wherein the back-end data pipeline processing unit (III) comprises: the system comprises a data sharing service part (8), a high resolution image reconstruction and display part (9) and a general-purpose computing GPU accelerated compressed sensing high-speed reconstruction part (10); wherein,

the data sharing service part (8) is used for receiving and storing low-resolution image data;

the high-resolution image reconstruction and display component (9) is used for continuously processing the low-resolution image data of the data sharing service component (8), and accelerating the compressed sensing high-speed reconstruction component (10) by a function call general purpose computing GPU to rapidly reconstruct a high-resolution image and display the high-resolution image in real time;

the general-purpose computing GPU accelerated compressed sensing high-speed reconstruction component (10) is used for realizing high-resolution image reconstruction by utilizing a low-resolution image, a block random matrix and a sparse basis and adopting a GPU accelerated compressed sensing high-speed reconstruction algorithm.

5. A parallel compressed-sensing GPU accelerated real-time imaging system according to claim 4, characterized in that the spatial light modulator (2) employs a digital micromirror device; the convergent light-receiving component (3) comprises a convergent lens and a diaphragm; wherein the converging lens is used for reflecting the spatial light modulator (2) to the photoelectric array detector (4); and the diaphragm is used for eliminating stray light.

6. A parallel compressed sensing GPU accelerated real-time imaging method, which is realized based on the system of claim 5 and comprises the following steps:

the field diaphragm and the imaging objective lens (1) collect optical signals transmitted, reflected or radiated by a target, the optical signals are imaged to the spatial light modulator (2), the spatial light modulator (2) conducts segmented parallel random modulation on target image signals under the control of the random number generator (5), light at different positions is reflected to the converging and light-receiving component (3), and the converging and light-receiving component (3) transmits the converged and collected light to the photoelectric array detector (4);

the signal synchronization control module (6) controls the photoelectric array detector (4), the random number generator (5) and the data acquisition cache (7) through signal synchronization, optical parallel complementary compressed sensing imaging is carried out, parallel complementary measurement is completed, and low-resolution image data are recorded in the data acquisition cache (7);

the data acquisition cache (7) instantly and continuously transmits the low-resolution image data to the data sharing service component (8), the high-resolution image reconstruction and display component (9) continuously processes the low-resolution image data of the data sharing service component (8), and the general-purpose computing GPU is called to accelerate the compressed sensing high-speed reconstruction component (10) to rapidly reconstruct the high-resolution image and display the high-resolution image in real time.

7. The parallel compressive sensing GPU acceleration real-time imaging method according to claim 6, wherein the optical parallel complementary compressive sensing imaging completes parallel complementary measurement, and specifically comprises:

dividing an N multiplied by N effective imaging area on a spatial light modulator (2) into a plurality of blocks, wherein the size of each block is C multiplied by C, imaging the blocks in parallel to N/C multiplied by N/C pixels of a photoelectric array detector (4), and acquiring a corresponding ith observation vector column Y on the photoelectric array detector (4)⁽ⁱ⁾Satisfies the following formula:

Y⁽ⁱ⁾＝Φ⁽ⁱ⁾(X⁽ⁱ⁾)+E⁽ⁱ⁾

wherein, X⁽ⁱ⁾To represent the ith block in the spatial light modulator (2) in a column-first manner, [ phi ]⁽ⁱ⁾A projection operator for the ith blocked sub-scene image;

is provided with

Is one of the blocks formed by the target scene on the spatial light modulator (2), and is provided with

For the complementary measurement binary patterns respectively displayed on the spatial light modulator (2),

for complementary compression measurements acquired by the photo array detector (4),

is a complementary compressed measurement differential vector, each measurement obtained on the photo array detector (4) is as follows:

wherein e is₁And e₂Representing two random noises collected by complementary compression measurement;

one measurement of the complementary matrix from the following equation gives Δ y as:

wherein,

representing the product of the elements.

8. The parallel compressed sensing GPU accelerated real-time imaging method according to claim 6, characterized in that the high resolution image reconstruction and display unit (9) continuously processes the low resolution image data of the data sharing service unit (8); the method specifically comprises the following steps:

the high resolution image reconstruction and display part (9) arranges parallel blocks in the measurement data of the data sharing service part (8) according to the column priority, a complementary measurement vector in each block consists of the measurement values of a specific block distributed at the corresponding position of each low resolution image, and the complementary measurement vector of each block is synchronized with each mask vector in a sub-measurement matrix to reconstruct a compressed block observation value vector of each original image block.

9. The method according to claim 8, wherein the fast reconstructing a high resolution image and displaying the image in real time comprises:

generating a full-sampling measurement matrix according to the set compression rate, the size of the reconstructed subblock and the measurement times, extracting and generating a sub-sampling measurement matrix from the full-sampling measurement matrix, and converting the sub-sampling measurement matrix into a CUDA sparse matrix format;

combining a plurality of frames of low-resolution images into one frame; obtaining complementary measurement values through differential calculation, partitioning each frame, and resetting each block according to column vectors;

and (3) reconstructing each image according to the CUDA sparse matrix format and the GPU kernel function definition, and splicing a plurality of images into one or more frames of high-resolution images for real-time display.

Images