CN111914786B - Finger vein recognition method and system - Google Patents

Abstract

The present invention discloses a finger vein recognition SOC system and method based on ARM Cortex-M3, which realizes the whole process of the finger vein recognition system, including finger vein image collection, ROI extraction, grayscale and size normalization, feature extraction, and connection Domain denoising, feature matching and other processes, and the image processing results of each process are displayed on the LCD. The hardware acceleration of ROI recognition is carried out through the ROI hardware extraction module to improve the speed of finger vein recognition. After a large number of tests, it is concluded that this system can fully realize the function of finger vein recognition, and the recognition accuracy rate is about 95%. Using FPGA to accelerate the ROI extraction module can increase the total recognition speed by more than 15%. The system can be used as a portable finger vein recognition system for product development, and can also be used as a demonstration system for finger vein recognition in experimental teaching, which has high practical value.

Description

A finger vein recognition method and system thereof

technical field

The invention relates to the technical field of finger vein recognition, in particular to a finger vein recognition method and system thereof.

Background technique

With the development of information technology, biometric technology has become a key technology in the era of Internet artificial intelligence. Nowadays, there are more and more researches on biometric technology and more and more applications. Common biometric technologies include: fingerprints, faces, irises, finger veins, etc. Although there have been many research results on finger vein recognition algorithms in the past ten years, there are not many finger vein recognition products on the market at present. The main reason is that the vein recognition algorithm is relatively complex, which leads to difficulties in hardware implementation and long recognition time.

There are currently three implementations on the market: implementing the entire recognition algorithm on local hardware, transmitting the original image data to a PC for implementation, and sending the original image data to a cloud server for implementation. For the portable finger vein recognition system, the solution of using PC and cloud server is not very convenient. Local hardware implementations include microprocessors (such as ARM) and FPGAs. The embedded microprocessor has the characteristics of low cost, low power consumption, and simple development. However, because the CPU executes instructions one by one when it works, it does not have an advantage in running time for image processing algorithms with many multiplication operations. For complex algorithms, FPGA has no advantages in terms of implementation difficulty and power consumption.

Contents of the invention

In view of this, the object of the present invention is to provide a finger vein recognition method and system thereof, which utilizes FPGA to accelerate the ROI extraction module to improve the recognition speed.

To achieve the above object, the present invention provides the following technical solutions:

The finger vein recognition method provided by the present invention comprises the following steps:

Collect finger vein images;

Obtain ROI data of the region of interest of the finger vein image through the ROI hardware extraction module;

Read the ROI data from the ROI hardware extraction module and normalize the gray size of the ROI data;

Extract the texture features of the normalized ROI data;

Perform median filtering and connected domain denoising processing on texture features;

Determine whether it is a registration process, if yes, store the texture feature data in FLASH;

If not, then read out the stored texture feature data from the FLASH to carry out feature matching;

Display the feature matching result through LCD.

Further, the results of normalizing the gray scale of the ROI data, extracting the texture features of the normalized ROI data, and performing median filtering and connected domain denoising processing on the texture features are input into the LCD for display. .

Further, the ROI hardware extraction module uses FPGA to preprocess the finger vein image, and saves the acquired ROI of the finger vein image into the dual-port RAM of the FPGA.

Further, the control of the acquisition of the finger vein image is realized by controlling the corresponding image acquisition device through the control instruction stored in the register in the system control module; the control of the LCD display module is realized through the register storage in the system control module Control instructions to control the corresponding LCD display module to achieve.

The present invention also provides a finger vein recognition system, comprising a finger vein image acquisition module, an ROI hardware extraction module, a gray scale normalization module, a texture feature extraction module, a filter denoising module, a registration judgment module, and an LCD display module;

The finger vein image acquisition module is used to acquire finger vein images;

The ROI hardware extraction module is used to extract the region of interest of the finger vein image;

The gray size normalization module is used to read the region of interest from the ROI hardware extraction module and normalize the gray size of the region of interest;

The texture feature extraction module is used to extract the texture features of the normalized ROI data;

The filtering and denoising module is used to perform median filtering and connected domain denoising processing on texture features;

The registration judging module is used to judge whether it is a registration process, if yes, store the texture feature data in the FLASH; if not, read the stored texture feature data from the FLASH to perform feature matching;

The LCD display module is used to display the intermediate results and feature matching results of image acquisition and processing.

Further, it also includes a two-level AHB bus, and the AHB bus includes a first-level AHB bus and a second-level AHB bus;

The first-level AHB bus is respectively connected to the instruction memory and the data memory through AHB_to_SRAM;

The DDR3 is connected to the first level AHB bus;

The UART is connected to the first-level AHB bus through APBInterconnect and AHP_to_APB in turn;

The second-level AHB bus is respectively connected with the finger vein image acquisition module, the ROI hardware extraction module, the LCD display module, the FLASH read-write control module, and the dual-port Block RAM through the system control module;

The buttons, LED indicators and buzzer are also connected to the second-level AHB bus through GPIO.

Further, the system control module is provided with hardware control LCD display registers, reading Block RAM registers, camera control registers, region of interest extraction control registers, pixel accumulation and lower limit threshold registers, pixel accumulation and upper limit threshold registers, writing FLASH registers, Read the FLASH register and FLASH erase register; the CPU controls the operation of the corresponding device by configuring the corresponding register.

Further, the ROI hardware extraction module includes the following states, respectively IDLE state, FIRST_CUT state, EDGE_POINT state, CORRECTION state, WIDTH_DEFINE state, HIGHT_DEFINE state and SECOND_CUT state:

In the IDLE state, if the camera captures the finger vein image and has stored it in the BlockRAM, enter the FIRST_CUT state;

The FIRST_CUT state is used for performing background interception on the finger vein image and storing the intercepted image in the Block RAM;

The EDGE_POINT state is used to read image data from the Block RAM and calculate the boundary point of the finger;

The CORRECTION state is used to calculate the deflection angle of the finger and the value of each line that needs to be translated according to the boundary points, and then read the image data from the RAM in turn and store them back in the RAM after being translated. The translation correction of the image is actually the storage of each pixel change of address;

The WIDTH_DEFINE state is used to find the column numbers that need to be intercepted on the left and right sides of the finger image according to the boundary points after translation correction, so as to determine the width of the ROI area;

The HIGHT_DEFINE state is used to calculate the area of the distal joint of the finger by using the sliding window method, so as to determine the height of the ROI area, that is, it is necessary to calculate the line numbers of the upper and lower intercepts;

The SECOND_CUT state is used to re-read and write the image data into the RAM according to the column numbers intercepted from left and right and the row numbers intercepted from top to bottom. Finally, the data stored in the RAM starting with 0 is the data of the ROI image.

The beneficial effects of the present invention are:

The finger vein recognition SOC system and method based on ARM Cortex-M3 provided by the present invention realizes the entire process of the finger vein recognition system, including finger vein image acquisition, ROI extraction, gray scale and size normalization, feature extraction, connected domain Denoising, feature matching and other processes, and the image processing results of each process are displayed on the LCD. Through the ROI hardware extraction module, the ROI recognition hardware accelerator is implemented to improve the speed of finger vein recognition. After a large number of tests, it is concluded that this system can fully realize the function of finger vein recognition, and the recognition accuracy rate is about 95%. Using FPGA to accelerate the ROI extraction module can increase the total recognition speed by more than 15%. The system can be used as a solution for portable finger vein recognition system for product development, and can also be used as a process demonstration system for finger vein recognition in experimental teaching, which has high practical value.

Other advantages, objects and features of the present invention will be set forth in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the investigation and research below, or can be obtained from It is taught in the practice of the present invention. The objects and other advantages of the invention may be realized and attained by the following specification.

Description of drawings

In order to make the purpose, technical scheme and beneficial effect of the present invention clearer, the present invention provides the following drawings for illustration:

Figure 1 shows the implementation process of finger vein recognition and the division of software and hardware.

Figure 2 is a flow chart of the system software.

Figure 3 is a block diagram of the system.

Figure 4 is a block diagram of the SOC system based on Cortex-M3.

Figure 5 shows the Master and Slave access relationship of L1AhbMtx.

Figure 6 shows the Master and Slave access relationship of L2AhbMtx.

Fig. 7 is a state transition diagram of ROI extraction.

Figure 8 is the algorithm flow of ROI extraction.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, so that those skilled in the art can better understand the present invention and implement it, but the examples given are not intended to limit the present invention.

As shown in Figure 1, the finger vein recognition method provided by the present embodiment is a finger vein recognition SOC method based on ARM Cortex-M3, comprising the following steps:

Collect finger vein images;

Obtain the ROI data of the finger vein image by the ROI hardware extraction module;

Read the ROI data from the ROI hardware extraction module and normalize the gray size of the ROI data;

Extract the texture features of the normalized ROI data;

Perform median filtering and connected domain denoising processing on texture features;

Determine whether it is a registration process, if yes, store the texture feature data in FLASH;

If not, then read out the stored texture feature data from the FLASH to carry out feature matching;

Display the feature matching result through LCD.

In this embodiment, the image processing results of each process are displayed through the LCD, specifically including normalizing the gray size of the ROI data, extracting the texture features of the normalized ROI data, and performing median filtering on the texture features and connected domain denoising processing results are input to the LCD for display;

The ROI hardware extraction module adopts FPGA to accelerate the ROI extraction module to realize finger vein image preprocessing. The FPGA in this embodiment adopts the XC7A75T of the Artix-7 series of Xilinx Company, adopts CMSDK to build the SOC system of Cortex-M3, and the ROI hardware The extraction module is used to obtain the region of interest of the finger vein image and save it in the dual-port RAM of the FPGA;

The FLASH provided in this embodiment is MT25QL128. It is more convenient and time-saving to implement the read-write control module of the FLASH by using hardware. Therefore, the read-write control of the FLASH is implemented by hardware logic circuits.

As shown in Figure 2, in order to visually display the results of image processing in each stage of the finger vein recognition process in this embodiment, the LCD needs to display the processed image in real time, so in the image acquisition and ROI extraction stages, the LCD display is controlled by the hardware. Take control.

In this embodiment, the collection of finger vein images adopts a camera to capture and display images in real time. The camera is directly connected to the IO port of the FPGA, which saves image acquisition time. The control of the camera is realized by hardware. When the camera module detects that a finger is inserted into the collection After the card slot of the device, the collected images are saved to the dual-port RAM of the FPGA.

As shown in Figure 3, the finger vein recognition system provided by the present embodiment is based on the SOC design of Xilinx Artix-7, and includes peripheral circuits, and the peripheral circuits include near-infrared LED light-emitting circuits, power conversion circuits, and buttons that are respectively connected to FPGAs. circuit, LED indicator circuit, buzzer circuit.

As shown in Figure 4, in this embodiment, the Cortex-M3 core of DesignStart Eval is used for development, and the development is carried out on the basis of the CMSDK bus and peripherals provided by ARM Company. The bus includes two levels of AHB bus, that is, the first level AHB bus and second level AHB bus. The first-level AHB bus is connected to the core processor;

In the figure, I-code represents the instruction bus, which is mainly used to fetch instructions from the memory; D-code represents the data bus, which is mainly used to fetch data from the memory; System represents the system bus, which is mainly used to access system peripheral modules.

The first-level AHB bus is connected to the second-level AHB bus through the AHB bridge; the second-level AHB bus is respectively connected to the finger vein image acquisition module, the ROI hardware extraction module, the LCD display module, and the FLASH read-write control module through the system control module. The module and the dual-port Block RAM are connected; the second-level AHB bus is connected with buttons, LED indicators and buzzers through GPIO.

The Cortex-M3 kernel provided by this embodiment is provided with an instruction memory and a data memory, and the instruction memory and the instruction data memory are connected to the first-level AHB bus through AHB_to_SRAM respectively; DDR3 is also connected to the first-level AHB bus ; The UART is connected to the first-level AHB bus through the APB bus and the AHB-to-APB bridge APBInterconnect in turn;

Among them, the instruction memory is used to store the instructions operated by the Cortex-M3 core; the instruction data memory represents the program memory, which is used to store the data operated by the Cortex-M3 core; DDR3 represents the memory, which is used for the cache of image acquisition; UART represents the universal asynchronous transceiver The device, that is, the serial port, is used to send the feature data and matching results to the host computer (such as a computer).

The finger vein image acquisition module, the LCD display control module, the FLASH read-write control module, the dual-port BlockRAM and the ROI hardware extraction module are all connected to the second-level AHB bus through the system control module. The device is also connected to the second-level AHB bus through GPIO.

Among them, GPIO represents a general-purpose input and output port, and there are 16 ports, which can be configured as input or output modes through registers.

As shown in Figure 5, the first-level AHB bus is provided with 5 Slave interfaces (S0~S4) and 5 Master interfaces (M0~M4), the S0 interface is connected to the I-code bus of the Cortex-M3 core, and the S1 The interface is connected to the D-code bus of the Cortex-M3 core, and the S3 interface is connected to the system bus of the Cortex-M3 core. S3 and S4 are reserved for future system expansion of DMA modules and coprocessors. The M0 and M1 interfaces are respectively connected to the instruction memory and the instruction data memory, M2 is connected to the AHB-to-APB bridge, M3 is connected to the DDR, and M4 is connected to the AHB-to-AHB bridge to expand the second-level AHB bus. The access relationship between each Slave and Master is shown in Figure 4.

As shown in FIG. 6, the second-level AHB bus is provided with one Slave interface (S0) and two Master interfaces (M0, M1). The S0 interface is connected to the AHB to AHB bridge, the M0 interface is connected to the system control module, and the M1 interface is connected to the GPIO.

The Cortex-M3 core provided in this embodiment needs to access modules on each bus, and must allocate address space to each module. The allocated address space is shown in Table 1.

Table 1 Address mapping of each module in the system

The system control module provided by this embodiment is an important control axis connecting software and hardware, and converts control instructions into hardware control signals, thereby realizing the control of the camera module, ROI accelerator module, LCD display module and FLASH read-write module.

The system control module provided in this embodiment has 9 registers, and software can control the work of corresponding hardware by configuring these 9 registers. These 9 registers are shown in Table 2.

Table 2 Register list of the system control module

This embodiment also provides a register setting method for controlling the finger vein image recognition chip, the register setting includes a hardware control LCD display register, a block RAM read register, a camera control register, an area of interest extraction control register, and a pixel accumulation register. And lower limit threshold register, pixel accumulation and upper limit threshold register, write FLASH register, read FLASH register and FLASH erase register; the specific settings are as follows:

The hardware control LCD display register (LCD_HW_CTRL_DISP) is configured with an LCD hardware control bit; when the LCD hardware control bit is configured as "1", the hardware logic circuit is started to control the signal of the display LCD and drive the LCD display; the LCD hardware control bit When the configuration is "0", the LCD display is controlled by software, and its detailed features are:

The address is 0x5000 0000, the software will access this register by reading and writing this address through the Cortex-M3 core.

The reset value is 0x0000 0000, that is, the software controls the LCD to display by default.

Bits[31:1] are reserved.

Bit [0] is the LCD hardware control bit, named lcd_hw_ctrl, this bit is readable and writable, when this bit is configured as "1", the LCD display is controlled by hardware, when this bit is configured as "0", the LCD is controlled by software show.

The use scenario and configuration method of this register are: when the system is turned on and in the registration and identification mode, first write 0x1 to the address 0x5000 0000, that is, configure the lcd_hw_ctrl bit of the register to "1", and start the FPGA hardware logic circuit to control the LCD LCD_CS, LCD_RS, LCD_WR, LCD_RD and other signals and drive the LCD display, so that the images collected by the camera are displayed on the LCD screen in real time. When the software detects that the ROI extraction is completed, write 0x0 into the address 0x50000000, that is, configure the lcd_hw_ctrl bit of the register as "0", then the control right of the LCD display is given to the software, and the LCD_CS, LCD_RS, LCD_WR, LCD_CS, LCD_RS, LCD_WR, LCD_RD and other signals and drive LCD display. The display of ROI extraction results, normalization, feature extraction, connected domain denoising, median filtering, and feature matching results are all controlled by software.

The read Block RAM register is configured with a Block RAM read control bit and a Block RAM address bit;

When the Block RAM read control bit is "1", the data in the corresponding address of the Block RAM is read back through software, and when the bit is configured as "0", the Block RAM is not read;

When the Block RAM address bit is 1, the data in the address specified by the bit will be read;

The read Block RAM register (READ_BRAM) is used to read data from the Block RAM inside the FPGA, and its use is divided into two steps of reading and writing. When writing to this register, its detailed characteristics are:

The address is 0x5000 0004, and the software will access this register by writing this address through the Cortex-M3 core.

The reset value is 0x0000 0000, that is, the Block RAM inside the FPGA is not read by default.

Bits[31:21] are reserved.

Bit [20] is the Block RAM read control bit, named bram_rd, this bit can only be written. When this bit is configured as "1", the software will read back the data in the corresponding address of the Block RAM; when this bit is configured as "0", the Block RAM will not be read.

Bits[19:17] are reserved.

Bits [16:0] are Block RAM address bits, named bram_addr, which can only be written. When the bram_rd bit is 1, the data at the address specified by this bit will be read. Since the image pixels captured by the camera are 320*240, the maximum value of the address represented by this bit is 320*240-1, which is 0x12BFF.

When reading this register, its detailed characteristics are

The address is 0x5000 0004, the software will access this register by reading this address through the Cortex-M3 core.

The reset value is 0x0000 0000, that is, the default read-back data is 0.

Bits[31:8] are reserved.

Bits [7:0] are read data bits, named bram_data, which can only be read. The data corresponding to the address of the read Block RAM will be stored in this bit.

The usage scenario and configuration method of this register are: after the ROI extraction hardware accelerator module in the FPGA completes the ROI data extraction, the ROI data is stored in the Block RAM in the FPGA, and the software needs to read the ROI data back for subsequent algorithm processing. When you need to configure this register to read the data in the Block RAM. For example, if you need to read the data in the address 0x1234, you first need to write the register to 0x00101234, and then read the register, the data in the bits [7:0] is the data in the Block RAM address 0x1234.

The camera control register (CAMERA_CTRL) is configured with an image pixel accumulation and bit, an image acquisition completion flag bit, and a camera control bit;

The cumulative sum of image pixels is used to calculate the cumulative sum of pixels of each frame of image collected,

The image acquisition completion flag is used to store the image information into the Block RAM and set the position to "1" when the camera detects that a finger is inserted into the card slot of the image acquisition device;

When the configuration bit of the camera control bit is "1", the camera is started to collect images, and when the configuration bit is "0", the camera is turned off;

Its detailed characteristics are

The address is 0x5000 0008, the software will access this register by reading and writing this address through the Cortex-M3 core.

The reset value is 0x0000 0000, that is, the camera is not turned on by default.

Bits[31:29] are reserved.

Bits [28:4] are the cumulative sum of image pixels, named pix_color_sum. When the camera captures images, the value of this bit will be updated after each frame of image to store the cumulative sum of pixels of each frame of image.

Bits[3:2] are reserved.

Bit [1] is the image acquisition completion flag, named camera_finish_flag, when the camera detects that a finger is inserted into the card slot of the image acquisition device, the camera will store the image information into the Block RAM, and set the bit to "1".

Bit [0] is the camera control bit, named camera_ctrl, when the bit is configured as "1", the camera is started to collect images, and when the bit is configured as "0", the camera is turned off.

The usage scenario and configuration method of this register are as follows: when the registration or recognition mode is turned on, first configure the camera_ctrl bit of the register to be "1", start the camera, and when the camera_finish_flag bit of the register is read as "1", it means that the finger The image has been captured, write the camera_ctrl bit and camera_finish_flag bit of this register to 0 to turn off the camera and clear the flag bit. The value of pix_color_sum of this register can be read when the camera is turned on to obtain the pixel accumulation value of the image when the finger is inserted into the card slot and not inserted into the card slot, so as to adjust the pixel for detecting whether the finger is inserted into the card slot according to this value Accumulate and Threshold.

The ROI extraction control register (ROI_CTRL) is configured with ROI width bits, ROI height bits, ROI extraction completion flag bits, and ROI extraction control bits;

The ROI width bit; used to store the width data of the ROI collected;

The height bit of the region of interest; used to store the collected height data of the region of interest;

The region of interest extraction completion flag bit; when the region of interest extraction is completed, the configuration bit is "1";

The region of interest extraction control bit; when the configuration bit is "1", the ROI hardware accelerator module is started to extract the region of interest, and when the configuration bit is "0", the ROI module is closed;

Its detailed characteristics are

The address is 0x5000 000C, the software will access this register by reading and writing this address through the Cortex-M3 core.

The reset value is 0x0000 0000, that is, the ROI extraction function is disabled by default.

Bits [31:24] are the width bits of the region of interest, namely ROI_width. After the ROI is extracted, the obtained width data of the region of interest will be stored into this bit.

Bits [23:16] are height bits of the region of interest, ie ROI_height. After the ROI is extracted, the obtained height data of the region of interest will be stored into this bit.

Bits[15:2] are reserved.

Bit [1] is the region of interest extraction completion flag, named ROI_finish_flag, when the ROI hardware accelerator module in the FPGA finishes running, that is, the region of interest extraction is completed, set the bit to "1".

Bit [0] is the region of interest extraction control bit, named ROI_ctrl, when the bit is configured as "1", the ROI hardware accelerator module is started to extract the region of interest, and when it is "0", the ROI module is turned off.

The usage scenario and configuration method of this register are: when the camera_finish_flag bit of the camera control register (CAMERA_CTRL) is read as "1", it means that the finger image has been captured. The ROI hardware accelerator module starts ROI extraction. When the ROI_finish_flag bit of the register is read as "1", it indicates that the region of interest has been extracted. Write the ROI_ctrl bit and ROI_finish_flag bit of the register to 0 to close the ROI module and clear the ROI. Zero flag, at this time, you can read the ROI_width and ROI_height bits of this register to get the width and height of the extracted region of interest, so that it can be used in subsequent algorithm processing.

The pixel accumulation and the lower limit threshold register (PIX_SUM_MIN) are configured with image pixel accumulation and lower limit threshold bits;

The image pixel accumulation and lower limit threshold bit; a frame of pixel accumulation lower limit threshold for storing the collected image;

Its detailed characteristics are

The address is 0x5000 0010, the software will access this register by reading and writing this address through the Cortex-M3 core.

The reset value is 0x 0040 0000, that is, the default is that when the cumulative sum of pixels of each frame image is less than 0x400000, no finger is inserted into the card slot of the image acquisition device.

Bits[31:25] are reserved.

Bits [24:0] are image pixel accumulation and lower limit threshold bits, named pix_sum_min, which are readable and writable. When the cumulative sum of pixels of a frame of the image captured by the camera is less than this threshold, it means that the finger is not inserted into the card slot; otherwise, if it is greater than or equal to this threshold and less than or equal to the value of the pix_sum_max bit in the pixel accumulation and upper limit threshold register, it means that the finger is inserted into the card slot of the image capture device.

The usage scenario and configuration method of this register are: if it is found that the detection of fingers extending into the card slot is inaccurate when debugging the system, then when the camera is turned on, by reading the pix_color_sum bit in the camera control register (CAMERA_CTRL), it is obtained that the near-infrared When the LED is turned off, the value of the sum of pixels when the finger is inserted into the card slot and when it is not inserted into the slot, and the pix_sum_min bit of this register is configured according to this value to set the threshold of the sum of image pixels.

Described pixel accumulation and upper limit threshold register (PIX_SUM_MAX) are configured with image pixel accumulation and upper limit threshold position;

The image pixel accumulation and upper limit threshold bit; used to store a frame of pixel accumulation upper limit threshold of the image collected;

Its detailed characteristics are

The address is 0x5000 0014, the software will access this register by reading and writing this address through the Cortex-M3 core.

The reset value is 0x 00B0 0000, that is, the default is that when the cumulative sum of pixels of each frame image is greater than 0xB00000, no finger is inserted into the card slot of the image acquisition device.

Bits[31:25] are reserved.

Bits [24:0] are image pixel accumulation and upper limit threshold bits, named pix_sum_max, which are readable and writable. When the cumulative sum of pixels of a frame of the image captured by the camera is greater than this threshold, it means that the finger is not inserted into the card slot. Otherwise, if it is less than or equal to this threshold and greater than or equal to the value of the pix_sum_min bit in the pixel accumulation and lower limit threshold register, it means that the finger is extended. into the card slot of the image capture device.

The usage scenario and configuration method of this register are: if it is found that the detection of fingers extending into the card slot is inaccurate when debugging the system, then when the camera is turned on, by reading the pix_color_sum bit in the camera control register (CAMERA_CTRL), it is obtained that the near-infrared The value of the sum of pixels when the finger is inserted into the card slot and not inserted into the card slot when the LED is turned on, and the pix_sum_max bit of this register is configured according to this value to set the upper threshold of the image pixel sum.

The write FLASH register (FLASH_WRITE),

Configured with FLASH write data bit, FLASH write address bit;

The FLASH write data bit is configured to write FLASH data;

The FLASH write address bit is configured to write the address of the FLASH;

Its detailed characteristics are

The address is 0x5000 0018, the software will access this register by reading and writing this address through the Cortex-M3 core.

The reset value is 0x 0080 0000, that is, the starting address of the default FLASH storage vein pattern feature is 0x00800000.

Bits [31:24] are FLASH write data bits, named flash_wdata, that is, the data that needs to be written into FLASH. This bit is readable and writable.

Bits [23:0] are FLASH write address bits, named flash_waddr, that is, the address that needs to be written to FLASH. This bit is readable and writable.

The usage scenario and configuration method of this register are as follows: if the current registration mode is used, after the connected domain denoising and median filtering of the vein pattern, the feature data needs to be stored in the FLASH. The data is stored in the corresponding address of FLASH. For example, the third pixel value of feature data is 0xFF, and the address to be stored is 0x800003, then configure this register as 0xFF80 0003. The FLASH in this embodiment uses MT25QL128, the storage space is 16M bytes, and its address range is 0x000000～0xFFFFFF, and there is still 8MB of storage space from 0x800000 to the end, and this 8MB is specially used for storing vein pattern features. A vein texture feature is 96*64 bytes, that is, 6144 bytes. In this embodiment, the first address stored in each vein pattern feature image is 8 KB aligned, and a total of 1024 vein pattern feature images can be stored. The first feature storage address of the first feature is 0x800000, the first feature storage address of the second feature is 0x802000, the first feature storage address of the third feature is 0x804000, and so on.

The read FLASH register (FLASH_READ is configured with a FLASH read address bit; the FLASH read address bit is configured for the address address of the FLASH read;

Its detailed characteristics are

The address is 0x5000 001C, the software will access this register by reading and writing this address through the Cortex-M3 core.

The reset value is 0x 0080 0000, that is, the default starting address for reading FLASH is 0x 00800000.

Bits[31:25] are reserved.

Bits [23:0] are the FLASH read address bits, named flash_raddr bit, that is, the address of the FLASH to be read. This bit is write only.

When reading this register, its detailed characteristics are

The address is 0x5000 001C, the software will access this register by reading and writing this address through the Cortex-M3 core.

The reset value is 0x 0000 0000, that is, the FLASH data read by default is 0x0.

Bits[31:8] are reserved.

Bits [7:0] are the FLASH read address bits, named flash_rdata bits, that is, the data corresponding to the read FLASH address. This bit is read only.

The usage scenario and configuration method of this register are as follows: if the current recognition mode is used, after the connected domain denoising and median filtering of the vein pattern, it is necessary to read out the feature data previously stored in the FLASH for feature matching. The software configures this register to read out the characteristic data of the FLASH corresponding address. For example, if you need to read the data in the FLASH address 0x801234, you first need to write the register to 0x00801234, and then read the register, the data in the bits [7:0] is the data in the FLASH address 0x801234.

The read FLASH eraser (FLASH_CLR) is configured with an erase selection bit, a starting address bit for FLASH erasing;

When the erasing selection bit is configured as "1", all the vein pattern features in the FLASH are erased; when configured as "0", only a texture feature image starting from the flash_caddr address is erased;

Its detailed characteristics are

The address is 0x5000 0020, the software will access this register by reading and writing this address through the Cortex-M3 core.

The reset value is 0x 0080 0000, that is, the default FLASH erase address is 0x 00800000, and only one texture feature image starting from this address will be erased.

Bits[31:25] are reserved.

Bit [24] is the erase selection bit, named CA. When this bit is configured as "1", all the vein pattern features in FLASH will be erased. When this bit is configured as "0", only the flash_caddr address will be erased. A texture feature image of . This bit is write only.

Bits [23:0] are the starting address of FLASH erasing, named flash_caddr bit, that is, the first address of FLASH to be erased. This bit is write only.

The usage scenario and configuration method of this register are: if you want to delete the vein pattern feature stored in FLASH, you need to configure this register. If the CA bit of this register is set to "1", no matter what the value of the flash_caddr bit is, all feature data after address 0x 00800000 in FLASH will be erased. If you only want to erase a certain characteristic image, configure the CA bit of this register as "0", and specify the starting address of erasing through flash_caddr. The 8KB data starting with flash_caddr address will be erased. At this time, the configuration of the flash_caddr bit requires 8KB alignment.

After the camera controller provided by this embodiment opens the camera module by configuring the camera control register (CAMERA_CTRL), the system control module generates a camera_start signal to start the camera to collect images in real time, and stores the collected image data into the Block RAM, and the LCD controller module Read data from Block RAM in real time for display.

The camera controller in this embodiment is provided with a finger position detection module for detecting whether the finger is inserted into the card slot of the acquisition device, and the finger position detection module detects according to the following steps:

Get vein images with fingers;

Calculate the pixel sum of the vein image;

Determine whether the pixel sum of the vein image is within the preset threshold range, if so, put your finger into the card slot;

If not, the finger is not inserted into the card slot;

Move your finger over the card slot and repeat the cycle.

The default range of the preset threshold provided in this embodiment is as follows: the lower threshold is 0x400000, and the upper threshold is 0xB00000.

When it is detected that the pixel accumulation value of the image is within this interval, it means that a finger has been inserted into the card slot, and then the image delayed by 100 frames is collected as the original image of the finger vein and the camera_finish signal is generated as a sign of the end of the collection. The effect of delaying 100 frames is to stabilize the image. The cumulative sum of image pixels can be read out through the pix_color_sum bit of the register CAMERA_CTRL, and the lower and upper thresholds for finger detection can be configured through the registers PIX_SUM_MIN and PIX_SUM_MAX.

As shown in Figure 7 and Figure 8, described ROI hardware extraction module adopts Verilog hardware description language to write ROI hardware accelerator, when camera collects finger vein image, ROI hardware extraction module begins to work,

The ROI algorithm in the present embodiment adopts hardware description language Verilog to carry out FPGA development, and main program adopts finite state machine (Finite-state machine, FSM) to design, and the extraction process of ROI is finished by 7 states, is IDLE state respectively: namely Idle state; FIRST_CUT state: initial interception state; EDGE_POINT state: finger boundary point detection state; CORRECTION state: rotation correction state; WIDTH_DEFINE state: width definition state; HIGHT_DEFINE state: height definition state; SECOND_CUT state: two Secondary interception status.

As shown in Figure 7, Figure 7 is a ROI extraction state transition diagram, the IDLE state, when the finger is not inserted into the card slot, is always in this state, will detect that the finger is inserted into the card slot and store the image into the Block RAM After that, enter the FIRST_CUT state;

The FIRST_CUT state is used for performing background interception on the finger vein image and storing the intercepted image in the Block RAM;

The purpose of the FIRST_CUT state is to cut off the background area and reduce the size of the image. After the image is cut, it enters the EDGE_POINT state. The image cut is completed by reading and writing Block RAM;

In the EDGE_POINT state, read the image data from the RAM in turn, and calculate the 4 boundary points of the finger;

In the CORRECTION state, first calculate the deflection angle of the finger and the value to be shifted for each line according to the boundary points obtained in the previous state, and then read the image data from the RAM in turn and store them back in the RAM after being shifted. The translation correction of the image is actually It is the change of the storage address of each pixel;

In the WIDTH_DEFINE state, according to the boundary points after translation correction, find out the column numbers that need to be intercepted on the left and right sides of the finger image to determine the width of the ROI area;

In the HIGHT_DEFINE state, use the sliding window method to find the area of the distal joint of the finger, so as to determine the height of the ROI area, that is, it is necessary to calculate the line number of the upper and lower intercepts;

In the SECOND_CUT state, the image data is re-read and written into the RAM according to the column numbers intercepted on the left and right and the row numbers intercepted on the top and bottom. Finally, the data stored in the RAM starting with 0 is the data of the ROI image;

The extraction of ROI is completed through the above 7 states, the extracted image data is stored in the Block RAM, and then the ROI_finish signal is generated to notify the software that the data can be read from the RAM for subsequent algorithm processing.

The ROI module is designed to take advantage of the speed of the hardware accelerator, and the software solution is used to implement the same ROI algorithm. The clock frequency of both hardware and software is 36MHz, and the running time of each step and the comparison of the total running time are shown in Table 3. It can be seen that the total running time of this ROI method is 3.822ms when FPGA is used, and 78.547ms is needed by software. At the same clock frequency, hardware is more than 20 times faster than software.

Table 3 Comparison of running time of ROI module software and hardware implementation methods (ms)

Algorithms after the ROI of this system, such as normalization, direction segmentation, connected domain denoising, etc., are all implemented in software. After MDK running tests, it can be seen that the total running time of subsequent algorithms is about 480ms, so it can be concluded that the ROI module Hardware acceleration can increase the overall recognition speed by more than 15%.

The above-mentioned embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Equivalent substitutions or transformations made by those skilled in the art on the basis of the present invention are all within the protection scope of the present invention. The protection scope of the present invention shall be determined by the claims.

Claims (9)

1. Finger vein recognition system, its characterized in that: the device comprises a finger vein image acquisition module, an ROI hardware extraction module, a gray scale size normalization module, a texture feature extraction module, a filtering denoising module, a registration judgment module and an LCD display module;

the finger vein image acquisition module is used for acquiring finger vein images;

the ROI hardware extraction module is used for extracting a region of interest of the finger vein image;

the gray scale size normalization module is used for reading the region of interest from the ROI hardware extraction module and normalizing the gray scale size of the region of interest;

the texture feature extraction module is used for extracting texture features of the normalized ROI data;

the filtering denoising module is used for carrying out median filtering and connected domain denoising on the texture characteristics;

the registration judging module is used for judging whether the registration process is adopted, and if so, the line characteristic data are stored in FLASH; if not, reading the stored line characteristic data from the FLASH to perform characteristic matching;

the LCD display module is used for displaying intermediate results and characteristic matching results in the image acquisition and processing process;

the ROI hardware extraction module adopts an FPGA to preprocess the finger vein image, and saves the acquired finger vein image interested region into a dual-port RAM of the FPGA;

the control of the acquisition of the finger vein image is realized by controlling corresponding image acquisition equipment through a control instruction stored in a register in a system control module; the control of the LCD display modules is realized by controlling the corresponding LCD display modules through control instructions stored in registers in the system control modules;

the ROI hardware extraction module comprises the following states, namely an IDLE state, a first_CUT state, an EDGE_POINT state, a CORRECTION state, a WIDTH_DEFINE state, a HIGHT_DEFINE state and a SECOND_CUT state:

in the IDLE state, if the camera acquires finger vein images and the finger vein images are stored in the Block RAM, the first_CUT state is entered;

the first_cut state is used for carrying out background interception on the finger vein image and storing the intercepted image into a Block RAM;

the EDGE_POINT state is used for reading image data from the Block RAM and calculating the boundary POINTs of the fingers;

the correct state is used for calculating the finger deflection angle and the value of each line to be translated according to the boundary point, then sequentially reading image data from the RAM, and storing the image data back into the RAM after translation, wherein the translation CORRECTION of the image is actually the change of the storage address of each pixel point;

the WIDTH_DEFINE state is used for finding out column numbers to be intercepted on the left and right sides of the finger image according to the boundary points after translational correction so as to determine the WIDTH of the ROI;

the HIGHT_DEfine state is used for solving the region of the far-end joint of the finger by adopting a sliding window method so as to determine the height of the region of the ROI, namely, the line number which needs to be intercepted up and down is calculated;

the second_cut state is used for re-reading and writing the image data into the RAM according to the left-right intercepted column number and the up-down intercepted line number, and finally the data stored in the RAM with 0 as the initial address is the data of the ROI image.

2. The system of claim 1, wherein: the system also comprises a two-level AHB bus, wherein the AHB bus comprises a first-level AHB bus and a second-level AHB bus;

the first-stage AHB bus is respectively connected with the instruction memory and the data memory through the AHB_to_SRAM;

DDR3 is connected with the first-stage AHB bus;

UART is connected with the first-stage AHB bus through APBInterconnect and AHP_to_APB in sequence;

the second-stage AHB bus is respectively connected with the finger vein image acquisition module, the ROI hardware extraction module, the LCD display module, the FLASH read-write control module and the dual-port Block RAM through the system control module;

the key, the LED indicator light and the buzzer are also hung on the second-stage AHB bus through the GPIO.

3. The system of claim 1, wherein: the system also comprises a kernel processor, wherein the first-stage AHB bus is connected with the kernel processor.

4. The system of claim 1, wherein: the core processor adopts an ARM Cortex-M3 core processor.

5. The system of claim 1, wherein: the system control module is provided with a hardware control LCD display register, a read Block RAM register, a camera control register, an interested region extraction control register, a pixel accumulation and lower limit threshold value register, a pixel accumulation and upper limit threshold value register, a write FLASH register, a read FLASH register and a FLASH erasure register; and each register is provided with a control instruction for controlling corresponding equipment.

6. A method for performing finger vein recognition using the finger vein recognition system of any one of claims 1-5, characterized by: the method comprises the following steps:

collecting finger vein images;

obtaining ROI data of a region of interest of the finger vein image through an ROI hardware extraction module;

reading the ROI data from the ROI hardware extraction module and normalizing the gray scale of the ROI data;

extracting texture characteristics of the normalized ROI data;

carrying out median filtering and connected domain denoising treatment on the grain characteristics;

judging whether the registration process is adopted, if so, storing the texture feature data into FLASH;

if not, reading the stored line characteristic data from the FLASH to perform characteristic matching;

and displaying the feature matching result through an LCD.

7. The method of claim 6, wherein: and (3) carrying out normalization processing on the gray scale of the ROI data, extracting texture characteristics of the normalized ROI data, and inputting results of median filtering and connected domain denoising processing on the texture characteristics into an LCD for display.

8. The method of claim 6, wherein: the ROI hardware extraction module adopts an FPGA to preprocess the finger vein image, and saves the acquired finger vein image region of interest into a dual-port RAM of the FPGA.

9. The method of claim 6, wherein: the control of the acquisition of the finger vein image is realized by controlling corresponding image acquisition equipment through a control instruction stored in a register in a system control module; the control of the LCD display modules is realized by controlling the corresponding LCD display modules through control instructions stored in registers in the system control modules.

Images