CN110414633B - System and recognition method for handwritten font recognition - Google Patents

Abstract

The invention discloses a system and a method for recognizing handwritten fonts. The method is based on the handwriting font recognition of the flexible wearable bracelet, the problems of environment light and resolution are solved, and the method is more stable and lower in cost; the invention uses the trained model to identify the handwritten font by means of the convolutional neural network, and has the advantages of short time consumption, high identification rate and stronger robustness; the invention can be suitable for various specific occasions by flexibly modifying the structure of the output layer and by transfer learning.

Description

System and recognition method for handwritten font recognition

technical field

The invention relates to a font recognition device, in particular to a system and a recognition method for handwritten font recognition.

Background technique

The earliest character recognition originated in the United States in the early 1950s. As an extremely important part of industrial applications, the development of character recognition has always attracted much attention.

At present, character recognition mainly uses Optical Character Recognition (OCR). The general process is: check the characters on the paper through electronic equipment (scanner, camera, etc.) Tilt correction, perform layout analysis such as segmenting and branching on the navigation picture, and then perform character segmentation, input the segmented single character into a specific character recognition program such as convolutional neural network, SVM support vector machine, etc., and finally output the recognized character.

At present, in this field, the problem of pixel distortion naturally exists due to the use of images obtained by shooting.

SUMMARY OF THE INVENTION

Purpose of the invention: The purpose of the present invention is to provide a system and a recognition method for handwritten font recognition, so as to solve the problems of being easily affected by the environment and having an unstable recognition rate.

Technical solution: The system for handwritten font recognition according to the present invention includes a flexible wearable wristband, a sensor signal acquisition and processing unit and a convolutional neural network unit, the flexible wearable wristband and the sensor signal acquisition unit and a convolutional neural network unit. The neural network units are all electrically connected.

The flexible wearable wristband includes a flexible substrate on which a pressure deformation sensor is arranged, and an insulating encapsulation layer is covered on the flexible substrate.

The recognition method of the system for handwritten font recognition according to the present invention comprises the following steps:

(1) The flexible wearable wristband obtains the wrist dynamic parameter information at the characteristic points when the wrist bends and moves through the change of the sensor resistance;

(2) The sensor signal acquisition processing unit converts the acquired wrist dynamic parameter information into electrical response signals and outputs them;

(3) The convolutional neural network unit trains the neural network model to obtain the optimal handwritten convolutional neural network discriminant model, and then the electrical response signal is input into the optimal handwritten convolutional neural network discriminant model, which is predicted after the middle layer of the network. Recognition of handwritten fonts.

Wherein, in the described step (3), it is specifically:

(a) Use different people to write with different pen-holding postures, different strengths, and different fonts to write unified characters, and build a dataset for training the neural network;

(b) constructing the structure of the convolutional neural network model, and setting the number of output units of the neural network to T, and using the network data set for pre-training to initialize the weight parameters;

(c) Input the constructed data set into the constructed convolutional neural network model, perform training to optimize the weight parameters, and finally obtain the optimal handwritten convolutional neural network discriminant model;

(d) Input the electrical response signal obtained by the sensor signal acquisition and processing unit into the optimal handwritten convolutional neural network discrimination model, and output the discrimination result.

The data set format in the step (a) is (source, target), where the source is the wrist dynamics parameter information at the feature points that have been collected and processed, and the target is a handwritten font encoded in one-hot encoding format.

In the step (c), in the training process, cross-validation is used to check whether the model is over-fitted, and the cross-entropy loss function softmax is used as the error loss function of the network to carry out back-propagation training, until the error no longer decreases for consecutive N training cycles. So far, the optimal handwriting discriminant model is obtained.

Beneficial effects: the present invention is based on the handwritten font recognition of the flexible wearable bracelet, eliminates the problem of viewing ambient light and resolution, is more stable, and has a lower cost; the present invention uses a convolutional neural network to recognize handwritten fonts with a trained model, The time is short, the recognition rate is high, and the robustness is stronger; the present invention can flexibly modify the output layer structure and transfer learning, and can be applied to various specific occasions, such as specific handwritten font recognition.

Description of drawings

Fig. 1 is the module schematic diagram of the present invention;

2 is a schematic structural diagram of a flexible wearable wristband;

Fig. 3 is the flow chart of the electrical response acquisition module;

Fig. 4 is the flow chart of neural network unit;

FIG. 5 is a working flow chart of the present invention.

Detailed ways

The present invention will be further described below with reference to the accompanying drawings.

As shown in Figure 1-2, for example, to realize the recognition of Chinese characters "east", "south", "big" and "learning", the system that can be used for handwritten font recognition includes:

(1) Flexible wearable bracelet

The flexible wearable wristband can obtain the wrist dynamic parameter information at the three characteristic points when the wrist is bent through the change of the sensor resistance. The flexible pressure/deformation sensor realizes the acquisition of feature point information, thereby realizing the acquisition of dynamic parameters that characterize wrist movement.

The flexible wearable wristband is composed of a pressure/deformation sensor 2, a VHB double-sided hydrogel flexible base layer 1 and an insulating encapsulation layer 3, wherein the pressure/deformation sensor 2 is a three-dimensional cross-linking of MXene and multi-walled carbon nanotubes (MWCNTs). network.

When the pressure/deformation sensor is deformed by pressure, the three-dimensional cross-linked network composed of MXene and multi-walled carbon nanotubes (MWCNTs) will generate lateral cracks, thereby reducing the conduction path, increasing the conduction resistance, and increasing the resistance. The degree of resistance increase is used to characterize the dynamic state of the wrist.

The preparation of the flexible wearable bracelet includes the following steps: cutting the VHB double-sided hydrogel into a fixed cuboid with a length of 18cm, a width of 1cm and a thickness of 5mm; using a laser engraving machine to engrave three feature points on a fixed position of the mask, The length of the mask is 18.2cm, the width is 1.2cm, and the thickness is 6mm. The positions of the three feature points are the middle of both sides of the wrist and the lower side. After that, the mask is fixed on the VHB double-sided hydrogel flexible base layer; The prepared MXene and multi-walled carbon nanotube dispersions are sprayed on the mask plate by spraying process, and the spraying is finished after 5-6 repetitions; PVP aqueous solution is sprayed on the other side of the VHB double-sided hydrogel flexible base layer , as an encapsulation layer and an electrical insulating layer; peel off the mask from the VHB double-sided hydrogel flexible base layer, and draw out lead wires at three feature points as a conductive layer;

(2) Sensor signal acquisition processing unit

The sensor signal acquisition and processing unit includes an electrical response acquisition module for acquiring the electrical signals sig_1, sig_2, sig_3 and an electrical response processing module for processing the electrical response signals sig_1, sig_2, sig_3. As shown in Figure 3, when the AI chip detects that one of the electrical signals sig_1, sig_2, and sig_3 has changed through the analog-to-digital converter, it will control the analog-to-digital converter to collect the electrical signals sig_1, sig_2, and sig_3, and store them in the buffer. , until the electrical signals sig_1, sig_2, and sig_3 remain unchanged, the collection work is ended, wherein the number of statistical collection points is W, and after processing by the electrical response processing module, the processed electrical signals are output through matrix splicing.

The electrical signal processing module uses the recorded initial voltage signal U ₀ to process the voltage signal obtained by the electrical response acquisition module as (U _i -U ₀ )/U ₀ , where U _i is the voltage signal obtained at the three characteristic points , i=1,2,...,W. The processed sig_1, sig_2 and sig_3 are spliced into a two-dimensional matrix, the dimension of which is [W, 3], where the first channel of the second dimension is the processed sig_1 electrical signal, and the second channel of the second dimension is the processed electrical signal. sig_2 electrical signal, the third channel of the second dimension is the processed sig_3 electrical signal. Finally, zeros will be added at the end of each row until the length of each row of the matrix is padded to 2000.

(3) Convolutional neural network unit

In order to achieve a higher recognition rate, the present invention uses a convolutional neural network with a special structure. According to this example, the network input is [2000, 3] and the softmax classification layer is a 4-dimensional vector, i.e. 4 classifiers.

The structure of the convolutional neural network is as follows:

[2000,3] Two-dimensional tensor input->convolutional layer->Relu->pooling layer->convolutional layer->ReLu->pooling layer->convolutional layer->ReLu->pooling layer- >->Convolutional layer->ReLu->Pooling layer->Fully connected layer1->Fully connected layer2->softmax layer[4] dimensional tensor output.

In order to further optimize the network structure, avoid overfitting, and speed up training, the convolutional layer adopts a one-dimensional convolution kernel with a length of 3; the pooling layer adopts a one-dimensional average pooling with a kernel length of 2; Constraints, where the dropout value of the first convolutional layer is 0.9, the dropout value of the second convolutional layer is 0.8, the dropout value of the third convolutional layer is 0.7, and the dropout value of the fourth convolutional layer is 0.6. The dropout value of fully connected layer 1 and fully connected layer 2 is 0.5.

In the process of network training, cross-validation is used to check whether the model is overfitting. The specific process is as follows: divide the training data into one or more data sets, train the model with a part of the data set, and verify the accuracy of the model with the other part. The results are very different on the training set and the test set, resulting in overfitting. Here we divide the samples in the database into 5 sub-data sets, each word data set is used as a test sample, and the other 4 word data sets are used as training samples. In this way, 5 classifiers and 5 test results are obtained, and the average of these 5 results is used to measure the piezoresistive matching performance of our proposed deep decomposable convolutional neural classification network model.

As shown in Figure 4, the work flow of the convolutional neural network unit mainly includes the following steps: using different people to write, with different pen holding postures, different strengths, and different fonts to write "east", "south", "big" ", "Learn" four words, construct the data set used for training the neural network. Among them, the data format is (source, target). The dimension of the source is [2000,3], and the target is the y value in the supervised learning process. All target categories are encoded in one-hot encoding format, and the format of the target after one-hot encoding corresponds to the following table 1:

Table 1 Format of target after one-hot encoding

writing font "East" "South" "Big" "study" one-hot encoding 1000 0100 0010 0001

Construct the structure of the convolutional neural network model, set the number of output units of the neural network to 4, and use the network data set for pre-training to initialize the weight parameters. The constructed dataset is input into the constructed convolutional neural network for training to optimize the weight parameters. Cross-validation is used to check whether the model has been over-fitted, and back-propagation training is performed as the error of the network until the error does not decrease for 5 consecutive training cycles, so as to obtain the optimal handwriting discrimination model. Finally, the AI chip controls the model. The signal obtained and processed by the digital conversion module is input into the optimal handwritten convolutional neural network discrimination model trained in step 3, and the discrimination result is output.

(4) Integrated circuit design module

The integrated circuit design module mainly realizes power supply for the flexible wearable bracelet, controls the analog-to-digital conversion module to collect three characteristic point signals, and uses the AI chip to realize the processing of characteristic electrical signals and the normal operation of the neural network.

As shown in Figure 5, the workflow of the present invention is:

1) Database construction, use the sensor signal acquisition processing unit to obtain the output signals sig_1, sig_2, sig_3 of the flexible wearable bracelet under a specific gesture, process it as a matrix of [2000, 3], as the source of the database unit, the corresponding handwriting The font is targeted as a database. Extract a certain number of (source, target) to build the database, in which each type of target is collected 50 times.

2) Using the data in the database unit, transfer learning is used to train the constructed pre-trained convolutional neural network, and cross-validation is used to check whether the model is overfitting.

3) The AI chip obtains and processes the electrical output signals sig_1, sig_2, sig_3 from the flexible pressure sensor array through the analog-to-digital conversion module through the dynamic signal acquisition processing unit, and then transmits the data to the data training neural network discrimination unit for pressure matching identification.

4) The value output by the neural network is the recognized handwritten font after one-hot inverse encoding.

Claims (4)

1. A recognition method for a system for handwritten font recognition, comprising the steps of:

(1) The identification system is constructed and comprises a flexible wearable bracelet, a sensor signal acquisition and processing unit and a convolutional neural network unit, wherein the flexible wearable bracelet is electrically connected with the sensor signal acquisition unit and the convolutional neural network unit;

the flexible wearable bracelet comprises a flexible substrate, wherein a pressure deformation sensor is arranged on the flexible substrate, and an insulating packaging layer covers the flexible substrate;

the flexible wearable bracelet consists of a pressure deformation sensor, a VHB double-sided hydrogel flexible substrate layer and an insulating packaging layer, wherein the pressure deformation sensor is an MXene and multi-walled carbon nanotube MWCNTs three-dimensional cross-linked network;

the preparation method of the flexible wearable bracelet comprises the following steps: cutting the VHB double-sided hydrogel into fixed cuboids; carving three characteristic points at the fixed position of the mask plate by using a laser carving machine, wherein the positions of the three characteristic points are the two sides and the middle of the lower side of the wrist; fixing a mask plate on the VHB double-sided hydrogel flexible substrate layer; spraying the prepared MXene and multi-walled carbon nanotube dispersion liquid on a mask plate by using a spraying process; spraying PVP aqueous solution on the other side of the VHB double-sided hydrogel flexible substrate layer to serve as an encapsulation layer and an electric insulation layer; stripping the mask plate from the VHB double-sided hydrogel flexible substrate layer, and leading out lead wires at three characteristic points to serve as a conductive layer;

(2) The flexible wearable bracelet acquires wrist dynamics parameter information at a characteristic point during wrist bending motion through the change of the resistance of the sensor;

(3) The sensor signal acquisition processing unit converts the acquired wrist dynamics parameter information into an electric response signal and outputs the electric response signal;

(4) And the convolutional neural network unit trains the convolutional neural network model to obtain an optimal handwriting convolutional neural network discrimination model, then an electric response signal is input into the optimal handwriting convolutional neural network discrimination model, and recognition of the handwriting font is realized after prediction of a network middle layer.

2. The recognition method of the system for handwritten font recognition according to claim 1, wherein said step (4) is specifically:

(a) Writing unified characters by different people in different pen holding postures, different strength and different fonts to construct a data set for training a neural network;

(b) Constructing a structure of a convolutional neural network model, setting the number of output units of a neural network as T, and adopting a network data set to perform pre-training to initialize weight parameters;

(c) Inputting the constructed data set into the constructed convolutional neural network model, training to optimize weight parameters, and finally obtaining an optimal handwritten convolutional neural network discrimination model;

(d) And inputting the electric response signal obtained by the sensor signal acquisition processing unit into the optimal handwritten convolutional neural network discrimination model, and outputting a discrimination result.

3. The recognition method of the system for handwritten font recognition according to claim 2, wherein the data set format in step (a) is (source, target), the source is used for collecting the wrist dynamics parameter information at the processed feature points, and the target is the handwritten font coded by a one-hot coding format.

4. The method of claim 2, wherein cross validation is used to verify whether the model is over-fitted during the training process in step (c), and cross entropy loss function is used as the error loss function of the network for back propagation training until the N training periods of error no longer decrease, thereby obtaining the optimal handwriting discrimination model.

Images