Disclosure of Invention
In order to solve the problem of lack of a method for normalizing analog domain signals, the application provides a method for normalizing analog domain signals of a sense calculation fusion processing scene and a device for normalizing analog domain signals of the sense calculation fusion processing scene.
The application provides a method for normalizing analog domain signals of a sense-in-store fusion processing scene, which adopts the following technical scheme:
In a first aspect, an analog domain signal normalization method for a sense-in-compute fusion processing scenario is provided, and is applied to sensor signal processing, and includes:
sampling to obtain an analog domain signal;
Sampling to obtain the mean value and standard deviation of the analog domain signal;
obtaining a first difference value by utilizing the difference between the average values of the analog domain signals;
And dividing the first difference value by the standard deviation of the analog domain signal to obtain a normalized value of the analog domain signal.
Optionally, the analog domain signal is a voltage signal.
Optionally, the sampling obtains an analog domain signal comprising sampling the voltage signal one or two times.
Optionally, when the voltage signal is sampled twice, then further comprising:
the first sampling voltage and the second sampling voltage which are respectively obtained by the two times of sampling are subjected to difference value to obtain a second difference value;
and subtracting the average value of the voltage signals from the second difference value, and dividing the second difference value by the standard deviation of the voltage signals to obtain the normalized value of the voltage signals.
In a second aspect, there is also provided an analog domain signal normalization apparatus for sensing a fusion processing scenario, applied to sensor signal processing, including:
the first sampling unit is used for sampling and obtaining analog domain signals, and sampling and obtaining the mean value and standard deviation of the analog domain signals, wherein the analog domain signals are voltage signals;
The computing unit is used for obtaining a first difference value by utilizing the difference between the analog domain signal and the average value of the analog domain signal, and dividing the first difference value by the standard deviation of the analog domain signal to obtain the normalized value of the analog domain signal.
Optionally, the first sampling unit includes:
the first switch SW1 is used for sampling the voltage signal and sending the voltage signal to the computing unit;
a second switch SW2 for sampling dark current information of the voltage signal and sending the information to the calculation unit;
A third switch SW3 for sampling the average value of the voltage signal and sending the average value to the calculation unit;
a second reference voltage switch SWN2 for feeding the reference voltage VREF into the calculation unit when the second switch SW2 is turned off;
The third reference voltage switch SWN3 is used for sending the reference voltage VREF into the calculation unit when the third switch SW3 is opened.
Optionally, the computing unit includes:
The fourth switch SW4 is used for sending the reference voltage VREF to the right polar plate of the fourth capacitor C4;
a fifth switch SW5 for connecting the inverting input terminal of the operational amplifier unit AMP and the output terminal of the operational amplifier unit AMP together and keeping the off, on state and the fourth switch SW4 in agreement;
a sixth switch SW6 for connecting the fourth capacitor C4 across the inverting input terminal and the output terminal of the op-AMP unit to form a feedback loop when the reference voltage VREF is turned off by the fourth switch SW 4;
the first capacitor C1 is used for receiving the sampling voltage signal sent by the first switch SW 1;
a second capacitor C2 for receiving dark current information of the sampling voltage signal sent from the second switch SW2 or receiving a reference voltage VREF sent from the second reference voltage switch SWN 2;
the third capacitor C3 is used for receiving the average value of the sampling voltage signal sent by the third switch SW3 or receiving the reference voltage VREF sent by the third reference voltage switch SWN 3;
a fourth capacitor C4 for receiving the reference voltage VREF sent by the fourth switch SW4 or being connected with the output end of the operational amplifier unit AMP to form a feedback loop between the inverting input end and the output end of the operational amplifier unit AMP;
The operational amplifier unit AMP is used for inputting a reference voltage VREF at a non-inverting input end, and connecting right polar plates of the first capacitor C1, the second capacitor C2 and the third capacitor C3 and left polar plates of the fourth capacitor C4 at an inverting input end.
In summary, the present application includes at least one of the following beneficial technical effects:
1. hardware for realizing normalization of simple analog domain signals;
2. And a normalization operator is provided for realizing sensor-end data compression in an analog domain, so that the calculation pressure (comprising power consumption, speed and area) of the analog-to-digital converter ADC is reduced.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings, fig. 1 to 4 and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
Term interpretation:
Standard deviation (Standard Deviation), which can reflect the degree of dispersion of one dataset, and the average number of two groups of data which are identical, the standard deviation is not necessarily identical. The standard deviation plays an important role in Batch Normalization (batch normalization). The standard deviation is the square root of the arithmetic mean from the mean square, reflecting the degree of dispersion of the dataset. In Batch Normalization, the standard deviation is used to calculate the distribution of each batch of data, and normalization is performed. Specifically, batchNorm makes the data follow a standard normal distribution (the mean is 0 and the standard deviation is 1) by estimating the mean and standard deviation of each batch of data and then normalizing the data. The standard deviation plays a key role in Batch Normalization, and helps BatchNorm to know the distribution condition of data, so that effective normalization processing is performed, and training efficiency and performance of the neural network are improved.
Dark current is a phenomenon in which in a physical device, even if light is not irradiated to a pixel, a charge is generated in a pixel unit. These charges form a so-called dark current, which is one of noises affecting the image quality of the CMOS sensor. The generation of dark current is related to various factors such as impurities, heat, etc.
The application provides a method for normalizing analog domain signals of a sense-in-store fusion processing scene, which adopts the following technical scheme:
In a first aspect, as shown in fig. 1, there is provided a method for normalizing analog domain signals of a scene of a sense computation fusion process, including:
S101, sampling to obtain an analog domain signal, wherein the analog domain signal is a signal which is continuous in time in general;
S201, sampling to obtain the mean value and standard deviation of the analog domain signal, wherein the mean value and standard deviation of the analog domain signal are both predicted in advance. For example, in the field of CMOS image sensors, the mean value and standard deviation of image signals are obtained in advance during the training process of the neural network algorithm based on the existing data set. Sampling the mean value and the standard deviation.
S301, obtaining a first difference value by utilizing the difference between the average values of the analog domain signals;
S401, dividing the first difference value by the standard deviation of the analog domain signal to obtain a normalized value of the analog domain signal. In the technical scheme, the normalized value of the analog domain signal can be obtained by dividing the difference between the average value of the analog domain signal obtained by sampling and the stored average value of the analog domain signal by the standard deviation of the stored analog domain signal. In the prior art, the analog domain signal is converted into a digital signal through an analog-to-digital converter ADC, and then the digital signal is processed to obtain a normalized value of the digital signal. According to the technical scheme, the analog domain signal stage is used for realizing the normalization of the analog domain signal, so that the processing cost of converting the analog domain signal into the digital signal is greatly saved, the calculation pressure (comprising power consumption, speed and area) of the analog-to-digital converter ADC is reduced, and the cost comprises the power consumption and the area of the digital signal processor. In particular, the fundamental features of traditional von neumann architecture computation and storage separation from each other, storage walls and power consumption walls, become key bottlenecks in computer architecture, especially neural network computation. Due to the implementation of the technical scheme, the required expenditure of data handling can be greatly reduced, and the method is helpful for the neural network deployment of the 'sense memory calculation' integrated chip. The "sense computation" is a sense signal, a storage signal, and a calculation signal.
Optionally, the analog domain signal is a voltage signal. The analog domain signals include a variety of signals that can be processed by the electronic device, including voltage signals, current signals. In the technical scheme, the voltage signal can be processed.
Optionally, the sampling obtains an analog domain signal comprising sampling the voltage signal one or two times. In the technical scheme, the voltage signal can be sampled once or twice. In the field of image sensors, the image signal perceived may also be processed and normalized once.
Optionally, as shown in fig. 2, when the voltage signal is sampled twice, the method further includes:
S202, obtaining a difference value between a first sampling voltage and a second sampling voltage which are respectively obtained by two times of sampling, and obtaining a second difference value; in the present solution, especially in CMOS image sensors, excessive noise, such as dark current noise, may occur due to sampling once. If the voltage signal is sampled twice and the subsequent processing is performed, dark current noise can be completely eliminated.
And S203, subtracting the average value of the voltage signals from the second difference value, and dividing the second difference value by the standard deviation of the voltage signals to obtain the normalized value of the voltage signals.
In a second aspect, as shown in fig. 3, there is further provided an analog domain signal normalization apparatus 1 for sensing a fusion processing scenario, including:
The sampling unit 101 is used for sampling and obtaining an analog domain signal, and sampling and obtaining the average value and the standard deviation of the analog domain signal, wherein the analog domain signal is a voltage signal;
The calculating unit 102 is configured to obtain a first difference value by using a difference between the analog domain signal and a mean value of the analog domain signal, and divide the first difference value by a standard deviation of the analog domain signal to obtain a normalized value of the analog domain signal.
Optionally, as shown in fig. 4, the first sampling unit 101 includes:
the first switch SW1 is used for sampling the voltage signal and sending the voltage signal to the computing unit;
a second switch SW2 for sampling dark current information of the voltage signal and sending the information to the calculation unit;
A third switch SW3 for sampling the average value of the voltage signal and sending the average value to the calculation unit;
a second reference voltage switch SWN2 for feeding the reference voltage VREF into the calculation unit when the second switch SW2 is turned off;
The third reference voltage switch SWN3 is used for sending the reference voltage VREF into the calculation unit when the third switch SW3 is opened.
Optionally, as shown in fig. 4, the computing unit 102 includes:
The fourth switch SW4 is used for sending the reference voltage VREF to the right polar plate of the fourth capacitor C4;
a fifth switch SW5 for connecting the inverting input terminal of the operational amplifier unit AMP and the output terminal of the operational amplifier unit AMP together and keeping the off, on state and the fourth switch SW4 in agreement;
The sixth switch SW6 is used for connecting the fourth capacitor C4 across the inverting input terminal and the output terminal of the op-AMP unit to form a feedback loop when the reference voltage VREF is turned off by the fourth switch SW4, and is also used for connecting the variable capacitor C4 into the feedback loop.
The first capacitor C1 is used for receiving the sampling voltage signal sent by the first switch SW 1;
a second capacitor C2 for receiving dark current information of the sampling voltage signal sent from the second switch SW2 or receiving a reference voltage VREF sent from the second reference voltage switch SWN 2;
the third capacitor C3 is used for receiving the average value of the sampling voltage signal sent by the third switch SW3 or receiving the reference voltage VREF sent by the third reference voltage switch SWN 3;
a fourth capacitor C4 for receiving the reference voltage VREF sent by the fourth switch SW4 or being connected with the output end of the operational amplifier unit AMP to form a feedback loop between the inverting input end and the output end of the operational amplifier unit AMP;
The operational amplifier unit AMP is used for inputting a reference voltage VREF at a non-inverting input end, and connecting right polar plates of the first capacitor C1, the second capacitor C2 and the third capacitor C3 and left polar plates of the fourth capacitor C4 at an inverting input end.
In summary, the present application includes at least one of the following beneficial technical effects:
1. hardware for realizing normalization of simple analog domain signals;
2. And a normalization operator is provided for realizing sensor-end data compression in an analog domain, so that the calculation pressure (comprising power consumption, speed and area) of the analog-to-digital converter ADC is reduced. The embodiment of the application relates to a normalization method and a normalization device for analog domain signals, and the implementation principle of the normalization method and the normalization device is as follows:
As shown in fig. 4, when sampling is performed once, a normalized voltage value can be obtained, and the working flow is as follows, firstly, the fourth switch SW4 and the fifth switch SW5 are closed, the sixth switch SW6 is opened, then the right polar plate access voltage of the fourth capacitor C4 is the reference voltage VREF, and because the ideal model of the operational amplifier AMP is in a virtual short state, i.e. the voltages of the inverting input end and the non-inverting input end are equal, the potential of the inverting input end of the operational amplifier AMP and the right polar plate of the fourth capacitor C4 is the reference voltage VREF;
when the first switch SW1 is closed, the sampling terminal vpixel_in is connected to the left plate of the first capacitor C1 to obtain a first sampling signal vpixel_in, the first reference voltage switch SWN1 complementary to the first switch SW1 is opened, the second switch SW2 and the third switch SW3 are opened, the second reference voltage switch SWN2 and the third reference voltage switch SWN3 complementary to the second switch SW2 and the third switch SW3 respectively are closed, and the respective left plates of the second capacitor C2 and the third capacitor C3 are connected to the reference voltage VREF.
At this time, the accumulated charge on the first capacitor C1 is C1 (vpixel_in-VREF), and the accumulated charge on the second and third capacitors C2 and C3 is zero. Accordingly, the total charge amount of the right plate of each of the first, second, and third capacitances C1, C2, and C3 and the left plate of the fourth capacitance C4 is c1× (vpixel_in-VREF).
Thereafter, the first switch SW1 is turned off, and at the same time, the first reference voltage switch SWN1 is turned on, the second switch SW2 and the third switch SW3 are turned on, and the second reference voltage switch SWN2 and the third reference voltage switch SWN3, which are respectively complementary to the second switch SW2 and the third switch SW3, are turned off, the DARK current information dark_in is input to the left plate of the second capacitor C2, and the average value norm_in of the voltage signal is input to the left plate of the third capacitor C3. At this time, the accumulated charge on C1 becomes 0, the accumulated charge on C2 becomes c2 (VDARK _in-VREF), VDARK _in= VDARK, i.e., the input voltage at the input terminal of pin VDARK _in is VDARK, and the accumulated charge on C3 becomes c3 (vnorm_in-VREF), i.e., vnorm_in=vnorm+vref, i.e., the input voltage at the input terminal of pin vnorm_in is VNORRM +vref. At this time, the total charge on the right plates of C1, C2 and C3 is c2× (VDARK-VREF) +c3× (vnorm+vref-VREF), and the charge on the left plate of the fourth capacitor C4 is c4× (v_out-VREF). The total charge is c2 x (VDARK-VREF) +c3 x vnorm+c4 x (v_out-VREF).
Since the right plate of the first capacitor C1, the right plate of the second capacitor C2, and the right plate of the third capacitor C3, and the left plate of the fourth capacitor C4 constitute a closed system, the closed system follows the principle of conservation of charge, and then the following formula of conservation of charge exists:
C1*(VPIXEL_IN-VREF)=C2*(VDARK-VREF)+C3*VNORM+C4*(V_OUT-VREF)
The above formula follows q=cu, where Q is the charge amount, C is the capacitance of the capacitor, and U is the voltage between the two plates of the capacitor. The left side of the charge conservation formula is the total amount of charges in the closed system when the first switch SW1 is closed, the fourth switch SW4 and the fifth switch SW5 are closed, the sixth switch SW6 is opened, the right polar plate access voltage of the fourth capacitor C4 is the reference voltage VREF, and the right side of the charge conservation formula is the total amount of charges in the closed system after the second switch SW2 and the third switch SW3 are closed. Obviously, the charge amounts on both sides are equal due to the closed system. Wherein, VREF may be zero or a certain reference value.
And, the capacitance values of the first capacitor C1, the second capacitor C2 and the third capacitor C3 may be set to be equal, and then the normalized value of the analog voltage signal may be calculated as follows:
V_OUT=Cunit/C4*(VPIXEL_IN-VDARK-VNORM)+VREF
Wherein c1=c2=c3= Cunit, VDARK is dark current information, and VNORM is the average value of the voltage signal. The fourth capacitor C4 may be a variable capacitor array, and the variable capacitor may be changed, so that the value Cunit/C4 may be adjusted, i.e. the capacitance ratio of the first capacitor C1, the second capacitor C2 or the third capacitor C3 to the variable capacitor array represents the normalized standard deviation. V_OUT is the normalized voltage value.
Since the application is mainly applied to the field of CMOS image sensors, the sampled voltage signal is typically sampled twice. The reason for sampling twice is to eliminate noise of the CMOS image sensor, such as dark current noise, to close the first switch SW1 again after sampling twice, to open the first reference voltage switch SWN1 complementary to the first switch SW1, to open the fourth switch SW4 and to open the fifth switch SW5, to close the sixth switch SW6, and to obtain the second sampling signal vpixel_in2 at the sampling end vpixel_in2 to be connected to the left plate of the first capacitor C1.
The following charge conservation formula can be obtained according to the charge conservation principle:
C1*(VPIXEL_IN1-VREF)=C1*(VPIXEL_IN2-VREF)+C2*(VDARK_IN-VREF)+C3*(VNORM_IN-VREF)+C4*(V_OUT-VREF)
Wherein c1=c2=c3= Cunit, VDARK _in= VDARK +vref, vnorm_in=vnorm+vref, VDARK is dark current information, and VNORM is the average value of the voltage signal. The fourth capacitor C4 may be a variable capacitor, and the value of the variable capacitor may be adjusted to Cunit/C4, that is, the capacitance ratio of the first capacitor C1, the second capacitor C2, or the third capacitor C3 and the variable capacitor array represents the normalized standard deviation. Vpixel_in1 is a sampling voltage obtained by the first sampling, and vpixel_in2 is a sampling voltage obtained by the second sampling. V_OUT is the normalized voltage value.
And, the capacitance values of the first capacitor C1, the second capacitor C2 and the third capacitor C3 may be set to be equal, and then the normalized value of the analog voltage signal may be calculated as follows:
V_OUT=Cunit/C4*(VPIXEL_IN1-VPIXEL_IN2-VDARK-VNORM)+VREF
where VREF may be 0 or some other reference value (e.g., a parameter near the threshold voltage Vth, etc.) that facilitates lower level circuit operation. CL is the load capacitance, which is the equivalent load capacitance of the next stage circuit.
The foregoing description of the preferred embodiments of the application is not intended to limit the scope of the application in any way, including the abstract and drawings, in which case any feature disclosed in this specification (including abstract and drawings) may be replaced by alternative features serving the same, equivalent purpose, unless expressly stated otherwise. That is, each feature is one example only of a generic series of equivalent or similar features, unless expressly stated otherwise.