JP2002358504A - Signal processing circuit and pattern recognition device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To stabilize the operation of a hierarchical parallel processing circuit composed of arithmetic elements receiving a plurality of local timing signals in a neural network for expressing and processing pattern information based on pulse phases and intervals, and to achieve consistent operation. . SOLUTION: A plurality of neurons connected to each other based on a predetermined rule and arranged in parallel, and a pulse-like signal is generated at a predetermined frequency in accordance with an input time-series signal pattern, and a predetermined group of nearby neurons is generated. A local timing signal generation circuit 1301 for outputting a pulse-like signal; and a synchronization detection circuit 7 for detecting synchronization between the output from the local timing signal generation circuit 1301 and the output from the neuron group. A predetermined calculation is executed based on the output of the above.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a signal processing circuit such as a neural network for performing parallel processing of signals, and a pattern recognition device using the same.

[0002]

2. Description of the Related Art Conventionally, in the field of image recognition and voice recognition, a type in which a recognition processing algorithm specialized for a specific recognition target is sequentially calculated and executed as computer software, or a dedicated parallel image processor (SIM) is used.
D, MIMD machine, etc.).

A typical example of the image recognition algorithm will be described below. First, as a feature to calculate and perform a feature amount related to the similarity with the recognition target model, model data of the recognition target is expressed as a template model, and similarity to the input image (or its feature vector) is determined by template matching or the like. Calculation of degree, calculation of higher-order correlation coefficient, etc., mapping of input pattern to eigenimage function space obtained by principal component analysis of target model image, and calculation of distance between model and feature space Method (Sirovic
h, et al., 1987, Low-dimensional procedure for the
characterization of human faces, J. Opt. Soc. Am.
[A], vol. 3, pp.519-524), a method of calculating the similarity by elastic graph matching by expressing a plurality of feature extraction results (feature vectors) and their spatial arrangement relationships in a graph (La
des et al. 1993, Distortion Invariant Object Recog
nition in the Dynamic Link Architecture, IEEE Tran
s. on Computers, vol.42, pp.300-311), a method of performing a predetermined transformation on an input image to obtain a position, rotation, and scale-invariant representation and then collating with a model (Seibert, et al.
1992, Learning and recognizing 3D objects from mul
tiple views in a neural system, in Neural Networks
for Perception, vol. 1 Human and MachinePerceptio
n (H. Wechsler Ed.) Academic Press, pp. 427-444).

As a pattern recognition method using a neural network model inspired by the information processing mechanism of a living body, a method of performing hierarchical template matching (Japanese Patent Publication No. 60-712, Fuku
shima & Miyake, 1982 Neocognitron: A new algorithm
for pattern recognition tolerant of deformation a
nd shifts in position, Pattern Recognition, vol.1
5, pp.455-469), a method of obtaining a scale and position-invariant representation of the center of an object using a dynamic routing neural network (Anderson, et al. 1995, Routing Networks in
Visual Cortex, in Handbook of Brain Theory and Neu
ral Networks (M.Arbib, Ed.), MIT Press, pp.823-82
6), other multilayer perceptrons, radial basis functions (Radial
Basis Function) There is a method using a network.

On the other hand, as an attempt to more faithfully incorporate an information processing mechanism based on a biological neural network, a neural network model circuit has been proposed in which information is transmitted and represented by a pulse train corresponding to an action potential (Mur).
ray et al., 1991 Pulse-Stream VLSI Neural Networks
Mixing Analog and Digital Techniques, IEEE Trans.
on Neural Networks, vol.2, pp.193-204.
No. 62157, JP-A-7-334478, JP-A-8-153148
And Japanese Patent No. 2879767).

As a method of recognizing and detecting a specific object by a neural network composed of pulse train generating neurons, a higher order (second or higher order) by Eckhorn et al. On the assumption of a linking input and a feeding input is used. ) Model (Eckhorn, et al. 1990, Feature linking via synchr
onization among distributed assemblies: Simulation
of results from cat cortex, Neural Computation, V
ol.2, pp.293-307), that is, a method using a pulse-coupled neural network (hereinafter abbreviated as PCNN) (USP56).
64065 and Broussard, et al. 1999, Physiolo
gically Motivated Image Fusion for Object Detectio
n using a Pulse Coupled Neural Network, IEEE Tran
s. on Neural Networks, vol. 10, pp. 554-563, etc.).

As a method of reducing the wiring problem in the neural network, there is a method of encoding the address of a so-called pulse output neuron in an event-driven manner (Address Event Representation: AER) (Lazzaro, et al. 1993). , Silicon Auditory
Processors as Computer Peripherals, In Touretzky,
D. (ed), Advances in Neural Information Processing
Systems 5. San Mateo, CA: Morgan Kaufmann Publisher
s). In this case, since the ID of the neuron on the output side of the pulse train is binary-coded as an address, even if output signals from different neurons are temporally arranged on a common bus, the neurons on the input side do not. The address of the original neuron can be automatically decoded.

On the other hand, in the neural network processor according to Japanese Patent No. 2741793, the number of neurons and the circuit scale are reduced by forming a multi-layer feedforward network in a systolic array architecture.

In the parallel processing multiprocessor system and the like according to Japanese Patent No. 2500038, the presence or absence of an error is detected by majority processing of a signature generated simultaneously with processing of an instruction set in a distributed parallel computing system.

In recent years, as the operating frequency of the microprocessor has been greatly increased, the chip is divided into a plurality of blocks and each block is asynchronously operated instead of operating the entire chip in synchronization with a single clock signal. An architecture that runs on a computer has been developed (Schuster, S. et al.,
“Asynchronous Interlocked Pipelined CMOS Circuits
Operating at 3.3-4.5GHz ”, 2000 IEEE Internationala
l Solid-State Circuits Conference (ISSCC2000), WA1
7.3, vol.43, pp.292-293, 2000).

[0011]

In the above-mentioned prior art, when a plurality of distributed local clock signals are used as a control clock signal as a hierarchical parallel signal processing circuit, stable operation can be performed without contradiction. There is no method that guarantees low power consumption.

[0012]

According to the present invention, there is provided a non-consistent operation of parallel pulse signal processing (for example, a method of using a plurality of asynchronous timing control signals by using a locally distributed clock signal generation circuit as a data driven type arithmetic element). The intended operation can be performed even if the operation is received), and stable operation and low power consumption operation are realized.

According to an embodiment of the present invention, the signal processing circuit is coupled to each other based on a predetermined rule and has a plurality of arithmetic elements arranged in parallel and a predetermined frequency according to an input time-series signal pattern. A plurality of local timing signal generation circuits for generating a pulse-like signal and outputting the pulse-like signal to a group of operation elements in a predetermined vicinity; and at least one of the signals output from the local timing signal generation circuit and the predetermined operation element A synchronization detection circuit for detecting synchronization with an output from the group, wherein the arithmetic element performs a predetermined operation based on an output of the synchronization detection circuit.

According to another aspect of the present invention, in the signal processing circuit, the first arithmetic element group and the local timing signal generating circuit form a synchronous cluster locally by interaction. .

According to another aspect of the present invention, a plurality of arithmetic elements which are coupled to each other based on a predetermined rule and arranged in a hierarchical and parallel manner, and an input time-series signal pattern A local timing signal generation circuit that generates a pulse-like signal at a predetermined frequency according to and outputs the pulse-like signal to the first and second arithmetic element groups in a predetermined vicinity,
A detection circuit that detects a correlation or synchronization between the pulse-like signal and the first processing element group, and wherein the second processing element group is based on an output from the detection circuit. And the local timing signal generating circuit is provided for each of a predetermined number of arithmetic element groups of the second arithmetic element group.

Further, according to another aspect of the present invention, in the signal processing circuit, the local timing signal generating circuit is configured so that the operation element at the representative position of the second operation element group receives an input from the first operation element. Characterized by receiving the output of

Further, according to another aspect of the present invention, the local timing signal generation circuit includes an operation element at a representative position of the first operation element group corresponding to the second operation element or one of the vicinity thereof. And receiving an output from the arithmetic element.

Further, according to another aspect of the present invention, the local timing signal generating circuit is configured such that an arithmetic element at a representative position of the first arithmetic element group corresponding to the second arithmetic element receives an input. Among the outputs from the arithmetic element group.

According to another aspect of the present invention, part or all of the coupling between the local timing signal generation circuit and the first arithmetic element is a mutual coupling.

According to another aspect of the present invention, an input means for inputting data of a predetermined dimension and a feature detection layer for detecting a plurality of features from the data input from the input means are provided to the pattern recognition device. A plurality of data processing modules, respectively; and output means for outputting a recognition result of a predetermined pattern based on outputs of the plurality of data processing modules, wherein the data processing modules are connected to each other by predetermined synapse connection means. A plurality of operation elements and a plurality of local timing signal generation elements, wherein the operation elements are based on input signals from the plurality of local timing signal generation elements and arrival time patterns of a plurality of pulses within a predetermined time window. A pulse-like signal is output at a predetermined frequency or timing.

[0021]

DESCRIPTION OF THE PREFERRED EMBODIMENTS (First Embodiment) Overview of Overall Configuration FIG. 1 shows an overall configuration of a pattern recognition apparatus according to the present invention.
It mainly deals with information related to recognition (detection) of objects or geometric features, and uses what is called a convolutional network structure (LeCun, Y. and Bengio, Y., 1995, “Convoluti
onal Networks for Images Speech, and Time Series ”
in Handbook of Brain Theory and Neural Networks
(M. Arbib, Ed.), MIT Press, pp. 255-258). However, it differs from the conventional one in that the interlayer connection in the same path can locally form mutual connection (described later). The final output corresponds to the recognition result, that is, the category of the recognized object.

The data input layer 1 is a CMOS sensor or a photoelectric conversion element such as a CCD element in the case of an image sensor means, and is a voice input sensor in the case of detecting and recognizing voice. Alternatively, high-dimensional data obtained from an analysis result (for example, principal component analysis, vector quantization, etc.) of the predetermined data analysis means may be input.

Hereinafter, the case of inputting an image will be described. The feature detection layer (1, 0) performs multi-resolution processing by Gabor wavelet transform or the like to apply local low-order features of the image pattern (which may include color component features in addition to geometric features) to each of the entire screen. At the position (or each of the predetermined sampling points over the entire screen), the same number of feature categories are detected at the same location at a plurality of scale levels or resolutions, and the types of the feature amounts (for example, a predetermined direction as a geometric feature are detected). When a line segment is extracted, it has a receptive field structure corresponding to the geometrical structure (slope of the line segment), and is composed of neuron elements that generate a pulse train according to the degree.

The feature detection layer (1, k) as a whole forms a processing channel at a plurality of resolutions (or scale levels) (where k ≧ 0). That is, in the case where the Gabor wavelet transform is performed in the feature detection layer (1, 0), a set of feature detection cells having a Gabor filter kernel having the same scale level and different direction selectivity in the receptive field structure is used for feature detection. The same processing channel is formed in the layer (1,0), and also in the subsequent layer (1,1), the feature detection cells receiving outputs from the feature detection cells (detecting higher-order features)
Belong to the same channel as the processing channel. Further, in the subsequent layer (1, k) (where k> 1), similarly, the feature detection cells receiving outputs from a plurality of feature integrated cells forming the same channel in the (2, k−1) layer are It is configured to belong to a channel. In each processing channel, processing at the same scale level (or resolution) proceeds, and detection and recognition from low-order features to high-order features are performed by hierarchical parallel processing.

The feature integration layer (2,0) has a predetermined receptive field structure.
(Hereinafter, the receptive field has a coupling range with the output element of the immediately preceding layer, and the receptive field structure means the distribution of the coupling load), has a neuron element that generates a pulse train, and has a feature detection layer ( (1, 0) are integrated (operations such as sub-sampling by local averaging, maximum value detection, and the like) in the same receptive field. Further, each receptive field of the neuron in the feature integration layer has a common structure among neurons in the same layer. Each feature detection layer (1, 1),
(1,2), ..., (1, N)) and each feature integration layer ((2,1),
(2, 2),..., (2, N)) each have a predetermined receptive field structure, and the former ((1, 1),.
Performs detection of a plurality of different features in each feature detection module, and the latter ((2, 1),...) Integrates detection results regarding a plurality of features from the preceding feature detection layer. However, the former feature detection layer is connected (wired) so as to receive the cell element output of the preceding feature integration layer belonging to the same channel. The sub-sampling, which is a process performed in the feature integration layer, is for averaging the output from a local region (local receptive field of the feature integration layer neuron) from the feature detection cell population of the same feature category. .

FIG. 2 is a diagram showing a configuration of a synapse circuit and a neuron element. As shown in FIG. 2A, the structure for connecting the neuron elements 201 between the layers includes a signal transmission unit 203 (wiring or delay line) corresponding to an axon or a dendrite of a nerve cell, and a synapse circuit. This is S202. FIG.
(A) relates to the output (input from the viewpoint of the cell N) of the feature integrated (detected) cell forming the receptive field for a certain characteristic detected (integrated) cell (N) from the neuron group (n _i ). 3 shows a configuration of a connection. The signal transmission unit 203 shown by a thick line forms a common bus line, and pulse signals from a plurality of neurons are transmitted on this signal transmission line in time series. The same configuration is adopted when receiving an input from a cell (N) of an output destination. In this case, the input signal and the output signal may be divided and processed on the time axis with exactly the same configuration, or two systems for input (dendritic side) and output (axon side) may be used. The processing may be performed by providing the same configuration as that of FIG.

The synapse circuit S202 has an interlayer connection (a connection between a neuron on the feature detection layer 102 and a neuron on the feature integration layer 103, and each layer has a connection to a subsequent layer and a preceding layer. May be involved)
Some are involved in connections between neurons in the same layer. The latter is mainly used, if necessary, mainly for coupling a pacemaker neuron described later with a feature detection or feature integration neuron.

In the synapse circuit S202, the so-called excitatory coupling amplifies the pulse signal, and the inhibitory coupling conversely attenuates. When transmitting information using a pulse signal, amplification and attenuation can be realized by any of amplitude modulation, pulse width modulation, phase modulation, and frequency modulation of the pulse signal.

In this embodiment, the synapse circuit S20
2 is mainly used as a phase modulation element of the pulse, the amplification of the signal is converted as a substantial advance as a characteristic quantity of the pulse arrival time characteristic, and the attenuation is converted as a substantial delay. That is,
The synaptic connection gives the arrival position (phase) on the time axis peculiar to the feature of the output neuron as described later, and qualitatively the excitatory connection leads the phase of the arrival pulse with respect to a certain reference phase. In the case of inhibitory coupling, a delay is similarly given.

In FIG. 2A, each neuron element
n _j outputs a pulse signal (spike train) and uses a so-called integral-and-fire type neuron element as described later. As shown in FIG. 2 (C),
The synapse circuit and the neuron element may be combined to form a circuit block.

The described neuron element then neurons constituting each layer. Each neuron element is an extended model based on the so-called integral-and-fire neuron, and fires when the result of spatiotemporal linear addition of the input signal (pulse train corresponding to the action potential) exceeds the threshold, and the pulse shape It is the same as a so-called integral-and-fire neuron in that it outputs a signal.

FIG. 2B shows an example of a basic configuration showing the operation principle of a pulse generation circuit (CMOS circuit) as a neuron element, and a known circuit (IEEE Trans. On Neural Net).
works Vol. 10, pp. 540). Here, it is configured to receive excitatory and inhibitory inputs as inputs.

Hereinafter, the principle of operation of the pulse generation circuit will be described. Capacitor C ₁ and resistor R excitatory input side
The time constant of _one circuit is smaller than the time constant of the capacitor C ₂ and the resistor R ₂ , and in a steady state, the transistors T ₁ ,
T ₂ and T ₃ are shut off. Note that the resistance is actually
It is composed of transistors coupled in diode mode.

The potential of the capacitor C ₁ is increased, the capacitor
Above than that of C ₂ by the threshold value of the transistors _{T 1,} transistors _{T 1} becomes active, further activates the transistor _T 2, _{T 3.} Transistors T ₂ , T
₃ constitutes a current mirror circuit, the output of the circuit of FIG. 2 (B), is output from the capacitor C ₁ side by the output circuit (not shown). When the charge storage amount of the capacitor C ₂ becomes maximum, transistors T ₁ is blocked, as a result transistors T ₂ and T ₃ are also cut off, the positive feedback is constituted as a 0.

The so-called refractory period, the capacitor C ₂ is discharged, the potential of the capacitor C ₁ is large than the potential of the capacitor C _2, as long as the difference does not exceed the threshold value of the transistor T _1, the neuron does not respond . Capacitors C ₁ and C
_A periodic pulse is output by repeating the alternate charging and discharging of No. ₂ and its frequency is generally determined according to the level of the excitatory input. However, depending on the presence of the refractory period, it is possible to limit the maximum value or to output a constant frequency.

The potential of the capacitor, and thus the amount of charge storage, is
It is temporally controlled by a reference voltage control circuit (time window weight function generation circuit) 204. Reflecting this control characteristic,
This is weighted addition within a time window to be described later for the input pulse (see FIG. 7). This reference voltage control circuit 204 is based on an input timing from a pacemaker neuron to be described later (or a mutual coupling input with a neuron in a subsequent layer).
A reference voltage signal (corresponding to the weight function of FIG. 7B) is generated.

In some embodiments, the input of suppressiveness is not always necessary. However, the divergence (saturation) of the output is prevented by making the input from the pacemaker neuron to the feature detection layer neuron to be suppressive as described later. be able to.

In general, the relationship between the sum of the input signals and the output level (pulse phase, pulse frequency, pulse width, etc.) changes depending on the sensitivity characteristics of the neuron.
The sensitivity characteristic can be changed by a top-down input from an upper layer. In the following, for convenience of explanation, it is assumed that the circuit parameters are set so that the frequency of the pulse output corresponding to the total value of the input signal rises sharply (thus, almost in binary in the frequency domain), and the output level is determined by pulse phase modulation. (Such as the timing at which phase modulation is applied).

As a means for modulating the pulse phase, a circuit as shown in FIG. 5 to be described later may be added and used.
As a result, the phase of the pulse output from the neuron changes as a result of the reference voltage being controlled by the weight function within the time window, and this phase can be used as the output level of the neuron.

The time τ _w1 corresponding to the maximum value of the weighting function as shown in FIG. 7B which gives the temporal integration characteristic (reception sensitivity characteristic) of the pulse subjected to the pulse phase modulation by the synaptic connection is generally _Is set earlier in time than the estimated arrival time τ _{si of the} pulse unique to the feature given by the synaptic connection. As a result, a pulse arriving earlier within a certain range from the expected arrival time (in the example of FIG. 7B, the pulse arriving too early is attenuated) is a pulse having a high output level in the neuron receiving it. It is integrated over time as a signal. The shape of the weight function is not limited to a symmetric shape such as Gaussian, but may be an asymmetric shape.

The phase of the neuron output (before synapse) is based on the beginning of the time window as described later, and the delay (phase) from the reference time is the time when a reference pulse (by a pacemaker output or the like) is received. It has output characteristics determined by the amount of charge storage. The details of the circuit configuration for providing such output characteristics will not be described because they are not the focus of the present invention. If the post-synaptic pulse phase is obtained by adding the pre-synaptic phase to the inherent phase modulation amount given by the synapse, an operation of multiplying the pre-synaptic signal by an amount corresponding to the synaptic load will be described later. Given by the weight function.

A known circuit configuration may be used in which an oscillation output is output with a predetermined timing delay when the sum of inputs obtained by using a window function or the like exceeds a threshold value.

The configuration of the neuron element is a neuron belonging to the feature detection layer or the feature integration layer, and receives a pulse output from the pacemaker neuron when the firing pattern is controlled based on a pacemaker neuron output timing described later. After that, the circuit configuration may be such that the feature detection neuron outputs a pulse with a phase delay corresponding to the input level (the above-described input sum value) received from the local receptive field of the preceding layer. In this case, before a pulse signal is input from a pacemaker neuron, there is a transitional transition state in which each neuron outputs pulses at random phases in accordance with the input level.

In general, adjacent feature detection layer neurons have overlapping receptive fields with respect to the preceding layer (such as the feature integration layer). In this case, as described later, the neurons are attached to the feature detection layer neurons. Pacemaker neurons have the same receptive fields as their feature detection layer neurons,
Receptive field structures for the immediately preceding layer between adjacent pacemaker neurons also overlap with each other. Therefore, a feature integration layer neuron belonging to a certain receptive field of a predetermined feature detection layer neuron may simultaneously belong to a plurality of feature detection layer neurons. As will be described later, since the feature integration layer neuron also receives a timing signal from a pacemaker neuron in a subsequent layer, in this case, it receives timing control signals from a plurality of pacemaker neurons.

In order to correctly integrate a pulse signal within a time window in a feature detection layer neuron to be described later, the feature integration layer neuron is switched to the feature detection layer neuron even when a plurality of timing control signals are received. The timing of the generation of the time window is set so that the flow of the pulse signal is appropriately performed.

For this purpose, a timing control signal output from a pacemaker neuron or a synchronous firing signal generated by network coupling between neurons (between a feature detection layer and a feature integration layer) and network dynamics is used as a reference, and a predetermined value is used as a reference. A circuit configuration is used in which the firing timing of the output pulse is controlled so that the phases are synchronized within the time width (for details, refer to the explanation of the operation principle of pattern detection).

As described above, the neurons in the feature detection layer have a receptive field structure corresponding to the feature category, and the input pulse signal (current value or potential) from the neuron in the preceding layer (input layer or feature integration layer) is obtained. When the total weight of the time window function (described later) is equal to or greater than a threshold value, a non-decreasing and non-linear function that asymptotically saturates to a certain level, such as a sigmoid function, according to the total value, that is, a so-called sq
A pulse is output with an output that takes a uashing function value (here, given by a phase change; a configuration that changes based on frequency, amplitude, and pulse width may be used).

The synapse circuit etc. Figure 4 synaptic to each neuron n _'j is the coupling destination of the neuron n _i at the synapse circuit 202 (S _i) to (meaning the size of the modulation about the phase delay, etc.) This shows that the given small circuits are arranged in a matrix.

When the network is configured to have a covalent connection form of connection weights (in the case where the distribution of synaptic connection weights of different neurons is represented by the same weight coefficient distribution of 1), the amount of delay at each synapse (described below). P _ij ) may be uniform in the same receptive field unlike the case of FIG. For example, the connection from the feature detection layer to the feature integration layer is performed when the feature integration layer performs subsampling by local averaging (however, uniform weighting) of the output of the feature detection layer that is the preceding layer. Such a configuration can be made regardless of the detection target (that is, regardless of the problem).

In this case, each small circuit 401 shown in FIG.
Requires only a single circuit _{Sk, i} as shown in FIG. 4C _, which is a particularly economical circuit configuration. On the other hand, if the connection from the feature integration layer (or sensor input layer) to the feature detection layer is like this, the feature detection neuron will detect
This is an event called simultaneous arrival (or almost simultaneous arrival) of pulses representing a plurality of different characteristic elements.

As shown in FIG. 4B, each synapse coupling small circuit 401 includes a learning circuit 402 and a phase delay circuit 403. The learning circuit 402 adjusts the delay amount by changing the characteristics of the phase delay circuit 403, and also stores the characteristic value (or its control value) on the floating gate element or on a capacitor coupled to the floating gate element. It is something to memorize. The phase delay circuit 403 is a pulse phase modulation circuit, and includes, for example, monostable multivibrators 506 and 507, resistors 501 and 504, capacitors 503 and 505 as shown in FIG.
There is a structure using the transistor 502. FIG. 5B shows a square wave P ₁ input to the monostable multivibrator 506 (FIG. 5B [1]), a square wave P ₂ output from the monostable multivibrator 506 (the same [2]), Monostable multivibrator 507
Represents the respective timings of the square wave P ₃ (the same [3]) output from.

[0052] Although not described details of the operating mechanism of the phase delay circuit 403, the pulse width of P ₁ is determined by the time to reach the threshold voltage of the capacitor 503 by the charging current is predetermined, P ₂ Is determined by the time constant of the resistor 504 and the capacitor 505. The pulse width of P ₂ (FIG. 5
The way) spreads the dotted square wave (B), the rise time of P3 deviates later time down the falling is also shifted the same amount, the pulse width of P ₃ does not change, resulting in the input pulse phase Is modulated and output.

A refresh circuit that uses the control voltage Ec as a reference voltage
The pulse phase (delay amount) can be controlled by changing the learning circuit 402 that controls the amount of charge stored in the capacitor 508 that applies the connection weight to the capacitor 509. In order to maintain the connection weight for a long period of time, after the learning operation, as a charge of a floating gate element (not shown) added to the outside of the circuit of FIG. The coupling load may be stored. In addition, a configuration devised to reduce the circuit scale (for example,
317, JP-A-10-327054)
For example, a known circuit configuration can be used.

FIG. 5 shows an example of a learning circuit at the synapse that achieves simultaneous arrival of pulses or a predetermined amount of phase modulation.
What has a circuit element as shown in FIG. That is, the learning circuit 402 is connected to the pulse propagation time measurement circuit 510.
(Here, the propagation time refers to a time difference between a pulse output time at a pre-synapse of a neuron of a certain layer and an arrival time of the pulse at an output destination neuron on the next layer), a time window generating circuit 511. , And a pulse phase modulation amount adjustment circuit 512 that adjusts the pulse phase modulation amount at the synapse so that the propagation time becomes a constant value.

As the propagation time measuring circuit 510, a clock pulse from a pacemaker neuron forming the same local receptive field as described later is input, and within a predetermined time width (time window: see FIG. 3B). A configuration in which the propagation time is obtained based on the output of the clock pulse from the counter circuit is used. By setting the time window based on the firing time of the output destination neuron, an extended Hebb's learning rule as described below is applied.

Processing at the feature detection layer (1,0) (Gabor wavelet
( Lower-order feature extraction by transformation, etc.) The feature detection layer (1, 0) has a pattern structure (lower-order feature) having a predetermined spatial frequency in a local area of a certain size and having a vertical directional component. Assuming that there is a neuron for detecting the characteristic, if there is a corresponding structure in the N1 receptive field on the data input layer 1, a pulse is output at a phase corresponding to the contrast. Such a function can be realized by a Gabor filter. Hereinafter, the feature detection layer (1, 0)
The feature detection filter function performed by each neuron will be described.

In the feature detection layer (1, 0), multi-scale
It is assumed that Gabor wavelet transform represented by a filter set of multi-directional components is performed, and each neuron (or each group including a plurality of neurons) in a layer has a predetermined G value.
It has an abor filter function. In the feature detection layer, a plurality of Gas with a fixed scale level (resolution) and different direction selectivities
A plurality of neuron groups consisting of neurons having a receptive field structure corresponding to the convolution operation kernel of the bor function are collectively formed to form one channel.

At this time, neuron groups forming the same channel have different direction selectivities, and neuron groups having the same size selectivity may be arranged at positions close to each other, or belong to the same feature category and differ. Neuron groups belonging to the processing channel may be arranged close to each other. This is because the arrangement configuration as shown in each drawing is easier to realize in terms of the circuit configuration for the sake of the combining process described later in the collective coding.

The details of the method of performing the Gabor wavelet conversion in a neural network are described in the literature by Daugman (1988) (IEEE Trans. On Acoustics, Speech, and Signal Pro
cessing, vol. 36, pp. 1169-1179).

Each neuron in the feature detection layer (1,0) is represented by g
_It has a receptive field structure corresponding to _mn . G _mn having the same scale index m has the same size receptive field, and the corresponding kernel g _mn size has a size corresponding to the scale index in operation. Here, 30 × 3 on the input image in order from the coarsest scale
The sizes were 0, 15 × 15, and 7 × 7. Each neuron outputs a non-linear squashing function of a wavelet transform coefficient value obtained by inputting a product sum of a distribution weight coefficient and image data (here, a phase reference is used;
The pulse output may be performed based on the frequency, amplitude, and pulse width. As a result, Gabor wavelet conversion is performed as the output of the entire layer (1, 0).

Processing at the feature detection layer (medium and high order feature extraction)
Out) subsequent feature detection layer ((1,1), (1,2), each neuron ...), unlike the feature detection layer (1,0), a unique feature in the pattern to be recognized Is formed by the so-called Hebb learning rule or the like. The size of the local region where the feature detection is performed gradually becomes closer to the size of the entire recognition target in a later layer, and a medium-order or higher-order feature is geometrically detected.

For example, in the case of performing face detection / recognition, the middle-order (or higher-order) features represent features at the level of a figure element such as an eye, a nose, and a mouth that constitute the face. If different processing channels have the same hierarchical level (the complexity of detected features is the same level), the detected features differ in the same category but are detected on different scales. It is in. For example, "eye" as a secondary feature
In different processing channels, detection is performed as "eyes" having different sizes. That is, detection is attempted in a plurality of processing channels with different scale level selectivity for a given size "eye" in the image.

Note that the feature detection layer neuron is generally
A mechanism may be provided to receive the coupling of the shunting inhibition (shunting inhibition) based on the output of the preceding layer in order to stabilize the output (independent of low-order and higher-order feature extraction). .

Processing in the Feature Integration Layer The neurons in the feature integration layer ((2,0), (2,1),...) Will be described. As shown in FIG. 1, a feature detection layer (for example,
The connection from (1,0)) to the feature integration layer (eg, (2,0)) is
The input of excitatory connections and the output of a pacemaker neuron (P
N _out1 ) are received on the excitatory input side in FIG. 2B, and the functions of the neurons in the integrated layer are, as described above, local averaging for each feature category, maximum value detection, and other subsampling. It is.

According to the former, a plurality of pulses of the same type of characteristic are inputted, and they are integrated in a local area (receptive field) and averaged (or a representative value such as a maximum value in the receptive field). ), The position fluctuation of the feature,
Deformation can be reliably detected. For this reason, the receptive field structure of the feature integration layer neuron is uniform regardless of the feature category (for example, each is a rectangular region of a predetermined size, and the sensitivity or the weight coefficient is uniformly distributed therein). You may comprise so that it may become.

In this embodiment , the pulse signal processing in the feature integration layer is such that the feature integration cell receives the timing control from the pacemaker neuron on the feature detection layer of the preceding layer number (1, k). Does not configure. This is because, in a feature-integrated cell, the phase (frequency, pulse width, or amplitude) determined by the input level (such as the sum of the input pulses over time) within a certain time range, rather than the arrival time pattern of the input pulse. However, since the pulse is output in the present embodiment, the timing is not so important because the pulse is output. Note that this is not intended to exclude a configuration in which the feature-integrated cells receive the timing control from the pacemaker neuron in the feature detection layer in the preceding layer, and it goes without saying that such a configuration is also possible.

[0067] The operating principle of the pattern detection Next, pulse code of two-dimensional figure pattern detection method will be described. FIG. 3 schematically shows the propagation of a pulse signal from the feature integration layer to the feature detection layer (for example, from layer (2,0) to layer (1,1) in FIG. 1). . Each neuron n _i on the feature integration layer side corresponds to a different feature amount (or feature element), and a neuron n ′ _j on the feature detection layer side is a higher order neuron obtained by combining the features in the same receptive field. Related to the detection of features (graphic elements).

The connection between each neuron is made during the pulse propagation.
Between and neuron n_iFrom neuron n ' _jSynaptic connection to
(S_{j, i}) Due to time delay etc. (specific to feature)
A delay occurs, resulting in neuron n '_jArrive at
Pulse train P_iIs a pulse output from each neuron in the feature integration layer.
As long as the power is exerted, the synaptic connections determined by learning
Are in a predetermined order (and interval) depending on the delay amount of
(In FIG. 3A, P_Four, P_Three, P_Two, P₁Shown to arrive in the order
Has been).

FIG. 3B shows a case where the phase window is synchronized between the output of the pacemaker neuron and the feature integration layer neuron when the time window synchronization control is performed using the timing signal from the pacemaker neuron described later. From the feature integrated cells n ₁ , n ₂ , and n ₃ on the layer number (2, k) (representing different types of characteristics), the layer number (1, k + 1)
The timing of pulse propagation to an upper certain feature detection cell (n ′ _j ) (which performs higher-order feature detection) and the like are shown.

In FIG. 6, the pacemaker neuron n
_p is associated with feature detecting neurons (n _j , _nk, etc.) that form the same receptive field and detect different types of features, form the same receptive field as them, and form a feature integration layer (or Input from the input layer). In addition, since the output from the pacemaker neuron is also output to the excitatory input of the feature integration layer neuron, there is a (loop-like) interconnection between the feature integration layer neuron group and the pacemaker neuron. The pacemaker neuron receives an input pulse at a predetermined timing determined by the sum of its inputs (or to control the state of the receptive field, such as the average activity level of the entire receptive field, to be dependent on a state representing an activity characteristic unique to the entire receptive field). Pulse output is performed so that the phase is synchronized with the signal. For this purpose, as shown in FIG. 13 (A), the pacemaker neuron circuit uses the above-mentioned neuron element circuit (FIG. 2 (B)) with the pacemaker input as the phase synchronization detection window signal input and the suppressive input as the fixed time. The configuration is such that a phase synchronization detection and control circuit 7 is added to a constant input (FIG. 13B) which is input as long as there is a pulse input from the feature integration neuron in the range.

The phase synchronization detection and control circuit 7 outputs an output pulse (FIG. 13) from the neuron element circuit (FIG. 13B).
(A) (PN _{out2, pre} ) and a plurality of pulses from the preceding layer of the feature integration layer neuron are input, and the pulse signal from the feature integration layer neuron based on the start time of the phase synchronization detection window described later is used as a reference. each phase (hereinafter referred to as P _I) PN ₂ of the pulse signal output (Fig. 15 corresponding _{to, out1,} and 13
PN _out1) of (A), i.e., a plurality of feature integration layer temporal variation width (pacemaker determined by output interval predetermined duration (T in FIG. 15) of the ensemble average of the phase P _I of the input pulse from the neuron ) Is below the reference value and converges to a predetermined value, and the phase of the pacemaker neuron output (PN _out1 ) is adjusted and output to the feature integration layer neuron. The details will be described below.

[0072] Figure 13 phase synchronization detection and control circuit 7 in (A) is composed of a phase synchronization detection window generating circuit and the pulse signal (PN _out1) generating circuit, and a control circuit as shown in FIG. 13 (C) A pulse signal having a predetermined width is output from the phase synchronization detection window generating circuit to the neuron element circuit.
In the neuron element circuit, the temporal integration of the input pulse signal from the feature integration layer neuron or the like is performed only when this pulse signal is ON, and when the pulse signal is equal to or larger than a predetermined threshold value, the pulse signal (PN _{out2, pre in} FIG. ) Is output.

The synchronization detection signal inducing circuit in the control circuit
Pulse signal PN_{out2, pre}Input within the specified time span
PN_out2Signal S for prompting the generator to output_pContinue to output
Ke, PN _out2The synchronization detection signal pulse (hereinafter
Description of PN_{2, out2}) Is output to the feature detection layer neuron.
You. PN_out2As a generating circuit, this PN_out2Signals and pulses
Signal (PN_out1) Are almost in phase (after synchronization is established)
Switch ON / OFF according to the output from the threshold processing circuit
A switch circuit that turns OFF or a pulse signal (P
N _out1) With a delay circuit that gives a fixed delay to
Is done. In addition, the convergence determination circuit uses a PN described below._out1of
This circuit performs convergence judgment when performing output timing control.
You.

Next, the operation of the feature integration layer neuron after receiving PN _{2, out1} will be described. The output of the feature integration layer neuron before synchronization with the pacemaker neuron is as described above (described in Pulse signal processing in the feature integration layer ), and the output pulse (PN
_{2, out1} ), the pulse output is performed when the integrated value of the pulse signal from the preceding feature detection layer exceeds a predetermined reference value (threshold value), while after receiving (plural) PN _out1 , integration is performed. The output timing from the layer neuron is the output timing within a predetermined time width from the previous pulse output in which the integrated value of the input pulse from the pacemaker neuron exceeds the threshold. The output from the feature integration layer neuron to the feature detection layer neuron is performed via a synapse connection circuit in FIG. 16 as described later.

FIG. 14A schematically shows the connection between the feature integration layer neuron and the feature detection layer neuron before and after the pacemaker neuron. Here, the pacemaker neuron PN ₂ is attached to the feature detection layer neuron N ′ _2D , and the feature integration layer neurons (N _1I ,...) Belonging to the same receptive field as N ′ _2D with respect to the feature integration layer receiving direct input. , N _4I ). Similarly, the pacemaker neuron PN ₃ is attached to the feature detection layer neuron N ′ _3D , and the feature integration layer neurons (N _3I ,...) Belonging to the same receptive field as N ′ _3D with respect to the feature integration layer receiving direct input. , N _{6 I} ). There is no input to N _1I , the input from N _2I to N _4I is strong, and N _5I , N
The input to _6I shall be weak. A connection is made between the pacemaker neuron and the feature integration layer neuron, and a signal flows in one direction from the pacemaker neuron to the subsequent feature detection layer neuron.

The connection from the feature integration layer neuron to the feature detection layer neuron is shown by a dotted line in FIG. 14A (the synapse connection is omitted). Further, in order to transmit a pulse signal by such a connection, only when there is a predetermined pulse output ( _{PN2, out2} below) from the pacemaker neuron, it is opened only when there is a corresponding pulse output. The gate circuit (or switch circuit) is set on the output side of the synapse connection circuit. FIG. 16 shows an outline of this configuration. Note that _{PN2, out1} and _{PN2 , out2} are both pulse outputs from the phase synchronization detection circuit.

When a pulse signal from the feature integration layer neuron (N _jI ) is input to the pacemaker neuron (PN), at a predetermined cycle (T) after at least one pulse input from the feature integration layer neuron is received. A time window signal having a very short time width (a phase synchronization detection window in FIG. 1D) is generated from the phase synchronization detection and control circuit 7, and the pacemaker neuron element circuit is configured to input a pulse signal only in the time window. I do. At this time, the pulse output of _{PN2, out1} is output from the phase synchronization detection and control circuit 7 to the feature integration layer neuron in accordance with the timing control method described below.

If the temporal integration value of the input pulse signal in the phase synchronization detection window is higher than a predetermined threshold _and the phase of PN _{out2, pre} is within the predetermined time width as described above, the pulse signal output (PN _{2, out2} output pulse) from the phase synchronization detection and control circuit 7 to the feature integration layer neuron and the feature detection layer neuron, and when this pulse signal (PN2 _{, out2} ) is input, the feature integration layer neuron Is subjected to modulation processing such as phase modulation in a synapse connection circuit. In the feature detection layer neuron, the temporal integration of the pulse signal via the synapse connection from the feature integration layer neuron is represented by the time axis integration shown in FIG. Done within the window. In addition,
The timings of _{PN2 , out1} and _{PN2 , out2} after the synchronization is detected are fixed, and here, there is no delay in order to match with FIG. 3B.

FIG. 15 shows the pulse output timing of each neuron corresponding to FIG. Pacemaker neurons, the pulse output (PN _out1) output phase of the timing pulses in accordance with a modulation rule described below the starting time of the phase synchronization detection window based on the timing (PN _{2, out1) (2}
πT _pi / T). Where the index i of T _pi
Represents the pulse output timing related to the i-th modulation. Presentation after the first pacemaker neuron output in FIG. 15 in the image (PN _{2, out1)} pulse integrated value of the time (transition time) synchronization establishment in the phase synchronization detection window after a T _p1 + T _p2 from after an The second output (PN2 _{, out2} ) is detected when the value exceeds the predetermined value. Note that the first PN _{2, out1} is output at the same timing as the phase synchronization detection window, and T 'is a fixed time which is the time from when the first synchronization is detected until the timing signal (PN _{2, out1} ) is generated, It is assumed that the output of _{PN2, out2} is controlled so that T ′ does not exist in a steady state in which the phases are synchronized after the first synchronization is detected.

Here, the output phase of the pacemaker neuron normalized by the time window generation cycle is represented by P _p (= 2πT
expressed as _{p /} T), The time t (time window generation of the pulse signals from a plurality of feature integration layer neurons in discrete time) that the period T as a unit phase P _I of the ensemble average <P _I>
t (For example, if the phase at time t of the output pulse from the integrated layer neuron n _I is P _I (t),

[Outside 1]

Is obtained. Here, N is the number of feature integrated layer neurons to be input. Time t per the predetermined time width
Is assumed to be Δ <P _I > _t and α is a positive constant less than 1, the phase modulation amount ΔP _p of the timing pulse output from the pacemaker neuron (PN) is expressed as follows, for example. You.

[0082] _{ΔP p (t + 1) =} -αP p (t) [Δ <
P _I > _t / ΔP _p (t)] Here, time t + 1 is represented by discrete time for convenience, and a unit discrete time elapses from time t at which at least one pulse input from the feature detection layer neuron is received. Indicates the time. According to this equation, P _p is controlled in the opposite direction by a factor proportional to Δ <P _I > corresponding to the unit change amount of the immediately preceding phase control amount.

[0083] By such modulation to be performed by the phase of the PN _out1 alternatives, changes the timing of the integration layer neuron output acts to integration layer neuron output entering the phase synchronization detection within the window is equal to or greater than a predetermined value.

It is needless to say that another modulation method may be used as long as the phase of the output pulse from the pacemaker neuron is modulated so that the fluctuation of the pulse phase from the integrated layer neuron is reduced. The convergence is determined based on whether or not the phase variation (for example, variance) of the pulse signal from the feature integration layer neuron is equal to or less than the time width of the time window (phase synchronization detection window) in which the pacemaker-curon occurs. The convergence determination circuit (FIG. 13C) of the phase synchronization detection and control circuit 7 performs a convergence determination and sends a signal for stopping phase modulation to the PN _out1 output circuit when the convergence is determined.

By performing such an adjustment in each pacemaker neuron, not only the phase synchronization of one pacemaker neuron and the pre-layer feature integration layer neuron, but also a synchronization cluster of the pacemaker neuron group and the feature integration layer neuron group is formed. You. That is, timing signals from the pacemaker neurons synchronized with each other are input to the integrated layer neurons corresponding to the overlapping receptive field portion.

As a prerequisite, the pulse output level (eg, pulse width) from the pacemaker neuron to the feature integration layer neuron is the pulse of the pulse from the feature detection layer neuron, which is another input component, until the detection of synchronization establishment shown below. Greater than the output level (similarly for example the pulse width)
Therefore, it is assumed that the relative contribution to the temporally integrated value (current base) of the pulse signal in each feature integration (detection) layer neuron is large.

Next, the process of establishing synchronization within the pacemaker neuron group will be described. Until the synchronization is established, for example, for a change in which the phase variation (variation amount of the ensemble average) Δ <P _I > of the pulse signals from the different feature integration layer neurons becomes large, each feature integration layer neuron performs _Are updated by the modulation formula of ΔP _p so that the phase difference between the pulse outputs of the above becomes small.

For example, the phase of each output (PN _out1 ) of the pacemaker neurons PN1 and PN2 at time t is represented by P
_{p1 (t),} when the _P p2 (t), the phase difference [delta] P of the outputs of modulated _{(t) (= P p2 (} t) -P p1 (t)) is, δP (t + 1) = δP (t) [1-αΔ <P _I > t /
ΔP _p (t)]. Therefore, αΔ <P _I > t / ΔP
_{If the} phase synchronization detection and control circuit 7 modulates ΔP _p (t) so that _p (t) is a positive number less than 1, the phase difference between the pacemaker neurons converges to zero.

After the synchronization is detected, the pacemaker output (PN
_out1 ) performs phase synchronization detection so that the output from the feature detection layer neuron is equal to or lower than the pulse output level of the feature integration layer neuron so that the output from the feature detection layer neuron is determined based on the temporal integration value regarding the pulse output of the feature integration layer neuron. The control circuit 7 controls the output. As described above, until the synchronization detection, the smaller the phase variation of the timing signal output from the plurality of pacemaker neurons that output to the overlapping receptive field, the smaller the phase variation of the timing signal output from the plurality of feature integrated layer neurons belonging to the same receptive field is output to the pacemaker neuron The phase difference between the pulses is reduced. As a result, the integration value in the phase synchronization detection window exceeds the threshold, and synchronization is detected. In this way, the sign of the phase change of the input pulse signal from the feature integration layer neuron that also receives the timing signal from another adjacent pacemaker neuron for each pacemaker neuron (for example, in the direction in which the phase average increases, that is, the phase becomes slower) When there is a pulse input of the component, the timing pulse signal from the pacemaker is modulated such that the timing pulse is output earlier and smaller (α is a positive constant less than 1). An (indirect) interaction with respect to the output pulse phase occurs, which results in a pull-in that results in phase synchronization between the pulse signal from the feature integration layer neuron and this timing clock pulse. It should be noted that the phase synchronization is not based on the pull-in by the timing control from such a pacemaker neuron, for example, as shown in FIG.
As shown in (1), the phase variation (for example, when there are a plurality of phases to be compared, the average value of the ensemble of the phases, etc.) is determined by using the phase comparator 8 for comparing the phase of the output pulse signal with the adjacent pacemaker neuron. The output pulse timing may be controlled so as to converge to a predetermined value. This concludes the description of the control of the local timing signal.

Referring again to FIG. 3, in the feature detection neuron, before there is an input from the pacemaker neuron, time window integration described later is not performed, and the same integration is performed with a pulse input from the pacemaker neuron as a trigger.

Here, a time window (different from the time window of the pacemaker neuron which is the phase synchronization detection window shown in FIG. 15) is determined for each characteristic detection cell (n ′ _i ), and the same receptive field is defined for the cell. It is common to each neuron in the forming feature integration layer and to the pacemaker neuron and provides a time range for time window integration.

The pacemaker neuron at layer number (1, k) (k is a natural number) outputs the pulse output from each feature-integrated cell of layer number (2, k-1) and the feature detection cell (layer) to which the pacemaker neuron belongs. The number (1, k)) provides a timing signal for the generation of a time window when the feature detection cells temporally add their inputs. The start time of this time window is a reference time for determining the arrival time of the pulse output from each integrated cell. That is, the pacemaker neuron provides a pulse output time from the feature-integrated cell and a reference pulse for time window integration in the feature-detected cell.

Each pulse is given a predetermined amount of phase delay when passing through the synapse circuit, and further, reaches a feature detecting cell through a signal transmission line such as a common bus. The arrangement of the pulses on the time axis at this time is indicated by the pulses (P ₁ , P ₂ , P ₃ ) indicated by dotted lines on the time axis of the feature detection cells.

[0094] Each in feature detection cells pulsed _(P 1, P
₂ , P ₃ ) time window integration (usually one integration; however, charge accumulation by multiple time window integrations or averaging of multiple time window integrations may be performed) As a result, when it becomes larger than the threshold value, a pulse output (P _d ) is made based on the end time of the time window. Note that the learning time window shown in the figure is referred to when a learning rule described later is executed.

Spatio- temporal integration of pulse output and network
Click properties Next calculation of spatial weighted sum when the input pulses (weighted sum) will be described. As shown in FIG. 7B, in each neuron, the weighted sum of the input pulses is calculated by a predetermined weighting function (for example, Gaussian) for each sub-time window (time slot), and the sum of the weighted sums is compared with a threshold. Is done. τ _j represents the center position of the weighting function of the sub-time window j, and is represented by the start time reference (elapsed time from the start time) of the time window. The weight function is generally a function of a distance (shift on a time axis) from a predetermined center position (representing a pulse arrival time when a feature to be detected is detected).

While phase modulation is performed at the synapse in accordance with the type of feature, multiplication according to the synapse load is performed on the neuron output level in the preceding layer by referring to the weight function value on the time axis. Is also good.

When the shape of the weighting function in each sub-time window is symmetric, the center position τ of the weighting function of each sub-time window (time slot) of the neuron is determined by the time delay between the neurons after learning. Then, a neural network that performs spatio-temporal weighted summation of input pulses (weighted sum) is a kind of time-domain radial basis function network (Radial Basis Fun Network).
ction Network; hereinafter abbreviated as RBF). The time window F _Ti of the neuron n _i using the weight function of the Gaussian function is expressed as σ for each sub-time window and b _ij for the coefficient factor.

[Outside 2] ... (1) Note that the weight function may take a negative value. For example, when the neurons in a feature detection layer is expected to detect the triangle Finally, if the graphic pattern is evident feature that is not a component of _(F fauls _e) is detected, In the input sum calculation processing, a negative contribution is given from the pulse corresponding to the feature (F _false ) so that the detection output of the triangle is not finally performed even if the contribution from other feature elements is large. Such weighting functions and feature detection (integration) coupling from cells can be provided.

[0098] space sum when the input signal to the neuron n _i of the feature detection layer X _i (t) is

[Outside 3] ... (2) Here, ε _j is the initial phase of the output pulse from the neuron n _j , and converges to 0 by synchronous firing with the neuron n _i , or changes the phase of the time window by the timing pulse input from the pacemaker neuron. When forcibly synchronizing to 0, ε _j may always be 0. FIG.
When the weighted function shown in FIG. 7B and the weighted function shown in FIG. 7B are executed, a temporal transition of the weighted sum value as shown in FIG. 7E is obtained. The feature detection neuron outputs a pulse when the weighted sum reaches a threshold value (Vt). The output pulse signal from the neuron n _i, as described above, the upper layer with a space-time sum squashing nonlinear function to become an output level and a given time delay by learning (the so-called total input sum) of the input signal (phase) Output to neuron (pulse output is fixed frequency (binary), and output is added by adding a phase modulation amount that becomes a squashing nonlinear function for spatiotemporal sum of input signal to a phase corresponding to a fixed delay amount determined by learning ).

Processing mainly performed in the feature detection layer (at the time of learning and recognition)
Will be described. In each feature detection layer, as described above, pulse signals related to a plurality of different features from the same receptive field are input in the processing channel set for each scale level, and a spatio-temporal weighted sum (weighted sum) is input.
Perform calculations and threshold processing. Pulses corresponding to each feature amount arrive at predetermined time intervals according to a delay amount (phase) predetermined by learning. Since the learning control of the pulse arrival time pattern is not the focus of the present invention and will not be described in detail, for example, it is assumed that a feature element constituting a certain figure pattern arrives earlier so that it is a feature that contributes most to the detection of the figure. In this case, competitive learning is introduced such that the feature elements whose pulse arrival times are almost equal arrive at a certain amount of time apart from each other. Alternatively, predetermined feature elements (feature elements constituting a recognition target, which are considered to be particularly important: for example, a feature having a large average curvature, a feature having a high linearity, etc.) arrive at different time intervals. It may be designed as follows.

In this embodiment, the neurons corresponding to the respective lower-order feature elements in the same receptive field on the feature integration layer, which is the preceding layer, are synchronously fired (pulse output) at predetermined phases.
Will do. In general, there is a connection to a feature detection neuron that detects the same higher-order feature that is different in position but different in neurons in the feature integration layer (in this case, the same higher-order feature is formed although the receptive field is different) To have a bond).
At this time, it goes without saying that synchronous firing occurs even with these feature detection neurons. However, the output level (here, a phase reference; however, a configuration based on a frequency, an amplitude, and a pulse width may be used) is a sum (or an average) of contributions from a plurality of pacemaker neurons provided for each receptive field of the feature detection neuron. Etc.). Also,
In each neuron on the feature detection layer, the calculation of the spatiotemporally weighted sum of the input pulses (the weighted sum) is performed only in a time window of a predetermined width for the pulse train that has arrived at the neuron. Means for implementing weighted addition within a time window are:
It goes without saying that the present invention is not limited to the neuron element circuit shown in FIG.

This time window corresponds to the refractory period of the actual neuron.
(refractory period) It corresponds to the time zone to some extent. In other words, there is no output from the neuron regardless of any input during the refractory period (time range other than the time window), but the firing according to the input level occurs in the time window other than that time range. Is similar to the neuron. FIG.
The refractory period shown in (B) is a time period from immediately after the firing of the feature detection cells to the next time window start time. Needless to say, the length of the refractory period and the width of the time window can be arbitrarily set, and the refractory period does not have to be shorter than the time window as shown in FIG.

In this embodiment, as schematically shown in FIG. 6, as a mechanism already described, for example, for each feature detection layer neuron, a pacemaker neuron that receives an input from the same receptive field (at a fixed frequency) By inputting the timing information (clock pulse) by the pulse output), the above-mentioned common start timing is brought about.

In the case of such a configuration, it is not necessary to perform the synchronization control of the time window (if it is necessary) over the entire network, and there is a fluctuation and fluctuation of the clock pulse as described above. Is also uniformly affected by the output from the local receptive field (the fluctuation of the position of the window function on the time axis is the same between neurons forming the same receptive field). The reliability of detection does not degrade. Since a highly reliable synchronous operation is enabled by such local circuit control, the tolerance of variation regarding circuit element parameters is also increased.

Hereinafter, a feature detecting neuron for detecting a triangle as a feature will be described for simplicity. The feature integration layer at the preceding stage provides L-shaped patterns (f ₁₁ , f ₁₂ ,...) Having various orientations as shown in FIG. 7C, and continuity (connectivity) with the L-shaped pattern. Line segment combination pattern (f
₂₁ , f ₂₂ ,...), A combination of two sides forming a triangle (f ₃₁ ,...), And the like.

Also, f ₄₁ , f ₄₂ , and f _{43 in} the figure are features constituting triangles having different directions, and indicate features corresponding to f ₁₁ , f ₁₂ , and f ₁₃ . As a result of setting a specific delay amount between neurons forming interlayer connection by learning, in a triangular feature detection neuron, each sub time window (time slot) (w ₁ , w ₂ , ...)
The setting is made in advance so that the pulses corresponding to the main and different features constituting the triangle arrive.

For example, w ₁ , w ₂ ,
..., the w _n as shown in FIG. 7 (A), pulses corresponding to the combination of a set of features as constituting a triangle as a whole arrive first. Here, the L-shaped pattern (f ₁₁ ,
f ₁₂ , f ₁₃ ) arrive within w ₁ , w ₂ , w ₃ respectively, and the feature element
pulse corresponding to _{(f 21, f 22, f} 23) , respectively w _1, w _2, w ₃
The amount of delay is set by learning so as to arrive within.

Pulses corresponding to the characteristic elements (f ₃₁ , f ₃₂ , f ₃₃ ) arrive in the same order. In the case of FIG. 7A, a pulse corresponding to one characteristic element arrives in one sub time window (time slot). The meaning of dividing into sub-time windows is that, by performing pulse detection (detection of characteristic elements) corresponding to different characteristic elements developed and expressed on the time axis in each sub-time window individually and reliably, those characteristics are obtained. The purpose of the present invention is to increase the changeability and adaptability of the processing mode, such as whether the conditions of all feature elements are to be detected or a certain percentage of features are to be detected. . For example, in a situation where the target of recognition (detection) is a face, and the search (detection) of the eyes, which is a component of the face, is important, it is necessary to set the priority of the eye pattern detection high in the visual search. Introducing feedback feedback from a higher-order feature detection layer can selectively enhance the reaction selectivity (detection sensitivity of a specific feature) corresponding to a feature element pattern constituting an eye. By doing so, it is possible to give higher importance to the lower-order feature elements constituting the higher-order feature elements (patterns) and detect them.

Further, if it is set in advance that a pulse arrives in a sub time window as soon as an important feature is reached, the weight function value in the sub time window must be larger than the value in the other sub time windows. Thereby, the feature having a higher importance can be more easily detected. This importance (detection priority between features) can be obtained by learning or can be defined in advance.

Therefore, as long as it is only necessary to detect a certain percentage of characteristic elements, the division into sub-time windows has little meaning and may be performed in one time window.

Note that a plurality (three) of different characteristic elements
The corresponding pulses arrive and add up
(FIG. 7D). That is, one sub time window (time
Slot) with multiple feature elements (Fig. 7 (D)) or optional
Before the pulses corresponding to the number of feature elements are input
It may be a stake. In this case, in FIG.
In the time window, the vertex of the triangle f₁₁Support detection of
Other characteristic element f₂₁, F₂₃The pulse corresponding to arrives
Similarly, in the second sub-time window, the vertex angle portion f ₁₂Inspection
Other features f that support₂₂, F₃₁Pal
Has arrived.

The number of divisions into sub-time windows (time slots), the width of each sub-time window (time slot), the class of the feature, and the assignment of the pulse time interval corresponding to the feature are not limited to those described above. Needless to say, it can be changed.

Application Example Mounted on Photographing Apparatus When pattern recognition (detection) apparatus according to the configuration of this embodiment is mounted on photographing means, focusing on a specific subject, color correction of a specific subject, and exposure control are performed. Will be described with reference to FIG. FIG. 12 is a diagram illustrating a configuration of an example in which the pattern detection (recognition) device according to the embodiment is used for an imaging device.

An image pickup apparatus 1101 shown in FIG. 12 includes an image forming optical system 1102 including a photographing lens and a drive control mechanism for zoom photographing.
D or CMOS image sensor 1103, imaging parameter measurement unit 1104, video signal processing circuit 1105, storage unit 1106, control signal generation unit 1107 that generates control signals such as control of imaging operation, control of imaging conditions, and viewfinders such as EVF And a recording medium 1110, and the above-described pattern detection device as a subject detection (recognition) device 1111.

The image pickup apparatus 1101 detects, for example, a face image of a person registered in advance (detection of the position and size of a person) from a captured image by a subject detection (recognition) apparatus 1111. When the position and size information of the person is input from the subject detection (recognition) device 1111 to the control signal generation unit 1107, the control signal generation unit 1107
Focus control for that person based on the output from 04,
A control signal for optimally controlling exposure condition control, white balance control, and the like is generated.

By using the above-described pattern detection (recognition) device for an image pickup device in this way, the function of reliably detecting (recognizing) the subject can be realized with low power consumption and high speed (real time). Detection and optimal control of photographing based on the detection (AF, AE, etc.).

(Second Embodiment) FIG. 8 shows a configuration example of a pacemaker neuron and another neuron (a neuron group of a feature integration layer and a detection layer) connected to the pacemaker neuron in this embodiment. The first embodiment is characterized in that there is one pacemaker neuron for each group obtained by dividing a feature detection layer neuron for detecting the same feature category into a predetermined number and to generate an independent timing pulse signal. And different.

In this way, the overlapping receptive field structure for the feature integration layer between adjacent feature detection layer neurons is eliminated (the feature integration layer neuron receives only the timing signal from one pacemaker neuron), The number of neurons can be reduced. Further, the transition time (T _p1 + T _{p2 in} FIG. 15) until the phase synchronization is established as in the previous embodiment can be eliminated, and the processing can be speeded up. FIG. 9 shows an example of the pulse output timing of each neuron corresponding to FIG. The pulse output (PN _out ) of the pacemaker neuron is PN
When the time integrated value of the input pulse within the time window (shown in FIG. 9) exceeds a predetermined threshold value, it is performed independently of other pacemaker neurons.

The PN time window is generated a predetermined time after the first pulse input from the feature integration layer neuron (or the first pulse after the pulse output from the pacemaker neuron when repeatedly output in a steady state). Is done. In order to generate a time axis integration window in the feature detection layer, two pulse outputs (PN _{2, out1} and PN _2,
out2 ) is unnecessary.

In the present embodiment, the pacemaker neuron does not always generate a timing pulse signal at a constant interval. Instead, the time of the pulse signal from the neuron group in the receptive field of the preceding layer (such as the feature integration layer) of the pacemaker neuron is used. The timing signal is generated only when the average number of pulses reaches a predetermined threshold level. By doing so, power consumption is reduced.

(Third Embodiment) FIGS. 10 (A), (B), and FIG. 11 show configuration examples relating to connections centering on pacemaker neurons in this embodiment. For example, as a representative position of the feature detection layer neuron group obtained a pacemaker neurons are divided similarly to the second embodiment, only the feature integration layer neuron that is input to the neuron (N _3I) which is the gravity center position or a nearest neighbor In response to the pulse signal, a timing pulse signal is generated under predetermined conditions.

[0121] pacemaker neuron PN in FIG. 10 (A) has a bound for outputting a pulse signal for the feature detection layer neuron _{(N '1D, N' 2D} , N '3D) timing control for. N ' _2D is located at the center of gravity of the neuron group of the feature detection layer that is connected to the PN, and the feature integrated layer neurons N _1I , N _2I , N _3I , and N _4I that are connected to the N' _2D are interconnected with the pacemaker neuron (Bonds are indicated by thick arrows). With such a configuration, the number of wires required for mutual coupling is reduced as compared with the second embodiment. In addition, even if the wiring is reduced as described above, the operation problem is caused by the elimination of overlapping receptive field structures and the use of a pacemaker neuron that generates an independent timing pulse (a pacemaker neuron caused by asynchronous input of a plurality of timing control signals). And a time window generation timing in the feature detection layer).

Similarly, in the configuration shown in FIG.
Only one of the neurons in the vicinity of the center of gravity (N _3I ) of the feature integrated layer neuron group to which N ′ _2D receives the input is interconnected. Such an arrangement results in further reduction of wiring for interconnection. Further, the pacemaker neuron may be configured to receive an input only from a neuron that outputs the maximum among the feature integration layer neurons that should receive the input in FIG. 10B.

For example, in the configuration shown in FIG. 11, a so-called Winner-Take-All circuit (hereinafter, referred to as a WTA circuit) for detecting the maximum output between the feature integration layer neuron group and the pacemaker neuron is set, and the pacemaker is set. The neuron receives the output from the WTA circuit. The connection between the feature integration layer neuron group and the feature detection layer neuron group is the same as in FIGS. 10A and 10B. By performing local timing control based on the output from the feature integration layer neuron that detects the most prominent feature in a local area within a predetermined range on input (image) data,
A stable operation of parallel pulse signal processing with a smaller circuit scale is realized.

[0124]

As described above, according to the present invention, a hierarchical parallel processing circuit which operates via a pulse signal under the control of a distributed local timing clock signal, particularly a plurality of local timing circuits Input a clock signal,
In a parallel processing circuit composed of a group of arithmetic elements for performing spatio-temporal integration of pulse signals and threshold processing, there is an effect that stable operation can be ensured and power consumption can be reduced.

[Brief description of the drawings]

FIG. 1 is a diagram showing an overall configuration diagram of a network.

FIG. 2 is a diagram showing a configuration of a synapse section and a neuron element section.

FIG. 3 is a diagram showing how multiple pulses propagate from a feature integration layer (or an input layer) to a feature detection layer neuron.

FIG. 4 is a diagram illustrating a configuration of a synapse circuit.

FIG. 5 is a configuration diagram of a synapse connection small circuit, and Embodiment 1
1 shows a configuration diagram of a pulse phase delay circuit used in FIG.

FIG. 6 is a diagram showing a network configuration in a case where an input from a pacemaker neuron is provided to a feature detection layer neuron.

FIG. 7 shows a configuration of a time window when processing a plurality of pulses corresponding to different feature elements input to the feature detection neuron,
It is a figure which shows the example of a weight function distribution, and the example of a characteristic element.

FIG. 8 is a schematic diagram of a connection structure centering on a pacemaker neuron according to a second embodiment.

FIG. 9 is a diagram showing pulse firing timing of each neuron.

FIG. 10 is a schematic diagram of a connection structure centering on a pacemaker neuron.

FIG. 11 is a schematic diagram of a connection structure centering on a pacemaker neuron.

FIG. 12 is a diagram illustrating a configuration example of a photographing device equipped with a pattern recognition device.

FIG. 13 is a diagram showing a configuration block of a pacemaker neuron circuit that generates a local timing signal.

FIG. 14 is a schematic diagram of an example of a connection structure centering on a pacemaker neuron.

FIG. 15 is a diagram showing pulse firing timing of each neuron.

FIG. 16 is a diagram schematically illustrating an example of a combination between a feature integration layer and a feature detection layer.

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 7/18 G10L 3/00 539

Claims (12)

[Claims] 1. A plurality of arithmetic elements coupled to each other based on a predetermined rule and arranged in parallel, and a pulse-like signal is generated at a predetermined frequency in accordance with an input time-series signal pattern, and an operation in a predetermined neighborhood is performed. A plurality of local timing signal generation circuits for outputting the pulse-like signal to a group of elements; a synchronization detection for detecting synchronization between at least one of the signals output from the local timing signal generation circuit and an output from the predetermined arithmetic element group A signal processing circuit, wherein the arithmetic element performs a predetermined operation based on an output of the synchronization detection circuit. 2. The signal processing circuit according to claim 1, wherein the arithmetic element group and the local timing signal generating circuit form at least one synchronous cluster locally by interaction. 3. A plurality of arithmetic elements which are coupled to each other based on a predetermined rule and are arranged in a hierarchical and parallel manner, and generate a pulse-like signal at a predetermined frequency in accordance with an input time-series signal pattern. A local timing signal generation circuit that outputs the pulse-like signal to the first and second arithmetic element groups in the vicinity of the detection circuit, and a detection circuit that detects the correlation or synchronization between the pulse-like signal and the first arithmetic element group. The second operation element group receives an output from the first operation element group based on an output from the detection circuit, and the local timing signal generation circuit performs a predetermined operation of the second operation element group. A signal processing circuit provided for each of a number of arithmetic element groups. 4. The local timing signal generation circuit according to claim 3, wherein an operation element at a representative position of the second operation element group receives an output from a first operation element which receives an input. A signal processing circuit as described. 5. The local timing signal generating circuit outputs an output from a first arithmetic element group which receives an input to an arithmetic element at a representative position of the first arithmetic element group corresponding to the second arithmetic element. 4. The signal processing circuit according to claim 3, wherein the signal processing circuit receives the signal. 6. The local timing signal generating circuit outputs an output from a first arithmetic element group which receives an input to an arithmetic element at a representative position of the first arithmetic element group corresponding to the second arithmetic element. 4. The signal processing circuit according to claim 3, wherein the signal processing circuit receives a maximum value. 7. The signal processing circuit according to claim 1, wherein a part or all of the connection between the local timing signal generation circuit and the first arithmetic element is a mutual connection. 8. The detection circuit for detecting synchronization of a local timing signal from the local timing signal generation circuit, wherein the synchronization is detected by inputting a pulse signal from another local timing signal generation circuit. 4. The signal processing circuit according to claim 3, wherein a phase difference between the pulse signal and the local timing signal is detected. 9. The local timing signal generation circuit includes an output pulse timing modulation circuit that modulates a pulse output timing based on a temporal variation pattern of pulse outputs from the plurality of first arithmetic elements. The signal processing circuit according to claim 3, wherein: 10. An image input apparatus comprising the signal processing circuit according to claim 1, wherein an image input operation is controlled based on an output of the signal processing circuit. 11. An input unit for inputting data of a predetermined dimension, a plurality of data processing modules each having a feature detection layer for detecting a plurality of features from data input from the input unit, and the plurality of data processing modules Output means for outputting a recognition result of a predetermined pattern based on the output of the data processing module. The data processing module has a plurality of arithmetic elements and a plurality of local timing signal generation elements which are connected to each other by predetermined synapse connection means. The arithmetic element outputs a pulse-like signal at a frequency or timing based on input signals from the plurality of local timing signal generation elements and arrival time patterns of a plurality of pulses within a predetermined time window. Pattern recognition device. 12. An image input device comprising the pattern recognition device according to claim 11, wherein an image input operation is controlled based on an output of the pattern recognition device.