CN107330915B - Target tracking method for AER image sensor - Google Patents

Abstract

The present invention relates to the field of AER image sensor image recognition. In order to meet the requirements of an AER image sensor-based recognition system for tracking algorithm event-driven, low computational complexity and providing target states, the present invention aims to propose a target for AER image sensors. tracking method. The technical solution adopted in the present invention is that the target tracking method for AER image sensor is based on event-driven, generating different trackers to distinguish different targets, and each tracker includes the following information: the center coordinate x _c of the tracker, the size R _c , the search range R _k , the tracker generation time T _c , the event time T _l finally accepted, and the activity value A a total of 6 parameters. The invention is mainly applied to the occasion of image recognition of the image sensor.

Description

Object tracking method for AER image sensor

technical field

The invention relates to the field of image recognition of AER image sensors, in particular to a target tracking method for AER image sensors.

Background technique

In the field of real-time target recognition, the explosive demand for high resolution and frame rate increases the pressure on information transmission and storage, and also puts forward huge requirements on the processing speed of recognition. The optimization of back-end algorithms and architecture cannot fundamentally solve many problems. The bottleneck greatly limits the development of the recognition system. AER (Address-Event Representation, AER, address-event representation) image sensor is a biomimetic vision sensor that simulates the asynchronous sparse event-driven signal processing on the biological visual cortex. The pixels of the sensor encode light intensity change information into events, and only pixels that detect light intensity changes can output events, which significantly reduces redundant data and provides a new idea for breaking through the bottleneck of real-time identification systems.

Different from the frame-driven traditional image sensor, the AER image sensor outputs an asynchronous event stream, each event contains the pixel address where the light intensity changes, the generation time and the event polarity (the light intensity increases or decreases). However, most of the existing image processing algorithms are based on frame driving and cannot be directly used for AER data processing. Dividing events into fixed time slices is a simple way to adapt to these algorithms, but it cannot take advantage of the AER sensor, which increases the processing time of each part of the algorithm, which is not conducive to the performance of real-time recognition systems.

In practical applications, the target tracking algorithm is indispensable, which not only needs to realize the function of target detection, but also gives the real-time position of the target, and also prevents the interference of noise. If there are multiple moving targets, in many cases, such as when the targets overlap, it is still possible to accurately locate different targets, which is the key to the realization of the tracking algorithm. Furthermore, the event-driven concept is particularly important for tracking algorithms based on AER image sensors. The time-slicing algorithm may cause an instantaneous action or target to be divided into two time-slices, affecting the tracking accuracy.

At present, there are many event-driven cluster tracking algorithms, but basically each algorithm can play the best performance in specific applications. The commonly used tracking algorithms are mainly to achieve target detection, but in the field of target recognition, different The state of the target determines whether the recognition requirements are met, which is particularly important in the continuous monitoring of moving targets. In addition, for the special application of target recognition, it is extremely necessary to simplify the algorithm complexity as much as possible and reduce the consumption of computing resources. It is extremely critical to design an event-driven target tracking algorithm for AER image sensor to provide target status information for back-end recognition.

SUMMARY OF THE INVENTION

In order to overcome the deficiencies of the prior art, according to the requirements of an AER image sensor-based recognition system for tracking algorithm event-driven, low computational complexity and providing target states, the present invention aims to propose a target tracking method for AER image sensors. The technical solution adopted in the present invention is that the target tracking method for AER image sensor is based on event-driven, generating different trackers to distinguish different targets, and each tracker includes the following information: the center coordinate x _c of the tracker, the size R _c , the search range R _k , the tracker generation time T _c , the event time T _l finally accepted, and the activity value A a total of 6 parameters;

According to the generation time of the event stream output by the AER sensor, each event is processed in sequence. For an event whose coordinate is x _e , the Euclidean distance R between it and the existing trackers is calculated first. If it is in the search of a tracker Within the range R _k , the parameters of the tracker will be updated. If multiple trackers meet the conditions, the tracker with the earliest generation time will be updated. (1) is the Euclidean distance calculation and tracker discrimination formula:

R=|x _c -x _e |<R _k (1)

If there is no tracker that satisfies the conditions, a tracker whose center coordinate is x _e and other parameters are default will be created. Update, the center coordinate of the tracker at time t will be moved from x _c (t) to x _c (t+△t), and its calculation formula is shown in (2):

x _c (t+Δt)=x _c (t)·α _x +x _e ·(1-α _x ) (2)

The size of α _x is related to the moving speed of the center of the tracker, Δt is the difference between the time of the current event and the time when the tracker finally accepted the event, the size of the tracker R _c is related to the calculated Euclidean distance R, the calculation formula As shown in (3):

R _c (t+Δt)=max(R _min ,R _c (t)·α _R +R·(1-α _R )) (3)

The size of α _R is related to how fast the size of the tracker changes, R _min is the minimum size of the tracker, the search range R _k of the tracker is proportional to the size R of the tracker, and the multiple is α _k . The calculation formula of the active value A of the tracker is shown in (4):

Where p is a constant, the tracker can be in the hidden layer or the visible layer according to the size of its active value; the active value of the tracker in the hidden layer is low, which is caused by noise or represents an incomplete target; the tracker in the visible layer conforms to Recognition conditions, the event to which the target belongs will be sent to the subsequent modules of the recognition system; the newly created trackers are all in the hidden layer, and when the tracker's active value is higher than A _up , the tracker enters the visible layer; if the tracker is tracked If the active value of the tracker is lower than A _up , the tracker will fall back to the hidden layer; at the same time, if the active value of the tracker is lower than A _down , the tracker will be deleted directly; if the two trackers overlap, the subsequent events The tracker with the earlier creation time will be preferentially selected, and if the tracker with the earlier creation time cannot receive new events for a long time, it will also be deleted.

A look-up table is used to store index values with a number of N according to the minimum time step t _min , and the required value is retrieved according to the calculated Δt.

The characteristics and beneficial effects of the present invention are:

The invention proposes a target tracking algorithm based on an AER image sensor, which is based on event driving and solves the problem that the traditional frame driving tracking algorithm does not adapt to AER data. For the special application of target recognition, the computational complexity is reduced as much as possible, the active value is used to distinguish the states of different targets to determine whether the recognition requirements are met, and the lookup table is used to reduce the amount of active value calculation. The target tracking algorithm is beneficial to Utilization of AER image sensor for multi-target recognition.

Description of drawings:

Figure 1 Schematic diagram of the center movement of the tracker.

Figure 2 Schematic diagram of the inter-layer variation of the tracker.

Detailed ways

The event-driven target tracking algorithm for the AER image sensor proposed by the present invention is introduced as follows: The algorithm is based on event-driven, and each event output by the sensor will trigger a series of operations. The algorithm generates different trackers to distinguish different targets, each tracker contains the following information: the center coordinate x _c of the tracker, the size R _c , the search range R _k , the tracker generation time T _c , the last receiving event time T _l , the activity The value A has a total of 6 parameters.

The algorithm processes each event in sequence according to the generation time of the event stream output by the AER sensor. For an event whose coordinate is x _e , first calculate the Euclidean distance R between it and the existing trackers. If it is within the search range R _k of a tracker, the parameters of the tracker will be updated. If multiple trackers meet the condition, the tracker with the earliest spawn time is updated. (1) is the Euclidean distance calculation and tracker discrimination formula.

R=|x _c -x _e |<R _k (1)

If there is no tracker that satisfies the conditions, a tracker with center coordinates x _e and other parameters are default will be created. For trackers that meet the conditions, all parameters are updated except the creation time that remains the same. The center coordinate of the tracker at time t will move from x _c (t) to x _c (t+Δt), the calculation formula is shown in (2), and the moving process is shown in Figure 1.

x _c (t+Δt)=x _c (t)·α _x +x _e ·(1-α _x ) (2)

The size of α _x is related to the moving speed of the center of the tracker, and Δt is the difference between the time of the current event and the time when the tracker finally accepted the event. The size R _c of the tracker is related to the calculated Euclidean distance R, and the calculation formula is shown in (3).

R _c (t+Δt)=max(R _min ,R _c (t)·α _R +R·(1-α _R )) (3)

The size of α _R is related to the speed of change of the tracker size, and R _min is the minimum size of the tracker. The search range R _k of the tracker is proportional to the tracker size R, and the multiple is α _k . The calculation formula of the active value A of the tracker is shown in (4).

where p is a constant that is related to the magnitude of each increase in the active value. The tracker can be in the hidden layer or the visible layer according to the size of its active value. The tracker activity value of the hidden layer is low, which is generally caused by noise or represents an incomplete target. The tracker of the visible layer meets the conditions for recognition, and the event to which the target belongs will be sent to the subsequent modules of the recognition system. The inter-layer variation of the tracker is shown in Figure 2. The newly created trackers are all in the hidden layer, when the active value of the tracker is higher than A _up , the tracker enters the visible layer. If the active value of the tracker is lower than A _up , the tracker falls back to the hidden layer. At the same time, if the active value of the tracker is lower than A _down , the tracker will be deleted directly. Trackers created incorrectly due to noise will have difficulty receiving new events and will be quickly deleted. If two trackers overlap, the subsequent event will choose the one with the earlier creation time, and if the tracker with the earlier creation time cannot receive new events for a long time, it will also be deleted quickly.

The active value calculation of the tracker involves index calculation, and the calculation amount is relatively large. Therefore, the present invention uses a look-up table to store index values with a number of N according to the minimum time step t _min , and retrieve the required value according to the calculated Δt. , which greatly reduces the computational complexity.

According to the target tracking algorithm based on the AER image sensor proposed in the present invention, for an image sensor with a resolution of 128×128, the following are the default parameters of the tracker: the tracker size R _c is 15 pixels, and the search range R _k is 18 pixels Pixels, the last receiving time Tl is _-1ns , and the initial active value is 1v. In order to make the tracker's movement and size change smoother, the coordinate movement coefficient α _x is 0.95, the size change coefficient α _R is 0.95, and the ratio α _k of the search range R _k to the size R _c is 1.2. The two thresholds for the active value are related to the event rate, which can be set to 50 for A _up and 0.1 for A _d o _wn , and the constant p can be set to 1. In using the lookup table to simplify the calculation of active values, the minimum time step _tmin is 20ns, and τ is 40000ns. The lookup table stores values in the range of 0ns to 200000ns, with _tmin as the step, and a total of 10000 values are stored. The target tracking algorithm proposed in the present invention is driven by events, has low computational complexity, and can meet the requirements of target recognition systems based on AER image sensors.

Claims (2)

1. A target tracking method for an AER image sensor, characterized in that, based on event-driven, different trackers are generated to distinguish different targets, and each tracker contains the following information: center coordinate x _c of the tracker, size R _c , the search range R _k , the tracker generation time T _c , the event time T _l finally accepted, and the activity value A a total of 6 parameters; According to the generation time of the event stream output by the AER sensor, each event is processed in sequence. For an event whose coordinate is x _e , the Euclidean distance R between it and the existing trackers is calculated first. If it is in the search of a tracker Within the range R _k , the parameters of the tracker will be updated. If multiple trackers meet the conditions, the tracker with the earliest generation time will be updated. (1) is the Euclidean distance calculation and tracker discrimination formula: R=|x _c -x _e |<R _k (1) If there is no tracker that satisfies the conditions, a tracker whose center coordinate is x _e and other parameters are default will be created. Update, the center coordinate of the tracker at time t will be moved from x _c (t) to x _c (t+△t), and its calculation formula is shown in (2): x _c (t+Δt)=x _c (t)·α _x +x _e ·(1-α _x ) (2) The size of α _x is related to the moving speed of the center of the tracker, Δt is the difference between the time of the current event and the time when the tracker finally accepted the event, the size of the tracker R _c is related to the calculated Euclidean distance R, the calculation formula As shown in (3): R _c (t+Δt)=max(R _min ,R _c (t)·α _R +R·(1-α _R )) (3) The size of α _R is related to the speed of the size change of the tracker, R _min is the minimum size of the tracker, the search range R _k of the tracker is proportional to the size R _c of the tracker, the multiple is α _k , the active value of the tracker A The calculation formula is shown in (4):

Where p is a constant, the tracker can be in the hidden layer or the visible layer according to the size of its active value; the active value of the tracker in the hidden layer is low, which is caused by noise or represents an incomplete target; the tracker in the visible layer conforms to Recognition conditions, the event to which the target belongs will be sent to the subsequent modules of the recognition system; the newly created trackers are all in the hidden layer, and when the tracker's active value is higher than A _up , the tracker enters the visible layer; if the tracker is tracked If the active value of the tracker is lower than A _up , the tracker will fall back to the hidden layer; at the same time, if the active value of the tracker is lower than A _down , the tracker will be deleted directly; if the two trackers overlap, the subsequent events The tracker with the earlier generation time will be preferentially selected, and if the tracker with the earlier generation time cannot receive new events for a long time, it will also be deleted. 2. The target tracking method for an AER image sensor as claimed in claim 1, wherein a look-up table is used to store an index value of N according to the minimum time step t _min , and according to the calculated Δt Recall the desired value.

Images