CN118134972A - Target tracking method based on fusion of intensity image and point cloud data - Google Patents

Abstract

A target tracking method based on fusion of an intensity image and point cloud data relates to the fields of computer graphics and three-dimensional point cloud target tracking, and solves the problems of reduced tracking precision and even tracking failure caused by shielding of an existing target by an obstacle. According to the method, a multi-feature fusion mode is adopted, and nuclear correlation filtering tracking is carried out on a target intensity image formed by the laser radar by fusing the HOG features and Fourier description sub-features of the target of interest; converting a target range profile formed by the laser radar into a three-dimensional point cloud, and predicting a track of a point cloud target by using a Kalman tracker; and effectively judging the target shielding state by combining PSR and ISS, providing an adaptive factor according to different target shielding conditions, and correcting the positions of the kernel correlation and the Kalman tracker by using the adaptive factor to obtain a final tracking position. The method can carry out steady tracking on the target, and the average time consumption of each frame of data is 39ms.

Description

Target tracking method based on fusion of intensity image and point cloud data

Technical Field

The present invention relates to the fields of computer graphics and three-dimensional point cloud target tracking, and in particular to a target tracking method based on the fusion of intensity image and point cloud data.

Background technique

With the rapid development of science and technology, unmanned combat vehicles have emerged on the battlefield. As an important component of unmanned combat vehicles, the tracking system directly affects the ability to accurately track and accurately strike enemy targets, and the tracking algorithm is the key to determining the excellent performance of the tracking system. Three-dimensional imaging laser radar is a sensor that uses laser beams as carriers for active detection. It can obtain three-dimensional point cloud images of targets and has all-day, high-precision imaging detection capabilities. Laser radars using Geiger-Mode Avalanche PhotoDiode (GM-APD) detectors can perform high-sensitivity single-photon imaging of long-distance targets and have broad application prospects in modern military warfare. Due to the complex battlefield environment, enemy targets such as tanks and tanks are easily blocked and interfered by objects such as buildings and rocks, resulting in the loss of target information obtained by the laser radar, which increases the difficulty of target tracking tasks. The tracking algorithm based on the kernel correlation filter has high tracking accuracy and fast speed, but the tracking accuracy will be greatly reduced in the case of occlusion due to the less target feature information obtained. When the target is completely blocked, the Kalman filter tracking algorithm can predict the target position.

Summary of the invention

The present invention aims to provide a target tracking method based on the fusion of intensity image and point cloud data to solve the problem that the target is blocked by obstacles, resulting in reduced tracking accuracy or even tracking failure.

The target tracking method based on the fusion of intensity image and point cloud data is implemented by the following steps:

Step 1: Fuse the HOG features and Fourier descriptor features of the target intensity image formed by the laser radar, and perform kernel correlation filter tracking;

Step 2: Convert the target range image formed by the laser radar into three-dimensional point cloud data, and use the Kalman tracker to predict the trajectory of the point cloud target;

Step 3: Combine the peak sidelobe ratio PSR and the internal shape descriptor ISS to effectively judge the target occlusion state. According to the different target occlusion conditions, an adaptive factor is proposed. The kernel correlation filter tracking and the Kalman tracker predicted position are corrected using the adaptive factor to obtain the final tracking position.

Beneficial effects of the present invention:

1. The present invention fuses the Histogram of Oriented Gradient (HOG) features of the target intensity image with the Fourier descriptor to provide a more comprehensive target description and improve the robustness of the kernel correlation filter tracking algorithm.

2. The present invention utilizes the point cloud data converted from intensity image and range image combined with the peak-to-sidelobe ratio and internal shape description to effectively judge the target occlusion state, and proposes an adaptive factor to weight the tracking results of kernel correlation filtering and Kalman filtering according to different target occlusion conditions to obtain the final target position information.

3. The algorithm proposed in the present invention is experimentally verified on the KITTI dataset. The algorithm proposed in the present invention can accurately track targets in different occlusion states. The tracking accuracy is improved by 21.67% compared with the Kalman filter algorithm, and the accuracy is improved by 7.94% compared with the extended Kalman filter. The average processing time per frame is 51ms. In the actual sampling scene of the GM-APD lidar, the method of the present invention can robustly track the target, and the average processing time per frame of data is 39ms.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a target tracking method based on fusion of intensity image and point cloud data according to the present invention;

Figure 2 is a schematic diagram of the Kalman filter principle;

FIG3 is a target peak response diagram, wherein (a) is a peak response diagram when the target is not blocked, and (b) is a peak response diagram when the target is blocked;

Figure 4 is a diagram of the ISS key point extraction results;

Figure 5 is a diagram of the occlusion threshold result; (a) is the peak sidelobe ratio PSR = 7.69, (b) is the ISS key point N = 9; (c) is the peak sidelobe ratio PSR = 5.62, (d) is the ISS key point N = 6; (e) is the peak sidelobe ratio PSR = 1.77, (f) ISS key point N = 0;

FIG6 is a diagram of the algorithm tracking results; wherein (a) is the intensity image tracking result 1, (b) is the final tracking result 1, (c) is the intensity image tracking result 2, (d) is the final tracking result 2, (e) is the intensity image tracking result 3, (f) is the final tracking result 3, (g) is the intensity image tracking result 4, and (h) is the final tracking result 4;

FIG7 is an error curve and a tracking trajectory diagram; wherein (a) is a center position error curve diagram, (b) is a Kalman filter tracking trajectory diagram, (c) is an extended Kalman tracking trajectory diagram, and (d) is a fusion tracking trajectory diagram;

Figure 8 is a diagram of an outdoor detection scene;

Figure 9 is a diagram of the GM-APD laser radar tracking results; wherein, (a) is the intensity image tracking result 1, (b) is the final tracking result 1, (c) is the intensity image tracking result 2, (d) is the final tracking result 2, (e) is the intensity image tracking result 3, (f) is the final tracking result 3, (g) is the intensity image tracking result 4, (h) is the final tracking result 4, (i) is the intensity image tracking result 5, and (j) is the final tracking result 5.

Detailed ways

In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with the accompanying drawings and embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

The target tracking method based on the fusion of intensity image and point cloud data, as shown in FIG1, specifically includes the following steps:

Step S1, using a multi-feature fusion method, the intensity image of the target formed by the laser radar is fused with the HOG feature and the Fourier descriptor feature of the target of interest, and kernel correlation filtering tracking is performed;

Step S11, using the ridge regression function to train the classifier model of the kernel correlation filter tracking algorithm;

Assume that there is a training set {(a ₁ ,b ₁ ),…(a _i ,b _i )}, for the ridge regression function f( _xi )= ^wTai , the training goal is to minimize the mean square error between the sample vector _ai and its corresponding label _{bi .} The optimization formula is as follows:

Among them, λ is the regularization parameter to prevent overfitting, w is a column vector representing the weight coefficient, ||w|| ² represents the 2-norm of w, which controls overfitting. Formula (1) has a unique optimal solution, which can be calculated by the following formula:

w＝( ^ATA +λI) ^-1ATb (2 ⁾

Replace formula (2) with its plural form:

w＝(A ^H A+λI) ^-1 A ^H b (3)

Wherein, A represents the circulant matrix obtained by cyclic shift of sample vector a _i , each element in b represents a sample label b _i , I represents the identity matrix, A ^H is the Hermitian transpose of A, A ^H =(A ^* ) ^T , and A ^* is the complex conjugate of A.

Step S12, expanding the training samples through the circulant matrix; obtaining the trained kernel correlation filter tracking model;

Assume that the vector a = [a ₁ , a ₂ , …, a _n ] ^T represents the image block features containing the object of interest, and use it as the basic sample. The basic sample is used as the positive sample, and the vector a is shifted to obtain the negative sample by left multiplication with the permutation matrix P. The permutation matrix P is shown in (4):

Pa=[a _N-1 ,a ₀ ,a ₁ ,…,a _N-2 ] is the vector obtained by translating vector a one unit to the right. Here, P ^N a is used to represent the vector obtained by shifting vector a N units to the right. If N takes a negative value, vector a is shifted in the opposite direction.

After performing a circular shift operation on vector a, the circulant matrix A can be obtained:

The circulant matrix can be obtained by performing a discrete Fourier transform (DFT) on the diagonal matrix (diag) and is independent of the basis vector a. This property can be expressed by the following formula:

Among them, F represents a constant matrix independent of the basis vector a, which is called the Fourier matrix. For any vector z to be detected, it can be obtained by Get its Fourier transform, * is the conjugate operator, /> is the discrete Fourier transform of a, and N represents the N×N dimensional sample matrix.

If the training data are all obtained by cyclic shift of the basic samples, the Fourier matrix diagonalization method can be used to perform element-level operations to obtain the final result. Incorporating the diagonalization into the covariance matrix yields:

Since FF ^H =I, the above formula (8) is simplified to:

Since the diagonal matrix is operated only on the diagonal elements, we can get:

Where ⊙ represents element-by-element multiplication (matrix corresponding positions multiplication). Substitute equations (7) and (10) into equation (3) to calculate:

Performing Fourier transform on both sides of the result of equation (11) simultaneously, the results are as follows:

in, is a vector, λ is a regularization parameter to prevent overfitting, and the above formula is simplified according to the properties of the diagonal matrix:

Step S13, quickly detecting and locating the target based on the kernel correlation filter tracking model;

In the kernel correlation filter tracking algorithm, in order to introduce the kernel technique, it is necessary to transform the parameter w to be optimized from the linear space (original space) to the dual space α;

Set the sample X = [x ₁ ,x ₂ ,…,x _n ] ^T for training, and Y = [y ₁ ,y ₂ ,…,y _n ] ^T as the sample label value. The nonlinear transformation from the sample space to the Hilbert feature space is The corresponding kernel function The optimal solution to the optimization problem is:

In the formula, are respectively the mapping functions from samples _xi and x′ to the Hilbert feature space, x is the training sample, and x′ is the sample to be tested; its solution w is located in the subspace spanned by sample _xi , and w can be expressed as:

At this time, the variable sought changes from w to α, which is called the solution of the original space in the dual space.

Therefore the function response f(z) is:

where κ( _xi , _zi ) represents the training samples in high-dimensional space With/> The inner product of .

The closed-form solution of the ridge regression kernelization form is as follows:

α＝(K+λI) ^-1y (17)

Among them, y represents the general meaning of a sample label value. The kernel matrices corresponding to the polynomial kernel, linear kernel, and Gaussian kernel are all circulant matrices. The kernel matrix K is expressed as:

K＝C(κ ^xx ) (18)

In the formula, κ ^xx represents the first row of the kernel matrix K. Calculating formula (17) yields:

In order to detect the target, it is necessary to train the classifier in the candidate area to obtain the required regression function. The model of the candidate area is constructed by the following formula:

K ^z ＝C(κ ^xz ) (20)

Where ^Kz is the kernel matrix composed of training samples and candidate regions, and ^κxz is a cross-correlation matrix. From this, we can get the relevant response of the candidate region (the response after function mapping):

f(z)＝K ^z α (21)

The diagonalization characteristics of the Fourier matrix are used to quickly detect the target. The vector z to be detected and the response value of the trained classifier are expressed in the frequency domain as follows:

in, The position corresponding to the maximum value is the predicted target position.

Step S14: multi-feature fusion;

First, calculate the HOG features of the target intensity image. The specific steps are as follows:

a. Capture the image of the tracking area and convert it to grayscale to reduce the amount of calculation.

b. Use the Gamma correction method to normalize the input image so that the extracted features are more robust to illumination and background noise. The normalization operation formula is shown in (23):

I(h _x , _hy )＝I(h _x , _hy ) ^Gamma (23)

Where I(h _x , _hy ) is the image matrix, and Gamma is usually 5.0.

c. Calculate the gradient information of the pixel points to obtain the contour information of the target. First, use the [-1, 0, 1] gradient operator to perform a convolution operation on the original image to obtain the gradient component in the horizontal direction, and then use the [-1, 0, 1] ^T gradient operator to perform a convolution operation on the original image to obtain the gradient component in the vertical direction. The calculation formulas are shown in (24) and (25):

_Gx ( _hx , _hy )=I(hx ₊₁ , _hy )-I(hx _-1 , _hy ) (24)

G _y ( h _x , h _y ) = I ( h _x , h _{y + 1} ) - I ( h _x , h _{y - 1} ) (25)

In the formula, _Gx ( _hx , _hy ) represents the horizontal gradient value of the image I( _hx , _hy ) and _Gy ( _hx , _hy ) represents the vertical gradient value of the image I( _hx , _hy ). Therefore, the gradient value and gradient direction at the image ( _hx , _hy ) coordinate can be expressed as:

d. Divide the image into many small cells and calculate the gradient information of each cell. Divide the gradient direction of each cell into 9 bins, i.e., the interval of each bin is 20°, and then map the gradient information to the corresponding bin according to the gradient direction.

e. Count the gradient information in the block composed of cells, and finally concatenate the gradient histograms in all blocks to obtain the gradient histogram features of the entire image.

In this implementation, the Fourier descriptor features are calculated, and the calculation steps are as follows:

a. First, grayscale the intensity image and perform gamma correction.

b. Use the Canny edge detection algorithm to extract the contour point set {(x _c ,y _c )|c＝1,2,…,m} of the target to be tracked.

c. Calculate the center coordinates of the contour points:

d. Convert the contour points from rectangular coordinates to polar coordinates.

by The coordinates of the contour point {(x _c ,y _c )|c＝1,2,…,m} are converted to the corresponding polar coordinates {(r _c ,θ _c )|c＝1,2,…,m}

Where r _c is the distance between the center contour points.

Sort r _c in ascending order to obtain the center point contour point distance sequence, denoted as D = [r ₁ , r ₂ , …, r _m ].

e. Perform fast Fourier transform on the center contour point distance sequence.

In the formula, j is the Fourier transform imaginary unit, and the final constructed Fourier descriptor is:

Here, |·| represents the Fourier spectrum.

Finally, the HOG features are serially fused with the Fourier descriptor to improve the robustness of the kernel correlation filter tracking algorithm.

Step S2, converting the target range image formed by the laser radar into a three-dimensional point cloud, and using the Kalman tracker to predict the trajectory of the point cloud target;

The Kalman filter is divided into two stages: prediction and update. As shown in Figure 2, in the prediction stage, the optimal estimate of the target state at time t-1 is used to calculate the estimated value of the target state at time t. Since the influence of system noise is ignored in the prediction stage, there is a deviation between the estimated value and the actual state value; in the update stage, the estimated value is corrected based on the observed value of the target state at time t and the state observation matrix, so as to obtain the optimal estimate of the target state. The specific process is:

Step S21, state prediction;

The target state prediction process at any time t is as follows:

in is the optimal estimate of the target state at the previous moment, At _is the state transfer matrix, /> is the estimated target state at the current moment, /> is the covariance matrix corresponding to the optimal estimate of the target state at the previous moment,/> is the covariance matrix corresponding to the estimated value of the target state at the current moment, and _Qt is the noise covariance matrix corresponding to _ωt .

Step S22: Status update;

The target state update process at any time t is as follows:

Where _Kt is the Kalman gain matrix for obtaining the optimal estimate of the target state, _Ht is the observation matrix, _Rt is the covariance matrix of the measurement noise _υt , is the optimal estimate of the target state at the current moment, /> is the covariance matrix corresponding to the optimal estimate of the target state at the current moment, Z _t is the observed value of the target state at the current moment, and I _t is the unit matrix.

In the present invention, the target of interest is represented as a 6-dimensional vector composed of (x, y, z, θ, v _x , _vy , v _z ), where (x, y, z) represent the coordinates of the center point of the vehicle, (v _x , _vy , v _z ) are the velocity components in the x, y, and z directions during the vehicle's travel, and θ is the vehicle's orientation. Considering that the orientation does not change significantly during vehicle tracking on adjacent point cloud frames, the present invention ignores the target angular velocity and uses a uniform velocity model to describe the vehicle motion between frames. The state of a certain vehicle target is shown in formula (35):

X＝[x,y,z,θ, _vx , _vy , _vz ] ^T (35)

It is known that the state of the vehicle at time t-1 is X _t-1 . The state X _t at time t can be calculated according to the kinematic formula, as shown in formula (36):

Among them, Xt _-1 is the target state of the vehicle at time t-1, _Xt is the target state of the vehicle at time t calculated according to the kinematic model, Δt is the time interval between the point clouds of the t-1th frame and the tth frame, _At is the state transfer matrix from _Xt-1 to _Xt , and _ωt is the noise in the motion equation and obeys the zero-mean Gaussian distribution.

In this embodiment, the target position is estimated using a three-dimensional vehicle target motion model, and the model output value is used as the state observation value to establish an observation equation, as shown in formula (37):

Wherein, Z _t is the vehicle state observation value at time t, H _t is the observation matrix from X _t to Z _t , and υ _t is the observation noise at time t, which is the observation error of the target model in the present invention.

Step S3, combining the Peak Sidelobe Ratio (PSR) and the Internal Shape Signatures (ISS) to effectively judge the target occlusion state, and according to different target occlusion situations, propose an adaptive factor ε, and use the adaptive factor ε to correct the kernel correlation and Kalman tracker position to obtain the final tracking position.

Step S31, calculating the peak sidelobe ratio;

The peak-to-sidelobe ratio is a physical quantity that describes the prominence of the main lobe relative to the side lobe, and can be used to evaluate the matching degree of two target signals in the correlation operation. It is defined as:

Where max(f(z)) is the peak value of the correlation response, μ and σ are the mean and standard deviation of the side lobes except the maximum response peak.

As shown in Figure 3, when the kernel correlation filter tracking algorithm tracks the target normally, its response graph is close to the ideal two-dimensional Gaussian distribution and has an obvious peak. However, in the presence of interference, especially occlusion, as shown in Figure 3 (b), the target features are destroyed, the response graph fluctuates, no longer follows the Gaussian distribution, but has multiple peaks, and the contrast between the sidelobe and the peak decreases, as shown in Figure 3 (a). When the target is not occluded, the response peak obviously reaches the maximum value.

This implementation pre-calculates the PSR average μ _PSR of the first five frames when the target is not occluded, as shown in formula (39), as the peak response standard when the target is not occluded. When the target is completely occluded, the peak response is not 0. After experimental testing, 0.3μ _PSR and 0.8μ _PSR are defined as the low threshold _TL and high threshold _TH of the occlusion detection mechanism, respectively.

Where n is the total number of image frames, and k is the number of image sequences.

Step S32, adaptively updating the kernel correlation filter model in combination with the peak sidelobe ratio;

In the kernel correlation filter tracking process, the target area information will change with the continuous advancement of the frame number. In order to adapt to the target appearance features that change with the change of video frames, this implementation method adaptively updates the appearance model in combination with the peak sidelobe ratio. α ^k , x ^k , α ^k-1 , and x ^k-1 represent the parameters and target models of the kth frame and the k-1th frame, respectively. γ ^k is a learning rate parameter that adaptively changes according to the calculated PSR value in each frame. β is a constant value set to 0.025:

Step S33, further using the number of key points of different occlusion conditions as a criterion for judging the occlusion state;

The specific ISS key point extraction process is as follows, and the result is shown in Figure 4:

Suppose the point cloud target has _Np points, and the coordinates of any point _pi are (x _i , y _i , z _i ).

1) Define a local coordinate system for each point _pi on the point cloud and give each point a search radius _rframe ;

2) Query all points within the radius r _frame around each point p _i in the point cloud data and calculate their weights, that is:

w _ij ＝1/|pi _{- pj} |,|pi _{- pj} |＜r _frame (41)

3) Calculate the covariance matrix of each point p _i :

4) Calculate the eigenvalues of the covariance matrix cov( _pi ) for each point p _i And arrange them in order from largest to smallest/>

5) Set the thresholds ε ₁ and ε ₂ , and the points satisfying equation (43) are regarded as ISS feature points;

6) Repeat the above steps until all points are completed;

Suppose the number of key points extracted just before the point cloud target is occluded is _Niss . Similarly, _0.3Niss and _0.8Niss are defined as the low threshold ( _TL ) and high threshold ( _TH ) of the occlusion detection mechanism, respectively.

The final occlusion detection strategy design is as follows:

Step S34: According to different target occlusion conditions, the present invention proposes an adaptive factor ε, and uses the adaptive factor ε to correct the kernel correlation and Kalman tracker position;

The calculation formula of the adaptive factor ε is as follows:

Where PSR ^k is the peak sidelobe ratio of the kth frame image, N ^k _iss is the number of key points in the kth frame image, μ _PSR is the average PSR of the first five frames before the target is occluded, and N _iss is the number of key points extracted just before the point cloud target is occluded. The adaptive factor ε is used to correct the kernel correlation and Kalman tracker positions to obtain the final tracking position:

In the formula, F _op (x, y, z) is the best target three-dimensional estimated position combined with Kalman filter and kernel correlation filter, F _KCF (x, y) is the target two-dimensional position calculated by kernel correlation filter, F _KM (x, y) is the target two-dimensional position calculated by Kalman filter, F(x, y) is the target two-dimensional coordinate position after weighted fusion, and F _KM (z) is the target third-dimensional position calculated by Kalman filter. It can be seen from the above formula that when the target is in an unobstructed state, ε focuses more on the tracking result of kernel correlation filter; when the target is in a completely obstructed state, ε focuses more on the tracking result of Kalman filter.

In order to verify the effectiveness of the target tracking algorithm proposed in the present invention, the present invention conducts experimental verification on the KITTI dataset, converts the color image in the KITTI dataset into a grayscale image to simulate the intensity image formed by the GM-APD laser radar, and then uses the kernel correlation filter and the Kalman filter to track the image and point cloud data respectively. The center position error (CLE) is used to evaluate the performance of the tracking algorithm. CLE refers to the Euclidean distance between the center position of the real target frame and the center position of the predicted target frame during the tracking process. The smaller the Euclidean distance, the higher the tracking accuracy. The CLE calculation formula is as follows:

Where CLE is the center error value, point p is the center position of the predicted target box, and point r is the center position of the real target box.

KITTI dataset tracking experiment:

The tracking scene is the KITTI dataset tracking 0001 scene, and the tracking target is the No. 86 white car. The motion state sequence of the car from unobstructed to obstructed by other cars and then to unobstructed is selected. When tracking the target, it is necessary to calculate the occlusion threshold each time in order to effectively fuse the intensity image and point cloud tracking position information in combination with the adaptive factor. The occlusion threshold obtained for some motion sequences is shown in Figure 5.

It can be seen from (a) and (b) of Figure 5 that when the target is not obscured, the peak sidelobe ratio PSR = 7.69, and the number of ISS key points is 9; when the target is partially obscured, such as (c) and (d), the peak sidelobe ratio drops to 5.62, and the number of ISS key points is reduced to 6, which meets the occlusion detection interval set by the present invention; when the target is completely obscured, such as (e) and (f), the peak sidelobe ratio is not 0 but drops to 1.77, and the number of ISS key points is 0, which also meets the threshold interval set by the present invention when the target is completely obscured, proving that the occlusion threshold set by the present invention is effective and meets the judgment of the target occlusion state by subsequent tracking tasks.

Figure 6 is a tracking result diagram of the present invention, (a), (c), (e), (g) are intensity image tracking results respectively; (b), (d), (f), (h) are final tracking results respectively. The red box is the real position of the target, and the yellow box is the algorithm tracking position. It can be seen that when the target is in an unobstructed state, the kernel correlation filter based on multi-feature fusion can effectively track the target of interest. When the target is partially obstructed, the tracking result based on the kernel correlation filter based on multi-feature fusion is biased, and the Kalman tracking algorithm can predict the target position with a small error.

As can be seen from Figure 7, (a) is the center position error curve, (b) is the Kalman filter tracking trajectory, (c) is the extended Kalman tracking trajectory, and (d) is the fusion tracking trajectory;

The running time of the extended Kalman filter algorithm is higher than that of the Kalman filter algorithm. By combining the kernel correlation and the Kalman tracker, the tracking accuracy is better than that of the Kalman filter and the extended Kalman filter when the target is in an unobstructed state. When the target is partially or completely obstructed, the adaptive factor is adjusted to maintain a high tracking accuracy. As can be seen from Table 1, the average center position error of the algorithm proposed in the present invention is 21.67% higher than that of the Kalman filter, and 7.94% higher than that of the extended Kalman filter, which proves that the method of the present invention can accurately track targets in different obstruction states.

Table 1

The above analysis experiment verifies the superiority of the algorithm proposed in the present invention through the KITTI dataset, which will be verified by experiments below.

LiDAR real scene tracking experiment:

The GM-APD laser radar detection field of view is fixed to detect and image the crossroad 600 meters away outdoors, and a tracking experiment is carried out on the point cloud data after the intensity image and distance image formed by the GM-APD laser radar are converted. The detection scene is shown in the figure.

It can be seen from Figure 9 that when the target is relatively complete in the intensity image, as shown in (a) and (c) in the figure, the kernel correlation filter tracking algorithm can effectively track the target of interest. As the tracked target gradually moves away from the detection field of view, as shown in (e), (g), and (i) in the figure, the tracking result based on the intensity image gradually becomes invalid. The target position is predicted by the Kalman filter through adaptive factor adjustment. It can be seen from (b), (d), (f), (h), and (j) in the point cloud result diagram that when the target gradually disappears from the detection field of view from complete information, the algorithm of the present invention can effectively track the target. Since the detection data is affected by noise, the average processing time per frame of the algorithm of the present invention is 39ms in this experiment.

The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims (7)

1. A target tracking method based on the fusion of intensity image and point cloud data, characterized in that the method is implemented by the following steps: Step 1: Fuse the HOG features and Fourier descriptor features of the target intensity image formed by the laser radar, and perform kernel correlation filter tracking; Step 2: Convert the target range image formed by the laser radar into three-dimensional point cloud data, and use the Kalman tracker to predict the trajectory of the point cloud target; Step 3: Combine the peak sidelobe ratio PSR and the internal shape descriptor ISS to effectively judge the target occlusion state. According to the different target occlusion conditions, an adaptive factor is proposed. The kernel correlation filter tracking and the Kalman tracker predicted position are corrected using the adaptive factor to obtain the final tracking position. 2. The target tracking method based on the fusion of intensity image and point cloud data according to claim 1 is characterized in that: the specific process of step 1 is: Step 11: Use ridge regression function to train the classifier model of kernel correlation filter tracking algorithm; Step 1 and 2: Expand the training samples through the circulant matrix to obtain the trained kernel correlation filter tracking model; Step 13: Use the kernel correlation filter tracking model to detect and locate the target. 3. The target tracking method based on the fusion of intensity image and point cloud data according to claim 1 is characterized in that: in step 1, the specific process of calculating the HOG feature of the target intensity image is: A. intercepting the tracking area image, graying the image, and normalizing the input image using a gamma correction method; B. Calculate the gradient information of the pixels of the normalized image to obtain the contour information of the target; C. Divide the image into multiple cells, count the gradient information of each cell, divide the gradient direction of each cell into 9 bins, that is, the interval of each bin is 20°, and then map the gradient information to the corresponding bin according to the gradient direction; D. Count the gradient information in the block composed of cells, and finally concatenate the gradient histograms in all blocks to obtain the gradient histogram features of the entire image. 4. The target tracking method based on the fusion of intensity image and point cloud data according to claim 1 is characterized in that: in step 1, the process of calculating the Fourier descriptor feature is: a. Grayscale the intensity image and perform gamma correction; b. Use the Canny edge detection algorithm to extract the contour point set of the target to be tracked; c. Calculate the center coordinates of the contour points, convert the contour points from the rectangular coordinate system to the polar coordinate system, and obtain the center point and contour point distance sequence; d. Perform fast Fourier transform on the center contour point distance sequence to construct the Fourier descriptor. 5. The target tracking method based on the fusion of intensity image and point cloud data according to claim 1 is characterized in that the specific process of step three is: Step 31: Calculate the peak sidelobe ratio; Step 32: adaptively updating the kernel correlation filter tracking model according to the peak sidelobe ratio calculated in step 31; Step 3. Use the ISS detection algorithm to extract the key points of the point cloud target and set the occlusion detection strategy; Step 3 and 4: According to the different occlusion conditions of the target, the adaptive factor ε is used to correct the kernel correlation and Kalman tracker position; The calculation formula of the adaptive factor ε is: Where PSR ^k is the peak sidelobe ratio of the k-th frame image, N ^k _iss is the number of key points in the k-th frame image, μ _PSR is the average PSR of the first five frames before the target is not occluded, and N _iss is the number of key points extracted just before the point cloud target is occluded; The adaptive factor ε is used to correct the positions of the kernel correlation filter tracker and the Kalman tracker to obtain the final tracking position: F(x,y)＝εF _KCF (x,y)+(1-ε)F _KM (x,y) F _op (x, y, z) = F (x, y, F _KM (z)) In the formula, F _op (x, y, z) is the best target three-dimensional estimated position combined with Kalman filter and kernel correlation filter, F _KCF (x, y) is the target two-dimensional position calculated by kernel correlation filter, F _KM (x, y) is the target two-dimensional position calculated by Kalman filter, F(x, y) is the target two-dimensional coordinate position after weighted fusion, and F _KM (z) is the target third-dimensional position calculated by Kalman filter. It can be seen from the above formula that when the target is in an unobstructed state, ε focuses on the tracking result of kernel correlation filter; when the target is in a completely obstructed state, ε focuses on the tracking result of Kalman filter. 6. The target tracking method based on the fusion of intensity image and point cloud data according to claim 5 is characterized in that: in step 32, the kernel phase filter model is adaptively updated in combination with the peak sidelobe ratio, α ^k and x ^k are set to represent the parameters and target model of the kth frame respectively, α ^k-1 and x ^k-1 are set to represent the parameters and target model of the k-1th frame respectively, and γ ^k is set to be a learning rate parameter that adaptively changes according to the calculated PSR value in each frame, then the following formula is obtained: α ^k =(1-γ ^k )α ^k-1 +γ ^k α ^k x ^k =(1-γ ^k )x ^k-1 +γ ^k x ^k In the formula, β is a constant value. 7. The target tracking method based on the fusion of intensity image and point cloud data according to claim 5 is characterized in that: in step 33, the specific process of extracting the key points of the point cloud target using the ISS detection algorithm is: Assume that the point cloud target has N _p points, and the coordinates of any point p _i are (x _i ,y _i ,z _i ); Define a local coordinate system for each point p _i of the point cloud target and give each point a search radius r _frame ; Query all points around each point p _i in _{the frame} of radius r and calculate their weights; Calculate the covariance matrix of each point p _i and the eigenvalues of the covariance matrix of each point p _i And arrange them in order from largest to smallest/> Set thresholds ε ₁ and ε ₂ to satisfy And/> The points are regarded as ISS feature points until all key points are extracted; Assume that the number of key points extracted before the point cloud target is occluded is _Niss , and define _0.3Niss and _0.8Niss as the low threshold _TL and high threshold _TH of the occlusion detection mechanism respectively; The final occlusion detection strategy is as follows: