CN105844667A - Structural target tracking method of compact color coding - Google Patents
Structural target tracking method of compact color coding Download PDFInfo
- Publication number
- CN105844667A CN105844667A CN201610178939.6A CN201610178939A CN105844667A CN 105844667 A CN105844667 A CN 105844667A CN 201610178939 A CN201610178939 A CN 201610178939A CN 105844667 A CN105844667 A CN 105844667A
- Authority
- CN
- China
- Prior art keywords
- target
- frame
- compact
- color
- tracking
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Image Analysis (AREA)
Abstract
Description
技术领域technical field
本发明涉及一种结构化目标跟踪方法，特别是一种紧凑颜色编码的结构化目标跟踪方法。The invention relates to a structured target tracking method, in particular to a compact color-coded structured target tracking method.
背景技术Background technique
目标跟踪的任务是在视频序列的每一帧图像中找到被跟踪目标所处的位置并标记出来，在军事制导、视频监控、医疗诊断、产品检测、虚拟现实等众多领域均有重要的应用和发展前景。由于目标自身变化的多样性(如：目标尺寸变化、姿态旋转、非刚体目标的形变等)和外部环境的复杂性(如：光照、背景扰动、相似物体干扰、目标之间相互遮挡、背景对目标的部分遮挡甚至完全遮挡等)，鲁棒、实时的目标跟踪一直是一个极富挑战性的问题。目前针对视觉跟踪的研究在某些条件下取得了良好的跟踪效果，但对于实际场景下长视频视觉跟踪中出现的严重遮挡、非刚体目标形变等问题，目标跟踪性能仍有待提高。The task of target tracking is to find and mark the position of the tracked target in each frame of the video sequence. It has important applications and Prospects. Due to the variety of changes in the target itself (such as: target size change, attitude rotation, deformation of non-rigid objects, etc.) and the complexity of the external environment (such as: lighting, background disturbance, similar object interference, mutual occlusion between Partial occlusion or even complete occlusion of the target, etc.), robust and real-time target tracking has always been a very challenging problem. The current research on visual tracking has achieved good tracking results under certain conditions, but the target tracking performance still needs to be improved for problems such as severe occlusion and non-rigid object deformation in long video visual tracking in actual scenes.
文献“Adaptive Color Attributes for Real-Time Visual Tracking,Martin Danelljan et al.,2014IEEE Conference on Computer Vision and Pattern Recognition(CVPR)[C],2014:1090–1097”公开了一种自适应颜色特征的实时目标跟踪方法。该方法提取目标与背景区域的颜色特征，使用基于核函数的相关滤波器来区分它们，进而实现视频序列的目标跟踪，取得较好的跟踪效果。但文献所述方法只简单使用了目标像素的颜色信息，并没有考虑目标的形状特征，使得特征的描述能力较弱；另外构造的基于相关滤波器的最小二乘分类器只能进行二分类，不能精确反映目标的遮挡信息；以上问题在复杂的实际跟踪场景中会造成视觉跟踪漂移，导致跟踪性能下降。The document "Adaptive Color Attributes for Real-Time Visual Tracking, Martin Danelljan et al., 2014IEEE Conference on Computer Vision and Pattern Recognition (CVPR) [C], 2014:1090–1097" discloses a real-time target with adaptive color features tracking method. This method extracts the color features of the target and the background area, uses a kernel function-based correlation filter to distinguish them, and then realizes the target tracking of the video sequence, and obtains a better tracking effect. However, the method described in the literature only simply uses the color information of the target pixel, and does not consider the shape feature of the target, which makes the description ability of the feature weak; in addition, the least squares classifier based on the correlation filter can only perform binary classification. It cannot accurately reflect the occlusion information of the target; the above problems will cause visual tracking drift in complex actual tracking scenarios, resulting in a decrease in tracking performance.
发明内容Contents of the invention
本发明的目的是要提供一种紧凑颜色编码的结构化目标跟踪方法，解决在复杂的实际跟踪场景中会造成视觉跟踪漂移，导致跟踪性能下降的问题。The purpose of the present invention is to provide a compact color-coded structured object tracking method, which can solve the problem of visual tracking drift and decrease in tracking performance in complex actual tracking scenes.
本发明的目的是这样实现的：该结构化目标跟踪方法，通过融合候选目标区域的形状与颜色特征，并使用哈希函数降低这种组合特征的维度，形成低维的紧凑颜色编码特征，进而采用结构化分类函数进行目标分类与预测，实现目标跟踪；The purpose of the present invention is achieved in this way: the structured target tracking method, by fusing the shape and color features of the candidate target area, and using a hash function to reduce the dimension of this combined feature, forms a low-dimensional compact color-coded feature, and then Use structured classification functions to classify and predict targets to achieve target tracking;
包括下述步骤：Include the following steps:
步骤一：初始化目标位置及结构化分类函数；Step 1: Initialize the target position and structured classification function;
在视频序列的初始帧，手动的给定需要跟踪目标的边界框B1＝(c1,r1,w1,h1)，B1用来表示目标的位置，其中c1,r1分别表示左上角的列和行坐标，w1,h1表示宽度和高度，边界框Bt表示第t帧时目标的位置，目标位移的偏移量表示为由于视频序列中第1帧的目标位置上的边界框已经手动给定，跟踪方法需要从第2帧开始跟踪目标，估计出准确的标识目标位置的边界框；第t帧中目标的边界框Bt可由以下方式得到：In the initial frame of the video sequence, manually give the bounding box B 1 = (c 1 , r 1 , w 1 , h 1 ) of the target to be tracked, and B 1 is used to represent the position of the target, where c 1 , r 1 are respectively Represents the column and row coordinates of the upper left corner, w 1 , h 1 represent the width and height, the bounding box B t represents the position of the target at frame t, and the offset of the target displacement is expressed as Since the bounding box on the target position in the first frame of the video sequence has been manually given, the tracking method needs to start tracking the target from the second frame, and estimate the accurate bounding box that identifies the target position; the bounding box B of the target in the tth frame t can be obtained by:
公式(1)中的f(xt,y)＝<w,Ψ(xt,y)>是结构化分类函数，其中xt表示视频序列中的第t帧图像，Ψ(xt,y)是一个k维的向量，表示候选目标区域的紧凑颜色编码特征，将由步骤三构建；参数w是一个k维的向量，使用k个介于0和1之间的随机实数对它初始化，它将在步骤四通过对每一帧样本的学习进行在线更新。f(x t ,y)=<w,Ψ(x t ,y)> in formula (1) is a structured classification function, where x t represents the tth frame image in the video sequence, Ψ(x t ,y ) is a k-dimensional vector, representing the compact color-coded features of the candidate target area, which will be constructed by step 3; the parameter w is a k-dimensional vector, initialized with k random real numbers between 0 and 1, it It will be updated online through the learning of each frame sample in step 4.
步骤二：生成目标训练样本，并提取样本的形状与颜色特征；Step 2: Generate target training samples, and extract the shape and color features of the samples;
(2-1)、使用密集采样方法在真实目标边界框附近采样，产生M个目标位置偏移量，并截取相应的图像区域作为训练样本，提取这些样本的形状与颜色特征；(2-1), use the dense sampling method to sample near the real target bounding box, generate M target position offsets, and intercept the corresponding image areas as training samples, and extract the shape and color features of these samples;
(2-2)、提取每个训练样本的形状特征，使用Haar-like特征来描述目标的形状信息；(2-2), extract the shape feature of each training sample, and use the Haar-like feature to describe the shape information of the target;
(2-3)、提取每个训练样本的颜色信息，与形状特征融合为一个新的特征向量。(2-3), extract the color information of each training sample, and fuse it with the shape feature into a new feature vector.
步骤三：构建紧凑编码特征；Step 3: Construct compact encoding features;
使用局部敏感哈希对步骤二得到的高维特征进行映射，生成紧凑颜色编码特征向量Ψ(xt,y)；Use locality-sensitive hashing to map the high-dimensional features obtained in step 2 to generate a compact color-coded feature vector Ψ(x t ,y);
当步骤二得到的特征向量的维度为d，即每个样本的特征都是一个d维向量，若m＝100，定义由m个哈希函数ha(·)组成的哈希函数族高维的d维向量映射为m(m＜＜d)维的紧凑二进制编码特征；更具体地，从d维高斯分布中生成一个随机向量作为超平面，则哈希函数hav(·)定义为：When the dimension of the feature vector obtained in step 2 is d, that is, the feature of each sample is a d-dimensional vector, if m=100, define a hash function family composed of m hash functions ha(·) High-dimensional d-dimensional vectors are mapped to m (m<<d)-dimensional compact binary encoded features; more specifically, from a d-dimensional Gaussian distribution Generate a random vector in As a hyperplane, the hash function ha v ( ) is defined as:
其中是单个样本通过权利要求1所述步骤二得到的特征向量，通过上述方法构建m个哈希函数，并将q依次代入这些哈希函数，得到m维的二进制编码串，即为构建的紧凑颜色编码特征向量，上述m个哈希函数仅在视频序列第1帧生成，在后继帧中将继续使用。in is the eigenvector obtained by a single sample through step 2 of claim 1, constructing m hash functions by the above method, and substituting q into these hash functions in turn to obtain an m-dimensional binary coded string, which is the compact color constructed Encoding feature vectors, the above m hash functions are only generated in the first frame of the video sequence, and will continue to be used in subsequent frames.
步骤四：学习与更新结构化分类函数；Step 4: Learning and updating the structured classification function;
使用步骤三中生成的样本的紧凑颜色编码特征更新公式(1)中结构化分类函数f(xt,y)的参数w，以便使用更新后的w在新的视频帧中估计最优目标位置；Update the parameter w of the structured classification function f(x t , y) in Equation (1) using the compact color-coded features of the samples generated in step 3, so that the optimal object position can be estimated in the new video frame using the updated w ;
目标跟踪是一个在线更新的过程，用于处理动态的数据流，目标跟踪方法需要从训练样本中学习并更新参数以适应新的数据；Target tracking is an online update process for processing dynamic data streams. The target tracking method needs to learn from training samples and update parameters to adapt to new data;
将用紧凑颜色编码特征表示的M个样本代入公式(3)，通过优化公式(3)得到新的参数w：Substitute M samples represented by compact color-coded features into formula (3), and obtain a new parameter w by optimizing formula (3):
其中λ为正则化系数，λ＝0.1，ξt为松弛变量，标记代价Δ用来度量边界框的覆盖率：Where λ is the regularization coefficient, λ=0.1, ξ t is the slack variable, and the marking cost Δ is used to measure the coverage of the bounding box:
使用次梯度下降法对上式进行迭代优化，最终确定新的参数w的值。Use the subgradient descent method to iteratively optimize the above formula, and finally determine the value of the new parameter w.
步骤五：生成候选目标区域，使用结构化分类函数估计最优目标区域，确定目标位置；当第t+1帧图像到来时，跟踪方法需要在上一帧目标出现位置附近采样，使用已更新过参数wt+1结构化分类函数估计出这些样本中分类分数最高的，这个样本指示的区域即为最优目标位置；得到新的目标位置之后，再转向步骤二继续执行，直至视频序列结束；Step 5: Generate candidate target areas, use the structured classification function to estimate the optimal target area, and determine the target position; when the t+1th frame image arrives, the tracking method needs to sample near the position where the target appeared in the previous frame, using the updated The parameter w t+1 structured classification function estimates the highest classification score among these samples, and the area indicated by this sample is the optimal target position; after obtaining the new target position, turn to step 2 and continue until the end of the video sequence;
使用如下方案：Use the following scheme:
(5-1)当上一帧计算的目标边界框为Bt，在以(0,0)为圆心、S为半径(S＝60)的圆内采样P个偏移由Bt与采样的P个偏移y相加(Bt+y)得到P个候选目标边界框，利用它们在当前第t+1帧图像xt+1上截取出相应的P个图像区域作为候选目标区域；(5-1) When the target bounding box calculated in the previous frame is B t , sample P offsets in a circle with (0,0) as the center and S as the radius (S=60) Add P t and sampled P offsets y (B t +y) to get P candidate target bounding boxes, and use them to intercept corresponding P image regions on the current t+1th frame image x t+1 as a candidate target area;
(5-2)所述步骤二、步骤三描述的特征生成方法，计算P个候选目标区域的紧凑颜色编码特征向量Ψ(xt,y)，根据上一帧目标边界框Bt与公式(1)计算的最优偏移得出当前帧目标的位置 (5-2) The feature generation method described in step 2 and step 3 calculates the compact color-coded feature vector Ψ(x t , y) of P candidate target regions, according to the target bounding box B t of the previous frame and the formula ( 1) Calculate the optimal offset to get the position of the target in the current frame
有益效果，由于采用了上述方案，通过融合候选目标区域的形状与颜色特征，并使用哈希函数降低这种组合特征的维度，形成低维的紧凑颜色编码特征，进而采用结构化分类函数进行目标分类与预测，实现了目标跟踪。本方法增强了目标的特征描述能力，同时结构化分类提高了目标分类的准确度，可有效避免视觉跟踪漂移，提高跟踪性能。Beneficial effects, due to the adoption of the above scheme, by fusing the shape and color features of the candidate target area, and using the hash function to reduce the dimension of this combined feature, a low-dimensional compact color-coded feature is formed, and then the structured classification function is used for target Classification and prediction, achieving target tracking. This method enhances the feature description ability of the target, and at the same time, the structured classification improves the accuracy of the target classification, which can effectively avoid the drift of visual tracking and improve the tracking performance.
由于提取了包含形状与颜色信息的目标区域特征，增强了目标外观的描述能力，从根本上提高了目标跟踪的鲁棒性；使用哈希函数对高维的形状与颜色信息进行紧凑二进制编码，有效的降低了跟踪算法的计算复杂度；使用结构化分类函数提高了目标分类的精确度，增强了对非刚体目标形变、目标遮挡的处理能力，提升了目标跟踪的性能。Due to the extraction of target area features containing shape and color information, the ability to describe the appearance of the target is enhanced, and the robustness of target tracking is fundamentally improved; the hash function is used to perform compact binary encoding of high-dimensional shape and color information, It effectively reduces the computational complexity of the tracking algorithm; the use of structured classification functions improves the accuracy of target classification, enhances the processing capabilities of non-rigid target deformation and target occlusion, and improves the performance of target tracking.
附图说明：Description of drawings:
图1为本发明提供的紧凑颜色编码的结构化目标跟踪方法的基本流程图。Fig. 1 is a basic flowchart of the compact color-coded structured object tracking method provided by the present invention.
具体实施方式detailed description
该结构化目标跟踪方法，通过融合候选目标区域的形状与颜色特征，并使用哈希函数降低这种组合特征的维度，形成低维的紧凑颜色编码特征，进而采用结构化分类函数进行目标分类与预测，实现目标跟踪；This structured target tracking method combines the shape and color features of the candidate target area, and uses the hash function to reduce the dimension of the combined feature to form a low-dimensional compact color-coded feature, and then uses the structured classification function for target classification and classification. Forecast, achieve target tracking;
包括下述步骤：Include the following steps:
步骤一：初始化目标位置及结构化分类函数；Step 1: Initialize the target position and structured classification function;
在视频序列的初始帧，手动的给定需要跟踪目标的边界框B1＝(c1,r1,w1,h1)，B1用来表示目标的位置，其中c1,r1分别表示左上角的列和行坐标，w1,h1表示宽度和高度，边界框Bt表示第t帧时目标的位置，目标位移的偏移量表示为由于视频序列中第1帧的目标位置上的边界框已经手动给定，跟踪方法需要从第2帧开始跟踪目标，估计出准确的标识目标位置的边界框；第t帧中目标的边界框Bt可由以下方式得到：In the initial frame of the video sequence, manually give the bounding box B 1 = (c 1 , r 1 , w 1 , h 1 ) of the target to be tracked, and B 1 is used to represent the position of the target, where c 1 , r 1 are respectively Represents the column and row coordinates of the upper left corner, w 1 , h 1 represent the width and height, the bounding box B t represents the position of the target at frame t, and the offset of the target displacement is expressed as Since the bounding box on the target position in the first frame of the video sequence has been manually given, the tracking method needs to start tracking the target from the second frame, and estimate the accurate bounding box that identifies the target position; the bounding box B of the target in the tth frame t can be obtained by:
公式(1)中的f(xt,y)＝<w,Ψ(xt,y)>是结构化分类函数，其中xt表示视频序列中的第t帧图像，Ψ(xt,y)是一个k维的向量，表示候选目标区域的紧凑颜色编码特征，将由步骤三构建；参数w是一个k维的向量，使用k个介于0和1之间的随机实数对它初始化，它将在步骤四通过对每一帧样本的学习进行在线更新。f(x t ,y)=<w,Ψ(x t ,y)> in formula (1) is a structured classification function, where x t represents the tth frame image in the video sequence, Ψ(x t ,y ) is a k-dimensional vector, representing the compact color-coded features of the candidate target area, which will be constructed by step 3; the parameter w is a k-dimensional vector, initialized with k random real numbers between 0 and 1, it It will be updated online through the learning of each frame sample in step 4.
步骤二：生成目标训练样本，并提取样本的形状与颜色特征；Step 2: Generate target training samples, and extract the shape and color features of the samples;
(2-1)、使用密集采样方法在真实目标边界框附近采样，产生M个目标位置偏移量，并截取相应的图像区域作为训练样本，提取这些样本的形状与颜色特征；(2-1), use the dense sampling method to sample near the real target bounding box, generate M target position offsets, and intercept the corresponding image areas as training samples, and extract the shape and color features of these samples;
(2-2)、提取每个训练样本的形状特征，使用Haar-like特征来描述目标的形状信息；(2-2), extract the shape feature of each training sample, and use the Haar-like feature to describe the shape information of the target;
(2-3)、提取每个训练样本的颜色信息，与形状特征融合为一个新的特征向量。(2-3), extract the color information of each training sample, and fuse it with the shape feature into a new feature vector.
步骤三：构建紧凑编码特征；Step 3: Construct compact encoding features;
使用局部敏感哈希对步骤二得到的高维特征进行映射，生成紧凑颜色编码特征向量Ψ(xt,y)；Use locality-sensitive hashing to map the high-dimensional features obtained in step 2 to generate a compact color-coded feature vector Ψ(x t ,y);
当步骤二得到的特征向量的维度为d，即每个样本的特征都是一个d维向量，若m＝100，定义由m个哈希函数ha(·)组成的哈希函数族高维的d维向量映射为m(m＜＜d)维的紧凑二进制编码特征；更具体地，从d维高斯分布中生成一个随机向量作为超平面，则哈希函数hav(·)定义为：When the dimension of the feature vector obtained in step 2 is d, that is, the feature of each sample is a d-dimensional vector, if m=100, define a hash function family composed of m hash functions ha(·) High-dimensional d-dimensional vectors are mapped to m (m<<d)-dimensional compact binary encoded features; more specifically, from a d-dimensional Gaussian distribution Generate a random vector in As a hyperplane, the hash function ha v ( ) is defined as:
其中是单个样本通过权利要求1所述步骤二得到的特征向量，通过上述方法构建m个哈希函数，并将q依次代入这些哈希函数，得到m维的二进制编码串，即为构建的紧凑颜色编码特征向量，上述m个哈希函数仅在视频序列第1帧生成，在后继帧中将继续使用。in is the eigenvector obtained by a single sample through step 2 of claim 1, constructing m hash functions by the above method, and substituting q into these hash functions in turn to obtain an m-dimensional binary coded string, which is the compact color constructed Encoding feature vectors, the above m hash functions are only generated in the first frame of the video sequence, and will continue to be used in subsequent frames.
步骤四：学习与更新结构化分类函数；Step 4: Learning and updating the structured classification function;
使用步骤三中生成的样本的紧凑颜色编码特征更新公式(1)中结构化分类函数f(xt,y)的参数w，以便使用更新后的w在新的视频帧中估计最优目标位置；Update the parameter w of the structured classification function f(x t , y) in Equation (1) using the compact color-coded features of the samples generated in step 3, so that the optimal object position can be estimated in the new video frame using the updated w ;
目标跟踪是一个在线更新的过程，用于处理动态的数据流，目标跟踪方法需要从训练样本中学习并更新参数以适应新的数据；Target tracking is an online update process for processing dynamic data streams. The target tracking method needs to learn from training samples and update parameters to adapt to new data;
将用紧凑颜色编码特征表示的M个样本代入公式(3)，通过优化公式(3)得到新的参数w：Substitute M samples represented by compact color-coded features into formula (3), and obtain a new parameter w by optimizing formula (3):
其中λ为正则化系数，λ＝0.1，ξt为松弛变量，标记代价Δ用来度量边界框的覆盖率：Where λ is the regularization coefficient, λ=0.1, ξ t is the slack variable, and the marking cost Δ is used to measure the coverage of the bounding box:
使用次梯度下降法对上式进行迭代优化，最终确定新的参数w的值。Use the subgradient descent method to iteratively optimize the above formula, and finally determine the value of the new parameter w.
步骤五：生成候选目标区域，使用结构化分类函数估计最优目标区域，确定目标位置；当第t+1帧图像到来时，跟踪方法需要在上一帧目标出现位置附近采样，使用已更新过参数wt+1结构化分类函数估计出这些样本中分类分数最高的，这个样本指示的区域即为最优目标位置；得到新的目标位置之后，再转向步骤二继续执行，直至视频序列结束；Step 5: Generate candidate target areas, use the structured classification function to estimate the optimal target area, and determine the target position; when the t+1th frame image arrives, the tracking method needs to sample near the position where the target appeared in the previous frame, using the updated The parameter w t+1 structured classification function estimates the highest classification score among these samples, and the area indicated by this sample is the optimal target position; after obtaining the new target position, turn to step 2 and continue until the end of the video sequence;
使用如下方案：Use the following scheme:
(5-1)当上一帧计算的目标边界框为Bt，在以(0,0)为圆心、S为半径(S＝60)的圆内采样P个偏移由Bt与采样的P个偏移y相加(Bt+y)得到P个候选目标边界框，利用它们在当前第t+1帧图像xt+1上截取出相应的P个图像区域作为候选目标区域；(5-1) When the target bounding box calculated in the previous frame is B t , sample P offsets in a circle with (0,0) as the center and S as the radius (S=60) Add P t and sampled P offsets y (B t +y) to get P candidate target bounding boxes, and use them to intercept corresponding P image regions on the current t+1th frame image x t+1 as a candidate target area;
(5-2)所述步骤二、步骤三描述的特征生成方法，计算P个候选目标区域的紧凑颜色编码特征向量Ψ(xt,y)，根据上一帧目标边界框Bt与公式(1)计算的最优偏移得出当前帧目标的位置 (5-2) The feature generation method described in step 2 and step 3 calculates the compact color-coded feature vector Ψ(x t , y) of P candidate target regions, according to the target bounding box B t of the previous frame and the formula ( 1) Calculate the optimal offset to get the position of the target in the current frame
为使本发明的目的、技术方案和优点更加清楚，下面将结合附图对本发明实施方式作进一步地详细描述。In order to make the object, technical solution and advantages of the present invention clearer, the implementation manner of the present invention will be further described in detail below in conjunction with the accompanying drawings.
实施例1：本发明的方法具体运行的硬件和编程语言并不限制，用任何语言编写都可以完成。图1为本发明所提供的方法的流程图，具体步骤如下：Embodiment 1: The specific operating hardware and programming language of the method of the present invention are not limited, and can be completed in any language. Fig. 1 is the flowchart of the method provided by the present invention, and concrete steps are as follows:
步骤一：初始化目标位置及结构化分类函数。Step 1: Initialize the target position and the structured classification function.
在视频序列的初始帧，手动的给定需要跟踪目标的边界框B1＝(c1,r1,w1,h1)，B1用来表示目标的位置，其中c1,r1分别表示左上角的列和行坐标，w1,h1表示宽度和高度，边界框Bt表示第t帧时目标的位置，目标位移的偏移量表示为由于视频序列中第1帧的目标位置上的边界框已经手动给定，跟踪方法需要从第2帧开始跟踪目标，估计出准确的标识目标位置的边界框。第t帧中目标的边界框Bt可由以下方式得到：In the initial frame of the video sequence, manually give the bounding box B 1 = (c 1 , r 1 , w 1 , h 1 ) of the target to be tracked, and B 1 is used to represent the position of the target, where c 1 , r 1 are respectively Represents the column and row coordinates of the upper left corner, w 1 , h 1 represent the width and height, the bounding box B t represents the position of the target at frame t, and the offset of the target displacement is expressed as Since the bounding box on the target position in the first frame of the video sequence has been manually given, the tracking method needs to track the target from the second frame and estimate the accurate bounding box that identifies the target position. The bounding box B t of the target in the tth frame can be obtained by:
公式(1)中的f(xt,y)＝<w,Ψ(xt,y)>是结构化分类函数，其中xt表示视频序列中的第t帧图像，Ψ(xt,y)是一个k维的向量，表示候选目标区域的紧凑颜色编码特征，将由步骤三构建；参数w是一个k维的向量，本发明使用k个介于0和1之间的随机实数对它初始化，它将在步骤四通过对每一帧样本的学习进行在线更新。f(x t ,y)=<w,Ψ(x t ,y)> in formula (1) is a structured classification function, where x t represents the tth frame image in the video sequence, Ψ(x t ,y ) is a k-dimensional vector, representing the compact color-coded features of the candidate target area, which will be constructed by step three; the parameter w is a k-dimensional vector, and the present invention uses k random real numbers between 0 and 1 to initialize it , it will be updated online in step 4 by learning each frame sample.
步骤二：生成目标训练样本，并提取样本的形状与颜色特征。Step 2: Generate target training samples, and extract the shape and color features of the samples.
2-1、使用密集采样方法在真实目标边界框附近采样，产生M个目标位置偏移量，并截取相应的图像区域作为训练样本，提取这些样本的形状与颜色特征。2-1. Use the dense sampling method to sample near the real target bounding box, generate M target position offsets, and intercept the corresponding image areas as training samples, and extract the shape and color features of these samples.
密集采样方法如下：当前第t帧的真实目标边界框为Bt，因此真实的目标偏移yt＝(0,0,Δwt,Δht)，在本发明中，固定目标大小，设置Δwt＝0,Δht＝0。以当前目标偏移为圆心、s为半径(本实例中s＝30)的圆内采样M个偏移根据公式(1)的定义，可由Bt与采样的M个偏移y相加得到M个目标边界框，利用它们在当前第t帧图像xt上截取出相应的M个图像区域作为训练样本。The dense sampling method is as follows: the real target bounding box of the current t-th frame is B t , so the real target offset y t = (0,0,Δw t ,Δh t ), in the present invention, the target size is fixed, and Δw is set t =0, Δh t =0. Sample M offsets in a circle with the current target offset as the center and s as the radius (s=30 in this example) According to the definition of formula (1), M target bounding boxes can be obtained by adding B t and the sampled M offsets y, and use them to intercept the corresponding M image regions on the current t-th frame image x t as training samples .
2-2、提取每个训练样本的形状特征，本发明使用Haar-like特征来描述目标的形状信息。Haar-like特征是一种目标跟踪领域常用的特征描述算子，本发明使用三种基本类型的特征，分为两矩形特征、三矩形特征、对角特征。分别将三类矩阵图像区域内一类矩形部分的所有像素值之和减去另一类矩形部分所有像素值之和，得到的就是单个特征值，本发明中使用积分图加速计算这个特征值。把所有这三类特征的特征值串联组成一个向量，即为该图像区域的Haar-like特征。2-2. Extract the shape feature of each training sample. The present invention uses Haar-like features to describe the shape information of the target. The Haar-like feature is a commonly used feature description operator in the field of target tracking. The present invention uses three basic types of features, which are divided into two-rectangle features, three-rectangle features, and diagonal features. The sum of all pixel values of one type of rectangular part in the three types of matrix image areas is subtracted from the sum of all pixel values of another type of rectangular part to obtain a single eigenvalue. In the present invention, the integral map is used to accelerate the calculation of this eigenvalue. The eigenvalues of all these three types of features are concatenated to form a vector, which is the Haar-like feature of the image area.
2-3、提取每个训练样本的颜色信息，与形状特征融合为一个新的特征向量。提取颜色信息的方法如下所述：把颜色分为11类(黑、蓝、棕、灰、绿、橙、粉红、紫、红、白、黄)，针对上一步中Haar-like特征的三种矩形，统计每个矩形内所有像素的RGB值映射为上述11类颜色的概率，把11个概率值形成颜色向量，放在已求得的Haar-like特征之后，即形成了包含形状与颜色信息的新特征。颜色向量CN，以及矩形I内所有像素的RGB值映射为11类颜色的概率p(cni|I)由公式(2)求得：2-3. Extract the color information of each training sample, and fuse it with the shape feature into a new feature vector. The method of extracting color information is as follows: Divide colors into 11 categories (black, blue, brown, gray, green, orange, pink, purple, red, white, yellow), for the three types of Haar-like features in the previous step Rectangle, count the probability that the RGB values of all pixels in each rectangle are mapped to the above 11 types of colors, form the 11 probability values into a color vector, and place it after the Haar-like feature that has been obtained, which forms a shape and color information. new features of . The color vector CN, and the probability p(cn i |I) that the RGB values of all pixels in the rectangle I are mapped to 11 types of colors are obtained by formula (2):
其中cni是11类颜色中的第i类颜色，c是矩形I内像素的坐标，N是矩形I内的像素总数，g(c)是像素c的Lab颜色空间值，由通用颜色名映射关系可得p(cni|g(c))。Where cn i is the i-th color among the 11 colors, c is the coordinate of the pixel in the rectangle I, N is the total number of pixels in the rectangle I, g(c) is the Lab color space value of the pixel c, mapped by the general color name The relationship can be p(cn i |g(c)).
步骤三：构建紧凑编码特征。步骤二提取的包含样本形状与颜色信息的特征向量具有很高的维度，直接使用这种特征会增加目标跟踪的计算复杂度，不利于实时跟踪。本发明使用局部敏感哈希对步骤二得到的高维特征进行映射，生成紧凑颜色编码特征向量Ψ(xt,y)。Step 3: Build compact encoding features. The feature vector extracted in step 2 containing sample shape and color information has a very high dimensionality, directly using this feature will increase the computational complexity of target tracking, which is not conducive to real-time tracking. The present invention uses local sensitive hashing to map the high-dimensional features obtained in step 2 to generate a compact color-coded feature vector Ψ(x t , y).
具体方法如下：设步骤二得到的特征向量的维度为d，即每个样本的特征都是一个d维向量。为了把高维的d维向量映射为m(m＜＜d)维的紧凑二进制编码特征，本实例中m＝100，定义由m个哈希函数ha(·)组成的哈希函数族更具体地，从d维高斯分布中生成一个随机向量作为超平面，则哈希函数hav(·)定义为：The specific method is as follows: Let the dimension of the feature vector obtained in step 2 be d, that is, the feature of each sample is a d-dimensional vector. In order to map high-dimensional d-dimensional vectors into m (m<<d)-dimensional compact binary coded features, m=100 in this example, define a hash function family composed of m hash functions ha( ) More specifically, from the d-dimensional Gaussian distribution Generate a random vector in As a hyperplane, the hash function ha v ( ) is defined as:
其中是单个样本通过步骤二得到的特征向量。通过上述方法构建m个哈希函数，并将q依次代入这些哈希函数，可得到m维的二进制编码串，即为构建的紧凑编码特征向量，注意上述m个哈希函数仅在视频序列第1帧生成，在后继帧中将继续使用。in is the feature vector obtained by a single sample through step 2. Construct m hash functions by the above method, and substituting q into these hash functions in turn, the m-dimensional binary coded string can be obtained, which is the constructed compact coded feature vector. Note that the above m hash functions are only in the first 1 frame is generated and will continue to be used in subsequent frames.
步骤四：学习与更新结构化分类函数。目标跟踪是一个在线更新的过程，用于处理动态的数据流，目标跟踪方法需要从训练样本中学习并更新参数以适应新的数据。本步骤使用步骤三中生成的样本的紧凑颜色编码特征更新公式(1)中结构化分类函数f(xt,y)的参数w，以便使用更新后的w在新的视频帧中估计最优目标位置。Step 4: Learn and update the structured classification function. Object tracking is an online update process for dealing with dynamic data streams. Object tracking methods need to learn from training samples and update parameters to adapt to new data. This step uses the compact color-coded features of the samples generated in step 3 to update the parameter w of the structured classification function f(x t ,y) in Eq. (1), so that the updated w can be used to estimate the optimal target location.
下面详细描述更新参数w的方法。将使用紧凑编码特征表示的M个样本代入公式(4)，通过优化公式(4)得到新的参数w：The method of updating the parameter w is described in detail below. Substitute M samples represented by compact coding features into formula (4), and obtain a new parameter w by optimizing formula (4):
其中λ为正则化系数，本实例中λ＝0.1，ξt为松弛变量，标记代价Δ用来度量边界框的覆盖率，定义为：Where λ is the regularization coefficient. In this example, λ=0.1, ξ t is the slack variable, and the marking cost Δ is used to measure the coverage of the bounding box, which is defined as:
使用次梯度下降法对公式(4)进行迭代优化，最终确定新的参数w的值。假设当前帧为第t帧，公式(4)关于参数wt的次梯度为：Use subgradient descent method to iteratively optimize formula (4), and finally determine the value of the new parameter w. Assuming that the current frame is the tth frame, the subgradient of the formula (4) with respect to the parameter w t is:
其中δΨt＝Ψ(xt,y)-Ψ(xt,yt)，是一个指示函数，如果条件满足返回1，否则返回0。这样，第t+1帧的结构化分类函数参数ηt＝1/(λt)是更新的步长，上式可写为：Where δΨ t = Ψ(x t ,y)-Ψ(x t ,y t ), is an indicator function that returns 1 if the condition is met, and 0 otherwise. In this way, the structured classification function parameters of frame t+1 η t =1/(λt) is the update step size, the above formula can be written as:
将步骤三计算的M个样本的特征向量Ψ(xt,y)分别代入公式(7)，重新计算后的wt+1即为更新之后的结构化分类函数参数。Substitute the eigenvectors Ψ(x t , y) of the M samples calculated in step 3 into formula (7), and the recalculated w t+1 is the updated structured classification function parameter.
步骤五：生成候选目标区域，使用结构化分类函数估计最优目标区域，确定目标位置。当第t+1帧图像到来时，跟踪方法需要在上一帧目标出现位置附近采样，使用已更新过参数wt+1结构化分类类函数估计出这些样本中分类分数最高的，这个样本指示的区域即为最优目标位置。得到新的目标位置之后，再转向步骤二继续执行，直至视频序列结束。具体方法如下所述：Step 5: Generate candidate target areas, use the structured classification function to estimate the optimal target area, and determine the target location. When the t+1th frame image arrives, the tracking method needs to sample near the position where the target appeared in the previous frame, and use the updated parameter w t+1 structured classification class function to estimate the highest classification score among these samples. This sample indicates The area is the optimal target location. After obtaining the new target position, turn to step 2 and continue until the end of the video sequence. The specific method is as follows:
5-1、假设上一帧计算的目标边界框为Bt，在以(0,0)为圆心、S为半径(本实例中S＝60)的圆内采样P个偏移可由Bt与采样的P个偏移y相加(Bt+y)得到P个候选目标边界框，利用它们在当前第t+1帧图像xt+1上截取出相应的P个图像区域作为候选目标区域。5-1. Assuming that the target bounding box calculated in the previous frame is B t , sample P offsets in a circle with (0,0) as the center and S as the radius (S=60 in this example) P candidate target bounding boxes can be obtained by adding B t and sampled P offsets y (B t +y), and use them to intercept corresponding P image regions on the current t+1th frame image x t+1 as a candidate target area.
5-2、使用步骤二、三描述的特征生成方法，计算P个候选目标区域的紧凑颜色编码特征向量Ψ(xt,y)，利用公式(8)计算最优偏移 5-2. Using the feature generation method described in steps 2 and 3, calculate the compact color-coded feature vector Ψ(x t , y) of P candidate target regions, and use formula (8) to calculate the optimal offset
可根据上一帧目标边界框Bt与公式(8)计算的最优偏移得出当前帧目标的位置 The position of the target in the current frame can be obtained according to the optimal offset calculated by the target bounding box B t in the previous frame and formula (8)
Claims (6)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610178939.6A CN105844667A (en) | 2016-03-25 | 2016-03-25 | Structural target tracking method of compact color coding |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610178939.6A CN105844667A (en) | 2016-03-25 | 2016-03-25 | Structural target tracking method of compact color coding |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105844667A true CN105844667A (en) | 2016-08-10 |
Family
ID=56583845
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610178939.6A Pending CN105844667A (en) | 2016-03-25 | 2016-03-25 | Structural target tracking method of compact color coding |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105844667A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106530327A (en) * | 2016-11-07 | 2017-03-22 | 北京航空航天大学 | Quick real-time discrimination type tracing method based on multi-local-feature learning |
| CN108280808A (en) * | 2017-12-15 | 2018-07-13 | 西安电子科技大学 | The method for tracking target of correlation filter is exported based on structuring |
| CN108596949A (en) * | 2018-03-23 | 2018-09-28 | 云南大学 | Video frequency object tracking state analysis method, device and realization device |
| CN109272533A (en) * | 2018-10-12 | 2019-01-25 | 山东大学 | Object robust tracking device and tracking method based on compact expression |
| CN114723759A (en) * | 2021-01-06 | 2022-07-08 | 阿里巴巴集团控股有限公司 | Video processing method and device, electronic equipment and storage medium |
| CN115454009A (en) * | 2022-11-11 | 2022-12-09 | 常熟理工学院 | Model Predictive Control Method of Component Formulation for Chemical Production |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140009682A1 (en) * | 2012-07-03 | 2014-01-09 | Motorola Solutions, Inc. | System for media correlation based on latent evidences of audio |
| WO2014133251A1 (en) * | 2013-02-27 | 2014-09-04 | 주식회사 케이쓰리아이 | Matching point extraction system using feature points of data inquiry result of lsh algorithm and method for same |
| CN104081435A (en) * | 2014-04-29 | 2014-10-01 | 中国科学院自动化研究所 | Image matching method based on cascading binary encoding |
| CN104463233A (en) * | 2014-12-30 | 2015-03-25 | 深圳市捷顺科技实业股份有限公司 | Vehicle logo recognition method and device |
| CN105095435A (en) * | 2015-07-23 | 2015-11-25 | 北京京东尚科信息技术有限公司 | Similarity comparison method and device for high-dimensional image features |
-
2016
- 2016-03-25 CN CN201610178939.6A patent/CN105844667A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140009682A1 (en) * | 2012-07-03 | 2014-01-09 | Motorola Solutions, Inc. | System for media correlation based on latent evidences of audio |
| WO2014133251A1 (en) * | 2013-02-27 | 2014-09-04 | 주식회사 케이쓰리아이 | Matching point extraction system using feature points of data inquiry result of lsh algorithm and method for same |
| CN104081435A (en) * | 2014-04-29 | 2014-10-01 | 中国科学院自动化研究所 | Image matching method based on cascading binary encoding |
| CN104463233A (en) * | 2014-12-30 | 2015-03-25 | 深圳市捷顺科技实业股份有限公司 | Vehicle logo recognition method and device |
| CN105095435A (en) * | 2015-07-23 | 2015-11-25 | 北京京东尚科信息技术有限公司 | Similarity comparison method and device for high-dimensional image features |
Non-Patent Citations (2)
| Title |
|---|
| MOSES S. CHARIKAR: "Similarity Estimation Techniques from Rounding Algorithms", 《PROCEEDINGS OF THE THIRY-FOURTH ANNUAL ACM SYMPOSIUM ON THEORY OF COMPUTING》 * |
| YAO RUI ET AL.: "Robust tracking via online Max-Margin structural learning with approximate sparse intersection kernel", 《NEUROCOMPUTING》 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106530327A (en) * | 2016-11-07 | 2017-03-22 | 北京航空航天大学 | Quick real-time discrimination type tracing method based on multi-local-feature learning |
| CN108280808A (en) * | 2017-12-15 | 2018-07-13 | 西安电子科技大学 | The method for tracking target of correlation filter is exported based on structuring |
| CN108596949A (en) * | 2018-03-23 | 2018-09-28 | 云南大学 | Video frequency object tracking state analysis method, device and realization device |
| CN108596949B (en) * | 2018-03-23 | 2020-06-12 | 云南大学 | Video target tracking state analysis method and device and implementation device |
| CN109272533A (en) * | 2018-10-12 | 2019-01-25 | 山东大学 | Object robust tracking device and tracking method based on compact expression |
| CN109272533B (en) * | 2018-10-12 | 2021-08-17 | 山东大学 | A robust object tracking device and tracking method based on compact representation |
| CN114723759A (en) * | 2021-01-06 | 2022-07-08 | 阿里巴巴集团控股有限公司 | Video processing method and device, electronic equipment and storage medium |
| CN115454009A (en) * | 2022-11-11 | 2022-12-09 | 常熟理工学院 | Model Predictive Control Method of Component Formulation for Chemical Production |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105844667A (en) | Structural target tracking method of compact color coding | |
| Liu et al. | A denoising fpn with transformer r-cnn for tiny object detection | |
| CN110378196B (en) | Road visual detection method combining laser point cloud data | |
| Liu et al. | A multikernel domain adaptation method for unsupervised transfer learning on cross-source and cross-region remote sensing data classification | |
| Zhou et al. | Research on pedestrian detection technology based on the SVM classifier trained by HOG and LTP features | |
| Enzweiler et al. | A mixed generative-discriminative framework for pedestrian classification | |
| CN114187450A (en) | A deep learning-based semantic segmentation method for remote sensing images | |
| CN103186775B (en) | Based on the human motion identification method of mix description | |
| CN105528794A (en) | Moving object detection method based on Gaussian mixture model and superpixel segmentation | |
| CN102800099B (en) | Multi-feature multi-level visible light and high-spectrum image high-precision registering method | |
| CN110458192B (en) | Visual saliency-based classification method and system for hyperspectral remote sensing images | |
| CN108596951A (en) | A kind of method for tracking target of fusion feature | |
| CN106056155A (en) | Super-pixel segmentation method based on boundary information fusion | |
| CN102819839B (en) | High-precision registration method for multi-characteristic and multilevel infrared and hyperspectral images | |
| CN102867195B (en) | Method for detecting and identifying a plurality of types of objects in remote sensing image | |
| CN102663436A (en) | Self-adapting characteristic extracting method for optical texture images and synthetic aperture radar (SAR) images | |
| CN113537173A (en) | A Face Image Authenticity Recognition Method Based on Facial Patch Mapping | |
| CN107316005A (en) | The Activity recognition method of son is described based on dense track core covariance | |
| CN104732248A (en) | Human body target detection method based on Omega shape features | |
| CN104200226B (en) | Particle filter method for tracking target based on machine learning | |
| CN109766850B (en) | Fingerprint Image Matching Method Based on Feature Fusion | |
| CN101650829A (en) | Method for tracing covariance matrix based on grayscale restraint | |
| CN113657225A (en) | A target detection method | |
| Sun et al. | Which target to focus on: Class-perception for semantic segmentation of remote sensing | |
| CN104778472A (en) | Extraction method for facial expression feature |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20160810 |
|
| WD01 | Invention patent application deemed withdrawn after publication |