[go: up one dir, main page]

CN114582011B - Pedestrian tracking method based on federal learning and edge calculation - Google Patents

Pedestrian tracking method based on federal learning and edge calculation

Info

Publication number
CN114582011B
CN114582011B CN202111612745.XA CN202111612745A CN114582011B CN 114582011 B CN114582011 B CN 114582011B CN 202111612745 A CN202111612745 A CN 202111612745A CN 114582011 B CN114582011 B CN 114582011B
Authority
CN
China
Prior art keywords
pedestrian
image
model
data
training
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111612745.XA
Other languages
Chinese (zh)
Other versions
CN114582011A (en
Inventor
曾锦超
黄裕
黄家铭
刘洋
覃雄宁
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangxi Zhuang Autonomous Region Public Information Industry Co ltd
Original Assignee
Guangxi Zhuang Autonomous Region Public Information Industry Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangxi Zhuang Autonomous Region Public Information Industry Co ltd filed Critical Guangxi Zhuang Autonomous Region Public Information Industry Co ltd
Priority to CN202111612745.XA priority Critical patent/CN114582011B/en
Publication of CN114582011A publication Critical patent/CN114582011A/en
Application granted granted Critical
Publication of CN114582011B publication Critical patent/CN114582011B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了一种基于联邦学习与边缘计算的行人追踪方法，包括以下步骤：步骤一，检测获取行人图像；步骤二，对行人图像进行跨模态处理；步骤三，提取行人特征；步骤四，匹配特征，确定目标行人。该方法基于联邦学习的模型训练能够有效保证数据隐私，解决数据孤岛问题；使用知识蒸馏与权重分配的方法，解决了由于多场景的数据异构性所带来的精度下降与模型收敛问题；使用跨模态处理技术对不同模态的图像进行处理，以满足跨模态行人追踪的需求。

The present invention discloses a pedestrian tracking method based on federated learning and edge computing, comprising the following steps: step one, detecting and acquiring pedestrian images; step two, performing cross-modal processing on pedestrian images; step three, extracting pedestrian features; step four, matching features and determining target pedestrians. The method can effectively ensure data privacy and solve the problem of data islands based on model training based on federated learning; use knowledge distillation and weight distribution methods to solve the problems of accuracy reduction and model convergence caused by data heterogeneity in multiple scenes; use cross-modal processing technology to process images of different modalities to meet the needs of cross-modal pedestrian tracking.

Description

Pedestrian tracking method based on federal learning and edge calculation
Technical Field
The invention belongs to the technical field of computer vision and federal learning, and particularly relates to a pedestrian tracking method based on federal learning and edge calculation.
Background
The existing pedestrian tracking method needs to train a model by using a large amount of pedestrian data, but a single data set usually only contains data of a specific scene, and the identification accuracy is often greatly reduced when the model trained based on the data set is applied to an actual scene. If the data collected by the video acquisition equipment in a plurality of different scenes are used for model training, data leakage can be caused, and the privacy and the data safety of a user can not be ensured. In addition, the existing pedestrian tracking technology can only be applied to RGB video images photographed in daytime, but pedestrian tracking is required to be applicable to night scenes as well as daytime use. In summary, the model obtained by training the single marked data set by the existing pedestrian tracking technology has the defects of poor generalization capability, incapability of adapting to different use scenes and incapability of processing cross-modal data. Therefore, a pedestrian tracking method capable of ensuring data security, adapting to different application scenes and realizing cross-modal identification is a key for solving the problem.
The federal learning is used as a distributed machine learning technology, so that a plurality of participants can perform machine learning under the condition of protecting data privacy and meeting legal compliance requirements, and the problem of data island is solved. The edge computing technology is mainly applied to the field of video processing, and has the advantages of low transmission delay, large bandwidth and high computing performance. The federal learning and edge computing are applied to the pedestrian tracking field, and the edge server and the cloud server are used for jointly performing model training based on federal learning, so that data privacy can be effectively ensured, and the problem of data island is solved.
The Chinese patent No. CN20191019437. X discloses a pedestrian tracking method, device and equipment, wherein the method comprises the steps of obtaining a video frame to be detected, detecting candidate pedestrians in the video frame to be detected, extracting candidate pedestrian characteristics of the candidate pedestrians, determining differences between the candidate pedestrian characteristics and stored characteristics in a characteristic queue, and determining the candidate pedestrians as target pedestrians when the differences meet preset conditions, wherein the stored characteristics in the characteristic queue are characteristics matched with the target pedestrians. The efficiency of the pedestrian tracking process can be improved. But this method can only be used in the daytime and cannot be applied to night scenes.
The Chinese patent CN201510548633.0 discloses a video pedestrian detection and tracking method based on motion information and track association. Detecting pedestrians, namely detecting motion by using a frame difference method, combining a morphological method in digital image processing, firstly detecting a motion region in a video, then extracting features in the motion region in a sliding window searching mode, and finally classifying by using a pre-trained pedestrian detection classifier to obtain a classification result. The tracking method is characterized in that the pedestrian detection result obtained in the last step is used as input of the step, a tracker is initialized for each detected pedestrian at the beginning, and each tracker contains historical motion information and appearance information of a target. When the current frame is processed, extracting position information and appearance information from each input detection result, and establishing an association matrix based on the position information and the appearance information to associate tracking targets of previous frames, so as to finally obtain the tracking track of pedestrians. The method has good real-time performance and good robustness in more complex scenes. But this method has a data security problem.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a pedestrian tracking method based on federal learning and edge calculation, which uses an edge server and a cloud server to jointly perform model training based on federal learning to effectively ensure data privacy and solve the problem of data island, uses a knowledge distillation and weight distribution method to solve the problems of precision reduction and model convergence caused by data isomerism of multiple scenes when federal learning is applied to pedestrian tracking, and uses a cross-modal processing technology to process images of different modes so as to meet the requirements of cross-modal pedestrian tracking.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
a pedestrian tracking method based on federal learning and edge calculation comprises the following steps:
Step one, detecting and acquiring a pedestrian image;
step two, performing cross-modal processing on the pedestrian image;
Step three, extracting pedestrian characteristics;
And step four, matching the characteristics and determining the target pedestrians.
In a further aspect of the present invention, the detecting and acquiring a pedestrian image in the first step specifically includes:
11 Processing the video to be detected to obtain a plurality of video frame images;
12 Inputting each video frame image of the video file into a pedestrian tracking network, determining whether a pedestrian appears in the current frame image through the pedestrian tracking network, and if the pedestrian is detected, acquiring and outputting the pedestrian image;
The detection is carried out to obtain a pedestrian image based on a detection frame of PP-YOLOv.
The invention further provides a specific flow of the cross-mode processing in the second step, which is as follows:
21 Judging the image data type, if the image is RGB image, executing step 22), if the image is IR image, executing step 23);
22 Converting the RGB image into a gray scale image, wherein the image is a single channel, creating a second channel and filling the second channel with a value of 0;
23 Taking the IR image itself as a second channel, filling the first channel with a value of 0;
24 The image data processed in the step 22) or the step 23) are unified into a double-channel structure and output to the step three for pedestrian feature extraction.
In the process of model training by using the training set, different nodes can selectively inactivate different modes, so that the effect of cross-mode re-identification is achieved.
The specific flow of extracting the pedestrian features in the third step is as follows:
31 Respectively inputting the pedestrian image to be queried and the pedestrian image detected from the video and subjected to cross-modal processing into a trained feature extraction model based on federal learning and edge calculation;
32 The feature extraction model extracts feature vectors of the pedestrian image to be queried and the detected pedestrian images respectively.
The model training method of federal learning and edge computing comprises the steps that firstly, a cloud server selects K participating models from all N edge servers to train to obtain an initial global model, and the cloud server transmits the global model to each edge server. And after receiving the global model, the edge server connects the global model with the local classifier obtained by the previous round of training to form a new local model, and uses the local data set to perform local updating training on the model. The edge server saves the local classifier and uploads the parameters of the backbone network to the cloud server. Finally, the cloud server aggregates the received model parameters uploaded by all edge servers to obtain a new global model, and the global model is continuously issued and the steps are repeated until the model converges to meet the accuracy requirement.
The invention further provides a specific process for determining the target pedestrian by the matching feature in the fourth step, which comprises the following steps:
41 Respectively comparing the feature vectors of the image to be queried with the feature vectors of the pedestrian image detected in the video, and counting
Calculating feature similarity, wherein the feature similarity calculation formula is as follows:
Wherein x is the characteristic of the pedestrian to be inquired, y is the characteristic of the pedestrian image detected in the video image, n is the characteristic dimension, and T x,y is the cosine value between the two characteristics;
42 A pedestrian in the detected image that satisfies the similarity threshold is taken as a target pedestrian in the video.
According to the method, the cloud server, the edge server and the data acquisition equipment are needed to be used in the training process of the feature extraction model, and the data are encrypted by using a homomorphic encryption algorithm and then transmitted when the information is transmitted between the edge server and the cloud server.
According to the method, the cloud server, the edge server and the data acquisition equipment are needed to be used in the training process of the feature extraction model, and the data are encrypted by using a homomorphic encryption algorithm and then transmitted when the information is transmitted between the edge server and the cloud server.
Further describing the invention, the training method of the feature extraction model is optimized through knowledge distillation and weight distribution.
The knowledge distillation optimization method includes the steps of firstly taking a data set as a shared data set, distributing the shared data set and initialized model parameters to edge servers by a cloud server, predicting the shared data set by each edge server through a local model obtained through local data training to obtain a prediction label, uploading updated weights of the local model and the obtained prediction label, carrying out weighted average on the received prediction label by the cloud server, and finally training a federal model by the cloud server through the shared data set and the prediction label, so that model instability is reduced, and better convergence of the model is achieved.
The invention further describes the optimization method of weight distribution, which uses cosine distance as a measurement standard, distributes larger weight to the local model with larger parameter change in each training, and leads more learned knowledge to be reflected in the global model, and comprises the following specific procedures:
51 The edge server randomly selects a group of training data D batch from the local training data;
52 In the next round of training, the edge server receives the model parameters issued by the cloud server, connects the global model with the local classifier obtained in the previous round of training to form a new local model, and sends the local model parameters Save with D batch as log
53 Edge server continues training to obtain new local modelNew model parametersSave with D batch as log
54 Edge server calculates weights by averaging cosine distances for each data point in D batch The calculation formula is as follows:
55 Edge server will weight) Uploading to a cloud server, wherein the cloud server uses the weight value as a weighted average of each model.
The invention has the following beneficial effects:
1. the invention processes images of different modes by using a cross-mode processing technology so as to meet the requirements of cross-mode pedestrian tracking.
2. The present invention is directed to a method for weight redistribution by using knowledge distillation. The method solves the problems of precision reduction and model convergence caused by data heterogeneity of multiple scenes when federal learning is applied to pedestrian tracking.
3. The invention completes all processes under the condition that the data does not go out of the local place by using the federal learning and the edge server to carry out the distributed training of the pedestrian tracking model, thereby protecting the safety of the monitoring data and the privacy of the user.
Drawings
Fig. 1 is a flowchart of pedestrian tracking.
FIG. 2 is a cross-modality processing module workflow diagram.
Fig. 3 is a schematic diagram of a training process of the pedestrian feature extraction module.
Detailed Description
The invention is further described below with reference to the accompanying drawings.
Example 1:
a pedestrian tracking method based on federal learning and edge calculation comprises the following steps:
Step one, detecting and acquiring a pedestrian image;
step two, performing cross-modal processing on the pedestrian image;
Step three, extracting pedestrian characteristics;
And step four, matching the characteristics and determining the target pedestrians.
Example 2:
a pedestrian tracking method based on federal learning and edge calculation comprises the following steps:
step one, detecting and acquiring a pedestrian image, which specifically comprises the following steps:
11 Processing the video to be detected to obtain a plurality of video frame images;
12 Inputting each video frame image of the video file into a pedestrian tracking network, determining whether a pedestrian appears in the current frame image through the pedestrian tracking network, and if the pedestrian is detected, acquiring and outputting the pedestrian image;
step two, performing cross-modal processing on the pedestrian image;
the specific flow of the cross-mode processing is as follows:
21 Judging the image data type, if the image is RGB image, executing step 22), if the image is IR image, executing step 23);
22 Converting the RGB image into a gray scale image, wherein the image is a single channel, creating a second channel and filling the second channel with a value of 0;
23 Taking the IR image itself as a second channel, filling the first channel with a value of 0;
24 The image data processed in the step 22) or the step 23) are unified into a double-channel structure and output to the step three for pedestrian feature extraction.
Step three, extracting pedestrian characteristics, wherein the specific flow is as follows:
31 Respectively inputting the pedestrian image to be queried and the pedestrian image detected from the video and subjected to cross-modal processing into a trained feature extraction model based on federal learning and edge calculation;
32 The feature extraction model extracts feature vectors of the pedestrian image to be queried and the detected pedestrian images respectively.
Step four, matching the characteristics and determining a target pedestrian, wherein the specific flow is as follows:
41 Respectively comparing the feature vectors of the image to be queried with the feature vectors of the pedestrian image detected in the video, and counting
Calculating feature similarity, wherein the feature similarity calculation formula is as follows:
Wherein x is the characteristic of the pedestrian to be inquired, y is the characteristic of the pedestrian image detected in the video image, n is the characteristic dimension, and T x,y is the cosine value between the two characteristics
42 A pedestrian in the detected image that satisfies the similarity threshold is taken as a target pedestrian in the video.
According to the method, the cloud server, the edge server and the data acquisition equipment are needed to be used in the training process of the feature extraction model, and the data are encrypted by using a homomorphic encryption algorithm and then transmitted when the information is transmitted between the edge server and the cloud server.
Example 3:
a pedestrian tracking method based on federal learning and edge calculation comprises the following steps:
Step one, detecting and acquiring a pedestrian image, which specifically comprises the following steps:
11 Processing the video to be detected to obtain a plurality of video frame images;
12 Inputting each video frame image of the video file into a pedestrian tracking network, determining whether a pedestrian appears in the current frame image through the pedestrian tracking network, and if the pedestrian is detected, acquiring and outputting the pedestrian image;
step two, performing cross-modal processing on the pedestrian image;
the specific flow of the cross-mode processing is as follows:
21 Judging the image data type, if the image is RGB image, executing step 22), if the image is IR image, executing step 23);
22 Converting the RGB image into a gray scale image, wherein the image is a single channel, creating a second channel and filling the second channel with a value of 0;
23 Taking the IR image itself as a second channel, filling the first channel with a value of 0;
24 The image data processed in the step 22) or the step 23) are unified into a double-channel structure and output to the step three for pedestrian feature extraction.
Step three, extracting pedestrian characteristics, wherein the specific flow is as follows:
31 Respectively inputting the pedestrian image to be queried and the pedestrian image detected from the video and subjected to cross-modal processing into a trained feature extraction model based on federal learning and edge calculation;
32 The feature extraction model extracts feature vectors of the pedestrian image to be queried and the detected pedestrian images respectively.
The method comprises the steps that a cloud server, an edge server and data acquisition equipment are needed in the training process of the feature extraction model, and when information is transmitted between the edge server and the cloud server, the data are encrypted by using a homomorphic encryption algorithm and then transmitted.
The training method of the feature extraction model is optimized through knowledge distillation and weight distribution;
The knowledge distillation optimization method comprises the steps of firstly taking a data set as a shared data set, distributing the shared data set and initialized model parameters to edge servers by a cloud server, predicting the shared data set by each edge server by using a local model obtained by local data training to obtain a prediction label, uploading updated weight of the local model and the obtained prediction label, and carrying out weighted average on the received prediction label by the cloud server;
the optimization method of the weight distribution is that cosine distance is used as a measurement standard, and the local model with larger parameter change in each training is distributed with larger weight, so that more learned knowledge is reflected in the global model, and the specific flow is as follows:
51 The edge server randomly selects a group of training data D batch from the local training data;
52 In the next round of training, the edge server receives the model parameters issued by the cloud server, connects the global model with the local classifier obtained in the previous round of training to form a new local model, and sends the local model parameters Save with D batch as log
53 Edge server continues training to obtain new local modelNew model parametersSave with D batch as log
54 Edge server calculates weights by averaging cosine distances for each data point in D batch The calculation formula is as follows:
55 Edge server will weight) Uploading to a cloud server, wherein the cloud server uses the weight value as a weighted average of each model.
Step four, matching the characteristics and determining a target pedestrian, wherein the specific flow is as follows:
41 Respectively comparing the feature vectors of the image to be queried with the feature vectors of the pedestrian image detected in the video, and counting
Calculating feature similarity, wherein the feature similarity calculation formula is as follows:
wherein x is the characteristic of the pedestrian to be inquired, y is the characteristic of the pedestrian image detected in the video image, n is the characteristic dimension, and T x,y is the cosine value between the two characteristics;
42 A pedestrian in the detected image that satisfies the similarity threshold is taken as a target pedestrian in the video.
The above embodiments are only exemplary embodiments of the present invention and are not intended to limit the present invention, the scope of which is defined by the claims. Various modifications and equivalent arrangements of parts may be made to the present invention within the spirit and scope of the invention, and such modifications and equivalents should be considered to fall within the scope of the invention.

Claims (6)

1.一种基于联邦学习与边缘计算的行人追踪方法，其特征在于包括以下步骤：1. A pedestrian tracking method based on federated learning and edge computing, characterized by comprising the following steps: 步骤一，检测获取行人图像；Step 1: Detect and obtain pedestrian images; 步骤二，对行人图像进行跨模态处理；Step 2: Perform cross-modal processing on pedestrian images; 所述跨模态处理的具体流程为：The specific process of the cross-modal processing is as follows: 21)判断图像数据类型：如果为RGB图像，执行步骤22)，如果为IR图像，执行步骤23)；21) Determine the image data type: if it is an RGB image, execute step 22); if it is an IR image, execute step 23); 22)将RGB图像转换为灰度图像，此时图像为单通道，创建第二个通道并将第二个通道填充满0值；22) Convert the RGB image to a grayscale image. At this time, the image is a single channel. Create a second channel and fill it with 0 values. 23)将IR图像自身作为第二个通道，将第一个通道填充满0值；23) Use the IR image itself as the second channel and fill the first channel with zero values; 24)经过步骤22)或者步骤23)处理后的图像数据统一为双通道结构并输出至步骤三中进行行人特征提取；24) The image data processed in step 22) or step 23) is unified into a dual-channel structure and output to step 3 for pedestrian feature extraction; 步骤三，提取行人特征，具体流程为：Step 3: Extract pedestrian features. The specific process is as follows: 31)将待查询行人图像与从视频中检测得到并经过跨模态处理的行人图像分别输入到训练好的基于联邦学习与边缘计算的特征提取模型中；31) Input the pedestrian image to be queried and the pedestrian image detected from the video and processed cross-modally into the trained feature extraction model based on federated learning and edge computing respectively; 32)特征提取模型分别提取出待查询行人图像和各检测行人图像的特征向量;32) The feature extraction model extracts the feature vectors of the pedestrian image to be queried and each detected pedestrian image respectively; 步骤四，匹配特征，确定目标行人，具体流程为：Step 4: Match features and determine the target pedestrian. The specific process is as follows: 41)将待查询图像的特征向量分别与视频中检测到的行人图像的特征向量进行对比，计算特征相似度；所述特征相似度计算公式为：41) The feature vector of the image to be queried is compared with the feature vector of the pedestrian image detected in the video, and the feature similarity is calculated; the feature similarity calculation formula is: 其中，x为待查询行人的特征，y为视频图像中检测出来的行人图像的特征，n为特征维度，Tx，y为两特征之间的余弦值；Where x is the feature of the pedestrian to be queried, y is the feature of the pedestrian image detected in the video image, n is the feature dimension, and T x, y is the cosine value between the two features; 42)将满足相似度阈值的检测图像中的行人作为视频中的目标行人。42) The pedestrians in the detected image that meet the similarity threshold are taken as the target pedestrians in the video. 2.根据权利要求1所述的基于联邦学习与边缘计算的行人追踪方法，其特征在于：所述步骤一中检测获取行人图像具体包括：2. The pedestrian tracking method based on federated learning and edge computing according to claim 1 is characterized in that: the step of detecting and acquiring pedestrian images in step 1 specifically includes: 11)对待检测视频进行处理，得到若干视频帧图像；11) Processing the video to be detected to obtain several video frame images; 12)将视频文件的每一视频帧图像输入行人追踪网络中，通过行人追踪网络确定当前帧图像中是否出现行人，如果检测到行人，则获取并输出该行人图像。12) Each video frame image of the video file is input into the pedestrian tracking network, and the pedestrian tracking network is used to determine whether a pedestrian appears in the current frame image. If a pedestrian is detected, the pedestrian image is obtained and output. 3.根据权利要求2所述的基于联邦学习与边缘计算的行人追踪方法，其特征在于：所述特征提取模型训练过程中需要用到云服务器、边缘服务器与数据采集设备；所述边缘服务器与云服务器进行信息传递时使用同态加密算法对数据进行加密之后再进行传输。3. According to claim 2, the pedestrian tracking method based on federated learning and edge computing is characterized in that: cloud servers, edge servers and data acquisition equipment are required in the feature extraction model training process; when the edge server and the cloud server transmit information, the data is encrypted using a homomorphic encryption algorithm before transmission. 4.根据权利要求3所述的基于联邦学习与边缘计算的行人追踪方法，其特征在于：所述特征提取模型的训练方法通过知识蒸馏及权重分配进行优化。4. According to claim 3, the pedestrian tracking method based on federated learning and edge computing is characterized in that the training method of the feature extraction model is optimized through knowledge distillation and weight distribution. 5.根据权利要求4所述的基于联邦学习与边缘计算的行人追踪方法，其特征在于：所述知识蒸馏的优化方法为，首先将一个数据集作为共享数据集，云服务器将共享数据集与初始化的模型参数一起分发给边缘服务器，每个边缘服务器使用本地数据训练得到的本地模型对共享数据集进行预测，得到预测标签；之后上传本地模型更新后的权重与得到的预测标签，云服务器将收到的预测标签进行加权平均；最后，云服务器使用共享数据集与预测标签对联邦模型进行训练，减少模型的不稳定，并使模型更好的收敛。5. According to the pedestrian tracking method based on federated learning and edge computing described in claim 4, it is characterized in that: the optimization method of the knowledge distillation is to first use a data set as a shared data set, and the cloud server distributes the shared data set together with the initialized model parameters to the edge server, and each edge server uses the local model trained with local data to predict the shared data set to obtain a predicted label; then upload the updated weights of the local model and the obtained predicted labels, and the cloud server performs weighted average on the received predicted labels; finally, the cloud server uses the shared data set and the predicted labels to train the federated model to reduce the instability of the model and make the model converge better. 6.根据权利要求5所述的基于联邦学习与边缘计算的行人追踪方法，其特征在于：所述权重分配的优化方法为，使用余弦距离作为度量标准，对每次训练中参数变化越大的本地模型，分配更大的权重，使更多新学到的知识反映在全局模型中；具体流程为：6. The pedestrian tracking method based on federated learning and edge computing according to claim 5 is characterized in that: the optimization method of weight allocation is to use cosine distance as a metric, and to assign a larger weight to the local model with a larger parameter change in each training, so that more newly learned knowledge is reflected in the global model; the specific process is: 51)边缘服务器从本地训练数据中随机选择一组训练数据Dbatch；51) The edge server randomly selects a set of training data D batch from the local training data; 52)下一轮训练中，边缘服务器收到云服务器下发的模型参数，将全局模型与上一轮训练得到的本地分类器连接起来形成新的本地模型，将本地模型参数与Dbatch保存为日志 52) In the next round of training, the edge server receives the model parameters sent by the cloud server, connects the global model with the local classifier obtained from the previous round of training to form a new local model, and sets the local model parameters Save as log with D batch 53)边缘服务器继续训练得到新的本地模型将新的模型参数与Dbatch保存为日志 53) The edge server continues to train to obtain a new local model The new model parameters Save as log with D batch 54)边缘服务器通过对Dbatch中每个数据点的余弦距离求平均值来计算权重计算公式如下：54) The edge server calculates the weight by averaging the cosine distance of each data point in the D batch The calculation formula is as follows: 55)边缘服务器将权重上传至云服务器，云服务器使用其作为各个模型加权平均的权重值。55) The edge server will weight Uploaded to the cloud server, the cloud server uses it as the weight value of the weighted average of each model.
CN202111612745.XA 2021-12-27 2021-12-27 Pedestrian tracking method based on federal learning and edge calculation Active CN114582011B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111612745.XA CN114582011B (en) 2021-12-27 2021-12-27 Pedestrian tracking method based on federal learning and edge calculation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111612745.XA CN114582011B (en) 2021-12-27 2021-12-27 Pedestrian tracking method based on federal learning and edge calculation

Publications (2)

Publication Number Publication Date
CN114582011A CN114582011A (en) 2022-06-03
CN114582011B true CN114582011B (en) 2025-07-18

Family

ID=81768836

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111612745.XA Active CN114582011B (en) 2021-12-27 2021-12-27 Pedestrian tracking method based on federal learning and edge calculation

Country Status (1)

Country Link
CN (1) CN114582011B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115221135B (en) * 2022-07-21 2023-05-09 陕西合友网络科技有限公司 Sharing method and system for industrial Internet data
CN115145966B (en) * 2022-09-05 2022-11-11 山东省计算中心（国家超级计算济南中心） Comparison federated learning method and system for heterogeneous data
CN116416280B (en) * 2023-04-14 2025-07-22 中国人民解放军国防科技大学 Cooperative target tracking method, device and equipment based on self-adaptive federal learning

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113642700A (en) * 2021-07-05 2021-11-12 湖南师范大学 Cross-platform multimodal public opinion analysis method based on federated learning and edge computing

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7685252B1 (en) * 1999-10-12 2010-03-23 International Business Machines Corporation Methods and systems for multi-modal browsing and implementation of a conversational markup language
EP1455282A1 (en) * 2003-03-06 2004-09-08 Alcatel Markup language extension enabling speech recognition for controlling an application
DE102008051756A1 (en) * 2007-11-12 2009-05-14 Volkswagen Ag Multimodal user interface of a driver assistance system for entering and presenting information
CN104331412B (en) * 2014-09-23 2018-03-09 北京工业大学 A kind of method that face retrieval is carried out in normalization three-dimensional face storehouse
CN105184303B (en) * 2015-04-23 2019-08-09 南京邮电大学 An Image Annotation Method Based on Multimodal Deep Learning
CN108388862B (en) * 2018-02-08 2021-09-14 西北农林科技大学 Face recognition method based on LBP (local binary pattern) characteristics and nearest neighbor classifier
CN113168669B (en) * 2018-09-18 2024-03-29 浙江宇视科技有限公司 Image processing method, device, electronic device and readable storage medium
CN109408384B (en) * 2018-10-16 2022-07-29 网易（杭州）网络有限公司 Software application testing method and device, processor and electronic device
WO2020160485A1 (en) * 2019-01-31 2020-08-06 Alfred E. Mann Institute For Biomedical Engineering At The University Of Southern California A hyperspectral imaging system
CN111444848A (en) * 2020-03-27 2020-07-24 广州英码信息科技有限公司 Specific scene model upgrading method and system based on federal learning
CN113128701A (en) * 2021-04-07 2021-07-16 中国科学院计算技术研究所 Sample sparsity-oriented federal learning method and system
CN113449623B (en) * 2021-06-21 2022-06-28 浙江康旭科技有限公司 A Lightweight Liveness Detection Method Based on Deep Learning
CN113361694B (en) * 2021-06-30 2022-03-15 哈尔滨工业大学 A hierarchical federated learning method and system applying differential privacy protection
CN113609521B (en) * 2021-07-27 2022-11-01 广州大学 A privacy protection method and system for federated learning based on adversarial training
CN113627332B (en) * 2021-08-10 2025-01-28 宜宾电子科技大学研究院 A distracted driving behavior recognition method based on gradient controlled federated learning

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113642700A (en) * 2021-07-05 2021-11-12 湖南师范大学 Cross-platform multimodal public opinion analysis method based on federated learning and edge computing

Also Published As

Publication number Publication date
CN114582011A (en) 2022-06-03

Similar Documents

Publication Publication Date Title
CN114582011B (en) Pedestrian tracking method based on federal learning and edge calculation
CN111931637B (en) Cross-modal pedestrian re-identification method and system based on double-flow convolutional neural network
CN109948561B (en) Method and system for unsupervised image and video pedestrian re-identification based on transfer network
CN104601964B (en) Pedestrian target tracking and system in non-overlapping across the video camera room of the ken
CN109684922B (en) A multi-model recognition method for finished dishes based on convolutional neural network
CN116403123B (en) Remote sensing image change detection method based on deep convolutional network
CN114359773B (en) Video Person Re-identification Method Based on Trajectory Fusion in Complex Underground Space
CN105404886A (en) Feature model generating method and feature model generating device
CN112016402B (en) Self-adaptive method and device for pedestrian re-recognition field based on unsupervised learning
CN110633643A (en) Abnormal behavior detection method and system for smart community
CN114139013B (en) Image searching method, device, electronic equipment and computer readable storage medium
CN107301376B (en) A Pedestrian Detection Method Based on Deep Learning Multi-layer Stimulation
CN108960142B (en) Pedestrian re-identification method based on global feature loss function
Tsintotas et al. DOSeqSLAM: Dynamic on-line sequence based loop closure detection algorithm for SLAM
WO2019167784A1 (en) Position specifying device, position specifying method, and computer program
CN111353496A (en) A real-time detection method for infrared weak and small targets
CN104599291B (en) Infrared motion target detection method based on structural similarity and significance analysis
CN104346630A (en) Cloud flower identifying method based on heterogeneous feature fusion
Niu et al. Boundary-aware RGBD salient object detection with cross-modal feature sampling
CN116721349A (en) Infrared small target detection method and device based on track assumption and track verification
CN107330363B (en) Rapid internet billboard detection method
CN110188718B (en) An unconstrained face recognition method based on keyframes and joint sparse representation
Zhao et al. A benchmark for vehicle re-identification in mixed visible and infrared domains
CN117373100B (en) Face recognition method and system based on differential quantization local binary pattern
CN116884030B (en) Cross-modal person re-identification method and system based on semantic alignment and relational reasoning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant