[go: up one dir, main page]

CN119759083A - A UAV autonomous target search system and method based on large language model - Google Patents

A UAV autonomous target search system and method based on large language model Download PDF

Info

Publication number
CN119759083A
CN119759083A CN202411936066.1A CN202411936066A CN119759083A CN 119759083 A CN119759083 A CN 119759083A CN 202411936066 A CN202411936066 A CN 202411936066A CN 119759083 A CN119759083 A CN 119759083A
Authority
CN
China
Prior art keywords
target
module
task
unmanned aerial
planning
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202411936066.1A
Other languages
Chinese (zh)
Inventor
宋伟
胡拓成
宋雨璠
朱世强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yuyao Robot Research Center
Zhejiang University ZJU
Original Assignee
Yuyao Robot Research Center
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yuyao Robot Research Center, Zhejiang University ZJU filed Critical Yuyao Robot Research Center
Priority to CN202411936066.1A priority Critical patent/CN119759083A/en
Publication of CN119759083A publication Critical patent/CN119759083A/en
Pending legal-status Critical Current

Links

Landscapes

  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

本发明属于无人机自主导航和目标搜寻技术领域,公开了一种基于大型语言模型的无人机自主目标搜寻系统,包括自然语言处理模块、多传感器融合模块、控制与飞行模块、目标检测模块、路径规划模块、任务规划与控制模块,所述系统通过利用通用大语言模型处理自然语言形式的任务描述和状态信息,实现无人机自主规划和执行复杂的搜寻任务;本发明的系统能够在各种复杂环境中运行,适用于多种应用场景,如灾害救援、环境监测、巡逻安防和物流运输等,具有广泛的应用前景和市场价值。通过结合自然语言处理、多传感器融合和深度学习等先进技术,通过各模块的有机结合,实现了无人机自主目标搜寻系统的智能化和高效化。

The present invention belongs to the technical field of autonomous navigation and target search of unmanned aerial vehicles, and discloses an autonomous target search system of unmanned aerial vehicles based on a large language model, including a natural language processing module, a multi-sensor fusion module, a control and flight module, a target detection module, a path planning module, and a task planning and control module. The system realizes autonomous planning and execution of complex search tasks by unmanned aerial vehicles by using a general large language model to process task descriptions and status information in natural language form; the system of the present invention can operate in various complex environments and is suitable for a variety of application scenarios, such as disaster relief, environmental monitoring, patrol security, and logistics transportation, etc., and has broad application prospects and market value. By combining advanced technologies such as natural language processing, multi-sensor fusion, and deep learning, and by organically combining various modules, the intelligentization and efficiency of the autonomous target search system of unmanned aerial vehicles are realized.

Description

Unmanned aerial vehicle autonomous target searching system and method based on large language model
Technical Field
The invention belongs to the technical field of unmanned aerial vehicle autonomous navigation and target searching, and particularly relates to an unmanned aerial vehicle autonomous target searching system and method based on a large language model.
Background
Along with the rapid development of unmanned aerial vehicle technology, unmanned aerial vehicle is more and more widely applied in fields such as military reconnaissance, search rescue, environmental monitoring, etc. Traditional unmanned aerial vehicle search systems mainly rely on preset flight paths and target positions, lack flexibility and autonomy, and are difficult to cope with complex and changeable environments and task demands. In recent years, with the development of artificial intelligence and deep learning technology, unmanned aerial vehicle autonomous navigation and target recognition capability are remarkably improved, but some technical bottlenecks still exist.
Conventional unmanned aerial vehicle systems generally rely on data of a single sensor for environment sensing and target recognition, and are easily affected by environmental noise and sensor errors, so that sensing accuracy and robustness are insufficient. Meanwhile, when the existing path planning algorithm is used for processing dynamic obstacles in a complex environment, the calculation efficiency and the instantaneity still need to be improved. In addition, the conventional unmanned aerial vehicle task planning system generally adopts traditional methods such as rules, finite state machines and the like, is difficult to process complex and unstructured task descriptions, and limits the autonomous decision making capability and task execution efficiency of the unmanned aerial vehicle.
Disclosure of Invention
The invention aims to provide an unmanned aerial vehicle autonomous target searching system and method based on a large language model, so as to solve the technical problems.
In order to solve the technical problems, the unmanned aerial vehicle autonomous target searching system and method based on the large language model have the following specific technical scheme:
The unmanned aerial vehicle autonomous target searching system based on the large language model comprises a natural language processing module, a multi-sensor fusion module, a control and flight module, a target detection module, a path planning module and a task planning and control module, wherein the system processes task description and state information in a natural language form by utilizing the general large language model to realize unmanned aerial vehicle autonomous planning and execute complex searching tasks;
The natural language processing module is composed of a main body by a general large language model, and outputs a task planning result according to a current task target and state information, wherein the task target is a text in a natural language form, the state information is usually a text in a natural language form for describing a current scene and other information related to the task, and the task planning result is a task frame generated by the general large language model on the basis of input information and comprises task steps and execution sequences;
The multi-sensor fusion module is used for receiving environmental data from a plurality of sensors and preprocessing the acquired sensor data;
The control and flight module is used for receiving flight instructions and controlling the movement of the unmanned aerial vehicle, adjusting the flight track of the unmanned aerial vehicle according to a preset path and a real-time position, adjusting the gesture of the unmanned aerial vehicle in real time to keep flight stability, and managing and adjusting the power system of the unmanned aerial vehicle to realize high-efficiency flight;
The target detection module acquires environment data from the multi-sensor fusion module, synchronizes and fuses the multi-sensor data, processes the fused data by using a deep learning algorithm to detect and identify a target, and determines the accurate coordinates of the target by calculation based on the position of the detected target in the environment;
The task planning and control module calls a corresponding path planning module and a corresponding control and flight module to execute corresponding task steps according to a task framework, and ensures the task to be carried out according to the conversion of the control state of the target detection result given by the target detection module.
The invention also discloses an unmanned aerial vehicle autonomous target searching method of the unmanned aerial vehicle autonomous target searching system based on the large language model, which comprises the following steps:
S1, controlling an unmanned aerial vehicle to perform environment map modeling by using a laser radar in a multi-sensor fusion module;
S2, the task planning and control module enters an initial waiting state and controls the unmanned aerial vehicle to fly to an initial appointed place;
S3, after judging that the unmanned aerial vehicle reaches an initial designated place, the task planning and control module enters a waiting task frame state, and receives task targets and state information in a natural language form by calling a natural language processing module to generate a task frame comprising task steps and execution sequences;
S4, after the task planning and control module receives the execution plan, entering an action execution state, and calling a path planning module according to steps in a task frame to enable the unmanned aerial vehicle to fly to a designated search place;
s5, after the task planning and control module judges that the unmanned aerial vehicle reaches a target place, the unmanned aerial vehicle enters a searching target state, invokes a path planning module and performs round-robin searching at the target place;
S6, the task planning and control module continuously calls the result of the target detection module, if the target is found, the step 7 is skipped, and if the target is not found after the target place searching is finished, the step 9 is skipped;
s7, after the unmanned aerial vehicle is judged to find a target, the task planning and control module enters a target positioning state, and a multi-sensor fusion module is called to determine target coordinates and report the target coordinates;
S8, after judging that the unmanned aerial vehicle reaches the upper air of the target place, the task planning and control module enters a reset state, invokes the path planning module, flies to the initial designated place, returns to the step 3, and waits for the input of the next target searching task;
s9, at the moment, after searching the target place, the designated target is not found, and the task planning and control module enters a waiting task framework state again.
Further, the state equation corresponding to the environment map modeling in the step1 is as follows:
xk=f(xk-1,uk)+wk
Where x k is the state vector at time k, f (·) is a nonlinear state transfer function, u k is the control input, w k is the process noise, and the observation equation is as follows:
zk=h(xk)+vk
where z k is the observation data at time k, h (·) is a nonlinear observation function, representing the relationship from the state vector to the observation data, and v k is the observation noise.
Further, in S2, the unmanned aerial vehicle flight control uses PID control to ensure the flight from the current position to the target site, and the PID control formula is as follows:
Wherein e (t) is the error between the current position and the target position, and K p,Ki,Kd is the proportional, integral and differential gains of the PID controller.
Further, the S3 receives the task object g t in natural language form and the status information S t including the history search information h t and the scene map information o t by calling the natural language processing module M n to generate an execution plan including n task steps and execution sequencesThe natural language processing module formula is as follows:
pt=Mn(gt,st)。
Further, after the S4 task planning and control module receives the execution plan p t, it enters an action execution state, and according to the steps in the task framework, invokes the path planning module to fly the unmanned aerial vehicle to the designated search location, where the formula of the path planning algorithm a is as follows:
f(n)=g(n)+h(n)
Where f (n) is the total estimated cost of the current node, g (n) is the actual cost from the starting node to the current node, and h (n) is the estimated cost from the current node to the target node.
Further, the path generation formula in S5 is as follows:
x(t)=r·cos(ωt)
y(t)=r·sin(ωt)
where r is the radius, ω is the angular velocity, and t is the time.
Further, the target detection module M d in S6 gives the target detection probability and the position of the target in the camera coordinate system, and the formula is as follows:
ximage,yimage,Pobj=Md(I)
Where (x image,yimage) represents the coordinates of the object in the image, P obj represents the probability that the object is present, and I is the input image.
Further, the camera internal reference matrix in S7 converts the pixel coordinates into normalized coordinates in the camera coordinate system, and the formula is as follows:
Here, K -1 is an inverse of the camera's reference matrix, and the camera's reference matrix converts three-dimensional coordinates in the camera's coordinate system into three-dimensional coordinates in the world coordinate system, and the formula is as follows:
Where R is the rotation matrix and t is the translation vector.
Further, in S9, the current task object and the execution action are added into the history search information h t+1, and then the task object g t+1 in the form of natural language and the state information S t+1 including the history search information h t+1 and the scene map information o t+1 are received again by calling the natural language processing module to generate an execution plan including n task steps and execution sequences for the next roundJump to step 4.
The unmanned aerial vehicle autonomous target searching system and method based on the large language model have the advantages that the universal large language model is adopted to process natural language input, so that the system can quickly and accurately understand task targets. By automatically generating the task frame and the steps, human intervention is reduced, continuous target searching tasks are automatically completed, and the efficiency and accuracy of target searching are improved. According to the invention, a plurality of sensor data are integrated through the multi-sensor fusion module, so that denoising, calibration and synchronization of the data are realized, and the accuracy and the robustness of environmental awareness are improved. The complementarity of the different sensors enhances the adaptability of the system to complex environments. The invention processes the multi-sensor fusion data by using a deep learning algorithm through the target detection module, and has high-precision and high-robustness target detection and recognition capability. Through accurate calculation of the target position, the capability of the unmanned aerial vehicle for autonomously searching the target in a complex environment is improved. According to the invention, the optimal flight path is generated by combining the real-time environment data with the preset target through the path planning module, and the dynamic obstacle avoidance is performed in the flight process, so that the unmanned aerial vehicle can safely and quickly reach the target position. The real-time adjustment capability of the module adapts to the dynamically-changing environment, and the autonomy and flexibility of the unmanned aerial vehicle are enhanced. According to the invention, the stability and the high efficiency of the unmanned aerial vehicle in the flight process are ensured by adjusting the flight track and the gesture in real time through the control and flight module. And the flight performance is optimized by managing the power system, so that the endurance time and the task execution capacity of the unmanned aerial vehicle are prolonged. The system can operate in various complex environments, is suitable for various application scenes such as disaster relief, environment monitoring, patrol security and logistics transportation and the like, and has wide application prospect and market value. By combining advanced technologies such as natural language processing, multi-sensor fusion, deep learning and the like and through the organic combination of the modules, the intelligent and high-efficiency of the unmanned aerial vehicle autonomous target search system is realized.
Drawings
FIG. 1 is a schematic diagram of a policy network based on a pre-trained language model in the present invention.
FIG. 2 is a schematic diagram of the process of the present invention.
FIG. 3 is a schematic diagram of a memory adapter modification action in accordance with the present invention.
Detailed Description
In order to better understand the purpose, structure and function of the present invention, the following describes in further detail an unmanned aerial vehicle autonomous target searching system and method based on a large language model with reference to the accompanying drawings.
As shown in FIG. 1, the unmanned aerial vehicle autonomous target searching system based on the large language model comprises a natural language processing module, a multi-sensor fusion module, a control and flight module, a target detection module, a path planning module and a task planning and control module.
The natural language processing module is composed of a main body by a general large language model, and outputs a task planning result according to a current task target and state information, wherein the task target is a text in a natural language form, the state information is usually a text in a natural language form for describing a current scene and other information related to the task, and the task planning result is a task frame generated by the general large language model on the basis of input information and comprises task steps and execution sequences;
The multi-sensor fusion module can receive environmental data from a plurality of sensors, including but not limited to cameras, lidars, infrared sensors and inertial measurement units, and can pre-process acquired sensor data, including denoising, calibration and synchronization;
the control and flight module can receive flight instructions and control the movement of the unmanned aerial vehicle, can adjust the flight track of the unmanned aerial vehicle according to a preset path and a real-time position, can adjust the attitude of the unmanned aerial vehicle in real time to maintain flight stability, and can manage and adjust the power system of the unmanned aerial vehicle to realize high-efficiency flight;
The target detection module acquires environment data from the multi-sensor fusion module, can synchronize and fuse the multi-sensor data to improve the accuracy and the robustness of environment sensing, can process the fused data by using a deep learning algorithm to detect and identify a target, and can determine the accurate coordinates of the target by calculation based on the position of the detected target in the environment;
the path planning module can generate an optimal path of the unmanned aerial vehicle based on the environmental data and the preset target generated by the multi-sensor fusion module, and detect and avoid obstacles in real time in the flight process of the unmanned aerial vehicle;
The task planning and control module can call the corresponding path planning module and control and flight module to execute corresponding task steps according to the task framework, and can control the state conversion according to the target detection result given by the target detection module, so that the task is ensured to be carried out according to the steps.
According to the system, a natural language processing module is used as a decision core, tasks of the unmanned aerial vehicle can be planned in real time through task targets and state information in a natural language form, and environment perception and target detection are realized by utilizing multi-sensor data and a target detection module, so that efficient and accurate target searching task execution is realized.
As shown in fig. 2, the unmanned aerial vehicle autonomous target searching method based on the large language model of the invention comprises the following steps:
S1, controlling the unmanned aerial vehicle to perform environment map modeling by using a laser radar in a multi-sensor fusion module. The corresponding state equation is as follows:
xk=f(xk-1,uk)+wk
Where x k is the state vector at time k, f (·) is a nonlinear state transfer function, u k is the control input, and w k is the process noise. The observation equation is as follows:
zk=h(xk)+vk
where z k is the observation data at time k, h (·) is a nonlinear observation function, representing the relationship from the state vector to the observation data, and v k is the observation noise.
S2, the task planning and control module enters an initial waiting state, and the unmanned aerial vehicle is controlled to fly to an initial appointed place. Unmanned aerial vehicle flight control typically uses PID control to ensure flight from a current location to a target site. The PID control formula is as follows:
Wherein e (t) is the error between the current position and the target position, and K p,Ki,Kd is the proportional, integral and differential gains of the PID controller.
S3, after judging that the unmanned aerial vehicle arrives at the initial designated place, the task planning and control module enters a waiting task framework state, and receives a task target g t in a natural language form and state information S t containing history search information h t and scene map information o t through calling a natural language processing module M n to generate an execution plan containing n task steps and execution sequencesThe natural language processing module formula is as follows:
pt=Mn(gt,st)
S4, after the task planning and control module receives the execution plan p t, the task planning and control module enters an action execution state, and calls the path planning module according to the steps in the task framework, so that the unmanned aerial vehicle flies to the appointed searching place. The path planning algorithm formula is as follows:
f(n)=g(n)+h(n)
Where f (n) is the total estimated cost of the current node, g (n) is the actual cost from the starting node to the current node, and h (n) is the estimated cost from the current node to the target node.
S5, after the task planning and control module judges that the unmanned aerial vehicle reaches the target place, the unmanned aerial vehicle enters a searching target state, and the path planning module is called to perform round-robin searching at the target place. The path generation formula is as follows:
x(t)=r·cos(ωt)
y(t)=r·sin(ωt)
where r is the radius, ω is the angular velocity, and t is the time.
S6, the task planning and control module continuously calls the result of the target detection module, if the target is found, the step 7 is skipped, and if the target is not found after the target place searching is finished, the step 9 is skipped. The object detection module M d gives the object detection probability and the position of the object in the camera coordinate system, and the formula is as follows:
ximage,yimage,Pobj=Md(I)
Where (x image,yimage) represents the coordinates of the object in the image, P obj represents the probability that the object is present, and I is the input image.
S7, after the unmanned aerial vehicle is judged to find the target, the task planning and control module enters a target positioning state, a multi-sensor fusion module is called to determine target coordinates and report the target coordinates, and then a path planning module is called to fly to a target place. The camera internal reference matrix converts the pixel coordinates into normalized coordinates in the camera coordinate system, and the formula is as follows:
Here K -1 is the inverse of the camera reference matrix. The camera's extrinsic matrix converts the three-dimensional coordinates in the camera's coordinate system to three-dimensional coordinates in the world's coordinate system, as follows:
Where R is the rotation matrix and t is the translation vector.
S8, after the unmanned aerial vehicle is judged to reach the upper air of the target place, the unmanned aerial vehicle enters a reset state, the path planning module is called, the unmanned aerial vehicle flies to the initial appointed place, the step 3 is returned, and the next input of the target searching task is waited.
S9, when no specified target is found after searching the target place, the task planning and control module enters a waiting task frame state again, adds the current task target and execution action into history searching information h t+1, then receives the task target g t+1 in natural language form and state information S t+1 containing history searching information h t+1 and scene map information o t+1 through calling the natural language processing module again, and generates an execution plan containing n task steps and execution sequence in the next roundJump to step 4.
As shown in fig. 2, the solution overall frame is built up in this form. The task planning and control module transmits instructions such as a construction map or acquiring environment data to the multi-sensor fusion module, then the multi-sensor fusion module returns an environment map or an environment photo, the channel 2 transmits current task targets and state information to the natural language processing module, then the natural language processing module returns a task frame containing task steps and execution sequences, the channel 3 transmits destination coordinates to the path planning module, then the path planning module returns a planning result, the channel 4 transmits the path planning to the control and flight module, then the control and flight module returns a flight state to the path planning module, and the channel 5 transmits search target information and environment picture data to the target detection module, and then the target detection module transmits the detection result to the task planning and control module.
As shown in fig. 3, one application scenario of the present invention is for a cargo-carrying unmanned aerial vehicle to search for a designated recipient at a designated receiving point. After the system receives the customer order, the logistics system generates a task instruction comprising goods information, a receiving address, an appointed addressee and the like. The natural language processing module analyzes the instruction and generates a task frame, wherein the unmanned aerial vehicle carries goods to take off from the warehouse A and fly to the receiving point B. And searching the receiver with the specified characteristics at the receiving point B, after the searching of the target is successful, flying to the target receiver, and then unloading the goods from the unmanned aerial vehicle by the receiver, and completing signing. Finally, the unmanned aerial vehicle completes the task and returns to warehouse a.
It will be understood that the application has been described in terms of several embodiments, and that various changes and equivalents may be made to these features and embodiments by those skilled in the art without departing from the spirit and scope of the application. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the application without departing from the essential scope thereof. Therefore, it is intended that the application not be limited to the particular embodiment disclosed, but that the application will include all embodiments falling within the scope of the appended claims.

Claims (10)

1.一种基于大型语言模型的无人机自主目标搜寻系统,其特征在于,包括自然语言处理模块、多传感器融合模块、控制与飞行模块、目标检测模块、路径规划模块、任务规划与控制模块,所述系统通过利用通用大语言模型处理自然语言形式的任务描述和状态信息,实现无人机自主规划和执行复杂的搜寻任务;1. An autonomous target search system for unmanned aerial vehicles based on a large language model, characterized in that it includes a natural language processing module, a multi-sensor fusion module, a control and flight module, a target detection module, a path planning module, and a task planning and control module. The system uses a universal large language model to process task descriptions and status information in natural language form, thereby enabling unmanned aerial vehicles to autonomously plan and execute complex search tasks; 所述自然语言处理模块由通用大语言模型构成主体,根据当前的任务目标和状态信息输出任务规划的结果,任务目标是自然语言形式的文本,状态信息通常是描述当前场景以及其他与任务相关信息的自然语言形式的文本,任务规划的结果是通用大语言模型在输入信息基础上生成的任务框架,包含任务步骤和执行顺序;The natural language processing module is mainly composed of a general large language model, and outputs the result of task planning according to the current task goal and state information. The task goal is a text in natural language form, and the state information is usually a text in natural language form describing the current scene and other task-related information. The result of task planning is a task framework generated by the general large language model based on the input information, including task steps and execution order; 所述多传感器融合模块用于从多个传感器接收环境数据,对获取的传感器数据进行预处理;The multi-sensor fusion module is used to receive environmental data from multiple sensors and pre-process the acquired sensor data; 所述控制与飞行模块用于接收飞行指令并控制无人机的运动,根据预定路径和实时位置调整无人机的飞行轨迹,实时调整无人机的姿态以保持飞行稳定性,管理和调节无人机的动力系统,以实现高效飞行;The control and flight module is used to receive flight instructions and control the movement of the UAV, adjust the flight trajectory of the UAV according to the predetermined path and real-time position, adjust the attitude of the UAV in real time to maintain flight stability, and manage and adjust the power system of the UAV to achieve efficient flight; 所述目标检测模块从多传感器融合模块中获取环境数据,并对多传感器数据进行同步和融合,使用深度学习算法对融合后的数据进行处理,以检测和识别目标,基于检测到的目标在环境中的位置,通过计算确定目标的精确坐标;The target detection module obtains environmental data from the multi-sensor fusion module, synchronizes and fuses the multi-sensor data, processes the fused data using a deep learning algorithm to detect and identify the target, and determines the precise coordinates of the target by calculation based on the position of the detected target in the environment; 所述路径规划模块基于多传感器融合模块生成的环境数据和预定目标生成无人机的最优路径以及在无人机飞行过程中实时检测和避开障碍物;所述任务规划与控制模块根据任务框架,调用相应的路径规划模块和控制与飞行模块执行相应任务步骤,根据目标检测模块给出的目标检测结果控制状态的转换,确保任务按步骤进行。The path planning module generates the optimal path for the UAV based on the environmental data and predetermined targets generated by the multi-sensor fusion module, and detects and avoids obstacles in real time during the flight of the UAV; the mission planning and control module calls the corresponding path planning module and the control and flight module to execute the corresponding mission steps according to the mission framework, and controls the state transition according to the target detection results given by the target detection module to ensure that the task is carried out step by step. 2.一种如权利要求1所述的基于大型语言模型的无人机自主目标搜寻系统的无人机自主目标搜寻方法,其特征在于,包括如下步骤:2. A method for autonomous target search of a drone of the autonomous target search system of a drone based on a large language model as claimed in claim 1, characterized in that it comprises the following steps: S1,控制无人机利用多传感器融合模块中的激光雷达进行环境地图建模;S1, controls the UAV to use the lidar in the multi-sensor fusion module to model the environment map; S2,任务规划与控制模块进入初始等待状态,控制无人机飞向初始指定地点;S2, the mission planning and control module enters the initial waiting state and controls the drone to fly to the initial designated location; S3,任务规划与控制模块在判断无人机到达初始指定地点后,进入等待任务框架状态,通过调用自然语言处理模块接收自然语言形式的任务目标和状态信息,生成包含任务步骤和执行顺序的任务框架;S3, after determining that the UAV has arrived at the initial designated location, the mission planning and control module enters the waiting mission framework state, receives the mission objectives and status information in natural language form by calling the natural language processing module, and generates a mission framework including mission steps and execution order; S4,任务规划与控制模块接受执行计划后,进入动作执行状态,根据任务框架中的步骤,调用路径规划模块,使无人机飞向指定搜索地点;S4, after the mission planning and control module receives the execution plan, it enters the action execution state and calls the path planning module according to the steps in the mission framework to make the drone fly to the designated search location; S5,任务规划与控制模块在判断无人机到达目标地点后,进入搜索目标状态,调用路径规划模块,在目标地点进行绕圆搜索;S5, after determining that the UAV has reached the target location, the mission planning and control module enters the target search state, calls the path planning module, and performs a circular search at the target location; S6,任务规划与控制模块不断调用目标检测模块的结果,若发现目标,则跳转到步骤7;若在目标地点搜寻结束未发现目标,则跳转到步骤9;S6, the mission planning and control module continuously calls the results of the target detection module. If the target is found, it jumps to step 7; if the target is not found after the search at the target location, it jumps to step 9; S7,任务规划与控制模块在判断无人机发现目标后,进入定位目标状态,通过调用多传感器融合模块确定目标坐标,并报告目标坐标;然后,调用路径规划模块,飞向目标地点;S7, after determining that the UAV has found the target, the mission planning and control module enters the target positioning state, determines the target coordinates by calling the multi-sensor fusion module, and reports the target coordinates; then, the path planning module is called to fly to the target location; S8,任务规划与控制模块在判断无人机到达目标地点上空后,进入重置状态,调用路径规划模块,飞向初始指定地点,并返回到步骤3,等待下一次目标搜寻任务的输入;S8, after determining that the UAV has reached the target location, the mission planning and control module enters a reset state, calls the path planning module, flies to the initial designated location, and returns to step 3 to wait for the input of the next target search mission; S9,此时在目标地点搜索后未发现指定目标,任务规划与控制模块再次进入等待任务框架状态。S9, at this time, after searching the target location, no designated target is found, and the mission planning and control module enters the waiting mission framework state again. 3.根据权利要求2所述的无人机自主目标搜寻方法,其特征在于,所述步骤1环境地图建模对应的状态方程如下:3. The autonomous target search method for unmanned aerial vehicles according to claim 2, characterized in that the state equation corresponding to the environmental map modeling in step 1 is as follows: xk=f(xk-1,uk)+wk x k =f(x k-1 , uk )+w k 其中,xk为时刻k的状态向量,f(·)是非线性状态转移函数,uk是控制输入,wk是过程噪声,观测方程如下:Where xk is the state vector at time k, f(·) is the nonlinear state transfer function, uk is the control input, wk is the process noise, and the observation equation is as follows: zk=h(xk)+vk z k =h(x k )+v k 其中,zk为时刻k的观测数据,h(·)是非线性观测函数,表示从状态向量到观测数据的关系,vk是观测噪声。Among them, zk is the observation data at time k, h(·) is the nonlinear observation function, which represents the relationship from the state vector to the observation data, and vk is the observation noise. 4.根据权利要求2所述的无人机自主目标搜寻方法,其特征在于,所述S2中无人机飞行控制使用PID控制,以确保从当前位置飞向目标地点,PID控制公式如下:4. The autonomous target search method for unmanned aerial vehicles according to claim 2 is characterized in that the flight control of the unmanned aerial vehicle in S2 uses PID control to ensure that it flies from the current position to the target location, and the PID control formula is as follows: 其中,e(t)为当前位置与目标位置之间的误差,Kp,Ki,Kd为PID控制器的比例、积分、微分增益。Wherein, e(t) is the error between the current position and the target position, Kp , Ki , Kd are the proportional, integral, and differential gains of the PID controller. 5.根据权利要求2所述的无人机自主目标搜寻方法,其特征在于,所述S3通过调用自然语言处理模块Mn接收自然语言形式的任务目标gt和包含历史搜索信息ht和场景地图信息ot的状态信息st,生成包含n个任务步骤和执行顺序的执行计划自然语言处理模块公式如下:5. The autonomous target search method for unmanned aerial vehicles according to claim 2 is characterized in that the step S3 receives the task target g t in natural language form and the state information s t including the historical search information h t and the scene map information o t by calling the natural language processing module M n , and generates an execution plan including n task steps and execution order. The formula of the natural language processing module is as follows: pt=Mn(gt,st)。 pt = Mn ( gt , st ). 6.根据权利要求2所述的无人机自主目标搜寻方法,其特征在于,所述S4任务规划与控制模块接受执行计划pt后,进入动作执行状态,根据任务框架中的步骤,调用路径规划模块,使无人机飞向指定搜索地点,A*路径规划算法公式如下:6. The autonomous target search method for unmanned aerial vehicles according to claim 2 is characterized in that after the S4 task planning and control module receives the execution plan pt , it enters the action execution state, calls the path planning module according to the steps in the task framework, and makes the unmanned aerial vehicle fly to the designated search location. The A* path planning algorithm formula is as follows: f(n)=g(n)+h(n)f(n)=g(n)+h(n) 其中,f(n)是当前节点的总估计代价,g(n)是从起始节点到当前节点的实际代价,h(n)是从当前节点到目标节点的估计代价。Among them, f(n) is the total estimated cost of the current node, g(n) is the actual cost from the start node to the current node, and h(n) is the estimated cost from the current node to the target node. 7.根据权利要求2所述的无人机自主目标搜寻方法,其特征在于,所述S5中路径生成公式如下:7. The autonomous target search method for unmanned aerial vehicles according to claim 2, characterized in that the path generation formula in S5 is as follows: x(t)=r·coS(ωt)x(t)=r·coS(ωt) y(t)=r·sin(ωt)y(t)=r·sin(ωt) 其中,r是半径,ω是角度速度,t是时间。Where r is the radius, ω is the angular velocity, and t is the time. 8.根据权利要求2所述的无人机自主目标搜寻方法,其特征在于,所述S6中目标检测模块Md给出目标检测概率以及目标在相机坐标系中的位置,公式如下:8. The autonomous target search method for unmanned aerial vehicles according to claim 2, characterized in that the target detection module Md in S6 gives the target detection probability and the position of the target in the camera coordinate system, and the formula is as follows: ximage,yimage,Pobj=Md(I)x image , y image , P obj =M d (I) 其中,(ximage,yimage)代表目标在图像中的坐标,Pobj表示目标存在的概率,I是输入图像。Among them, (x image , y image ) represents the coordinates of the target in the image, P obj represents the probability of the target existence, and I is the input image. 9.根据权利要求2所述的无人机自主目标搜寻方法,其特征在于,所述S7中相机内参矩阵将像素坐标转换为相机坐标系中的归一化坐标,公式如下:9. The autonomous target search method for unmanned aerial vehicles according to claim 2, characterized in that the camera intrinsic parameter matrix in S7 converts pixel coordinates into normalized coordinates in the camera coordinate system, and the formula is as follows: 这里的K-1是相机内参矩阵的逆矩阵,相机的外参矩阵来将相机坐标系中的三维坐标转换为世界坐标系中的三维坐标,公式如下:Here K -1 is the inverse matrix of the camera's intrinsic parameter matrix. The camera's extrinsic parameter matrix is used to convert the three-dimensional coordinates in the camera coordinate system into the three-dimensional coordinates in the world coordinate system. The formula is as follows: 其中,R是旋转矩阵,t是平移向量。Where R is the rotation matrix and t is the translation vector. 10.根据权利要求2所述的无人机自主目标搜寻方法,其特征在于,所述S9中将当前任务目标和执行动作添加入历史搜索信息ht+1,然后再次通过调用自然语言处理模块接收自然语言形式的任务目标gt+1和包含历史搜索信息ht+1和场景地图信息ot+1的状态信息st+1,生成下一轮包含n个任务步骤和执行顺序的执行计划跳转到步骤4。10. The autonomous target search method for unmanned aerial vehicles according to claim 2, characterized in that, in said S9, the current task target and execution action are added to the historical search information h t+1 , and then the task target g t+1 in natural language form and the state information s t+1 including the historical search information h t+1 and the scene map information o t +1 are received again by calling the natural language processing module, and the next round of execution plan including n task steps and execution order is generated. Skip to step 4.
CN202411936066.1A 2024-12-26 2024-12-26 A UAV autonomous target search system and method based on large language model Pending CN119759083A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202411936066.1A CN119759083A (en) 2024-12-26 2024-12-26 A UAV autonomous target search system and method based on large language model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202411936066.1A CN119759083A (en) 2024-12-26 2024-12-26 A UAV autonomous target search system and method based on large language model

Publications (1)

Publication Number Publication Date
CN119759083A true CN119759083A (en) 2025-04-04

Family

ID=95192173

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202411936066.1A Pending CN119759083A (en) 2024-12-26 2024-12-26 A UAV autonomous target search system and method based on large language model

Country Status (1)

Country Link
CN (1) CN119759083A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120066113A (en) * 2025-04-24 2025-05-30 中航(成都)无人机系统股份有限公司 Unmanned aerial vehicle heuristic path planning method, equipment and medium based on large language model

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117151338A (en) * 2023-09-08 2023-12-01 安徽大学 Multi-unmanned aerial vehicle task planning method based on large language model
CN117723058A (en) * 2023-10-31 2024-03-19 中国人民解放军32806部队 Autonomous target searching method and system of unmanned aerial vehicle
CN117993422A (en) * 2023-12-27 2024-05-07 南京航空航天大学 Robot character stream distribution method and logistics distribution robot based on natural language
CN118502456A (en) * 2024-04-01 2024-08-16 浙江大学 A method and system for realizing perception and control of unmanned aerial vehicle flight assistant
WO2024182721A1 (en) * 2023-03-01 2024-09-06 Google Llc Using scene understanding to generate context guidance in robotic task execution planning
CN118822193A (en) * 2024-07-16 2024-10-22 智洋创新科技股份有限公司 A planning method, system and device for complex tasks of substation drone inspection
CN119090307A (en) * 2024-08-29 2024-12-06 南京智子跳动科技有限公司 A method for autonomous mission planning and execution of unmanned equipment based on a large model agent framework

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024182721A1 (en) * 2023-03-01 2024-09-06 Google Llc Using scene understanding to generate context guidance in robotic task execution planning
CN117151338A (en) * 2023-09-08 2023-12-01 安徽大学 Multi-unmanned aerial vehicle task planning method based on large language model
CN117723058A (en) * 2023-10-31 2024-03-19 中国人民解放军32806部队 Autonomous target searching method and system of unmanned aerial vehicle
CN117993422A (en) * 2023-12-27 2024-05-07 南京航空航天大学 Robot character stream distribution method and logistics distribution robot based on natural language
CN118502456A (en) * 2024-04-01 2024-08-16 浙江大学 A method and system for realizing perception and control of unmanned aerial vehicle flight assistant
CN118822193A (en) * 2024-07-16 2024-10-22 智洋创新科技股份有限公司 A planning method, system and device for complex tasks of substation drone inspection
CN119090307A (en) * 2024-08-29 2024-12-06 南京智子跳动科技有限公司 A method for autonomous mission planning and execution of unmanned equipment based on a large model agent framework

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
CUI JINQIANG, ETAL: "TPML:Task Planning for Multi-UAV System with Large Language Models", 2024 IEEE18TH INTERNATIONAL CONFERENCE ON CONTROL & AUTOMATION, 13 November 2024 (2024-11-13), pages 886 - 891 *
秦龙: "基于大语言模型的复杂任务自主规划处理框架", 自动化学报, vol. 50, no. 4, 30 April 2024 (2024-04-30), pages 862 - 870 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120066113A (en) * 2025-04-24 2025-05-30 中航(成都)无人机系统股份有限公司 Unmanned aerial vehicle heuristic path planning method, equipment and medium based on large language model
CN120066113B (en) * 2025-04-24 2025-07-29 中航(成都)无人机系统股份有限公司 Unmanned aerial vehicle heuristic path planning method, equipment and medium based on large language model

Similar Documents

Publication Publication Date Title
CN110969655B (en) Method, device, equipment, storage medium and vehicle for detecting parking space
Padhy et al. Deep neural network for autonomous UAV navigation in indoor corridor environments
CN102628690B (en) Task collaborative visual navigation method of two unmanned aerial vehicles
Fu et al. Monocular visual-inertial SLAM-based collision avoidance strategy for Fail-Safe UAV using fuzzy logic controllers: comparison of two cross-entropy optimization approaches
Chen et al. Robot navigation with map-based deep reinforcement learning
CN112378397A (en) Unmanned aerial vehicle target tracking method and device and unmanned aerial vehicle
Suzuki Recent researches on innovative drone technologies in robotics field
CN115494866A (en) A method and system for multi-UAV global and local path intelligent planning
CN112596071A (en) Unmanned aerial vehicle autonomous positioning method and device and unmanned aerial vehicle
CN117685953A (en) UWB and vision fusion positioning method and system for multi-UAV collaborative positioning
CN119759083A (en) A UAV autonomous target search system and method based on large language model
CN112380933A (en) Method and device for identifying target by unmanned aerial vehicle and unmanned aerial vehicle
CN118672284A (en) Unmanned aerial vehicle autonomous environment sensing, path planning and dynamic landing method and system
Xing et al. Autonomous landing of a micro aerial vehicle on a moving platform using a composite landmark
CN117270565A (en) Airborne autonomous sensing and flight system based on vision
CN117745765A (en) An indoor drone target tracking system based on depth vision
CN118502457A (en) Track planning method, device and autonomous system
CN115267725B (en) A mapping method, device, and storage medium based on single-line lidar
Aspragkathos et al. Event-triggered image moments predictive control for tracking evolving features using uavs
Li et al. UAV obstacle avoidance by human-in-the-loop reinforcement in arbitrary 3D environment
CN116225022A (en) Optimizing the Heterogeneous Cooperative Roundup Method of UAV and Unmanned Ship Based on Pigeon Thinking
CN115903880A (en) Unmanned aerial vehicle autonomous image navigation and obstacle avoidance method based on improved reinforcement learning
CN119002514A (en) Unmanned aerial vehicle autonomous inspection method and device considering inspection requirement emergency degree
US20250093876A1 (en) Robot local planner selection
Chen et al. Vision-based formation control of multiple UAVs with event-triggered integral sliding mode control

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination