CN117901892A - Automatic driving control method, device, electronic device and storage medium - Google Patents

Abstract

The application discloses an automatic driving control method, an automatic driving control device, electronic equipment and a storage medium, and relates to the technical field of vehicles; the automatic driving control method includes the steps of: acquiring in-vehicle and out-of-vehicle monitoring data, and extracting current cognitive layer characteristics from the in-vehicle and out-of-vehicle monitoring data through the cognitive layer; inputting the current cognitive layer characteristics into the decision layer, and determining automatic driving control parameters; and generating driving behavior interpretation according to the current cognitive layer characteristics and the automatic driving control parameters. The application improves the interpretability of the end-to-end perception decision framework, reduces the negative moods of users such as confusion, panic, anxiety and the like, and improves the experience of the users.

Description

Automatic driving control method, device, electronic device and storage medium

Technical Field

The present application relates to the field of vehicle technology, and in particular to an automatic driving control method, device, electronic device and storage medium.

Background Art

With the continuous development of science and technology, continuous breakthroughs in computing power, and the rapid development of autonomous driving, autonomous driving can provide people with a safer, more convenient and efficient way of travel, while also having a profound impact on the entire transportation system and urban planning.

The mainstream technical solutions for autonomous driving systems are mainly divided into two types: rule-based modular architecture and data-driven end-to-end perception decision architecture. Although the rule-based modular architecture has been put into mass production due to its advantages such as low computing power requirements, simple deployment, and explainable processes, it is difficult to achieve full scene coverage, and the results of modular solutions are not the global optimal solution. The end-to-end solution uses raw sensor data as input and generates planning and/or low-level control actions as output. It can perform joint and global optimization, and the accuracy of autonomous driving decisions is relatively high.

However, the end-to-end perception decision-making architecture has poor interpretability. Therefore, when controlling the vehicle to perform sudden behaviors, it is easy for users riding in the autonomous driving vehicle to not understand the driving behavior of the autonomous driving vehicle and experience negative emotions such as confusion, panic, and anxiety, resulting in a poor user experience.

Summary of the invention

The main purpose of this application is to provide an autonomous driving control method, device, electronic device and storage medium, aiming to solve the technical problem of poor interpretability of end-to-end perception decision-making solutions in related technologies.

To achieve the above-mentioned purpose, the present application provides an automatic driving control method, wherein the automatic driving control method adopts a cognitive decision model, wherein the cognitive decision model includes a cognitive layer and a decision layer; the automatic driving control method includes the following steps:

Acquire vehicle interior and exterior monitoring data, and extract current cognitive layer features from the vehicle interior and exterior monitoring data through the cognitive layer;

Inputting the current cognitive layer features into the decision layer to determine the automatic driving control parameters;

Generate a driving behavior explanation based on the current cognitive layer characteristics and the autonomous driving control parameters.

The present application also provides an automatic driving control device, on which a cognitive decision model is deployed, the cognitive decision model includes a cognitive layer and a decision layer, and the automatic driving control device includes:

A cognitive module, used for acquiring in-vehicle and out-vehicle monitoring data, and extracting current cognitive layer features from the in-vehicle and out-vehicle monitoring data through the cognitive layer;

A decision module, used for inputting the current cognitive layer features into the decision layer to determine the automatic driving control parameters;

An explanation module is used to generate a driving behavior explanation based on the current cognitive layer characteristics and the automatic driving control parameters.

The present application also provides an electronic device, which is a physical device, and includes: a memory, a processor, and a program of the autonomous driving control method stored in the memory and executable on the processor. When the program of the autonomous driving control method is executed by the processor, the steps of the autonomous driving control method as described above can be implemented.

The present application also provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores a program for implementing an autonomous driving control method. When the program of the autonomous driving control method is executed by a processor, the steps of the autonomous driving control method as described above are implemented.

The present application provides an automatic driving control method, device, electronic device and storage medium, wherein the automatic driving control method adopts a cognitive decision model, wherein the cognitive decision model includes a cognitive layer and a decision layer. First, by acquiring the monitoring data inside and outside the vehicle, the current cognitive layer features are extracted from the monitoring data inside and outside the vehicle through the cognitive layer, thereby realizing the recognition of the current situation of the automatic driving vehicle; then, by inputting the environmental cognitive features and the user demand features into the decision layer, the automatic driving control parameters are determined, thereby realizing accurate automatic driving decisions and determining automatic driving control parameters based on the recognition of the current situation; then, by generating a driving behavior explanation based on the current cognitive layer features and the automatic driving control parameters, the purpose of explaining the automatic driving decision is realized. Since the automatic driving control parameters are determined based on the cognitive layer features, there is a certain causal relationship between the automatic driving control parameters and the cognitive layer features, and the reason why the user cannot understand the driving behavior of the automatic driving vehicle is that before the user feels the sudden driving behavior of the automatic driving vehicle, the user's attention may be focused on different matters from the driving vehicle. Therefore, when the user feels the sudden driving behavior of the automatic driving vehicle, what he feels is the final decision made by the automatic driving vehicle, and he may not see the factors affecting the decision, and thus cannot infer the reason for its decision, so he will have negative emotions such as doubt, panic, and anxiety. Therefore, this application interprets the autonomous driving control parameters based on the cognitive layer characteristics, generates a driving behavior explanation, can actively interact with the user, explain the reasons for the driving behavior to the user, and eliminate the user's doubts about the driving behavior. Therefore, it overcomes the poor interpretability of the end-to-end perception decision architecture. Therefore, when controlling the vehicle to perform sudden behaviors, it is easy to cause users riding in the autonomous driving vehicle to be unable to understand the driving behavior of the autonomous driving vehicle and generate negative emotions such as doubt, panic, and uneasiness, and the technical defects of poor user experience. It improves the interpretability of the end-to-end perception decision architecture, reduces the user's negative emotions such as doubt, panic, and uneasiness caused by the unknown, and improves the user experience.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

FIG1 is a schematic flow chart of a first embodiment of an automatic driving control method of the present application;

FIG2 is a schematic diagram of a structure of a possible implementation method of a cognitive decision-making model in an embodiment of the present application;

FIG. 3 is a flow chart of a second embodiment of the automatic driving control method of the present application.

FIG4 is a schematic diagram of a flow chart of a third embodiment of the automatic driving control method of the present application;

FIG5 is a schematic diagram of the structure of an automatic driving control device in an embodiment of the present application;

FIG6 is a schematic diagram of the device structure of the hardware operating environment involved in the automatic driving control method in an embodiment of the present application.

The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

DETAILED DESCRIPTION

In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

Embodiment 1

The present application provides an automatic driving control method. Referring to FIG. 1 , in the first embodiment of the automatic driving control method of the present application, the automatic driving control method adopts a cognitive decision model, and the cognitive decision model includes a cognitive layer and a decision layer. The automatic driving control method includes the following steps:

Step S10, acquiring vehicle interior and exterior monitoring data, and extracting current cognitive layer features from the vehicle interior and exterior monitoring data through the cognitive layer;

The executor of the method of this embodiment can be an autonomous driving control device, or an autonomous driving control terminal device or server. This embodiment takes an autonomous driving control device as an example. The autonomous driving control device can be integrated in a terminal device such as a vehicle with data processing function, an on-board terminal, a smart phone, a computer, etc.

In this embodiment, it should be noted that the autonomous driving control method is applied to a cognitive decision-making model, which is used to achieve end-to-end driving behavior control of the autonomous driving vehicle. The cognitive decision-making model uses an attention mechanism and can be a deep learning model based on a Transformer structure or other deep learning neural network models using an attention mechanism. The specific design can be based on actual conditions, and this embodiment does not impose any restrictions on this.

The cognitive decision model includes at least a cognitive layer and a decision layer, wherein the cognitive layer is used to extract features, fuse, encode and infer the acquired in-vehicle and out-vehicle monitoring data to obtain the risk features required by the decision layer; the decision layer is used to plan the driving behavior of the autonomous driving vehicle at the next time step based on the risk features transmitted by the cognitive layer, so that the autonomous driving vehicle can avoid risks and drive safely to the destination.

The in-vehicle and out-vehicle monitoring data refers to the data required for automatic driving control that can be obtained, including in-vehicle monitoring data and cockpit monitoring data. The in-vehicle and out-vehicle monitoring data can be one or more modes of image data, text data, sound data, signal data, etc. Exemplarily, the in-vehicle monitoring data can be in-vehicle image data collected by a camera, in-vehicle sound data collected by a microphone, signal light countdown data obtained by a communication module, etc. Exemplarily, the cockpit monitoring data can be vehicle driving data such as vehicle speed and acceleration obtained from a vehicle controller or a controller area network bus, or can be cockpit image data collected by a camera, driver image data, cockpit sound data collected by a microphone, text data obtained by converting the collected voice, text data extracted from the collected image, etc. These in-vehicle and out-vehicle monitoring data can be collected in real time, so they can carry real-time information at the current moment about the status of other traffic participants, the status of the driver, the road condition, and other information that may change.

Different models can be used to extract features from in-vehicle and out-of-vehicle monitoring data of different modalities. For example, convolutional neural networks can be used to extract features from image data, and long short-term memory networks can be used to extract features from time series data. Different models can also be used for feature processing to extract different risk features required by the decision-making layer. Therefore, multiple models can be deployed in the cognitive layer for feature processing. The specific model structure and model type can be determined based on actual conditions, and this embodiment does not impose any restrictions on this.

The cognitive layer features may include at least one of environmental cognitive features and user demand features.

The environmental cognition feature is used to characterize the cognition of the autonomous driving vehicle to the current environment, such as the cognition of traffic facilities, ground obstacles, the driving status of the vehicle, other traffic participants, the risk of the scene, the risk items of the scene, etc. The main purpose of the behavior control of the autonomous driving vehicle is to make the autonomous driving vehicle reach the destination safely, that is, to control the vehicle to drive to the destination while avoiding risks. In an implementable manner, the autonomous driving control method can be to predict the high-risk driving area and/or low-risk driving area of the autonomous driving vehicle in the next time step, so as to plan the driving behavior of the autonomous driving vehicle to avoid the high-risk driving area or pass through the low-risk driving area in the next time step. The environmental cognition feature is an objective representation of the current environment of the autonomous driving vehicle. Therefore, the more accurate the extracted environmental cognition feature is, the more accurate the current risk judgment of the autonomous driving vehicle is, and the higher the safety of the autonomous driving control is. The environmental cognition features may include scene cognition features, map features, risk target detection and tracking features, risk target motion features, low-risk driving area features, etc., and may also include other features. The cognitive layer may be a multi-layer structure, that is, the input of any cognitive layer can be the output of other cognitive layers, and the output of any cognitive layer can also be used as the input of other cognitive layers, thereby realizing the fusion and reasoning of multiple features, capturing the correlation and dependency between the various features extracted by the cognitive layer, and achieving a more comprehensive understanding of the current actual driving situation and deeper logical reasoning.

The user demand feature is used to characterize the needs of users riding in the autonomous driving vehicle, such as the user's needs for the destination, the user's needs for the route to the destination, the user's needs for being in a hurry, the user's needs for smooth driving to relieve motion sickness, etc. The user demand feature can capture user demand information by detecting user operations, collecting user images, etc., and then extract user demand features therefrom. For example, the user image can be collected by a camera. When the image recognition technology is used to identify that the user is currently in a sleeping state in the user image, the user's sleeping state usually implies the user's need for the vehicle to drive smoothly. For example, the user's voice information can also be collected by a microphone. When it is recognized that the user says "drive faster", it can be known that the user has a need to accelerate.

The cognitive layer can adopt an attention mechanism. In an implementable manner, the cognitive layer can adopt a transformer structure. The attention mechanism can be used to perform feature extraction, fusion, encoding and reasoning on various types of vehicle monitoring data of various modalities, extract the real-time information of the information that may change at the current moment, and form an information sequence with the historical information of a period of time before the current moment, and capture the correlation and dependency between different types of information, perform feature reasoning, and extract risk features with deep logical information for the decision-making layer to perform autonomous driving control on the autonomous driving vehicle. In this way, the mutual influence between the information that may change, multiple risk items and user needs can be fully considered in the process of predicting the driving risk of the autonomous driving vehicle, and the prediction accuracy of the driving risk of the autonomous driving vehicle can be improved, so that the autonomous driving vehicle can avoid more risks, and at the same time better meet user needs and improve the safety and comfort of autonomous driving control. Among them, the correlation and dependency between different types of information may have a greater impact on the autonomous driving control of the autonomous driving vehicle. For example, traffic participants may interact with each other, traffic participants may interact with traffic facilities, and traffic participants may interact with scenes. For example, at an intersection, the vehicle is traveling in the east-west direction, and there is a second vehicle traveling in the north-south direction and pedestrians walking in the east-west direction nearby. Assuming that the interaction between traffic participants is not considered, the vehicle and the second vehicle will not collide at the current speed. However, if the second vehicle slows down or stops to avoid pedestrians, it may cause a collision between the vehicle and the second vehicle. Therefore, considering the interaction between traffic participants, the vehicle still needs to slow down or plan other driving routes; and if it is assumed that the second vehicle is in a turning lane, considering the interaction between traffic participants and traffic facilities, if the turning trajectory of the second vehicle does not intersect with the driving trajectory of the vehicle, the vehicle does not need to slow down; in addition, considering the interaction between traffic participants and the scene, in the current scenario of passing through an intersection, due to the complexity of the intersection scene, the risk factor of people and vehicles at the intersection can be increased, so it may be necessary to control the vehicle to slow down.

As an example, step S10 includes: collecting sensor data at the current moment as vehicle interior and exterior monitoring data through one or more sensors provided on the autonomous driving vehicle, and also communicating with external devices through a communication module on the autonomous driving vehicle to obtain external data provided by the external devices as vehicle interior and exterior monitoring data, such as traffic light signals, traffic accident information, and roadside monitoring data collected by roadside monitoring equipment. The vehicle interior and exterior monitoring data are input into the cognitive layer, and multi-level progressive feature processing such as feature extraction, feature fusion, feature encoding, and feature reasoning is performed to obtain cognitive layer features.

Optionally, the cognitive layer includes a user feature extraction model; the current cognitive layer features include user demand features, wherein the user demand features include user implicit demand features and user explicit demand features; the in-vehicle and out-of-vehicle monitoring data include user operation data and user status data;

The step of extracting current cognitive layer features from the vehicle internal and external monitoring data through the cognitive layer includes:

The user feature extraction model is used to extract user explicit demand features from the user operation data, and/or the user feature extraction model is used to extract user implicit demand features from the user status data.

In this embodiment, it should be noted that user operation data refers to data generated by users actively interacting with the autonomous vehicle, such as the user issuing voice commands to the autonomous vehicle, or actively clicking buttons on the display interface of the vehicle terminal, etc. User status data refers to user-related information actively collected by the autonomous vehicle. For example, when the user is asleep, the user does not actively interact with the autonomous vehicle. At this time, the user's image data can be collected through the camera to identify the user's sleeping state. For another example, when the user is calling a friend to express the state of nervousness and anxiety about riding in the autonomous vehicle, the user does not actively interact with the autonomous vehicle. At this time, the user's voice data can be collected through the microphone to identify the user's nervousness and anxiety.

As an example, the user operation data can be input into the user feature extraction model to perform multi-level progressive feature processing such as feature extraction, feature fusion, feature encoding, and feature reasoning to obtain user display demand features. The user status data can also be input into the user feature extraction model to perform multi-level progressive feature processing such as feature extraction, feature fusion, feature encoding, and feature reasoning to obtain user implicit demand features.

In this embodiment, the user can actively interact with the autonomous driving vehicle and actively put forward demands. The autonomous driving vehicle can also respond to the user's interactive operations, meet the user's needs, and improve the interactivity between the autonomous driving vehicle and the user. The autonomous driving vehicle can also actively discover user needs and actively make autonomous driving behaviors that better meet user needs, thereby reducing user discomfort.

Optionally, the cognitive layer includes a scene cognitive model, a map construction model, a risk target detection and tracking model, and a motion prediction model; the current cognitive layer features include environmental cognitive features, wherein the environmental cognitive features include scene cognitive features, map features, risk target detection and tracking features, and risk target motion features;

The step of extracting current cognitive layer features from the vehicle internal and external monitoring data through the cognitive layer includes:

Step A10, extracting scene recognition features from the in-vehicle and out-vehicle monitoring data by using the scene recognition model, extracting map features from the in-vehicle and out-vehicle monitoring data by using the map building model, and extracting risk target detection and tracking features from the in-vehicle and out-vehicle monitoring data by using the risk target detection and tracking model;

Step A20, using the attention mechanism through the motion prediction model, based on the scene recognition features, the map features and the risk target detection and tracking features, performs risk target motion prediction to obtain risk target motion features.

In this embodiment, it should be noted that the cognitive layer includes a scene cognitive model, a map construction model, a risk target detection and tracking model, and a motion prediction model, wherein a risk target refers to an object that may cause risk to the driving of an autonomous vehicle, and may be one or more of a traffic participant, a traffic infrastructure, etc. The scene cognitive features, map features, and risk target detection and tracking features output by the scene cognitive model, the map construction model, and the risk target detection and tracking model all have a certain influence on predicting the motion of the risk target at the next time step, and there is a certain correlation and dependency between the scene cognitive features, the map features, and the risk target detection and tracking features, so they can all be used as inputs to the motion prediction model. The motion prediction model uses an attention mechanism to capture the correlation and dependency between these features, and further feature fusion, encoding, and reasoning are performed to determine the risk target motion features of the risk target.

The risk features include scene recognition features, map features, risk target detection and tracking features, and risk target motion features. Scene recognition features are used to characterize the risk recognition of the scene in which the vehicle is currently located, such as attention distribution, semantic understanding of risk items in the scene, etc. Map features are used to characterize map information such as ground facilities and road elements in the scene in which the vehicle is located. Risk target detection and tracking features are used to characterize the identification of risk targets and the location information of risk targets within a period of time before the current moment. Risk target motion features are used to characterize the motion information of risk targets within a period of time after the current moment. The motion information may include location information, trajectory information, motion intention, motion trend, etc.

As an example, the steps A10-A20 include: inputting the vehicle interior and exterior monitoring data into the scene recognition model, performing multi-level progressive feature processing such as feature extraction, feature fusion, feature coding, and feature reasoning, and extracting scene recognition features; synchronously, inputting the vehicle interior and exterior monitoring data into the map construction model, performing multi-level progressive feature processing such as feature extraction, feature fusion, feature coding, and feature reasoning, and extracting map features; synchronously, inputting the vehicle interior and exterior monitoring data into the risk target detection and tracking model, performing multi-level progressive feature processing such as feature extraction, feature fusion, feature coding, and feature reasoning, and extracting risk target detection and tracking features. Further, the scene recognition features, the map features, and the risk target detection and tracking features are input into the motion prediction model, and the motion prediction model is used to capture the correlation and dependency between the scene recognition features, the map features, and the risk target detection and tracking features by using the attention mechanism. Based on these correlations and dependencies, risk target operation prediction can be performed more accurately, and the risk target motion features of at least one risk target can be determined.

In an implementable manner, the risk target detection and tracking feature includes at least one risk target detection and tracking sub-feature; the motion prediction model includes an attention layer, a multi-layer perceptron layer, and a prediction layer;

The step of using the attention mechanism through the motion prediction model to predict the motion of the risk target based on the scene recognition features, the map features and the risk target detection and tracking features to obtain the motion features of the risk target includes:

Step A21, inputting the scene recognition feature, the map feature and the risk target detection and tracking feature into the attention layer, using the attention mechanism to determine the risk target interaction feature between each of the risk target detection and tracking sub-features, the scene interaction feature between each of the risk target detection and tracking sub-features and the scene recognition feature, and the map interaction feature between each of the risk target detection and tracking sub-features and the map feature;

Step A22, inputting the risk target interaction feature, the scene interaction feature and the map interaction feature into a multi-layer perceptron layer to obtain a scene risk feature and a motion query feature;

Step A23, inputting the scene risk feature and the motion query feature into the prediction layer, performing risk target motion prediction, and obtaining risk target motion features.

In this embodiment, it should be noted that for complex scenarios, multiple risk targets may be involved. When multiple risk targets are identified, the risk target detection and tracking model can identify multiple risk targets and track multiple risk targets simultaneously, so that the risk target detection and tracking features obtained will contain relevant information of multiple risk targets, that is, the risk target detection and tracking features include at least one risk target detection and tracking sub-feature. In this case, motion prediction can be performed for each risk target separately. By adopting the attention mechanism, the correlation and dependency between each risk target and other risk targets, the correlation and dependency between each risk target and map elements, and the correlation and dependency between each risk target and scene cognition can be captured. Based on these correlations and dependencies, risk target operation prediction can be performed more accurately for each risk target, and the risk target motion characteristics of each risk target can be determined.

The scene risk features and motion query features represent the risks that already exist in the current scene. Based on the scene risk features and motion query features, the motion information of the risk target after the current moment can be further predicted.

As an example, the steps A21-A23 include: inputting the scene recognition feature, the map feature and the risk target detection and tracking feature into the attention layer, using the attention mechanism to capture the dependency between each of the risk target detection and tracking sub-features to determine the mutual influence between each risk target and other risk targets, generating a risk target interaction feature, capturing the dependency between each of the risk target detection and tracking sub-features and the scene recognition feature to determine the mutual influence between each risk target and scene recognition, generating a scene interaction feature, capturing the dependency between each of the risk target detection and tracking sub-features and the map feature to determine the mutual influence between each risk target and the map element, generating a scene interaction feature. Then, the risk target interaction feature, the scene interaction feature and the map interaction feature are spliced and input into the multi-layer perceptron layer to obtain the scene risk feature and the motion query feature. Then, the scene risk feature and the motion query feature are input into the prediction layer to perform risk target motion prediction on each risk target to obtain the risk target motion feature of each risk target.

In one feasible manner, the multi-layer perceptron layer can also decode scene risk features and motion query features, and output the decoded scene risk prediction information and motion information of the risk target as intermediate results for user viewing.

Step S20, inputting the current cognitive layer features into the decision layer to determine the automatic driving control parameters;

In this embodiment, it should be noted that the autonomous driving control parameters may include driving trajectory, vehicle control parameters, etc., wherein the driving trajectory refers to the trajectory of the autonomous driving vehicle a period of time after the current moment, and the autonomous driving vehicle can be controlled to drive based on the driving trajectory. The vehicle control parameters include speed, accelerator pedal opening, steering wheel angle, etc.

As an example, the step S20 includes: inputting the environmental cognition characteristics and the user demand characteristics into the decision layer to determine the autonomous driving control parameters of the autonomous driving vehicle at the current moment or within a period of time after the current moment.

Step S30, generating a driving behavior explanation based on the current cognitive layer characteristics and the automatic driving control parameters.

In this embodiment, it should be noted that the autonomous driving control parameters may include driving trajectory, vehicle control parameters, etc., wherein the driving trajectory refers to the trajectory of the autonomous driving vehicle a period of time after the current moment, and the autonomous driving vehicle can be controlled to drive based on the driving trajectory. The vehicle control parameters include speed, accelerator pedal opening, steering wheel angle, etc.

As an example, step S30 includes: decoding information about factors affecting the autonomous driving control parameters output by the decision-making layer from the current cognitive layer features, fusing the autonomous driving control parameters and the information about the factors corresponding to the autonomous driving control parameters, and generating a driving behavior explanation.

In an operative manner, after the step of generating the driving behavior explanation according to the current cognitive layer characteristics and the automatic driving control parameters, the step may further include: outputting the behavior explanation information. That is, the driving behavior explanation is outputted in the form of images, sounds, texts, etc., so that the user can understand the reasons for the driving behavior. For example, if an automatic driving vehicle needs to stop at a red light intersection, the automatic driving control parameter may be to stop, and the information of the influencing factors may be the location information of the red light at the intersection ahead and the stop line. After aggregating the two, a driving behavior explanation "Because the light ahead is red, the vehicle stops before the stop line" can be generated, and a voice broadcast of "Because the light ahead is red, the vehicle stops before the stop line" can be outputted through a speaker.

In one feasible manner, the driving behavior explanation may be output before the autonomous driving vehicle executes the autonomous driving control parameters. This allows the user to predict the driving behavior that the vehicle is about to perform and will not panic due to the sudden driving behavior of the autonomous driving vehicle.

Optionally, the autonomous driving control parameter includes a plurality of autonomous driving control sub-parameters;

The step of generating a driving behavior explanation according to the current cognitive layer characteristics and the automatic driving control parameters comprises:

Step S31, determining the target decision feature corresponding to each of the autonomous driving control sub-parameters according to the current cognitive layer feature;

Step S32, aggregating each of the autonomous driving control sub-parameters with the corresponding target decision features to obtain a step-by-step explanation of the driving behavior;

Step S33, aggregating the step-by-step explanations of the driving behavior to obtain a driving behavior explanation.

In this embodiment, it should be noted that the driving behavior may be achieved in one step or may require multiple steps to achieve. For driving behaviors that require multiple steps to achieve, the automatic driving control sub-parameters corresponding to each step can be determined respectively, and then arranged in chronological order to form an automatic driving control parameter sequence. For example, for the driving behavior of accelerating lane change, the automatic driving control parameter sequence can be {τ ₁ (turn on the right turn signal), τ ₂ (acceleration), τ ₃ (change lane to the right)}, where τ represents the automatic driving control sub-parameter. For driving behaviors that require multiple steps to achieve, each step has its execution reasons and key influencing factors, so each step can be explained more accurately step by step.

As an example, steps S31-S33 include: decoding from the current cognitive layer features information on influencing factors that affect the output of each of the autonomous driving control sub-parameters at the decision layer, splicing the information on the influencing factors corresponding to each of the autonomous driving control sub-parameters into a target decision feature, aggregating each of the autonomous driving control sub-parameters with their respective corresponding target decision features, generating a step-by-step explanation of driving behavior, and further aggregating each of the step-by-step explanations of driving behavior into a driving behavior explanation.

In one implementable manner, the target decision feature may include a shallow target decision feature and a deep target decision feature, and the step of determining the target decision feature corresponding to each of the autonomous driving control sub-parameters according to the current cognitive layer feature may include: determining the driving behavior step-by-step explanation type corresponding to each of the autonomous driving control sub-parameters according to the current cognitive layer feature through a classification model or by adopting an attention mechanism, etc., wherein the driving behavior step-by-step explanation type includes a shallow explanation and a deep explanation; when it is determined that the driving behavior step-by-step explanation type is a shallow explanation, determining the shallow target decision feature corresponding to each of the autonomous driving control sub-parameters according to the current cognitive layer feature; when it is determined that the driving behavior step-by-step explanation type is a deep explanation, determining the deep target decision feature corresponding to each of the autonomous driving control sub-parameters according to the current cognitive layer feature. Among them, shallow explanation refers to the spatial explanation of driving behavior, and deep explanation refers to the spatiotemporal explanation of driving behavior. The series of steps corresponding to each driving behavior have different importance for the explanation of driving behavior. For example, in the driving behavior of accelerating and changing lanes, since lane changing usually requires deceleration before changing lanes, when accelerating and changing lanes, users will have great doubts about the "acceleration" step; and, for autonomous driving vehicles encountering different actual situations in different scenarios, users' needs for driving behavior explanations are also different. For example, for the same lane change, the driving behavior of decelerating and changing lanes is in line with conventional cognition, so it is not easy to cause doubts, and a shallow explanation is sufficient. The driving behavior of accelerating and changing lanes usually occurs in special scenarios, so a deep explanation of the cause and effect may be required.

Optionally, the step of aggregating the step-by-step explanations of the driving behaviors to obtain the driving behavior explanation comprises:

Step S331, obtaining the step-by-step explanation weight corresponding to each of the autonomous driving control sub-parameters;

Step S332: performing weighted aggregation on each of the step-by-step explanations of the driving behavior based on the weight of each of the step-by-step explanations to obtain a driving behavior explanation.

In this embodiment, it should be noted that the actual needs of the user for the explanation of different steps are different. For example, for the driving behavior of accelerating lane change, the automatic driving control parameter sequence {τ ₁ (turn on the right turn signal), τ ₂ (accelerate), τ 3 (accelerate), τ ₄ (accelerate), τ 5 (accelerate), τ 6 (accelerate), τ 7 (accelerate), τ 8 (accelerate), τ 9 (accelerate), In the example (changing lanes to the right), the user's perception of whether the turn signal is on is not obvious, but the reason for changing lanes to the right is usually obvious, and the user may understand it by looking up. The operation of acceleration may not necessarily occur during the lane change process. For safety reasons, it is usually necessary to slow down and turn on the turn signal to remind the following vehicle first, then change lanes, and then accelerate after changing lanes. Therefore, the situation of accelerating before changing lanes is easy to arouse the user's vigilance. Therefore, for the explanation of the driving behavior of accelerating lane change, the explanation of the acceleration operation is more important. Therefore, by allocating weights, the explanation of the driving behavior can be divided into priorities, and the order of the step-by-step explanation of the driving behavior, the level of detail of the description, etc. can also be determined according to the weights. Among them, the size of the weight is positively correlated with the order of precedence, and the size of the weight is also positively correlated with the level of detail of the description, so that the distribution explanation of the driving behavior with a higher weight can be output earlier and more detailed, while the distribution explanation of the driving behavior with a lower weight can be simply output later. In this way, it can ensure that the more important steps are explained in detail and in a timely manner, and it can also avoid the detailed explanation of all steps that makes the user feel verbose and noisy and produce disgusting emotions.

The step-by-step explanation weight refers to the importance of explaining each autonomous driving control sub-parameter in the autonomous driving control parameter sequence relative to explaining the driving behavior corresponding to the entire autonomous driving control parameter sequence. It can be set in advance by manual or corresponding models according to test results, actual needs, etc., or it can be determined by using an attention mechanism based on the step-by-step explanation of each driving behavior and the attention distribution characteristics determined in the previous time step. This embodiment does not limit this.

As an example, the steps S331-S332 include: obtaining the step-by-step explanation weights corresponding to each autonomous driving control sub-parameter in the currently determined autonomous driving control parameters, and performing weighted aggregation on each step-by-step explanation of the driving behavior based on each step-by-step explanation weight to obtain a driving behavior explanation.

In an practicable manner, the step of using the attention mechanism to determine the step-by-step explanation weight corresponding to each of the autonomous driving control sub-parameters based on the step-by-step explanation of each driving behavior and the attention distribution characteristics of the previous time step can be expressed as:

Where J refers to the driving behavior explanation, _βt refers to the step-by-step explanation weight corresponding to the behavior time t, yt refers to the step-by-step explanation of the driving behavior corresponding to the behavior time t, t∈[t ₀ ,T].

Optionally, the step of obtaining the step-by-step interpretation weight corresponding to each of the autonomous driving control sub-parameters includes:

Step S3311, obtaining the attention distribution feature of the previous time step;

Step S3312, using the attention mechanism, based on the step-by-step interpretation of each driving behavior and the attention distribution characteristics of the previous time step, determines the step-by-step interpretation weight corresponding to each of the autonomous driving control sub-parameters.

As an example, steps S3311-S3312 include: obtaining the attention distribution characteristics of the previous time step, and then using the attention mechanism to perform feature fusion, feature encoding, feature reasoning, etc. on the step-by-step interpretation of each driving behavior and the attention distribution characteristics of the previous time step, determine the step-by-step interpretation weight of the current time step, and determine the step-by-step interpretation weight of the current time step as the step-by-step interpretation weight corresponding to each of the autonomous driving control sub-parameters in the currently determined autonomous driving control parameters.

In one implementable manner, referring to Figure 2, the autonomous driving control device includes a cockpit domain and an intelligent driving domain human-computer interaction interface, a cognitive decision-making model and a vehicle driving behavior embodiment module, wherein the cockpit domain and the intelligent driving domain human-computer interaction interface is used to receive user demand information and environmental cognition information obtained by the cockpit domain and the intelligent driving domain, the cognitive decision-making model is used to determine the autonomous driving control parameters, and the vehicle driving behavior embodiment module is used to control the autonomous driving vehicle to perform corresponding driving behaviors based on the autonomous driving control parameters. After performing the driving behavior, the user's feedback information on the driving behavior can be obtained through the cockpit domain and the intelligent driving domain human-computer interaction interface. Such a cycle can enable the autonomous driving vehicle to perform driving behaviors that adapt to user needs.

In this embodiment, the automatic driving control method adopts a cognitive decision model, and the cognitive decision model includes a cognitive layer and a decision layer. First, by acquiring the monitoring data inside and outside the vehicle, the cognitive layer extracts the current cognitive layer features from the monitoring data inside and outside the vehicle, so as to realize the recognition of the current situation of the automatic driving vehicle; then, by inputting the environmental cognitive features and the user demand features into the decision layer, the automatic driving control parameters are determined, so as to realize the accurate automatic driving decision and determine the automatic driving control parameters based on the recognition of the current situation; then, by generating the driving behavior explanation according to the current cognitive layer features and the automatic driving control parameters, the purpose of explaining the automatic driving decision is realized. Since the automatic driving control parameters are determined based on the cognitive layer features, there is a certain causal relationship between the automatic driving control parameters and the cognitive layer features. The reason why the user cannot understand the driving behavior of the automatic driving vehicle is that before the user feels the sudden driving behavior of the automatic driving vehicle, the user's attention may be focused on different matters from the driving vehicle. Therefore, when the user feels the sudden driving behavior of the automatic driving vehicle, the user feels the final decision made by the automatic driving vehicle, and may not see the factors affecting the decision, so as to be unable to infer the reason for its decision, so as to generate negative emotions such as doubt, panic, and uneasiness. Therefore, this application interprets the autonomous driving control parameters based on the cognitive layer characteristics, generates a driving behavior explanation, can actively interact with the user, explain the reasons for the driving behavior to the user, and eliminate the user's doubts about the driving behavior. Therefore, it overcomes the poor interpretability of the end-to-end perception decision architecture. Therefore, when controlling the vehicle to perform sudden behaviors, it is easy to cause users riding in the autonomous driving vehicle to be unable to understand the driving behavior of the autonomous driving vehicle and generate negative emotions such as doubt, panic, and uneasiness, and the technical defects of poor user experience. It improves the interpretability of the end-to-end perception decision architecture, reduces the user's negative emotions such as doubt, panic, and uneasiness caused by the unknown, and improves the user experience.

Embodiment 2

Further, in the second embodiment of the present application, the same or similar contents as those in the above embodiment can be referred to the above description, and will not be described in detail later. On this basis, referring to FIG. 3 , the current cognitive layer features include the features of the vehicle's external environment; the target decision features include the shallow target decision features;

The step of determining the target decision feature corresponding to each of the autonomous driving control sub-parameters according to the current cognitive layer feature comprises:

Step B10, determining at least one shallow-layer sub-parameter to be explained from each of the autonomous driving control sub-parameters according to the current cognitive layer characteristics, and the spatial attention weight distribution corresponding to each of the shallow-layer sub-parameters to be explained;

In this embodiment, it should be noted that the more explanations of autonomous driving behaviors are provided, the better. When users cannot understand the driving behavior of the autonomous driving vehicle, explaining the driving behavior can effectively alleviate the user's negative emotions. However, if a detailed explanation is still given for driving behaviors that are easy to understand, on the one hand, the more detailed the explanation, the more analysis will be required and the more computing power will be consumed. On the other hand, a very detailed explanation of driving behaviors that are easy to understand may easily cause users to feel that the explanation is verbose, noisy, and unable to distinguish the key points, which may lead to negative emotions.

The spatial attention weight distribution is used to characterize the attention distribution required for the autonomous driving vehicle to achieve safe driving in the current scene and the current step. Positions with high attention weights are significant for the task.

As an example, the step B10 includes: determining at least one shallow sub-parameter to be interpreted from each of the autonomous driving control sub-parameters according to the current cognitive layer characteristics through a classification model or an attention mechanism, and simultaneously, outputting the spatial attention weight distribution corresponding to each of the shallow sub-parameters to be interpreted based on the current cognitive layer characteristics through an activation function.

Step B20, determining the shallow target decision features corresponding to each of the shallow sub-parameters to be interpreted according to the spatial attention weight distribution and the external vehicle environment characteristics.

As an example, step B20 includes: fusing the spatial attention weight distribution corresponding to each of the shallow sub-parameters to be interpreted and the corresponding external vehicle environment characteristics, so as to obtain the shallow target decision features corresponding to each of the shallow sub-parameters to be interpreted.

In one possible implementation, the step of determining the shallow target decision features corresponding to each of the shallow to-be-interpreted sub-parameters according to the spatial attention weight distribution and the vehicle exterior environment features can be expressed as:

Among them, s represents the shallow layer, refers to the shallow sub-parameter to be explained at the time of behavior t, is the spatial attention weight distribution diagram of the ith autonomous driving control sub-parameter at behavior time t, and satisfies x _t,i is the external environment characteristic corresponding to the i-th autonomous driving control sub-parameter at behavior time t, i = {1, 2, … l}, l refers to the number of autonomous driving control sub-parameters.

Optionally, before the step of acquiring the vehicle interior and exterior monitoring data and extracting the current cognitive layer features from the vehicle interior and exterior monitoring data through the cognitive layer, the step further includes:

Step C10, obtaining the in-vehicle and out-vehicle monitoring data and the autonomous driving behavior label of the training sample, and extracting the current cognitive layer features of the training sample from the in-vehicle and out-vehicle monitoring data of the training sample through the cognitive layer;

Step C20, inputting the current cognitive layer features of the training samples into the decision layer, and determining multiple automatic driving control sub-parameters of the training samples;

Step C30, determining at least one shallow sub-parameter to be explained of the training sample from each of the autonomous driving control sub-parameters of the training sample according to the current cognitive layer characteristics of the training sample, and the spatial attention weight distribution of the training samples corresponding to each of the shallow sub-parameters to be explained of the training sample;

Step C40, determining the shallow attention loss based on the autonomous driving control sub-parameters of each training sample, the autonomous driving behavior labels, and the spatial attention weight distribution of each training sample, and iteratively optimizing the cognitive decision-making model based on the shallow attention loss.

In this embodiment, it should be noted that before the cognitive decision model is applied, the cognitive decision model needs to be trained. In each round of iterative optimization of the model training, the cognitive decision model can be updated based on the calculated model loss until the cognitive decision model converges. The training process of the cognitive decision model can be carried out on the device of the final application. Considering the demand for model training computing power, the trained cognitive decision model can also be deployed to the application device for application after the training is completed on other devices. For example, the cognitive decision model can be trained on the server, and then the trained cognitive decision model can be deployed on the autonomous driving vehicle for application. The way of making cognitive decisions based on training samples during the model training process is similar to the model application process. For the same or similar content as the above embodiment, please refer to the above introduction, and no further description will be given later. In the case where the target decision feature includes a shallow target decision feature, since the autonomous driving behavior prediction is closely related to the spatial attention weight distribution, the detection accuracy of the spatial attention weight distribution of the training sample can be used to correct the autonomous driving behavior prediction loss, thereby reducing the error of the single calculation of the driving behavior prediction loss.

As an example, the steps C10-C40 include: obtaining in-vehicle and out-vehicle monitoring data and autonomous driving behavior labels of the training samples, extracting current cognitive layer features of the training samples from the in-vehicle and out-vehicle monitoring data of the training samples through the cognitive layer; inputting the current cognitive layer features of the training samples into the decision layer to determine multiple autonomous driving control sub-parameters of the training samples; determining at least one shallow sub-parameter to be explained of the training samples from each of the autonomous driving control sub-parameters of the training samples according to the current cognitive layer features of the training samples, and the spatial attention weight distribution of the training samples corresponding to each of the shallow sub-parameters to be explained of the training samples; determining at least one based on the difference between each of the autonomous driving control sub-parameters of the training samples and the corresponding autonomous driving behavior labels; The initial prediction loss corresponding to each behavior moment is determined, and the spatial attention loss is determined based on the spatial attention weight distribution of the training samples corresponding to each of the behavior moments. The initial prediction loss corresponding to each behavior moment is corrected based on the spatial attention loss to obtain the shallow attention loss after correction. Then, whether the cognitive decision model has converged is judged based on the shallow attention loss. When it is determined that the cognitive decision model has converged, it can be determined that the model training is completed to obtain a trained cognitive decision model. When it is determined that the cognitive decision model has not converged, a round of updating of the cognitive decision model can be performed based on the shallow attention loss according to the gradient descent method, and the step of obtaining the in-vehicle and out-of-vehicle monitoring data and the autonomous driving behavior labels of the training samples is returned to perform the next round of training.

In an operative manner, the method for correcting the respective corresponding initial prediction losses based on the spatial attention loss can be, based on the spatial attention weight distribution of the training samples corresponding to each of the behavior moments, calculating the attention weight entropy value, determining the attention weight entropy value as the spatial attention loss, and aggregating the spatial attention loss with each of the initial prediction losses to achieve correction of each of the initial prediction losses. It should be noted that the attention weight distribution is used to represent the importance of each element in the input image, and the attention weight entropy value refers to the entropy value of the attention weight distribution, which is used to measure the uncertainty of the model's attention to each element in the input image. If the attention weight distribution is concentrated on one or a few elements, the calculated attention weight entropy value will be relatively low, indicating that the model's attention to the input image is highly certain. On the contrary, if the attention weight distribution is relatively uniform, the calculated attention weight entropy value will be relatively high, indicating that the model's attention to the input image is uncertain.

Optionally, the step of determining the shallow attention loss based on each of the training sample autonomous driving control sub-parameters, each of the autonomous driving behavior labels, and each of the training sample spatial attention weight distributions includes:

Input each of the training sample autonomous driving control sub-parameters, each of the autonomous driving behavior labels, and each of the training sample spatial attention weight distributions into a shallow attention loss function to determine a shallow attention loss, wherein the shallow attention loss function is:

in, is the shallow attention loss, _τt is the autonomous driving behavior label at behavior time t, is the sub-parameter of the autonomous driving control of the training sample at the behavior time t, H is the entropy function used to calculate the entropy of the spatial attention weight distribution of the training sample, λ _s is the preset first hyperparameter, is the spatial attention weight distribution of the training sample at behavior time t, t∈[t ₀ ,T].

In this embodiment, through the shallow explanation mechanism, a shallow explanation at the spatial level can be made for driving behaviors that are easy for users to understand. On the one hand, the explanation at the spatial level only needs to use the cognitive layer features of the current moment, without the need for time dimension analysis. On the one hand, it can reduce computing power consumption and improve the efficiency of generating driving behavior explanations. On the other hand, it can simplify the driving behavior explanations to avoid users feeling verbose, noisy, and unable to distinguish the key points and generating negative emotions. For example, for the driving behavior of "stopping at a traffic light intersection", the key influencing factor of the decision is the red light ahead, and the larger weight in the spatial attention weight distribution will also be concentrated on the position of the traffic light. Therefore, a shallow explanation at the spatial level "because the red light ahead, the vehicle stops before the stop line" can be made.

Embodiment 3

Further, in the third embodiment of the present application, the same or similar contents as those in the above embodiment can be referred to the above description, and will not be described in detail later. On this basis, referring to FIG. 4 , the current cognitive layer features include the features of the vehicle's external environment; the target decision features include the shallow target decision features;

The step of determining the target decision feature corresponding to each of the autonomous driving control sub-parameters according to the current cognitive layer feature comprises:

Step D10, determining at least one deep-level sub-parameter to be explained from each of the autonomous driving control sub-parameters according to the current cognitive layer characteristics;

In this embodiment, it should be noted that if the driving behavior decisions made in complex scenarios are only explained superficially at the spatial level, the user needs to think and associate the spatiotemporal logic by himself, and some historical information may be missing. For example, if a cat suddenly runs across the road, when the user feels the brakes and looks up, the cat may have run away. As a result, the user may not be able to accurately understand the driving behavior and may have negative emotions such as doubt, panic, and anxiety, resulting in a poor user experience.

As an example, the step D10 includes: determining at least one deep-level sub-parameter to be explained from each of the autonomous driving control sub-parameters according to the current cognitive layer characteristics through a classification model or an attention mechanism.

Step D20, obtaining historical cognitive layer features extracted before the current moment;

Step D30, adopting the spatiotemporal attention mechanism, based on the current cognitive layer features and the historical cognitive layer features, extracts the deep target decision features corresponding to each of the deep sub-parameters to be explained.

In this embodiment, it should be noted that the historical cognitive layer features refer to the cognitive layer features extracted before the current moment. The deep interpretation is to interpret the two dimensions of time and space for those driving behaviors that cannot be clearly and intuitively explained through the spatial dimension. The deep target decision-making features contain spatial information and time information of the factors that affect the decision, such as motion trend information, motion trajectory information, etc. If the time dimension interpretation is to be performed, whether it is to analyze from the past to the present or to predict the future, it is necessary to perform trend analysis based on the historical cognitive layer features generated from the past to the present in order to more accurately determine the key factors that affect driving behavior. Therefore, it is necessary to obtain the historical cognitive layer features and analyze them in combination with the current cognitive layer features. For example, the driving behavior of "accelerating and changing lanes to avoid obstacles in a mixed traffic scene" involves many influencing factors and the vehicle performs an unconventional operation of accelerating before changing lanes. If only a shallow explanation of the spatial dimension is performed, "there is someone ahead, so accelerate and change lanes to avoid obstacles", but because there is someone ahead, you should usually slow down and give way, so users will still be confused about the driving behavior of the autonomous vehicle. At this time, explanations in both time and space dimensions are needed. The generated deep explanation can be "because the movement trajectory of the pedestrian will intersect with the trajectory of this vehicle, there is a risk of collision, and this lane cannot slow down to avoid collision, so accelerate and change lanes to avoid obstacles". It can also output a spatial attention weight distribution map with a large weight concentrated on the position of pedestrians that cannot be avoided, allowing users to more intuitively see the pedestrians that cannot be avoided, and thus better understand driving behavior.

As an example, step D20 includes: a spatiotemporal attention mechanism can be used to perform feature fusion, feature encoding, feature reasoning and other analyses on the current cognitive layer features, the historical cognitive layer features and each of the deep-layer sub-parameters to be interpreted, so as to determine the deep target decision features corresponding to each of the deep-layer sub-parameters to be interpreted.

Optionally, the step of determining a shallow attention loss based on each of the training sample autonomous driving control sub-parameters, each of the autonomous driving behavior labels, and each of the training sample spatial attention weight distributions, and iteratively optimizing the cognitive decision model based on the shallow attention loss includes:

Step C41, determining at least one training sample deep-layer to-be-explained sub-parameter from each of the training sample autonomous driving control sub-parameters according to the current cognitive layer characteristics of the training sample, and the spatial attention weight distribution of the training samples corresponding to each of the training sample deep-layer to-be-explained sub-parameters;

Step C42, obtaining the historical cognitive layer features of the training samples extracted before the current moment, using the spatiotemporal attention mechanism, based on the current cognitive layer features of the training samples and the historical cognitive layer features of the training samples, determining the spatiotemporal attention weight distribution of the training samples corresponding to the deep-layer to-be-explained sub-parameters of the training samples;

Step C43, determining the deep attention loss based on the difference between the spatial attention weight distribution of the training samples corresponding to the deep sub-parameters to be interpreted of the training samples and the corresponding spatiotemporal attention weight distribution of the training samples, and determining the shallow attention loss based on the difference between the spatial attention weight distribution of the training samples and the autonomous driving control sub-parameters of the training samples and the corresponding autonomous driving behavior labels;

Step C44, aggregating the deep attention loss and the shallow attention loss to obtain the autonomous driving behavior prediction loss, and iteratively optimizing the cognitive decision model based on the autonomous driving behavior prediction loss.

In this embodiment, it should be noted that when the target decision feature includes a deep target decision feature, since it is necessary to determine the key factors affecting the decision regardless of whether the analysis is in the spatial dimension or in the time and space dimensions, this key factor is usually determined based on the attention weight distribution. Therefore, by comparing the difference between the spatial attention weight distribution determined from the spatial dimension and the time and space attention weight distribution determined from the time and space dimension, the time and space attention mechanism can refer to the significant object of attention, that is, the position with a higher weight in the spatial attention weight distribution determined in the spatial dimension, and interpret the vehicle driving behavior output by the cognitive decision model in the time and space dimensions.

As an example, the steps C41-C44 include: determining at least one deep sub-parameter to be interpreted of the training samples and the spatial attention weight distribution of the training samples corresponding to each of the deep sub-parameters to be interpreted of the training samples from each of the autonomous driving control sub-parameters of the training samples according to the current cognitive layer characteristics of the training samples; obtaining the historical cognitive layer characteristics of the training samples extracted before the current moment, and using the spatiotemporal attention mechanism to determine the spatiotemporal attention weight distribution of the training samples corresponding to each of the deep sub-parameters to be interpreted of the training samples based on the current cognitive layer characteristics of the training samples and the historical cognitive layer characteristics of the training samples; determining the deep attention weight distribution based on the difference between the spatial attention weight distribution of the training samples corresponding to each of the deep sub-parameters to be interpreted of the training samples and the spatiotemporal attention weight distribution of the training samples corresponding to each of the deep sub-parameters to be interpreted of the training samples. Loss, based on the spatial attention weight distribution of each training sample and the difference between each training sample autonomous driving control sub-parameter and the corresponding autonomous driving behavior label, determine the shallow attention loss; then aggregate the deep attention loss and the shallow attention loss to obtain the autonomous driving behavior prediction loss; then, judge whether the cognitive decision model has converged based on the autonomous driving behavior prediction loss. When it is determined that the cognitive decision model has converged, it can be determined that the model training is completed, and a trained cognitive decision model is obtained; when it is determined that the cognitive decision model has not converged, a round of updating of the cognitive decision model can be performed based on the autonomous driving behavior prediction loss according to the gradient descent method, and return to execute the step of obtaining the in-vehicle and out-of-vehicle monitoring data and autonomous driving behavior labels of the training samples to perform the next round of training.

Optionally, the step of determining the deep attention loss based on the difference between the spatial attention weight distribution of the training samples corresponding to each of the deep sub-parameters to be interpreted of the training samples and the spatiotemporal attention weight distribution of the training samples corresponding to each of the deep sub-parameters to be interpreted of the training samples comprises:

Input the spatial attention weight distribution of each training sample and the spatiotemporal attention weight distribution of each training sample into the deep attention loss function to determine the deep attention loss, wherein the deep attention loss function is:

Where D _KL represents the divergence function, is the spatial attention weight distribution of the training sample corresponding to the deep-layer to-be-explained sub-parameter of the i-th training sample at behavior time t, is the spatiotemporal attention weight distribution of the training samples corresponding to the deep-layer to-be-explained sub-parameters of the i-th training sample at behavior time t, _λd is the preset second hyperparameter, i∈[1,l].

Optionally, the step of aggregating the deep attention loss and the shallow attention loss to obtain the autonomous driving behavior prediction loss includes:

Step E10, generating at least one shallow driving behavior step-by-step explanation of the training sample according to the shallow sub-parameters to be explained of each training sample and the current cognitive layer characteristics of the training sample, and generating at least one deep driving behavior step-by-step explanation of the training sample according to the deep sub-parameters to be explained of each training sample and the current cognitive layer characteristics of the training sample;

Step E20, aggregating the shallow driving behavior step-by-step explanations of each training sample, the shallow sub-parameters to be explained of each training sample, the deep driving behavior step-by-step explanations of each training sample, and the deep sub-parameters to be explained of each training sample to obtain the driving behavior explanations of the training samples;

Step E30, obtaining the historical training sample driving behavior explanation, the historical training sample spatial attention weight distribution, and the historical training sample spatiotemporal attention weight distribution of the previous time step, and determining the minimum negative log-likelihood loss based on the historical training sample driving behavior explanation, the historical training sample spatial attention weight distribution, and the historical training sample spatiotemporal attention weight distribution, as well as the training sample driving behavior explanation, the training sample spatial attention weight distribution, and the training sample spatiotemporal attention weight distribution of the current time step;

Step E40, aggregating the deep attention loss, the shallow attention loss and the minimized negative log-likelihood loss to obtain the autonomous driving behavior prediction loss.

In this embodiment, it should be noted that in order to uniformly optimize the cognitive decision-making model so that the overall loss of the cognitive decision-making model is minimized and the best interpretation performance can be obtained, the shallow attention loss and the deep attention loss are combined for training, and the model is fine-tuned by minimizing the negative log-likelihood function.

As an example, the steps E10-E40 include: generating a step-by-step explanation of the shallow driving behavior of the training sample corresponding to each shallow sub-parameter to be explained of the training sample and the current cognitive layer characteristics of the training sample according to the shallow sub-parameter to be explained of each training sample and the current cognitive layer characteristics of the training sample; generating a step-by-step explanation of the deep driving behavior of the training sample corresponding to each deep sub-parameter to be explained of the training sample according to the deep sub-parameter to be explained of each training sample and the current cognitive layer characteristics of the training sample; further, aggregating the step-by-step explanation of the shallow driving behavior of each training sample and the corresponding shallow sub-parameter to be explained of the training sample into a shallow step-by-step explanation of the driving behavior of the training sample, and aggregating the step-by-step explanation of the deep driving behavior of each training sample and the corresponding deep sub-parameter to be explained of the training sample into a deep step-by-step explanation of the driving behavior of the training sample, and further aggregating the step-by-step explanation of the deep driving behavior of each training sample and the corresponding deep sub-parameter to be explained of the training sample into a deep step-by-step explanation of the driving behavior of the training sample, and further aggregating the step-by-step explanation of the deep driving behavior of each training sample and the corresponding deep sub-parameter to be explained of the training sample The shallow step-by-step explanation of the driving behavior and the deep step-by-step explanation of the driving behavior of each training sample are aggregated to obtain the explanation of the driving behavior of the training sample; then, the driving behavior explanation of the historical training samples, the spatial attention weight distribution of the historical training samples, and the spatiotemporal attention weight distribution of the historical training samples at the previous time step are obtained, and the driving behavior explanation of the historical training samples, the spatial attention weight distribution of the historical training samples, and the spatiotemporal attention weight distribution of the historical training samples at the previous time step and the driving behavior explanation of the training samples, the spatial attention weight distribution of the training samples, and the spatiotemporal attention weight distribution of the training samples at the current time step are input into the minimized negative log-likelihood function to calculate the minimized negative log-likelihood loss; then, the deep attention loss, the shallow attention loss, and the minimized negative log-likelihood loss are aggregated to obtain the automatic driving behavior prediction loss.

In one practicable manner, the loss function of the autonomous driving behavior prediction loss can be expressed as:

in,

in, Predicting losses for autonomous driving behaviors, represents shallow attention loss, represents the deep attention loss, k represents the current time step, k-1 represents the previous time step, represents the attention weight distribution of the current time step, including the spatial attention weight distribution and the spatiotemporal attention weight distribution, J _k represents the driving behavior explanation of the current time step, and o _k represents the interpretable output of the current time step, including and J _k , _ok-1 represents the interpretable output of the previous time step, Represents the probability of determining the interpretable output at the current time step given the interpretable output at the previous time step.

In this embodiment, through the deep interpretation mechanism, the driving behavior that is difficult for users to understand in more complex scenarios can be deeply interpreted in the time dimension and space dimension, explaining the causes and consequences of the driving behavior to users, eliminating users' doubts about the driving behavior, improving the interpretability of the end-to-end perception decision architecture, reducing users' negative emotions such as doubt, panic, and anxiety caused by the unknown, and improving user experience.

Embodiment 4

Furthermore, an embodiment of the present application further provides an automatic driving control device. Referring to FIG. 5 , a cognitive decision model is deployed on the automatic driving control device. The cognitive decision model includes a cognitive layer and a decision layer. The automatic driving control device includes:

A cognitive module 10, used to obtain in-vehicle and out-vehicle monitoring data, and extract current cognitive layer features from the in-vehicle and out-vehicle monitoring data through the cognitive layer;

A decision module 20, configured to input the current cognitive layer features into the decision layer to determine the automatic driving control parameters;

The explanation module 30 is used to generate a driving behavior explanation based on the current cognitive layer characteristics and the automatic driving control parameters.

Optionally, the autonomous driving control parameter includes a plurality of autonomous driving control sub-parameters;

The interpretation module 30 is further used for:

Determining target decision features corresponding to each of the autonomous driving control sub-parameters according to the current cognitive layer features;

Aggregating each of the autonomous driving control sub-parameters with the corresponding target decision features to obtain a step-by-step explanation of the driving behavior;

The step-by-step explanations of the driving behavior are aggregated to obtain a driving behavior explanation.

Optionally, the current cognitive layer features include features of the vehicle's external environment; the target decision features include shallow target decision features;

The interpretation module 30 is further used for:

Determine, according to the current cognitive layer characteristics, at least one shallow-layer sub-parameter to be explained from each of the autonomous driving control sub-parameters, and a spatial attention weight distribution corresponding to each of the shallow-layer sub-parameters to be explained;

The shallow target decision features corresponding to each of the shallow sub-parameters to be interpreted are determined according to the spatial attention weight distribution and the external vehicle environment characteristics.

Optionally, the automatic driving control device further includes a model training module, and the model training module is used to:

Acquire the in-vehicle and out-vehicle monitoring data and the autonomous driving behavior labels of the training samples, and extract the current cognitive layer features of the training samples from the in-vehicle and out-vehicle monitoring data of the training samples through the cognitive layer;

Inputting the current cognitive layer features of the training samples into the decision layer to determine a plurality of automatic driving control sub-parameters of the training samples;

Determining at least one shallow sub-parameter to be explained of the training sample from each of the autonomous driving control sub-parameters of the training sample according to the current cognitive layer characteristics of the training sample, and the spatial attention weight distribution of the training samples corresponding to each of the shallow sub-parameters to be explained of the training sample;

The shallow attention loss is determined based on the autonomous driving control sub-parameters of each training sample, the autonomous driving behavior labels, and the spatial attention weight distribution of each training sample, and the cognitive decision-making model is iteratively optimized based on the shallow attention loss.

Optionally, the model training module is further used to:

Input each of the training sample autonomous driving control sub-parameters, each of the autonomous driving behavior labels, and each of the training sample spatial attention weight distributions into a shallow attention loss function to determine a shallow attention loss, wherein the shallow attention loss function is:

in, is the shallow attention loss, _τt is the autonomous driving behavior label at behavior time t, is the training sample automatic driving control sub-parameter at behavior time t, H is the entropy function, λ _s is the preset first hyperparameter, is the spatial attention weight distribution of the training sample at behavior time t, t∈[t ₀ ,T].

Optionally, the driving behavior explanation includes a deep driving behavior explanation; the target decision feature includes a deep target decision feature;

The interpretation module 30 is further used for:

Determining at least one deep-level sub-parameter to be interpreted from each of the autonomous driving control sub-parameters according to the current cognitive layer characteristics;

Obtain the historical cognitive layer features extracted before the current moment;

The spatiotemporal attention mechanism is adopted to extract the deep target decision features corresponding to each of the deep sub-parameters to be explained based on the current cognitive layer features and the historical cognitive layer features.

Optionally, the model training module is further used to:

Determining at least one training sample deep-layer to-be-explained sub-parameter from each of the training sample autonomous driving control sub-parameters according to the current cognitive layer characteristics of the training sample, and the spatial attention weight distribution of the training samples corresponding to each of the training sample deep-layer to-be-explained sub-parameters;

Obtain the historical cognitive layer features of the training samples extracted before the current moment, and use the spatiotemporal attention mechanism to determine the spatiotemporal attention weight distribution of the training samples corresponding to the deep-layer to-be-explained sub-parameters of each training sample based on the current cognitive layer features of the training samples and the historical cognitive layer features of the training samples;

Determine the deep attention loss based on the difference between the spatial attention weight distribution of the training samples corresponding to the deep sub-parameters to be interpreted of the training samples and the corresponding spatiotemporal attention weight distribution of the training samples; determine the shallow attention loss based on the difference between the spatial attention weight distribution of the training samples and the autonomous driving control sub-parameters of the training samples and the corresponding autonomous driving behavior labels;

The deep attention loss and the shallow attention loss are aggregated to obtain the autonomous driving behavior prediction loss, and the cognitive decision model is iteratively optimized based on the autonomous driving behavior prediction loss.

Optionally, the model training module is further used to:

Input the spatial attention weight distribution of each training sample and the spatiotemporal attention weight distribution of each training sample into the deep attention loss function to determine the deep attention loss, wherein the deep attention loss function is:

Where D _KL represents the divergence function, is the spatial attention weight distribution of the training sample corresponding to the deep-layer to-be-explained sub-parameter of the i-th training sample at behavior time t, is the spatiotemporal attention weight distribution of the training samples corresponding to the deep-layer to-be-explained sub-parameters of the i-th training sample at behavior time t, _λd is the preset second hyperparameter, i∈[1,l].

Optionally, the model training module is further used to:

Generate at least one shallow driving behavior step-by-step explanation of the training sample according to the shallow sub-parameters to be explained of each training sample and the current cognitive layer characteristics of the training sample, and generate at least one deep driving behavior step-by-step explanation of the training sample according to the deep sub-parameters to be explained of each training sample and the current cognitive layer characteristics of the training sample;

Aggregating the shallow driving behavior step-by-step explanations of each training sample, the shallow sub-parameters to be explained of each training sample, the deep driving behavior step-by-step explanations of each training sample, and the deep sub-parameters to be explained of each training sample to obtain the driving behavior explanations of the training samples;

Obtain the historical training sample driving behavior explanation, the historical training sample spatial attention weight distribution, and the historical training sample spatiotemporal attention weight distribution of the previous time step, and determine the minimum negative log-likelihood loss based on the historical training sample driving behavior explanation, the historical training sample spatial attention weight distribution, the historical training sample spatiotemporal attention weight distribution, and the training sample driving behavior explanation, the training sample spatial attention weight distribution, and the training sample spatiotemporal attention weight distribution of the current time step;

The deep attention loss, the shallow attention loss and the minimized negative log-likelihood loss are aggregated to obtain the autonomous driving behavior prediction loss.

Optionally, the interpretation module 30 is further configured to:

Obtaining a step-by-step explanation weight corresponding to each of the autonomous driving control sub-parameters;

The driving behavior explanation is obtained by weighting and aggregating the step-by-step explanations of the driving behavior based on the weights of the step-by-step explanations.

Optionally, the interpretation module 30 is further configured to:

Get the attention distribution features of the previous time step;

An attention mechanism is adopted to determine the step-by-step interpretation weight corresponding to each of the autonomous driving control sub-parameters based on the step-by-step interpretation of each driving behavior and the attention distribution characteristics of the previous time step.

Optionally, the cognitive layer includes a user feature extraction model; the current cognitive layer features include user demand features, wherein the user demand features include user implicit demand features and user explicit demand features; the in-vehicle and out-of-vehicle monitoring data include user operation data and user status data;

The cognitive module 10 is also used for:

The user display demand features are extracted from the user operation data by using the user feature extraction model, and/or the user implicit demand features are extracted from the user status data by using the user feature extraction model.

Optionally, the cognitive layer includes a scene cognitive model, a map construction model, a risk target detection and tracking model, and a motion prediction model; the current cognitive layer features include environmental cognitive features, wherein the environmental cognitive features include scene cognitive features, map features, risk target detection and tracking features, and risk target motion features;

The cognitive module 10 is also used for:

Extracting scene recognition features from the vehicle interior and exterior monitoring data by using the scene recognition model, extracting map features from the vehicle interior and exterior monitoring data by using the map construction model, and extracting risk target detection and tracking features from the vehicle interior and exterior monitoring data by using the risk target detection and tracking model;

The motion prediction model adopts the attention mechanism to perform risk target motion prediction based on the scene recognition features, the map features and the risk target detection and tracking features to obtain risk target motion features.

The automatic driving control device provided by the present invention adopts the automatic driving control method in the above embodiment, and solves the technical problem of poor interpretability of the end-to-end perception decision-making scheme in the related art. Compared with the related art, the benefits of the automatic driving control device provided by the embodiment of the present invention are the same as the benefits of the automatic driving control method provided by the above embodiment, and the other technical features in the automatic driving control device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

Embodiment 5

Furthermore, an embodiment of the present application also provides an automatic driving control device, on which a cognitive decision model is deployed, the cognitive decision model includes a cognitive layer and a decision layer, the cognitive layer is pre-trained using the automatic driving control method as described above, and the automatic driving control device includes:

The second acquisition module is used to acquire the vehicle internal and external monitoring data to be predicted;

A second cognitive module is used to extract risk features to be predicted from the vehicle internal and external monitoring data through the cognitive layer, wherein the risk features to be predicted include scene cognitive features to be predicted;

The decision module is used to input the risk characteristics to be predicted into the decision layer and determine the automatic driving control parameters.

Optionally, the cognitive layer includes a scene cognitive model, a map construction model, a risk target detection and tracking model, and a motion prediction model; the risk features to be predicted also include map features to be predicted, risk target detection and tracking features to be predicted, and risk target motion features to be predicted;

The second cognitive model is further used for:

Extracting the scene recognition features to be predicted from the vehicle interior and exterior monitoring data to be predicted by the scene recognition model, extracting the map features to be predicted from the vehicle interior and exterior monitoring data to be predicted by the map construction model, and extracting the risk target detection and tracking features to be predicted from the vehicle interior and exterior monitoring data to be predicted by the risk target detection and tracking model;

The motion prediction model adopts the attention mechanism to perform risk target motion prediction based on the cognitive features of the scene to be predicted, the map features to be predicted and the detection and tracking features of the risk target to be predicted, so as to obtain the motion features of the risk target to be predicted.

Optionally, the risk target detection and tracking feature to be predicted includes at least one risk target detection and tracking sub-feature; the motion prediction model includes an attention layer, a multi-layer perceptron layer and a prediction layer;

The second cognitive model is further used for:

Inputting the scene recognition feature to be predicted, the map feature to be predicted and the risk target detection and tracking feature to be predicted into the attention layer, and using the attention mechanism to determine the risk target interaction feature between each of the risk target detection and tracking sub-features, the scene interaction feature between each of the risk target detection and tracking sub-features and the scene recognition feature to be predicted, and the map interaction feature between each of the risk target detection and tracking sub-features and the map feature to be predicted;

Inputting the risk target interaction feature, the scene interaction feature and the map interaction feature into a multi-layer perceptron layer to obtain a scene risk feature and a motion query feature;

The scene risk feature and the motion query feature are input into the prediction layer to perform risk target motion prediction to obtain the risk target motion feature to be predicted.

The automatic driving control device provided by the present invention adopts the automatic driving control method in the above embodiment, and solves the technical problem of poor interpretability of the end-to-end perception decision-making scheme in the related art. Compared with the related art, the benefits of the automatic driving control device provided by the embodiment of the present invention are the same as the benefits of the automatic driving control method provided by the above embodiment, and the other technical features in the automatic driving control device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

Embodiment 6

Furthermore, an embodiment of the present invention provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the autonomous driving control method in the above embodiment.

Referring to FIG6 below, a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present disclosure is shown. The electronic device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as Bluetooth headsets, mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG6 is only an example and should not bring any limitation to the functions and scope of use of the embodiment of the present disclosure.

As shown in Figure 6, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage device into a random access memory (RAM). In the RAM, various programs and arrays required for the operation of the electronic device are also stored. The processing device, ROM, and RAM are connected to each other via a bus. An input/output (I/O) interface is also connected to the bus.

Typically, the following systems can be connected to the I/O interface: input devices including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; output devices including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices including, for example, a magnetic tape, a hard disk, etc.; and communication devices. The communication device can allow the electronic device to communicate with other devices wirelessly or by wire to exchange arrays. Although the figure shows an electronic device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have instead.

In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

The electronic device provided by the present invention adopts the automatic driving control method in the above embodiment, and solves the technical problem of poor interpretability of the end-to-end perception decision-making scheme in the related art. Compared with the related art, the benefits of the electronic device provided by the embodiment of the present invention are the same as the benefits of the automatic driving control method provided by the above embodiment, and the other technical features in the electronic device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

It should be understood that the various parts of the present disclosure can be implemented with hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Embodiment 7

Furthermore, this embodiment provides a computer-readable storage medium having computer-readable program instructions stored thereon, and the computer-readable program instructions are used to execute the automatic driving control method in the above embodiment.

The computer-readable storage medium provided in the embodiment of the present invention may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

The computer-readable storage medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by an electronic device, the electronic device: obtains in-vehicle and out-vehicle monitoring data and scene labels, and obtains target human driving experience scene cognition characteristics corresponding to the scene labels; extracts risk characteristics from the in-vehicle and out-vehicle monitoring data through the cognitive layer; and iteratively optimizes the cognitive layer according to the risk characteristics and the target human driving experience scene cognition characteristics.

Alternatively, the computer-readable storage medium carries one or more programs. When the one or more programs are executed by an electronic device, the electronic device: obtains the vehicle's internal and external monitoring data to be predicted; extracts risk features to be predicted from the vehicle's internal and external monitoring data through the cognitive layer, wherein the risk features to be predicted include scene cognitive features to be predicted; and inputs the risk features to be predicted into the decision layer to determine the automatic driving control parameters.

Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

The computer-readable storage medium provided by the present invention stores computer-readable program instructions for executing the above-mentioned automatic driving control method, which solves the technical problem of poor interpretability of the end-to-end perception decision-making scheme in the related art. Compared with the related art, the benefits of the computer-readable storage medium provided by the embodiment of the present invention are the same as the benefits of the automatic driving control method provided by the above-mentioned embodiment, which will not be repeated here.

The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent processing scope of the present application.

Claims (16)

1. An automatic driving control method, characterized in that the automatic driving control method adopts a cognitive decision model, the cognitive decision model includes a cognitive layer and a decision layer; the automatic driving control method includes the following steps: Acquire vehicle interior and exterior monitoring data, and extract current cognitive layer features from the vehicle interior and exterior monitoring data through the cognitive layer; Inputting the current cognitive layer features into the decision layer to determine the automatic driving control parameters; Generate a driving behavior explanation based on the current cognitive layer characteristics and the autonomous driving control parameters. 2. The automatic driving control method according to claim 1, wherein the automatic driving control parameter comprises a plurality of automatic driving control sub-parameters; The step of generating a driving behavior explanation according to the current cognitive layer characteristics and the automatic driving control parameters comprises: Determining target decision features corresponding to each of the autonomous driving control sub-parameters according to the current cognitive layer features; Aggregating each of the autonomous driving control sub-parameters with the corresponding target decision features to obtain a step-by-step explanation of the driving behavior; The step-by-step explanations of the driving behavior are aggregated to obtain a driving behavior explanation. 3. The automatic driving control method according to claim 2, wherein the current cognitive layer features include external environment features; the target decision features include shallow target decision features; The step of determining the target decision feature corresponding to each of the autonomous driving control sub-parameters according to the current cognitive layer feature comprises: Determine, according to the current cognitive layer features, at least one shallow-layer sub-parameter to be explained from each of the autonomous driving control sub-parameters, and a spatial attention weight distribution corresponding to each of the shallow-layer sub-parameters to be explained; The shallow target decision features corresponding to each of the shallow sub-parameters to be interpreted are determined according to the spatial attention weight distribution and the external vehicle environment characteristics. 4. The automatic driving control method according to claim 3, characterized in that before the step of acquiring the vehicle interior and exterior monitoring data and extracting the current cognitive layer features from the vehicle interior and exterior monitoring data through the cognitive layer, it also includes: Acquire the in-vehicle and out-vehicle monitoring data and the autonomous driving behavior labels of the training samples, and extract the current cognitive layer features of the training samples from the in-vehicle and out-vehicle monitoring data of the training samples through the cognitive layer; Inputting the current cognitive layer features of the training samples into the decision layer to determine a plurality of automatic driving control sub-parameters of the training samples; Determining at least one shallow sub-parameter to be explained of the training sample from each of the autonomous driving control sub-parameters of the training sample according to the current cognitive layer characteristics of the training sample, and the spatial attention weight distribution of the training samples corresponding to each of the shallow sub-parameters to be explained of the training sample; The shallow attention loss is determined based on the autonomous driving control sub-parameters of each training sample, the autonomous driving behavior labels, and the spatial attention weight distribution of each training sample, and the cognitive decision-making model is iteratively optimized based on the shallow attention loss. 5. The automatic driving control method according to claim 4, characterized in that the step of determining the shallow attention loss based on the automatic driving control sub-parameters of each training sample, the automatic driving behavior labels and the spatial attention weight distribution of each training sample comprises: Input each of the training sample autonomous driving control sub-parameters, each of the autonomous driving behavior labels, and each of the training sample spatial attention weight distributions into a shallow attention loss function to determine a shallow attention loss, wherein the shallow attention loss function is: in, is the shallow attention loss, _τt is the autonomous driving behavior label at behavior time t, is the training sample automatic driving control sub-parameter at behavior time t, H is the entropy function, λ _s is the preset first hyperparameter, is the spatial attention weight distribution of the training sample at behavior time t, t∈[t ₀ ,T]. 6. The automatic driving control method according to claim 5, wherein the driving behavior explanation comprises a deep driving behavior explanation; the target decision feature comprises a deep target decision feature; The step of determining the target decision feature corresponding to each of the autonomous driving control sub-parameters according to the current cognitive layer feature comprises: Determining at least one deep-level sub-parameter to be interpreted from each of the autonomous driving control sub-parameters according to the current cognitive layer characteristics; Obtain the historical cognitive layer features extracted before the current moment; The spatiotemporal attention mechanism is adopted to extract the deep target decision features corresponding to each of the deep sub-parameters to be explained based on the current cognitive layer features and the historical cognitive layer features. 7. The automatic driving control method according to claim 6, characterized in that the step of determining the shallow attention loss based on the automatic driving control sub-parameters of each training sample, the automatic driving behavior labels and the spatial attention weight distribution of each training sample, and iteratively optimizing the cognitive decision model based on the shallow attention loss comprises: Determining at least one training sample deep-layer to-be-explained sub-parameter from each of the training sample autonomous driving control sub-parameters according to the current cognitive layer characteristics of the training sample, and the spatial attention weight distribution of the training samples corresponding to each of the training sample deep-layer to-be-explained sub-parameters; Obtain the historical cognitive layer features of the training samples extracted before the current moment, and use the spatiotemporal attention mechanism to determine the spatiotemporal attention weight distribution of the training samples corresponding to the deep-layer to-be-explained sub-parameters of each training sample based on the current cognitive layer features of the training samples and the historical cognitive layer features of the training samples; Determine the deep attention loss based on the difference between the spatial attention weight distribution of the training samples corresponding to the deep sub-parameters to be interpreted of the training samples and the corresponding spatiotemporal attention weight distribution of the training samples; determine the shallow attention loss based on the difference between the spatial attention weight distribution of the training samples and the autonomous driving control sub-parameters of the training samples and the corresponding autonomous driving behavior labels; The deep attention loss and the shallow attention loss are aggregated to obtain the autonomous driving behavior prediction loss, and the cognitive decision model is iteratively optimized based on the autonomous driving behavior prediction loss. 8. The automatic driving control method according to claim 7, characterized in that the step of determining the deep attention loss based on the difference between the spatial attention weight distribution of the training samples corresponding to each of the deep to-be-interpreted sub-parameters of the training samples and the spatiotemporal attention weight distribution of the training samples corresponding to each of the deep to-be-interpreted sub-parameters of the training samples comprises: Input the spatial attention weight distribution of each training sample and the spatiotemporal attention weight distribution of each training sample into the deep attention loss function to determine the deep attention loss, wherein the deep attention loss function is: Where D _KL represents the divergence function, is the spatial attention weight distribution of the training sample corresponding to the deep-layer to-be-explained sub-parameter of the i-th training sample at behavior time t, is the spatiotemporal attention weight distribution of the training samples corresponding to the deep-layer to-be-explained sub-parameters of the i-th training sample at behavior time t, _λd is the preset second hyperparameter, i∈[1,l]. 9. The automatic driving control method according to claim 7, wherein the step of aggregating the deep attention loss and the shallow attention loss to obtain the automatic driving behavior prediction loss comprises: Generate at least one shallow driving behavior step-by-step explanation of the training sample according to the shallow sub-parameters to be explained of each training sample and the current cognitive layer characteristics of the training sample, and generate at least one deep driving behavior step-by-step explanation of the training sample according to the deep sub-parameters to be explained of each training sample and the current cognitive layer characteristics of the training sample; Aggregating the shallow driving behavior step-by-step explanations of each training sample, the shallow sub-parameters to be explained of each training sample, the deep driving behavior step-by-step explanations of each training sample, and the deep sub-parameters to be explained of each training sample to obtain the driving behavior explanations of the training samples; Obtain the historical training sample driving behavior explanation, the historical training sample spatial attention weight distribution, and the historical training sample spatiotemporal attention weight distribution of the previous time step, and determine the minimum negative log-likelihood loss based on the historical training sample driving behavior explanation, the historical training sample spatial attention weight distribution, the historical training sample spatiotemporal attention weight distribution, and the training sample driving behavior explanation, the training sample spatial attention weight distribution, and the training sample spatiotemporal attention weight distribution of the current time step; The deep attention loss, the shallow attention loss and the minimized negative log-likelihood loss are aggregated to obtain the autonomous driving behavior prediction loss. 10. The automatic driving control method according to claim 2, wherein the step of aggregating the step-by-step explanations of the driving behaviors to obtain the driving behavior explanations comprises: Obtaining a step-by-step explanation weight corresponding to each of the autonomous driving control sub-parameters; The driving behavior explanation is obtained by weighting and aggregating the step-by-step explanations of the driving behavior based on the weights of the step-by-step explanations. 11. The automatic driving control method according to claim 10, wherein the step of obtaining the step-by-step interpretation weight corresponding to each of the automatic driving control sub-parameters comprises: Get the attention distribution features of the previous time step; An attention mechanism is adopted to determine the step-by-step interpretation weight corresponding to each of the autonomous driving control sub-parameters based on the step-by-step interpretation of each driving behavior and the attention distribution characteristics of the previous time step. 12. The automatic driving control method according to claim 1, characterized in that the cognitive layer includes a user feature extraction model; the current cognitive layer features include user demand features, wherein the user demand features include user implicit demand features and user explicit demand features; the in-vehicle and out-of-vehicle monitoring data include user operation data and user status data; The step of extracting current cognitive layer features from the vehicle internal and external monitoring data through the cognitive layer includes: The user display demand features are extracted from the user operation data by using the user feature extraction model, and/or the user implicit demand features are extracted from the user status data by using the user feature extraction model. 13. The automatic driving control method according to claim 1, characterized in that the cognitive layer includes a scene cognitive model, a map construction model, a risk target detection and tracking model, and a motion prediction model; the current cognitive layer features include environmental cognitive features, wherein the environmental cognitive features include scene cognitive features, map features, risk target detection and tracking features, and risk target motion features; The step of extracting current cognitive layer features from the vehicle internal and external monitoring data through the cognitive layer includes: Extracting scene recognition features from the vehicle interior and exterior monitoring data by using the scene recognition model, extracting map features from the vehicle interior and exterior monitoring data by using the map construction model, and extracting risk target detection and tracking features from the vehicle interior and exterior monitoring data by using the risk target detection and tracking model; The motion prediction model adopts the attention mechanism to perform risk target motion prediction based on the scene recognition features, the map features and the risk target detection and tracking features to obtain risk target motion features. 14. An automatic driving control device, characterized in that a cognitive decision model is deployed on the automatic driving control device, the cognitive decision model includes a cognitive layer and a decision layer, and the automatic driving control device includes: A cognitive module, used for acquiring in-vehicle and out-vehicle monitoring data, and extracting current cognitive layer features from the in-vehicle and out-vehicle monitoring data through the cognitive layer; A decision module, used for inputting the current cognitive layer features into the decision layer to determine the automatic driving control parameters; An explanation module is used to generate a driving behavior explanation based on the current cognitive layer characteristics and the automatic driving control parameters. 15. An electronic device, characterized in that the electronic device comprises: at least one processor; and, a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by at least one of the processors, and the instructions are executed by at least one of the processors to enable at least one of the processors to perform the steps of the automatic driving control method as described in any one of claims 1 to 13. 16. A storage medium, characterized in that the storage medium is a computer-readable storage medium, and a program for implementing an automatic driving control method is stored on the computer-readable storage medium, and the program for implementing the automatic driving control method is executed by a processor to implement the steps of the automatic driving control method as described in any one of claims 1 to 13.