Disclosure of Invention
In view of the above, it is necessary to provide an automatic driving method, an automatic driving apparatus, a computer device, and a storage medium, which can improve driving safety and increase diversity of driving route selection modes during automatic driving.
An autonomous driving method comprising:
acquiring an indication sign of a driving direction on a current driving lane and a state sign of a front traffic signal lamp;
sensing a distance between a current vehicle and a surrounding object;
and acquiring an action indicating signal of the current automatic driving of the vehicle according to the indicating mark, the state mark and the distance.
In one embodiment, the automatic driving method, which acquires a first video image of a road sign indication in front of a vehicle on a current driving lane and a second video image of a traffic signal lamp, comprises:
the method comprises the steps of collecting a first video image indicated by a road sign in front of a vehicle on a current driving lane and a second video image of a traffic signal lamp, and automatically adjusting the frequency of collecting the first video image and the second video image according to the current vehicle speed.
In one embodiment, the automatic driving method, before obtaining the indication sign of the driving direction on the current driving lane and the status sign of the front traffic signal lamp, comprises:
the method comprises the steps of collecting a first video image indicated by a road sign in front of a vehicle on a current driving lane and a second video image of a traffic signal lamp.
In one embodiment, the automatic driving method, which obtains the indication sign of the driving direction on the current driving lane and the status sign of the front traffic signal lamp, includes:
and acquiring an indication sign of the driving direction on the current driving lane by analyzing the first video image, and acquiring a state sign of a front traffic signal lamp by analyzing the second video image.
In one embodiment, the automatic driving method, which obtains the indication sign of the driving direction on the current driving lane by analyzing the first video image and obtains the status sign of the front traffic signal lamp by analyzing the second video image, includes:
acquiring the indicator by inputting the first video image into a first analysis model established in advance; the first analysis model is obtained by training a first image sample indicated by a road sign on the lane;
acquiring the state mark by inputting the second video image into a pre-established second analysis model; the second analysis model is obtained by training a second image sample of the traffic signal lamp.
In one embodiment, the automatic driving method, the sensing a distance between the current vehicle and a surrounding object, includes:
the distance between the current vehicle and surrounding objects is measured through a laser range finder arranged on the roof of the current vehicle and a radar arranged on the head of the current vehicle.
In one embodiment, the automatic driving method further includes:
and according to the action indication signal, indicating a controller on the current vehicle to control the current vehicle to automatically drive.
An autopilot device comprising:
the acquisition module is used for acquiring an indication sign of the driving direction on the current driving lane and a state sign of a front traffic signal lamp;
the distance sensing module is used for sensing the distance between the current vehicle and the surrounding objects;
and the indicating module is used for acquiring an action indicating signal of the current automatic driving of the vehicle according to the indicating mark, the state mark and the distance.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
acquiring an indication sign of a driving direction on a current driving lane and a state sign of a front traffic signal lamp;
sensing a distance between a current vehicle and a surrounding object;
and acquiring an action indicating signal of the current automatic driving of the vehicle according to the indicating mark, the state mark and the distance.
A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:
acquiring an indication sign of a driving direction on a current driving lane and a state sign of a front traffic signal lamp;
sensing a distance between a current vehicle and a surrounding object;
and acquiring an action indicating signal of the current automatic driving of the vehicle according to the indicating mark, the state mark and the distance.
According to the automatic driving method, the automatic driving device, the automatic driving computer equipment and the storage medium, the indication mark of the driving direction on the current driving lane and the state mark of the front traffic light are obtained, the distance between the current vehicle and the surrounding objects is sensed, the action indication signal of the automatic driving of the current vehicle is obtained according to the indication mark, the state mark and the distance, when the driving route is selected, the distance between the current vehicle and the surrounding objects is considered, the indication mark of the driving direction on the current driving lane and the state mark of the front traffic light are combined to obtain the action indication signal of the automatic driving of the current vehicle, the driving safety is improved, and the diversity of the driving route selection mode is increased.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
The automatic driving method provided by the application can be applied to the application environment shown in fig. 1. Wherein the terminal 102 and the server 104 communicate through a wired or wireless network. The terminal 102 may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented by a server stored locally or a server cluster formed by a plurality of servers stored remotely.
In one embodiment, as shown in fig. 2, an automatic driving method is provided, which is described by taking the method as an example applied to the server in fig. 1, and includes the following steps:
step 202, acquiring an indication sign of the driving direction on the current driving lane and a state sign of a front traffic light.
Specifically, the indication sign of the driving direction on the current driving lane and the status sign of the front traffic signal lamp can be acquired through a camera preset on the vehicle. The cameras for acquiring the state signs of the traffic signal lamps in front can be multiple, and one camera for acquiring the indication sign of the driving direction on the current driving lane can be provided.
Step 204, the distance between the current vehicle and the surrounding objects is sensed.
Specifically, the distance information between the current vehicle and the surrounding object (obstacle) may be acquired by a ranging radar, a laser range finder, or the like preset on the vehicle.
And step 206, acquiring an action indication signal of the current automatic driving of the vehicle according to the indication mark, the state mark and the distance.
Specifically, the action indicating signal of the current vehicle automatic driving may include one of four states of "ready to run", "ready to stop", and "stop".
According to the automatic driving method, the indication mark of the driving direction on the current driving lane and the state mark of the front traffic light are obtained, the distance between the current vehicle and the surrounding objects is sensed, the action indication signal of the automatic driving of the current vehicle is obtained according to the indication mark, the state mark and the distance, and when the driving route is selected, the distance between the current vehicle and the surrounding objects is considered, the action indication signal of the automatic driving of the current vehicle is obtained by combining the indication mark of the driving direction on the current driving lane and the state mark of the front traffic light, so that the driving safety is improved, and the diversity of the driving route selection mode is increased.
In one embodiment, step 202 may be preceded by the steps of: the method comprises the steps of collecting a first video image indicated by a road sign in front of a vehicle on a current driving lane and a second video image of a traffic signal lamp.
In the automatic driving method, when the ground road sign indicating sign is collected, a road sign indicating video image signal on the current driving lane can be obtained through a preset video collecting device (such as a camera) on the vehicle, and the frequency of the collected image is automatically adjusted according to the current vehicle speed in the process of obtaining the video image. When the state mark of the traffic signal lamp in the front is collected, the video image signal of the traffic signal lamp in the front can be acquired through the preset video acquisition equipment on the vehicle, more than one video acquisition equipment of the traffic signal lamp can be set, and distance detection and signal lamp identification are realized according to the pre-established binocular vision model.
In the embodiment, the indication sign of the driving direction on the current driving lane and the state sign of the front traffic light are acquired, the distance between the current vehicle and the surrounding object is sensed, the action indication signal of the automatic driving of the current vehicle is acquired according to the indication sign, the state sign and the distance, and when the driving route is selected, the distance between the current vehicle and the surrounding object is considered, the action indication signal of the automatic driving of the current vehicle is acquired by combining the indication sign of the driving direction on the current driving lane and the state sign of the front traffic light, so that the driving safety is improved, and the diversity of the driving route selection mode is increased.
In one embodiment, when the first video image of the road sign indication in front of the vehicle on the current driving lane and the second video image of the traffic signal lamp are collected, the frequency of collecting the first video image and the second video image can be automatically adjusted according to the current vehicle speed.
In the above embodiment, the frequency of acquiring the first video image and the second video image is automatically adjusted according to the current vehicle speed, so that memory resources are not consumed on the premise of acquiring necessary video images.
In the embodiment, the indication sign of the driving direction on the current driving lane and the state sign of the front traffic light are acquired, the distance between the current vehicle and the surrounding object is sensed, the action indication signal of the automatic driving of the current vehicle is acquired according to the indication sign, the state sign and the distance, and when the driving route is selected, the distance between the current vehicle and the surrounding object is considered, the action indication signal of the automatic driving of the current vehicle is acquired by combining the indication sign of the driving direction on the current driving lane and the state sign of the front traffic light, so that the driving safety is improved, and the diversity of the driving route selection mode is increased.
In one embodiment, the indication sign of the driving direction on the current driving lane and the status sign of the traffic signal light ahead may be acquired by: and acquiring an indication sign of the driving direction on the current driving lane by analyzing the first video image, and acquiring a state sign of a front traffic signal lamp by analyzing the second video image.
In the above embodiment, when the indication sign of the driving direction on the current driving lane is obtained, the road sign indication on the current driving lane may be obtained according to the pre-established machine learning analysis model, and the traffic light state of the current driving intersection may be obtained according to the pre-established machine learning analysis model.
In the embodiment, the indication sign of the driving direction on the current driving lane and the state sign of the front traffic light are acquired, the distance between the current vehicle and the surrounding object is sensed, the action indication signal of the automatic driving of the current vehicle is acquired according to the indication sign, the state sign and the distance, and when the driving route is selected, the distance between the current vehicle and the surrounding object is considered, the action indication signal of the automatic driving of the current vehicle is acquired by combining the indication sign of the driving direction on the current driving lane and the state sign of the front traffic light, so that the driving safety is improved, and the diversity of the driving route selection mode is increased.
In one embodiment, the indicator of the driving direction on the current driving lane and the status flag of the traffic signal lamp in front may be acquired by performing the following steps: acquiring an indication mark by inputting a first video image into a first analysis model established in advance; the first analysis model is obtained by training a first image sample indicated by a road sign on the lane; acquiring a state mark by inputting a second video image into a pre-established second analysis model; the second analysis model is trained from a second image sample of the traffic signal lamp.
The ground road markings in the first image sample may include: the left turn, the straight line, the right turn, the left turn plus the straight line, the right turn plus the straight line and the identification with uncertain types are totally 6 types; the traffic signal status in the second image sample may include: circular lamps, left-turn arrow lamps, straight arrow lamps, right-turn arrow lamps, signal lamps with uncertain types and non-signal lamps are 6 types. As shown in fig. 3, the ground road sign indication and the traffic light status can be obtained by the following steps:
step 302, preliminarily detecting a road sign image area aiming at a first video image; and aiming at the second video image, preliminarily detecting the lamp panel position of the traffic signal lamp in the image plane by using a monocular vision model and an active learning method.
The detection classifier of the traffic signal lamp panel can be trained by pre-establishing a two-dimensional depth network according to the monocular vision model, and the candidate area of the traffic signal lamp panel is found out preliminarily by performing traversal search on a single image. The traffic signal lamp panel detection classifier is composed of a visual layer, a hidden layer 1, a hidden layer 2 and a label layer, wherein the visual layer and the hidden layer are connected through a group of weights, and the network weights are obtained through unsupervised training and supervised training.
And 304, acquiring the depth information of the detected traffic signal lamp by using the binocular vision model, and screening out a correct traffic signal lamp panel for subsequent further identification.
The correct traffic signal lamp panel area can be screened out by establishing a position information model based on the traffic signal lamps in an image coordinate system and a world coordinate system according to a binocular vision model for candidate areas possibly having the traffic signal lamp panel, and the approximate position relation between the traffic signal lamps and the vehicles can be obtained.
And step 306, identifying the ground road sign indication and the traffic light state through the established machine learning model.
The ground road sign indicating and the screened traffic signal lamp panel area can be trained off line respectively, a ground road sign indicating training model and a traffic signal lamp training model are established, a ground road sign indicating classifier and a traffic signal lamp state recognition classifier are obtained, and then relevant recognition information is obtained.
In the embodiment, the indication sign of the driving direction on the current driving lane and the state sign of the front traffic light are acquired, the distance between the current vehicle and the surrounding object is sensed, the action indication signal of the automatic driving of the current vehicle is acquired according to the indication sign, the state sign and the distance, and when the driving route is selected, the distance between the current vehicle and the surrounding object is considered, the action indication signal of the automatic driving of the current vehicle is acquired by combining the indication sign of the driving direction on the current driving lane and the state sign of the front traffic light, so that the driving safety is improved, and the diversity of the driving route selection mode is increased.
In one embodiment, the distance between the current vehicle and the surrounding objects may be perceived by: the distance between the current vehicle and surrounding objects is measured through a laser range finder arranged on the roof of the current vehicle and a radar arranged on the head of the current vehicle.
In the above embodiment, as shown in fig. 4, a laser range finder may be disposed at the roof position, a plurality of radars may be disposed at the head position, a plurality of traffic signal light video capture cameras may be disposed at the roof position, and a ground road sign capture camera may be disposed at the head position.
In the embodiment, the indication sign of the driving direction on the current driving lane and the state sign of the front traffic light are acquired, the distance between the current vehicle and the surrounding object is sensed, the action indication signal of the automatic driving of the current vehicle is acquired according to the indication sign, the state sign and the distance, and when the driving route is selected, the distance between the current vehicle and the surrounding object is considered, the action indication signal of the automatic driving of the current vehicle is acquired by combining the indication sign of the driving direction on the current driving lane and the state sign of the front traffic light, so that the driving safety is improved, and the diversity of the driving route selection mode is increased.
In one embodiment, the automatic driving method may further include the steps of: and according to the action indication signal, indicating a controller on the current vehicle to control the current vehicle to automatically drive.
In the above embodiment, the action indication signal may be obtained according to the decision table shown in table 1. The next driving action of the vehicle can be acquired according to the state condition information. The vehicle state condition information may include a traffic light turning state and a distance from a preceding obstacle, and the driving action may include a left turn, a straight run, and a right turn.
TABLE 1
In the embodiment, the indication sign of the driving direction on the current driving lane and the state sign of the front traffic light are acquired, the distance between the current vehicle and the surrounding object is sensed, the action indication signal of the automatic driving of the current vehicle is acquired according to the indication sign, the state sign and the distance, and when the driving route is selected, the distance between the current vehicle and the surrounding object is considered, the action indication signal of the automatic driving of the current vehicle is acquired by combining the indication sign of the driving direction on the current driving lane and the state sign of the front traffic light, so that the driving safety is improved, and the diversity of the driving route selection mode is increased.
It should be understood that although the steps in the flowcharts of fig. 2 and 3 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2 and 3 may include multiple sub-steps or multiple stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or stages is not necessarily sequential, but may be performed alternately or alternately with other steps or at least some of the sub-steps or stages of other steps.
In one embodiment, as shown in fig. 5, there is provided an automatic driving apparatus including: the system comprises an acquisition module 502, a distance sensing module 504 and an indication module 506, wherein the acquisition module 502 and the distance sensing module 504 are respectively connected with the indication module 506.
An obtaining module 502, configured to obtain an indication sign of a driving direction on a current driving lane and a status sign of a traffic signal light in front;
a distance sensing module 504 for sensing a distance between the current vehicle and a surrounding object;
and the indicating module 506 is used for acquiring an action indicating signal of the current automatic driving of the vehicle according to the indicating mark, the state mark and the distance.
In one embodiment, as shown in fig. 6, there is further provided an automatic driving apparatus, wherein the obtaining module 502 includes: a video acquisition module 5021 and an image analysis module 5022. In addition, a control module 508 may be included. The video capture module 5021 is connected to the image analysis module 5022, and the image analysis module 5022, the distance sensing module 504 and the control module 508 are respectively connected to the indication module 506.
The video collecting module 5021 is used for collecting a first video image indicated by a road sign in front of a vehicle on a current driving lane and a second video image of a traffic signal lamp.
The image analysis module 5022 is used for obtaining the indication sign of the driving direction on the current driving lane by analyzing the first video image and obtaining the state sign of the front traffic signal lamp by analyzing the second video image.
And the control module 508 is used for instructing a controller on the current vehicle to control the current vehicle to automatically drive according to the action indication signal.
For specific limitations of the automatic driving device, reference may be made to the above limitations of the automatic driving method, which are not described in detail herein. The various modules in the autopilot unit described above may be implemented in whole or in part by software, hardware, and combinations thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.
It should be noted that the terms "first \ second \ third" related to the embodiments of the present invention are only used for distinguishing similar objects, and do not represent a specific ordering for the objects, and it should be understood that "first \ second \ third" may exchange a specific order or sequence order if allowed. It should be understood that the terms first, second, and third, as used herein, are interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in other sequences than those illustrated or otherwise described herein.
The terms "comprises" and "comprising," and any variations thereof, of embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or (module) elements is not limited to only those steps or elements but may alternatively include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
Reference herein to "a plurality" means two or more. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.
In one embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 7. The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used to store the autopilot data. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement an autopilot method.
Those skilled in the art will appreciate that the architecture shown in fig. 7 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the following steps when executing the computer program:
acquiring an indication sign of a driving direction on a current driving lane and a state sign of a front traffic signal lamp;
sensing a distance between a current vehicle and a surrounding object;
and acquiring the action indication signal of the current automatic driving of the vehicle according to the indication mark, the state mark and the distance.
In one embodiment, a computer-readable storage medium is provided, having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring an indication sign of a driving direction on a current driving lane and a state sign of a front traffic signal lamp;
sensing a distance between a current vehicle and a surrounding object;
and acquiring the action indication signal of the current automatic driving of the vehicle according to the indication mark, the state mark and the distance.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), synchronous link (S7nchlink) DRAM (SLDRAM), Rambus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.