CN111353453B - Obstacle detection method and device for vehicle - Google Patents

Abstract

The embodiment of the present application discloses an obstacle detection method and device for vehicles, which relate to the technical field of automatic driving, especially the technical field of autonomous parking. The above-mentioned vehicle includes an image acquisition device. A specific implementation of the above method includes: acquiring the calibration data of the image acquisition device; acquiring the labeling frame for the obstacle in the image collected by the image acquisition device; determining the estimated distance between the obstacle and the vehicle according to the calibration data and the labeling frame ; According to the position information of the vehicle and the estimated distance, determine multiple candidate points around the vehicle; determine the optimal distance between the obstacle and the vehicle according to the multiple candidate points and the marked frame. This embodiment can improve the accuracy of obstacle distance detection.

Description

Obstacle detection method and device for vehicle

technical field

The embodiments of the present application relate to the field of computer technology, and in particular to an obstacle detection method and device for vehicles.

Background technique

In order to adapt to the requirements of dynamic scenes, self-driving vehicles must be able to detect and identify the surrounding environment, and take corresponding decisions and path planning according to the surrounding environment. The static environment includes buildings, roadsides, trees, etc.; while the dynamic environment includes moving obstacles such as vehicles and people. The identification of obstacles includes the discrimination of the type and size of obstacles, and the detection includes the estimation of obstacle position and attitude. Among them, the estimation of the obstacle position is often simplified as the estimation of the distance between the obstacle and the vehicle, so the obstacle ranging scheme is a key problem and difficulty in automatic driving.

For different sensors, there are different ranging optimization schemes. In the automatic driving scheme based on lidar or millimeter-wave radar, since the radar has the ability of direct ranging, the ranging accuracy itself is high, but the amount of raw data is large. Therefore, the optimization of the ranging scheme is biased towards the calculation time and speed, as well as the elimination of gross errors. In the binocular-based autonomous driving solution, ranging is mainly affected by calibration and stereo matching, and wrong matching often leads to wrong depth calculations. Therefore, the optimization of the ranging scheme is biased towards the optimization of matching accuracy and online and offline calibration. Whether it is a laser or a binocular solution, the sensor cost is relatively large.

With the development of vision, solutions based on monocular sensors are more and more widely used. Obtaining depth through a monocular is a difficult problem in the computer vision community due to the lack of sufficient geometric constraints for the monocular.

Contents of the invention

Embodiments of the present application propose an obstacle detection method and device for vehicles.

In the first aspect, the embodiment of the present application provides an obstacle detection method for a vehicle. The above-mentioned vehicle is equipped with an image acquisition device. The above-mentioned method includes: acquiring the calibration data of the above-mentioned image acquisition device; The annotation frame of the obstacle in the image; according to the above-mentioned calibration data and the above-mentioned annotation frame, determine the estimated distance between the above-mentioned obstacle and the above-mentioned vehicle; according to the position information of the above-mentioned vehicle and the above-mentioned estimated distance, determine a plurality of candidates around the above-mentioned vehicle point; determining an optimal distance between the obstacle and the vehicle according to the plurality of candidate points and the marked frame.

In some embodiments, the above-mentioned marked frame includes at least two touchdown points of the above-mentioned obstacle; and the above-mentioned determining the estimated distance between the above-mentioned obstacle and the above-mentioned vehicle according to the above-mentioned calibration data and the above-mentioned marked frame includes: according to the above-mentioned calibration data and the at least two touchdown points, determining the distance between each touchdown point and the vehicle; and determining the estimated distance according to the distance between each touchdown point and the vehicle.

In some embodiments, the determining a plurality of candidate points around the vehicle according to the position information of the vehicle and the estimated distance includes: determining the selection range of candidate points according to the position information of the vehicle and the estimated distance; The point selection range selects multiple points as candidate points; and determines the three-dimensional coordinate information of the above-mentioned candidate points according to the high-precision map.

In some embodiments, determining the optimal distance between the obstacle and the vehicle based on the plurality of candidate points and the labeling frame includes: projecting to the image coordinate system according to the three-dimensional coordinate information of the candidate points to obtain the above The two-dimensional coordinate information of the candidate point in the above-mentioned image coordinate system; according to the three-dimensional coordinate information of the candidate point, the two-dimensional coordinate information and the two-dimensional coordinate information of the above-mentioned label frame in the above-mentioned image coordinate system, determine the three-dimensional coordinate information of the above-mentioned label frame ; According to the above-mentioned three-dimensional coordinate information, determine the above-mentioned optimization distance.

In some embodiments, determining the three-dimensional coordinate information of the above-mentioned annotation frame based on the three-dimensional coordinate information of the candidate point, the two-dimensional coordinate information, and the two-dimensional coordinate information of the above-mentioned annotation frame in the above-mentioned image coordinate system includes: according to the above-mentioned candidate point The two-dimensional coordinate information of the above-mentioned candidate points and the above-mentioned label frame are determined; the above-mentioned distances are sorted according to the order of small to large, and the candidate points of the previous target number are used as target candidate points; according to the three-dimensional target candidate points The coordinate information, the two-dimensional coordinate information, and the two-dimensional coordinate information of the above-mentioned annotation frame in the above-mentioned image coordinate system determine the three-dimensional coordinate information of the above-mentioned annotation frame.

In some embodiments, the vehicle is equipped with a first image acquisition device and a second image acquisition device with different imaging principles; and the method further includes: according to the estimated distance and the first image acquisition device and the second image acquisition device According to the imaging principle, the target image acquisition device is determined from the first image acquisition device and the second image acquisition device.

In a second aspect, an embodiment of the present application provides an obstacle detection device for a vehicle, where the vehicle is equipped with an image acquisition device, and the device includes: a first acquisition unit configured to acquire calibration data of the image acquisition device; The second acquisition unit is configured to acquire a labeling frame for the obstacle in the image captured by the image acquisition device; the distance estimation unit is configured to determine the distance between the obstacle and the vehicle according to the above-mentioned calibration data and the above-mentioned labeling frame The estimated distance of the candidate point; the candidate point determination unit is configured to determine a plurality of candidate points around the vehicle according to the position information of the vehicle and the estimated distance; the distance optimization unit is configured to , to determine the optimal distance between the obstacle and the vehicle.

In some embodiments, the above-mentioned labeled frame includes at least two touchdown points of the above-mentioned obstacle; and the above-mentioned distance estimation unit is further configured to: according to the above-mentioned calibration data and the above-mentioned at least two touchdown points, determine the distance between each touchdown point and the above-mentioned vehicle the distance between each touchdown point and the above-mentioned vehicle to determine the above-mentioned estimated distance.

In some embodiments, the above-mentioned candidate point determination unit is further configured to: determine the candidate point selection range according to the above-mentioned vehicle position information and the above-mentioned estimated distance; select multiple points in the above-mentioned candidate point selection range as candidate points; The map determines the three-dimensional coordinate information of the above-mentioned candidate points.

In some embodiments, the above-mentioned distance optimization unit is further configured to: project to the image coordinate system according to the three-dimensional coordinate information of the above-mentioned candidate points to obtain the two-dimensional coordinate information of the above-mentioned candidate points in the above-mentioned image coordinate system; The three-dimensional coordinate information, the two-dimensional coordinate information, and the two-dimensional coordinate information of the above-mentioned annotation frame in the above-mentioned image coordinate system are used to determine the three-dimensional coordinate information of the above-mentioned annotation frame; according to the above-mentioned three-dimensional coordinate information, the above-mentioned optimization distance is determined.

In some embodiments, the above-mentioned distance optimization unit is further configured to: determine the distance between the above-mentioned candidate point and the above-mentioned annotation frame according to the two-dimensional coordinate information of the above-mentioned candidate point; sort the above-mentioned distances in order of small to large , using the candidate points of the previous target number as target candidate points; according to the three-dimensional coordinate information, two-dimensional coordinate information of the target candidate point and the two-dimensional coordinate information of the above-mentioned label frame in the above-mentioned image coordinate system, determine the three-dimensional coordinate information of the above-mentioned label frame .

In some embodiments, the above-mentioned vehicle is equipped with a first image acquisition device and a second image acquisition device with different imaging principles; and the above-mentioned device further includes a device determination unit configured to: According to the imaging principle of the second image acquisition device, the target image acquisition device is determined from the first image acquisition device and the second image acquisition device.

In the third aspect, the embodiment of this application provides an electronic device, including: one or more processors; a storage device, on which one or more programs are stored, when the above one or more programs are The processor executes, so that the above-mentioned one or more processors implement the method described in any embodiment of the first aspect.

In a fourth aspect, an embodiment of the present application provides a computer-readable medium, on which a computer program is stored, and when the program is executed by a processor, the method as described in any embodiment of the first aspect is implemented.

The above embodiments of the present application provide an obstacle detection method and device for a vehicle, where an image acquisition device is installed on the vehicle. The method and device of this embodiment can firstly acquire the calibration data of the image acquisition device. And obtain the label frame for the obstacle in the image collected by the image collection device. Then, according to the calibration data and the labeled frame, the estimated distance between the obstacle and the vehicle is determined. Then, according to the position of the vehicle and the estimated distance, a plurality of candidate points around the vehicle are determined. Finally, the optimal distance between the obstacle and the vehicle is determined according to the multiple candidate points and the labeled frame. The method of this embodiment can improve the accuracy of obstacle distance detection.

Description of drawings

Other characteristics, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

FIG. 1 is an exemplary system architecture diagram to which an embodiment of the present application can be applied;

FIG. 2 is a flowchart of an embodiment of an obstacle detection method for a vehicle according to the present application;

FIG. 3 is a schematic diagram of an application scenario of an obstacle detection method for a vehicle according to the present application;

FIG. 4 is a flow chart of another embodiment of an obstacle detection method for a vehicle according to the present application;

Fig. 5 is a schematic structural diagram of an embodiment of an obstacle detection device for a vehicle according to the present application;

Fig. 6 is a schematic structural diagram of a computer system suitable for implementing the electronic device of the embodiment of the present application.

Detailed ways

The application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain related inventions, rather than to limit the invention. It should also be noted that, for the convenience of description, only the parts related to the related invention are shown in the drawings.

It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and embodiments.

FIG. 1 shows an exemplary system architecture 100 to which embodiments of the obstacle detection method for a vehicle or the obstacle detection device for a vehicle of the present application can be applied.

As shown in FIG. 1 , a system architecture 100 may include a vehicle 101 , a network 102 and a server 103 . The network 102 serves as a medium for providing a communication link between the vehicle 101 and the server 103 . Network 102 may include various types of wireless connections.

The vehicle 101 can interact with the server 105 during driving to receive or send messages and the like. Various sensors may be installed on the vehicle 101, such as a monocular camera, a speed sensor, and the like.

The vehicle 101 may be hardware or software. When the vehicle 101 is hardware, it may be various vehicles capable of driving, including automatic driving vehicles, semi-automatic driving vehicles, human-driving vehicles and the like. When the vehicle 101 is software, it can be installed in the vehicles listed above. It can be implemented as a plurality of software or software modules (for example, to provide distributed services), or as a single software or software module. No specific limitation is made here.

The server 103 may be a server that provides various services, for example, a background server that processes information collected during the driving of the vehicle 101 . The background server can analyze and process the received data, and feed back the processing result (such as obstacle distance) to the vehicle 101 .

It should be noted that the server 103 may be hardware or software. When the server 103 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server 103 is software, it may be implemented as multiple software or software modules (for example, for providing distributed services), or as a single software or software module. No specific limitation is made here.

It should be noted that the obstacle detection method for a vehicle provided in the embodiment of the present application may be executed by the vehicle 101 or by the server 103 . Correspondingly, the obstacle detection device for the vehicle can be set in the vehicle 101 or in the server 103 .

It should be understood that the numbers of vehicles, networks and servers in Figure 1 are merely illustrative. There can be any number of vehicles, networks and servers depending on implementation needs.

Continuing to refer to FIG. 2 , a flow 200 of an embodiment of an obstacle detection method for a vehicle according to the present application is shown. In this embodiment, an image acquisition device is installed on the vehicle, and the image acquisition device may include various monocular cameras, such as a monocular wide-angle camera and a monocular fisheye camera. The obstacle detection method for a vehicle of this embodiment may include the following steps:

Step 201, acquiring calibration data of an image acquisition device.

In this embodiment, the executing subject of the vehicle obstacle detection method (such as the vehicle 101 and the server 103 shown in FIG. 1 ) can obtain the calibration data of the image acquisition device in various ways. Here, the calibration data may include internal parameters and external parameters. Wherein, the internal parameters are parameters related to the characteristics of the image acquisition device itself, such as the focal length and pixel size of the camera. External parameters are parameters in the world coordinate system, such as camera position, rotation direction, etc. Calibration data can be obtained by calibrating the image acquisition device.

Step 202, obtaining a label frame for an obstacle in an image captured by an image capture device.

The execution subject may also acquire the annotation frame for the obstacle in the image captured by the image capture device. Specifically, a trained obstacle recognition model can be set inside the execution body, and the above obstacle recognition model can recognize and mark obstacles in the image. Labeling boxes can be used for labeling, and different types of obstacles can be represented by labeling boxes of different colors. The image acquired by the image acquisition device may be input into the above-mentioned obstacle recognition model to obtain the annotation frame of the obstacle in the above-mentioned image. It can be understood that the above-mentioned obstacle recognition model can also be installed in other electronic equipment, and the execution subject can send the image collected by the image acquisition device to the above-mentioned electronic equipment, and receive the annotation frame of the obstacle sent by the above-mentioned electronic equipment.

Step 203, determine the estimated distance between the obstacle and the vehicle according to the calibration data and the marked frame.

In this embodiment, after the execution subject obtains the calibration data and the label frame, the estimated distance between the obstacle and the vehicle can be determined. Specifically, the execution subject can select two points from the label box based on the inverse perspective projection, and project these two points to the plane of Z=1 to obtain the coordinates (x ₁ ,y ₁ )(x ₂ ,y ₂ ) . Combined with the height of the vehicle, using geometric principles, the distances S ₁ and S ₂ from the two points to the vehicle can be calculated. Then, calculate the average value of _S1 and _S2 , and use the average value as the estimated distance between the obstacle and the vehicle.

In some optional implementations of this embodiment, the label frame includes at least two grounding points of the obstacle, then in the above-mentioned step 203, the execution subject may project the above-mentioned at least two grounding points onto the plane of Z=1, Get the distance between at least two touchdown points and the vehicle. And according to the above at least two distances, an estimated distance is determined.

Step 204, according to the position information of the vehicle and the estimated distance, determine a plurality of candidate points around the vehicle.

After the estimated distance is calculated, the execution subject can also obtain the location information of the vehicle. And according to the location information and the estimated distance, multiple candidate points around the vehicle are determined. Specifically, the execution subject may use the vehicle as the center of the circle, use the estimated radius as the radius, and select multiple points from the obtained circle as candidate points. When selecting multiple points, it can be selected randomly or uniformly. The above-mentioned candidate points may be some landmark points on the road surface, such as lane lines, stop lines, and so on. It can be understood that the above candidate points may include three-dimensional coordinate information of the candidate points. The above coordinate information can be obtained from a high-precision map. High-precision maps can include lanes in different directions, lane lines, lane boundaries, stop lines, pedestrian crossings, speed bumps, traffic lights, traffic signs, warning signs, etc., and can also include location information of the above objects.

Step 205, determine the optimal distance between the obstacle and the vehicle according to the plurality of candidate points and the marked boxes.

In this embodiment, after determining a plurality of candidate points, the execution subject can determine the optimal distance between the obstacle and the vehicle in combination with the marked frame. Specifically, the execution subject may calculate the two-dimensional coordinates of multiple candidate points in the image coordinate system, and then combine the two-dimensional coordinates of the annotation frame in the image coordinate system to determine the relative positional relationship between the candidate point and the annotation frame. Then, according to the three-dimensional coordinates of the candidate points and the above-mentioned relative positional relationship, the three-dimensional coordinates of the label frame are determined. An optimal distance between obstacles and the vehicle can thus be determined.

Continue to refer to FIG. 3 , which is a schematic diagram of an application scenario of the obstacle detection method for vehicles according to this embodiment. In the application scenario of FIG. 3 , a monocular camera is installed on the self-driving vehicle 301 and images are collected during driving. After the self-driving vehicle 301 drives near the parking space, it needs to collect images around the parking space. After the processing of steps 201-205, the distance between the front vehicle, the rear vehicle and the self-driving vehicle is obtained, so as to guide the self-driving vehicle Drive to stop exactly at the parking spot.

In the obstacle detection method for vehicles provided by the above-mentioned embodiments of the present application, firstly, the calibration data of the image acquisition device can be acquired. And obtain the label frame for the obstacle in the image collected by the image collection device. Then, according to the calibration data and the labeled frame, the estimated distance between the obstacle and the vehicle is determined. Then, according to the position of the vehicle and the estimated distance, a plurality of candidate points around the vehicle are determined. Finally, the optimal distance between the obstacle and the vehicle is determined according to the multiple candidate points and the labeled frame. The method of this embodiment can improve the accuracy of obstacle distance detection.

Continue referring to FIG. 4 , which shows a process 400 of another embodiment of the obstacle detection method for vehicles according to the present application. In this embodiment, a first image acquisition device and a second image acquisition device with different imaging principles may be installed on the vehicle. The first image device may be a monocular wide-angle camera, and the second image acquisition device may be a monocular fisheye camera. As shown in Figure 4, the obstacle detection method for the vehicle of the present embodiment may include the following steps:

Step 401, acquiring calibration data of an image acquisition device.

When two image acquisition devices are installed on the vehicle, the calibration data of the two image acquisition devices can be obtained here, that is, the calibration data of the first image acquisition device and the second image acquisition device are obtained respectively.

Step 402, obtaining a label frame for an obstacle in an image captured by an image capture device.

In this embodiment, the labeling frame of the obstacle in the first image captured by the first image capturing device and the labeling frame of the obstacle in the second image captured by the second image capturing device may be obtained respectively.

Step 403: Determine the estimated distance between the obstacle and the vehicle according to the calibration data and the marked frame.

The principle of this step is similar to that of step 203, and will not be repeated here.

Step 404: Determine the target image acquisition device from the first image acquisition device and the second image acquisition device according to the estimated distance and the imaging principles of the first image acquisition device and the second image acquisition device.

Different image sensors have different accuracy at different ranging distances due to their different imaging principles. When a monocular wide-angle camera and a monocular fisheye camera are installed in the vehicle at the same time, the appropriate image sensor should be selected according to the characteristics of different image sensors and the distribution range of obstacles. For example, the monocular fisheye camera has a higher ranging accuracy at 2-6 meters, and the monocular wide-angle camera has better ranging accuracy at 6-20 meters. When obstacles appear within a distance of about 14 meters from the vehicle, a monocular wide-angle camera should be used as the main image sensor for ranging, that is, the target image acquisition device.

Step 405, according to the location information of the vehicle and the estimated distance, determine the selection range of the candidate points.

In this embodiment, the execution subject can also acquire the location information of the vehicle, and then combine the estimated distance to determine the selection range of the candidate points. Specifically, the execution subject can use the position of the vehicle as the center of the circle, use twice or three times the estimated distance as the radius, and use the obtained circle as the selection range of candidate points.

Step 406, selecting a plurality of points in the candidate point selection range as candidate points.

After the executive body determines the selection range of candidate points, it can select multiple points as candidate points. Specifically, the execution subject may uniformly select multiple points in the candidate point selection range as candidate points. For example, sample at intervals of 20 cm.

In some optional implementation manners of this embodiment, after determining the selection range of candidate points, the execution subject may also determine the three-dimensional features within the range of selection of candidate points according to the high-precision map. The above-mentioned three-dimensional ground objects may include lane lines. Then select candidate points in the above-mentioned three-dimensional object and candidate point selection range.

Step 407, determine the three-dimensional coordinate information of the candidate points according to the high-precision map.

After selecting the candidate points, the execution subject can combine the high-precision map to determine the three-dimensional coordinate information of the candidate points.

Step 408, according to the three-dimensional coordinate information of the candidate point, project to the image coordinate system to obtain the two-dimensional coordinate information of the candidate point in the image coordinate system.

In this embodiment, the three-dimensional coordinate information of the candidate points is in the earth coordinate system. The execution subject can first transform the three-dimensional coordinate information in the earth coordinate system into the vehicle coordinate system. Then project the three-dimensional coordinate information of the candidate points in the vehicle coordinate system to the image coordinate system. Since the image coordinate system is two-dimensional, you can see the two-dimensional coordinate information of the candidate points in the image coordinate system.

Step 409, according to the three-dimensional coordinate information of the candidate point, the two-dimensional coordinate information and the two-dimensional coordinate information of the annotation frame in the image coordinate system, determine the three-dimensional coordinate information of the annotation frame.

Afterwards, the execution subject can determine the three-dimensional coordinate information of the label frame according to the three-dimensional coordinate information of the candidate point, the two-dimensional coordinate information and the two-dimensional coordinate information of the label frame in the image coordinate system. Specifically, the coordinates of each point of the annotation frame in the image coordinate system are also two-dimensional, and the execution subject can determine the proportional difference between the two-dimensional coordinates. Combined with the three-dimensional coordinate information of the candidate points, the three-dimensional coordinate information of the label frame can be obtained. It can be understood that, here, the three-dimensional coordinate information of the label frame is in the vehicle coordinate system.

In some optional implementations of this embodiment, the execution subject can also implement step 409 through the following steps not shown in FIG. 2: determine the distance between the candidate point and the label frame according to the two-dimensional coordinate information of the candidate point Distance; sort the distances in order from small to large, and use the number of candidate points of the previous target as target candidate points; according to the three-dimensional coordinate information, two-dimensional coordinate information of the target candidate point and the two-dimensional coordinates of the label frame in the image coordinate system Coordinate information, to determine the three-dimensional coordinate information of the label box.

In this implementation manner, after obtaining the two-dimensional coordinate information of each candidate point, the execution subject may calculate the distance between each candidate point and the marked frame. Then the nearest N candidate points are taken as target candidate points. In some application scenarios, N is 4. Finally, the three-dimensional coordinate information of the label frame can be determined according to the three-dimensional coordinate information of the target candidate point, the two-dimensional coordinate information and the two-dimensional coordinate information of the label frame in the image coordinate system. This can effectively reduce the workload of calculation.

Step 410, determine the optimal distance according to the three-dimensional coordinate information.

After obtaining the three-dimensional coordinate information of the label frame, the execution subject can combine the position of the vehicle to calculate the optimal distance between the vehicle and the obstacle.

The obstacle detection method for vehicles provided by the above-mentioned embodiments of the present application can use the candidate points of known three-dimensional coordinate information to determine the three-dimensional coordinate information of the label frame, so that the estimated distance can be optimized to make the obtained distance more accurate .

Further referring to FIG. 5 , as an implementation of the methods shown in the above figures, the present application provides an embodiment of an obstacle detection device for vehicles, which corresponds to the method embodiment shown in FIG. 2 , the device can be specifically applied to various electronic devices. In this embodiment, the vehicle is equipped with an image acquisition device.

As shown in FIG. 5 , the obstacle detection device 500 for vehicles in this embodiment includes: a first acquisition unit 501 , a second acquisition unit 502 , a distance estimation unit 503 , a candidate point determination unit 504 and a distance optimization unit 505 .

The first acquisition unit 501 is configured to acquire calibration data of an image acquisition device.

The second obtaining unit 502 is configured to obtain a label frame for an obstacle in an image collected by the image collection device.

The distance estimating unit 503 is configured to determine the estimated distance between the obstacle and the vehicle according to the calibration data and the marked frame.

The candidate point determining unit 504 is configured to determine a plurality of candidate points around the vehicle according to the position information of the vehicle and the estimated distance.

The distance optimization unit 505 is configured to determine an optimal distance between the obstacle and the vehicle according to the plurality of candidate points and the marked frame.

In some optional implementation manners of this embodiment, the label frame includes at least two grounding points of the obstacle. The distance estimating unit 503 may be further configured to: determine the distance between each touch point and the vehicle according to the calibration data and at least two touch points; and determine the estimated distance according to the distance between each touch point and the vehicle.

In some optional implementations of this embodiment, the candidate point determination unit 504 may be further configured to: determine the candidate point selection range according to the vehicle's position information and the estimated distance; select multiple points within the candidate point selection range as candidate points point; according to the high-precision map, determine the three-dimensional coordinate information of the candidate point.

In some optional implementations of this embodiment, the distance optimization unit 505 may be further configured to: project to the image coordinate system according to the three-dimensional coordinate information of the candidate point to obtain the two-dimensional coordinate information of the candidate point in the image coordinate system ; According to the three-dimensional coordinate information of the candidate point, the two-dimensional coordinate information and the two-dimensional coordinate information of the label frame in the image coordinate system, determine the three-dimensional coordinate information of the label frame; according to the three-dimensional coordinate information, determine the optimal distance.

In some optional implementations of this embodiment, the distance optimization unit 505 may be further configured to: determine the distance between the candidate point and the label frame according to the two-dimensional coordinate information of the candidate point; The order of the target is sorted, and the candidate points of the previous target number are used as target candidate points; according to the three-dimensional coordinate information of the target candidate point, the two-dimensional coordinate information and the two-dimensional coordinate information of the label frame in the image coordinate system, determine the three-dimensional coordinates of the label frame information.

In some optional implementation manners of this embodiment, the vehicle is installed with a first image acquisition device and a second image acquisition device with different imaging principles. The device 600 may further include a device determination unit not shown in FIG. 5, configured to: according to the estimated distance and the imaging principle of the first image capture device and the second image capture device, from the first image capture device and the second image capture device The target image acquisition device is determined in the acquisition device.

It should be understood that the units 501 to 505 described in the obstacle detection device 500 for vehicles respectively correspond to the steps in the method described with reference to FIG. 2 . Therefore, the operations and features described above for the obstacle detection method for vehicles are also applicable to the device 500 and the units contained therein, and will not be repeated here.

Referring now to FIG. 6 , it shows a schematic structural diagram of an electronic device (such as the server in FIG. 1 or the terminal device installed in a vehicle) 600 suitable for implementing an embodiment of the present disclosure. The electronic device shown in FIG. 6 is only an example, and should not limit the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG. 6, an electronic device 600 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which may be randomly accessed according to a program stored in a read-only memory (ROM) 602 or loaded from a storage device 608. Various appropriate actions and processes are executed by programs in the memory (RAM) 603 . In the RAM 603, various programs and data necessary for the operation of the electronic device 600 are also stored. The processing device 601 , ROM 602 and RAM 603 are connected to each other through a bus 604 . An input/output (I/O) interface 605 is also connected to the bus 604 .

Typically, the following devices can be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speaker, vibration an output device 607 such as a computer; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While FIG. 6 shows electronic device 600 having various means, it should be understood that implementing or having all of the means shown is not a requirement. More or fewer means may alternatively be implemented or provided. Each block shown in FIG. 6 may represent one device, or may represent multiple devices as required.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, where the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via communication means 609 , or from storage means 608 , or from ROM 602 . When the computer program is executed by the processing device 601, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are executed. It should be noted that the computer-readable medium described in the embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the embodiments of the present disclosure, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device. The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device: acquires the calibration data of the image acquisition device; acquires the The labeling frame of the obstacle; determine the estimated distance between the obstacle and the vehicle according to the calibration data and the labeling frame; determine multiple candidate points around the vehicle according to the position information of the vehicle and the estimated distance; according to multiple candidate points and the labeling frame , to determine the optimal distance between the obstacle and the vehicle.

Computer program code for carrying out operations of embodiments of the present disclosure may be written in one or more programming languages, or combinations thereof, including object-oriented programming languages—such as Java, Smalltalk, C++, Also included are conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through an Internet service provider). Internet connection).

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.

The units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. The described units can also be set in a processor, for example, it can be described as: a processor includes a first acquisition unit, a second acquisition unit, a distance estimation unit, a candidate point determination unit and a distance optimization unit. Wherein, the names of these units do not constitute a limitation on the unit itself under certain circumstances, for example, the first acquisition unit may also be described as "a unit that acquires the calibration data of the image acquisition device".

The above description is only a preferred embodiment of the present disclosure and an illustration of the applied technical principle. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solution formed by the specific combination of the above-mentioned technical features, but also covers the above-mentioned invention without departing from the above-mentioned inventive concept. Other technical solutions formed by any combination of technical features or equivalent features. For example, a technical solution formed by replacing the above-mentioned features with technical features with similar functions disclosed in (but not limited to) the embodiments of the present disclosure.

Claims (10)

1. An obstacle detection method for a vehicle, the vehicle is equipped with an image acquisition device, the method comprising: Acquiring calibration data of the image acquisition device; Acquiring a label frame for obstacles in the image captured by the image capture device; Determining an estimated distance between the obstacle and the vehicle according to the calibration data and the marked frame includes: determining a distance between each touchdown point and the vehicle according to the calibration data and at least two touchdown points The distance between; according to the distance between each touchdown point and the vehicle, determine the estimated distance; Determining a plurality of candidate points around the vehicle according to the position information of the vehicle and the estimated distance includes: determining a selection range of candidate points according to the position information of the vehicle and the estimated distance; Selecting multiple points as candidate points in the selection range; determining the three-dimensional coordinate information of the candidate points according to the high-precision map; An optimal distance between the obstacle and the vehicle is determined according to the plurality of candidate points and the labeled frame. 2. The method according to claim 1, wherein said determining an optimal distance between said obstacle and said vehicle according to said plurality of candidate points and said labeling frame comprises: According to the three-dimensional coordinate information of the candidate point, project to the image coordinate system to obtain the two-dimensional coordinate information of the candidate point in the image coordinate system; Determine the three-dimensional coordinate information of the label frame according to the three-dimensional coordinate information of the candidate point, the two-dimensional coordinate information, and the two-dimensional coordinate information of the label frame in the image coordinate system; The optimal distance is determined according to the three-dimensional coordinate information. 3. The method according to claim 2, wherein said according to the three-dimensional coordinate information of the candidate point, the two-dimensional coordinate information and the two-dimensional coordinate information of the label frame in the image coordinate system, determine the coordinates of the label frame 3D coordinate information, including: determining the distance between the candidate point and the label frame according to the two-dimensional coordinate information of the candidate point; Sorting the distances in ascending order, using the candidate points of the previous target number as target candidate points; The three-dimensional coordinate information of the labeling frame is determined according to the three-dimensional coordinate information of the target candidate point, the two-dimensional coordinate information, and the two-dimensional coordinate information of the labeling frame in the image coordinate system. 4. The method according to claim 1, wherein the vehicle is equipped with a first image capture device and a second image capture device with different imaging principles; and The method also includes: A target image acquisition device is determined from the first image acquisition device and the second image acquisition device according to the estimated distance and the imaging principles of the first image acquisition device and the second image acquisition device. 5. An obstacle detection device for a vehicle, the vehicle is equipped with an image acquisition device, the device comprising: a first acquisition unit configured to acquire calibration data of the image acquisition device; A second acquisition unit configured to acquire a label frame for an obstacle in an image captured by the image capture device; The distance estimating unit is configured to determine an estimated distance between the obstacle and the vehicle according to the calibration data and the labeled frame, and is further configured to: determine according to the calibration data and at least two touchdown points the distance between each touchdown point and the vehicle; determining the estimated distance according to the distance between each touchdown point and the vehicle; The candidate point determining unit is configured to determine a plurality of candidate points around the vehicle according to the position information of the vehicle and the estimated distance, and is further configured to: determine according to the position information of the vehicle and the estimated distance Candidate point selection range; select multiple points in the candidate point selection range as candidate points; determine the three-dimensional coordinate information of the candidate points according to the high-precision map; The distance optimization unit is configured to determine an optimal distance between the obstacle and the vehicle according to the plurality of candidate points and the labeled frame. 6. The apparatus according to claim 5, wherein the distance optimization unit is further configured to: According to the three-dimensional coordinate information of the candidate point, project to the image coordinate system to obtain the two-dimensional coordinate information of the candidate point in the image coordinate system; Determine the three-dimensional coordinate information of the label frame according to the three-dimensional coordinate information of the candidate point, the two-dimensional coordinate information, and the two-dimensional coordinate information of the label frame in the image coordinate system; The optimal distance is determined according to the three-dimensional coordinate information. 7. The apparatus according to claim 6, wherein the distance optimization unit is further configured to: determining the distance between the candidate point and the label frame according to the two-dimensional coordinate information of the candidate point; Sorting the distances in ascending order, using the candidate points of the previous target number as target candidate points; The three-dimensional coordinate information of the labeling frame is determined according to the three-dimensional coordinate information of the target candidate point, the two-dimensional coordinate information, and the two-dimensional coordinate information of the labeling frame in the image coordinate system. 8. The device according to claim 5, wherein the vehicle is equipped with a first image capture device and a second image capture device with different imaging principles; and The device also includes a device determination unit configured to: A target image acquisition device is determined from the first image acquisition device and the second image acquisition device according to the estimated distance and the imaging principles of the first image acquisition device and the second image acquisition device. 9. An electronic device comprising: one or more processors; a storage device on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors are made to implement the method according to any one of claims 1-4. 10. A computer-readable medium, on which a computer program is stored, wherein, when the program is executed by a processor, the method according to any one of claims 1-4 is implemented.