CN113682319B - Camera adjustment method and device, electronic equipment and storage medium - Google Patents

Abstract

The application discloses a camera adjusting method and device, electronic equipment and storage medium, wherein the camera adjusting method comprises the following steps: determining an abnormal camera and a normal camera in N cameras, wherein N is more than or equal to 2; acquiring parameters of an abnormal camera and a normal camera; calculating a blind area caused by the abnormal camera according to parameters of the abnormal camera and the normal camera; if the maximum visible area of the normal camera can cover the blind area caused by the abnormal camera, calculating parameters to be adjusted by the normal camera to obtain a calculation result; and adjusting the normal camera according to the calculation result. According to the technical scheme, the vehicle can be controlled to continue running according to the adjusted visual field coverage range of the normal camera, or the vehicle is controlled to run to a safety area for stopping, so that the safety problems of traffic regulation violation, traffic accidents and the like caused by direct in-situ stopping of the vehicle are avoided.

Description

Camera adjustment method and device, electronic equipment and storage medium

technical field

The present application relates to the technical field of self-driving vehicles, and in particular to a camera adjustment method and device, electronic equipment, and a storage medium.

Background technique

At present, in the field of artificial intelligence technology, cameras are usually used. For example, in the application scenario of automatic driving, the camera is usually used as a sensing unit, so that the central control computer of the vehicle can control the movement of the vehicle according to the sensing results reported by the sensing unit. However, when the camera fails, the faulty camera will not be able to provide input signals for the corresponding equipment, or the effectiveness of the input signal for the corresponding equipment will be reduced, thus causing the equipment to fail to work normally.

Therefore, when the camera fails, how to ensure the normal operation of the corresponding equipment becomes an urgent problem to be solved.

Contents of the invention

In order to solve the above technical problems, embodiments of the present application provide a method and device for adjusting a camera, electronic equipment, and a storage medium.

According to one aspect of the present application, a camera adjustment method is provided, including: determining abnormal cameras and normal cameras among N cameras, wherein N≥2; obtaining parameters of abnormal cameras and normal cameras; Calculate the blind area caused by the abnormal camera by parameter calculation; if the maximum visible area of the normal camera can cover the blind area caused by the abnormal camera, calculate the parameters that need to be adjusted by the normal camera, and obtain the calculation result; adjust the normal camera according to the calculation result.

According to another aspect of the present application, a camera adjustment device is provided, including: a determination module, used to determine an abnormal camera and a normal camera among N cameras, where N≥2; an acquisition module, used to obtain the abnormal camera and the normal camera The parameters of the camera; the calculation module is used to calculate the blind area caused by the abnormal camera according to the parameters of the abnormal camera and the normal camera; if the maximum visible area of the normal camera can cover the blind area caused by the abnormal camera, calculate the parameters that need to be adjusted by the normal camera, and get the calculation Result; an adjustment module, used to adjust the normal camera according to the calculation result.

According to another aspect of the present application, an electronic device is provided, including: a processor; and a memory, in which computer program instructions are stored, and when the computer program instructions are executed by the processor, the processor executes the above-mentioned camera adjustment method .

According to still another aspect of the present application, a computer-readable storage medium is provided. Computer program instructions are stored on the computer-readable storage medium. When executed by a processor, the computer program instructions cause the processor to execute the above camera adjustment method.

The embodiment of the present application provides a camera adjustment method and device, electronic equipment and storage medium, by determining the blind area of the abnormal camera according to the parameters of the abnormal camera, and calculating the parameters that need to be adjusted by the normal camera according to the parameters of the normal camera and the blind area of the abnormal camera , so as to adjust the normal camera according to the parameters that need to be adjusted, so that the central control computer can control the vehicle to continue driving according to the adjusted field of view coverage of the normal camera, or control the vehicle to drive to a safe area to stop, so as to avoid the violation caused by the vehicle directly parking on the spot Traffic rules, and safety issues such as traffic accidents.

Description of drawings

The above and other objects, features and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification, and are used together with the embodiments of the present application to explain the present application, and do not constitute limitations to the present application. In the drawings, the same reference numerals generally represent the same components or steps.

Fig. 1 is a schematic diagram of a system architecture of a camera adjustment system provided by an exemplary embodiment of the present application.

Fig. 2 is a schematic flowchart of a camera adjustment method provided by an exemplary embodiment of the present application.

Fig. 3 is a schematic flowchart of calculating blind spots caused by abnormal cameras provided by another exemplary embodiment of the present application.

Fig. 4 is a schematic flowchart of a camera adjustment method provided by another exemplary embodiment of the present application.

Fig. 5 is a schematic flowchart of adjusting a normal camera provided by another exemplary embodiment of the present application.

Fig. 6 is a schematic diagram of a field of view coverage of a vehicle camera provided by an exemplary embodiment of the present application.

Fig. 7 is a schematic diagram of the field of view coverage of the vehicle camera when one camera of the vehicle is abnormal according to an exemplary embodiment of the present application.

Fig. 8 is a schematic diagram of a field of view coverage of a vehicle camera after a normal camera is adjusted by using a camera adjustment method provided by an exemplary embodiment of the present application.

Fig. 9 is a schematic structural diagram of a camera adjustment device provided by an exemplary embodiment of the present application.

Fig. 10 is a schematic structural diagram of a calculation module of a camera adjustment device provided by another exemplary embodiment of the present application.

Fig. 11 is a schematic structural diagram of a camera adjustment device provided by another exemplary embodiment of the present application.

Fig. 12 is a schematic structural diagram of an adjustment module of a camera adjustment device provided in another exemplary embodiment of the present application.

Fig. 13 is a block diagram of an electronic device provided by an exemplary embodiment of the present application.

Detailed ways

Hereinafter, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Apparently, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described here.

Application overview

In L4 (highly automated) and above automatic driving scenarios, multiple cameras are usually set up around the vehicle as perception units, so that the vehicle's central control computer can make decisions based on the perception results reported by the perception unit and then control the vehicle movement. However, during the driving process, if a certain camera of the vehicle is abnormal, there will be a blind spot in the field of vision, which will lead to safety hazards in the subsequent automatic driving process of the vehicle.

Aiming at the problem of potential safety hazards in automatic driving due to abnormalities in the camera, the existing solution is to stop the vehicle on the spot to prevent the vehicle from continuing to drive. However, parking directly on the spot may violate traffic regulations (such as parking in a place where parking is prohibited), hinder traffic, and even cause more dangerous consequences due to inappropriate parking positions (such as parking in the middle of the road may be rear-ended by the car behind) .

exemplary system

Fig. 1 is a schematic diagram of the system architecture of a camera adjustment system 1 provided by an exemplary embodiment of the present application. As shown in FIG. 1 , the camera adjustment system 1 includes an automatic driving vehicle 10 , and the automatic driving vehicle 10 includes a first camera 11 , a second camera 12 and a central control computer 13 . The central control computer 13 communicates with the first camera 11 and the second camera 12 . The first camera 11 and the second camera 12 can be installed at different positions of the self-driving vehicle 10 to collect images related to the surrounding environment of the self-driving vehicle 10 , wherein the first camera 11 and the second camera 12 have different field of view coverages.

The central control computer 13 is used to perceive the surrounding environment according to the images collected by the first camera 11 and the second camera 12 , and make corresponding decisions to control the driving process of the self-driving vehicle 10 . When there is an abnormal camera in the first camera 11 and the second camera 12, the central control computer 13 can determine the difference between the abnormal camera in the first camera 11 and the second camera 12 according to the images collected by the first camera 11 and the second camera 12. camera, and determine the parameters that need to be adjusted for the normal camera according to the parameters of the abnormal camera and the normal camera, and then adjust the normal camera according to the parameters that need to be adjusted.

In an embodiment, there may be three or more cameras communicatively connected to the central control computer 13, wherein there may be one or more abnormal cameras.

It should be noted that the above application scenarios are only shown for the convenience of understanding the principles of the present application, and the embodiments of the present application are not limited thereto. On the contrary, the embodiments of the present application can be applied to any scene that may be applicable.

exemplary method

Fig. 2 is a schematic flowchart of a camera adjustment method provided by an exemplary embodiment of the present application. The execution subject of this embodiment may be an electronic device, such as the central control computer in FIG. 1 . As shown in FIG. 2 , the method includes the following steps.

Step 110: Determine abnormal cameras and normal cameras among the N cameras, where N≥2.

Specifically, the N cameras can be installed around the vehicle according to certain preset positions, and the surroundings can refer to the surroundings of the roof, such as the front of the roof, the rear of the roof, the left side of the roof, the right side of the roof, and the like. The cameras may be used to capture images of the surroundings of the vehicle, where the vehicle may be an autonomous vehicle, such as an autonomous vehicle with a driver, or an autonomous vehicle without a driver. The abnormality of the camera may be caused by the lens being blocked, external damage, device aging and other reasons.

In an embodiment, there is no problem with the hardware of the abnormal camera, and the image can be collected, but there is a problem with the collected image. At this time, the abnormal image may be determined according to the output images collected by the N cameras, and then the camera corresponding to the abnormal image may be determined as the abnormal camera. For example, the reason for the abnormality of the camera may be that the lens of the camera is partially blocked or completely blocked. In this embodiment, if the lens of the camera is blocked or partially blocked, the frame-by-frame comparison method can be used to determine the amount of image change between multiple frames; or the pixel value distribution detection method, that is, the statistical distribution of the Y value of the pixel is significantly abnormal, etc. Algorithm detected. It should be noted that abnormal images collected by the camera include but are not limited to lens occlusion faults, such as out of focus, abnormal color, etc., which will also make the camera image unable to meet the required imaging requirements. All in all, there are mature algorithms that can detect abnormal images.

In another embodiment, there is a hardware problem with the abnormal camera, which makes it impossible to capture images. At this time, the abnormal camera can be determined by detecting the running status of the camera. For example, when the central control computer detects that the communication bus data of the camera is interrupted or the camera fails to respond to the access signal of the central control computer, the central control computer can determine the camera as an abnormal camera.

Of course, in addition to the above method, other suitable methods may also be used to determine the abnormal camera, which is not limited in this embodiment of the present application.

Step 120: Obtain the parameters of the abnormal camera and the normal camera.

Specifically, obtaining the parameters of the abnormal camera and the normal camera includes: obtaining the current pose and viewing angle of the abnormal camera and the normal camera.

The parameters of the camera can include the current pose and perspective. Pose can refer to the position and angle of the camera. In general, pose can refer to the installation position and installation angle of the camera, which can be entered into the central control computer when the camera is installed. Optionally, although the installation position of the camera generally does not change, the camera may rotate during use, so the central control computer can monitor the angle of the camera in real time, so as to obtain the current pose of the camera.

The angle of view may include a horizontal angle of view and a vertical angle of view, and the angle of view may be used to characterize the coverage of the camera's field of view. When installing the camera, the viewing angle of the camera can be determined according to the model of the camera.

Step 130: Calculate the blind area caused by the abnormal camera according to the parameters of the abnormal camera and the normal camera.

In an embodiment, different cameras have different field of view coverage areas, so the blind area caused by the abnormal camera can be directly determined according to the parameters of the abnormal camera. For example, the spatial range of the blind spot is determined according to the current pose and viewing angle of the abnormal camera.

In another embodiment, the number of cameras installed around the vehicle is large, so that the coverage areas of the adjacent cameras overlap. In this way, part of the field of vision coverage of the abnormal camera can be covered by adjacent cameras, so at this time, the blind area caused by the abnormal camera can be calculated according to the parameters of the normal camera and the abnormal camera.

Step 140: If the maximum visible area of the normal camera can cover the blind area caused by the abnormal camera, calculate the parameters that need to be adjusted by the normal camera, and obtain the calculation result.

Each camera corresponds to a visible area, and the visible area may be determined according to parameters of the camera, for example, determined according to the position and angle of the camera.

In an embodiment, the visible area corresponding to each camera can be entered into the central control computer in advance, and the central control computer sums the visible areas of each normal camera to obtain the maximum visible area.

In another embodiment, the maximum visible area of the normal camera may be calculated according to parameters of the normal camera. Specifically, during the use of the vehicle, the angle of the camera may change, thereby causing a change in the viewing area of the camera. Therefore, the central control computer can detect the parameters of the cameras in real time, and then determine the visible area of each normal camera according to the parameters of each normal camera, and sum the visible areas of all normal cameras to obtain the maximum visible area. In this way, the accurate maximum visible area can be obtained in real time, which is convenient for subsequent adoption of appropriate security policies.

Step 150: Adjust the normal camera according to the calculation result.

When the maximum visible area of the normal camera can cover the blind spot caused by the abnormal camera, it means that most of the surrounding environment of the vehicle can be perceived by the central control computer of the vehicle. At this time, the central control computer can adjust the parameters of the normal camera so that the normal camera’s The field of view covers the most important orientations. For example, the abnormal camera is located on the right side of the vehicle. At this time, the vehicle needs to merge into the right lane, so the right side of the vehicle is the most important orientation. The central control computer can adjust the angle of the normal camera so that the field of view on the right side of the vehicle can be covered by the camera. .

When the maximum visible area of the normal camera cannot cover the blind area caused by the abnormal camera, it means that most of the surrounding environment of the vehicle cannot be sensed by the central control computer of the vehicle. At this time, the risk of continuing to drive the vehicle is high, so the central control computer can directly control the vehicle Parking, or implementing other preset safety policies.

The embodiment of the present application provides a camera adjustment method, by determining the blind area of the abnormal camera according to the parameters of the abnormal camera, and calculating the parameters that need to be adjusted by the normal camera according to the parameters of the normal camera and the blind area of the abnormal camera, so as to adjust the parameters according to the needs Normal camera, so that the central control computer can control the vehicle to continue driving according to the adjusted normal camera's field of view coverage, or control the vehicle to drive to a safe area to stop, avoiding the violation of traffic rules caused by the vehicle directly parked in place, and traffic accidents and other safety issues. question.

Fig. 3 is a schematic flowchart of calculating blind spots caused by abnormal cameras provided by another exemplary embodiment of the present application. On the basis of the embodiment shown in Fig. 2 of the present application, the embodiment shown in Fig. 3 of the present application is extended, and the difference between the embodiment shown in Fig. 3 and the embodiment shown in Fig. 2 is emphasized below, and the similarities are no longer repeat.

As shown in FIG. 3, in the camera adjustment method provided in the embodiment of the present application, the blind area caused by the abnormal camera is calculated according to the parameters of the abnormal camera and the normal camera (ie step 130), including:

Step 131: Calculate the invisible area according to the current pose and viewing angle of the abnormal camera.

Step 132: Calculate the visible area according to the current pose and viewing angle of the normal camera.

According to the current pose and angle of view of the camera, the field of view coverage of the camera can be determined. When the camera is a normal camera, the field of view coverage is the visible area of the normal camera. When the camera is an abnormal camera, the field of view coverage is invisible to the abnormal camera. area.

The viewing angle of the camera is a fixed parameter of the camera and is used to characterize the field of view coverage of the camera. For example, the viewing angle of the camera can be determined according to the model of the camera. Combining with the viewing angle, installation position and installation angle of the camera, the visible area of the camera installed on the vehicle can be determined. For example, the coordinates of the installation position of the camera can be used as the center of the circle, and a fan-shaped plane can be obtained by drawing a radius along the center of the circle according to the viewing angle and installation angle of the camera, and the fan-shaped plane can represent the visible area. The central control computer can construct a coordinate system (such as a polar coordinate system), and determine the installation position coordinates (center coordinates) of each camera in this coordinate system, and then can express the visible area of a normal camera in the form of coordinates or coordinate equations , similarly, the invisible area of the abnormal camera can also be expressed in the form of coordinates or coordinate equations.

Step 133: Determine the blind area caused by the abnormal camera according to the visible area and the invisible area.

Specifically, when the number of cameras installed around the vehicle is large, the coverage areas of the adjacent cameras may overlap. Part of the field of view coverage of the abnormal camera can be covered by the adjacent camera, and this part of the field of view covered by the adjacent camera is visible and does not belong to the blind spot. Therefore, the central control computer can use the remaining part of the field of vision coverage of the abnormal camera after removing the part covered by the adjacent normal camera as a blind area. In this way, the scope of the blind area can be accurately determined, and the parameters that need to be adjusted by the normal camera, such as the rotation angle, can be determined more reasonably, so that the adjusted field of view coverage of the normal camera can be as large as possible while compensating for the lack of vision in the blind area.

According to an embodiment of the present application, calculating the parameters that need to be adjusted by the normal camera includes: calculating the rotation angle that needs to be adjusted for at least one normal camera that needs to be adjusted, so that the adjusted field of view of all normal cameras covers the blind area caused by the abnormal camera.

Specifically, when the blind spot caused by the abnormal camera is very important for the next driving strategy, the central control computer can adjust the rotation of at least one normal camera so that the adjusted field of view of all normal cameras covers the blind spot caused by the abnormal camera.

In an embodiment, the rotation angles that other normal cameras need to adjust when a certain camera is abnormal can be set in advance. For example, when the camera directly in front of the roof is abnormal, the rotation angle of other normal cameras is preset. When the camera directly in front of the roof is abnormal, the central control computer can directly control the rotation of the normal camera according to the preset rotation angle, so that the adjusted The field of view of all normal cameras covers the blind spots caused by abnormal cameras.

In another embodiment, the central control computer can determine the range of the visible area of the normal camera adjacent to the abnormal camera that needs to be adjusted in the coordinate system based on the range of the blind area of the abnormal camera in the coordinate system, and adjust the range as needed The range of the viewable area in the coordinate system calculates the rotation angle of the normal camera. The central control computer controls the rotation of the normal camera according to the rotation angle, so that the visible area of the rotated normal camera covers the blind area.

For example, the abnormal camera is located on the right side of the vehicle. At this time, the central control computer can determine the corresponding security policy according to the position or pose of the abnormal camera. The security policy can include parking on the right side of the vehicle and eliminating the blind spot on the right side of the vehicle. The central control computer can determine the rotation angle of at least one normal camera according to the security policy, so that the adjusted field of view of all normal cameras can cover the blind spot on the right side of the vehicle caused by the abnormal camera.

For another example, the abnormal camera is located on the left side of the vehicle, and the vehicle is driving in the rightmost lane. At this time, the central control computer can determine the corresponding security policy according to the position (or pose) of the abnormal camera and the current driving lane. The security policy can be This includes continuing to drive in the current lane and eliminating blind spots on the left side of the vehicle. The central control computer can determine the rotation angle of at least one normal camera according to the security policy, so that the adjusted field of view of all normal cameras can cover the blind spot on the left side of the vehicle caused by the abnormal camera.

In this embodiment, because part of the field of view around the vehicle is blocked, there is a risk in the subsequent driving process of the vehicle. At this time, the central control computer is used to determine the corresponding security strategy according to the position or pose of the abnormal camera, and determine at least one The rotation angle of the normal camera can eliminate blind spots in time, reduce driving risks, and avoid accidents.

According to an embodiment of the present application, the calculation result includes the rotation angle of at least one normal camera that needs to be adjusted, and adjusting the normal camera according to the calculation result includes: adjusting the normal camera according to the rotation angle of at least one normal camera that needs to be adjusted.

Specifically, the calculation result includes the rotation angle of at least one normal camera that needs to be adjusted. The rotation of the camera can be driven by the steering mechanism, and the central control computer can transmit the rotation angle of the normal camera to the steering mechanism, and the steering mechanism can drive the camera to rotate according to the rotation angle. Optionally, the steering mechanism may include a motor, and the central control computer may transmit the number of rotations of the motor corresponding to the rotation angle of the normal camera to the steering mechanism, and the steering mechanism controls the rotation of the motor according to the number of rotations of the motor to drive the camera to rotate according to the rotation angle.

Fig. 4 is a schematic flowchart of a camera adjustment method provided by another exemplary embodiment of the present application. On the basis of the embodiment shown in Fig. 2 of the present application, the embodiment shown in Fig. 4 of the present application is extended, and the difference between the embodiment shown in Fig. 4 and the embodiment shown in Fig. 2 is emphasized below, and the similarities are no longer repeat.

As shown in Figure 4, in the camera adjustment method provided in the embodiment of the present application, it also includes:

Step 160: Identify whether the road environment satisfies the parking condition according to the output images collected by the N cameras.

Specifically, the central control computer can determine whether the current road environment meets the parking conditions according to the output image of the normal camera, or determine whether the current road environment Meet the parking conditions.

In one embodiment, identifying whether the road environment meets the parking conditions includes: identifying whether there is an emergency lane or a parking space available for parking on the left or right side of the vehicle according to at least one output image collected by a normal camera; When there is an emergency lane or a parking space available for parking on the left or right side of the vehicle, it is determined that the road environment meets the parking conditions.

If it is recognized that parking is possible in the road environment, then execute step 170 , if not, execute step 180 .

Step 170: Adjust the normal camera according to the rotation angle of at least one normal camera that needs to be adjusted.

In one embodiment, if it is recognized that there is an emergency lane or a parking space available for parking on the left side of the vehicle, the central control computer can determine that the safety strategy is to park on the left side of the vehicle and eliminate the blind spot on the left side of the vehicle. At this time, if the blind spot is On the left side of the vehicle, the central control computer determines the rotation angle of at least one normal camera according to the security policy, so that the adjusted field of view of all normal cameras can cover the blind spot on the left side of the vehicle. If the blind spot is on the right side of the vehicle, the central control computer determines that the rotation angle of at least one normal camera is 0 according to the security policy, that is, the angle of the normal camera is not adjusted, because there is no blind spot on the left side of the vehicle at this time.

The situation that there is an emergency lane or a parking space available for parking on the right side of the vehicle is similar to that on the left side, and will not be repeated here to avoid repetition.

Step 180: Continue to acquire the output image captured by at least one normal camera among the N cameras in real time.

If it is determined that the current road environment does not meet the parking conditions, the central control computer can determine that the safety strategy is to continue driving, and continue to obtain the output image collected by the normal camera at the next moment, and identify whether the road environment meets the parking condition based on the output image at the next moment , if the parking condition is met, execute step 170, otherwise execute step 180. That is, the central control computer can repeatedly execute step 160 and step 180 until it recognizes that the current road environment allows parking.

In this embodiment, the central control computer can identify whether the current road environment meets the parking conditions according to the output image collected by the camera, specifically, it can identify whether there is a parking space in the current road environment, such as an emergency lane or a parking space available for parking , and then the central control computer can determine the corresponding security strategy, and adjust the normal camera rotation angle and control the vehicle to continue driving or pull over according to the security strategy, so as to avoid the risk caused by the vehicle parking in place.

In other embodiments, the current road environment can be determined according to the vehicle's navigation and positioning system. The current road environment can include the current road type, for example, the current road type can be urban roads, expressways, mountain roads, viaducts, etc. The central control computer can further determine whether the current road environment meets the parking conditions according to the current road environment determined by the navigation and positioning system. For example, the current road environment is a mountain road, and mountain roads are generally one-way streets. Parking on mountain roads is more dangerous. Therefore, the central control computer can determine that the current road environment does not meet the parking conditions. At this time, the central control computer can determine that the safety strategy is to continue driving.

In this embodiment, the central control computer can identify whether the current road environment meets the parking conditions according to the current road type determined by the navigation and positioning system, and then the central control computer can determine the corresponding security strategy, and adjust the rotation of the normal camera according to the security strategy Angle and control the vehicle to continue driving or pull over, so as to avoid the risk caused by the vehicle stopping in place.

The central control computer can determine different safety strategies according to different road environments to reduce the risk of vehicle driving. The specific security policy can be set according to the actual situation, which is not limited in this embodiment of the present application.

Optionally, the central control computer can also determine the corresponding security policy according to the fault type of the abnormal camera. For example, the fault type of the abnormal camera can include single surround view camera abnormality, fisheye camera abnormality, multi-surround view camera abnormality, front view camera abnormality, etc. .

Different types of faults correspond to different security strategies. Taking the abnormality of multiple surround-view cameras as an example, when there are multiple abnormalities in the cameras, it means that the vehicle's perception ability is basically disabled at this time, and it is necessary to stop immediately and warn other vehicles. Therefore, the security strategy corresponding to the abnormality of the multi-surround view camera is to stop immediately and turn on the warning lights. Specific security policies corresponding to different fault types may be set according to actual conditions, which is not limited in this embodiment of the present application.

Optionally, the central control computer can determine the corresponding security strategy in combination with the fault type of the abnormal camera and the current road environment. For example, the fault type of the abnormal camera is an abnormality of the single surround view camera, and the current road environment is a single-lane mountain road. The central control computer determines the corresponding safety policy based on the abnormality of the single surround-view camera and the single-lane mountain road to remind the driver to drive carefully and give priority to driving out of the mountain road. Because the blind spot caused by the abnormality of the single-lane view camera does not necessarily affect the normal driving of the vehicle, and the single-lane mountain road generally does not have parking conditions, so the safety strategy corresponding to the abnormality of the single-lane view camera and the single-lane mountain road is to remind the driver to drive carefully and give priority to exit. mountain road. It should be understood that specific safety policies corresponding to different fault types and road environments may be set according to actual conditions.

Fig. 5 is a schematic flowchart of adjusting a normal camera provided by another exemplary embodiment of the present application. The embodiment shown in FIG. 5 of the present application is extended on the basis of the embodiment shown in FIG. repeat.

As shown in Figure 5, in the camera adjustment method provided in the embodiment of the present application, the normal camera is adjusted according to the rotation angle of at least one normal camera that needs to be adjusted (that is, step 170), including:

Step 171: Adjust the first adjacent camera according to the rotation angle that the first adjacent camera needs to adjust.

Specifically, the at least one normal camera that needs to be adjusted includes at least one first adjacent camera that is adjacent to the abnormal camera.

Because the number of cameras installed around the vehicle can be multiple, when a certain camera has a blind spot due to abnormality, in order to make the field of vision of the normal camera cover the blind spot, the central control computer will adjust the normal camera adjacent to the abnormal camera (section An angle of an adjacent camera), so that the field of view of the first adjacent camera covers the blind area. For example, the central control computer can determine the rotation angle of the adjacent camera on one side of the abnormal camera, so as to adjust the rotation of the adjacent camera so that its field of view covers the blind area; or, the central control computer can determine the rotation angle of the two adjacent cameras on both sides of the abnormal camera The rotation angle of the camera, so that the rotation of the two adjacent cameras is adjusted so that the fields of view of both cover the blind area.

Fig. 6 is a schematic diagram of a field of view coverage of a vehicle camera provided by an exemplary embodiment of the present application. There are generally multiple cameras installed around the vehicle, usually 8 to 13 cameras. Only part of the cameras are shown in FIG. 6 , including cameras directly in front of the roof, directly behind the roof, right in front of the roof, and right behind the roof. The fields of view of these four cameras can cover the front, rear and right sides of the vehicle.

When a certain camera of the vehicle is abnormal, blind spots will appear around the vehicle. For example, the camera on the right rear of the roof is abnormal, and a blind spot appears on the right side of the vehicle, as shown in Figure 7.

At this time, in order to ensure the safety of the vehicle and not affect the traffic, the central control computer will determine the safety strategy according to the position of the abnormal camera to park on the right side of the vehicle and eliminate the blind spot on the right side of the vehicle. Because parking on the right side of the vehicle needs to ensure that there is no blind spot in the right side of the vehicle, so as to further ensure the safety of parking on the right side of the vehicle. The central control computer can determine the rotation angles of the two first adjacent cameras (the camera on the right front of the roof and the camera directly behind the roof) adjacent to the abnormal camera according to the security policy, so that the two first adjacent cameras after rotation The field of view can cover the blind spot caused by the camera on the right rear of the roof.

Further, this embodiment may further include step 172 .

Step 172: Adjust the second adjacent camera according to the rotation angle of the second adjacent camera that needs to be adjusted.

Specifically, the at least one normal camera that needs to be adjusted further includes a second adjacent camera that is adjacent to the first adjacent camera. If the central control computer controls the first adjacent camera to rotate according to the determined rotation angle to cover the blind area caused by the abnormal camera, a new blind area will appear in the original visible area of the first adjacent camera. The central control computer can determine the rotation angle of the second adjacent camera according to the new blind spot caused by the rotation of the first adjacent camera, and control the rotation of the second adjacent camera according to the rotation angle. The process of determining the rotation angle of the second adjacent camera here may refer to the above process of determining the rotation angle of the normal camera adjacent to the abnormal camera.

As shown in Figure 8, after the central control computer controls the right front camera on the roof to rotate to the blind spot caused by the abnormal camera, the area covered by the right front camera on the roof will become a new blind spot. In order to avoid a new blind spot on the right side of the vehicle , the central control computer can further determine the rotation angle of the second adjacent camera (camera directly in front of the roof) adjacent to the first adjacent camera, so that the fields of view of the rotated first adjacent camera and the second adjacent camera The area to the right of the vehicle can be covered.

After the central control computer controls the right front camera on the roof, the rear camera on the roof and the front camera on the roof to rotate according to a certain angle, although the visible area behind the vehicle becomes smaller and the visible area in front of the vehicle becomes smaller, for example The area at the left rear of the vehicle and the area at the left front of the vehicle, but the area on the right side of the vehicle is all visible, which can ensure that the central control computer controls the vehicle to park safely on the right side of the vehicle.

In this embodiment, the central control computer can determine the security policy according to the position of the abnormal camera, and determine the rotation angles of the adjacent and next-adjacent cameras to the abnormal camera according to the security policy, or determine the surrounding area of the abnormal camera from near to far according to the security policy. The rotation angle of the camera, so that the camera whose angle needs to be adjusted can be quickly determined, and the reliable execution of the security policy can be ensured.

Exemplary device

Fig. 9 is a schematic structural diagram of a camera adjustment device provided by an exemplary embodiment of the present application. As shown in FIG. 9 , the apparatus 900 includes: a determination module 910 , a first acquisition module 920 , a calculation module 930 and an adjustment module 940 .

The determination module 910 is used to determine the abnormal camera and the normal camera in the N cameras, wherein N≥2; the first acquisition module 920 is used to obtain the parameters of the abnormal camera and the normal camera; Calculate the blind area caused by the abnormal camera. If the maximum visible area of the normal camera can cover the blind area caused by the abnormal camera, calculate the parameters that need to be adjusted by the normal camera to obtain the calculation result; the adjustment module 940 is used to adjust the normal camera according to the calculation result.

The embodiment of the present application provides a camera adjustment device, by determining the blind area of the abnormal camera according to the parameters of the abnormal camera, and calculating the parameters that need to be adjusted by the normal camera according to the parameters of the normal camera and the blind area of the abnormal camera, so as to adjust the parameters according to the needs Normal camera, so that the central control computer can control the vehicle to continue driving according to the adjusted normal camera's field of view coverage, or control the vehicle to drive to a safe area to stop, avoiding the violation of traffic rules caused by the vehicle directly parked in place, and traffic accidents and other safety issues. question.

According to an embodiment of the present application, the first acquiring module 920 is configured to acquire the current pose and angle of view of the abnormal camera and the normal camera.

Fig. 10 is a schematic structural diagram of a calculation module 930 of a camera adjustment device provided in another exemplary embodiment of the present application. On the basis of the embodiment shown in FIG. 9 of the present application, the embodiment shown in FIG. 10 of the present application is extended. The difference between the embodiment shown in FIG. 10 and the embodiment shown in FIG. 9 will be emphasized below, and the similarities will not be repeated. .

As shown in FIG. 10 , the calculating module 930 includes: a first calculating unit 931 , a second calculating unit 932 and a first determining unit 933 .

The first calculation unit 931 is used to calculate the invisible area according to the current pose and angle of view of the abnormal camera.

The second calculation unit 932 is used to calculate the visible area according to the current pose and angle of view of the normal camera.

The first determination unit 933 is configured to determine the blind area caused by the abnormal camera according to the visible area and the invisible area.

According to an embodiment of the present application, the calculation module 930 is configured to calculate the rotation angle of at least one normal camera that needs to be adjusted, so that the adjusted field of view of all normal cameras covers the blind area caused by the abnormal camera.

According to an embodiment of the present application, the calculation result includes the rotation angle of at least one normal camera that needs to be adjusted, and the adjustment module 940 is configured to adjust the normal camera according to the rotation angle of the at least one normal camera that needs to be adjusted.

Fig. 11 is a schematic structural diagram of a camera adjustment device provided by another exemplary embodiment of the present application. On the basis of the embodiment shown in FIG. 9 of the present application, the embodiment shown in FIG. 11 of the present application is extended. The difference between the embodiment shown in FIG. 11 and the embodiment shown in FIG. 9 will be emphasized below, and the similarities will not be repeated. .

As shown in FIG. 11 , the device 900 further includes: an identification module 950 configured to identify whether the road environment satisfies the parking condition according to the output images collected by the N cameras. If it is recognized that parking is possible in the road environment, the adjustment module 940 executes the step of adjusting the normal camera according to the rotation angle of at least one normal camera that needs to be adjusted.

According to an embodiment of the present application, as shown in FIG. 11 , the device 900 further includes: a second acquiring module 960 . If it is recognized that the road environment is not suitable for parking, the second acquisition module 960 continues to acquire the output image captured by at least one normal camera among the N cameras in real time.

Specifically, the first obtaining module 920 and the second obtaining module 960 may be the same module or different modules.

According to an embodiment of the present application, the identification module 950 is configured to identify whether there is an emergency lane or a parking space available for parking on the left or right side of the vehicle according to at least one output image collected by a normal camera, wherein, after identifying the left side of the vehicle When there is an emergency lane or a parking space available for parking on the side or right side, it is determined that the road environment meets the parking conditions.

Fig. 12 is a schematic structural diagram of an adjustment module 940 of a camera adjustment device provided in another exemplary embodiment of the present application. The embodiment shown in FIG. 12 of the present application is extended on the basis of the embodiment shown in FIG. 11 of the present application. The difference between the embodiment shown in FIG. 12 and the embodiment shown in FIG. 11 will be emphasized below, and the similarities will not be repeated. .

In this embodiment, the at least one normal camera to be adjusted includes at least one first adjacent camera that is adjacent to the abnormal camera. As shown in FIG. 12 , the adjustment module 940 includes: a first adjustment unit 941 , configured to adjust the first adjacent camera according to the rotation angle that needs to be adjusted by the first adjacent camera.

According to an embodiment of the present application, the at least one normal camera to be adjusted further includes a second adjacent camera adjacent to the first adjacent camera. As shown in FIG. 12 , the adjustment module 940 further includes: a second adjustment unit 942 configured to adjust the second adjacent camera according to the rotation angle that needs to be adjusted by the second adjacent camera.

It should be understood that the determination module 910, the first acquisition module 920, the calculation module 930, the adjustment module 940, the first calculation unit 931, the second calculation unit 932, the first determination unit 933, the identification module 950, the second For the operations and functions of the acquisition module 960, the first adjustment unit 941, and the second adjustment unit 942, reference may be made to the descriptions in the camera adjustment methods provided in FIGS.

Exemplary electronic device

Next, an electronic device according to an embodiment of the present application will be described with reference to FIG. 13 . The electronic device 90 can execute the above camera adjustment method.

FIG. 13 illustrates a block diagram of an electronic device 90 according to an embodiment of the present application.

As shown in FIG. 13 , electronic device 90 includes one or more processors 91 and memory 92 .

The processor 91 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 90 to perform desired functions.

Memory 92 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and/or cache memory (cache). The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, and the like. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 91 may execute the program instructions to implement the camera adjustment method of each embodiment of the present application described above and/or other expected functionality. Various contents such as an input signal, a signal component, a noise component, an image signal, etc. may also be stored in the computer-readable storage medium.

In one example, the electronic device 90 may further include: an input device 93 and an output device 94, and these components are interconnected through a bus system and/or other forms of connection mechanisms (not shown).

The input device 93 includes but not limited to keyboard, mouse, camera and so on.

The output device 94 can output various information to the outside, including the determined blind spot caused by the abnormal camera, the parameters that need to be adjusted by the normal camera, and the like. The output device 94 may include, for example, a display, a speaker, a printer, a communication network and its connected remote output devices, and the like.

Of course, for simplicity, only some of the components related to the present application in the electronic device 90 are shown in FIG. 13 , and components such as bus, input/output interface, etc. are omitted. In addition, according to specific application conditions, the electronic device 90 may also include any other suitable components.

Exemplary computer program product and computer readable storage medium

In addition to the above-mentioned methods and devices, embodiments of the present application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the above-mentioned "exemplary method" of this specification. The steps in the camera adjustment method according to various embodiments of the present application described in the section.

The computer program product can be written in any combination of one or more programming languages for executing the program codes for the operations of the embodiments of the present application, and the programming languages include object-oriented programming languages, such as Java, C++, etc. , also includes conventional procedural programming languages, such as the "C" language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server to execute.

In addition, the embodiments of the present application may also be a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the processor executes the above-mentioned "Exemplary Method" section of this specification. The steps in the camera adjustment method according to various embodiments of the present application described in .

The computer readable storage medium may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof, for example. More specific examples (non-exhaustive list) of readable storage media include: electrical connection with one or more conductors, portable disk, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

The basic principles of the present application have been described above in conjunction with specific embodiments, but it should be pointed out that the advantages, advantages, effects, etc. mentioned in the application are only examples rather than limitations, and these advantages, advantages, effects, etc. Various embodiments of this application must have. In addition, the specific details disclosed above are only for the purpose of illustration and understanding, rather than limitation, and the above details do not limit the application to be implemented by using the above specific details.

The block diagrams of devices, devices, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, devices, systems may be connected, arranged, configured in any manner. Words such as "including", "comprising", "having" and the like are open-ended words meaning "including but not limited to" and may be used interchangeably therewith. As used herein, the words "or" and "and" refer to the word "and/or" and are used interchangeably therewith, unless the context clearly dictates otherwise. As used herein, the word "such as" refers to the phrase "such as but not limited to" and can be used interchangeably therewith.

It should also be pointed out that in the devices, equipment and methods of the present application, each component or each step can be decomposed and/or reassembled. These decompositions and/or recombinations should be considered equivalents of this application.

The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the application to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims (8)

1. A camera adjustment method, comprising:

determining an abnormal camera and a normal camera in N cameras, wherein N is more than or equal to 2;

Acquiring the current pose and view angles of the abnormal camera and the normal camera;

calculating a blind area caused by the abnormal camera according to the current pose and the visual angle of the abnormal camera and the current pose and the visual angle of the normal camera;

if the maximum visible area of the normal camera can cover a blind area caused by the abnormal camera, calculating the rotation angle required to be adjusted by the normal camera to obtain a calculation result, wherein the maximum visible area comprises an area determined according to the current pose and the visual angle of the normal camera;

and adjusting the normal camera according to the calculation result.

2. The camera adjustment method according to claim 1, wherein the calculating the blind area caused by the abnormal camera according to the current pose and the view angle of the abnormal camera and the normal camera includes:

calculating an invisible area according to the current pose and the view angle of the abnormal camera;

calculating a visual area according to the current pose and the visual angle of the normal camera;

and determining blind areas caused by the abnormal cameras according to the visible areas and the invisible areas.

3. The camera adjustment method according to claim 1, wherein the calculating the rotation angle that the normal camera needs to adjust includes:

Calculating the rotation angle to be adjusted of at least one normal camera to be adjusted so that the adjusted visual field of all the normal cameras covers the blind area caused by the abnormal camera, wherein,

the adjusting the normal camera according to the calculation result comprises the following steps:

and adjusting the normal camera according to the rotation angle to be adjusted of the at least one normal camera to be adjusted.

4. The camera adjustment method according to claim 3, further comprising:

identifying whether the road environment meets the parking condition according to the output images acquired by the N cameras;

and if the road environment is identified to be capable of stopping, executing the step of adjusting the normal cameras according to the rotation angle to be adjusted of the at least one normal camera to be adjusted.

5. The camera adjustment method according to claim 4, further comprising:

if the road environment is identified to be incapable of stopping, continuously acquiring the output image acquired by at least one normal camera in the N cameras in real time.

6. A camera adjustment device, comprising:

the determining module is used for determining an abnormal camera and a normal camera in the N cameras, wherein N is more than or equal to 2;

The acquisition module is used for acquiring the current pose and the visual angle of the abnormal camera and the normal camera;

the calculating module is used for calculating a blind area caused by the abnormal camera according to the current pose and the visual angle of the abnormal camera and the normal camera; if the maximum visible area of the normal camera can cover a blind area caused by the abnormal camera, calculating the rotation angle required to be adjusted by the normal camera to obtain a calculation result, wherein the maximum visible area comprises an area determined according to the current pose and the visual angle of the normal camera;

and the adjusting module is used for adjusting the normal camera according to the calculation result.

7. An electronic device, comprising:

a processor; and

a memory in which computer program instructions are stored which, when executed by the processor, cause the processor to perform the camera adjustment method according to any one of claims 1 to 5.

8. A computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the camera adjustment method of any of claims 1 to 5.