[go: up one dir, main page]

CN116205829A - Rearview mirror image fusion method, device, vehicle-mounted equipment and storage medium - Google Patents

Rearview mirror image fusion method, device, vehicle-mounted equipment and storage medium Download PDF

Info

Publication number
CN116205829A
CN116205829A CN202211742598.2A CN202211742598A CN116205829A CN 116205829 A CN116205829 A CN 116205829A CN 202211742598 A CN202211742598 A CN 202211742598A CN 116205829 A CN116205829 A CN 116205829A
Authority
CN
China
Prior art keywords
rearview mirror
image data
mirror image
image
fusion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211742598.2A
Other languages
Chinese (zh)
Inventor
刘永学
徐超
胡金
王冉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Hangsheng Automobile Electronics Co ltd
Shenzhen Hangsheng Electronic Co Ltd
Original Assignee
Wuhan Hangsheng Automobile Electronics Co ltd
Shenzhen Hangsheng Electronic Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Hangsheng Automobile Electronics Co ltd, Shenzhen Hangsheng Electronic Co Ltd filed Critical Wuhan Hangsheng Automobile Electronics Co ltd
Priority to CN202211742598.2A priority Critical patent/CN116205829A/en
Publication of CN116205829A publication Critical patent/CN116205829A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02TCLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00Road transport of goods or passengers
    • Y02T10/10Internal combustion engine [ICE] based vehicles
    • Y02T10/40Engine management systems

Landscapes

  • Closed-Circuit Television Systems (AREA)

Abstract

本申请公开了一种后视镜图像融合方法、装置、车载设备以及存储介质,其后视镜图像融合方法包括:获取汽车电子后视镜的后视镜图像数据,其中,所述电子后视镜包括电子内后视镜和左右电子外后视镜;将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像。本申请解决了驾驶员在行车过程中需要同时观察三种显示设备,容易在判断外界场景和方位时造成失误的问题,实现系统的高度集成化,提升行车的安全性。

Figure 202211742598

The application discloses a rearview mirror image fusion method, device, vehicle-mounted equipment, and storage medium. The rearview mirror image fusion method includes: acquiring rearview mirror image data of an automobile electronic rearview mirror, wherein the electronic rearview The mirror includes an electronic interior rearview mirror and left and right electronic exterior mirrors; the image data of the rearview mirror is input into the FPGA chip for deep image fusion to generate a fused image. This application solves the problem that the driver needs to observe three kinds of display devices at the same time during the driving process, and easily causes mistakes when judging the external scene and orientation, realizes the high integration of the system, and improves the safety of driving.

Figure 202211742598

Description

后视镜图像融合方法、装置、车载设备以及存储介质Rearview mirror image fusion method, device, vehicle-mounted equipment and storage medium

技术领域technical field

本申请涉及图像处理技术领域,尤其涉及一种后视镜图像融合方法、装置、车载设备以及存储介质。The present application relates to the technical field of image processing, and in particular to a rearview mirror image fusion method, device, vehicle-mounted equipment, and storage medium.

背景技术Background technique

传统的物理外后视镜容易受极端天气、光线强弱、反光等因素的影响,使行车安全存在一定的隐患。随着汽车行业的工业化以及新四化的推进,汽车厂商安全意识的提升,电子内外后视镜逐渐出现在新的高端车型中。现有的电子外后视镜是在车辆的左右两边安装摄像头,通过在车内安装两个独立的显示屏来进行车外两侧图像的低延时实景显示。而电子内后视镜(即流媒体后视镜),是将摄像头功能与传统车内后视镜相结合,在车身鱼鳍天线处安装后视摄像头,并将摄像头采集到的图像数据通过同轴电缆双向超长距离传输到车内的后视镜上,经过Image Signal Processor芯片(图像信号处理器,简称ISP)处理以及图像算法增强处理后,在后视镜上投射车后的低延时实景图像。Traditional physical exterior rearview mirrors are easily affected by factors such as extreme weather, light intensity, and reflections, which pose certain hidden dangers to driving safety. With the industrialization of the automobile industry and the promotion of the new four modernizations, the safety awareness of automobile manufacturers has improved, and electronic interior and exterior mirrors have gradually appeared in new high-end models. In the existing electronic exterior rearview mirror, cameras are installed on the left and right sides of the vehicle, and two independent display screens are installed in the vehicle to perform low-delay real-time display of images on both sides of the vehicle. The electronic interior rearview mirror (that is, the streaming media rearview mirror) combines the camera function with the traditional interior rearview mirror, installs the rearview camera at the fish fin antenna of the body, and passes the image data collected by the camera through the same The shaft cable is transmitted to the rearview mirror in the car in two directions over a long distance. After processing by the Image Signal Processor chip (ISP for short) and image algorithm enhancement processing, the low-latency image behind the car is projected on the rearview mirror. Live imagery.

上述方案在车内需要放置三种不同类型的显示设备,驾驶员在左右变道或倒车等场景时,除了需要观察车内电子内后视镜外,还需要观察左右两个电子外后视镜,容易在判断外界场景和方位时造成失误,存在安全隐患。The above solution needs to place three different types of display devices in the car. When the driver is changing lanes left or right or reversing, the driver needs to observe the two electronic exterior mirrors in addition to the electronic interior rearview mirror inside the car. , It is easy to cause mistakes in judging the external scene and orientation, and there are potential safety hazards.

发明内容Contents of the invention

本申请的主要目的在于提供一种后视镜图像融合方法、装置、车载设备以及存储介质,旨在解决驾驶员在行车过程中需要同时观察三种显示设备,容易在判断外界场景和方位时造成失误的问题,实现系统的高度集成化,提升行车的安全性。The main purpose of this application is to provide a rearview mirror image fusion method, device, vehicle-mounted equipment and storage medium, aiming at solving the problem that the driver needs to observe three kinds of display devices at the same time during driving, which is easy to cause problems when judging the external scene and orientation. The problem of mistakes can be realized, and the system can be highly integrated to improve the safety of driving.

为实现上述目的,本申请提供一种后视镜图像融合方法,所述后视镜图像融合方法包括:In order to achieve the above object, the application provides a rearview mirror image fusion method, the rearview mirror image fusion method comprising:

获取汽车电子后视镜的后视镜图像数据,其中,所述电子后视镜包括电子内后视镜和左右电子外后视镜;Acquiring the rearview mirror image data of the electronic rearview mirror of the automobile, wherein the electronic rearview mirror includes an electronic inner rearview mirror and a left and right electronic outer rearview mirror;

将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像。The image data of the rearview mirror is input into the FPGA chip for deep image fusion to generate a fused image.

可选地,所述获取汽车电子后视镜的后视镜图像数据的步骤之后,还包5括:Optionally, after the step of obtaining the rearview mirror image data of the automobile electronic rearview mirror, it also includes:

将所述后视镜图像数据进行数据解串处理,以将串行的后视镜图像数据转换为并行的后视镜图像数据;Carrying out data deserialization processing on the rearview mirror image data, to convert the serial rearview mirror image data into parallel rearview mirror image data;

所述将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像的步骤包括:0将所述并行的后视镜图像数据输入至FPGA芯片中进行图像深度融合,The described rearview mirror image data is input into the FPGA chip to carry out image depth fusion, and the step of generating the fused picture image comprises: 0. The parallel rearview mirror image data is input into the FPGA chip to carry out image depth fusion ,

生成融合后的画面图像。A fused frame image is generated.

可选地,所述将所述后视镜图像数据进行数据解串处理,以将串行的后视镜图像数据转换为并行的后视镜图像数据的步骤之后,还包括:5对所述并行的后视镜图像数据进行图像信号处理,得到处理后的并行的后视镜图像数据。Optionally, after the step of performing data deserialization processing on the rearview mirror image data to convert the serial rearview mirror image data into parallel rearview mirror image data, it also includes: 5 pairs of the rearview mirror image data Image signal processing is performed on the parallel rearview mirror image data to obtain processed parallel rearview mirror image data.

可选地,所述将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像的步骤包括:Optionally, the described rearview mirror image data is input into the FPGA chip to carry out image depth fusion, and the step of generating a fused picture image includes:

0获取输入至所述FPGA芯片中的所述后视镜图像数据;0 to obtain the image data of the rearview mirror input into the FPGA chip;

对所述后视镜图像数据进行图像校正,得到校正后的后视镜图像数据;Carrying out image correction to the rearview mirror image data to obtain corrected rearview mirror image data;

获取汽车三维模型和汽车参数;Obtain the 3D model of the car and the parameters of the car;

将所述校正后的后视镜图像数据和所述汽车参数投影到所述汽车三维模型上,得到投影的图像数据;Projecting the corrected rearview mirror image data and the car parameters onto the three-dimensional car model to obtain projected image data;

5根据所述投影的图像数据对图像的连接处进行拼接处理,得到拼接后的图像数据;5. Perform splicing processing on the joints of the images according to the projected image data to obtain spliced image data;

对所述拼接后的图像数据进行渲染处理,得到所述融合后的画面图像。Rendering is performed on the spliced image data to obtain the fused picture image.

可选地,所述获取输入至所述FPGA芯片中的所述后视镜图像数据的步0骤之前,还包括:Optionally, before the step 0 of obtaining the rearview mirror image data input into the FPGA chip, it also includes:

加载图像处理的全局配置,初始化OpenGL ES组件和摄像头设备驱动。Load the global configuration of image processing, initialize OpenGL ES components and camera device drivers.

可选地,所述获取汽车电子后视镜的后视镜图像数据的步骤之前,还包括:Optionally, before the step of obtaining the rearview mirror image data of the automotive electronic rearview mirror, it also includes:

采集得到所述后视镜图像数据,包括:The image data of the rearview mirror is collected, including:

通过后视摄像头采集得到汽车后方的后视镜图像数据;The rearview mirror image data of the rear of the car is collected by the rearview camera;

通过安装在所述左右电子外后视镜处的左右两路摄像头采集得到左右两侧的后视镜图像数据。The image data of the rearview mirrors on the left and right sides are collected by the left and right cameras installed at the left and right electronic exterior mirrors.

可选地,所述将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像的步骤之后,还包括:Optionally, after the described rearview mirror image data is input into the FPGA chip to carry out image depth fusion, after the step of generating the fused picture image, it also includes:

将所述融合后的画面图像显示于单个显示设备上。and displaying the fused image on a single display device.

本申请实施例还提出一种后视镜图像融合装置,所述后视镜图像融合装置包括:The embodiment of the present application also proposes a rearview mirror image fusion device, and the rearview mirror image fusion device includes:

图像获取模块,用于获取汽车电子后视镜的后视镜图像数据,其中,所述电子后视镜包括电子内后视镜和左右电子外后视镜;The image acquisition module is used to acquire the rearview mirror image data of the electronic rearview mirror of the automobile, wherein the electronic rearview mirror includes an electronic inner rearview mirror and a left and right electronic outer rearview mirror;

图像融合模块,用于将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像。The image fusion module is used to input the image data of the rearview mirror into the FPGA chip for image depth fusion to generate a fused picture image.

本申请实施例还提出一种车载设备,所述车载设备包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的后视镜图像融合程序,所述后视镜图像融合程序被所述处理器执行时实现如上所述的后视镜图像融合方法的步骤。The embodiment of the present application also proposes a vehicle-mounted device, the vehicle-mounted device includes a memory, a processor, and a rearview mirror image fusion program stored in the memory and operable on the processor, the rearview mirror image When the fusion program is executed by the processor, the steps of the above-mentioned rearview mirror image fusion method are realized.

本申请实施例还提出一种计算机可读存储介质,所述计算机可读存储介质上存储有后视镜图像融合程序,所述后视镜图像融合程序被处理器执行时实现如上所述的后视镜图像融合方法的步骤。The embodiment of the present application also proposes a computer-readable storage medium, the computer-readable storage medium stores a rearview mirror image fusion program, and when the rearview mirror image fusion program is executed by a processor, the rearview mirror image fusion program as described above is implemented. Steps of the mirror image fusion method.

本申请实施例提出的后视镜图像融合方法、装置、车载设备以及存储介质,通过获取汽车电子后视镜的后视镜图像数据,其中,所述电子后视镜包括电子内后视镜和左右电子外后视镜;将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像。通过集成电子内后视镜画面和电子外后视镜画面并进行画面的全景融合,能够将三路视频融合成一个显示画面,实现车后方无死角的画面展示,提高行车的安全性。The rearview mirror image fusion method, device, vehicle-mounted equipment, and storage medium proposed in the embodiments of the present application obtain rearview mirror image data of an electronic rearview mirror of an automobile, wherein the electronic rearview mirror includes an electronic interior rearview mirror and an electronic rearview mirror. The left and right electronic exterior mirrors; the image data of the rearview mirrors are input to the FPGA chip for deep image fusion to generate a fused image. By integrating the image of the electronic interior rearview mirror and the image of the electronic exterior mirror and performing panoramic fusion of the images, the three-way video can be fused into one display image, realizing the image display without dead angles behind the car, and improving driving safety.

附图说明Description of drawings

图1为本申请后视镜图像融合装置所属车载设备的功能模块示意图;Fig. 1 is the schematic diagram of the functional modules of the vehicle equipment belonging to the rearview mirror image fusion device of the present application;

图2为本申请后视镜图像融合方法第一示例性实施例的流程示意图;Fig. 2 is the schematic flow chart of the first exemplary embodiment of the rearview mirror image fusion method of the present application;

图3为本申请后视镜图像融合方法第二示例性实施例的流程示意图;Fig. 3 is the schematic flow chart of the second exemplary embodiment of the rearview mirror image fusion method of the present application;

图4为本申请后视镜图像融合方法第三示例性实施例的流程示意图。Fig. 4 is a schematic flowchart of a third exemplary embodiment of a rearview mirror image fusion method of the present application.

本申请目的的实现、功能特点及优点将结合实施例,参照附图做进一步说明。The realization, functional features and advantages of the present application will be further described in conjunction with the embodiments and with reference to the accompanying drawings.

具体实施方式Detailed ways

应当理解,此处所描述的具体实施例仅仅用以解释本申请,并不用于限定本申请。It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

本申请实施例的主要解决方案是:获取汽车电子后视镜的后视镜图像数据,其中,所述电子后视镜包括电子内后视镜和左右电子外后视镜;将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像。通过集成电子内后视镜画面和电子外后视镜画面并进行画面的全景融合,能够将三路视频融合成一个显示画面,实现车后方无死角的画面展示,提高行车的安全性。The main solution of the embodiment of the present application is to obtain the rearview mirror image data of the automobile electronic rearview mirror, wherein the electronic rearview mirror includes an electronic interior rearview mirror and a left and right electronic exterior rearview mirror; The mirror image data is input to the FPGA chip for deep image fusion to generate a fused image. By integrating the image of the electronic interior rearview mirror and the image of the electronic exterior mirror and performing panoramic fusion of the images, the three-way video can be fused into one display image, realizing the image display without dead angles behind the car, and improving driving safety.

具体地,参照图1,图1为本申请后视镜图像融合装置所属车载设备的功能模块示意图。该后视镜图像融合装置可以为独立于车载设备的、能够进行图片处理、图片深度融合的装置,其可以通过硬件或软件的形式承载于车载设备上。Specifically, referring to FIG. 1 , FIG. 1 is a schematic diagram of the functional modules of the vehicle-mounted equipment to which the rearview mirror image fusion device of the present application belongs. The rearview mirror image fusion device can be a device independent of the vehicle-mounted equipment, capable of image processing and deep image fusion, and can be carried on the vehicle-mounted equipment in the form of hardware or software.

在本实施例中,该后视镜图像融合装置所属车载设备至少包括输出模块110、处理器120、存储器130以及通信模块140。In this embodiment, the vehicle-mounted device to which the rearview mirror image fusion device belongs includes at least an output module 110 , a processor 120 , a memory 130 and a communication module 140 .

存储器130中存储有操作系统以及后视镜图像融合程序,后视镜图像融合装置可以将获取的汽车电子后视镜的后视镜图像数据,将后视镜图像数据输入至FPGA芯片中进行图像深度融合生成的融合后的画面图像,将后视镜Stored in the memory 130 are an operating system and a rearview mirror image fusion program, and the rearview mirror image fusion device can input the rearview mirror image data into the FPGA chip for the rearview mirror image data of the obtained automotive electronic rearview mirror. The fused picture image generated by deep fusion, the rearview mirror

图像数据进行数据解串处理后得到的并行的后视镜图像数据,对并行的后视5镜图像数据进行图像信号处理得到的处理后的并行的后视镜图像数据等信息存储于该存储器130中;输出模块110可为显示屏等。通信模块140可以包括CAN模块、WIFI模块、移动通信模块以及蓝牙模块等,通过通信模块140与外部设备或服务器进行通信。Information such as the parallel rearview mirror image data obtained after the image data is deserialized, and the processed parallel rearview mirror image data obtained by performing image signal processing on the parallel rearview 5-mirror image data is stored in the memory 130 Middle; the output module 110 may be a display screen or the like. The communication module 140 may include a CAN module, a WIFI module, a mobile communication module, a Bluetooth module, etc., and communicate with external devices or servers through the communication module 140 .

其中,存储器130中的后视镜图像融合程序被处理器执行时实现以下步0骤:Wherein, when the rearview mirror image fusion program in the memory 130 is executed by the processor, the following steps are realized:

获取汽车电子后视镜的后视镜图像数据,其中,所述电子后视镜包括电子内后视镜和左右电子外后视镜;Acquiring the rearview mirror image data of the electronic rearview mirror of the automobile, wherein the electronic rearview mirror includes an electronic inner rearview mirror and a left and right electronic outer rearview mirror;

将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像。The image data of the rearview mirror is input into the FPGA chip for deep image fusion to generate a fused image.

5进一步地,存储器130中的后视镜图像融合程序被处理器执行时还实现以下步骤:5 Further, when the rearview mirror image fusion program in the memory 130 is executed by the processor, the following steps are also realized:

将所述后视镜图像数据进行数据解串处理,以将串行的后视镜图像数据转换为并行的后视镜图像数据;Carrying out data deserialization processing on the rearview mirror image data, to convert the serial rearview mirror image data into parallel rearview mirror image data;

将所述处理后的并行的后视镜图像数据输入至FPGA芯片中进行图像深0度融合,生成融合后的画面图像。The processed parallel rearview mirror image data is input into the FPGA chip for image depth 0-degree fusion to generate a fused image.

进一步地,存储器130中的后视镜图像融合程序被处理器执行时还实现以下步骤:Further, when the rearview mirror image fusion program in the memory 130 is executed by the processor, the following steps are also realized:

对所述并行的后视镜图像数据进行图像信号处理,得到处理后的并行的后视镜图像数据。Image signal processing is performed on the parallel rearview mirror image data to obtain processed parallel rearview mirror image data.

5进一步地,存储器130中的后视镜图像融合程序被处理器执行时还实现以下步骤:5 Further, when the rearview mirror image fusion program in the memory 130 is executed by the processor, the following steps are also realized:

获取输入至所述FPGA芯片中的所述后视镜图像数据;Obtain the image data of the rearview mirror input into the FPGA chip;

对所述后视镜图像数据进行图像校正,得到校正后的后视镜图像数据;Carrying out image correction to the rearview mirror image data to obtain corrected rearview mirror image data;

获取汽车三维模型和汽车参数;Obtain the 3D model of the car and the parameters of the car;

0将所述校正后的后视镜图像数据和所述汽车参数投影到所述汽车三维模型上,得到投影的图像数据;0 projecting the corrected rearview mirror image data and the car parameters onto the three-dimensional car model to obtain projected image data;

根据所述投影的图像数据对图像的连接处进行拼接处理,得到拼接后的图像数据;splicing the joints of the images according to the projected image data to obtain the spliced image data;

对所述拼接后的图像数据进行渲染处理,得到所述融合后的画面图像。Rendering is performed on the spliced image data to obtain the fused picture image.

进一步地,存储器130中的后视镜图像融合程序被处理器执行时还实现以下步骤:Further, when the rearview mirror image fusion program in the memory 130 is executed by the processor, the following steps are also realized:

加载图像处理的全局配置,初始化OpenGL ES组件和摄像头设备驱动。Load the global configuration of image processing, initialize OpenGL ES components and camera device drivers.

进一步地,存储器130中的后视镜图像融合程序被处理器执行时还实现以下步骤:Further, when the rearview mirror image fusion program in the memory 130 is executed by the processor, the following steps are also realized:

采集得到所述后视镜图像数据,包括:The image data of the rearview mirror is collected, including:

通过后视摄像头采集得到汽车后方的后视镜图像数据;The rearview mirror image data of the rear of the car is collected by the rearview camera;

通过安装在所述左右电子外后视镜处的左右两路摄像头采集得到左右两侧的后视镜图像数据。The image data of the rearview mirrors on the left and right sides are collected by the left and right cameras installed at the left and right electronic exterior mirrors.

进一步地,存储器130中的后视镜图像融合程序被处理器执行时还实现以下步骤:Further, when the rearview mirror image fusion program in the memory 130 is executed by the processor, the following steps are also realized:

将所述融合后的画面图像显示于单个显示设备上。and displaying the fused image on a single display device.

本实施例通过上述方案,具体通过获取汽车电子后视镜的后视镜图像数据,其中,所述电子后视镜包括电子内后视镜和左右电子外后视镜;将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像。通过集成电子内后视镜画面和电子外后视镜画面并进行画面的全景融合,能够将三路视频融合成一个显示画面,实现车后方无死角的画面展示,提高行车的安全性。This embodiment adopts the above scheme, specifically by acquiring the rearview mirror image data of the electronic rearview mirror of the automobile, wherein the electronic rearview mirror includes an electronic interior rearview mirror and a left and right electronic exterior rearview mirror; The image data is input to the FPGA chip for deep image fusion to generate a fused image. By integrating the image of the electronic interior rearview mirror and the image of the electronic exterior mirror and performing panoramic fusion of the images, the three-way video can be fused into one display image, realizing the image display without dead angles behind the car, and improving driving safety.

基于上述车载设备架构但不限于上述架构,提出本申请方法实施例。Based on but not limited to the above-mentioned vehicle-mounted device architecture, the method embodiments of the present application are proposed.

参照图2,图2为本申请后视镜图像融合方法第一示例性实施例的流程示意图。所述后视镜图像融合方法包括:Referring to FIG. 2 , FIG. 2 is a schematic flowchart of a first exemplary embodiment of a rearview mirror image fusion method of the present application. Described rearview mirror image fusion method comprises:

步骤S10,获取汽车电子后视镜的后视镜图像数据,其中,所述电子后视镜包括电子内后视镜和左右电子外后视镜;Step S10, acquiring rearview mirror image data of the automobile electronic rearview mirror, wherein the electronic rearview mirror includes an electronic interior rearview mirror and left and right electronic exterior mirrors;

步骤S20,将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像。Step S20, inputting the image data of the rearview mirror into the FPGA chip for deep image fusion to generate a fused image.

本实施例方法的执行主体可以是一种后视镜图像融合装置,也可以是一种后视镜图像融合车载设备或服务器,本实施例以后视镜图像融合装置进行举例,该后视镜图像融合装置可以集成在具有数据处理功能的车载设备上。The execution subject of the method in this embodiment can be a rearview mirror image fusion device, or a rearview mirror image fusion vehicle-mounted device or server. The rearview mirror image fusion device in this embodiment is an example. The rearview mirror image The fusion device can be integrated on the vehicle equipment with data processing function.

在本实施例中,获取汽车的电子后视镜的后视镜图像数据,其中,该电子后视镜可以包括电子内后视镜和左右电子外后视镜;然后,将获取到的左、后和右的后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成超广角的融合后的画面图像。本实施例通过集成在一个FPGA芯片上,通过图像融合算法,在满足三路视频输入都能显示的前提下,能够减少SOC的使用,实现成本更优,重量更轻,行车时更安全的优点。In this embodiment, the rearview mirror image data of the electronic rearview mirror of the automobile is obtained, wherein the electronic rearview mirror may include an electronic interior rearview mirror and a left and right electronic exterior rearview mirror; The rear and right rearview mirror image data is input to the FPGA chip for deep image fusion to generate a super wide-angle fused image. This embodiment can reduce the use of SOC by integrating it on an FPGA chip, and through the image fusion algorithm, on the premise that all three video inputs can be displayed, and realize the advantages of better cost, lighter weight, and safer driving. .

需要说明的是,FPGA相比较传统同等价格SOC方案,有着图像处理性能强,对于画面处理到显示时延更小的优点,并且同时支持多路图像输入。可选地,本实施例选用的FPGA为赛灵思的ZU3EG芯片作为主芯片。It should be noted that, compared with traditional SOC solutions of the same price, FPGA has the advantages of strong image processing performance, less delay from image processing to display, and supports multiple image inputs at the same time. Optionally, the FPGA selected in this embodiment is the ZU3EG chip of Xilinx as the main chip.

进一步地,作为一种实施方式,在上述步骤S101,获取汽车电子后视镜的后视镜图像数据之前,还可以包括:Further, as an implementation manner, before the above step S101, before obtaining the rearview mirror image data of the electronic rearview mirror of the automobile, it may also include:

采集得到所述后视镜图像数据,包括:The image data of the rearview mirror is collected, including:

通过后视摄像头采集得到汽车后方的后视镜图像数据;The rearview mirror image data of the rear of the car is collected by the rearview camera;

通过安装在所述左右电子外后视镜处的左右两路摄像头采集得到左右两侧的后视镜图像数据。The image data of the rearview mirrors on the left and right sides are collected by the left and right cameras installed at the left and right electronic exterior mirrors.

具体地,对于单独的电子内后视镜来说,由于摄像头安装位置的限制,通常能够凸显的视野范围在80°左右,而对于左右两侧仍然存在视野盲区,因此无法对车辆左右两侧后方的视野盲区进行全覆盖。在本实施例中,通过结合后视摄像头采集得到汽车后方的后视镜图像数据,以及安装在左右电子外后视镜处的左右两路摄像头采集得到左右两侧的后视镜图像数据,可以采集车辆左右侧和后侧全景,避免车后方存在视野死角。Specifically, for a single electronic interior rearview mirror, due to the limitation of the installation position of the camera, the field of view that can be highlighted is usually about 80°, but there are still blind spots on the left and right sides of the vehicle, so it is impossible to view the rear of the left and right sides of the vehicle. Full coverage of blind spots in the field of vision. In this embodiment, the rearview mirror image data at the rear of the car is collected by combining the rearview camera, and the rearview mirror image data on the left and right sides are collected by the left and right cameras installed at the left and right electronic exterior mirrors. Collect the left, right and rear panorama of the vehicle to avoid blind spots behind the vehicle.

进一步地,作为一种实施方式,在上述步骤S102,将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像的之后,还可以包括:Further, as an implementation manner, in the above step S102, after inputting the rearview mirror image data into the FPGA chip for image depth fusion and generating the fused picture image, it may also include:

将所述融合后的画面图像显示于单个显示设备上。and displaying the fused image on a single display device.

通过将集成了电子内外后视镜画面的融合后的画面图像显示在单个显示设备上,能够提高汽车的集成化程度,降低整车重量,降低整车制造成本,为整车的轻量化指标做出贡献。By displaying the fused image integrated with the images of the electronic interior and exterior mirrors on a single display device, the degree of integration of the vehicle can be improved, the weight of the vehicle can be reduced, the manufacturing cost of the vehicle can be reduced, and it can contribute to the lightweight index of the vehicle. contribute.

本实施例通过上述方案,具体通过获取汽车电子后视镜的后视镜图像数据,其中,所述电子后视镜包括电子内后视镜和左右电子外后视镜;将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像。通过集成电子外后视镜画面并进行画面的全景融合实现车后方无死角的画面,提高行车的安全性;通过将电子内外后视镜的融合实现了系统的高度集成化,降低整车制造成本。This embodiment adopts the above scheme, specifically by acquiring the rearview mirror image data of the electronic rearview mirror of the automobile, wherein the electronic rearview mirror includes an electronic interior rearview mirror and a left and right electronic exterior rearview mirror; The image data is input to the FPGA chip for deep image fusion to generate a fused image. Through the integration of the electronic exterior mirror picture and the panoramic fusion of the picture, the picture without dead angle behind the car can be realized, and the safety of driving can be improved; the high integration of the system can be realized by the fusion of the electronic interior and exterior mirror, and the manufacturing cost of the whole vehicle can be reduced. .

参照图3,图3为本申请后视镜图像融合方法第二示例性实施例的流程示意图。基于上述图2所示的实施例,在本实施例中,在获取汽车电子后视镜的后视镜图像数据的步骤之后,所述后视镜图像融合方法还包括:Referring to FIG. 3 , FIG. 3 is a schematic flowchart of a second exemplary embodiment of a rearview mirror image fusion method of the present application. Based on the embodiment shown in above-mentioned Fig. 2, in the present embodiment, after obtaining the step of the rearview mirror image data of automobile electronic rearview mirror, described rearview mirror image fusion method also includes:

步骤S101,将所述后视镜图像数据进行数据解串处理,以将串行的后视镜图像数据转换为并行的后视镜图像数据。Step S101, performing data deserialization processing on the rearview mirror image data, so as to convert the serial rearview mirror image data into parallel rearview mirror image data.

具体地,由于获取到的汽车电子后视镜的后视镜图像数据为串行数据,在输入FPGA芯片之前,将该后视镜图像数据进行数据解串处理,以便可以将串行的后视镜图像数据转换为并行的后视镜图像数据。Specifically, since the acquired rearview mirror image data of the automobile electronic rearview mirror is serial data, before inputting the FPGA chip, the rearview mirror image data is subjected to data deserialization processing, so that the serial rearview mirror can be The mirror image data is converted into parallel mirror image data.

然后,执行步骤S102,将所述并行的后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像。Then, step S102 is executed to input the parallel rearview mirror image data into the FPGA chip for deep image fusion to generate a fused image.

进一步地,作为一种实施方式,在上述步骤S101,将所述后视镜图像数据进行数据解串处理,以将串行的后视镜图像数据转换为并行的后视镜图像数据之后,还可以包括:Further, as an implementation manner, in the above step S101, after performing data deserialization processing on the rearview mirror image data to convert the serial rearview mirror image data into parallel rearview mirror image data, further Can include:

步骤S103,对所述并行的后视镜图像数据进行图像信号处理,得到处理后的并行的后视镜图像数据。Step S103, performing image signal processing on the parallel rearview mirror image data to obtain processed parallel rearview mirror image data.

具体地,对数据解串处理后得到的并行的后视镜图像数据进行图像信号处理,得到处理后的并行的后视镜图像数据,提升后续用户的观察效果。可选地,图像信号处理可以包括但不限于:线性纠正、噪点去除、坏点修补、颜色插值、白平衡矫正和曝光校正等。Specifically, image signal processing is performed on the parallel rearview mirror image data obtained after data deserialization processing, and the processed parallel rearview mirror image data is obtained, so as to improve the observation effect of subsequent users. Optionally, the image signal processing may include but not limited to: linearity correction, noise removal, dead point repair, color interpolation, white balance correction, exposure correction, etc.

本实施例通过对后视镜图像数据进行图像信号处理,可以在环境恶劣天气条件下,对画面进行增强处理,以此来规避恶劣环境下的行车安全问题,进一步提升行车的安全性。In this embodiment, by performing image signal processing on the image data of the rearview mirror, the image can be enhanced under severe weather conditions, so as to avoid driving safety problems in harsh environments and further improve driving safety.

本实施例通过上述方案,具体通过获取汽车电子后视镜的后视镜图像数据,其中,所述电子后视镜包括电子内后视镜和左右电子外后视镜;将所述后视镜图像数据进行数据解串处理,以将串行的后视镜图像数据转换为并行的后视镜图像数据;对所述并行的后视镜图像数据进行图像信号处理,得到处理后的并行的后视镜图像数据;将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像。通过集成电子内后视镜画面和电子外后视镜画面并进行画面的全景融合,能够将三路视频融合成一个显示画面,实现车后方无死角的画面展示,提高行车的安全性。This embodiment adopts the above scheme, specifically by acquiring the rearview mirror image data of the electronic rearview mirror of the automobile, wherein the electronic rearview mirror includes an electronic interior rearview mirror and a left and right electronic exterior rearview mirror; Carry out data deserialization processing on the image data, to convert the serial rearview mirror image data into parallel rearview mirror image data; carry out image signal processing to the parallel rearview mirror image data, obtain the parallel rearview mirror image data after processing Mirror image data; the rearview mirror image data is input into the FPGA chip for image depth fusion to generate a fused picture image. By integrating the image of the electronic interior rearview mirror and the image of the electronic exterior mirror and performing panoramic fusion of the images, the three-way video can be fused into one display image, realizing the image display without dead angles behind the car, and improving driving safety.

参照图4,图4为本申请后视镜图像融合方法第三示例性实施例的流程示意图。基于上述图2和图3所示的实施例,在本实施例中,上述步骤S20,将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像可以包括:Referring to FIG. 4 , FIG. 4 is a schematic flowchart of a third exemplary embodiment of a rearview mirror image fusion method of the present application. Based on the embodiment shown in above-mentioned Fig. 2 and Fig. 3, in the present embodiment, above-mentioned step S20, described rearview mirror image data is input in the FPGA chip and carries out image depth fusion, and the screen image after generating fusion can comprise:

步骤S201,获取输入至所述FPGA芯片中的所述后视镜图像数据;Step S201, obtaining the image data of the rearview mirror input into the FPGA chip;

步骤S202,对所述后视镜图像数据进行图像校正,得到校正后的后视镜图像数据;Step S202, performing image correction on the rearview mirror image data to obtain corrected rearview mirror image data;

步骤S203,获取汽车三维模型和汽车参数;Step S203, obtaining the three-dimensional model of the vehicle and the parameters of the vehicle;

步骤S204,将所述校正后的后视镜图像数据和所述汽车参数投影到所述汽车三维模型上,得到投影的图像数据;Step S204, projecting the corrected rearview mirror image data and the car parameters onto the three-dimensional car model to obtain projected image data;

步骤S205,根据所述投影的图像数据对图像的连接处进行拼接处理,得到拼接后的图像数据;Step S205, performing splicing processing on the joints of the images according to the projected image data to obtain spliced image data;

步骤S206,对所述拼接后的图像数据进行渲染处理,得到所述融合后的画面图像。Step S206, performing rendering processing on the spliced image data to obtain the fused picture image.

具体地,获取输入至FPGA芯片中的电子内后视镜的后视镜图像数据和左右电子外后视镜的后视镜图像数据。然后,对获取到的三路后视镜图像数据进行图像校正,得到校正后的后视镜图像数据。接着,获取预先存储的汽车三维模型和汽车参数;可选地,该汽车参数可以包括但不限于汽车颜色。将校正后的后视镜图像数据和汽车参数投影到该汽车三维模型上,形成投影的图像数据。然后根据投影的图像数据对图像中的连接处进行拼接处理,包括连接图像的亮度调整等,得到拼接后的图像数据。最后,对拼接后的图像数据进行图像渲染处理,得到该融合后的画面图像。Specifically, the rearview mirror image data of the electronic interior rearview mirror and the rearview mirror image data of the left and right electronic exterior rearview mirrors input to the FPGA chip are acquired. Then, image correction is performed on the acquired three-way rearview mirror image data to obtain corrected rearview mirror image data. Next, the pre-stored three-dimensional model of the car and car parameters are acquired; optionally, the car parameters may include but not limited to car color. The corrected rearview mirror image data and vehicle parameters are projected onto the three-dimensional model of the vehicle to form projected image data. Then according to the projected image data, splicing processing is performed on the joints in the images, including brightness adjustment of the connected images, etc., to obtain the spliced image data. Finally, image rendering processing is performed on the spliced image data to obtain the fused image.

进一步地,作为一种实施方式,在上述步骤S201,获取输入至所述FPGA芯片中的所述后视镜图像数据之前,还可以包括:Further, as an implementation manner, before the above step S201, before obtaining the rearview mirror image data input to the FPGA chip, it may also include:

加载图像处理的全局配置,初始化OpenGL ES组件和摄像头设备驱动。Load the global configuration of image processing, initialize OpenGL ES components and camera device drivers.

本实施例通过上述方案,具体通过获取汽车电子后视镜的后视镜图像数据,其中,所述电子后视镜包括电子内后视镜和左右电子外后视镜;将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像。通过集成电子内后视镜画面和电子外后视镜画面并进行画面的全景融合,能够将三路视频融合成一个显示画面,实现车后方无死角的画面展示,提高行车的安全性。This embodiment adopts the above scheme, specifically by acquiring the rearview mirror image data of the electronic rearview mirror of the automobile, wherein the electronic rearview mirror includes an electronic interior rearview mirror and a left and right electronic exterior rearview mirror; The image data is input to the FPGA chip for deep image fusion to generate a fused image. By integrating the image of the electronic interior rearview mirror and the image of the electronic exterior mirror and performing panoramic fusion of the images, the three-way video can be fused into one display image, realizing the image display without dead angles behind the car, and improving driving safety.

此外,本申请实施例还提出一种后视镜图像融合装置,所述后视镜图像融合装置包括:In addition, the embodiment of the present application also proposes a rearview mirror image fusion device, and the rearview mirror image fusion device includes:

图像获取模块,用于获取汽车电子后视镜的后视镜图像数据,其中,所述电子后视镜包括电子内后视镜和左右电子外后视镜;The image acquisition module is used to acquire the rearview mirror image data of the electronic rearview mirror of the automobile, wherein the electronic rearview mirror includes an electronic inner rearview mirror and a left and right electronic outer rearview mirror;

图像融合模块,用于将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像。The image fusion module is used to input the image data of the rearview mirror into the FPGA chip for image depth fusion to generate a fused picture image.

本实施例实现后视镜图像融合的原理及实施过程,请参照上述各实施例,在此不再一一赘述。For the principle and implementation process of realizing the fusion of rearview mirror images in this embodiment, please refer to the above-mentioned embodiments, and details will not be repeated here.

此外,本申请实施例还提出一种车载设备,所述车载设备包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的后视镜图像融合程序,所述后视镜图像融合程序被所述处理器执行时实现如上所述的后视镜图像融合方法的步骤。In addition, the embodiment of the present application also proposes a vehicle-mounted device, which includes a memory, a processor, and a rearview mirror image fusion program stored in the memory and operable on the processor. When the mirror image fusion program is executed by the processor, the steps of the above-mentioned rearview mirror image fusion method are realized.

由于本后视镜图像融合程序被处理器执行时,采用了前述所有实施例的全部技术方案,因此至少具有前述所有实施例的全部技术方案所带来的所有有益效果,在此不再一一赘述。Since this rearview mirror image fusion program is executed by the processor, it adopts all the technical solutions of all the foregoing embodiments, so at least it has all the beneficial effects brought by all the technical solutions of all the foregoing embodiments, and will not repeat them one by one here. repeat.

此外,本申请实施例还提出一种计算机可读存储介质,所述计算机可读存储介质上存储有后视镜图像融合程序,所述后视镜图像融合程序被处理器执行时实现如上所述的后视镜图像融合方法的步骤。In addition, the embodiment of the present application also proposes a computer-readable storage medium, on which a rearview mirror image fusion program is stored, and when the rearview mirror image fusion program is executed by a processor, the above-mentioned The steps of the rearview mirror image fusion method.

由于本后视镜图像融合程序被处理器执行时,采用了前述所有实施例的全部技术方案,因此至少具有前述所有实施例的全部技术方案所带来的所有有益效果,在此不再一一赘述。Since this rearview mirror image fusion program is executed by the processor, it adopts all the technical solutions of all the foregoing embodiments, so at least it has all the beneficial effects brought by all the technical solutions of all the foregoing embodiments, and will not repeat them one by one here. repeat.

相比现有技术,本申请实施例提出的后视镜图像融合方法、装置、车载设备以及存储介质,通过获取汽车电子后视镜的后视镜图像数据,其中,所述电子后视镜包括电子内后视镜和左右电子外后视镜;将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像。通过集成电子内后视镜画面和电子外后视镜画面并进行画面的全景融合,能够将三路视频融合成一个显示画面,实现车后方无死角的画面展示,提高行车的安全性。Compared with the prior art, the rearview mirror image fusion method, device, vehicle-mounted equipment and storage medium proposed in the embodiments of the present application obtain the rearview mirror image data of the automobile electronic rearview mirror, wherein the electronic rearview mirror includes The electronic interior rearview mirror and the left and right electronic exterior mirrors; the image data of the rearview mirrors are input to the FPGA chip for deep image fusion to generate a fused image. By integrating the image of the electronic interior rearview mirror and the image of the electronic exterior mirror and performing panoramic fusion of the images, the three-way video can be fused into one display image, realizing the image display without dead angles behind the car, and improving driving safety.

需要说明的是,在本文中,术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含,从而使得包括一系列要素的过程、方法、物品或者系统不仅包括那些要素,而且还包括没有明确列出的其他要素,或者是还包括为这种过程、方法、物品或者系统所固有的要素。在没有更多限制的情况下,由语句“包括一个……”限定的要素,并不排除在包括该要素的过程、方法、物品或者系统中还存在另外的相同要素。It should be noted that, as used herein, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or system comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or system. Without further limitations, an element defined by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article or system comprising that element.

上述本申请实施例序号仅仅为了描述,不代表实施例的优劣。The serial numbers of the above embodiments of the present application are for description only, and do not represent the advantages and disadvantages of the embodiments.

通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现,当然也可以通过硬件,但很多情况下前者是更佳的实施方式。基于这样的理解,本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品存储在如上的一个存储介质(如ROM/RAM、磁碟、光盘)中,包括若干指令用以使得一台设备执行本申请每个实施例的方法。Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence or in other words, the part that contributes to the prior art, and the computer software product is stored in one of the above storage media (such as ROM/RAM, magnetic disc, optical disc), several instructions are included to make a device execute the method of each embodiment of the present application.

以上仅为本申请的优选实施例,并非因此限制本申请的专利范围,凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换,或直接或间接运用在其他相关的技术领域,均同理包括在本申请的专利保护范围内。The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structures or equivalent process transformations made by using the description of the application and the accompanying drawings are directly or indirectly used in other related technical fields. , are all included in the patent protection scope of the present application in the same way.

Claims (10)

1.一种后视镜图像融合方法,其特征在于,所述后视镜图像融合方法包括:1. a rearview mirror image fusion method is characterized in that, the rearview mirror image fusion method comprises: 获取汽车电子后视镜的后视镜图像数据,其中,所述电子后视镜包括电子内后视镜和左右电子外后视镜;Acquiring the rearview mirror image data of the electronic rearview mirror of the automobile, wherein the electronic rearview mirror includes an electronic inner rearview mirror and a left and right electronic outer rearview mirror; 将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像。The image data of the rearview mirror is input into the FPGA chip for deep image fusion to generate a fused image. 2.如权利要求1所述的后视镜图像融合方法,其特征在于,所述获取汽车电子后视镜的后视镜图像数据的步骤之后,还包括:2. rearview mirror image fusion method as claimed in claim 1, is characterized in that, after the step of the rearview mirror image data of described acquisition electronic rearview mirror of automobile, also comprises: 将所述后视镜图像数据进行数据解串处理,以将串行的后视镜图像数据转换为并行的后视镜图像数据;Carrying out data deserialization processing on the rearview mirror image data, to convert the serial rearview mirror image data into parallel rearview mirror image data; 所述将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像的步骤包括:The described rearview mirror image data is input in the FPGA chip to carry out image depth fusion, and the step of generating the picture image after fusion comprises: 将所述并行的后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像。The parallel rearview mirror image data is input into the FPGA chip for deep image fusion to generate a fused image. 3.如权利要求2所述的后视镜图像融合方法,其特征在于,所述将所述后视镜图像数据进行数据解串处理,以将串行的后视镜图像数据转换为并行的后视镜图像数据的步骤之后,还包括:3. the rearview mirror image fusion method as claimed in claim 2, is characterized in that, described rearview mirror image data is carried out data deserialization processing, to convert serial rearview mirror image data into parallel After the step of rearview mirror image data, it also includes: 对所述并行的后视镜图像数据进行图像信号处理,得到处理后的并行的后视镜图像数据。Image signal processing is performed on the parallel rearview mirror image data to obtain processed parallel rearview mirror image data. 4.如权利要求1所述的后视镜图像融合方法,其特征在于,所述将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像的步骤包括:4. rearview mirror image fusion method as claimed in claim 1, is characterized in that, described rearview mirror image data is input in the FPGA chip and carries out image depth fusion, and the step of generating the picture image after fusion comprises: 获取输入至所述FPGA芯片中的所述后视镜图像数据;Obtain the image data of the rearview mirror input into the FPGA chip; 对所述后视镜图像数据进行图像校正,得到校正后的后视镜图像数据;Carrying out image correction to the rearview mirror image data to obtain corrected rearview mirror image data; 获取汽车三维模型和汽车参数;Obtain the 3D model of the car and the parameters of the car; 将所述校正后的后视镜图像数据和所述汽车参数投影到所述汽车三维模型上,得到投影的图像数据;Projecting the corrected rearview mirror image data and the car parameters onto the three-dimensional car model to obtain projected image data; 根据所述投影的图像数据对图像的连接处进行拼接处理,得到拼接后的图像数据;splicing the joints of the images according to the projected image data to obtain the spliced image data; 对所述拼接后的图像数据进行渲染处理,得到所述融合后的画面图像。Rendering is performed on the spliced image data to obtain the fused picture image. 5.如权利要求4所述的后视镜图像融合方法,其特征在于,所述获取输入至所述FPGA芯片中的所述后视镜图像数据的步骤之前,还包括:5. rearview mirror image fusion method as claimed in claim 4, is characterized in that, before the step of described rearview mirror image data that described acquisition is inputted in the described FPGA chip, also comprises: 加载图像处理的全局配置,初始化OpenGL ES组件和摄像头设备驱动。Load the global configuration of image processing, initialize OpenGL ES components and camera device drivers. 6.如权利要求1所述的后视镜图像融合方法,其特征在于,所述获取汽车电子后视镜的后视镜图像数据的步骤之前,还包括:6. rearview mirror image fusion method as claimed in claim 1, is characterized in that, before the step of the rearview mirror image data of described obtaining automobile electronic rearview mirror, also comprises: 采集得到所述后视镜图像数据,包括:The image data of the rearview mirror is collected, including: 通过后视摄像头采集得到汽车后方的后视镜图像数据;The rearview mirror image data of the rear of the car is collected by the rearview camera; 通过安装在所述左右电子外后视镜处的左右两路摄像头采集得到左右两侧的后视镜图像数据。The image data of the rearview mirrors on the left and right sides are collected by the left and right cameras installed at the left and right electronic exterior mirrors. 7.如权利要求1所述的后视镜图像融合方法,其特征在于,所述将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像的步骤之后,还包括:7. rearview mirror image fusion method as claimed in claim 1, is characterized in that, described rearview mirror image data is input in FPGA chip and carries out image depth fusion, after the step of the picture image after generating fusion, Also includes: 将所述融合后的画面图像显示于单个显示设备上。and displaying the fused image on a single display device. 8.一种后视镜图像融合装置,其特征在于,所述后视镜图像融合装置包括:8. A rearview mirror image fusion device is characterized in that, said rearview mirror image fusion device comprises: 图像获取模块,用于获取汽车电子后视镜的后视镜图像数据,其中,所述电子后视镜包括电子内后视镜和左右电子外后视镜;The image acquisition module is used to acquire the rearview mirror image data of the electronic rearview mirror of the automobile, wherein the electronic rearview mirror includes an electronic inner rearview mirror and a left and right electronic outer rearview mirror; 图像融合模块,用于将所述后视镜图像数据输入至FPGA芯片中进行图像深度融合,生成融合后的画面图像。The image fusion module is used to input the image data of the rearview mirror into the FPGA chip for image depth fusion to generate a fused picture image. 9.一种车载设备,其特征在于,所述车载设备包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的后视镜图像融合程序,所述后视镜图像融合程序被所述处理器执行时实现如权利要求1-7中任一项所述的后视镜图像融合方法的步骤。9. A vehicle-mounted device, characterized in that, said vehicle-mounted device comprises a memory, a processor and a rearview mirror image fusion program stored on said memory and operable on said processor, said rearview mirror image When the fusion program is executed by the processor, the steps of the rearview mirror image fusion method according to any one of claims 1-7 are realized. 10.一种计算机可读存储介质,其特征在于,所述计算机可读存储介质上存储有后视镜图像融合程序,所述后视镜图像融合程序被处理器执行时实现如权利要求1-7中任一项所述的后视镜图像融合方法的步骤。10. A computer-readable storage medium, characterized in that, the computer-readable storage medium is stored with a rearview mirror image fusion program, and when the rearview mirror image fusion program is executed by a processor, it is realized as claimed in claim 1- The step of the rearview mirror image fusion method described in any one of 7.
CN202211742598.2A 2022-12-30 2022-12-30 Rearview mirror image fusion method, device, vehicle-mounted equipment and storage medium Pending CN116205829A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211742598.2A CN116205829A (en) 2022-12-30 2022-12-30 Rearview mirror image fusion method, device, vehicle-mounted equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211742598.2A CN116205829A (en) 2022-12-30 2022-12-30 Rearview mirror image fusion method, device, vehicle-mounted equipment and storage medium

Publications (1)

Publication Number Publication Date
CN116205829A true CN116205829A (en) 2023-06-02

Family

ID=86516489

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211742598.2A Pending CN116205829A (en) 2022-12-30 2022-12-30 Rearview mirror image fusion method, device, vehicle-mounted equipment and storage medium

Country Status (1)

Country Link
CN (1) CN116205829A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116729260A (en) * 2023-08-02 2023-09-12 广州勘帝德电子科技有限公司 An electronic rearview mirror CMS image correction system and device
CN117237393A (en) * 2023-11-06 2023-12-15 深圳金语科技有限公司 Image processing method and device based on streaming media rearview mirror and computer equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1333626A (en) * 2000-07-07 2002-01-30 松下电器产业株式会社 Image synthesizing device and method
CN104590118A (en) * 2014-11-28 2015-05-06 广东好帮手电子科技股份有限公司 Driving environment panoramic display system and method based on rearview mirror box
CN112373395A (en) * 2020-11-24 2021-02-19 马鞍山采倪汽车科技有限公司 Panoramic reverse image system
WO2022052789A1 (en) * 2020-09-08 2022-03-17 恒大新能源汽车投资控股集团有限公司 Full-field dynamic display method for in-vehicle rearview mirror, storage medium, and electronic device
CN216331763U (en) * 2021-08-30 2022-04-19 鹰驾科技(深圳)有限公司 Intelligent automobile electronic rearview mirror equipment of integrated panorama function and BSD function
CN114821544A (en) * 2022-06-29 2022-07-29 小米汽车科技有限公司 Perception information generation method and device, vehicle, electronic equipment and storage medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1333626A (en) * 2000-07-07 2002-01-30 松下电器产业株式会社 Image synthesizing device and method
CN104590118A (en) * 2014-11-28 2015-05-06 广东好帮手电子科技股份有限公司 Driving environment panoramic display system and method based on rearview mirror box
WO2022052789A1 (en) * 2020-09-08 2022-03-17 恒大新能源汽车投资控股集团有限公司 Full-field dynamic display method for in-vehicle rearview mirror, storage medium, and electronic device
CN112373395A (en) * 2020-11-24 2021-02-19 马鞍山采倪汽车科技有限公司 Panoramic reverse image system
CN216331763U (en) * 2021-08-30 2022-04-19 鹰驾科技(深圳)有限公司 Intelligent automobile electronic rearview mirror equipment of integrated panorama function and BSD function
CN114821544A (en) * 2022-06-29 2022-07-29 小米汽车科技有限公司 Perception information generation method and device, vehicle, electronic equipment and storage medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
卢纪博;: "云度π3车全景影像功能无法正常使用", 汽车维护与修理, no. 08, 15 April 2020 (2020-04-15) *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116729260A (en) * 2023-08-02 2023-09-12 广州勘帝德电子科技有限公司 An electronic rearview mirror CMS image correction system and device
CN117237393A (en) * 2023-11-06 2023-12-15 深圳金语科技有限公司 Image processing method and device based on streaming media rearview mirror and computer equipment
CN117237393B (en) * 2023-11-06 2024-05-17 深圳金语科技有限公司 Image processing method and device based on streaming media rearview mirror and computer equipment

Similar Documents

Publication Publication Date Title
JP7397262B2 (en) 3D rendering for surround views using predefined viewpoint lookup tables
CN114913506B (en) A 3D object detection method and device based on multi-view fusion
JP7164706B2 (en) Image stitching method and device, in-vehicle image processing device, electronic device, storage medium
US20190349571A1 (en) Distortion correction for vehicle surround view camera projections
CN116205829A (en) Rearview mirror image fusion method, device, vehicle-mounted equipment and storage medium
CN110728638A (en) Image distortion correction method, vehicle machine and vehicle
CN113490879B (en) Using real-time ray tracing for lens remapping
CN102045546A (en) Panoramic parking assist system
US11341607B2 (en) Enhanced rendering of surround view images
CN204258976U (en) A kind of vehicle-mounted panoramic visible system
CN114863096A (en) Semantic map construction and positioning method and device for indoor parking lot
CN107240065A (en) A kind of 3D full view image generating systems and method
CN113837936A (en) Panoramic image generation method and device
CN116238414A (en) Projection method, device, equipment and storage medium of a smart car lamp
WO2017043331A1 (en) Image processing device and image processing method
CN114872631A (en) Method and system for realizing functions of transparent chassis
Turturici et al. Low-power DSP system for real-time correction of fish-eye cameras in automotive driver assistance applications
CN105774657B (en) Single-camera panoramic reverse image system
CN113315961B (en) Method and device for transparent A-pillar image block mapping
CN115690733A (en) Method, device, electronic device and storage medium for constructing parking map
WO2023095340A1 (en) Image processing method, image displaying method, image processing device, and image displaying device
CN114663521A (en) A Surround View Splicing Processing Method for Assisted Parking
JP6977725B2 (en) Image processing device and image processing method
CN116228876A (en) Camera calibration method, device, controller, vehicle and storage medium
CN115431879A (en) Vehicle rear image display method, device, equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination