JP2022106769A - Circuit device and electronic apparatus - Google Patents

Abstract

A circuit device, an electronic device, an error detection method, and the like capable of appropriately detecting an error in an image displayed on a head-up display are provided. A circuit device (100) includes an image processing circuit (135) and a comparison circuit (145). The image processing circuit 135 performs a first mapping process of mapping the input first image IMA1 to a second image IMA2 to be projected onto the object to be projected, and a reverse mapping process of the first mapping process. and a second mapping process for converting the second image IMA2 into a third image IMA3. The comparison circuit 145 performs a comparison between the first image IMA1 and the third image IMA3, and outputs the result of the comparison as information for error detection of the second image IMA2. [Selection diagram] Fig. 1

Description

The present invention relates to circuit devices, electronic devices, and the like.

A head-up display (HUD) is known in which a two-dimensional image is projected onto a projected object and the projected image is presented to a user. The projected object is, for example, the windshield of an automobile, and often has a non-planar shape. Therefore, the head-mounted display controller (HUDC: Head Up Display Controller) performs image mapping processing so that an undistorted image is presented to the user when the image is projected on a non-plane surface.

The prior art of mapping processing is disclosed in, for example, Patent Document 1. Patent Document 1 describes a pixel mapping method between a source array and a destination array, and particularly describes a method of inverse mapping from a destination array to a source array in detail. The source array is an array of pixels before the mapping process, and the destination array is an array of pixels that is the destination when the source array is coordinate-transformed by the mapping process.

Japanese Unexamined Patent Publication No. 10-49665

When the head-up display controller performs the mapping process as described above, there is a problem that the image displayed on the head-up display has appropriate display contents. That is, there is a problem that it is desired to confirm whether or not the map data used for the mapping process and the coordinate transformation using the map data are normal. In the prior art of mapping processing, the mapping method itself is disclosed, but the method for confirming whether the mapping processing is normal is not disclosed.

One aspect of the present invention is a first mapping process for mapping an input first image to a second image for projecting onto a projected object, and a reverse mapping process for the first mapping process. An image processing circuit that performs a second mapping process for converting the second image into a third image is compared with the first image and the third image, and the result of the comparison is obtained. The present invention relates to a circuit device including a comparison circuit that outputs information for detecting an error in the second image.

Further, in one aspect of the present invention, the comparison circuit is based on the pixel value of the first image and the pixel value of the third image, or the pixel value of the edge image of the first image and the first. An index indicating the degree of coincidence between the first image and the third image may be obtained as a result of the comparison based on the pixel value of the edge image of the third image.

Further, in one aspect of the present invention, the image processing circuit performs the first mapping process using the first map data generated from the map data corresponding to the projected object, and is the map data. The second mapping process may be performed using the map data of 2.

Another aspect of the present invention is the first aspect of mapping the input first image to the second image for projecting onto the projected object based on the map data corresponding to the projected object. An image processing circuit that performs a mapping process and a second mapping process for converting the first image into a third image by a second mapping process different from the first mapping process based on the map data. A circuit including a comparison circuit that makes a comparison between the second image and the third image and outputs the result of the comparison as information for performing error detection of the second image. Related to the device.

Further, in another aspect of the present invention, the comparison circuit is based on the pixel value of the second image and the pixel value of the third image, or the pixel value of the edge image of the second image and the said. An index indicating the degree of coincidence between the second image and the third image may be obtained as a result of the comparison based on the pixel value of the edge image of the third image.

Further, in another aspect of the present invention, the image processing circuit performs the first mapping process using the first map data generated from the map data, and obtains the second map data which is the map data. You may use it to perform the second mapping process.

Further, in another aspect of the present invention, the image processing circuit generates the third image having a resolution lower than that of the first image, and the comparison circuit generates the third image with respect to the second image. The resolution may be reduced to match the resolution of the image of 3, and the third image may be compared with the second image after the reduction in resolution.

Further, in another aspect of the present invention, an error detection circuit that detects an error in the second image based on the result of the comparison may be included.

Further, in another aspect of the present invention, an operation mode setting register for setting the operation mode of the circuit device when an error is determined by the error detection circuit may be included.

In another aspect of the present invention, the operation mode setting register has a mode of notifying the external device of the circuit device of the error detection result, a mode of hiding the second image, or a specific image. The mode for displaying the above may be set as the operation mode.

Further, in another aspect of the present invention, the error detection circuit may perform the error detection by comparing the result of the comparison with a threshold value for determining an error in the second image.

In another aspect of the present invention, a threshold register in which the threshold is set may be included.

Yet another aspect of the present invention relates to an electronic device including the circuit device according to any of the above.

A first configuration example of the circuit device of this embodiment. A modification of the circuit device of the first configuration example. The figure which schematically showed the operation of the circuit apparatus of the 1st configuration example. A second configuration example of the circuit device of this embodiment. Detailed configuration example of the comparison circuit. A third configuration example of the circuit device of this embodiment. The figure which schematically showed the operation of the circuit apparatus of the 3rd configuration example. A fourth configuration example of the circuit device of this embodiment. The first example of the analysis image. An example of a reference image. An averaged image of the reference image. The second example of the analysis image. A third example of an analysis image. Fourth example of the analysis image. An example of an analysis image in a region of interest. An example of the edge value calculated from the analysis image. Histogram of each channel of YCbCr in the region of interest. Cross-correlation value obtained by performing cross-correlation calculation on the histogram. An example of a histogram of the analysis image and the reference image. An example of the cross-correlation value of the histogram of the analysis image and the reference image. Configuration example of electronic equipment.

Hereinafter, preferred embodiments of the present invention will be described in detail. The present embodiment described below does not unreasonably limit the content of the present invention described in the claims, and all the configurations described in the present embodiment are indispensable as a means for solving the present invention. Not necessarily.

1. 1. First and Second Configuration Examples FIG. 1 is a first configuration example of the circuit device of the present embodiment. The circuit device 100 includes an interface 110 (first interface), a preprocessing circuit 125, an image processing circuit 135, an interface 140 (second interface), a comparison circuit 145, a register circuit 170, and an interface 190 (third interface). include. The circuit device 100 is, for example, an integrated circuit device (IC).

The interface 110 receives, for example, image data transmitted from the processing device 200 or the like to the circuit device 100, and converts the received image data into a format used inside the circuit device 100. For example, the interface 110 is an Open LDI (Open LVDS Display Interface), and converts a serial signal received by LVDS (Low Voltage Differential Signaling) into an RGB parallel signal. The processing device 200 is, for example, a SoC (System on a Chip), an MCU (Micro Control Unit), a CPU (Central Processing Unit), or the like.

The preprocessing circuit 125 performs preprocessing on the image data input from the interface 110, and outputs the image IMA1 after the preprocessing. For example, the preprocessing circuit 125 performs gamma correction, FRC (Frame Rate Control), white balance processing, and the like as preprocessing. The preprocessing circuit 125 is a circuit that acquires an image input to the image processing circuit 135, and is also called an image acquisition circuit.

The image processing circuit 135 includes a mapping processing unit WEA1 that maps an image according to the surface shape of the projected object, and a mapping processing unit WEA2 that performs reverse mapping thereof. Specifically, the mapping processing unit WEA1 performs mapping processing using the map data MPA1 on the image IMA1 output by the preprocessing circuit 125, and outputs the image IMA2 after the mapping processing. This mapping process is a process of transforming the image IMA1 so that the image projected on the projected object is not distorted when viewed from the user. The mapping processing unit WEA2 performs mapping processing using the map data MPA2 on the image IMA2 output by the mapping processing unit WEA1, and outputs the image IMA3 after the mapping processing. This mapping process corresponds to the inverse conversion of the mapping process performed by the mapping processing unit WEA1. That is, it is a conversion that returns the image IMA2 deformed according to the projected object to the image before the deformation.

If there is no error in the mapping process, the degree of matching between the image IMA3 and the IMA1 is high. Therefore, by comparing the image IMA3 and the IMA1 with the comparison circuit 145 described later, it can be determined whether the image IMA2 which is the display image is a normal image. Can be detected. The images IMA3 and IMA1 do not necessarily have to completely match as image data, and the images may be similar to the extent that they can be compared by the comparison circuit 145. That is, the mapping process performed by the mapping processing unit WEA2 is an inverse conversion to the mapping processing performed by the mapping processing unit WEA1, but it does not have to be an inverse conversion such that the image IMA3 and the IMA1 completely match.

The projected object is an object on which the image IAM2 is projected or displayed. That is, the image IMA2 is output from the interface 140 to the projection device, and the projection device projects the image IMA2 onto the projected object. The image projected on the projected object is presented to the user. The projection device includes, for example, a liquid crystal display panel, a display driver for driving the liquid crystal display panel, a light source, and a lens. The display driver displays an image on the liquid crystal display panel based on the received image data, the light source outputs light to the liquid crystal display panel, and the light passing through the liquid crystal display panel is projected onto the object to be projected by the lens. The projected object has a reflecting surface that reflects the projected light, for example, the reflecting surface is the surface of the projected object. The reflecting surface is a surface that reflects at least a part of the incident light, and the reflection may be either diffuse reflection or specular reflection. The projected object is an object on which the head-up display projects an image. For example, in the case of an in-vehicle head-up display, the projected object is an automobile windshield or the like. The projected object is not limited to this example, and may be any object whose image is projected or displayed by the projection device.

The mapping process is a process of coordinate-transforming the pixel position of an image based on the map data. The mapping process can include a pixel value interpolation process and the like as a process associated with the coordinate transformation. There are forward mapping and reverse mapping as the mapping process. In the forward mapping, the pixel position of the image before processing is coordinate-converted to the pixel position of the image after processing, and the image after processing is generated using the result of the coordinate conversion. That is, the pixels of the image before processing are sequentially selected, the pixel positions (xa, ya) of the selected pixels are coordinate-converted, and the pixel values of the pixels are converted into the pixel positions (ua, ua, after coordinate conversion. Let it be the pixel value in va). On the other hand, in reverse mapping, the pixel position of the processed image is coordinate-converted to the pixel position of the image before processing, and the processed image is generated using the result of the coordinate conversion. That is, the pixel positions of the processed image are sequentially selected, the selected pixel positions (ub, vb) are coordinate-converted to the pixel positions of the image before processing, and the pixel positions (xb, yb) are converted into coordinates. The pixel value of the pixel is taken as the pixel value at the pixel position (ub, vb) of the processed image.

The map data is data showing coordinate conversion corresponding to the shape of the reflecting surface of the projected object, and is table data that associates the pixel position of the image before the mapping process with the pixel position of the image after the mapping process. .. The map used for forward mapping is called a forward map, and in the map data, the pixel positions (ua, va) of the image after the mapping process are associated with the pixel positions (xa, ya) of the image before the mapping process. This is the table data that was created. The map used for reverse mapping is called a reverse map, and in the map data, the pixel positions (xb, yb) of the image before the mapping process are associated with the pixel positions (ub, vb) of the image after the mapping process. This is the table data that was created. The map data MPA1 and MPA2 used by the mapping processing units WEA1 and WEA2 are input to the mapping processing units WEA1 and WEA2 from, for example, the processing device 200 via the interface 190.

The interface 140 outputs the image IMA2 to the outside of the circuit device 100. The outside of the circuit device 100 is, for example, a display driver that drives a display panel. For example, the interface 140 is an LVDS interface and converts RGB parallel signals from the image processing circuit 135 into LVDS serial signals.

The comparison circuit 145 performs comparison processing between the image IMA1 and the image IMA3, and outputs the comparison result. This comparison result is used to detect an error in image IMA2. That is, it is used to verify whether or not the mapping processing performed by the map data MPA1 and the mapping processing unit WEA1 is normal. Specifically, the comparison circuit 145 obtains an index indicating the degree of agreement between the image IMA1 and the image IMA3. This index corresponds to the shape index or the shape index described later, and is an index whose value changes according to the shape of what is displayed on the images IMA1 and IMA3 or the degree of coincidence of the edges thereof. Alternatively, a visibility index described later may be obtained as an index. The visibility index is an index whose value changes according to the degree of dissimilarity between the image of the region of interest in which the icon or the like is displayed and the image of the other background region. The image of the region of interest is also called the foreground image. Specifically, by comparing the histograms of the pixel values of the images IMA1 and IMA3, an index showing the contrast between the foreground image and the background image is obtained.

The register circuit 170 is configured to be accessible from the processing apparatus 200 via the interface 190. The register circuit 170 includes a comparison result register 175 that stores the comparison result information output by the comparison circuit 145, and the processing device 200 reads the comparison result information from the comparison result register 175 via the interface 190. The processing device 200 includes an error detection circuit that detects an error based on the information of the comparison result. By the above image comparison and error detection, error detection of the image IMA2 projected on the projected object is realized. That is, it is detected whether or not the mapping process by the mapping process unit WEA1 is normally performed.

In the above configuration, the comparison result information output by the comparison circuit 145 is stored in the register circuit 170, the processing device 200 reads the comparison result information from the register circuit 170 via the interface 190, and the comparison result information is read in the processing device 200. The error detection circuit detects the error, but the present invention is not limited to this. That is, when an error detection circuit is included in the circuit device 100 as shown in FIG. 4, for example, even if the error detection circuit in the circuit device 100 performs error detection based on the comparison result information output by the comparison circuit 145. good. In this case, after the error detection circuit in the circuit device 100 detects the error, the circuit device 100 may interrupt the processing device 200 to notify the error.

The interface 190 communicates setting information, control information, and the like between the circuit device 100 and the processing device 200. For example, the interface 190 is a serial communication interface such as an SPI (Serial Peripheral Interface) system or an I2C system. The setting information and control information from the processing device 200 are written to, for example, the register circuit 170, and the circuit device 100 operates according to the setting information and control information.

The logic circuit included in the circuit device 100 may be configured as an individual circuit, for example, or may be configured as an integrated circuit by automatic arrangement and wiring or the like. The logic circuit is, for example, a preprocessing circuit 125, an image processing circuit 135, and a comparison circuit 145. Further, a part or all of these logic circuits may be realized by a processor such as a DSP (Digital Signal Processor). In this case, a program or instruction set in which the function of each circuit is described is stored in the memory, and the processor executes the program or instruction set to realize the function of each circuit.

The above-mentioned preprocessing circuit 125 is a circuit for acquiring an image input to the image processing circuit 135, but the present invention is not limited to this. FIG. 2 is a modification of the circuit device of the first configuration example. For example, the image data input from the interface 110 is set to IMA1', and the mapping processing unit WEA1 performs mapping processing using the map data MPA1 on the IMA1'. After that, the post-processing circuit 126 may perform gamma correction, FRC, white balance processing, and the like as post-processing on the image data after the mapping processing by the mapping processing unit WEA1. The post-processing circuit 126 corresponds to the pre-processing circuit 125 of FIG. The mapping processing unit WEA2 performs mapping processing using the map data MPA2 on the image data IMA2'output by the post-processing circuit 126. In this case, in addition to being able to confirm whether or not the mapping process is normal, it is also possible to confirm that the image has not been significantly changed by post-processing such as gamma correction, FRC, and white balance processing.

Next, the operation of the circuit device 100 of FIG. 1 will be described. FIG. 3 is a diagram schematically showing the operation of the circuit device 100 of FIG.

As shown in FIG. 3, the reference map data is subjected to map conversion processing to generate map A, and the reference map data is used as map B. Specifically, the non-volatile memory 210 shown in FIG. 1 stores the reference map data, and the processing device 200 reads the reference map data from the non-volatile memory 210 and performs map conversion processing to correspond to the map A. Generates the map data MPA1 to be used. The map data MPA1 is input to the mapping processing unit WEA1 via the interface 190. The map conversion process includes, for example, conversion between a forward map and a reverse map, rotation of a map, translation of a map, and the like. Further, the processing device 200 outputs the reference map data as the map data MPA2 corresponding to the map B. The map data MPA2 is input to the mapping processing unit WEA2 via the interface 190.

The warp engine A corresponds to the mapping processing unit WEA1, and maps the image IMA1 using the map A to generate the image IMA2. The mapping process is also called a warp process. Further, the warp engine B corresponds to the mapping processing unit WEA2, and the image IMA2 is reverse-mapped using the map B to generate the image IMA3. This inverse mapping process may include rotation transformation and translation of an image corresponding to the inverse transformation of map rotation and translation in the map transformation process.

Since the warp engine A generates the image IMA2 for display, it performs a higher quality mapping process than the warp engine B. For example, high-resolution mapping processing is performed according to the display resolution. In addition, pixel value interpolation processing is performed to ensure display quality. On the other hand, since the warp engine B generates the image IMA3 for error detection, the mapping process is simplified as compared with the warp engine A. For example, by thinning out the target pixels for coordinate transformation, an image IMA3 having a resolution lower than that of the image IMA1 is generated. Further, the pixel value interpolation process may be omitted.

In the comparison process performed by the comparison circuit 145, the image IMA1 is thinned out as described later, the pixels of the image IMA1 and the IMA3 are aligned, and the index is obtained from the image after the alignment.

In the above operation, the map A is generated by performing map conversion processing on the reference map data. Therefore, if an abnormality occurs in the map A due to an error in the map conversion process, an abnormality may occur in the image IMA2 generated by the warp engine A using the map A. Further, even if there is no abnormality in the map A, if an abnormality occurs in the mapping process by the warp engine A, there is a possibility that an abnormality occurs in the image IMA2 generated by the warp engine A. In order to detect such an abnormality, it is necessary to detect the error from the image deformed by the warp engine A, and it is difficult to detect the error by the prior art. For example, when data error detection such as CRC is used, the CRC value of the image before being converted by the warp engine A is used as a reference. Therefore, the CRC value obtained from the image after being converted by the warp engine A is It doesn't match in the first place.

In the present embodiment, the image IMA2 is subjected to reverse mapping processing by the warp engine B, and the image IMA3 after the reverse mapping processing is compared with the original image IMA1 to obtain an index indicating the degree of agreement. When there is no abnormality in the map A and the warp engine A, the degree of coincidence between the image IMA3 and the IMA1 is high, so that the error of the image IMA2 can be detected by the index. Since the index is a value that changes according to the degree of coincidence, the images IMA1 and IMA3 do not have to be a perfect match. That is, even if there is a slight abnormality in the map A or the warp engine A, if the image IMA2 is visible to the user, it may be determined as non-error. For example, in error detection, an index is determined as a threshold value, and by setting the threshold value, it is possible to adjust how much the degree of agreement is determined as an error.

FIG. 4 is a second configuration example of the circuit device of the present embodiment. In FIG. 4, the circuit device 100 includes an error detection circuit 150. Further, the register circuit 170 includes an error detection result register 176, an operation mode setting register 177, and a threshold value register 178. The same components as those already described are designated by the same reference numerals, and the description of the components will be omitted as appropriate.

The comparison circuit 145 outputs the comparison result of the images IMA1 and IMA3 to the error detection circuit 150. The error detection circuit 150 performs error detection of the image IMA2 for display based on the comparison result. When the comparison result is the above-mentioned index, the error detection circuit 150 performs error detection by comparing the index and the threshold value. The threshold value for the shape index is a threshold value indicating how much similarity is acceptable. The threshold value for the visibility index is a threshold value indicating how much visibility is acceptable.

When an error is detected by the error detection circuit 150, the image processing circuit 135 stops the output of the image IMA2 to the interface 140. Alternatively, the interface 140 stops the output of the image IMA2 when an error is detected by the error detection circuit 150. The interface 140 may output the image IMA2 together with the error information, and the display driver receiving the error information may perform an operation based on the error information. The error information is, for example, an error determination flag, an index, or the like. The operation based on the error information is, for example, stopping the display.

The error detection result register 176 stores the error detection result output by the error detection circuit 150. The error detection result is, for example, an error determination flag indicating whether or not the display image is determined to be an error.

Mode setting information is set in the operation mode setting register 177 from the processing device 200 via the interface 190. The mode setting information is information for setting the operation mode of the circuit device 100 when the error detection result indicates an error, and is information for setting what kind of operation is performed as the operation at the time of error detection. The image processing circuit 135 performs the above-mentioned error detection operation based on the mode setting information. The processing device 200 may be configured to perform the operation at the time of error detection. In this case, the processing device 200 reads the error detection result from the error detection result register 176 via the interface 190. When the error detection result indicates an error, the processing device 200 performs the operation at the time of error detection. The operation at the time of error detection is, for example, stopping the transmission of image data to the circuit device 100, transmitting predetermined image data to the circuit device 100, or the like. The predetermined image data is image data of an image presented to the user when an error is detected.

The threshold value register 178 is set from the processing device 200 via the interface 190. The error detection circuit 150 compares the index with the threshold value set in the threshold value register 178 to perform error detection. That is, the error detection circuit 150 compares the shape index with the threshold value for the shape index, compares the visibility index with the threshold value for the visibility index, performs error detection based on the comparison result, and obtains the result as an error. Output as a detection result. For example, if at least one of the comparison results indicates an error, it is determined that an error exists in the second image.

Also in the configuration of FIG. 4, similarly to the configuration of FIG. 1, by comparing the image IMA1 and the IMA3, the error detection of the image IMA2 after the mapping process projected on the projected object is detected based on the comparison result. It can be carried out.

Although the case where the circuit device 100 outputs the error detection result to the processing device 200 has been described above as an example, the circuit device 100 does not output the error detection result to the outside of the circuit device 100, and the error detection result is output to the circuit device. It may be used only inside 100. For example, the circuit device 100 may shift to safe mode when an error is detected. In the safe mode, for example, a predetermined error notification image may be transmitted to the display driver, or control may be performed to turn off the backlight of the liquid crystal display device. As a result, the error status can be directly notified to the user by the control from the circuit device 100 instead of the control from the processing device 200.

As the circuit device 100 of FIGS. 1 and 4 described above, a head-up display controller that controls a head-up display and a display controller that controls a display driver can be assumed. However, the circuit apparatus to which the method of this embodiment can be applied is not limited to these. For example, the circuit device may be a display driver that includes the functionality of a display controller. When the circuit device is a head-up display controller, a display controller, or a display driver, the circuit device is, for example, an integrated circuit device (IC). The circuit device may include a plurality of integrated circuit devices. For example, the circuit device includes a head-up display controller which is a first integrated circuit device and a processing device which is a second integrated circuit device. In this case, the head-up display controller includes a comparison circuit that performs comparison processing between the images IMA1 and IMA3, and the processing device includes an error detection circuit that performs error detection based on the comparison result received from the head-up display controller.

FIG. 5 is a detailed configuration example of the comparison circuit 145. The comparison circuit 145 includes a pixel arrangement processing unit 146 and an index acquisition unit 147.

The pixel arrangement processing unit 146 performs pixel arrangement processing for comparing the images IMA1 and IMA3, and outputs the processed images IMA1 and IMA3 to the index acquisition unit 147. Specifically, since the image IMA3 has a lower resolution than the image IMA1 as described above, the pixel arranging processing unit 146 performs a subsampling process that matches the number of pixels of the image IMA1 with the number of pixels of the image IMA3. Further, for example, rotation processing, translation processing, interpolation processing, and the like may be performed so that the pixels are arranged at the same pixel positions in the images IMA1 and IMA3.

The index acquisition unit 147 obtains an index for detecting an error in the image IMA2 based on the pixel-arranged images IMA1 and IMA3. As described above, the index is a shape index, and a visibility index may be further acquired. Details of the shape index and visibility index will be described later. The index output by the index acquisition unit 147 is stored in the comparison result register 175 as information on the comparison result. Alternatively, the error detection circuit 150 performs error detection based on the index.

According to the above embodiment, the image processing circuit 135 has a first mapping process for mapping the input first image to a second image for projecting onto the projected object, and a first mapping process. The second mapping process of converting the second image into the third image by the reverse mapping process of the process is performed. The comparison circuit 145 makes a comparison between the first image and the third image, and outputs the result of the comparison as information for performing error detection of the second image.

In FIGS. 1 to 5, IMA1 is a first image, IMA2 is a second image, and IMA3 is a third image. The mapping process performed by the mapping processing unit WEA1 is the first mapping process, and the mapping process performed by the mapping processing unit WEA2 is the second mapping process. The information for performing error detection is information on the comparison result of the images IMA1 and IMA3, and corresponds to, for example, a shape index or a visibility index, or a shape index and a visibility index described later.

As described with reference to FIG. 3 and the like, when an abnormality occurs in the mapping process or the map data, an abnormality occurs in the second image, but it has been difficult to detect the abnormality by the prior art. In this regard, according to the present embodiment, in the second mapping process, the first image and the third image are obtained by converting the second image into the third image by the inverse mapping process of the first mapping process. It becomes possible to compare with the image. Since the third image is generated from the second image, it can be determined that there is no error in the second image if the degree of agreement between the third image and the reference first image is high. In this way, it is possible to detect whether or not the image projected on the projected object such as a head-up display has appropriate display contents.

Further, in the present embodiment, the comparison circuit 145 is based on the pixel value of the first image and the pixel value of the third image, or the pixel value of the edge image of the first image and the edge image of the third image. As a result of comparison, an index indicating the degree of matching between the first image and the third image is obtained based on the pixel value of.

Here, the pixel value of the edge image means the amount of edges in the pixel, and is a signal value obtained by edge extraction. The index showing the degree of agreement is a shape index, and the details thereof will be described later.

In this way, the error detection of the third image can be performed based on the index indicating the degree of agreement between the first image and the third image, instead of the error detection in bit units such as CRC. can. If no error is detected in the third image, it can be determined that the mapping process has been performed normally, so that an error in the second image generated by the mapping process can be detected. For example, on an in-vehicle head-up display or the like, an icon or the like to be presented to the user is displayed. According to the present embodiment, such an icon can be presented to the user when the shape can be correctly recognized without stopping the display due to a 1-bit error or the like.

Further, in the present embodiment, the image processing circuit 135 performs the first mapping process using the first map data generated from the map data corresponding to the projected object, and obtains the second map data which is the map data. The second mapping process is performed using the data.

In FIGS. 1 to 4, the map data corresponding to the projected object is the map data stored in the non-volatile memory 210. The first map data is the map data MPA1, which is the data of the map A generated by the map conversion process. The second map data is the map data MPA2, which is the data of the map B which is the map data read from the non-volatile memory 210.

When the first map data is generated from the map data corresponding to the projected object, if an abnormality occurs in the processing, there is a possibility that an abnormality may occur in the second image to be mapped using the first map data. be. According to the present embodiment, the reference map data is used as it is as the second map data to generate a third image which is an image for comparison. As a result, it becomes possible to determine whether or not the first mapping process using the first map data was normal based on the comparison result between the third image and the first image.

Further, in the present embodiment, the circuit device 100 includes an error detection circuit 150 that detects an error in the second image based on the result of comparison by the comparison circuit 145.

In this way, the error detection circuit 150 built in the circuit device 100 can detect the error in the displayed image. Then, the circuit device 100 can perform the operation at the time of error detection based on the error detection result, or the error detection result is output to the processing device 200 or the like, and the processing device 200 or the like outputs an error based on the error detection result. The operation at the time of detection can be performed.

Further, in the present embodiment, the circuit device 100 includes an operation mode setting register 177 in which the operation mode of the circuit device 100 when an error is determined by the error detection circuit 150 is set.

In this way, the operation mode of the circuit device 100 when the second image is determined to be an error based on the comparison result of the first image and the second image is set to the operation mode setting register 177 via the interface. Can be set to. For example, the processing device 200 can set the operation mode in the operation mode setting register 177 via the interface 190. For example, it is possible to make the circuit device 100 perform the operation at the time of error detection desired by the user.

Further, in the present embodiment, the operation mode setting register 177 has a mode of notifying the external device of the circuit device 100 of the error detection result, a mode of hiding the second image, or a mode of displaying a specific image. , Set as the operating mode.

The external device is, for example, a processing device 200, such as a SoC or a CPU. The error detection result is output to the external device via, for example, the interface 190. Depending on the interface, the circuit device 100 cannot send a notification to the processing device 200. In that case, the circuit device 100 notifies the processing device 200 by an interrupt, and the processing device sends a notification from the circuit device via the interface. Detailed information can be read. Hiding means setting the display panel to not display an image, for example, setting the entire display area of the display panel to black or white. The specific image is an image different from the display image targeted for error detection, and is an image to be displayed at the time of error detection. For example, it is an image displaying a message, a symbol, a color, or the like presented to the user when an error is detected. In the case of color, for example, a predetermined color such as red is displayed in all or a part of the display area of the display panel.

In this way, the operation mode of the circuit device 100 when the second image is determined to be an error can be set to any of the above three modes. For example, as an operation at the time of error detection desired by the user, it is possible to cause the circuit device 100 to perform any of the above three modes of operation.

Further, in the present embodiment, the error detection circuit 150 performs error detection by comparing the result of the comparison with the threshold value for determining the error of the second image.

In FIG. 4, the threshold value for determining the error of the second image is the threshold value to be compared with the index which is the comparison result.

In CRC, whether or not an error exists in the image data is detected by comparing the CRC value received together with the image data with the CRC value calculated from the received image data. On the other hand, the index that is the comparison result in the present embodiment is not an index that simply indicates whether or not there is an error, but the value changes according to the degree of matching between the first image and the third image. By comparing this index with the threshold value, error detection can be performed even when a data error detection method such as CRC cannot be used. That is, when it is determined that the shape similarity cannot be ensured based on the degree of coincidence, it is possible to determine an error.

Further, in the present embodiment, the circuit device 100 includes a threshold register 178 in which a threshold value is set.

In this way, the threshold value for comparison with the index which is the comparison result can be set in the threshold value register 178 via the interface. For example, the processing device 200 can set a threshold value in the threshold value register 178 via the interface 190. By setting the threshold value variably, it becomes variable how much the shape similarity is judged to be an error. By providing the threshold value register 178, for example, the user can arbitrarily set a threshold value which is an error determination criterion.

2. Third and Fourth Configuration Examples FIG. 6 is a third configuration example of the circuit device of the present embodiment. In FIG. 6, the image processing circuit 135 includes a mapping processing unit WEB1 that maps an image according to the surface shape of the projected object, and a mapping processing unit WEB2 that performs mapping in the same direction as the mapping processing unit WEB1. Then, the comparison circuit 145 compares the images IMB2 and IMB3 output by the mapping processing units WEB1 and WEB2. The same components as those already described are designated by the same reference numerals, and the description of the components will be omitted as appropriate.

The preprocessing circuit 125 outputs the preprocessed image IMB1. The content of the preprocessing is the same as the content described with reference to FIG.

The mapping processing unit WEB1 performs mapping processing using the map data MPB1 on the image IMB1 output by the preprocessing circuit 125, and outputs the image IMB2 after the mapping processing. This mapping process is the same as the mapping process performed by the mapping processing unit WEA1 of FIG. The mapping processing unit WEB2 performs mapping processing using the map data MPB2 on the image IMB1 output by the preprocessing circuit 125, and outputs the image IMB3 after the mapping processing. This mapping process is a process of mapping an image according to the surface shape of the projected object, but it does not have to be the same as the mapping process performed by the mapping processing unit WEB1. That is, at least one of the map data and the mapping process may be different from the mapping process unit WEB1. For example, the mapping processing unit WEB2 generates an image IMB3 having a resolution lower than that of the image IMB2. Further, the pixel value interpolation process may be omitted.

The comparison circuit 145 performs comparison processing between the image IMB2 and the image IMB3, and outputs the comparison result. This comparison result is used to detect an error in the image IMB2. That is, it is used to verify whether or not the mapping process performed by the map data MPB1 and the mapping processing unit WEB1 is normal. Specifically, the comparison circuit 145 obtains an index indicating the degree of agreement between the image IMB2 and the image IMB3. This index corresponds to the shape index or the shape index described later. Alternatively, a visibility index described later may be obtained as an index.

The register circuit 170 includes a comparison result register 175 that stores information on the comparison result output by the comparison circuit 145. The processing device 200 performs error detection based on the information of the comparison result read from the comparison result register 175 via the interface 190. By the above image comparison and error detection, error detection of the image IMB2 projected on the projected object is realized. That is, it is detected whether or not the mapping process by the mapping process unit WEB1 is normally performed.

Next, the operation of the circuit device 100 of FIG. 6 will be described. FIG. 7 is a diagram schematically showing the operation of the circuit device 100 of FIG.

As shown in FIG. 7, the reference map data is subjected to map conversion processing to generate map C, and the reference map data is used as map D. Specifically, the non-volatile memory 210 shown in FIG. 6 stores the reference map data, and the processing device 200 reads the reference map data from the non-volatile memory 210 and performs map conversion processing to correspond to the map C. Generates the map data MPB1 to be used. The map data MPB1 is input to the mapping processing unit WEB1 via the interface 190. Further, the processing device 200 outputs the reference map data as the map data MPB2 corresponding to the map D. The map data MPB2 is input to the mapping processing unit WEB2 via the interface 190.

The warp engine C corresponds to the mapping processing unit WEB1, and maps the image IMB1 using the map C to generate the image IMB2. Further, the warp engine D corresponds to the mapping processing unit WEB2, and maps the image IMB1 using the map D to generate the image IMB3. This mapping process may include rotation conversion and translation of an image corresponding to rotation and translation of the map in the map conversion process.

Since the warp engine C generates the image IMB2 for display, it performs a higher quality mapping process than the warp engine D. For example, high-resolution mapping processing is performed according to the display resolution. In addition, pixel value interpolation processing is performed to ensure display quality. On the other hand, since the warp engine D generates the image IMB3 for error detection, the mapping process is simplified as compared with the warp engine C. For example, by thinning out the target pixels for coordinate transformation, an image IMB3 having a resolution lower than that of the image IMB2 is generated. Further, the pixel value interpolation process may be omitted.

In the comparison process performed by the comparison circuit 145, the image IMB2 is thinned out, the pixels of the image IMB2 and the IMB3 are aligned, and the index is obtained from the image after the alignment. The detailed configuration of the comparison circuit 145 is the same as that in FIG. That is, in FIG. 5, IMA1 and IMA3 may be read as IMB2 and IMB3.

In the present embodiment, the warp engine D performs mapping processing in the same direction as the warp engine C, compares the images IMB2 and IMB3 after the mapping processing, and obtains an index indicating the degree of agreement. When there is no abnormality in the map C and the warp engine C, the degree of coincidence between the image IMB2 and the IMB3 is high, so that an error in the image IMB2 can be detected by the index. Since the index is a value that changes according to the degree of coincidence, the images IMB2 and IMB do not have to be exactly coincident. That is, even if there is a slight abnormality in the map C or the warp engine C, if the image IMB2 is visible to the user, it may be determined as non-error.

FIG. 8 is a fourth configuration example of the circuit device of the present embodiment. In FIG. 8, the circuit device 100 includes an error detection circuit 150. Further, the register circuit 170 includes an error detection result register 176, an operation mode setting register 177, and a threshold value register 178. The same components as those already described are designated by the same reference numerals, and the description of the components will be omitted as appropriate.

The comparison circuit 145 outputs the comparison result of the images IMB2 and IMB3 to the error detection circuit 150. The error detection circuit 150 performs error detection of the image IMB2 for display based on the comparison result. When the comparison result is the above-mentioned index, the error detection circuit 150 performs error detection by comparing the index and the threshold value.

Also in the configuration of FIG. 8, similarly to the configuration of FIG. 6, by comparing the image IMB2 and the IMB3, the error detection of the image IMB2 projected on the projected object after the mapping process is detected based on the comparison result. It can be carried out.

Although the case where the circuit device 100 outputs the error detection result to the processing device 200 has been described above as an example, the circuit device 100 does not output the error detection result to the outside of the circuit device 100, and the error detection result is output to the circuit device. It may be used only inside 100.

As the circuit device 100 of FIGS. 6 and 8 described above, a head-up display controller that controls the head-up display and a display controller that controls the display driver can be assumed. However, the circuit apparatus to which the method of this embodiment can be applied is not limited to these. For example, the circuit device may be a display driver that includes the functionality of a display controller. When the circuit device is a head-up display controller, a display controller, or a display driver, the circuit device is, for example, an integrated circuit device (IC). The circuit device may include a plurality of integrated circuit devices. For example, the circuit device includes a head-up display controller which is a first integrated circuit device and a processing device which is a second integrated circuit device. In this case, the head-up display controller includes a comparison circuit that performs comparison processing of the images IMB2 and IMB3, and the processing device includes an error detection circuit that performs error detection based on the comparison result received from the head-up display controller.

According to the above embodiment, the image processing circuit 135 maps the input first image to the second image for projecting on the projected object based on the map data corresponding to the projected object. The first mapping process is performed, and the second mapping process of converting the first image into the third image by the second mapping process different from the first mapping process is performed based on the map data. The comparison circuit 145 makes a comparison between the second image and the third image, and outputs the result of the comparison as information for performing error detection of the second image.

In FIGS. 6 to 8, IMB1 is the first image, IMB2 is the second image, and IMB3 is the third image. Further, the mapping process performed by the mapping processing unit WEB 1 is the first mapping process, and the mapping process performed by the mapping processing unit WEB 2 is the second mapping process. The information for performing error detection is information on the comparison result of the images IMB2 and IMB3, and corresponds to, for example, a shape index or a visibility index, or a shape index and a visibility index described later.

Here, the difference between the first and second mapping processes is that the hardware or algorithm is different. For example, in FIGS. 6 and 8, the mapping processing units WEB1 and WEB2 may be configured by separate mapping processing circuits. Alternatively, the difference between the first and second mapping processes means that at least one of the map data and the contents of the mapping process is different. For example, as described with reference to FIG. 7, the map C used by the warp engine C corresponding to the first mapping process is a map conversion process of the reference map data, and the warp engine corresponding to the second mapping process. The map D used by D may be the reference map data itself. Alternatively, the first mapping process may be one of the forward mapping and the reverse mapping, and the second mapping process may be the other. Alternatively, the first and second mapping processes may have different resolutions. Alternatively, the first mapping process may include the interpolation process, and the second mapping process may not include the interpolation process.

According to this embodiment, the first image is converted into the second and third images by the first and second mapping processes. The first and second mapping processes are mapping processes in the same direction that perform the same deformation on the first image. This makes it possible to compare the second image with the third image. If the degree of coincidence between the second image and the third image is high, it can be determined that there is no error in the second image. In this way, it is possible to detect whether or not the image projected on the projected object such as a head-up display has appropriate display contents. Further, since the second mapping process is a mapping process different from the first mapping process, it is possible to avoid the same abnormality from occurring in the first and second mapping processes, and the error detection accuracy can be improved. Can be improved.

Further, in the present embodiment, the comparison circuit 145 is based on the pixel value of the second image and the pixel value of the third image, or the pixel value of the edge image of the second image and the edge image of the third image. As a result of comparison, an index indicating the degree of matching between the second image and the third image is obtained based on the pixel value of.

In this way, the error detection of the second image can be performed based on the index indicating the degree of agreement between the second image and the third image, instead of the error detection in bit units such as CRC. can. If no error is detected in this process, it can be determined that the mapping process has been performed normally, so that an error in the second image generated by the mapping process can be detected. For example, on an in-vehicle head-up display or the like, an icon or the like to be presented to the user is displayed. According to the present embodiment, such an icon can be presented to the user when the shape can be correctly recognized without stopping the display due to a 1-bit error or the like.

Further, in the present embodiment, the image processing circuit 135 performs the first mapping process using the first map data generated from the map data corresponding to the projected object, and obtains the second map data which is the map data. The second mapping process is performed using the data.

In FIGS. 6 to 8, the map data corresponding to the projected object is the map data stored in the non-volatile memory 210. The first map data is the map data MPB1 and is the data of the map C generated by the map conversion process. The second map data is the map data MPB2, which is the map D data which is the map data read from the non-volatile memory 210.

According to the present embodiment, the reference map data is used as it is as the second map data to generate a third image which is an image for comparison. As a result, it becomes possible to determine whether or not the first mapping process using the first map data was normal based on the comparison result between the third image and the second image.

Further, in the present embodiment, the image processing circuit 135 generates a third image having a resolution lower than that of the first image. The comparison circuit 145 reduces the resolution of the second image to match the resolution of the third image, and compares the third image with the second image after the reduction.

When the configuration of FIG. 5 is applied to the comparison circuit 145 of FIGS. 6 and 8, the pixel arranging processing unit 146 reduces the resolution of the second image to match the resolution of the third image. The index acquisition unit 147 compares the third image with the second image after the resolution is reduced to acquire the index.

According to the present embodiment, by lowering the resolution of the third image, which is an image for comparison, the second mapping process for generating the third image can be made into a low-resolution mapping process. As a result, the processing load of the second mapping process can be reduced, or the circuit scale of the hardware that performs the second mapping process can be reduced. Then, by lowering the resolution of the second image to match the resolution of the third image, it is possible to compare the third image with the second image.

3. 3. Comparison processing, error detection processing Hereinafter, the comparison processing performed by the comparison circuit 145 and the error detection processing performed by the error detection circuit 150 will be described. Here, a case where an index is acquired in the comparison process will be described as an example, and the comparison process will be referred to as an index acquisition process. However, the comparison process is not limited to this, and various image comparisons can be used.

In an image processing system that displays content on a display, it may be necessary to check whether a predetermined area of the image matches the original intention. For example, consider the case of displaying an important image on a cluster display of an automobile system. A cluster display is a meter panel display. At this time, it is necessary to display predetermined important information via a visible image superimposed on the existing content displayed on the screen. The following describes some methods for detecting whether an image is displayed correctly. Detection is performed by analyzing the region of interest and deriving several key indicators of how well the region is displayed. In the present embodiment, the region of interest is the entire image, but a part of the region of the image may be the region of interest. Hereinafter, the region of interest is also referred to as ROI (Region Of Interest).

In the following, the image to be analyzed is referred to as an analysis image, and the image as a reference for analysis is referred to as a reference image. In FIGS. 1 and 4, the image IMA3 is an analysis image, and the image IMA1 is a reference image. In FIGS. 6 and 8, the image IMB2 is an analysis image and the image IMB3 is a reference image. Since the comparison process is a mutual comparison of two images, the analysis image and the reference image may be exchanged.

3.1. The first calculation method of the shape index (first index) The shape index is an index indicating whether or not the shapes of the analysis image and the reference image in the region of interest match. Hereinafter, the calculation method of the shape index will be described.

FIG. 9 is a first example of the analysis image. A1 is an area of interest and A2 is an icon. The dotted line indicating the region of interest is not actually drawn on the display image. Further, although the region of interest is a part of the analysis image here, the region of interest may be the entire analysis image.

First, the ROI pixel blocks of the analysis image are averaged so that the final averaged image has m × n pixels. This subsampling process is performed because a small number of pixel errors are not detected as important errors, these errors are ignored, and the overall shape of the reference image and the analysis image is confirmed. Errors that you want to ignore are, for example, color shifts, small distortions, and so on. The resolution of the subsampled image can be increased to obtain an exact match. The value of m × n can be selected according to the application. When used in connection with a reference image as described below, the m × n value is selected based on sample data observations.

When the region of interest of the analysis image is u × v pixels, the averaged block size is u / m × v / n pixels. When the reference background information is not available, the pixels of the analysis image in the portion where the reference pixels do not exist are deleted. This corresponds to standard foreground masking. This is done because it is necessary to baseline (align, make the same conditions) the background pixels between the reference image and the analysis image. Making a baseline means aligning or making the same conditions. Therefore, the value of the background pixel is set to the same value in both the analysis image and the reference image.

The averaging of the reference image is also performed so that the number of pixels is m × n. The averaging is done separately for each channel. FIG. 10 is an example of a reference image. The foreground F1 which is the icon of the reference image RIA is colored, and the background which is an area other than the icon is colorless such as black. In FIG. 10, the size of the reference image RIA is 256 × 256 pixels. FIG. 11 is an averaged image of the reference image. In FIG. 11, m = n = 16, and the size of the averaged image SRef is 16 × 16 pixels. When the background of the reference image and its averaged image is colorless, the background of the area of interest of the analysis image may also be converted to colorless to obtain an averaged image of the area of interest. For example, the background may be converted to colorless by deleting the background.

Next, the averaged image of the reference image (SRef m × n) and the averaged image of the region of interest (SANz m × n) of the analysis image are compared for each pixel using the distance reference, and the following equation (1) Find the distance D as in. The distance D is a three-dimensional distance. In this embodiment, the distance reference is the square of the Cartesian distance, but similar parameters can be obtained with other distance criteria.

c represents a channel, x represents a horizontal pixel position in the averaged image, and y represents a vertical pixel position in the averaged image. The horizontal direction is also called the horizontal direction, and the vertical direction is also called the vertical direction. m and n are the sizes of the averaged image. R _xyc represents the pixel value at the position (x, y) of the averaged image of the reference image on the channel c. R'c represents the average value of R _xy pixels in channel _c . The average value of the R _xy pixels is the average value of the R xy _c in the averaged image. A _xyc represents the pixel value at the position (x, y) of the averaged image of the analysis image on the channel c. A'c represents the average value of A _xy pixels in channel _c . The average value of A _xy pixels is the average value of A xy _c in the averaged image.

The reason for subtracting the average value in each channel is to prevent a small color shift between the reference image and the analysis image from being treated as an error. If an exact match is required, the mean value can be set to 0. In this case, the match of shape and color is checked by the distance reference.

The shape index S is derived from the distance parameter by the following equations (2) and (3). The shape index S is also called a shape parameter. T is a threshold value, and any value can be adopted. When D <T, T / D = 1, and the shape index S does not change.

The function f is selected so that it can be easily implemented in hardware. For example, the function f may be a scaling function K such that the range 0 to 1 is scaled to 0 to k. In the example described below, the function f is a unit function. That is, S = T / D. The shape index S indicates the degree of coincidence of the shape between the reference image and the analysis image. If the images do not match, this value tends to decrease to zero. An example is described below.

In FIG. 9, the icon of the reference image is correctly displayed on the analysis image. In this case, the shape index is S = 1, and in FIG. 9, Shape: 1.000 is shown.

FIG. 12 is a second example of the analysis image. B1 indicates a region of interest. As shown in B2 of FIG. 12, the icon of the reference image is obscured in the analysis image. That is, some of the reference pixels do not exist in the analysis image, and when the function f is a unit function, the shape index S is less than 1. In the case of such an unclear foreground, the shape index becomes a small value. The visibility index described later also has a small value.

FIG. 13 is a third example of the analysis image. E1 indicates a region of interest. As shown in E2 of FIG. 13, the icon of the reference image is rotated in the analysis image. In this example, since the shape is rotated from the reference, the shape index S is less than 1 when the function f is a unit function. When the foreground is rotated in this way, the shape index becomes a small value. The visibility index, which will be described later, also has a large value. The visibility index will be described later, but by combining the visibility index and the shape index, it is possible to perform appropriate error detection in various foreground states, and the accuracy of error detection can be improved.

The above shape index only checks for base signal matching. In the case of an image with low visibility, the edge detection kernel can be convoluted with the region of interest and the reference image of the analysis image to generate a first-order gradient image, and then the parameters can be obtained by a shape calculation algorithm. The edge detection kernel is, for example, Laplacian or Sobel. With the obtained parameters, the false positives obtained by the shape index can be eliminated. By doing so, it is possible to obtain a correct error detection result even in the case of an image having low visibility.

3.2. Second Calculation Method for Shape Index FIG. 14 is a fourth example of an analysis image. In FIG. 14, the icon ICA is superimposed on the dashboard image DIM. The icon image is blended with the dashboard image with some transparency.

In this embodiment, the edge detection technique is used to detect the edges of the analysis image and the reference image in the region of interest. The edge detection technique is, for example, an edge detection technique using a Sobel edge detection convolution operator.

FIG. 15 is an example of an analysis image in the region of interest. The image CIB is an image of the region of interest in which the dashboard image DIM is blended with the reference image ICB. In the icon part, the dashboard image DIM can be seen through by blending. If there is no error, the reference image in the region of interest is expected to be the same image as in FIG.

FIG. 16 is an example of the edge value calculated from the analysis image. The ECIB is an edge image of the image CIB of FIG. For the sake of illustration, the edges are shown by black lines and gray lines, but in reality, the strength of the edges can be shown by gray scale. White indicates high intensity edges and black indicates no edges. This edge detection is performed on the luminance channel. Similarly, edge detection is performed on a color channel or in a color space such as YCbCr. If there is no error, the edge image of the reference image in the region of interest is expected to be the same image as in FIG.

The edges in the foreground region and the background region are calculated for the reference image and the analysis image, and the shape index is calculated by calculating the similar amount as shown in the following equations (4) to (15). The Match of the following formula (15) is a shape index. The shape index is also called a conforming value. In the following, it is assumed that the reference image has a size of m × n pixels and the region of interest of the display image is also m × n pixels.

Equation (4) below is an operator of the horizontal sobel kernel, that is, the sobel filter that detects the horizontal edge. Equation (5) below is a vertical Sobel kernel, that is, an operator of a Sobel filter that detects vertical edges.

As shown in the following equations (6) to (11), the edge value is calculated for each pixel position in the region of interest of the reference image and the display image. "*" Is a convolution operator. N is a normalization coefficient for keeping the value between 0 and 1, where N = 4. IRef is the luminance (Y) channel of the reference image. IRef _{(x, y)} is a pixel at the position x, y of the luminance channel of the reference image. x is an integer of 0 <x ≦ m, and y is an integer of 0 <y ≦ n. IRen is the luminance channel of the displayed image in the region of interest. IREN _{(x, y)} is a 3 × 3 pixel centered on the positions x and y of the luminance channel of the display image in the region of interest.

As shown in the following equations (12) to (15), the shape index Match (matching value) is obtained from the above edge values. "・" Represents the inner product operator.

When the above calculation is applied to FIGS. 15 and 16, for example, Match = 0.78.

When it is required to calculate the conforming value without analyzing the background, the calculation shown in the following equations (16) to (21) is used.

M _{(x, y)} is 1. Note that M _{(x, y)} may be a mask pixel that defines which pixel is to be compared. This mask can be realized by a simple 1-bit mask in which pixels to be compared are defined by 0 and pixels to be compared are defined by 1. For example, when it is desired to compare an image of only a part of an area of interest, such as comparing only icons, a mask corresponding to the image may be set.

According to the above embodiment, the comparison circuit 145 is based on the pixel value of the analysis image and the pixel value of the reference image, or based on the pixel value of the edge image of the analysis image and the pixel value of the edge image of the reference image. Then, an index showing the degree of matching between the analysis image and the reference image is obtained. The error detection circuit 150 detects an error in the displayed image based on the index.

In this way, it is possible to perform error detection of the analysis image based on an index indicating the degree of agreement between the analysis image and the reference image, instead of error detection in bit units such as CRC. When the analysis image has a high degree of agreement with the reference image, it is highly possible that the analysis image looks visually the same as the reference image. That is, according to this method, it is possible to determine an error when the shape of the analysis image is not displayed correctly. If no error is detected in the analyzed image, it can be determined that the mapping process has been performed normally. Therefore, by detecting the error in the analyzed image, it is possible to detect the error in the image projected on the projected object.

Here, the first calculation method shown in the above equations (2) to (4) corresponds to the case where the index (S) is obtained based on the pixel value of the analysis image and the pixel value of the reference image. Further, the second calculation method shown in the above equations (5) to (21) corresponds to a case where an index (Match) is obtained based on the pixel value of the edge image of the analysis image and the pixel value of the edge image of the reference image. do. The pixel value of the edge image corresponds to the edge amount of the above equations (6), (9), (16), and (17).

The degree of matching is, for example, the degree of matching of the shapes of icons, characters, figures, marks, etc. (hereinafter referred to as icons). More specifically, it is the degree of matching between the contour and the direction of the icon or the like. Further, it may include the degree of coincidence of the state inside the contour such as the icon, for example, whether or not the inside of the contour is filled. For example, the index indicating the degree of matching increases as the degree of matching between the foreground image and the background image increases.

Further, in the present embodiment, as shown in the above equations (2) and (3), the comparison circuit 145 obtains the index (S) from the value obtained by dividing the given threshold value (T) by the distance information (D).

Since the distance (D) becomes smaller as the degree of shape matching increases, an index (S) whose value increases as the degree of shape matching increases can be obtained by dividing a given threshold value by the distance information.

Further, in the present embodiment, the comparison circuit 145 performs a product-sum calculation (the above equation (12)) between the pixel value of the edge image of the analysis image and the pixel value of the edge image of the reference image, and is based on the result of the product-sum calculation. Find the index (Equation (15) above).

The edge image is an image in which the edge amount is defined as the pixel value of each pixel. When the shapes match, when the edge image of the analysis image and the edge image of the reference image are compared with the same pixels, the edge amounts should be the same (including substantially the same). On the contrary, when the shapes do not match, the edge positions do not match between the analysis image and the reference image. Therefore, for example, even if the edge image of the analysis image has a large amount of edges, the same pixels of the edge image of the reference image The amount of edges is zero. Therefore, when the edge amounts of the same pixels are summed, the sum of products results in a large value when the shapes match, and the sum of products results in a small value when the shapes do not match. Become. Therefore, by using the product-sum calculation of the edge amount, the degree of matching of the shapes can be appropriately evaluated.

Here, in the above equation (12), the "product" of the sum of products is the inner product of the vectors, but the "product" is not limited to this. For example, when the edge amount is defined by a scalar, the "product" is the product of the scalars.

3.3. First calculation method for obtaining the visibility index (second index) In the calculation of the visibility index, the terms of the analysis image and the reference image are used as in the calculation of the shape index. In FIGS. 1 and 4, the image IMA3 is an analysis image, and the image IMA1 is a reference image. In FIGS. 6 and 8, the image IMB2 is an analysis image and the image IMB3 is a reference image. Since the comparison process is a mutual comparison of two images, the analysis image and the reference image may be exchanged.

FIG. 17 is a histogram of each channel of YCbCr in the region of interest. Further, FIG. 18 is a cross-correlation value obtained by performing a cross-correlation operation on the histograms of the analysis image and the reference image.

As shown in FIG. 17, for each channel of the YCbCr image, a histogram is obtained using n binaries. For example, 256 binaries can be used to generate a histogram with different sets of binaries.

The histogram counts the number of times a particular value occurs in a region of interest. That is, for each channel of the YCbCr image, the number of pixels having the value indicated by each binary is counted in the region of interest. The histogram is then normalized to a value between 0 and a. The value "a" can be selected in consideration of ease of implementation. For example, a can be set to 1 or 255. In FIG. 17, a = 1. Next, in each channel, the histograms of the analysis image and the reference image are cross-correlated. Then, the cross-correlation signal is used for the subsequent analysis. As shown in FIG. 18, the cross-correlation signal is normalized so that the peak value at zero delay is 1 or a preset value.

The cross-correlation value is obtained by the following equation (22). f and g represent functions to be correlated. One of f and g is a histogram of the analysis image, and the other of f and g is a histogram of the reference image. f * g represents a correlation operation between the function f and the function g. f * represents the complex conjugate of the function f, and in this embodiment f * = f. m represents the binary number of the histogram. n represents a delay (lag), and in FIG. 18, n is an integer from -255 to +255.

In the histogram of FIG. 17, since 256 binaries are normalized between 0 and 1, the horizontal axis is 0 to 1. As for the correlation value in FIG. 18, since the correlation value is obtained while changing the delay for each binary, the horizontal axis is − (256-1) to + (256-1).

As shown in FIG. 18, when a two-color image exists in the region of interest, a sideband is obtained by the correlation calculation. The distance of the delay from the center where the peak occurs indicates the contrast between colors. The center corresponds to zero delay. The contrast allows the human eye to identify the features of the image, so check for peaks on all three channels. The contrast is, for example, luminance contrast or color contrast. In FIG. 18, the Y channel is shown by a dotted line, the Cb channel is shown by a fine solid line, and the Cr channel is shown by a thick solid line. The check is performed by setting a peak search threshold so as not to pick up the noise of the cross-correlation signal. For example, the minimum peak threshold is set to 0.05. Find the peak in the signal to find the local maximum. The peak to be searched is a peak having a peak value larger than the threshold value.

In addition, in order to avoid in-band signal peaks, the minimum distance between consecutive peaks can be set to a predetermined value. These thresholds are adjustable values and are selected according to the application.

After finding all the peaks of the cross-correlation signal that exceeds the noise threshold for all channels, to find a visibility index that indicates whether the identifiable image is shown on a background of two or more colors, then the peaks. The maximum distance at which is generated, that is, the maximum delay is obtained. The maximum value of the delays at which peaks occur in the three channels is selected as an index indicating visibility.

In the correlation plot shown in FIG. 18, the peaks are circled. In the illustrated example, the Cr channel shows maximum separation and the distance is 184. Normalize the above values to the maximum possible delay. For example, the maximum possible delay is 256, which is the number of binaries in the histogram. Therefore, the index value is 184/255 = 0.722. In the image described above in FIG. 9, the index value is shown as a Vis parameter. The above operation shows one example.

In FIG. 9, the inside of the icon A2, which is the portion shown in black, is red, and the background, which is the portion shown in white, is green.

In the image of FIG. 9, since there are two color pixel groups of red and green in the region of interest, in the histogram shown in FIG. 17, two large and small peaks occur in each channel of YCbCr. For example, in the Cr channel, peaks occur in binary Ba and Bb. The distance between these two peaks represents the contrast between the color of the foreground icon, which is the icon, and the background color, which means that the larger the distance, the more different the foreground and background colors. The distance between the two peaks in the histogram is the distance of the delay at which the peaks occur at the cross-correlation values shown in FIG. In the image of FIG. 9, since the foreground icon, which is an icon, is red and the background is green, the distance between the two peaks of the Cr channel is the maximum distance in the histogram shown in FIG. 17, and the distance is | Ba-Bb | It is × 255. This is detected as the maximum distance at which the peak occurs in the cross-correlation value, and the normalized index value is | Ba-Bb |. Therefore, the greater the contrast between the color of the foreground icon, which is the icon, and the color of the background, the larger the index value of visibility.

The error detection circuit 150 performs error detection based on the visibility index obtained as described above. For example, the visibility index is compared with a given threshold value, and if the visibility index is smaller than the given threshold value, an error is determined. Alternatively, the visibility index may be output to the outside of the circuit device 100 as an error detection result.

3.4. Second to Fourth Calculation Method for Finding Visibility Index In the second calculation method, instead of finding the distance of the peak from the center of the cross-correlation signal, does the cross-correlation signal have a peak exceeding a predetermined threshold value? Find out if it isn't. When such a peak is present, the reference image and the analysis image are in good agreement as long as the pixel distribution is taken into consideration. This makes it possible to perform first-level error detection on the analyzed image. This parameter does not show spatial correlation, only pixel distribution correlation. The index in this case may be the peak value itself, not the distance of the peak from the center.

FIG. 19 is an example of a histogram of an analysis image and a reference image. FIG. 20 is an example of the cross-correlation value of the histogram of FIG. Here, one channel of the color image will be described, but the same processing is performed for a plurality of channels. For example, the maximum peak value among the peaks of the cross-correlation values of a plurality of channels may be adopted.

As shown in FIG. 19, the histograms of the analysis image and the reference image have three or more peaks. In the example of FIG. 19, four peaks occur. It is assumed that the peak of the histogram of the analysis image and the peak of the histogram of the reference image are deviated by Bn. In this case, as shown in FIG. 20, a large peak appears at the delay Bn in the cross-correlation value. When the peak value of this peak is larger than the threshold value Thr, for example, the peak value is adopted as an index value of visibility.

In the third calculation method, the contrast ratio between the foreground and the background is obtained as an index value of visibility.

In the first calculation method, the difference | Ba-Bb | of the binary Ba and Bb in which the peak occurs in the histogram of the Cr channel is used as an index value of visibility.

In the third calculation method, the contrast ratio | Ba-Bb | / Ba or | Ba-Bb | / Bb is obtained and used as an index value of visibility. Alternatively, when a reference image such as the second calculation method is used, C1 = | Ba-Bb | in the analysis image and C2 = | Ba-Bb | in the reference image are obtained, and the contrast ratio C1 / C2 or C2 / C1 is obtained and used as an index value of visibility.

In the fourth calculation method, a multidimensional histogram is generated to obtain a visibility index.

In the first calculation method, a one-dimensional histogram of each channel is used for the analysis of visibility.

On the other hand, in the fourth calculation method, a multidimensional histogram is generated from signals of a plurality of channels, and a multidimensional correlation calculation is performed on the multidimensional histogram to obtain a visibility index. The multidimensional cross-correlation operation is a multidimensional cross-correlation operation. This may allow better simulation of contrast detection by the human eye. Better performance may be obtained by using the 3D color histogram.

According to the above embodiment, the comparison circuit 145 statistically obtains an index based on the pixel values of the analysis image and the reference image.

To obtain the index statistically means that a plurality of pixel values included in the analysis image are used as the population of the first statistic, and a plurality of pixel values included in the reference image are used as the population of the second statistic. The index is obtained by processing using the method. Specifically, a histogram is generated from each of the analysis image and the reference image, and an index is obtained based on the histogram.

According to the present embodiment, by statistically obtaining an index, it is possible to obtain an index indicating the degree of dissimilarity between the foreground image and the background image. That is, instead of detecting data defects as in CRC, the degree of dissimilarity between the foreground image and the background image is evaluated by a statistical method, and whether or not an error is determined based on the degree of dissimilarity is determined. Can be decided.

Further, in the present embodiment, the comparison circuit 145 obtains a histogram of the pixel values of the analysis image and the reference image, and performs a correlation calculation using the histogram. In the above example, the pixel value is the pixel value of YCbCr. The comparison circuit 145 uses a visibility index indicating the degree of dissimilarity between the foreground image, which is an image of the region of interest in the analysis image, and the background image corresponding to the background of the foreground image in the analysis image as the result of the correlation calculation. Find based on. The error detection circuit 150 performs error detection based on the index.

When the foreground image has a high degree of dissimilarity to the background image, it is highly likely that the foreground image is visually distinguished from the background image, so that the visibility of the foreground image is considered to be high. That is, according to this method, it is possible to determine an error when the visibility of the foreground image is low. For example, on an in-vehicle instrument panel or the like, an icon or the like for warning the user is displayed. According to the present embodiment, it is possible to warn the user by displaying such an icon as much as possible when the display is not stopped due to a 1-bit error or the like and the visibility is ensured.

Here, the foreground image is an image of a region of the analysis image in which the degree of dissimilarity with the background image is to be determined by an index. Also, that area is a given area. The background image is a part or the whole of the display image excluding the foreground image. That is, the region of interest, which is the region including the foreground image) is set as a part or the whole of the display image, and the image of the region of interest excluding the foreground image is the background image.

The degree of dissimilarity is the degree of dissimilarity in each component of the color space. Components are also called channels. For example, in the YCbCr space, it is a degree indicating how much the brightness of the foreground image and the brightness of the background image are different, or how much the color of the foreground image and the color of the background image are different. Alternatively, in the RGB space, it is a degree indicating how much the color of the foreground image and the color of the background image are different.

Further, in the present embodiment, the comparison circuit 145 obtains a histogram of each component of the constituent components of the color space, performs a cross-correlation calculation on the histogram of each component, and obtains the distance at which the peak of the cross-correlation occurs for each component. The index is calculated based on the maximum distance obtained. The maximum distance is | Ba-Bb | in FIG. 18, and the index here is a visibility index.

In this way, among the constituent components of the color space, the index can be obtained from the component having the largest difference between the foreground image and the background image. Since the component having the largest difference between the foreground image and the background image is considered to have a large difference visually, the degree of dissimilarity between the background and the foreground can be evaluated by obtaining an index from the component. Then, by using this index, the visibility of the foreground can be appropriately evaluated.

Here, the index may be a value obtained based on the maximum distance | Ba-Bb |. For example, in the first calculation method, the index is the maximum distance | Ba-Bb | itself. Further, in the second calculation method, the index is the contrast ratio based on the maximum distance | Ba-Bb |. The contrast ratio is, for example, | Ba-Bb | / Ba or the like.

Further, in the present embodiment, as described with reference to FIGS. 19 and 20, the comparison circuit 145 obtains the first histogram of each component of the color space component as a histogram from the analysis image, and obtains the first histogram of each component from the reference image. Find the histogram of 2. The comparison circuit 145 performs a cross-correlation calculation between the first histogram and the second histogram for each component, and obtains an index based on the peak value of the peak of the cross-correlation.

In this way, even when the analysis image and the reference image are multitone including two or more colors, an index indicating the degree of dissimilarity between the foreground image and the background image can be obtained. That is, two or more peaks occur in the histogram of the reference image, but when the same pattern as this histogram is included in the histogram of the analysis image, an image similar to the reference image in at least the color or brightness pattern is the analysis image. Will be included in. In this case, a large peak should occur in the result of the cross-correlation calculation, so the visibility of the foreground can be appropriately evaluated by obtaining an index from the peak value.

Further, in the present embodiment, the comparison circuit 145 obtains a shape index which is a first index and a visibility index which is a second index. The error detection circuit 150 performs error detection based on the first index and the second index.

In this way, error detection of the projected image can be performed by combining two indexes evaluated for different properties. That is, by combining a visibility index showing the degree of brightness and color dissimilarity between the analysis image and the reference image and a shape index showing the degree of shape matching between the analysis image and the reference image, the projected image Error detection can be performed with higher accuracy.

4. Electronic device FIG. 21 is a configuration example of an electronic device including the circuit device of this embodiment. The electronic device 300 includes a processing device 310, a circuit device 320, a projection device 380, a storage device 350, an operation device 360, and a communication device 370. The processing device 310 is, for example, an MCU or the like. The circuit device 320 corresponds to the circuit device 100 of FIGS. 1, 4, 6, and 8, and is, for example, a head-up display controller or a display controller.

The projection device 380 includes a display driver 330, a display panel 340, a light source 335, and a lens 345. The display driver 330 drives the display panel 340 to display an image. The light source 335 outputs light for projection to the display panel 340, and the light that has passed through the display panel 340 or is reflected from the display panel 340 is incident on the lens 345. The lens 345 forms an image on the projected object.

The processing device 310 transfers the image data stored in the storage device 350 or the image data received by the communication device 370 to the circuit device 320. The circuit device 320 performs image processing on the image data, display timing control, error detection processing of the image data to be transferred to the display driver, and the like. In the error detection process, the visibility index and the shape index are calculated, and the error detection based on these indexes is performed. The display driver 330 drives the display panel 340 to display an image based on the image data transferred from the circuit device 320 and the display timing control by the circuit device 320. The display panel 340 is, for example, a liquid crystal display panel. The storage device 350 is, for example, a memory, a hard disk drive, an optical disk drive, or the like. The operation device 360 is a device for a user to operate the electronic device 300, and is, for example, a button, a touch panel, a keyboard, or the like. The communication device 370 is, for example, a device for performing wired communication (or a device for performing wireless communication. The wired communication is, for example, LAN, USB, etc.) The wireless communication is, for example, a wireless LAN, wireless proximity communication, or the like. be.

In FIG. 21, the case where the electronic device is a head-up display has been described as an example, but the electronic device including the circuit device of the present embodiment is not limited to this. As the electronic device including the circuit device of the present embodiment, various devices such as a head-mounted display and a projector that project an image onto a projected object can be assumed. The configuration of the electronic device is not limited to FIG. 21, and various configurations can be adopted depending on the application. For example, in an in-vehicle electronic device, a circuit device 320, a projection device 380, and an operation device 360 are incorporated in an instrument panel, and a processing device 310, a storage device 350, and a communication device 370 are incorporated in an ECU (Electronic Control Unit). In this case, the instrument panel corresponds to an electronic device including the circuit device of the present embodiment.

Although the present embodiment has been described in detail as described above, those skilled in the art will easily understand that many modifications that do not substantially deviate from the novel matters and effects of the present invention are possible. Therefore, all such modifications are included in the scope of the present invention. For example, a term described at least once in a specification or drawing with a different term in a broader or synonymous manner may be replaced by that different term anywhere in the specification or drawing. All combinations of the present embodiment and modifications are also included in the scope of the present invention. Further, the configuration and operation of the circuit device and the electronic device are not limited to those described in the present embodiment, and various modifications can be performed.

100 ... circuit device, 110 ... interface, 125 ... preprocessing circuit, 135 ... image processing circuit, 140 ... interface, 145 ... comparison circuit, 146 ... pixel arrangement processing unit, 147 ... index acquisition unit, 150 ... error detection circuit, 170 ... Register circuit, 175 ... Comparison result register, 176 ... Error detection result register, 177 ... Operation mode setting register, 178 ... Threshold register, 190 ... Interface, 200 ... Processing device, 210 ... Non-volatile memory, 300 ... Electronic equipment, 310 ... Processing device, 320 ... Circuit device, 330 ... Display driver, 335 ... Light source, 340 ... Display panel, 345 ... Lens, 350 ... Storage device, 360 ... Operating device, 370 ... Communication device, 380 ... Projection device, IMA1 to IMA3 ... image, IMB1 to IMB3 ... image, MPA1, MPA2 ... map data, MPB1, MPB2 ... map data, WEA1, WEA2 ... mapping processing unit, WEB1, WEB2 ... mapping processing unit

Claims (13)

The second image is seconded by the first mapping process of mapping the input first image to the second image for projecting onto the projected object and the inverse mapping process of the first mapping process. An image processing circuit that performs a second mapping process that converts to the image of 3 and
A comparison circuit that makes a comparison between the first image and the third image and outputs the result of the comparison as information for performing error detection of the second image.
A circuit device characterized by including. In claim 1,
The comparison circuit
Based on the pixel value of the first image and the pixel value of the third image, or based on the pixel value of the edge image of the first image and the pixel value of the edge image of the third image. , A circuit device characterized in that an index indicating a degree of agreement between the first image and the third image is obtained as a result of the comparison. In claim 1 or 2,
The image processing circuit
The first mapping process is performed using the first map data generated from the map data corresponding to the projected object, and the second mapping process is performed using the second map data which is the map data. A circuit device characterized by doing. Based on the map data, the first mapping process of mapping the input first image to the second image for projecting on the projected object based on the map data corresponding to the projected object, and the map data. An image processing circuit that performs a second mapping process for converting the first image into a third image by a second mapping process different from the first mapping process.
A comparison circuit that makes a comparison between the second image and the third image and outputs the result of the comparison as information for performing error detection of the second image.
A circuit device characterized by including. In claim 4,
The comparison circuit
Based on the pixel value of the second image and the pixel value of the third image, or based on the pixel value of the edge image of the second image and the pixel value of the edge image of the third image. , A circuit device characterized in that an index indicating a degree of agreement between the second image and the third image is obtained as a result of the comparison. In claim 4 or 5,
The image processing circuit
The first mapping process is performed using the first map data generated from the map data, and the second mapping process is performed using the second map data which is the map data. Circuit equipment. In any of claims 4 to 6,
The image processing circuit
Generate the third image with a lower resolution than the first image.
The comparison circuit
The second image is reduced in resolution to match the resolution of the third image, and the third image is compared with the second image after the reduction in resolution. Circuit equipment. In any of claims 1 to 7,
A circuit device including an error detection circuit that detects an error in the second image based on the result of the comparison. In claim 8.
A circuit device including an operation mode setting register in which an operation mode of the circuit device is set when an error is determined by the error detection circuit. In claim 9.
The operation mode setting register has
A mode for notifying the external device of the circuit device of the result of the error detection, a mode for hiding the second image, or a mode for displaying a specific image is set as the operation mode. Circuit equipment to do. In any of claims 8 to 10,
The error detection circuit
A circuit device characterized in that the error detection is performed by comparing the result of the comparison with a threshold value for determining an error in the second image. 11.
A circuit device including a threshold register in which the threshold is set. An electronic device comprising the circuit device according to any one of claims 1 to 12.

Images