WO2018197984A1

WO2018197984A1 - Display system and mobile body

Info

Publication number: WO2018197984A1
Application number: PCT/IB2018/052611
Authority: WO
Inventors: 山崎舜平; 宮口厚
Original assignee: 株式会社半導体エネルギー研究所
Priority date: 2017-04-28
Filing date: 2018-04-16
Publication date: 2018-11-01
Also published as: JP2023036577A; JPWO2018197984A1; JP2024116154A; JP7707366B2

Abstract

Provided are: a display system that has a novel configuration; and a mobile body. A display system that has an imaging device, a display device, a feature value output circuit, an image processing circuit, and a server. The imaging device can acquire imaging data. The feature value output circuit can acquire feature value data for the imaging data. A database includes correction data and detection data and can output correction data to the image processing circuit in accordance with the feature value data. The database selects the correction data in accordance with machine learning or with coincidence with or similarity to the feature value data. The image processing circuit can generate image data by correcting the imaging data on the basis of the correction data. The display device can produce a display in accordance with the image data.

Description

Display system and moving body

One embodiment of the present invention relates to a display system and a moving object.

Vehicles equipped with an imaging device for imaging information around a vehicle and a display device for displaying information obtained by imaging are widely used (for example, Patent Document 1).

JP 2017-5678 A

In a situation where the periphery of the vehicle is imaged during night driving, a light source with high illuminance (dazzling light) such as a vehicle headlight (headlight) approaches. Therefore, there is a problem that the obtained image data is excessively exposed around the light source, and the display of the area corresponding to the periphery of the light source becomes white. On the other hand, by suppressing the exposure, although the exposure around the light source can be adjusted, the displayed image becomes dark, and it becomes difficult to grasp the size and shape of the entire vehicle. That is, the information obtained is limited.

An object of one embodiment of the present invention is to provide a novel display system, a moving object, and the like.

Alternatively, one embodiment of the present invention provides a novel display system, a moving body, and the like that can perform normal display even if a defect (unclear) portion occurs in acquired captured data due to an imaging environment. This is one of the issues. Another object of one embodiment of the present invention is to provide a novel display system, a moving object, and the like that can improve visibility.

In the display system of one embodiment of the present invention, in the imaging data obtained by the imaging device, the feature amount is extracted from part of the information of the subject that is clearly imaged, and the corresponding correction data is selected by collating in the database. Thus, the gist is to correct the unclear area of the imaging data based on the correction data.

One embodiment of the present invention includes an imaging device, a display device, a feature amount output circuit, an image processing circuit, and a database. The imaging device has a function of outputting imaging data. The feature amount output circuit includes imaging data. And the database has correction data and detection data, and has a function of outputting correction data to the image processing circuit in accordance with the feature data. The image processing circuit has a function of generating image data by correcting imaging data based on correction data, and the display device is a display system having a function of performing display according to image data.

In one embodiment of the present invention, the database is preferably a display system having a function of selecting detection data that matches or resembles feature data and outputting correction data corresponding to the detection data to the image processing circuit.

In one embodiment of the present invention, the database can update detection data serving as a weight parameter by machine learning using feature data as learning data, and infer correction data that matches or is similar to the feature data. A display system with functionality is preferred.

One embodiment of the present invention includes an imaging device, a display device, a feature amount output circuit, an image processing circuit, and a transmission / reception circuit, the imaging device has a function of acquiring imaging data, and the feature amount output circuit includes: The image processing circuit has a function of acquiring feature amount data of imaging data, and the feature amount data has a function of being transmitted to a database having correction data and detection data via a transmission / reception circuit. Has a function of receiving correction data from a database via a transmission / reception circuit and generating image data by correcting imaging data based on the correction data. It is a moving body having a function of performing display.

In one embodiment of the present invention, it is preferable that the correction data is data corresponding to the detection data, and the detection data is data that matches or is similar to the feature data.

In one embodiment of the present invention, the correction data is obtained by inferring by inputting feature amount data in a database in which detection data serving as a weight parameter is updated by machine learning using the feature amount data as learning data. A mobile that is data is preferred.

Other aspects of the present invention are described in “DETAILED DESCRIPTION OF THE INVENTION” and “Drawings” described below.

One embodiment of the present invention can provide a novel display system, a moving object including the display system, and the like.

Alternatively, one embodiment of the present invention provides a novel display system, a moving body, and the like that can perform normal display even if a defect (unclear) portion occurs in acquired captured data due to an imaging environment. be able to. Alternatively, according to one embodiment of the present invention, a novel display system, a moving object, and the like that can improve visibility can be provided.

The block diagram for demonstrating a display system. The block diagram for demonstrating a display system. The conceptual diagram for demonstrating a display system. The block diagram for demonstrating a display system. The block diagram for demonstrating a display system. The flowchart for demonstrating a display system. The flowchart for demonstrating a display system. The block diagram for demonstrating a display system. The figure for demonstrating a display system. The figure for demonstrating a display system. The figure for demonstrating the specific example of operation | movement of a display system. The figure for demonstrating the specific example of operation | movement of a display system. The figure for demonstrating the specific example of operation | movement of a display system. The figure for demonstrating the specific example of operation | movement of a display system. The figure for demonstrating the specific example of operation | movement of a display system. FIG. 9 illustrates a configuration example of a semiconductor device. The figure which shows the structural example of a memory cell. The figure which shows the structural example of an offset circuit. Timing chart. The figure which shows the example of application of a product difference calculating circuit. FIG. 14 is a top view illustrating one embodiment of a display device. FIG. 14 is a cross-sectional view illustrating one embodiment of a display device. FIG. 14 is a cross-sectional view illustrating one embodiment of a display device. 4A and 4B illustrate a display device in a moving object. The figure explaining an example of a moving body.

Hereinafter, one embodiment of the present invention will be described with reference to the drawings. However, one embodiment of the present invention can be implemented in many different modes, and it is easily understood by those skilled in the art that the modes and details can be variously changed without departing from the spirit and scope thereof. Is done. Accordingly, the present invention should not be construed as being limited to the following description.

(Embodiment 1)
<Configuration of display system>
A display system of one embodiment of the present invention is described.

FIG. 1 is a block diagram for explaining an example of a display system which is one embodiment of the present invention. The display system 10 illustrated in FIG. 1 includes an imaging device 11, a feature amount output circuit 12, a database 13, an image processing circuit 14, and a display device 15.

The imaging device 11 is specifically a camera module attached to a moving body such as an automobile. The imaging device 11 has a function of outputting the imaging data 16. The imaging data 16 is output to the feature amount output circuit 12 and the image processing circuit 14.

The imaging device 11 preferably has an imaging element with a wide dynamic range. For example, by using the imaging device 11 including an imaging element having selenium, it is possible to reduce unclear portions of imaging data when imaging a subject having a large contrast.

The feature amount output circuit 12 extracts the feature amount in the imaging data 16 and outputs it as feature amount data. It is effective to extract the feature amount in combination with imaging data obtained by the imaging device 11 and data that can be acquired from a sensor or the like used to detect the relative speed or relative position of the subject.

The feature amount includes an appearance feature amount (appearance feature amount). For example, the appearance feature amount in an automobile includes, for example, the body color of the own vehicle, the number of the license plate, the vehicle type, the vehicle width, the vehicle height, the blinker lamp, and the headlight. The feature amount data of these feature amounts is output to the database 13 as encoded data or data cut out from the original imaging data.

¡By default, the feature amount is set so that it can be easily detected by image recognition, thereby reducing the load of image recognition processing. It is also effective to exclude the blinker lamp and the brake lamp from being subject to image recognition because lighting or blinking changes depending on the situation.

The unclear area of the imaging data 16 is an area where it is difficult to extract a feature amount because the contrast ratio between light and dark is small. Therefore, the feature quantity output circuit 12 can reduce the data of the unclear area of the imaged data by extracting the feature quantity.

Further, the feature quantity output circuit 12 extracts, for example, a Haar-like feature, a HOG (Histograms of Oriented Gradients) feature, a SIFT (Scaled Invariance Feature Transform) feature, or a SURF (Speeded Up Robust feature). It is also effective to extract and output feature amount data at a desired location by combining the above. It is also effective to smooth the imaged data 16 with a Gaussian filter and select characteristic area information.

It is also effective to perform object detection that combines local gradient feature extraction technology and SVM (Support Vector Machine) learning, and extract and output feature data at a desired location. The feature quantity output circuit 12 may output, as feature quantity data, a feature quantity that is automatically calculated by CNN (Convolutional Neural Network) learning, for example.

The feature amount output circuit 12 is mainly configured as a microcomputer, and includes a processor, a memory, an I / O, and a bus connecting them. In the feature amount output circuit 12, various functions can be realized by executing a program stored in advance in a memory by a processor based on information of various sensors in addition to the imaging device 11.

The database 13 stores feature data 17, correction data 18, and detection data 19. The correction data 18 is data for correcting the imaging data 16. The detection data 19 is data for selecting the correction data 18 based on the feature data.

The correction data 18 is data for correcting the imaging data 16 in the image processing circuit 14. The correction data 18 is preferably data that can be displayed with a higher resolution than the imaging data 16. With this configuration, the resolution of an image obtained by correcting the imaging data 16 can be increased, and the visibility of display on the display device 15 can be increased.

Note that the image data obtained by the correction of the imaging data using the correction data 18 may be different from the image actually obtained by visual observation. For example, there may be image data in which the vehicle type, the shape of the vehicle body, etc. are partially different. Alternatively, correction data may be selected from subjects having the same feature data. Even in such a case, since the entire information of the subject can be removed as compared with the image data without correction, the situation can be grasped more efficiently by the projected image.

The detection data 19 is data that can select the correction data 18 based on the feature value data 17 input from the feature value output circuit 12. For example, the detection data can be feature amount data such as an appearance feature amount extracted based on the correction data 18. With this configuration, it is possible to select the desired correction data 18 by selecting the detection data 19 that matches or is similar to the feature data 17.

Alternatively, the detection data 19 may be data serving as a weight parameter in CNN learning. In this case, the CNN learning in the database 13 is a detection parameter which is a weighting parameter using, as learning data, data obtained by adding a correct label corresponding to the correction data 18 to the feature data 17 input from the feature data output circuit 12 collected in advance. The data 19 is updated. The detection data 19 is updated by repeating the CNN learning. By repeating the update, the accuracy can be improved. By using the CNN using the updated detection data 19 as a weight parameter, the correction data 18 corresponding to the input of the feature data 17 can be selected with high accuracy.

The database 13 has a function of storing the correction data 18 and the detection data 19, a memory for storing a program for selecting the correction data 18 corresponding to the feature data 17, and a processor for executing the program It has a function as a computer equipped with. In the database 13, various functions can be realized by executing a program stored in advance in a computer.

The image processing circuit 14 is a circuit having a function of correcting data in a blurred region of the imaging data 16 based on the correction data 18 and outputting the corrected imaging data as image data. Note that the unclear area of the imaging data 16 corresponds to an area where it is difficult to extract a feature amount, such as an area where the contrast ratio between light and dark is small. For this reason, a configuration in which a region where feature amount extraction is difficult can be corrected based on the correction data 18 can be given as an example. The correction data 18 is particularly effective when correcting the entire contour of the subject in the imaging data 16.

It is preferable that the image processing circuit 14 holds the correction data once acquired. With this configuration, once the correction data 18 is held, it can be repeatedly used for correction of imaging data.

Note that the correction data 18 preferably has a higher resolution than the imaging data 16. When the resolution of the imaging data obtained by the imaging device 11 is smaller than the resolution of the display, the correction data 18 can be used for the super-resolution processing of the imaging data, and the corrected and displayed image becomes clearer. be able to.

The display device 15 performs display based on the image data generated by the image processing circuit. The display device 15 can be used in place of a room mirror in the case of an automobile. As the display device 15, in addition to a liquid crystal display device, a self-luminous display device having an electroluminescence element can be used.

The display system 10 includes an imaging device 11 and a display device 15. The display system 10 extracts feature data from a part of information of the subject in the imaging data 16 obtained by the imaging device 11 and collates it in the database 13 to select correction data 18. Originally, the unclear area of the imaging data 16 is corrected.

The display system 10 is preferably applied to a moving body such as an automobile. Specifically, it is particularly suitable in a situation where the periphery of the automobile is imaged by an imaging device such as a camera and the state is visually recognized by a display device. The moving body is a vehicle that moves like an automobile. Accordingly, the moving body is not limited to a car, but includes a bus, a train, an airplane, and the like.

As described above, in an automobile equipped with an imaging device and a display device, an unclear area may occur in imaging data obtained by the imaging device according to changes in conditions such as day and night, and fine rain. For example, a headlight of a rear automobile or water droplets adhering to the imaging device causes a blurred area. When there is an unclear area in the imaging data obtained in this way, it is difficult to grasp the size and shape of the entire subject, and the obtained information is limited. As described above, in the display system 10 according to one aspect of the present invention, the feature data is extracted from the information on a part of the subject and collated in the database 13 to select the correction data 18 so that the imaging data 16 is unclear. Correct the area. Therefore, it is effective for improving visibility.

The application of the display system 10 of the present invention is not limited to a mobile object. This is effective when image data captured by the imaging device is displayed on the display device. For example, it is effective when correcting unclear portions of imaging data obtained by an imaging device such as a security camera or a smartphone.

In addition, what is necessary is just to make the database 13 like the block diagram of the display system shown in FIG. A display system 10A shown in FIG. 2 has a configuration in which a transmission / reception circuit 20 and a network 21 are added to the configuration of the display system 10 of FIG. That is, data output from the feature amount output circuit 12 to the database 13 and data output from the database to the image processing circuit 14 are performed via the transmission / reception circuit 20 and the network 21.

The configuration of the display system 10A in FIG. 2 can be described with reference to the conceptual diagram illustrated in FIG. That is, the display system 10 A is configured to transmit / receive data to / from the database 13 via the relay station 24 from the automobile 22 A to which the transmission / reception circuit 20 is attached.

Note that the configuration of the display system 10 in FIG. 1 can be described with reference to the conceptual diagram illustrated in FIG. That is, the imaging device 11, the feature amount output circuit 12, the database 13, the image processing circuit 14, and the display device 15 that constitute the display system 10 are configured to be included in the automobile 22. The feature amount output circuit 12, the database 13, and the image processing circuit 14 are illustrated as a data processing circuit 23.

Further, in the configuration of the display system 10 in FIG. 1, a plurality of configurations may be provided. For example, FIG. 4 illustrates a block diagram of a display system 10B including a plurality of imaging devices 11_1-11_n (n is a natural number) and a display device 15_1-15_n. In the configuration of FIG. 4, there are a plurality of pieces of data output from the feature amount output circuit 12, the database 13, and the image processing circuit 14 in addition to the plurality of pieces of imaging data 16 output from the imaging devices 11_1-11_n. Note that the display device 15 can include a large display portion so that image data obtained by imaging with the plurality of imaging devices 11_1-11_n can be displayed side by side.

Further, in the configuration of the display system 10 in FIG. For example, FIG. 5 illustrates a block diagram of a display system 10C in which a sensor 25 is added to the configuration of the display system 10 of FIG.

In the configuration of the display system 10C in FIG. 5, the sensor 25 can control whether or not to stop the function of the feature value output circuit 12 according to the output. For example, when the sensor 25 is an illuminance sensor, the output of feature amount data from the feature amount output circuit 12 is stopped when the illuminance is high, and the output of feature amount data from the feature amount output circuit 12 is started when the illuminance is low. can do.

The switching of the output of the feature amount data from the feature amount output circuit 12 may be performed by turning on and off the switching function by the user. In this case, the configuration may be such that the function can be switched between start and stop in accordance with input from a touch sensor or the like.

The sensor 25 may be configured to be used in combination with other sensors such as a millimeter wave radar that can be used to detect the relative speed and relative position of the subject.

In the display system of one embodiment of the present invention described above, even if a defect (unclear) portion occurs in the acquired imaging data due to the imaging environment, the imaging data is corrected based on the correction data, and normal Image data for display can be generated. Therefore, the visibility of the image displayed on the display device can be improved.

<Operation of display system>
FIG. 6 is a flowchart for explaining an operation example of the display system 10 described in FIG.

In the operation of the display system described with reference to FIG. 6, feature amount data is input to the database 13 by extracting the feature amount of the imaging data 16, and detection data 19 that matches or resembles the feature amount data 17 is compared. The correction data 18 is selected. Then, the selected correction data 18 is used by the image processing circuit 14 to correct the imaging data 16.

First, step S 11 is a step of acquiring the imaging data 16 by the imaging device 11. The imaging data 16 is output to the feature amount output circuit 12. The same imaging data 16 is also output to the image processing circuit 14.

Step S12 is a step of determining whether or not the input imaging data 16 has a display abnormality. Step S12 may be performed together with the feature amount extraction in step S13. When there is a display abnormality, extraction of feature quantities such as appearance feature quantities is partially difficult. For this reason, it is effective to perform display abnormality determination based on changes in the feature amount data. Or the structure which determines display abnormality according to the output of sensors, such as an illumination intensity sensor, may be sufficient. The user may determine the display abnormality by looking at the display based on the imaging data 16 and perform switching using a sensor or the like. If it is determined that there is no display abnormality (No), the process proceeds to the step of displaying on the display device 15 in step S17. If it is determined that there is a display abnormality (Yes), the process proceeds to step S13.

Step S13 is a step in which the feature quantity output circuit 12 performs feature quantity extraction. The obtained feature quantity is output to the database 13 as feature quantity data 17. The feature amount data 17 is data obtained by combining the appearance feature amount based on the image recognition process and the feature amount obtained by the local gradient feature extraction technique of the characteristic region.

Step S14 is a step of selecting (searching) the detection data 19 stored in the database 13 corresponding to the feature data obtained in Step S13. As described above, the detection data 19 is data for selecting the correction data 18 based on the feature amount data 17.

Step S15 is a step for causing the image processing circuit 14 to output the correction data 18 corresponding to the detection data 19 that matches or is similar to (or hits) the feature data 17 by the search in step S14. As described above, the correction data 18 is data for correcting a missing (unclear) region of the imaging data 16.

Step S16 is a step of correcting the imaging data 16 to generate image data. The image data obtained on the basis of the corrected imaging data 16 is data in which excessive exposure around the light source and information on the size and shape of the entire subject are corrected. That is, the information obtained can be complemented by the correction data 18.

Step S17 is a step of performing display based on the image data on the display device 15.

In the series of operations of the display system described with reference to FIG. 6, the correction data 18 can be selected by comparing the detection data 19 that matches or is similar to the feature data 17. The selected correction data 18 is used to correct the imaging data 16 by the image processing circuit 14, thereby generating image data for normal display when a missing (unclear) portion occurs in the acquired imaging data. can do. Therefore, the visibility of the image displayed on the display device can be improved.

In the operation of the display system, the correction data 18 can be selected from the feature amount data by extracting the feature amount of the imaging data 16 into the database 13. The database 13 is trained so as to infer the correction data 18 corresponding to the feature data using the detection data 19 as a weight parameter in CNN learning. Then, the selected correction data 18 can be used for correction of the imaging data 16 by the image processing circuit 14.

FIG. 7 shows a flowchart for explaining an example of learning of the display system 10 when the detection data 19 is used as a weight parameter in CNN learning.

Step S21 performs the same feature amount extraction as step S13. That is, in step S12, the feature amount of the imaging data 16 having display abnormality is extracted. The feature amount data obtained in step S21 is training data and test data. The original imaging data is preferably different data, and it is preferable to generate and prepare a large amount of feature amount data.

Step S22 is a step of assigning a correct answer label to the feature data. The correct answer label corresponds to correction data 18 that is optimal for correcting the imaging data 16 corresponding to the feature data.

Step S23 is a step in which feature data for training is given and the detection data 19 serving as a weight parameter is updated. It is effective to update the weight parameter using a known learning method such as an error back propagation method.

Step S24 is a step of inputting feature amount data. The feature data input in step S24 is test data with a correct answer label.

In step S25, it is determined whether the output signal obtained in step S24 corresponds to the correct label attached earlier (Yes or No). If not, the parameter update in step S23 is repeated.

Step S26 is a determination (Yes or No) as to whether or not the accuracy of whether the correction data 18 that matches or is similar to the feature data 17 is obtained is equal to or less than the set value. If the accuracy is less than or equal to the set value, the parameter update in step S23 is repeated.

Detecting data 19 is updated by repeating CNN learning. By repeating the update, the accuracy of the correction data 18 corresponding to the input of the feature data 17 can be improved. By using the CNN using the updated detection data 19 as a weight parameter, the correction data 18 corresponding to the input of the unknown feature data 17 can be selected with high accuracy.

FIG. 8 is a block diagram for explaining a configuration example of the CNN (convolutional neural network) explained in FIG.

The convolutional neural network 30 shown in FIG. 8 includes an input layer 31, an intermediate layer 32, and an output layer 33.

FIG. 8 illustrates the feature data 17 that is input data in the input layer 31.

Further, in FIG. 8, in the intermediate layer 32, the filter 34, the convolution data 35, the pooling data 36, the filter 37, the convolution data 38, and the pooling data 39 are illustrated. Note that the configuration of the intermediate layer 32 is an example, and a configuration in which pooling processing and convolution calculation processing by the

filters

34 and 37 are performed in multiple layers, or a configuration in which calculation processing such as padding and stride is performed may be employed.

Further, in FIG. 8, in the output layer 33, all combined data 40 and output data 41 are illustrated. The output data 41 is obtained by processing the obtained all combined data 40 (y1-ym in the figure) with a softmax function to obtain an output corresponding to the correct answer label.

In the CNN learning in the flowchart described in FIG. 7, the parameters (weight data) of the

filters

34 and 37 are updated, and output data 41 (O (y1) −O (ym) in the figure) with labels is obtained. So that they can learn and infer. By inputting the feature data 17 to the convolutional neural network 30, the corresponding correction data 18 can be output to the image processing circuit 14.

A specific example of the display system 10 described above and its operation will be described. 9 and 10, an example in which the display system 10 is applied to an automobile will be described.

FIG. 9A is a schematic view seen from the rear of the automobile 22. In FIG. 9A, a camera that is the imaging device 11 and a room mirror that is the display device 15 are illustrated.

FIG. 9B shows a car 22 and another car 42 behind it. The automobile 42 illustrates a state in which a headlight as a light source 43 is emitted toward the automobile 22 in the irradiation direction 44. FIG. 9B shows a state in which the imaging device 11 of the automobile 22 is imaged toward the automobile 42 in the imaging direction 45.

In the example of FIG. 9B, the automobile 42 is illustrated as the subject to be imaged by the imaging device 11, but the subject is not limited to the automobile. For example, the subject may be any object that can select correction data by extracting feature values from imaged data such as pedestrians, bicycles, street lamps, and signs. If it is a pedestrian, it is also effective to extract a feature amount from features of parts such as feet and hands, and to correct a part of the head or upper body with correction data. Visibility can be enhanced by displaying image data of the entire pedestrian on the display device.

In the case of FIG. 9B, the imaging data 16 obtained by the imaging device 11 can be expressed as shown in FIG. That is, as shown in FIG. 9C, the illuminance of the light source 43 increases. Although the shape of the area around the light source 43 is recognized by suppressing exposure, it is difficult to recognize the shape of the entire automobile 42. For this reason, as shown in FIG. 9D, when an automobile having a light source with high illuminance is captured during night driving or the like, a clear area 48 and an unclear area 49 are mixed in the imaging data.

As another example, FIG. 10A is a schematic view seen from the rear of the automobile 22A. FIG. 10A illustrates a camera that is the imaging device 11 A and a room mirror that is the display device 15. FIG. 10A illustrates an enlarged schematic diagram of the imaging device 11A, and illustrates a lens 46 and a water droplet 47. In the case of rain or the like, water droplets 47 or the like may adhere to the surface of the lens 46, resulting in unclear image data. 10A shows a configuration in which the display device 15 is arranged at the position of the room mirror, but it is also effective for a configuration in which a display device at the position of the side mirror or the dashboard is arranged.

FIG. 10B shows a car 22A and another car 42 behind the car 22A. FIG. 10B illustrates a state where the imaging device 11 A of the automobile 22 A is imaged toward the automobile 42 in the imaging direction 45.

In the case of FIG. 10B, the imaging data 16 obtained by the imaging device 11 can be represented as shown in FIG. That is, as shown in FIG. 10C, when the water droplet 47 is imaged, the image of the automobile 42 becomes unclear, and it becomes difficult to recognize the shape of the entire automobile 42. Therefore, as illustrated in FIG. 10D, when an automobile is imaged during rainy weather driving or the like, a clear area 48 and an unclear area 49 are mixed in the image data.

In the display system 10 according to one aspect of the present invention described above, a feature amount is extracted from part of the information of the automobile 42 in the clear region 48 of the imaging data 16, and the feature amount data 17 is collated in the database 13 for correction. The data 18 can be selected and the unclear area of the imaging data 16 can be corrected.

Therefore, in a car equipped with the display system 10, a corrected image can be visually recognized even if an unclear area is generated in the imaging data obtained by the imaging device in accordance with changes in conditions such as day and night and fine rain. For example, even if a headlight of a vehicle behind the vehicle or a water droplet attached to the imaging device causes a blurred region, correction data corresponding to the feature amount data can be selected and the imaging data can be corrected. Therefore, it becomes easy to grasp the size and shape of the entire subject. Moreover, since the information obtained can be increased, it is effective in improving visibility.

<Specific example for explaining operation of display system>
FIG. 11 is a flowchart for explaining a more specific example of the operation of the display system. FIG. 11 illustrates the data flow in the imaging device 11, the feature amount output circuit 12, the database 13, the image processing circuit 14, and the display device 15 in the display system 10 described in FIG.

In step S31 shown in FIG. 11, the imaging data is acquired by the imaging device 11 and the imaging data is output to the feature amount output circuit 12. In step S31 shown in FIG. 11, correction data and detection data are stored in the database 13. The correction data 18 and the detection data 19 in the database 13 may be stored in advance.

Step S31 shown in FIG. 11 can be illustrated as in the block diagram shown in FIG. In FIG. 12, the imaging data 16 illustrated in FIG. 9C is illustrated for easy understanding. In FIG. 12, a plurality of automobile images 18 A are shown as the correction data 18 for easy understanding. In FIG. 12, for easy understanding, a headlight image 19 A, which is a part of an image of an automobile, is illustrated as detection data.

In step S32 shown in FIG. 11, the feature amount data is calculated by the feature amount output circuit 12, and the feature amount data is output from the feature amount output circuit 12 to the database 13.

Step S32 shown in FIG. 11 can be illustrated as in the block diagram shown in FIG. In FIG. 13, in order to facilitate understanding, a headlight image 17 A is illustrated as a part of an automobile image that is an appearance feature amount as the feature amount data 17.

In step S33 shown in FIG. 11, the feature amount data 17 is input from the feature amount output circuit 12 to the database 13, and the detection is performed by searching for the detection data 19 that matches or resembles the feature amount data using the headlight image 17A. The data 19 is selected and the correction data 18 corresponding to the detection data 19 is output to the image processing circuit 14.

Step S33 shown in FIG. 11 can be illustrated as in the block diagram shown in FIG. In FIG. 14, for the sake of easy understanding, it is assumed that the feature data of the headlight image 17A and the feature data of the headlight image 19A coincide with each other with correction data 18 output to the image processing circuit 14. An image 18A is illustrated.

In step S34 shown in FIG. 11, the imaging data 16 from the imaging device 11 as correction source data and the correction data 18 from the database 13 as correction data are input to the image processing circuit 14, and imaging is performed. Data 16 is corrected. The corrected imaging data is output from the image processing circuit 14 to the display device 15 as image data. The display device 15 receives image data from the image processing circuit 14 and can perform a clear display.

Step S34 shown in FIG. 11 can be illustrated as in the block diagram shown in FIG. FIG. 15 illustrates the imaging data 16 illustrated in FIG. 9C for easy understanding. In FIG. 15, an image 18 A is illustrated as correction data 18 output to the image processing circuit 14 for easy understanding. In FIG. 15, image data 51 corresponding to the image data output to the display device 15 is illustrated for easy understanding.

In the above-described display system 10 according to an aspect of the present invention, as illustrated in FIG. 11, the feature amount is extracted from a part of information of the imaging data 16, and the feature amount data 17 is collated in the database 13. 18 is selected and the unclear area of the imaged data 16 is corrected. Therefore, it is effective for improving visibility.

(Embodiment 2)
In this embodiment, a configuration example of a semiconductor device that can be used for the neural network 30 described in the above embodiment will be described. In particular, it can be used in product-sum operation circuits such as the intermediate layer 32 and the output layer 33.

<Configuration example of semiconductor device>
FIG. 16 shows a configuration example of the semiconductor device MAC having a function of performing a neural network operation. The semiconductor device MAC can be used for at least a part of the neural network 30 described in the above embodiment. The semiconductor device MAC has a function of performing a product-sum operation on the first data corresponding to the connection strength (weight) between the neurons and the second data corresponding to the input data. Note that the first data and the second data can be analog data or multivalued data (discrete data), respectively. Further, the semiconductor device MAC has a function of converting data obtained by the product-sum operation using an activation function.

The semiconductor device MAC includes a cell array CA, a current source circuit CS, a current mirror circuit CM, a circuit WDD, a circuit WLD, a circuit CLD, an offset circuit OFST, and an activation function circuit ACTV.

The cell array CA includes a plurality of memory cells MC and a plurality of memory cells MCref. In FIG. 16, the cell array CA has m rows and n columns (m and n are integers of 1 or more) memory cells MC (MC [1,1] to [m, n]) and m memory cells MCref (MCref 2 shows an example of a configuration having [1] to [m]). Memory cell MC has a function of storing first data. The memory cell MCref has a function of storing reference data used for product-sum operation. The reference data can be analog data or multi-value data.

The memory cell MC [i, j] (i is an integer of 1 to m, j is an integer of 1 to n) includes the wiring WL [i], the wiring RW [i], the wiring WD [j], and the wiring BL [J]. The memory cell MCref [i] is connected to the wiring WL [i], the wiring RW [i], the wiring WDref, and the wiring BLref. Here, a current flowing between the memory cell MC [i, j] and the wiring BL [ _j] is expressed as I _{MC [i, j],} and a current flowing between the memory cell MCref [i] and the wiring BLref is expressed as I _{MCref [ i]} .

FIG. 17 shows a specific configuration example of the memory cell MC and the memory cell MCref. FIG. 17 shows memory cells MC [1,1], [2,1] and memory cells MCref [1], [2] as representative examples, but the same applies to other memory cells MC and memory cells MCref. Can be used. Each of the memory cell MC and the memory cell MCref includes transistors Tr11 and Tr12 and a capacitor C11. Here, the case where the transistors Tr11 and Tr12 are n-channel transistors will be described.

In the memory cell MC, the gate of the transistor Tr11 is connected to the wiring WL, one of the source and the drain is connected to the gate of the transistor Tr12 and the first electrode of the capacitor C11, and the other of the source or the drain is connected to the wiring WD. Has been. One of a source and a drain of the transistor Tr12 is connected to the wiring BL, and the other of the source and the drain is connected to the wiring VR. The second electrode of the capacitor C11 is connected to the wiring RW. The wiring VR is a wiring having a function of supplying a predetermined potential. Here, as an example, a case where a low power supply potential (such as a ground potential) is supplied from the wiring VR will be described.

A node connected to one of the source and the drain of the transistor Tr11, the gate of the transistor Tr12, and the first electrode of the capacitor C11 is a node NM. The nodes NM of the memory cells MC [1,1] and [2,1] are denoted as nodes NM [1,1] and [2,1], respectively.

The memory cell MCref has the same configuration as the memory cell MC. However, the memory cell MCref is connected to the wiring WDref instead of the wiring WD, and is connected to the wiring BLref instead of the wiring BL. In the memory cells MCref [1] and [2], a node connected to one of the source and the drain of the transistor Tr11, the gate of the transistor Tr12, and the first electrode of the capacitor C11 is a node NMref [1]. , [2].

The node NM and the node NMref function as a memory cell MC and a holding node for the memory cell MCref, respectively. The node NM holds first data, and the node NMref holds reference data. Further, currents IMC _[1,1] and _{IMC [2,1]} flow from the wiring BL [1] to the transistors Tr12 of the memory cells MC [1,1] and [2,1], respectively. Further, currents I _{MCref [1]} and I _{MCref [2]} flow from the wiring BLref to the transistors Tr12 of the memory cells MCref [1] and [2], respectively.

Since the transistor Tr11 has a function of holding the potential of the node NM or the node NMref, the off-state current of the transistor Tr11 is preferably small. Therefore, an OS transistor with an extremely low off-state current is preferably used as the transistor Tr11. As a result, fluctuations in the potential of the node NM or the node NMref can be suppressed, and the calculation accuracy can be improved. In addition, the frequency of the operation of refreshing the potential of the node NM or the node NMref can be suppressed, and power consumption can be reduced.

The transistor Tr12 is not particularly limited, and for example, a Si transistor or an OS transistor can be used. When an OS transistor is used as the transistor Tr12, the transistor Tr12 can be manufactured using the same manufacturing apparatus as the transistor Tr11, and manufacturing cost can be reduced. Note that the transistor Tr12 may be an n-channel type or a p-channel type.

The current source circuit CS is connected to the wirings BL [1] to [n] and the wiring BLref. The current source circuit CS has a function of supplying current to the wirings BL [1] to [n] and the wiring BLref. Note that the current value supplied to the wirings BL [1] to [n] may be different from the current value supplied to the wiring BLref. Here, the current supplied from the current source circuit CS to the wirings BL [1] to [n] is expressed as I _C , and the current supplied from the current source circuit CS to the wiring BLref is expressed as I _Cref .

The current mirror circuit CM includes wirings IL [1] to [n] and wiring ILref. The wirings IL [1] to [n] are connected to the wirings BL [1] to [n], respectively, and the wiring ILref is connected to the wiring BLref. Here, connection points between the wirings IL [1] to [n] and the wirings BL [1] to [n] are denoted as nodes NP [1] to [n]. Further, a connection point between the wiring ILref and the wiring BLref is referred to as a node NPref.

The current mirror circuit CM has a function of flowing a current I _CM corresponding to the potential of the node NPref to the wiring ILref and a function of flowing the current I _CM to the wirings IL [1] to [n]. Figure 16 is discharged current _{I CM} from the wiring BLref to the wiring ILref, wiring BL [1] to the wiring from the [n] IL [1] to [n] to the current _{I CM} is an example to be discharged . Further, currents flowing from the current mirror circuit CM to the cell array CA via the wirings BL [1] to [n] are denoted as I _B [1] to [n]. A current flowing from the current mirror circuit CM to the cell array CA via the wiring _BLref is denoted as I _Bref .

The circuit WDD is connected to the wirings WD [1] to [n] and the wiring WDref. The circuit WDD has a function of supplying a potential corresponding to the first data stored in the memory cell MC to the wirings WD [1] to [n]. Further, the circuit WDD has a function of supplying a potential corresponding to reference data stored in the memory cell MCref to the wiring WDref. The circuit WLD is connected to the wirings WL [1] to [m]. The circuit WLD has a function of supplying a signal for selecting the memory cell MC or the memory cell MCref to which data is written to the wirings WL [1] to [m]. The circuit CLD is connected to the wirings RW [1] to [m]. The circuit CLD has a function of supplying a potential corresponding to the second data to the wirings RW [1] to [m].

The offset circuit OFST is connected to the wirings BL [1] to [n] and the wirings OL [1] to [n]. The offset circuit OFST has a function of detecting a current amount flowing from the wirings BL [1] to [n] to the offset circuit OFST and / or a change amount of a current flowing from the wirings BL [1] to [n] to the offset circuit OFST. Have The offset circuit OFST has a function of outputting the detection result to the wirings OL [1] to [n]. Note that the offset circuit OFST may output a current corresponding to the detection result to the wiring OL, or may convert a current corresponding to the detection result into a voltage and output the voltage to the wiring OL. The current flowing between the cell array CA and the offset circuit OFST is expressed as I _α [1] to [n].

FIG. 18 shows a configuration example of the offset circuit OFST. The offset circuit OFST illustrated in FIG. 18 includes circuits OC [1] to [n]. Each of the circuits OC [1] to [n] includes a transistor Tr21, a transistor Tr22, a transistor Tr23, a capacitor C21, and a resistor R1. The connection relationship of each element is as shown in FIG. Note that a node connected to the first electrode of the capacitor C21 and the first terminal of the resistor element R1 is referred to as a node Na. A node connected to the second electrode of the capacitor C21, one of the source or the drain of the transistor Tr21, and the gate of the transistor Tr22 is referred to as a node Nb.

The wiring VrefL has a function of supplying the potential Vref, the wiring VaL has a function of supplying the potential Va, and the wiring VbL has a function of supplying the potential Vb. The wiring VDDL has a function of supplying the potential VDD, and the wiring VSSL has a function of supplying the potential VSS. Here, the case where the potential VDD is a high power supply potential and the potential VSS is a low power supply potential will be described. The wiring RST has a function of supplying a potential for controlling the conduction state of the transistor Tr21. The transistor Tr22, the transistor Tr23, the wiring VDDL, the wiring VSSL, and the wiring VbL constitute a source follower circuit.

Next, an operation example of the circuits OC [1] to [n] will be described. In addition, although the operation example of circuit OC [1] is demonstrated here as a typical example, circuit OC [2] thru | or [n] can be operated similarly. First, when a first current flows through the wiring BL [1], the potential of the node Na becomes a potential corresponding to the first current and the resistance value of the resistance element R1. At this time, the transistor Tr21 is on, and the potential Va is supplied to the node Nb. Thereafter, the transistor Tr21 is turned off.

Next, when a second current flows through the wiring BL [1], the potential of the node Na changes to a potential corresponding to the second current and the resistance value of the resistance element R1. At this time, since the transistor Tr21 is in an off state and the node Nb is in a floating state, the potential of the node Nb changes due to capacitive coupling as the potential of the node Na changes. Here, if the change in the potential of the node Na is ΔV _Na and the capacitive coupling coefficient is 1, the potential of the node Nb is Va + ΔV _Na . When the threshold voltage of the transistor Tr22 and _{V th,} the potential Va + ΔV _Na -V _th is output from the wiring OL [1]. Here, by Va _{= V th,} it is possible to output the potential [Delta] V _Na from the wiring OL [1].

The potential ΔV _Na is determined according to the amount of change from the first current to the second current, the resistance element R1, and the potential Vref. Here, since the resistance element R1 and the potential Vref is known, it is possible to determine the amount of change current flowing from the potential [Delta] V _Na wiring BL.

The signal corresponding to the amount of current detected by the offset circuit OFST and / or the amount of change in current as described above is input to the activation function circuit ACTV via the wirings OL [1] to [n].

The activation function circuit ACTV is connected to the wirings OL [1] to [n] and the wirings NIL [1] to [n]. The activation function circuit ACTV has a function of performing an operation for converting the signal input from the offset circuit OFST according to a predefined activation function. As the activation function, for example, a sigmoid function, a tanh function, a softmax function, a ReLU function, a threshold function, or the like can be used. The signal converted by the activation function circuit ACTV is output as output data to the wirings NIL [1] to [n].

<Operation example of semiconductor device>
Using the semiconductor device MAC, the product-sum operation of the first data and the second data can be performed. Hereinafter, an operation example of the semiconductor device MAC when performing the product-sum operation will be described.

FIG. 19 shows a timing chart of an operation example of the semiconductor device MAC. 19 includes the wiring WL [1], the wiring WL [2], the wiring WD [1], the wiring WDref, the node NM [1,1], the node NM [2,1], and the node NMref [1] in FIG. , The transition of the potential of the node NMref [2], the wiring RW [1], and the wiring RW [2], and the transition of the values of the current I _B [1] −I _α [1] and the current I _Bref . . The current I _B [1] −I _α [1] corresponds to the sum of currents flowing from the wiring BL [1] to the memory cells MC [1,1] and [2,1].

Here, as a representative example, the operation will be described focusing on the memory cells MC [1,1] and [2,1] and the memory cells MCref [1] and [2] shown in FIG. The MC and the memory cell MCref can be operated similarly.

[Storage of first data]
First, at time T01-T02, the potential of the wiring WL [1] becomes high level, the potential of the wiring WD [1] becomes V _PR −V _{W [1,1]} higher than the ground potential (GND), and the wiring potential of WDref becomes the _{V PR} greater potential than the ground potential. Further, the potentials of the wiring RW [1] and the wiring RW [2] are the reference potential (REFP). Note that the potential V _{W [1, 1]} is a potential corresponding to the first data stored in the memory cell MC [1, 1]. The potential _VPR is a potential corresponding to the reference data. Accordingly, the transistor Tr11 included in the memory cell MC [1,1] and the memory cell MCref [1] is turned on, and the potential of the node NM [1,1] is V _PR −V _{W [1,1]} and the node NMref. The potential of [1] becomes _VPR .

At this time, a current I _{MC [1,1], 0} flowing from the wiring BL [1] to the transistor Tr12 of the memory cell MC [1,1] can be expressed by the following equation. Here, k is a constant determined by the channel length, channel width, mobility, capacitance of the gate insulating film, and the like of the transistor Tr12. V _th is the threshold voltage of the transistor Tr12.

Further, the current I _{MCref [1], 0} flowing from the wiring BLref to the transistor Tr12 of the memory cell MCref [1] can be expressed by the following equation.

Next, at time T02 to T03, the potential of the wiring WL [1] is at a low level. Accordingly, the transistor Tr11 included in the memory cell MC [1,1] and the memory cell MCref [1] is turned off, and the potentials of the node NM [1,1] and the node NMref [1] are held.

As described above, an OS transistor is preferably used as the transistor Tr11. Accordingly, leakage current of the transistor Tr11 can be suppressed, and the potentials of the node NM [1,1] and the node NMref [1] can be accurately held.

Next, at time T03-T04, the potential of the wiring WL [2] is at a high level, the potential of the wiring WD [1] is V _PR −V _{W [2,1]} higher than the ground potential, and the wiring WDref The potential becomes a potential _VPR larger than the ground potential. Note that the potential V _{W [2, 1]} is a potential corresponding to the first data stored in the memory cell MC [2, 1]. Accordingly, the transistor Tr11 included in the memory cell MC [2,1] and the memory cell MCref [2] is turned on, and the potential of the node NM [2,1] is V _PR −V _{W [2,1]} and the node NMref. The potential of [2] becomes _VPR .

At this time, the current I _{MC [2,1], 0} flowing from the wiring BL [1] to the transistor Tr12 of the memory cell MC [2,1] can be expressed by the following equation.

Further, the current I _{MCref [2], 0} flowing from the wiring BLref to the transistor Tr12 of the memory cell MCref [2] can be expressed by the following equation.

Next, at time T04 to T05, the potential of the wiring WL [2] is at a low level. Accordingly, the transistor Tr11 included in the memory cell MC [2,1] and the memory cell MCref [2] is turned off, and the potentials of the node NM [2,1] and the node NMref [2] are held.

Through the above operation, the first data is stored in the memory cells MC [1,1] and [2,1], and the reference data is stored in the memory cells MCref [1] and [2].

Here, currents flowing through the wiring BL [1] and the wiring BLref from time T04 to T05 are considered. A current is supplied from the current source circuit CS to the wiring BLref. Further, the current flowing through the wiring BLref is discharged to the current mirror circuit CM and the memory cells MCref [1] and [2]. When the current supplied from the current source circuit CS to the wiring BLref is I _Cref and the current discharged from the wiring BLref to the current mirror circuit CM is I _{CM, 0} , the following equation is established.

A current from the current source circuit CS is supplied to the wiring BL [1]. Further, the current flowing through the wiring BL [1] is discharged to the current mirror circuit CM and the memory cells MC [1,1], [2,1]. In addition, a current flows from the wiring BL [1] to the offset circuit OFST. When the current supplied from the current source circuit CS to the wiring BL [1] is I _{C, 0} and the current flowing from the wiring BL [1] to the offset circuit OFST is I _{α, 0} , the following equation is established.

[Product-sum operation of first data and second data]
Next, at time T05 to T06, the potential of the wiring RW [1] is V _{X [1]} larger than the reference potential. At this time, the potential V _{X [1]} is supplied to the respective capacitive elements C11 of the memory cell MC [1,1] and the memory cell MCref [1], and the potential of the gate of the transistor Tr12 is increased by capacitive coupling. Note that the potential V _{x [1]} is a potential corresponding to the second data supplied to the memory cell MC [1, 1] and the memory cell MCref [1].

The amount of change in the potential of the gate of the transistor Tr12 is a value obtained by multiplying the amount of change in the potential of the wiring RW by the capacitive coupling coefficient determined by the configuration of the memory cell. The capacitive coupling coefficient is calculated by the capacitance of the capacitive element C11, the gate capacitance of the transistor Tr12, the parasitic capacitance, and the like. Hereinafter, for the sake of convenience, description will be made assuming that the amount of change in potential of the wiring RW and the amount of change in potential of the gate of the transistor Tr12 are the same, that is, the capacitive coupling coefficient is 1. Actually, the potential V _x may be determined in consideration of the capacitive coupling coefficient.

When the potential V _{X [1]} is supplied to the capacitor C11 of the memory cell MC [1] and the memory cell MCref [1], the potentials of the node NN [1] and the node NMref [1] are V _{X [1],} respectively _. To rise.

Here, from time T05 to T06, the current I _{MC [1,1], 1} flowing from the wiring BL [1] to the transistor Tr12 of the memory cell MC [1,1] can be expressed by the following equation.

That is, by supplying the potential V _{X [1]} to the wiring RW [1], the current flowing from the wiring BL [1] to the transistor Tr12 of the memory cell MC [1,1] is ΔI _{MC [1,1]} = I _{MC [1,1], 1} −I _{MC [1,1], 0 is} increased.

Further, current I _{MCref [1], 1} flowing from the wiring BLref to the transistor Tr12 of the memory cell MCref [1] from time T05 to T06 can be expressed by the following equation.

That is, by supplying the potential V _{X [1]} to the wiring RW [1], the current flowing from the wiring BLref to the transistor Tr12 of the memory cell MCref [1] is ΔI _{MCref [1]} = I _{MCref [1], 1} -I _{MCref [1],} incremented by ₀ .

Consider the current flowing through the wiring BL [1] and the wiring BLref. A current I _Cref is supplied from the current source circuit CS to the wiring BLref. Further, the current flowing through the wiring BLref is discharged to the current mirror circuit CM and the memory cells MCref [1] and [2]. When the current discharged from the wiring BLref to the current mirror circuit CM is I _{CM, 1} , the following equation is established.

The wiring BL [1], the current _{I C} is supplied from the current source circuit CS. Further, the current flowing through the wiring BL [1] is discharged to the current mirror circuit CM and the memory cells MC [1,1], [2,1]. Further, a current also flows from the wiring BL [1] to the offset circuit OFST. When the current flowing from the wiring BL [1] to the offset circuit OFST is I _{α, 1} , the following equation is established.

From the equations (E1) to (E10) _, the difference between the current I _{α, 0} and the current I _{α, 1} (differential current ΔI _α ) can be expressed by the following equation.

Thus, the differential current ΔI _α has a value corresponding to the product of the potentials V _{W [1, 1]} and V _{X [1]} .

Thereafter, at time T06 to T07, the potential of the wiring RW [1] becomes the ground potential, and the potentials of the node NM [1,1] and the node NMref [1] are the same as those of the time T04-T05.

Next, at time T07 to T08, the potential of the wiring RW [1] is V _{X [1]} larger than the reference potential, and the potential of the wiring RW [2] is V _{X [2]} larger than the reference potential. Become. As a result, the potential V _{X [1]} is supplied to the respective capacitive elements C11 of the memory cell MC [1,1] and the memory cell MCref [1], and the node NM [1,1] and the node NMref [ 1] is increased by V _{X [1]} . In addition, the potential V _{X [2]} is supplied to the respective capacitor C11 of the memory cell MC [2,1] and the memory cell MCref [2], and the node NM [2,1] and the node NMref [2 _] are connected by capacitive coupling. ] Increases by V _{X [2]} .

Here, at time T07-T08, the current I _{MC [2,1], 1} flowing from the wiring BL [1] to the transistor Tr12 of the memory cell MC [2,1] can be expressed by the following equation.

That is, by supplying the potential V _{X [2]} to the wiring RW [2], the current flowing from the wiring BL [1] to the transistor Tr12 of the memory cell MC [2, 1] is ΔI _{MC [2,1]} = I _{MC [2,1], 1} −I _{MC [2,1], 0} increases.

Further, current I _{MCref [2], 1} flowing from the wiring BLref to the transistor Tr12 of the memory cell MCref [2] from time T05 to T06 can be expressed by the following equation.

That is, by supplying the potential V _{X [2]} to the wiring RW [2], the current flowing from the wiring BLref to the transistor Tr12 of the memory cell MCref [2] is ΔI _{MCref [2]} = I _{MCref [2], 1} -I _{MCref [2],} incremented by ₀ .

Consider the current flowing through the wiring BL [1] and the wiring BLref. A current I _Cref is supplied from the current source circuit CS to the wiring BLref. Further, the current flowing through the wiring BLref is discharged to the current mirror circuit CM and the memory cells MCref [1] and [2]. When the current discharged from the wiring BLref to the current mirror circuit CM is I _{CM, 2} , the following equation is established.

The wiring BL [1], the current _{I C} is supplied from the current source circuit CS. Further, the current flowing through the wiring BL [1] is discharged to the current mirror circuit CM and the memory cells MC [1,1], [2,1]. Further, a current also flows from the wiring BL [1] to the offset circuit OFST. Assuming that the current flowing from the wiring BL [1] to the offset circuit OFST is I _{α, 2} , the following equation is established.

Then, from the equations (E1) to (E8) and the equations (E12) to (E15) _, the difference between the current I _{α, 0} and the current I _{α, 2} (differential current ΔI _α ) is expressed by the following equation. be able to.

As described above, the differential current ΔI _α is obtained by adding the product of the potential V _{W [1, 1]} and the potential V _{X [1] and} the product of the potential V _{W [2, 1]} and the potential V _{X [2].} The value depends on the combined result.

After that, at times T08 to T09, the potentials of the wirings RW [1] and [2] become the ground potential, and the potentials of the nodes NM [1,1] and [2,1] and the nodes NMref [1] and [2] are It becomes the same as time T04-T05.

As shown in the equations (E9) and (E16), the difference current ΔI _α input to the offset circuit OFST includes the potential V _X corresponding to the first data (weight) and the second data (input data). ) corresponding to a value corresponding to the combined result plus the product of the potential V _W. That is, by measuring the differential current ΔI _α with the offset circuit OFST, it is possible to obtain a product-sum operation result of the first data and the second data.

In the above description, the memory cells MC [1,1] and [2,1] and the memory cells MCref [1] and [2] are particularly focused. However, the number of the memory cells MC and the memory cells MCref should be arbitrarily set. Can do. The differential current ΔIα when the number of rows m of the memory cell MC and the memory cell MCref is an arbitrary number can be expressed by the following equation.

Also, by increasing the number of columns n of the memory cells MC and the memory cells MCref, the number of product-sum operations executed in parallel can be increased.

As described above, by using the semiconductor device MAC, the product-sum operation of the first data and the second data can be performed. Note that by using the structure shown in FIG. 17 as the memory cell MC and the memory cell MCref, a product-sum operation circuit can be formed with a small number of transistors. Therefore, the circuit scale of the semiconductor device MAC can be reduced.

When the semiconductor device MAC is used for computation in the neural network, the number of rows m of the memory cells MC corresponds to the number of input data supplied to one neuron, and the number of columns n of the memory cells MC corresponds to the number of neurons. Can do. For example, consider a case where a product-sum operation using the semiconductor device MAC is performed in the intermediate layer. At this time, the number of rows m of the memory cells MC is set to the number of input data (number of neurons in the input layer) supplied from the input layer, and the number of columns n of the memory cells MC is set to the number of neurons in the intermediate layer. Can be set.

As described above, by using the semiconductor device MAC, the product-sum operation of the neural network can be performed. Further, by using the memory cell MC and the memory cell MCref shown in FIG. 17 for the cell array CA, an integrated circuit capable of improving calculation accuracy, reducing power consumption, or reducing the circuit scale can be provided. .

The product difference calculation can be performed by setting “positive” when the potential V _X corresponding to the first data (weight) is equal to or higher than an arbitrary reference potential, and “negative” when the potential is smaller. Therefore, the semiconductor device MAC can function as a product difference arithmetic circuit.

積 Sudden noise can be removed using a product difference arithmetic circuit. As an example, FIGS. 20A and 20B are shown. 20A and 20B are images showing a vehicle that is traveling in a dark place such as in a tunnel. FIG. 20A shows that the vehicle 110 positioned outside the tunnel can be visually recognized due to the influence of the headlight 105 of the vehicle 100 that is suddenly turned on, but the visibility of the pedestrian 120 and the pedestrian 130 inside the tunnel is reduced. Shows the state.

In FIG. 20B, the image processing using the product difference arithmetic circuit reduces the influence of the headlight 105 that is suddenly turned on, and not only the vehicle 110 located outside the tunnel but also the pedestrian 120 inside the tunnel. In addition, the visibility of the pedestrian 130 is also improved. Note that difference information between the current frame image and the previous frame image can be acquired by the product difference calculation circuit, and image processing can be performed using the difference information.

In recent years, in-vehicle image sensors are expected for automated driving and as side mirrors and rearview mirrors. By combining one embodiment of the present invention with a mechanism for adjusting sensitivity for each pixel, an obstacle or a person can be accurately detected even in a situation where the lights of other vehicles are dazzling.

This embodiment mode can be combined with any of the other embodiment modes as appropriate.

(Embodiment 3)
In this embodiment, an example of the display device 15 illustrated in the above embodiment will be described below with reference to FIGS.

FIG. 21 is a top view showing a display device 700 applicable to the display device 15 exemplified in the previous embodiment. A display device 700 illustrated in FIG. 21 includes a pixel portion 702 provided over a first substrate 701, a demultiplexer 703 provided in the first substrate 701, a source driver 704 and a gate driver 706, a pixel portion 702, The sealant 712 is disposed so as to surround the demultiplexer 703 and the gate driver 706, and the second substrate 705 is provided so as to face the first substrate 701. Note that the first substrate 701 and the second substrate 705 are sealed with a sealant 712. That is, the pixel portion 702, the demultiplexer 703, and the gate driver 706 are sealed with the first substrate 701, the sealant 712, and the second substrate 705. Note that although not illustrated in FIG. 21, a display element is provided between the first substrate 701 and the second substrate 705.

In addition, the display device 700 includes a pixel portion 702, a demultiplexer 703, a source driver 704, and a gate driver 706 in different regions from the region surrounded by the sealant 712 on the first substrate 701. FPC terminal portion 708 (FPC: Flexible printed circuit) is provided. In addition, an FPC 716 is connected to the FPC terminal portion 708, and various signals are supplied to the pixel portion 702, the demultiplexer 703, the source driver 704, and the gate driver 706 by the FPC 716. A signal line 710 is connected to each of the pixel portion 702, the demultiplexer 703, the source driver 704, the gate driver 706, and the FPC terminal portion 708. Various signals and the like supplied from the FPC 716 are supplied to the pixel portion 702, the demultiplexer 703, the source driver 704, the gate driver 706, and the FPC terminal portion 708 through the signal line 710.

Further, a plurality of gate drivers 706 may be provided in the display device 700. In addition, as the display device 700, an example in which the gate driver 706 is formed over the same first substrate 701 as the pixel portion 702 and the source driver 704 is a source driver IC is shown; however, the structure is not limited to this. For example, the source driver 704 may be formed on the first substrate 701. Note that the source driver IC can be provided by a COG (Chip On Glass) method, a wire bonding method, or the like. Further, the demultiplexer 703 can be omitted.

In addition, the display device 700 can have various elements. Examples of the element include, for example, an electroluminescence (EL) element (an EL element including an organic substance and an inorganic substance, an organic EL element, an inorganic EL element, an LED, etc.), a light-emitting transistor (a transistor that emits light in response to a current), and electron emission. Element, liquid crystal element, electronic ink element, electrophoretic element, electrowetting element, plasma display panel (PDP), MEMS (micro electro mechanical system) display (for example, grating light valve (GLV), digital micromirror device (DMD), digital micro shutter (DMS) element, interferometric modulation (IMOD) element, etc.), piezoelectric ceramic display, and the like.

An example of a display device using an EL element is an EL display. As an example of a display device using an electron-emitting device, there is a field emission display (FED), a SED type flat display (SED: Surface-conduction Electron-emitter Display), or the like. As an example of a display device using a liquid crystal element, there is a liquid crystal display (a transmissive liquid crystal display, a transflective liquid crystal display, a reflective liquid crystal display, a direct view liquid crystal display, a projection liquid crystal display) and the like. An example of a display device using an electronic ink element or an electrophoretic element is electronic paper. Note that in the case of realizing a transflective liquid crystal display or a reflective liquid crystal display, part or all of the pixel electrode may have a function as a reflective electrode. For example, part or all of the pixel electrode may have aluminum, silver, or the like. Further, in that case, a memory circuit such as an SRAM can be provided under the reflective electrode. Thereby, power consumption can be further reduced.

Note that as a display method in the display device 700, a progressive method, an interlace method, or the like can be used. Further, the color elements controlled by the pixels when performing color display are not limited to three colors of RGB (R represents red, G represents green, and B represents blue). For example, it may be composed of four pixels: an R pixel, a G pixel, a B pixel, and a W (white) pixel. Alternatively, as in a pen tile arrangement, one color element may be configured by two colors of RGB, and two different colors may be selected and configured depending on the color element. Alternatively, one or more colors such as yellow, cyan, and magenta may be added to RGB. The size of the display area may be different for each dot of the color element. Note that the disclosed invention is not limited to a display device for color display, and can be applied to a display device for monochrome display.

In addition, a colored layer (also referred to as a color filter) may be used in order to display white light (W) in a backlight (an organic EL element, an inorganic EL element, an LED, a fluorescent lamp, or the like) and display a full color display device. Good. For example, red (R), green (G), blue (B), yellow (Y), and the like can be used in appropriate combination for the colored layer. By using the colored layer, the color reproducibility can be increased as compared with the case where the colored layer is not used. At this time, white light in a region having no colored layer may be directly used for display by arranging a region having a colored layer and a region having no colored layer. By disposing a region that does not have a colored layer in part, a decrease in luminance due to the colored layer can be reduced during bright display, and power consumption can be reduced by about 20% to 30%. However, when a full color display is performed using a self-luminous element such as an organic EL element or an inorganic EL element, R, G, B, Y, and W may be emitted from elements having respective emission colors. By using a self-luminous element, power consumption may be further reduced as compared with the case where a colored layer is used.

In addition, as a colorization method, in addition to a method (color filter method) in which part of the light emission from the white light emission described above is converted into red, green, and blue through a color filter, red, green, and blue light emission is performed. A method of using each (three-color method) or a method of converting a part of light emission from blue light emission into red or green (color conversion method, quantum dot method) may be applied.

In this embodiment, a structure in which a liquid crystal element and an EL element are used as display elements will be described with reference to FIGS. Note that FIG. 22 is a cross-sectional view taken along one-dot chain line QR shown in FIG. FIG. 23 is a cross-sectional view taken along the alternate long and short dash line QR shown in FIG. 21, and includes an EL element as a display element.

First, common parts shown in FIGS. 22 and 23 will be described first, and then different parts will be described below.

<Description of common parts of display device>
A display device 700 illustrated in FIGS. 22 and 23 includes a lead wiring portion 711, a pixel portion 702, a demultiplexer 703, and an FPC terminal portion 708. Further, the lead wiring portion 711 includes a signal line 710. In addition, the pixel portion 702 includes a transistor 750 and a capacitor 790. In addition, the demultiplexer 703 includes a transistor 752.

The transistor 750 and the transistor 752 may be any of a top gate type, a bottom gate type, a channel etch type, and a channel protection type. 22 and 23 illustrate a top gate type. For the semiconductor layers of the transistor 750 and the transistor 752, a silicon-based semiconductor (amorphous silicon, polycrystalline silicon, or the like), an oxide semiconductor (zinc oxide, indium oxide, or the like), or the like can be used. 22 and 23 illustrate the case where an oxide semiconductor is used.

The capacitor 790 includes a first oxide semiconductor film included in the transistor 750, a lower electrode formed through a step of processing the same oxide semiconductor film, and a conductive material functioning as a source electrode and a drain electrode included in the transistor 750. A film and an upper electrode formed through a process of processing the same conductive film. In addition, a step of forming the same insulating film as the second insulating film and the insulating film functioning as the third insulating film of the transistor 750 between the lower electrode and the upper electrode is formed. An insulating film formed through the above is provided. That is, the capacitor 790 has a stacked structure in which an insulating film functioning as a dielectric is sandwiched between a pair of electrodes.

22 and 23, a planarization insulating film 770 is provided over the transistor 750, the transistor 752, and the capacitor 790.

As the planarization insulating film 770, an organic material having heat resistance such as polyimide resin, acrylic resin, polyimide amide resin, benzocyclobutene resin, polyamide resin, or epoxy resin can be used. Note that the planarization insulating film 770 may be formed by stacking a plurality of insulating films formed using these materials. Further, the planarization insulating film 770 may be omitted.

Further, the signal line 710 is formed through the same process as the conductive film functioning as the source and drain electrodes of the

transistors

750 and 752. Note that the signal line 710 is a conductive film formed through a different process from the source and drain electrodes of the

transistors

750 and 752, for example, an oxide semiconductor formed through the same process as an oxide semiconductor film functioning as a gate electrode. A membrane may be used. For example, when a material containing a copper element is used as the signal line 710, signal delay due to wiring resistance is small and display on a large screen is possible.

The FPC terminal portion 708 includes a connection electrode 760, an anisotropic conductive film 780, and an FPC 716. Note that the connection electrode 760 is formed through the same process as the conductive film functioning as the source and drain electrodes of the

transistors

750 and 752. The connection electrode 760 is electrically connected to a terminal included in the FPC 716 through an anisotropic conductive film 780.

Further, as the first substrate 701 and the second substrate 705, for example, glass substrates can be used. Further, as the first substrate 701 and the second substrate 705, flexible substrates may be used. Examples of the flexible substrate include a plastic substrate.

In addition, a structure body 778 is provided between the first substrate 701 and the second substrate 705. The structure body 778 is a columnar spacer obtained by selectively etching an insulating film, and is provided to control the distance (cell gap) between the first substrate 701 and the second substrate 705. Note that a spherical spacer may be used as the structure body 778.

Further, on the second substrate 705 side, a light shielding film 738 functioning as a black matrix, a colored film 736 functioning as a color filter, and an insulating film 734 in contact with the light shielding film 738 and the colored film 736 are provided.

<Configuration Example of Display Device Using Liquid Crystal Element>
A display device 700 illustrated in FIG. 22 includes a liquid crystal element 775. The liquid crystal element 775 includes a conductive film 772, a conductive film 774, and a liquid crystal layer 776. The conductive film 774 is provided on the second substrate 705 side and functions as a counter electrode. The display device 700 illustrated in FIG. 22 can display an image by controlling transmission and non-transmission of light by changing the alignment state of the liquid crystal layer 776 depending on voltages applied to the

conductive films

772 and 774.

The conductive film 772 is connected to a conductive film functioning as a source electrode and a drain electrode of the transistor 750. The conductive film 772 is formed over the planarization insulating film 770 and functions as a pixel electrode, that is, one electrode of a display element. The conductive film 772 functions as a transparent electrode. A display device 700 illustrated in FIG. 22 is a so-called transmissive color liquid crystal display device that transmits light from a backlight through a liquid crystal layer 776 and displays the light through a colored film 736.

As the conductive film 772, a conductive film that is transparent to visible light or a conductive film that is reflective to visible light can be used. As the conductive film that transmits visible light, for example, a material containing one kind selected from indium (In), zinc (Zn), and tin (Sn) may be used. As the conductive film having reflectivity in visible light, for example, a material containing aluminum or silver is preferably used.

Although not shown in FIG. 22, an alignment film may be provided on each side of the

conductive films

772 and 774 in contact with the liquid crystal layer 776. Although not shown in FIG. 22, an optical member (an optical substrate) such as a polarizing member, a retardation member, or an antireflection member may be provided as appropriate. For example, circularly polarized light using a polarizing substrate and a retardation substrate may be used. Further, a backlight, a sidelight, or the like may be used as the light source.

When a liquid crystal element is used as the display element, a thermotropic liquid crystal, a low molecular liquid crystal, a polymer liquid crystal, a polymer dispersed liquid crystal, a ferroelectric liquid crystal, an antiferroelectric liquid crystal, or the like can be used. These liquid crystal materials exhibit a cholesteric phase, a smectic phase, a cubic phase, a chiral nematic phase, an isotropic phase, and the like depending on conditions.

In addition, when the horizontal electric field method is adopted, a liquid crystal exhibiting a blue phase without using an alignment film may be used. The blue phase is one of the liquid crystal phases. When the temperature of the cholesteric liquid crystal is increased, the blue phase appears immediately before the transition from the cholesteric phase to the isotropic phase. Since the blue phase appears only in a narrow temperature range, a liquid crystal composition mixed with several percent by weight or more of a chiral agent is used for the liquid crystal layer in order to improve the temperature range. A liquid crystal composition containing a liquid crystal exhibiting a blue phase and a chiral agent has a short response speed and is optically isotropic, so that alignment treatment is unnecessary. Further, since it is not necessary to provide an alignment film, a rubbing process is not required, so that electrostatic breakdown caused by the rubbing process can be prevented, and defects or breakage of the liquid crystal display device during the manufacturing process can be reduced. . A liquid crystal material exhibiting a blue phase has a small viewing angle dependency.

In addition, when a liquid crystal element is used as a display element, a TN (Twisted Nematic) mode, an IPS (In-Plane-Switching) mode, an FFS (Fringe Field Switching) mode, an ASM (Axially Symmetrical Aligned MicroOcell) mode. A Compensated Birefringence mode, an FLC (Ferroelectric Liquid Crystal) mode, an AFLC (Antiferroelectric Liquid Crystal) mode, and the like can be used.

Alternatively, a normally black liquid crystal display device such as a transmissive liquid crystal display device employing a vertical alignment (VA) mode may be used. There are several examples of the vertical alignment mode. For example, an MVA (Multi-Domain Vertical Alignment) mode, a PVA (Patterned Vertical Alignment) mode, an ASV mode, and the like can be used.

<Display device using light emitting element>
A display device 700 illustrated in FIG. 23 includes a light-emitting element 782. The light-emitting element 782 includes a conductive film 784, an EL layer 786, and a conductive film 788. The display device 700 illustrated in FIG. 23 can display an image when the EL layer 786 included in the light-emitting element 782 emits light.

Further, the conductive film 784 is connected to a conductive film functioning as a source electrode and a drain electrode of the transistor 750. The conductive film 784 is formed over the planarization insulating film 770 and functions as a pixel electrode, that is, one electrode of a display element. As the conductive film 784, a conductive film that transmits visible light or a conductive film that reflects visible light can be used. As the conductive film that transmits visible light, for example, a material containing one kind selected from indium (In), zinc (Zn), and tin (Sn) may be used. As the conductive film having reflectivity in visible light, for example, a material containing aluminum or silver is preferably used.

Further, in the display device 700 illustrated in FIG. 23, the insulating film 730 is provided over the planarization insulating film 770 and the conductive film 784. The insulating film 730 covers part of the conductive film 784. Note that the light-emitting element 782 has a top emission structure. Therefore, the conductive film 788 has a light-transmitting property and transmits light emitted from the EL layer 786. In the present embodiment, the top emission structure is illustrated, but is not limited thereto. For example, a bottom emission structure in which light is emitted to the conductive film 784 side or a dual emission structure in which light is emitted to both the conductive film 784 and the conductive film 788 can be used.

Further, a colored film 736 is provided at a position overlapping with the light emitting element 782, and a light shielding film 738 is provided at a position overlapping with the insulating film 730, the routing wiring portion 711, and the source driver 704. Further, the coloring film 736 and the light shielding film 738 are covered with an insulating film 734. A space between the light emitting element 782 and the insulating film 734 is filled with a sealing film 732. Note that in the display device 700 illustrated in FIG. 23, the structure in which the colored film 736 is provided is illustrated, but the present invention is not limited to this. For example, in the case where the EL layer 786 is formed by separate coating, the coloring film 736 may not be provided.

The structure described in this embodiment can be combined as appropriate with any of the structures described in the other embodiments.

(Embodiment 4)
In this embodiment, application examples of the display device described in the above embodiment to a vehicle and other moving objects will be described below with reference to FIGS.

<Example of application of display system to moving body>
An example in which the display device included in the above-described display system is provided in the vicinity of a driver's seat of an automobile that is a moving body will be described.

For example, FIG. 24A shows the interior of an automobile in which bench seats are used for the driver's seat and the passenger seat. FIG. 24A illustrates a display device 52A provided at the door, a display device 52B provided at the handle, and a display device 52C provided at the center of the seat surface of the bench seat.

The display device 52A can complement the view blocked by the door, for example, by displaying an image from an imaging device provided on the vehicle body on the display unit.

The

display devices

52B and 52C have various other information such as navigation information, meters such as a speedometer and a tachometer, travel distance, oil supply amount, gear state, and air conditioner settings in addition to the image from the image pickup device provided on the vehicle body. Can be provided. In addition, display items, layouts, and the like displayed on the display device can be changed as appropriate according to user preferences. The

display devices

52B and 52C can also be used as lighting devices.

FIG. 24B is a diagram showing the periphery of the windshield in the interior of a car. FIG. 24B illustrates the display device 53A attached to the dashboard.

The display device 53A can provide various other information such as navigation information, a speedometer and a tachometer, a travel distance, an oil supply amount, a gear state, and an air conditioner setting. In addition, the display items, layout, and the like displayed on the display device can be appropriately changed according to the user's preference, and the design can be improved. The display device 53A can also be used as a lighting device.

Also, the display device 53A can complement the field of view (dead angle) obstructed by the vehicle body by displaying an image from the imaging means provided on the vehicle body. That is, by displaying an image from the imaging means provided outside the automobile, the blind spot can be compensated and safety can be improved. Also, by displaying a video that complements the invisible part, it is possible to confirm the safety more naturally and without a sense of incongruity. The display device 53A can also be used as a lighting device.

<Example of moving body>
An example of the moving body will be described.

The display system according to one embodiment of the present invention can be used not only for automobiles but also for various mobile objects. Specific examples of these moving objects are shown in FIGS.

FIG. 25A shows the bus 302. The moving body according to one embodiment of the present invention can be used for the bus 302. The display system can take an image outside the bus 302 and improve the visibility of the image when viewing the image. Therefore, the bus 302 can be improved in safety.

FIG. 25B shows the train 303. The moving body according to one embodiment of the present invention can be used for the train 303. The display system captures an image outside the train 303 and can improve the visibility of the image when viewing the image. Therefore, the train 303 can be improved in safety.

FIG. 25C shows the airplane 304. The moving body according to one embodiment of the present invention can be used for the airplane 304. The display system can take an image outside the airplane 304 and improve the visibility of the image when viewing the image. Therefore, the airplane 304 can be improved in safety.

<Additional notes regarding the description of this specification>
In this specification and the like, the ordinal numbers “first”, “second”, and “third” are given to avoid confusion between components. Therefore, the number of components is not limited. Further, the order of the components is not limited.

In this specification and the like, in the block diagram, the components are classified by function and shown as independent blocks. However, in an actual circuit or the like, it is difficult to separate the components for each function, and there may be a case where a plurality of functions are involved in one circuit or a case where one function is involved over a plurality of circuits. Therefore, the blocks in the block diagram are not limited to the components described in the specification, and can be appropriately rephrased depending on the situation.

Note that in the drawings, the same element, an element having a similar function, an element of the same material, or an element formed at the same time may be denoted by the same reference numeral, and repeated description thereof may be omitted.

In this specification and the like, in describing connection relations of transistors, “one of a source and a drain” (or a first electrode or a first terminal), “the other of a source and a drain” (or a second electrode, or a second Terminal). This is because the source and drain of a transistor vary depending on the structure or operating conditions of the transistor. Note that the names of the source and the drain of the transistor can be appropriately rephrased depending on the situation, such as a source (drain) terminal or a source (drain) electrode.

In addition, in this specification and the like, voltage and potential can be described as appropriate. The voltage is a potential difference from a reference potential. For example, when the reference potential is a ground potential (ground potential), the voltage can be rephrased as a potential. The ground potential does not necessarily mean 0V. Note that the potential is relative, and the potential applied to the wiring or the like may be changed depending on the reference potential.

In this specification and the like, a switch refers to a switch that is in a conductive state (on state) or a non-conductive state (off state) and has a function of controlling whether or not to pass a current. Alternatively, the switch refers to a switch having a function of selecting and switching a current flow path.

As an example, an electrical switch or a mechanical switch can be used. That is, the switch is not limited to a specific one as long as it can control the current.

Note that when a transistor is used as a switch, the “conducting state” of the transistor means a state in which the source and drain of the transistor can be regarded as being electrically short-circuited. In addition, the “non-conducting state” of a transistor refers to a state where the source and drain of the transistor can be regarded as being electrically cut off. Note that when a transistor is operated as a simple switch, the polarity (conductivity type) of the transistor is not particularly limited.

In this specification and the like, the term “A and B are connected” includes not only those in which A and B are directly connected, but also those that are electrically connected. Here, A and B are electrically connected. When there is an object having some electrical action between A and B, it is possible to send and receive electrical signals between A and B. It says that.

10: Display system, 10A: Display system, 10B: Display system, 10C: Display system, 11: Imaging device, 11A: Imaging device, 12: Feature output circuit, 13: Database, 14: Image processing circuit, 15: Display Device: 16: imaging data, 17: feature data, 18: correction data, 19: detection data, 20: transmission / reception circuit, 21: network, 22: automobile, 22A: automobile, 23: data processing circuit, 24: Relay station, 25: sensor, S11-S17: step, S21-S26: step, 30: neural network, 31: input layer, 32: intermediate layer, 33: output layer, 34: filter, 35: convolution data, 36: Pooling data, 37: filter, 38: convolution data, 39: pooling data, 40: all combined data, 4 : Output data, 42: automobile, 43: light source, 44: irradiation direction, 45: imaging direction, 46: lens, 47: water drop, 48: clear region, 49: unclear region, 11_1-11_n: imaging device, 15_1-15_n : Display device, S31-S34: Step, 18A: Image, 19A: Image, 17A: Feature data, 51: Image data, 700: Display device, 701: Substrate, 702: Pixel unit, 703: Demultiplexer, 704: Source driver 705: Substrate 706: Gate driver 708: FPC terminal portion 710: Signal line 711: Wiring portion 712: Seal material 716: FPC 730: Insulating film 732: Sealing film 734: Insulating film, 736: Colored film, 738: Light shielding film, 750: Transistor, 752: Transistor, 760: Connection electrode, 770: Flat Insulating film, 772: conductive film, 774: conductive film, 775: liquid crystal element, 776: liquid crystal layer, 778: structure, 780: anisotropic conductive film, 782: light emitting element, 784: conductive film, 786: EL layer 788: conductive film, 790: capacitive element, 52A-52C: display device, 53A: display device, 302: bus, 303: train, 304: airplane

Claims

An imaging device, a display device, a feature output circuit, an image processing circuit, and a database;
The imaging device has a function of outputting imaging data,
The feature amount output circuit has a function of outputting feature amount data of the imaging data,
The database includes correction data and detection data, and has a function of outputting the correction data to the image processing circuit according to the feature amount data.
The image processing circuit has a function of generating image data by correcting the imaging data based on the correction data;
The display system has a function of performing display according to the image data.
In claim 1,
The display system has a function of selecting the detection data that matches or is similar to the feature data and outputting the correction data corresponding to the detection data to the image processing circuit.
In claim 1,
The database has a function of updating the detection data serving as a weight parameter by machine learning using the feature data as learning data, and inferring the correction data that matches or is similar to the feature data. A display system comprising:
An imaging device, a display device, a feature amount output circuit, an image processing circuit, and a transmission / reception circuit;
The imaging device has a function of acquiring imaging data,
The feature amount output circuit has a function of acquiring feature amount data of the imaging data,
The feature amount data has a function of being transmitted to a database having correction data and detection data via a transmission / reception circuit,
The image processing circuit has a function of receiving the correction data from the database via the transmission / reception circuit, and generating image data by correcting the imaging data based on the correction data,
The display device has a function of performing display according to the image data.
In claim 4,
The correction data is data corresponding to the detection data,
The moving object, wherein the detection data is data that matches or is similar to the feature data.
In claim 4,
The correction data is data obtained by inference by inputting the feature amount data in a database in which the detection data serving as a weighting parameter is updated by machine learning using the feature amount data as learning data. A moving object characterized by that.