CN104977590A - Optical device and method for detecting surrounding environment - Google Patents

Abstract

An optical device and a method for detecting the surrounding environment. The optical device is used for detecting a surrounding environment and comprises an image extraction unit, an image processing unit and an analysis unit. The image extraction unit is used for emitting a light ray to the surrounding environment in a scanning mode and generating a plurality of depth images corresponding to different time points according to the light ray reflected by the surrounding environment. The image processing unit is used for generating the speed of an object at different time points according to the depth images and predicting the future speed of the object according to the previous speed change situation of the object. The analysis unit is used for generating related warning messages according to the previous speed change situation and the future speed of the object.

Description

Optical device and method for detecting surrounding environment

technical field

The present invention relates to an optical device, and in particular to an optical device for detecting surrounding environment.

Background technique

Cycling activities have become popular in recent years. Traveling or exercising leisurely and comfortably by bicycle is an activity that is widely loved by the public. However, there are also many potential dangers in cycling activities. Since the body of the cyclist is not protected by the external metal body, once an accident occurs, it often causes serious consequences. The main cause of accidents often comes from the cyclist's inattention, failing to notice changes in the surrounding environment in time, and the cyclist's vision is limited, unable to take into account possible dangers in all directions.

Therefore, how to effectively detect the surrounding environment so as to provide higher safety for cyclists is one of the topics that the industry is currently working on.

Contents of the invention

The invention relates to an optical device and a method for detecting the surrounding environment.

According to a first aspect of the present invention, an optical device is provided for detecting a surrounding environment, and the optical device includes an image extraction unit, an image processing unit, and an analysis unit. The image capturing unit is used to emit a light beam to the surrounding environment in a scanning manner, and the image capturing unit generates multiple depth images corresponding to different time points according to the light reflected by the surrounding environment. The image processing unit is used to generate the speed of an object at different time points according to the depth images, and predict the future speed of the object according to the previous speed change of the object. The analysis unit is used for generating relevant warning messages according to the previous speed change and the future speed of the object.

According to another aspect of the present invention, a method for detecting the surrounding environment is proposed, which is applied to an optical device. The method includes the following steps: emit a light, and shoot it to the surrounding environment in a scanning manner; Multiple depth images corresponding to different time points; according to these depth images, the speed of an object at different time points is generated, and the future speed of the object is predicted according to the previous speed change of the object; and according to the previous speed change of the object According to the situation and future speed, relevant warning messages will be generated.

In order to have a better understanding of the above-mentioned and other aspects of the present invention, the preferred embodiments are specifically cited below, together with the accompanying drawings, and are described in detail as follows:

Description of drawings

FIG. 1 shows a schematic diagram of an optical device according to an embodiment of the present invention

FIG. 2 is a schematic diagram of an image extraction unit according to an embodiment of the present invention.

FIG. 3 shows waveforms of laser emission and light detection.

FIG. 4 is a schematic diagram of an optical device according to another embodiment of the present invention.

FIG. 5 is a schematic diagram of an example of practical use of the optical device.

FIG. 6 is a flowchart of a method for detecting surrounding environment according to an embodiment of the present invention.

【Symbol Description】

1, 2: Optical device

10: Image extraction unit

12: Image processing unit

14: Analysis unit

31~34: Steps

100: MEMS

102: Infrared laser transmitter

104: Infrared light detector

160, 161, 162: Visible laser light source

B: mobile body

D: depth image

M: warning message

P: projected image

Y: light

Y': reflected light

Detailed ways

FIG. 1 is a schematic diagram of an optical device according to an embodiment of the present invention. The optical device 1 comprises an image capture unit 10 , an image processing unit 12 and an analysis unit 14 . The image extraction unit 10 is used to emit a light ray Y, which is emitted to the surrounding environment in a scanning manner, and the surrounding environment reflects the light ray Y, and the image extraction unit 10 receives the reflected light ray Y' to generate multiple depth images D corresponding to different time points . The image processing unit 12 is used to generate the velocity of an object at different time points according to the depth images D, and predict the future velocity of the object according to the previous velocity variation of the object. The analysis unit 14 is used for generating relevant warning messages M according to the previous speed change and the future speed of the object. Now, the functions of each component will be described in detail as follows.

FIG. 2 is a schematic diagram of an image extraction unit according to an embodiment of the present invention. The image extraction unit 10 is used for generating a plurality of depth images D corresponding to different time points of the surrounding environment. The image capture unit 10 includes a MEMS 100 , an infrared laser emitter 102 and an infrared photodetector 104 . The infrared laser emitter 102 is used for emitting light Y, and the light Y is infrared light. The MEMS 100 makes the light Y scan the surrounding environment.

The MEMS 100 includes, for example, a micro-scanning mirror. The light Y emitted by the infrared laser emitter 102 hits the scanning mirror, and is reflected so that the light Y is emitted to the surrounding environment. The MEMS 100 drives the scanning mirror to rotate at different angles, so that the emitted light Y scans a plane in the surrounding environment. As shown in FIG. 2 , the MEMS 100 enables the light Y to scan a plane in the surrounding environment column by column in a scanning manner.

When the light Y is emitted to the surrounding environment, it will be reflected by the objects in the surrounding environment. The infrared light detector 104 is used to receive the reflected light Y', and calculate the time difference of each pixel in the image according to the time difference between the infrared light emitting and being received. "Depth" refers to the distance d between the object in the environment and the optical device 1 . The formula for calculating the distance d is as follows:

Where c represents the speed of light (3×10 ⁸ m/s), t _receiver represents the time when the infrared light detector 104 receives the reflected light Y′, and t _emitter represents the time when the infrared laser emitter 102 emits the light Y'.

FIG. 3 shows waveforms of laser emission and light detection. As shown in FIG. 3 , laser emission represents the pulse waveform of the infrared laser emitter 102 , light detection represents the light pulse waveform sensed by the infrared light detector 104 , and there is a clock signal oscillating at a fixed frequency in the system. The time difference (Δt) between the laser emission pulse and the light detection pulse can be calculated according to the clock signal, and the distance between the object in the image and the optical device 1 can be obtained according to the time difference.

The depth of the current pixel can be known according to the distance, wherein the position of the current pixel can be known according to the control of the MEMS 100 . When the infrared light emitted by the infrared laser emitter 102 scans a plane, the infrared light detector 104 can generate a depth image D to record the depth of each pixel in the image. Since the image capturing unit 10 repeatedly scans the surrounding environment, multiple depth images D corresponding to different time points can be generated.

In an implementation manner, 55 depth images D are generated every second, and the resolution of each depth image D is 90×70 pixels. However, the present disclosure is not limited thereto, as long as a depth image with sufficient information can be generated, the actual frame rate and image resolution will depend on design requirements and actual hardware limitations.

The image processing unit 12 can calculate the velocity of the object at different time points according to these depth images D. Specifically, these depth images D represent the depth of each pixel in the picture at different time points, and the speed can be calculated according to the depth and time in the picture at different time points. The image processing unit 12 also records the previous speed change of the object, and predicts the future speed of the object according to the previous speed change of the object. For example, if the calculated speed at the previous time point t1 is v1, and the calculated speed at the previous time point t2 is v2, then the speed v3 at the future time point t3 is predicted according to the speed v1 and the speed v2.

In one implementation, the image processing unit 12 calculates the future velocity v3 of the object according to the weighted sum of the first previous velocity v1 and the second previous velocity v2 of the object, that is, v3=w1×v1+w2×v2. The sum of the weight coefficients w1 and w2 is equal to 1. Therefore, the future velocity v3 is directly related to the first previous velocity v1 and the second previous velocity v2, and the image processing unit 12 predicts the future velocity of the object according to the previous movement trend of the object.

The weight coefficients w1 and w2 can also be adjusted dynamically. Continuing from the above example, the future velocity at the predicted calculation time point t3 is v3; and at the actual time point t3, the velocity value obtained by the image processing unit 12 according to the actual measurement and calculation of the depth image D is v3'. The image processing unit 12 uses the difference between the predicted future speed v3 and the actually calculated speed v3' as a reference basis for adjusting the weight coefficients w1 and w2. In this way, the image processing unit 12 has the capability of correcting and learning, and can gradually adjust the weight coefficients w1 and w2, so that the prediction of the future speed v3 can be more accurate.

The analysis unit 14 generates a relevant warning message M according to the previous speed variation and the predicted future speed of the object output by the image processing unit 12 . For example, the optical device 1 is configured on a mobile body B, such as in front of a bicycle or motorcycle, or on a cyclist, for example, the cyclist wears the optical device 1 on his head. The analysis unit 14 can predict the possible trajectory of the object in the environment according to the previous speed change of the object and the predicted future speed, and because the optical device 1 is configured on the mobile body B, it can know the position of the object in the environment relative to the mobile body B. Therefore, the time and probability of collision between objects in the environment and the mobile body B can be predicted.

When the analysis unit 14 judges that there is danger according to the collision time and probability of objects in the environment and the mobile body B, the analysis unit 14 can also determine the direction in which the mobile body B can dodge. At this moment, the analysis unit 14 will send out a warning message M, and the warning message M includes the information of the direction in which the mobile body B can dodge determined by the analysis unit 14 . For example, when there is a locomotive approaching rapidly in front of the rider on the right side, the analysis unit 14 predicts that there will be a high probability of collision between the locomotive and the bicycle in a short period of time. According to the speed direction of the oncoming vehicle, the analysis unit 14 decides to dodge to the left , a warning message M is issued, informing the cyclist to dodge to the left immediately.

The analysis unit 14 can also search for the direction of the nearby open space in the process of determining the direction in which the mobile body B can dodge, so as to help determine a better dodge direction and further improve safety. Searching for an open space can also be based on the depth image D to find out the direction with fewer obstacles in the environment and a lower speed of the object.

FIG. 4 is a schematic diagram of an optical device according to another embodiment of the present invention. The optical device 2 in this embodiment is different from the previous embodiment in that the optical device 2 further includes a plurality of visible laser light sources 160 , 161 , 162 . The MEMS 100 generates a projected image P by scanning the light emitted by a plurality of visible light laser sources 160 , 161 , 162 . The MEMS 100 is configured in the image capture unit 10 , that is, the infrared laser emitter 102 and a plurality of visible laser light sources 160 , 161 , 162 share the MEMS 100 .

For example, the visible laser light source 160 can be a red laser light source, the visible light laser light source 161 can be a green laser light source, the visible light laser light source 162 can be a blue laser light source, and the red, The combined green and blue light beams and the visible light beams can be reflected into the space by the scanning mirror of the MEMS 100 to scan to form a projected image P. The projected image P may include, for example, a map of the surrounding environment, road conditions, weather, current driving speed, or related driving information.

FIG. 5 is a schematic diagram of an example of practical use of the optical device. In this example, the mobile body B is a cyclist, and the optical device 2 is worn on the head of the cyclist, that is, the optical device 2 is worn in the same way as wearing glasses. Within the visible range of the cyclist, the optical device 2 can project a projected image P to display road condition information. The cyclist does not need to lower his head, as long as he maintains the normal viewing angle when riding a bicycle, he can see the projected image P to ensure safety . When the analysis unit 14 of the optical device 2 detects that an object is approaching or dangerous, it sends out a warning message M to remind the cyclist. The warning message M includes, for example, the direction and speed of the approaching vehicle, and the direction that the cyclist can avoid. The warning message M can be displayed in the projected image P so that the cyclist can directly see it. Or the warning message M can also be a voice message to directly remind the cyclist of the danger information, so that the cyclist can detect the danger in the shortest time and make an immediate response.

The above examples illustrate the practical application of the optical device of the present disclosure. Of course, the optical device of the present disclosure is not limited to being worn on a cyclist. Since the warning message can be a voice message, the optical device of the present disclosure can also be worn on a blind person, and can inform the blind person of information about the surrounding environment, and can dodge The direction of the object can even tell the speed of the object's approach, the time and the probability of collision.

The invention also proposes a method for detecting the surrounding environment, which is applied to an optical device. Please refer to FIG. 6 , which shows a flowchart of a method for detecting surrounding environment according to an embodiment of the present invention, including the following steps. Firstly, step 31 is executed to emit a light beam to the surrounding environment in a scanning manner. Next, step 32 is executed to generate a plurality of depth images corresponding to different time points according to the light reflected by the surrounding environment. Then, enter step 33, generate the speed of an object at different time points according to the multiple depth images, and predict the future speed of the object according to the previous speed change of the object. Then, step 34 is executed to generate a relevant warning message according to the previous velocity variation of the object and the predicted future velocity.

The order of the steps in FIG. 6 is only for illustration, and the order is not fixed. Depending on the surrounding environment, as well as the scanning image resolution and frame rate, the steps may overlap each other, that is, multiple steps may be executed at the same time, or the same step may be repeated many times before entering the next step. For example, when the danger level of the surrounding environment is very low, steps 31-33 may be repeated several times before entering step 34 .

In the above-mentioned optical device and the method for detecting the surrounding environment in the embodiment of the present invention, because not only the current position and speed of the surrounding environment objects can be known, but also the future speed of the surrounding environment objects can be predicted, and the time of collision can be predicted and probability, so danger can be detected early.

When worn on the human body or configured on a vehicle, it can automatically pay attention to changes in the surrounding environment. Since it scans the environment completely and continuously, it can make up for the lack of human's limited vision and effectively solve the inevitable distraction of human beings. And lack of concentration. In addition, based on the extracted images, the optical device is more able to predict possible upcoming accidents in advance, and can actively prompt the direction that should be avoided. The optical device provides clear instructions, which can overcome the problem that human beings tend to panic and cannot think in emergency situations. Provides higher security.

In addition, multiple optical devices can also be used. Each optical device detects different directions of the surrounding environment. , the device directly sends a warning message, or after the devices communicate and integrate with each other, an image is projected from the optical device in front of the field of view to provide a warning message.

In summary, although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. Those skilled in the art to which the present invention belongs may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention should be defined by the appended claims.

Claims (20)

1. An optical device for detecting a surrounding environment, comprising: An image extraction unit is used to emit a light, and the light is directed to the surrounding environment in a scanning manner, and the image extraction unit generates a plurality of depth images corresponding to different time points according to the light reflected by the surrounding environment; An image processing unit, used to generate the speed of an object at different time points according to the depth images, and predict a future speed of the object according to a previous speed change of the object; and An analysis unit is used for generating a related warning message according to the previous speed change and the future speed of the object. 2. The optical device according to claim 1, wherein the image extraction unit comprises: an infrared laser transmitter for emitting the light, the light is an infrared light; a MEMS for enabling the infrared light to scan the surrounding environment; and An infrared light detector is used to receive the infrared light reflected from the object in the surrounding environment at different time points, and obtain the depth images according to the time difference between the infrared light being emitted and received. 3. The optical device as claimed in claim 2, further comprising a plurality of visible light laser sources, and the micro-electromechanical system enables the light emitted by the visible light laser sources to generate a projected image in a scanning manner. 4. The optical device as claimed in claim 3, wherein the warning message is displayed in the projected image. 5. The optical device as claimed in claim 1, wherein the warning message is a voice message. 6. The optical device as claimed in claim 1, wherein the image processing unit calculates the future velocity of the object according to a weighted sum of a first previous velocity and a second previous velocity of the object. 7. The optical device as claimed in claim 1, wherein the optical device is configured on a mobile body, and the analysis unit predicts that the object collides with the mobile body according to the previous velocity change and the future velocity of the object time and probability. 8. The optical device as claimed in claim 7, wherein the analysis unit further determines a direction in which the mobile body can dodge, and the warning message includes information on the determined direction in which the mobile body can dodge. 9. The optical device as claimed in claim 8, wherein the analysis unit is also used to search the direction of the nearby empty space to determine the direction in which the moving body can avoid. 10. The optical device of claim 1, wherein the optical device is a wearable optical device. 11. A method of detecting surrounding environment, applied to an optical device, comprising: emit a light, and the light is directed towards the surrounding environment in a scanning manner; generating a plurality of depth images corresponding to different time points according to the light reflected by the surrounding environment; Generate the velocity of an object at different time points according to the depth images, and predict a future velocity of the object according to a previous velocity change of the object; and According to the previous speed variation and the future speed of the object, a relevant warning message is generated. 12. The method of claim 11, wherein the step of generating the depth images comprises: emitting the light from an infrared laser emitter, the light being an infrared light; scanning the surrounding environment with a MEMS; and An infrared light detector is used to receive the infrared light reflected from the object in the surrounding environment at different time points, and the depth images are obtained according to the time difference between when the infrared light is emitted and received. 13. The method as claimed in claim 12, further comprising generating a projected image by using the MEMS to scan the light emitted by the plurality of visible light laser sources. 14. The method of claim 13, wherein the warning message is displayed in the projected image. 15. The optical device as claimed in claim 11, wherein the warning message is a voice message. 16. The method as claimed in claim 11, wherein in the step of predicting the future velocity of the object, the object is calculated based on a weighted sum of a first previous velocity and a second previous velocity of the object future speed. 17. The method as claimed in claim 11, wherein the optical device is configured on a mobile body, and the method further comprises predicting the collision time between the object and the mobile body according to the previous speed variation and the future speed of the object time and chance. 18. The method according to claim 17, further comprising determining a direction in which the mobile body can dodge, and the warning message includes information on the determined direction in which the mobile body can dodge. 19. The method as claimed in claim 18, further comprising searching the direction of nearby open space to determine the direction in which the mobile body can dodge. 20. The method of claim 11, wherein the optical device is a wearable optical device.