KR102724415B1 - Method for fusion visible image and thermal image and fire-fighting lantern applied with this method - Google Patents

Abstract

The present invention relates to a method for fusion of a visual image and a thermal image, and a firefighting lantern applying the method, and more particularly, to a method for fusion of a visual image and a thermal image, and a firefighting lantern applying the method, which mounts a thermal image camera and a visual image camera on a lantern that firefighters, etc. always carry when dispatched to a fire or disaster scene, and fuses images taken by the mounted cameras based on the scene temperature and displays them in real time, thereby providing a scene where visibility is not secured due to a power outage, smoke, etc., to be clearly identified together with temperature data, and at the same time, supports the scene image to be suitable for life rescue or for judgment of a dangerous situation depending on the scene temperature.

Description

Method for fusion of visible image and thermal image and fire-fighting lantern applied with this method

The present invention relates to a method for fusion of a visual image and a thermal image, and to a fire lantern applying the method. More specifically, the present invention relates to a method for fusion of a visual image and a thermal image, and to a fire lantern applying the method, which comprises mounting a visible light camera and a thermal image camera on a lantern which firefighters and the like always carry when dispatched to a fire or disaster scene, and fusion of images captured by the mounted cameras and displaying them in real time, thereby enabling firefighters who cannot secure a field of vision due to a power outage, smoke, or the like to clearly identify a fire scene.

Firefighting activities to rescue people at fire scenes often involve artificially cutting off electricity in buildings to prevent electric shock from firefighters using firefighters to fight fires, or power outages depending on the fire situation. In addition, because of the toxic smoke generated during a fire, it is more difficult for firefighters to secure visibility, making it very difficult for firefighters to identify the location of people in need of rescue with their own eyes.

To solve these problems, various technologies have been developed recently to secure the field of vision of firefighters at fire scenes. Korean Patent No. 2069094 discloses a method of implementing and providing Human Augmented Reality to facilitate avoidance of danger and identification of rescuers by confirming the presence of objects in smoke through a lidar sensor. However, in the case of the above prior art, there is a disadvantage in that it is difficult to perform missions efficiently because each firefighter must carry separate equipment equipped with at least one lidar sensor and display.

For example, a fire lantern used to illuminate a dark fire scene to search for rescuers or other materials is essential equipment for firefighters, and they must carry it in their hands to quickly and effectively illuminate the desired location. However, if a separate terminal that implements human augmented reality is equipped to confirm the presence of objects in smoke, there is a problem that firefighters have to carry the equipment in both hands or change the equipment as needed, which interferes with the performance of their duties and hinders smooth fire extinguishing and rescue operations.

In particular, in the case of the above prior art, since high-spec hardware is required to implement human augmented reality, it is an expensive product that requires a lot of cost during production, so there is a practical problem that it is difficult for all firefighters deployed to the field to equip it, and as a result, many firefighters share one piece of equipment, making it difficult for all firefighters deployed to the field to quickly grasp the scene.

In addition, in areas where fires or toxic gases are generated, high processing speeds and high-quality image processing are required to quickly support fire scene images so that firefighters can quickly and accurately recognize the on-site situation. However, as the image quality increases, high-spec hardware is required, so there is a problem that the processing speed of objects is not satisfactory.

1. Korean Patent No. 2069094 (Registration Date: January 16, 2020) “Method for Detecting Space in Smoke Situation Using Lidar Sensor”

The present invention is to solve the problems of the above-mentioned prior art. That is, the purpose of the present invention is to provide a method for fusion of real-image and thermal images, which enables firefighters and the like to carry a lantern when dispatched to a fire or disaster scene, to fuse images captured by the mounted cameras based on the scene temperature and display them in real time, thereby providing a scene where visibility is not secured due to a power outage, smoke, etc., to be clearly identified together with temperature data, and at the same time to support the scene image to be suitable for life rescue or for assessing a dangerous situation depending on the scene temperature, and a fire lantern applying the method.

According to one embodiment of the present invention for achieving the above purpose, a method for fusion of a real-world image and a thermal image, performed by a device including a photographing device and a processor, comprises the steps of: acquiring a real-world image captured by a visible-light camera mounted on the photographing device and a thermal image captured by a thermal imaging camera, respectively; detecting a distance to a subject using a distance measuring device mounted on the photographing device, and adjusting, by a processor, a scale of the thermal image to match that of the real-world image according to the detected distance to the subject; adjusting transparency of the thermal image based on a temperature of a center point of the thermal image; and overlaying and merging the thermal image on the real-world image.

In the step of adjusting the transparency of the thermal image based on the center point temperature of the thermal image, if the center point temperature of the thermal image is lower than a preset temperature standard, the transparency of the thermal image is adjusted based on a high gain mode standard, and if the center point temperature of the thermal image is higher than a preset temperature standard, the transparency of the thermal image is adjusted based on a low gain mode standard.

The above preset temperature standard is characterized by being 100℃.

In the step of adjusting the transparency of the thermal image based on the high gain mode, the transparency is preset to a maximum value at 36℃, and as the temperature moves away from 36℃, the transparency of the thermal image is gradually reduced, and as the temperature moves away from 36℃, the thermal image is emphasized and fused compared to the real image.

In the step of adjusting the transparency of the thermal image based on the above low gain mode, the transparency becomes a preset minimum value at 100℃, and as the temperature increases based on 100℃, the transparency of the thermal image is gradually increased, and as the temperature increases based on 100℃, the real image is emphasized and fused compared to the thermal image.

A fire lantern according to the present invention comprises: a bar-shaped lantern body; an LED lighting unit provided on the upper end of the lantern body; a visible light camera provided on the front of the lantern body adjacent to the LED lighting unit; a thermal imaging camera provided on the front of the lantern body adjacent to the LED lighting unit; a rangefinder provided on the front of the lantern body adjacent to the thermal imaging camera; a display provided on the rear of the lantern body; a switch provided on the side of the lantern body; a battery provided on one side of the lantern body; and a processor configured to merge a real-world image acquired through the visible light camera and a thermal image acquired through the thermal imaging camera and display the result through the display.

The above processor is characterized by merging a real image and a thermal image by adjusting the transparency of the thermal image based on the temperature of the center point of the thermal image.

The method for fusion of a real-world image and a thermal image according to the present invention and the fire lantern applying the method have the advantage of providing a visible-light camera, a thermal-image camera, and a display on a lantern that firefighters necessarily carry on site, so that even in situations where visibility is not secured due to a power outage, smoke, etc., the on-site situation can be clearly identified together with temperature data, while reducing the weight of the entire equipment carried by firefighters, thereby increasing the efficiency of firefighting activities.

In addition, the thermal image is fused onto the real-world image by gradually adjusting the transparency of the image, so the compositing process is simple and the processing speed is fast. In addition, the real-world image that is identical to the on-site situation is displayed as is, so firefighters can clearly recognize objects in the on-site situation, enabling them to quickly search for the location of rescuers and identify risk factors.

In addition, by supporting the fusion of on-site images based on on-site temperature data to suit rescue operations or judgment of dangerous situations, it is advantageous for firefighters to quickly assess the on-site situation, thereby enabling early discovery of rescuers and assessment of risk factors.

Figure 1 is a flowchart sequentially showing a method for fusion of a real-world image and a thermal image according to an embodiment of the present invention.
Figure 2 is an exemplary diagram showing the stages of a typical compartment fire progression.
FIG. 3 is a perspective view of a fire lantern to which a method for fusion of real-world images and thermal images according to an embodiment of the present invention is applied.
Figure 4 is a schematic internal configuration diagram of the fire lantern illustrated in Figure 3.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprises" or "has" and the like are intended to specify the presence of a feature, number, step, operation, component, part or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

Unless otherwise defined herein, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and should not be interpreted in an idealized or overly formal sense, unless expressly defined otherwise herein.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

FIG. 1 is a flowchart sequentially showing a method for fusion of a real-world image and a thermal image according to one embodiment of the present invention.

As illustrated in FIG. 1, a method for merging a real-time image and a thermal image according to the present invention first acquires a real-time image through a visible-light camera, and then acquires a thermal image through a thermal image camera (S210).

A thermal image is an image that is expressed in general colors by relying on visible light, while a thermal image is an image that detects heat energy, that is, infrared heat radiation energy, rather than visible light, and is expressed in different colors depending on the temperature.

These thermal images express the captured object through the distribution of temperature emitted by the captured object. The higher the heat of the object, the redder it appears, and the lower the heat of the object, the bluer it appears. With thermal images, you can check the temperature information of the environment or object in which the image was captured.

Next, the scale of the thermal image is adjusted to match the scale of the visual image using the distance information measured by the distance meter (S220).

The rangefinder measures the distance from the thermal imaging camera to the subject being photographed, and can be composed of a distance measuring module capable of measuring the distance between a reference point and a target, such as a laser distance measuring sensor, a depth camera, or a 3D camera.

The focus and scale of a visual image captured by a visible light camera and a thermal image captured by a thermal imaging camera may differ due to differences in the mounting positions of each camera and differences in camera resolution.

Accordingly, in the present invention, the visual image is captured while the field of view (FOV) of the visible light camera is fixed, and the thermal image is captured by adjusting the scale of the thermal image to match that of the visual image using distance information between the subject and the thermal image camera acquired through a distance measuring device, thereby making the scale of the visual image and the scale of the thermal image match each other.

In this way, when synthesizing and outputting real-world images and thermal images with different focal points and scales captured by a visible-light camera and a thermal-image camera, the degree of fusion of the two images can be improved by adjusting the scale of the thermal image to match that of the real-world image.

In addition, the present invention can also correct temperature errors in thermal images by using distance information acquired through a distance measuring device.

In general, thermal imaging cameras have errors depending on the emissivity of the object being measured, the distance between the thermal imaging camera and the object, the size of the object, and the number of pixels on the camera sensor. In other words, depending on the measurement distance or the size of the object being measured, the measurement may be higher or lower than the actual temperature, resulting in a temperature error.

Accordingly, a distance compensation operation is performed on the temperature value of the thermal image using the distance information between the thermal image camera and the subject from the distance measuring device, and the distance compensation value is applied to the temperature value output in the thermal image, so that the thermal image ultimately shows an accurate temperature value with the temperature error corrected.

Next, the transparency of the thermal image is adjusted based on the temperature of the center point of the thermal image (S230).

In general, compartment fires are maintained at high temperatures as heat release accumulates over time. Figure 2 illustrates the progression stages of a compartment fire in time. As illustrated in Figure 2, a compartment fire progresses through an ignition stage in which combustion begins, a growth stage in which flames are formed over combustibles, a flashover stage in which the surfaces of all combustible objects in the compartment change to a state related to fire, a peak stage in which all combustible materials in the compartment are related to fire, and a decline stage in which the fire decreases.

Here, the flashover stage is when the temperature rises rapidly, and at this time, the temperature inside the compartment exceeds 400˚C. If the compartment door is opened during this flashover stage, oxygen may rapidly flow into the closed compartment, and if a large amount of oxygen is temporarily supplied to a compartment that is lacking oxygen or in a smoldering state, the combustion gas may ignite instantaneously, causing a phenomenon such as backdraft with strong explosive power. Therefore, firefighters must check the temperature inside the compartment before opening the door.

On the other hand, before flashover occurs, it is possible to search for rescuers, and at this time, firefighters need to be provided with field of vision information suitable for searching for people.

In the present invention, if the center point temperature of a thermal image is lower than a preset temperature standard, the transparency of the thermal image is adjusted based on a high gain mode suitable for human search, and if the center point temperature of the thermal image is higher than a preset temperature standard, the transparency of the thermal image is adjusted based on a low gain mode suitable for determining a dangerous situation such as a backdraft.

In the present invention, a preset temperature standard is set to 100℃, and when the temperature of the center point of a thermal image is 100℃ or lower, the transparency of the thermal image is adjusted in high gain mode to provide an image suitable for searching for rescuers, and when it is 100℃ or higher, the transparency of the thermal image is adjusted in low gain mode to provide an image suitable for judging a dangerous situation.

Generally, the normal body temperature of a human is in the range of 35 to 37.5℃, so in high gain mode, the transparency of the thermal image is gradually reduced as the temperature increases or decreases around 36℃, which is the human body temperature.

In other words, in high gain mode, the farther the temperature is from 36℃, the more the transparency of the thermal image is reduced, so that the thermal image information becomes more opaque as the temperature gets farther away from 36℃, emphasizing the thermal image compared to the visible image, thereby providing the convenience of universal exploration. On the other hand, the closer it gets to 36℃, the more the transparency of the thermal image is increased, so that the closer the temperature is to 36℃, the more the thermal image information becomes transparent, emphasizing the visible image compared to the thermal image, thereby improving the resolution of human life.

The thermal image transparency setting according to the thermal image center point temperature in the high gain mode according to one embodiment of the present invention can be applied as shown in [Table 1] below. At this time, the thermal image becomes most transparent when the transparency is 90%, and most opaque when the transparency is 10%.

Temperature (°C) Transparency of thermal image (%) Above 51°C 10% Above 49°C ~ Below 51°C 20% Above 47°C ~ Below 49°C 30% Above 45°C ~ Below 47°C 40% Above 43°C ~ Below 45°C 50% Above 41°C ~ Below 43°C 60% Above 39°C ~ Below 41°C 70% Above 37°C ~ Below 39°C 80% Above 35°C ~ Below 37°C 90% Above 33°C ~ Below 35°C 80% Above 31°C ~ Below 33°C 70% Above 29°C ~ Below 31°C 60% Above 27°C ~ Below 29°C 50% Above 25°C ~ Below 27°C 40% Above 23°C ~ Below 25°C 30% Above 20°C ~ Below 23°C 20% below 20°C 10%

When the temperature of the center point of a thermal image is 100℃ or higher, the present invention adjusts the transparency of the thermal image in a low gain mode to provide an image suitable for assessing a dangerous situation such as a backdraft.

Thermal images in low gain mode have lower resolution than those in high gain mode because many pixels are measured at high temperatures above 100℃. Therefore, in order to improve the readability of the image, the transparency of the thermal image is gradually increased as the temperature increases based on the preset temperature standard of 100℃.

In other words, in low gain mode, as the temperature increases from 100℃, the transparency of the thermal image is gradually increased, so that as the temperature increases from 100℃, the thermal image becomes more transparent, emphasizing the real image compared to the thermal image, and as it gets closer to 100℃, the transparency of the thermal image is reduced, so that as it gets closer to 100℃, the thermal image becomes more opaque, emphasizing the thermal image compared to the real image.

The thermal image transparency setting according to the thermal image center point temperature in the low gain mode according to one embodiment of the present invention can be applied as shown in [Table 2] below. At this time, the thermal image becomes most transparent when the transparency is 90%, and most opaque when the transparency is 10%.

Temperature (°C) Transparency of thermal image (%) 500°C or higher 90% Above 450°C ~ Below 500°C 80% Above 400°C ~ Below 450°C 70% 350°C or above ~ 400°C or below 60% 300°C or above ~ 350°C or below 50% 250°C or above ~ 300°C or below 40% 200°C or above ~ 250°C or below 30% Above 100°C ~ Below 200°C 20% below 100°C 10%

Finally, the real-world image is placed in the background, and the thermal image with adjusted transparency is covered from above to merge into a single image (S240).

As described above, the present invention provides a method for clearly identifying a fire scene where visibility is not secured due to a power outage, smoke, etc., and at the same time, supports firefighters to quickly determine the suitability of searching for rescuers and judging a dangerous situation using only the temperature data of the fused image. That is, in a temperature situation before a flashover occurs at a fire scene where visibility is not secured due to a power outage, smoke, etc., the image is processed and provided so that the presence or absence of a rescuer can be quickly and clearly distinguished, so that firefighters can effectively utilize it for rescuing rescued people at the fire scene, and in a high-temperature environment with low resolution, the real-world image is emphasized to increase the visibility of objects according to temperature, so that firefighters can effectively utilize it for judging and responding to risk factors such as obstacles at the fire scene in advance.

FIG. 3 is a perspective view of a fire lantern to which a method for fusion of a real-world image and a thermal image according to an embodiment of the present invention is applied, and FIG. 4 is a schematic internal configuration diagram of the fire lantern illustrated in FIG. 3.

As shown in FIGS. 3 and 4, a fire lantern (100) according to the present invention is configured to include an LED lighting unit (120), a photographing device (130), a display (140), a switch (150), a battery (160), and a processor (170) in a lantern body (110), and the photographing device (130) is equipped with a visible light camera (133), a thermal imaging camera (136), and a distance measuring device (139).

Accordingly, a visible light camera (133) and a thermal imaging camera (136) are attached to a fire lantern (100) that firefighters can most easily carry, thereby expanding the available range and replacing three pieces of equipment that firefighters had to carry with one piece of equipment, thereby reducing the weight of the equipment they carry. In addition, since each firefighter can now have one thermal imaging camera (136), which was previously used by many people, the efficiency of firefighters' work activities can be improved.

Looking specifically at the configuration of the fire lantern (100) according to the present invention, the fire lantern (100) is provided with a bar-shaped lantern body (110).

The upper part of the lantern body (110) is provided with an LED lighting unit (120) made up of LEDs positioned toward the front.

According to one embodiment, the LED lighting unit (120) may be provided with a main LED (122) located at the center and a plurality of auxiliary LEDs (124) radially distributed around the main LED (122) centered on the main LED (122). At this time, both the main LED and auxiliary LED of the LED lighting unit (120) are arranged facing forward, and for example, the brightness and blinking of the LED lighting can be selectively controlled by the user in three stages (High, Low, and off) of the light amount of the main LED, and in the case of the auxiliary LED, the brightness of the LED lighting can be controlled in two stages (ON, OFF) of blinking.

A photographing device (130) is provided on the front of the lantern body (110) adjacent to the LED lighting unit, and the photographing device (130) includes a visible light camera (133), a thermal imaging camera (136), and a distance measuring device (139).

A visible light camera (133) is a device that captures a real-world image expressed in general colors by relying on visible light, and a thermal imaging camera (136) is a device that detects thermal energy, i.e. infrared thermal radiation energy, rather than visible light, and captures a thermal image expressed in different colors depending on temperature.

The range finder (139) is a distance measuring module capable of measuring the distance between a reference point and a target, and is a device that measures the distance between a thermal imaging camera (136) and a subject, and may be composed of a laser distance measuring sensor, a depth camera, a 3D camera, etc.

A display (140) is provided on the rear of the lantern body (110) to display an image that is a fusion of a real-world image and a thermal image.

A switch (150) is provided on the side of the lantern body (110).

The above switch (150) is provided on one side of the lantern body (110), and the switch (150) may be a switch that can be operated by pressing. The switch may be configured in two or more forms, including a main switch and an auxiliary switch.

Accordingly, the user can clearly perform step-by-step brightness and blinking operations of the LED lighting unit (120) by simply pressing the switch (150) briefly or long pressing it for a certain period of time (e.g., 3 seconds or more) based on the tactile sensation of the fingertips in a dark and urgent rescue situation.

A battery (160) that can be arbitrarily removed from the lantern body (110) by the user is provided on one side of the lantern body (110). In addition, a battery remaining amount display unit (190) that calculates remaining amount information of the installed battery and visually displays the calculated remaining amount of the battery may be provided on the front or rear of the lantern body (110).

In addition, a motion sensor (not shown) for detecting the user's movement or vibration may be provided on one side or inside of the lantern body (110), and a buzzer (180) may be further provided to notify surrounding colleagues of the user's location when no movement or vibration is detected by the motion sensor.

The processor (170) controls each of the above devices, and fuses the real-world image acquired through the visible-light camera (133) and the thermal image acquired through the thermal image camera (136) and displays them through the display (140), thereby providing a fire scene where visibility is not secured due to a power outage, smoke, etc., so that it can be identified with the naked eye.

To this end, the processor (170) receives a visual image through a visible light camera (133), a thermal image through a thermal image camera (136), and at the same time receives distance information between the thermal image camera (136) and the subject measured through a distance measuring device (139).

The processor (170) uses the distance information received from the distance measuring device (139) to adjust the scale of the thermal image to match the scale of the real image.

As explained above, the focus and scale of the real-world image captured by the visible-light camera (133) and the thermal image captured by the thermal image camera (136) may differ due to differences in the positions where each camera is mounted and differences in the resolution of the cameras.

Accordingly, the processor (170) adjusts the scale of the thermal image to match the scale of the real image by using the real image captured and transmitted while the field of view (FOV) of the visible light camera (133) is fixed and the distance information between the thermal image camera (136) and the subject transmitted through the distance measuring device (139), thereby making the scale of the real image and the scale of the thermal image match.

Additionally, the processor (170) can also correct the temperature error of the thermal image using the distance information transmitted through the distance measuring device (139).

In general, thermal imaging cameras (136) have errors depending on the emissivity of the object to be measured, the distance between the thermal imaging camera and the object, the size of the object, and the number of pixels of the camera sensor. In other words, depending on the measurement distance or the size of the object to be measured, a measurement that is higher or lower than the actual temperature is made, resulting in a temperature error.

Accordingly, the processor (170) performs a distance compensation operation on the temperature value of the thermal image using the distance information between the thermal image camera (136) and the subject transmitted from the distance measuring device (139) and applies the distance compensation value to the temperature value output in the thermal image, thereby finally displaying an accurate temperature value with the temperature difference corrected in the thermal image.

In particular, the processor (170) according to the present invention adjusts the transparency of the thermal image based on the temperature of the center point of the thermal image, places the real image as the background, and covers the thermal image with the adjusted transparency from above to merge it into one image.

That is, the processor (170) adjusts the transparency of the thermal image based on a high gain mode suitable for human search when the center point temperature of the thermal image is lower than a preset temperature standard, and adjusts the transparency of the thermal image based on a low gain mode suitable for determining a dangerous situation such as a backdraft when the center point temperature of the thermal image is higher than a preset temperature standard.

In the present invention, a preset temperature standard is set to 100℃, and when the temperature of the center point of a thermal image is 100℃ or lower, the transparency of the thermal image is adjusted in high gain mode to provide an image suitable for searching for rescuers, and when it is 100℃ or higher, the transparency of the thermal image is adjusted in low gain mode to provide an image suitable for judging a dangerous situation.

In high gain mode, the farther the temperature gets from 36℃, which is the body temperature of a person, the less transparent the thermal image becomes, so that the thermal image becomes more opaque as the temperature gets farther away from 36℃, emphasizing the thermal image compared to the real image, thereby providing the convenience of universal exploration. On the other hand, the closer it gets to 36℃, the more transparent the thermal image becomes, so that the real image becomes more transparent as the temperature gets closer to 36℃, thereby enhancing the resolution of human life.

In low gain mode, the higher the temperature is from 100℃, the more transparent the thermal image becomes, emphasizing the real image compared to the thermal image. As the temperature approaches 100℃, the transparency of the thermal image decreases, emphasizing the real image compared to the thermal image. As the temperature approaches 100℃, the transparency of the thermal image decreases, emphasizing the real image compared to the thermal image.

Accordingly, the processor (170) can provide a clear identification of a fire scene where visibility is not secured due to a power outage, smoke, etc., and can also support firefighters to quickly determine the suitability of searching for rescuers and judging dangerous situations using only the temperature data of the fused image.

Additionally, the processor (170) receives an operation signal from the switch (150) and controls the LED lighting unit (120).

In one embodiment of the present invention, the processor (170) can set the main LED (122) to 1-stage LED High (high brightness) in response to a signal for pressing the switch (150) once briefly, set the main LED (122) to 2-stage LED Low (low brightness) in response to a signal for pressing the switch (150) twice in succession, and set the main LED (122) to 3-stage LED OFF (main LED off) in response to a signal for pressing the switch three times in succession.

Meanwhile, in the case of a long press operation signal for a certain period of time (e.g., 3 seconds or more) rather than a short press operation signal of the switch (150), the processor (170) can turn off the main LED (122) and turn off all power of the fire lantern (100) of the present invention, including the auxiliary LED (124), the photographing device (130), and the display (140).

In addition, the processor (170) controls the flashing of the auxiliary LED (124) in multiple stages according to the number of inputs of a separately provided auxiliary switch (not shown), and when the input time of the auxiliary switch is operated for a certain period of time (e.g., 3 seconds or more), it can be switched to 'alarm mode'.

That is, the processor (170) can set the auxiliary LED (124) to 1-stage LED ON (auxiliary LED turned on) according to a signal for briefly pressing the auxiliary switch once, and can set the auxiliary LED (124) to 2-stage LED OFF (auxiliary LED turned off) according to a signal for pressing the auxiliary switch twice in succession.

Meanwhile, the processor (170) can switch the fire lantern (100) of the present invention to 'alarm mode' in the case of a long-press operation signal for a certain period of time (e.g., 3 seconds or more) rather than a short-press operation signal of the auxiliary switch.

After switching to 'alarm mode', if no movement or vibration is detected by the motion sensor of the fire lantern (100) for a certain period of time (e.g., 10 seconds or more), the processor (170) determines that the user's condition is abnormal and generates an alarm sound through the buzzer (180).

According to one embodiment, the alarm sound generated through the buzzer (180) can have a frequency that varies in the range of 2000 to 3000 Hz over time so that colleagues in the vicinity can better recognize it, and at the same time, the auxiliary LED (124) can be set to blink repeatedly over time.

As described above, the fire lantern (100) according to the present invention is equipped with a visible light camera (133), a thermal imaging camera (136), and a display (140) on a lantern that firefighters necessarily carry on site, so that even in situations where visibility is not secured due to a power outage, smoke, etc., the on-site situation can be clearly identified along with temperature data, and at the same time, the weight of the entire equipment carried by firefighters is reduced, thereby increasing the efficiency of firefighting activities.

In addition, the images are fused by gradually adjusting the transparency of the thermal image over the real-world image, so the compositing process is simple and the processing speed is fast. In addition, the real-world image that is identical to the on-site situation is displayed as is, so firefighters can clearly recognize objects in the on-site situation, enabling them to quickly search for the location of rescuers and identify risk factors.

In addition, by supporting the fusion of on-site images based on on-site temperature data to suit rescue operations or judgment of dangerous situations, firefighters can quickly assess the on-site situation, which is advantageous for early detection of rescuers and judgment of risk factors.

In addition, the present invention is not limited to only the above-described embodiment, and the same effect can be created even when the detailed configuration, number, and arrangement structure of the device are changed. Therefore, it is stated that those with ordinary skill in the art can add, delete, and modify various configurations within the scope of the technical idea of the present invention.

100 : Fire Lantern 110 : Lantern Body
120: LED lighting section 122: Main LED
124 : Auxiliary LED 130 : Shooting device
133 : Visible light camera 136 : Thermal imaging camera
139 : Distance Meter 140 : Display
150 : Switch 160 : Battery
170 : Processor 180 : Buzzer
190: Battery level indicator

Claims (16)

A method for fusion of a visual image and a thermal image performed by a device including a photographing device and a processor,
A step of acquiring a visual image captured by a visible light camera mounted on a shooting device and a thermal image captured by a thermal imaging camera, respectively;
A step of detecting a distance to a subject using a range finder mounted on a photographing device, and a processor adjusting the scale of the thermal image to match the real image according to the detected distance to the subject;
The transparency of the thermal image is adjusted based on the temperature of the center point of the thermal image. However, if the temperature of the center point of the thermal image is lower than the preset temperature standard, the transparency of the thermal image is adjusted based on the high gain mode in which the real image is emphasized and fused compared to the thermal image, centered on the human body temperature range of 35 to 37℃.
A step of adjusting the transparency of the thermal image based on a low gain mode criterion in which the transparency of the thermal image increases as the center point temperature of the thermal image increases if the center point temperature of the thermal image is higher than a preset temperature criterion; and
A method for fusion of a real-world image and a thermal image, comprising the step of covering and fusion of the thermal image over the real-world image.
delete In paragraph 1,
A method for fusion of a visual image and a thermal image, characterized in that the preset temperature standard is 100℃.
In paragraph 1,
In the step of adjusting the transparency of the thermal image based on the above high gain mode,
A method for merging a thermal image and a real image, characterized in that when the center point temperature of the thermal image is 35 to 37℃, the transparency of the thermal image is adjusted to a preset maximum value, so that the thermal image information becomes transparent, thereby emphasizing and merging the real image compared to the thermal image.
In paragraph 1,
In the step of adjusting the transparency of the thermal image based on the above high gain mode,
At 36℃, the transparency reaches its preset maximum value,
A method for merging a real-world image and a thermal image, characterized in that the transparency of a thermal image is gradually reduced as the temperature increases away from 36℃, and the thermal image information becomes gradually more opaque as the temperature increases away from 36℃, thereby emphasizing and merging the thermal image compared to the real-world image.
In paragraph 1,
In the step of adjusting the transparency of the thermal image based on the above low gain mode,
At 100℃, the transparency reaches the preset minimum value,
A method for merging a real-world image and a thermal image, characterized in that as the temperature increases from 100℃, the transparency of the thermal image is gradually increased, and as the temperature increases from 100℃, the thermal image becomes gradually more transparent, thereby emphasizing and merging the real-world image compared to the thermal image.
Bar-shaped lantern body;
LED lighting unit provided on the top of the lantern body;
A visible light camera provided on the front of the lantern body adjacent to the LED lighting unit;
A thermal imaging camera provided on the front of the lantern body adjacent to the LED lighting unit;
A rangefinder provided on the front of the lantern body adjacent to the thermal imaging camera;
A display provided on the rear of the above lantern body;
A switch provided on the side of the above lantern body;
A battery provided on one side of the above lantern body; and
A processor that merges a visual image acquired through the visible light camera and a thermal image acquired through the thermal image camera and displays the result through the display;
Consisting of including,
The above processor,
Adjust the transparency of the thermal image based on the center point temperature of the thermal image.
If the center point temperature of the above thermal image is lower than the preset temperature standard, the transparency of the thermal image is adjusted based on the high gain mode standard in which the real image is emphasized and fused compared to the thermal image, centered on the human body temperature range of 35 to 37℃.
A fire lantern characterized in that the transparency of the thermal image is adjusted based on a low gain mode criterion in which the transparency of the thermal image increases as the center point temperature of the thermal image increases when the center point temperature of the thermal image is higher than a preset temperature standard.
In Article 7,
The above processor,
A fire lantern characterized in that the distance to a subject is detected using the above range finder, and the scale of the thermal image is adjusted to match the real image according to the distance to the detected subject.
delete delete In Article 7,
A fire lantern, characterized in that the preset temperature standard is 100℃.
In Article 7,
When adjusting the transparency of the thermal image based on the above high gain mode,
A fire lantern characterized in that when the center point temperature of the thermal image is 35 to 37℃, the transparency of the thermal image is adjusted to a preset maximum value, so that the thermal image information becomes transparent and the real image is emphasized and fused compared to the thermal image.
In Article 7,
When adjusting the transparency of the thermal image based on the above high gain mode,
At 36℃, the transparency reaches its preset maximum value,
A fire lantern characterized by gradually reducing the transparency of a thermal image as the temperature increases away from 36℃, thereby emphasizing and merging the thermal image compared to the real image as the thermal image information becomes more opaque as the temperature increases away from 36℃.
In Article 7,
When adjusting the transparency of the thermal image based on the above low gain mode,
At 100℃, the transparency reaches the preset minimum value,
A fire lantern characterized by gradually increasing the transparency of a thermal image as the temperature rises from 100℃, and by gradually increasing the transparency of a thermal image as the temperature rises from 100℃, thereby emphasizing and fusing a real image compared to a thermal image.
In Article 7,
A fire lantern, characterized in that the above fire lantern further includes a motion sensor and a buzzer.
In Article 7,
The above LED lighting part,
A fire lantern characterized by having a main LED as the center and a plurality of auxiliary LEDs radially distributed around the main LED.