DE102023212381A1 - COMPUTER-IMPLEMENTED METHOD FOR DETECTING LIVING BEINGS IN A STATIONARY VEHICLE - Google Patents

Abstract

A computer-implemented method 100 for detecting living beings in a stationary vehicle 142 comprises the following steps: - receiving 102 data from an inertial sensor 140, characterized by a time-dependent amplitude 112, 114; - determining 104 at least one dominant frequency 118, 120 from the time-dependent amplitude 118; - providing 106 reference values 122, 124 such that they indicate the presence or absence of living beings in the vehicle 142; and - comparison 108 of the at least one dominant frequency 118,120 with the reference values 122,124 to determine whether a living being is located in the vehicle 142, and, if the comparison 108 indicates a living being in the vehicle 142: - generation and output 110 of a signal based on the comparison 108 of the at least one dominant frequency 118,120 with the reference values 122,124.

Description

Technical area

The present invention relates to the presence detection of living beings in a vehicle. In particular, the present invention relates to a computer-implemented method for detecting living beings in a stationary vehicle. Furthermore, the present invention relates to a data processing device, a computer program product, a computer-readable storage medium, a data carrier signal, and a vehicle.

Technical background and task

Children, animals, or adults with physical or mental disabilities are exposed to particular health risks when left unattended in a closed vehicle. Excessive exposure to sunlight on a closed vehicle increases the risk of overheating, especially for infants, which can lead to death in the worst case scenario. Efforts to minimize associated risks are being promoted, for example, by Euro NCAP (European New Car Assessment Programme).

The methods and systems disclosed in the prior art for detecting people or animals in a vehicle typically operate based on radar sensors, infrared sensors, or utilize ultra-wideband (UWB) technology to detect living beings. However, these methods and systems have the disadvantage that sensors specifically for this purpose must be installed in the vehicle interior. This generates high costs and also increases the complexity of development and production.

It is therefore the object of the present invention to provide a computer-implemented method for detecting living beings in a stationary vehicle that eliminates the aforementioned disadvantages. Furthermore, it is the object of the invention to provide a corresponding data processing device, a computer program product, a computer-readable storage medium, a data carrier signal, and a vehicle.

Disclosure of the invention

The object is achieved according to the invention by a computer-implemented method and a computer program product, a computer-readable storage medium, a data carrier signal, a data processing device, and a vehicle according to the respective main claims. Advantageous embodiments can be found in the subclaims.

The invention is based on the finding that living beings, through their movements (e.g., movement of the chest during breathing), cause a stationary vehicle to oscillate periodically. As a system capable of oscillation, a car does indeed have eigenmodes. However, these modes can be shifted in their dominant frequency by vibrations caused by living beings. This shift can then be used to draw conclusions about the presence of living beings in the vehicle.

According to a first aspect of the invention, a computer-implemented method for detecting living beings in a stationary vehicle comprises a step of receiving data from an inertial sensor, said data being characterized by a time-dependent amplitude. The time-dependent amplitude can correspond, for example, to an angular velocity or an acceleration. A stationary vehicle is understood to be a vehicle in which vibrations caused by internal vehicle components are largely absent. A vehicle with an internal combustion engine cannot therefore be stationary as long as the engine is switched on. However, a vehicle with an electric motor can also be stationary with the engine activated.

Preferably, the initial sensor is designed as an acceleration sensor and/or yaw rate sensor. This design has the advantage that corresponding components are inexpensive and reliable, for example, in the form of a MEMS (micro-electro-mechanical system) 3-axis acceleration sensor. The inertial sensor can also consist of a plurality of sensors that work together to provide data in the form of a time-dependent amplitude.

Furthermore, in the computer-implemented method according to the first aspect of the invention, at least one dominant frequency is determined from the time-dependent amplitude. In the simplest case, a frequency-dependent amplitude is first generated via a Fourier transform of the time-dependent amplitude. A dominant frequency can then be identified in the frequency-dependent amplitude as the frequency at which a local maximum can be observed.

Advantageously, the frequency-dependent amplitude is determined by a discrete Fourier transform or a fast Fourier transform These are common and proven numerical methods for calculating the corresponding spectrum from a time series.

In a next step of the computer-implemented method according to the first aspect of the invention, reference values are provided. These reference values are selected such that they indicate or characterize the presence or absence of a living being in the vehicle. A reference value can be implemented, for example, by an interval or a threshold value.

The reference values can be selected based on empirical values, for example, acquired through series of measurements. The dominant frequencies of the natural vibrations of a stationary vehicle are known in advance. However, they shift to lower frequencies when additional weight, such as a living being, is present in the vehicle. The presence of a dominant frequency in a certain interval can therefore indicate the presence of a living being.

Multiple reference values can also be selected in such a way that conclusions can be drawn about the type of living being. This is based on the knowledge that the shift in the dominant frequency of the natural vibrations of the stationary vehicle depends on the weight. The more weight is added to the vehicle, the lower the dominant frequency will be. Since adults are heavier than, for example, children or smaller animals, the shift can be used to draw conclusions about whether an adult or a child is the cause of the shift. Furthermore, the vehicle appears to be empty when no shift occurs. The corresponding frequency intervals, which enable a classification of the measured dominant frequency to species of living being, can be found, for example, through a series of measurements.

In a further step of the computer-implemented method according to the first aspect of the invention, a comparison of the at least one dominant frequency with the reference values is performed. The comparison provides information about whether a living being is present in the vehicle and what type of living being it is.

Since the oscillation frequency decreases with the increasing weight of the entire system, consisting of living beings and vehicle, it is also possible to determine the number of living beings in the vehicle. Such detection can be ensured by calibrating the inertial sensor accordingly.

If the comparison of the at least one dominant frequency with the reference values indicates the presence of a living being in the vehicle, the computer-implemented method according to the first aspect of the invention comprises a further step in which a signal is generated and output based on the comparison. This signal can be used for multiple purposes.

Preferably, a vehicle function is executed based on the signal to warn a vehicle occupant. This can, for example, consist of a visual, haptic, or acoustic warning being issued inside or outside the vehicle. For example, the vehicle horn is activated, a warning light in the vehicle begins to illuminate, or the vehicle's hazard lights are activated. Furthermore, it is possible for a web service to be started via the on-board network in order to contact and warn the driver of the vehicle via their mobile phone, for example. Some types of warning, such as the automatic activation of the vehicle horn, are also suitable for informing bystanders about the presence of a living being in the vehicle. This can be particularly helpful if a vehicle occupant is either not directly reachable or does not have access to the vehicle, for example because, as a passenger, they do not have a key.

The vehicle function can be tailored to the type of living being detected. For example, if a child is detected in the vehicle, the warning can be more attentive than if an animal is detected.

In a preferred embodiment, the reference values are determined by a trained neural network. The reference values, which serve to classify or assign the dominant frequencies to different living beings, are thus determined by a correspondingly trained neural network.

Several methods from the field of machine learning are known that are suitable for classifying the dominant frequency into reference values. Examples include a support vector machine (SVM), logistic regression, or a random forest.

Suitable training data can be provided, for example, by the FFT-transformed time-dependent amplitude of a vehicle's inertial sensor. The data should allow for the correct classification, for example, whether an adult, a child, an animal, or even an adult and a child are in the vehicle. The data can also be further processed, for example, by using a bandpass filter or averaging using different window widths.

According to a second aspect of the invention, a computer program product comprises instructions which, when executed by a computer, cause the computer to carry out a computer-implemented method as described above.

These commands can be implemented in a programming language such as C, C++ or Python.

According to a third aspect of the invention, a computer-readable storage medium comprises instructions which, when executed by a computer, cause the computer to perform a computer-implemented method as described above.

Such a computer-readable storage medium can, for example, be a form of RAM (Random Access Memory), flash memory or an SSD (Solid State Drive).

According to a fourth aspect of the invention, a data carrier signal transmits the computer program product described above. The data carrier signal can be transmitted either wirelessly or via wired systems such as a CAN bus (Controller Area Network).

According to a fifth aspect of the invention, a data processing device for detecting living beings in a stationary vehicle comprises a processor configured to perform the steps of a computer-implemented method as described above.

Furthermore, the device comprises a computer-readable storage medium as described above, which is communicatively connected to the processor. Furthermore, the device comprises an inertial sensor, which is also communicatively connected to the processor.

Using this device, it is therefore possible to carry out a computer-implemented method as described above.

In a preferred embodiment, the data processing device is characterized in that the inertial sensor is designed as an acceleration sensor and/or rotation rate sensor.

Inertial sensors that are already installed in the vehicle for other purposes can be used particularly advantageously. For example, airbag control units use inertial sensors. The data from these already installed sensors can be readily used for the process described here, eliminating the need to install additional, specialized sensors in the vehicle. The inertial sensor can, for example, also be installed in a high-performance computer (HPC) and simultaneously used for the process proposed here.

According to a sixth aspect of the invention, a vehicle comprises a data processing device as described above.

With the device proposed in this invention, the entire vehicle interior can be monitored and recorded, requiring only a few sensors to achieve this purpose. Blind spots, such as those encountered with optical sensors (e.g., cameras or radar sensors), are eliminated.

Summary of the characters

• 1: Ein Flussdiagramm des computerimplementierten Verfahrens zur Erkennung von Lebewesen in einem ruhenden Fahrzeug;
• 2: Eine Ausführung des computerimplementierten Verfahrens aus 1;
• 3: Eine Vorrichtung zur Datenverarbeitung, die das computerimplementierte Verfahren aus 1 ausführen kann; und
• 4: Ein Fahrzeug mit der Vorrichtung zur Datenverarbeitung aus 3.

The invention is explained in more detail below using exemplary embodiments with the aid of figures. The figures show:

• 1 : A flowchart of the computer-implemented method for detecting living beings in a stationary vehicle;
• 2 : An execution of the computer-implemented method from 1 ;
• 3 : A data processing device which carries out the computer-implemented method of 1 can perform; and
• 4 : A vehicle with the device for data processing from 3 .

Detailed description of the characters

1 shows a flowchart of the computer-implemented method 100 for detecting living beings in a stationary vehicle 142.

In a first step of the method, receiving step 102, which is carried out continuously (in 1 (indicated by the circular arrow at reference numeral 102), data from an inertial sensor 140 is received. The data is characterized by a time-dependent amplitude 112, 114. The time-dependent amplitude 112, 114 can, for example, consist of a time series of measured values from an acceleration sensor.

In a determination step 104, a dominant frequency 118, 120 is determined from the time-dependent amplitude 112, 114. In the simplest case, this can be done, for example, using a fast Fourier transform, which first yields a frequency-dependent amplitude 116. Dominant frequencies 118, 120 can then be determined at the local maxima in the frequency-dependent amplitude 116.

In a next step, provision step 106, reference values 122, 124 are provided. These are selected such that they characterize the presence or absence of a living being in the vehicle 142.

The reference values 122, 124 can be based, for example, on the fact that different types of living beings shift the vibration frequencies of the vehicle 142 differently. Living beings with greater weight, such as adults, shift the vibration frequencies of the vehicle 142 to lower frequencies than children.

In a further step of the computer-implemented method 100, comparison step 108, a comparison of the at least one dominant frequency 118, 120 with the reference values 122, 124 is carried out in order to determine whether a living being is located in the vehicle 142.

If the comparison step 108 does not provide any indication of a living being in the vehicle 142 (f-branch in 1 ), the computer-implemented method 100 is terminated. However, by continuously executing the receiving step 102, the computer-implemented method 100 is automatically restarted on an ongoing basis.

However, if the comparison step 108 indicates a living being in the vehicle 142 (t -branch in 1 ), in a generation step 110 a signal is generated based on the comparison of the at least one dominant frequency 118,120 with the reference values 122,124.

The signal can then be used, for example, to send a warning to the driver of vehicle 142.

2a) -d) show an embodiment of the computer-implemented method 100 from 1 .

2a) shows a first time-dependent amplitude 112 of an inertial sensor 140 of a vehicle 142. The first time-dependent amplitude 112 corresponds to the measurement data of the inertial sensor 140, wherein the vehicle 142 is empty. Accordingly, a time t is shown on the x-axis, and the acceleration a measured by the inertial sensor 140 is shown on the y-axis.

2b) shows a second time-dependent amplitude 114 of an inertial sensor 140 of a vehicle 142, with persons in the vehicle. In the example of 2b) There is an adult and a child in the vehicle 142. However, this is not recognizable in the second time-dependent amplitude 114. As in 2a) a time t is shown on the x-axis and the acceleration a measured by the inertial sensor 140 is shown on the y-axis.

Using known methods, such as a fast Fourier transformation, the second time-dependent amplitude 114 can be converted into a frequency-dependent amplitude 116. This is 2c ). Here, a frequency f is shown on the x-axis and the Fourier-transformed time-dependent amplitude 114 or frequency-dependent amplitude 116, FFT(a), is shown on the y-axis.

The frequency-dependent amplitude 116 shows two relevant pronounced local maxima: a first local maximum at a first dominant frequency 118, and a second local maximum at a second dominant frequency 120. 2c ) further local maxima were observed, but these were not observed in the example of 2c ) is not further considered.

2d ) shows the dominant frequencies 118,120 as a function of the weight supplied to the vehicle 142. Accordingly, a weight w is shown on the x-axis and the dominant frequencies 118,120 f _d are shown on the y-axis. The measured values can be obtained, for example, by a measurement in which the weight supplied to the vehicle 142 in the form of people was known.

The measured values in 2d ) can be used to generate first reference values 122 and second reference values 124. The first reference values 122 represent the functional relationship between the first dominant frequency 118 and the additional weight in the vehicle 142. The second reference values 124 represent the functional relationship between the second dominant frequency 120 and the additional weight in the vehicle 142. The reference values 122, 124 have been created in the form of best-fit lines. However, it is also possible to create them using other means, for example, using an appropriately trained neural network.

If a dominant frequency 118,120 is now determined in the vehicle 142 as in the method from 1 shown, the reference values 112,124 can be used to determine how much additional weight is in the vehicle 142. The amount of weight thus determined can in turn be used to draw conclusions about the number and/or type of people in the vehicle.

3 shows a data processing device 134 that implements the computer-implemented method 100 of 1 can execute.

The device 134 has a processor 136 which is configured to carry out the steps of the computer-implemented method 100 from 1 Furthermore, the device 134 has a computer-readable storage medium 138 on which a computer program product is stored, which has instructions which, when the program is executed by the processor 136, cause the processor 136 to carry out the steps of the computer-implemented method from 1 to execute.

Furthermore, the device 134 has an inertial sensor 140. This provides the time-dependent amplitude 112,114, which is required for the execution of the computer-implemented method from 1 is required. All components 136, 138, 140 of the device 134 are communicatively connected to one another.

The device 134 does not have to be embodied as a single unit. Rather, it is possible for the processor 136 and storage medium 138 to be part of a high-performance computer or central control unit already installed in the vehicle 142. The inertial sensor 140 can be part of another component in the vehicle 142, for example, a control unit for the vehicle airbags, which typically has inertial sensors 140.

4 shows a vehicle 142 with the device 134 from 3 .

List of reference symbols

100

Computer-implemented method

102

Receiving step

104

Determination step

106

Deployment step

108

Comparison step

110

Generation step

112

First time-dependent amplitude

114

Second time-dependent amplitude

116

Frequency-dependent amplitude

118

First dominant frequency

120

Second dominant frequency

122

First reference values

124

Second reference values

134

device

136

processor

138

Storage medium

140

Inertial sensor

142

vehicle

Claims (11)

A computer-implemented method (100) for detecting living beings in a stationary vehicle (142), comprising the steps of: a) receiving (102) data from an inertial sensor (140), characterized by a time-dependent amplitude (112, 114), b) determining (104) at least one dominant frequency (118, 120) from the time-dependent amplitude (112, 114), c) providing (106) reference values (122, 124) such that they indicate the presence or absence of a living being in the vehicle (142), and d) comparing (108) the at least one dominant frequency (118, 120) with the reference values (122, 124), and, if the comparison (108) indicates a living being in the vehicle (142), e) generating and Outputting (110) a signal based on the comparison (108) of the at least one dominant frequency (118,120) with the reference values (122,124). Computer-implemented method according to Claim 1 , characterized in that on the basis of the signal a vehicle function is carried out to warn an occupant of the vehicle (142). Computer-implemented method according to Claim 1 or 2 , characterized in that when comparing (108) the dominant frequency (118,120) with the reference values (122,124) a type of living being is additionally determined and the generation and output (110) of the signal is additionally based on the determined type of living being. Computer-implemented method according to one of the preceding claims, characterized in that the determination (104) of the dominant frequency (118, 120) is achieved by means of a discrete Fourier transform or a fast Fourier transform. Computer-implemented method according to one of the preceding claims, characterized characterized in that the reference values (122,124) are determined by a trained neural network. A computer program product comprising instructions which, when executed by a computer, cause the computer to execute a computer-implemented method according to any one of the preceding claims. A computer-readable storage medium (138) comprising instructions which, when executed by a computer, cause the computer to perform a computer-implemented method according to one of the Claims 1 until 5 to execute. Data carrier signal which the computer program product Claim 6 transmits. A data processing device (134) for detecting living beings in a stationary vehicle (142), comprising a) a processor (136) configured to carry out the steps of a computer-implemented method according to one of the Claims 1 until 5 b) a computer-readable storage medium (138) according to Claim 7 which is communicatively connected to the processor (136), and c) an inertial sensor (140) which is communicatively connected to the processor (136). Device for data processing according to Claim 9 , characterized in that the inertial sensor (140) is designed as an acceleration sensor and/or rotation rate sensor. Vehicle (142) which comprises a data processing device (134) according to Claim 9 or 10 has.

Images