KR102397311B1 - Method of Recognizing Daily Activities of Living-alone Person Using Channel State Information between WiFi Transmitter and WiFi Receiver - Google Patents

Abstract

Disclosed is a method for recognizing daily activities of indoor occupants using channel state information between Wi-Fi transceivers. A channel state information (CSI) signal communicated between a WiFi transmitter and a WiFi receiver respectively installed at a predetermined location in an indoor space is collected, high-frequency noise is removed, and only waveforms related to human behavior are extracted. From the extracted CSI signal waveforms, sub-carrier signals that reflect the movement of the occupant's body are selected and normalized, and a waveform of a predetermined comparison section is extracted from the normalized CSI signal waveform to obtain a previously known reference waveform for each occupant's behavior. Based on the comparison of the waveform similarity, the current behavioral state of the occupant in the indoor space is grasped. In the activity pattern comparison step, the reference waveform is reflected in the established learning model secured through machine learning, and the extracted waveform of the predetermined comparison section is provided as an input to the learning model. The learning model may be built using a Support Vector Machine (SVM) algorithm. It is possible to recognize the daily activities of indoor occupants by comparing the pattern of the waveform generated by the daily activities of the occupants extracted from the WiFi CSI signal for each time period with the pattern of the reference waveform for each daily activity.

Description

{Method of Recognizing Daily Activities of Living-alone Person Using Channel State Information between WiFi Transmitter and WiFi Receiver}

The present invention relates to a method for recognizing the daily activities of daily living of an occupant living in an indoor space, and more particularly, to the It relates to a method for recognizing Activities of Daily Life.

The rapid aging of the population worldwide is emerging as a social problem. Korea will also enter an aged society in 2018, in which the population aged 65 and over will account for more than 14% of the total population. Degenerative brain diseases such as dementia are representative cognitive impairment diseases that require long-term treatment, and are expected to increase rapidly with the aging population. Since this is directly related to the cost that society as a whole has to bear, efforts to reduce it are necessary.

The elderly are more likely to fall into an emergency situation, and their ability to respond to emergency situations on their own decreases. If appropriate countermeasures are not taken, serious situations can arise. In particular, falls can occur due to various diseases, and can also occur from simple old age. Therefore, it is necessary to prevent it in advance and respond quickly in case of a fall accident. In particular, in the case of a single-person elderly household, it is very difficult for the elderly person to detect and respond to symptoms caused by the occurrence of a dysfunctional disease on their own. Therefore, the demand for a smart home healthcare system that enables the elderly to self-monitor their health at all times is also increasing.

For this purpose, research is being actively conducted to collect data related to the behavior of occupants in an indoor space and to confirm the relevance to the occupant's health status or to extract behavioral patterns. Recently, as healthcare technology is combined with IoT technology, research on the identification of existing human behavior is also being conducted. Among such studies, there is behavior recognition using wearable sensors and acoustic signals, or behavior recognition systems using multi-mode sensors composed of acceleration, angular velocity, and altitude sensors. However, existing studies mainly focus on detecting the movement and posture of occupants by using sensors, such as capturing the occupant’s daily life or collecting occupant’s physiological data in real time by having the occupant wear a wearable device directly. are aligning

These dedicated sensors can achieve fine-grained activity recognition. However, it is usually accompanied by a high cost, which greatly reduces the cost and accessibility of installing non-scalable dedicated sensor devices. However, the method of using a wearable sensor has disadvantages such as rejection and inconvenience of the elderly for having to install additional devices such as sensors, and disadvantages such as excessive exposure of personal information. Therefore, in order to compensate for these shortcomings, a technology capable of extracting and monitoring daily life activities without directly dealing with additional devices is needed.

Korean Patent Publication No. 10-2009-0025924 (published on March 11, 2009)

One object of the present invention is to collect and analyze information on the occupant's daily life behavioral change pattern by using CSI between a WiFi transmitter and a receiver, which is a non-intrusive sensing method in a residential space, and analyze the occupant's behavior and state. This is to provide a way to recognize it.

The problem to be solved by the present invention is not limited to the above problems, and may be variously expanded without departing from the spirit and scope of the present invention.

A method for recognizing an indoor occupant's daily life activities using channel state information between WiFi transceivers according to embodiments for realizing an object of the present invention includes one or more WiFi transmitters installed at predetermined locations in an indoor space, respectively. In the WiFi communication system including the above WiFi receiver, a method performed through execution of a computer program in a computing device, (a) a channel state information (Channel State Information, CSI) signal communicated between the WiFi transmitter and the WiFi receiver activity data collection step to collect; (b) an activity data pre-processing step of removing high-frequency noise included in the collected CSI signal and extracting only waveforms related to the behavior of occupants in the indoor space; (c) selecting and normalizing sub-carrier signals reflecting the movement of the occupant's body from the extracted CSI signal waveforms; And (d) extracting the waveform of a predetermined comparison section from the normalized CSI signal waveform and comparing the previously known reference waveform for each behavior of the occupant with the waveform similarity to determine the current behavioral state of the occupant in the indoor space It includes a step of comparing activity patterns to identify. According to this, the daily living activities of indoor occupants can be recognized by comparing the pattern of the waveform generated by the daily living activity of the occupant extracted from the WiFi CSI signal for each time period with the pattern of the reference waveform for each daily living activity.

In exemplary embodiments, amplitude information may be extracted from the CSI signal and used as the 'waveform related to human behavior'.

In example embodiments, the high-frequency noise may be removed by using a low-pass filter with a cut-off frequency of 2 Hz or less.

In example embodiments, the comparison period may be determined as a period in which the change of the CSI signal is greater than a set threshold value.

In exemplary embodiments, in the activity pattern comparison step, the reference waveform is reflected in a built-up learning model secured through machine learning, so the extracted waveform of the predetermined comparison section is an input of the learning model. can be provided.

In example embodiments, the learning model may be an SVM learning model constructed using a support vector machine (SVM) algorithm.

In exemplary embodiments, the method for recognizing an indoor occupant's daily life activity includes: securing a plurality of sample data in which amplitude change according to the occupant's activity occurs in each of the plurality of normalized CSI signal waveforms; extracting features of each of the secured plurality of sample data; assigning a label to each sample data based on a camera-captured image corresponding to each of the secured sample data; and generating the SVM learning model by learning the secured characteristics and label information of each sample data based on the SVM algorithm.

In example embodiments, the normalized signal of the sub-carrier signals may be a signal in which all of the selected sub-carrier signals are superimposed.

According to exemplary embodiments of the present invention, a computer-executable program stored in a computer-readable recording medium may be provided to perform a 'method for recognizing an indoor occupant's daily activities.'

According to exemplary embodiments of the present invention, a computer-readable recording medium in which a computer-executable program for performing 'a method for recognizing an indoor occupant's daily life activities' is recorded may be provided.

The method for recognizing an indoor occupant's daily life activity using the channel state information between the WiFi transceivers according to exemplary embodiments of the present invention detects minute movements of the body in the daily activities of a occupant of a living space using WiFi CSI information, and By observing the pattern of the waveform, it is possible to detect behavioral changes only with the WiFi signal. In particular, the daily life activity pattern of the elderly living alone can be identified by time, and when a sudden change in the pattern is detected, it can be determined as a signal for a change in the health status of the occupant and whether an emergency has occurred can be determined.

It can contribute to improving the quality of life related to health by providing information on daily life activity patterns by time period to the occupants and guardians, and related specific effects are as follows. First, since the final result of the present invention, information on daily life activity by time zone, has the same level of granularity as information through the use of an existing direct attachable sensor, it is possible to collect information in consideration of the occupant's living convenience. Second, it is possible to provide a customizable residential space use profile for the occupant's living pattern by utilizing the daily life activity pattern information, which is the final result derived from the present invention. Third, if abnormal (long-term inactivity, rapid unlabeled waveform fluctuations, etc.) occurs when the derived pattern is evaluated, the quality of life is improved by notifying the occupant, or, in the case of the elderly, the relevant organization is notified. It can be used for health condition diagnosis information.

In addition, it is possible to obtain information about the occupant's daily life activities without directly responding to additional devices. Therefore, the elderly occupant does not cause any discomfort or rejection as a result of directly dealing with additional devices.

1 is a diagram exemplarily illustrating a multi-path propagation state of a WiFi signal in an indoor environment in which an indoor occupant's daily life activity recognition system is installed, according to an exemplary embodiment of the present invention.
2 is a block diagram showing the configuration of a WiFi receiver for recognizing an indoor occupant's daily activities according to an exemplary embodiment of the present invention.
3 is a graph illustrating amplitude data of a raw CSI signal of one sub-carrier signal and amplitude data of a filtered CSI signal.
4 is a flowchart schematically illustrating a method for recognizing an indoor occupant's daily life activities using channel state information between WiFi transceivers according to an exemplary embodiment of the present invention.
5a and 5b respectively show plan views of different experimental test beds for testing the method according to the invention.
6A shows the raw CSI amplitude data of 30 subcarriers captured when an indoor occupant sequentially performs four activities of absence, typing, walking, and falling, and FIG. 6B shows the ratio of 30 subcarriers with respect to walking activity among them. It is a graph showing the processed CSI amplitude data in more detail.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. The same reference numerals are used for the same components in the drawings, and repeated descriptions of the same components are omitted.

1 exemplarily shows a multipath propagation state of a WiFi signal in a system 100 for recognizing indoor occupants' daily life activities configured to implement the method of the present invention in order to identify indoor occupants' daily living activities.

Referring to FIG. 1 , a system 100 implementing the method of the present invention includes an access point (AP) device 110 serving as a WiFi transmitter in an indoor space 10 and one or more WiFi receivers 120 capable of WiFi communication. -1, 120-1).

One or more WiFi receivers 120 - 1 and 120 - 1 may perform WiFi communication with the AP device 110 . A multiple input and multiple output (MIMO) router having a plurality of antennas may be used as the AP device 110 . The MIMO router is configured to support 802.11n AP mode, for example at a frequency of 5 GHz, and can transmit Internet Control Message Protocol (ICMP) packets at a sampling rate of 10 Hz (10 packets per second). The one or more WiFi receivers 120-1 and 120-1 include a wireless receiver capable of receiving a WiFi wireless signal transmitted from the AP device 110, and processing the received signal in a manner described below to derive a desired result. It may be a computing device having an arithmetic processor, a memory, a data storage, and the like.

Referring to FIG. 2 , the WiFi receiver 120 includes a WiFi wireless receiver 310 , a CSI data collection unit 320 , a CSI data processing unit 330 , and an activity classification unit 340 for recognizing the daily activities of indoor occupants. , and the learning model 350 may be included. In an exemplary embodiment, the WiFi receiver 120 may be implemented in hardware and software.

The WiFi wireless receiver 310 may be implemented with a WiFi antenna and a wireless reception circuit to receive a WiFi wireless signal transmitted by the AP device 110 . The WiFi wireless signal transmitted by the AP device 110 is propagated through various propagation paths 212 and 214 in the indoor space 10 . The WiFi wireless receiver 310 receives the WiFi wireless signal radiated from the AP device 110 through a direct path 212, that is, a line of sight (LOS), or a path reflected by a wall or floor ( 214) that is, a signal passing through NLOS (None LOS) may be received.

The CSI data collection unit 320 may receive the WiFi wireless signal from the WiFi wireless receiver 310 and collect CSI data included in the signal.

The CSI data processing unit 330 may receive the CSI data collected from the CSI data collection unit 320 and perform signal processing for extracting information corresponding to the activity of a single occupant in the indoor space 10 . To this end, the CSI data processing unit 330 may include a filtering unit 332 , a subcarrier selection unit 334 , a normalization unit 336 , and a segmentation unit 338 .

Occupant activity has a very low frequency component. The raw CSI data collected from the CSI data collection unit 320 includes, in addition to signal components representing the activity of the occupant 20 , various high-frequency noises irrelevant to the activity of the occupant. The filtering unit 332 may extract only a signal related to the activity of the occupant 20 by removing high-frequency noise included in the CSI data. The filtering unit 332 may be implemented as, for example, a low-pass filter such as a Butterworth filter. The filtering unit 332 may be configured to maintain a sampling rate of, for example, 10 packets/sec. The cutoff frequency of the filtering unit 332 implemented as a low-pass filter may be set to 1-2 Hz. Various activities of the occupant 20 can be represented by a signal having a frequency component of 1 Hz or less in most cases, and most activities can be safely represented by setting 2 Hz as a cutoff frequency. Variations included in the filtered CSI data can be distinguished from other activities.

The graph of FIG. 3 compares amplitude data of a raw CSI signal of one sub-carrier signal with amplitude data of a filtered CSI signal. From the graph, it can be seen that the amplitude waveform of the filtered CSI signal (Raw Amplitude in the graph) has a much smoother amplitude waveform (Filtered Amplitude in the graph) after the high-frequency noise component included therein is removed.

The subcarrier selector 334 may select a subcarrier that well reflects the minute movement of the body from the CSI signal of the low frequency component filtered by the filtering unit 332 . For example, between the AP device 110 and the WiFi receiver 120 , about 30 subcarrier signals may be selected from among about 100 subcarrier signals.

The normalizer 336 may normalize the selected subcarrier signals. Normalization determines which data among the 30 subcarrier signals to use. There may be several methods for this. In an exemplary embodiment, a signal waveform set in which all 30 subcarrier signal waveforms are superimposed may be created as a normalized signal.

The segment unit 338 may set a comparison section in order to extract necessary data corresponding to the activity of the occupant from the amplitude waveform of the CSI signal. Human activity causes a change in the CSI signal, and the CSI signal change has a relatively larger value when the human activity is occurring. If the degree of CSI signal change is greater than a threshold value determined empirically, the CSI data may be viewed as an active signal interval containing specific human activity information, and if less than the threshold value, the CSI data may be viewed as an inactive signal interval. Based on this reference, an activity signal section can be extracted from the CSI signal.

The activity classification unit 340 performs machine learning using the CSI data on the activity of the occupant generated by the CSI data processing unit 330 to establish a waveform standard for each activity, and a learning unit to build the learning model 350 ( 342), and an activity determination unit 344 that inputs CSI data related to the occupant's activity into the learning model 350 and compares the input CSI data with a waveform reference for each activity to determine which activity the occupant's activity is. may include

Next, the flowchart of FIG. 4 schematically shows a method of recognizing an indoor occupant's daily life activities using channel state information between WiFi transceivers according to an exemplary embodiment of the present invention.

(a) Collection of activity data;

First, it is necessary to perform machine learning using CSI data obtained through experiments, and to build a learning model 350 as a result. 5a and 5b illustrate a plan view of an experimental test bed for testing the method according to the invention. In an exemplary embodiment, two different house environments were selected as test beds. 5A is a low-rise housing environment of a wooden frame, and FIG. 5B may be a high-rise apartment environment of a reinforced concrete frame. In the housing environment of FIG. 5A , the AP device 110 is installed in the living room and the WiFi receiver 120 is installed in a kitchen provided as a space separate from the living room. In the housing environment of FIG. 5B , the AP device A case in which 110 is installed in the living room and the WiFi receiver 120 is installed in the kitchen provided in the same space as the living room is exemplified. In both environments, large household items such as refrigerators in the kitchen, dining tables and desks in the living room are contained, and there are more small and medium-sized household items in the environment of FIG. 5B than the environment of FIG. 5A.

As shown, in the indoor space 10 provided with the occupant's daily life activity recognition system 100, for the experiment, several subjects performed periodic activities including the same activity content, for example, ten times. As an experiment, each subject sat down on a sofa in the living room or a chair in the kitchen, etc. and engaged in daily life activities such as eating and typing, and moving between the living room and kitchen (walking). One subject at a time performed the activity. Each subject walked ten times along the path indicated by arrows, and performed activities such as eating and typing ten times in the dining room and living room, respectively. Each time there was 10 seconds of activity and 20 seconds of inactivity. A 20-second interval between activities can make the distinction between activity cycles clear. Arrows indicate the gait trajectory of the occupant.

During the experiment, the camera can be used to record the activities of the occupants 20 and to be time-stamped to set basic information about the activities. CSI data for 30 subcarriers for the first AP device 110-WiFi receiver 120 antenna pair may be captured and extracted. Then, the CSI amplitude data can be used to perform data processing for occupant activity recognition.

In an exemplary embodiment, in order to derive daily daily life activities of the occupant 20 , a change in behavior of the occupant 20 may be recognized and distinguished by using channel state information of WiFi, that is, CSI. That is, while the WiFi signal is propagated from the AP device 110 as a WiFi transmitter to the WiFi receiver 120 , the CSI data collection unit 320 of the WiFi receiver 120 receives CSI data through the WiFi wireless reception unit 310 . can be collected (step S10). The collected CSI data includes information about the activities of the occupants 20 .

The CSI signal may be delivered via multipath 210 as illustrated in FIG. 1 . That is, the CSI signal is transmitted directly from the AP device 110, which is a WiFi transmitter, to the WiFi receiver 120 through the direct transmission path 212 according to the characteristics of each path, and the wall 30, floor, or ceiling of the room. signal transmitted via path 214 reflected by a structure such as

In the frequency domain, when the signal Y received by the WiFi radio receiver 310 is expressed as the signal X transmitted from the AP device 110, it can be expressed as in Equation 1 below. The subscript i denotes a data packet, i∈[1, N]. However, the value of N indicates the number of correctly received packets.

Here, X _i : a transmission signal vector, Y _i : a reception signal vector, and H _i : a CSI matrix of the i -th packet, indicating a channel response characteristic matrix. N _i : White Gaussian noise and level constant, representing noise in the channel.

According to an exemplary embodiment, in WiFi communication between the AP device 110 and the WiFi receiver 120, one channel is divided into a plurality of subcarriers and an orthogonal frequency division multiplexing scheme is used to minimize the interval between the divided carriers. Multiplexing: OFDM) transmission method can be used. The CSI matrix H _i may have dozens of subcarriers, for example, 30 subcarriers. Whenever the occupant 20 takes an action, the WiFi signal transmitted from the transmitting antenna of the AP device 110 is reflected or refracted to form a delayed signal component, which is included in the CSI immediately and the WiFi wireless receiver 310 ) can be captured by the receiving antenna of

6A shows raw CSI amplitude data of 30 subcarriers captured when a person sequentially performs four activities of absence, typing, walking, and falling in the indoor space 20, and FIG. 6b is the We present the raw CSI amplitude data of 30 subcarriers in more detail for moderate walking activity.

Referring to FIGS. 6A and 6B , 30 subcarriers displayed in different colors have different amplitude values but show similar trends. In Figure 6a, when a person is not in the indoor space 20, there is little change in the CSI amplitude, when typing, the CSI amplitude change occurs relatively uniformly, when the walking behavior, the degree of amplitude change occurs relatively irregularly, , in the case of a fall, a very large irregular amplitude change occurs at the time of the fall, and then the amplitude change calms down immediately and maintains a uniform state. That is, it can be seen that the CSI amplitude change according to human activity or behavior is different from each other in the CSI amplitude change pattern for each behavior. Since each behavior and the CSI amplitude change pattern can be uniquely matched, it can be recognized as a specific behavior by using it. Therefore, a CSI signal is collected, and amplitude data is extracted from the CSI signal and used.

(b) activity data preprocessing

The collected CSI data may include noise unrelated to the occupant's activities. It is necessary to remove the noise included in the CSI data and go through the process of extracting only the waveform of the occupant's activity.

Specifically, it is necessary to remove unnecessary signal components in order to accurately grasp the fine movement according to the change in the activity of the occupant 20 . Therefore, the filtering unit 332 may remove high-frequency noise from the CSI data provided from the CSI data collection unit 320 by using a high-frequency removal filter such as a second-order low-pass Butterworth filter. (Step S20).

By removing the high-frequency noise component, only data related to the behavior of the occupants can be extracted. In general, since human behavior has a low cycle of 1-2 Hz or less, the cut-off frequency in the filtering unit 332 may be set to a value of 1-2 Hz. To effectively detect low frequency activity, the Sampling Frequency can be configured to hold 10 packets/s. As can be seen from the graph shown in FIG. 3 mentioned above, the CSI amplitude waveform subjected to high frequency rejection filtering has a much smoother shape than the original CSI amplitude waveform before filtering.

(c) Classification of activity data

The sub-carrier selector 334 may receive the CSI amplitude data from which the high-frequency noise has been removed from the filtering unit 332 and select a sub-carrier that well reflects the minute movement of the body. From among a large number of sub-carrier signals wirelessly communicated between the AP device 110 and the WiFi receiver 120 , for example, about 30 may be selected. In addition, the normalizer 336 may normalize the amplitude waveform of the 30 sub-carrier signals in order to determine which data among the selected 30 sub-carriers to use (step S30). The waveform shown in FIG. 6 shows a case in which a signal waveform set in which all 30 subcarrier signal waveforms are superimposed is made into a normalized signal.

The amplitude waveform of the normalized subcarrier signal contains information about occupant activity. In other words, it includes both amplitude waveforms when the occupant engages in any activity and when there is no activity. The segment unit 338 may divide the normalized amplitude waveform of the subcarrier signal into a static section, that is, an inactive section and an active section, in order to extract necessary data corresponding to the activity of the occupant. That is, in the waveform of the normalized subcarrier signal, it is possible to determine an activity section (comparison section) for comparing whether or not there is a pattern change (step S40).

The segment unit 338 may process a signal waveform of a section subdivided into an activity area to extract features to be used for learning. The amplitude change of CSI data has a relatively high value when activity occurs. When the amplitude change of the CSI data is greater than an empirically determined threshold, the change in the amplitude of the CSI data may be considered to include a specific activity. Human activity has small body movements, and occupant activity can be subdivided by referring to actual data recorded in video. Through this process, hundreds to thousands of samples of amplitude changes of CSI data can be obtained for each human activity.

(d) comparison and classification of activity patterns

The activity classifying unit 340 may first perform learning to build the learning model 350 through the learning unit 342 . That is, the learning unit 342 may perform machine learning based on a predetermined algorithm on the waveform of the comparison period, that is, the activity period, and the waveform of the already known activity pattern. Through the machine learning, a reference pattern regarding the amplitude change of CSI data can be prepared for each human activity. In an exemplary embodiment, the machine learning may be performed using an SVM algorithm, and an SVM model may be built through this (step S50).

In an exemplary embodiment, in building the SVM learning model 350, the SVM model requires the user to input features and labels to train the model. In this case, the label may be classified from the actual data recorded in the image captured by the camera, and the feature may be extracted from the input data (ie, segmented activity data). For example, in segmented activity data, it can be classified using the method of Sheheryar Arshad (2017)'s 6 Wi-Chase features to be used for learning. That is, in the segmented activity data, the CSI data of the first data packet may be expressed as H _a (s), which is an S _b x 1-dimensional vector. Here, S _b represents 30 subcarriers, and s represents the sequence of Na consecutive data packets. The six characteristic features present in all subcarriers of the CSI amplitude used for training are: (1) the mean of H _k , (2) the standard deviation of H _k , and (3) the 25th percentile of H _k ( s), (4) the 75th percentile of H _k , (5) the maximum value of H _k , and (6) the mean absolute deviation of H _k . Some of the amplitude change samples of the CSI data obtained in step S40 may be used for learning, and the rest may be used for an accuracy test of the learning model. The SVM shows the expected labels for the activity data, so the labels generated by the SVM can be compared with the labels of the ground truth data in determining model accuracy.

For each of the entire sample data, learning can be performed based on the SVM algorithm on the data whose characteristics and labels are determined. When the learning model 350 regarding the amplitude change pattern of CSI data regarding human activity is built through this learning process, the activity of the occupant 20 can be grasped using the learning model 350 . That is, the CSI data processing unit 330 may secure the waveform of the comparison section determined through steps S10 to S40 and provide it to the activity determination unit 344 . The activity determination unit 344 may provide the waveform of the comparison section as an input to the learning model 350 to determine the state of the occupant or the type of activity in real time (step S60). In an exemplary embodiment, the type of the current activity of the occupant 20 may be determined by comparing the waveform similarity between the waveform of the comparison section provided as an input and the waveforms of the behavior known in advance by learning. In an exemplary embodiment, the SVM machine learning algorithm may be used to find out the similarity of the patterns of amplitude waveforms of two signals to be compared.

As such, by extracting the amplitude waveform pattern generated by the occupant's daily life activities from the WiFi CSI signal for each time period, and inputting the amplitude waveform pattern of the extracted CSI data into the machine-learning model 350, the pre-learned daily life activity By comparing the reference pattern of the amplitude waveform of each CSI data, it is possible to grasp what kind of activity the indoor occupant is currently doing. This may be performed in real time. According to the experiment, the activity recognition accuracy of the SVM-based learning model 350 showed an accuracy of 94.38% in the (A) environment of FIG. 5 and 87.50% in the (B) environment.

As described above, according to an exemplary embodiment of the present invention, it is possible to provide a customizable residential space use profile for the living pattern of the occupant by utilizing the daily life activity pattern information derived as a final result. And if abnormal activity occurs when the derived occupant's life pattern is evaluated (eg, time-activity mismatch, period-activity mismatch, frequency-activity mismatch, etc.), by notifying the occupant, the occupant can be used to improve the quality of life of Furthermore, if the occupant is an elderly person, it can be used for health condition diagnosis information by notifying the relevant institution.

The system 100 for recognizing an indoor occupant's daily life activities described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include

The method for recognizing an indoor occupant's daily life activities according to the embodiments of the present invention described above may be implemented as software that can be executed by a computing device including an arithmetic processing unit and a memory device. The software may include a computer program, code, instructions, or a combination of one or more thereof, which, independently or collectively, configures a processing device to operate as desired. The processing unit can be commanded. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or apparatus, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method for recognizing an indoor occupant's daily life activities according to embodiments of the present invention may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

The present invention can be applied to human data collection and evaluation using an indirect sensing method in addition to fields such as construction equipment and smart home in a residential space.

Although the embodiments have been described with reference to the limited drawings as described above, those skilled in the art can variously modify and change the present invention within the scope without departing from the spirit and scope of the present invention described in the claims below. You will understand that you can. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result. Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

100: indoor occupant daily life activity recognition system
110: AP device (WiFi transmitter) 120: WiFi receiver
310: WiFi wireless receiver 320: CSI data collection unit
330: CSI data processing unit 340: activity classification unit
350: learning model

Claims (10)

In the WiFi communication system including one or more WiFi transmitters and one or more WiFi receivers respectively installed at predetermined positions in an indoor space, a method performed by executing a computer program in a computing device, the method comprising:
An activity data collection step of collecting a channel state information (CSI) signal communicated between the WiFi transmitter and the WiFi receiver while a plurality of subjects repeatedly perform a plurality of types of daily living activities indoors;
an activity data pre-processing step of removing high-frequency noise included in the collected CSI signal and extracting only waveforms related to activities of occupants in the indoor space;
Normalizing a set of superimposed signal waveforms by selecting a plurality of sub-carrier signals reflecting the movement of the occupant's body from among a plurality of sub-carrier signals of the extracted CSI signal waveforms;
setting a comparison section for determining whether a pattern is changed in the normalized sub-carrier signal waveform;
extracting amplitude change samples of CSI data for each activity type of occupants based on the comparison section;
The extracted amplitude change samples of the CSI data are machine-learned based on a predetermined algorithm together with a known activity pattern corresponding to each amplitude change sample, and a reference pattern for the amplitude change of the CSI data for each activity is supported. Building a vector machine (Support Vector Machine: SVM) learning model; And
By extracting the amplitude waveform pattern of the CSI data in the comparison section from the WiFi CSI signal collected in real time and filtered with high-frequency noise over time and providing it as an input to the SVM learning model, Comprising an activity pattern comparison step to determine the current activity state in real time,
In the step of extracting the amplitude change sample, the activity type of the occupant is determined using actual measurement data recorded as a video. delete The method of claim 1, wherein the high-frequency noise is removed by using a low-pass filter with a cut-off frequency of 2 Hz or less. The method of claim 1, wherein the comparison section is defined as a section in which the change in the CSI signal is greater than a set threshold value. The method of claim 1, wherein the current activity state of the occupant is determined by comparing the waveform similarity between the amplitude waveform of the CSI data of the comparison section provided as an input and waveforms of the occupant's activity known in advance by the machine learning. A method for recognizing daily living activities of indoor occupants, characterized in that it is determined. The method of claim 1, wherein in the setting of the comparison period, if the degree of variation of the CSI signal is greater than a predetermined threshold, it is regarded as an active signal period of the occupant, and if it is less than the threshold, it is inactive. A method for recognizing daily life activities of indoor occupants, comprising the step of considering it as a signal section. The method of claim 1 , wherein the building of the SVM learning model comprises: securing a plurality of sample data in which amplitude changes according to an occupant's activity occur in each of the plurality of normalized CSI signal waveforms; extracting features of each of the secured plurality of sample data; assigning a label to each sample data based on a camera-captured image corresponding to each of the secured sample data; and generating the SVM learning model by learning the secured characteristics and label information of each sample data based on the SVM algorithm. The method of claim 1, wherein the normalized signal of the sub-carrier signals is a signal superimposed on all of the selected sub-carrier signals. A computer-executable program stored in a computer-readable recording medium to perform the 'method of recognizing an indoor occupant's daily life activities' according to any one of claims 1 to 8. A computer-readable recording medium in which a computer-executable program for performing the 'method for recognizing an indoor occupant's daily life activities' according to any one of claims 1 to 8 is recorded.

Images