JP2025036178A - COMPUTER PROGRAM, INFORMATION PROCESSING METHOD, AND INFORMATION PROCESSING APPARATUS - Google Patents

Abstract

[Problem] To provide a computer program, information processing method, and information processing device for monitoring with improved visibility of highly urgent information and improved viewability of information. [Solution] A sensor is used to acquire data for each subject, and a status image showing the subject's condition based on the acquired data is displayed for each subject, and when an arbitrary subject is selected, an instruction is received to set whether or not to display the history of each of multiple types of conditions for the selected subject, settings are made according to the received instruction, and a status history is displayed that includes the history of the conditions that are set to be displayed among the multiple types of conditions for the subject, and does not include the history of the conditions that are set not to be displayed. [Selected Figure] Figure 1

Description

The present invention relates to a computer program, an information processing method, and an information processing device for monitoring.

Conventionally, monitoring systems have been used to monitor a subject. The monitoring system uses sensors to acquire data related to the subject's condition, such as vital signs, and displays the subject's condition based on the acquired data on a terminal device. For example, in a care facility, a sensor is attached to each bed, and care is provided by a caregiver who checks the subject's condition displayed on a terminal device. The displayed status of the subject includes whether or not the subject is in bed, whether or not they are asleep, increases or decreases in heart rate, or increases or decreases in respiratory rate. Patent Document 1 discloses an example of such a monitoring system.

JP 2021-178198 A

In a monitoring system, various types of information are displayed on a terminal device, so the visibility of highly urgent information, such as increases or decreases in heart rate or respiratory rate, can be poor. In addition, the viewability of the displayed information can be impaired.

The present invention has been made in consideration of the above circumstances, and its purpose is to provide a computer program, an information processing method, and an information processing device that performs monitoring with improved visibility of highly urgent information and listability of information.

A computer program according to one embodiment of the present invention is characterized in that it causes a computer to execute a process of acquiring data for each subject using a sensor, displaying for each subject a status image showing the subject's status based on the acquired data, and, when an arbitrary subject is selected, accepting an instruction to set whether or not to display the history of each of multiple types of status of the selected subject, making settings according to the accepted instruction, and displaying a status history that includes the history of the statuses that are set to be displayed among the multiple types of status of the subject, and does not include the history of the statuses that are set not to be displayed.

An information processing method according to one embodiment of the present invention is characterized in that it acquires data for each subject using a sensor, displays for each subject a status image showing the subject's status based on the acquired data, and when an arbitrary subject is selected, receives an instruction to set whether or not to display the history of each of multiple types of status of the selected subject, performs settings according to the received instruction, and displays a status history that includes the history of the statuses that are set to be displayed among the multiple types of status of the subject, and does not include the history of the statuses that are set not to be displayed.

An information processing device according to one aspect of the present invention includes a calculation unit, which acquires data for each subject using a sensor, displays a status image showing the subject's status based on the acquired data on a display unit for each subject, and when an arbitrary subject is selected, accepts an instruction to set whether or not to display the history of each of multiple types of status of the selected subject, performs settings according to the accepted instruction, and displays a status history that includes the history of the status that is set to be displayed among the multiple types of status of the subject, and does not include the history of the status that is set not to be displayed.

In one embodiment of the present invention, a status image showing the subject's condition is displayed for each subject based on data acquired using a sensor. In addition, it is set whether or not to display the history of each of the subject's multiple types of conditions. If displayed, the history of the set condition is included, and if not displayed, the history of the set condition is not included. Since the history of some types of conditions is displayed and the history of other types of conditions is not displayed, the diversity of the displayed information is reduced and the visibility of the information is improved. If the history of a specific type of condition is displayed and the history of other types of conditions is not displayed, the visibility of the specific type of condition in the status history is improved.

The present invention has excellent effects, such as improving the visibility of information necessary for the user and improving the visibility of information that is important to the user in the status history.

FIG. 1 is a schematic diagram showing a configuration example of a monitoring system. 1 is a block diagram showing an example of an internal configuration of an information processing device; 2 is a block diagram showing an example of the internal configuration of a terminal device; 11 is a schematic diagram showing an example in which the information processing device displays information indicating the condition of a subject on a terminal device. FIG. 11 is a flowchart illustrating an example of a procedure for a process of setting standards for a heart rate and a respiratory rate, which is performed by the information processing device. FIG. 13 is a schematic diagram showing an example of a setting screen for setting standards for heart rate and respiratory rate. 13 is a flowchart showing an example of a processing procedure for displaying a status image. 11 is a schematic diagram showing a first example of a displayed image showing an abnormality in the respiratory rate per unit time of a subject as a status image; FIG. FIG. 13 is a schematic diagram showing a second example of a displayed image showing an abnormality in the respiratory rate per unit time of a subject as a status image. FIG. 13 is a schematic diagram showing a third example of a status image in which an image showing an abnormality in the subject's respiratory rate per unit time is displayed. FIG. 13 is a schematic diagram showing a fourth example in which an image showing an abnormality in the respiratory rate per unit time of a subject is displayed as a status image. 13 is a flowchart showing an example of a processing procedure for displaying details of a subject's condition. FIG. 13 is a schematic diagram showing an example in which a first graph, a second graph, and a state history are displayed. 1 is a schematic diagram showing a first example of displaying details of an abnormality in a subject's heart rate per unit time; FIG. FIG. 13 is a schematic diagram showing a second example of displaying details of an abnormality in a subject's heart rate per unit time. FIG. 4 is a schematic diagram showing a display example of a captured image. FIG. 13 is a schematic diagram showing a first example of an adjusted state history. FIG. 13 is a schematic diagram showing a second example of an adjusted state history. 13 is a schematic diagram showing an example of a first graph, a second graph, and a state history in which highlighting has been stopped; FIG.

The present invention will now be described in detail with reference to the drawings showing embodiments thereof.
FIG. 1 is a schematic diagram showing a configuration example of a monitoring system 100. The monitoring system 100 is a system that performs processing for monitoring the state of a target person 51. The monitoring system 100 includes an information processing device 1, a plurality of sensor devices 2, a plurality of camera devices 7, and a plurality of terminal devices 3. The plurality of sensor devices 2 are connected to the information processing device 1 via a communication network 4 such as a LAN (Local Area Network). The sensor device 2 may communicate with the information processing device 1 without the communication network 4. The plurality of camera devices 7 are each capable of communicating with the information processing device 1 using wired communication or wireless communication. The plurality of terminal devices 3 are each capable of communicating with the information processing device 1 using wired communication or wireless communication. The terminal device 3 may communicate with the information processing device 1 via a communication network, or may communicate with the information processing device 1 without the communication network. Each of the terminal devices 3 is used by a user 53.

The sensor device 2 is equipped with one or more sensors that measure various phenomena caused by the subject 51. The sensors measure vibrations, sounds, temperatures, pressure changes, load changes, body movements, and the like as phenomena caused by the subject 51. The sensors are, for example, piezoelectric sensors, microphone sensors, Doppler sensors, and the like. The sensor device 2 may be equipped with a sensor that directly measures the vital signs of the subject 51, such as heart rate or respiratory rate. The sensor device 2 transmits data related to the state of the subject 51 according to the measurement results of the sensor to the information processing device 1. The information processing device 1 determines the state of the subject 51, such as whether the subject 51 is staying in bed 52, based on the data from the sensor device 2. The information processing device 1 outputs information indicating the state of the subject 51 using the terminal device 3. The user 53 checks the state of the subject 51 using the terminal device 3.

The sensor device 2 is attached to a bed 52 used by a subject 51 to be monitored. For example, a facility such as a nursing home or a hospital has multiple beds 52, and a sensor device 2 is attached to each bed 52. There is a one-to-one correspondence between the sensor device 2 and the subject 51. The sensor device 2 may be attached to something other than the bed 52. For example, the sensor device 2 may be attached to a futon or clothing. When the subject 51 sleeps without using a bed 52, the sensor device 2 may be attached to the floor.

The camera device 7 is a camera for capturing an image of the person 51 being monitored. The camera device 7 may communicate with the information processing device 1 via a communication network. The camera device 7 may capture an image of the bed 52 or the room used by the person 51. For example, the camera device 7 is installed on the bed 52. The camera device 7 may also be installed somewhere other than the bed 52. The camera device 7 has an image sensor for capturing images, a storage unit for storing captured images created by capturing images, and a communication unit for communicating. The camera device 7 captures images continuously or intermittently, and stores the captured images in association with the time when the images were captured. The camera device 7 may transmit the captured images to the information processing device 1, and the captured images created by the camera device 7 may be stored in the information processing device 1.

FIG. 2 is a block diagram showing an example of the internal configuration of the information processing device 1. The information processing device 1 is configured using a computer such as a server device, a PC (personal computer), a smartphone, or a general-purpose tablet PC. The information processing device 1 executes an information processing method. The information processing device 1 includes a calculation unit 11, a memory 12, a storage unit 13, a reading unit 14, and a communication unit 15. The calculation unit 11 is configured using, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a multi-core CPU. The calculation unit 11 may be configured using a quantum computer. The memory 12 stores temporary data generated in association with the calculation. The memory 12 is, for example, a RAM (Random Access Memory). The storage unit 13 is non-volatile, for example, a hard disk or a non-volatile semiconductor memory. The reading unit 14 reads information from a recording medium 10 such as an optical disk or a portable memory. The communication unit 15 communicates with the outside of the information processing device 1 by wired communication or wireless communication. Specifically, the communication unit 15 receives data from the sensor device 2 via the communication network 4 and communicates with the terminal device 3.

The calculation unit 11 causes the reading unit 14 to read the computer program 131 recorded on the recording medium 10, and stores the read computer program 131 in the storage unit 13. The calculation unit 11 executes processing for realizing the functions of the information processing device 1 in accordance with the computer program 131. The computer program 131 may be a program product. The computer program 131 may be stored in advance in the storage unit 13, or may be downloaded from outside the information processing device 1. In this case, the information processing device 1 does not need to be equipped with the reading unit 14.

The computer program 131 can be deployed to run on a single computer, or on multiple computers located at one site or distributed across multiple sites and interconnected by a communications network. That is, the information processing device 1 may be composed of multiple computers, and the computer program 131 may be executed on multiple computers connected via a communications network.

The storage unit 13 stores setting data that records various settings for determining and displaying the state of the subject 51. The setting data records a one-to-one correspondence between the sensor device 2 and the subject 51. For example, the setting data records subject information for identifying each subject 51 and identification information for identifying the sensor device 2 that transmits data related to the state of the subject 51 according to the sensor measurement results to the information processing device 1, in association with each other. The subject information includes the name of the subject 51, the location of the room used by the subject 51, the location of the bed 52 used by the subject 51, etc. The identification information includes the serial number of the sensor device 2, etc.

Figure 3 is a block diagram showing an example of the internal configuration of the terminal device 3. The terminal device 3 is a computer such as a PC, a smartphone, or a tablet PC. The terminal device 3 includes a calculation unit 31, a memory 32, a storage unit 33, an operation unit 34, a display unit 35, and a communication unit 36. The calculation unit 31 is configured using, for example, a CPU, a GPU, or a multi-core CPU. The calculation unit 31 may also be configured using a quantum computer. The memory 32 stores temporary data generated in association with the calculation. The memory 32 is, for example, a RAM. The storage unit 33 is non-volatile, for example, a hard disk or a non-volatile semiconductor memory.

The operation unit 34 receives instructions from the user 53 by receiving operations from the user 53, and also receives input of information. The operation unit 34 is, for example, a touch panel, a keyboard, or a pointing device. The display unit 35 displays images. The display unit 35 is, for example, a liquid crystal display or an EL display (Electroluminescent Display). The operation unit 34 and the display unit 35 may be integrated. The communication unit 36 communicates with the outside of the terminal device 3 by wired or wireless communication. Specifically, the communication unit 36 communicates with the information processing device 1. The terminal device 3 may be equipped with an audio output unit such as a speaker.

The storage unit 33 stores the computer program 331. For example, the computer program 331 is downloaded using the communication unit 36 and stored in the storage unit 33. For example, the computer program 331 may be stored in the storage unit 33 in advance. For example, the computer program 331 may be read into the terminal device 3 via an interface unit (not shown) and stored in the storage unit 33. The calculation unit 31 executes processing in accordance with the computer program 331.

FIG. 4 is a schematic diagram showing an example in which the information processing device 1 displays information indicating the condition of the subject 51 on the terminal device 3. For each subject 51, a status image 61 indicating the condition of the subject 51 is displayed. A subject column 62 is displayed in addition to the status image 61, and includes subject information such as the name or the position of the bed 52 for identifying the subject 51.

In the example shown in FIG. 4, six status images 61 indicating the status of six subjects 51 are displayed. One status image 61 indicates that the subject 51 is in the "rising" status, standing up from the bed 52. One status image 61 indicates that the subject 51 is in the "starting to get up" status, starting to get up from the bed 52. One status image 61 indicates that the subject 51 is in the "in bed" status, staying in the bed 52. One status image 61 indicates that the subject 51 is in the "absent" status, not being in the bed 52. One status image 61 indicates that the subject 51 is in the "body movement" status, making subtle body movements while sleeping.

As shown in FIG. 4, the information processing device 1 causes the terminal device 3 to display a status image 61 indicating a state related to the behavior of the subject 51, such as "wake up", "get up", "absent", "in bed", "body movement", or "in bed" in which the subject 51 is in bed 52. The information processing device 1 can also display, on the terminal device 3, a status image 61 indicating a state related to the sleep of the subject 51, such as "sleeping" in which the subject 51 is asleep, "light sleep" in which the subject 51 is in a light sleep state, or "awake" in which the subject 51 wakes up from a sleep state. Furthermore, the information processing device 1 can display, on the terminal device 3, a status image indicating a state corresponding to the heart rate per unit time of the subject 51 and a status image indicating a state corresponding to the respiratory rate per unit time of the subject 51.

The information processing device 1 judges the state of the subject 51 according to the heart rate and respiration rate based on the set criteria. The set criteria are determined by the subject 51's threshold heart rate per unit time and the subject 51's threshold respiration rate per unit time. The information processing device 1 performs processing to set the heart rate and respiration rate criteria for each subject 51. FIG. 5 is a flowchart showing an example of the processing procedure for setting the heart rate and respiration rate criteria performed by the information processing device 1. Hereinafter, steps are abbreviated as S. The calculation unit 11 executes information processing according to the computer program 131, and the information processing device 1 executes the following processing.

The information processing device 1 causes the terminal device 3 to display a setting screen for setting the heart rate and respiratory rate standards (S11). In S11, the calculation unit 11 causes the communication unit 15 to transmit data for the setting screen to the terminal device 3, the terminal device 3 receives the data via the communication unit 36, and the calculation unit 31 displays the setting screen on the display unit 35 based on the data. For example, the user 53 operates the operation unit 34 to identify one subject 51 from among multiple subjects 51, input an instruction to perform setting processing, and the information processing device 1 starts setting processing for the specific subject 51.

Figure 6 is a schematic diagram showing an example of a setting screen for setting the standards for heart rate and respiration rate. The setting screen includes a switch for instructing whether to perform heart rate detection or respiration detection. Heart rate detection corresponds to determining an abnormality in the heart rate and displaying a status image showing a state in which there is an abnormality in the heart rate per unit time. Respiration detection corresponds to determining an abnormality in the respiration rate and displaying a status image showing a state in which there is an abnormality in the respiration rate per unit time.

Also included are input fields for inputting the upper and lower limits of the heart rate per unit time and the upper and lower limits of the respiration rate per unit time as thresholds for the heart rate and respiration rate. The upper limit of the heart rate is the threshold at which it is determined that the heart rate per unit time is increasing, and the lower limit of the heart rate is the threshold at which it is determined that the heart rate per unit time is decreasing. The upper limit of the respiration rate is the threshold at which it is determined that the respiration rate per unit time is increasing, and the lower limit of the respiration rate per unit time is the threshold at which it is determined that the respiration rate is decreasing. In the example shown in FIG. 6, an instruction is given to not detect the heart rate but to detect respiration, and the upper and lower limits of the respiration rate per unit time are specified.

The information processing device 1 accepts an instruction to set the standard for the heart rate and respiration rate (S12). In S12, the user 53, who has visually confirmed the setting screen, operates the operation unit 34 to input an instruction as to whether or not to perform heart rate detection or respiration detection to the terminal device 3. The user 53 also operates the operation unit 34 to input the upper and lower limits of the heart rate per unit time, or the upper and lower limits of the respiration rate per unit time to the terminal device 3. The calculation unit 31 causes the communication unit 36 to transmit the input contents to the information processing device 1, and the information processing device 1 receives them at the communication unit 15. At this time, the calculation unit 11 accepts an instruction as to whether or not to perform heart rate detection or respiration detection. The calculation unit 11 also receives the upper and lower limits of the heart rate per unit time, or the upper and lower limits of the respiration rate per unit time at the communication unit 15.

The information processing device 1 performs setting according to the received instruction (S13). In S13, the calculation unit 11 performs setting by recording in the setting data whether or not to perform heart rate detection or respiration detection in association with the specific subject 51. The calculation unit 11 also performs setting by recording in the setting data the upper and lower limits of the heart rate per unit time, or the upper and lower limits of the respiration rate per unit time. If the value of the heart rate or respiration rate is within the range from the lower limit to the upper limit, the heart rate or respiration rate meets the standard. If the value of the heart rate or respiration rate is below the lower limit or exceeds the upper limit, the heart rate or respiration rate does not meet the standard. In this way, the standard for the heart rate and respiration rate is set. After S13 is completed, the information processing device 1 ends the process of setting the standard.

The information processing device 1 executes the processes of S11 to S13 for each subject 51. Through the processes of S11 to S13, standards for heart rate and respiration rate are set for each subject 51. Appropriate ranges for heart rate and respiration rate may differ from person to person. By setting standards for each subject 51, it is possible to set the standards for heart rate and respiration rate to different appropriate standards for each subject 51. Furthermore, even for the same subject 51, appropriate standards for heart rate and respiration rate may differ depending on the physical condition. By changing and resetting the standards for heart rate and respiration rate, it is possible to set appropriate standards according to the physical condition of the subject 51. Note that the threshold for the heart rate per unit time or the threshold for the respiration rate per unit time may be only an upper limit or only a lower limit.

In addition, by the processing of S11 to S13, whether or not to perform heart rate detection or respiration detection is set for each subject 51. That is, the information processing device 1 sets whether or not to display an image indicating a state in which there is an abnormality in the heart rate or respiration rate per unit time as a status image. Similarly, the information processing device 1 can perform processing to set whether or not to display a status image indicating a state related to the behavior of the subject 51. Similarly, the information processing device 1 can perform processing to set whether or not to display a status image indicating a state related to the sleep of the subject 51. The settings are recorded in the setting data. Whether or not to display a status image indicating a state related to the behavior of the subject 51 and whether or not to display a status image indicating a state related to the sleep of the subject 51 may be set in advance.

The information processing device 1 performs processing to display multiple status images showing the status of multiple subjects 51. FIG. 7 is a flowchart showing an example of the processing procedure for displaying status images. The information processing device 1 acquires data from the sensor device 2 (S21). The sensor device 2 transmits data related to the status of the subject 51 according to the sensor measurement results to the information processing device 1 at any time. The sensor device 2 may transmit the value measured by the sensor as data as it is.

The sensor device 2 may transmit data obtained by processing values measured by the sensor. For example, a piezoelectric sensor included in the sensor device 2 measures a phenomenon caused by the subject 51, such as vibration or sound, and generates a measurement signal representing a value obtained by the measurement. The sensor device 2 performs signal processing such as noise removal, amplification, or frequency separation on the measurement signal to generate a feature signal representing a feature of the state of the subject 51. For example, the feature signal includes a heartbeat signal indicating the heartbeat of the subject 51, a breathing signal indicating the breathing vibration of the subject 51, a body movement signal indicating the body movement of the subject 51, or a snoring signal indicating the snoring of the subject 51. The sensor device 2 transmits data including the feature signal to the information processing device 1. The data transmitted by the sensor device 2 may include the measurement signal.

The information processing device 1 acquires data from the sensor device 2 by receiving the data transmitted from the sensor device 2 via the communication unit 15. The information processing device 1 receives data from multiple sensor devices 2 at any time. The calculation unit 11 stores the received data in the memory unit 13 in association with a time. The time may include the year, month, and day.

The information processing device 1 determines the state of the subject 51 based on the acquired data (S22). In S22, the calculation unit 11 determines the state of the subject 51's behavior, such as "waking up" or "rising up", based on the acquired data. The calculation unit 11 also determines the state of the subject 51's sleep, such as "asleep" or "awake".

For example, the calculation unit 11 determines the state of the subject 51 using a trained model that outputs information indicating the behavior and sleep state of the subject 51 when a feature signal is input. The trained model is realized by the calculation unit 11 executing processing according to the computer program 131. The memory unit 13 stores data necessary to realize the trained model. For example, the trained model is realized using a neural network.

The trained model is generated by learning using training data including a feature signal that represents the characteristics of the state of the subject 51 and information indicating the state related to the behavior and sleep of the subject 51. The trained model is trained in advance to output information indicating the state related to the behavior and sleep of the subject 51 when a feature signal is input. The trained model may be trained by the information processing device 1. The trained model for determining the state related to the behavior of the subject 51 and the trained model for determining the state related to the sleep of the subject 51 may be different.

In S22, the calculation unit 11 inputs the body movement signal or snoring signal contained in the data acquired from the sensor device 2 to the trained model. The trained model performs calculations in response to the input of the body movement signal or snoring signal, and outputs information indicating the behavior and sleep state of the subject 51. The calculation unit 11 acquires the information indicating the behavior and sleep state of the subject 51 outputted by the trained model. The calculation unit 11 determines the behavior and sleep state of the subject 51 as indicated by the acquired information.

The calculation unit 11 may determine the behavior and sleep state of the subject 51 without using a learned model. For example, the calculation unit 11 determines the behavior and sleep state of the subject 51 according to a predetermined rule. The calculation unit 11 may determine that the subject 51 is in bed when a measurement signal included in the acquired data changes significantly and exceeds a predetermined threshold. The calculation unit 11 may determine that the subject 51 is in bed when at least one of a heart rate signal and a respiratory signal is obtained from the acquired data. The calculation unit 11 may determine that the subject 51 is in a body movement state when a prominent body movement signal is obtained.

The calculation unit 11 may determine that the state of the subject 51 is in the sitting state when a signal indicating the initial movement of the subject 51 to sit up is obtained from the body movement signal included in the acquired data. The calculation unit 11 may determine that the state of the subject 51 is in the sitting state when the body movement signal indicates a stable value below a predetermined threshold value after the signal indicating the initial movement is obtained. The calculation unit 11 may determine that the state of the subject is in the absent state when the body movement signal remains significantly below the predetermined threshold value after the signal indicating the initial movement is obtained.

The calculation unit 11 may determine that the state of the subject 51 is in bed when the measurement signal exceeds a predetermined threshold and then falls below the threshold. The calculation unit 11 may determine that the state of the subject 51 is in bed when the heart rate signal and the respiratory signal included in the acquired data fall below a predetermined threshold. The calculation unit 11 may determine that all states other than the in-bed state, body movement state, sitting up state, sitting up state, and getting out of bed state are in bed. When the calculation unit 11 determines that the state of the subject 51 is in bed, it may determine that the subject is in a sleeping state or an awake state according to the pattern of the characteristic signal including the heart rate signal, the respiratory signal, and the body movement signal.

The monitoring system 100 may be configured such that the sensor device 2 determines the behavior and sleep state of the subject 51 based on the sensor measurement results, and transmits information indicating the determined state of the subject 51 from the sensor device 2 to the information processing device 1. In this configuration, in S22, the sensor device 2 determines the behavior and sleep state of the subject 51 by performing the determination process described above based on the sensor measurement results, and transmits information indicating the determined state of the subject 51 to the information processing device 1. The information processing device 1 receives the information indicating the state of the subject 51 transmitted from the sensor device 2 via the communication unit 15, thereby determining the behavior and sleep state of the subject 51.

Furthermore, in S22, the calculation unit 11 determines the state of the subject 51 according to the heart rate per unit time and the state according to the respiratory rate per unit time of the subject 51. The calculation unit 11 determines the heart rate per unit time of the subject 51 based on the acquired data. For example, the calculation unit 11 determines the heart rate per unit time of the subject 51 based on the heart rate signal included in the acquired data. If the determined heart rate meets the set criteria, the calculation unit 11 determines that the state of the subject 51 is in a state where there is no abnormality in the heart rate. More specifically, if the determined heart rate is within the range from the lower limit to the upper limit recorded in the set data, the calculation unit 11 determines that there is no abnormality in the heart rate. Also, if the determined heart rate falls outside the set criteria, the calculation unit 11 determines that the state of the subject 51 is in a state where there is an abnormality in the heart rate. More specifically, if the identified heart rate is below the lower limit or exceeds the upper limit recorded in the setting data, the calculation unit 11 determines that the heart rate is abnormal.

The calculation unit 11 determines the respiratory rate per unit time of the subject 51 based on the acquired data. For example, the calculation unit 11 determines the respiratory rate per unit time of the subject 51 based on a respiratory signal included in the acquired data. If the determined respiratory rate meets the set criteria, the calculation unit 11 determines that the subject 51 is in a state where there is no abnormality in the respiratory rate. More specifically, if the determined respiratory rate is within the range from the lower limit to the upper limit recorded in the set data, the calculation unit 11 determines that there is no abnormality in the respiratory rate. Furthermore, if the determined respiratory rate falls outside the set criteria, the calculation unit 11 determines that the subject 51 is in a state where there is an abnormality in the respiratory rate. More specifically, if the determined respiratory rate is a value below the lower limit or above the upper limit recorded in the set data, the calculation unit 11 determines that there is an abnormality in the respiratory rate.

In S22, the calculation unit 11 judges the state of each of the multiple subjects 51. The calculation unit 11 also records the history of the judged state of the subjects 51. The storage unit 13 stores history data that records the history of the state of the subjects 51. In the history data, the judged state of the subject 51 is recorded in association with a time for each subject 51. The time associated with the state of the subject 51 is the time when the state of the subject 51 is judged, or the time when the data on which the state of the subject 51 is judged is acquired by the sensor device 2 or the information processing device 1. The time may include the year, month, and day. The calculation unit 11 records the history of the state of the subject 51 by associating the judged state of the subject 51 with a time and recording it in the history data for each subject 51.

The information processing device 1 displays a status image showing the state of the determined subject 51 on the terminal device 3 (S23). The storage unit 13 stores status images showing the respective states of the subjects 51. In S23, the calculation unit 11 causes the communication unit 15 to transmit the status image showing the determined state of the subject 51 to the terminal device 3. The terminal device 3 receives the status image via the communication unit 36, and the calculation unit 31 displays the received status image on the display unit 35. The information processing device 1 transmits status images for multiple subjects 51, and the terminal device 3 displays the multiple status images. The information processing device 1 may generate an image including multiple status images and transmit it to the terminal device 3, and the terminal device 3 may display the received image. The information processing device 1 transmits information showing each status to the terminal device 3, and the terminal device 3 may store status images showing the respective states of the subjects 51 in advance in the storage unit 33, and display a status image according to the received information on the display unit 35.

The information processing device 1 displays, among the status images, an image showing the behavioral status of the subject 51 and an image showing the sleep status of the subject 51 on the display unit 35 of the terminal device 3 at different positions for each subject 51. FIG. 4 shows an example in which an image showing the behavioral status of the subject 51 or an image showing the sleep status of the subject 51 is displayed as a status image 61. The information processing device 1 displays the status image 61, which is an image showing the behavioral status of the subject 51 or an image showing the sleep status of the subject 51, in a first display area provided for each subject 51 on the display unit 35. The position where each status image 61 is arranged in FIG. 4 corresponds to the first display area.

The information processing device 1 displays an image showing an abnormality in the subject's 51 heart rate per unit time or an abnormality in the subject's 51 respiratory rate per unit time in a form different from the image showing the subject's 51 behavioral state and the image showing the subject's 51 sleep state. FIG. 8 is a schematic diagram showing a first example of an image showing an abnormality in the subject's 51 respiratory rate per unit time as a state image. As a state image 61, an image showing the subject's 51 behavioral state or an image showing the subject's 51 sleep state is displayed in the first display area. Furthermore, the information processing device 1 causes the display unit 35 of the terminal device 3 to display a state image 63 showing an abnormality in the subject's 51 respiratory rate per unit time, superimposed on the state image 61 displayed in the first display area. At this time, the information processing device 1 makes the area of the state image 63 smaller than the area of the state image 61.

A status image 63 indicating that the subject 51 has an abnormality in the respiration rate per unit time is displayed superimposed on a status image 61 indicating the behavioral or sleeping status of the subject 51. By visually checking the status image 63, the user 53 can confirm that the subject 51 has an abnormality in the respiration rate per unit time. The status image 63 being superimposed on the status image 61 makes it clear that the subject 51 has an abnormal respiration rate. As the area of the status image 63 is smaller than the area of the status image 61, the user 53 can visually check both the status image 61 and the status image 63. Therefore, the user 53 can confirm both the behavioral or sleeping status of the subject 51 and the abnormality in the respiration rate per unit time of the subject 51.

Figure 9 is a schematic diagram showing a second example in which an image showing an abnormality in the respiratory rate per unit time of the subject 51 is displayed as a status image. The information processing device 1 causes the display unit 35 of the terminal device 3 to display a status image 63 showing an abnormality in the respiratory rate per unit time of the subject 51, superimposed on the status image 61. At this time, the status image 63 is displayed as a transparent image on the display unit 35 of the terminal device 3. The area of the status image 63 may be equal to the area of the status image 61, or may be larger than the area of the status image 61.

In the second example, the user 53 can also confirm that the subject 51 has an abnormality in his/her breathing rate per unit time by visually checking the status image 63. The status image 63 is superimposed on the status image 61, making it clear that the subject 51 has an abnormal breathing rate. The status image 63 is transparent, allowing the user 53 to visually check both the status image 61 and the status image 63. Thus, the user 53 can confirm both the behavioral or sleeping status of the subject 51 and the status of the subject 51 having an abnormality in his/her breathing rate per unit time.

Figure 10 is a schematic diagram showing a third example in which an image indicating that the subject 51 has an abnormal breathing rate per unit time is displayed as a status image. The information processing device 1 pops up an image 64 including a status image 63 indicating that the subject 51 has an abnormal breathing rate per unit time on the display unit 35 of the terminal device 3 in a position independent of the subject 51. The information processing device 1 also causes the pop-up image 64 to include a subject field 641 including subject information for identifying the subject 51, and a warning message 642 notifying the subject 51 that there is an abnormality in his breathing.

In the third example as well, the user 53 can visually confirm that there is an abnormality in the respiratory rate per unit time of the subject 51 by visually checking the status image 63. The status image 63 is displayed as a pop-up in a position independent of the subject 51, making it easier for the user 53 to recognize the status image 63.

FIG. 11 is a schematic diagram showing a fourth example in which an image showing an abnormality in the respiratory rate per unit time of the subject 51 is displayed as a status image. The information processing device 1 causes the display unit 35 of the terminal device 3 to display a status image 63 showing an abnormality in the respiratory rate per unit time of the subject 51 in a second display area provided for each subject 51 different from the first display area in which the status image 61 is displayed. The position where the status image 63 is arranged in FIG. 11 corresponds to the second display area. The information processing device 1 may also cause the terminal device 3 to display an image showing a normal respiratory rate per unit time of the subject 51 in the second display area. Furthermore, the information processing device 1 may cause the terminal device 3 to display an image showing the respiratory rate per unit time of the subject 51 in the second display area.

In the fourth example, the user 53 can also confirm that the subject 51 has an abnormality in the respiratory rate per unit time by visually checking the status image 63. By displaying the status image 63 at a position different from the status image 61, the user 53 can visually check both the status image 61 and the status image 63. In addition, the status image 63 is displayed in a second display area provided for each subject 51, so that the subject 51 with an abnormal respiratory rate becomes clear.

8 to 11, the information processing device 1 also displays on the display unit 35 of the terminal device 3 a status image showing a state in which the subject 51 has an abnormal heart rate per unit time. The information processing device 1 may display on the display unit 35 of the terminal device 3 an image showing a state in which the subject 51 has a normal heart rate per unit time, or may display on the display unit 35 of the terminal device 3 an image showing the subject 51's respiratory rate per unit time. The user 53 can confirm that the subject 51 has an abnormal heart rate per unit time by visually checking the displayed status image. The state in which the subject 51 has an abnormal heart rate or respiratory rate per unit time is of higher urgency to be notified to the user 53 than the state related to the subject 51's behavior or sleep. By displaying an image showing a state in which the heart rate or respiratory rate is abnormal in a form different from the state related to the subject 51's behavior or sleep, the user 53 can be impressed with information of high urgency to be notified. This improves the visibility of information with high urgency, making it possible to respond quickly to such information and provide appropriate monitoring.

The criteria for determining whether there is an abnormality in the heart rate or respiration rate per unit time are set to appropriate criteria for each subject 51, so that the subject 51 can accurately determine that there is an abnormality in the heart rate or respiration rate. Even if the criteria differ depending on the physical condition of the subject 51, by setting appropriate criteria according to the physical condition, it is possible to accurately determine that there is an abnormality in the heart rate or respiration rate according to the physical condition of the subject 51. This improves the accuracy of the display on the terminal device 3 that there is an abnormality in the heart rate or respiration rate per unit time of the subject 51. This makes it possible to monitor the subject 51's condition and accurately confirm it.

In S23, the information processing device 1 may cause the terminal device 3 to output an alarm in synchronization with the display of an image indicating an abnormality in heart rate or respiratory rate as a status image. For example, the terminal device 3 may output an alarm by changing the color of part or all of the image displayed on the display unit 35 to a specific color, or by blinking part or all of the image. For example, the terminal device 3 may be equipped with an audio output unit and output an alarm by outputting audio. In this way, an alarm indicating that the subject 51 is abnormal is notified to the user 53 in synchronization with the display of the status image.

When heart rate detection is not set, the information processing device 1 does not perform the process of displaying on the terminal device 3 a status image indicating that the subject 51 has an abnormal heart rate per unit time in S23. Similarly, when respiration detection is not set, the information processing device 1 does not perform the process of displaying on the terminal device 3 a status image indicating that the subject 51 has an abnormal respiration rate per unit time. Furthermore, the information processing device 1 may perform the process of S23 for multiple terminal devices 3, or may perform the process of S23 for a specific terminal device 3.

The information processing device 1 waits for confirmation from the user 53 regarding the display of the status image showing the status of the subject 51 (S24). The user 53 visually confirms the status image displayed on the display unit 35 and inputs that the status of the subject 51 has been confirmed by operating the operation unit 34. In S24, the calculation unit 31 accepts the input by the user 53 operating the operation unit 34, and causes the communication unit 36 to transmit information indicating that the confirmation of the status of the subject 51 has been accepted to the information processing device 1, and the information processing device 1 receives the information at the communication unit 15. If there is no acceptance of confirmation from the user 53 (S24: NO), the information processing device 1 repeats S24. Note that the information processing device 1 may end the process if a predetermined time has elapsed without accepting confirmation from the user 53. Also, in S24, when the information processing device 1 displays a status image different from the one displayed without accepting confirmation from the user 53, it may wait for confirmation from the user 53 regarding the display of the different status image.

If confirmation is received from the user 53 (S24: YES), the information processing device 1 records that confirmation has been completed (S25). In S25, the calculation unit 11 records in the history data that confirmation has been completed, in association with the state of the subject 51. After S25 ends, the information processing device 1 ends the process of displaying the state image. The information processing device 1 executes the processes of S21 to S25 as needed.

The information processing device 1 performs a process to display details of the subject 51's condition. FIG. 12 is a flowchart showing an example of the procedure of the process to display details of the subject 51's condition. The information processing device 1 selects an arbitrary subject 51 (S301). In S301, the user 53 operates the operation unit 34, whereby the terminal device 3 accepts the selection of the arbitrary subject 51, and transmits the content of the accepted selection to the information processing device 1. For example, the user 53 operates the operation unit 34 to specify any of the status images 61 or subject columns 62 as shown in FIG. 4 and FIG. 8 to FIG. 11, thereby accepting the selection of the arbitrary subject 51 on the terminal device 3. The information processing device 1 receives the content of the selection via the communication unit 15. The calculation unit 11 selects the subject 51 according to the received content of the selection.

The information processing device 1 causes the terminal device 3 to display the first graph, the second graph, and the status history for the selected subject 51 (S302). FIG. 13 is a schematic diagram showing an example in which the first graph 65, the second graph 66, and the status history 67 are displayed. The first graph 65 is a scatter plot showing the heart rate and respiration rate per unit time based on data acquired at each time for the selected subject 51. In the figure, "heart rate" refers to the heart rate, and "respiration" refers to the respiration rate. The horizontal axis of the first graph 65 indicates time, and the vertical axis indicates the heart rate or respiration rate per unit time. The heart rate and respiration rate are numerical values based on data acquired by the information processing device 1 from the sensor device 2 and stored in the memory unit 13.

The first graph 65 displays the standards for heart rate and respiration rate that are set at each time. The first graph 65 displays a line 654 indicating the upper limit of the heart rate, a line 655 indicating the lower limit of the heart rate, a line 656 indicating the upper limit of the respiration rate, and a line 657 indicating the upper limit of the respiration rate, thereby indicating the standards for the heart rate and respiration rate that have been set. In the example shown in FIG. 13, the value corresponding to the line 654 indicating the upper limit of the heart rate differs before and after the time 5:00. This indicates that the upper limit of the heart rate has been changed due to the settings after the time 5:00.

The second graph 66 is a bar graph showing the state of the selected subject 51 at each time based on data acquired at each time for the subject 51. The horizontal axis of the second graph 66 shows the same time as the first graph 65. In the example shown in FIG. 13, the second graph 66 shows whether the state of the subject 51 at each time was in bed, absent, or moving. The second graph 66 may also show other states of the subject 51 at each time.

The status history 67 is a history of each of a plurality of types of status of the selected subject 51. The status history 67 includes the status of the subject 51 at each time, associated with a time. The time associated with the status of the subject 51 is the time associated with the status of the subject 51 in the history data. In addition, in the status history 67, the status of the subject 51 at each time is associated with whether the status of the subject 51 is unconfirmed or confirmed. In addition, a camera icon that is a link to a photographed image of the subject 51 at the time associated with the status of the subject 51 included in the status history 67 is displayed in association with the status of the subject 51 included in the status history 67. The link to the photographed image may be displayed in a form other than a camera icon. Furthermore, when a setting is made to not notify the subject 51 of a specific status so that a status image indicating the status is not displayed, the status history 67 associates the specific status of the subject 51 with being not notified.

In S302, the calculation unit 11 generates a first graph 65 based on the data obtained from the sensor device 2 and the setting data for the selected subject 51, generates a second graph 66 based on the history data, and generates a state history 67 based on the setting data and the history data. The calculation unit 11 causes the communication unit 15 to transmit the generated first graph 65, second graph 66, and state history 67 to the terminal device 3. The terminal device 3 receives the first graph 65, second graph 66, and state history 67 via the communication unit 36, and the calculation unit 31 displays the received first graph 65, second graph 66, and state history 67 on the display unit 35. The information processing device 1 may generate an image including the first graph 65, second graph 66, and state history 67 and transmit it to the terminal device 3, and the terminal device 3 may display the received image. The information processing device 1 may transmit data necessary to generate the first graph 65, the second graph 66, and the state history 67 to the terminal device 3, and the terminal device 3 may generate the first graph 65, the second graph 66, and the state history 67 based on the received data and display them on the display unit 35.

By displaying the first graph 65, the user 53 can confirm the subject's 51 heart rate or respiration rate per unit time at each time, as well as the standard heart rate and respiration rate set at each time. By displaying the second graph 66, the user 53 can confirm the time distribution of the subject's 51's condition. By displaying the condition history 67, the selected condition of the subject 51 is displayed in association with the time at which the subject 51 entered that condition. The user 53 can confirm what condition the subject 51 was in at each time.

The information processing device 1 sets whether or not to display the subject's 51 condition history for each condition category that categorizes multiple types of conditions of the subject 51. One condition category includes one or more types of conditions. In this embodiment, one of the condition categories is a first condition category that includes conditions related to the behavior of the subject 51. One of the condition categories is a second condition category that includes conditions related to the sleep of the subject 51. One of the condition categories is a third condition category that includes conditions corresponding to the subject's 51 heart rate or respiratory rate per unit time. The setting contents are recorded in the setting data.

The state history 67 includes a setting field 671 that displays a setting of whether or not to display the state history of the subject 51 included in each state category. FIG. 13 shows an example in which the state history of the subject 51's behavior, the state of the subject 51's sleep, and the state according to the subject 51's heart rate or respiration rate per unit time are set to be displayed. For this reason, the state history 67 shown in FIG. 13 includes the history of the states of "body movement," "getting up," "getting up," and "not being there," which are one of the states of the subject 51's behavior. As the state of the subject 51's behavior, the state history 67 may include the state of "in bed" or "in bed." As the state of the subject 51's sleep, the state history 67 may include the history of the state of "sleeping" and the state of "awake." As the state of the subject 51's sleep, the state history 67 may include the state of "light sleep." In addition, in the setting field 671, the first state category including the state of the subject 51's behavior may be displayed as "behavior," "getting out of bed," or "being there/not being there." In the setting field 671, a second status category including a state related to the sleep of the subject 51 may be displayed as "Sleep." In the setting field 671, a third status category including a state according to the heart rate or respiration rate per unit time of the subject 51 may be displayed as "Heart Rate/Respiration," "Notification," "Alert," or "Abnormal."

The state history 67 shown in FIG. 13 also includes a state of "heart rate increase" in which the heart rate per unit time of the subject 51 is increasing as a state corresponding to the heart rate or respiratory rate per unit time of the subject 51. The state history 67 may also include a state in which the heart rate per unit time is decreasing, a state in which the respiratory rate per unit time is increasing, or a state in which the respiratory rate per unit time is decreasing as a state corresponding to the heart rate or respiratory rate per unit time of the subject 51.

The information processing device 1 determines whether or not it is set to perform heartbeat detection or respiration detection (S303). In S303, the calculation unit 11 determines whether or not it is set to perform heartbeat detection or respiration detection based on the setting data. If it is set to not perform either heartbeat detection or respiration detection (S303: NO), the information processing device 1 proceeds to S310, which will be described later.

If it is set to perform heart rate detection or respiration detection (S303: YES), the information processing device 1 determines whether it is set to display the state history 67 according to the subject 51's heart rate or respiration rate per unit time (S304). In S304, the calculation unit 11 makes a determination based on the setting data. If it is set not to display the state history 67 according to the heart rate or respiration rate (S304: NO), the information processing device 1 proceeds to S310 described below.

When the state history 67 is set to display the state history according to the subject 51's heart rate or respiration rate per unit time (S304: YES), the information processing device 1 highlights the portions of the first graph 65 and the second graph 66 that correspond to the heart rate or respiration rate that deviates from the standard (S305). In S305, the calculation unit 11 adjusts the first graph 65 so as to highlight the portions in the first graph 65 that indicate the heart rate or respiration rate per unit time that deviates from the set standard. The calculation unit 11 causes the communication unit 15 to transmit the adjusted first graph 65 to the terminal device 3. The terminal device 3 receives the adjusted first graph 65 and displays it on the display unit 35. In this way, the information processing device 1 highlights the portions in the first graph 65 that indicate the heart rate or respiration rate per unit time that deviates from the set standard.

In addition, in S305, the calculation unit 11 adjusts the second graph 66 so as to highlight portions showing times when the heart rate or respiration rate per unit time deviates from the set standard. The calculation unit 11 causes the communication unit 15 to transmit the adjusted second graph 66 to the terminal device 3. The terminal device 3 receives the adjusted second graph 66 and displays it on the display unit 35. In this way, the information processing device 1 highlights portions in the second graph 66 showing times when the heart rate or respiration rate per unit time deviates from the set standard.

In the example shown in FIG. 13, the part of the first graph 65 that indicates the heart rate per unit time that deviates from the standard is the highlighted part 651. Also, the part of the second graph 66 that indicates the time when the heart rate per unit time deviates from the standard is the highlighted part 661. The information processing device 1 highlights the part in the highlighted part 651 by changing the color, size, or shape of the point that indicates the heart rate or respiration rate that deviates from the standard from that of the other parts, or by blinking the point that indicates the heart rate or respiration rate that deviates from the standard. The information processing device 1 highlights the part in the highlighted part 661 that corresponds to the heart rate or respiration rate that deviates from the standard by changing the color, size, or shape of the part that corresponds to the heart rate or respiration rate that deviates from the standard from that of the other parts, or by blinking the part that corresponds to the heart rate or respiration rate that deviates from the standard.

The highlighting improves visibility of the occurrence of an abnormality in the subject's 51 heart rate or respiration rate per unit time that deviates from the standard, and visually notifies the user that an abnormality has occurred. The user 53 can easily confirm that an abnormality has occurred in the subject's 51 heart rate or respiration rate per unit time that deviates from the standard. The user 53 can also easily confirm the time at which the abnormality occurred.

The information processing device 1 waits for the acceptance of the designation of the highlighted portion in the first graph 65 or the second graph (S306). The user 53 operates the operation unit 34 to designate the highlighted portion in the first graph 65 or the second graph. For example, the highlighted portion 651 or 661 is designated by placing the cursor over the highlighted portion 651 or 661. In S24, the calculation unit 31 accepts the designation of the highlighted portion 651 or 661 by the user 53 operating the operation unit 34, and causes the communication unit 36 to transmit information indicating that the designation of the highlighted portion 651 or 661 has been accepted to the information processing device 1. The information processing device 1 receives the information at the communication unit 15. If the designation of the highlighted portion has not been accepted (S306: NO), the information processing device 1 proceeds to the process at S310 described later.

When the designation of the highlighted portion is accepted (S306: YES), the information processing device 1 causes the terminal device 3 to display details of the state in which the subject 51 has an abnormal heart rate or respiration rate per unit time (S307). At this time, the information processing device 1 also causes the terminal device 3 to display the set reference value of the heart rate or respiration rate and a link to the captured image. FIG. 14 is a schematic diagram showing a first example of displaying details of the state in which the subject 51 has an abnormal heart rate per unit time. The highlighted portion 651 is designated by placing a cursor 652 on the highlighted portion 651 in the first graph 65, and a display field 653 of the details of the abnormality is displayed. In the display field 653, the time corresponding to the position of the cursor 652 and the numerical value of the heart rate at that time are displayed as details of the state in which the heart rate is abnormal.

Display field 653 displays the set value that is the basis for notifying of an abnormality, i.e., the preset reference value for the heart rate or respiration rate. FIG. 14 shows an example in which the heart rate exceeds the upper limit, and the set upper limit value for the heart rate is displayed as the set value for notifying of an abnormality. By displaying the reference value for the heart rate or respiration rate, user 53 can confirm how much the abnormal heart rate or respiration rate deviates from the reference. Also displayed in display field 653 is a camera icon. The camera icon is a link to a captured image of subject 51 at the time corresponding to specified highlighted portion 651. The link to the captured image may be in a form other than a camera icon, such as a link consisting of text.

15 is a schematic diagram showing a second example of displaying details of an abnormal heart rate per unit time state of the subject 51. The highlighted portion 661 in the second graph 66 is designated by placing a cursor 662 over the highlighted portion 661, and a display field 663 for displaying details of the abnormality is displayed. In the display field 663, the time corresponding to the position of the cursor 662 and the heart rate value at that time are displayed as details of the abnormal heart rate state. In addition, the display field 663 displays the setting value that is the basis for notifying the abnormality. Specifically, the upper limit of the heart rate that is set is displayed. Furthermore, the display field 663 displays a camera icon that is a link to a captured image. Even if the subject 51 has an abnormal respiration rate per unit time, details of the abnormal respiration rate state are displayed in the same manner as in FIG. 14 or FIG. 15.

In addition, along with details of the state in which there is an abnormality in the heart rate or respiration rate, only one of the set value for notifying the abnormality and the reference value for the heart rate or respiration rate may be displayed. Alternatively, only details of the state in which there is an abnormality in the heart rate or respiration rate may be displayed, and the set value for notifying the abnormality and the reference value for the heart rate or respiration rate may not be displayed.

In S307, the calculation unit 11 corrects the first graph 65 or the second graph 66 based on the data obtained from the sensor device 2 and the setting data for the selected subject 51. At this time, the calculation unit 11 corrects the first graph 65 or the second graph 66 so as to add details of the state in which the heart rate or respiration rate is abnormal, the set reference value of the heart rate or respiration rate, and a link to the captured image. The calculation unit 11 causes the communication unit 15 to transmit the corrected first graph 65 or the second graph 66 to the terminal device 3. The terminal device 3 receives the corrected first graph 65 or the second graph 66 via the communication unit 36, and the calculation unit 31 displays the corrected first graph 65 or the second graph 66 on the display unit 35. The information processing device 1 may transmit information including the time and the value of the heart rate or respiration rate to the terminal device 3, and the terminal device 3 may correct the first graph 65 or the second graph 66 based on the received information. Through the processing of S306 to S307, the user 53 can specify the highlighted portion 651 or 661 and check the details of the abnormality in the subject's 51 heart rate per unit time.

The information processing device 1 waits for the acceptance of the designation of a link to the photographed image (S308). With the display field 653 or 663 displayed, the user 53 operates the operation unit 34 to designate a link to the photographed image. For example, the link to the photographed image is designated by hovering the cursor over the camera icon displayed in the display field 653 or 663. In S308, the calculation unit 31 accepts the designation of a link to the photographed image by the user 53 operating the operation unit 34, and causes the communication unit 36 to transmit information indicating that the designation of a link to the photographed image has been accepted to the information processing device 1. The information processing device 1 receives the information at the communication unit 15. If the designation of a link to the photographed image has not been accepted (S308: NO), the information processing device 1 proceeds to the process of S310 described later.

When the designation of the link to the photographed image is accepted (S308: YES), the information processing device 1 displays the photographed image of the subject 51 on the terminal device 3 (S309). In S309, the calculation unit 11 transmits a request for a photographed image of the subject 51 at the time corresponding to the designated highlighted portion 651 or 661 to the camera device 7. The camera device 7 selects the requested photographed image from among the photographed images stored, and transmits the selected photographed image to the information processing device 1. The information processing device 1 receives the photographed image transmitted from the camera device 7 via the communication unit 15, and the calculation unit 11 causes the communication unit 15 to transmit the received photographed image to the terminal device 3. In a form in which the photographed image is stored in the information processing device 1, the information processing device 1 selects the requested photographed image from among the photographed images stored, and transmits the selected photographed image to the terminal device 3. The terminal device 3 receives the photographed image via the communication unit 36, and the calculation unit 31 displays the photographed image on the display unit 35.

FIG. 16 is a schematic diagram showing an example of a display of a captured image. A captured image of the subject 51 captured by the camera device 7 is displayed. More specifically, a captured image of the subject 51 when there is an abnormality in the heart rate or respiratory rate per unit time is displayed. The user 53 can check the condition of the subject 51 when there is an abnormality in the heart rate or respiratory rate.

The monitoring system 100 may be configured such that the terminal device 3 and the camera device 7 communicate with each other in a peer-to-peer manner, and the captured image is displayed on the terminal device 3 without the intervention of the information processing device 1. In this configuration, in S308, the terminal device 3 waits for a link to the captured image to be accepted. When a link to the captured image is accepted, the calculation unit 31 transmits a request for a captured image of the subject 51 at the time corresponding to the specified highlighted portion 651 or 661 to the camera device 7. In S309, the camera device 7 transmits the requested captured image to the terminal device 3, the terminal device 3 receives the captured image via the communication unit 36, and the calculation unit 31 displays the captured image on the display unit 35.

The monitoring system 100 may perform processing to display the heart rate or respiratory condition of the subject 51 even when a portion other than the highlighted portion of the first graph 65 or the second graph 66 is specified. For example, when a portion other than the highlighted portion of the first graph 65 or the second graph 66 is specified, the heart rate or respiratory rate value or a link to a captured image may be displayed. For example, when a link to a captured image is specified, a captured image of the subject 51 at the time corresponding to the portion specified in the first graph 65 or the second graph 66 may be displayed.

The information processing device 1 waits for a setting instruction to be received as to whether or not to display the condition history of the subject 51 (S310). In S310, the user 53 who has visually recognized the condition history 67 operates the operation unit 34, and the terminal device 3 receives the setting instruction. For example, as shown in FIG. 13, the setting field 671 includes a check box for each condition category, and the user 53 uses the check box to input a setting instruction as to whether or not to display the condition history included in each condition category. The terminal device 3 transmits the setting instruction to the information processing device 1, and the information processing device 1 receives the setting instruction via the communication unit 15. By receiving the setting instruction, the calculation unit 11 accepts the setting instruction. If the setting instruction has not been accepted (S310: NO), the information processing device 1 proceeds to S315, which will be described later.

If a setting instruction is received (S310: YES), the information processing device 1 performs setting according to the received instruction (S311). In S311, the calculation unit 11 performs setting by recording in the setting data whether or not to display the history of the states included in each state category according to the received instruction. The information processing device 1 then adjusts the state history 67 according to the setting (S312). In S312, the calculation unit 11 adjusts the state history 67 so that the history of the states set to be displayed among the multiple types of states of the subject 51 is displayed and the history of the states set to not be displayed is not displayed. The calculation unit 11 causes the communication unit 15 to transmit the adjusted state history 67 to the terminal device 3, the terminal device 3 receives the adjusted state history 67 via the communication unit 36, and the calculation unit 31 displays the adjusted state history 67 on the display unit 35. The information processing device 1 may generate an image including the adjusted state history 67 and transmit it to the terminal device 3, and the terminal device 3 may display the received image. The information processing device 1 may transmit data necessary to generate the adjusted state history 67 to the terminal device 3, and the terminal device 3 may generate the state history 67 based on the received data and display it on the display unit 35.

FIG. 17 is a schematic diagram showing a first example of an adjusted state history 67. FIG. 17 shows state history 67 that is set to display state history related to the behavior of subject 51 and state history corresponding to the heart rate or respiratory rate of subject 51 per unit time, and not display state history related to the sleep of subject 51. The wording "heart rate/respiration" included in setting field 671 indicates a state corresponding to the heart rate or respiratory rate of subject 51 per unit time. State history 67 includes state history related to the behavior of subject 51 and state history corresponding to the heart rate or respiratory rate of subject 51 per unit time, and does not include other state history.

Figure 18 is a schematic diagram showing a second example of an adjusted state history 67. Figure 18 shows state history 67 that is set to display state history according to the subject 51's heart rate or respiratory rate per unit time, and not display state history related to the subject 51's sleep and state history related to the subject 51's behavior. State history 67 includes state history according to the subject 51's heart rate or respiratory rate per unit time, and does not include other state history.

As shown in Figures 17 and 18, a status history 67 is displayed that includes the history of some types of statuses of the subject 51, but does not include the history of other types of statuses. If the history of all types of statuses were included in the status history 67, information with too much diversity would be displayed, compromising the visibility of the information. By displaying the history of some types of statuses and not displaying the history of other types of statuses, the diversity of the displayed information is reduced and the visibility of the information is improved. By limiting the status categories that include the statuses of the subject 51 to be displayed, the user 53 can improve the visibility of the required information and easily obtain the required information.

A state corresponding to the subject's 51 heart rate or respiratory rate per unit time has a greater impact on the subject 51 than other states, and is therefore of higher importance. As shown in FIG. 17, by reducing the number of types of the subject's 51 states that are displayed, it becomes easier for the user 53 to check the history of states of high importance. Also, as shown in FIG. 18, when the history of states corresponding to the subject's 51 heart rate or respiratory rate per unit time is displayed and the history of other states is not displayed, the visibility of information of high importance is improved. The user 53 can easily check the history of states of high importance.

The information processing device 1 determines whether the settings have been changed so that the history of the state according to the heart rate or respiration rate per unit time of the subject 51 is not displayed (S313). In S313, the calculation unit 11 makes a determination based on the setting data. If the settings have not been changed so that the history of the state according to the heart rate or respiration rate is not displayed (S313: NO), the information processing device 1 proceeds to S315, which will be described later.

When the settings are changed so that the history of the state according to the heart rate or respiration rate is not displayed (S313: YES), the information processing device 1 stops highlighting the first graph 65 and the second graph 66 (S314). In S314, the calculation unit 11 adjusts the first graph 65 and the second graph 66 so that the first graph 65 and the second graph 66 are not highlighted. The calculation unit 11 causes the communication unit 15 to transmit the adjusted first graph 65 and second graph 66 to the terminal device 3. The terminal device 3 receives the adjusted first graph 65 and second graph 66 and displays them on the display unit 35.

FIG. 19 is a schematic diagram showing an example of the first graph 65, the second graph 66, and the state history 67 with highlighting disabled. The state history 67 is set to display the state history related to the sleep of the subject 51 and the state history related to the behavior of the subject 51, and not to display the state history according to the subject 51's heart rate or respiration rate per unit time. The state history 67 does not include the state history according to the subject 51's heart rate or respiration rate per unit time. The first graph 65 does not include the highlighted portion 651, and is similar to the other portions in that it shows a heart rate or respiration rate that deviates from the standard. The second graph 66 does not include the highlighted portion 661.

In this way, when the information processing device 1 does not display the history of the state according to the heart rate or respiration rate, it does not highlight the first graph 65 and the second graph 66. When the history of the state according to the heart rate or respiration rate is not displayed, there is no need to pay attention to the state according to the heart rate or respiration rate. Information that does not require attention is not highlighted, making it easier for the user 53 to check other information that requires attention.

The information processing device 1 waits for the acceptance of the designation of a link to a photographed image associated with the state of the subject 51 included in the state history 67 (S315). The user 53 designates a link to a photographed image associated with the state of the subject 51 by operating the operation unit 34. For example, the link to the photographed image is designated by hovering the cursor over one of the camera icons associated with the state of the subject 51 at each time included in the state history 67. In S315, the calculation unit 31 accepts the designation of a link to a photographed image by the user 53 operating the operation unit 34, and transmits information indicating that the designation of the link to the photographed image has been accepted to the information processing device 1. The information processing device 1 receives the information at the communication unit 15. If the designation of a link to a photographed image has not been accepted (S315: NO), the information processing device 1 ends the process of displaying the details of the state of the subject 51.

When a link to a captured image is accepted (S315: YES), the information processing device 1 causes the terminal device 3 to display the captured image of the subject 51 (S316). In S316, the calculation unit 11 transmits a request for a captured image of the subject 51 at the time associated with the state of the subject 51 to which the specified link is associated, to the camera device 7, and the camera device 7 transmits the requested captured image to the information processing device 1. The information processing device 1 receives the captured image, and the calculation unit 11 causes the communication unit 15 to transmit the captured image to the terminal device 3. In a form in which the captured image is stored in the information processing device 1, the information processing device 1 transmits the stored captured image to the terminal device 3. The terminal device 3 receives the captured image via the communication unit 36, and the calculation unit 31 displays the captured image on the display unit 35.

The monitoring system 100 may be in a form in which the terminal device 3 and the camera device 7 communicate with each other in a peer-to-peer manner, and the captured image is displayed on the terminal device 3 without the intervention of the information processing device 1. In this form, in S315, the terminal device 3 waits for the acceptance of a designation of a link to a captured image. When the designation of a link to a captured image is accepted, the calculation unit 31 transmits to the camera device 7 a request for a captured image of the subject 51 at the time associated with the state of the subject 51 associated with the designated link. In S316, the camera device 7 transmits the requested captured image to the terminal device 3, the terminal device 3 receives the captured image via the communication unit 36, and the calculation unit 31 displays the captured image on the display unit 35.

As in the example shown in FIG. 16, the captured image is displayed. The captured image captured by the camera device 7 of the subject 51 when the subject 51 is in each of the states displayed in the status history 67 is displayed. When the subject 51 is absent, a captured image showing the bed 52 or the room used by the subject 51 may be displayed. The user 53 can check the state of the subject 51 when the subject 51 is in each of the states displayed in the status history 67. After S316 is completed, the information processing device 1 ends the process of displaying the details of the subject 51's status.

As described above in detail, in this embodiment, the information processing device 1 causes the terminal device 3 to display the status image 63 indicating the state of the subject 51 according to the heart rate or respiratory rate per unit time in a manner different from the status image 61 indicating other states of the subject 51. Meanwhile, the information processing device 1 causes the terminal device 3 to display the status history 67, thereby displaying the history of the state of the subject 51 according to the heart rate or respiratory rate per unit time and the history of other states of the subject 51 in the same manner. In a scene in which a status image indicating the state of the subject 51 is displayed in real time, the visibility of information with a high degree of urgency of notification is improved, whereas in a scene in which the status history 67 is displayed, since there is no urgency, the user 53 can equally confirm multiple types of states of the subject 51.

In this embodiment, an example is shown in which a standard is set and a status image 63, a first graph 65, and a status history 67 are displayed for both a state according to the subject 51's heart rate per unit time and a state according to the subject 51's respiratory rate per unit time. The information processing device 1 may be configured to set a standard and display a status image 63, a first graph 65, and a status history 67 for only one of a state according to the subject 51's heart rate per unit time and a state according to the subject 51's respiratory rate per unit time. The information processing device 1 may also be configured to set a standard and display a status image 63, a first graph 65, and a status history 67 for a specific type of numerical value other than the subject 51's heart rate and respiratory rate based on data acquired using the sensor device 2. For example, the information processing device 1 may be configured to set a standard and display a status image 63, a first graph 65, and a status history 67 for the subject 51's pulse rate or body temperature.

In the present embodiment, an example is shown in which the state of the subject 51 regarding the behavior, the state of the subject 51 regarding the sleep, and the state according to the heart rate or respiratory rate of the subject 51 are determined, and a status image is displayed. The monitoring system 100 may be configured to determine other types of states of the subject 51 and display a status image. In the present embodiment, a form is shown in which the information processing device 1 causes the terminal device 3 to display various information by the processes of S11 to S13, S21 to S25, and S301 to S316. The information processing device 1 may be provided with a display unit and may display various information by itself. Alternatively, the monitoring system 100 may be configured in such a way that the terminal device 3 executes part or all of the processing of the information processing device 1 described above. In this form, the calculation unit 31 executes processing corresponding to part or all of the processing of the information processing device 1 according to the computer program 331.

The present invention is not limited to the contents of the above-described embodiment, and various modifications are possible within the scope of the claims. In other words, embodiments obtained by combining technical means that are appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

The matters described in each embodiment can be combined with each other. In addition, the independent claims and dependent claims described in the claims can be combined with each other in any and all combinations regardless of the citation format. Furthermore, the claims use a format in which a claim cites two or more other claims (multi-claim format), but this is not limited to this. They may also be written in a format in which multiple claims cite at least one other claim (multi-multi-claim).

REFERENCE SIGNS LIST 100 Monitoring system 1 Information processing device 10 Recording medium 11 Calculation unit 13 Storage unit 131 Computer program 2 Sensor device 3 Terminal device 31 Calculation unit 33 Storage unit 331 Computer program 4 Communication network 51 Target person 53 User 61, 63 Status image 65 First graph 66 Second graph 67 Status history

Claims (19)

Data is collected from each subject using sensors,
A status image showing the status of each subject based on the acquired data is displayed for each subject;
When an arbitrary target person is selected, an instruction is received to set whether or not to display each history of a plurality of types of status of the selected target person;
Follow the instructions to set up the device.
A computer program that causes a computer to execute a process of displaying a status history that includes a history of a status that is set to be displayed among a plurality of types of status of a subject, and does not include a history of a status that is set not to be displayed. The computer program product according to claim 1, further comprising a program for causing a computer to execute a process of displaying, as the status history, a history associating the status of the selected subject with a time at which the selected subject entered the status. receiving a confirmation of the display of the status image or a notification synchronized with the display of the status image;
The computer program product according to claim 1 , further comprising: displaying information indicating whether or not the confirmation has been received in association with the status history. receiving, for each subject, an instruction to set whether or not to display the status image or to notify the subject in synchronization with the display of the status image, for each of a plurality of types of status of the subject;
Follow the instructions to set up the device.
The computer program product according to claim 1 , further comprising: displaying information indicating whether or not to display the status image or to provide a notification synchronized with the display of the status image in association with the status history. When an arbitrary subject is selected, for each status category categorized to include one or more types of status of the subject, an instruction is received to set whether or not to display a history of the status of the subject included in the status category;
The computer program product according to claim 1 , further comprising: a setting unit configured to set the setting unit in accordance with the received instruction. one of the status categories is a first status category including a status related to the subject's behavior,
The computer program according to claim 5, characterized in that the computer is caused to execute a process of displaying the status history including at least one of a status history of a status in which the subject is present and a status history in which the subject is absent when the setting is set to display the status history of the subject included in the first status category. one of the condition categories is a second condition category including a condition related to the subject's sleep;
The computer program according to claim 5, characterized in that the computer is caused to execute a process of displaying the condition history including at least one of a sleep state and a wakefulness state history when the condition history of the subject included in the second condition category is set to be displayed. one of the condition categories is a third condition category including at least one of a condition according to a heart rate per unit time of the subject and a condition according to a respiratory rate per unit time of the subject;
The computer program according to claim 5, characterized in that the computer is caused to execute a process of displaying the condition history including a history of at least one of an increasing heart rate, a decreasing heart rate, an increasing respiratory rate, and a decreasing respiratory rate when the condition history of the subject included in the third condition category is set to be displayed. receiving instructions to set criteria for an increased heart rate and a decreased heart rate for each subject;
The computer program product according to claim 8, further comprising a step of causing a computer to execute a process of performing settings according to the received instructions. receiving instructions to set criteria for an increased respiratory rate and a decreased respiratory rate for each subject;
The computer program product according to claim 8, further comprising a step of causing a computer to execute a process of performing settings according to the received instructions. the state of the subject shown in the state image includes at least one of a state according to the subject's heart rate per unit time and a state according to the subject's respiratory rate per unit time, a state related to the subject's behavior, and a state related to the subject's sleep;
displaying, as the status image, an image showing a state related to the subject's behavior or an image showing a state related to the subject's sleep in a first display area provided for each subject;
The computer program according to claim 1, characterized in that the computer is caused to execute a process of displaying, as the status image, at least one of an image showing a state of the subject depending on the heart rate per unit time of the subject and an image showing a state of the subject depending on the respiratory rate per unit time of the subject, superimposed on the image displayed in the first display area. The computer program according to claim 11, characterized in that the computer is caused to execute a process of displaying at least one of an image showing a state of the subject corresponding to the heart rate per unit time and an image showing a state of the subject corresponding to the respiratory rate per unit time in an area smaller than an image showing a state related to the subject's behavior and an image showing a state related to the subject's sleep. The computer program according to claim 11, characterized in that the computer is caused to execute a process of displaying at least one of an image indicating a state corresponding to the subject's heart rate per unit time and an image indicating a state corresponding to the subject's respiratory rate per unit time as an image having at least a portion that is transparent so that the image displayed in the first display area can be viewed. the state of the subject shown in the state image includes at least one of a state according to the subject's heart rate per unit time and a state according to the subject's respiratory rate per unit time, a state related to the subject's behavior, and a state related to the subject's sleep;
displaying, as the status image, an image showing a state related to the subject's behavior or an image showing a state related to the subject's sleep in a first display area provided for each subject;
The computer program according to claim 1, characterized in that the computer is caused to execute a process of displaying, as the status image, at least one of an image showing a state of the subject corresponding to the heart rate per unit time of the subject and an image showing a state of the subject corresponding to the respiratory rate per unit time of the subject, in a pop-up display at a position independent of the subject. the state of the subject shown in the state image includes at least one of a state according to the subject's heart rate per unit time and a state according to the subject's respiratory rate per unit time, a state related to the subject's behavior, and a state related to the subject's sleep;
displaying, as the status image, an image showing a state related to the subject's behavior or an image showing a state related to the subject's sleep in a first display area provided for each subject;
The computer program according to claim 1, characterized in that the computer is caused to execute a process of displaying, as the status image, at least one of an image showing a state of the subject depending on the heart rate per unit time of the subject and an image showing a state of the subject depending on the respiratory rate per unit time of the subject in a second display area provided for each subject, different from the first display area. The state of the subject shown in the state image includes at least one of a state according to the subject's heart rate per unit time and a state according to the subject's respiratory rate per unit time, and other states;
As the status image, at least one of an image showing a state according to the subject's heart rate per unit time and an image showing a state according to the subject's respiratory rate per unit time is displayed in a format different from that of the images showing the other states;
The computer program according to claim 1, characterized in that, when displaying the condition history, the computer is caused to execute a process of displaying at least one of the condition history according to the subject's heart rate per unit time and the condition history according to the subject's respiratory rate per unit time, and the other condition history in the same format. displaying a link to a captured image of the subject at a time associated with the state of the subject included in the state history;
The computer program product according to claim 1 , further comprising: when the link specification is accepted, causing the computer to execute a process of displaying the photographed image. Data is collected from each subject using sensors,
A status image showing the status of each subject based on the acquired data is displayed for each subject;
When an arbitrary target person is selected, an instruction is received to set whether or not to display each history of a plurality of types of status of the selected target person;
Follow the instructions to set up the device.
An information processing method comprising: displaying a status history including a history of a status that is set to be displayed among a plurality of types of status of a subject, and not including a history of a status that is set not to be displayed. A calculation unit is provided,
The calculation unit is
Data is collected from each subject using sensors,
A status image showing the status of the subject based on the acquired data is displayed on a display unit for each subject;
When an arbitrary target person is selected, an instruction is received to set whether or not to display each history of a plurality of types of status of the selected target person;
Follow the instructions to set up the device.
An information processing device that displays a status history including a history of a status that is set to be displayed among a plurality of types of status of a subject, and does not include a history of a status that is set not to be displayed.

Images