KR102669060B1 - Method and apparatus for estimating location of tracing target - Google Patents

Abstract

According to an embodiment of the present invention, a method for estimating a position of a tracking target to provide a care service by a robot comprises the steps of: generating map information for an area where the care service is provided; generating face information, body information, and clothing information from input image information, and identifying the tracking target based on at least one of the face information, the body information, and the clothing information; and specifying a position of the tracking target on the map information based on a position and a direction in which the image information is input on the map information.

Description

Method and apparatus for estimating the location of a tracking target {METHOD AND APPARATUS FOR ESTIMATING LOCATION OF TRACING TARGET}

The present invention relates to a method and device for estimating the location of a tracking target to provide care services based on map information, image information, device signals, etc.

In accordance with technological advancements in the fields of various sensors and artificial intelligence, labor shortages, and the aging of society, it is necessary to perform repetitive tasks while moving within a defined space, provide services tailored to various situations, or provide customized care services to the elderly, etc. Robotic technology is attracting attention.

For example, in the case of a robot that provides care services to the elderly, it moves within the activity space where the care recipient lives, providing various content to the care recipient, including various daily life reminders, medication reminders, and cognitive training, as well as abnormal activities and emergency situations. It can provide comprehensive functions such as monitoring and reporting.

For robots that provide interpersonal services, including the provision of care services, basic technology is required to recognize the activity space and move while quickly and accurately identifying and locating service recipients within that space.

KR 10-2397078 B1 (2022.05.09.)

A method of identifying a tracking target to provide care services in space based on map information and image information about the area providing care services and estimating the location of the tracking target by specifying the location of the tracking target on the map information. and devices.

An object is to provide a method and device for estimating the location of a tracking target that can specify the location of the tracking target on map information based on map information and a signal strength indicator from a device worn or possessed by the tracking target.

However, the technical challenges that this embodiment aims to achieve are not limited to the technical challenges described above, and other technical challenges may exist.

As a means of achieving the above-described technical problem, a method of estimating the location of a tracking target to provide care services by a robot according to an embodiment of the present invention generates map information about the area providing the care services. steps; Generating face information, body information, and clothing information from input image information, and identifying the tracking target based on at least one of the face information, the body information, and the clothing information; and specifying the location of the tracking target on the map information based on the location and direction of receiving the image information on the map information.

An apparatus for estimating the location of a tracking target to provide a care service according to an embodiment of the present invention includes a map information generator that generates map information about an area providing the care service; a tracking object identification unit that generates face information, body information, and clothing information from input image information, and identifies a tracking target based on at least one of the face information, the body information, and the clothing information; and a location specification unit that specifies the location of the tracking target on the map information based on the position and direction of receiving the image information on the map information.

In addition, a method of estimating the location of a tracking target to provide care services by a robot according to another embodiment of the present invention generates a group of location candidates for the tracking target based on map information about the area providing the care service. steps; Measuring a signal strength index of a device worn or possessed by the tracking target at a plurality of points on the map information; updating the location candidates based on a change in location between the plurality of points and a change in the signal strength index; and specifying the location of the tracking target on the map information based on the updated location candidate group.

An apparatus for estimating the location of a tracking target to provide a care service according to another embodiment of the present invention includes a location candidate group generator that generates a location candidate group of the tracking target based on map information about the area providing the care service. ; a signal strength indicator measuring unit that measures the signal strength indicator of a device worn or possessed by the tracking target at a plurality of points on the map information; a location candidate group updating unit that updates the location candidate group based on a change in position between the plurality of points and a change in the signal strength index; and a location specification unit that specifies the location of the measurement object on the map information based on the updated location candidate group.

The above-described means for solving the problem are merely illustrative and should not be construed as limiting the present invention. In addition to the exemplary embodiments described above, there may be additional embodiments described in the drawings and detailed description of the invention.

According to one of the above-described means for solving the problems of the present invention, a robot identifies a tracking target to provide care services in space based on map information and image information about the area where care services are provided, and tracks its location using map information. By specifying the image, the location of the tracking target can be accurately estimated. In particular, by combining face information, body information, and clothing information to identify the tracking target, reliability of identification and location estimation can be secured in various situations.

In addition, the location of the tracking target can be determined by specifying the location of the tracking target in space on the map information based on map information about the area where the robot provides care services and the signal strength index from the device worn or carried by the tracking target. It can be estimated accurately.

1 is a configuration diagram of a care service provision system according to an embodiment of the present invention.
Figure 2 is a flowchart of a method for estimating the location of a tracking target according to an embodiment of the present invention.
Figure 3 is a diagram specifically showing the step of identifying a tracking target in one embodiment of the present invention.
Figure 4 is a diagram illustrating a process of estimating the location of a tracking target by a robot in a space where an elderly person to be tracked resides according to an embodiment of the present invention.
Figure 5 is a configuration diagram of a device for estimating the location of a tracking target according to an embodiment of the present invention.
Figure 6 is a flowchart of a method for estimating the location of a tracking object according to another embodiment of the present invention.
Figure 7 is a diagram illustrating a process of updating a candidate location of a tracking target by measuring a signal strength index by a robot in a space where an elderly person to be tracked resides in accordance with another embodiment of the present invention.
Figure 8 is a configuration diagram of a device for estimating the location of a tracking target according to another embodiment of the present invention.

Below, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts unrelated to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this is not only the case when it is "directly connected", but also when another component is interposed in between, or "with another element in between." Also includes cases of being “electrically connected.” In addition, when a part is said to "include" a certain component, this means that it does not exclude other components, but may further include other components, unless specifically stated to the contrary, and one or more other features. It should be understood that it does not exclude in advance the presence or addition of numbers, steps, operations, components, parts, or combinations thereof.

Throughout this specification, when a member is said to be located “on” another member, this includes not only cases where a member is in contact with another member, but also cases where another member exists between the two members.

In this specification, 'part' includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more pieces of hardware, and two or more units may be realized using one piece of hardware.

In this specification, some of the operations or functions described as being performed by a terminal or device may instead be performed on a server connected to the terminal or device. Likewise, some of the operations or functions described as being performed by the server may also be performed on a terminal or device connected to the server.

Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

The method of estimating the location of the tracking target according to the present invention can be performed by a robot that provides services while moving in an indoor space, or a server or terminal connected to the robot and their linkage, and the tracking target is active in an indoor space, etc. Alternatively, the person may be an elderly person or a disabled person who will receive residential care services, but is not necessarily limited to this.

In addition, in the present invention, the care service may include providing services such as providing various content including various daily life reminders, medication reminders, and cognitive training, and monitoring and reporting the occurrence of abnormal activities and emergency situations to the care recipient who is the target of tracking.

1 is a configuration diagram of a care service provision system according to an embodiment of the present invention.

Referring to FIG. 1, the care service providing system 1 may include a robot 100.

As shown in FIG. 1, the robot 100 and the server 200 may be connected to a network simultaneously or at time intervals. In addition, a network refers to a connection structure that allows information exchange between nodes such as terminals and servers, including a local area network (LAN), a wide area network (WAN), and the Internet (WWW). : World Wide Web), wired and wireless data communication networks, telephone networks, and wired and wireless television communication networks. Examples of wireless data communication networks include 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Bluetooth communication, infrared communication, and ultrasound. This includes, but is not limited to, communication, Visible Light Communication (VLC), LiFi, etc.

The robot 100 operates based on a built-in power source or an external power source and may be exemplified as a robot for communicating or interacting with the user 10. More specifically, the robot 100 may include a body part 110 and a moving part 120, and may also include other additional components necessary for driving the robot.

The body portion 110 has major components of the robot 100, such as a CPU (not shown), a battery (not shown), and a communication unit 117, mounted inside or other components connected to the outside. As a central component, a collection unit 111 for capturing the user or sensing the user's voice or surrounding sounds to generate related information, a display unit 115 for outputting an image or text, and a speaker unit for outputting a voice (Mido Poetry) may be further included.

The collection unit 111 may photograph the user 10 or the environment around the user through a visual sensor such as a camera. Here, the components included in the visual sensor include a camera that generates shooting information of general RGB pixels, an infrared camera for shooting a subject in a dark environment, and a shooting device such as a depth camera that can generate depth information about the distance to the shooting subject. These can be exemplified.

Additionally, the collection unit 111 may receive sounds from the user 10 or the environment around the user through a sound sensor such as a microphone. Here, the user's voice or surrounding sounds are important information that is treated as user input in interaction with the user, so it is desirable to place the microphone in a location where there are no elements that interfere with receiving the user's voice or surrounding sounds. . For example, if placed right behind the hole of the speaker unit formed in the front of the body unit 110, the user's voice or surrounding sounds can be directly received.

Meanwhile, the microphone may not be an independent structure disposed separately from the speaker unit, which will be described later, but may be formed integrally with the speaker unit. In this case, there will be no need to consider the location of the microphone described above, and space utilization inside the body portion 110 will also be improved.

Additionally, the collection unit 111 can receive device information generated from another smart device through a communication unit to be described later, and a detailed description of this will be provided later.

In addition, the collection unit 111 includes one or more devices for sensing at least one of information within the robot 100 (in particular, the operating state of the care service provision device), surrounding environment information surrounding the care service provision device, location information, and user information. May include sensors.

For example, the collection unit 111 may include a proximity sensor, a laser scanner (LIDAR sensor), an RGBD sensor, a geomagnetic sensor, an ultrasonic sensor, an inertial sensor, a UWB sensor, etc. in addition to the above sensors.

The display unit 115 is a type of liquid crystal display device, and can output one or more of text, images, and videos of predetermined information. Here, the predetermined information may include status information of the robot 100, such as communication status strength information, remaining battery information, and wireless Internet ON/OFF information. The display unit 115 can display media content displayed by the robot 100 in response to situation information, which will be described later, and can also display information converted into text by the voice output by the robot 100 through the speaker unit, which will be described later. You can. For example, if the robot 100 sets a wake-up alarm in the process of providing an appropriate service to the user through situation recognition, the corresponding text is output on the display unit 115 as “wake up at 7:00 a.m.” It can be.

The text output to the display unit 115 may be one of the pieces of information described above repeatedly, or a plurality of pieces of information may be displayed alternately, or specific information may be displayed as a default. For example, status information of the robot 100, such as communication status strength information, battery remaining amount information, wireless Internet ON/OFF information, etc., is continuously output by default as small text at the top or bottom of the display unit 115, Other information may be output alternately.

Meanwhile, the display unit 115 is said to be capable of outputting one or more of images and videos. In this case, the display unit 115 has a higher resolution and a larger size liquid crystal than when only text is output to improve visibility. It would be desirable to implement it as a display device, and the display unit 115 of FIG. 1 should be viewed as an example.

The speaker unit can output various sounds including voice. Here, the voice corresponds to auditory information output by the robot 100 to interact with the user, and the type of voice can be selected through a media-only application installed on the user's terminal (not shown) or through direct control of the robot 100. can be set in various ways.

For example, the type of voice output through the speaker unit can be selected from among various voices such as male voice, female voice, adult voice, and child voice, and even language types such as Korean, English, Japanese, and French can be selected. there is.

Meanwhile, although the speaker unit outputs sound, it does not mean that it does not perform the role that a typical speaker performs, for example, universal sound output, and can also perform this in the same way. For example, if a user wants to listen to music through the robot 100, music can be output through the speaker unit, and if a video is output through the display unit 115, the video is synchronized with the video. Sound may be output through the speaker unit.

The moving unit 120 can move the robot 100. More specifically, the moving unit 120 provides a means for the body part 110 of the robot to move within a specific space in relation to driving according to a movement command from the control device. More specifically, the moving part includes a motor and a plurality of wheels, and these can be combined to perform the functions of driving, changing direction, and rotating the robot 100.

The functions performed by the body unit 110 and the moving unit 120 are performed by control signals output from a control unit (not shown) within the body unit, but for convenience of explanation, they are performed by the robot 100 below. Let's continue the explanation.

Figure 2 is a flowchart of a method for estimating the location of a tracking target according to an embodiment of the present invention. Figure 3 is a diagram specifically showing the step of identifying a tracking target in one embodiment of the present invention. In addition, Figure 4 is a diagram illustrating a process of tracking the location of a tracking target by a robot in a space where the tracking target, an elderly person, resides, according to an embodiment of the present invention.

Referring to FIG. 2, the method for estimating the location of the tracking target according to this embodiment includes a step of generating map information (S110), a step of identifying the tracking target (S120), and the location of the tracking target on the map information. It includes a step (S150) of specifying. In addition, the method of this embodiment further includes tracking a change in the location of the tracking target (S130), specifying the location of the tracking target as one (S140), and generating or updating location pattern information (S160). can do.

In the step of generating map information (S110), map information about the area providing care services may be generated. Specifically, map information may be generated based on information input by various sensors such as depth cameras or lidar mounted on robots. Alternatively, map information for this embodiment may be generated by receiving map data from an external device, etc.

The step of identifying the tracking target (S120) is to accurately detect the identity of the tracking target. According to this embodiment, face information, body information, and clothing information are generated from the input image information, and at least one of these information is identified. It is performed to identify the tracking target based on

Here, the image information may be an image captured by a camera, and the camera may be mounted on the robot described above.

When movement is detected in the image captured by the robot's camera, for example, in Figure 4, the tracking object comes out of the room and exists in the same space as the robot (living room in Figure 4) or begins to move, the tracking object is identified. Step S120 may be initiated.

Hereinafter, the step (S120) of identifying the tracking target based on face information, body information, and clothing information will be described in detail with reference to FIG. 3. Here, as shown in FIG. 4, face information, body information, and clothing information may be information about at least some areas of the tracking target.

First, a step of identifying the tracking target based on face information, which includes the step of identifying the tracking target (S120), the step of extracting a bounding box (S121), and the step of identifying the face of the tracking target (S121). S122) may be included.

In the step of extracting the bounding box (S121), the bounding box for the human face may be extracted from image information input by the face detection model. The face detection model can be a convolutional neural network (CNN)-based detection model, and for example, object detection algorithms such as RCNN, Fast RCNN, YOLO, Single Shot Detector (SSD), Retina-Net, and Pyramid Net can be used. It can be.

In the step of identifying the face of the tracking target (S122), feature points may be extracted from the image of the bounding box extracted by the identity recognition model and stored in the database, and feature points extracted from the image of the bounding box may be used as feature points previously stored in the database. By comparing with , the face of the tracking target can be identified. The identity recognition model may be a convolutional neural network-based model, and for example, algorithms such as VGG-Face, FaceNet, OpenFace, DeepFace, and ArcFace may be used.

Next, a step of identifying the tracking target based on body information. The step of identifying the tracking target (S120) includes extracting skeletal information (S123) and identifying the body of the tracking target (S124). It can be included.

In the step of extracting skeletal information (S123), skeletal information including the positions of joints may be extracted from image information using a skeletal information extraction (Pose Estimation) model.

In the step of identifying the body of the tracking target (S124), at least one of the height, bone length, and body ratio of the tracking target may be calculated based on the skeletal information and stored in the database, and the calculated value may be a value already stored in the database. By comparing with , the body of the tracking target can be identified.

In the step of identifying the tracking target based on body information, a convolutional neural network-based feature point detection model can be applied as a skeletal information extraction model, for example, MoveNet, PoseNet, OpenPose, MediaPipe, AlphaPose, Nuitrack, Kinecct SDK, etc. The algorithm can be used. Skeleton information and values calculated based on it do not change in a short period of time and can be effectively used to identify tracking targets.

Furthermore, as a step of identifying the tracking target based on clothing information, the step of identifying the tracking target (S120) includes extracting a clothing image (S125) and identifying the clothing of the tracking target (S126). can do.

Specifically, in the step of extracting a clothing image (S125), a clothing image may be extracted from the input image information. In the step of identifying the clothing to be tracked (S126), the clothing to be tracked can be identified by extracting feature points from the clothing image, storing them in a database, and comparing them. At this time, types of clothing can be classified into tops and bottoms using models such as machine learning, deep neural networks, and convolutional neural networks, and feature points of the clothing can be extracted.

Meanwhile, as shown in FIG. 3, according to this embodiment, the step of identifying the tracking target (S120) includes the step of identifying the tracking target by matching or synthesizing face information, body information, and clothing information (S127). More may be included.

Specifically, at least two of the identified face information, body information, and clothing information may be matched with each other and stored in a database. Afterwards, if there is at least one of the newly created and identified face information, body information, and clothing information, the tracking target can be identified by comparing it with the information stored in the database. Therefore, even if any type of information is lacking, reliability in identifying the tracking target can be secured.

In addition, the tracking target can be identified by sequentially identifying the facial information, body information, and clothing information according to their mutual priorities, or by combining, for example, summing, based on their respective weights.

Meanwhile, in the method of estimating the location of the tracking target according to this embodiment, the step of tracking the change in the location of the tracking target (S130) includes detecting and tracking the movement of the identified tracking target in the area providing care services. It can be. Specifically, as in the previous embodiment, for an identified tracking target, video or image information for a certain period of time is input by the robot's camera, etc., and the location change of the tracking target over time is tracked in the input image information, You can track the direction of movement or movement status. As shown in FIG. 4, when the tracking target moves to another space, the change in location can be tracked using image information received from a camera, etc.

On the other hand, the step (S140) of specifying one location of the tracking target may be performed when multiple locations of the tracking target are estimated. In the process of identifying the tracking target or tracking a change in the location of the tracking target described above, it may be calculated that the tracking target exists at a plurality of locations.

In this case, in this embodiment, one tracking target can be identified based on additionally input voice information. Voice information can be collected from a microphone mounted on a mobile robot, etc., as described above, and the location of the tracking target can be selected and specified among a plurality of locations where voice information is collected and confirmed. As in the embodiment of FIG. 4, voice information may be obtained by inducing the tracking target to utter a voice, such as by asking a question.

As another alternative, search image information may be additionally input and a single tracking target may be specified. The search image information may be of the same type as the image information input to identify the tracking target, but may also be a different type of image obtained from another camera, etc. The robot moves to a plurality of locations where the tracking target is estimated to be located, and the location of the tracking target can be selected and specified through search image information directly captured at the corresponding locations.

Voice information or navigation image information may be obtained from a plurality of locations in space, and as described above, may be information obtained while the robot moves in an area providing care services.

As described above, when the tracking target is identified, or when the tracking target is identified and the location of the tracking target is specified as one, the location of the tracking target may be specified on the map information (S150). Specifically, the location of the tracking target on the map information may be specified based on information about the location and direction of receiving image information on the map information, and further may be specified based on changes in the location of the tracked identification target. You can.

Meanwhile, according to this embodiment, information about the location of a specific and accumulated tracking target can be created and updated as location pattern information (S160). Specifically, the location pattern information may be a record of the frequency of the location of the tracking target specified on the map information, the frequency of the location according to time zone, etc. Furthermore, records of the time and location at which the robot's functions were operated by the tracking target calling or manipulating the robot can be utilized.

This updated location pattern information can be used in the step of identifying the tracking target (S120). For example, if the face information of the tracking target is not accurately recognized, the tracking target can be identified as a likely person based on location pattern information.

According to an embodiment of the present invention, by comprehensively utilizing face information, body information, and clothing information among image information, the tracking target can be accurately recognized. In addition, the location of the tracking target identified on the generated map information can be tracked and specified using various types of information, thereby ensuring accuracy and laying the foundation for providing advanced services targeting people.

Figure 5 is a configuration diagram of a device for estimating the location of a tracking target according to an embodiment of the present invention. The device for estimating the location of the tracking target according to this embodiment may be a robot or a component of a robot that provides services while moving in an indoor space, etc., or a component implemented in a server or terminal connected to the robot and a system combining them. .

Referring to FIG. 5, the device 100 for estimating the location of a tracking target to provide care services according to this embodiment includes a map information generator 110 that generates map information about the area providing care services; , a tracking target identification unit 120 that generates face information, body information, and clothing information from the input image information, and identifies the tracking target based on at least one of the face information, body information, and clothing information, and an image on the map information. It may include a location specification unit 130 that specifies the location of the tracking target on the map information based on the location and direction of receiving the information. Additionally, the device 100 may further include a location pattern information generator 140 that generates or updates location pattern information that accumulates information about the location of the tracking target specified on the map information.

In addition, the steps for estimating the location of the tracking target described with reference to FIGS. 2 and 3 may be components of the device 100 for estimating the location of the tracking target according to an embodiment of the present invention, individually or in combination. .

Next, Figure 6 is a flowchart of a method for estimating the location of a tracking object according to another embodiment of the present invention. Figure 7 is a diagram illustrating a process of updating a candidate location of a tracking target by measuring a signal strength index by a robot in a space where an elderly person to be tracked resides in accordance with another embodiment of the present invention.

Hereinafter, a method for estimating the location of a tracking target according to another embodiment of the present invention will be described with reference to FIGS. 6 and 6 . Another embodiment of the present invention is configured to estimate the location of the tracking target based on the signal and its strength received from a device held or worn by the tracking target, such as an elderly person, for example, various wearable devices.

Referring to FIG. 6, the method for estimating the location of the tracking target according to this embodiment includes the steps of generating a location candidate group of the tracking target (S210), measuring the signal strength index of the device (S220), and tracking It includes a step of updating the target location candidates (S230) and a step of specifying the location of the tracking target on the map information (S250). In addition, a step of confirming the location of the tracking target with respect to the updated location candidate group (S240) and a step of generating or updating location pattern information (S260) may be further included.

In this embodiment, map information may be generated based on the robot's depth camera or lidar as in the previous embodiment, or may be generated by receiving map data from an external device, etc.

In the step of generating a location candidate group for the tracking target (S210), the location candidate group may be generated based on map information. As in the example of FIG. 7 , the location candidate group may include at least one location candidate in which there is a high possibility that a tracking target exists on map information.

In the step of measuring the signal strength indicator of the device (S220), the signal strength indicator (RSSI, Receiver Signal Strength Indicator) of the device worn or possessed by the tracking target can be measured while moving a plurality of points on the map information. Specifically, the signal strength index may be measured for a plurality of points on the path along which the robot moves so that the distance to at least one of the location candidates changes. The signal strength index may be measured multiple times in the area where the robot provides care services or on the path it moves on the map information generated based thereon.

In the step of updating the location candidates of the tracking target (S230), the location candidates may be updated based on the change in location between a plurality of points and the change in the measured signal strength index.

For example, when there is a change in the measurement position along the robot's movement path as shown in (a) of Figure 7, if the signal intensity index changes to increase, candidate 3, which is the location where the signal intensity index should decrease, is a position candidate group. may be excluded from. Next, if a change such that the signal intensity index increases is detected on the robot's movement path as shown in (b) of FIG. 7, candidate 1, which is the location where the signal intensity index should decrease, can be excluded from the position candidate group. The judgment of predicting an increase or decrease in the signal strength index for each location candidate may be based on the candidate location and the signal strength index measurement location on the map information.

In other words, the step of updating the location candidates of the tracking target (S230) is performed by comparing the direction of the movement path on the map information and the increase or decrease in the measured signal strength index, thereby excluding unlikely location candidates from the location candidates. It can be.

In the step of specifying the location of the tracking target on the map information (S250), the location of the tracking target on the map information may be specified based on the updated location candidate group. Based on the location of the specified tracking target, care services, etc. can be provided to the tracking target by the movable robot as an example in this embodiment.

Meanwhile, in at least one of the step of generating a location candidate group of the tracking target (S210) and the step of updating the location candidate group of the tracking target (S230), the location candidate group is generated based on at least one of the input voice information and image information. Or it can be updated. Voice information or image information may be collected by a microphone or camera mounted on the robot described in this embodiment.

Furthermore, in at least one of the step of generating a location candidate group of the tracking target (S210) and the step of updating the location candidate group of the tracking target (S230), the sensor or tracking target whose location is specified on the map information is further carried or worn and A location candidate group may be created or updated based on additional detection information received from an additional device.

Specifically, if the tracking target possesses or wears an additional device, including a mobile phone or other IoT device, the accuracy of location specification is improved by receiving additional detection information from the device and updating the location candidate group based on this. It can be.

In addition, additional detection information received from a sensor whose location is specified on the map information, for example, an entry/exit detection sensor installed at a specific location in space, can also be used to create or update a location candidate group.

On the other hand, in the step of confirming the location of the tracking target with respect to the updated location candidate group (S240), additionally input voice information and image information may be used. That is, if there are multiple location candidates in the updated location candidate group, the location of the tracking target can be confirmed again based on at least one of voice information, image information, and a signal strength index searched by the robot in close proximity to the location candidate. .

Also, in other embodiments of the present invention, location pattern information may be created or updated as in one embodiment of the present invention. In the step of generating or updating location pattern information (S260), information about the location of the tracking target specified on the map information may be accumulated to generate and update location pattern information. Furthermore, records of the time and location at which the robot's functions were operated by the tracking target calling or manipulating the robot can be utilized.

The information created or updated in this way can be used as basic information in at least one of the step of generating a location candidate group of the tracking target (S210) and the step of updating the location candidate group of the tracking target (S230).

According to another embodiment of the present invention, recognition of the tracking target can be accurately performed by utilizing map information and signal strength indicators of devices such as wearable devices. In addition, the location of the tracking target identified on the generated map information can be tracked and specified using additional information, thereby ensuring accuracy, thereby laying the foundation for providing advanced care services for people.

Figure 8 is a configuration diagram of a device for estimating the location of a tracking target according to another embodiment of the present invention. The device for estimating the location of the tracking target according to this embodiment may be a robot or a component of a robot that provides services while moving in an indoor space, etc., or a component implemented in a server or terminal connected to the robot and a system combining them. .

Referring to FIG. 8, the device 200 for estimating the location of a tracking target to provide care services according to the present embodiment generates a location candidate group of the tracking target on map information for the area providing care services. A generator 210, a signal strength indicator measurement unit 220 that measures the signal strength indicator of a device worn or possessed by a tracking target at a plurality of points on the map information, and a signal strength indicator and a change in position between the plurality of points. It may include a location candidate update unit 230 that updates the location candidate group of the tracking target based on changes in , and a location specification unit 240 that specifies the location of the measurement target on map information based on the updated location candidate group. .

In addition, the steps for tracking the location of the tracking target described through FIGS. 6 to 6 may be functions performed by the device 200 for tracking the location of the tracking target according to another embodiment of the present invention, individually or in combination. You can.

Furthermore, in the above description, the steps of the method for estimating the location of the tracking object according to one embodiment and other embodiments of the present invention are further divided into additional steps or fewer steps, depending on the implementation of the present invention. can be combined. Additionally, some steps may be omitted or the order between steps may be switched as needed.

The description of the present invention described above is for illustrative purposes, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential features of the present invention. will be. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. For example, each component described as single may be implemented in a distributed manner, and similarly, components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention.

100, 200: Device for estimating the location of the tracking target
110: Map information generation unit
120: Tracking target identification unit
130, 240: Location specification unit
140: Location pattern information generation unit
210: Location candidate group generation unit
220: Signal strength indicator measurement unit
230: Location candidate update unit

Claims (19)

In a method of estimating the location of a tracked object to provide care services by a robot,
Generating map information about an area providing the care service;
Generating face information, body information, and clothing information from image information input from a visual sensor mounted on the robot, and identifying the tracking target based on at least one of the face information, the body information, and the clothing information;
Specifying the location of the tracking target on the map information based on the location and direction of receiving the image information on the map information; and
Comprising: generating or updating location pattern information by accumulating information about the location of the tracking target specified on the map information,
The step of identifying the tracking target is,
Extracting a clothing image classifying the type of clothing from the image information using a model based on any one of machine learning, deep neural network, and convolutional neural network, extracting feature points from the clothing image,
Identifying a tracking target by adding up the face information, body information, and clothing information based on respective weights,
Frequency of the location of the tracking target specified on the map information, frequency according to time zone for the location of the tracking target specified on the map information, and the function of the robot is operated by the tracking target operating the robot. identify the tracking object further based on the location pattern information including time and location;
When the location of the tracking target is estimated to be multiple, the robot induces the tracking target to utter a voice, and the voice information is confirmed among the plurality of locations based on the voice information input from the sound sensor mounted on the robot. Selecting and specifying the location of the tracking target,
A method of estimating the location of a tracked object.
According to claim 1,
The step of identifying the tracking target based on the face information includes:
Extracting a bounding box for a human face from the image information using a face detection model; and
Using an identity recognition model, extracting feature points from the image of the bounding box and storing them in a database, and comparing them with feature points stored in the database to identify the face of the tracking target.
A method of estimating the location of a tracking object.
According to claim 1,
The step of identifying the tracking target based on the body information includes:
Comprising the step of extracting skeletal information including the position of a joint from the image information using a skeletal information extraction model,
A method of estimating the location of a tracking object.
According to claim 3,
The step of identifying the tracking target based on the body information includes:
Calculating at least one of the height, bone length, and body ratio of the tracking target based on the skeletal information and storing it in a database, and comparing it with the value stored in the database to identify the body of the tracking target, further comprising:
A method of estimating the location of a tracking object.
delete According to claim 1,
The step of identifying the tracking target is,
Store one of the face information, the body information, and the clothing information in a database,
Identifying a tracking target by comparing at least one of the face information, the body information, and the clothing information with information stored in the database,
A method of estimating the location of a tracking object.
delete According to claim 1,
For the identified tracking target, further comprising tracking a change in the location of the tracking target over time in an area providing the care service from the input image information,
A method of estimating the location of a tracking object.
delete According to claim 1,
When the locations of the tracking target are estimated to be plural, receiving search image information taken by moving to the estimated plurality of locations of the tracking target and specifying the locations of the tracking target as one, further comprising:
A method of estimating the location of a tracking object.
delete In the device for estimating the location of a tracking target to provide care services,
a map information generator that generates map information about an area providing the care service;
Generate face information, body information, and clothing information from image information input from a visual sensor mounted on the robot providing the care service, and identify a tracking target based on at least one of the face information, the body information, and the clothing information. A tracking object identification unit that identifies the target;
a location specification unit that specifies the location of the tracking target on the map information based on the location and direction of receiving the image information on the map information; and
A location pattern information generator that generates or updates location pattern information by accumulating information about the location of the tracking target specified on the map information,
The tracking target identification unit,
Extracting a clothing image classifying the type of clothing from the image information using a model based on any one of machine learning, deep neural network, and convolutional neural network, extracting feature points from the clothing image,
Identifying a tracking target by adding up the face information, body information, and clothing information based on respective weights,
Frequency of the location of the tracking target specified on the map information, frequency according to time zone for the location of the tracking target specified on the map information, and the function of the robot is operated by the tracking target operating the robot. identify the tracking object further based on the location pattern information including time and location;
When the location of the tracking target is estimated to be multiple, the location specifying unit induces the robot to utter a voice to the tracking target and selects one of the plurality of locations based on voice information input from a sound sensor mounted on the robot. Selecting and specifying the location of the tracking target as the place where the voice information is confirmed,
A device that estimates the location of a tracking target.
delete delete delete delete delete delete delete

Images