JP2019079419A - Robot management system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To appropriately determine a service desired by a person staying in a facility. SOLUTION: One aspect of the present invention is a robot provided with a movable robot, a camera for taking a picture, and a person detection unit for detecting a person from a shot image, and the robot provides a service to the detected person. A management system, an action pattern table in which an action pattern defined by a person's rest time, operation time, and a combination of the rest time and the operation time is repeated and a service provided to the person are registered in association with each other. When 111g and the behavior of the person detected from the captured image match the behavior pattern registered in the behavior pattern table 111g, the behavior pattern that determines the service that the robot should provide to the person based on the corresponding behavior pattern. A determination unit 111f is provided. [Selection diagram] Fig. 3

Description

  The present invention relates to a robot management system that manages robots that service people.

  Conventionally, the time in which a person photographed with a camera stayed in a specific area is measured, and movement or stationary is performed in a first area in contact with the area where the article is placed and a second area in contact with the first area based on the stay time. There is a technique in which a mobile robot provides guidance to a person who does the job (for example, see Patent Document 1).

Unexamined-Japanese-Patent No. 2007-152442

  However, just by measuring the time when a person stayed in a specific area (dwelling) as described in Patent Document 1, the factor of the stagnation can not be determined, and the service desired by the stagnated person can not be appropriately provided. .

  The present invention has been made in consideration of the above situation, and enables a robot to appropriately provide a service desired by a person to a person staying in a certain area.

A robot management system according to an aspect of the present invention includes a movable robot, a camera for shooting, and a person detection unit for detecting a person from an image photographed by the camera, and the person detected by the person detection unit The robot is a robot management system that provides services.
The robot management system includes an action pattern table and an action pattern determination unit. In the behavior pattern table, the behavior time defined by the resting time of the person, the operation time, and the repetition of the combination of the resting time and the operation time are registered in association with the service provided to the person. The action pattern determination unit provides the person to the person from the action pattern table based on the corresponding action pattern when the action of the person detected from the video of the camera matches the action pattern registered in the action pattern table. It has a function to determine the service to be.

According to at least one aspect of the present invention, it is possible to appropriately provide the staying person with a service desired by the robot.
Problems, configurations, and effects other than those described above will be apparent from the description of the embodiments below.

FIG. 1 is a block diagram showing an example of the overall configuration of a robot management system according to an embodiment of the present invention. It is a functional block diagram showing an example of composition of a robot concerning one example of an embodiment of the present invention. It is a functional block diagram showing an example of composition of a robot control device concerning one example of an embodiment of the present invention. It is a functional block diagram showing an example of composition of a robot surveillance center concerning one example of an embodiment of the present invention. It is a functional block diagram showing an example of composition of a camera control device concerning one example of an embodiment of the present invention. It is an explanatory view showing the inside (the 1st example) of the building where the robot concerning one example of an embodiment of the present invention, and surveillance cameras are arranged. It is an explanatory view showing the inside (the 2nd example) of the building where the robot and surveillance camera concerning one example of an embodiment of the present invention are arranged. It is an explanatory view showing an example of data structure of a robot management table concerning an example of an embodiment of the present invention. It is an explanatory view showing an example of data structure of a surveillance camera management table concerning an example of an embodiment of the present invention. It is an explanatory view showing an example of data structure of an action pattern table concerning an example of an embodiment of the present invention. It is an explanatory view showing an example of data structure of an action judging result database concerning an example of an embodiment of the present invention. It is a flowchart which shows the example of a procedure of the service provision based on the action pattern of the stay person by the robot management system which concerns on the example of one Embodiment of this invention.

  Hereinafter, examples of modes for carrying out the present invention (hereinafter, referred to as “embodiments”) will be described with reference to the attached drawings. The components having substantially the same function or configuration in the present specification and the attached drawings will be denoted by the same reference numerals and redundant description will be omitted.

<1. One embodiment>
[Overall configuration of robot management system]
First, with reference to FIG. 1, an example of the overall configuration of a robot management system according to an embodiment of the present invention (hereinafter also referred to as "this example") will be described. FIG. 1 is a block diagram showing an example of the overall configuration of a robot management system 1.

  As shown in FIG. 1, the robot management system 1 includes a plurality of robots 100, a robot control device 110, and a robot monitoring center 120. In the robot management system 1 of this example, the robot 100, the robot control device 110, the monitoring camera 200, and the camera control device 210 are disposed in the building 300.

  The robot 100 and a robot control device 110 that controls the robot 100 are connected via, for example, a wireless local area network (LAN). In addition, one or more monitoring cameras 200 can capture the inside of the building 300. The camera control device 210 records an image captured by the monitoring camera 200 via a wired LAN or the like. The robot control device 110 and the camera control device 210 are connected via a wired LAN or the like, and can mutually transmit and receive data. The robot control device 110 and the camera control device 210 may be connected by wireless communication means such as a wireless LAN when the distance between them is short.

  The robot 100 is an autonomous mobile proposal robot that autonomously moves toward a person (user) in the building 300, and performs guidance in the building 300, proposal of product information, and the like to the user. The product information is, for example, information on a product handled by a store or the like installed in the building 300. Hereinafter, an action such as guidance or suggestion performed by the robot 100 to the user is referred to as a “service”. The robot 100 normally stands by at a fixed position on the floor in the building 300. The robot 100 communicates with a robot control device 110 installed in a building 300, and is configured to be wirelessly controlled by the robot control device 110.

  For example, when the robot control device 110 detects a user and issues a movement instruction to the robot 100, a route along which the robot 100 moves from the standby position to the destination is set in the robot 100 according to the instruction from the robot control device 110. The robot 100 autonomously moves to the position of the designated user, and provides services such as guidance and suggestion. When the service to the user ends, the robot 100 moves autonomously to return to the original standby position or moves to the next user.

  Note that the autonomous movement in the robot 100 of this example means that the robot itself moves and moves, and means that the robot itself performs all the judgments necessary for movement (judgement of a route, etc.) autonomously. It is not something to do. That is, the route along which the robot 100 moves is set by the instruction from the robot control device 110 as described above, and the route along which the robot 100 moves autonomously is not determined. However, all determinations necessary for movement of the robot 100 may be made.

  In addition to instructing the robot 100 to execute a service, the robot control device 110 communicates with the robot monitoring center 120 installed outside the building 300 via the network N such as the Internet.

  The robot monitoring center 120 normally communicates with the robot control devices 110 installed in a plurality of buildings 300, and monitors the management status and the like of the robots 100 disposed in each building 300.

  One or more surveillance cameras 200 are installed at the entrance of the building 300, a common area, a restaurant / shop area, and the like. That is, the monitoring camera 200 is installed at a place where people in the building 300 go and go, a passage where people pass, and the like. The surveillance camera 200 continues to capture a person appearing in each imaging area (imaging space), and transmits video data obtained by imaging to the camera control device 210 via the wired LAN. Of course, the monitoring camera 200 can change the shooting direction and the shooting magnification arbitrarily and shoot based on an instruction from the camera control device 210. In the following description, when the monitoring camera 200 does not distinguish between a common area which is a photographing area (photographing place) to be photographed and a restaurant / shop area or the like, the area may be collectively referred to as an “area”.

  The camera control device 210 records the video data acquired from the monitoring camera 200 and supplies the video data to the robot control device 110 via the wired LAN. The data recorded in the camera control device 210 is supplied to the robot monitoring center 120 via the robot control device 110 and the network N. In this example, the analysis of the video data is performed by the robot control device 110 as described later, but the camera control device 210 may be configured to analyze the video data.

  In the present embodiment, the installation location of the monitoring camera 200 is inside the building 300, but the installation location is not limited to indoors. For example, the monitoring camera 200 may be installed at a position where it can capture an outside of an outdoor facility or a building belonging to any facility.

Robot Configuration
FIG. 2 is a functional block diagram showing an example of the internal configuration of the robot 100. As shown in FIG.
As shown in FIG. 2, the robot 100 includes a CPU 104, and is configured of a computer device in which each process is executed under the control of the CPU 104, and peripheral devices connected to the computer device. The robot 100 includes a main storage device 101, an input / output device 102, and a communication interface 103 connected via a bus line. The robot 100 can move in a predetermined area shown in FIGS. 7 and 8 described later.

  The main storage device 101 includes a drive control unit 101a, an interaction control unit 101b, and an input / output unit 101c. The drive control unit 101a, the dialogue control unit 101b, and the input / output unit 101c are functions of a computer program that operates based on an instruction from the CPU 104. The computer program is recorded in a non-volatile mass storage device (not shown), and the necessary computer program is read out from the mass storage device to the main storage device 101.

  The drive control unit 101 a performs drive control to move the robot 100 autonomously. For example, when the robot 100 moves, the drive control unit 101a determines the situation around the robot 100 from the image captured by the camera 102a of the robot 100 or the position of the robot itself detected by the ranging sensor 102d. The data on the distance to the object is used to move the robot 100 avoiding obstacles such as people and walls. Further, the drive control unit 101a detects the tilt of the body of the robot 100 using the gyro sensor 102c, and performs control to move the robot 100 without falling down.

The interaction control unit 101 b controls interaction with the subject by voice using the microphone 102 b and the speaker 102 e included in the input / output device 102. As an example, the dialogue control unit 101b outputs, from the speaker 102e, speech information such as an utterance necessary for proposal (service execution), and obtains an answer by the speech from the subject through the microphone 102b. This shows an example of a method for obtaining a response from the subject. As an example of alternative means, a display panel or touch panel is attached to the robot 100, and guidance information or product information is displayed through the display panel or touch panel. May be The subject can obtain an answer or desired information by operating the touch panel.
The input / output unit 101 c performs data input / output operations with the input / output device 102, and also performs data input / output operations with the robot control apparatus 110 via the communication interface 103.

  The input / output device 102 includes a camera 102a, a microphone 102b, a gyro sensor 102c, a range sensor 102d, a speaker 102e, and a drive mechanism 102f.

  The camera 102 a shoots a shooting area (shooting space) around the robot 100, and sends video data obtained by shooting to the main storage device 101 or a large capacity recording device (not shown). The camera 102 a is configured using an imaging device such as a CCD or a CMOS. Generally, the camera 102a of the robot 100 is disposed at a position lower than the height of a person to be photographed (for example, the average height of a general person (including children)), but the present invention is not limited to this example.

The microphone 102 b acquires audio information from the subject. Various data such as image information acquired by the camera 102 a and audio information acquired by the microphone 102 b are supplied to the robot control device 110 via the communication interface 103.
The gyro sensor 102 c detects the inclination or the like of the robot 100 from the angular acceleration applied to the robot 100, and supplies detection data to the robot control device 110 via the communication interface 103.

The ranging sensor 102 d is a sensor for specifying the position of the robot 100, and also detects the surrounding environment of the robot 100. The range sensor 102d emits laser light, infrared light, and the like, and detects reflection from a surrounding object. The range measuring sensor 102 d measures the position of the robot 100 and the shape of the surrounding space including an obstacle and the like, and the measured data is sent to the robot control device 110 via the communication interface 103.
The speaker 102 e outputs, to the subject, speech for interaction, such as a fixed phrase, which is generated by the dialogue control unit 101 b and is necessary for proposing to the staying person who is the service target.

  The drive mechanism 102 f moves the robot 100 based on an instruction from the drive control unit 101 a. The drive mechanism 102 f includes a motor that drives at least a wheel. Alternatively, when the robot 100 is a humanoid robot, an actuator for driving a member corresponding to a leg is provided to perform movement by walking.

  As shown in FIG. 1, the communication interface 103 is connected to the communication interface 112 (see FIG. 3) of the robot control apparatus 110 by a wireless LAN or the like. The communication interface 103 outputs the data collected by the input / output device 102 to the robot controller 110 and receives an instruction from the robot controller 110.

  In addition, when the robot 100 moves, the drive control unit 101a determines an image captured by the camera 102a or a surrounding condition detected by the range measurement sensor 102d, and stops the movement or avoids an obstacle or the like. Control the operation. In the case of this example, the movable range in which the robot 100 can move is limited to a predetermined range (such as a photographing area described later and its periphery). That is, the current position of the robot 100 determined by the drive control unit 101a remains at the position within the movable range.

[Configuration of robot controller]
FIG. 3 is a functional block diagram showing an example of the internal configuration of the robot control device 110. As shown in FIG. The robot control device 110 is configured by a computer device in which each process is executed under the control of the CPU 113.
The CPU 113 is connected to the main storage device 111 and the communication interface 112 connected via a bus line.

  The main storage device 111 includes an input / output data processing unit 111a, a movement instructing unit 111b, and a robot management table 111c that operate based on an instruction of the CPU 113. The main storage device 111 further includes a person detection unit 111d, an action analysis unit 111e, an action pattern determination unit 111f, an action pattern table 111g, and a scenario storage unit 111h. The behavior analysis unit 111 e and the behavior pattern determination unit 111 f are each an example of an element configuring the service determination function.

  The input / output data processing unit 111 a performs data transmission / reception processing with the robot 100, the camera control device 210, and the robot monitoring center 120 via the communication interface 112.

  The movement instruction unit 111b instructs the robot 100 to move to the position of the person (service target person) who is determined to be the staying person by the behavior analysis unit 111e. This instruction includes the route along which the robot 100 moves. As an example of the route, a route for guiding the robot 100 to the place where the staying person is present, a route for returning the robot 100 to the original waiting position when the service to the staying person is finished, or the robot up to the next staying person There is a route that leads 100.

  Management data of the robot 100 to be managed by the robot control apparatus 110 is stored in the robot management table 111 c. Details of the robot management table 111c will be described later with reference to FIG.

  The person detection unit 111 d detects a person based on the video captured by the monitoring camera 200. The person in the image can be detected, for example, by detecting the face of the person, that is, the face image of the person or the gait of the person. Here, "gait" refers to the state of physical exercise when a person walks, and refers to the way of walking that appears to the eye. When the person detection unit 111d detects a person from the video, it temporarily assigns a person ID (identification information) for uniquely identifying the person, and calculates the position (two-dimensional coordinates or three-dimensional coordinates) of the person from the video. Do. Thus, the person detection unit 111d can identify the position of the detected person by the coordinates.

  The behavior analysis unit 111e includes the items of the rest time of the person detected from the image by the person detection unit 111d, the operation time, and the combination of the rest time and the operation time (number of behavior cycles) (action pattern table 111g of FIG. 10). Analyze the behavior of the target person by calculating the reference). Based on the calculation result of each item and the action pattern stored in the action pattern table 111g, the action pattern determination unit 111f detects the staying person. In this example, one combination of the resting time and the operation time is taken as one action cycle, and the number of repetitions of one action cycle is taken as the action cycle number. It should be noted that although the order of the stationary time and the operation time may be earlier, it can be said that the flow of calculating the operating time after judging the presence / absence of stationary (calculating the stationary time) is natural.

  Here, in the present specification, the term "static" means that a person is standing still or staying within a very narrow range. In addition, the staying person is a person who is stationary or who is in a stationary state and who is periodically performing an operation, and who meets a preset condition (action pattern). For example, the behavior analysis unit 111e compares a plurality of images captured continuously at the same place, and detects a person who has remained at a predetermined position or a predetermined range for a predetermined time.

  The aim of the behavior analysis unit 111 e to detect the retention of the person is to improve the accuracy of the service provided to the person in the imaging area. A person who is stationary or a person who repeats movement with motion is likely to have some interest or trouble. Therefore, the robot 100 in the present embodiment is controlled not to provide a service to a person passing by. The person who is the object of the service provision is only the person who has a long rest time in the shooting area or who repeats the rest and motion.

  The behavior pattern determination unit 111f compares the behavior analysis result by the behavior analysis unit 111e with the behavior pattern stored in the behavior pattern table 111g, detects the staying person and determines the behavior pattern of the staying person, and based on the determination result, the robot Decide what services to provide to the residents. The behavior pattern determination unit 111f can also be referred to as a service determination unit in that the behavior pattern determination unit 111f has a function of determining the service to be provided from the robot to the staying person. In addition, the behavior pattern determination unit 111 f supplies the person ID of the person who is determined to be the staying person and various data when it is determined to be the staying person to the robot monitoring center 120 via the communication interface 112. Then, the person ID of the staying person and various data determined to be the staying person are registered in the action pattern table 111g.

  The behavior pattern table 111g stores information on behavior patterns of staying people registered in advance for each building. The action pattern table 111g is referred to when the action pattern determination unit 111f determines the action pattern of the staying person.

  The scenario storage unit 111 h stores scenario data in which the content of the service provided to the staying person by the robot 100 is described, which corresponds to the factor / service determined by the action pattern determining unit 111 f.

  Furthermore, the robot control device 110 has a communication interface 112 capable of mutually communicating with the robot 100 and the network N. The communication interface 112 communicates with the robot monitoring center 120 in addition to performing wireless communication with the robot 100 via the network N.

[Configuration of robot monitoring center]
FIG. 4 is a functional block diagram showing an example of the internal configuration of the robot monitoring center 120. As shown in FIG. The robot monitoring center 120 is configured of a computer device that executes each process under the control of the CPU 124. The CPU 124 controls the main storage device 121, the operation unit 122, and the communication interface 123 connected via a bus line.

  The main storage device 121 includes a video data storage unit 121a, an action data storage unit 121b, and a robot management table 121c.

The video data storage unit 121 a stores video captured by the monitoring camera 200.
The action data storage unit 121b stores an action determination result database 121bD (see FIG. 11) in which various data transmitted from the robot control device 110 and determined when the action pattern determination unit 111f determines that there is a staying person are stored. Ru. Management data of the robots 100 managed by each of the plurality of robot control devices 110 are collectively stored in the robot management table 121 c.
Each data stored in the video data storage unit 121a, the action data storage unit 121b, and the robot management table 121c may be stored in a large capacity storage device (not shown).

For example, a keyboard, a mouse, a touch panel, or the like is used for the operation unit 122, and a system administrator can perform a predetermined operation and input an instruction.
The communication interface 123 communicates with the robot control device 110 via the network N.

[Configuration of surveillance camera]
The monitoring camera 200 is a camera that can be operated from a remote location, and is a so-called network camera. The monitoring camera 200 includes an imaging device (not shown) such as a CCD or CMOS, processes an image captured by the imaging device, and transmits the processed image to the camera control device 210 via a communication interface (not shown). Generally, the monitoring camera 200 includes a video processing unit that converts raw data (RAW image) output from the imaging device into a data format that can be processed by the camera control device 210.

[Configuration of Camera Controller]
FIG. 5 is a functional block diagram showing an example of the internal configuration of the camera control device 210. As shown in FIG.
The camera control device 210 is configured of a computer that executes each process under the control of the CPU 213. The CPU 213 controls the main storage device 211 and the communication interface 212 connected via a bus line.

The main storage device 211 includes an input / output data processing unit 211a, a video data storage unit 211b, and a monitoring camera management table 211c that operate based on an instruction from the CPU 213.
The input / output data processing unit 211 a inputs / outputs data communicated between the monitoring camera 200 and the robot control apparatus 110 via the communication interface 212.
The video captured by the monitoring camera 200 is recorded in the video data storage unit 211 b.
Management data of each monitoring camera 200 is stored in the monitoring camera management table 211 c. Details of the monitoring camera management table 211c will be described later with reference to FIG.

  The communication interface 212 communicates with the robot control device 110 via a wired LAN or the like, and also communicates with the monitoring camera 200 via the wired LAN or the like.

In the above description, the robot management system 1 according to the present embodiment is configured such that the robot 100, the robot control device 110, the robot monitoring center 120, and the camera control device 210 share the functions shown in the block diagrams of FIGS. Is realized. However, the functions of the robot 100, the robot control device 110, the robot monitoring center 120, and the camera control device 210 are not necessarily fixed. In particular, it is also possible to have the robot monitoring center 120 or the robot 100 have a part of the functions of the robot control device 110. However, since the robot 100 is configured as a single unit, it is desirable to consider the system operator's request that the robot 100 does not want to include many functions.

[Arrangement Example of Monitoring Camera] FIG. 6 is a sketch showing the inside (first example) of a building in which the robot 100 and the monitoring camera 200 are arranged. FIG. 7 is a sketch showing the inside of a building (second example). 6 and 7 show an airport 301 as an example of a building 300.

  On the first floor of the airport 301 shown in FIG. 6, a surveillance camera 200-1 for photographing an international arrival lobby as a first photographing area is installed. The robot 100-1 is in charge of providing services to persons in the international arrivals lobby.

  On the second floor of the airport 301 shown in FIG. 7, there are a monitoring camera 200-2 for photographing a restaurant / shop area as a second photographing area and a monitoring camera 200-3 for photographing an international departure lobby as a third photographing area. is set up. The robot 100-2 is in charge of providing services to persons in the restaurant / shop area, and the robot 100-3 is in charge of providing services to persons in the international departure lobby. The monitoring cameras 200-1, 200-2, and 200-3 are referred to as a monitoring camera 200 when it is not necessary to distinguish them in particular. The monitoring camera 200 is installed at a position where it can look over the imaging area.

[Data structure of various tables]
(Robot Management Table) FIG. 8 shows an example of the data structure of the robot management tables 111c and 121c.
The robot management tables 111 c and 121 c include items of a customer ID, a facility ID, a global IP, a robot ID, a local IP, and an area.

The “customer ID” is an ID (identification information) for uniquely identifying a customer who uses the robot or the business model according to the present embodiment based on a contract exchanged with the provider of the robot 100.
The “facility ID” is an ID for uniquely identifying the facility where the robot is disposed. The facility may be indoor or outdoor, for example, all or part of a section of a building to give an example of indoor.
“Global IP” is a global IP (Internet Protocol) address of a communication device provided in a target facility.
The “robot ID” is an ID for uniquely identifying the robot 100.
The “local IP” is a local IP address of the robot 100 (communication interface 103).
The “area” represents an area where the robot 100 is present in the target facility, ie, an area in which the robot 100 can move. In the example of FIG. 6 and FIG. 7, the international arrival lobby, the restaurant / shop area, the international departure lobby, etc. correspond. In the present example, the “area” of the robot management tables 111 c and 121 c is matched with the location indicated by the “area” of the monitoring camera management table 211 c described later.

  According to the example of FIG. 8, for example, a robot provider makes a robot provision contract with a customer of a customer ID “001”, and three robots 100 are used in a facility having a facility ID “001”. In the facility having the facility ID "001", the robot 100 having the robot ID "001" is disposed in the area "B3", the robot 100 having the robot ID "002" is disposed in the area "B2", and the robot having the robot ID "003" 100 is arranged in the area "C2".

  The robot management table 111 c of the robot controller 110 manages information of only the robot 100 controlled by the robot controller 110. On the other hand, the robot management table 121c of the robot monitoring center 120 manages information of the robots 100 controlled by all the robot control devices 110 monitored by the robot monitoring center 120. When the robot 100 has the drive mechanism 102 f, the area representing the position of the robot 100 is a range sensor 102 d of the robot 100, a movement history (travel history) of the drive mechanism 102 f of the robot 100, a beacon (not shown), etc. Obtained using

(Monitoring Camera Management Table) FIG. 9 shows an example of the data structure of the monitoring camera management table 211c.
The monitoring camera management table 211 c includes items of a customer ID, a facility ID, a global IP, a monitoring camera ID, a local IP, and an area. Hereinafter, each item of the monitoring camera management table 211c will be described, but description of items overlapping the items of the robot management table 121c will be simplified.

The “customer ID” is an ID uniquely identifying a customer.
The “facility ID” is an ID that uniquely identifies a facility.
"Global IP" is the global IP address of the communication device in the target facility.
The “monitoring camera ID” is an ID for uniquely identifying the monitoring camera 200.
The “local IP” is a local IP address of the monitoring camera 200 (communication interface not shown).
The “area” represents the position of the monitoring camera 200 disposed in the target facility, that is, the area that the monitoring camera 200 is to capture.

  According to the example of FIG. 9, for example, with respect to a robot providing facility in which a robot provider has made a contract with a customer of a customer ID "001", three monitoring cameras 200 are arranged in the facility of facility ID "001". In the facility having the facility ID "001", the monitoring camera 200 of the monitoring camera ID "001" is disposed in the area "B3", the monitoring camera 200 of the monitoring camera ID "002" is disposed in the area "B2", and the monitoring camera ID The surveillance camera 200 of "003" is disposed in the area "C2".

(Action pattern table)
FIG. 10 shows an example of the data structure of the action pattern table 111g.
The action pattern table 111g includes items of NO, facility ID, facility name, monitoring camera ID, target time zone, action pattern name, resting time, operation time, number of action cycles, factors and services.

"NO" is an arbitrary number of data stored in the action pattern table 111g.
The “facility ID” is the same as that stored in the robot management tables 111 c and 121 c and the monitoring camera management table 211 c.
"Facilities name" represents the name of the facility indicated by the facility ID.
“Monitoring camera ID” is the same as that stored in the monitoring camera management table 211 c.
The “target time zone” is a time zone targeted by the corresponding behavior pattern, in other words, a time zone to which the corresponding behavior pattern is applied.
The “action pattern name” represents the name of an action pattern registered in association with a facility ID.
The "rest time" represents a continuous time during which a person is at rest.
"Operation time" represents a continuous time during which a person is operating.
The “number of activity cycles” represents the number of repetitions of the stillness and the action.
“Factor / service” indicates a service provided by the robot 100 to the target person (stayer) when the target person's action matches the registered action pattern.

  For example, a person desiring to guide often sees information on a surrounding display board and an electronic display device in many cases. That is, when guidance is desired, the stationary time tends to be short and the repetitive action tends to be large. In addition, a person who is in poor physical condition tends to have a longer rest time and less repetitive behavior due to actions such as slow motion and curling. When such a poor physical condition person is detected, it is necessary to rush the robot 100 to the target person.

  In the example of the action pattern table 111g illustrated in FIG. 10, the action pattern is set in consideration of the traffic volume of the user by time zone and the specification of the target location for each facility. The behavior pattern in the shooting area of the surveillance camera ID "001" (international arrival lobby) assumes that there are many people in the morning (assuming congestion when there are a large number of arrival flights in the morning). In addition, the behavior pattern in the shooting area (restaurant / shop area) of the monitoring camera ID "002" assumes that there are many people who search for a meal place at noon. Furthermore, the behavior pattern in the shooting area of the surveillance camera ID "003" (international departure lobby) assumes that there is a lot of people coming and going in the afternoon (assuming congestion when there are many departure flights in the afternoon).

  Whether to register these behavior patterns can be determined in consideration of various conditions. For example, in the photographing area of the monitoring camera ID "001", if there are few people in the afternoon and it is not necessary to determine the staying person, the behavior pattern may not be registered. In addition, although there are few people, they detect staying people and take action by the robot 100, but if the tendency for people to stay is different from the tendency in the morning, target time zone, stationary time and operating time (and number of activity cycles) It is desirable to change and register a new behavior pattern.

  Moreover, the scenario is classified and registered so that it can respond appropriately according to the purpose and the content with respect to the guidance request of the staying person. In the example of FIG. 10, a service (scenario guidance A) is assumed in which a person arriving at the airport A is given guidance on the inside and the outside of the airport for the shooting area of the surveillance camera ID "001" (international arrival lobby). ing. In addition, a service (scenario guidance B) supposing guidance on a meal place in the airport is registered in the photographing area (restaurant / shop area) of the monitoring camera ID "002". In addition, a service (scenario guidance C) supposing guidance about a registration procedure to a person departing from the airport A is registered in the photographing area (international departure lobby) of the monitoring camera ID "003".

  Further, a service (scenario poor physical condition) in which a call to a person with poor physical condition, a call of a rescue person, etc. is assumed is registered in the photographing area of the monitoring camera ID "001". The type of scenario for poor health may also be classified and registered.

  As described above, by registering behavior patterns and factors / services for each facility (facility ID), shooting location (corresponding to surveillance camera ID), or time zone (target time zone), retention factor and service accuracy Can be improved. In particular, user traffic in the facility changes depending on the shooting location and time zone, so by registering the action pattern for each monitoring camera ID or target time zone, the robot control device 110 optimizes the robot 100 in a timely manner. It is possible to indicate various services and responses.

  In addition, after the start of operation of the robot management system 1, the operation time of the action pattern table 111g can be arbitrarily changed by operating the operation unit 122 of the robot monitoring center 120. In addition, applicable factors and services can be similarly changed.

(Behavior judgment result database)
FIG. 11 shows an example of the data structure of the action determination result database 121bD stored in the action data storage unit 121b.
The action determination result database 121bD includes items of NO, facility ID, facility name, shooting date, determination time, person ID, surveillance camera ID, staying person location, factor and service.

"NO" is an arbitrary number of data (record) stored in the action determination result database 121bD.
The "facility ID" is the same as that of the behavior pattern table 111g.
The "facility name" is the same as that of the behavior pattern table 111g.
The “photographing date” is a date when the surveillance camera 200 photographed the staying person.
The “determination time” is the time when the behavior analysis unit 111 e determines that there is a staying person in the image captured by the monitoring camera 200.
The “person ID” is an ID uniquely identifying a person in the video. Person IDs are numbered in the order of detection of persons as an example.
"Monitoring camera ID" indicates a monitoring camera ID that has captured the staying person.
The “staying place” indicates the place (position) where the staying person exists in the facility. The dwelling place is represented by, for example, a coordinate system (two-dimensional coordinates in this example) based on a certain position in the facility, or latitude and longitude.
“Factor / service” is the same as that of the action pattern table 111g.

  It is desirable that the data registered in the behavior determination result database 121bD be deleted after a predetermined period has elapsed. As a result, the amount of data stored in the storage area of the storage device of the robot monitoring center 120 can be reduced, and a certain free space can be secured.

[Procedure of service provision process for staying people]
Next, the flow of processing from detection of a staying person to instructing the robot 100 will be described with reference to FIG. FIG. 12 is a flowchart showing a procedure example of service provision based on the behavior pattern of the staying person by the robot management system 1.

  First, the administrator of the system registers in advance an action pattern that is determined to be a resident in the action pattern table 111g (S1). At this time, the system administrator registers the behavior pattern for each of the facilities and the shooting location of the monitoring camera 200. The shooting location corresponds to the surveillance camera ID.

  Next, after starting operation of the robot management system 1, the person detection unit 111d of the robot control device 110 detects a person from the image captured by the monitoring camera 200 by image processing technology, and assigns a person ID to the person. (S2). Next, the behavior analysis unit 111e calculates the stillness time and the movement time of the person detected by the person detection unit 111d by the image processing technique (S3).

  Next, the action pattern determination unit 111f determines whether the pattern of the target person's stillness and motion is repeated based on the target person's still time and the operation time calculated by the action analysis unit 111e (S4). Here, when the action pattern of the target person's stillness and the action is repeated (YES in S4), the action pattern determination unit 111f determines that the repeat action is present (S5). On the other hand, when the patterns of the stillness and the motion of the target person are not repeated (NO in S4), the behavior pattern determination unit 111f determines that there is no repetitive behavior (S6).

  Next, the action pattern determination unit 111f determines whether the action pattern of the target person's stillness and movement corresponds to the action pattern of the staying person (S7). Specifically, the action pattern determination unit 111f refers to the action pattern table 111g to search for an action pattern that matches the action pattern of the target person's stillness and action. Then, when there is no action pattern matching the action of the target person in the action pattern table 111g (NO in S7), the action pattern determination unit 111f returns to step S2 and uses the image captured by the monitoring camera 200. Continue processing to detect people.

  On the other hand, when there is an action pattern matching the action of the target person in the action pattern table 111g, the action pattern determination unit 111f determines that the target person is a staying person (YES in S7), and the staying factor is It is determined with reference to the item of factor and service in the action pattern table 111g. For example, the behavior pattern determination unit 111f determines whether the staying cause is poor health or not from the content of the item of the factor / service corresponding to the matched behavior pattern (S8).

  If the staying factor is poor health (YES in S8), the behavior pattern determining unit 111f sends a command to the robot 100 located at or near the target imaging area to move the robot 100 to the person of the staying person. Then, the action pattern determination unit 111 f reads out scenario data corresponding to poor physical condition from the scenario storage unit 111 h and transmits the scenario data to the robot 100. Accordingly, the dialogue control unit 101b and the drive control unit 101a of the robot 100 ask the staying person about the physical condition based on the received scenario data, and carry out the rescuer's call (S9).

  On the other hand, when the staying cause is not poor physical condition (NO in S8), the behavior pattern determination unit 111f then determines whether the staying cause is guidance request (S10). The behavior pattern determination unit 111f sends a command to the robot 100 located at or near the target imaging area, and moves the robot 100 to the person of the staying person, when the staying factor is the guidance request (YES in S10). Then, the action pattern determination unit 111 f reads out scenario data corresponding to the request for guidance from the scenario storage unit 111 h and transmits the scenario data to the robot 100. Accordingly, the dialogue control unit 101b and the drive control unit 101a of the robot 100 inquire the purpose of the staying person based on the received scenario data, and carry out facility guidance (S11).

  After the process of step S9 or step S10 is completed, or when the staying factor is not a guide request in step S10 (NO in S10), the robot 100 returns to the original standby position. Furthermore, when there is a next staying person, the robot 100 moves to the next staying person based on the instruction of the robot controller 110.

  In the present embodiment, when it is determined in step S7 that the target person is the staying person, the robot control device 110 performs various data when it is determined that the staying person is present in parallel with controlling the robot 100. It transmits to the monitoring center 120, and accumulate | stores data in action determination result database 121bD.

  As described above, according to the present embodiment, an activity pattern expected to be a cause of retention of a person is registered in the activity pattern table 111g in advance for each facility and shooting location (building 300, each shooting area, etc.) There is. The robot control device 110 determines whether or not the person photographed by the monitoring camera 200 installed at the site matches the condition of the registered action pattern. Accordingly, the robot control device 110 determines the presence or absence of a person (staying person) corresponding to the registered action pattern from the image captured by the monitoring camera 200 and the staying factor thereof. Then, the robot control device 110 can cause the robot 100 to provide a service to the staying person according to the staying cause by detecting the staying person in the building 300 and specifying the staying cause.

  Therefore, the robot 100 can appropriately provide the service desired by the staying person depending on the staying cause of the staying person. Furthermore, by providing the staying person with the service desired by the staying person, it is possible to implement support for eliminating the cause of the staying.

  Further, in the present embodiment, since the stationary time and the operating time registered in the behavior pattern table 111g can be arbitrarily changed, and the corresponding factors and services can be arbitrarily changed as well, the facility (for example, the building 300) itself can be used It is possible to flexibly respond to the determination criteria according to the current usage situation. The usage status is, for example, information as to whether the building 300 is used for an office, a supermarket, a commercial facility, a housing complex, or other purpose. As a result, it is possible to provide optimal service and response to people who have stagnated using the facility.

<2. Modified example>
When the number of persons constituting the group (group) to which the person detected from the video of the monitoring camera 200 belongs is equal to or more than a predetermined number, the action pattern determination unit 111 f does not determine the service to be provided. Do not offer services to the person in question. It is often the case that members of the group can solve the request for guidance of people belonging to the group and the treatment of poor physical condition. Therefore, the service such as calling by the robot 100 is self-solicited, and provision of unsolicited services is prevented. The group to which the detected person belongs can be detected by the person detection unit 111d. For example, when the person detection unit 111d detects a plurality of persons from the video and the plurality of persons move or stop in the same direction within a predetermined time, the plurality of persons (staying Person) is considered to form a group.

  In the embodiment described above, the person in the imaging area is photographed by the monitoring camera 200. However, the person may be photographed by the camera 102a included in the robot 100, and the presence or absence of the staying person may be determined from the photographed image.

  In addition, when there are a plurality of staying people in the same area of the building 300, the robot control device 110 may control the robot 100 to go to the source of the person who has been determined to be the staying person first. Alternatively, the robot control device 110 may control the robot 100 to be preferentially directed to a person who is in poor physical condition among a plurality of staying people.

  In the embodiment described above, an example is described in which the monitoring camera 200 is disposed in a facility (such as an airport, a station, a commercial facility, a supermarket, etc.) where it is assumed that there are many people coming and going and many staying people. It may be installed in a facility or place (such as an apartment house, staff passage, restroom, etc.) where there are few people coming and going and there are few or few people staying. By thus installing the monitoring camera 200 in a place where it is estimated that it will be difficult or unlikely to occur by the staying person, it is possible to prevent a person who has fallen ill due to poor physical condition from being left for a long time. In addition, when a staying person is detected in an environment where the staying person is unlikely to occur, the possibility of a person with poor physical condition or a suspicious person is high, so the present invention also contributes to crime prevention. By properly defining the behavior pattern, the accuracy of suspicious person detection is improved.

  Furthermore, the present invention is not limited to the embodiments described above, and it goes without saying that various other applications and modifications can be taken without departing from the scope of the present invention described in the claims. is there.

  For example, the above-described embodiment is a detailed and specific description of the configuration of the apparatus and system for the purpose of easy understanding of the present invention, and is not necessarily limited to one having all the components described. In addition, it is possible to replace part of the configuration of one embodiment with the component of another embodiment. In addition, it is also possible to add components of other example embodiments to the configuration of one example embodiment. Moreover, it is also possible to add, delete, and replace other components for part of the configuration of each embodiment.

  Further, some or all of the above-described components, functions, processing units, processing means, etc. may be realized by hardware, for example, by design of an integrated circuit. Further, each component, function, and the like described above may be realized by software by a processor interpreting and executing a program that realizes each function. Information such as a program, a table, and a file for realizing each function can be placed in a memory, a hard disk, a recording device such as a solid state drive (SSD), or a recording medium such as an IC card, an SD card, or a DVD.

  Further, control lines and information lines indicate what is considered to be necessary for the description, and not all control lines and information lines in the product are necessarily shown. In practice, almost all components may be considered to be connected to each other.

  Furthermore, in the present specification, processing steps that describe time-series processing are parallel or individual processing that is not necessarily performed chronologically, as well as processing performed chronologically in the order described. Processing (eg, parallel processing or processing by an object).

  DESCRIPTION OF SYMBOLS 1 ... Robot management system, 100, 100-1-100-3 ... Robot, 110 ... Robot control apparatus, 111 d ... Person detection part 111 d, 111 e ... Action analysis part, 111 f ... Action pattern determination part, 111 g ... Action pattern table, 111h Scenario storage unit 120 Robot monitoring center 200 200-1 to 200-3 Monitoring camera 210 Camera control device

Claims (7)

The robot includes a movable robot, a camera for photographing, and a person detection unit for detecting a person from a video taken by the camera, and the robot provides a service to the person detected by the person detection unit. A robot management system,
A behavior pattern table in which a behavior time defined by the resting time of the person, an operation time, and a combination of the resting time and the operation time and a service provided to the person are registered in association with each other;
When the action of the person detected from the video matches the action pattern registered in the action pattern table, the robot determines the service to be provided to the person based on the corresponding action pattern A robot management system comprising: an action pattern determination unit. When the action of the person detected from the image corresponds to the action pattern registered in the action pattern table, the action pattern determination unit determines that the person is a person who is staying at the place The robot management system according to claim 1. The robot management system according to claim 2, wherein the behavior pattern and the service are registered in the behavior pattern table for each facility. The robot management system according to claim 2, wherein the behavior pattern and the service are registered in the behavior pattern table for each imaging location in a facility. The robot management system according to claim 4, wherein the behavior pattern and the service are registered in the behavior pattern table for each time zone. The robot management system according to claim 2, wherein the behavior pattern determination unit does not cause the robot to perform the service when the number of persons constituting a group to which a person detected from the video belongs is equal to or more than a predetermined number. The robot control device sends a command to the robot based on the determination of the behavior pattern determination unit, and controls an operation of the robot.
The robot management system according to any one of claims 1 to 6, wherein the behavior pattern table and the behavior pattern determination unit are provided in the robot control device connected to the robot via a network.

Images