JP6131822B2 - Information processing apparatus, information processing method, and program - Google Patents

Description

  The present disclosure relates to an information processing apparatus, an information processing method, and a program for sensor output of an acceleration sensor.

  In recent years, information sensors such as mobile phone terminals have been equipped with various sensors as their functions have increased. For example, a device having an acceleration sensor determines the current state of the device based on the output value of the sensor.

  Patent Documents 1 to 3 propose a method for determining a walking state of a user who carries an information processing device using an output value of an acceleration sensor. In these documents, acceleration information in the vertical direction (vertical direction or gravity direction) of the information processing device is calculated using the output value of the acceleration sensor, or similarity is evaluated by comparing two time-series data. .

JP 2010-257395 A JP 2012-63208 A JP 2012-247991 A

  However, in order to determine the operating state of the user carrying the information processing apparatus, it is desirable to detect not only the vertical acceleration magnitude but also the horizontal acceleration magnitude and the longitudinal acceleration magnitude.

  Some information processing apparatuses such as mobile phone terminals include a main microcomputer for processing user operations and executing communication, and a sub-microcomputer for processing output information from an acceleration sensor or the like. The main microcomputer is high-performance because it controls not only the entire apparatus but also applications such as games and browsers that are executed by users. In general, it can be considered that the performance of the microcomputer and the power consumption are in a proportional relationship. In the information processing apparatus, driving time (battery duration) is one of important factors. If power consumption in the information processing apparatus is suppressed, the drive time of the information processing apparatus can be extended. Therefore, the main microcomputer is put into a sleep state as much as possible. For this reason, a configuration in which the output of the acceleration sensor is not monitored for a long time by the main microcomputer is usually employed.

  Therefore, a sub-microcomputer of the information processing apparatus is used. In general, the power consumption of the sub-microcomputer is about 1/200 to 1/1000 of the main microcomputer. That is, even if the sub-microcomputer is operated for a long time, there is no problem from the viewpoint of power consumption. However, the performance of the sub-microcomputer is weak, and it can be executed if it is a simple process such as four arithmetic operations of integers and memory operations. In order to evaluate a user's behavior using a sub-microcomputer, processing is performed based on the simple calculation as described above.

For example, when a value output in the coordinate system of the acceleration sensor is converted into a value in a so-called global coordinate system in the vertical and horizontal directions, such conversion can be performed by performing an operation using a rotation matrix. Such conversion is called coordinate conversion, and is usually performed using a trigonometric function. However, as described above, a sub-microcomputer capable of processing four arithmetic operations with integers and a memory operation cannot perform trigonometric functions with a high processing load.

  The technology of the present disclosure has been made in view of the above, and an object thereof is to determine an operation state of a user carrying the information processing apparatus by simple arithmetic processing.

  In one aspect, the information processing apparatus according to the present disclosure includes an acceleration sensor that detects acceleration in a three-dimensional space, a low-pass filter that extracts a gravity component vector signal from the sensor output of the acceleration sensor, and time for the gravity component vector signal. A time delay unit for adding a delay, an inner product value calculation unit for calculating an inner product value of the signal of the gravity component vector extracted by the low-pass filter and the signal of the gravity component vector given the time delay by the time delay unit, and a predetermined By performing template matching on the temporal change of the inner product value calculated by the inner product value calculation unit over a predetermined time using the template storage unit that stores in advance a template indicating the temporal change of the inner product value in the operation of An operation determining unit that determines whether or not a predetermined operation has occurred.

  According to the technique of the present disclosure, it is possible to determine the operation state of the user carrying the information processing apparatus by simple arithmetic processing.

FIG. 1 is a schematic configuration diagram illustrating a hardware configuration of a mobile phone terminal according to an embodiment. FIG. 2 is a functional configuration diagram illustrating a functional unit of the mobile phone terminal according to the embodiment. FIG. 3 is a schematic configuration diagram of a sub-microcomputer according to an embodiment. FIG. 4A is a diagram illustrating a sensor output vector of the acceleration sensor according to the embodiment. FIG. 4B is a diagram illustrating a sensor output vector of the acceleration sensor according to the embodiment. FIG. 5A is a schematic diagram illustrating a user's walking state according to an embodiment. Drawing 5B is a mimetic diagram showing a user's walk state in one embodiment. FIG. 6A is a sequence diagram illustrating an operation of the low-pass filter according to the embodiment. FIG. 6B is a diagram illustrating a part of a flowchart of processing executed in the low-pass filter according to the embodiment. FIG. 7A is a sequence diagram illustrating an operation of the delay circuit according to the embodiment. FIG. 7B is a diagram illustrating a part of a flowchart of processing executed in the delay circuit according to the embodiment. FIG. 8 is a sequence diagram showing the operation of the correlation calculation processing unit in one embodiment. FIG. 9 is a schematic diagram illustrating the configuration of the buffer of the correlation calculation processing unit in one embodiment. FIG. 10 is a table illustrating a configuration example of the operation determination template according to the embodiment. FIG. 11 is a diagram illustrating a part of a flowchart of processing executed in the correlation calculation processing unit in the embodiment. FIG. 12 is a graph illustrating processing executed in the correlation calculation processing unit in one embodiment. FIG. 13 is a diagram illustrating a part of a flowchart of processing executed in the correlation calculation processing unit in one embodiment.

  Hereinafter, a mobile phone terminal and a control method thereof according to an embodiment will be described with reference to the drawings.

As shown in FIG. 1, the mobile phone terminal 10 in this embodiment includes an antenna 101, an RF (Radio Frequency) unit 102, a main microcomputer 103, a sub-microcomputer 104, and a button 10.
5, a display 106, a touch panel 107, a ROM (Read Only Memory) 108, a RAM (Random-Access Memory) 109, and an acceleration sensor 110. Note that the mobile phone terminal 10 corresponds to an example of an information processing apparatus. However, the technology disclosed in the present embodiment is not limited to the mobile phone terminal 10 and can be applied to information processing apparatuses in general. In the present embodiment, the mobile phone terminal 10 is carried by the user.

  The mobile phone terminal 10 performs wireless communication with an external wireless transmission / reception device (not shown) such as a base station via the antenna 101. In transmission / reception with an external wireless transmission / reception device, the RF unit 102 modulates a baseband signal to a frequency band (RF band) at the time of transmission, and demodulates a signal of these frequency bands to a baseband signal at the time of reception.

  The main microcomputer 103 executes the control program 108a stored in the ROM 108, and realizes various functions of the mobile phone terminal 10. The sub-microcomputer 104 executes the acceleration sensor output processing program 104a incorporated in the sub-microcomputer 104, performs arithmetic processing on the sensor output of the acceleration sensor 110, and performs the coordinate system of the user of the mobile phone terminal 10, that is, the global coordinate system. Convert to acceleration. Further, the sub-microcomputer 104 executes the operation state determination program 104b to generate a determination result of the operation state of the user of the mobile phone terminal 10. In addition, it is good also as a structure which performs the various arithmetic processing in this embodiment using a 1 or more submicrocomputer. Further, the acceleration sensor output processing program 104a and the operation state determination program 104b may be configured as one program.

  The button 105 is a hardware key that the user uses to operate the mobile phone terminal 10. The display 106 displays various information such as the results of various processes executed in the mobile phone terminal 10. The display 106 displays processing results, processing states, contents, and the like of various processes executed by the main microcomputer 103 and the sub-microcomputer 104. The touch panel 107 receives a user's touch operation and the like. The touch panel 107 is disposed so as to overlap the display 106. The user selects a display item such as an icon displayed on the display 106 by touching the touch panel 107 with a finger or the like.

  The ROM 108 stores various programs executed in the mobile phone terminal 10 and data of an operation determination template described later. The ROM 108 is an example of a template storage unit. The RAM 109 is used as a temporary storage area when the main microcomputer 103 and the sub-microcomputer 104 execute various processes. That is, the RAM 109 is used as a work area, and stores unique information of the mobile phone terminal such as various programs and terminal numbers. The RAM 109 stores settings input by the user, program information, data provided to the user via the display 106, and the like.

The main microcomputer 103 can execute high-speed arithmetic processing, but consumes a large amount of power. Therefore, since it is not preferable to keep the operation state for a long time, a certain restriction is imposed on the operation time of the main microcomputer 103, and the operation state is temporarily shifted to the rest state. By the way, when determining a user's operation state, a user's operation data is acquired over a long time, and an operation state is determined based on the acquired operation data.

  Therefore, in the present embodiment, the sub-microcomputer 104 is kept in an operating state at least during a user's operation, receives the sensor output of the acceleration sensor 110, performs arithmetic processing on the sensor output, and performs acceleration in the global coordinate system. Generate data. The sub-microcomputer 104 executes processing with a low processing load such as simple arithmetic processing such as four arithmetic operations and access processing to a memory such as the ROM 108 and the RAM 109. However, since the sub-microcomputer 104 has low power consumption, it can execute the acceleration sensor output processing program 104a and the operation state determination program 104b for a long time.

  Therefore, it is suppressed that the main microcomputer 103 becomes an operation state, and the sub microcomputer 104 converts the sensor output into a value that is easy for the user to understand based on the determination result of the user's operation state or creates statistical information. The generated information is displayed on the display 106. The generated information may be transmitted to the outside via the antenna 101.

  In the present embodiment, an application or the like stored in the ROM 108 of the mobile phone terminal 10 is executed by the sub-microcomputer 104. By executing such an application, as shown in FIG. 2, the sub-microcomputer 104 of the mobile phone terminal 10 functions as a time delay unit 201, an inner product value calculation unit 202, and an operation determination unit 203. However, any one or more of the time delay unit 201, the inner product value calculation unit 202, and the operation determination unit 203 may be a hardware circuit.

  Next, in the present embodiment, a configuration, algorithm, and processing for detecting a predetermined operation performed by the user, which is executed in the sub-microcomputer 104 of the mobile phone terminal 10 will be described. FIG. 3 shows an outline of a part of the configuration of the sub microcomputer 104 and the algorithm executed by the sub microcomputer 104 in the present embodiment. As shown in FIG. 3, the sub-microcomputer 104 includes a low-pass filter (LPF) 104c, a delay circuit 104d, an inner product calculation processing unit 104e, and a correlation calculation processing unit 104f. The delay circuit 104d is an example of a time delay unit. The inner product calculation processing unit 104e is an example of an inner product value calculation unit. Furthermore, the correlation calculation processing unit 104f is an example of an operation determination unit.

  In the present embodiment, the gravity direction vector G is detected by passing the LPF 104c with respect to the sensor output A of the acceleration sensor 110. On the other hand, the LPF 104c is connected to the delay circuit 104d. Since the delay circuit 104d outputs the input signal after a predetermined time, the past gravity direction vector G 'is output from the delay circuit 104d. Then, the inner product calculation processing unit 104e calculates the inner product of the gravity direction vector G detected by the LPF 104c and the gravity direction vector G ′ output from the delay circuit 104d. The inner product value p obtained by the calculation of the inner product calculation processing unit 104e is sent to the correlation calculation processing unit 104f. The correlation calculation processing unit 104f performs a correlation calculation on the inner product value p output from the inner product calculation processing unit 104e using the template stored in the ROM 108, and determines the user's operation.

  Here, the advantage of the present embodiment will be described regarding the process of converting the sensor output A of the acceleration sensor 110 into the acceleration in the coordinate system of the user carrying the mobile phone terminal 10, that is, the so-called global coordinate system.

4A and 4B show an example of a method for extracting vertical (vertical) acceleration information in the user's coordinate system from the sensor output of the acceleration sensor when the posture of the user carrying the mobile phone terminal is unknown. FIG. Here, it is assumed that the user is walking. As shown in FIG. 4A, the X axis, the Y axis, and the Z axis orthogonal to each other constitute a local coordinate system of the mobile phone terminal. In FIG. 4A, the sensor output A of the acceleration sensor is given by A = (x, y, z) ^T. T means a transposed matrix. As shown in FIG. 4A, the sensor output A of the acceleration sensor can be considered as a combination of vibration S and gravity acceleration G accompanying the user's walking.

  Here, we outline human walking. 5A and 5B are diagrams schematically illustrating the movement of the trunk and both feet in human walking. FIG. 5A is a diagram of a human walking viewed from directly above (vertically upward), and FIG. 5B is a diagram of a human walking viewed from right side (horizontal direction). The trunk 20 corresponds to the portion from the human hip joint to the head, and the legs 20L and 20R correspond to the left leg and the right leg, respectively. As shown in FIG. 5A and FIG. 5B, during walking, the human right and left legs 20L and 20R alternately touch the ground, and the trunk 20 swings up and down, left and right, and front and rear. Further, as shown in FIGS. 5A and 5B, the period of the vertical movement of the trunk 20 is a period from when one of the left and right legs 20L and 20R arrives again from the ground and then arrives again after the other leaves the ground. Matches. On the other hand, the left and right shaking of the trunk 20 swings to the left of the person while the left leg 20L is on the ground, and swings to the right of the person while the right leg 20R is on the ground. Therefore, the period of the left and right shaking of the trunk 20 during human walking is twice as long as the period of vertical movement, that is, a frequency of 1/2.

  In addition, when a human is walking, the moment of landing the left leg 20L or the right leg 20R on the ground has the greatest downward acceleration. Then, after the landing of the left leg 20L or the right leg 20R, an upward acceleration is generated when the trunk 20 is raised, and the upward acceleration is minimized at the highest point reached by the trunk 20, and thereafter Acceleration in the downward direction occurs. As for the left-right acceleration, when the left leg 20L is landed, leftward acceleration is generated, and when the right leg 20R is landed, rightward acceleration is generated.

Returning to FIGS. 4A and 4B, as described with reference to FIGS. 5A and 5B, it can be said that the vibration S accompanying the walking of the user is a repetitive motion with a predetermined cycle. Therefore, when the sensor output A of the acceleration sensor is passed through an LPF or high-pass filter (HPF) having an appropriate cutoff frequency, the sensor output A is separated into a vibration component vector of vibration S and a gravity component vector of gravitational acceleration G. Can do. The vertical acceleration associated with walking can be expressed by a vector V parallel to the gravity component vector of the gravitational acceleration G in the vibration component vector of the vibration S. When the magnitude of the vector V is v and the angle between the vibration component vector of the vibration S and the gravity component vector of the gravitational acceleration G is θ, v is given by the following equation.
v = | S | cosθ (1)

Consider the inner product of vibration S and gravitational acceleration G. The inner product is obtained by multiplying the x, y, and z components of the vector and then calculating the sum of the multiplied components. Note that the inner product of the vibration component vector of the vibration S and the gravity component vector of the gravitational acceleration G is given by the following expression from the definition.
S + G = | S | | G | cosθ (2)

When equation (2) is applied to equation (1), the following equation (3) is obtained.
v = S · G / | G | (3)

Here, the magnitude | G | of the gravitational acceleration G is substantially constant on the earth. In other words, the sensor output A of the acceleration sensor is passed through LPF and HPF, and the inner product of the output of each filter is taken to obtain a value proportional to the acceleration in the upward direction accompanying the user's walking motion and proportional to the acceleration in the downward direction You can get a numerical value. In the above description, it is assumed that the user is walking. However, even when the user performs an operation other than walking, based on the above algorithm, the vibration component vector and the gravity component vector can be obtained by passing the LPF and HPF through the sensor output A of the acceleration sensor.

  By the way, in the above method, the vibration component vector of the vibration S is calculated by passing the sensor output of the acceleration sensor through the HPF. However, human motion includes motion accompanied by rotational motion of the trunk, such as bowing or lying down. In such an operation, the direction of the gravitational direction viewed from the acceleration sensor greatly changes with the movement of the trunk. In addition, if the terminal position is at the rotational center of the rotational movement of the trunk when such an operation occurs, the position of the terminal does not change from the rotational center. It cannot be detected.

  On the other hand, in order to detect the rotational motion associated with such an operation, a configuration in which an angular velocity sensor is provided in the terminal and rotation information in the global coordinate system is acquired using an appropriate coordinate transformation for the output of the angular velocity sensor is conceivable. However, since the angular velocity sensor generates a fine movement internally in order to detect the Coriolis force, it generally requires a larger electric power than the acceleration sensor. Therefore, from the viewpoint of the power consumption of the terminal, it is not preferable to employ the angular velocity sensor for analyzing the user's movement. Therefore, in the present embodiment, the amount of rotational motion necessary for determining various user operations is extracted from the sensor output of the acceleration sensor using the configuration described below. As a result, in this embodiment, the user's motion determination can be performed by performing a simple operation using four arithmetic operations without using a complicated operation such as a trigonometric function, thereby reducing power consumption associated with the operation. Can do.

  6A and 6B show a configuration example of the LPF 104c and processing executed in the LPF 104c in the present embodiment. As shown in FIG. 6A, in the LPF 104c, the sensor output value X of the acceleration sensor 110 is stored in n buffers BF0 to BFn−1 in synchronization with the clock CLK synchronized with the sampling clock of the acceleration sensor 110. . The clock CLK is an example of a predetermined time interval that determines the execution timing of the operation determination process in the present embodiment. Then, an average value of n consecutive sensor values X is calculated by an averaging process using the adder 104g and the divider 104h that multiplies the input value by 1 / n. That is, the LPF 104c can be realized by executing an averaging process for obtaining an average of the values X of n sensor outputs. Therefore, when executing the acceleration sensor output processing program 104a, the sub-microcomputer 104 stores the sensor output value X in the buffer one by one, adds the previous n sensor output values, and divides by n. , The LPF 104c is processed.

  As shown in FIG. 6B, in the LPF 104c, when the value X is input, when the input is the first time, the value held by each buffer is set to X through S101 to S107, and the total value is calculated. The process of multiplying the total value by 1 / n is executed. In the figure, in S101 to S113, “index” indicates an index set in each buffer. “Buf [index]” indicates an input value held by a buffer having an index indicated by the value of “index”. In the figure, other symbols follow so-called C language rules. The same applies to the description of other flowcharts below. When the input of the value X is not the first time, S108 to S113 are repeatedly executed in units of clocks, whereby an averaging process using values sequentially input after the input is executed. The output of the divider 104h is sent to the delay circuit 104d.

  Next, FIG. 7A and FIG. 7B show a configuration example of the delay circuit 104d and processing executed in the delay circuit 104d in the present embodiment. As shown in FIG. 7A, the delay circuit 104d has m buffers BF0 to BFm-1. The number of buffers is appropriately determined according to the delay time of the gravity direction vector G by the delay circuit 104d. As shown in FIG. 7A, in this embodiment, m buffers are connected.

  As shown in FIG. 7B, in the delay circuit 104d, when the value X is input, when the input is the first time, initialization is performed to set the value held in each buffer to 0 through S201 to S206. When the input of the value X is not the first time, S207 to S211 are repeatedly executed in units of clock CLK, so that the input value X is sequentially transferred from the buffer BF0 having the index 0 to the subsequent buffers BF1 to BFm-1. To go. Further, the value output from the buffer BFm-1 having the index M-1 is sent to the inner product calculation processing unit 104e. In the present embodiment, the value output from the buffer BFm−1 is the past gravity direction vector G ′.

As shown in FIG. 3, the inner product calculation processing unit 104e receives the gravity direction vector G output from the LPF 104c and the past gravity direction vector G ′ output from the delay circuit 104d. The inner product calculation processing unit 104e obtains an inner product value p of the vector G and the vector G ′ by the following equation (4).
p = G ・ G '= | G | | G' | cosθ (4)
Here, the angle between the vector G and the vector G ′ is θ. The inner product value p output from the inner product calculation processing unit 104e is sent to the correlation calculation processing unit 104f.

  By the way, when a person walks as an example, when the so-called pitch axis, roll axis, and yaw axis are set, the hip joint rotates greatly in the pitch direction, and the rotation in the roll direction is smaller than the rotation in the pitch direction. . That is, in determining various motion states including the user's walking from the sensor output of the acceleration sensor 110, even if the rotation around the pitch axis cannot be calculated, the rotation around the axis parallel to the horizontal plane can be calculated. This rotation can be regarded as a rotation about the pitch axis.

  Since the vector G and the vector G ′ are also gravity vectors, the magnitude thereof is almost the gravitational acceleration. From this, the inner product of G and G ′ may be considered to be a constant (square of gravity acceleration) times the cosine of the angle formed by these two vectors. Further, the cosine decreases monotonically from 0 ° to 180 °. In other words, the cosine value itself can be used for the operation determination process without converting the expression (4) into an angle θ expression using an inverse function, as shown below. Therefore, in this embodiment, it is possible to determine the user's motion without performing an arithmetic operation such as a trigonometric function using the angular velocity sensor.

  FIG. 8 shows a part of the configuration of the correlation calculation processing unit 104f in the present embodiment. As shown in FIG. 8, the correlation calculation processing unit 104f includes a plurality of template data storage units 104i, 104j, and 104k. The template data storage units 104i, 104j, and 104k are input with values stored in a template value column for operation determination described later. The values stored in the template data storage units 104i, 104j, and 104k are input to different subtracters 104l, 104m, and 104n, respectively. Although FIG. 8 shows three template data storage units and three subtracters, the number of template data storage units and subtracters can be changed as appropriate according to the number of indexes of the template for operation determination described later. .

  The correlation calculation processing unit 104f includes k buffers BF0 to BFk-1. The inner product value p ("input X" in the figure) output from the inner product operation processing unit 104e is stored in the buffer BF0, and when the next inner product value is input to the correlation operation processing unit 104f, it is stored in the buffer BF0. The value is passed to the subsequent buffer BF1. In this way, each time the inner product value is input to the correlation calculation processing unit 104f, the values stored in each buffer are sequentially passed to the subsequent buffer. Note that the value stored in the buffer BFk-1 is updated with the value input to the buffer BFk-1.

An example of the operation determination process of the correlation calculation processing unit 104f in the present embodiment will be described with reference to FIGS. FIG. 9 shows buffers BF0 to BF of the correlation calculation processing unit 104f.
An example of the inner product value p stored in k−1 is shown. The inner product value p output from the inner product calculation processing unit 104e is sequentially stored from a buffer having a small index value. In the case of FIG. 9, the inner product value p is sequentially output as 3, 2, 5, 5 from the inner product calculation processing unit 104e, and is sequentially stored in the correlation calculation processing unit 104f from the buffer whose index is index0. When the inner product value p is stored in the buffer whose index is indexK-1, the inner product value p that is input next to the correlation calculation processing unit 104f is stored back to the buffer whose index is index0.

  FIG. 10 shows an example of the data structure of the operation determination template. In FIG. 10, each value of “sampling index” and “time (t)” is used to designate a subtractor that obtains a value from a buffer, as will be described later. The “value (d)” is a value stored in the template data storage unit. The values “sampling index”, “time (t)”, and “value (d)” are used in combination. That is, the value of “value (d)” is input to the subtracter specified by “sampling index” and “time (t)”. A specific example will be shown when explaining a flowchart described later.

  FIG. 11 is a flowchart of the operation determination process executed in the correlation calculation processing unit 104f in the present embodiment. In the correlation calculation processing unit 104f, when the value X is input, when the input is the first time, initialization is performed to set the value held by each buffer to 0 through S301 to S306. If the input of the value X is not the first time, the degree of coincidence with the template for operation determination is calculated by repeatedly executing S307 to S316 in units of clock CLK.

  Here, the degree of coincidence with the action determination template is the coincidence between the change of the inner product value p input to the correlation calculation processing unit 104f in the past predetermined time range and the change of the value indicated by the action determination template. It is an index indicating the degree of. Here, the past predetermined time range is determined by the time range in which the value changes in the operation determination template to be used.

  In S307, the value X is stored in the buffer corresponding to the currently designated index. Here, the currently designated index means that when the value X is input for the first time, that is, when no value is stored in any of the buffers, the values of the respective buffers are initialized by S301 to S306. Since the process of “index = 0” is executed, it is index0. When the value X is not input for the first time, that is, when the value is already stored in the buffer, the currently designated index is the index incremented in S314. Next, the process proceeds to S308.

  In S308 and S309, the value of the matching degree COR and the value of the variable j are each set to 0. Here, the variable j is the value of the sampling index in FIG. Next, the process proceeds to S310. In S310 to S313, the difference between the absolute values of the buffer value input at the time determined by the value of the sampling index of the motion determination template and the value of the motion determination template is sequentially calculated. First, in S311, the position (i) of the input buffer is determined at a time traced back by the time (t) paired with the sampling index value, starting from the buffer specified by the current index value. . If the value of i is calculated by the formula shown in S311, the process proceeds to S312. In S312, the difference between the value (d) paired with the value of the sampling index in the operation determination template and the value of the buffer at the position determined by S311 is integrated as the matching degree COR.

In the present embodiment, as shown in FIG. 10, the operation determination template has M sampling indexes of 0 to M−1. Therefore, the processing of S310 to S313 is M.
After being repeated a number of times, the process proceeds to S314. In S314 to S316, when the inner product value p input to the correlation calculation processing unit 104f is stored in the right end buffer (index is K-1, see FIG. 9), the inner product value p input next is The index value is changed so as to be stored in the buffer (index is 0, see FIG. 9).

  As an example, in the case shown in FIGS. 9 and 10, it is assumed that the value “5” stored in the buffer whose index is index4 is the input value of the latest inner product p. In this case, by the processing of S310 to S313, first, the degree of coincidence is calculated by using the sampling index value and the set of time (t) and value (d) “0, 0, 0” in the operation determination template. Is done. Since the value of time (t) is “0”, the buffer determined by S311 is a buffer whose index is index4. Further, since the value (d) is 0, the matching degree COR calculated in S312 is COR = | 0-0 | = 0.

  Next, through S313 and S310, when j is incremented by 1 and S311 and S312 are executed again, the set “1, 3, 10” of the sampling index value, time (t) and value (d) is set. Using this, the degree of coincidence is calculated. The buffer whose time (t) value is determined by “3” S311 is a buffer whose index is index1. Further, since the value (d) is “10”, the matching degree COR calculated in S312 is 0 as described above, so that COR = 0 + | 5−10 | = 5 It is. In this way, the degree of coincidence COR is integrated using the values of the sampling index and the values of each set of time (t) and value (d). Thus, as the degree of coincidence COR decreases, the time change of the inner product value calculated from the sensor output of the acceleration sensor 110 approaches the time change of the inner product value indicated by the motion determination template.

  FIG. 12 is a diagram schematically showing the process of calculating the degree of coincidence in the present embodiment. As shown in FIG. 12, it is assumed that a change in the inner product value in a predetermined time range is given by a curve C for the operation to be determined. Then, in curve C, K sampling positions on the time axis are determined. The sampling index values are sequentially assigned from 0 in order of increasing proximity to the sampling position from the current time. Also, the time difference from the current time at each sampling position is “time (t)”, and the inner product value taken by the curve C is “value (d)”. A template for determining an operation is created by combining a set of the value of the sampling index, the value of “time (t)”, and the value of “value (d)”. Then, by executing the processing of the above flowchart, the difference between the inner product value indicated by the operation determination template and the value actually input to the correlation calculation processing unit 104f is calculated retroactively from the present to the past. The coincidence degree COR indicating the correlation between these values is calculated.

  FIG. 13 shows a flowchart of the operation determination process executed by the correlation calculation processing unit 104f in the present embodiment. This flowchart is started each time the integration of the degree of coincidence COR is completed through S315 or S316 according to the flowchart shown in FIG. As shown in FIG. 13, in S401, it is determined whether or not the value of the degree of coincidence COR integrated by the flowchart shown in FIG. 11 is equal to or less than a predetermined threshold Th. The operation determination template used in the present embodiment is a template for determining whether or not a user carrying the mobile phone terminal 10 bends as an example. Therefore, when the value of the matching degree COR is equal to or less than the predetermined threshold Th (S401: Y), it is determined that the user has bent down. Further, when the value of the matching degree COR exceeds the predetermined threshold Th (S401: N), it is determined that the user is not performing an action of bending his / her waist.

  The result of the determination process according to the flowchart shown in FIG. 13 can be stored in the RAM 109 of the mobile phone terminal 10, displayed on the display 106, or transmitted to an external device by the RF unit 102 and the antenna 101.

  The above is the description regarding the present embodiment, but the configuration and processing of the information processing apparatus are not limited to the above-described embodiment, and may be variously within a range that does not lose the technical idea of the present invention. Can be changed. For example, in the above description, one template for determining whether or not the user has bent down is used as a template for motion determination, but the number and types of templates are not limited to this. For example, there are individual differences in the behavior of bending the waist depending on the gender and physique of the user. In other words, a plurality of templates for determining whether or not the hips are bent are stored in the ROM 108 or the like, and the above processing is repeated for each template, so that it is possible to determine how to bend multiple patterns of the hips. it can. Further, not only a template for determining whether or not the user is bent, but also a template for determining various operations such as sitting, bowing, jumping, and the like can be stored in the ROM 108 or the like for each operation. Thereby, it can be set as the structure which can determine various operation | movement of the user who carries the mobile telephone terminal 10. FIG.

<Computer-readable recording medium>
A management tool for setting the information processing apparatus in a computer or other machine or device (hereinafter referred to as a computer or the like), a program for realizing an OS or the like can be recorded on a computer-readable recording medium. Here, the setting means, for example, assignment of button functions. The function can be provided by causing a computer or the like to read and execute the program of the recording medium. Here, the computer is, for example, a mobile phone terminal.

  Here, a computer-readable recording medium is a recording medium that stores information such as data and programs by electrical, magnetic, optical, mechanical, or chemical action and can be read from a computer or the like. Say. Examples of such a recording medium that can be removed from a computer or the like include a flexible disk, a magneto-optical disk, a CD-ROM, a CD-R / W, a DVD, a Blu-ray disk, a DAT, an 8 mm tape, a flash memory, and the like. There are cards. Moreover, there are a hard disk, a ROM, and the like as a recording medium fixed to a computer or the like.

  Regarding the above embodiment, the following additional notes are disclosed.

(Appendix 1)
An acceleration sensor for detecting acceleration in a three-dimensional space;
A low-pass filter for extracting a gravity component vector signal from the sensor output of the acceleration sensor;
A time delay unit for adding a time delay to the signal of the gravity component vector;
An inner product value calculation unit for calculating an inner product value of the signal of the gravity component vector extracted by the low-pass filter and the signal of the gravity component vector given a time delay by the time delay unit;
A template storage unit that stores in advance a template indicating a temporal change of the inner product value in a predetermined operation;
An operation determination unit that determines whether or not the predetermined operation has occurred by performing template matching on a temporal change of the inner product value calculated by the inner product value calculation unit over a predetermined time using the template; ,
An information processing apparatus comprising:

(Appendix 2)
In the template matching, the motion determination unit calculates, as the degree of coincidence, a value obtained by integrating the difference between the inner product value calculated by the inner product value calculation unit and the inner product value indicated by the template over the predetermined time. Supplementary note 1 wherein if the degree of coincidence is less than a predetermined threshold, it is determined that the predetermined action has occurred, and if the calculated degree of coincidence is not less than a predetermined threshold, it is determined that the predetermined action has not occurred. The information processing apparatus described.

(Appendix 3)
The motion determination unit stores the inner product value calculated by the inner product value calculation unit in a time series at predetermined time intervals, and newly stores the inner product value calculated by the inner product value calculation unit. The information processing apparatus according to appendix 2, wherein the degree of coincidence is calculated with respect to a temporal change of an inner product value including the inner product value stored in the item 2.

(Appendix 4)
Detecting acceleration in a three-dimensional space;
Extracting a gravity component vector signal from the acceleration;
Providing a time delay to the signal of the gravity component vector;
Calculating an inner product value of the extracted gravity component vector signal and the time delay-added gravity component vector signal;
It is determined whether or not the predetermined operation has occurred by performing template matching on the temporal change of the calculated inner product value over a predetermined time using a template indicating the temporal change of the inner product value in a predetermined operation. A determining step;
An information processing method comprising:

(Appendix 5)
In the template matching, the step of determining whether or not the predetermined operation has occurred is calculated as a degree of coincidence by integrating a difference between the calculated inner product value and the inner product value indicated by the template over the predetermined time. If the calculated degree of coincidence is less than a predetermined threshold, it is determined that the predetermined action has occurred. If the calculated degree of coincidence is equal to or greater than the predetermined threshold, it is determined that the predetermined action has not occurred. The information processing method according to appendix 4.

(Appendix 6)
The step of determining whether or not the predetermined operation has occurred stores the calculated inner product value in a time series at predetermined time intervals, and newly stores the calculated inner product value when the calculated inner product value is newly stored. The information processing apparatus according to appendix 5, wherein the degree of coincidence is calculated with respect to a temporal change of the inner product value including the inner product value stored in the item 5.

(Appendix 7)
Detecting acceleration in a three-dimensional space;
Extracting a gravity component vector signal from the acceleration;
Providing a time delay to the signal of the gravity component vector;
Calculating an inner product value of the extracted gravity component vector signal and the time delay-added gravity component vector signal;
It is determined whether or not the predetermined operation has occurred by performing template matching on the temporal change of the calculated inner product value over a predetermined time using a template indicating the temporal change of the inner product value in a predetermined operation. A determining step;
A program that causes a computer to execute.

(Appendix 8)
In the template matching, the step of determining whether or not the predetermined operation has occurred is calculated as a degree of coincidence by integrating a difference between the calculated inner product value and the inner product value indicated by the template over the predetermined time. If the calculated degree of coincidence is less than a predetermined threshold, it is determined that the predetermined action has occurred. If the calculated degree of coincidence is equal to or greater than the predetermined threshold, it is determined that the predetermined action has not occurred. The program according to appendix 7.

(Appendix 9)
The step of determining whether or not the predetermined operation has occurred stores the calculated inner product value in a time series at predetermined time intervals, and newly stores the calculated inner product value when the calculated inner product value is newly stored. The program according to appendix 8, wherein the degree of coincidence is calculated with respect to a temporal change in the inner product value including the inner product value stored in the item 8.

104 Sub-microcomputer 104a Acceleration sensor output processing program 104b Operation state determination program 104c LPF
104d Delay circuit 104e Inner product calculation processing unit 104f Correlation calculation processing unit 104g, 104o Adder 104h Dividers 104i-104k Template data storage units 104l-104n Subtractor 108 ROM
109 RAM
DESCRIPTION OF SYMBOLS 110 Acceleration sensor 201 Time delay part 202 Inner product value calculation part 203 Motion determination part

Claims (5)

An acceleration sensor for detecting acceleration in a three-dimensional space;
A low-pass filter for extracting a gravity component vector signal from the sensor output of the acceleration sensor;
A time delay unit for adding a time delay to the signal of the gravity component vector;
An inner product value calculation unit for calculating an inner product value of the signal of the gravity component vector extracted by the low-pass filter and the signal of the gravity component vector given a time delay by the time delay unit;
A template storage unit that stores in advance a template indicating a temporal change of the inner product value in a predetermined operation;
An operation determination unit that determines whether or not the predetermined operation has occurred by performing template matching on a temporal change of the inner product value calculated by the inner product value calculation unit over a predetermined time using the template; ,
An information processing apparatus comprising:   In the template matching, the motion determination unit calculates, as the degree of coincidence, a value obtained by integrating the difference between the inner product value calculated by the inner product value calculation unit and the inner product value indicated by the template over the predetermined time. The determination is made that the predetermined operation has occurred if the degree of coincidence is less than a predetermined threshold value, and it is determined that the predetermined operation has not occurred if the calculated degree of coincidence is equal to or greater than the predetermined threshold value. The information processing apparatus described in 1.   The motion determination unit stores the inner product value calculated by the inner product value calculation unit in a time series at predetermined time intervals, and newly stores the inner product value calculated by the inner product value calculation unit. The information processing apparatus according to claim 2, wherein the degree of coincidence is calculated with respect to a temporal change in the inner product value including the inner product value stored in the item. Detecting acceleration in a three-dimensional space;
Extracting a gravity component vector signal from the acceleration;
Providing a time delay to the signal of the gravity component vector;
Calculating an inner product value of the extracted gravity component vector signal and the time delay-added gravity component vector signal;
It is determined whether or not the predetermined operation has occurred by performing template matching on the temporal change of the calculated inner product value over a predetermined time using a template indicating the temporal change of the inner product value in a predetermined operation. A determining step;
An information processing method comprising: Detecting acceleration in a three-dimensional space;
Extracting a gravity component vector signal from the acceleration;
Providing a time delay to the signal of the gravity component vector;
Calculating an inner product value of the extracted gravity component vector signal and the time delay-added gravity component vector signal;
It is determined whether or not the predetermined operation has occurred by performing template matching on the temporal change of the calculated inner product value over a predetermined time using a template indicating the temporal change of the inner product value in a predetermined operation. A determining step;
A program that causes a computer to execute.

Images