KR102635657B1 - Segway Robot Based on Reinforcement Learning and Its Method for Motion Control - Google Patents

Abstract

One embodiment of the present invention is a reinforcement learning-based robot driving state control system, which collects driving data including inertial data measured by the IMU sensor of the robot and rotation data of the motor measured by the encoder of the robot. Driving data collection unit; A control logic operation unit that calculates a robot state value using the driving data obtained from the driving information collection unit and performs reinforcement learning by comparing it with a target robot state value; A robot driving state control system can be provided, including a robot driving control unit that generates a robot control signal that controls the driving conditions of the robot based on the robot state value obtained from the control logic calculation unit.

Description

Segway robot based on reinforcement learning and its motion control method {Segway Robot Based on Reinforcement Learning and Its Method for Motion Control}

This embodiment relates to a reinforcement learning-based Segway-type robot and its motion control method. More specifically, it relates to a motion control method for self-balancing and autonomous driving of a Segway-type robot applying deep learning reinforcement learning.

Segway is a means of transport that uses an electric motor to transport users, and is attracting attention as a means of transport to replace the internal combustion engine as the next-generation mobility.

Unlike conventional four-wheel drive vehicles, Segway uses one or two wheels, and the driver's safety is greatly affected by changes in the mobility driving environment. For example, when driving conditions change due to a change in the center of gravity or wheel spin, there is a need for Segway to learn on its own to search for optimal driving conditions.

In particular, since there is a difference between the driving conditions when slippage (spin) of the Segway's wheels occurs and the driving conditions under normal circumstances, it is necessary to appropriately reflect these driving characteristics.

In the driving process of conventional Segway-type robots, changes in physical characteristics of the robot (for example, changes in mass, speed, acceleration, etc.) cannot be sensed in real time, so the data that is the basis for the robot's self-balancing and autonomous driving is used. There is a problem with collecting and updating.

In addition, conventional Segway-type robots such as KR 10-1816136 B1 do not comprehensively consider control parameters such as mass (m) and rotational inertia (I), and have limitations in designing sophisticated learning models.

Against this background, the purpose of this embodiment is, in one aspect, to apply a reinforcement learning algorithm that reflects parameters related to the mass and rotation of a Segway-type robot and to control the robot's motion by reflecting real-time driving situations. The purpose is to provide a robot and its motion control method.

The purpose of this embodiment is to provide a Segway-type robot and a method of controlling its motion that can perform learning and the driving state of the robot by reflecting whether the wheels of the Segway-type robot slip (spin) or not.

In another aspect, the purpose of this embodiment is to provide a Segway-type robot capable of self-balancing and autonomous driving and a method for controlling its motion by applying a deep learning algorithm that predicts the driving state of the robot.

In order to achieve the above-described object, an embodiment of the present invention provides a reinforcement learning-based robot driving state control system, inertial data measured by the IMU sensor of the robot and rotation data of the motor measured by the encoder of the robot. A driving data collection unit that collects driving data including; A control logic operation unit that calculates a robot state value using the driving data obtained from the driving information collection unit and performs reinforcement learning by comparing it with a target robot state value; A robot driving state control system can be provided, including a robot driving control unit that generates a robot control signal that controls the driving conditions of the robot based on the robot state value obtained from the control logic calculation unit.

In the robot driving state control system, the robot is a Segway, the inertial data includes angular velocity and acceleration values for the x-axis, y-axis, and z-axis of the robot, and the rotation data is a rotation angle value for each wheel of the robot. It includes, the posture of the robot can be recognized based on the inertial data, and the movement of the wheels can be recognized based on the rotation data.

In the robot driving state control system, the control logic calculation unit obtains the robot state value by calculating the robot's mass value (m) and rotational inertia value (I) by setting the linear motion acceleration value (a) of the robot as an input value. can do.

In the robot driving state control system, the reinforcement learning calculates the difference between the acquired robot state value and the preset target robot state value, and ends learning when the difference value is recognized as 0 or within the standard range, and the difference If the value is outside the standard range, the robot status value can be updated.

In the robot driving state control system, the reinforcement learning can be performed by a simulation program, or by receiving a robot state estimate value transmitted from a user terminal.

In the robot driving state control system, the robot state value may include one or more of the robot's mass, rotational inertia, tilt, tilt change amount, speed, acceleration, wheel torque, and wheel angular velocity.

In the robot driving state control system, the control logic operation unit estimates the robot state value by acquiring linear motion speed data through the sum of the speeds of the left and right wheels of the robot and rotating motion speed data through the speed difference between the left and right wheels of the robot. can do.

The robot driving state control system may further include a robot driving control unit that controls the operation of the motor by transmitting a robot control signal corresponding to the robot state value calculated by the control logic operation unit to a motor driver.

In order to achieve the above-described object, another embodiment of the present invention provides a method for determining and controlling the driving state of a Segway, comprising the steps of collecting driving data of the Segway; determining whether the wheel spins based on the driving data; Performing control logic to calculate a Segway state value and compare it with a target Segway state value when the wheel does not spin; A Segway driving state control method can be provided, including the step of controlling the driving of the motor to correspond to the Segway state value.

In the Segway driving state control method, the Segway includes sensors that collect driving data, and the driving data includes inertia data including angular velocity and acceleration values of the Segway and rotation data including rotation angle values for each wheel of the Segway. It can be included.

In the Segway driving state control method, the Segway includes an encoder that measures rotation angle displacement of the wheel; And it may further include a processor that determines the rotation of the wheel by comparing the wheel rotation speed measured by the encoder with the target wheel rotation speed.

In the Segway driving state control method, whether the wheels spin is determined by comparing the straight-line movement distance and the wheel rotation distance, and if the wheel rotation distance is greater than the straight-line movement distance, it can be determined that spin exists.

In the Segway driving state control method, the Segway includes a hub motor that provides driving force for the wheels; And it may further include a processor that obtains acceleration data by applying the output data of the hub motor to a preset algorithm.

The Segway driving state control method may further include performing a control algorithm that determines whether a disturbance occurs due to a change in mass (m) or rotational inertia (I) that occurs during the driving process of the Segway.

In the Segway driving state control method, if it is determined that there is a difference between the Segway state value and the target Segway state value, reinforcement learning can be performed to iterate the Segway state value to converge to the target Segway state value.

In the Segway driving state control method, the driving of the wheels can be individually controlled by transmitting a motor control signal corresponding to the Segway state value to a motor driver.

As described above, according to an embodiment of the present invention, the operation of the robot can be controlled by reflecting the real-time driving situation by applying a reinforcement learning algorithm that reflects parameters related to the mass and rotation of the Segway-type robot.

According to an embodiment of the present invention, the driving state of the Segway-type robot can be controlled by reflecting whether the wheels of the Segway type robot slip (spin) and changes in the driving state.

According to an embodiment of the present invention, the amount of calculation of the processor for self-balancing and autonomous driving can be reduced by extracting and learning some parameters based on the modeling parameters of the Segway-type robot, and reinforcement learning for self-balancing and autonomous driving can improve the speed.

Figure 1 is a diagram showing a data communication method between Segway and a processor according to an embodiment of the present invention.
Figure 2 is a diagram explaining the operation sequence of a processor according to an embodiment of the present invention.
Figure 3 is a diagram explaining the operation control sequence of Segway according to an embodiment of the present invention.
Figure 4 is a configuration diagram of a Segway according to an embodiment of the present invention.
Figure 5 is a diagram explaining modeling parameters of Segway according to an embodiment of the present invention.
Figure 6 is a diagram illustrating a method of calculating modeling parameters of a Segway according to an embodiment of the present invention.
Figure 7 is a diagram illustrating a reinforcement learning method according to an embodiment of the present invention.
Figure 8 is a flowchart explaining a method of generating a control signal for a Segway according to an embodiment of the present invention.

Hereinafter, some embodiments of the present invention will be described in detail through illustrative drawings. When adding reference numerals to components in each drawing, it should be noted that the same components are given the same reference numerals as much as possible even if they are shown in different drawings. Additionally, in describing the present invention, detailed descriptions of related known configurations or functions that are judged to be likely to obscure the gist of the present invention will be omitted.

Additionally, in describing the components of the present invention, terms such as first, second, a, and b may be used. These terms are only used to distinguish the component from other components, and the nature, order, or order of the component is not limited by the term. When a component is described as being “connected,” “coupled,” or “connected” to another component, that component may be directly connected or connected to that other component, but there is another component between each component. It will be understood that elements may be “connected,” “combined,” or “connected.”

Additionally, the terms 'system', 'server', etc. used in this specification may be defined as a computer program or device that transmits and receives information to a client over a network for data storage and calculation, but is not limited thereto.

In addition, the terms 'Segway', 'Segway-type robot', and 'robot' used in this specification can be defined as a device that provides driving force to transport a user, and each term can be used interchangeably as needed. .

Figure 1 is a diagram showing a data communication method between Segway and a processor according to an embodiment of the present invention.

Referring to FIG. 1, the Segway 10 and the processor 20 according to one embodiment can control the operation of the Segway through data communication.

Segway 10 may be a robot for user transportation that includes one or more wheels. For example, the Segway 10 includes two wheels and a motor, and may include a motor driver for controlling the motor of each wheel, and transmits and receives control signals from the motor driver according to the calculation results of the processor 20. can do.

The Segway 10 may further include an IMU sensor (not shown) capable of measuring inertial data and an encoder (not shown) capable of measuring rotation data of the motor.

The IMU sensor (not shown) is an inertial measurement unit (IMU) that collects inertial data such as angular velocity and acceleration values for the three axes of the robot - for example, the x-axis, y-axis, and z-axis. It can be measured. Additionally, the IMU sensor can measure the robot's tilt and posture in each direction by combining a 3-axis acceleration and 3-axis gyro sensor. For example, an IMU sensor can obtain measured values by applying a Kalman filter, but is not limited to this.

The encoder (not shown) can measure rotation data such as the number of revolutions, rotation speed, rotation angle, and direction of rotation of the motor. Additionally, the encoder can measure the robot's wheel movements based on rotation data. For example, the encoder may adopt various measurement methods such as optical or magnetic encoders, but is not limited thereto.

The processor 20 may be a device for calculation, such as a server separate from the Segway 10, but is included in the Segway 10 and includes a program that performs calculations to control the operation of the Segway 10. It may be a device that does this.

The Segway 10 can transmit information about the driving state to the processor 20 (S101), and the processor 20 receives and learns information about the driving state, etc. and controls the driving state of the Segway 10. The control signal can be transmitted to the Segway 10 (S102).

The processor 20 may be implemented as a physical computing device, but is not limited to this and may be implemented to receive information about the driving state of the Segway 10, etc. from a cloud server, etc., and perform calculations.

Figure 2 is a diagram explaining the operation sequence of a processor according to an embodiment of the present invention.

Referring to FIG. 2, the processor 20 may include a driving data collection unit 21, a control logic operation unit 22, a robot driving control unit 23, etc. Blocks that implement each operation of the processor 20 may be physically or conceptually separate blocks.

The driving data collection unit 21 can receive driving data of the Segway from each sensor and generate a robot control signal to control the robot.

Segway's driving data may include inertial data measured by an IMU sensor and motor rotation data measured by an encoder.

The control logic operation unit 22 can perform reinforcement learning by calculating the robot state value using the driving data obtained from the driving information collection unit and comparing it with the target robot state value.

The robot state value may be a parameter representing the running state of the robot, such as the robot's mass, rotational inertia, tilt, tilt change amount, speed, acceleration, wheel torque, and wheel angular velocity.

For example, the control logic calculation unit 22 sets the robot's linear motion acceleration value () as an input value and calculates the robot's mass value (m) and rotational inertia value (I) to obtain the robot state value of the robot. You can.

The control logic operation unit 22 can obtain the robot state value through a preset algorithm, and, if necessary, perform machine learning - for example, reinforcement learning - to obtain the target robot state value. .

Reinforcement learning performed by the control logic operation unit 22 calculates the difference between the acquired robot state value and the preset target robot state value, and learning can be terminated when the difference value is recognized as 0 or within the standard range. , If the difference value is outside the standard range, the robot status value can be updated repeatedly.

Reinforcement learning performed by the control logic operation unit 22 may be performed by a simulation program or by receiving a robot state estimate value transmitted from a user terminal. In cases where reinforcement learning does not reach the convergence value, or by performing learning based on the input data of the driving state value transmitted from the user terminal to reflect the driving conditions desired by the user, the offset of the robot's driving state is adjusted or the user Convenience can be improved.

The control logic operation unit 22 estimates the robot state value by acquiring linear motion speed data through the sum of the speeds of the left and right wheels of the robot, or obtains rotational motion speed data through the speed difference between the left and right wheels of the robot to estimate the robot state value. It can be estimated. In this case, parameters related to the driving state - for example, the robot linear motion speed and the robot rotational angular velocity in FIG. 6 - can be calculated using a preset calculation formula.

The robot driving control unit 24 can control the driving conditions of the robot based on the robot state value obtained from the control logic calculation unit. The control logic operation unit 22 updates the robot state value acquired through repeated learning, and based on this, the robot control signal for controlling components such as the robot's motor can be changed in real time.

The robot driving control unit 24 can transmit a robot control signal to control the robot only when it is determined that the current robot state value and the target robot state value match or are within a standard error range.

The robot driving control unit 24 may control the operation of the motor by transmitting a robot control signal corresponding to the robot state value calculated by the control logic operation unit 22 to the motor driver.

Figure 3 is a diagram explaining the operation control sequence of Segway according to an embodiment of the present invention.

Referring to FIG. 3, the operation of the Segway 10 can be controlled by receiving the operation result of the processor 20.

Segway 10 may include an IMU sensor 11, a motor encoder 12, a motor driver 13, a left motor 14, a right motor 15, etc.

The IMU sensor 11 may be a device that measures the rotational angular velocity values for the three axes of the robot and the acceleration values in each axis direction, and the motor encoder 12 may measure the rotation angle relative to the angular reference point of the Segway wheels, etc. It may be a device that measures .

Segway 10 may include a left motor 14 corresponding to the left and right wheels, a right motor 15, and a motor driver 13 that controls each motor.

The motor driver 13 can receive a robot control signal from the processor 20 and control driving conditions such as rotation speed, rotation speed, rotation acceleration, and rotation angle of the left and right motors 14 and 15.

The processor 20 includes an inertial data collection unit 21-1, a rotation data collection unit 21-2, a control logic operation unit 22, a robot state value estimation unit 23, and a robot driving control unit 24. can do.

The inertial data collection unit 21-1 can collect the inertial data measured by the IMU sensor 11 and determine the robot's posture, such as tilt.

The rotation data collection unit 21-2 can collect rotation data measured by the motor encoder 12 and determine the movement of the wheel.

The control logic calculation unit 22 can collect driving data such as inertial data and rotation data and calculate a driving state value related to the driving state of the robot. In addition, the control logic operation unit 22 can update the driving state value through time-series data learning to generate a driving state value for the robot driving control unit 24 to determine the robot's operation.

The control logic operation unit 22 can learn the input driving data according to a preset learning algorithm, and can further determine whether to continue learning by determining the accuracy of the learning.

The control logic operation unit 22 can determine whether to end learning by comparing the target driving state value, which is data about the driving state of the robot desired by the user, with the driving state value, which is data about the current driving state of the robot.

The robot state value estimation unit 23 can reduce the amount of learning by determining the learning order by considering the priorities of the parameters and the correlation between the parameters without performing calculations on all parameters. For example, as shown in Figure 7, the acceleration obtained by the robot's IMU sensor and the target acceleration can be compared first, and the angular acceleration obtained by the motor encoder and the target angular acceleration can be compared next.

If the parameter estimation of the robot state value estimation unit 23 is not completed or it is determined that the estimated parameters are different from the target value, the result value can be transmitted to the control logic calculation unit 22 to perform learning again.

The robot state value estimation unit 23 may be included in a lower block of the control logic operation unit 22, but if necessary, it may be defined as a separate block to perform sequential data operations and updates.

The robot travel control unit 24 may generate a travel control signal based on the robot state value determined and transmitted by the control logic operation unit 22 and transmit it to the motor driver 13. Since the robot travel control unit 24 generates a travel control signal based on the calculation results of the control logic calculation unit 22, more precise travel control is possible.

Here, the configuration of the Segway 10 and the processor 20 is divided to explain the process of collecting and calculating data and transmitting control signals, and is all or part of the configuration included in one robot, and the technical idea of the present invention is It is not limited to Figure 3.

Figure 4 is a configuration diagram of a Segway according to an embodiment of the present invention.

Referring to FIG. 4, the Segway 100 may include a left wheel 101, a right wheel 102, a main body 103, a handle 104, etc.

Depending on the rotation state of the wheels 101 and 102 of the Segway 100, the linear and rotational movements of the Segway 100 can be determined by dynamically modeling them.

For example, if the center of mass of the Segway 100 is located adjacent to the main body 103, the movement of the Segway 100 can be replaced by the movement of the main body 103, or the position of the center of gravity can be defined and used in calculations. You can.

For example, if the rotation distance of the wheels 101 and 102 is the same as the straight-line movement distance of the Segway 100, it is determined that the wheel slippage (spin) phenomenon has not occurred, and the rotation data of the wheels and the straight-line movement of the Segway are determined. Correlation of distance data can be defined.

In addition, if the current revolutions per minute (RPM) of the wheels 101 and 102 are the same as the target revolutions per minute (RPM), it is determined that the slippage (spin) phenomenon of the wheels has not occurred, and the rotation data of the wheels and the Segway Correlation of straight line travel distance data can be defined.

The speed in the direction of linear movement or the speed in the direction of rotation of the Segway 100 can be defined by using the sum or difference in speed of the left and right sides of the wheels 101 and 102.

The handle portion 104 or the main body portion 103 may be used to determine the tilt of the robot. For example, the tilt of the pillar of the handle portion 104 can be defined as the tilt of the robot, and the tilt of one plane of the main body portion 103 can be defined as the tilt of the robot.

Figure 4 is for explaining a simplification method and modeling method of each component of the Segway 100, and the technical idea of the present invention is not limited thereto.

Figure 5 is a diagram explaining modeling parameters of Segway according to an embodiment of the present invention.

Referring to FIG. 5, the configuration of the Segway 100 can be simplified and modeled.

Speed of linear movement of Segway (100) ( ), acceleration ( ) can be defined, and the rotation angle of rotational movement ( ), rotational angular speed ( ) can be defined as exercise data.

The rotation angle of the left wheel 101 of the Segway 100 ( ), rotational angular speed ( ), rotational inertia ( ), angular acceleration ( ) etc. can be defined as rotation data, and the rotation angle of the right wheel 102 ( ), rotational angular speed ( ), rotational inertia ( ), angular acceleration ( ), etc. can be defined as rotation data.

The rotation angle in the tilt direction of the Segway 100 ( ), angular velocity ( ), talk( ), angular acceleration ( ), etc. can be defined as slope data.

In addition, the total mass of Segway 100 ( ), total rotational inertia ( ), total angular momentum ( ), the distance from one plane of the main body 103 to the center of mass (not shown), etc. can be defined and used in the calculation of each parameter.

In order to obtain the state driving value according to one embodiment, the slope parameter of the Segway 100 may be a solution obtained by substituting the equation of motion of a rotary inverted pendulum, and the solution of the nonlinear equation may be obtained through learning such as iterative calculation. can be obtained.

Figure 6 is a diagram illustrating a method of calculating modeling parameters of a Segway according to an embodiment of the present invention.

Referring to FIG. 6, estimated values regarding the driving state of the robot can be calculated using the Segway modeling parameters of FIG. 5.

The linear motion speed of a Segway can be derived using the sum of the speeds of the left and right wheels.

For example, the robot's linear motion speed ( ) is the radius of the wheel ( ) and the average rotation speed of each wheel ( ) can be defined as the product of the linear motion acceleration of the robot ( ) can be calculated by differentiating the linear motion speed of the robot with time.

The angular speed of Segway's rotational motion can be derived using the speed difference between the left and right wheels.

For example, the rotational movement angular velocity of the robot ( ) is the deviation of the rotation speed of each wheel ( ) to the radius of the wheel ( ), then multiply it by the distance between the wheels ( ) can be defined as the divided value, and the robot's rotation angle can be calculated by integrating the robot's rotational motion angular speed with time.

Figure 6 is for explaining a method of calculating parameters related to the driving state of the Segway 100, and the technical idea of the present invention is not limited thereto.

Figure 7 is a diagram illustrating a reinforcement learning method according to an embodiment of the present invention.

Referring to FIG. 7, the method 200 of reinforcement learning and robot motion control according to an embodiment includes collecting sensor data (S201), calculating the acceleration of the robot (S202), current acceleration and target acceleration. Comparing (S203), calculating the angular velocity of the wheels (S204), comparing the current angular velocity and the target angular velocity (S205), calculating the mass of the robot (S206), calculating the angular acceleration of the robot. (S207), comparing the current angular acceleration and target angular acceleration (S208), calculating angular momentum and rotational inertia (S209), calculating control parameters (S210), controlling the robot's motion (S211), etc. may include.

The step of collecting sensor data (S201) may be a step of measuring and collecting various types of sensing values in the form of data by sensors of the robot.

The step of calculating the acceleration of the robot (S202) may be a step of determining the measured value of the IMU sensor as acceleration.

The step of comparing the current acceleration and the target acceleration (S203) may be a step of comparing the calculated acceleration of the robot in the current state and the target acceleration. If the current acceleration and the target acceleration are the same or within a preset reference range, the angular acceleration calculation step (S207) can be performed immediately. If the current acceleration and the target acceleration are not the same or are outside a preset reference range, a step (S204) of calculating the angular velocity can be performed.

The step of calculating the angular velocity of the wheel (S204) may be a step of calculating the angular velocity of the left and right wheels/motor, and the measured value of the motor encoder can be used to calculate the angular velocity.

The step of comparing the current angular velocity and the target angular velocity (S205) may be a step of comparing the calculated angular velocity of the robot in the current state and the target angular velocity. If the current angular velocity and the target angular velocity are the same or within a preset reference range, the step of calculating the total mass of the robot (S206) can be performed immediately. If the current angular velocity and the target angular velocity are not the same or are outside a preset reference range, a step (S210) of calculating control parameters may be performed.

The step of calculating the mass of the robot (S206) may be a step of calculating the total mass of the robot based on equations related to classical mechanics such as Newton's laws of motion.

The step of calculating the angular acceleration of the robot (S207) may be a step of acquiring and calculating the angular acceleration measurement value on the IMU sensor.

The step of comparing the current angular acceleration and the target angular acceleration (S208) may be a step of comparing the calculated angular acceleration of the robot in the current state and the target angular acceleration. If the current angular acceleration and the target angular acceleration are the same or within a preset reference range, the robot's control parameter calculation (S210) can be performed immediately. If the current angular acceleration and the target angular acceleration are not the same or are outside a preset reference range, a step (S209) of calculating angular momentum and rotational inertia can be performed.

The step of calculating angular momentum and rotational inertia (S209) may be a step of calculating the angular momentum and rotational inertia of the robot based on equations related to classical mechanics such as Newton's laws of motion. Calculate the torque value from the rotary inverted pendulum equation of motion, and calculate the rotational inertia ( ) can be calculated. Angular momentum ( ) can be calculated using the rotational inertia formula and the parallel axis theorem.

The step of calculating control parameters (S210) may be a step of determining parameters obtained through the above-described logic and deciding whether to end learning.

The step of controlling the operation of the robot (S211) may be a step of controlling the operation of the robot by generating and transmitting a robot control signal corresponding to the robot state value obtained based on the determined parameters.

Figure 8 is a flowchart explaining a method of generating a control signal for a Segway according to an embodiment of the present invention.

Referring to FIG. 8, the method 300 of determining and controlling the driving state of the Segway includes collecting driving data of the Segway (S301), determining whether the wheels spin based on the driving data (S302), and determining whether the Segway is spinning. It may include a step of determining whether a disturbance occurs during the driving process of the Segway (S303), a step of performing Segway control logic (S304), and a step of controlling the driving of the motor by generating a motor control signal of the Segway (S305). You can.

The step of collecting driving data of the Segway (S301) may be a step of collecting data generated during the driving process of the Segway. Data collected through sensors can be calculated in the form of parameters to define and control the driving state of the Segway.

Driving data may include various data sets, such as inertia data including angular velocity and acceleration values of the Segway, and rotation data including rotation angle values for each wheel of the Segway. For example, if the motor that provides driving force to the wheels of a Segway is a hub motor, data such as the rotational speed and rotational acceleration of the motor can be obtained based on the hub axis. Additionally, separate converted data can be obtained by applying the motor's output data to a preset algorithm.

The step of determining whether the wheel spins based on the driving data (S302) may be a step of determining whether the Segway wheel spins based on the obtained driving data. If the spin of the Segway wheel occurs, the consistency or correlation of the obtained driving data becomes inaccurate, so to simplify and improve the accuracy of driving modeling, it is possible to first determine whether the Segway wheel spins.

Only when it is determined that spin of the Segway wheel does not exist, it is possible to proceed to the step of calculating parameters and calculating the Segway driving state value.

The Segway according to one embodiment may further include an encoder that measures the rotation angle displacement of the wheel, and a processor that determines the rotation of the wheel by comparing the wheel rotation speed measured by the encoder with the target wheel rotation speed.

According to another embodiment, whether the Segway spins is determined by comparing the straight-line movement distance of the Segway and the rotation distance of the wheels, and if the rotation distance of the wheels is greater than the straight-line movement distance, it can be determined that spin exists.

The step of determining whether a disturbance occurs during the driving process of the Segway (S303) is a step of determining whether a disturbance occurs by monitoring data changes such as mass (m) or rotational inertia (I) that occur during the driving process of the Segway. It can be. In this process, a preset control algorithm can be performed, or a change value that exceeds the standard value can be judged to be a disturbance.

Additionally, in the process of determining whether a disturbance has occurred, the weight of the rider can be excluded and the disturbance can be judged based on the weight of the Segway itself.

The step of performing the Segway control logic (S304) may be a step of performing data learning by applying control logic that calculates the Segway state value and compares it with the target Segway state value when the wheel spin does not occur.

If necessary, a step of performing dynamic modeling of the Segway and extracting parameters required for modeling may be performed in advance before performing the Segway control logic. Since the degree of computational freedom increases depending on the number of parameters used in the learning algorithm, the number of parameters can be limited to a number within a preset range.

The control logic may be a step of comparing the current Segway status value with the target Segway status value corresponding to the user's input target value and calculating parameters according to a set order.

The control logic can repeatedly update parameters to improve data accuracy, and repeat learning until a preset target deviation or target ratio is reached.

Additionally, if it is determined that there is a difference between the Segway state value and the target Segway state value, the control logic can perform reinforcement learning to converge to the target Segway state value by repeatedly calculating the Segway state value using an iteration method. For example, a cyclical iteration can be performed in which all or part of the obtained output values are used as input data.

The step of controlling the driving of the motor by generating the Segway motor control signal (S305) may be a step of controlling the driving of the motor simultaneously or at different times according to the Segway motor control signal. The driving of the motor can be controlled independently depending on the motion state of the Segway - for example, linear motion or rotational motion.

Through the above method, it is possible to perform a self-balancing function that maintains constant parameters such as direction and speed required during the Segway's operation process, and an autonomous driving function that controls Segway operation by autonomously updating parameters without continuous operation by the rider. can be implemented.

The Segway driving state control method 300 according to the technical idea of the present invention is not limited to the sequence and steps of FIG. 8, and some of the sequence of each step may be omitted or the sequence of each step may be changed.

Claims (16)

In the reinforcement learning-based Segway type robot driving state control system,
a driving data collection unit that collects driving data including inertial data measured by the IMU sensor of the robot and rotation data of the motor measured by the encoder of the robot;
A control logic operation unit that performs reinforcement learning to calculate a robot state value using the driving data obtained from the driving data collection unit and compares it with a target robot state value; and
A robot driving state control system comprising a robot driving control unit that generates a robot control signal that controls driving conditions of the robot based on the robot state value obtained from the control logic calculation unit. According to paragraph 1,
The robot is a Segway, the inertial data includes angular velocity and acceleration values for the x-axis, y-axis, and z-axis of the robot, and the rotation data includes rotation angle values for each wheel of the robot,
A robot driving state control system that recognizes the posture of the robot based on the inertial data and recognizes wheel movement based on the rotation data. According to paragraph 2,
The control logic calculation unit takes the linear motion acceleration value (a) of the robot as an input value and calculates the mass value (m) and rotational inertia value (I) of the robot to obtain the robot state value, controlling the robot driving state. system. According to paragraph 1,
The reinforcement learning calculates the difference between the acquired robot state value and the preset target robot state value, and ends learning when the difference value is recognized as 0 or within the standard range. If the difference value is outside the standard range, the reinforcement learning ends. A robot driving state control system that updates the robot state value in this case. According to paragraph 1,
A robot driving state control system in which the reinforcement learning is performed by a simulation program or by receiving a robot state estimate value transmitted by a user terminal. According to paragraph 1,
The robot state value includes one or more of the robot's mass, rotational inertia, tilt, tilt change amount, speed, acceleration, wheel torque, and wheel angular velocity. According to paragraph 1,
The control logic operation unit obtains linear motion speed data through the sum of the speeds of the left and right wheels of the robot, and obtains rotational movement speed data through the speed difference between the left and right wheels of the robot to estimate the robot state value. Robot driving state control. system. According to paragraph 1,
A robot driving state control system further comprising a robot driving control unit that controls the operation of the motor by transmitting a robot control signal corresponding to the robot state value calculated by the control logic operation unit to a motor driver. In the method of determining and controlling the driving state of a Segway,
Collecting driving data of Segway;
determining whether the wheel spins based on the driving data;
Performing control logic to calculate a Segway state value and compare it with a target Segway state value when the wheel does not spin; and
A Segway driving state control method comprising controlling the driving of the motor to correspond to the Segway state value. According to clause 9,
The Segway includes sensors that collect driving data,
The driving data includes inertia data including angular velocity and acceleration values of the Segway, and rotation data including rotation angle values for each wheel of the Segway. According to clause 9,
The Segway includes an encoder that measures rotation angle displacement of the wheel; and
Segway driving state control method further comprising a processor that determines the rotation of the wheel by comparing the wheel rotation speed measured by the encoder with the target wheel rotation speed. According to clause 9,
A Segway driving state control method wherein whether the wheel spins is determined by comparing the straight-line movement distance and the wheel rotation distance, and when the wheel rotation distance is greater than the straight-line movement distance, it is determined that spin exists. According to clause 9,
The Segway includes a hub motor that provides driving force for the wheels; and
Segway driving state control method further comprising a processor that obtains acceleration data by applying the output data of the hub motor to a preset algorithm. According to clause 9,
A Segway driving state control method further comprising the step of performing a control algorithm that determines whether a disturbance occurs based on a change in mass (m) or rotational inertia (I) that occurs during the driving process of the Segway. According to clause 9,
When it is determined that there is a difference between the Segway state value and the target Segway state value, a Segway driving state control method that performs reinforcement learning to iterate the Segway state value to converge to the target Segway state value. According to clause 9,
A Segway driving state control method that individually controls the driving of the wheels by transmitting a motor control signal corresponding to the Segway state value to a motor driver.