KR102730196B1 - Music feedback system using the removal or addition of music instrument for the purposed movement - Google Patents

Abstract

A music feedback system utilizing instrument exclusion or addition for inducing movement includes a camera configured to capture a user's body and body movements; a wearable sensor attached to a body segment location of the user and configured to detect movement of a specific body part; and a control unit operably coupled with the camera and the wearable sensor and configured to play music that induces movement of the body part, wherein the control unit can compare a reference movement with real-time movement to determine whether to exclude or add a specific instrument from music selected by the user.

Description

{Music feedback system using the removal or addition of music instrument for the purposed movement}

The present invention relates to a medical device, and more specifically, to a music feedback system for inducing movement that can be utilized in exercise-related businesses/culture and sports.

Medical devices and fitness equipment that induce correct body movements can use feedback to suppress incorrect movements. Feedback on real-time body movements is effective in inducing correct movements. Most feedback consists of monotonous audio signals.

Monotonous feedback can cause boredom in users and be a factor that hinders continuous exercise. It is necessary to add entertainment elements beyond the original purpose of feedback. Prior patent document 1 relates to a real-time user-participatory music generation method and device therefor (Publication number: 10-2021-0152847 (2021.12.16.)). Prior patent document 1 is a device that generates music using real-time body movements. It may be somewhat similar in that it controls music through body movements. However, the present invention is different in that it uses music as feedback by judging the correctness of real-time body movements. In addition, the operating algorithm is different in that it removes instruments from existing music rather than changing the modulation or performance method of music.

Prior patent document 2 relates to a music modulation auditory feedback system using wireless earphones and a smartphone (Registration number: 10-2301575 (2021.09.07)). Prior patent document 2 is a device that maintains body symmetry with real-time auditory feedback. Prior patent document 2 measures the body inclination with a sensor attached to the earphone and provides feedback, but the present invention estimates the positions of all body segments using a camera and provides feedback. In addition, the auditory feedback method is different. The present invention provides feedback by removing or adding instruments to music, but the existing study is different in that it uses a method of selecting a different music type.

Method and device for generating real-time user-participatory music (Publication number: 10-2021-0152847 (2021.12.16.)) Music Modulation Auditory Feedback System Using Wireless Earphones and Smartphones (Registration Number: 10-2301575 (2021.09.07))

An object of the present invention is to provide a music feedback system utilizing the exclusion or addition of instruments to induce movement.

In addition, an object of the present invention is to induce correct movements of a user by removing or adding instruments to music according to body movements recognized by a wearable sensor (IMU) or a camera linked to a smart phone.

Furthermore, it is an object of the present invention to provide greater motivation to users who aim to improve their fitness, sports and dance performance.

A music feedback system utilizing instrument exclusion or addition for movement induction according to an embodiment of the present invention comprises: a camera configured to photograph a user's body and body movements; a wearable sensor attached to a body segment location of the user and configured to detect movement of a specific body part; and a control unit operably coupled with the camera and the wearable sensor and configured to play music that induces movement of the body part, wherein the control unit can compare a reference movement with real-time movement to determine whether to exclude or add a specific instrument from music selected by the user.

According to one embodiment, if negative feedback is selected, the control unit can control the instrument that was excluded to be added back to the music when the user's body segment position returns to the reference position, and if positive feedback is selected, the control unit can control the instrument that was added to be excluded from the music when the user's body segment position moves away from the reference position.

According to one embodiment, the control unit extracts sound sources using MFCC (Mel-Frequency Cepstral Coefficient), classifies the extracted sound sources and musical instruments, and then controls the instruments to be added to or excluded from the music based on the classification, and adjusts the volume and beats of the sound sources of the music for each specific time section so that the sound sources of the music divided for each specific time section are similar to the extracted sound sources within a predetermined range.

According to one embodiment, the control unit may control to exclude or add a first instrument from the music when the user's knee position deviates from the reference knee position by a specific value or more in the compound joint motion, and may control to exclude or add a second instrument from the music when the x-axis position of the user's knee deviates from the x-axis position of the reference knee by a specific value or more in the single joint motion.

According to one embodiment, the control unit can control instruments assigned to each body joint to be added to or excluded from the music according to the difference between the current position of each body joint and the reference position when the compound joint movement is performed.

According to one embodiment, the control unit controls the music to be played in a state where instruments are designated for each body part, and when exercising while listening to the music, if the knee moves out of a position indicated by the device during the exercise motion, the first instrument is removed from the music and played, and if the waist moves out of the position indicated by the device, the control unit controls the music to be removed and played, and if the position of the body segment returns to the position indicated by the device, at least one of the first instrument and the second instrument that was removed is added again and the music is played.

According to one embodiment, the control unit may control the addition of an instrument assigned to each joint through comparison between a position estimated in real time and a reference position, control music so that only a first instrument is played when a squat motion is first performed, control the music so that a second instrument is added when a knee position is similar to a reference position within a predetermined range, control the music so that a third instrument is added when a pelvis position is similar to a reference position within a predetermined range, and control the music so that all instruments are added when all body positions are similar to a reference motion within a predetermined range.

According to one embodiment, the control unit estimates a three-dimensional body position using body segment trajectory, angle, and acceleration data or images input to a smart phone, and compares the positions of each body joint estimated in real time with a reference position to calculate the difference between the positions of each joint and the reference position for each dimension.

According to one embodiment, the control unit can control the first instrument to be excluded from the music when a difference in the x-axis positions of each body joint occurs, and can control the second instrument to be excluded from the music when a difference in the y-axis positions of each body joint occurs.

According to one embodiment, the control unit controls the music so that only the first instrument is played when the specific motion of each joint of the body is first performed, controls so that a second instrument is added when the difference in the x-axis positions is less than a predetermined range when the specific motion of each joint of the body is performed, controls so that a third instrument is added when the difference in the y-axis positions is less than a predetermined range, controls so that a fourth instrument is added when the difference in the z-axis positions is less than a predetermined range, and controls so that music with all instruments added is played when the positions of the specific motion of each joint of the body are similar to the reference position within a predetermined range on all axes.

According to one embodiment, the control unit may control to select settings related to motions of each joint of the user's body, music, instruments, feedback types, and feedback intensities, compare each dimensional position of each joint of the body with each dimensional position of a reference motion, and control to add or exclude at least one of the instruments from the music, control to play the music from a speaker or earphone that is located in a direction in which each joint of the body is determined to be inconsistent with a specific motion among a plurality of speakers and earphones that support spatial audio in a space where the user is located, and control to adjust the volume and beat of the music based on the feedback intensities.

According to an embodiment of the present invention, body movements during exercise are detected in real time, and feedback is given to the user based on the detection results. Through the feedback, the user recognizes incorrect movements and puts more effort into achieving correct movements.

According to an embodiment of the present invention, target muscles can be efficiently activated by inducing correct movements, and movements that reduce pain can be learned.

According to an embodiment of the present invention, by using music to break away from a monotonous feedback format, user boredom can be reduced and interest can be increased. This can increase the user's exercise duration.

According to an embodiment of the present invention, the musculoskeletal condition of a user can be diagnosed based on whether or not the user can perform correct movements.

According to an embodiment of the present invention, the use of the medical device and fitness equipment can enhance understanding of correct movements.

Most medical devices and fitness equipment that use feedback to induce correct body movements induce single-joint movements. According to an embodiment of the present invention, compared to single-joint movements, complex joint movements are effective in improving body coordination and have high exercise efficiency. It is expected that inducing correct complex joint movements will have a greater effect on the user's return to daily life and injury prevention.

Further scope of applicability of the present invention will become apparent from the detailed description below. However, since various changes and modifications within the spirit and scope of the present invention will become apparent to those skilled in the art, it should be understood that the detailed description and specific embodiments, such as preferred embodiments of the present invention, are given by way of example only.

Figure 1 is a drawing illustrating a single joint motion according to the present invention.
FIG. 2 is a drawing illustrating a composite joint movement according to the present invention.
FIG. 3 is a diagram showing the configuration of a music feedback system utilizing the exclusion or addition of musical instruments for motion induction according to the present invention.
FIG. 4 is a flowchart of a method for implementing music feedback implemented by a music feedback system utilizing instrument exclusion or addition for motion induction according to the present invention.
FIG. 5 illustrates a flow diagram of a user input-based music feedback method implemented by a music feedback system utilizing instrument exclusion or addition for motion induction according to the present invention.

Specific structural or functional descriptions of embodiments according to the concept of the present invention disclosed in this specification are merely exemplified for the purpose of explaining embodiments according to the concept of the present invention, and embodiments according to the concept of the present invention may be implemented in various forms and are not limited to the embodiments described in this specification.

Since embodiments according to the concept of the present invention can have various changes and can have various forms, embodiments are illustrated in the drawings and described in detail in the present specification. However, this is not intended to limit embodiments according to the concept of the present invention to specific disclosed forms, but includes all changes, equivalents, or substitutes included in the spirit and technical scope of the present invention.

The terms first or second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are only intended to distinguish one component from another, for example, without departing from the scope of the invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

When it is said that a component is "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components in between. On the other hand, when it is said that a component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Other expressions that describe the relationship between components, such as "between" and "directly between" or "adjacent to" and "directly adjacent to", should be interpreted similarly.

The terminology used herein is only used to describe particular embodiments and is not intended to be limiting of the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. It should be understood that, as used herein, the terms "comprises" or "has" and the like are intended to specify the presence of a feature, number, step, operation, component, part, or combination thereof described in the present specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms defined in commonly used dictionaries, such as those defined in common usage, should be interpreted as having a meaning consistent with the meaning they have in the context of the relevant art, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings attached to this specification. However, the scope of the patent application is not limited or restricted by these embodiments. The same reference numerals presented in each drawing represent the same components.

Hereinafter, a music feedback system utilizing the exclusion or addition of an instrument for inducing movement according to the present invention will be described in detail with reference to the drawings. In this regard, FIG. 1 illustrates a drawing illustrating a single joint movement according to the present invention. FIG. 2 illustrates a drawing illustrating a compound joint movement according to the present invention.

Referring to Figure 1, an elbow joint flexion movement is shown as an example of a single joint movement. In this regard, the hand position can be estimated through the elbow joint flexion and compared with the stored reference position. If a position difference occurs, the instrument assigned to each dimension (x-axis, y-axis, z-axis) can be excluded or added to the music.

Referring to Figure 2, a squat movement is shown as an example of a complex joint movement. In this regard, the positions of the head, pelvis, and knees can be estimated through a specific squat movement, and compared with the stored reference positions. When a change in the positions of the head, pelvis, and knees occurs, the instruments assigned to each dimension (x-axis, y-axis, z-axis) can be excluded or added to the music.

Hereinafter, the configuration of a music feedback system utilizing the exclusion or addition of musical instruments for inducing movement according to the present invention will be described in detail. Fig. 3 shows a configuration diagram of a music feedback system utilizing the exclusion or addition of musical instruments for inducing movement according to the present invention.

Referring to FIG. 3, the music feedback system may be configured to include a camera (110), a wearable sensor (120), a control unit (130), a memory (140), and an audio output unit (150). The camera (110) may be provided in a user terminal, such as a smart phone such as a mobile terminal, or may be provided separately therefrom. The wearable sensor (120) may be implemented as at least one sensor and may be attached to the user's body (e.g., each joint part). The control unit (130) may correspond to a processor of a user terminal, such as a smart phone such as a mobile terminal. The memory (140) may be operably coupled with the control unit (130) and configured to store information about a specific body location and music feedback. The audio output unit (150) may be implemented as a Bluetooth wireless earphone and/or a stereo speaker. The Bluetooth wireless earphone and/or the stereo speaker may be implemented as two or more on the left and/or right sides to provide music-type feedback to the left and/or right sides of the user's body.

The camera (110) is configured to capture the user's body and body movements. The wearable sensor (120) may be attached to a body segment location of the user and configured to detect the movement of a specific body part. The control unit (130) may be operably coupled with the camera (110) and the wearable sensor (120) and configured to play music that induces the movement of the body part.

The control unit (130) can compare the reference movement with the real-time movement to determine whether to exclude or add a specific instrument from the music selected by the user. If negative feedback is selected, the control unit (130) can control the instrument that was excluded to be added to the music again when the user's body segment position returns to the reference position. If positive feedback is selected, the control unit (130) can control the instrument that was added to be excluded from the music when the user's body segment position moves away from the reference position.

In this regard, the main task of the present invention is to induce correct movement of the user by removing or adding an instrument to music according to the body movement recognized by a wearable sensor (IMU) or a camera linked to a smart phone. Meanwhile, the detailed task of the present invention can be summarized as follows. In this regard, FIG. 4 shows a flowchart of a music feedback implementation method implemented by a music feedback system utilizing the exclusion or addition of an instrument for movement induction according to the present invention.

1. In the sound source extraction/instrument classification process (S100), the sound source is extracted using MFCC (Mel-Frequency Cepstral Coefficient), the instrument and sound source are classified, and then the instrument addition/subtraction is implemented.

2. In the instrument addition/exclusion process (S200) during compound joint movement, the instrument assigned to each body joint during compound joint movement is added to or removed from the music according to the body position. The instrument addition/exclusion process (S200) during compound joint movement can be configured to include a body segment position estimation process (S210), a negative feedback implementation process (S220), and a positive feedback implementation process (S230).

2-1. In the body segment position estimation process (S210), the body segment position is estimated in real time using body segment trajectory, angle, and acceleration data or images input to a smart phone.

2-2. (Negative feedback) In the negative feedback implementation process (S220), the instrument assigned to each joint is removed by comparing the estimated position in real time with the reference position. For example, music is played while the instruments for each body part are assigned. When exercising while listening to music, if the knee deviates from the position indicated by the device during the exercise motion, the drum is removed from the music and played, and if the waist deviates from the position indicated by the device, the guitar is removed from the music and played. When the position of the body segment returns to the position indicated by the device, the removed instrument is added again and the original music is played.

2-3. (Positive feedback) In the positive feedback implementation process (S230), the instrument assigned to each joint is added by comparing the estimated position in real time with the reference position. For example, in the case of a squat movement, only drums are played when the movement is first performed. If the real-time knee position is similar to the reference position, a guitar is added, and if the pelvis position is similar to the reference position, a bass is added. If the reference movement and all body positions are similar, music with all instruments added is played.

3. In the instrument addition/exclusion process (S300) during single-joint movement, the instrument assigned to each dimension of the body position is added to or removed from the music according to the body position. The instrument addition/exclusion process (S300) during single-joint movement can be configured to include a three-dimensional body position estimation process (S310), a dimension-by-dimension position difference calculation process (S320), a negative feedback implementation process (S330), and a positive feedback implementation process (S340).

3-1. In the 3D body position estimation process (S310), the 3D body position is estimated using body segment trajectory, angle, and acceleration data or images input to the smartphone.

3-2. In the process of calculating the position difference by dimension (S320), the position difference for each dimension is calculated by comparing the estimated position in real time with the reference position.

3-3. (Negative Feedback) In the negative feedback implementation process (S330), the instruments assigned to each dimension are excluded from the music according to the position difference. For example, if there is a position difference in the x-axis, the drum is excluded from the music, and if there is a position difference in the y-axis, the guitar is excluded from the music.

3-4. (Positive feedback) In the positive feedback implementation process (S340), the instruments assigned to each dimension are added by comparing the estimated position in real time with the reference position. For example, when performing the elbow bending motion, only drums are played when the motion is first performed. When performing the elbow bending motion, if the difference in the x-axis position is small, a guitar is added, the y-axis adds a bass, and the z-axis adds a keyboard. If the reference and position are similar, music is played with all instruments added.

Meanwhile, the configuration and operation of a music feedback system utilizing the exclusion or addition of a musical instrument for inducing movement according to the present invention will be described in detail with reference to FIGS. 1 to 4. In this regard, the configuration of a music feedback system utilizing the exclusion or addition of a musical instrument for inducing movement according to the present invention can be classified in detail as follows, but is not limited thereto.

1) Camera (110) equipped on a smart phone

2) Wearable Sensor (IMU) (120)

3) Algorithm for estimating body segment locations using images

4) Body segment position estimation algorithm using orbit, angle and acceleration data

5) MFCC (Mel-Frequency Cepstral Coefficient) Algorithm for classifying instruments and sound sources in music

6) Algorithm that can individually remove or add instruments to the music based on body position.

In this regard, the algorithms 3) to 6) can be implemented by a control unit (130) and a memory (140) which are processors equipped in a smart phone or separate smart device.

7) Application of smart device that can select movement, music, instrument, type of feedback, and feedback intensity. The application can be installed in memory (140), which is a processor equipped in a smart phone or a separate smart device, and executed by the control unit (130).

8) Bluetooth wireless earphones and stereo speakers. In this regard, the sound output unit (150) may be implemented as Bluetooth wireless earphones and/or stereo speakers. The Bluetooth wireless earphones and/or stereo speakers may be implemented as two or more on the left and/or right sides to provide music-type feedback to the left and/or right sides of the user's body.

With reference to the above description, the operation and function of a music feedback system utilizing the exclusion or addition of musical instruments for inducing movement according to the present invention are described in detail.

The user installs the smartphone in a location where the entire user's body can be captured, or attaches wearable sensors (IMUs) to body segments suggested according to the movement. The number of wearable sensors (IMUs) is determined according to the intended movement, and the wearable sensors (IMUs) can be clothing that includes multiple wearable sensors (IMUs) as needed.

The input body image produces body segment location information through a body segment location estimation algorithm, which is then used as a reference for music feedback.

Users can choose negative or positive feedback.

The body segment position information is compared in real time with the reference movement to determine whether to exclude or add a specific instrument from the music selected by the user. For example, in a compound joint movement, if the user's knee position deviates from the reference knee position by a certain amount, the drum is excluded or added to the music. In a single joint movement, if the user's knee's x-axis position deviates from the reference knee's x-axis position by a certain amount, the guitar is excluded or added to the music.

If negative feedback is selected, the instrument that was excluded is added back to the music when the user's body segment position returns to the reference position. If positive feedback is selected, the instrument that was added is excluded from the music when the user's body segment position moves away from the reference position.

Meanwhile, a detailed description of a user input-based music feedback method implemented by a music feedback system utilizing instrument exclusion or addition for motion induction according to the present invention is as follows. In this regard, FIG. 5 shows a flowchart of a user input-based music feedback method implemented by a music feedback system utilizing instrument exclusion or addition for motion induction according to the present invention.

1) The user selects a symptom through a smartphone application.

2) The application suggests recommended actions based on symptoms.

3) The user selects actions and music through the application.

4) Users receive feedback through the application. The types of feedback are divided into negative feedback and positive feedback. Negative feedback removes instruments from the music when an incorrect action is performed, and positive feedback adds instruments to the music when a correct action is performed.

In relation to 1) to 4), the symptom/action/music selection process (S510) is performed based on user input via a smart phone application.

5) If you select single joint motion, assign instruments to the x, y, and z axes.

6) If you select a compound joint motion, assign an instrument to the corresponding joint.

5) and 6), an instrument allocation process (S520) according to single/complex joint motion is performed.

7) Place the smartphone in a location where it can capture the entire body. Or, attach a wearable sensor (IMU) to the body segment suggested by the movement.

8) The reference motion corresponding to the selected motion is played through the application.

9) The user tries to follow the presented reference movements identically.

7) to 9), a reference motion reproduction/user motion acquisition process (S530) can be performed.

10) If negative feedback is selected, when you deviate from the suggested action, the instrument assigned to the joint or position dimension is removed from the music that flows. If positive feedback is selected, when you perform an action similar to the suggested action, the instrument assigned to the joint or position dimension is added to the music.

11) If negative feedback is selected, the removed instrument is restored when the presented action is deviated from and then returned to the presented action. If positive feedback is selected, the added instrument is removed from the music when the presented action is deviated from.

10) and 11), it can be determined whether negative feedback is selected (S540). If negative feedback is selected (S540) and it is determined that the user's motion deviates from the reference motion (S560), a process of removing an instrument assigned to a specific joint/position (S570) is performed. If positive feedback is selected (S550) and it is determined that the user's motion has returned to the reference motion (S580), a process of adding the corresponding instrument to the specific joint/position (S590) is performed.

If negative feedback is selected (S540) and the user's motion is determined to have deviated from the reference motion (S550), and then returned to the reference motion (S580), a process of adding a corresponding instrument to a specific joint/position (S590) is performed. If positive feedback is selected (S550) and the motion is determined to have deviated from the reference motion, a process of removing a corresponding instrument from a specific joint/position (S570) is performed.

Hereinafter, with reference to FIGS. 1 to 5, the detailed operation of a music feedback system utilizing the exclusion or addition of musical instruments for inducing movement according to the present invention will be described.

The control unit (130) can extract sound sources using MFCC (Mel-Frequency Cepstral Coefficient). MFCC can be divided into six steps as follows, but is not limited thereto. 1) Segment the sound signal of the input time domain into small-sized frames, and 2) calculate the Periodogram estimate (Periodogram Spectral Estimate) of the Power Spectrum for each frame. 3) Apply the Mel Filter bank to the Power Spectrum obtained in 2), and add the energy of each filter. 4) Take the Log of all the filter bank energies obtained in 30. 5) Apply the DCT to the value obtained in 4), and 6) discard the rest except for some coefficients of the DCT value.

The control unit (130) can classify the extracted sound sources and instruments, and then control the instruments to be added to or excluded from the music based on the classification. The control unit (130) can adjust the volume and beats of the sound sources of the music divided into specific time sections so that the sound sources of the music are similar to the extracted sound sources within a predetermined range.

The control unit (130) can control to exclude or add a first instrument from the music if the user's knee position deviates from the reference knee position by a specific value or more in a compound joint movement. The control unit (130) can control to exclude or add a second instrument from the music if the x-axis position of the user's knee deviates from the x-axis position of the reference knee by a specific value or more in a single joint movement.

The control unit (130) can control the musical instruments assigned to each body joint to be added to or excluded from the music according to the difference between the current position of each body joint and the reference position when a compound joint movement is performed.

The control unit (130) can control the music to be played in a state where instruments are designated for each body part. When exercising while listening to music, the control unit (130) can control the first instrument to be removed and played from the music when the knee moves out of the position indicated by the device during the exercise motion, and the second instrument to be removed and played from the music when the waist moves out of the position indicated by the device. When the position of the body segment returns to the position indicated by the device, the control unit (130) can control the music to be played by adding at least one of the first instrument and the second instrument that was removed again.

The control unit (130) can control the addition of an instrument assigned to each joint by comparing the estimated position in real time with the reference position. The control unit (130) can control the music so that only the first instrument is played when the squat motion is first performed. The control unit (130) can control the music so that the second instrument is added if the knee position is similar to the reference position within a predetermined range. The control unit (130) can control the music so that the third instrument is added if the pelvis position is similar to the reference position within a predetermined range. The control unit (130) can control the music so that all instruments are added if all body positions are similar to the reference motion within a predetermined range.

The control unit (130) can estimate a three-dimensional body position using body segment trajectory, angle, and acceleration data or images input to a smart phone. The control unit (130) can calculate the difference between the position of each joint and the reference position by comparing the position of each body joint estimated in real time with the reference position.

The control unit (130) can control the first instrument to be excluded from the music when a difference in the x-axis position of each joint of the body occurs. The control unit (130) can control the second instrument to be excluded from the music when a difference in the y-axis position of each joint of the body occurs.

The control unit (130) can control the music so that only the first instrument is played when a specific motion of each joint of the body is first performed. The control unit (130) can control the music so that when the specific motion of each joint of the body is performed, if the difference in the x-axis positions is less than a predetermined range, the second instrument is added, if the difference in the y-axis positions is less than a predetermined range, the third instrument is added, and if the difference in the z-axis positions is less than a predetermined range, the fourth instrument is added.

The control unit (130) can control so that settings regarding the motion of each joint of the user's body, music, instruments, feedback type, and feedback intensity are selected. The control unit (130) can compare the position of each dimension of each joint of the body with the position of each dimension of a reference motion, and control so that at least one of the instruments is added or excluded from the music. The control unit (130) can control so that the music is played from a speaker located in a direction where each body joint is determined to be inconsistent with a specific motion among a plurality of speakers in a space where the user is located, as a feedback type. The control unit (130) can control so that the volume and beat of the music are adjusted as a feedback intensity.

In the above, the music feedback system utilizing the exclusion or addition of musical instruments for inducing movement according to the present invention has been described. The technical effects of the music feedback system utilizing the exclusion or addition of musical instruments for inducing movement according to the present invention can be summarized as follows.

According to an embodiment of the present invention, body movements during exercise are detected in real time, and feedback is given to the user based on the detection results. Through the feedback, the user recognizes incorrect movements and puts more effort into achieving correct movements.

According to an embodiment of the present invention, target muscles can be efficiently activated by inducing correct movements, and movements that reduce pain can be learned.

According to an embodiment of the present invention, by using music to break away from a monotonous feedback format, boredom of the user can be reduced and interest can be increased. This can increase the duration of the user's exercise.

According to an embodiment of the present invention, the musculoskeletal condition of a user can be diagnosed based on whether or not the user can perform correct movements.

According to an embodiment of the present invention, the use of the medical device and fitness equipment can enhance understanding of correct movements.

Most medical devices and fitness equipment that use feedback to induce correct body movements induce single-joint movements. According to an embodiment of the present invention, compared to single-joint movements, complex joint movements are effective in improving body coordination and have high exercise efficiency. It is expected that inducing correct complex joint movements will have a greater effect on the user's return to daily life and injury prevention.

Further scope of applicability of the present invention will become apparent from the detailed description below. However, since various changes and modifications within the spirit and scope of the present invention will become apparent to those skilled in the art, it should be understood that the detailed description and specific embodiments, such as preferred embodiments of the present invention, are given by way of example only.

The features and effects of the present invention described above will become more apparent through the following detailed description with reference to the attached drawings, so that a person having ordinary skill in the art to which the present invention pertains can easily practice the technical idea of the present invention.

The present invention can have various modifications and various embodiments, and specific embodiments are illustrated in the drawings and specifically described in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but should be understood to include all modifications, equivalents, or substitutes included in the spirit and technical scope of the present invention.

In terms of software implementation, the procedures and functions described herein, as well as the design and parameter optimization for each component, can be implemented as separate software modules. The software code can be implemented as a software application written in a suitable programming language. The software code can be stored in memory and executed by a controller or a processor.

110: Camera
120: Wearable Sensors
130: Processor
140: Memory
150: Audio output section

Claims (10)

In a music feedback system that utilizes the exclusion or addition of instruments to induce movement,
A camera configured to capture the user's body and body movements;
A wearable sensor attached to a body segment location of the user and configured to detect movement of a specific body part; and
A control unit operatively coupled with the camera and the wearable sensor and configured to play music that induces movement of the body part;
The above control unit,
Compares reference movements with real-time movements to determine whether to exclude or add specific instruments from the user's selected music.
If negative feedback is selected by the user, the excluded instrument is controlled to be added back to the music when the user's body segment position returns to the reference position.
If positive feedback is selected by the user, the added instrument is controlled to be excluded from the music when the user's body segment position deviates from the reference position.
Extract sound sources using MFCC (Mel Frequency Cepstral Coefficient),
After classifying the extracted sound sources and instruments, the instruments are controlled to be added or excluded from the music based on the classification.
A music feedback system that adjusts the volume and beats of the music sound sources divided into specific time sections so that the music sound sources divided into specific time sections are similar to the extracted sound sources within a predetermined range. delete In the first paragraph,
The above control unit,
In a compound joint movement, if the user's knee position deviates from the reference knee position by a certain value or more, the first instrument is controlled to be excluded or added to the music.
A music feedback system that controls a second instrument to be excluded or added to the music when the x-axis position of the user's knee deviates from the x-axis position of the reference knee by a specific value or more in a single-joint movement. In the third paragraph,
The above control unit,
A music feedback system that controls the addition or exclusion of musical instruments assigned to each body joint to the music according to the difference between the current position of each body joint and the reference position when the above compound joint movement is performed. In the third paragraph,
The above control unit,
Control the music to be played while the instruments for each body part are designated.
When exercising while listening to the above music, if the knee deviates from the position indicated by the device during the exercise movement, the first instrument from the above music is removed and played, and if the waist deviates from the position indicated by the device, the second instrument from the above music is removed and played.
A music feedback system that controls the music to be played by adding back at least one of the first instrument and the second instrument that was removed when the position of the body segment returns to the position indicated by the device. In the third paragraph,
The above control unit,
Controls the addition of instruments assigned to each joint by comparing the estimated position in real time with the reference position,
Control the music so that only the first instrument plays when the squat movement is first performed.
Control the music so that a second instrument is added when the knee position is similar to the reference position within a predetermined range,
Control the music so that a third instrument is added when the pelvic position is similar to the reference position within a certain range,
A music feedback system that controls the music so that all instruments are added when all body positions are similar to the reference movement within a predetermined range. In the third paragraph,
The above control unit,
Estimate 3D (x, y, z axis) body position using body segment trajectory, angle, and acceleration data or images input to a smartphone.
A music feedback system that calculates the difference between the position of each joint and the reference position for each dimension by comparing the position of each body joint estimated in real time with the reference position. In Article 7,
The above control unit,
If there is a difference in the position of the x-axis of each joint of the above body, the first instrument is controlled to be excluded from the music.
A music feedback system that controls a second instrument to be excluded from the music when a difference in the y-axis position of each joint of the body occurs. In Article 7,
The above control unit,
Control the music so that only the first instrument is played when a specific movement of each joint of the body is first performed,
When a specific movement of each joint of the above body is performed, if the difference in the x-axis position is less than a predetermined range, the second instrument is controlled to be added, if the difference in the y-axis position is less than a predetermined range, the third instrument is controlled to be added, and if the difference in the z-axis position is less than a predetermined range, the music is controlled to be added, and
A music feedback system that controls all instruments to play added music when the position of a specific movement of each joint of the body is similar to the reference position within a predetermined range on all axes. In the third paragraph,
The above control unit,
Controls the selection of settings for each joint of the user's body, music, instruments, feedback type, and feedback intensity.
By comparing the x-axis, y-axis and z-axis positions of each joint of the body with the x-axis, y-axis and z-axis positions of the reference motion, at least one of the instruments is controlled to be added or excluded from the music,
Controlling the music to be played from a speaker or earphone located in a direction where each body joint is judged to be inconsistent with a specific movement among multiple speakers and earphones supporting spatial audio in the space where the user is located with the above feedback type,
A music feedback system that controls the volume and beat of the music by the feedback intensity.