KR101553484B1 - Apparatus for detecting hand motion and method thereof - Google Patents

Abstract

The present invention relates to a hand gesture recognition apparatus and a method thereof. A hand-operated recognition apparatus according to the present invention includes: a sound analyzer for analyzing a sound sensed by a sound sensor; A motion recognition unit for analyzing motion and vibration of a user sensed by the motion sensor; A database unit for storing respective sounds, motions, and vibration values corresponding to various hand motions of the user; A hand gesture recognition unit for recognizing a user's hand gesture from the database unit using the analyzed sound, motion, and vibration; And a control unit for generating a control signal corresponding to the recognized hand movement.
As described above, according to the present invention, the accuracy of the motion can be improved by using the motion of the hand and the additional sound. In addition, it is possible to make various input patterns in the interface device according to the presence or absence of sound and the number of occurrences of sound, as compared with the case of using only hand motion recognition.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a hand-

The present invention relates to a hand-operated recognition apparatus and method, and more particularly, to a sensor and a method for recognizing a motion of a hand and a fricative sound such as an applause sound according to a hand-operated operation.

You will be able to pull out the keyboard and mouse as a representative device for the input interface. If the keyboard and mouse were the main input devices of a desktop computer, the touch screen input interface came to the fore in mobile computing. Among them, the pen-touch method of MS Windows phone and the hand-touch method of Apple iPhone are mainstream of the input method of the personal wireless terminal.

However, in order for a mobile device derived from mobility to have true mobility, a more convenient user interface device is required. In this regard, an input interface that utilizes human hand motions in 3D free space is a field of interest in recent years.

FIG. 1 shows a screen control according to the conventional handwriting recognition apparatus.

The input interface device using the motion of the hand expresses the motion of the hand using an RGB camera that distinguishes colors and an IR camera that distinguishes depths. It is possible to control the electronic map and various application programs through the input interface using the motion of the hand.

In addition, an interface device capable of recognizing both the motion of the hand and the human voice is used as an input device of a game machine. However, if a fricative sound such as a sound due to a hand gesture, such as an applause sound, is additionally recognized in the motion of the hand, certainty may be given to the interface device. For example, when a specific command is performed by an interface device through an applause operation, if an applause having a certain frequency is used without an individual difference for each person rather than an uncertain hand motion, Can be guaranteed.

When using only the motion of the hand, its accuracy may be limited. For example, according to the conventional hand motion sensor, it is difficult to distinguish between the hand gestures interrupted by applause and the hand gestures influenced by applause. In the process of moving the hand, the sound generated by the contact between the hand and the hand or between the hand and the object may be an important factor in the input interface.

The technology that constitutes the background of the present invention is described in Korean Patent Laid-Open Publication No. 2013-0138519 (Dec. 19, 2013).

A problem to be solved by the present invention is to provide a sensor and a method for recognizing sound and movement according to a hand movement.

According to one embodiment of the present invention, there is provided an apparatus for recognizing a hand gesture to generate a control signal, the apparatus including: a sound analyzer for analyzing a sound sensed by the sound sensor; A motion recognition unit for analyzing motion and vibration of a user sensed by the motion sensor; A database unit for storing respective sounds, motions, and vibration values corresponding to various hand motions of the user; A hand gesture recognition unit for recognizing a user's hand gesture from the database unit using the analyzed sound, motion, and vibration; And a control unit for generating a control signal corresponding to the recognized hand movement.

In addition, the sound analyzer may analyze the frequency spectrum of the sensed sound to distinguish apples, finger snaps, and sound generated when an object is hit.

The sound analyzing unit may analyze the sensed sound only when the motion recognizing unit recognizes vibration of a threshold value or more.

The sound analysis unit may further include: a microphone for converting sound into an electric signal; An amplifier for amplifying the electric signal to a predetermined level; A band pass filter for filtering a signal component corresponding to a frequency band of the sound from the signal amplified at a predetermined level through the amplifier and outputting the filtered signal component; An analog-to-digital converter for converting the signal components filtered by the band-pass filter into a digital signal; And analyzing a frequency spectrum of the digital signal to discriminate a sound according to a hand gesture.

According to an embodiment of the present invention, there is provided a method of recognizing a hand gesture using an apparatus for recognizing a hand gesture and generating a control signal, comprising the steps of: analyzing a sensed sound from a sound sensor; Analyzing the motion and vibration of the user sensed from the motion sensor; Storing respective sounds, motions, and vibration values corresponding to various hand motions of a user; Recognizing a user's hand gesture from the stored sound, motion, and vibration values using the analyzed sound, motion, and vibration; And generating a control signal corresponding to the recognized hand gesture.

According to the hand-operated recognition apparatus and method of the present invention, motion accuracy can be improved by using motion of the hand and additional sound. In addition, it is possible to make various input patterns in the interface device according to the presence or absence of sound and the number of occurrences of sound, as compared with the case of using only hand motion recognition.

FIG. 1 shows a screen control according to the conventional handwriting recognition apparatus.
Fig. 2 shows an example of a hand gesture in which sound is generated.
FIG. 3 is a block diagram illustrating a configuration of a hand movement recognition apparatus according to an embodiment of the present invention.
FIG. 4 is a diagram showing the configuration of the sound analysis unit in FIG. 3;
5 is a flowchart of a hand movement recognition method according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

Throughout the specification, when an element is referred to as "including" an element, it does not exclude other elements unless specifically stated to the contrary, But do not preclude the presence or addition of one or more of the other features, numbers, steps, operations, elements, parts, or combinations thereof.

Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS. 2 to 5. FIG.

The hand gesture recognizing apparatus 100 according to the embodiment of the present invention recognizes the sound with motion of the hand and generates a control signal corresponding to the recognized motion and sound of the hand. Here, sound can be limited to sounds that can be generated by using the hand. According to the existing technology, a device for recognizing the motion of the voice and the hand has already been disclosed.

Fig. 2 shows an example of a hand gesture in which sound is generated.

As shown in FIG. 2, sounds that can be generated by using the hand can be roughly classified into three types. Fig. 2 (b) shows a finger snap action due to the friction of the finger, Fig. 2 (c) shows a finger snap action when the palm of the hand touches another object, .

In the above picture, the sound due to the friction generated in each case generally keeps the frequency range of the sound in a certain range, and in the cases of (a) to (c) Since there are differences, (a) to (c) can be distinguished from each other.

FIG. 3 is a diagram illustrating a configuration of a hand-operated recognition apparatus according to an embodiment of the present invention, and FIG. 4 is a diagram illustrating a configuration of a sound analysis unit in FIG.

3, the handwriting recognition apparatus 100 according to the embodiment of the present invention includes a sound analysis unit 110, a motion recognition unit 120, a database unit 130, a handwriting recognition unit 140, and a controller 150 ).

4, the sound analysis unit 110 includes an amplifier 111, a band-pass filter 112, an analog-to-digital converter 113, and a digital signal processor 114. [

A sound sensor (not shown) receives a sound, converts the sound into an electric signal, and transmits the electric signal to the sound analysis unit 110.

The sound analysis unit 110 amplifies the inputted electric signal through the amplifier 111. [ The amplified electric signal is filtered through a band-pass filter to a certain frequency range. The electric signal filtered to a certain extent is converted into a digital signal through the analog / digital converter 113. Finally, the digital signal processor 114 analyzes the spectrum of the digitized electrical signal and compares it with a spectrum according to the type of sound of the pre-stored database unit 130 to determine whether the electrical signal is due to clapping, finger snapping or friction with an object . Here, the digital signal processor 114 analyzes the spectrum of the inputted electric signal according to time, and can determine the number of times of sound generation, that is, the number of applause, when sound is generated continuously.

In order to distinguish sounds generated by hand movements from sounds other than sounds generated through sound by a sound sensor (not shown), the sound analysis unit 110 may perform a sensing operation only when a vibration exceeding a threshold value is recognized by the motion recognition unit 120 Can be analyzed. When sound is generated by hand movements such as applause, finger snap, or object striking, a certain degree of vibration occurs in the hand. Therefore, in order to analyze only the sound due to the hand movement, the sound analysis unit 110 can be operated only when the motion recognition unit 120 recognizes the vibration of the threshold value or more.

The motion recognition unit 120 analyzes motion and vibration of the user sensed by the motion sensor. That is, the motion recognition unit 120 can analyze the patterns of motions of the user sensed using the patterns of motions pre-stored in the database unit 130. Here, the patterns of motion stored in the database unit 130 may be stored in a tree structure.

The motion recognition unit 120 can analyze a pattern of a user's hand gesture using an image representing a hand gesture of a user photographed through a photographing apparatus (not shown). In addition, the motion recognition unit 120 can detect joints of a user's hand in an image of a user's hand gesture, and recognize hand gestures varying according to movement of joints in different patterns. Since this is a technical matter widely known in the field of motion recognition apparatus, a detailed description will be omitted.

The database unit 130 stores respective sounds, motions, and vibration values corresponding to various hand motions of the user. That is, the database unit 130 stores the frequency spectrum of the corresponding sound signal for each type of sound that can be generated by the hand gesture, and analyzes the sound by comparing the frequency spectrum with the frequency spectrum of the input sound. In addition, the database unit 130 stores a pattern of motion and a critical vibration value as well as a database of sounds.

The hand gesture recognition unit 140 recognizes the user's hand gesture from the database unit 130 using the analyzed sound, motion, and vibration. That is, the handgrip recognition unit 140 recognizes the handgrip by using the sound analyzed by the sound analysis unit 110 and the motion and the vibration recognized by the motion recognition unit 120. FIG. For example, in the case where sound is analyzed, the handwriting recognition unit 140 outputs a pattern most similar to the input hand motion together with information values of the type of sound, the time of occurrence of sound, 130) and output the corresponding information value.

That is, the hand-operation recognition unit 140 receives the image related to the motion of the hand photographed in the motion recognition unit 120, maps the trajectory according to the motion of the hand onto the virtual three-dimensional space, Can be output. Here, the information value includes information about the type of sound, the time of sound generation, and the frequency of occurrence of sound.

The control unit 150 generates a control signal corresponding to the recognized hand operation. Here, the control unit 150 generates a control signal corresponding to the information value transmitted from the handgrip recognition unit 140. For example, if the transmitted information value includes an information value that calculates a trajectory of a hand moving in a three-dimensional space, information type about the type of sound according to the generation of sound, time of sound generation, and frequency of occurrence of sound , The controller 150 can control the operation of the pointer in conjunction with the motion of the hand on the computer screen using the information value of the locus calculated. In addition, the control unit 150 may cause an event corresponding to a sound of a specific frequency band to be executed, an event to be executed at a generation time, or a different control value signal according to the number of occurrences.

Hereinafter, a hand motion recognition method according to an embodiment of the present invention will be described.

5 is a flowchart of a hand movement recognition method according to an embodiment of the present invention.

For convenience of explanation, a case where the hand-operated recognition apparatus 100 according to the present invention is used as an interface device of a desktop computer such as a mouse or a keyboard will be described as an example.

First, the sound analysis unit 110 analyzes the sensed sound from the sound sensor (S510).

When the user performs the finger snap in the motion of the hand or using the stopped hand, the sound analysis unit 110 amplifies the sensed sound from the sound sensor (not shown), performs filtering, And analyzes the frequency spectrum of the converted signal to analyze the sound according to an operation. Here, since the sound of applause, the sound of finger snap, and the fricative when hitting the table correspond to sounds of different frequency bands, they can be distinguished from each other by using a frequency spectrum. Here, the sound analysis unit 110 may analyze the number of finger snaps through the frequency spectrum.

The motion recognition unit 120 analyzes motion and vibration of the user sensed by the motion sensor (S520).

The reason why the motion recognition unit 120 analyzes the presence or absence of vibration is that when a sound is generated by a hand operation, a vibration of a threshold value or more is accompanied by a hand movement. Therefore, it is analyzed whether or not vibration is generated so as to prevent the erroneous input due to the sound other than the sound due to the hand movement inputted to the sound sensor. The motion recognition unit 120 analyzes the motion of the user using the camera and the image captured through the infrared camera according to the conventional technique. In this case, the motion of the analyzed hand can be represented using the information value of the corresponding numerical value.

Motion, and vibration values corresponding to various hand motions of the user in advance in the database unit 130 (S530). That is, the frequency spectrum of sounds that can be generated when a variety of clapping sounds, finger snap sounds, and hitting of a table or a plurality of objects that may occur due to a hand gesture may be stored. Then, as described above, the critical vibration value can be stored in order to determine whether or not the vibration exceeds the threshold value. In addition, the information value of the motion of the hand which distinguishes the motion of the hand according to the pattern using the tree structure can be stored.

The handwriting recognition unit 140 recognizes the hand operation of the user from the database unit 130 using the analyzed sound, motion, and vibration (S540). That is, the handwriting recognition unit 140 compares the information value of the motion trajectory, the sound generation, the type of sound, and the number of sounds in the three-dimensional space with the value stored in the database unit 130, .

The control unit 150 generates a control signal corresponding to the recognized hand operation (S550). For example, the controller 150 may generate a control signal for moving the mouse pointer according to the locus of the user's hand operation, or may generate a control signal Lt; / RTI > In addition, a number of times corresponding to the number of occurrences of the finger snap sound may be input, or a control signal may be generated to execute a command corresponding to the number of times.

As described above, according to the hand gesture recognition apparatus and method of the present invention, motion accuracy can be improved by using hand motion and additional sound. In addition, it is possible to make various input patterns in the interface device according to the presence or absence of sound and the number of occurrences of sound, as compared with the case of using only hand motion recognition.

The present invention has been described above with reference to the embodiments. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. Therefore, the scope of the present invention is not limited to the above-described embodiments, but should be construed to include various embodiments within the scope of the claims and equivalents thereof.

100: a hand movement recognition device, 110: a sound analysis section,
111: amplifier, 112: bandpass filter,
113: analog / digital converter, 114: digital signal processor,
120: motion recognition unit, 130: database unit,
140: a hand movement recognition unit, 150: a control unit

Claims (10)

An apparatus for recognizing a hand gesture to generate a control signal,
A sound analysis unit for analyzing the sensed sound from the sound sensor;
A motion recognition unit for analyzing a motion of a user sensed by the motion sensor;
A database unit storing respective sounds and motions corresponding to various hand motions of the user;
A handgrip recognition unit for recognizing a user's hand gesture from the database unit using the analyzed sound and motion; And
And a control unit for generating a control signal corresponding to the recognized hand movement,
Wherein the sound analysis unit comprises:
Wherein the motion recognition unit analyzes the sensed sound only when a vibration of a threshold value or more is recognized by the motion recognition unit. The method according to claim 1,
Wherein the motion recognition unit comprises:
Further analyzing the vibration of the user sensed from the motion sensor,
The database unit,
Further storing respective vibration values corresponding to various hand motions of the user,
The hand-
And further recognizing a user's hand gesture from the database using the analyzed vibration. The method according to claim 1,
Wherein the sound analysis unit comprises:
A hand gesture recognizing device for discriminating clapping, finger snapping, and sound generated when an object is hit by analyzing the frequency spectrum of the sensed sound (1140). delete The method according to claim 1,
Wherein the sound analysis unit comprises:
An amplifier for amplifying the electric signal transmitted from the sound sensor to a predetermined level;
A band pass filter for filtering a signal component corresponding to a frequency band of the sound from the signal amplified at a predetermined level through the amplifier and outputting the filtered signal component;
An analog-to-digital converter for converting the signal components filtered by the band-pass filter into a digital signal; And
And a digital signal processor for analyzing a frequency spectrum of the digital signal to discriminate a sound according to a hand movement. A hand movement recognition method using a device for recognizing a hand movement and generating a control signal,
Analyzing the sensed sound from the sound sensor;
Analyzing the motion of the user sensed from the motion sensor;
Storing respective sounds and motions corresponding to various hand motions of a user in a database;
Recognizing a user's hand gesture from the stored sound and motion using the analyzed sound and motion; And
Generating a control signal corresponding to the recognized hand gesture,
Wherein the analyzing the sensed sound comprises:
And analyzing the sensed sound only when the motion recognition unit recognizes vibration of a threshold value or more. The method according to claim 6,
Wherein the analyzing of the motion of the sensed user comprises:
Further analyzing the vibration of the user sensed from the motion sensor,
The step of storing the sound and the motion includes:
Further storing respective vibration values corresponding to various hand motions of the user,
The step of recognizing the hand-
And further using the analyzed vibration to recognize a user's hand gesture from the database. The method according to claim 6,
Wherein the analyzing the sensed sound comprises:
And analyzing the frequency spectrum of the sensed sound to distinguish applause, finger snap, and sound generated when an object is hit. delete The method according to claim 6,
Wherein the analyzing the sensed sound comprises:
Converting the sound into an electrical signal;
Amplifying the electric signal to a constant level;
Filtering the signal component corresponding to the frequency band of the sound from the signal amplified at the predetermined level and outputting the signal component;
Converting the filtered signal component to a digital signal; And
And analyzing a frequency spectrum of the digital signal to discriminate a sound according to a hand movement.

Images