KR101573868B1 - Method for displaying music lyrics automatically, server for recognizing music lyrics and system for displaying music lyrics automatically comprising the server - Google Patents

Abstract

The method includes the steps of: (a) receiving song data information corresponding to an audio signal input via a microphone; (b) identifying a corresponding song from received song data information; and (c) Generating lyric data information including lyric data; and (d) transmitting the lyric data information.
By using the present invention, it is possible to acquire information about a song that is heard from the outside and display the synchronized lyrics.

Description

METHOD FOR DISPLAYING MUSIC LYRICS AUTOMATICALLY, SERVER FOR RECOGNIZING MUSIC LYRICS AND SYSTEM FOR DISPLAYING MUSIC LYRICS AUTOMATICALLY COMPRISING THE SERVER [0001] This invention relates to a song lyrics automatic display system,

The present invention relates to a song lyrics automatic display method, a server for recognizing song lyrics, and a song lyrics automatic display system including the server, which can display lyrics in real time from a song received from the outside and synchronize lyrics and songs A server for recognizing song lyrics, and a song lyrics automatic display system including the server.

We come to sing in various places around life. Generally, you can listen to songs at a coffee shop, a restaurant, a concert, or listen to songs from devices such as radio, TV, and YouTube. And listeners who listen to the song here want to know what song they are listening to and especially want to know what song lyrics are.

However, it is unreasonable to obtain information about the song being heard. If the listener can know the title of the song, it will be possible to obtain information such as lyrics for the song by searching through the Internet. It is a reality that the search through the Internet is inconvenient for the search input or confirmation depending on the situation of the listener (for example, during driving).

Therefore, there is a need for a device or a method that enables listeners to easily acquire information about a song they are listening to.

Meanwhile, as the technology has developed, various types of devices capable of transmitting and receiving data by being connected to the Internet have been introduced. A device connected to the Internet can also be operated according to an installed application program including a processor internally, and examples thereof include a smart phone, a smart TV, a personal computer, an electronic photo frame, and a tablet PC.

And, the device has a display that can conveniently and intuitively provide information about the song to the listener if the display can display song information such as song lyrics.

The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to provide a song lyrics automatic display method, a server for recognizing song lyrics, and a server for extracting information related to a song from an audio signal input through a microphone, The goal is to provide a song lyrics automatic display system that includes.

The present invention also provides a song lyrics automatic display method, a server for recognizing song lyrics, and a song lyrics automatic display system including the server, by which it is possible to identify lyrics corresponding to input audio signals to display corresponding lyrics The purpose is to do.

It is another object of the present invention to provide a song lyrics automatic display method, a server for recognizing song lyrics, and a song lyrics automatic display system including the server, in which the lyrics can be displayed in real time.

The present invention also provides a song lyrics automatic display method, a server for recognizing song lyrics, and a song lyrics automatic display system including the server, in which the display of the lyrics can be changed according to changes in the input audio signal, There is a purpose.

The present invention also provides a song lyrics automatic display method, a server for recognizing song lyrics, and a song lyrics automatic including the server so that the display of the input audio signal and lyrics can be synchronized in consideration of the processing time and the delay time of the network And a display system.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are not restrictive of the invention, unless further departing from the spirit and scope of the invention as defined by the appended claims. It will be possible.

According to another aspect of the present invention, there is provided a method of automatically displaying a song lyrics, comprising: (a) receiving song data information corresponding to an audio signal input through a microphone; and (b) (C) generating lyric data information including lyric data corresponding to the identified tune; and (d) transmitting the lyric data information.

In order to achieve the above object, a server for recognizing song lyrics includes a communication unit for receiving song data information corresponding to an audio signal inputted through a microphone and transmitting lyric data information, And transmits the generated lyric data information to the communication unit.

In order to achieve the above-mentioned object, a song lyrics automatic display system including a server includes a server for recognizing song lyrics, and a server for recognizing song lyrics includes song data information corresponding to an audio signal inputted through a microphone And a control unit for identifying the corresponding song from the song data information received through the communication unit and transmitting the generated song data information including the lyrics data corresponding to the identified song to the communication unit .

The automatic song-song display method, the server for recognizing song lyrics, and the automatic song-song display system including the server according to the present invention can extract information related to a song from an audio signal input through a microphone or the like It is effective.

Also, the automatic song song display method according to the present invention as described above, the server for recognizing the song lyrics, and the song lyrics automatic display system including the server can identify the lyrics corresponding to the inputted audio signals and display the corresponding lyrics .

In addition, the automatic song-song display method, the server for recognizing the song lyrics, and the automatic song-song display system including the server according to the present invention can display the lyrics in real time.

In addition, the automatic song-song display method, the server for recognizing song lyrics, and the automatic song-song display system including the server according to the present invention as described above can change the display of the lyrics according to the change of the input audio signal .

Also, the automatic song-song display method, the server for recognizing song lyrics, and the automatic song-song display system including the server according to the present invention, as described above, allow the input audio signal and the display of the lyrics to take into account the processing time and the delay time of the network So that they can be synchronized.

The effects obtained by the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those skilled in the art from the following description will be.

1 is a diagram showing an exemplary system configuration of a song lyrics automatic display system for recognizing a song and displaying the lyrics of the recognized song.
2 is a block diagram illustrating an example of a song recognition server.
3 is a diagram showing an exemplary control flow performed in the song recognition server.
4 is a block diagram showing an example of a personal apparatus.
5 is a diagram illustrating an exemplary control flow performed in a personal device.

The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings, in which: It can be easily carried out. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram showing an exemplary system configuration of a song lyrics automatic display system for recognizing a song and displaying the lyrics of the recognized song.

1, the song lyrics automatic display system includes a sound source 100, a personal device 200, a communication network 300, and a song recognition server 400. Some of the blocks of such a song lyrics automatic display system may be omitted. For example, the source source 100 may be omitted according to a variant.

Referring to each constituent block of the song lyrics automatic display system according to FIG. 1, a sound source 100 is a device capable of reproducing a song and outputting it as audio. For example, the source 100 may be a radio, a TV, a sound deck, a speaker connected to a sound deck, and the like.

Such a sound source 100 includes at least a speaker and outputs audio as a specific song (music) being reproduced so that the user can listen to the surrounding listener or a user who uses the song lyrics automatic display system of the present invention.

Personal device 200 is a personally available device. For example, the personal device 200 may be a mobile phone, a smart phone, a tablet PC, a personal computer, a photo frame, and the like. And the personal device 200 includes a display that includes a microphone capable of converting at least audio to electrical signals and is capable of displaying an image.

The personal device 200 can directly or indirectly recognize (through the song recognition server 400) the song output from the sound source 100 using the microphone and the display and display the lyrics of the recognized song. A more detailed description of the personal device 200 will be given with reference to FIGS. 4 and 5. FIG.

The song recognition server 400 is connected to one or more personal devices 200 to recognize songs and extract lyrics. The song recognition server 400 is capable of communicating with an external personal device 200 according to a designated communication protocol (for example, a communication protocol of an application layer among OSI (Open System Interconnection) 7 layers).

The song recognition server 400 can receive the song data information used for song recognition from the connected personal device 200 and identify the lyrics of the corresponding song and song from the received song data information using the database . The identified songs and lyrics may then be delivered to the personal device 200 by the song recognition server 400. The personal apparatus 200 receiving the information can output the lyrics through the display.

A more detailed description of the song recognition server 400 will be given with reference to FIGS. 2 and 3. FIG.

The communication network 300 is a network capable of transmitting and receiving data between one or more personal devices 200 and the song recognition server 400. The communication network 300 includes a wired communication network such as a wired LAN, and a wireless communication network such as an LTE or wireless LAN. Or the communication network 300 may be a combination of a wired communication network and a wireless communication network. Such a communication network 300 preferably constitutes the Internet.

1, it is shown that the sound source 100 exists. The sound source 100 may be omitted and the personal device 200 may serve as a sound source. Accordingly, the personal apparatus can output the stored specific song through the speaker and enable the recognition of the song or the display of the lyrics by the input through the microphone or by the internal direct delivery.

2 is a block diagram showing an exemplary block configuration of the song recognition server 400. As shown in FIG.

2, the song recognition server 400 includes a communication unit 410, a storage unit 420, a control unit 430, and a connection unit 440. Some of the blocks may be configured to include other blocks that are omitted or not shown in accordance with the type or variant of the song recognition server 400, in this block diagram. FIG. 2 shows only a configuration necessary for song recognition or lyrics recognition, and the components not related to the song recognition or the lyrics recognition are omitted because they may obscure the gist of the present invention.

Referring to the respective blocks of the song recognition server 400, the communication unit 410 allows the communication network 300 to transmit and receive data. The communication unit 410 is configured to be capable of data communication with one or more personal devices 200.

The communication unit 410 may be connected to the communication network 300 and is configured to output and receive a wired or wireless analog signal designated according to the communication type.

For example, the communication unit 410 can receive data from the communication network 300 and transmit the data to the control unit 430 in the song recognition server 400 and output the data from the control unit 430 to the communication network 300 .

The storage unit 420 stores various data and programs including a mass storage medium such as a hard disk, a volatile memory, a non-volatile memory, and the like. The storage unit 420 may include a database. For example, the storage unit 420 includes a music DB and a lyrics DB.

The song DB and the lyrics DB are used to identify songs from the song data information received from the connected personal device 200 and to extract the lyrics.

The song DB is a database storing various pieces of information related to songs that can be recognized by the song recognition server 400 for each song. Each song-specific information includes information such as, for example, beat information, melody information, a singer, a voice characteristic of a singer, a composer, a title, one or more representative words, Such information can be previously configured at the time of designing or installing the tune DB.

Here, the melody information or the voice characteristic of the singer may be defined in the frequency domain or the time domain, and may be composed of a so-called characteristic vector widely used in the field of music recognition or a combination of characteristic vectors.

The lyrics DB is a database that stores song lyrics. The lyrics of each song can be transmitted to the personal device 200 and displayed. In particular, the lyrics of a song can be divided into several pieces of measure data and managed in the lyrics DB. The measure data may indicate a portion of the lyrics to be displayed on the same line when displayed on the display of the personal device 200 or the node of the song.

In addition, the lyrics per song included in the lyrics DB may include a plurality of measure data and a plurality of time information corresponding to each word. Such time information may include information such as the start time and end time of the corresponding measure data, or the start time and the playing time of the measure data. And the time information may be transmitted to the personal device 200 connected via the communication network 300. The time information of each measure data can be expressed in relative time (time) after the start of the performance of the music piece.

As described above, the plurality of measure data represents one piece of the lyrics, and information such as the posterior relationship between the measure data and the time at which each measure data is played is provided to the personal device 200 or the song recognition server 400 .

The time information included in the lyrics DB may be configured in advance and may be used to highlight specific measure data as time elapses when displaying the lyrics represented by a plurality of measure data in the personal device 200 .

In addition, the lyrics DB may further include characteristic vectors corresponding to the lyrics (or measure data) of the songs. These feature vectors can be used for speech recognition.

The controller 430 controls the blocks in the song recognition server 400. The control unit 430 may be a processor capable of executing instructions of a program stored in the storage unit 420. A processor may be a so-called CPU, which may include one or more Execution Units, a Micro Processor Unit (MPU), or the like.

The control unit 430 preferably controls other blocks using various programs stored in the storage unit 420. [ For example, the control unit 430 can identify and extract songs and lyrics from the song data information received from the remote personal device 200 using the database stored in the storage unit 420. [

The control flow performed by the controller 430 will be described in detail with reference to FIG.

The connection unit 440 allows data to be transmitted or received between blocks in the song recognition server 400. [ This connection 440 may be a parallel bus, a serial bus, a single signal line, or a combination of such buses or signal lines.

FIG. 3 is a diagram illustrating an exemplary control flow performed in the song recognition server 400. FIG. The control flow of FIG. 3 is performed by controlling the other blocks by the control unit 430, and is preferably performed by using a program stored in the storage unit 420.

First, the control flow of FIG. 3 may start (S100) according to a connection request from a remote personal device 200 via the communication network 300 and end (S200) according to a termination request.

Upon receipt of the connection request, the controller 430 performs communication setup with the remote personal device 200. For example, the control unit 430 may assign an ID (e.g., a session ID) to the personal device 200 using a program executed in the application layer of the OSI layer 7.

In step S101, the control unit 430 receives the song data information through the communication unit 410. [ The received song data information may include audio data generated from audio input through the microphone of the personal device 200. [ The audio data may be data representing one measure identified in the personal device 200 or data received via a microphone during an appointed time. And the audio data may be compressed in any desired format. The promised time or the promised format can be determined at the time of communication setting or preset in the program itself.

In step S103, the control unit 430 identifies a song corresponding to the audio data of the song data information by using the song DB and the lyrics DB of the storage unit 420. [ For example, the control unit 430 can identify a specific music by comparing audio data with various information included in the music DB.

To this end, the control unit 430 separates the audio data and the melody data from the received audio data and extracts the time information. Generally, the voice data and the melody data are separable because the characteristics of the voice and the musical instrument (for example, the piano) are different.

The voice data or the melody data can be converted into the same form as the voice characteristic or the melody information of the mantissa stored in the music DB.

The controller 430, which has obtained such data or information, identifies one or more specific tunes in the tuning DB by using the separated tunes, melody data, and time information.

For example, the control unit 430 can recognize the voice data and compare the recognized contents with representative words stored in the song DB to determine similarity with each song. Alternatively, the control unit 430 may compare the melody information and the beat information of the music DB using the separated melody data and beat information to determine whether or not the songs are similar to each other. Alternatively, the control unit 430 may compare the voice characteristics extracted from the voice data and the voice characteristics of the singers stored in the music DB to determine whether they are similar to the respective songs. Alternatively, the control unit 430 can determine whether or not the music piece is similar to a specific piece of music by a combination of voice recognition, melody information, beat information, and voice characteristic color. Based on the determination of similarity, it is possible to identify songs with a certain threshold similarity or higher.

This process, of course, requires efficient search and comparison. Especially since digital signal processing is required at the song recognition server 400. [ Accordingly, it is necessary to narrow the search range. For this purpose, the control unit 430 may be configured to sequentially select and apply various parameters (such as voice or melody, beat, voice characteristic, etc.) for music identification.

For example, the control unit 430 may select a plurality of songs having a predetermined degree of similarity or more from the melody data in the music DB using the separated melody data, and identify whether the recognized contents exist from the separated voice data in the plurality of songs The song can be identified. In this process, the control unit 430 can compare the recognized word with the representative word. Alternatively, the control unit 430 may compare the recognized content with lyrics of a plurality of measure data corresponding to a plurality of songs, and identify the songs from the lyrics.

Through this process, the controller 430 can identify one or a plurality of tunes. It is generally expected that one song can be identified, but multiple songs may be identified or selected if the discernment is poor. Information or lyric information on the plurality of songs may be transmitted to the personal device 200 and configured to display a specific one of the songs according to the selection in the personal device 200. [

Thereafter, in step S105, the control unit 430 generates lyric data information corresponding to the identified music using the lyric DB. The lyrics data information may include lyrics corresponding to the identified songs.

More specifically, the control unit 430 extracts time information of each of the plurality of measure data and the lyric data including a plurality of measure data corresponding to the music identified in the lyric DB, and generates the lyric data information including the time information . The time information of the measure data can be expressed by information such as the start time of the measure data, the performance time, and the like during the performance of the song. This time information can then be used by the personal device 200 to know when the corresponding measure data should be emphasized and further to be used for time synchronization of the song and the lyrics.

In this manner, the controller 430 can transmit information related to all the lyrics of the song recognized from the received audio data and the time at which the lyrics data in the lyrics should be displayed to the personal device 200. [

In addition, the control unit 430 may compare the received audio data with the entire lyrics of the identified song to match a specific piece of lyrics (e.g., measure data). For example, the control unit 430 compares received audio data or separated voice data with each of a plurality of measure data, and identifies a portion matching the predetermined threshold similarity or more.

The respective measure data and the audio data of the received audio data can be compared but need not be limited. For example, the control unit 430 can compare the voice data with two consecutive measure data (for example, measure data of one measure and measure data of a succeeding measure immediately following the measure) to identify a matching portion.

The control unit 430 may include the information indicating the matched part or the matching measure data in the lyric data information. The matching information allows the personal apparatus 200 to know what kind of lyrics to emphasize as the song progresses.

Here, the matching information may further consider the network delay time required for reception of the audio data information from the personal device 200 and transmission of the lyric data information, and the elapsed time according to comparison of the plurality of measure data in step S103 and the like.

Accordingly, the matching information may reflect the network delay time and the internal elapsed time. For example, the control unit 430 may change or set the matched portion from the corresponding measure data of the audio data to the corresponding measure data after the network delay time and the internal processing elapsed time.

The network delay time can be simply determined based on the difference between the transmission time of the data packet transmitted through the communication network 300 and the reception time of the response packet of this data packet. And the elapsed time can be easily determined by the control unit 430 as the time required for the steps S103 and S105.

The application of the delay time and the like to the original matching information may be omitted depending on the modification and may be performed by the personal device 200. The determination of the matching measure data in the personal device 200 will be described with reference to FIG.

And the lyric data information further includes other information included in the tune DB. For example, the lyric data information may further include information such as a title, a singer, and the like.

Thereafter, in step S107, the controller 430 transmits the generated lyric data information to the personal device 200 via the communication unit 410 via the set communication channel.

These steps S101 to S107 may be repeatedly performed, and more accurate measure data or part matching may be performed according to the repetitive execution.

In this way, it is possible to deliver the lyrics to the personal apparatus 200 requesting the lyrics in real time by the control flow shown in FIG. 3, so that the lyrics and song can be synchronized visually.

4 is a block diagram illustrating an exemplary configuration of the personal device 200. As shown in FIG.

4, the personal device 200 includes an audio receiving unit 210, an input unit 220, an output unit 230, a communication unit 240, a storage unit 250, a control unit 260, and a connection unit 270 do. Some of the blocks may be configured to include other blocks that are omitted or not shown in accordance with the type or modification of the personal device 200, in addition to the block diagram. FIG. 4 shows only the necessary configuration for song recognition and lyrics display, and components not related to song recognition and lyrics display are omitted because they may obscure the gist of the present invention.

Referring to each block of the personal device 200, the audio receiver 210 receives an audio signal from a microphone including a microphone and an ADC (Analog Digital Converter). The audio receiving unit 210 is configured to convert an analog audio signal received from a microphone into a digital audio signal through an ADC and output the digital audio signal.

The input unit 220 receives a user input including a button, a touch panel, and the like. The user input through the input unit 220 enables the start of the program used in the song recognition and the lyrics display according to the present invention.

The output unit 230 includes a liquid crystal display (LCD), a light emitting diode (LED) display, and the like, and outputs image data and the like generated by the control unit 260. A plurality of programs can be displayed through the output unit 230 and the lyric data of the received lyric data information can be outputted according to the driving of the user.

The communication unit 240 allows the communication network 300 to transmit and receive data. The communication unit 240 is configured to be capable of data communication with the song recognition server 400.

The communication unit 240 may be connected to a communication network 300 such as a wired LAN or an LTE or wireless LAN, and is configured to output and receive signals according to the communication type.

The storage unit 250 stores various data and programs including a volatile memory and / or a nonvolatile memory. The storage unit 250 includes an application program for performing a connection request to the song recognition server 400 and generating audio data and displaying lyrics accordingly.

The application program may be configured differently depending on the type of the personal device 200 or on the OS of the personal device 200. [ For example, an application may be an application for a so-called smartphone (App).

The control unit 260 controls the blocks in the personal device 200. The control unit 260 may be a processor capable of executing instructions of a program stored in the storage unit 250. [ A processor may be a so-called CPU, a microprocessor unit (MPU), an application processor (AP), etc., including one or more execution units.

The control unit 260 preferably controls other blocks by using various programs stored in the storage unit 250. For example, the control unit 260 can be connected to the remote song recognition server 400 by driving an application program stored in the storage unit 250. The song recognition server 400 receives the lyric data according to the data communication and displays the lyric data through the output unit 230.

The control flow performed by the control unit 260 will be described in detail with reference to FIG.

The connection unit 270 allows data to be transmitted or received between the blocks in the personal device 200. [ The connection unit 270 may be a parallel bus, a serial bus, a single signal line, or a combination of such buses and signal lines.

FIG. 5 is a diagram illustrating an exemplary control flow performed in the personal device 200. FIG. The control flow of FIG. 5 is performed by controlling the other blocks by the control unit 260, and is preferably performed by using an application program stored in the storage unit 250.

The control flow of FIG. 5 may be started (S300) by driving an application program designated as a user input through the input unit 220 and may be terminated (S400) according to a termination request through a user input.

After the start, the control unit 260 makes a connection request to the remote song recognition server 400 through the communication unit 240 and sets a communication channel through the connection.

In step S301, the audio receiving unit 210 converts the analog audio signal from the microphone into a digital audio signal. More specifically, the microphone converts the sound waves in the air into an electrical signal, and the analog audio signal converted into an electrical signal is converted into a digital audio signal through the ADC. Then, the digital audio signal is transmitted to the control unit 260 through the designated bus (I2S, etc.).

The digital audio signal generated in step S301 may be a signal output continuously in time.

In step S303, the control unit 260 receives the digital audio signal from the audio receiving unit 210 and generates audio data based on the received digital audio signal. The audio data may be data representing one measure identified from successive digital audio signals or may be data of an audio signal continuously received for a certain period of time.

As an example of a method of identifying data indicating one measure, the control unit 260 may detect a predetermined time (for example, one second) from the point where no speech is detected for a predetermined time using a voice detection in a digital audio signal The data of the audio signal up to the point in time when the audio signal is not detected can be simply identified by one measure.

The control unit 260 may compress the digital audio signal in a format agreed with the remote song recognition server 400 to generate audio data. Such compression may be a compressed format used for music or speech.

In step S305, the control unit 260 transmits the song data information including the generated audio data to the connected song recognition server 400 through the communication unit 240. [ The song data information may be configured in a format agreed with the song recognition server 400 and may further include various information in addition to audio data.

Thereafter, in step S307, the control unit 260 receives the lyrics data information corresponding to the transmitted song data information from the song recognition server 400. [ The received lyric data information includes the lyric information of the song recognized through the microphone.

Specifically, the lyric data information includes a plurality of measure data indicating the total lyrics of the recognized music piece and time information related to each measure data. In addition, the lyric data information may include matching information indicating measure data matched with the audio data transmitted in step S303. The matching information is used to determine what words to emphasize when the lyrics are displayed through the output unit 230 under the control of the control unit 260. [

Then, in step S309, the control unit 260 outputs the lyric data included in the lyric data information through the output unit 230 and displays it on an output screen composed of an LCD, an LED, or the like. Where the output screen represents the LCD or LED display itself, or represents a certain area within the display that is available for displaying the lyrics by an application program, or an entire area of the display that is controllable by the application.

Specifically, the control unit 260 selects one or more measure data to be displayed at present from a plurality of measure data. This selection can be determined based on the matching information included in the lyric data information. The plurality of measure data may represent the entire lyrics and the display of the personal device 200 may not be able to display all the lyrics due to visibility. Accordingly, it is necessary to configure such that only a part of the lyrics is displayed at a specific time, and other words are displayed at a subsequent time.

Then, the control unit 260 may determine a particular one of the one or more measure data selected based on the matching information as measure data corresponding to the audio signal input through the microphone. This measure data is emphasized so that the audio signals of the lyrics and songs can be synchronized.

Accordingly, the control unit 260 converts one or more measure data from the plurality of measure data into image data, displays it through the output unit 230, and emphasizes a certain one measure data among the one or more measure data Or the like) and displays it through the output unit 230.

This process may be performed upon receipt of the housework data information, and may be performed by changing only the image of the modifying data to be changed and emphasized over time. The change of the measure data to be emphasized is performed using the time information of each measure data included in the song data information.

On the other hand, the matching information indicating the bar data to be matched in response to the audio data transmitted in step S305 may be information considering the network delay time and the processing time in the song recognition server 400, .

If the delay time and the elapsed time in the song recognition server 400 are considered, the control unit 260 identifies the corresponding measure data from the matching information, and indicates one or more measure data including the identified measure data, And output image data for emphasizing the decoded measure data through the output unit 230.

If the delay time and the elapsed time are not considered, the control unit 260 can calculate this time by itself.

For example, the control unit 260 can calculate itself by using the time at which the audio signal is received via the microphone. The control unit 260 records the reception time of the digital audio signal used for generating audio data in the storage unit 250 in step S303. The reception time stored as described above may include a reception start time, a reception end time, or a reception time.

Thereafter, the control unit 260 also stores the time at which the ladder data information was received in the storage unit 250. [ Then, a difference value between the time of receiving the lyric data information and the reception time (reception start time or reception end time) of the audio signal is calculated. The control unit 260 can determine the measure data to be emphasized using the difference value and the matching measure data of the matching information included in the lyric data information.

For example, when the difference value exceeds the performance time of the time information of the measure data of the received matching information, it may be determined as the measure data to be emphasized on the measure data which is played after the measure data and is determined according to the difference value. Alternatively, even if the difference value is within the performance time, it can be determined as the measure data for emphasizing the subsequent measure data in the case where the difference value is equal to or greater than a certain ratio according to the ratio of the difference value and the performance time.

The measure data to be emphasized in this way can be dynamically determined by the remote song recognition server 400 or by the personal device 200 in various ways. And the measure data to be emphasized is determined in consideration of the elapsed time and the network delay time in the song recognition server 400. This enables dynamic synchronization of songs and lyrics in real time.

On the other hand, in step S309, the control unit 260 may change one or more measure data to be emphasized or measure data to be emphasized according to the passage of time determined within the personal device 200. [

In this process, the controller 260 uses the time information of each measure data included in the lyric data information.

For example, the control unit 260 selects one or more measure data according to the reception of the ladder data information, determines the measure data to be emphasized, and displays the measure data on the output screen through the output unit 230.

Thereafter, the control unit 260 measures a time lapse using an internal timer (not shown) and determines whether the elapsed time after display exceeds the playing time of the measure data to be emphasized. If so, the measure data to be emphasized will be determined as the following measure data, emphasizing the measure data and eliminating the emphasis of the previous measure data.

If the last measure data of the selected one or more measure data is emphasized, the controller 260 selects one or more measure data subsequent to the one or more measure data and displays the selected one or more measure data to be displayed on the output screen, ) Controls the output unit (230).

And steps S301 to S309 may be repeatedly performed at a predetermined period. For example, it is possible to transmit the song data information to the song recognition server 400 again after a predetermined time elapses. Or when the song recognition server 400 does not correctly recognize the bar data to be matched, the song data information can be repeatedly transmitted until it is possible to recognize the bar data to be matched.

On the other hand, in step 309, it is described that the measure data to be emphasized as time elapses is changed to another measure and displayed. This description allows scrolling of the measure data to be emphasized. In other words, when several measure data is displayed on the output screen, the output position of the measure data is fixed and the measure data to be emphasized is selected and highlighted.

You can also display the lyrics in various ways. For example, the location of the measure data to be emphasized may be altered in one or more measure data to be displayed over time in a fixed state on the output screen.

That is, there may be a method in which the controller 260 changes one or more measure data to be displayed on the output screen every time the measure data to be emphasized is changed over time, and the changed one or more measure data is displayed again on the output screen . Such a configuration induces a change in the display position of existing measure data (e.g., scroll-up or scroll-down) and fixes the position of the highlighted measure data that is synchronized with the current song.

In this way, the control unit 260 can synchronize the lyrics and music in real time, and can acquire various information as well as the lyrics related to the songs heard from the sound source 100 such as a sound device.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. The present invention is not limited to the drawings.

100: Source source 200: Personal device
210: Audio receiving unit 220: Input unit
230: output unit 240: communication unit
250: storage unit 260: control unit
270:
300: communication network 400: song recognition server
410: communication unit 420: storage unit
430: control unit 440:

Claims (18)

CLAIMS 1. A song lyrics automatic display method,
(a) receiving song data information corresponding to an audio signal input through a microphone;
(b) identifying a corresponding song from the received song data information;
(c) generating lyric data information including lyric data corresponding to the identified tune; And
(d) transmitting the lyric data information,
Wherein the step (c) comprises the steps of: generating lyrics data including a plurality of measure data corresponding to the identified music, time information of each measure data, and information indicating measure data matching the audio data of the song data information among a plurality of measure data Generating lyric data information,
Automatic display method of song lyrics. The method according to claim 1,
The step (b) includes the steps of: (b-1) separating the audio data and the melody data from the audio data of the song data information; And (b-2) identifying the song using the separated voice data and the separated melody data.
Automatic display method of song lyrics. 3. The method of claim 2,
(B-2) selecting a plurality of songs using the separated melody data; And determining one or more songs using the lyrics recognized from the separated voice data in the selected plurality of songs.
Automatic display method of song lyrics. delete The method according to claim 1,
The steps (a) to (d) are performed in a server for recognizing song lyrics,
The method of automatic song lyrics display may further comprise: (e) receiving the lyric data information by a personal device connected to the server; And (f) displaying lyric data of the lyric data information on an output screen.
Automatic display method of song lyrics. 6. The method of claim 5,
Wherein the step (f) comprises: displaying one or more measure data of a plurality of measure data included in the lyric data on an output screen; And highlighting the measure data determined based on the matched measure data,
Wherein the emphasized measure data is changed using time information of each measure data as time elapses,
Automatic display method of song lyrics. The method according to claim 1,
Wherein the information indicating the matched measure data is determined by using an elapsed time according to a comparison between a plurality of measure data and voice data separated from audio data of the song data information and a network delay time between the server and the personal apparatus,
Automatic display method of song lyrics. The method according to claim 6,
And recording, by the personal device, the time at which the audio signal was received via the microphone,
Wherein the step of emphasizing the measure data of the step (f) comprises the steps of determining the measure data to be emphasized by using the difference between the reception time of the lyric data information and the recorded time and the information indicating the matched measure data,
Automatic display method of song lyrics. 6. The method of claim 5,
Converting, by the personal device, an analog signal received from the microphone into a digital audio signal; Generating audio data from the converted digital audio signal; And transmitting the song data information including the generated audio data to the server,
The audio data is data representing a measure of the music or data received during a predetermined time,
Wherein the step of transmitting the song data information to the server is repeatedly performed,
Automatic display method of song lyrics. A server for recognizing song lyrics,
A communication unit that receives song data information corresponding to an audio signal input through a microphone and transmits lyric data information;
A control unit for identifying the corresponding music piece from the song data information received through the communication unit, generating the lyric data information including the lyric data corresponding to the identified music piece, and transmitting the generated lyric data information to the communication unit; And
And a storage unit including a tune DB and a lyrics DB,
Wherein the controller extracts a plurality of measure data corresponding to the identified music piece and time information of each measure data from the lyrics DB and determines measure data to be matched with the audio data of the song data information among a plurality of measure data,
Wherein the control unit generates lyric data information including information indicating a plurality of pieces of measure data and time data of each piece of measure data,
server. 11. The method of claim 10,
Wherein the control unit separates the audio data and the melody data from the audio data of the song data information and identifies the music using the separated melody data and the separated audio data.
server. 12. The method of claim 11,
And a storage unit including a tune DB and a lyrics DB,
Wherein the control unit selects a plurality of songs by comparing the melody data separated from the melody data stored in the music DB and compares the lyrics recognized from the separated voice data with the lyrics stored in the lyrics DB for the selected plurality of songs Determining one or more songs,
server. delete delete delete A communication unit that receives song data information corresponding to an audio signal input through a microphone and transmits lyric data information; And a control unit for identifying the corresponding song from the song data information received through the communication unit and for generating the lyric data information including the lyric data corresponding to the identified song and transmitting the generated lyric data information to the communication unit server; And
And a personal device connected to the server,
The personal device includes: a communication unit for receiving the lyric data information; And an output unit for outputting lyric data of the lyric data information; And a control unit for controlling the output unit,
Wherein the controller of the personal device transmits image data for emphasizing the measure data determined from among the one or more measure data to at least one of the plurality of measure data included in the lyric data information to the output unit,
Song lyrics automatic display system. 17. The method of claim 16,
The personal device comprising: a storage for storing data; And an audio receiver for receiving an audio signal from the microphone,
The control unit of the personal apparatus stores the time at which the audio signal was received in the storage unit and uses the difference between the reception time of the lyric data information received through the communication unit and the time stored in the storage unit, To determine,
Song lyrics automatic display system. 18. The method of claim 17,
The audio receiver converts an audio signal received from a microphone into a digital audio signal,
The control unit of the personal device generates audio data from the converted digital audio signal and transmits the song data information including the generated audio data to the communication unit so that the song data information is transmitted to the server,
Wherein the audio data is data representing a measure of the music or data received via the audio receiver for a predetermined time,
Wherein the song data information is repeatedly transmitted,
Song lyrics automatic display system.

Images