CN106228986A - The automated testing method of a kind of speech recognition engine, device and system - Google Patents

Abstract

The invention discloses an automatic testing method, device and system of a speech recognition engine. The method includes: playing the current test voice signal, making the specified terminal equipment as the receiving end receive the voice signal and send it to the voice recognition engine and obtain the voice recognition result from the voice recognition engine; Speech recognition result information corresponding to the speech signal; verifying the speech recognition result information to implement a speech recognition test for the speech recognition engine. In summary, the present invention plays the voice signal in the voice library that needs to be detected, and the designated device as the receiving end will send the voice signal to the voice recognition engine, receive the returned recognition result, complete the verification of the voice recognition result, and automatically According to the verification results, it is judged whether to proceed to the next test, and the automatic test of the speech recognition engine is realized, eliminating the time-consuming and labor-intensive manual test.

Description

An automated testing method, device and system for a speech recognition engine

technical field

The invention relates to the technical field of testing, in particular to an automatic testing method, device and system for a speech recognition engine.

Background technique

With the advancement of science and technology, machines and electronic equipment are becoming more and more intelligent, especially the development of voice recognition technology, which enables people to communicate with machines and equipment through voice, so that machines and equipment can understand what you say. In the past two decades, speech recognition technology has made remarkable progress, and it has begun to move from the laboratory to the market. It is expected that in the next 10 years, speech recognition technology will enter industries, home appliances, communications, automotive electronics, medical care, home services, and consumer electronics products. and other fields. However, the popularization of this technology is inseparable from a speech recognition engine with excellent performance. The speech recognition engine must realize the recognition of such different speeches, be able to recognize sounds of different speeches, different semantics, and even different accents, and its recognition results meet the recognition standards; And it also needs the support of a huge voice database, which can accommodate a variety of voices. Therefore, it is impossible to detect such a speech recognition engine with excellent performance with a huge speech database through traditional manual detection methods. How to realize the automatic monitoring of speech recognition engine has become an urgent problem to be solved.

Contents of the invention

In view of the above problems, the present invention is proposed to provide an automated testing method, device and system for a speech recognition engine that overcomes the above problems or at least partially solves the above problems.

According to one aspect of the present invention, a kind of automatic testing method of speech recognition engine is provided, wherein, this method comprises:

Play the current voice signal for testing, so that the designated terminal device as the receiving end receives the voice signal and sends it to the voice recognition engine and obtains the voice recognition result from the voice recognition engine;

receiving voice recognition result information corresponding to the current test voice signal output by the designated terminal device;

The speech recognition result information is verified to realize the speech recognition test of the speech recognition engine.

Optionally, the method further includes:

Judging according to the verification result, whether the recognition of the current test voice signal reaches the preset standard, if so, playing the next test voice signal, otherwise playing the current test voice signal again.

Optionally, the method further includes:

When the number of times the recognition of the current voice signal for testing fails to meet the preset standard reaches a preset value, the next voice signal for testing is played.

Optionally, the playing the current test voice signal includes:

Traverse each voice signal in the test voice library in turn, and play the currently traversed voice signal as the current test voice signal;

or,

Each text data in the text database is traversed sequentially, and the currently traversed text data is converted into a voice signal by using the text-to-speech TTS technology, and then played as the current test voice signal.

Optionally, the method further includes:

A speech recognition test report of the speech recognition engine is generated according to verification results of the speech recognition information corresponding to each test speech signal.

Optionally, the receiving the speech recognition result information corresponding to the current test speech signal output by the designated terminal device includes:

receiving the voice signal output by the specified terminal device and including the recognition result information through the sound receiving module;

and / or,

The wireless signal including the identification result information output by the designated terminal device is received through the wireless communication module or the network module.

Optionally, the verifying the speech recognition result information includes:

saving the expected identification information of the voice signal for testing in advance;

The speech recognition result information is compared with the expected recognition information corresponding to the current test speech signal.

Optionally, the specified terminal device sends the test voice signal to the voice recognition engine through the voice recognition service interface of the voice recognition engine;

or,

The specified terminal device sends the test voice signal to the voice recognition engine through a specific interface provided by the voice recognition engine.

Optionally, the method further includes:

receiving the semantic recognition result information corresponding to the test voice signal output by the specified terminal device; wherein, the specified terminal device receives the semantic recognition of the test voice signal by the voice recognition engine for the specified terminal device After the operation, generate semantic recognition result information corresponding to the operation;

The semantic recognition result information is verified to realize the semantic recognition test of the speech recognition engine.

According to another aspect of the present invention, a kind of automatic testing device of speech recognition engine is provided, wherein, this device comprises:

The playback unit is adapted to play the current test voice signal, so that the designated terminal device as the receiving end receives the voice signal and sends it to the voice recognition engine and obtains the voice recognition result from the voice recognition engine;

The receiving unit is adapted to receive the speech recognition result information corresponding to the current test speech signal output by the designated terminal device;

The test verification unit is adapted to verify the speech recognition result information, so as to implement the speech recognition test of the speech recognition engine.

Optionally, the device further includes:

The judging unit is adapted to judge according to the verification result of the verification unit, whether the recognition of the current test voice signal reaches a preset standard, if so, notify the playback unit to play the next test voice signal, otherwise notify the The playback unit plays the current voice signal for testing again.

Optionally, the judging unit is further adapted to notify the playing unit to play the next test voice signal when the number of times the recognition of the current test voice signal fails to meet the preset standard reaches a preset value.

Optionally, the playback unit is adapted to sequentially traverse each voice signal in the test voice database, and play the currently traversed voice signal as the current test voice signal; or, is adapted to sequentially traverse each voice signal in the text database. For text data, the currently traversed text data is converted into a voice message by using the text-to-speech TTS technology, and then played as the current test voice signal.

Optionally, the device further includes:

The report generation unit is adapted to generate the speech recognition test report of the speech recognition engine according to the verification result of the speech recognition information corresponding to each test speech signal.

Optionally, the receiving unit is adapted to receive the voice signal output by the designated terminal device through the sound receiving module, including the recognition result information; and/or, receive the voice signal output by the designated terminal device through the wireless communication module or the network module. Wireless signal of recognition result information.

Optionally, the test verification unit is adapted to store the expected recognition information of the test voice signal in advance, and compare the voice recognition result information with the expected recognition information corresponding to the current test voice signal.

Optionally, the receiving unit is further adapted to receive semantic recognition result information corresponding to the test voice signal output by the designated terminal device; After the semantic recognition of the voice signal is performed on the specified terminal equipment, the semantic recognition result information corresponding to the operation is generated;

The test verification unit is further adapted to verify the semantic recognition result information, so as to implement the semantic recognition test of the speech recognition engine.

According to another aspect of the present invention, an automated test system for a speech recognition engine is provided, wherein the system includes a terminal device as a sending end and a terminal device as a receiving end;

The terminal device as the sending end includes the automatic testing device for the speech recognition engine described in any one of the foregoing.

Optionally, the terminal device serving as the receiving end includes:

The receiving unit is adapted to receive the test voice signal sent by the automatic test device of the voice recognition engine;

The speech recognition processing unit is suitable for sending the speech signal for testing to the speech recognition engine and obtaining the speech recognition result from the speech recognition engine;

The output unit is adapted to output speech recognition result information corresponding to the current test speech signal.

Optionally, the speech recognition processing unit is adapted to send the test speech signal to the speech recognition engine through an interface of the speech recognition engine that provides a speech recognition service; or, send the speech signal to the speech recognition engine through a specific interface provided by the speech recognition engine. The test voice signal is sent to the speech recognition engine.

Optionally, the speech recognition processing unit is adapted to generate semantics corresponding to the operation after the terminal device serving as the receiving end receives the operation performed on the terminal device after the speech recognition engine performs semantic recognition on the test speech signal. Identification result information;

The output unit is further adapted to output semantic recognition result information corresponding to the current test speech signal.

It can be seen that the terminal device as the sending end generates the voice library that needs to be detected, and plays the currently traversed voice in the voice library through the voice playback module of the third-party application. After receiving the current voice signal, the designated terminal device as the receiving end plays it. Send it to the speech recognition engine, the speech recognition engine will recognize the speech signal it receives, and return the recognition result to the designated terminal device as the receiving end, and then verify and judge according to the speech recognition result output by the designated terminal device as the receiving end The correctness or accuracy rate of the speech recognition result, the correctness or accuracy rate judged by it can reflect the performance of the current speech recognition engine, realize the test of the speech recognition engine, and judge whether it is correct or not according to the verification result Play the next voice to be detected, or replay the voice, and finally form a test report for the developer to judge whether to further optimize the tested voice and video engine, which realizes the automated test of the voice recognition engine. In summary, the technical solution of the present invention realizes automatic testing of the speech recognition engine by introducing the terminal device as the sending end and the terminal device as the receiving end to test the speech recognition engine, eliminating the time-consuming and labor-intensive manual testing, Make speech recognition engine testing smarter.

The above description is only an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, it can be implemented according to the contents of the description, and in order to make the above and other purposes, features and advantages of the present invention more obvious and understandable , the specific embodiments of the present invention are enumerated below.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiment. The drawings are only for the purpose of illustrating a preferred embodiment and are not to be considered as limiting the invention. Also throughout the drawings, the same reference numerals are used to designate the same components. In the attached picture:

Fig. 1 shows the flow chart of the automated testing method of a kind of speech recognition engine according to an embodiment of the present invention;

Fig. 2 shows a schematic diagram of an automated testing device for a speech recognition engine according to an embodiment of the present invention;

FIG. 3 shows a schematic diagram of an automated testing device for a speech recognition engine according to another embodiment of the present invention;

Fig. 4 shows a schematic diagram of a designated device as a receiving end according to an embodiment of the present invention;

Fig. 5 shows a schematic diagram of an automated test system for a speech recognition engine according to an embodiment of the present invention.

detailed description

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided for more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

FIG. 1 shows a flow chart of an automatic testing method for a speech recognition engine according to an embodiment of the present invention, and illustrates the implementation process of this solution from the side of the terminal device as the sending end. As shown in Figure 1, the method includes:

Step S110, playing the current voice signal for testing, so that the designated terminal device as the receiving end receives the voice signal and sends it to the voice recognition engine and obtains a voice recognition result from the voice recognition engine.

In the embodiment of the present invention, the terminal device as the sending end that plays the current test voice signal includes, but is not limited to, any device that can generate and play voice, such as a mobile phone, a tablet computer, or a desktop computer. First generate the voice library to be detected, and play the currently traversed voice in the voice library through the voice playback module of the third-party application. After the designated terminal device as the receiving end receives the current voice signal, it sends it to the voice recognition Engine, the speech recognition engine will recognize the current speech signal it receives, and return the recognition result to the designated terminal device as the receiving end. Among them, the designated terminal device as the receiving end includes but is not limited to mobile phones, tablet computers, desktop computers and other devices that can receive voice signals, send voice signals to the voice recognition engine and output the recognition results at the same time. When testing the speech recognition engine, there are not only a few voices that need to be detected, but it is necessary to traverse the voices of various situations that it will encounter in future application scenarios as much as possible, so the generated voices that need to be detected Not just a few lines, but thousands or even tens of thousands of lines, which is a voice library that needs to be detected.

Step S120, receiving the speech recognition result information corresponding to the current test speech signal output by the designated terminal device.

In an embodiment of the present invention, after playing the voice for current detection, a voice recognition result corresponding to the voice currently played will be received from the receiving specified terminal device, the voice recognition result is the result information returned by the voice recognition engine, At the same time, the next logical verification will be carried out based on the speech recognition result.

Step S130, verifying the speech recognition result information, so as to realize the speech recognition test of the speech recognition engine.

In the embodiment of the present invention, verify according to the current speech recognition result that it receives, judge the correctness of this speech recognition result, or its accuracy rate, and carry out the next procedure according to judgment result, this speech recognition result is speech The correctness or accuracy rate of the result returned by the recognition engine can reflect the performance of the current speech recognition engine and realize the test of the speech recognition engine. Because a speech library that needs to be detected is generated, it will be verified according to the current speech recognition result it receives, and it will judge whether to play the next speech to be detected or replay the current speech according to the verification result. voice, which realizes the automated testing of the speech recognition engine.

It can be seen that the terminal device as the sending end generates the voice library that needs to be detected, and plays the currently traversed voice in the voice library through the voice playback module of the third-party application. After the designated terminal device as the receiving end receives the current voice signal, it will It is sent to the speech recognition engine, and the speech recognition engine will recognize the speech signal it receives, and return the recognition result to the designated terminal device as the receiving end, and then verify it according to the speech recognition result output by the designated terminal device as the receiving end, Judging the correctness of the speech recognition result, or its correct rate, the correctness or correct rate can reflect the performance of the current speech recognition engine, realize the test of the speech recognition engine, and will automatically Judging whether to play the next voice to be detected, or to replay the voice, and finally form a test report for developers to judge whether to further optimize the voice and video engine for testing, which realizes the automated testing of the voice recognition engine. The technical scheme of the present invention realizes the automated testing of the speech recognition engine by introducing the terminal device as the sending end and the terminal device as the receiving end to test the speech recognition engine, which eliminates the time-consuming and labor-intensive manual testing, and makes the speech recognition engine test Smarter.

In one embodiment of the present invention, the method shown in Fig. 1 further comprises: judging according to the verification result, whether the recognition of the current test voice signal reaches the preset standard, if so, playing the next test voice signal, otherwise The current test audio signal is played again.

During the testing process of the speech recognition engine, while generating the speech bank to be detected, a preset correct result after each speech recognition corresponding to it is also generated, and a preset standard is set, then according to the receiving end The recognition result information output by the specified terminal device is compared with the preset correct result. After the comparison, if the recognition information is consistent with the correct result, or reaches the preset standard, then the next voice will be played automatically. If the difference is large after the comparison, And if the preset standard is not reached, the current voice will be played again for recognition again. For example, when performing a test, the correct result of generating a speech is ten characters, and the preset standard is set to 90%, then when verifying the recognition result, ten characters are returned, and there are The next speech can only be played if the nine characters match the correct result; if the returned result is ten characters, but only eight characters match, then replay, or the returned recognition result is nine characters, that is also To replay the voice.

In one embodiment of the present invention, the method shown in FIG. 1 further includes: playing the next test voice signal when the number of times the recognition of the current test voice signal fails to meet the preset standard reaches a preset value. Because during the test of the speech recognition engine, a speech that the engine cannot recognize is encountered, that is, it cannot be recognized correctly after replaying it many times. If the number of replays is not limited, the program will continue indefinitely. The test process It will not proceed further. Therefore, while generating the correct result of speech recognition and setting the preset standard, it is also necessary to set a preset value to limit the number of replays, so that when the speech recognition engine cannot recognize the current speech, it can also perform other tasks. Voice recognition. For example, after a ten-character voice is generated, the preset standard is 90%, and the default value is 3 times. Then, when verifying the recognition result returned by the speech recognition engine, the result always returns only nine characters. After 3 times, if the recognition result cannot be matched correctly, the current voice will not be replayed, but the next voice will be played, and the voice will be marked as an unrecognizable voice, and will be recorded in the final generated test report .

Because when testing the speech recognition engine, the speech that it needs to detect is not only a small number, but also needs to traverse the speech of various situations that it will encounter in future application scenarios as much as possible, so the generated speech that needs to be detected Voice is not just a few lines, it may be thousands of lines, it is a voice library that needs to be detected. In one embodiment of the present invention, playing the current test voice signal in step S110 shown in Fig. 1 comprises: traversing each voice signal in the test voice library in turn, using the current voice signal traversed as the current test voice signal The audio signal is played.

Or, the voice signal in the voice bank that it generates is not the form of audio frequency, but text database, so just need to introduce the application of a third party, promptly can carry out the speech synthesis software (TTS, Text toSpeech) of text-to-speech, will The current text data in the text database is converted into a voice signal through TTS and then played through the voice playback module. In one embodiment of the present invention, playing the current voice signal for testing in step S110 shown in FIG. 1 includes: sequentially traversing each text data in the text database, converting the currently traversed text data using text-to-speech TTS technology After being converted into a voice signal, it will be played as the current test voice signal.

In an embodiment of the present invention, the method shown in FIG. 1 further includes: generating a speech recognition test report of the speech recognition engine according to verification results of the speech recognition information corresponding to each test speech signal. After the speech recognition engine is tested, a test result report will be generated, which shows the total recognition accuracy of the speech in the test speech database, the accuracy of each speech recognition result, and the unrecognized voice recording. The detection result will show the performance of the current speech recognition engine, and it will be convenient for developers to find unrecognized speech, so as to optimize the speech recognition engine.

In the embodiment of the present invention, the speech recognition result information corresponding to the current test speech signal output by the designated terminal device may be a speech signal or a wireless signal. Then, in one embodiment of the present invention, in step S120 of the method shown in FIG. 1 , receiving the voice recognition result information corresponding to the current test voice signal output by the specified terminal device includes: through a sound collection module, such as a microphone, receiving The voice signal containing the recognition result information output by the designated terminal device; and/or, through the wireless communication module or network module, such as Bluetooth module, WiFi module, etc., receiving the wireless signal containing the recognition result information output by the designated terminal device.

In one embodiment of the present invention, verifying the speech recognition result information in step S130 of the method shown in FIG. 1 includes: pre-saving the expected recognition information of the speech signal for testing; The expected identification information corresponding to the speech signal is compared. Wherein, the expected recognition information is the correct result of the aforementioned speech, and each piece of speech or text data in the test speech database has its corresponding expected recognition result.

In one embodiment of the present invention, the designated terminal device as the receiving end will install a service software corresponding to the speech recognition engine in advance to realize the provision of speech signals to the speech recognition engine, specifically providing speech recognition services through the speech recognition engine The interface of the test will send the speech signal to the speech recognition engine.

Alternatively, the speech recognition engine as a test version will provide a specific interface, and the designated terminal device as the receiving end will send the speech signal for testing to the speech recognition engine through the specific interface provided by the speech recognition engine.

In the process of voice dialogue with equipment, machines, etc., not only the voice itself is involved, but also the voice with command meaning is involved, and the machine or device needs to perform corresponding operations according to the voice with command meaning, that is, semantics identify. Then in one embodiment of the present invention, the method shown in Figure 1 further includes: receiving the semantic recognition result information corresponding to the voice signal for testing output by the specified terminal equipment; wherein, the specified terminal equipment receives the voice recognition engine for testing After the semantic recognition of the signal is performed on the specified terminal equipment, the semantic recognition result information corresponding to the operation is generated and output; the semantic recognition result information is verified to realize the semantic recognition test of the speech recognition engine.

In a specific example, the current test voice signal played is "turn on bluetooth", then the recognition result information of the test voice signal "turn on bluetooth" by the voice recognition engine receiving the output of the specified terminal device includes: whether the voice result It is the four words "turn on bluetooth", and whether the semantic recognition engine has performed the operation of "turn on bluetooth" as the designated terminal device. When verifying the recognition result, it is necessary not only to match the returned voice result, but also to verify whether the operation of "turning on Bluetooth" is performed. If both match, the next one will be played; if the returned semantic recognition result information is "Turn on Bluetooth", but it does not contain the information that the operation of "Turn on Bluetooth" has been performed, then play again. Wherein, when the test voice of "turn on Bluetooth" is generated, the expected recognition result and the corresponding expected operation result are also preset, and both are verified during verification.

In another specific example, the current test voice signal is played as "How are you?", and when the test voice signal "How are you?" is generated, the expected recognition result will also be preset. If the result is "I'm fine" or "I'm not good", then the recognition result information of the voice signal "How are you?" for the test voice signal "How are you?" by the voice recognition engine receiving the output of the designated terminal device includes: whether the voice result is "I'm fine" " or "I'm not good", if it matches, the next one will be played; if it doesn't match, it will be played again.

Fig. 2 shows a schematic diagram of an automatic testing device for a speech recognition engine according to an embodiment of the present invention. As shown in Figure 2, the automatic testing device 200 of this speech recognition engine comprises:

The playing unit 210 is adapted to play the current test voice signal, so that the designated terminal device as the receiving end receives the voice signal and sends it to the voice recognition engine and obtains a voice recognition result from the voice recognition engine.

The receiving unit 220 is adapted to receive speech recognition result information corresponding to the current test speech signal output by the designated terminal device.

The test verification unit 230 is adapted to verify the speech recognition result information, so as to implement the speech recognition test of the speech recognition engine.

It can be seen that the terminal device as the sending end generates the voice library that needs to be detected, and plays the currently traversed voice in the voice library through the voice playback module of the third-party application. After receiving the current voice signal, the designated terminal device as the receiving end plays it. Send it to the speech recognition engine, the speech recognition engine will recognize the speech signal it receives, and return the recognition result to the designated terminal device as the receiving end, and then verify and judge according to the speech recognition result output by the designated terminal device as the receiving end The correctness or accuracy rate of the speech recognition result, the correctness or accuracy rate judged by it can reflect the performance of the current speech recognition engine, realize the test of the speech recognition engine, and judge whether it is correct or not according to the verification result Play the next voice to be detected, or replay the voice, and finally form a test report for the developer to judge whether to further optimize the tested voice and video engine, which realizes the automated test of the voice recognition engine. In summary, the technical solution of the present invention realizes the automated testing of the speech recognition engine by introducing the terminal device as the sending end and the terminal device as the receiving end to test the speech recognition engine, thereby eliminating the time-consuming and labor-intensive manual testing, and making the speech recognition engine Test smarter.

Fig. 3 shows a schematic diagram of an automatic testing device for a speech recognition engine according to another embodiment of the present invention. As shown in FIG. 3 , the automatic testing device 300 of the speech recognition engine includes: a playback unit 310 , a receiving unit 320 , a test verification unit 330 , a judging unit 340 and a report generating unit 350 . Wherein, the playback unit 310, the reception unit 320, the test verification unit 330 and the playback unit 210, the reception unit 220, the test verification unit 230 of the apparatus shown in FIG.

The judging unit 340 is suitable for judging according to the verification result of the verification unit, whether the recognition of the current test voice signal reaches the preset standard, if so, then notify the playback unit to play the next test voice signal, otherwise notify the playback unit to play the current test voice signal again. audio signal for testing.

The report generation unit 350 is adapted to generate the speech recognition test report of the speech recognition engine according to the verification result of the speech recognition information corresponding to each test speech signal. The test report shows the total recognition accuracy rate of the speech in the test speech library, the accuracy rate of each speech recognition result, and the record of unrecognized speech. The result will show the performance of the current speech recognition engine, and facilitate developers to find unrecognizable speech in order to optimize the speech recognition engine.

In an embodiment of the present invention, the judging unit 340 is further adapted to notify the playback unit to play the next test voice signal when the number of times the recognition of the current test voice signal fails to meet the preset standard reaches a preset value.

In one embodiment of the present invention, the playback unit 310 is adapted to sequentially traverse each speech signal in the test speech library, and plays the currently traversed speech signal as the current test speech signal; or, is suitable for sequentially traversing the text For each text data in the database, the currently traversed text data is converted into a voice message by using the text-to-speech TTS technology, and then played as the current test voice signal.

In one embodiment of the present invention, the receiving unit 320 is adapted to receive the voice signal containing the recognition result information output by the designated terminal device through the sound receiving module, including a microphone, etc.; and/or, through the wireless communication module or network module, including The bluetooth module, the WiFi module, etc. receive the wireless signal output by the specified terminal device that includes the identification result information.

In one embodiment of the present invention, the test verification unit 330 is adapted to pre-save the expected recognition information of the test voice signal, and compare the voice recognition result information with the expected recognition information corresponding to the current test voice signal.

In one embodiment of the present invention, the receiving unit 310 is further adapted to receive the semantic recognition result information corresponding to the test voice signal output by the designated terminal device; After the operation performed on the specified terminal device is identified, semantic recognition result information corresponding to the operation is generated.

The test verification unit 330 is further adapted to verify the semantic recognition result information, so as to implement the semantic recognition test of the speech recognition engine.

It should be noted that the embodiments of the apparatus shown in FIGS. 2-3 are correspondingly the same as the embodiments of the method shown in FIG. 1 , which have been described in detail above and will not be repeated here.

Fig. 4 shows a schematic diagram of a designated device as a receiving end according to an embodiment of the present invention. As shown in Figure 4, the specified device 400 as the receiving end includes:

The receiving unit 410 is adapted to receive the test voice signal sent by the automatic test device of the voice recognition engine.

The voice recognition processing unit 420 is adapted to send the test voice signal to the voice recognition engine and obtain a voice recognition result from the voice recognition engine.

The output unit 430 is adapted to output speech recognition result information corresponding to the current test speech signal. The result information can be a voice signal or a wireless signal, which will be output to the terminal device as the sending end through the voice playback module, bluetooth module, WiFi module, etc.

In one embodiment of the present invention, the voice recognition processing unit 420 is adapted to send the test voice signal to the voice recognition engine through the interface of the voice recognition engine that provides voice recognition services; The test voice signal is sent to the speech recognition engine.

In one embodiment of the present invention, the speech recognition processing unit 420 is adapted to generate a corresponding response to the operation after the terminal device as the receiving end accepts the operation performed on the terminal device after the speech recognition engine performs semantic recognition on the test speech signal. Semantic recognition result information.

Then the output unit 430 is further adapted to output semantic recognition result information corresponding to the current test speech signal.

It should be noted that the embodiments of the apparatus shown in FIG. 4 are correspondingly the same as the embodiments of the method shown in FIG. 1 , which have been described in detail above and will not be repeated here.

Fig. 5 shows a schematic diagram of an automated test system for a speech recognition engine according to an embodiment of the present invention. As shown in FIG. 5 , the automatic test system 500 of the speech recognition engine includes a terminal device 510 as a sending end and a terminal device 520 as a receiving end.

The terminal device 510 as the sending end has the corresponding same function as the automatic test device 200/300 of the speech recognition engine above, and the terminal device 520 as the receiving end has the same function as the specified device 400 as the receiving end above, and the same parts I won't repeat them here.

In the embodiment of the present invention, the automatic test system of the speech recognition engine introduces two kinds of devices as the terminal device of the sending end and the terminal device as the receiving end, respectively, as the sending end to send the voice and verify the speech recognition result, and as the receiving end to receive Speech and output speech recognition results to ensure that various speech recognition results that may appear in the speech recognition engine can be traversed to realize the automation of speech recognition engine testing.

It can be seen that the device as the sending end generates the voice library that needs to be detected, and plays the currently traversed voice in the voice library through the voice playback module of the third-party application. After receiving the current voice signal, the designated terminal device as the receiving end sends it to to the speech recognition engine, the speech recognition engine will recognize the speech signal it receives, and return the recognition result to the designated terminal device as the receiving end, and then verify it according to the speech recognition result output by the designated terminal device as the receiving end, and judge the The correctness of the speech recognition result, or its accuracy rate, the correctness or accuracy rate can reflect the performance of the current speech recognition engine, realize the test of the speech recognition engine, and judge whether it is Play the next voice to be detected, or replay the voice, and finally form a test report for the developer to judge whether to further optimize the tested voice and video engine, which realizes the automated test of the voice recognition engine. The technical scheme of the present invention realizes the automated testing of the speech recognition engine by introducing the terminal device as the sending end and the terminal device as the receiving end to test the speech recognition engine, which eliminates the time-consuming and labor-intensive manual testing, and makes the speech recognition engine test Smarter.

It should be noted:

The algorithms and displays presented herein are not inherently related to any particular computer, virtual appliance, or other device. Various general purpose devices can also be used with the teachings based on this. The structure required to construct such an apparatus will be apparent from the foregoing description. Furthermore, the present invention is not specific to any particular programming language. It should be understood that various programming languages can be used to implement the contents of the present invention described herein, and the above description of specific languages is for disclosing the best mode of the present invention.

In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure the understanding of this description.

Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, in order to streamline this disclosure and to facilitate an understanding of one or more of the various inventive aspects, various features of the invention are sometimes grouped together in a single embodiment, figure, or its description. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art can understand that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. Modules or units or components in the embodiments may be combined into one module or unit or component, and furthermore may be divided into a plurality of sub-modules or sub-units or sub-assemblies. All features disclosed in this specification (including accompanying claims, abstract and drawings) and any method or method so disclosed may be used in any combination, except that at least some of such features and/or processes or units are mutually exclusive. All processes or units of equipment are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, those skilled in the art will understand that although some embodiments described herein include some features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of the invention. and form different embodiments. For example, in the following claims, any of the claimed embodiments may be used in any combination.

The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should be understood that can use microprocessor or digital signal processor (DSP) to realize some or some or all parts in the automatic testing device and system of speech recognition engine according to the embodiment of the present invention in practice Full functionality. The present invention can also be implemented as an apparatus or an apparatus program (for example, a computer program and a computer program product) for performing a part or all of the methods described herein. Such a program for realizing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet site, or provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. does not indicate any order. These words can be interpreted as names.

The invention discloses A1, an automated testing method for a speech recognition engine, wherein the method includes:

Play the current voice signal for testing, so that the designated terminal device as the receiving end receives the voice signal and sends it to the voice recognition engine and obtains the voice recognition result from the voice recognition engine;

receiving voice recognition result information corresponding to the current test voice signal output by the designated terminal device;

The speech recognition result information is verified to implement the speech recognition test of the speech recognition engine.

A2. The method as described in A1, wherein the method further comprises:

Judging according to the verification result, whether the recognition of the current test voice signal reaches the preset standard, if so, playing the next test voice signal, otherwise playing the current test voice signal again.

A3, the method as described in A2, wherein, the method further comprises:

When the number of times the recognition of the current voice signal for testing fails to meet the preset standard reaches a preset value, the next voice signal for testing is played.

A4, the method as described in A1, wherein, described playing current test voice signal comprises:

Traverse each voice signal in the test voice library in turn, and play the currently traversed voice signal as the current test voice signal;

or,

Each text data in the text database is traversed sequentially, and the currently traversed text data is converted into a voice signal by using the text-to-speech TTS technology, and then played as the current test voice signal.

A5. The method as described in A4, wherein the method further comprises:

A speech recognition test report of the speech recognition engine is generated according to verification results of the speech recognition information corresponding to each test speech signal.

A6. The method as described in A1, wherein said receiving the speech recognition result information corresponding to the current test speech signal output by the designated terminal device includes:

receiving the voice signal output by the specified terminal device and including the recognition result information through the sound receiving module;

and / or,

The wireless signal including the identification result information output by the designated terminal device is received through the wireless communication module or the network module.

A7. The method as described in A1, wherein the verifying the speech recognition result information includes:

saving the expected identification information of the voice signal for testing in advance;

The speech recognition result information is compared with the expected recognition information corresponding to the current test speech signal.

A8. The method of any one of A1-A7, wherein,

The designated terminal device sends the test voice signal to the voice recognition engine through the voice recognition service interface of the voice recognition engine;

or,

The specified terminal device sends the test voice signal to the voice recognition engine through a specific interface provided by the voice recognition engine.

A9. The method according to any one of A1-A7, wherein the method further comprises:

receiving the semantic recognition result information corresponding to the test voice signal output by the specified terminal device; wherein, the specified terminal device receives the semantic recognition of the test voice signal by the voice recognition engine for the specified terminal device After the operation, generate semantic recognition result information corresponding to the operation;

The semantic recognition result information is verified to realize the semantic recognition test of the speech recognition engine.

The present invention also discloses B10, an automated testing device for a speech recognition engine, wherein the device includes:

The playback unit is adapted to play the current test voice signal, so that the designated terminal device as the receiving end receives the voice signal and sends it to the voice recognition engine and obtains the voice recognition result from the voice recognition engine;

The receiving unit is adapted to receive the speech recognition result information corresponding to the current test speech signal output by the designated terminal device;

The test verification unit is adapted to verify the speech recognition result information, so as to implement the speech recognition test of the speech recognition engine.

B11. The device as described in B10, wherein the device further comprises:

The judging unit is adapted to judge according to the verification result of the verification unit, whether the recognition of the current test voice signal reaches a preset standard, if so, notify the playback unit to play the next test voice signal, otherwise notify the The playback unit plays the current voice signal for testing again.

B12. The device of B11, wherein,

The judging unit is further adapted to notify the playback unit to play the next test voice signal when the number of times the recognition of the current test voice signal fails to meet the preset standard reaches a preset value.

B13. The device of B10, wherein,

The playback unit is suitable for sequentially traversing each voice signal in the test voice database, playing the currently traversed voice signal as a current test voice signal; or, being suitable for sequentially traversing each text data in the text database, and The currently traversed text data is converted into a voice message using the text-to-speech TTS technology and then played as the current test voice signal.

B14. The device as described in B13, wherein the device further comprises:

The report generation unit is adapted to generate the speech recognition test report of the speech recognition engine according to the verification result of the speech recognition information corresponding to each test speech signal.

B15. The device of B10, wherein,

The receiving unit is adapted to receive the voice signal containing the recognition result information output by the specified terminal device through the sound receiving module; and/or receive the voice signal containing the recognition result information output by the specified terminal device through the wireless communication module or the network module. wireless signal.

B16. The device of B10, wherein,

The test verification unit is adapted to store expected recognition information of the test voice signal in advance, and compare the voice recognition result information with the expected recognition information corresponding to the current test voice signal.

B17. The device of any one of B10-B16, wherein,

The receiving unit is further adapted to receive semantic recognition result information corresponding to the test voice signal output by the specified terminal device; wherein, the specified terminal device receives the voice recognition engine to perform semantic recognition on the test voice signal After identifying the operation performed on the designated terminal device, generate semantic recognition result information corresponding to the operation;

The test verification unit is further adapted to verify the semantic recognition result information, so as to implement the semantic recognition test of the speech recognition engine.

The present invention also discloses C18, an automatic test system for a speech recognition engine, wherein the system includes a terminal device as a sending end and a terminal device as a receiving end;

The terminal device as the sending end includes the automatic testing device for the speech recognition engine described in any one of B10-B17.

C19. The system as described in C18, wherein the terminal device as the receiving end includes:

The receiving unit is adapted to receive the test voice signal sent by the automatic test device of the voice recognition engine;

The speech recognition processing unit is suitable for sending the speech signal for testing to the speech recognition engine and obtaining the speech recognition result from the speech recognition engine;

The output unit is adapted to output speech recognition result information corresponding to the current test speech signal.

C20. The system of C19, wherein,

The speech recognition processing unit is adapted to send the speech signal for testing to the speech recognition engine through the interface of the speech recognition engine that provides speech recognition services; or send the speech signal for testing to the speech recognition engine through a specific interface provided by the speech recognition engine sent to the speech recognition engine.

C21. The system of C19, wherein,

The speech recognition processing unit is adapted to generate semantic recognition result information corresponding to the operation after the terminal device as the receiving end receives the operation performed on the terminal device after the speech recognition engine performs semantic recognition of the test voice signal;

The output unit is further adapted to output semantic recognition result information corresponding to the current test speech signal.

Claims (10)

1. A method for automated testing of a speech recognition engine, wherein the method comprises: Play the current voice signal for testing, so that the designated terminal device as the receiving end receives the voice signal and sends it to the voice recognition engine and obtains the voice recognition result from the voice recognition engine; receiving voice recognition result information corresponding to the current test voice signal output by the designated terminal device; The speech recognition result information is verified to implement the speech recognition test of the speech recognition engine. 2. The method of claim 1, wherein the method further comprises: Judging according to the verification result, whether the recognition of the current test voice signal reaches the preset standard, if so, playing the next test voice signal, otherwise playing the current test voice signal again. 3. The method of claim 2, wherein the method further comprises: When the number of times the recognition of the current voice signal for testing fails to meet the preset standard reaches a preset value, the next voice signal for testing is played. 4. method as claimed in claim 1, wherein, the current test speech signal of described playing comprises: Traverse each voice signal in the test voice library in turn, and play the currently traversed voice signal as the current test voice signal; or, Each text data in the text database is traversed sequentially, and the currently traversed text data is converted into a voice signal by using the text-to-speech TTS technology, and then played as the current test voice signal. 5. An automated testing device for a speech recognition engine, wherein the device comprises: The playback unit is adapted to play the current test voice signal, so that the designated terminal device as the receiving end receives the voice signal and sends it to the voice recognition engine and obtains the voice recognition result from the voice recognition engine; The receiving unit is adapted to receive the speech recognition result information corresponding to the current test speech signal output by the designated terminal device; The test verification unit is adapted to verify the speech recognition result information, so as to implement the speech recognition test of the speech recognition engine. 6. The device of claim 5, wherein the device further comprises: The judging unit is adapted to judge according to the verification result of the verification unit, whether the recognition of the current test voice signal reaches a preset standard, if so, notify the playback unit to play the next test voice signal, otherwise notify the The playback unit plays the current voice signal for testing again. 7. The apparatus of claim 6, wherein, The judging unit is further adapted to notify the playback unit to play the next test voice signal when the number of times the recognition of the current test voice signal fails to meet the preset standard reaches a preset value. 8. An automated test system for a speech recognition engine, wherein the system includes a terminal device as a sending end and a terminal device as a receiving end; The terminal device as the sending end includes the automatic testing device for the speech recognition engine according to any one of claims 5-7. 9. The system according to claim 8, wherein the terminal device as the receiving end comprises: The receiving unit is adapted to receive the test voice signal sent by the automatic test device of the voice recognition engine; The speech recognition processing unit is suitable for sending the speech signal for testing to the speech recognition engine and obtaining the speech recognition result from the speech recognition engine; The output unit is adapted to output speech recognition result information corresponding to the current test speech signal. 10. The system of claim 9, wherein, The speech recognition processing unit is adapted to send the speech signal for testing to the speech recognition engine through the interface of the speech recognition engine that provides speech recognition services; or send the speech signal for testing to the speech recognition engine through a specific interface provided by the speech recognition engine sent to the speech recognition engine.