CN108763329A - Evaluating method, device and the computer equipment of voice interactive system IQ level - Google Patents

Abstract

The present invention relates to the evaluating method of voice interactive system IQ level, device and computer equipment, the method includes：Test problem is obtained, the test problem is phonetic problem；Obtain the instruction cycles of test problem described in the voice interactive system success response；Wherein, an instruction cycle is the voice interactive system and an extraneous dialog turns；The evaluation result of the voice interactive system IQ level is determined by described instruction periodicity.The embodiment of the present invention solves the problems, such as that the evaluation result reliability of voice interactive system IQ level is low, and evaluation and test realization process is simple, and evaluation result is more objective.

Description

Evaluation method, device and computer equipment for IQ level of voice interaction system

technical field

The invention relates to the technical field of evaluation, in particular to an evaluation method, device, computer equipment and storage medium for the IQ level of a voice interaction system.

Background technique

With the development of mobile Internet technology and intelligent hardware devices, human beings are no longer satisfied with human-computer interaction methods such as keyboard input and handwriting input. Voice interaction system technology has been applied in various fields such as electronic information, Internet, medical care, education, and office. It is widely used, covering intelligent voice input system, intelligent voice assistant, intelligent speaker, vehicle voice system, intelligent voice-assisted medical system, intelligent spoken language evaluation system, intelligent conference system and other products.

Based on the voice interaction system, it can make smart devices have the same listening ability as humans, and at the same time automatically convert the natural semantics expressed in human language into structured semantics that computers can understand and operate, and complete the human-computer interaction function. For the performance evaluation of the voice interaction system, the common practice in the industry is to evaluate the matching degree between the recognition result of the voice interaction system and the standard voice recognition result.

In the process of realizing the present invention, the inventor found that there are following problems in the prior art. The traditional performance evaluation method needs to pre-build a database of test questions and standard speech recognition results. Due to the limitation of samples in the database, a given Standard speech recognition results cannot take into account actual situations such as language expression habits, diversity of answers to questions, and subjective arbitrariness, which leads to low reliability of the evaluation results of the IQ level (or comprehension ability, intelligence degree, etc.) of the voice interaction system.

Contents of the invention

Based on this, it is necessary to provide a method, device, computer equipment and storage medium for evaluating the IQ level of a voice interactive system in view of the low reliability of the evaluation results of the IQ level of the voice interactive system in the existing method.

The first aspect of the embodiments of the present invention provides a method for evaluating the IQ level of a voice interaction system, including:

Obtain a test question, the test question is a speech question;

Acquiring the number of instruction cycles for the voice interaction system to successfully respond to the test question; wherein, one instruction cycle is a dialogue round between the voice interaction system and the outside world;

The evaluation result of the IQ level of the voice interaction system is determined by the number of instruction cycles.

In one of the embodiments, the determination of the evaluation result of the IQ level of the voice interaction system through the number of instruction cycles includes:

Determining the instruction cycle cost corresponding to the test question by the number of instruction cycles;

The evaluation result of the IQ level of the voice interaction system is determined by the instruction cycle cost.

In one of the embodiments, said obtaining test questions includes: obtaining a plurality of preset test questions;

The evaluation result of determining the IQ level of the voice interaction system through the instruction cycle cost includes:

According to the instruction cycle cost corresponding to each of the plurality of test questions, the score of the evaluation result of the IQ level of the voice interaction system is determined.

In one of the embodiments, it also includes: determining the evaluation result score of the IQ level of the voice interaction system according to the instruction cycle cost corresponding to each of the plurality of test questions, including:

Acquiring respective weights corresponding to the plurality of test questions;

According to the instruction cycle cost and weight corresponding to each of the plurality of test questions, the score of the evaluation result of the IQ level of the voice interaction system is determined.

In one of the embodiments, the obtaining the corresponding weights of the plurality of test questions includes:

Obtaining the corresponding types of the plurality of test questions;

Obtain the weights corresponding to each type as the weights corresponding to the corresponding test questions.

In one of the embodiments, the determining the instruction cycle cost corresponding to the test question through the number of instruction cycles includes:

Get the preset cycle number limit;

The instruction cycle cost corresponding to the test question is determined through the number of instruction cycles and the limit value of the number of cycles.

In one of the embodiments, the determining the instruction cycle cost corresponding to the test question through the number of instruction cycles and the limit value of the number of cycles includes:

If the number of instruction cycles is N1, determine that the instruction cycle cost corresponding to the test question is cost score S1;

If the number of instruction cycles is N2, determine that the instruction cycle cost corresponding to the test question is a cost score S2;

If the number of instruction cycles is N3, determine that the instruction cycle cost corresponding to the test question is a cost score S3;

or,

If the number of instruction cycles is N1, determine that the instruction cycle cost corresponding to the test question is cost level L1;

If the number of instruction cycles is N2, determine that the instruction cycle cost corresponding to the test question is cost level L2;

If the number of instruction cycles is N3, determine that the instruction cycle cost corresponding to the test question is cost level L3;

Among them, 0<N1<N2<M≤N3, N1 is the preset number of cycles, M represents the limit value of the number of cycles; S3<S2<S1, the levels corresponding to L1, L2, and L3 decrease in turn.

In one embodiment, when the number of instruction cycles is between N1 and M, as the number of instruction cycles increases, the value of the cost score S2 decreases.

In one of the embodiments, the decreasing value of the cost score S2 includes: at least one of linear decreasing, exponential decreasing, and logarithmic decreasing.

The second aspect of the embodiment of the present invention provides a device for evaluating the IQ level of a voice interaction system, including:

A question acquiring module, configured to acquire a test question, the test question being a voice question;

The cycle counting module is used to obtain the number of instruction cycles for the voice interaction system to successfully respond to the test question; wherein, one instruction cycle is a dialogue round between the voice interaction system and the outside world;

as well as,

The result determination module is used to determine the evaluation result of the IQ level of the voice interaction system through the number of instruction cycles.

The third aspect of the embodiments of the present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and operable on the processor, and the processor implements the following when executing the computer program: A method for evaluating the IQ level of the above-mentioned voice interactive system.

The fourth aspect of the embodiments of the present invention provides a computer storage medium, on which a computer program is stored, and when the program is executed by a processor, the method for evaluating the IQ level of the voice interaction system described above is implemented.

Implement the embodiment provided by the present invention, based on the preset test questions, obtain the number of dialogue rounds in which the voice interaction system successfully responds to the test questions, the data can reflect the command cycle cost, and then determine the IQ level of the voice interaction system based on the number of dialogue rounds Evaluation results; since the evaluation core refers to the number of dialogue rounds in which the voice interaction system successfully responds to the test questions, it can be compatible with actual conditions such as multilingual expression habits, diversity of question answers, and subjective randomness, which is conducive to improving the reliability of evaluation results sex.

Description of drawings

Fig. 1 is the application environment diagram of the evaluation method of voice interaction system IQ level in an embodiment;

Fig. 2 is the schematic flowchart of the evaluation method of the intelligence quotient level of the speech interactive system of an embodiment;

Fig. 3 is the schematic flowchart of the evaluation method of the intelligence quotient level of the speech interactive system of another embodiment;

Fig. 4 is a schematic structural diagram of an evaluation device for an IQ level of a voice interactive system according to an embodiment;

Figure 5 is an internal block diagram of a computer device in one embodiment.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The occurrences of this phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is understood explicitly and implicitly by those skilled in the art that the embodiments described herein can be combined with other embodiments.

The method for evaluating the IQ level of a voice interactive system provided in this application can be applied to the application environment shown in FIG. 1 . Wherein, the smart terminal 102 is a terminal equipped with a voice interaction system, for example, a terminal with a voice interaction function such as a smart speaker, a driverless car, and a smart voice assistant. The evaluation terminal 104 is used to evaluate the performance of the smart terminal 102. In one embodiment, the evaluation terminal 104 can output voice information, and according to the response of the smart terminal 102 to the voice information, the voice interaction system of the smart terminal 102 IQ level is assessed. In other embodiments, the step of outputting voice information may also be performed by other third-party terminals or users.

In one embodiment, as shown in Figure 2, a method for evaluating the IQ level of a voice interaction system is provided, and the method is applied to the evaluation terminal in Figure 1 as an example for illustration, including the following steps:

S110. Obtain a test question, where the test question is a speech question.

Wherein, the test question refers to the voice information that needs to be sent to the voice interaction system, for example, "call XXX", "query today's weather" and so on.

S120. Obtain the number of instruction cycles for the voice interaction system to successfully respond to the test question; wherein, one instruction cycle is a dialogue round between the voice interaction system and the outside world.

In the embodiment of the invention, for the following scenarios:

User: Call XXX. (user command)

System: Ok, do you want to call the contact XXX now? (system execution)

User: Yes. (user command)

System: calling. (system execution)

The instruction cycle can be understood as: the above human-computer interaction process includes two instruction cycles, that is, the concept of a human-computer dialogue round.

In the above scenario, the test question is: "Call XXX". The number of instruction cycles for the voice interactive system to successfully respond to the test question is 2.

In another scenario:

User: Call XXX. (user command)

System: Sorry, I didn't hear you clearly. (system execution)

User: Call XXX. (user command)

System: Ok, do you want to call the contact XXX now? (system execution)

User: Yes. (user command)

System: calling. (system execution)

In the above scenario, for the same test question ("call XXX"), the number of instruction cycles for the voice interaction system to successfully respond to the test question is 3.

Based on the above example, the number of command cycles for the voice interaction system to successfully respond to the test question refers to the conversation rounds from the voice interaction system receiving the voice of the test question to executing the instruction corresponding to the test question.

S130. Determine the evaluation result of the IQ level of the voice interaction system according to the number of instruction cycles.

In the embodiment of the present invention, when the IQ level of the voice interaction system is low, the number of instruction cycles for successfully responding to the same test question is likely to be more; when the IQ level of the voice interaction system is high, it can successfully respond to the same test. The number of instruction cycles in question will most likely be less.

Through the method of the above-mentioned embodiment, based on the preset test questions, the number of instruction cycles for the voice interaction system to successfully respond to the test questions is obtained. This data can reflect the cost of the instruction cycle, and then determine the evaluation of the IQ level of the voice interaction system based on the number of instruction cycles. Results; Since the evaluation core reference is the number of command cycles in which the voice interaction system successfully responds to the test questions, it can be compatible with the actual conditions such as multi-language expression habits, diversity of question answers, and subjective randomness, which is conducive to improving the reliability of the evaluation results. .

In addition, by evaluating the number of instruction cycles spent by the voice interaction system to successfully respond to test questions, it is possible to comprehensively evaluate the speech recognition ability, semantic understanding ability, and semantic processing ability of the voice interaction system. reliability of the results.

In addition, there is no need to pre-build a standard language recognition result database, which also reduces the complexity of the evaluation process.

In an embodiment, the above step S130 includes: determining the instruction cycle cost corresponding to the test question according to the number of instruction cycles, and then determining the evaluation result of the IQ level of the voice interaction system through the instruction cycle cost.

The number of instruction cycles for the voice interaction system to successfully respond to the same test question is more, and it takes more time, and the task execution volume of the voice interaction system is also more, so the corresponding instruction cycle cost is higher. Therefore, the number of instruction cycles can reflect the level of the instruction cycle cost, and then more intuitively determine the evaluation result of the IQ level of the voice interaction system through the level of the instruction cycle cost.

In an embodiment, there may be multiple test questions participating in the evaluation, and the multiple test questions are pre-stored in a corresponding database. When evaluating the voice interaction system, each test question can be obtained from the database to evaluate the voice interaction system, or some test questions can be selected according to certain rules to evaluate the voice interaction system. The rules also include random selection.

In other embodiments, it can also be completely random, without the need for preset test questions and storage, and the user, evaluation terminal or third-party terminal randomly sends out the voice information of the test questions, and then comprehensively evaluates based on the evaluation information corresponding to each test question Performance of voice interaction systems.

Further, the evaluation result of the IQ level of the voice interaction system can be determined according to the instruction cycle costs corresponding to each of the multiple test questions. For example: obtaining the corresponding weights of the plurality of test questions; determining the evaluation result of the IQ level of the voice interaction system according to the instruction cycle costs and weights corresponding to the plurality of test questions. Among them, the type can be divided according to the complexity of question understanding, or according to the minimum number of instruction cycles required to successfully respond to the question, or according to the voice type or accent corresponding to the question (for example, English type and Chinese type, Mandarin type, Cantonese type, etc.) type or other dialect types, etc.). For different types of test questions, set corresponding weights, so that the response of the voice interaction system to different types of voices can be obtained, and based on this, the IQ level of the voice interaction system can be comprehensively evaluated.

Evaluating the voice interaction system through multiple test questions can further improve the reliability of the evaluation results.

In an embodiment, the specific method of determining the instruction cycle cost corresponding to the test question through the number of instruction cycles may include: obtaining a preset limit value of the number of cycles; using the number of instruction cycles and the limit value of the number of cycles , to determine the instruction cycle cost corresponding to the test question. E.g:

If the number of instruction cycles is N1, determine that the instruction cycle cost corresponding to the test question is a cost score S1; if the instruction cycle number is N2, determine that the instruction cycle cost corresponding to the test question is a cost score S2; The number of instruction cycles is N3, and the instruction cycle cost corresponding to the test problem is determined as cost score S3.

Wherein, 0<N1<N2<M≤N3, N1 is the preset number of cycles, M represents the limit value of the number of cycles; S3<S2<S1. In the embodiment of the present invention, the higher the cost score of the instruction cycle cost, the lower the cost for the voice interaction system to successfully respond to the test question. Wherein, the preset number of cycles is determined according to the minimum number of instruction cycles consumed to successfully respond to a test question, and N1 may be 1 or other integers, depending on the actual situation.

Correspondingly, the evaluation result of the IQ level of the voice interaction system can be determined according to the cost score of the instruction cycle cost corresponding to the test question.

In other embodiments, determining the instruction cycle cost corresponding to the test question may also be in other ways than the cost score, such as a level, for example:

If the instruction cycle number is N1, determine that the instruction cycle cost corresponding to the test question is cost level L1; if the instruction cycle number is N2, determine that the instruction cycle cost corresponding to the test question is cost level L2; The number of instruction cycles is N3, and the instruction cycle cost corresponding to the test problem is determined to be cost level L3; wherein, the levels corresponding to L1, L2, and L3 are successively decreased. In the embodiment of the present invention, the higher the cost level of the instruction cycle cost, the lower the cost for the voice interaction system to successfully respond to the test question. Wherein, the level may be determined based on the numerical value of the level, for example, level 1 is lower than level 2, or level 1 is greater than level 2, which may be determined according to the actual situation.

Correspondingly, the evaluation result of the IQ level of the voice interaction system can be determined according to the cost level of the instruction cycle cost corresponding to the test question.

On the basis of the above-mentioned embodiments, in conjunction with FIG. 3 , the method is applied to the evaluation terminal in FIG. 1 as an example to further illustrate the evaluation method of the IQ level of the voice interaction system. Among them, there are multiple test questions participating in the evaluation, and the corresponding evaluation method of the IQ level of the voice interaction system includes the following steps:

Step S301, setting multiple test questions and storing them in the question database.

Step S302, classify the multiple set test questions into types, set corresponding weights for the test questions corresponding to each type, and save the corresponding relationship.

In step S303, a test question is obtained from the question database, and a voice corresponding to the test question is issued.

The voice corresponding to the test question can be issued by the evaluation terminal, the test user, or a third-party device, for example, by playing the recorded voice through the user's mobile phone.

Step S304, obtaining the number of instruction cycles for which the voice interaction system successfully responds to the test question.

For different voice interaction systems, due to differences in IQ levels, voice interaction systems cannot accurately complete tasks when recognizing, understanding, and processing voice commands. In this case, the voice interaction system will give wrong answers, perform wrong operations, or fail to perform operations. At this time, the user needs to re-issue the voice command, such as reducing the speech rate, changing the expression mode of the voice command, etc., so that the voice interaction system can better recognize and understand the user's voice. It will increase the number of instruction cycles required to solve a test problem, increasing the instruction cycle cost. Therefore, when the voice interaction system successfully responds to a voice command, the more command cycles it consumes, it reflects the lower IQ level of the voice interaction system.

Step S305, determine the instruction cycle cost corresponding to the test question according to the number of instruction cycles.

For example: assume that the minimum number of instruction cycles required by the voice interaction system to successfully respond to the test question is 1, and M instruction cycles are used as the cycle number limit. Then, if the voice interaction system cannot provide the correct answer or reasonable response within M instruction cycles, the cost score of the task is 0; if the voice interaction system provides the correct answer or reasonable response within the first instruction cycle, then The cost score of this task is full score S ₀ ; if the voice interaction system provides the correct answer or reasonable response within the Nth (1<N<M) instruction cycle, the cost score of this task is S _N (0<S _N <S). Among them, there is a decreasing relationship between S ₀ and _SN , including but not limited to linear decreasing, exponential decreasing, logarithmic decreasing or other types of decreasing relationship, which can be selected according to the needs of the test object and the characteristics of the problem set.

Step S306, whether there are other test questions in the question database that have not participated in the evaluation, if so, return to step S303, and obtain the next test question for evaluation; if not, go to the next step.

Step S307, obtaining weights corresponding to each test question participating in the evaluation, and determining the evaluation result score of the IQ level of the voice interaction system according to the instruction cycle costs and weights corresponding to each of the multiple test questions.

That is, the final score of the evaluation result of the IQ level of the voice interaction system is obtained through the comprehensive calculation of the cost scores and their weights corresponding to various questions. Comprehensive operations include, but are not limited to, operations such as accumulation and multiplication.

In the method of the above-mentioned embodiment, by introducing the concept of instruction cycle cost, the instruction cycle cost consumed by the successful execution of a task (that is, successfully responding to a test question) of the voice interaction system is used as an index to measure its IQ level, and the Its IQ level is quantified. Due to the characteristics of easy operation and digitalization of instruction cycle cost calculation, it can accurately distinguish the IQ level of each voice interaction system. At the same time, it can avoid the limitation problem caused by the standard answer matching evaluation method, and make the evaluation result more reasonable and objective with higher reliability.

It can be understood that the method for evaluating the IQ level of the voice interaction system in the above embodiment is applicable to terminals with intelligent voice interaction functions such as smart speakers, driverless cars, and smart voice assistants.

It should be understood that, for the foregoing method embodiments, although the steps in the flow chart are shown sequentially as indicated by the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flow chart of the method embodiment may include multiple sub-steps or multiple stages, these sub-steps or stages are not necessarily executed at the same time, but may be executed at different time, these sub-steps Or the order of execution of the stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least a part of other steps or substeps of other steps or stages.

Based on the same idea as the method for evaluating the IQ level of the voice interaction system in the above embodiments, this paper also provides an evaluation device for the IQ level of the voice interaction system.

In one embodiment, as shown in Figure 4, the evaluation device of the IQ level of the voice interaction system of this embodiment includes: a question acquisition module 410, a cycle counting module 420 and a result determination module 430, each module is described in detail as follows:

The above-mentioned question acquisition module 410 is configured to acquire a test question, and the test question is a speech question.

The above-mentioned cycle counting module 420 is used to obtain the number of instruction cycles for the voice interaction system to successfully respond to the test question; wherein, one instruction cycle is a dialogue round between the voice interaction system and the outside world.

The above-mentioned result determination module 430 is configured to determine the evaluation result of the IQ level of the voice interaction system through the number of instruction cycles.

In one embodiment, the above-mentioned result determination module 430 includes:

The cost accounting sub-module is used to determine the instruction cycle cost corresponding to the test question through the number of instruction cycles. The result determination sub-module is used to determine the evaluation result of the IQ level of the voice interaction system through the instruction cycle cost.

In an embodiment, the above-mentioned question obtaining module 410: the application obtains a plurality of preset test questions. Correspondingly, the above-mentioned result determination module 430 is configured to determine the evaluation result score of the IQ level of the voice interaction system according to the instruction cycle costs corresponding to each of the plurality of test questions.

In one embodiment, the above-mentioned result determination submodule includes:

The weight acquisition unit is configured to acquire the respective weights corresponding to the plurality of test questions. And, the result calculation unit is used to determine the evaluation result score of the IQ level of the voice interaction system according to the instruction cycle cost and weight corresponding to each of the plurality of test questions.

In an embodiment, the above-mentioned weight acquisition unit is configured to acquire the types corresponding to each of the plurality of test questions; and acquire the weights corresponding to each type as the weights corresponding to the corresponding test questions.

In one embodiment, the above-mentioned cost accounting sub-module includes:

a limit acquisition unit, configured to acquire a preset limit of the number of cycles; and,

A cost calculation unit, configured to determine the instruction cycle cost corresponding to the test question through the instruction cycle number and the cycle number limit.

In one embodiment, the above-mentioned cost calculation unit includes:

The first cost calculation unit is configured to determine that the instruction cycle cost corresponding to the test question is a cost score S1 if the instruction cycle number is N1; the second cost calculation unit is configured to determine that the instruction cycle number is N2. The instruction cycle cost corresponding to the test question is a cost score S2; the third cost calculation unit is configured to determine that the instruction cycle cost corresponding to the test question is a cost score S3 if the number of instruction cycles is N3.

Wherein, 0<N1<N2<M≤N3, N1 is the preset number of cycles, M represents the limit value of the number of cycles; S3<S2<S1.

In another embodiment, the first cost calculation unit is configured to determine that the instruction cycle cost corresponding to the test problem is cost level L1 if the instruction cycle number is N1; the second cost calculation unit is configured to determine The number of instruction cycles is N2, and it is determined that the instruction cycle cost corresponding to the test question is cost level L2; the third cost calculation unit is used to determine that the instruction cycle cost corresponding to the test question is cost if the instruction cycle number is N3 Level L3;

Among them, 0<N1<N2<M≤N3, N1 is the preset number of cycles, M represents the limit value of the number of cycles; the levels corresponding to L1, L2, and L3 decrease in turn.

For the specific limitations of the evaluation device for the IQ level of the voice interaction system, please refer to the above definition of the evaluation method for the IQ level of the voice interaction system, which will not be repeated here. Each module in the evaluation device for the IQ level of the above-mentioned voice interactive system can be fully or partially realized by software, hardware and combinations thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of the above modules.

In addition, in the implementation of the evaluation device for the IQ level of the voice interaction system in the above example, the logical division of each program module is only an example, and in actual applications, it can be based on the needs, such as the configuration requirements of the corresponding hardware or the convenience of software implementation. It is considered that the above function distribution is completed by different program modules, that is, the internal structure of the IQ level evaluation device of the voice interaction system is divided into different program modules to complete all or part of the functions described above.

In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure may be as shown in FIG. 5 . The computer device includes a processor, a memory, a display screen and an input device connected by a system bus. Wherein, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. When the computer program is executed by a processor, a method for evaluating the IQ level of a voice interaction system is realized. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a button, a trackball or a touch pad provided on the casing of the computer device , and can also be an external keyboard, touchpad, or mouse.

Those skilled in the art can understand that the structure shown in Figure 5 is only a block diagram of a part of the structure related to the solution of this application, and does not constitute a limitation to the computer equipment on which the solution of this application is applied. The specific computer equipment can be More or fewer components than shown in the figures may be included, or some components may be combined, or have a different arrangement of components.

In one embodiment, a computer device is provided, including a memory and a processor, and a computer program is stored in the memory, and when the processor executes the computer program, the method for evaluating the IQ level of a voice interaction system in any of the above embodiments can be realized. step.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented through computer programs to instruct related hardware, and the computer programs can be stored in a non-volatile computer-readable memory In the medium, when the computer program is executed by the processor, it can realize the steps of the method for evaluating the IQ level of the voice interaction system in any of the above embodiments.

Based on this, in one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the steps of the method for evaluating the IQ level of a voice interaction system in any of the above-mentioned embodiments are implemented. .

Wherein, any references to memory, storage, database or other media used in the various embodiments provided in the present application may include non-volatile and/or volatile memory. Nonvolatile memory can include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Chain Synchlink DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

The technical features of the above embodiments can be combined arbitrarily. To make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be It is considered to be within the range described in this specification.

In the foregoing embodiments, the descriptions of each embodiment have their own emphases, and for parts not described in detail in a certain embodiment, reference may be made to relevant descriptions of other embodiments.

The terms "comprising" and "having" and any variations thereof in the embodiments herein are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or (module) units is not limited to the listed steps or units, but optionally also includes steps or units not listed, or optionally It also includes other steps or elements inherent to these processes, methods, products or devices.

The "plurality" mentioned herein means two or more. "And/or" describes the association relationship of associated objects, indicating that there may be three types of relationships, for example, A and/or B may indicate: A exists alone, A and B exist simultaneously, and B exists independently. The character "/" generally indicates that the contextual objects are an "or" relationship.

The "first\second" mentioned in this article is only to distinguish similar objects, and does not represent a specific ordering of objects. Understandably, "first\second" can be interchanged with specific sequence or sequence. It should be understood that the terms "first\second" are interchangeable under appropriate circumstances such that the embodiments described herein can be practiced in sequences other than those illustrated or described herein.

The above-mentioned embodiments only represent several implementation modes of the present application, and the description thereof is relatively specific and detailed, but it should not be construed as limiting the scope of the patent for the invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the scope of protection of the patent application should be based on the appended claims.

Claims (12)

1. A method for evaluating the IQ level of a voice interactive system, characterized in that it comprises: Obtain a test question, the test question is a speech question; Acquiring the number of instruction cycles for the voice interaction system to successfully respond to the test question; wherein, one instruction cycle is a dialogue round between the voice interaction system and the outside world; The evaluation result of the IQ level of the voice interaction system is determined by the number of instruction cycles. 2. The method according to claim 1, wherein the determination of the evaluation result of the IQ level of the voice interaction system by the number of instruction cycles includes: Determining the instruction cycle cost corresponding to the test question by the number of instruction cycles; The evaluation result of the IQ level of the voice interaction system is determined by the instruction cycle cost. 3. The method according to claim 2, wherein said obtaining test questions comprises: obtaining a plurality of preset test questions; The evaluation result of determining the IQ level of the voice interaction system through the instruction cycle cost includes: According to the instruction cycle cost corresponding to each of the plurality of test questions, the score of the evaluation result of the IQ level of the voice interaction system is determined. 4. The method according to claim 3, further comprising: determining the evaluation result score of the IQ level of the voice interaction system according to the instruction cycle cost corresponding to each of the plurality of test questions, including: Acquiring respective weights corresponding to the plurality of test questions; The evaluation result of the IQ level of the voice interaction system is determined according to the instruction cycle cost and weight corresponding to each of the plurality of test questions. 5. The method according to claim 4, wherein said obtaining the respective weights corresponding to said plurality of test questions comprises: Obtaining the corresponding types of the plurality of test questions; Obtain the weights corresponding to each type as the weights corresponding to the corresponding test questions. 6. The method according to any one of claims 2 to 5, wherein the determining the instruction cycle cost corresponding to the test question through the number of instruction cycles includes: Obtain the preset cycle number limit; The instruction cycle cost corresponding to the test question is determined through the number of instruction cycles and the limit value of the number of cycles. 7. The method according to claim 6, wherein the determining the instruction cycle cost corresponding to the test question through the number of instruction cycles and the limit value of the number of cycles includes: If the number of instruction cycles is N1, determine that the instruction cycle cost corresponding to the test question is cost score S1; If the number of instruction cycles is N2, determine that the instruction cycle cost corresponding to the test question is a cost score S2; If the number of instruction cycles is N3, determine that the instruction cycle cost corresponding to the test question is a cost score S3; or, If the number of instruction cycles is N1, determine that the instruction cycle cost corresponding to the test question is cost level L1; If the number of instruction cycles is N2, determine that the instruction cycle cost corresponding to the test question is cost level L2; If the number of instruction cycles is N3, determine that the instruction cycle cost corresponding to the test question is cost level L3; Among them, 0<N1<N2<M≤N3, N1 is the preset number of cycles, M represents the limit value of the number of cycles; S3<S2<S1, the levels corresponding to L1, L2, and L3 decrease in turn. 8. The method of claim 7, wherein, When the number of instruction cycles is between N1 and M, as the number of instruction cycles increases, the value of the cost score S2 decreases. 9 . The method according to claim 8 , wherein the decreasing value of the cost score S2 includes at least one of linear decreasing, exponential decreasing, and logarithmic decreasing. 10. An evaluation device for the IQ level of a voice interactive system, characterized in that it comprises: A question acquiring module, configured to acquire a test question, the test question being a voice question; The cycle counting module is used to obtain the number of instruction cycles for the voice interaction system to successfully respond to the test question; wherein, one instruction cycle is a dialogue round between the voice interaction system and the outside world; as well as, The result determination module is used to determine the evaluation result of the IQ level of the voice interaction system through the number of instruction cycles. 11. A computer device, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, characterized in that, when the processor executes the program, any one of claims 1 to 9 is realized. steps of the method described above. 12. A computer-readable storage medium, on which a computer program is stored, characterized in that, when the program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are realized.